From 7d73191613f2f1b467b94fab8c4d079d958abe02 Mon Sep 17 00:00:00 2001 From: Doug Mealing Date: Fri, 9 Oct 2026 09:56:47 -0400 Subject: [PATCH 01/18] docs(plan): FR-044 zero rows from a dimension's entity and a default for an empty measure Design for two additions to the 1.1 reporting vocabulary, in all five ports: @spine on object.report (the report's rows come from a dimension's entity, so a tuple with no fact rows still has a row) and @default on measure.aggregate and measure.ratio (the integer a measure reads when it would be null). The plan settles where each attribute lives and what was rejected, the loader rules (R8, R9, M7, M8) and their error codes, the view SQL per dialect, what a measure reads in a row with no facts, the derived shape's nullability and the row type in every port, and filter and sort on the derived fields. The SQL shapes were executed on Postgres 16, SQLite and MySQL 8.4. The spec gains the ADR-0023 register amendment (3.2), requirements R8 and R9, two mapping rows, one acceptance bullet and the parked list. metamodelVersion stays 1.1. No code changes in this commit. --- ...9-fr-044-zero-rows-and-measure-defaults.md | 915 ++++++++++++++++++ ...2026-10-02-fr-044-core-reporting-design.md | 159 ++- 2 files changed, 1073 insertions(+), 1 deletion(-) create mode 100644 docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md diff --git a/docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md b/docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md new file mode 100644 index 000000000..773b9d7a8 --- /dev/null +++ b/docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md @@ -0,0 +1,915 @@ +# FR-044 — Zero rows from a dimension's entity, and a default for an empty measure + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** Add two attributes to the FR-044 reporting vocabulary before 1.1.0 ships, in all five ports: `@spine` on `object.report` (the report's rows come from a dimension's entity, so a tuple with no fact rows still has a row) and `@default` on `measure.aggregate` and `measure.ratio` (the value a measure reads when it would be null). A model that uses neither generates byte-for-byte what it does now, and `metamodelVersion` stays `1.1`. + +**Architecture:** Both attributes are registered once in `spec/metamodel/` and copied or embedded into each port, as Plan 1 did. Four cross-node rules join the existing `validateReporting` pass in each port; the value type of `@default` is the registered `attr.int`, so the existing attribute type check enforces it. The derived read shape (`reportShape`, Plan 2 Table B) gains two nullability rules, copied to every port and pinned by `report-shapes.json`. TypeScript alone changes the SQL: with a spine the view selects `FROM` the spine entity's table and `LEFT JOIN`s the fact table, and a defaulted measure is wrapped in `COALESCE`. No other port emits SQL, no route generator changes, and no runtime changes: every port already reads a report view through its shape. + +**Tech Stack:** TypeScript (Bun), C# (.NET, EF Core), Java (Maven, OMDB), Kotlin (Exposed), Python (pytest). Postgres 16, SQLite ≥ 3.37, MySQL 8.4. + +**Spec:** `docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md`, amended by this plan's change: §3.2 (the ADR-0023 register amendment), R8 (`@spine`), R9 (`@default`), two §5 mapping rows, one §7 acceptance bullet and the §9 parked list. Earlier plans: [Plan 1](2026-10-03-fr-044-plan-1-reporting-vocabulary.md) (vocabulary and loader rules), [Plan 2](2026-10-03-fr-044-plan-2-report-view-lowering.md) (lowering; its contract tables A to H are amended here, not replaced), [Plan 3](2026-10-04-fr-044-plan-3-report-read-routes.md) (read routes). What those shipped: `docs/features/reporting.md`. + +**This plan sits between Plan 3 and Plan 4.** It is not one of the five: it closes two gaps a reference adopter's report inventory found (its gaps "no zero rows from a dimension's entity" and "no default for an empty measure") while the vocabulary can still change without a second metamodel move. Out of scope: cross-fact reports, anti-join, joins on a natural key, nested aggregation, renaming exposed fields, the calendar spine and the `reporting` library (R7), the Cube and dbt exporters, the client hook, `measure.derived`, #395, #222, #8, #393. + +## How this plan was verified + +Every path, function and test file cited below was read in the tree at `fff01da85`, by hand or by a read-only mapping pass over all five ports, unless it is marked **UNVERIFIED**. Every SQL expression in Tables D and E was executed on Postgres 16, SQLite 3.37.2 and MySQL 8.4 (default `ONLY_FULL_GROUP_BY`) against three programs, one of them with no weeks, plus a week whose reference is null, and returned the values Table E shows. The column types in Table E's last rows were read back from `information_schema` (Postgres, MySQL) and `typeof()` (SQLite). + +Things marked UNVERIFIED are collected in [Unverified items](#unverified-items). The first step of the task that touches one is to read the code and confirm or correct it. + +## Global Constraints + +- **Exactly two names are registered:** `spine` on `object.report`, and `default` on `measure.aggregate` and `measure.ratio`. Nothing else. `measure.derived` stays unregistered. +- **`metamodelVersion` stays `1.1`.** 1.1 has not shipped, so the additions join it. `node scripts/check-metamodel-version.mjs` must pass without `--set`, and `--explain` must classify every change ADDITIVE. +- **No-churn.** A model that declares neither attribute produces byte-identical output in every port: generated code, `canonical/schema.postgres.sql` statements, `report-shapes.json` entries, the api-contract `report/` schema, the `codegen-noop` corpus. Existing fixture files gain entries; no existing entry changes. +- **No query-time engine.** A report is still one compiled view. +- **Every measure belongs to `@from`, and every path is to-one.** `@spine` is a to-one path, as `@via` is. Do not admit a to-many hop, a cross-fact measure, an anti-join or a join on a natural key. +- **No SQL string enters the metamodel.** +- **View DDL is produced by TypeScript only** (ADR-0015). No other port emits SQL for a report. +- **A report lowers only with `source.rdb` `@kind: view`.** A sourceless report with `@spine` or a defaulted measure stays inert. +- ADR-0039: read effective properties with resolving accessors. Any `own*()` call carries a comment naming its sanctioned case. +- TS: named constants for metamodel strings, no `any`, never `instanceof` a node from another package. +- Public repo: no private project names, no absolute home paths, in code, fixtures, docs or commit messages. The adopter inventory that motivated this is private: write "a reference adopter" and generic entity names. +- **Another change is in flight** on the TypeScript report generators (a generated list hook), the spec's `daysEngaged` worked example and its fixtures, and a ratio-on-SQLite route check. Expect rebases. Do not edit R2's example in the spec, and add new fixture entries at the end of a file rather than between existing ones. +- This machine is loaded. Run scoped tests while iterating and the full `scripts/ci-local.sh` once, on the final tree. A lone timeout is rerun in isolation before it is believed. +- Do not push until every port is green. + +## Review Focus + +1. **A spine row whose facts are all filtered out keeps its row.** The report's `@segment` and `@filter` go into the join condition, never a `WHERE`. A `WHERE` on a fact column silently turns the outer join back into an inner one, and the zero rows vanish with no error. (Task 5 test `the report scope is in the join condition`; scenario `report-spine-scoped`.) +2. **A measure whose condition is true for a row of nulls.** `{ refundedAt: { isNull: true } }` is true for the null-extended row of a spine entity with no facts. Every aggregate must still read `0` or null there, because it ignores a null `@of` column, on all three engines. (Task 6 value test `an isNull condition does not count the empty row`.) +3. **A defaulted measure is never null, in the database and in every typed row.** `COALESCE` in the view, `required: true` in the shape, a non-nullable member in C#, Kotlin, Java, Python and the TypeScript read schema. A port that types it nullable compiles and passes the value scenarios, so only the shape artifact and the per-port row tests catch it. (Tasks 4 and 9 to 12.) +4. **A model that uses neither attribute is untouched.** No existing view statement, shape entry, generated file or fixture expectation changes. (Task 13, and the no-churn table at the end.) +5. **Hop names are compared as written.** A spine written with the relationship name (`Purchase.program`) and a dimension written with the reference name (`Purchase.fkProgram`) name the same join and are still refused, because the lowering merges joins by hop name. The error must say to write the same hops. (Task 3 fixture `error-report-spine-dimension-other-path`.) + +--- + +## The design, and what was rejected + +### What merging this plan decides + +Four choices here would be expensive to change after 1.1.0, because each changes what a report returns. They are stated first so they are ruled on, not discovered. + +1. **The grain does not change.** A report is still one row per distinct dimension tuple. `@spine` changes where the tuples come from (the spine entity's rows, not the fact rows). A report gets exactly one row per row of the spine entity when it lists a dimension over that entity's identity; if it lists only a title and two rows share the title, they are one row, as they would be in any report. +2. **The report's row scope filters facts, never spine rows.** `@segment` and `@filter` on a `@spine` report decide which rows of `@from` are aggregated. +3. **A fact row with no spine row is in no row of the report.** A null reference, or one that matches nothing, has no spine row to sit in. (Without `@spine`, the same rows form a null group.) +4. **A ratio's operand carries its own `@default` into the ratio.** `revenue` with `@default: 0` over `buyers` reads `0`, not null, for a group with buyers and no revenue. A ratio's own `@default` then applies to the quotient. + +### Decision 1 — where the zero rows are declared: `@spine` on the report + +`"@spine": "Purchase.program"`: a to-one path from `@from`, in `@via`'s grammar. The entity at its end supplies the rows. + +| Alternative | Why it was rejected | +|---|---| +| On the **dimension** (`dimension.attribute … "@zeroFill": true`) | One dimension is listed by both kinds of report. The adopter's sales table lists only programs that sold; its catalogue table lists every program. A flag on the dimension would force two dimensions that differ in nothing else, and two names for one field. | +| On the **dimension reference** (`"@dimensions": ["program!"]`, or `program:all`) | A third item grammar inside a string; `name:grain` already owns the colon. It also cannot say which entity when the dimension has no `@via` (a reference column names a row, it is not one). | +| On the **measure** | A measure does not know which report lists it or which rows that report shows. | +| The value is an **entity name** (`"@spine": "Program"`) | An entity can be reached from `@from` by more than one reference (`Transfer.fromAccount`, `Transfer.toAccount`), so the name does not say which rows are meant. A path does, and the grammar already exists. | +| The value is a **listed dimension's name** | A dimension with no `@via` reaches no entity; several dimensions can reach one entity, so naming one of them is arbitrary; and it hides the join the report depends on. | +| A new **subtype or `@kind`** of report | ADR-0037: the derived shape, the routes and the read path are the same. What differs is one reference, which is configuration: an attribute. | +| Leave it to `object.projection` with `origin.aggregate` (one row per base row already) | It restates every measure in a second vocabulary that has no ratio, no tuple distinct count, no relative date and no segment. That duplication is what FR-044 exists to remove. | +| Reverse the declaration (`@from` the dimension entity, measures reached to-many) | It breaks the standing rule that every measure belongs to `@from` and every path is to-one, which is what keeps `SUM` free of fan-out. | +| Group by the spine entity's key always, whatever is listed | Rows the caller cannot tell apart (two rows titled the same), and the one invariant every report has (a row is a distinct tuple) would have an exception. | + +Names considered: `@spine` (chosen), `@rowsFrom`, `@zeroRows`, `@per`, `@everyRowOf`. `@rowsFrom` reads as a second `@from`. `@zeroRows` names an effect and reads as a boolean. `@per` describes every dimension. "Spine" is the term the spec's own R7 uses ("time-spine"), the term the adopter inventory used, and the term in dbt MetricFlow, which this FR exports to. The registry description states it in one plain sentence for a reader who has never met the word. + +**What a `@spine` report may list.** Every listed dimension must be reached through the spine: its `@via` begins with the spine's hops. So it is a column of the spine entity or of an entity to-one from it. Three things follow, each refused at load so that admitting it later is additive: + +- **A dimension read from the fact row** (no `@via`, or a `@via` through another reference) is refused: it has no value in a row with no facts. +- **A time dimension alongside** follows the same rule and needs no rule of its own. Over a column of the spine entity or beyond (the month a program was published) it is legal. Over a fact column (the month of a purchase) it is refused; a program with no purchases would need a row with a null month, and zero-filled time buckets are R7's calendar spine. +- **More than one zero-row entity** is not expressible: `@spine` is one path. Entities to-one beyond the spine are reachable as ordinary dimensions (a workout's week, the week's program), which covers a chart of every scheduled day. Two independent spines would be a cross join the size of both tables multiplied; parked in spec §9. + +A report with `@spine` lists at least one dimension. With none it would be the totals row computed over a join, which differs from the plain totals report only by silently dropping facts whose reference is null. + +### Decision 2 — where the empty value is declared: `@default` on the measure + +`"@default": 0` on a `measure.aggregate` or a `measure.ratio`: an integer. + +| Alternative | Why it was rejected | +|---|---| +| On the **report** (`"@defaults": { "revenue": 0 }`) | The same measure would read null in one report and zero in another, and an exporter would have nowhere to put it: both Cube and MetricFlow attach the fill to the measure or metric. | +| On the **measure reference** (`"@measures": ["revenue=0"]`) | A new item grammar, and the same objection. | +| A **boolean** (`"@zeroWhenEmpty": true`) | It cannot say a sentinel (`-1` for "not applicable"), and it is no simpler to lower or to type. | +| A **new name** (`@whenEmpty`, `@ifNull`, `@fillNullsWith`, `@coalesce`) | ADR-0037: same concept, same attribute name. A field's `@default` is the value used when none is supplied; a measure's is the value used when there is none to aggregate. `@coalesce` and `@ifNull` also name SQL functions. | +| An **untyped** value following the measure's type, as a field's `@default` follows the field's | A fractional default needs a rule per measure type (an integer for a `long` or `currency` sum, a decimal for an `avg`) and a number spelling that five canonical serializers and three SQL dialects agree on. They do not agree today: no fixture round-trips a non-integer number through any attribute, C# and Python print a whole-number double as an integer, and Java has no such guard. Every case found is zero, MetricFlow's `fill_nulls_with` is an integer, and an integer is a valid value of every numeric type a measure can have. Registered as `attr.int`; widening is additive. | +| A default on **non-numeric** `min` / `max` (a date, a string) | "Last activity" with no activity is null, and a made-up date would be a wrong answer. It also needs a typed literal per dialect. Refused at load. | +| A default on `count` accepted and ignored | An attribute that can never apply is a false statement in the model. Refused at load. | + +--- + +## Contract tables (what the ports copy) + +Tables A, B and C are implemented in all five ports. Tables D and E are implemented in TypeScript only; the other ports never emit SQL, but their readers depend on the columns these tables produce. Each table amends the Plan 2 table named in its heading; a row not shown here is unchanged. + +### Table A — what is registered + +`spec/metamodel/object.json`, in the `object.report` children, after `filter`: + +```json +{ "type": "attr", "subType": "string", "name": "spine", "min": 0, "max": 1, "description": "Optional to-one path from @from to the entity whose rows supply the report's rows (e.g. 'Purchase.program'), written like a dimension's @via. With it the report has one row per distinct dimension tuple among THAT entity's rows, including the ones no row of @from refers to: a count there is 0 and any other measure is null unless it declares @default. Every listed dimension must be reached through this path. @segment and @filter still scope the rows of @from and never remove a row. A row of @from whose reference is null or matches nothing is in no row of the report." } +``` + +`spec/metamodel/reporting.json`, appended to the children of both `measure.aggregate` and `measure.ratio`: + +```json +{ "type": "attr", "subType": "int", "name": "default", "min": 0, "max": 1, "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } +``` + +No existing description is edited. No error code is added: the rules below reuse `ERR_INVALID_REPORT`, `ERR_INVALID_MEASURE` and `ERR_BAD_ATTR_VALUE`, so `fixtures/conformance/ERROR-CODES.json` is not touched. + +### Table B — loader rules (extends the Plan 1 rule table) + +| Id | Rule | Code | Fixture | +|---|---|---|---| +| R8 | A report's `@spine` is `Owner.hop[.hop…]`. `Owner` resolves in the report's package and is `@from` or an entity it extends. Every hop is a `relationship.*` with `@cardinality: one` or an `identity.reference`, and its target resolves. This is rule D2's walk, started at `@from`. | `ERR_INVALID_REPORT` | `error-report-spine-to-many`, `error-report-spine-not-from` | +| R9 | A report with `@spine` lists at least one dimension, and every listed dimension has an `@via` whose hop names begin with the spine's hop names. The names are compared as written; the owner segment is not compared. | `ERR_INVALID_REPORT` | `error-report-spine-no-dimensions`, `error-report-spine-dimension-off-spine`, `error-report-spine-dimension-other-path` | +| M7 | A `measure.aggregate` with `@agg: count` declares no `@default`. | `ERR_INVALID_MEASURE` | `error-measure-default-on-count` | +| M8 | A `measure.aggregate` with `@agg: min` or `max` over a field that is not numeric (rule M4's set: `int, long, double, float, decimal, currency`) declares no `@default`. | `ERR_INVALID_MEASURE` | `error-measure-default-non-numeric` | +| (type) | `@default` is an integer. Enforced by the registered `attr.int`, through the attribute type check every port already runs. | `ERR_BAD_ATTR_VALUE` | `error-measure-default-not-integer` | + +Order and skipping, so one mistake gives one error: + +- R8 and R9 run after R1 and need a resolved `@from`. R9 is skipped when R8 failed. R9 reports each offending dimension once, on the report node, naming the dimension and the spine. +- M7 and M8 run after M1 to M4 and only when none of them fired. A ratio is checked by neither (its `@default` needs only the type check). +- `error-report-spine-dimension-off-spine` lists a **time** dimension over a fact column (`purchasedAt:day`), so the "time dimension alongside" case is the fixture. `error-report-spine-dimension-other-path` writes the spine with the relationship's name (`Purchase.program`) and lists a dimension whose `@via` uses the reference's name for the same join (`Purchase.fkProgram`): the one case where a port that resolved joins instead of comparing names would accept what the others refuse. A dimension through a second reference to the same entity is a unit test in each port. + +Each error fixture's input differs from the positive fixture `reporting-spine-and-default` by the one change that breaks its rule. A second positive fixture, `reporting-spine-inherited`, declares the reference, the dimensions and a defaulted measure on an abstract base, writes the spine with the base as its owner (`BaseEvent.program`), and reports over the concrete entity. + +Message texts are part of the port contract (every port uses the same words). The TypeScript texts in Task 2 are the reference. + +### Table C — derived fields (amends Plan 2 Table B) + +Two rules change. Field names, order, subtypes and `typeSource` are untouched. + +| Item | `required` | +|---|---| +| A dimension of a report **without** `@spine` | unchanged: `true` only when the dimension has no `@via` and the `@of` field's effective `@required` is `true` | +| A dimension of a report **with** `@spine`, whose `@via` hops equal the spine's hops (a column of the spine entity itself) | `true` when the `@of` field's effective `@required` is `true`, **or** the field is one of the `identity.primary` `@fields` of the entity `@of` names (under rules D1 and R9 that is the spine entity, or one it extends) | +| A dimension of a report **with** `@spine`, reached beyond the spine | `false` (its join is `LEFT OUTER`) | +| `measure.aggregate` `count` | unchanged: `true` | +| Any other `measure.aggregate`, or a `measure.ratio`, **with** `@default` | `true` | +| Any other `measure.aggregate`, or a `measure.ratio`, without `@default` | unchanged: `false` | + +The identity clause exists because the dimension over the spine entity's key is the one every `@spine` report lists, a key column can never be null, and models rarely write `@required` on an `id`. It applies only under `@spine`: applying it to every report would change the shape of a report that exists today. + +The shape artifact `fixtures/persistence-conformance/report-shapes.json` keeps its format. Its existing six entries stay byte-identical; the new canonical reports append entries. + +### Table D — the view (amends Plan 2 Tables C and F) + +**With `@spine`.** `S` is the spine entity, `F` is `@from`. + +| Part | Rule | +|---|---| +| Aliases | computed exactly as for the same report without `@spine`: `F` takes `shortAliasFor(from.name)`, and every hop takes the alias the join tree gives it. No measure reference moves. | +| `FROM` | `S`'s table, under its join alias | +| The spine chain | the spine's hops walked **from `S` back to `F`**, one `LEFT OUTER JOIN` each, with the same `ON` predicate the forward join renders today. The last one introduces `F`'s table under the base alias. | +| Row scope | the report's `@segment` filter, then its `@filter`, ANDed onto the `ON` of the join that introduces `F`. **There is no `WHERE`.** | +| Onward joins | a dimension beyond the spine joins from `S`'s alias through the existing hop logic. In a `@spine` report every join is `LEFT OUTER`: the #209 `INNER` rule is not applied, so no join can drop a spine row. | +| `SELECT`, `GROUP BY` | unchanged: one column per item, grouped by each dimension's `SELECT` expression | + +The join tree has one root under `@spine` (rule R9 puts every dimension's path through it), and each node on the spine chain has exactly one child on that chain, so the chain is unambiguous. + +`meta migrate` refuses a derived `@spine` report, naming the report, in three more cases beside the existing ones: the spine entity or an entity on the chain has no table (abstract, or no writable `source.rdb`); one of them is a TPH subtype (it shares its base's table with every other subtype, so the report would get a row per row of all of them); a hop has no declared `identity.reference` (the existing hop error). + +**With `@default: n`.** `E` is the measure's full Plan 2 Table C expression, condition and cast included. + +| Measure | Postgres | SQLite / D1 | MySQL | +|---|---|---|---| +| `measure.aggregate` whose derived type is `int`, `long` or `currency` | `COALESCE(E, n)` | `COALESCE(E, n)` | `COALESCE(E, n)` | +| `measure.aggregate` whose derived type is `decimal`, `double` or `float` | `COALESCE(E, n)` | `COALESCE(E, n.0)` | `COALESCE(E, n)` | +| `measure.ratio` | `COALESCE( / NULLIF(, 0), n)` with the existing cast on `` | the same, with `n.0` | the same | + +`` and `` are the operands' full expressions **including their own `COALESCE`** when the operand declares a `@default`. SQLite gets `n.0` so a `REAL` column has one storage class in every row; without it the defaulted rows read `integer` from `typeof()` and the others `real`. + +Expected bodies for the canonical reports of Task 6, Postgres, `literal` naming (the goldens for Task 5): + +```sql +-- v_program_roster + SELECT + p."id" AS "programKey", + p."title" AS "programTitle", + COUNT(w."id") AS "weeks", + CAST(SUM(w."durationMinutes") AS BIGINT) AS "totalMinutes", + COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero", + CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0) AS "longShare", + COALESCE(CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0), 0) AS "longShareOrZero" + FROM "programs" p + LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" + GROUP BY p."id", p."title" + +-- v_program_long_weeks (report @segment: long) + SELECT + p."id" AS "programKey", + COUNT(w."id") AS "weeks", + COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero" + FROM "programs" p + LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" AND w."durationMinutes" >= 60 + GROUP BY p."id" + +-- v_fitness_totals_filled (no @spine: a default alone) + SELECT + COUNT(w."id") AS "weeks", + COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero", + COALESCE(CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0), 0) AS "longShareOrZero" + FROM "weeks" w +``` + +`v_program_roster` on SQLite differs from Postgres only as Plan 2 Table C says (no `BIGINT` cast, `CASE WHEN` for the condition, `REAL` for the ratio's cast) and in the last column's literal: + +```sql + COALESCE(CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0), 0.0) AS "longShareOrZero" +``` + +A two-hop spine (`Session` → `Week` → `Program`, spine `Session.week.program`) was run in the same shape and is the Task 5 unit case: + +```sql + FROM "programs" p + LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" + LEFT OUTER JOIN "sessions" s ON w."id" = s."weekId" AND s."minutes" >= 10 +``` + +### Table E — what a measure reads + +Executed on Postgres 16, SQLite 3.37.2 and MySQL 8.4; all three agreed on every cell, including an operand's default inside a ratio and a row scope with an `or` inside the join condition. "Empty" is any case with nothing to aggregate: a spine row with no facts, a spine row whose facts the report scope filtered out, a totals report over an empty table, or a measure `@filter` / `@segment` that matched none of a group's rows. + +| Measure | Rows to aggregate | Empty, no `@default` | Empty, `@default: 0` | +|---|---|---|---| +| `count`, `count` + `@distinct`, tuple count | the count | `0` | not legal (M7) | +| a `count` whose condition is `isNull: true` on a fact column | the count | `0` (the null-extended row has a null `@of`) | not legal (M7) | +| `sum` | the sum | null | `0` | +| `avg` | the average | null | `0` | +| `min` / `max` of a numeric field | the value | null | `0` | +| `min` / `max` of any other field | the value | null | not legal (M8) | +| `measure.ratio`, denominator not zero | the quotient | null when the numerator is null | the numerator's `@default` over the denominator when the numerator declares one; else the ratio's `@default` | +| `measure.ratio`, denominator zero or null | n/a | null | the ratio's `@default` | + +Column types with a default, read back from the engines: a defaulted integral `sum` stays `bigint` on Postgres and MySQL; a defaulted `avg` and ratio stay `numeric` / `decimal`; MySQL reports every defaulted column `NOT NULL`. Row count: the view over three programs, one with no weeks, returned three rows on every engine, and the week whose reference is null appeared in none of them. + +### Table F — what each port's row type does with it + +No generator is edited for this. Each port's detached read model sets `@required` on a derived field from the shape (`report-read-model.ts:72`, `ReportRows.cs:162`, `ReportReadModel.java:199`, `report_read_model.py:79`), and each port's existing read-only generators already type a required field as not nullable. The port tasks prove it with a test per port; they do not change it. + +| Port | A measure without `@default` (other than `count`) | The same measure with `@default` | A dimension over the spine entity's key | +|---|---|---|---| +| TypeScript | Zod `.nullable()`; row type `T \| null` | no `.nullable()`; `T` | `T` | +| C# | nullable member of the keyless row class (`long?`, `decimal?`) | `long`, `decimal` | not nullable | +| Java | nullable DTO component | as the port types a required field | as the port types a required field | +| Kotlin | nullable Exposed column and nullable data class property | `Column`, `Long` | not nullable | +| Python | `Optional[...]` on the Pydantic row model | the bare type | the bare type | + +**UNVERIFIED:** the exact spelling each generator uses for a required field of a read model (a Java record component, a Pydantic field). Each port task's first step reads the generated output for an existing `count` measure, which is `required: true` today, and uses that as the expected form. + +### Table G — filter and sort on the derived fields + +Nothing is added to a filter or sort allowlist rule: every derived field with a filter band stays filterable and sortable, by the operators of its type. + +| Case | Behaviour | +|---|---| +| The empty rows of a `@spine` report | present in the list and counted by `withCount`. `?filter[purchases][gt]=0` removes them at request time, so one report serves the page that shows them and the page that does not. | +| A filter on a defaulted measure | the default is the value: `?filter[revenue][eq]=0` matches the empty rows. `isNull=true` matches nothing; `isNull=false` matches every row. | +| A filter on a measure without a default | unchanged: an empty row is null, matched by `isNull=true` and by no comparison. | +| A sort on a defaulted measure | deterministic on every engine: the empty rows sort as their default. | +| A sort on a measure without a default | unchanged, and engine-dependent for the null rows (Postgres puts nulls last ascending; SQLite and MySQL first). The corpus does not sort a nullable measure across a null row. | +| A filter on a spine-entity column that is not listed | refused as today (`400 invalid_filter_field`). A row scope on the spine entity is listed as a dimension and filtered on the request (spec §9). | + +### Table H — fixtures and gates + +| Gate | Path | Ports | +|---|---|---| +| Registry | `fixtures/registry-conformance/expected-registry.json` (two attributes added, three sites), `coverage-report.json`, `fixtures/metamodel-docs/expected/**` | all five byte-match | +| Loader, positive | `fixtures/conformance/reporting-spine-and-default/`, `fixtures/conformance/reporting-spine-inherited/` | all five | +| Loader, errors | the eight `error-*` fixtures of Table B | all five | +| Canonical reports (model) | `fixtures/persistence-conformance/canonical/meta.fitness.json`: `ProgramRoster`, `ProgramLongWeeks`, `FitnessTotalsFilled` | all (shared input) | +| Report shapes | `fixtures/persistence-conformance/report-shapes.json` (three entries appended) | TS produces; C#, Java, Kotlin (through Java), Python byte-match | +| Canonical schema | `fixtures/persistence-conformance/canonical/schema.postgres.sql` (three views appended) | all execute it | +| `queries/report-spine-zero-rows.yaml` | a program with no weeks has a row: `weeks` `0`, `totalMinutes` null, `totalMinutesOrZero` `0`, `longShare` null, `longShareOrZero` `0`; `count` equals the number of programs; filter and sort on the defaulted measure | all five | +| `queries/report-spine-scoped.yaml` | `ProgramLongWeeks`: a program whose only weeks are short keeps its row with `weeks` `0` | all five | +| `queries/report-default-empty.yaml` | `FitnessTotalsFilled` over an empty table: `{ weeks: "0", totalMinutesOrZero: "0", longShareOrZero: "0" }`, the twin of `report-totals-empty` | all five | +| Emitter goldens | `server/typescript/packages/codegen-ts/test/projection/report-ddl-emit.test.ts`, `extract-report-spec.test.ts` (Table D) | TS | +| Idempotence and values | `server/typescript/packages/integration-tests/test/report-views-pg.test.ts`, `report-views-sqlite.test.ts`, `report-views-mysql.test.ts` | TS | +| REST | `fixtures/api-contract-conformance/report/`: `Product`, `Sale`, report `ProductRevenue`; scenarios `list-spine.yaml`, `filter-on-defaulted-measure.yaml`, `sort-on-defaulted-measure.yaml` | generated lane, all five | +| Inert | `fixtures/codegen-noop/reporting/with/`: a sourceless `@spine` report and a defaulted measure | all five inert tests | +| Metamodel version | `node scripts/check-metamodel-version.mjs` (no `--set`) | gates lane | + +The conformance corpus goes from 364 fixtures to 374 (37 concern reporting, up from 27). The persistence corpus goes from 39 scenarios to 42. The `report/` REST sub-corpus goes from 13 scenarios to 16. + +--- + +## File structure + +**Shared, modified:** `spec/metamodel/object.json`, `spec/metamodel/reporting.json`; `fixtures/registry-conformance/expected-registry.json` and `coverage-report.json`; `fixtures/metamodel-docs/expected/**`; `site-reference/**`; `fixtures/persistence-conformance/canonical/meta.fitness.json`, `canonical/schema.postgres.sql`, `report-shapes.json`, `README.md`; `fixtures/api-contract-conformance/report/{meta.json,seed.json,schema.postgres.sql,README.md}`; `fixtures/codegen-noop/reporting/with/meta.shop.json` and `README.md`. + +**Shared, new:** ten directories under `fixtures/conformance/` (Table B); three files under `fixtures/persistence-conformance/queries/`; three under `fixtures/api-contract-conformance/report/scenarios/`. + +**TypeScript, modified** (all under `server/typescript/packages/`): + +| File | Change | +|---|---| +| `metadata/src/core/object/object-constants.ts` | `OBJECT_REPORT_ATTR_SPINE` | +| `metadata/src/core/reporting/reporting-constants.ts` | `REPORTING_ATTR_DEFAULT` | +| `metadata/src/core/reporting/meta-measure.ts` | `defaultValue()` | +| `metadata/src/core/reporting/report-accessors.ts` | `reportSpine()`, `reportSpineHops()` | +| `metadata/src/core/reporting/report-shape.ts` | Table C; exports `measureDerivedSubType()` | +| `metadata/src/core/reporting/report-describe.ts` | the wording for a spine and a default | +| `metadata/src/loader/reporting-validation.ts` | R8, R9, M7, M8 | +| `metadata/src/core/{object,reporting}/*-definition.embedded.ts` | regenerated | +| `codegen-ts/src/projection/report-spec.ts` | `defaultValue` on an aggregate and a ratio; `spineDepth` on the view spec | +| `codegen-ts/src/projection/extract-report-spec.ts` | the spine path, forced `LEFT OUTER`, the three refusals, defaults | +| `codegen-ts/src/projection/report-ddl-emit.ts` | the spine `FROM` and chain, the scope in `ON`, `COALESCE` | +| `codegen-ts/src/generators/report-doc.ts`, `docs-site/src/builders/report-data.ts` | print the new wording | + +No TypeScript file is created. No route, queries, hook, entity or runtime file is edited in any port. + +**Other ports:** listed in Tasks 9 to 12. + +## Task order and parallelism + +| Task | Depends on | Can run in parallel with | +|---|---|---| +| 1 register (TS) | none | none | +| 2 loader rules (TS) | 1 | 4 | +| 3 conformance fixtures | 2 | 4, 5 | +| 4 `reportShape` (TS) | 1 | 2, 3 | +| 5 lowering (TS) | 4 | 3 | +| 6 canonical reports, artifacts, value tests | 5 | none | +| 7 persistence scenarios, TS reads, docs wording | 6 | 8 | +| 8 REST sub-corpus, TS generated lane | 6 | 7 | +| 9 C# | 3, 7, 8 | 10, 12 | +| 10 Java | 3, 7, 8 | 9, 12 | +| 11 Kotlin | 10 (shares the Java loader and `ReportShape`) | 9, 12 | +| 12 Python | 3, 7, 8 | 9, 10, 11 | +| 13 no-churn proof | 9 to 12 | 14 | +| 14 docs, skills, changelog, counts | 8 | 9 to 13 | +| 15 full CI, review, gate | all | none | + +Tasks 1 to 8 are one TypeScript track and the critical path. Each task gets a fresh implementer and a reviewer. A subagent's claim of a "pre-existing failure" is re-run on a clean build of `origin/main` before it is accepted. + +--- + +### Task 1: Declare both attributes and register them in TypeScript + +**Files:** +- Modify: `spec/metamodel/object.json` (`object.report` children), `spec/metamodel/reporting.json` (`measure.aggregate` and `measure.ratio` children) — the two entries of Table A, verbatim +- Regenerate: `server/typescript/packages/metadata/src/core/object/object-definition.embedded.ts`, `server/typescript/packages/metadata/src/core/reporting/reporting-definition.embedded.ts` +- Modify: `server/typescript/packages/metadata/src/core/object/object-constants.ts`, `core/reporting/reporting-constants.ts`, `core/reporting/meta-measure.ts`, `core/reporting/report-accessors.ts`, and the `index.ts` / constants barrel exports beside the existing reporting ones +- Regenerate: `fixtures/registry-conformance/expected-registry.json`, `fixtures/metamodel-docs/expected/**`, `site-reference/**` +- Test: `server/typescript/packages/metadata/test/reporting-registry.test.ts`, `test/object-definition-completeness.test.ts` (`REPORT_ATTRS`, line 80), `test/report-accessors` cases beside the existing ones + +**Interfaces:** +- Produces: `OBJECT_REPORT_ATTR_SPINE = "spine"`; `REPORTING_ATTR_DEFAULT = "default"`. +- Produces: `MetaMeasure.defaultValue(): number | undefined` (resolving `attr()`; the value when it is an integer, else `undefined`). +- Produces: `reportSpine(obj: MetaData): string | undefined` (resolving `attr()`). + +- [ ] **Step 1: Write the failing tests.** In `reporting-registry.test.ts`, assert through the accessor that file already uses that `object.report` has an attr `spine` of value type `string`, not required, and that `measure.aggregate` and `measure.ratio` each have an attr `default` of value type `int`, not required; keep the existing `measure.derived` assertion. In `object-definition-completeness.test.ts` add `spine: { valueType: "string", required: false }` to `REPORT_ATTRS`. Add accessor cases: `defaultValue()` is `0` for `"@default": 0`, `-1` for `-1`, `undefined` when absent; `reportSpine` returns the string. +- [ ] **Step 2: Run and see them fail.** `cd server/typescript && bun test packages/metadata/test/reporting-registry.test.ts packages/metadata/test/object-definition-completeness.test.ts` +- [ ] **Step 3: Edit the two spec files** with Table A's entries. Do not change any existing description. +- [ ] **Step 4: Constants and accessors.** + +```ts +// meta-measure.ts +/** `@default`: the integer this measure reads when it would otherwise be null (rule M7/M8 + * decide where it is legal). ADR-0039: resolving, so an inherited measure keeps it. */ +defaultValue(): number | undefined { + const v = this.attr(REPORTING_ATTR_DEFAULT); + return typeof v === "number" && Number.isInteger(v) ? v : undefined; +} + +// report-accessors.ts +/** `@spine`: the to-one path to the entity whose rows supply the report's rows. */ +export function reportSpine(obj: MetaData): string | undefined { + const v = obj.attr(OBJECT_REPORT_ATTR_SPINE); + return typeof v === "string" && v !== "" ? v : undefined; +} +``` + +- [ ] **Step 5: Regenerate.** + +```bash +bun run scripts/generate-embedded-metamodel.ts +bun run scripts/regen-expected-registry.ts +bun run scripts/regen-metamodel-docs.ts +bun scripts/build-site-reference.ts +node scripts/check-metamodel-version.mjs # passes with no --set: 1.1 stays +node scripts/check-metamodel-version.mjs --explain # every change ADDITIVE +``` + +Read the `expected-registry.json` diff by hand: three attribute entries added (one `spine`, two `default`), nothing else. `default` on a measure must print `valueType` `int`, not `null` (`null` is what a field's untyped `@default` prints). + +- [ ] **Step 6: Run.** `cd server/typescript && bun test packages/metadata/test/reporting-registry.test.ts packages/metadata/test/object-definition-completeness.test.ts packages/metadata/test/registry-conformance.test.ts packages/metadata/test/metamodel-docs-conformance.test.ts packages/metadata/test/object-definition-embed.test.ts` — PASS. `registry-coverage.test.ts` lists the two attributes as untested until Task 3; confirm the ratchet passes. +- [ ] **Step 7: Commit (local).** `git commit -m "feat(metamodel): register @spine on object.report and @default on measures (TypeScript)"`. Stage the files by name; never `git add -A`. + +--- + +### Task 2: TypeScript loader rules R8, R9, M7, M8 + +**Files:** +- Modify: `server/typescript/packages/metadata/src/loader/reporting-validation.ts` (`checkReport`, `checkMeasure`, `checkAggregateColumns`, `walkToOneVia`) +- Test: `server/typescript/packages/metadata/test/reporting-validation.test.ts` + +**Interfaces:** +- Consumes: Task 1's constants and accessors. +- Produces: nothing exported. `walkToOneVia` gains a wording argument; `checkAggregateColumns` returns the single `@of` field when rules M1 to M4 all passed, else `undefined`. + +- [ ] **Step 1: Write the failing tests,** one per Table B row plus the order rules, using the builders the file already has. Assert the code and that the message names the report or measure. Cases: a to-many spine hop (R8); a spine whose owner is another entity (R8); a spine hop that names nothing (R8); no dimensions (R9); a dimension with no `@via` (R9); a time dimension over a fact column (R9); a dimension through a second reference to the same entity (R9); the same join written with the relationship name on one side and the reference name on the other (R9, Review Focus 5); a legal time dimension over a spine-entity column; a two-hop spine with a dimension beyond it; `@default` on a `count` (M7); on a `max` of a timestamp (M8); on a `sum`, an `avg`, a `min` of an int and a ratio (clean); a measure that breaks M4 **and** declares `@default` reports M4 only; a report whose `@spine` fails R8 does not also report R9; an inherited spine (owner written as the abstract base). +- [ ] **Step 2: Run and see them fail.** `cd server/typescript && bun test packages/metadata/test/reporting-validation.test.ts` +- [ ] **Step 3: Implement.** In `checkReport`, after R7 and before the `@filter` check: + +```ts + // R8 — @spine is a to-one path from @from: rule D2's walk, started at @from. + const spine = reportSpine(report); + if (spine !== undefined) { + const terminal = walkToOneVia( + { root, host: from, declaring: report, label, suffix: "", sink }, + spine, + (message) => err(`: ${message}`), + { attr: OBJECT_REPORT_ATTR_SPINE, start: `@from '${fromKey}'` }, + ); + // R9 — every listed dimension is reached through the spine. Skipped when R8 failed. + if (terminal !== undefined) { + const spineHops = splitDotted(spine)?.path ?? []; + const items = reportDimensionItems(report); + if (items.length === 0) { + err( + `: @spine '${spine}' needs at least one dimension. The report's rows are the dimension tuples of ` + + `'${terminal.resolutionKey()}'; with no dimension it would be one totals row.`, + ); + } + for (const item of items) { + const dim = childOfType(from, TYPE_DIMENSION, item.name); + if (!(dim instanceof MetaDimension)) continue; // R2 already reported it + const via = dim.via(); + const hops = via === undefined ? undefined : splitDotted(via)?.path; + if (hops !== undefined && spineHops.every((h, i) => hops[i] === h)) continue; + err( + via === undefined + ? `: dimension '${item.name}' is read from @from '${fromKey}', so it has no value in a row that has ` + + `no facts. With @spine '${spine}' every dimension must be reached through it: declare the ` + + `dimension over a field of '${terminal.resolutionKey()}' (or an entity to-one from it) with an ` + + `@via that begins '${spine}'.` + : `: dimension '${item.name}' is reached by @via '${via}', which does not begin with the hops of ` + + `@spine '${spine}'. Hop names are compared as written; write the same hops.`, + ); + } + } + } +``` + +`walkToOneVia`'s fourth argument replaces the two places it says `@via` and "the owning entity '…'". Its default is the present wording, and **every existing D2 message stays byte-identical** (the existing tests and the ports assert them). + +In `checkMeasure`, after `checkAggregateColumns`: + +```ts + // M7 / M8 — where a @default can apply. Only when M1–M4 passed: one mistake, one error. + if (clean && measure.attr(REPORTING_ATTR_DEFAULT) !== undefined) { + const agg = measure.agg(); + if (agg === AGG_COUNT) { + err(`@default cannot apply to @agg: count. A count is never null (it is 0 when nothing matches); remove @default.`); + } else if ((agg === AGG_MIN || agg === AGG_MAX) && ofField !== undefined && !NUMERIC_FIELD_SUBTYPES.includes(ofField.subType)) { + err( + `@default is a number, but @agg '${agg}' of '${measure.ofColumns()[0]}' is a field.${ofField.subType}. ` + + `A default is supported on numeric measures only.`, + ); + } + } +``` + +`clean` and `ofField` come from the changed return of `checkAggregateColumns`. The presence test reads the raw attribute, not `defaultValue()`, so a mistyped value on a `count` still reports M7 beside the type error. + +- [ ] **Step 4: Run.** `cd server/typescript && bun test packages/metadata` — PASS, every existing loader test included. +- [ ] **Step 5: Commit (local).** `git commit -m "feat(metadata): validate @spine and a measure @default at load (TypeScript)"`. + +--- + +### Task 3: Shared conformance fixtures + +**Files:** +- Create: `fixtures/conformance/reporting-spine-and-default/{input/meta.shop.json,expected.json,providers.json}` +- Create: `fixtures/conformance/reporting-spine-inherited/…` +- Create: the eight `fixtures/conformance/error-*/{input/…,expected-errors.json,providers.json}` of Table B +- Modify: `fixtures/registry-conformance/coverage-report.json`; `fixtures/conformance/CAPABILITIES.json` +- Modify (counts, 364 → 374): `AGENTS.md:78`, `README.md:264`, `docs/CONFORMANCE.md` (the metamodel row, its `###` heading, the totals line, and the reporting fixture-prefix row), `examples/showcase/site-payload.json` + +- [ ] **Step 1: Write the positive input,** package `acme::shop`, `providers.json` = `["metaobjects-core-types","metaobjects-db"]`: + - `Catalog` (`id`, `name` required), `Program` (`id`, `title` required, `publishedAt` timestamp, `catalogId`, `identity.reference` `fkCatalog`, `relationship.association` `catalog` to-one), `Purchase` (`id`, `programId`, `customerEmail`, `amountCents` currency, `status`, `purchasedAt`, `identity.reference` `fkProgram`, `relationship.association` `program` to-one). + - On `Purchase`: segment `active`; dimensions `programId` (`Program.id` via `Purchase.program`), `programTitle`, `publishedAt` (time, `Program.publishedAt` via `Purchase.program`, grains `month, year`), `purchasedAt` (time, on the fact, grains `day, month`), `programByRef` (`Program.title` via `Purchase.fkProgram`), `catalogName` (`Catalog.name` via `Purchase.program.catalog`); measures `purchases` (count), `revenue` (sum, `@default: 0`), `avgAmount` (avg, `@default: 0`), `smallest` (min of `amountCents`, `@default: -1`), `lastPurchaseAt` (max of `purchasedAt`, no default), `revenuePerPurchase` (ratio, `@default: 0`). + - Reports: `ProgramSales` (`@spine: "Purchase.program"`, dimensions `programId, programTitle, publishedAt:month, catalogName`, every measure, `@segment: active`, a `source.rdb` view); `CatalogSales` (`@spine: "Purchase.program.catalog"`, dimension `catalogName`, measures `purchases, revenue`); `SalesByDay` (no spine: `purchasedAt:day`, `revenue`), so a default without a spine is in the fixture. +- [ ] **Step 2: Produce `expected.json`** from the TypeScript canonical serializer as `spec/conformance-tests.md` describes, and read it by hand: `@spine` is a string, each `@default` is a JSON integer (`-1` included). +- [ ] **Step 3: `reporting-spine-inherited`:** an abstract `BaseEvent` declaring the reference, the two dimensions and a defaulted `sum`; `WorkoutEvent extends BaseEvent`; a report over `WorkoutEvent` with `@spine: "BaseEvent.program"`. +- [ ] **Step 4: Each error fixture** is the positive input with one change, and `expected-errors.json` in the format of `fixtures/conformance/error-report-segment-unresolved/expected-errors.json`: + +| Fixture | The one change | Code, on | +|---|---|---| +| `error-report-spine-to-many` | add a to-many relationship, declared as `error-dimension-via-to-many` declares its own, and point `@spine` at it | `ERR_INVALID_REPORT`, the report | +| `error-report-spine-not-from` | `@spine: "Program.catalog"` on `ProgramSales` | `ERR_INVALID_REPORT`, the report | +| `error-report-spine-no-dimensions` | remove `@dimensions` from `CatalogSales` | `ERR_INVALID_REPORT`, the report | +| `error-report-spine-dimension-off-spine` | add `purchasedAt:day` to `ProgramSales` | `ERR_INVALID_REPORT`, the report | +| `error-report-spine-dimension-other-path` | add `programByRef` to `ProgramSales` | `ERR_INVALID_REPORT`, the report | +| `error-measure-default-on-count` | `@default: 0` on `purchases` | `ERR_INVALID_MEASURE`, the measure | +| `error-measure-default-non-numeric` | `@default: 0` on `lastPurchaseAt` | `ERR_INVALID_MEASURE`, the measure | +| `error-measure-default-not-integer` | `revenue` `@default: 0.5` | `ERR_BAD_ATTR_VALUE`, the measure | + +**UNVERIFIED:** that a fractional number on an `attr.int` is `ERR_BAD_ATTR_VALUE` in TypeScript (the existing `error-attr-wrong-type` fixture covers a string on an `attr.int`, not a fraction). Run it first. If TypeScript accepts `0.5`, tighten its integer check in the same task and say so in the commit; every other port then meets the fixture in its own task. + +- [ ] **Step 5: Run.** `cd server/typescript && bun test packages/metadata/test/conformance.test.ts`. A jsonPath mismatch means the error is attached to the wrong node: fix the validator's `source`, not the fixture. +- [ ] **Step 6: Counts and coverage.** + +```bash +bun server/typescript/packages/conformance/bin/conformance.ts manifest fixtures/conformance +bun run site:payload && bun scripts/build-site-payload.ts --check +cd server/typescript && MO_UPDATE_COVERAGE_SNAPSHOT=1 bun test packages/metadata/test/registry-coverage.test.ts +``` + +Edit the four count sites to 374 first. `AGENTS.md` is loaded into every agent session: change the number and nothing else. + +- [ ] **Step 7: Commit (local).** `git commit -m "test(conformance): @spine and measure @default fixtures (2 positive, 8 error cases)"`. The other ports' conformance lanes are red from here until their own tasks. + +--- + +### Task 4: `reportShape` — Table C in TypeScript + +**Files:** +- Modify: `server/typescript/packages/metadata/src/core/reporting/report-shape.ts`, `core/reporting/report-accessors.ts` +- Test: `server/typescript/packages/metadata/test/report-shape.test.ts`, `test/report-read-model.test.ts` + +**Interfaces:** +- Produces: `reportSpineHops(report: MetaObject, from: MetaObject, root: MetaRoot): string[] | undefined` — the hop names of `@spine`, read as `reportingViaHops` reads a `@via` (owner resolved in the report's package, must be `from` or an entity it extends). `undefined` when there is no `@spine`. +- Produces: `measureDerivedSubType(measure: MetaMeasure, from: MetaObject, root: MetaRoot): string` — Table B's subtype for one measure, listed in a report or not. `measureField` calls it; Task 5 calls it for a ratio operand. +- `ReportShape` and `ReportField` keep their members. `required` changes value only as Table C says. + +- [ ] **Step 1: Write the failing tests:** each Table C row, with an inline model. Include: a dimension over the spine entity's `id` (no `@required`, a member of `identity.primary`) is required; over a `@required` title is required; over an optional column is not; a dimension one hop beyond the spine is not, even when its field is `@required`; the **same dimensions in a report without `@spine`** keep today's values (the `id` one is `false`); a `sum`, an `avg`, a `min` and a ratio with `@default` are required, without it are not; `count` is required either way. In `report-read-model.test.ts`: a defaulted measure's detached field has `@required: true`. +- [ ] **Step 2: Run and see them fail.** `cd server/typescript && bun test packages/metadata/test/report-shape.test.ts packages/metadata/test/report-read-model.test.ts` +- [ ] **Step 3: Implement.** In `dimensionField`: + +```ts + const spine = reportSpineHops(report, from, root); + let required: boolean; + if (spine === undefined) { + required = vialess && of.attr(FIELD_ATTR_REQUIRED) === true; // unchanged + } else { + const via = dim.via(); + const hops = via === undefined ? undefined : reportingViaHops(via, reportingMemberOwner(dim, from), from, root); + const onSpine = hops !== undefined && hops.length === spine.length && hops.every((h, i) => h === spine[i]); + required = onSpine && (of.attr(FIELD_ATTR_REQUIRED) === true || isPrimaryKeyField(named, of)); + } +``` + +`named` is the entity `@of` names (the object `resolveReportingFieldRef` already resolves; return it beside the field). `isPrimaryKeyField(named, of)` is true when `of.name` is in the `@fields` of `named`'s `identity.primary`, read through resolving accessors (ADR-0039: an identity inherited from an abstract base counts). In `measureField`, every non-`count` return becomes `required: m.defaultValue() !== undefined`. + +- [ ] **Step 4: Run.** `cd server/typescript && bun test packages/metadata` — PASS. `report-shapes.json` is not regenerated here; `report-shapes-artifact.test.ts` must still pass unchanged, which is the no-churn proof for the six existing shapes. +- [ ] **Step 5: Commit (local).** `git commit -m "feat(metadata): a spine key and a defaulted measure are not nullable in a report's shape"`. + +--- + +### Task 5: Lowering — Table D in TypeScript + +**Files:** +- Modify: `server/typescript/packages/codegen-ts/src/projection/report-spec.ts`, `extract-report-spec.ts`, `report-ddl-emit.ts` +- Test: `server/typescript/packages/codegen-ts/test/projection/extract-report-spec.test.ts`, `report-ddl-emit.test.ts` + +**Interfaces:** +- `ReportAggregate` gains `readonly defaultValue?: { readonly value: number; readonly real: boolean }` (`real`: the derived subtype is `decimal`, `double` or `float`). +- The `ratio` arm of `ReportColumn` gains `readonly defaultValue?: number` (a ratio is always `real`). +- `ReportViewSpec` gains `readonly spineDepth?: number`: how many hops of the join tree's single root chain are the spine. Absent for a report without `@spine`. +- `spec.where` keeps its meaning (the report scope); the emitter decides where it is written. + +- [ ] **Step 1: Write the failing tests.** Extract: a spine report yields `spineDepth`, one root join, every join `left`; aliases equal those of the same report with `@spine` removed; the three refusals of Table D each throw naming the report; a spine hop with no `identity.reference` throws the existing hop error; an aggregate and a ratio carry `defaultValue`; an operand that is not listed in `@measures` carries its own. Emit, per dialect: the three bodies of Table D as goldens; `the report scope is in the join condition` (a spine report with `@segment` and `@filter` has no `WHERE`, and an `or` scope is parenthesised inside the `ON`); the two-hop chain; an onward join from the spine alias is `LEFT OUTER` even over a required reference; `COALESCE` wraps the cast (`COALESCE(CAST(SUM(x) AS BIGINT), 0)`); a negative default (`-1`, and `-1.0` on SQLite for a `real` measure); a defaulted operand inside a ratio; a one-to-one hop whose reference is held by the far entity, reversed. And **every existing golden in both files passes unedited**. +- [ ] **Step 2: Run and see them fail.** `cd server/typescript && bun test packages/codegen-ts/test/projection/extract-report-spec.test.ts packages/codegen-ts/test/projection/report-ddl-emit.test.ts` +- [ ] **Step 3: `extractReportSpec`.** After the existing `@from` checks: read `reportSpineHops`; walk it with the existing `walkViaPath([from.name, ...hops].join("."), root, packageOf(from), ctx)` and require the whole chain (the existing `viaHopError` otherwise); for each step's target entity, refuse one with no table (`isAbstract || !hasWritableRdbSource`) and one that `isTphSubtype`, naming the report, the spine and the entity. Pass `[spinePath, ...dimensionPaths]` to `pathsToJoins`, then set `joinType: "left"` on every node of the tree. `aggregateOf` adds `defaultValue` from `measure.defaultValue()` and `measureDerivedSubType`. Set `spineDepth: spinePath.length`. +- [ ] **Step 4: `emitReportViewDdl`.** `aggregate()` ends with: + +```ts +function withDefault(sql: string, dv: { value: number; real: boolean } | undefined, d: ReportDialect): string { + if (dv === undefined) return sql; + // SQLite: a REAL column keeps one storage class, so its default is a REAL literal. + return `COALESCE(${sql}, ${d === "sqlite" && dv.real ? `${dv.value}.0` : String(dv.value)})`; +} +``` + +The ratio arm wraps its whole quotient the same way with `real: true`. For the `FROM`, when `spec.spineDepth !== undefined`: + +```ts + // The spine chain: the single root, then its child on the chain, down to the spine entity. + const chain: JoinNode[] = []; + for (let n = spec.joinTree.joins[0]; n !== undefined && chain.length < spec.spineDepth; n = n.children[0]) chain.push(n); + const spine = chain[chain.length - 1]!; + let sql = ` FROM ${q(tableOf(spine.targetEntity), d)} ${spine.alias}`; + for (let i = chain.length - 1; i >= 0; i--) { + const parentAlias = i === 0 ? spec.joinTree.baseAlias : chain[i - 1]!.alias; + const parentTable = i === 0 ? options.baseTableName : tableOf(chain[i - 1]!.targetEntity); + // The row scope rides on the join that introduces @from: it filters facts, never spine rows. + const scope = i === 0 && spec.where !== undefined ? ` AND ${cond(spec.where, d)}` : ""; + sql += `\n LEFT OUTER JOIN ${q(parentTable, d)} ${parentAlias} ON ${onPredicate(chain[i]!, parentAlias, d)}${scope}`; + } + for (const child of spine.children) sql += "\n" + renderJoin(child, spine.alias, options); +``` + +`onPredicate` is the `ON` text `renderJoin` builds today, extracted so both callers share it; `tableOf` is its table lookup with the same "no table registered" error. No `WHERE` is written in this branch. The branch without `spineDepth` is the present code, untouched. + +- [ ] **Step 5: Run.** `cd server/typescript && bun test packages/codegen-ts/test/projection` — PASS. +- [ ] **Step 6: Commit (local).** `git commit -m "feat(codegen-ts): lower a report @spine and a measure @default to view SQL"`. + +--- + +### Task 6: Canonical reports, artifacts, value and idempotence tests + +**Files:** +- Modify: `fixtures/persistence-conformance/canonical/meta.fitness.json` +- Regenerate: `fixtures/persistence-conformance/canonical/schema.postgres.sql` (`bun run gen:schema`), `fixtures/persistence-conformance/report-shapes.json` (`bun run gen:report-shapes`), both from `server/typescript/packages/integration-tests` +- Modify: `server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts` (the hard-coded "six", line 39), `report-views-pg.test.ts`, `report-views-sqlite.test.ts`, `report-views-mysql.test.ts` + +- [ ] **Step 1: Extend the canonical model.** Append to `Week`'s children, after the last existing member: + +```json +{ "dimension.attribute": { "name": "programKey", "@of": "Program.id", "@via": "Week.fkProgram" } }, +{ "measure.aggregate": { "name": "totalMinutesOrZero", "@agg": "sum", "@of": "Week.durationMinutes", "@default": 0 } }, +{ "measure.ratio": { "name": "longShareOrZero", "@numerator": "longWeeks", "@denominator": "weeks", "@default": 0 } } +``` + +Append after `AssetActivity`, the last report: + +```json +{ "object.report": { "name": "ProgramRoster", "@from": "Week", "@spine": "Week.fkProgram", + "@dimensions": ["programKey", "programTitle"], + "@measures": ["weeks", "totalMinutes", "totalMinutesOrZero", "longShare", "longShareOrZero"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_roster" } } ] } }, +{ "object.report": { "name": "ProgramLongWeeks", "@from": "Week", "@spine": "Week.fkProgram", + "@dimensions": ["programKey"], "@measures": ["weeks", "totalMinutesOrZero"], "@segment": "long", + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_long_weeks" } } ] } }, +{ "object.report": { "name": "FitnessTotalsFilled", "@from": "Week", + "@measures": ["weeks", "totalMinutesOrZero", "longShareOrZero"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_fitness_totals_filled" } } ] } } +``` + +No existing node is edited. + +- [ ] **Step 2: Regenerate both artifacts and read the diffs.** `schema.postgres.sql`: three views appended with their fingerprint comments, bodies equal to Table D, no existing line changed. `report-shapes.json`: three entries appended; `ProgramRoster` reads `programKey` long required, `programTitle` string required, `weeks` required, `totalMinutes` not, `totalMinutesOrZero` required, `longShare` not, `longShareOrZero` required. +- [ ] **Step 3: Value tests, hand-computed.** Seed: program 1 with weeks of 30, 60, 90 and 60 minutes; program 2 with one week of 45; program 3 with none. + +| View | Expected rows | +|---|---| +| `v_program_roster` | `(1, Foundations, 4, 240, 240, 0.75, 0.75)`, `(2, Strength, 1, 45, 45, 0, 0)`, `(3, , 0, null, 0, null, 0)` | +| `v_program_long_weeks` | `(1, 3, 210)`, `(2, 0, 0)`, `(3, 0, 0)` | +| `v_fitness_totals_filled`, empty `weeks` | `(0, 0, 0)` | + +Add to `report-views-pg.test.ts` and `report-views-sqlite.test.ts`: those three; `the view has one row per program` (its row count equals `SELECT count(*) FROM programs`); `an isNull condition does not count the empty row` (inline model: a `count` and a `sum` whose `@filter` is `{ label: { isNull: true } }` read `0` and null for program 3); `with @spine a fact whose reference is null is in no row` (inline model with a nullable reference, the twin of the existing "NULL group" test); `a two-hop spine`; column types of the defaulted columns (`bigint` and `numeric` on Postgres; one `typeof()` per column on SQLite). Update the convergence tests from six views to nine: migrate from empty, then a second and third migrate propose nothing. +- [ ] **Step 4: MySQL.** In `report-views-mysql.test.ts`, the roster rows through `buildReportViews(root, { dialect: "mysql" })`, and that the defaulted columns are `NOT NULL` in `information_schema`. +- [ ] **Step 5: Run.** `cd server/typescript && bun test packages/integration-tests/test/report-shapes-artifact.test.ts packages/integration-tests/test/schema-artifact.test.ts packages/integration-tests/test/report-views-sqlite.test.ts packages/integration-tests/test/report-views-pg.test.ts packages/integration-tests/test/report-views-mysql.test.ts` — PASS. The Postgres and MySQL files need a database (`METAOBJECTS_TEST_PG_URL` / `METAOBJECTS_TEST_MYSQL_URL`, or the docker CLI); run them one at a time on this machine. +- [ ] **Step 6: Commit (local).** `git commit -m "test(persistence): canonical @spine and @default reports, value and idempotence tests"`. C#'s committed row classes and Kotlin's reference tables are stale from here until Tasks 9 and 11. + +--- + +### Task 7: Shared persistence scenarios, the TypeScript read, and the docs wording + +**Files:** +- Create: `fixtures/persistence-conformance/queries/report-spine-zero-rows.yaml`, `report-spine-scoped.yaml`, `report-default-empty.yaml` +- Modify: `fixtures/persistence-conformance/README.md` (the scenario list and count) +- Modify: `fixtures/codegen-noop/reporting/with/meta.shop.json` and `README.md` +- Modify: `server/typescript/packages/metadata/src/core/reporting/report-describe.ts`, `server/typescript/packages/codegen-ts/src/generators/report-doc.ts`, `server/typescript/packages/docs-site/src/builders/report-data.ts` +- Test: `server/typescript/packages/integration-tests/test/query.test.ts` (runner, expected unchanged), `metadata/test/report-describe.test.ts` (the described-attr lists, lines 190 to 191), `docs-site/test/reporting-site.test.ts`, `docs-site/test/coverage.test.ts`, `cli/test/unit/reporting-inert.test.ts` + +- [ ] **Step 1: Write the scenarios** in the format of `report-grouped-measures.yaml`, seed inline. Wire values follow the column types as in that file (a `bigint` is a string, a decimal a canonical decimal string): + +```yaml +name: report-spine-zero-rows +queries: + - name: every-program-has-a-row + op: list + entity: ProgramRoster + sort: [{ field: programKey, dir: asc }] + expect: + - { programKey: "1", programTitle: "Foundations", weeks: "4", totalMinutes: "240", totalMinutesOrZero: "240", longShare: "0.75", longShareOrZero: "0.75" } + - { programKey: "2", programTitle: "Strength", weeks: "1", totalMinutes: "45", totalMinutesOrZero: "45", longShare: "0", longShareOrZero: "0" } + - { programKey: "3", programTitle: "Mobility", weeks: "0", totalMinutes: null, totalMinutesOrZero: "0", longShare: null, longShareOrZero: "0" } + - name: count-is-the-number-of-programs + op: count + entity: ProgramRoster + expect: 3 + - name: the-default-is-filterable + op: list + entity: ProgramRoster + filter: { totalMinutesOrZero: { eq: 0 } } + expect: [ <the program 3 row> ] + - name: a-count-filter-drops-the-empty-rows + op: count + entity: ProgramRoster + filter: { weeks: { gt: 0 } } + expect: 2 + - name: sort-on-the-defaulted-measure + op: list + entity: ProgramRoster + sort: [{ field: totalMinutesOrZero, dir: asc }] + limit: 1 + expect: [ <the program 3 row> ] +``` + +`report-spine-scoped.yaml` lists `ProgramLongWeeks` (the Task 6 rows). `report-default-empty.yaml` lists `FitnessTotalsFilled` over an empty `weeks` table and expects `{ weeks: "0", totalMinutesOrZero: "0", longShareOrZero: "0" }`. No scenario sorts a nullable measure across a null row (Table G). +- [ ] **Step 2: Run the TypeScript runner.** `cd server/typescript && bun test packages/integration-tests/test/query.test.ts` — the three new scenarios PASS with no runtime change. If one fails, the defect is in Tasks 4 to 6, not in the runtime. +- [ ] **Step 3: The inert model.** Add to `fixtures/codegen-noop/reporting/with/meta.shop.json` a `@default` on one existing non-count measure, one dimension reached by `@via`, and one **sourceless** report with `@spine`. Run `cd server/typescript && bun test packages/cli/test/unit/reporting-inert.test.ts` — PASS unchanged: the generated tree still equals `without/`. +- [ ] **Step 4: The wording.** `report-describe.ts` is the one home for what a report page says. Add, with tests first: a defaulted measure's description ends `; \`0\` when there is nothing to aggregate` (the declared integer), and a report with `@spine` gains a rows sentence, `one row per distinct dimension tuple among the rows of \`Program\`, reached by \`Purchase.program\`, including those no \`Purchase\` refers to`, and its row scope reads `aggregating only <scope>` instead of `scoped to <scope>`. `report-doc.ts` and `report-data.ts` print them. Update the described-attr lists in `report-describe.test.ts` and whatever `docs-site/test/coverage.test.ts` and `reporting-site.test.ts` need so both attributes count as rendered. + +**UNVERIFIED:** the exact present phrases in `report-describe.ts` for a row scope, and the docs goldens that pin them. Read the file and its test before writing the new sentences; keep every existing sentence byte-identical for a report that uses neither attribute. + +- [ ] **Step 5: Run.** `cd server/typescript && bun test packages/metadata/test/report-describe.test.ts packages/docs-site/test packages/codegen-ts/test/generators` — PASS. +- [ ] **Step 6: Commit (local).** `git commit -m "test(persistence): @spine and @default read scenarios; describe them in meta docs"`. + +--- + +### Task 8: The REST sub-corpus and the TypeScript generated lane + +**Files:** +- Modify: `fixtures/api-contract-conformance/report/meta.json`, `seed.json`, `README.md` +- Regenerate: `fixtures/api-contract-conformance/report/schema.postgres.sql` (`bun run gen:report-api-schema` from `server/typescript/packages/integration-tests`) +- Create: `fixtures/api-contract-conformance/report/scenarios/list-spine.yaml`, `filter-on-defaulted-measure.yaml`, `sort-on-defaulted-measure.yaml` +- Modify: `server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts` (the hard-coded "thirteen", line 48), `api-contract-report.test.ts` (the report → route table) + +- [ ] **Step 1: Extend the model** by appending, never editing: entities `Product` (`id`, `name` required, primary identity) and `Sale` (`id`, `productId` required, `amountCents` long required, `identity.reference` `fkProduct`), with on `Sale` the dimensions `productId` (`Product.id` via `Sale.fkProduct`) and `productName`, the measures `sales` (count), `revenueCents` (sum) and `revenueOrZero` (sum, `@default: 0`), and the report `ProductRevenue` (`@from: Sale`, `@spine: "Sale.fkProduct"`, dimensions `productId, productName`, the three measures, view `v_product_revenue`). The `Invoice` entity, its four reports and their views are untouched. +- [ ] **Step 2: Seed.** `seed.json` gains `products` (three, the third with no sale), `sales`, and `reports.ProductRevenue`: + +```json +[ { "productId": 1, "productName": "Atlas", "sales": 2, "revenueCents": 30000, "revenueOrZero": 30000 }, + { "productId": 2, "productName": "Beacon", "sales": 1, "revenueCents": 4500, "revenueOrZero": 4500 }, + { "productId": 3, "productName": "Cinder", "sales": 0, "revenueCents": null, "revenueOrZero": 0 } ] +``` + +- [ ] **Step 3: Scenarios,** using existing assertion keys only: + - `list-spine.yaml`: `GET /api/product_revenues?sort=productId:asc` → the three rows; `withCount=1` → `total: 3`. + - `filter-on-defaulted-measure.yaml`: `?filter[revenueOrZero][eq]=0` → the `Cinder` row; `?filter[revenueCents][isNull]=true` → the same row; `?filter[sales][gt]=0&sort=productId:asc` → the other two. + - `sort-on-defaulted-measure.yaml`: `?sort=revenueOrZero:asc&limit=1` → the `Cinder` row; `?sort=revenueOrZero:desc&limit=1` → the `Atlas` row. + +**UNVERIFIED:** the served segment for `ProductRevenue`. Read it from the generated route file; do not assume `/api/product_revenues`. + +**UNVERIFIED:** how each lane's harness loads the base tables of `seed.json` (the present file has one, `invoices`). If a harness names `invoices`, it learns to load every top-level key but `reports`, parents first (`products` before `sales`). + +- [ ] **Step 4: Run the TypeScript lane.** `cd server/typescript && bun test packages/integration-tests/test/api-contract-report-corpus.test.ts packages/integration-tests/test/api-contract-report.test.ts` and the generated-lane test that runs `report/` — sixteen scenarios PASS, and the test that holds `seed.json`'s `reports` equal to what the views return passes for `ProductRevenue`. No generator is edited: if the generated read schema types `revenueOrZero` nullable, the defect is in Task 4. +- [ ] **Step 5: Commit (local).** `git commit -m "test(api-contract): a @spine report with a defaulted measure in the report/ sub-corpus"`. + +--- + +### Task 9: C# port + +**Files:** +- Modify (byte copies): `server/csharp/MetaObjects/SpecMetamodel/reporting.json`, `SpecMetamodel/object.json` +- Modify: `server/csharp/MetaObjects/Core/Reporting/ReportingConstants.cs`, `Core/Object/ObjectConstants.cs`, `Core/Reporting/ReportingSchema.cs` (`MeasureAggregateAttrs`, `MeasureRatioAttrs`, `ReportAttrs`), `Meta/MetaMeasure.cs`, `Core/Reporting/ReportAccessors.cs`, `Core/Reporting/ReportShape.cs` (`DimensionField`, `MeasureField`), `Loader/ValidationPasses.Reporting.cs` (`WalkToOneVia`, `CheckMeasure`, `CheckAggregateColumns`, `CheckReport`) +- Regenerate: `server/csharp/MetaObjects.IntegrationTests/Generated/` (three new row classes and `AppDbContext.g.cs`) +- Modify: `server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs` and `ReportGeneratedServerFactory.cs` if they name the `invoices` table +- Test: `server/csharp/MetaObjects.Conformance.Tests/{ReportingValidationTests,ReportingAccessorsTests,ReportShapeTests}.cs` (the hard-coded "six", line 48), `server/csharp/MetaObjects.Codegen.Tests/ReportRowCodegenTests.cs` + +- [ ] **Step 1:** Copy the two JSON files; run `dotnet test server/csharp --filter "FullyQualifiedName~Conformance"` — expected FAIL: `SpecMetamodelEmbedTests` passes, the registry manifest and the ten new fixtures fail. +- [ ] **Step 2: Register** the two attributes in `ReportingSchema.cs`; add the constants and the accessors (`DefaultValue`, `ReportSpine`). Registry-conformance PASS. +- [ ] **Step 3: Port R8, R9, M7, M8** rule for rule from `reporting-validation.ts`, with the same message texts. Add the unit cases of Task 2 Step 1 to `ReportingValidationTests.cs`. +- [ ] **Step 4: Port Table C** into `ReportShape.cs`. C# has no `reportingViaHops`: add the hop split (the owner ends at the first `.` after the last `::`) beside the existing field-reference resolvers. `ReportShapeTests` byte-matches the regenerated `report-shapes.json`. +- [ ] **Step 5: Row types.** First read the generated member for an existing `count` (required today). Then add to `ReportRowCodegenTests.cs`: a defaulted `sum` is `long`, not `long?`; a defaulted ratio is `decimal`; the spine key is not nullable; the same measure without `@default` stays nullable. Regenerate `Generated/` and confirm `IntegrationFixtureDriftTests` passes with only the three new files and the `AppDbContext` additions. +- [ ] **Step 6: Lanes.** `dotnet test server/csharp` — the persistence query lane (three new scenarios), the `report/` REST lane (sixteen) and `ReportingInertTests` PASS. +- [ ] **Step 7: Commit (local).** `git commit -m "feat(csharp): @spine and measure @default"`. + +--- + +### Task 10: Java port + +**Files** (under `server/java/`): +- Modify: `metadata/src/main/java/com/metaobjects/reporting/ReportingConstants.java`, `object/MetaObject.java` (the `object.report` registration, lines 250 to 263, and an `ATTR_REPORT_SPINE` constant), `reporting/AggregateMeasure.java`, `reporting/RatioMeasure.java` (registration), `reporting/MetaMeasure.java`, `reporting/ReportAccessors.java`, `reporting/ReportShape.java` (`dimensionField`, `measureField`), `loader/ReportingValidation.java` (`walkToOneVia`, `checkMeasure`, `checkReport`) +- Modify: `metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java` (see the trap below) +- Modify: `integration-tests/src/test/java/com/metaobjects/integration/api/ReportCorpus.java` and `generated/InMemoryReportRepositorySource.java` if they name `InvoiceStatusTotals` and its siblings one by one +- Test: `metadata/src/test/java/com/metaobjects/loader/ReportingValidationTest.java` (its fixture-to-code map, lines 150 to 176), `reporting/ReportingTest.java`, `reporting/ReportShapeTest.java`, `reporting/ReportReadModelTest.java`, `codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringReportRestSurfaceTest.java`, `omdb/src/test/java/com/metaobjects/manager/db/ReportReadTest.java` + +**The trap.** `RegistryManifest.java` prints `valueType: null` for any attribute **named** `default` (lines 267 and 635 to 637), because a field's `@default` is registered as a string and re-inferred from its text. A measure's `default` is registered as an `int` and must print `int`. Narrow the special case to the field registration, by owner type and name, not by name alone. The registry-conformance test is what catches it. + +- [ ] **Step 1:** `cd server/java && mvn -q -pl metadata -am test -Dtest='*Registry*Conformance*,*Conformance*'` — expected FAIL (the spec JSON is copied at build, so the manifest and the ten fixtures fail at once). Use `MAVEN_ARGS` for a repo override, never `MAVEN_OPTS`. +- [ ] **Step 2: Register** both attributes; fix the manifest special case; add the accessors. Registry-conformance PASS. +- [ ] **Step 3: Port R8, R9, M7, M8** with the same message texts; extend the fixture-to-code map and the unit cases. +- [ ] **Step 4: Port Table C** into `ReportShape.java` (add the hop split, as in C#). `ReportShapeTest` byte-matches `report-shapes.json`. +- [ ] **Step 5: Row types and reads.** A DTO component is boxed and `@NotNull` is the marker (`SpringDtoGenerator.java`, `validationAnnotations`): assert it is present on a defaulted measure and the spine key and absent on the same measure without `@default`. `ReportReadTest` reads a `@spine` view with an empty row through OMDB. +- [ ] **Step 6: Lanes.** `mvn -q -pl metadata,omdb,codegen-spring,integration-tests -am test` — persistence (three new scenarios), `report/` REST (sixteen) and `ReportingInertTest` PASS. +- [ ] **Step 7: Commit (local).** `git commit -m "feat(java): @spine and measure @default"`. + +--- + +### Task 11: Kotlin port + +Kotlin has no loader and no report-shape code of its own: it calls Java's (`KotlinExposedTableGenerator.kt:392`, `KotlinGenUtil.kt:683`). So this task is tests and reference tables. + +**Files** (under `server/java/`): +- Create: `integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/{ProgramRoster,ProgramLongWeeks,FitnessTotalsFilled}View.kt`, hand-written like the six beside them, with nullability spelled out per Table C +- Modify: the Kotlin `report/` lane's seeded repositories under `integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/` if they name reports one by one +- Test: `codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/{KotlinReportTableGeneratorTest,KotlinReportRestSurfaceTest,ReportingInertTest,RegistryManifestConformanceTest}.kt`, `integration-tests-kotlin/.../KotlinCodegenMatchesReferenceTest.kt`, `QueryScenarioRunner.kt` + +- [ ] **Step 1:** Add to `KotlinReportTableGeneratorTest`: a defaulted measure's Exposed column has no `.nullable()` and its data class property is not nullable; the spine key likewise; the same measure without `@default` keeps `.nullable()`. +- [ ] **Step 2:** Write the three reference tables; `KotlinCodegenMatchesReferenceTest` PASS. +- [ ] **Step 3: Lanes.** `mvn -q -pl codegen-kotlin,integration-tests-kotlin -am test` — registry manifest, persistence (three new scenarios), `report/` REST (sixteen), inert PASS. +- [ ] **Step 4: Commit (local).** `git commit -m "test(kotlin): @spine and measure @default through Exposed"`. + +--- + +### Task 12: Python port + +**Files** (under `server/python/`): +- Modify (byte copies): `src/metaobjects/spec_metamodel/reporting.json`, `spec_metamodel/object.json` +- Modify: `src/metaobjects/meta/core/reporting/reporting_constants.py`, `meta/core/object/object_constants.py`, `core_types.py` (the hand-declared report, `measure.aggregate` and `measure.ratio` attrs), `meta/core/reporting/meta_measure.py`, `report_accessors.py`, `report_shape.py` (`_dimension_field`, `_measure_field`), `loader/validate_reporting.py` (`_walk_to_one_via`, `_check_measure`, `_check_report`) +- Modify: `tests/integration/generated_report_app.py` if it names the `invoices` table or the reports one by one +- Test: `tests/unit/test_reporting_accessors.py`, `tests/test_report_shape.py` (the hard-coded "six", line 121), `tests/test_report_read_model.py`, `tests/codegen/test_report_router.py`, `tests/runtime/test_object_manager_report.py`, `tests/test_reporting_inert.py`; a new `tests/unit/test_reporting_validation_spine_default.py` for the Task 2 unit cases (Python has no reporting validation unit file today) + +Watch the naming inversion: Python `attr()` is OWN. Use the resolving form everywhere the TypeScript code calls `attr()`. And `bool` is a subclass of `int`: `default_value()` must refuse `True`. + +- [ ] **Step 1:** Copy the two JSON files; `cd server/python && uv run pytest -q tests/conformance` — expected FAIL (registry, ten fixtures). +- [ ] **Step 2: Register** both attributes in `core_types.py` (`value_type` `int` for `default`); add constants and accessors. Registry-conformance PASS. +- [ ] **Step 3: Port R8, R9, M7, M8** with the same message texts. +- [ ] **Step 4: Port Table C** into `report_shape.py` (add the hop split). `test_report_shape.py` byte-matches `report-shapes.json`. +- [ ] **Step 5: Row types and reads.** A defaulted measure and the spine key have no `| None` in the Pydantic row model (`entity_model.py`, `_field_line`); the same measure without `@default` keeps it. The `ObjectManager` reads a `@spine` view with an empty row. +- [ ] **Step 6: Lanes.** `uv run pytest -q` scoped to conformance, the query scenarios, `tests/integration/test_api_contract_report.py` and the inert test — PASS. +- [ ] **Step 7: Commit (local).** `git commit -m "feat(python): @spine and measure @default"`. + +--- + +### Task 13: The no-churn proof + +**Files:** none modified. This task produces evidence and stops the branch if any of it is missing. + +- [ ] **Step 1: Appended, never edited.** For each shared artifact, compare the branch to `origin/main` structurally: + +```bash +# SQL: no line of an existing statement removed or changed. +git diff origin/main -- fixtures/persistence-conformance/canonical/schema.postgres.sql \ + fixtures/api-contract-conformance/report/schema.postgres.sql | grep -c '^-[^-]' # must print 0 +# JSON: the first N entries are deep-equal to origin/main's. +bun -e 'const {execSync}=require("child_process");const p="fixtures/persistence-conformance/report-shapes.json"; +const a=JSON.parse(execSync(`git show origin/main:${p}`)).reports,b=require("./"+p).reports; +if(JSON.stringify(b.slice(0,a.length))!==JSON.stringify(a))throw new Error("an existing shape changed")' +``` + +The same deep-equal check for the existing children of `Week`, `Program` and the six reports in `meta.fitness.json`, and for `Invoice`, its four reports, `invoices` and the three existing `reports` entries in the `report/` corpus. + +- [ ] **Step 2: No existing expectation edited.** `git diff origin/main --stat` over `fixtures/**/expected*`, `fixtures/**/scenarios/**`, `fixtures/**/queries/**`, every committed generated tree (`server/csharp/MetaObjects.IntegrationTests/Generated/`, the Kotlin `tables/`) and every golden test: only new files, the `AppDbContext.g.cs` additions, and count literals (six → nine, thirteen → sixteen, 364 → 374). +- [ ] **Step 3: Inert.** The five inert tests pass: a sourceless `@spine` report and a defaulted measure on an entity generate nothing in any port. +- [ ] **Step 4: Codegen-compile gate.** It generates from `meta.fitness.json`, which now carries three more served reports: run each port's gate and confirm the new row types compile. +- [ ] **Step 5:** Record the four results in the PR description. No commit. + +--- + +### Task 14: Docs, skills, changelog, counts + +**Files:** +- Modify: `docs/features/reporting.md` — the registered table (`@spine`, `@default`); the example; two new sections, "Rows from a dimension's entity" and "A default for an empty measure", holding Tables D, E and G in prose; "The columns you get" (the two never-null rows); the rule tables (R8, R9, M7, M8, the integer check); "What differs by engine" (SQLite's `0.0`); "Known limits" (the spec §9 list); "What the corpus gates" (nine persistence scenarios, sixteen REST) +- Modify: `docs/ports/{typescript,csharp,java,kotlin,python}.md` (each port's Reports section; `kotlin.md` states the nullable rule and must gain the two exceptions), `docs/features/api-contract.md` (Reports), `docs/recipes/mysql.md` (Reports: a `@spine` body is valid MySQL as emitted) +- Modify: `agent-context/skills/metaobjects-authoring/references/reporting.md` ("the columns you get", "nulls and zeros", "known limits") and `SKILL.md` (the reporting section); the five `agent-context/skills/metaobjects-codegen/references/*.md` only where they state a report column's nullability +- Regenerate: `fixtures/agent-context-conformance/**` with `bun server/typescript/packages/sdk/scripts/regen-agent-context-conformance.ts` +- Modify: `CHANGELOG.md` `[Unreleased]` — one new entry; `docs/CONFORMANCE.md` (persistence 39 → 42, api-contract 78 → 81 with report 13 → 16); `fixtures/persistence-conformance/README.md` +- Check, and edit only what this change made wrong: `spec/roadmap.md` (the FR-044 rows), `.claude/rules/cross-language-porting.md` (it says "the 27 `reporting` conformance fixtures"), the project's own requirements ledger + +- [ ] **Step 1:** Write the docs. The authoring skill must teach the two questions an agent gets wrong: *do I want the rows that have no facts?* (then `@spine`, and every dimension goes through it) and *is zero true here, or is null?* (then `@default`, and never on a count). +- [ ] **Step 2:** The CHANGELOG entry states: the two attributes; `metamodelVersion` stays 1.1; nothing changes for a model that uses neither; the four decisions of "What merging this plan decides"; and that an adopter with owned generators needs no resync for this (no generator changed). +- [ ] **Step 3: Run.** `bun run site:payload && bun scripts/build-site-payload.ts --check && bun test scripts/site`, `cd server/typescript && bun test packages/sdk/test/agent-context-conformance.test.ts`. +- [ ] **Step 4: Commit (local).** `git commit -m "docs(reporting): @spine and measure @default"`. + +--- + +### Task 15: Full CI, review, gate + +- [ ] **Step 1:** Rebase on `origin/main`. Another change is editing the TypeScript report generators and the spec's worked example; resolve by keeping both, and re-run Tasks 6 and 8's regenerations if a canonical model moved under them. +- [ ] **Step 2:** `scripts/ci-local.sh` (full, no flags), once. Expected green in every lane, including `gates` (metamodel-version, site payload, leak scan, publish-set parity). +- [ ] **Step 3:** A fresh reviewer over `git diff origin/main..HEAD`, briefed with this plan's Review Focus and nothing else. Fix what it finds. +- [ ] **Step 4:** Hand the branch to the validation gate. Do not push to `main`. + +--- + +## No-churn and back-compat proof + +| Claim | What proves it | +|---|---| +| A model that declares neither attribute generates the same files | No generator is edited in any port. The `without/` model of `fixtures/codegen-noop/reporting/` is compared file by file in all five inert tests | +| A sourceless `@spine` report and a defaulted measure are inert | The `with/` model gains both and still equals `without/` in every port's inert test | +| The six existing report views keep their SQL | The lowering's branch without `spineDepth` is the present code; every golden in `report-ddl-emit.test.ts` and `extract-report-spec.test.ts` passes unedited; Task 13 Step 1 finds no removed line in either `schema.postgres.sql` | +| The six existing shapes keep their bytes | Table C changes `required` only under `@spine` or `@default`; `report-shapes-artifact.test.ts` passes after Task 4 before anything is regenerated; Task 13's deep-equal check | +| Existing loader behaviour is unchanged | R8, R9, M7 and M8 fire only when one of the two attributes is present; every existing conformance fixture and every D2 message stays byte-identical | +| The canonical format is unchanged | Two attributes serialise as a string and an integer, forms every port already prints | +| `metamodelVersion` stays `1.1` | `check-metamodel-version.mjs` passes without `--set`; `--explain` classifies ADDITIVE; 1.1 has not been released | +| Existing REST scenarios are unchanged | The thirteen scenario files are not edited; `Invoice` and its reports are not edited; no runner changes its assertions | +| No runtime changes | No `ObjectManager`, OMDB, EF Core or Exposed read path is edited; the three new persistence scenarios pass on the existing read code in five ports | +| **Behaviour change** (unreleased vocabulary only) | A report that declares `@spine`, or lists a measure with `@default`, lowers to different SQL and a tighter row type. Nothing released carries either attribute | + +## Unverified items + +Each is the first step of the task that touches it. + +| Item | Task | +|---|---| +| That a fractional number on an `attr.int` is `ERR_BAD_ATTR_VALUE` in each port (the existing fixture covers a string, not a fraction) | 3, 9, 10, 12 | +| The present row-scope phrases in `report-describe.ts`, and the docs goldens that pin them | 7 | +| What `docs-site/test/coverage.test.ts` needs for a new attribute to count as rendered | 7 | +| The served segment for `ProductRevenue` | 8 | +| How each lane's harness loads the base tables of `report/seed.json`, and whether the Java, Kotlin and Python lanes name the reports one by one | 8, 9 to 12 | +| The exact generated spelling of a required read-model field in each port (Table F) | 9 to 12 | +| A one-to-one hop whose reference is held by the far entity, reversed in a spine chain: not executed on an engine, only reasoned from the shared `ON` predicate | 5 | +| Whether `integration-tests-kotlin`'s reference tables are checked structurally or byte for byte | 11 | +| What `scripts/site/counts.ts` counts | 3 | +| Whether any entry of the project's own requirements ledger describes report rows or measure nulls | 14 | + +## Open questions for the captain + +None blocks the build. The four choices under "What merging this plan decides" are the ones that would be costly to reverse after 1.1.0; each has a recommendation and the reasons above, and merging this plan rules on them. Stated as questions: + +1. **The grain under `@spine`.** One row per distinct dimension tuple among the spine entity's rows (recommended: no exception to what a report row is), or one row per spine row whatever is listed (rows a caller cannot tell apart)? +2. **A ratio's operand keeps its `@default` inside the ratio** (recommended: a measure reads the same wherever it is used, and both exporter targets behave this way), or the operand reads null there? +3. **`@default` is an integer** (recommended: every case found is zero, and widening later is additive), or a number typed per measure? +4. **The names `@spine` and `@default`.** Renaming either before 1.1.0 is a search and replace across this plan's change; after it, a deprecation. +5. **What this does not cover for the reference adopter.** Its day-by-day chart joins events to the schedule on three natural-key columns. With `@spine` the chart is expressible once the event carries a declared reference to the scheduled day; joining on the natural key stays out, as ruled. That is a data model change on the adopter's side, not something this plan can remove. diff --git a/docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md b/docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md index 81c63777c..fe4936731 100644 --- a/docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md +++ b/docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md @@ -7,6 +7,12 @@ who need a full semantic layer get one through exporters to established tools. O the maintainer accepted all six open decisions as recommended (§8) and gave the explicit ADR-0023 agreement for the new vocabulary listed in §3.1, with the justifications written there. Ready for an implementation plan. +**Amended 2026-10-09:** two additions to the 1.1 vocabulary, requested by the maintainer after +a reference adopter's report pages were measured against Plans 1 to 3: a report's rows can come +from a dimension's entity (`@spine`, R8), and a measure can declare its value when it is null +(`@default`, R9). The register entries are in §3.2. Both are additive and `metamodelVersion` +stays `1.1`. Plan: +`docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md`. **Target:** metamodel `1.1` (additive vocabulary is a MINOR — `docs/compatibility-policy.md`, `scripts/check-metamodel-version.mjs`). Nothing here is a PATCH. **Relates to:** [ADR-0037](../../../spec/decisions/ADR-0037-metamodel-vocabulary-expansion-decision-framework.md) @@ -138,6 +144,24 @@ port's provider and in `fixtures/registry-conformance/expected-registry.json`. scope is the existing `@filter` (which carries R4's relative-date values), so no new attribute is needed for it. +### 3.2 ADR-0023 register amendment (2026-10-09) + +The maintainer asked for these two capabilities on 2026-10-09, for the 1.1 release, so that +vocabulary a first adopter needs does not force a second metamodel move straight after 1.1. +The names, the placement and the rules are this amendment's proposal (R8 and R9 below, and +the plan named in the header). Merging that plan is the ADR-0023 agreement to them. Nothing +else is added: `measure.derived` stays absent. + +| New name | Kind | Why it cannot be computed from existing metadata | +|---|---|---| +| `@spine` | attr, `string`, on `object.report` | Whether a report shows a dimension tuple that has no fact rows is the report author's choice, and the same dimensions serve both kinds of report: a sales table that lists only products with sales, and a catalogue table that lists every product with zero beside the ones that never sold. Nothing in the model says which a given report wants. The value is a to-one path because an entity can be reached from `@from` by more than one reference, so the entity's name alone would not say which rows are meant. | +| `@default` | attr, `int`, on `measure.aggregate` and `measure.ratio` | Null and zero mean different things ("no purchases" against "not applicable"), and which one a measure reads when it has nothing to aggregate is the author's statement about that measure. A `count` already derives its own zero, so the attribute is refused there. | + +`@default` reuses the name a field already has, under ADR-0037's "same concept, same +attribute name": on a field it is the value used when none is supplied, on a measure the +value used when there is none to aggregate. Unlike a field's, which follows the field's +type, a measure's is registered as an integer (see R9 for why). + ## 4. Requirements The examples use the reference adopter's model: `Purchase`, `Program` and `WorkoutEvent`. @@ -328,6 +352,115 @@ A FR-043 library, `stability: preview`, opt-in as `"libraries": ["reporting"]`: Composes only R1–R5 vocabulary; adds none. +### R8 — `@spine`: a report's rows from a dimension's entity (added 2026-10-09) + +Part of the 1.1 change set, with R1 to R5. + +```yaml +# on Purchase +- dimension.attribute: { name: programId, "@of": "Program.id", "@via": "Purchase.program" } +- dimension.attribute: { name: programTitle, "@of": "Program.title", "@via": "Purchase.program" } + +- object.report: + name: ProgramSales + "@from": Purchase + "@spine": "Purchase.program" + "@dimensions": [programId, programTitle] + "@measures": [purchases, buyers, revenue] + "@segment": active +``` + +**The gap.** A report has one row per dimension tuple that has fact rows. A reference +adopter's pages need the rows that have none: a published program with no purchases still +belongs in the per-program table, and a scheduled day nobody completed is an empty bar in a +chart whose whole point is the empty bars. R7's calendar spine covers the time case only. + +**Proposed shape.** One optional attribute on `object.report`. Without it nothing changes. +With it, the report's dimension tuples come from the rows of the entity at the end of the +path (the *spine entity*) instead of from the fact rows: one row per distinct dimension +tuple among **that entity's** rows, whether or not any row of `@from` refers to it. + +- `@spine` is a to-one path with `@via`'s grammar, `Owner.hop[.hop…]`. `Owner` is `@from` + (or an entity it extends) and every hop is to-one, exactly as for a dimension's `@via`. + The lowering needs a declared `identity.reference` on every hop, as it does for a dimension. +- **Every listed dimension is reached through the spine:** its `@via` begins with the + spine's hops. So a dimension is a column of the spine entity, or of an entity to-one from + it. A dimension read from the fact row (no `@via`, or a `@via` that leaves `@from` through + another reference) is refused, because it has no value in a row that has no facts. A + report with `@spine` lists at least one dimension. +- **A time dimension follows the same rule.** It is legal when its column belongs to the + spine entity or beyond (the month a program was published), and refused over a fact column + (the month of a purchase). Zero-filled time buckets remain R7's calendar spine. +- **`@segment` and `@filter` on the report scope fact rows, never spine rows.** A spine row + whose facts are all filtered out keeps its row. Both still name fields of `@from`. +- **A row with no facts** reads `0` for a `count`, null for `sum`, `avg`, `min` and `max` + (unless the measure declares R9's `@default`), and a ratio is computed from its operands as + they read. +- **A fact row whose reference is null, or matches no spine row, is in no row of the + report.** Without `@spine` such rows form a null group; with it there is no spine row to + hold them. +- The grain is unchanged: one row per distinct dimension tuple. To get exactly one row per + row of the spine entity, list a dimension over its identity (`programId` above). +- One spine per report. + +**ADR-0037 walk.** (0) Not derivable: the same dimensions serve a report that hides the empty +tuples and one that shows them, so the choice is the report's. (1) Not physical-only: it +changes the row set. (2) It configures an existing type and needs a reference (the path), so +it is an **attribute**; it is not a structural variant with its own generated shape, so not a +`@kind`, and it owns no behaviour of its own, so not a subtype. It lives on the report, not +on the dimension, because one dimension is listed by both kinds of report. + +**Admission.** RDB: the spine entity's table is the `FROM` and the fact table is +`LEFT JOIN`ed, with the report's row scope in the join condition. MongoDB: `$lookup` from +the spine collection, then `$group`. In memory: iterate the spine rows. Search: refused (no +join). + +### R9 — `@default`: a measure's value when it is null (added 2026-10-09) + +Part of the 1.1 change set, with R1 to R5. + +```yaml +- measure.aggregate: { name: revenue, "@agg": sum, "@of": "Purchase.amountCents", + "@segment": active, "@default": 0 } +- measure.ratio: { name: revenuePerBuyer, "@numerator": revenue, + "@denominator": buyers, "@default": 0 } +``` + +**The gap.** A `sum` of nothing and a ratio over zero are null by contract (R2), so every +caller defaults in the client where a hand-written view would have written +`COALESCE(…, 0)`. + +**Proposed shape.** One optional integer attribute on `measure.aggregate` and +`measure.ratio`: the value the measure reads when it would otherwise be null. That is the +case when nothing matched (an empty table, a row with no facts under R8, a measure `@filter` +or `@segment` that matched none of a group's rows), when every matched value is null, and +for a ratio whose denominator is zero or null. + +- Legal on a `sum` or `avg`, on a `min` or `max` over a numeric field, and on a ratio. +- **Refused on a `count`**: a count is never null (it is `0`), so a default could never + apply. +- **Refused on a `min` or `max` over a field that is not numeric** (a string, an enum, a + date): a default is a number. +- A measure with a `@default` is never null, so its derived report field is not nullable in + any port. +- A ratio's operand carries its own `@default` into the ratio: each operand is its full + expression, as it already is for conditions. + +**Why an integer.** Every case found is zero. An integer is a valid value of every numeric +type a measure can have (`long`, `currency` minor units, `decimal`, `double`), so the value +needs no rule per measure type. dbt MetricFlow's `fill_nulls_with` takes an integer, so the +§5 mapping stays lossless. A fractional default would need a spelling that five canonical +serializers and three SQL dialects agree on, for a case nobody has. Widening it later is +additive. + +**ADR-0037 walk.** (0) Not derivable: null and zero mean different things, and which one a +measure reads is the author's statement. (1) Not physical-only: it changes the value and the +nullability on the wire. (2) It configures an existing type: an **attribute**, under the +name fields already use for the same concept. + +**Admission.** RDB `COALESCE`; MongoDB `$ifNull`; Search the aggregation's `missing` value or +adapter code; in memory trivial. + ## 5. Mapping contract (core → Cube → MetricFlow) | MetaObjects | Cube | dbt MetricFlow | @@ -345,6 +478,8 @@ Composes only R1–R5 vocabulary; adds none. | `segment` | segment | metric `filter` / saved query filter | | relative filter `{ now: "-P7D" }` | query `dateRange` "last 7 days" | `{{ TimeDimension(...) }} >= dateadd(...)` | | `object.report` | a pre-aggregation (rollup) | a saved query | +| measure `@default: n` (R9) | a `number` measure `COALESCE({measure}, n)` | `fill_nulls_with: n` on the metric | +| report `@spine` (R8) | the spine entity's cube joins the fact cube `one_to_many`, and the rollup is rooted on it | none: `join_to_timespine` covers time only. The exporter must refuse a `@spine` report, never drop the attribute silently (§3 obligation 3) | ## 6. Dependencies and order @@ -354,7 +489,8 @@ Composes only R1–R5 vocabulary; adds none. 2. **R1, R2 (aggregate + ratio), R3, R4, R5** — one metamodel `1.1` change set: register in five ports, registry-conformance, loader fixtures, TS lowering for Postgres, SQLite/D1 and MySQL, persistence-conformance (read each report view in every port), api-contract - `report/` sub-corpus (list, filter, sort, paging, 405 on writes) in every port. + `report/` sub-corpus (list, filter, sort, paging, 405 on writes) in every port. **R8 and + R9** (added 2026-10-09) join this change set and are gated the same way. 3. **R2 `measure.derived`** — once FR-037 R5's arithmetic lands. 4. **R6** — Cube exporter. MetricFlow on first adopter demand (D5). 5. **R7** — the library. @@ -377,6 +513,13 @@ Composes only R1–R5 vocabulary; adds none. result equals the report view result on the conformance data. (MetricFlow: golden fixtures and `dbt parse`, when it is built.) - No-churn: the existing corpora produce byte-identical output. +- R8 and R9: loader error fixtures for a `@spine` over a to-many hop, a listed dimension not + reached through the spine, a `@spine` report with no dimensions, a `@default` on a `count`, + on a non-numeric `min`/`max`, and one that is not an integer. Value tests on Postgres and + SQLite with a spine row that has no facts: its row is present, a `count` is `0`, a `sum` + and a ratio are null without a `@default` and the declared value with one, and the view has + exactly as many rows as the spine entity has distinct dimension tuples. Every port reads + those rows and types a defaulted measure as not nullable. - The reference adopter's admin analytics are rebuilt from declarations, and its hand-written report code shrinks accordingly (measured and recorded in the release notes). @@ -403,3 +546,17 @@ The maintainer accepted every recommendation on 2026-10-03. - **Percentile/median, approximate distinct, conversion/funnel metrics, rolling windows, period-over-period**: advanced features present in only one or two surveyed tools. Available through the exported semantic layer. + +Parked with R8 and R9 (2026-10-09). Each is refused at load today, so admitting it later is +additive: + +- **Two spines in one report** (every program against every customer): a cross join whose + size is the product of both tables. Re-entry: an adopter case a single spine with onward + dimensions cannot serve. +- **A row scope on the spine entity** (only published programs). A report `@filter` names + `@from`'s fields. Until then, list the column as a dimension and filter it on the request. +- **A fact-row dimension beside a spine** (every program, split by purchase month). The row + for a program with no purchases would carry a null month. Zero-filled time buckets are R7. +- **A dimension over the reference column itself** (`Purchase.programId`) in a `@spine` + report. Declare the dimension over the spine entity's identity instead. +- **A fractional or non-numeric `@default`.** From 2ccebc44dfa016c30117318f9b54b7a74fa5ca69 Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 18:58:50 -0400 Subject: [PATCH 02/18] feat(metamodel): register @spine on object.report and @default on measures (TypeScript) --- fixtures/metamodel-docs/expected/providers.md | 2 +- .../metamodel-docs/expected/types/measure.md | 2 + .../metamodel-docs/expected/types/object.md | 1 + .../registry-conformance/coverage-report.json | 18 +++++++++ .../expected-registry.json | 21 ++++++++++ .../src/core/object/object-constants.ts | 2 + .../core/object/object-definition.embedded.ts | 8 ++++ .../src/core/reporting/meta-measure.ts | 8 ++++ .../src/core/reporting/report-accessors.ts | 7 ++++ .../src/core/reporting/reporting-constants.ts | 2 + .../reporting-definition.embedded.ts | 16 ++++++++ .../typescript/packages/metadata/src/index.ts | 1 + .../object-definition-completeness.test.ts | 3 +- .../metadata/test/reporting-registry.test.ts | 16 ++++++++ .../test/reporting-validation.test.ts | 39 +++++++++++++++++++ site-reference/providers.html | 2 +- site-reference/types/measure.html | 18 +++++++++ site-reference/types/object.html | 9 +++++ spec/metamodel/object.json | 3 +- spec/metamodel/reporting.json | 6 ++- 20 files changed, 178 insertions(+), 6 deletions(-) diff --git a/fixtures/metamodel-docs/expected/providers.md b/fixtures/metamodel-docs/expected/providers.md index c9e56802c..b27115a0a 100644 --- a/fixtures/metamodel-docs/expected/providers.md +++ b/fixtures/metamodel-docs/expected/providers.md @@ -31,7 +31,7 @@ Core metaobjects metamodel types and subtypes. - `identity.secondary`: `@fields` - `object.entity`: `@discriminator`, `@discriminatorValue` - `object.projection`: `@filter` -- `object.report`: `@dimensions`, `@filter`, `@from`, `@measures`, `@segment` +- `object.report`: `@dimensions`, `@filter`, `@from`, `@measures`, `@segment`, `@spine` - `origin.aggregate`: `@agg`, `@distinct`, `@filter`, `@of`, `@orderBy`, `@via` - `origin.computed`: `@expr` - `origin.first`: `@filter`, `@of`, `@orderBy`, `@via` diff --git a/fixtures/metamodel-docs/expected/types/measure.md b/fixtures/metamodel-docs/expected/types/measure.md index 1166f9e73..93dc113b7 100644 --- a/fixtures/metamodel-docs/expected/types/measure.md +++ b/fixtures/metamodel-docs/expected/types/measure.md @@ -21,6 +21,7 @@ A named aggregate over the declaring entity's own rows (FR-044). @agg count with | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | | `@agg` | string | yes | | `count`, `sum`, `avg`, `min`, `max` | — | The aggregate function. sum/avg need a numeric field; min/max refuse boolean, object and map fields. | +| `@default` | int | no | | | — | Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio. | | `@distinct` | boolean | no | | | — | Count distinct values. Legal only with @agg: count. | | `@filter` | filter | no | | | — | Optional row scope (a portable attr.filter over the declaring entity's fields). May use relative-date values ({ now: "-P7D" }). Combines with @segment by AND. | | `@of` | string[] | yes | | | — | Dotted Entity.field reference(s) on the declaring entity. A bare string is one column; more than one requires @agg: count and @distinct: true. | @@ -40,6 +41,7 @@ A named quotient of two measure.aggregate siblings (FR-044), lowered as numerato | Attribute | Type | Required | Default | Allowed values | Provider | Description | | --- | --- | --- | --- | --- | --- | --- | +| `@default` | int | no | | | — | Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio. | | `@denominator` | string | yes | | | — | Name of a measure.aggregate on the same entity. | | `@numerator` | string | yes | | | — | Name of a measure.aggregate on the same entity. | diff --git a/fixtures/metamodel-docs/expected/types/object.md b/fixtures/metamodel-docs/expected/types/object.md index 5c138b439..1645748e4 100644 --- a/fixtures/metamodel-docs/expected/types/object.md +++ b/fixtures/metamodel-docs/expected/types/object.md @@ -102,6 +102,7 @@ A declared report (FR-044): a fixed combination of dimensions and measures of ON | `@from` | string | yes | | | metaobjects-core-types | The object.entity whose rows the report aggregates. | | `@measures` | string[] | yes | | | metaobjects-core-types | Measure names of @from. | | `@segment` | string | no | | | metaobjects-core-types | Optional segment of @from scoping the rows. | +| `@spine` | string | no | | | metaobjects-core-types | Optional to-one path from @from to the entity whose rows supply the report's rows (e.g. 'Purchase.program'), written like a dimension's @via. With it the report has one row per distinct dimension tuple among THAT entity's rows, including the ones no row of @from refers to: a count there is 0 and any other measure is null unless it declares @default. Every listed dimension must be reached through this path. @segment and @filter still scope the rows of @from and never remove a row. A row of @from whose reference is null or matches nothing is in no row of the report. | **Allowed children** diff --git a/fixtures/registry-conformance/coverage-report.json b/fixtures/registry-conformance/coverage-report.json index 631dfcee5..6f9f29fac 100644 --- a/fixtures/registry-conformance/coverage-report.json +++ b/fixtures/registry-conformance/coverage-report.json @@ -300,6 +300,24 @@ "using" ] }, + { + "key": "measure.aggregate", + "untestedAttrs": [ + "default" + ] + }, + { + "key": "measure.ratio", + "untestedAttrs": [ + "default" + ] + }, + { + "key": "object.report", + "untestedAttrs": [ + "spine" + ] + }, { "key": "relationship.aggregation", "untestedAttrs": [ diff --git a/fixtures/registry-conformance/expected-registry.json b/fixtures/registry-conformance/expected-registry.json index ec78892d0..9ae0cd63d 100644 --- a/fixtures/registry-conformance/expected-registry.json +++ b/fixtures/registry-conformance/expected-registry.json @@ -3145,6 +3145,13 @@ ], "description": "The aggregate function. sum/avg need a numeric field; min/max refuse boolean, object and map fields." }, + { + "name": "default", + "valueType": "int", + "isArray": false, + "required": false, + "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." + }, { "name": "distinct", "valueType": "boolean", @@ -3182,6 +3189,13 @@ "description": "A named quotient of two measure.aggregate siblings (FR-044), lowered as numerator / NULLIF(denominator, 0) and typed decimal. A zero denominator yields null.", "whenToUse": "Averages per unit that are not a plain avg: average days engaged per starter.", "attrs": [ + { + "name": "default", + "valueType": "int", + "isArray": false, + "required": false, + "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." + }, { "name": "denominator", "valueType": "string", @@ -3484,6 +3498,13 @@ "isArray": false, "required": false, "description": "Optional segment of @from scoping the rows." + }, + { + "name": "spine", + "valueType": "string", + "isArray": false, + "required": false, + "description": "Optional to-one path from @from to the entity whose rows supply the report's rows (e.g. 'Purchase.program'), written like a dimension's @via. With it the report has one row per distinct dimension tuple among THAT entity's rows, including the ones no row of @from refers to: a count there is 0 and any other measure is null unless it declares @default. Every listed dimension must be reached through this path. @segment and @filter still scope the rows of @from and never remove a row. A row of @from whose reference is null or matches nothing is in no row of the report." } ], "children": [ diff --git a/server/typescript/packages/metadata/src/core/object/object-constants.ts b/server/typescript/packages/metadata/src/core/object/object-constants.ts index 50c55be27..ac4cbfccf 100644 --- a/server/typescript/packages/metadata/src/core/object/object-constants.ts +++ b/server/typescript/packages/metadata/src/core/object/object-constants.ts @@ -66,3 +66,5 @@ export const OBJECT_REPORT_ATTR_MEASURES = "measures"; export const OBJECT_REPORT_ATTR_SEGMENT = "segment"; /** Optional row scope over @from's fields (an attr.filter). */ export const OBJECT_REPORT_ATTR_FILTER = "filter"; +/** Optional to-one path from @from to the entity whose rows supply the report's rows (R8). */ +export const OBJECT_REPORT_ATTR_SPINE = "spine"; diff --git a/server/typescript/packages/metadata/src/core/object/object-definition.embedded.ts b/server/typescript/packages/metadata/src/core/object/object-definition.embedded.ts index dc19e244c..e95ed2dda 100644 --- a/server/typescript/packages/metadata/src/core/object/object-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/core/object/object-definition.embedded.ts @@ -202,6 +202,14 @@ export const OBJECT_DEFINITION: ProviderDefinition = { "min": 0, "max": 1, "description": "Optional row scope over @from's fields; may use relative-date values." + }, + { + "type": "attr", + "subType": "string", + "name": "spine", + "min": 0, + "max": 1, + "description": "Optional to-one path from @from to the entity whose rows supply the report's rows (e.g. 'Purchase.program'), written like a dimension's @via. With it the report has one row per distinct dimension tuple among THAT entity's rows, including the ones no row of @from refers to: a count there is 0 and any other measure is null unless it declares @default. Every listed dimension must be reached through this path. @segment and @filter still scope the rows of @from and never remove a row. A row of @from whose reference is null or matches nothing is in no row of the report." } ] } diff --git a/server/typescript/packages/metadata/src/core/reporting/meta-measure.ts b/server/typescript/packages/metadata/src/core/reporting/meta-measure.ts index 9274d401c..3e914f286 100644 --- a/server/typescript/packages/metadata/src/core/reporting/meta-measure.ts +++ b/server/typescript/packages/metadata/src/core/reporting/meta-measure.ts @@ -7,6 +7,7 @@ import { MEASURE_AGGS, MEASURE_SUBTYPE_RATIO, REPORTING_ATTR_AGG, + REPORTING_ATTR_DEFAULT, REPORTING_ATTR_DENOMINATOR, REPORTING_ATTR_DISTINCT, REPORTING_ATTR_FILTER, @@ -55,6 +56,13 @@ export class MetaMeasure extends MetaData { return typeof v === "object" && v !== null && !Array.isArray(v) ? (v as Record<string, unknown>) : undefined; } + /** `@default`: the integer this measure reads when it would otherwise be null (rule M7/M8 + * decide where it is legal). ADR-0039: resolving, so an inherited measure keeps it. */ + defaultValue(): number | undefined { + const v = this.attr(REPORTING_ATTR_DEFAULT); + return typeof v === "number" && Number.isInteger(v) ? v : undefined; + } + /** Name of the `measure.aggregate` sibling used as the numerator (`measure.ratio` only). */ numerator(): string | undefined { return optionalString(this.attr(REPORTING_ATTR_NUMERATOR)); diff --git a/server/typescript/packages/metadata/src/core/reporting/report-accessors.ts b/server/typescript/packages/metadata/src/core/reporting/report-accessors.ts index 58e118f71..30b91a37a 100644 --- a/server/typescript/packages/metadata/src/core/reporting/report-accessors.ts +++ b/server/typescript/packages/metadata/src/core/reporting/report-accessors.ts @@ -7,6 +7,7 @@ import { OBJECT_REPORT_ATTR_DIMENSIONS, OBJECT_REPORT_ATTR_FROM, OBJECT_REPORT_ATTR_MEASURES, + OBJECT_REPORT_ATTR_SPINE, } from "../object/object-constants.js"; import { CHILD_REF_SEPARATOR } from "../../shared/structural.js"; import { REPORT_DIMENSION_GRAIN_SEPARATOR } from "./reporting-constants.js"; @@ -27,6 +28,12 @@ export function reportFrom(obj: MetaData): string | undefined { return typeof v === "string" ? v : undefined; } +/** `@spine`: the to-one path to the entity whose rows supply the report's rows. */ +export function reportSpine(obj: MetaData): string | undefined { + const v = obj.attr(OBJECT_REPORT_ATTR_SPINE); + return typeof v === "string" && v !== "" ? v : undefined; +} + /** The `@dimensions` items, each `name` or `name:grain`. */ export function reportDimensionItems(obj: MetaData): ReportDimensionItem[] { return stringList(obj.attr(OBJECT_REPORT_ATTR_DIMENSIONS)).map((raw) => { diff --git a/server/typescript/packages/metadata/src/core/reporting/reporting-constants.ts b/server/typescript/packages/metadata/src/core/reporting/reporting-constants.ts index fbf575d28..2de709e44 100644 --- a/server/typescript/packages/metadata/src/core/reporting/reporting-constants.ts +++ b/server/typescript/packages/metadata/src/core/reporting/reporting-constants.ts @@ -57,6 +57,8 @@ export const REPORTING_ATTR_SEGMENT = "segment"; /** `measure.ratio` operand names. */ export const REPORTING_ATTR_NUMERATOR = "numerator"; export const REPORTING_ATTR_DENOMINATOR = "denominator"; +/** Integer a measure reads when it would otherwise be null (`measure.aggregate` / `measure.ratio`). */ +export const REPORTING_ATTR_DEFAULT = "default"; // --------------------------------------------------------------------------- // Closed sets — mirrored by `allowedValues` in spec/metamodel/reporting.json. diff --git a/server/typescript/packages/metadata/src/core/reporting/reporting-definition.embedded.ts b/server/typescript/packages/metadata/src/core/reporting/reporting-definition.embedded.ts index f6a7457c9..6de64b8a3 100644 --- a/server/typescript/packages/metadata/src/core/reporting/reporting-definition.embedded.ts +++ b/server/typescript/packages/metadata/src/core/reporting/reporting-definition.embedded.ts @@ -127,6 +127,14 @@ export const REPORTING_DEFINITION: ProviderDefinition = { "min": 0, "max": 1, "description": "Optional name of a segment declared on the same entity. Combines with @filter by AND." + }, + { + "type": "attr", + "subType": "int", + "name": "default", + "min": 0, + "max": 1, + "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } ] }, @@ -151,6 +159,14 @@ export const REPORTING_DEFINITION: ProviderDefinition = { "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." + }, + { + "type": "attr", + "subType": "int", + "name": "default", + "min": 0, + "max": 1, + "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } ] }, diff --git a/server/typescript/packages/metadata/src/index.ts b/server/typescript/packages/metadata/src/index.ts index d60d550c6..63e296e01 100644 --- a/server/typescript/packages/metadata/src/index.ts +++ b/server/typescript/packages/metadata/src/index.ts @@ -48,6 +48,7 @@ export { MetaMeasure } from "./core/reporting/meta-measure.js"; export { MetaSegment } from "./core/reporting/meta-segment.js"; export { reportFrom, + reportSpine, reportDimensionItems, reportMeasureNames, reportMeasureItemName, diff --git a/server/typescript/packages/metadata/test/object-definition-completeness.test.ts b/server/typescript/packages/metadata/test/object-definition-completeness.test.ts index 0ceeffd04..edc56a02e 100644 --- a/server/typescript/packages/metadata/test/object-definition-completeness.test.ts +++ b/server/typescript/packages/metadata/test/object-definition-completeness.test.ts @@ -75,7 +75,7 @@ const PROJECTION_FILTER: Record<string, ExpectedAttr> = { filter: { valueType: "filter", required: false }, }; -// FR-044 — object.report carries its five declaration attrs. The array-valued ones +// FR-044 — object.report carries its six declaration attrs (five, plus the optional @spine). The array-valued ones // (@dimensions, @measures) are `string` attrs with isArray, so valueType stays "string". const REPORT_ATTRS: Record<string, ExpectedAttr> = { from: { valueType: "string", required: true }, @@ -83,6 +83,7 @@ const REPORT_ATTRS: Record<string, ExpectedAttr> = { measures: { valueType: "string", required: true }, segment: { valueType: "string", required: false }, filter: { valueType: "filter", required: false }, + spine: { valueType: "string", required: false }, }; function expectedAttrsFor(subType: string): Record<string, ExpectedAttr> { diff --git a/server/typescript/packages/metadata/test/reporting-registry.test.ts b/server/typescript/packages/metadata/test/reporting-registry.test.ts index 61348de36..528bb3298 100644 --- a/server/typescript/packages/metadata/test/reporting-registry.test.ts +++ b/server/typescript/packages/metadata/test/reporting-registry.test.ts @@ -26,6 +26,22 @@ describe("FR-044 reporting vocabulary registration", () => { expect(registry.find(TYPE_MEASURE, "derived")).toBeUndefined(); }); + test("object.report registers an optional string @spine (R8)", () => { + const attr = registry.find(TYPE_OBJECT, OBJECT_SUBTYPE_REPORT)!.attributes.find((a) => a.name === "spine"); + expect(attr).toBeDefined(); + expect(attr!.valueType).toBe("string"); + expect(attr!.required).toBe(false); + }); + + test("measure.aggregate and measure.ratio each register an optional int @default (R9)", () => { + for (const sub of [MEASURE_SUBTYPE_AGGREGATE, MEASURE_SUBTYPE_RATIO]) { + const attr = registry.find(TYPE_MEASURE, sub)!.attributes.find((a) => a.name === "default"); + expect(attr).toBeDefined(); + expect(attr!.valueType).toBe("int"); + expect(attr!.required).toBe(false); + } + }); + test("closed sets match the spec", () => { expect([...TIME_GRAINS]).toEqual(["hour", "day", "week", "month", "quarter", "year"]); expect([...MEASURE_AGGS]).toEqual(["count", "sum", "avg", "min", "max"]); diff --git a/server/typescript/packages/metadata/test/reporting-validation.test.ts b/server/typescript/packages/metadata/test/reporting-validation.test.ts index 21d68a664..ab9f75581 100644 --- a/server/typescript/packages/metadata/test/reporting-validation.test.ts +++ b/server/typescript/packages/metadata/test/reporting-validation.test.ts @@ -13,6 +13,7 @@ import { ISO_DURATION_RE } from "../src/core/reporting/reporting-constants.js"; import { reportDerivedFieldName, reportDimensionItems, + reportSpine, } from "../src/core/reporting/report-accessors.js"; import { MetaMeasure } from "../src/core/reporting/meta-measure.js"; import type { MetaData } from "../src/shared/meta-data.js"; @@ -1099,4 +1100,42 @@ describe("report accessors", () => { "WorkoutEvent.dayNumber", ]); }); + + test("MetaMeasure.defaultValue: the declared integer (0 and negatives included), else undefined", async () => { + const m = edit((model) => { + patchMember(model, "Purchase", "revenue", { "@default": 0 }); + setChild(model, "Purchase", "net", { + "measure.aggregate": { name: "net", "@agg": "avg", "@of": "Purchase.amountCents", "@default": -1 }, + }); + patchMember(model, "WorkoutEvent", "avgDaysPerStarter", { "@default": 0 }); + }); + const { root, errors } = await loadInline(m); + expect(errors).toEqual([]); + const measure = (entity: string, name: string): MetaMeasure => + root.children().find((c) => c.name === entity)!.children().find((c) => c.name === name) as MetaMeasure; + expect(measure("Purchase", "revenue").defaultValue()).toBe(0); + expect(measure("Purchase", "net").defaultValue()).toBe(-1); + expect(measure("WorkoutEvent", "avgDaysPerStarter").defaultValue()).toBe(0); + expect(measure("Purchase", "purchases").defaultValue()).toBeUndefined(); + expect(measure("WorkoutEvent", "avgDaysPerStarter").isRatio()).toBe(true); + }); + + test("reportSpine returns the declared path, and undefined when the report declares none", async () => { + const m = edit((model) => { + model["metadata.root"].children.push({ + "object.report": { + name: "ProgramPurchases", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": ["programTitle"], + "@measures": ["purchases", "revenue"], + }, + }); + }); + const { root, errors } = await loadInline(m); + expect(errors).toEqual([]); + const report = (name: string) => root.children().find((c) => c.name === name)!; + expect(reportSpine(report("ProgramPurchases"))).toBe("Purchase.program"); + expect(reportSpine(report("DailyRevenue"))).toBeUndefined(); + }); }); diff --git a/site-reference/providers.html b/site-reference/providers.html index e6fe700aa..f565f02df 100644 --- a/site-reference/providers.html +++ b/site-reference/providers.html @@ -68,7 +68,7 @@ <h2>metaobjects-core-types</h2> <li><code>identity.secondary</code>: <code>@fields</code></li> <li><code>object.entity</code>: <code>@discriminator</code>, <code>@discriminatorValue</code></li> <li><code>object.projection</code>: <code>@filter</code></li> -<li><code>object.report</code>: <code>@dimensions</code>, <code>@filter</code>, <code>@from</code>, <code>@measures</code>, <code>@segment</code></li> +<li><code>object.report</code>: <code>@dimensions</code>, <code>@filter</code>, <code>@from</code>, <code>@measures</code>, <code>@segment</code>, <code>@spine</code></li> <li><code>origin.aggregate</code>: <code>@agg</code>, <code>@distinct</code>, <code>@filter</code>, <code>@of</code>, <code>@orderBy</code>, <code>@via</code></li> <li><code>origin.computed</code>: <code>@expr</code></li> <li><code>origin.first</code>: <code>@filter</code>, <code>@of</code>, <code>@orderBy</code>, <code>@via</code></li> diff --git a/site-reference/types/measure.html b/site-reference/types/measure.html index ee77c5484..72cb720e6 100644 --- a/site-reference/types/measure.html +++ b/site-reference/types/measure.html @@ -76,6 +76,15 @@ <h3>measure.aggregate</h3> <td>The aggregate function. sum/avg need a numeric field; min/max refuse boolean, object and map fields.</td> </tr> <tr> +<td><code>@default</code></td> +<td>int</td> +<td>no</td> +<td></td> +<td></td> +<td>—</td> +<td>Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio.</td> +</tr> +<tr> <td><code>@distinct</code></td> <td>boolean</td> <td>no</td> @@ -131,6 +140,15 @@ <h3>measure.ratio</h3> </tr> </thead> <tbody><tr> +<td><code>@default</code></td> +<td>int</td> +<td>no</td> +<td></td> +<td></td> +<td>—</td> +<td>Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio.</td> +</tr> +<tr> <td><code>@denominator</code></td> <td>string</td> <td>yes</td> diff --git a/site-reference/types/object.html b/site-reference/types/object.html index 44c207d52..5355296b7 100644 --- a/site-reference/types/object.html +++ b/site-reference/types/object.html @@ -214,6 +214,15 @@ <h3>object.report</h3> <td>metaobjects-core-types</td> <td>Optional segment of @from scoping the rows.</td> </tr> +<tr> +<td><code>@spine</code></td> +<td>string</td> +<td>no</td> +<td></td> +<td></td> +<td>metaobjects-core-types</td> +<td>Optional to-one path from @from to the entity whose rows supply the report's rows (e.g. 'Purchase.program'), written like a dimension's @via. With it the report has one row per distinct dimension tuple among THAT entity's rows, including the ones no row of @from refers to: a count there is 0 and any other measure is null unless it declares @default. Every listed dimension must be reached through this path. @segment and @filter still scope the rows of @from and never remove a row. A row of @from whose reference is null or matches nothing is in no row of the report.</td> +</tr> </tbody></table> <p><strong>Allowed children</strong></p> <ul> diff --git a/spec/metamodel/object.json b/spec/metamodel/object.json index cb5f9946f..76187a94a 100644 --- a/spec/metamodel/object.json +++ b/spec/metamodel/object.json @@ -64,7 +64,8 @@ { "type": "attr", "subType": "string", "name": "dimensions", "isArray": true, "min": 0, "max": 1, "description": "Dimension names of @from; a time dimension is written 'name:grain'. Absent means one global row." }, { "type": "attr", "subType": "string", "name": "measures", "isArray": true, "min": 1, "max": 1, "description": "Measure names of @from." }, { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional segment of @from scoping the rows." }, - { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope over @from's fields; may use relative-date values." } + { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope over @from's fields; may use relative-date values." }, + { "type": "attr", "subType": "string", "name": "spine", "min": 0, "max": 1, "description": "Optional to-one path from @from to the entity whose rows supply the report's rows (e.g. 'Purchase.program'), written like a dimension's @via. With it the report has one row per distinct dimension tuple among THAT entity's rows, including the ones no row of @from refers to: a count there is 0 and any other measure is null unless it declares @default. Every listed dimension must be reached through this path. @segment and @filter still scope the rows of @from and never remove a row. A row of @from whose reference is null or matches nothing is in no row of the report." } ] } ] diff --git a/spec/metamodel/reporting.json b/spec/metamodel/reporting.json index 024196d63..019dda238 100644 --- a/spec/metamodel/reporting.json +++ b/spec/metamodel/reporting.json @@ -24,14 +24,16 @@ { "type": "attr", "subType": "string", "name": "of", "isArray": true, "min": 1, "max": 1, "description": "Dotted Entity.field reference(s) on the declaring entity. A bare string is one column; more than one requires @agg: count and @distinct: true." }, { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Count distinct values. Legal only with @agg: count." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope (a portable attr.filter over the declaring entity's fields). May use relative-date values ({ now: \"-P7D\" }). Combines with @segment by AND." }, - { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional name of a segment declared on the same entity. Combines with @filter by AND." } + { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional name of a segment declared on the same entity. Combines with @filter by AND." }, + { "type": "attr", "subType": "int", "name": "default", "min": 0, "max": 1, "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } ] }, { "type": "measure", "subType": "ratio", "description": "A named quotient of two measure.aggregate siblings (FR-044), lowered as numerator / NULLIF(denominator, 0) and typed decimal. A zero denominator yields null.", "whenToUse": "Averages per unit that are not a plain avg: average days engaged per starter.", "children": [ { "type": "attr", "subType": "string", "name": "numerator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." }, - { "type": "attr", "subType": "string", "name": "denominator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." } + { "type": "attr", "subType": "string", "name": "denominator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." }, + { "type": "attr", "subType": "int", "name": "default", "min": 0, "max": 1, "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } ] }, { "type": "segment", "subType": "filter", "description": "A named, reusable row filter on the declaring entity (FR-044). Measures and reports reference it by name; exporters emit it as a named segment.", From 3826a15dc81372be517d85275505f5809a935884 Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 19:08:05 -0400 Subject: [PATCH 03/18] feat(metadata): validate @spine and a measure @default at load (TypeScript) --- .../src/loader/reporting-validation.ts | 134 ++++++- .../test/reporting-validation.test.ts | 350 ++++++++++++++++++ 2 files changed, 465 insertions(+), 19 deletions(-) diff --git a/server/typescript/packages/metadata/src/loader/reporting-validation.ts b/server/typescript/packages/metadata/src/loader/reporting-validation.ts index 838a4cc94..674a77b11 100644 --- a/server/typescript/packages/metadata/src/loader/reporting-validation.ts +++ b/server/typescript/packages/metadata/src/loader/reporting-validation.ts @@ -1,5 +1,6 @@ // FR-044 — cross-node rules for the reporting vocabulary. The rule ids (D1…F2) // match the rule table in docs/superpowers/plans/2026-10-03-fr-044-plan-1-reporting-vocabulary.md +// (R8, R9, M7 and M8: Table B of docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md) // and the error fixtures in fixtures/conformance/error-*. Every port implements // the same table WITH THE SAME MESSAGE TEXT; the fixtures are the contract. // @@ -38,6 +39,7 @@ import { OBJECT_SUBTYPE_REPORT, OBJECT_REPORT_ATTR_FILTER, OBJECT_REPORT_ATTR_SEGMENT, + OBJECT_REPORT_ATTR_SPINE, } from "../core/object/object-constants.js"; import { FIELD_SUBTYPE_INT, @@ -83,6 +85,8 @@ import { GRAIN_HOUR, REPORTING_ATTR_NUMERATOR, REPORTING_ATTR_DENOMINATOR, + REPORTING_ATTR_DEFAULT, + REPORTING_ATTR_VIA, MEASURE_SUBTYPE_AGGREGATE, } from "../core/reporting/reporting-constants.js"; import { MetaDimension } from "../core/reporting/meta-dimension.js"; @@ -93,6 +97,7 @@ import { reportDimensionItems, reportFrom, reportMeasureNames, + reportSpine, } from "../core/reporting/report-accessors.js"; // --------------------------------------------------------------------------- @@ -300,7 +305,8 @@ interface MemberCtx { readonly root: MetaData; /** The entity whose children() the member was reached through. */ readonly host: MetaData; - /** The entity that declares the member (the host, or an ancestor of it). */ + /** The entity that declares the member (the host, or an ancestor of it). For a report's @spine + * walk (R8) it is the report, so the spine's Owner resolves in the report's package. */ readonly declaring: MetaData; /** `<kind> '<name>' on entity '<declaring FQN>'` — every member message starts with it. */ readonly label: string; @@ -313,9 +319,12 @@ interface MemberCtx { * The FQN a member message names for `entity`: the DECLARING entity in place of * the host, so a failure is worded identically whichever entity reached the * member (the suffix names the inheritor) and the ErrorSink repeat test holds. + * A report's @spine walk (R8) has no declaring entity (`declaring` is the + * report), so there every entity names itself. */ function shown(ctx: MemberCtx, entity: MetaData): string { - return (entity === ctx.host ? ctx.declaring : entity).resolutionKey(); + const declaredByEntity = isSelfOrAncestor(ctx.declaring, ctx.host); + return (entity === ctx.host && declaredByEntity ? ctx.declaring : entity).resolutionKey(); } function checkEntityMembers(root: MetaData, entity: MetaData, declaredHere: boolean, sink: ErrorSink): void { @@ -397,20 +406,35 @@ function checkDimension(ctx: MemberCtx, dim: MetaDimension): void { } } +/** How a to-one walk's messages name it: the attribute that holds the path (`via`, `spine`) and + * what its Owner must be (`the owning entity '<FQN>'`, `@from '<FQN>'`). */ +interface WalkWording { + readonly attr: string; + readonly start: string; +} + /** * D2 — walk `Owner.hop[.hop...]`: Owner is the owning entity, and every hop is a * to-one `relationship.*` or an `identity.reference`. Returns the terminal - * entity, or undefined after reporting the first failure. + * entity, or undefined after reporting the first failure. R8 runs the same walk + * over a report's @spine, started at @from; `wording` names it (the default is + * D2's own wording). */ -function walkToOneVia(ctx: MemberCtx, via: string, err: (message: string) => void): MetaData | undefined { +function walkToOneVia( + ctx: MemberCtx, + via: string, + err: (message: string) => void, + wording: WalkWording = { attr: REPORTING_ATTR_VIA, start: `the owning entity '${ctx.declaring.resolutionKey()}'` }, +): MetaData | undefined { + const named = `@${wording.attr} '${via}'`; const parts = splitDotted(via); if (parts === undefined) { - err(`@via '${via}' must be Owner.hop[.hop...], starting at the owning entity.`); + err(`${named} must be Owner.hop[.hop...], starting at the owning entity.`); return undefined; } const owner = resolveObjectRef(ctx.root, parts.owner, pkgOf(ctx.declaring)).node; if (!isSelfOrAncestor(owner, ctx.host)) { - err(`@via '${via}' must start at the owning entity '${ctx.declaring.resolutionKey()}'.`); + err(`${named} must start at ${wording.start}.`); return undefined; } let current = ctx.host; @@ -422,7 +446,7 @@ function walkToOneVia(ctx: MemberCtx, via: string, err: (message: string) => voi .find((c) => c.type === TYPE_IDENTITY && c.subType === IDENTITY_SUBTYPE_REFERENCE && c.name === hopName); if (hop === undefined) { err( - `@via '${via}' names '${hopName}', which is not a relationship or identity.reference of ` + + `${named} names '${hopName}', which is not a relationship or identity.reference of ` + `'${shown(ctx, current)}'.`, ); return undefined; @@ -430,7 +454,7 @@ function walkToOneVia(ctx: MemberCtx, via: string, err: (message: string) => voi const isReference = hop.type === TYPE_IDENTITY; if (!isReference && hop.attr(RELATIONSHIP_ATTR_CARDINALITY) !== CARDINALITY_ONE) { err( - `@via '${via}' crosses relationship '${hopName}' on '${shown(ctx, current)}', which is not to-one. ` + + `${named} crosses relationship '${hopName}' on '${shown(ctx, current)}', which is not to-one. ` + `A dimension follows only @cardinality: one relationships and identity.reference hops, so grouping ` + `can never multiply the measured rows.`, ); @@ -441,7 +465,7 @@ function walkToOneVia(ctx: MemberCtx, via: string, err: (message: string) => voi const target = typeof targetRef === "string" ? resolveObjectRef(ctx.root, targetRef, pkgOf(current)).node : undefined; if (target === undefined) { - err(`@via '${via}' hop '${hopName}' on '${shown(ctx, current)}' targets no object.`); + err(`${named} hop '${hopName}' on '${shown(ctx, current)}' targets no object.`); return undefined; } current = target; @@ -459,7 +483,30 @@ function checkMeasure(ctx: MemberCtx, measure: MetaMeasure): void { } if (measure.subType !== MEASURE_SUBTYPE_AGGREGATE) return; - checkAggregateColumns(ctx, measure, err); + const checked = checkAggregateColumns(ctx, measure, err); + const clean = checked !== undefined; + const ofField = checked?.field; + + // M7 / M8 — where a @default can apply. Only when M1–M4 passed: one mistake, one error. + // The presence test reads the raw attribute, so a mistyped value on a count still reports M7. + if (clean && measure.attr(REPORTING_ATTR_DEFAULT) !== undefined) { + const agg = measure.agg(); + if (agg === AGG_COUNT) { + err( + `@default cannot apply to @agg: count. A count is never null (it is 0 when nothing matches); ` + + `remove @default.`, + ); + } else if ( + (agg === AGG_MIN || agg === AGG_MAX) && + ofField !== undefined && + !NUMERIC_FIELD_SUBTYPES.includes(ofField.subType) + ) { + err( + `@default is a number, but @agg '${agg}' of '${measure.ofColumns()[0]}' is a field.${ofField.subType}. ` + + `A default is supported on numeric measures only.`, + ); + } + } // M5 — @segment names a segment of the owning entity. const segment = measure.segmentName(); @@ -474,8 +521,16 @@ function checkMeasure(ctx: MemberCtx, measure: MetaMeasure): void { } } -/** M1–M4, in order; the first failure stops the chain (no M2+M3 double report). */ -function checkAggregateColumns(ctx: MemberCtx, measure: MetaMeasure, err: (message: string) => void): void { +/** + * M1–M4, in order; the first failure stops the chain (no M2+M3 double report). + * Returns undefined when one of them fired; otherwise `field` is the single + * resolved @of field (undefined for a tuple), for M7/M8. + */ +function checkAggregateColumns( + ctx: MemberCtx, + measure: MetaMeasure, + err: (message: string) => void, +): { readonly field: MetaData | undefined } | undefined { const agg = measure.agg(); const columns = measure.ofColumns(); @@ -485,7 +540,7 @@ function checkAggregateColumns(ctx: MemberCtx, measure: MetaMeasure, err: (messa const parts = splitDotted(item); if (parts === undefined || parts.path.length !== 1) { err(`@of '${item}' must be Entity.field.`); - return; + return undefined; } const named = resolveObjectRef(ctx.root, parts.owner, pkgOf(ctx.declaring)).node; if (!isSelfOrAncestor(named, ctx.host)) { @@ -493,12 +548,12 @@ function checkAggregateColumns(ctx: MemberCtx, measure: MetaMeasure, err: (messa `@of '${item}' must name a field of the owning entity '${ctx.declaring.resolutionKey()}'. ` + `A measure aggregates its own entity's rows; declare it on the entity that owns the column.`, ); - return; + return undefined; } const field = fieldOf(ctx.host, parts.path[0]!); if (field === undefined) { err(`@of '${item}' names no field '${parts.path[0]!}' on '${ctx.declaring.resolutionKey()}'.`); - return; + return undefined; } fields.push(field); } @@ -509,29 +564,31 @@ function checkAggregateColumns(ctx: MemberCtx, measure: MetaMeasure, err: (messa `@of lists ${columns.length} columns; a tuple is legal only with @agg: count and @distinct: true ` + `(a distinct count of the tuple).`, ); - return; + return undefined; } // M3 — @distinct is a count modifier. if (measure.distinct() && agg !== undefined && agg !== AGG_COUNT) { err(`@distinct: true requires @agg: count, not '${agg}'.`); - return; + return undefined; } // M4 — the aggregate must be meaningful for the column's type. const field = fields.length === 1 ? fields[0] : undefined; - if (field === undefined || agg === undefined) return; + if (field === undefined || agg === undefined) return { field }; const item = columns[0]!; if ((agg === AGG_SUM || agg === AGG_AVG) && !NUMERIC_FIELD_SUBTYPES.includes(field.subType)) { err( `@agg '${agg}' needs a numeric field (field.int, long, double, float, decimal or currency), ` + `but '${item}' is field.${field.subType}.`, ); - return; + return undefined; } if ((agg === AGG_MIN || agg === AGG_MAX) && UNORDERED_FIELD_SUBTYPES.includes(field.subType)) { err(`@agg '${agg}' cannot order '${item}', a field.${field.subType}.`); + return undefined; } + return { field }; } /** M6 — each operand names a measure.aggregate of the same entity. */ @@ -757,6 +814,45 @@ function checkReport(root: MetaData, report: MetaData, sink: ErrorSink): void { err(`: @segment '${segment}' names no segment of @from '${fromKey}'.`); } + // R8 — @spine is a to-one path from @from: rule D2's walk, started at @from. + const spine = reportSpine(report); + if (spine !== undefined) { + const terminal = walkToOneVia( + { root, host: from, declaring: report, label, suffix: "", sink }, + spine, + (message) => err(`: ${message}`), + { attr: OBJECT_REPORT_ATTR_SPINE, start: `@from '${fromKey}'` }, + ); + // R9 — every listed dimension is reached through the spine. Skipped when R8 failed. + if (terminal !== undefined) { + const spineHops = splitDotted(spine)?.path ?? []; + const items = reportDimensionItems(report); + if (items.length === 0) { + err( + `: @spine '${spine}' needs at least one dimension. The report's rows are the dimension tuples of ` + + `'${terminal.resolutionKey()}'; with no dimension it would be one totals row.`, + ); + } + for (const item of items) { + const dim = childOfType(from, TYPE_DIMENSION, item.name); + if (!(dim instanceof MetaDimension)) continue; // R2 already reported it + const via = dim.via(); + const hops = via === undefined ? undefined : splitDotted(via)?.path; + // Hop names are compared as written; the owner segment is not compared. + if (hops !== undefined && spineHops.every((h, i) => hops[i] === h)) continue; + err( + via === undefined + ? `: dimension '${item.name}' is read from @from '${fromKey}', so it has no value in a row that has ` + + `no facts. With @spine '${spine}' every dimension must be reached through it: declare the ` + + `dimension over a field of '${terminal.resolutionKey()}' (or an entity to-one from it) with an ` + + `@via that begins '${spine}'.` + : `: dimension '${item.name}' is reached by @via '${via}', which does not begin with the hops of ` + + `@spine '${spine}'. Hop names are compared as written; write the same hops.`, + ); + } + } + } + // S1 / F2 — the report's row scope over @from. const filter = report.attr(OBJECT_REPORT_ATTR_FILTER); if (isPlainObject(filter)) checkFilter(filter, from, fromKey, label, report, "", sink); diff --git a/server/typescript/packages/metadata/test/reporting-validation.test.ts b/server/typescript/packages/metadata/test/reporting-validation.test.ts index ab9f75581..993832d6f 100644 --- a/server/typescript/packages/metadata/test/reporting-validation.test.ts +++ b/server/typescript/packages/metadata/test/reporting-validation.test.ts @@ -1066,6 +1066,356 @@ describe("validateReporting — inherited members", () => { }); }); +// --------------------------------------------------------------------------- +// R8 / R9 — a report's @spine (docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md, Table B) +// --------------------------------------------------------------------------- + +/** Append report `ProgramPurchases` over Purchase with `@spine: Purchase.program`; `attrs` override + * (undefined deletes). */ +function addSpineReport(m: Model, attrs: Body = {}): void { + m["metadata.root"].children.push({ + "object.report": { + name: "ProgramPurchases", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": ["programTitle"], + "@measures": ["purchases", "revenue"], + }, + }); + patchObject(m, "ProgramPurchases", attrs); +} + +/** Program -> Coach -> Agency, both to-one, plus Purchase dimensions at the end of each hop. */ +function addCoachChain(m: Model): void { + childrenOf(m, "Program").push( + field("long", "coachId"), + { "identity.reference": { name: "coachRef", "@references": "Coach", "@fields": ["coachId"] } }, + { "relationship.association": { name: "coach", "@objectRef": "Coach", "@cardinality": "one" } }, + ); + m["metadata.root"].children.push( + { + "object.entity": { + name: "Coach", + children: [ + { "source.rdb": { "@table": "coaches" } }, + field("long", "id"), + field("string", "name"), + field("long", "agencyId"), + primary(), + { "identity.reference": { name: "agencyRef", "@references": "Agency", "@fields": ["agencyId"] } }, + { "relationship.association": { name: "agency", "@objectRef": "Agency", "@cardinality": "one" } }, + ], + }, + }, + { + "object.entity": { + name: "Agency", + children: [{ "source.rdb": { "@table": "agencies" } }, field("long", "id"), field("string", "name"), primary()], + }, + }, + ); + childrenOf(m, "Purchase").push( + { "dimension.attribute": { name: "coachName", "@of": "Coach.name", "@via": "Purchase.program.coach" } }, + { "dimension.attribute": { name: "agencyName", "@of": "Agency.name", "@via": "Purchase.program.coach.agency" } }, + ); +} + +/** A dimension of Purchase reaching Program.title through the identity.reference, not the relationship. */ +function programTitleByRef(): Wrapper { + return { + "dimension.attribute": { name: "programTitleByRef", "@of": "Program.title", "@via": "Purchase.programRef" }, + }; +} + +describe("validateReporting — @spine (R8, R9)", () => { + test("a spine report whose dimensions are all reached through the spine loads clean", async () => { + const m = edit((x) => addSpineReport(x)); + expect(await codes(m)).toEqual([]); + }); + + test("R8: a to-many spine hop is refused, naming the report and the hop's entity", async () => { + const m = edit((x) => { + setChild(x, "Program", "programs", { + "measure.aggregate": { name: "programs", "@agg": "count", "@of": "Program.id" }, + }); + x["metadata.root"].children.push({ + "object.report": { + name: "ProgramReach", + "@from": "Program", + "@spine": "Program.purchases", + "@measures": ["programs"], + }, + }); + }); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toBe( + "report 'acme::shop::ProgramReach': @spine 'Program.purchases' crosses relationship 'purchases' on " + + "'acme::shop::Program', which is not to-one. A dimension follows only @cardinality: one relationships " + + "and identity.reference hops, so grouping can never multiply the measured rows.", + ); + }); + + test("R8: a spine whose owner is another entity is refused", async () => { + const m = edit((x) => addSpineReport(x, { "@spine": "Program.purchases" })); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toBe( + "report 'acme::shop::ProgramPurchases': @spine 'Program.purchases' must start at @from 'acme::shop::Purchase'.", + ); + }); + + test("R8: a spine hop that names nothing is refused, naming @from (not the report) as the hop's entity", async () => { + const m = edit((x) => addSpineReport(x, { "@spine": "Purchase.nope" })); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toBe( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase.nope' names 'nope', which is not a relationship " + + "or identity.reference of 'acme::shop::Purchase'.", + ); + }); + + test("R8: a spine with no hop is refused", async () => { + const m = edit((x) => addSpineReport(x, { "@spine": "Purchase" })); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toContain("report 'acme::shop::ProgramPurchases': @spine 'Purchase' must be Owner.hop[.hop...]"); + }); + + test("D2's own wording is unchanged by the walk's wording argument", async () => { + const d2 = async (via: string): Promise<string> => + single(edit((x) => patchMember(x, "Purchase", "programTitle", { "@via": via })), "ERR_INVALID_DIMENSION"); + const head = "dimension 'programTitle' on entity 'acme::shop::Purchase': "; + expect(await d2("Purchase")).toBe( + `${head}@via 'Purchase' must be Owner.hop[.hop...], starting at the owning entity.`, + ); + expect(await d2("WorkoutEvent.program")).toBe( + `${head}@via 'WorkoutEvent.program' must start at the owning entity 'acme::shop::Purchase'.`, + ); + expect(await d2("Purchase.nope")).toBe( + `${head}@via 'Purchase.nope' names 'nope', which is not a relationship or identity.reference of ` + + `'acme::shop::Purchase'.`, + ); + }); + + test("R8: the error source is the report node", async () => { + const m = edit((x) => addSpineReport(x, { "@spine": "Purchase.nope" })); + const { errors } = await loadInline(m); + expect(jsonPathOf(errors[0]!.source)).toMatch(/\['object\.report'\]$/); + }); + + test("R8 failing skips R9: a broken spine with an off-spine dimension is ONE error", async () => { + const m = edit((x) => addSpineReport(x, { "@spine": "Purchase.nope", "@dimensions": ["purchasedAt:day"] })); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toContain("@spine 'Purchase.nope' names 'nope'"); + }); + + test("R8: an inherited spine (owner written as the abstract base) loads clean", async () => { + const m = inheritedModel(); + m["metadata.root"].children.push({ + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + field("long", "id"), + field("string", "title"), + primary(), + ], + }, + }); + (objectBody(m, "BaseEvent").children as Wrapper[]).push( + field("long", "programId"), + { "identity.reference": { name: "programRef", "@references": "Program", "@fields": ["programId"] } }, + { "relationship.association": { name: "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "BaseEvent.program" } }, + ); + m["metadata.root"].children.push({ + "object.report": { + name: "ProgramEvents", + "@from": "WorkoutEvent", + "@spine": "BaseEvent.program", + "@dimensions": ["programTitle"], + "@measures": ["events"], + }, + }); + expect(await codes(m)).toEqual([]); + }); + + test("R9: a spine report with no dimensions is refused", async () => { + const m = edit((x) => addSpineReport(x, { "@dimensions": undefined })); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toBe( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase.program' needs at least one dimension. The " + + "report's rows are the dimension tuples of 'acme::shop::Program'; with no dimension it would be one " + + "totals row.", + ); + }); + + test("R9: a dimension with no @via (read from the fact row) is refused", async () => { + const m = edit((x) => addSpineReport(x, { "@dimensions": ["programTitle", "program"] })); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toBe( + "report 'acme::shop::ProgramPurchases': dimension 'program' is read from @from 'acme::shop::Purchase', so " + + "it has no value in a row that has no facts. With @spine 'Purchase.program' every dimension must be " + + "reached through it: declare the dimension over a field of 'acme::shop::Program' (or an entity to-one " + + "from it) with an @via that begins 'Purchase.program'.", + ); + }); + + test("R9: a time dimension over a fact column is refused", async () => { + const m = edit((x) => addSpineReport(x, { "@dimensions": ["programTitle", "purchasedAt:day"] })); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toContain("report 'acme::shop::ProgramPurchases': dimension 'purchasedAt' is read from @from"); + }); + + test("R9: each offending dimension is reported once, even when listed at two grains", async () => { + const m = edit((x) => + addSpineReport(x, { "@dimensions": ["program", "purchasedAt:day", "purchasedAt:week", "programTitle"] }), + ); + const { errors } = await loadInline(m); + expect(errors.map((e) => e.code)).toEqual(["ERR_INVALID_REPORT", "ERR_INVALID_REPORT"]); + expect(errors[0]!.message).toContain("dimension 'program' is read from @from"); + expect(errors[1]!.message).toContain("dimension 'purchasedAt' is read from @from"); + }); + + test("R9: a dimension through a second reference to the same entity is refused", async () => { + const m = edit((x) => { + childrenOf(x, "Purchase").push( + field("long", "giftProgramId"), + { "identity.reference": { name: "giftProgramRef", "@references": "Program", "@fields": ["giftProgramId"] } }, + programTitleByRef(), + { + "dimension.attribute": { + name: "giftProgramTitle", + "@of": "Program.title", + "@via": "Purchase.giftProgramRef", + }, + }, + ); + addSpineReport(x, { "@spine": "Purchase.programRef", "@dimensions": ["programTitleByRef", "giftProgramTitle"] }); + }); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toBe( + "report 'acme::shop::ProgramPurchases': dimension 'giftProgramTitle' is reached by @via " + + "'Purchase.giftProgramRef', which does not begin with the hops of @spine 'Purchase.programRef'. Hop names " + + "are compared as written; write the same hops.", + ); + }); + + test("R9: the same join, named by the relationship in @spine and by the reference in @via, is refused", async () => { + const m = edit((x) => { + childrenOf(x, "Purchase").push(programTitleByRef()); + addSpineReport(x, { "@spine": "Purchase.program", "@dimensions": ["programTitleByRef"] }); + }); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toContain("dimension 'programTitleByRef' is reached by @via 'Purchase.programRef'"); + expect(msg).toContain("@spine 'Purchase.program'"); + }); + + test("R9: the owner segment is not compared (an FQN spine owner, a bare dimension @via)", async () => { + const m = edit((x) => addSpineReport(x, { "@spine": "acme::shop::Purchase.program" })); + expect(await codes(m)).toEqual([]); + }); + + test("R9: a time dimension over a column of the spine entity is legal", async () => { + const m = edit((x) => { + childrenOf(x, "Program").push(field("timestamp", "publishedAt")); + childrenOf(x, "Purchase").push({ + "dimension.time": { + name: "programPublishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": ["month"], + }, + }); + addSpineReport(x, { "@dimensions": ["programTitle", "programPublishedAt:month"] }); + }); + expect(await codes(m)).toEqual([]); + }); + + test("R9: a two-hop spine with a dimension at it and one beyond it is legal", async () => { + const m = edit((x) => { + addCoachChain(x); + addSpineReport(x, { "@spine": "Purchase.program.coach", "@dimensions": ["coachName", "agencyName"] }); + }); + expect(await codes(m)).toEqual([]); + }); + + test("R9: a dimension that stops short of a two-hop spine is refused", async () => { + const m = edit((x) => { + addCoachChain(x); + addSpineReport(x, { "@spine": "Purchase.program.coach", "@dimensions": ["coachName", "programTitle"] }); + }); + const msg = await single(m, "ERR_INVALID_REPORT"); + expect(msg).toContain("dimension 'programTitle' is reached by @via 'Purchase.program'"); + }); +}); + +// --------------------------------------------------------------------------- +// M7 / M8 — where a measure's @default can apply (Table B) +// --------------------------------------------------------------------------- + +describe("validateReporting — measure @default (M7, M8)", () => { + test("M7: @default on a count is refused, on the measure node", async () => { + const m = edit((x) => patchMember(x, "Purchase", "purchases", { "@default": 0 })); + const { errors } = await loadInline(m); + expect(errors.map((e) => e.code)).toEqual(["ERR_INVALID_MEASURE"]); + expect(errors[0]!.message).toBe( + "measure 'purchases' on entity 'acme::shop::Purchase': @default cannot apply to @agg: count. A count is " + + "never null (it is 0 when nothing matches); remove @default.", + ); + expect(jsonPathOf(errors[0]!.source)).toContain("['measure.aggregate']"); + }); + + test("M7: @default on a distinct count of a tuple is refused", async () => { + const m = edit((x) => patchMember(x, "WorkoutEvent", "daysEngaged", { "@default": 0 })); + const msg = await single(m, "ERR_INVALID_MEASURE"); + expect(msg).toContain("measure 'daysEngaged'"); + expect(msg).toContain("@default cannot apply to @agg: count"); + }); + + test("M8: @default on a max of a timestamp is refused", async () => { + const m = edit((x) => patchMember(x, "WorkoutEvent", "lastActivityAt", { "@default": 0 })); + const msg = await single(m, "ERR_INVALID_MEASURE"); + expect(msg).toBe( + "measure 'lastActivityAt' on entity 'acme::shop::WorkoutEvent': @default is a number, but @agg 'max' of " + + "'WorkoutEvent.occurredAt' is a field.timestamp. A default is supported on numeric measures only.", + ); + }); + + test("@default on a sum, an avg, a min of an int and a ratio is fine", async () => { + const m = edit((x) => { + patchMember(x, "Purchase", "revenue", { "@default": 0 }); + setChild(x, "Purchase", "avgRevenue", { + "measure.aggregate": { name: "avgRevenue", "@agg": "avg", "@of": "Purchase.amountCents", "@default": 0 }, + }); + setChild(x, "WorkoutEvent", "firstDay", { + "measure.aggregate": { name: "firstDay", "@agg": "min", "@of": "WorkoutEvent.dayNumber", "@default": 1 }, + }); + patchMember(x, "WorkoutEvent", "avgDaysPerStarter", { "@default": 0 }); + }); + expect(await codes(m)).toEqual([]); + }); + + test("a measure that breaks M4 and declares @default reports M4 only", async () => { + const m = edit((x) => patchMember(x, "Purchase", "revenue", { "@of": "Purchase.status", "@default": 0 })); + const msg = await single(m, "ERR_INVALID_MEASURE"); + expect(msg).toContain("field.string"); + expect(msg).not.toContain("@default"); + }); + + test("a measure that breaks M1 and declares @default on a count reports M1 only", async () => { + const m = edit((x) => patchMember(x, "Purchase", "purchases", { "@of": "Purchase.nope", "@default": 0 })); + const msg = await single(m, "ERR_INVALID_MEASURE"); + expect(msg).toContain("names no field 'nope'"); + expect(msg).not.toContain("@default"); + }); + + test("M7 on a count declared on an abstract base is reported ONCE", async () => { + const m = inheritedModel(); + const base = objectBody(m, "BaseEvent").children as Wrapper[]; + base[3] = { "measure.aggregate": { name: "events", "@agg": "count", "@of": "BaseEvent.id", "@default": 0 } }; + const msg = await single(m, "ERR_INVALID_MEASURE"); + expect(msg).toContain("measure 'events' on entity 'acme::shop::BaseEvent'"); + }); +}); + // --------------------------------------------------------------------------- // Accessors (Task 1 follow-up) // --------------------------------------------------------------------------- From 68c551dae9a17cd5be3dfa352c63b852eb668cbc Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 19:16:57 -0400 Subject: [PATCH 04/18] fix(metadata): clearer @spine messages and one error per mistake (TypeScript) --- .../src/loader/reporting-validation.ts | 150 ++++++++++++------ .../test/reporting-validation.test.ts | 49 +++++- 2 files changed, 142 insertions(+), 57 deletions(-) diff --git a/server/typescript/packages/metadata/src/loader/reporting-validation.ts b/server/typescript/packages/metadata/src/loader/reporting-validation.ts index 674a77b11..38e200ad5 100644 --- a/server/typescript/packages/metadata/src/loader/reporting-validation.ts +++ b/server/typescript/packages/metadata/src/loader/reporting-validation.ts @@ -6,9 +6,11 @@ // // Two design rules hold throughout, so one broken rule yields exactly one error: // - No cascades. A member that fails a structural rule is not checked further -// (a dimension whose @via fails D2 skips D1/D3/D4; a report whose @from fails -// R1 skips R2/R3/R6/R7 and its @filter; an invalid @dimensions/@measures item -// derives no report field for R6). +// (a dimension whose @via fails D2 skips D1/D3/D4; a measure that fails one +// of M1–M4 skips M7/M8; a report whose @from fails R1 skips R2/R3/R6/R7, +// R8/R9 and its @filter; a report whose @spine fails R8 skips R9, and R9 +// skips a dimension whose @via fails D2; an invalid @dimensions/@measures +// item derives no report field for R6). // - Each error's `source` is the offending node (the dimension / measure / // segment / report, or for R4/R5 the declared child), so a conformance // fixture's jsonPath points at it. @@ -305,8 +307,7 @@ interface MemberCtx { readonly root: MetaData; /** The entity whose children() the member was reached through. */ readonly host: MetaData; - /** The entity that declares the member (the host, or an ancestor of it). For a report's @spine - * walk (R8) it is the report, so the spine's Owner resolves in the report's package. */ + /** The entity that declares the member (the host, or an ancestor of it). */ readonly declaring: MetaData; /** `<kind> '<name>' on entity '<declaring FQN>'` — every member message starts with it. */ readonly label: string; @@ -319,12 +320,9 @@ interface MemberCtx { * The FQN a member message names for `entity`: the DECLARING entity in place of * the host, so a failure is worded identically whichever entity reached the * member (the suffix names the inheritor) and the ErrorSink repeat test holds. - * A report's @spine walk (R8) has no declaring entity (`declaring` is the - * report), so there every entity names itself. */ function shown(ctx: MemberCtx, entity: MetaData): string { - const declaredByEntity = isSelfOrAncestor(ctx.declaring, ctx.host); - return (entity === ctx.host && declaredByEntity ? ctx.declaring : entity).resolutionKey(); + return (entity === ctx.host ? ctx.declaring : entity).resolutionKey(); } function checkEntityMembers(root: MetaData, entity: MetaData, declaredHere: boolean, sink: ErrorSink): void { @@ -360,7 +358,7 @@ function checkDimension(ctx: MemberCtx, dim: MetaDimension): void { let ofEntity = ctx.host; const via = dim.via(); if (via !== undefined) { - const terminal = walkToOneVia(ctx, via, err); + const terminal = walkToOneVia(ctx.root, ctx.host, via, err, d2Walk(ctx.declaring)); if (terminal === undefined) return; ofEntity = terminal; } @@ -406,38 +404,82 @@ function checkDimension(ctx: MemberCtx, dim: MetaDimension): void { } } -/** How a to-one walk's messages name it: the attribute that holds the path (`via`, `spine`) and - * what its Owner must be (`the owning entity '<FQN>'`, `@from '<FQN>'`). */ -interface WalkWording { +/** + * What a to-one walk resolves against and how its messages name it. D2 walks a + * dimension's @via (`d2Walk`); R8 walks a report's @spine from @from + * (`spineWalk`). Every difference between the two is a field here, so each port + * copies one explicit rule. + */ +interface ToOneWalk { + /** The attribute holding the path: `via` (D2) or `spine` (R8). */ readonly attr: string; + /** The package Owner resolves in (ADR-0042): the declaring entity's (D2) or the report's (R8). */ + readonly ownerPkg: string; + /** The FQN named for the walk's first entity: the declaring entity (D2) or @from (R8). */ + readonly hostName: string; + /** What Owner must be: `the owning entity '<FQN>'` (D2) or `@from '<FQN>'` (R8). */ readonly start: string; + /** The same, in the malformed-path message: `the owning entity` (D2) or `@from '<FQN>'` (R8). */ + readonly startShort: string; + /** The sentence that ends the to-many message: why only to-one hops are followed. */ + readonly toOneReason: string; +} + +/** D2 — a dimension's @via, declared on `declaring`. Reproduces D2's messages exactly. */ +function d2Walk(declaring: MetaData): ToOneWalk { + const declaringKey = declaring.resolutionKey(); + return { + attr: REPORTING_ATTR_VIA, + ownerPkg: pkgOf(declaring), + hostName: declaringKey, + start: `the owning entity '${declaringKey}'`, + startShort: "the owning entity", + toOneReason: + "A dimension follows only @cardinality: one relationships and identity.reference hops, so grouping " + + "can never multiply the measured rows.", + }; +} + +/** R8 — a report's @spine, started at @from (`fromKey`); Owner resolves in the report's package. */ +function spineWalk(report: MetaData, fromKey: string): ToOneWalk { + return { + attr: OBJECT_REPORT_ATTR_SPINE, + ownerPkg: pkgOf(report), + hostName: fromKey, + start: `@from '${fromKey}'`, + startShort: `@from '${fromKey}'`, + toOneReason: + "A @spine follows only @cardinality: one relationships and identity.reference hops, so each fact row " + + "joins at most one row of the spine entity and is never counted twice.", + }; } /** - * D2 — walk `Owner.hop[.hop...]`: Owner is the owning entity, and every hop is a - * to-one `relationship.*` or an `identity.reference`. Returns the terminal - * entity, or undefined after reporting the first failure. R8 runs the same walk - * over a report's @spine, started at @from; `wording` names it (the default is - * D2's own wording). + * Walk `Owner.hop[.hop...]` from `host`: Owner is `host` or an entity it + * extends, and every hop is a to-one `relationship.*` or an + * `identity.reference`. Returns the terminal entity, or undefined after + * reporting the first failure. D2 and R8 both run it; `walk` says which. */ function walkToOneVia( - ctx: MemberCtx, + root: MetaData, + host: MetaData, via: string, err: (message: string) => void, - wording: WalkWording = { attr: REPORTING_ATTR_VIA, start: `the owning entity '${ctx.declaring.resolutionKey()}'` }, + walk: ToOneWalk, ): MetaData | undefined { - const named = `@${wording.attr} '${via}'`; + const named = `@${walk.attr} '${via}'`; + const nameOf = (entity: MetaData): string => (entity === host ? walk.hostName : entity.resolutionKey()); const parts = splitDotted(via); if (parts === undefined) { - err(`${named} must be Owner.hop[.hop...], starting at the owning entity.`); + err(`${named} must be Owner.hop[.hop...], starting at ${walk.startShort}.`); return undefined; } - const owner = resolveObjectRef(ctx.root, parts.owner, pkgOf(ctx.declaring)).node; - if (!isSelfOrAncestor(owner, ctx.host)) { - err(`${named} must start at ${wording.start}.`); + const owner = resolveObjectRef(root, parts.owner, walk.ownerPkg).node; + if (!isSelfOrAncestor(owner, host)) { + err(`${named} must start at ${walk.start}.`); return undefined; } - let current = ctx.host; + let current = host; for (const hopName of parts.path) { const hop = childOfType(current, TYPE_RELATIONSHIP, hopName) ?? @@ -447,25 +489,24 @@ function walkToOneVia( if (hop === undefined) { err( `${named} names '${hopName}', which is not a relationship or identity.reference of ` + - `'${shown(ctx, current)}'.`, + `'${nameOf(current)}'.`, ); return undefined; } const isReference = hop.type === TYPE_IDENTITY; if (!isReference && hop.attr(RELATIONSHIP_ATTR_CARDINALITY) !== CARDINALITY_ONE) { err( - `${named} crosses relationship '${hopName}' on '${shown(ctx, current)}', which is not to-one. ` + - `A dimension follows only @cardinality: one relationships and identity.reference hops, so grouping ` + - `can never multiply the measured rows.`, + `${named} crosses relationship '${hopName}' on '${nameOf(current)}', which is not to-one. ` + + walk.toOneReason, ); return undefined; } const targetRef = hop.attr(isReference ? IDENTITY_REFERENCE_ATTR_REFERENCES : RELATIONSHIP_ATTR_OBJECT_REF); // ADR-0042 — a hop target resolves in the package of the entity declaring the hop. const target = - typeof targetRef === "string" ? resolveObjectRef(ctx.root, targetRef, pkgOf(current)).node : undefined; + typeof targetRef === "string" ? resolveObjectRef(root, targetRef, pkgOf(current)).node : undefined; if (target === undefined) { - err(`${named} hop '${hopName}' on '${shown(ctx, current)}' targets no object.`); + err(`${named} hop '${hopName}' on '${nameOf(current)}' targets no object.`); return undefined; } current = target; @@ -502,8 +543,8 @@ function checkMeasure(ctx: MemberCtx, measure: MetaMeasure): void { !NUMERIC_FIELD_SUBTYPES.includes(ofField.subType) ) { err( - `@default is a number, but @agg '${agg}' of '${measure.ofColumns()[0]}' is a field.${ofField.subType}. ` + - `A default is supported on numeric measures only.`, + `@default is a number, but '${measure.ofColumns()[0]}', the @of of @agg '${agg}', is a ` + + `field.${ofField.subType}. A default is supported on numeric measures only.`, ); } } @@ -728,8 +769,8 @@ function checkReport(root: MetaData, report: MetaData, sink: ErrorSink): void { } } - // R1 — @from resolves to an object.entity. Without it, R2/R3/R6/R7 and the - // @filter have nothing to resolve against, so they are skipped. + // R1 — @from resolves to an object.entity. Without it, R2/R3/R6/R7, R8/R9 and + // the @filter have nothing to resolve against, so they are skipped. const fromRef = reportFrom(report); if (fromRef === undefined) return; // missing @from is ERR_MISSING_REQUIRED_ATTR const from = resolveObjectRef(root, fromRef, pkgOf(report)).node; @@ -817,12 +858,7 @@ function checkReport(root: MetaData, report: MetaData, sink: ErrorSink): void { // R8 — @spine is a to-one path from @from: rule D2's walk, started at @from. const spine = reportSpine(report); if (spine !== undefined) { - const terminal = walkToOneVia( - { root, host: from, declaring: report, label, suffix: "", sink }, - spine, - (message) => err(`: ${message}`), - { attr: OBJECT_REPORT_ATTR_SPINE, start: `@from '${fromKey}'` }, - ); + const terminal = walkToOneVia(root, from, spine, (message) => err(`: ${message}`), spineWalk(report, fromKey)); // R9 — every listed dimension is reached through the spine. Skipped when R8 failed. if (terminal !== undefined) { const spineHops = splitDotted(spine)?.path ?? []; @@ -833,21 +869,33 @@ function checkReport(root: MetaData, report: MetaData, sink: ErrorSink): void { `'${terminal.resolutionKey()}'; with no dimension it would be one totals row.`, ); } + // One verdict per dimension, however many grains list it. + const checkedDimensions = new Set<string>(); for (const item of items) { + if (checkedDimensions.has(item.name)) continue; + checkedDimensions.add(item.name); const dim = childOfType(from, TYPE_DIMENSION, item.name); if (!(dim instanceof MetaDimension)) continue; // R2 already reported it const via = dim.via(); - const hops = via === undefined ? undefined : splitDotted(via)?.path; + if (via === undefined) { + err( + `: dimension '${item.name}' is read from @from '${fromKey}', so it has no value in a row that has ` + + `no facts. With @spine '${spine}' every dimension must be reached through it: declare the ` + + `dimension over a field of '${terminal.resolutionKey()}' (or an entity to-one from it) with an ` + + `@via that begins '${spine}'.`, + ); + continue; + } + // A @via that does not walk is D2's error, on the dimension; R9 does not report it again. + const discard = (): void => undefined; + if (walkToOneVia(root, from, via, discard, d2Walk(dim.parent ?? from)) === undefined) continue; // Hop names are compared as written; the owner segment is not compared. - if (hops !== undefined && spineHops.every((h, i) => hops[i] === h)) continue; + const hops = splitDotted(via)?.path ?? []; + if (spineHops.every((h, i) => hops[i] === h)) continue; err( - via === undefined - ? `: dimension '${item.name}' is read from @from '${fromKey}', so it has no value in a row that has ` + - `no facts. With @spine '${spine}' every dimension must be reached through it: declare the ` + - `dimension over a field of '${terminal.resolutionKey()}' (or an entity to-one from it) with an ` + - `@via that begins '${spine}'.` - : `: dimension '${item.name}' is reached by @via '${via}', which does not begin with the hops of ` + - `@spine '${spine}'. Hop names are compared as written; write the same hops.`, + `: dimension '${item.name}' is reached by @via '${via}', which does not begin with the hops of ` + + `@spine '${spine}'. Hop names are compared as written: if both name the same join, write the same ` + + `hops; otherwise the dimension is not reached through the spine.`, ); } } diff --git a/server/typescript/packages/metadata/test/reporting-validation.test.ts b/server/typescript/packages/metadata/test/reporting-validation.test.ts index 993832d6f..efec5d6eb 100644 --- a/server/typescript/packages/metadata/test/reporting-validation.test.ts +++ b/server/typescript/packages/metadata/test/reporting-validation.test.ts @@ -1150,8 +1150,9 @@ describe("validateReporting — @spine (R8, R9)", () => { const msg = await single(m, "ERR_INVALID_REPORT"); expect(msg).toBe( "report 'acme::shop::ProgramReach': @spine 'Program.purchases' crosses relationship 'purchases' on " + - "'acme::shop::Program', which is not to-one. A dimension follows only @cardinality: one relationships " + - "and identity.reference hops, so grouping can never multiply the measured rows.", + "'acme::shop::Program', which is not to-one. A @spine follows only @cardinality: one relationships " + + "and identity.reference hops, so each fact row joins at most one row of the spine entity and is never " + + "counted twice.", ); }); @@ -1175,7 +1176,23 @@ describe("validateReporting — @spine (R8, R9)", () => { test("R8: a spine with no hop is refused", async () => { const m = edit((x) => addSpineReport(x, { "@spine": "Purchase" })); const msg = await single(m, "ERR_INVALID_REPORT"); - expect(msg).toContain("report 'acme::shop::ProgramPurchases': @spine 'Purchase' must be Owner.hop[.hop...]"); + expect(msg).toBe( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase' must be Owner.hop[.hop...], starting at " + + "@from 'acme::shop::Purchase'.", + ); + }); + + test("D2's to-many wording is unchanged by the walk's wording argument", async () => { + const m = edit((x) => + setChild(x, "Program", "buyerEmail", { + "dimension.attribute": { name: "buyerEmail", "@of": "Purchase.customerEmail", "@via": "Program.purchases" }, + }), + ); + expect(await single(m, "ERR_INVALID_DIMENSION")).toBe( + "dimension 'buyerEmail' on entity 'acme::shop::Program': @via 'Program.purchases' crosses relationship " + + "'purchases' on 'acme::shop::Program', which is not to-one. A dimension follows only @cardinality: one " + + "relationships and identity.reference hops, so grouping can never multiply the measured rows.", + ); }); test("D2's own wording is unchanged by the walk's wording argument", async () => { @@ -1294,10 +1311,20 @@ describe("validateReporting — @spine (R8, R9)", () => { expect(msg).toBe( "report 'acme::shop::ProgramPurchases': dimension 'giftProgramTitle' is reached by @via " + "'Purchase.giftProgramRef', which does not begin with the hops of @spine 'Purchase.programRef'. Hop names " + - "are compared as written; write the same hops.", + "are compared as written: if both name the same join, write the same hops; otherwise the dimension is " + + "not reached through the spine.", ); }); + test("R9: a dimension whose own @via fails D2 is reported once, by D2, not also by R9", async () => { + const m = edit((x) => { + patchMember(x, "Purchase", "programTitle", { "@via": "Purchase.programm" }); + addSpineReport(x); + }); + const msg = await single(m, "ERR_INVALID_DIMENSION"); + expect(msg).toContain("@via 'Purchase.programm' names 'programm'"); + }); + test("R9: the same join, named by the relationship in @spine and by the reference in @via, is refused", async () => { const m = edit((x) => { childrenOf(x, "Purchase").push(programTitleByRef()); @@ -1374,8 +1401,18 @@ describe("validateReporting — measure @default (M7, M8)", () => { const m = edit((x) => patchMember(x, "WorkoutEvent", "lastActivityAt", { "@default": 0 })); const msg = await single(m, "ERR_INVALID_MEASURE"); expect(msg).toBe( - "measure 'lastActivityAt' on entity 'acme::shop::WorkoutEvent': @default is a number, but @agg 'max' of " + - "'WorkoutEvent.occurredAt' is a field.timestamp. A default is supported on numeric measures only.", + "measure 'lastActivityAt' on entity 'acme::shop::WorkoutEvent': @default is a number, but " + + "'WorkoutEvent.occurredAt', the @of of @agg 'max', is a field.timestamp. A default is supported on " + + "numeric measures only.", + ); + }); + + test("a non-integer @default on a count reports M7 and the attribute type error", async () => { + const m = edit((x) => patchMember(x, "Purchase", "purchases", { "@default": "zero" })); + const { errors } = await loadInline(m); + expect(errors.map((e) => e.code).sort()).toEqual(["ERR_BAD_ATTR_VALUE", "ERR_INVALID_MEASURE"]); + expect(errors.find((e) => e.code === "ERR_INVALID_MEASURE")!.message).toContain( + "measure 'purchases' on entity 'acme::shop::Purchase': @default cannot apply to @agg: count.", ); }); From 19504f15458f24874b99c1fd619782f86ccf1692 Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 19:25:53 -0400 Subject: [PATCH 05/18] test(conformance): @spine and measure @default fixtures (2 positive, 8 error cases) Two positive fixtures (reporting-spine-and-default, reporting-spine-inherited) and the eight error fixtures of Table B, each the positive input with one change and one expected error: R8 (to-many hop, owner not @from), R9 (no dimensions, a time dimension over a fact column, a dimension through the reference's name for the spine's join), M7 (@default on a count), M8 (@default on a max of a timestamp) and the type check (@default: 0.5). attr.int and attr.long now refuse a fractional number in TypeScript: a fractional number in any int/long attribute fails to load with ERR_BAD_ATTR_VALUE (input with no valid meaning; it used to pass the attribute type check unchanged). attr.double is unchanged. No fixture, example or test model carried such a value. Metamodel corpus count 364 -> 374. The registry coverage snapshot is regenerated: spine and default are now exercised, and the stale `sortable` entries on field.double and field.float are dropped (already exercised by the api-contract projection fixtures). CAPABILITIES.json regenerates unchanged. The C#, Java, Kotlin and Python conformance lanes are red on the new fixtures until their own tasks. --- AGENTS.md | 2 +- README.md | 2 +- docs/CONFORMANCE.md | 6 +- examples/showcase/site-payload.json | 2 +- .../expected-errors.json | 15 + .../input/meta.shop.json | 321 +++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 + .../input/meta.shop.json | 320 +++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 + .../input/meta.shop.json | 321 +++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 + .../input/meta.shop.json | 321 +++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 + .../input/meta.shop.json | 321 +++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 + .../input/meta.shop.json | 317 +++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 + .../input/meta.shop.json | 320 +++++++++++++++++ .../providers.json | 1 + .../expected-errors.json | 15 + .../input/meta.shop.json | 327 +++++++++++++++++ .../error-report-spine-to-many/providers.json | 1 + .../reporting-spine-and-default/expected.json | 332 ++++++++++++++++++ .../input/meta.shop.json | 320 +++++++++++++++++ .../providers.json | 1 + .../reporting-spine-inherited/expected.json | 141 ++++++++ .../input/meta.shop.json | 139 ++++++++ .../reporting-spine-inherited/providers.json | 1 + .../registry-conformance/coverage-report.json | 20 -- .../metadata/src/core/attr/meta-attr.ts | 10 +- .../metadata/test/meta-attr-behavior.test.ts | 29 ++ 37 files changed, 3674 insertions(+), 27 deletions(-) create mode 100644 fixtures/conformance/error-measure-default-non-numeric/expected-errors.json create mode 100644 fixtures/conformance/error-measure-default-non-numeric/input/meta.shop.json create mode 100644 fixtures/conformance/error-measure-default-non-numeric/providers.json create mode 100644 fixtures/conformance/error-measure-default-not-integer/expected-errors.json create mode 100644 fixtures/conformance/error-measure-default-not-integer/input/meta.shop.json create mode 100644 fixtures/conformance/error-measure-default-not-integer/providers.json create mode 100644 fixtures/conformance/error-measure-default-on-count/expected-errors.json create mode 100644 fixtures/conformance/error-measure-default-on-count/input/meta.shop.json create mode 100644 fixtures/conformance/error-measure-default-on-count/providers.json create mode 100644 fixtures/conformance/error-report-spine-dimension-off-spine/expected-errors.json create mode 100644 fixtures/conformance/error-report-spine-dimension-off-spine/input/meta.shop.json create mode 100644 fixtures/conformance/error-report-spine-dimension-off-spine/providers.json create mode 100644 fixtures/conformance/error-report-spine-dimension-other-path/expected-errors.json create mode 100644 fixtures/conformance/error-report-spine-dimension-other-path/input/meta.shop.json create mode 100644 fixtures/conformance/error-report-spine-dimension-other-path/providers.json create mode 100644 fixtures/conformance/error-report-spine-no-dimensions/expected-errors.json create mode 100644 fixtures/conformance/error-report-spine-no-dimensions/input/meta.shop.json create mode 100644 fixtures/conformance/error-report-spine-no-dimensions/providers.json create mode 100644 fixtures/conformance/error-report-spine-not-from/expected-errors.json create mode 100644 fixtures/conformance/error-report-spine-not-from/input/meta.shop.json create mode 100644 fixtures/conformance/error-report-spine-not-from/providers.json create mode 100644 fixtures/conformance/error-report-spine-to-many/expected-errors.json create mode 100644 fixtures/conformance/error-report-spine-to-many/input/meta.shop.json create mode 100644 fixtures/conformance/error-report-spine-to-many/providers.json create mode 100644 fixtures/conformance/reporting-spine-and-default/expected.json create mode 100644 fixtures/conformance/reporting-spine-and-default/input/meta.shop.json create mode 100644 fixtures/conformance/reporting-spine-and-default/providers.json create mode 100644 fixtures/conformance/reporting-spine-inherited/expected.json create mode 100644 fixtures/conformance/reporting-spine-inherited/input/meta.shop.json create mode 100644 fixtures/conformance/reporting-spine-inherited/providers.json diff --git a/AGENTS.md b/AGENTS.md index 71af89cbf..36cdac57a 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -75,7 +75,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken. - **Kotlin** — `codegen-kotlin` (KotlinPoet on JVM): entity + Exposed table + Spring controller + payload + relations + filter allowlist + validator + stored-proc + output-parser generators. `integration-tests-kotlin` runs the persistence-conformance corpus through Exposed against Testcontainers Postgres. **Cross-port conformance corpora** (every port runs the shared corpus): -- Metamodel: `fixtures/conformance/` (364 fixtures; 28 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. +- Metamodel: `fixtures/conformance/` (374 fixtures; 28 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. - Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical. - Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) - API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived. diff --git a/README.md b/README.md index e5f52a94b..36beca94d 100644 --- a/README.md +++ b/README.md @@ -261,7 +261,7 @@ metaobjects/ ├── CLAUDE.md # project instructions for Claude ├── spec/ # canonical metamodel docs, ADRs, roadmap ├── fixtures/ # 25 cross-language conformance corpora — the oracle -│ ├── conformance/ # metamodel (loader + serializer + navigation), 364 fixtures +│ ├── conformance/ # metamodel (loader + serializer + navigation), 374 fixtures │ ├── yaml-conformance/ # YAML authoring desugar │ ├── render-conformance/ # FR-004 byte-identical render oracle │ ├── verify-conformance/ # FR-004 template-drift gate diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 96f595768..5cd772b91 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -26,7 +26,7 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor | Corpus | Fixtures | TS | Java | Kotlin | C# | Python | |---|---|---|---|---|---|---| -| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 364 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | +| [`fixtures/conformance/`](../fixtures/conformance/) (metamodel) | 374 | ✓ | ✓ | inherits via `metadata-ktx` | ✓ | ✓ | | [`fixtures/yaml-conformance/`](../fixtures/yaml-conformance/) | 16 | 16 / 16 | 15 / 16 (1 ledgered: `yaml-quoted-leading-zero` — Java pipeline strips quotes off `"007"`) | inherits via Java | 15 / 16 (1 ledgered: `error-yaml-coerced-hex-in-string` — YamlDotNet doesn't coerce `0xFF`) | 16 / 16 | | [`fixtures/verify-conformance/`](../fixtures/verify-conformance/) | 31 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/verify-strict-conformance/`](../fixtures/verify-strict-conformance/) | 1 | ✓ | — | — | — | ✓ | @@ -221,7 +221,7 @@ unit-test runners (`bun test`, `dotnet test`, `pytest`, `mvn test`) pull Docker. ## Fixture-to-doc mapping -### `fixtures/conformance/` — metamodel loader + canonical serializer (364) +### `fixtures/conformance/` — metamodel loader + canonical serializer (374) | Fixture prefix | Feature doc | |---|---| @@ -459,7 +459,7 @@ Kotlin's identity function is the Java one; the names its generator emits are as ## Orphaned fixtures (tested but not yet documented) -The fixtures in the nine corpora mapped above (metamodel 364 + yaml 16 + verify 31 +The fixtures in the nine corpora mapped above (metamodel 374 + yaml 16 + verify 31 + render 15 + persistence 39 + api-contract 78 + source-resolution 25 + scope 10 + dependency 23) each map to a feature doc, and so do the two requirement corpora, whose case lists live in their own READMEs. None are orphaned today. The remaining diff --git a/examples/showcase/site-payload.json b/examples/showcase/site-payload.json index e84e9bead..0239b3ed9 100644 --- a/examples/showcase/site-payload.json +++ b/examples/showcase/site-payload.json @@ -7,7 +7,7 @@ "metamodel": "1.1" }, "counts": { - "fixtures": 364, + "fixtures": 374, "corpora": 28, "baseTypes": 17 }, diff --git a/fixtures/conformance/error-measure-default-non-numeric/expected-errors.json b/fixtures/conformance/error-measure-default-non-numeric/expected-errors.json new file mode 100644 index 000000000..98816d49a --- /dev/null +++ b/fixtures/conformance/error-measure-default-non-numeric/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_MEASURE", + "source": { + "format": "json", + "files": [ + "meta.shop.json" + ], + "jsonPath": "$['metadata.root'].children[2]['object.entity'].children[21]['measure.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-measure-default-non-numeric/input/meta.shop.json b/fixtures/conformance/error-measure-default-non-numeric/input/meta.shop.json new file mode 100644 index 000000000..6328887d2 --- /dev/null +++ b/fixtures/conformance/error-measure-default-non-numeric/input/meta.shop.json @@ -0,0 +1,321 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id" + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt", + "@default": 0 + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": [ + "catalogName" + ], + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-measure-default-non-numeric/providers.json b/fixtures/conformance/error-measure-default-non-numeric/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-measure-default-non-numeric/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-measure-default-not-integer/expected-errors.json b/fixtures/conformance/error-measure-default-not-integer/expected-errors.json new file mode 100644 index 000000000..5ca695bf3 --- /dev/null +++ b/fixtures/conformance/error-measure-default-not-integer/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_BAD_ATTR_VALUE", + "source": { + "format": "json", + "files": [ + "meta.shop.json" + ], + "jsonPath": "$['metadata.root'].children[2]['object.entity'].children[18]['measure.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-measure-default-not-integer/input/meta.shop.json b/fixtures/conformance/error-measure-default-not-integer/input/meta.shop.json new file mode 100644 index 000000000..138c4e15e --- /dev/null +++ b/fixtures/conformance/error-measure-default-not-integer/input/meta.shop.json @@ -0,0 +1,320 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id" + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0.5 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt" + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": [ + "catalogName" + ], + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-measure-default-not-integer/providers.json b/fixtures/conformance/error-measure-default-not-integer/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-measure-default-not-integer/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-measure-default-on-count/expected-errors.json b/fixtures/conformance/error-measure-default-on-count/expected-errors.json new file mode 100644 index 000000000..57f749961 --- /dev/null +++ b/fixtures/conformance/error-measure-default-on-count/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_MEASURE", + "source": { + "format": "json", + "files": [ + "meta.shop.json" + ], + "jsonPath": "$['metadata.root'].children[2]['object.entity'].children[17]['measure.aggregate']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-measure-default-on-count/input/meta.shop.json b/fixtures/conformance/error-measure-default-on-count/input/meta.shop.json new file mode 100644 index 000000000..49facbfc1 --- /dev/null +++ b/fixtures/conformance/error-measure-default-on-count/input/meta.shop.json @@ -0,0 +1,321 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt" + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": [ + "catalogName" + ], + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-measure-default-on-count/providers.json b/fixtures/conformance/error-measure-default-on-count/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-measure-default-on-count/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-report-spine-dimension-off-spine/expected-errors.json b/fixtures/conformance/error-report-spine-dimension-off-spine/expected-errors.json new file mode 100644 index 000000000..a9b4e17c5 --- /dev/null +++ b/fixtures/conformance/error-report-spine-dimension-off-spine/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_REPORT", + "source": { + "format": "json", + "files": [ + "meta.shop.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.report']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-report-spine-dimension-off-spine/input/meta.shop.json b/fixtures/conformance/error-report-spine-dimension-off-spine/input/meta.shop.json new file mode 100644 index 000000000..6ed05fc61 --- /dev/null +++ b/fixtures/conformance/error-report-spine-dimension-off-spine/input/meta.shop.json @@ -0,0 +1,321 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id" + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt" + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName", + "purchasedAt:day" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": [ + "catalogName" + ], + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-report-spine-dimension-off-spine/providers.json b/fixtures/conformance/error-report-spine-dimension-off-spine/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-report-spine-dimension-off-spine/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-report-spine-dimension-other-path/expected-errors.json b/fixtures/conformance/error-report-spine-dimension-other-path/expected-errors.json new file mode 100644 index 000000000..a9b4e17c5 --- /dev/null +++ b/fixtures/conformance/error-report-spine-dimension-other-path/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_REPORT", + "source": { + "format": "json", + "files": [ + "meta.shop.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.report']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-report-spine-dimension-other-path/input/meta.shop.json b/fixtures/conformance/error-report-spine-dimension-other-path/input/meta.shop.json new file mode 100644 index 000000000..d42d3d5b5 --- /dev/null +++ b/fixtures/conformance/error-report-spine-dimension-other-path/input/meta.shop.json @@ -0,0 +1,321 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id" + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt" + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName", + "programByRef" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": [ + "catalogName" + ], + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-report-spine-dimension-other-path/providers.json b/fixtures/conformance/error-report-spine-dimension-other-path/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-report-spine-dimension-other-path/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-report-spine-no-dimensions/expected-errors.json b/fixtures/conformance/error-report-spine-no-dimensions/expected-errors.json new file mode 100644 index 000000000..c9e5884c3 --- /dev/null +++ b/fixtures/conformance/error-report-spine-no-dimensions/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_REPORT", + "source": { + "format": "json", + "files": [ + "meta.shop.json" + ], + "jsonPath": "$['metadata.root'].children[4]['object.report']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-report-spine-no-dimensions/input/meta.shop.json b/fixtures/conformance/error-report-spine-no-dimensions/input/meta.shop.json new file mode 100644 index 000000000..ae6fd4b0f --- /dev/null +++ b/fixtures/conformance/error-report-spine-no-dimensions/input/meta.shop.json @@ -0,0 +1,317 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id" + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt" + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-report-spine-no-dimensions/providers.json b/fixtures/conformance/error-report-spine-no-dimensions/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-report-spine-no-dimensions/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-report-spine-not-from/expected-errors.json b/fixtures/conformance/error-report-spine-not-from/expected-errors.json new file mode 100644 index 000000000..a9b4e17c5 --- /dev/null +++ b/fixtures/conformance/error-report-spine-not-from/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_REPORT", + "source": { + "format": "json", + "files": [ + "meta.shop.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.report']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-report-spine-not-from/input/meta.shop.json b/fixtures/conformance/error-report-spine-not-from/input/meta.shop.json new file mode 100644 index 000000000..2f2a850ff --- /dev/null +++ b/fixtures/conformance/error-report-spine-not-from/input/meta.shop.json @@ -0,0 +1,320 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id" + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt" + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Program.catalog", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": [ + "catalogName" + ], + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-report-spine-not-from/providers.json b/fixtures/conformance/error-report-spine-not-from/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-report-spine-not-from/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/error-report-spine-to-many/expected-errors.json b/fixtures/conformance/error-report-spine-to-many/expected-errors.json new file mode 100644 index 000000000..a9b4e17c5 --- /dev/null +++ b/fixtures/conformance/error-report-spine-to-many/expected-errors.json @@ -0,0 +1,15 @@ +{ + "errors": [ + { + "code": "ERR_INVALID_REPORT", + "source": { + "format": "json", + "files": [ + "meta.shop.json" + ], + "jsonPath": "$['metadata.root'].children[3]['object.report']" + } + } + ], + "warnings": [] +} diff --git a/fixtures/conformance/error-report-spine-to-many/input/meta.shop.json b/fixtures/conformance/error-report-spine-to-many/input/meta.shop.json new file mode 100644 index 000000000..085863924 --- /dev/null +++ b/fixtures/conformance/error-report-spine-to-many/input/meta.shop.json @@ -0,0 +1,327 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + }, + { + "relationship.association": { + "name": "purchases", + "@objectRef": "Purchase", + "@cardinality": "many" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id" + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt" + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program.purchases", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": [ + "catalogName" + ], + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/error-report-spine-to-many/providers.json b/fixtures/conformance/error-report-spine-to-many/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/error-report-spine-to-many/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/reporting-spine-and-default/expected.json b/fixtures/conformance/reporting-spine-and-default/expected.json new file mode 100644 index 000000000..1c3cbfc49 --- /dev/null +++ b/fixtures/conformance/reporting-spine-and-default/expected.json @@ -0,0 +1,332 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@fields": [ + "catalogId" + ], + "@references": "Catalog" + } + }, + { + "relationship.association": { + "name": "catalog", + "@cardinality": "one", + "@objectRef": "Catalog" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@fields": [ + "programId" + ], + "@references": "Program" + } + }, + { + "relationship.association": { + "name": "program", + "@cardinality": "one", + "@objectRef": "Program" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": { + "eq": "active" + } + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@grains": [ + "month", + "year" + ], + "@of": "Program.publishedAt", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@grains": [ + "day", + "month" + ], + "@of": "Purchase.purchasedAt" + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": [ + "Purchase.id" + ] + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@default": 0, + "@of": [ + "Purchase.amountCents" + ] + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@default": 0, + "@of": [ + "Purchase.amountCents" + ] + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@default": -1, + "@of": [ + "Purchase.amountCents" + ] + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": [ + "Purchase.purchasedAt" + ] + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@default": 0, + "@denominator": "purchases", + "@numerator": "revenue" + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName" + ], + "@from": "Purchase", + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "@spine": "Purchase.program", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@dimensions": [ + "catalogName" + ], + "@from": "Purchase", + "@measures": [ + "purchases", + "revenue" + ], + "@spine": "Purchase.program.catalog" + } + }, + { + "object.report": { + "name": "SalesByDay", + "@dimensions": [ + "purchasedAt:day" + ], + "@from": "Purchase", + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/reporting-spine-and-default/input/meta.shop.json b/fixtures/conformance/reporting-spine-and-default/input/meta.shop.json new file mode 100644 index 000000000..52e17863a --- /dev/null +++ b/fixtures/conformance/reporting-spine-and-default/input/meta.shop.json @@ -0,0 +1,320 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Catalog", + "children": [ + { + "source.rdb": { + "@table": "catalogs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "name", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "field.timestamp": { + "name": "publishedAt" + } + }, + { + "field.long": { + "name": "catalogId" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkCatalog", + "@references": "Catalog", + "@fields": [ + "catalogId" + ] + } + }, + { + "relationship.association": { + "name": "catalog", + "@objectRef": "Catalog", + "@cardinality": "one" + } + } + ] + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + { + "source.rdb": { + "@table": "purchases" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.string": { + "name": "customerEmail" + } + }, + { + "field.currency": { + "name": "amountCents" + } + }, + { + "field.string": { + "name": "status" + } + }, + { + "field.timestamp": { + "name": "purchasedAt" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "segment.filter": { + "name": "active", + "@filter": { + "status": "active" + } + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program" + } + }, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": [ + "month", + "year" + ] + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": [ + "day", + "month" + ] + } + }, + { + "dimension.attribute": { + "name": "programByRef", + "@of": "Program.title", + "@via": "Purchase.fkProgram" + } + }, + { + "dimension.attribute": { + "name": "catalogName", + "@of": "Catalog.name", + "@via": "Purchase.program.catalog" + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id" + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "avgAmount", + "@agg": "avg", + "@of": "Purchase.amountCents", + "@default": 0 + } + }, + { + "measure.aggregate": { + "name": "smallest", + "@agg": "min", + "@of": "Purchase.amountCents", + "@default": -1 + } + }, + { + "measure.aggregate": { + "name": "lastPurchaseAt", + "@agg": "max", + "@of": "Purchase.purchasedAt" + } + }, + { + "measure.ratio": { + "name": "revenuePerPurchase", + "@numerator": "revenue", + "@denominator": "purchases", + "@default": 0 + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programId", + "programTitle", + "publishedAt:month", + "catalogName" + ], + "@measures": [ + "purchases", + "revenue", + "avgAmount", + "smallest", + "lastPurchaseAt", + "revenuePerPurchase" + ], + "@segment": "active", + "children": [ + { + "source.rdb": { + "@kind": "view", + "@view": "v_program_sales" + } + } + ] + } + }, + { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": [ + "catalogName" + ], + "@measures": [ + "purchases", + "revenue" + ] + } + }, + { + "object.report": { + "name": "SalesByDay", + "@from": "Purchase", + "@dimensions": [ + "purchasedAt:day" + ], + "@measures": [ + "revenue" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/reporting-spine-and-default/providers.json b/fixtures/conformance/reporting-spine-and-default/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/reporting-spine-and-default/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/conformance/reporting-spine-inherited/expected.json b/fixtures/conformance/reporting-spine-inherited/expected.json new file mode 100644 index 000000000..e621c266d --- /dev/null +++ b/fixtures/conformance/reporting-spine-inherited/expected.json @@ -0,0 +1,141 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "BaseEvent", + "abstract": true, + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.int": { + "name": "minutes" + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@fields": [ + "programId" + ], + "@references": "Program" + } + }, + { + "relationship.association": { + "name": "program", + "@cardinality": "one", + "@objectRef": "Program" + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "BaseEvent.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "BaseEvent.program" + } + }, + { + "measure.aggregate": { + "name": "totalMinutes", + "@agg": "sum", + "@default": 0, + "@of": [ + "BaseEvent.minutes" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "WorkoutEvent", + "extends": "BaseEvent", + "children": [ + { + "source.rdb": { + "@table": "workout_events" + } + }, + { + "field.string": { + "name": "eventType" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramMinutes", + "@dimensions": [ + "programId", + "programTitle" + ], + "@from": "WorkoutEvent", + "@measures": [ + "totalMinutes" + ], + "@spine": "BaseEvent.program" + } + } + ] + } +} diff --git a/fixtures/conformance/reporting-spine-inherited/input/meta.shop.json b/fixtures/conformance/reporting-spine-inherited/input/meta.shop.json new file mode 100644 index 000000000..86b629e5f --- /dev/null +++ b/fixtures/conformance/reporting-spine-inherited/input/meta.shop.json @@ -0,0 +1,139 @@ +{ + "metadata.root": { + "package": "acme::shop", + "children": [ + { + "object.entity": { + "name": "Program", + "children": [ + { + "source.rdb": { + "@table": "programs" + } + }, + { + "field.long": { + "name": "id" + } + }, + { + "field.string": { + "name": "title", + "@required": true + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.entity": { + "name": "BaseEvent", + "abstract": true, + "children": [ + { + "field.long": { + "name": "id" + } + }, + { + "field.long": { + "name": "programId" + } + }, + { + "field.int": { + "name": "minutes" + } + }, + { + "identity.reference": { + "name": "fkProgram", + "@references": "Program", + "@fields": [ + "programId" + ] + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one" + } + }, + { + "dimension.attribute": { + "name": "programId", + "@of": "Program.id", + "@via": "BaseEvent.program" + } + }, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "BaseEvent.program" + } + }, + { + "measure.aggregate": { + "name": "totalMinutes", + "@agg": "sum", + "@of": "BaseEvent.minutes", + "@default": 0 + } + } + ] + } + }, + { + "object.entity": { + "name": "WorkoutEvent", + "extends": "BaseEvent", + "children": [ + { + "source.rdb": { + "@table": "workout_events" + } + }, + { + "field.string": { + "name": "eventType" + } + }, + { + "identity.primary": { + "name": "id", + "@fields": [ + "id" + ] + } + } + ] + } + }, + { + "object.report": { + "name": "ProgramMinutes", + "@from": "WorkoutEvent", + "@spine": "BaseEvent.program", + "@dimensions": [ + "programId", + "programTitle" + ], + "@measures": [ + "totalMinutes" + ] + } + } + ] + } +} diff --git a/fixtures/conformance/reporting-spine-inherited/providers.json b/fixtures/conformance/reporting-spine-inherited/providers.json new file mode 100644 index 000000000..f9bb435a3 --- /dev/null +++ b/fixtures/conformance/reporting-spine-inherited/providers.json @@ -0,0 +1 @@ +["metaobjects-core-types","metaobjects-db"] diff --git a/fixtures/registry-conformance/coverage-report.json b/fixtures/registry-conformance/coverage-report.json index 6f9f29fac..22bfd0312 100644 --- a/fixtures/registry-conformance/coverage-report.json +++ b/fixtures/registry-conformance/coverage-report.json @@ -73,7 +73,6 @@ "formExclude", "instruction", "mutability", - "sortable", "sortableDefaultOrder", "unique", "xmlText" @@ -121,7 +120,6 @@ "formExclude", "instruction", "mutability", - "sortable", "sortableDefaultOrder", "unique", "xmlText" @@ -300,24 +298,6 @@ "using" ] }, - { - "key": "measure.aggregate", - "untestedAttrs": [ - "default" - ] - }, - { - "key": "measure.ratio", - "untestedAttrs": [ - "default" - ] - }, - { - "key": "object.report", - "untestedAttrs": [ - "spine" - ] - }, { "key": "relationship.aggregation", "untestedAttrs": [ diff --git a/server/typescript/packages/metadata/src/core/attr/meta-attr.ts b/server/typescript/packages/metadata/src/core/attr/meta-attr.ts index da305f1b1..86ceb9530 100644 --- a/server/typescript/packages/metadata/src/core/attr/meta-attr.ts +++ b/server/typescript/packages/metadata/src/core/attr/meta-attr.ts @@ -95,7 +95,15 @@ export class MetaAttr extends MetaData implements DataTypeAware { if (dt === DATA_TYPE_STRING) { return typeof value === "string" ? [] : [this._typeError("string", value)]; } - if (dt === DATA_TYPE_INT || dt === DATA_TYPE_LONG || dt === DATA_TYPE_DOUBLE) { + if (dt === DATA_TYPE_INT || dt === DATA_TYPE_LONG) { + if (typeof value !== "number") return [this._typeError("number", value)]; + // A fraction coerces through unchanged (coerce keeps a number as-is), and an + // int/long attr has no meaning for it: refuse it here, never round it. + return Number.isInteger(value) + ? [] + : [{ message: `attribute '@${this.name}' must be of type '${this.subType}' but got non-integer number ${value}` }]; + } + if (dt === DATA_TYPE_DOUBLE) { return typeof value === "number" ? [] : [this._typeError("number", value)]; } if (dt === DATA_TYPE_BOOLEAN) { diff --git a/server/typescript/packages/metadata/test/meta-attr-behavior.test.ts b/server/typescript/packages/metadata/test/meta-attr-behavior.test.ts index c316a3fe3..8d4f1dd69 100644 --- a/server/typescript/packages/metadata/test/meta-attr-behavior.test.ts +++ b/server/typescript/packages/metadata/test/meta-attr-behavior.test.ts @@ -8,6 +8,8 @@ import { TYPE_ATTR, ATTR_SUBTYPE_STRING, ATTR_SUBTYPE_INT, + ATTR_SUBTYPE_LONG, + ATTR_SUBTYPE_DOUBLE, ATTR_SUBTYPE_BOOLEAN, ATTR_SUBTYPE_STRINGARRAY, ATTR_SUBTYPE_FILTER, @@ -52,6 +54,33 @@ describe("MetaAttr.coerce", () => { }); }); +describe("MetaAttr.validateValue — int and long hold whole numbers", () => { + it("int and long accept whole numbers, negative included", () => { + for (const subType of [ATTR_SUBTYPE_INT, ATTR_SUBTYPE_LONG]) { + expect(attr(subType).validateValue(0)).toEqual([]); + expect(attr(subType).validateValue(-1)).toEqual([]); + expect(attr(subType).validateValue(42)).toEqual([]); + } + }); + it("int and long refuse a fractional number (it coerces through unchanged, so the check must catch it)", () => { + for (const subType of [ATTR_SUBTYPE_INT, ATTR_SUBTYPE_LONG]) { + const a = attr(subType, "default"); + const errors = a.validateValue(a.coerce(0.5)); + expect(errors).toEqual([ + { message: `attribute '@default' must be of type '${subType}' but got non-integer number 0.5` }, + ]); + } + }); + it("int still refuses a non-number with the existing message", () => { + expect(attr(ATTR_SUBTYPE_INT, "default").validateValue("zero")).toEqual([ + { message: "attribute '@default' must be of type 'int' but got string" }, + ]); + }); + it("double keeps accepting a fractional number", () => { + expect(attr(ATTR_SUBTYPE_DOUBLE).validateValue(0.5)).toEqual([]); + }); +}); + describe("StringArrayAttr", () => { it("dataType is string", () => { expect(new StringArrayAttr(new TypeId(TYPE_ATTR, ATTR_SUBTYPE_STRINGARRAY), "fields").dataType).toBe(DATA_TYPE_STRING); From fc7978d929732d005961d7a452fe25cd2d82be3d Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 19:31:32 -0400 Subject: [PATCH 06/18] fix(metadata): refuse a fractional measure @default without changing attr.int (TypeScript) Reverts the attr.int/attr.long tightening from the previous commit. It also refused a fraction in validator @min/@max, which are registered attr.int but documented as numeric values, so `validator.numeric @min: 0.5` over a double field stopped loading. That changed released vocabulary, which this plan must not do. A fractional number in an int/long attribute loads again as before. A measure's @default is now checked in validateReporting instead: a number that is not an integer on a measure.aggregate or measure.ratio is ERR_BAD_ATTR_VALUE on the measure node, with the text "@default '<value>' is not an integer. A measure's @default is a whole number (for example 0)." The check runs first; when it fires, M7/M8 are skipped. A non-number stays the attribute type check's error alone. error-measure-default-not-integer is unchanged and still yields exactly one error. The registry coverage snapshot gets back the two `sortable` entries (field.decimal, field.float) that the regeneration had dropped, so the file no longer differs from main. --- .../registry-conformance/coverage-report.json | 2 + .../metadata/src/core/attr/meta-attr.ts | 10 +--- .../src/loader/reporting-validation.ts | 29 +++++++++-- .../metadata/test/meta-attr-behavior.test.ts | 29 ----------- .../test/reporting-validation.test.ts | 48 +++++++++++++++++++ 5 files changed, 75 insertions(+), 43 deletions(-) diff --git a/fixtures/registry-conformance/coverage-report.json b/fixtures/registry-conformance/coverage-report.json index 22bfd0312..631dfcee5 100644 --- a/fixtures/registry-conformance/coverage-report.json +++ b/fixtures/registry-conformance/coverage-report.json @@ -73,6 +73,7 @@ "formExclude", "instruction", "mutability", + "sortable", "sortableDefaultOrder", "unique", "xmlText" @@ -120,6 +121,7 @@ "formExclude", "instruction", "mutability", + "sortable", "sortableDefaultOrder", "unique", "xmlText" diff --git a/server/typescript/packages/metadata/src/core/attr/meta-attr.ts b/server/typescript/packages/metadata/src/core/attr/meta-attr.ts index 86ceb9530..da305f1b1 100644 --- a/server/typescript/packages/metadata/src/core/attr/meta-attr.ts +++ b/server/typescript/packages/metadata/src/core/attr/meta-attr.ts @@ -95,15 +95,7 @@ export class MetaAttr extends MetaData implements DataTypeAware { if (dt === DATA_TYPE_STRING) { return typeof value === "string" ? [] : [this._typeError("string", value)]; } - if (dt === DATA_TYPE_INT || dt === DATA_TYPE_LONG) { - if (typeof value !== "number") return [this._typeError("number", value)]; - // A fraction coerces through unchanged (coerce keeps a number as-is), and an - // int/long attr has no meaning for it: refuse it here, never round it. - return Number.isInteger(value) - ? [] - : [{ message: `attribute '@${this.name}' must be of type '${this.subType}' but got non-integer number ${value}` }]; - } - if (dt === DATA_TYPE_DOUBLE) { + if (dt === DATA_TYPE_INT || dt === DATA_TYPE_LONG || dt === DATA_TYPE_DOUBLE) { return typeof value === "number" ? [] : [this._typeError("number", value)]; } if (dt === DATA_TYPE_BOOLEAN) { diff --git a/server/typescript/packages/metadata/src/loader/reporting-validation.ts b/server/typescript/packages/metadata/src/loader/reporting-validation.ts index 38e200ad5..274999109 100644 --- a/server/typescript/packages/metadata/src/loader/reporting-validation.ts +++ b/server/typescript/packages/metadata/src/loader/reporting-validation.ts @@ -7,8 +7,9 @@ // Two design rules hold throughout, so one broken rule yields exactly one error: // - No cascades. A member that fails a structural rule is not checked further // (a dimension whose @via fails D2 skips D1/D3/D4; a measure that fails one -// of M1–M4 skips M7/M8; a report whose @from fails R1 skips R2/R3/R6/R7, -// R8/R9 and its @filter; a report whose @spine fails R8 skips R9, and R9 +// of M1–M4, or whose @default is a fraction, skips M7/M8; a report whose +// @from fails R1 skips R2/R3/R6/R7, R8/R9 and its @filter; a report whose +// @spine fails R8 skips R9, and R9 // skips a dimension whose @via fails D2; an invalid @dimensions/@measures // item derives no report field for R6). // - Each error's `source` is the offending node (the dimension / measure / @@ -134,6 +135,7 @@ const ERR_INVALID_MEASURE: ErrorCode = "ERR_INVALID_MEASURE"; const ERR_INVALID_REPORT: ErrorCode = "ERR_INVALID_REPORT"; const ERR_REPORT_FOREIGN_MEASURE: ErrorCode = "ERR_REPORT_FOREIGN_MEASURE"; const ERR_BAD_ATTR_FILTER: ErrorCode = "ERR_BAD_ATTR_FILTER"; +const ERR_BAD_ATTR_VALUE: ErrorCode = "ERR_BAD_ATTR_VALUE"; // --------------------------------------------------------------------------- // Entry point @@ -518,6 +520,22 @@ function checkMeasure(ctx: MemberCtx, measure: MetaMeasure): void { const err = (message: string): void => ctx.sink.push(measure, ERR_INVALID_MEASURE, ctx.label, `: ${message}`, ctx.suffix); + // The type rule — a measure's @default is a whole number. Checked here, not by + // tightening attr.int, whose other users (validator @min/@max) take a fraction. + // A non-number is the attribute type check's error and is not repeated. It runs + // first; when it fires, M7/M8 are skipped (one mistake, one error). + const declaredDefault = measure.attr(REPORTING_ATTR_DEFAULT); + const fractionalDefault = typeof declaredDefault === "number" && !Number.isInteger(declaredDefault); + if (fractionalDefault) { + ctx.sink.push( + measure, + ERR_BAD_ATTR_VALUE, + ctx.label, + `: @default '${declaredDefault}' is not an integer. A measure's @default is a whole number (for example 0).`, + ctx.suffix, + ); + } + if (measure.isRatio()) { checkRatioOperands(ctx, measure, err); return; @@ -528,9 +546,10 @@ function checkMeasure(ctx: MemberCtx, measure: MetaMeasure): void { const clean = checked !== undefined; const ofField = checked?.field; - // M7 / M8 — where a @default can apply. Only when M1–M4 passed: one mistake, one error. - // The presence test reads the raw attribute, so a mistyped value on a count still reports M7. - if (clean && measure.attr(REPORTING_ATTR_DEFAULT) !== undefined) { + // M7 / M8 — where a @default can apply. Only when M1–M4 passed and the @default is + // not a fraction: one mistake, one error. The presence test reads the raw attribute, + // so a non-number value on a count still reports M7 beside the attribute type error. + if (clean && !fractionalDefault && declaredDefault !== undefined) { const agg = measure.agg(); if (agg === AGG_COUNT) { err( diff --git a/server/typescript/packages/metadata/test/meta-attr-behavior.test.ts b/server/typescript/packages/metadata/test/meta-attr-behavior.test.ts index 8d4f1dd69..c316a3fe3 100644 --- a/server/typescript/packages/metadata/test/meta-attr-behavior.test.ts +++ b/server/typescript/packages/metadata/test/meta-attr-behavior.test.ts @@ -8,8 +8,6 @@ import { TYPE_ATTR, ATTR_SUBTYPE_STRING, ATTR_SUBTYPE_INT, - ATTR_SUBTYPE_LONG, - ATTR_SUBTYPE_DOUBLE, ATTR_SUBTYPE_BOOLEAN, ATTR_SUBTYPE_STRINGARRAY, ATTR_SUBTYPE_FILTER, @@ -54,33 +52,6 @@ describe("MetaAttr.coerce", () => { }); }); -describe("MetaAttr.validateValue — int and long hold whole numbers", () => { - it("int and long accept whole numbers, negative included", () => { - for (const subType of [ATTR_SUBTYPE_INT, ATTR_SUBTYPE_LONG]) { - expect(attr(subType).validateValue(0)).toEqual([]); - expect(attr(subType).validateValue(-1)).toEqual([]); - expect(attr(subType).validateValue(42)).toEqual([]); - } - }); - it("int and long refuse a fractional number (it coerces through unchanged, so the check must catch it)", () => { - for (const subType of [ATTR_SUBTYPE_INT, ATTR_SUBTYPE_LONG]) { - const a = attr(subType, "default"); - const errors = a.validateValue(a.coerce(0.5)); - expect(errors).toEqual([ - { message: `attribute '@default' must be of type '${subType}' but got non-integer number 0.5` }, - ]); - } - }); - it("int still refuses a non-number with the existing message", () => { - expect(attr(ATTR_SUBTYPE_INT, "default").validateValue("zero")).toEqual([ - { message: "attribute '@default' must be of type 'int' but got string" }, - ]); - }); - it("double keeps accepting a fractional number", () => { - expect(attr(ATTR_SUBTYPE_DOUBLE).validateValue(0.5)).toEqual([]); - }); -}); - describe("StringArrayAttr", () => { it("dataType is string", () => { expect(new StringArrayAttr(new TypeId(TYPE_ATTR, ATTR_SUBTYPE_STRINGARRAY), "fields").dataType).toBe(DATA_TYPE_STRING); diff --git a/server/typescript/packages/metadata/test/reporting-validation.test.ts b/server/typescript/packages/metadata/test/reporting-validation.test.ts index efec5d6eb..306d1d728 100644 --- a/server/typescript/packages/metadata/test/reporting-validation.test.ts +++ b/server/typescript/packages/metadata/test/reporting-validation.test.ts @@ -1416,6 +1416,54 @@ describe("validateReporting — measure @default (M7, M8)", () => { ); }); + test("a fractional @default on a sum is ERR_BAD_ATTR_VALUE on the measure node, once", async () => { + const m = edit((x) => patchMember(x, "Purchase", "revenue", { "@default": 0.5 })); + const { errors } = await loadInline(m); + expect(errors.map((e) => e.code)).toEqual(["ERR_BAD_ATTR_VALUE"]); + expect(errors[0]!.message).toBe( + "measure 'revenue' on entity 'acme::shop::Purchase': @default '0.5' is not an integer. A measure's " + + "@default is a whole number (for example 0).", + ); + expect(jsonPathOf(errors[0]!.source)).toContain("['measure.aggregate']"); + }); + + test("a fractional @default on a ratio is ERR_BAD_ATTR_VALUE on the ratio", async () => { + const m = edit((x) => patchMember(x, "WorkoutEvent", "avgDaysPerStarter", { "@default": 0.5 })); + const { errors } = await loadInline(m); + expect(errors.map((e) => e.code)).toEqual(["ERR_BAD_ATTR_VALUE"]); + expect(errors[0]!.message).toBe( + "measure 'avgDaysPerStarter' on entity 'acme::shop::WorkoutEvent': @default '0.5' is not an integer. " + + "A measure's @default is a whole number (for example 0).", + ); + expect(jsonPathOf(errors[0]!.source)).toContain("['measure.ratio']"); + }); + + test("the integer check runs first: a fractional @default on a count skips M7", async () => { + const m = edit((x) => patchMember(x, "Purchase", "purchases", { "@default": 0.5 })); + const msg = await single(m, "ERR_BAD_ATTR_VALUE"); + expect(msg).toContain("@default '0.5' is not an integer"); + }); + + test("the integer check runs first: a fractional @default on a max of a timestamp skips M8", async () => { + const m = edit((x) => patchMember(x, "WorkoutEvent", "lastActivityAt", { "@default": -1.5 })); + const msg = await single(m, "ERR_BAD_ATTR_VALUE"); + expect(msg).toBe( + "measure 'lastActivityAt' on entity 'acme::shop::WorkoutEvent': @default '-1.5' is not an integer. " + + "A measure's @default is a whole number (for example 0).", + ); + }); + + test("a fractional @default on a sum declared on an abstract base is reported ONCE", async () => { + const m = inheritedModel(); + const base = objectBody(m, "BaseEvent").children as Wrapper[]; + base[3] = { "measure.aggregate": { name: "events", "@agg": "sum", "@of": "BaseEvent.id", "@default": 0.5 } }; + const msg = await single(m, "ERR_BAD_ATTR_VALUE"); + expect(msg).toBe( + "measure 'events' on entity 'acme::shop::BaseEvent': @default '0.5' is not an integer. A measure's " + + "@default is a whole number (for example 0).", + ); + }); + test("@default on a sum, an avg, a min of an int and a ratio is fine", async () => { const m = edit((x) => { patchMember(x, "Purchase", "revenue", { "@default": 0 }); From 764b7a84196b36565a62538cdd60fad63b06d00e Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 19:39:25 -0400 Subject: [PATCH 07/18] feat(metadata): a spine key and a defaulted measure are not nullable in a report's shape --- .../src/core/reporting/report-shape.ts | 144 ++++++-- .../typescript/packages/metadata/src/index.ts | 2 + .../metadata/test/report-read-model.test.ts | 29 ++ .../metadata/test/report-shape.test.ts | 313 ++++++++++++++++++ 4 files changed, 463 insertions(+), 25 deletions(-) diff --git a/server/typescript/packages/metadata/src/core/reporting/report-shape.ts b/server/typescript/packages/metadata/src/core/reporting/report-shape.ts index e7c04ac27..582adcb6c 100644 --- a/server/typescript/packages/metadata/src/core/reporting/report-shape.ts +++ b/server/typescript/packages/metadata/src/core/reporting/report-shape.ts @@ -1,6 +1,8 @@ // Table B of docs/superpowers/plans/2026-10-03-fr-044-plan-2-report-view-lowering.md: // a report's derived fields. The single definition; every port has a rule-for-rule copy, -// gated by fixtures/persistence-conformance/report-shapes.json. +// gated by fixtures/persistence-conformance/report-shapes.json. `required` follows Table C +// of docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md, which +// amends Table B for a report with `@spine` and a measure with `@default`. import type { MetaData } from "../../shared/meta-data.js"; import type { MetaRoot } from "../../shared/meta-root.js"; @@ -22,6 +24,7 @@ import { } from "../field/field-constants.js"; import { MetaDimension } from "./meta-dimension.js"; import { MetaMeasure } from "./meta-measure.js"; +import { identityEffectiveFields } from "../identity/validate-identity-passthrough.js"; import { reportDerivedFieldName, reportDimensionItems, @@ -29,6 +32,7 @@ import { reportMeasureItemName, reportMeasureItemOwner, reportMeasureNames, + reportSpine, } from "./report-accessors.js"; import { AGG_AVG, @@ -111,6 +115,20 @@ export function resolveReportingFieldRef( root: MetaRoot, host?: MetaObject, ): MetaField | undefined { + return resolveReportingFieldRefNamed(ref, declaring, root, host)?.field; +} + +/** + * {@link resolveReportingFieldRef}, also returning `named`: the entity the reference's + * entity half names (which, with `host`, may be an ancestor of the entity `field` is read + * from). Table C reads `named`'s primary identity. + */ +function resolveReportingFieldRefNamed( + ref: string, + declaring: MetaData, + root: MetaRoot, + host?: MetaObject, +): { readonly named: MetaObject; readonly field: MetaField } | undefined { // `Entity.field`; a package qualifier uses `::`, so the member separator is the LAST dot. const dot = ref.lastIndexOf(CHILD_REF_SEPARATOR); if (dot <= 0) return undefined; @@ -118,7 +136,8 @@ export function resolveReportingFieldRef( if (!isMetaObject(named)) return undefined; if (host !== undefined && !isSelfOrAncestor(named, host)) return undefined; // ADR-0039: resolving, so a field inherited through extends is found. - return (host ?? named).fields().find((f) => f.name === ref.slice(dot + 1)); + const field = (host ?? named).fields().find((f) => f.name === ref.slice(dot + 1)); + return field === undefined ? undefined : { named, field }; } /** @@ -150,6 +169,30 @@ function unresolved(reportName: string, what: string): Error { return new Error(`report '${reportName}': ${what} does not resolve.`); } +/** + * The hop names of a report's `@spine` (`Owner.hop[.hop...]`), read as + * {@link reportingViaHops} reads a `@via`, the owner resolving in the REPORT's package + * (loader rule R8). Undefined when the report declares no `@spine`. Throws a plain Error + * naming the report when a declared `@spine` does not resolve (a report that passed + * `validateReporting` always resolves): reading it as "no spine" would claim a column + * non-null that a spine row with no facts leaves null. + */ +export function reportSpineHops(report: MetaObject, from: MetaObject, root: MetaRoot): string[] | undefined { + const spine = reportSpine(report); + if (spine === undefined) return undefined; + const hops = reportingViaHops(spine, report, from, root); + if (hops === undefined) throw unresolved(report.name, `@spine '${spine}'`); + return hops; +} + +/** True when `field` is one of the `@fields` of `entity`'s `identity.primary`. */ +function isPrimaryKeyField(entity: MetaObject, field: MetaField): boolean { + // ADR-0039: resolving — an identity.primary (and its @fields) inherited from an + // abstract base counts. + const pk = entity.primaryIdentity(); + return pk !== undefined && (identityEffectiveFields(pk) ?? []).includes(field.name); +} + function declaredMember<T extends MetaData>( from: MetaObject, type: string, @@ -164,19 +207,46 @@ function isTimeGrain(grain: string | undefined): grain is TimeGrain { return grain !== undefined && (TIME_GRAINS as readonly string[]).includes(grain); } +/** + * Table C (`required` of a dimension). Without `@spine`: only a dimension with no `@via` + * over an `@of` field whose effective `@required` is true. With `@spine`: only a dimension + * whose `@via` hops equal the spine's (a column of the spine entity itself, whose rows are + * the report's rows) over an `@of` field that is `@required` or a primary-key column of the + * entity `@of` names. A dimension beyond the spine is reached by a LEFT OUTER join. + */ +function dimensionRequired( + dim: MetaDimension, + named: MetaObject, + of: MetaField, + from: MetaObject, + root: MetaRoot, + spine: readonly string[] | undefined, + reportName: string, +): boolean { + const via = dim.via(); + if (spine === undefined) return via === undefined && of.attr(FIELD_ATTR_REQUIRED) === true; + if (via === undefined) return false; // a column of @from: null in a spine row with no facts + const hops = reportingViaHops(via, reportingMemberOwner(dim, from), from, root); + if (hops === undefined) throw unresolved(reportName, `dimension '${dim.name}' @via '${via}'`); + const onSpine = hops.length === spine.length && hops.every((h, i) => h === spine[i]); + return onSpine && (of.attr(FIELD_ATTR_REQUIRED) === true || isPrimaryKeyField(named, of)); +} + function dimensionField( item: { name: string; grain?: string }, from: MetaObject, root: MetaRoot, reportName: string, + spine: readonly string[] | undefined, ): ReportField { const dim = declaredMember(from, TYPE_DIMENSION, item.name, MetaDimension); if (dim === undefined) throw unresolved(reportName, `dimension '${item.name}' on '${from.name}'`); const vialess = dim.via() === undefined; - const of = resolveReportingFieldRef(dim.of() ?? "", reportingMemberOwner(dim, from), root, vialess ? from : undefined); - if (of === undefined) throw unresolved(reportName, `dimension '${item.name}' @of`); + const ref = resolveReportingFieldRefNamed(dim.of() ?? "", reportingMemberOwner(dim, from), root, vialess ? from : undefined); + if (ref === undefined) throw unresolved(reportName, `dimension '${item.name}' @of`); + const of = ref.field; const name = reportDerivedFieldName(item); - const required = vialess && of.attr(FIELD_ATTR_REQUIRED) === true; + const required = dimensionRequired(dim, ref.named, of, from, root, spine, reportName); if (dim.isTime()) { // Loader rule R2 guarantees a grain from the closed set; a tree built in code does not. const grain = item.grain; @@ -206,40 +276,64 @@ function measureField(item: string, report: MetaObject, from: MetaObject, root: } const m = declaredMember(from, TYPE_MEASURE, name, MetaMeasure); if (m === undefined) throw unresolved(reportName, `measure '${item}' on '${from.name}'`); - if (m.isRatio()) { - return { name, role: "measure", subType: FIELD_SUBTYPE_DECIMAL, required: false, measure: m }; - } + const derived = deriveMeasure(m, from, root); + if (derived === undefined) throw unresolved(reportName, `measure '${name}' @of`); + // Table C: a count is never null; any other measure is not null when it has a @default. + const required = (!m.isRatio() && m.agg() === AGG_COUNT) || m.defaultValue() !== undefined; + const { subType, typeSource } = derived; + // The key set (no `typeSource` key when there is none) and its order are as before. + return typeSource === undefined + ? { name, role: "measure", subType, required, measure: m } + : { name, role: "measure", subType, required, typeSource, measure: m }; +} + +/** One measure's Table B subtype, and the `@of` field it carries type attrs from (if any). */ +interface MeasureDerivation { + readonly subType: string; + readonly typeSource?: MetaField; +} + +/** Table B for one measure. Undefined when its `@of` does not resolve. */ +function deriveMeasure(m: MetaMeasure, from: MetaObject, root: MetaRoot): MeasureDerivation | undefined { + if (m.isRatio()) return { subType: FIELD_SUBTYPE_DECIMAL }; const agg = m.agg(); - if (agg === AGG_COUNT) { - return { name, role: "measure", subType: FIELD_SUBTYPE_LONG, required: true, measure: m }; - } + if (agg === AGG_COUNT) return { subType: FIELD_SUBTYPE_LONG }; const of = resolveReportingFieldRef(m.ofColumns()[0] ?? "", reportingMemberOwner(m, from), root, from); - if (of === undefined) throw unresolved(reportName, `measure '${name}' @of`); + if (of === undefined) return undefined; const src = of.subType; if (agg === AGG_SUM) { - if (src === FIELD_SUBTYPE_CURRENCY) { - return { name, role: "measure", subType: FIELD_SUBTYPE_CURRENCY, required: false, typeSource: of, measure: m }; - } - const subType = SUM_LONG.has(src) ? FIELD_SUBTYPE_LONG : FLOATING.has(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL; - return { name, role: "measure", subType, required: false, measure: m }; - } - if (agg === AGG_AVG) { - const subType = FLOATING.has(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL; - return { name, role: "measure", subType, required: false, measure: m }; + if (src === FIELD_SUBTYPE_CURRENCY) return { subType: FIELD_SUBTYPE_CURRENCY, typeSource: of }; + return { + subType: SUM_LONG.has(src) ? FIELD_SUBTYPE_LONG : FLOATING.has(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL, + }; } + if (agg === AGG_AVG) return { subType: FLOATING.has(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL }; // min / max keep the source field's type. - return { name, role: "measure", subType: src, required: false, typeSource: of, measure: m }; + return { subType: src, typeSource: of }; +} + +/** + * Table B's subtype for one measure of `from`, whether or not a report lists it (a + * ratio's operands are measures a report need not list). The same rule `reportShape` + * applies to a listed measure. Throws a plain Error naming the measure when its `@of` + * does not resolve (a measure that passed `validateReporting` always resolves). + */ +export function measureDerivedSubType(measure: MetaMeasure, from: MetaObject, root: MetaRoot): string { + const derived = deriveMeasure(measure, from, root); + if (derived === undefined) throw new Error(`measure '${measure.name}' on '${from.name}': @of does not resolve.`); + return derived.subType; } -/** Table B. Throws a plain Error naming the report when a reference does not resolve - * (a report that passed `validateReporting` always resolves). */ +/** Table B, with Table C's `required`. Throws a plain Error naming the report when a + * reference does not resolve (a report that passed `validateReporting` always resolves). */ export function reportShape(report: MetaObject, root: MetaRoot): ReportShape { const fromName = reportFrom(report); if (fromName === undefined) throw unresolved(report.name, "@from"); const from = resolveObjectRef(root, fromName, packageOfKey(report.resolutionKey())).node; if (!isMetaObject(from)) throw unresolved(report.name, `@from '${fromName}'`); + const spine = reportSpineHops(report, from, root); const fields = [ - ...reportDimensionItems(report).map((item) => dimensionField(item, from, root, report.name)), + ...reportDimensionItems(report).map((item) => dimensionField(item, from, root, report.name, spine)), ...reportMeasureNames(report).map((item) => measureField(item, report, from, root)), ]; return { report, from, fields }; diff --git a/server/typescript/packages/metadata/src/index.ts b/server/typescript/packages/metadata/src/index.ts index 63e296e01..63d4f39bd 100644 --- a/server/typescript/packages/metadata/src/index.ts +++ b/server/typescript/packages/metadata/src/index.ts @@ -57,7 +57,9 @@ export { type ReportDimensionItem, } from "./core/reporting/report-accessors.js"; export { + measureDerivedSubType, reportShape, + reportSpineHops, reportingMemberOwner, reportingViaHops, resolveReportingFieldRef, diff --git a/server/typescript/packages/metadata/test/report-read-model.test.ts b/server/typescript/packages/metadata/test/report-read-model.test.ts index 2f98198b8..c895d9db9 100644 --- a/server/typescript/packages/metadata/test/report-read-model.test.ts +++ b/server/typescript/packages/metadata/test/report-read-model.test.ts @@ -208,6 +208,35 @@ describe("reportReadModel (FR-044 Table B as a detached read model)", () => { expect(resolveTableName(model(root, "AssetActivity"))).toBe("v_asset_activity"); }); + test("a measure with @default yields a detached field with @required: true (Table C)", async () => { + const result = await new MetaDataLoader().load([ + new InMemoryStringSource( + JSON.stringify({ + "metadata.root": { + package: "acme", + children: [ + { "object.entity": { name: "Sale", children: [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { name: "id" } }, + { "field.currency": { name: "amountCents", "@currency": "EUR" } }, + { "identity.primary": { name: "pk", "@fields": "id", "@generation": "increment" } }, + { "measure.aggregate": { name: "revenue", "@agg": "sum", "@of": "Sale.amountCents", "@default": 0 } }, + { "measure.aggregate": { name: "revenueRaw", "@agg": "sum", "@of": "Sale.amountCents" } }, + ] } }, + { "object.report": { name: "Revenue", "@from": "Sale", "@measures": ["revenue", "revenueRaw"] } }, + ], + }, + }), + ), + ]); + expect(result.errors.map((e) => e.message)).toEqual([]); + const m = model(result.root, "Revenue"); + expect(m.fields().map((f) => [f.name, f.attr(FIELD_ATTR_REQUIRED)])).toEqual([ + ["revenue", true], + ["revenueRaw", false], + ]); + }); + test("a sourceless report yields a model with the fields and no source", async () => { const result = await new MetaDataLoader().load([ new InMemoryStringSource( diff --git a/server/typescript/packages/metadata/test/report-shape.test.ts b/server/typescript/packages/metadata/test/report-shape.test.ts index c9c3b1960..f24749094 100644 --- a/server/typescript/packages/metadata/test/report-shape.test.ts +++ b/server/typescript/packages/metadata/test/report-shape.test.ts @@ -4,11 +4,16 @@ import { pathToFileURL } from "node:url"; import { InMemoryStringSource, MetaDataLoader, + MetaMeasure, OBJECT_REPORT_ATTR_DIMENSIONS, OBJECT_REPORT_ATTR_MEASURES, + OBJECT_REPORT_ATTR_SPINE, + TYPE_MEASURE, loadUris, + measureDerivedSubType, reportMeasureItemName, reportShape, + reportSpineHops, type MetaObject, type MetaRoot, } from "../src/index.js"; @@ -186,3 +191,311 @@ describe("reportShape reference resolution", () => { ); }); }); + +// --------------------------------------------------------------------------- +// Table C of docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md: +// under @spine a column of the spine entity is not nullable when it is @required or a +// primary-key column; a dimension beyond the spine is; a measure with @default is not. +// --------------------------------------------------------------------------- + +const catalog = { + "object.entity": { + name: "Catalog", + children: [ + { "source.rdb": { "@table": "catalogs" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "name", "@required": true } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + ], + }, +}; +const program = { + "object.entity": { + name: "Program", + children: [ + { "source.rdb": { "@table": "programs" } }, + // No @required: a key column is not nullable because it is the key. + { "field.long": { name: "id" } }, + { "field.string": { name: "title", "@required": true } }, + { "field.string": { name: "subtitle" } }, + { "field.timestamp": { name: "publishedAt", "@required": true } }, + { "field.long": { name: "catalogId" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "identity.reference": { name: "fkCatalog", "@references": "Catalog", "@fields": ["catalogId"] } }, + { "relationship.association": { name: "catalog", "@objectRef": "Catalog", "@cardinality": "one" } }, + ], + }, +}; +const purchase = { + "object.entity": { + name: "Purchase", + children: [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "programId" } }, + { "field.int": { name: "minutes", "@required": true } }, + { "field.currency": { name: "amountCents", "@currency": "USD" } }, + { "field.double": { name: "score" } }, + { "field.timestamp": { name: "purchasedAt" } }, + { "identity.primary": { name: "pk", "@fields": ["id"] } }, + { "identity.reference": { name: "fkProgram", "@references": "Program", "@fields": ["programId"] } }, + { "relationship.association": { name: "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { name: "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "dimension.attribute": { name: "programSubtitle", "@of": "Program.subtitle", "@via": "Purchase.program" } }, + { "dimension.time": { name: "publishedAt", "@of": "Program.publishedAt", "@via": "Purchase.program", "@grains": ["hour", "month"] } }, + { "dimension.attribute": { name: "catalogId", "@of": "Catalog.id", "@via": "Purchase.program.catalog" } }, + // Catalog.name is @required, but one hop beyond a Purchase.program spine its join is LEFT OUTER. + { "dimension.attribute": { name: "catalogName", "@of": "Catalog.name", "@via": "Purchase.program.catalog" } }, + { "dimension.attribute": { name: "minutes", "@of": "Purchase.minutes" } }, + { "measure.aggregate": { name: "purchases", "@agg": "count", "@of": "Purchase.id" } }, + { "measure.aggregate": { name: "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } }, + { "measure.aggregate": { name: "revenueRaw", "@agg": "sum", "@of": "Purchase.amountCents" } }, + { "measure.aggregate": { name: "avgMinutes", "@agg": "avg", "@of": "Purchase.minutes", "@default": 0 } }, + { "measure.aggregate": { name: "avgMinutesRaw", "@agg": "avg", "@of": "Purchase.minutes" } }, + { "measure.aggregate": { name: "minMinutes", "@agg": "min", "@of": "Purchase.minutes", "@default": -1 } }, + { "measure.aggregate": { name: "minMinutesRaw", "@agg": "min", "@of": "Purchase.minutes" } }, + { "measure.aggregate": { name: "totalScore", "@agg": "sum", "@of": "Purchase.score" } }, + { "measure.aggregate": { name: "avgScore", "@agg": "avg", "@of": "Purchase.score" } }, + { "measure.aggregate": { name: "lastAt", "@agg": "max", "@of": "Purchase.purchasedAt" } }, + { "measure.ratio": { name: "share", "@numerator": "revenue", "@denominator": "purchases", "@default": 0 } }, + { "measure.ratio": { name: "shareRaw", "@numerator": "revenue", "@denominator": "purchases" } }, + ], + }, +}; +const ALL_MEASURES = [ + "purchases", + "revenue", + "revenueRaw", + "avgMinutes", + "avgMinutesRaw", + "minMinutes", + "minMinutesRaw", + "share", + "shareRaw", +]; +const spineSales = { + "object.report": { + name: "SpineSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle", "programSubtitle", "publishedAt:hour", "publishedAt:month", "catalogId", "catalogName"], + "@measures": ALL_MEASURES, + }, +}; +const plainSales = { + "object.report": { + name: "PlainSales", + "@from": "Purchase", + "@dimensions": ["programId", "programTitle", "programSubtitle", "catalogId", "catalogName", "minutes"], + "@measures": ALL_MEASURES, + }, +}; +const catalogSales = { + "object.report": { + name: "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": ["catalogId", "catalogName"], + "@measures": ["purchases", "revenue"], + }, +}; +const salesModel = () => loadInline([file("acme", [catalog, program, purchase, spineSales, plainSales, catalogSales])]); +const requiredOf = (root: MetaRoot, name: string) => + Object.fromEntries(reportShape(report(root, name), root).fields.map((f) => [f.name, f.required])); +const measureNamed = (root: MetaRoot, entity: string, name: string): MetaMeasure => { + const m = report(root, entity) + .children() + .find((c) => c.type === TYPE_MEASURE && c.name === name); + if (!(m instanceof MetaMeasure)) throw new Error(`no measure ${entity}.${name}`); + return m; +}; + +describe("reportShape Table C (FR-044 @spine and measure @default)", () => { + test("under @spine, a column of the spine entity is required when it is a key or @required", async () => { + const required = requiredOf(await salesModel(), "SpineSales"); + expect(required["programId"]).toBe(true); // Program.id: no @required, in identity.primary + expect(required["programTitle"]).toBe(true); // Program.title: @required + expect(required["programSubtitle"]).toBe(false); // Program.subtitle: optional + // A time dimension over a @required spine column, at either grain path (timestamp / date). + expect(required["publishedAtHour"]).toBe(true); + expect(required["publishedAtMonth"]).toBe(true); + }); + + test("under @spine, a dimension one hop beyond the spine is not required, even over a key or a @required field", async () => { + const required = requiredOf(await salesModel(), "SpineSales"); + expect(required["catalogId"]).toBe(false); // Catalog.id: a key, beyond the spine + expect(required["catalogName"]).toBe(false); // Catalog.name: @required, beyond the spine + }); + + test("a two-hop spine: a dimension whose @via equals the whole spine is on it", async () => { + const required = requiredOf(await salesModel(), "CatalogSales"); + expect(required["catalogId"]).toBe(true); + expect(required["catalogName"]).toBe(true); + }); + + test("the same dimensions in a report without @spine keep today's values (the key one is not required)", async () => { + const required = requiredOf(await salesModel(), "PlainSales"); + expect(required["programId"]).toBe(false); + expect(required["programTitle"]).toBe(false); + expect(required["programSubtitle"]).toBe(false); + expect(required["catalogId"]).toBe(false); + expect(required["catalogName"]).toBe(false); + expect(required["minutes"]).toBe(true); // no @via, @of @required: unchanged + }); + + test("a sum, an avg, a min and a ratio are required with @default and not without; a count always is", async () => { + const root = await salesModel(); + for (const name of ["SpineSales", "PlainSales"]) { + const required = requiredOf(root, name); + expect([name, required]).toEqual([ + name, + expect.objectContaining({ + purchases: true, + revenue: true, + revenueRaw: false, + avgMinutes: true, + avgMinutesRaw: false, + minMinutes: true, + minMinutesRaw: false, + share: true, + shareRaw: false, + }), + ]); + } + }); + + test("a count with a @default (past the loader, which refuses it) is still required", async () => { + const withCountDefault = JSON.parse(JSON.stringify(purchase)) as typeof purchase; + const count = withCountDefault["object.entity"].children.find( + (c) => "measure.aggregate" in c && c["measure.aggregate"].name === "purchases", + ) as { "measure.aggregate": Record<string, unknown> }; + count["measure.aggregate"]["@default"] = 5; + const { root, errors } = await new MetaDataLoader().load([ + file("acme", [catalog, program, withCountDefault, plainSales]), + ]); + expect(errors.map((e) => e.message.includes("@default cannot apply to @agg: count"))).toEqual([true]); + expect(requiredOf(root, "PlainSales")["purchases"]).toBe(true); + }); + + test("subtypes and type sources are untouched by @default", async () => { + const root = await salesModel(); + const shape = reportShape(report(root, "SpineSales"), root).fields; + expect(shape.map((f) => [f.name, f.subType, f.typeSource?.name])).toEqual([ + ["programId", "long", "id"], + ["programTitle", "string", "title"], + ["programSubtitle", "string", "subtitle"], + ["publishedAtHour", "timestamp", "publishedAt"], + ["publishedAtMonth", "date", undefined], + ["catalogId", "long", "id"], + ["catalogName", "string", "name"], + ["purchases", "long", undefined], + ["revenue", "currency", "amountCents"], + ["revenueRaw", "currency", "amountCents"], + ["avgMinutes", "decimal", undefined], + ["avgMinutesRaw", "decimal", undefined], + ["minMinutes", "int", "minutes"], + ["minMinutesRaw", "int", "minutes"], + ["share", "decimal", undefined], + ["shareRaw", "decimal", undefined], + ]); + }); + + test("an identity.primary inherited from an abstract base makes the key column required (ADR-0039)", async () => { + const keyed = { + "object.entity": { + name: "Keyed", + abstract: true, + children: [{ "field.long": { name: "id" } }, { "identity.primary": { name: "pk", "@fields": ["id"] } }], + }, + }; + const inheritedProgram = { + "object.entity": { + name: "Program", + extends: "Keyed", + children: program["object.entity"].children.filter( + (c) => !("identity.primary" in c) && !("field.long" in c && c["field.long"].name === "id"), + ), + }, + }; + const root = await loadInline([file("acme", [keyed, catalog, inheritedProgram, purchase, spineSales])]); + expect(requiredOf(root, "SpineSales")["programId"]).toBe(true); + }); + + test("under @spine, a dimension whose @via does not resolve throws, naming the report", async () => { + const broken = JSON.parse(JSON.stringify(purchase)) as typeof purchase; + const dim = broken["object.entity"].children.find( + (c) => "dimension.attribute" in c && c["dimension.attribute"].name === "programSubtitle", + ) as { "dimension.attribute": Record<string, unknown> }; + // An owner that is not @from (or an entity it extends): reportingViaHops does not resolve it. + dim["dimension.attribute"]["@via"] = "Program.catalog"; + // Past the loader, which refuses the @via under rule D2. + const { root, errors } = await new MetaDataLoader().load([file("acme", [catalog, program, broken, spineSales])]); + expect(errors.length).toBe(1); + expect(() => reportShape(report(root, "SpineSales"), root)).toThrow( + "report 'SpineSales': dimension 'programSubtitle' @via 'Program.catalog' does not resolve.", + ); + }); + + test("a spine written with an abstract base as its owner, over a concrete @from (the inherited fixture)", async () => { + const fixture = join(REPO_ROOT, "fixtures", "conformance", "reporting-spine-inherited", "input", "meta.shop.json"); + const { root, errors } = await loadUris([pathToFileURL(fixture).href]); + expect(errors).toEqual([]); + expect(reportShape(report(root, "ProgramMinutes"), root).fields.map((f) => [f.name, f.required])).toEqual([ + ["programId", true], + ["programTitle", true], + ["totalMinutes", true], + ]); + }); +}); + +describe("reportSpineHops", () => { + test("undefined without @spine; the hop names with one", async () => { + const root = await salesModel(); + const from = report(root, "Purchase"); + expect(reportSpineHops(report(root, "PlainSales"), from, root)).toBeUndefined(); + expect(reportSpineHops(report(root, "SpineSales"), from, root)).toEqual(["program"]); + expect(reportSpineHops(report(root, "CatalogSales"), from, root)).toEqual(["program", "catalog"]); + }); + + test("the owner may be an entity @from extends, resolved in the report's package", async () => { + const fixture = join(REPO_ROOT, "fixtures", "conformance", "reporting-spine-inherited", "input", "meta.shop.json"); + const { root } = await loadUris([pathToFileURL(fixture).href]); + expect(reportSpineHops(report(root, "ProgramMinutes"), report(root, "WorkoutEvent"), root)).toEqual(["program"]); + }); + + test("a @spine that does not resolve throws, naming the report (a tree built in code)", async () => { + const root = await salesModel(); + const from = report(root, "Purchase"); + // Past the loader, which refuses these under rule R8. + for (const spine of ["Program.catalog", "Purchase", "Purchase..program"]) { + const r = withAttr(report(root, "SpineSales"), OBJECT_REPORT_ATTR_SPINE, spine); + expect(() => reportSpineHops(r, from, root)).toThrow(`report 'SpineSales': @spine '${spine}' does not resolve.`); + expect(() => reportShape(r, root)).toThrow(`report 'SpineSales': @spine '${spine}' does not resolve.`); + } + }); +}); + +describe("measureDerivedSubType (Table B for one measure, listed in a report or not)", () => { + test("each aggregate and the ratio", async () => { + const root = await salesModel(); + const from = report(root, "Purchase"); + const subType = (name: string) => measureDerivedSubType(measureNamed(root, "Purchase", name), from, root); + expect(subType("purchases")).toBe("long"); // count + expect(subType("revenue")).toBe("currency"); // sum of a currency + expect(subType("totalScore")).toBe("double"); // sum of a double + expect(subType("avgMinutes")).toBe("decimal"); // avg of an int + expect(subType("avgScore")).toBe("double"); // avg of a double + expect(subType("minMinutes")).toBe("int"); // min keeps the source type + expect(subType("lastAt")).toBe("timestamp"); // max keeps the source type + expect(subType("share")).toBe("decimal"); // ratio + }); + + test("agrees with reportShape for every measure a report lists", async () => { + const root = await salesModel(); + const from = report(root, "Purchase"); + for (const f of reportShape(report(root, "SpineSales"), root).fields) { + if (f.measure !== undefined) expect([f.name, measureDerivedSubType(f.measure, from, root)]).toEqual([f.name, f.subType]); + } + }); +}); From 3ee8515149a0b5e4b273df74ad667aa132032e4a Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 19:54:44 -0400 Subject: [PATCH 08/18] feat(codegen-ts): lower a report @spine and a measure @default to view SQL FR-044 Table D in TypeScript (Postgres, SQLite/D1, MySQL). A @spine report reads FROM the spine entity and walks the spine's hops back to @from, every join LEFT OUTER, with the report scope ANDed onto the ON of the join that introduces @from (no WHERE). Aliases are those of the same report without @spine. A measure or ratio @default wraps its full expression in COALESCE (a REAL literal on SQLite for a real measure). The extractor refuses a spine that reaches an entity with no table or a TPH subtype, reports a hop with no identity.reference as @spine's hop error, and refuses a dimension not reached through the spine. A model declaring neither attribute emits byte-identical SQL. --- .../src/projection/extract-report-spec.ts | 111 ++++- .../src/projection/report-ddl-emit.ts | 100 ++++- .../codegen-ts/src/projection/report-spec.ts | 11 +- .../projection/extract-report-spec.test.ts | 310 ++++++++++++++ .../test/projection/report-ddl-emit.test.ts | 401 +++++++++++++++++- 5 files changed, 901 insertions(+), 32 deletions(-) diff --git a/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts b/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts index 250c25731..8b805b6bd 100644 --- a/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts @@ -8,6 +8,7 @@ import { FIELD_ATTR_LOCAL_TIME, FIELD_SUBTYPE_CURRENCY, FIELD_SUBTYPE_DATE, + FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_ENUM, FIELD_SUBTYPE_FLOAT, @@ -27,7 +28,10 @@ import { TYPE_MEASURE, TYPE_RELATIONSHIP, TYPE_SEGMENT, + measureDerivedSubType, reportShape, + reportSpine, + reportSpineHops, reportingMemberOwner, reportingViaHops, resolveObjectRef, @@ -68,6 +72,8 @@ export function temporalOf(field: MetaField): ReportTemporal { const INTEGRAL_SUM: ReadonlySet<string> = new Set([FIELD_SUBTYPE_INT, FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_CURRENCY]); const FLOATING_SUM: ReadonlySet<string> = new Set([FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT]); +/** Table D: a measure whose derived subtype is one of these reads a REAL `@default` on SQLite. */ +const REAL_SUBTYPES: ReadonlySet<string> = new Set([FIELD_SUBTYPE_DECIMAL, FIELD_SUBTYPE_DOUBLE, FIELD_SUBTYPE_FLOAT]); function isPlainObject(v: unknown): v is Record<string, unknown> { return typeof v === "object" && v !== null && !Array.isArray(v); @@ -206,12 +212,18 @@ function aggregateOf( resolveReportFilter(measure.filter(), from, baseAlias, ctx, `${where} @filter`), ]); const cast = castFor(agg, fields[0]); + // Table D: `@default` is read through the measure itself, so a ratio operand that the report + // does not list still carries its own. + const value = measure.defaultValue(); + const defaultValue = + value === undefined ? undefined : { value, real: REAL_SUBTYPES.has(measureDerivedSubType(measure, from, root)) }; return { agg, distinct: measure.distinct(), refs: fields.map((f) => `${baseAlias}.${sourceColumnNameFor(f, ctx)}`), ...(filter !== undefined ? { filter } : {}), ...(cast !== undefined ? { cast } : {}), + ...(defaultValue !== undefined ? { defaultValue } : {}), }; } @@ -229,13 +241,13 @@ function aliasAtEndOf(path: Path, joins: readonly JoinNode[]): string { } /** - * Why a dimension's `@via` walk stopped at `hop`: the error names the hop, the entity it was - * looked up on, and what the model is missing. The loader (rule D2) accepts a to-one - * `relationship.*` with no `identity.reference` behind it, so the missing-foreign-key case is - * reachable from a model that loads clean. + * Why a dimension's `@via` (or a report's `@spine`, `attr`) walk stopped at `hop`: the error + * names the hop, the entity it was looked up on, and what the model is missing. The loader + * (rules D2 and R8) accepts a to-one `relationship.*` with no `identity.reference` behind it, + * so the missing-foreign-key case is reachable from a model that loads clean. */ -function viaHopError(where: string, via: string, hop: string, at: MetaData, root: MetaRoot): Error { - const head = `${where} @via '${via}' cannot be joined at hop '${hop}' on '${at.resolutionKey()}'`; +function viaHopError(where: string, attr: string, via: string, hop: string, at: MetaData, root: MetaRoot): Error { + const head = `${where} ${attr} '${via}' cannot be joined at hop '${hop}' on '${at.resolutionKey()}'`; // ADR-0039: resolving children(), so an inherited relationship or reference is found. const node = at .children() @@ -260,6 +272,59 @@ function viaHopError(where: string, via: string, hop: string, at: MetaData, root ); } +/** The object a join step lands on, by the resolution key the walk recorded. */ +function stepTarget(step: Path[number], root: MetaRoot): MetaObject | undefined { + return root.objects().find((o) => o.resolutionKey() === step.targetEntity); +} + +/** + * Table D: a report's `@spine`, walked from `@from` as a dimension's `@via` is, and required + * whole. Every entity on the chain contributes rows through its own table, so one with no table + * (abstract, or no writable source.rdb) or a TPH subtype (it shares its base's table with every + * other subtype) is refused, naming the report, the spine and the entity. + */ +function spinePathOf( + report: MetaObject, + hops: readonly string[], + from: MetaObject, + root: MetaRoot, + ctx: ExtractContext, +): Path { + const spine = reportSpine(report) ?? ""; + const where = `report '${report.name}':`; + const path = walkViaPath([from.name, ...hops].join("."), root, packageOf(from), ctx); + if (path.length !== hops.length) { + const last = path[path.length - 1]; + const at = last === undefined ? from : stepTarget(last, root); + throw viaHopError(where, "@spine", spine, hops[path.length]!, at ?? from, root); + } + for (const step of path) { + const entity = stepTarget(step, root); + if (entity === undefined) continue; // the walk resolved it, so it is in the root + const head = `${where} @spine '${spine}' reaches '${entity.name}'`; + if (entity.isAbstract || !hasWritableRdbSource(entity)) { + throw new Error( + `${head}, which has no table (it is abstract or declares no writable source.rdb), so its rows cannot ` + + `be the report's rows. Give '${entity.name}' a source, or end @spine at an entity that has one.`, + ); + } + if (isTphSubtype(entity)) { + throw new Error( + `${head}, a TPH subtype: it shares the table of '${tphDiscriminatorBase(entity)?.name ?? ""}' with every ` + + `other subtype, so the report would have a row for each row of all of them. End @spine at an entity ` + + `with a table of its own.`, + ); + } + } + return path; +} + +/** Table D: under `@spine` every join is LEFT OUTER (the #209 INNER rule is not applied), so no + * join can drop a spine row. */ +function allLeft(nodes: readonly JoinNode[]): JoinNode[] { + return nodes.map((n) => ({ ...n, joinType: "left", children: allLeft(n.children) })); +} + export function extractReportSpec(report: MetaObject, root: MetaRoot, ctx: ExtractContext): ReportViewSpec { const shape = reportShape(report, root); const from = shape.from; @@ -287,13 +352,25 @@ export function extractReportSpec(report: MetaObject, root: MetaRoot, ctx: Extra const used = new Set<string>(); const baseAlias = shortAliasFor(from.name, used); + // Table D: the @spine's own path. Walked before any dimension, so a broken spine is reported + // as the spine's error rather than as the first dimension that crosses it. + const spineHops = reportSpineHops(report, from, root); + const spinePath = spineHops === undefined ? undefined : spinePathOf(report, spineHops, from, root, ctx); + // Rule R9, which the loader enforces: under @spine every listed dimension is reached through it. + // A tree built past the loader is refused here rather than given a join the view cannot place. + const notThroughSpine = (where: string, why: string): Error => + new Error(`${where} is not reached through @spine '${reportSpine(report) ?? ""}': ${why}.`); + // One path per LISTED dimension that has @via (Table F); an unlisted dimension adds no join. const pathOf = new Map<ReportField, Path>(); for (const f of shape.fields) { const dim = f.dimension; const via = dim?.via(); - if (dim === undefined || via === undefined) continue; const where = `report '${report.name}': dimension '${f.name}'`; + if (dim !== undefined && via === undefined && spineHops !== undefined) { + throw notThroughSpine(where, `it has no @via, so it reads @from '${from.name}', which a spine row with no facts lacks`); + } + if (dim === undefined || via === undefined) continue; // The loader's rule D2: the owner half resolves in the DECLARING entity's package and must be // `from` or an entity it extends; the walk then starts AT `from`. const hops = reportingViaHops(via, reportingMemberOwner(dim, from), from, root); @@ -302,6 +379,12 @@ export function extractReportSpec(report: MetaObject, root: MetaRoot, ctx: Extra `${where} @via '${via}' must be Owner.hop[.hop...], starting at @from '${from.name}' or an entity it extends.`, ); } + if (spineHops !== undefined && !spineHops.every((h, i) => hops[i] === h)) { + throw notThroughSpine( + where, + `its @via '${via}' does not begin with the spine's hops, so the view has no join to place it on`, + ); + } // The head is `from`'s SHORT name, resolved in `from`'s own package: a bare name binds the // referrer's package first, so it is `from` itself even when another package has an entity // of that name. Never its resolution key: walkViaPath splits on every `.`, and a package @@ -312,11 +395,18 @@ export function extractReportSpec(report: MetaObject, root: MetaRoot, ctx: Extra if (path.length !== hops.length) { const last = path[path.length - 1]; const at = last === undefined ? from : root.objects().find((o) => o.resolutionKey() === last.targetEntity); - throw viaHopError(where, via, hops[path.length]!, at ?? from, root); + throw viaHopError(where, "@via", via, hops[path.length]!, at ?? from, root); } pathOf.set(f, path); } - const joins = pathsToJoins([...pathOf.values()], used); + // Under @spine its path goes in FIRST, so it is the first (and, by R9, the only) child at every + // level of the trie down to the spine entity. Every dimension path begins with it, so inserting + // it first creates no node and assigns no alias that the first dimension path would not have: + // the aliases are exactly those of the same report without @spine, and no measure reference + // moves. The emitter still checks the chain is single (spineFrom) rather than trust the order. + const dimensionPaths = [...pathOf.values()]; + const tree = pathsToJoins(spinePath === undefined ? dimensionPaths : [spinePath, ...dimensionPaths], used); + const joins = spinePath === undefined ? tree : allLeft(tree); const columns = shape.fields.map((f): ReportColumn => { const dbColAlias = columnNameFromField(f.name, ctx.columnNamingStrategy); @@ -353,12 +443,14 @@ export function extractReportSpec(report: MetaObject, root: MetaRoot, ctx: Extra } return aggregateOf(m, report, from, baseAlias, root, ctx); }; + const defaultValue = measure.defaultValue(); return { kind: "ratio", fieldName: f.name, dbColAlias, numerator: operand(measure.numerator()), denominator: operand(measure.denominator()), + ...(defaultValue !== undefined ? { defaultValue } : {}), }; } return { kind: "aggregate", fieldName: f.name, dbColAlias, aggregate: aggregateOf(measure, report, from, baseAlias, root, ctx) }; @@ -375,5 +467,6 @@ export function extractReportSpec(report: MetaObject, root: MetaRoot, ctx: Extra joinTree: { baseEntity: from.resolutionKey(), baseAlias, joins }, columns, ...(where !== undefined ? { where } : {}), + ...(spinePath !== undefined ? { spineDepth: spinePath.length } : {}), }; } diff --git a/server/typescript/packages/codegen-ts/src/projection/report-ddl-emit.ts b/server/typescript/packages/codegen-ts/src/projection/report-ddl-emit.ts index e40a45654..349eeb152 100644 --- a/server/typescript/packages/codegen-ts/src/projection/report-ddl-emit.ts +++ b/server/typescript/packages/codegen-ts/src/projection/report-ddl-emit.ts @@ -73,7 +73,15 @@ function castType(cast: "bigint" | "double", d: ReportDialect): string | undefin } } -/** One aggregate (Table C): the bare aggregate, the condition by dialect, the cast last. */ +/** Table D: `COALESCE(E, n)` for a measure with a `@default`; `E` unchanged without one. */ +function withDefault(sql: string, dv: ReportAggregate["defaultValue"], d: ReportDialect): string { + if (dv === undefined) return sql; + // SQLite: a REAL column keeps one storage class in every row, so its default is a REAL literal. + return `COALESCE(${sql}, ${d === "sqlite" && dv.real ? `${dv.value}.0` : String(dv.value)})`; +} + +/** One aggregate (Table C): the bare aggregate, the condition by dialect, the cast, then the + * Table D default around all of it. */ function aggregate(a: ReportAggregate, d: ReportDialect): string { const refs = a.refs.map((r) => ref(r, d)); const c = a.filter === undefined ? undefined : cond(a.filter, d); @@ -106,7 +114,7 @@ function aggregate(a: ReportAggregate, d: ReportDialect): string { else sql = `${fn}(${distinct}CASE WHEN ${c} THEN ${x} END)`; } const type = a.cast === undefined ? undefined : castType(a.cast, d); - return type === undefined ? sql : `CAST(${sql} AS ${type})`; + return withDefault(type === undefined ? sql : `CAST(${sql} AS ${type})`, a.defaultValue, d); } interface RenderedColumn { @@ -126,37 +134,82 @@ function column(c: ReportColumn, d: ReportDialect): RenderedColumn { case "aggregate": return { expr: aggregate(c.aggregate, d), alias, grouped: false }; case "ratio": { - // Each operand is its FULL Table C expression (condition and cast included). + // Each operand is its FULL Table C expression (condition and cast included), and its own + // Table D default; the ratio's default wraps the whole quotient, which is always real. const num = aggregate(c.numerator, d); const den = aggregate(c.denominator, d); const top = d === "postgres" ? "NUMERIC" : d === "sqlite" ? "REAL" : undefined; - return { - expr: `${top === undefined ? num : `CAST(${num} AS ${top})`} / NULLIF(${den}, 0)`, - alias, - grouped: false, - }; + const quotient = `${top === undefined ? num : `CAST(${num} AS ${top})`} / NULLIF(${den}, 0)`; + const dv = c.defaultValue === undefined ? undefined : { value: c.defaultValue, real: true }; + return { expr: withDefault(quotient, dv, d), alias, grouped: false }; } } } -function renderJoin(node: JoinNode, parentAlias: string, options: ReportEmitOptions): string { - const table = options.joinTables[node.targetEntity]; +/** The table of a joined entity, refused by name when none is registered. */ +function tableOf(entity: string, options: ReportEmitOptions): string { + const table = options.joinTables[entity]; if (!table) { - throw new Error(`report-ddl-emit: no table name registered for joined entity "${node.targetEntity}".`); + throw new Error(`report-ddl-emit: no table name registered for joined entity "${entity}".`); } - const d = options.dialect; + return table; +} + +/** The `ON` predicate of the hop `node` from `parentAlias`. An equality, so the same text + * serves the hop walked backwards (Table D's spine chain). */ +function onPredicate(node: JoinNode, parentAlias: string, d: ReportDialect): string { const fk = q(node.fkColumn, d); const pk = q(node.pkColumn, d); // referenceHolder "source": FK on the parent (belongs-to); "target": FK on the child (has-many). - const on = node.referenceHolder === "source" + return node.referenceHolder === "source" ? `${node.alias}.${pk} = ${parentAlias}.${fk}` : `${node.alias}.${fk} = ${parentAlias}.${pk}`; +} + +function renderJoin(node: JoinNode, parentAlias: string, options: ReportEmitOptions): string { + const table = tableOf(node.targetEntity, options); + const d = options.dialect; const kw = node.joinType === "inner" ? "INNER JOIN" : "LEFT OUTER JOIN"; - let sql = ` ${kw} ${q(table, d)} ${node.alias} ON ${on}`; + let sql = ` ${kw} ${q(table, d)} ${node.alias} ON ${onPredicate(node, parentAlias, d)}`; for (const child of node.children) sql += "\n" + renderJoin(child, node.alias, options); return sql; } +/** + * Table D, a report with `@spine`: FROM the spine entity S, then the spine's hops walked from S + * back to @from, one LEFT OUTER JOIN each with the ON its forward join renders; the last one + * introduces @from's table under the base alias. The report scope is ANDed onto that join's ON: + * it filters facts, never spine rows, so there is no WHERE. Onward joins hang off S's alias. + */ +function spineFrom(spec: ReportViewSpec, depth: number, options: ReportEmitOptions): string { + const d = options.dialect; + // The chain: the single root, then its single child, down to S. Rule R9 routes every dimension + // through the spine, so nothing branches off above S; a tree that does would hold a join this + // FROM has no place for, so it is refused rather than half-rendered. + const chain: JoinNode[] = []; + let level = spec.joinTree.joins; + while (chain.length < depth && level.length === 1) { + chain.push(level[0]!); + level = level[0]!.children; + } + if (depth < 1 || chain.length !== depth) { + throw new Error( + `report-ddl-emit: view '${spec.viewName}' declares a @spine of ${depth} hop(s), but its join tree is not ` + + `one chain of that many hops from @from, so the spine cannot be placed.`, + ); + } + const spine = chain[chain.length - 1]!; + let sql = ` FROM ${q(tableOf(spine.targetEntity, options), d)} ${spine.alias}`; + for (let i = chain.length - 1; i >= 0; i--) { + const parentAlias = i === 0 ? spec.joinTree.baseAlias : chain[i - 1]!.alias; + const parentTable = i === 0 ? options.baseTableName : tableOf(chain[i - 1]!.targetEntity, options); + const scope = i === 0 && spec.where !== undefined ? ` AND ${cond(spec.where, d)}` : ""; + sql += `\n LEFT OUTER JOIN ${q(parentTable, d)} ${parentAlias} ON ${onPredicate(chain[i]!, parentAlias, d)}${scope}`; + } + for (const child of spine.children) sql += "\n" + renderJoin(child, spine.alias, options); + return sql; +} + export function emitReportViewDdl(spec: ReportViewSpec, options: ReportEmitOptions): string { const d = options.dialect; // Rendered once: a dimension's SELECT expression is its GROUP BY term, so they cannot differ. @@ -164,13 +217,18 @@ export function emitReportViewDdl(spec: ReportViewSpec, options: ReportEmitOptio const select = cols.map((c) => ` ${c.expr} AS ${c.alias}`).join(",\n"); const groupBy = cols.filter((c) => c.grouped).map((c) => c.expr); - const base = spec.joinTree.baseAlias; - const joins = spec.joinTree.joins.map((j) => renderJoin(j, base, options)).join("\n"); - const body = - ` SELECT\n${select}\n FROM ${q(options.baseTableName, d)} ${base}` + - (joins === "" ? "" : `\n${joins}`) + - (spec.where === undefined ? "" : `\n WHERE ${cond(spec.where, d)}`) + - (groupBy.length === 0 ? "" : `\n GROUP BY ${groupBy.join(", ")}`); + let from: string; + if (spec.spineDepth === undefined) { + const base = spec.joinTree.baseAlias; + const joins = spec.joinTree.joins.map((j) => renderJoin(j, base, options)).join("\n"); + from = + ` FROM ${q(options.baseTableName, d)} ${base}` + + (joins === "" ? "" : `\n${joins}`) + + (spec.where === undefined ? "" : `\n WHERE ${cond(spec.where, d)}`); + } else { + from = spineFrom(spec, spec.spineDepth, options); + } + const body = ` SELECT\n${select}\n${from}` + (groupBy.length === 0 ? "" : `\n GROUP BY ${groupBy.join(", ")}`); if (options.bodyOnly) return body; return `CREATE VIEW ${q(spec.viewName, d)} AS\n${body};`; diff --git a/server/typescript/packages/codegen-ts/src/projection/report-spec.ts b/server/typescript/packages/codegen-ts/src/projection/report-spec.ts index 5fcc0f770..d2257040e 100644 --- a/server/typescript/packages/codegen-ts/src/projection/report-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/report-spec.ts @@ -27,6 +27,9 @@ export interface ReportAggregate { readonly filter?: ViewFilterClause; /** The Table C cast: integral sum → "bigint", floating sum → "double". */ readonly cast?: "bigint" | "double"; + /** FR-044 Table D: the measure's `@default`, read when the aggregate is null. `real` when its + * derived subtype is decimal, double or float (SQLite then gets a REAL literal). */ + readonly defaultValue?: { readonly value: number; readonly real: boolean }; } export type ReportColumn = @@ -51,6 +54,8 @@ export type ReportColumn = readonly dbColAlias: string; readonly numerator: ReportAggregate; readonly denominator: ReportAggregate; + /** FR-044 Table D: the ratio's own `@default` (a ratio is always real). */ + readonly defaultValue?: number; }; export interface ReportViewSpec { @@ -58,6 +63,10 @@ export interface ReportViewSpec { readonly joinTree: JoinTree; /** One column per `@dimensions` item then one per `@measures` item, in listed order. */ readonly columns: readonly ReportColumn[]; - /** The report's `@segment` filter, then its `@filter`, ANDed. Absent when neither is declared. */ + /** The report's `@segment` filter, then its `@filter`, ANDed. Absent when neither is declared. + * The emitter decides where it is written: a WHERE, or with `spineDepth` the join to @from. */ readonly where?: ViewFilterClause; + /** FR-044 Table D: how many hops of the join tree's single root chain are the report's + * `@spine` (the chain's last node is the spine entity). Absent for a report without `@spine`. */ + readonly spineDepth?: number; } diff --git a/server/typescript/packages/codegen-ts/test/projection/extract-report-spec.test.ts b/server/typescript/packages/codegen-ts/test/projection/extract-report-spec.test.ts index 3e72c8ea1..ba9fc4314 100644 --- a/server/typescript/packages/codegen-ts/test/projection/extract-report-spec.test.ts +++ b/server/typescript/packages/codegen-ts/test/projection/extract-report-spec.test.ts @@ -9,6 +9,7 @@ import { MetaDataLoader, InMemoryStringSource, OBJECT_REPORT_ATTR_FILTER, + OBJECT_REPORT_ATTR_SPINE, reportReadModel, type MetaObject, type MetaRoot, @@ -616,3 +617,312 @@ describe("extractReportSpec: refusals that name what is wrong", () => { ); }); }); + +// --------------------------------------------------------------------------- +// FR-044 Table D: a report's @spine and a measure's @default. +// --------------------------------------------------------------------------- + +const table = (t: string): Json => ({ "source.rdb": { "@table": t } }); +const longField = (name: string, extra: Json = {}): Json => ({ "field.long": { name, ...extra } }); +const idPk = (): Json => ({ "identity.primary": { name: "pk", "@fields": ["id"] } }); +const reference = (name: string, field: string, target: string): Json => + ({ "identity.reference": { name, "@fields": [field], "@references": target } }); +const entity = (name: string, children: Json[], extra: Json = {}): Json => + ({ "object.entity": { name, ...extra, children } }); +const viewReport = (name: string, attrs: Json): Json => + ({ "object.report": { name, ...attrs, children: [view(`v_${name}`)] } }); + +/** Owner <- Program <- Week: the fitness shape, plus an owner one hop beyond the spine. */ +function roster(reports: Json[], opts: { entities?: Json[]; week?: Json[] } = {}): InMemoryStringSource { + return file("acme", [ + entity("Owner", [table("owners"), longField("id"), { "field.string": { name: "name" } }, idPk()]), + entity("Program", [ + table("programs"), + longField("id"), + { "field.string": { name: "title", "@required": true } }, + longField("ownerId", { "@required": true }), + idPk(), + reference("ownerRef", "ownerId", "Owner"), + ]), + entity("Week", [ + table("weeks"), + longField("id"), + longField("programId", { "@required": true }), + { "field.string": { name: "label" } }, + { "field.int": { name: "durationMinutes", "@required": true } }, + idPk(), + reference("fkProgram", "programId", "Program"), + { "segment.filter": { name: "long", "@filter": { durationMinutes: { gte: 60 } } } }, + { "dimension.attribute": { name: "programKey", "@of": "Program.id", "@via": "Week.fkProgram" } }, + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "Week.fkProgram" } }, + { "dimension.attribute": { name: "ownerName", "@of": "Owner.name", "@via": "Week.fkProgram.ownerRef" } }, + { "dimension.attribute": { name: "label", "@of": "Week.label" } }, + { "measure.aggregate": { name: "weeks", "@agg": "count", "@of": "Week.id" } }, + { "measure.aggregate": { name: "longWeeks", "@agg": "count", "@of": "Week.id", "@segment": "long" } }, + { "measure.aggregate": { name: "totalMinutes", "@agg": "sum", "@of": "Week.durationMinutes" } }, + { "measure.aggregate": { name: "totalMinutesOrZero", "@agg": "sum", "@of": "Week.durationMinutes", "@default": 0 } }, + { "measure.aggregate": { name: "avgMinutesOrNone", "@agg": "avg", "@of": "Week.durationMinutes", "@default": -1 } }, + { "measure.ratio": { name: "longShare", "@numerator": "longWeeks", "@denominator": "weeks" } }, + { "measure.ratio": { name: "longShareOrZero", "@numerator": "longWeeks", "@denominator": "weeks", "@default": 0 } }, + { "measure.ratio": { name: "minutesPerWeek", "@numerator": "totalMinutesOrZero", "@denominator": "weeks" } }, + ...(opts.week ?? []), + ]), + ...(opts.entities ?? []), + ...reports, + ]); +} + +const ownerRoster = (spine: boolean): Json => + viewReport("OwnerRoster", { + "@from": "Week", + ...(spine ? { "@spine": "Week.fkProgram" } : {}), + "@dimensions": ["programTitle", "ownerName"], + "@measures": ["weeks", "totalMinutesOrZero"], + "@segment": "long", + }); + +type Joins = ReportViewSpec["joinTree"]["joins"]; + +function joinsOfTree(nodes: Joins): unknown[] { + return nodes.map((n) => [n.relationship, n.alias, joinsOfTree(n.children)]); +} + +function joinTypes(nodes: Joins): string[] { + return nodes.flatMap((n) => [n.joinType, ...joinTypes(n.children)]); +} + +/** Every alias the spec names: the base, each join with its hop, and every column reference. */ +function aliasesOf(s: ReportViewSpec): unknown[] { + const refs = s.columns.map((c) => + c.kind === "aggregate" ? c.aggregate.refs : c.kind === "ratio" ? [c.numerator.refs, c.denominator.refs] : c.ref, + ); + return [s.joinTree.baseAlias, joinsOfTree(s.joinTree.joins), refs]; +} + +describe("extractReportSpec: @spine (Table D)", () => { + test("a spine report has spineDepth, one root join, and every join LEFT", async () => { + const root = await loadFiles([roster([ownerRoster(true)])]); + const s = extractReportSpec(root.findObject("OwnerRoster")!, root, CTX); + expect(s.spineDepth).toBe(1); + expect(joinsOfTree(s.joinTree.joins)).toEqual([["fkProgram", "p", [["ownerRef", "o", []]]]]); + // Both foreign keys are @required: without @spine both hops are INNER (#209), with it neither. + expect(joinTypes(s.joinTree.joins)).toEqual(["left", "left"]); + }); + + test("without @spine the same report has no spineDepth and keeps the #209 join types", async () => { + const root = await loadFiles([roster([ownerRoster(false)])]); + const s = extractReportSpec(root.findObject("OwnerRoster")!, root, CTX); + expect("spineDepth" in s).toBe(false); + expect(joinTypes(s.joinTree.joins)).toEqual(["inner", "inner"]); + }); + + test("aliases equal those of the same report with @spine removed, and the scope is the same clause", async () => { + const withSpine = await loadFiles([roster([ownerRoster(true)])]); + const without = await loadFiles([roster([ownerRoster(false)])]); + const a = extractReportSpec(withSpine.findObject("OwnerRoster")!, withSpine, CTX); + const b = extractReportSpec(without.findObject("OwnerRoster")!, without, CTX); + expect(aliasesOf(a)).toEqual(aliasesOf(b)); + expect(aliasesOf(a)).toEqual([ + "w", + [["fkProgram", "p", [["ownerRef", "o", []]]]], + ["p.title", "o.name", ["w.id"], ["w.duration_minutes"]], + ]); + expect(a.where).toEqual({ kind: "cmp", ref: "w.duration_minutes", op: "gte", value: 60 }); + expect(a.where).toEqual(b.where); + }); + + test("a two-hop spine has spineDepth 2 and one chain", async () => { + const root = await loadFiles([ + file("acme", [ + entity("Program", [table("programs"), longField("id"), { "field.string": { name: "title" } }, idPk()]), + entity("Week", [table("weeks"), longField("id"), longField("programId"), idPk(), reference("program", "programId", "Program")]), + entity("Session", [ + table("sessions"), + longField("id"), + longField("weekId"), + idPk(), + reference("week", "weekId", "Week"), + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "Session.week.program" } }, + { "measure.aggregate": { name: "sessions", "@agg": "count", "@of": "Session.id" } }, + ]), + viewReport("SessionsByProgram", { + "@from": "Session", "@spine": "Session.week.program", "@dimensions": ["programTitle"], "@measures": ["sessions"], + }), + ]), + ]); + const s = extractReportSpec(root.findObject("SessionsByProgram")!, root, CTX); + expect(s.spineDepth).toBe(2); + expect(joinsOfTree(s.joinTree.joins)).toEqual([["week", "w", [["program", "p", []]]]]); + expect(joinTypes(s.joinTree.joins)).toEqual(["left", "left"]); + }); + + /** A Week report whose @spine is `Week.<hop>`, grouped by `<dim>` reached through it. */ + const spineTo = (hop: string, dim: string): Json => + viewReport("Spined", { "@from": "Week", "@spine": `Week.${hop}`, "@dimensions": [dim], "@measures": ["weeks"] }); + + test("a spine entity with no writable source is refused, naming the report, the spine and the entity", async () => { + const root = await loadFiles([ + roster([spineTo("ghostRef", "ghostName")], { + entities: [entity("Ghost", [longField("id"), { "field.string": { name: "name" } }, idPk()])], + week: [ + longField("ghostId"), + reference("ghostRef", "ghostId", "Ghost"), + { "dimension.attribute": { name: "ghostName", "@of": "Ghost.name", "@via": "Week.ghostRef" } }, + ], + }), + ]); + expect(() => extractReportSpec(root.findObject("Spined")!, root, CTX)).toThrow( + "report 'Spined': @spine 'Week.ghostRef' reaches 'Ghost', which has no table (it is abstract or declares " + + "no writable source.rdb), so its rows cannot be the report's rows. Give 'Ghost' a source, or end " + + "@spine at an entity that has one.", + ); + }); + + test("an abstract spine entity is refused", async () => { + const root = await loadFiles([ + roster([spineTo("catalogRef", "catalogName")], { + entities: [ + entity("Catalog", [table("catalogs"), longField("id"), { "field.string": { name: "name" } }, idPk()], { abstract: true }), + ], + week: [ + longField("catalogId"), + reference("catalogRef", "catalogId", "Catalog"), + { "dimension.attribute": { name: "catalogName", "@of": "Catalog.name", "@via": "Week.catalogRef" } }, + ], + }), + ]); + expect(() => extractReportSpec(root.findObject("Spined")!, root, CTX)).toThrow( + /^report 'Spined': @spine 'Week.catalogRef' reaches 'Catalog', which has no table \(it is abstract/, + ); + }); + + test("an entity on the chain with no table is refused, not only the last one", async () => { + const root = await loadFiles([ + file("acme", [ + entity("Program", [table("programs"), longField("id"), { "field.string": { name: "title" } }, idPk()]), + entity("Week", [longField("id"), longField("programId"), idPk(), reference("program", "programId", "Program")]), + entity("Session", [ + table("sessions"), + longField("id"), + longField("weekId"), + idPk(), + reference("week", "weekId", "Week"), + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "Session.week.program" } }, + { "measure.aggregate": { name: "sessions", "@agg": "count", "@of": "Session.id" } }, + ]), + viewReport("SessionsByProgram", { + "@from": "Session", "@spine": "Session.week.program", "@dimensions": ["programTitle"], "@measures": ["sessions"], + }), + ]), + ]); + expect(() => extractReportSpec(root.findObject("SessionsByProgram")!, root, CTX)).toThrow( + /^report 'SessionsByProgram': @spine 'Session.week.program' reaches 'Week', which has no table/, + ); + }); + + test("a TPH subtype on the spine is refused, naming the report, the spine, the subtype and its base", async () => { + const root = await loadFiles([ + roster([spineTo("adminRef", "adminName")], { + entities: [ + entity( + "User", + [table("users"), longField("id"), { "field.string": { name: "kind" } }, { "field.string": { name: "name" } }, idPk()], + { "@discriminator": "kind" }, + ), + entity("Admin", [], { extends: "User", "@discriminatorValue": "ADMIN" }), + ], + week: [ + longField("adminId"), + reference("adminRef", "adminId", "Admin"), + { "dimension.attribute": { name: "adminName", "@of": "Admin.name", "@via": "Week.adminRef" } }, + ], + }), + ]); + expect(() => extractReportSpec(root.findObject("Spined")!, root, CTX)).toThrow( + "report 'Spined': @spine 'Week.adminRef' reaches 'Admin', a TPH subtype: it shares the table of 'User' " + + "with every other subtype, so the report would have a row for each row of all of them. End @spine at " + + "an entity with a table of its own.", + ); + }); + + test("a spine hop with no identity.reference is refused with the hop error, naming @spine", async () => { + // The loader (rule R8, like D2) accepts a to-one relationship with no identity.reference behind it. + const root = await loadFiles([ + roster([spineTo("coach", "coachName")], { + entities: [entity("Coach", [table("coaches"), longField("id"), { "field.string": { name: "name" } }, idPk()])], + week: [ + { "relationship.association": { name: "coach", "@objectRef": "Coach", "@cardinality": "one" } }, + { "dimension.attribute": { name: "coachName", "@of": "Coach.name", "@via": "Week.coach" } }, + ], + }), + ]); + expect(() => extractReportSpec(root.findObject("Spined")!, root, CTX)).toThrow( + "report 'Spined': @spine 'Week.coach' cannot be joined at hop 'coach' on 'acme::Week': the model declares " + + "no foreign key for it. A view joins a hop through an identity.reference; declare one on 'Week' whose " + + "@references is 'Coach' (with the foreign-key field in @fields).", + ); + }); + + test("a dimension that is not reached through the spine is refused (a tree built past the loader)", async () => { + const byLabel = viewReport("ByLabel", { "@from": "Week", "@dimensions": ["label"], "@measures": ["weeks"] }); + const root = await loadFiles([roster([ownerRoster(false), byLabel])]); + const report = withAttr(root.findObject("OwnerRoster")!, OBJECT_REPORT_ATTR_SPINE, "Week.fkProgram.ownerRef"); + expect(() => extractReportSpec(report, root, CTX)).toThrow( + "report 'OwnerRoster': dimension 'programTitle' is not reached through @spine 'Week.fkProgram.ownerRef': " + + "its @via 'Week.fkProgram' does not begin with the spine's hops, so the view has no join to place it on.", + ); + const labels = withAttr(root.findObject("ByLabel")!, OBJECT_REPORT_ATTR_SPINE, "Week.fkProgram"); + expect(() => extractReportSpec(labels, root, CTX)).toThrow( + "report 'ByLabel': dimension 'label' is not reached through @spine 'Week.fkProgram': it has no @via, so " + + "it reads @from 'Week', which a spine row with no facts lacks.", + ); + }); +}); + +describe("extractReportSpec: a measure @default (Table D)", () => { + const defaults = viewReport("Defaults", { + "@from": "Week", + "@measures": ["totalMinutes", "totalMinutesOrZero", "avgMinutesOrNone", "longShare", "longShareOrZero", "minutesPerWeek"], + }); + const columnOf = async (name: string) => { + const root = await loadFiles([roster([defaults])]); + const s = extractReportSpec(root.findObject("Defaults")!, root, CTX); + return s.columns.find((c) => c.fieldName === name)!; + }; + + test("an aggregate carries its default, and whether its derived type is real", async () => { + const sum = await columnOf("totalMinutesOrZero"); + if (sum.kind !== "aggregate") throw new Error("expected an aggregate"); + // sum of an int derives long: an integral default. + expect(sum.aggregate.defaultValue).toEqual({ value: 0, real: false }); + expect(sum.aggregate.cast).toBe("bigint"); + const avg = await columnOf("avgMinutesOrNone"); + if (avg.kind !== "aggregate") throw new Error("expected an aggregate"); + // avg of an int derives decimal: a real default. + expect(avg.aggregate.defaultValue).toEqual({ value: -1, real: true }); + }); + + test("an aggregate with no @default has no defaultValue key", async () => { + const plain = await columnOf("totalMinutes"); + if (plain.kind !== "aggregate") throw new Error("expected an aggregate"); + expect("defaultValue" in plain.aggregate).toBe(false); + }); + + test("a ratio carries its own default; its operands carry none they do not declare", async () => { + const ratio = await columnOf("longShareOrZero"); + if (ratio.kind !== "ratio") throw new Error("expected a ratio"); + expect(ratio.defaultValue).toBe(0); + expect("defaultValue" in ratio.numerator).toBe(false); + expect("defaultValue" in ratio.denominator).toBe(false); + const plain = await columnOf("longShare"); + expect("defaultValue" in plain).toBe(false); + }); + + test("an operand that is not listed in @measures carries its own default", async () => { + const ratio = await columnOf("minutesPerWeek"); + if (ratio.kind !== "ratio") throw new Error("expected a ratio"); + expect("defaultValue" in ratio).toBe(false); + expect(ratio.numerator.defaultValue).toEqual({ value: 0, real: false }); + expect("defaultValue" in ratio.denominator).toBe(false); + }); +}); diff --git a/server/typescript/packages/codegen-ts/test/projection/report-ddl-emit.test.ts b/server/typescript/packages/codegen-ts/test/projection/report-ddl-emit.test.ts index d5ace4927..48c6f47ca 100644 --- a/server/typescript/packages/codegen-ts/test/projection/report-ddl-emit.test.ts +++ b/server/typescript/packages/codegen-ts/test/projection/report-ddl-emit.test.ts @@ -1,7 +1,8 @@ // FR-044 Plan 2 Task 5 — emitReportViewDdl. Specs are hand-built (not extracted): this // file pins the TEXT of contract Tables C, E, F and the golden bodies of Table G. import { describe, test, expect } from "bun:test"; -import type { TimeGrain } from "@metaobjectsdev/metadata"; +import { InMemoryStringSource, MetaDataLoader, type TimeGrain } from "@metaobjectsdev/metadata"; +import { buildReportViews } from "../../src/projection/build-projection-views.js"; import { emitReportViewDdl, type ReportEmitOptions } from "../../src/projection/report-ddl-emit.js"; import { emitViewDdl } from "../../src/projection/view-ddl-emit.js"; import type { @@ -483,3 +484,401 @@ describe("emitReportViewDdl — rules", () => { ); }); }); + +// ── FR-044 Table D: a measure @default and a report @spine ─────────────────────────── + +const sumOf = (ref: string, extra: Partial<ReportAggregate> = {}): ReportAggregate => + ({ agg: "sum", distinct: false, refs: [ref], ...extra }); + +describe("emitReportViewDdl — Table D, a measure @default", () => { + test("COALESCE wraps the cast, per dialect", () => { + const spec = baseSpec("w", "Week", [ + col("t", sumOf("w.m", { cast: "bigint", defaultValue: { value: 0, real: false } })), + ]); + expect(emitReportViewDdl(spec, pg("weeks"))).toContain(` COALESCE(CAST(SUM(w."m") AS BIGINT), 0) AS "t"`); + expect(emitReportViewDdl(spec, sqlite("weeks"))).toContain(` COALESCE(SUM(w."m"), 0) AS "t"`); + expect(emitReportViewDdl(spec, mysql("weeks"))).toContain(" COALESCE(CAST(SUM(w.`m`) AS SIGNED), 0) AS `t`"); + }); + + test("COALESCE wraps the condition too: the full Table C expression is E", () => { + const spec = baseSpec("w", "Week", [ + col("t", sumOf("w.m", { cast: "bigint", filter: longWeek, defaultValue: { value: 0, real: false } })), + ]); + expect(emitReportViewDdl(spec, pg("weeks"))) + .toContain(`COALESCE(CAST(SUM(w."m") FILTER (WHERE w."durationMinutes" >= 60) AS BIGINT), 0) AS "t"`); + expect(emitReportViewDdl(spec, sqlite("weeks"))) + .toContain(`COALESCE(SUM(CASE WHEN w."durationMinutes" >= 60 THEN w."m" END), 0) AS "t"`); + }); + + test("a negative default: -1, and -1.0 on SQLite only for a real measure", () => { + const real = baseSpec("w", "Week", [ + col("a", { agg: "avg", distinct: false, refs: ["w.m"], defaultValue: { value: -1, real: true } }), + ]); + expect(emitReportViewDdl(real, pg("weeks"))).toContain(` COALESCE(AVG(w."m"), -1) AS "a"`); + expect(emitReportViewDdl(real, sqlite("weeks"))).toContain(` COALESCE(AVG(w."m"), -1.0) AS "a"`); + expect(emitReportViewDdl(real, mysql("weeks"))).toContain(" COALESCE(AVG(w.`m`), -1) AS `a`"); + const integral = baseSpec("w", "Week", [ + col("t", sumOf("w.m", { cast: "bigint", defaultValue: { value: -1, real: false } })), + ]); + expect(emitReportViewDdl(integral, sqlite("weeks"))).toContain(` COALESCE(SUM(w."m"), -1) AS "t"`); + }); + + test("a defaulted operand inside a ratio keeps its COALESCE, under the ratio's cast", () => { + const num = sumOf("w.m", { cast: "bigint", defaultValue: { value: 0, real: false } }); + const ratio: ReportColumn = { kind: "ratio", fieldName: "r", dbColAlias: "r", numerator: num, denominator: count("w.id") }; + const spec = baseSpec("w", "Week", [ratio]); + expect(emitReportViewDdl(spec, pg("weeks"))).toContain( + ` CAST(COALESCE(CAST(SUM(w."m") AS BIGINT), 0) AS NUMERIC) / NULLIF(COUNT(w."id"), 0) AS "r"`); + expect(emitReportViewDdl(spec, sqlite("weeks"))).toContain( + ` CAST(COALESCE(SUM(w."m"), 0) AS REAL) / NULLIF(COUNT(w."id"), 0) AS "r"`); + expect(emitReportViewDdl(spec, mysql("weeks"))).toContain( + " COALESCE(CAST(SUM(w.`m`) AS SIGNED), 0) / NULLIF(COUNT(w.`id`), 0) AS `r`"); + // The ratio's own default wraps the whole quotient, a REAL literal on SQLite. + const both = baseSpec("w", "Week", [{ ...ratio, defaultValue: 0 }]); + expect(emitReportViewDdl(both, pg("weeks"))).toContain( + ` COALESCE(CAST(COALESCE(CAST(SUM(w."m") AS BIGINT), 0) AS NUMERIC) / NULLIF(COUNT(w."id"), 0), 0) AS "r"`); + expect(emitReportViewDdl(both, sqlite("weeks"))).toContain( + ` COALESCE(CAST(COALESCE(SUM(w."m"), 0) AS REAL) / NULLIF(COUNT(w."id"), 0), 0.0) AS "r"`); + expect(emitReportViewDdl(both, mysql("weeks"))).toContain( + " COALESCE(COALESCE(CAST(SUM(w.`m`) AS SIGNED), 0) / NULLIF(COUNT(w.`id`), 0), 0) AS `r`"); + }); +}); + +/** Week -> Program, the spine of the hand-built specs below. */ +const spineProgram = (children: readonly JoinNode[] = []): JoinNode => ({ + relationship: "fkProgram", targetEntity: "Program", alias: "p", cardinality: "one", + fkColumn: "programId", pkColumn: "id", referenceHolder: "source", joinType: "left", children, +}); + +function spineSpec(where: ViewFilterClause | undefined, joins: readonly JoinNode[] = [spineProgram()], spineDepth = 1): ReportViewSpec { + return { + ...baseSpec("w", "Week", [ + { kind: "dimension", fieldName: "programTitle", dbColAlias: "programTitle", ref: "p.title" }, + col("weeks", count("w.id")), + ], where === undefined ? { joins } : { joins, where }), + spineDepth, + }; +} + +describe("emitReportViewDdl — Table D, a report @spine", () => { + const tables = { Program: "programs", Week: "weeks" }; + + test("FROM the spine entity, the hop reversed, the fact table LEFT OUTER, no WHERE", () => { + expect(emitReportViewDdl(spineSpec(undefined), pg("weeks", tables))).toBe(lines( + ` SELECT`, + ` p."title" AS "programTitle",`, + ` COUNT(w."id") AS "weeks"`, + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` GROUP BY p."title"`, + )); + }); + + test("the report scope is in the join condition", () => { + const segmentAndFilter: ViewFilterClause = { + kind: "and", clauses: [cmp("w.durationMinutes", "gte", 60), cmp("w.label", "eq", "x")], + }; + const sql = emitReportViewDdl(spineSpec(segmentAndFilter), pg("weeks", tables)); + expect(sql).not.toContain("WHERE"); + expect(sql).toContain(` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" AND (w."durationMinutes" >= 60 AND w."label" = 'x')\n`); + // An `or` scope is parenthesised inside the ON, so it cannot capture the join predicate. + const or: ViewFilterClause = { kind: "or", clauses: [cmp("w.label", "eq", "a"), cmp("w.label", "eq", "b")] }; + const mySql = emitReportViewDdl(spineSpec(or), mysql("weeks", tables)); + expect(mySql).not.toContain("WHERE"); + expect(mySql).toContain(" LEFT OUTER JOIN `weeks` w ON p.`id` = w.`programId` AND (w.`label` = 'a' OR w.`label` = 'b')\n"); + }); + + test("an onward join hangs off the spine alias", () => { + const owner: JoinNode = { + relationship: "ownerRef", targetEntity: "Owner", alias: "o", cardinality: "one", + fkColumn: "ownerId", pkColumn: "id", referenceHolder: "source", joinType: "left", children: [], + }; + const sql = emitReportViewDdl(spineSpec(undefined, [spineProgram([owner])]), sqlite("weeks", { ...tables, Owner: "owners" })); + expect(sql).toContain(lines( + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` LEFT OUTER JOIN "owners" o ON o."id" = p."ownerId"`, + )); + }); + + test("a spec whose join tree is not one chain of spineDepth hops is refused, not half-rendered", () => { + expect(() => emitReportViewDdl(spineSpec(undefined, [spineProgram()], 2), pg("weeks", tables))) + .toThrow(/v_test.*@spine/); + const other: JoinNode = { ...spineProgram(), relationship: "other", alias: "p0" }; + expect(() => emitReportViewDdl(spineSpec(undefined, [spineProgram(), other]), pg("weeks", tables))) + .toThrow(/v_test.*@spine/); + expect(() => emitReportViewDdl(spineSpec(undefined, [spineProgram()], 0), pg("weeks", tables))) + .toThrow(/v_test.*@spine/); + }); + + test("a spine entity with no registered table is refused", () => { + expect(() => emitReportViewDdl(spineSpec(undefined), pg("weeks", { Week: "weeks" }))).toThrow(/Program/); + }); +}); + +// ── Table D goldens, end to end: an inline model through buildReportViews ──────────── + +type Json = Record<string, unknown>; + +const tableSrc = (t: string): Json => ({ "source.rdb": { "@table": t } }); +const viewSrc = (v: string): Json => ({ "source.rdb": { "@kind": "view", "@view": v } }); +const field = (subType: string, name: string, extra: Json = {}): Json => ({ [`field.${subType}`]: { name, ...extra } }); +const idPk: Json = { "identity.primary": { name: "id", "@fields": "id" } }; +const ent = (name: string, children: Json[]): Json => ({ "object.entity": { name, children } }); +const rpt = (name: string, attrs: Json, view: string): Json => + ({ "object.report": { name, ...attrs, children: [viewSrc(view)] } }); + +/** The canonical fitness model's Program and Week, with Table D's three reports (Task 6). */ +const fitness = (extra: Json[] = [], programExtra: Json[] = [], weekExtra: Json[] = []): Json[] => [ + ent("Program", [ + tableSrc("programs"), + field("long", "id"), + field("string", "title", { "@required": true, "@maxLength": 200 }), + idPk, + ...programExtra, + ]), + ent("Week", [ + tableSrc("weeks"), + field("long", "id"), + field("long", "programId", { "@required": true }), + field("string", "label", { "@maxLength": 80 }), + field("int", "durationMinutes", { "@required": true }), + idPk, + { "identity.reference": { name: "fkProgram", "@fields": "programId", "@references": "Program" } }, + { "segment.filter": { name: "long", "@filter": { durationMinutes: { gte: 60 } } } }, + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "Week.fkProgram" } }, + { "measure.aggregate": { name: "weeks", "@agg": "count", "@of": "Week.id" } }, + { "measure.aggregate": { name: "longWeeks", "@agg": "count", "@of": "Week.id", "@segment": "long" } }, + { "measure.aggregate": { name: "totalMinutes", "@agg": "sum", "@of": "Week.durationMinutes" } }, + { "measure.ratio": { name: "longShare", "@numerator": "longWeeks", "@denominator": "weeks" } }, + { "dimension.attribute": { name: "programKey", "@of": "Program.id", "@via": "Week.fkProgram" } }, + { "measure.aggregate": { name: "totalMinutesOrZero", "@agg": "sum", "@of": "Week.durationMinutes", "@default": 0 } }, + { "measure.ratio": { name: "longShareOrZero", "@numerator": "longWeeks", "@denominator": "weeks", "@default": 0 } }, + ...weekExtra, + ]), + rpt("ProgramRoster", { + "@from": "Week", "@spine": "Week.fkProgram", "@dimensions": ["programKey", "programTitle"], + "@measures": ["weeks", "totalMinutes", "totalMinutesOrZero", "longShare", "longShareOrZero"], + }, "v_program_roster"), + rpt("ProgramLongWeeks", { + "@from": "Week", "@spine": "Week.fkProgram", "@dimensions": ["programKey"], + "@measures": ["weeks", "totalMinutesOrZero"], "@segment": "long", + }, "v_program_long_weeks"), + rpt("FitnessTotalsFilled", { + "@from": "Week", "@measures": ["weeks", "totalMinutesOrZero", "longShareOrZero"], + }, "v_fitness_totals_filled"), + ...extra, +]; + +async function viewsOf(children: Json[], dialect: "postgres" | "sqlite" | "d1" | "mysql"): Promise<Record<string, string>> { + const source = new InMemoryStringSource(JSON.stringify({ "metadata.root": { package: "fitness", children } })); + const { root, errors } = await new MetaDataLoader().load([source]); + expect(errors).toEqual([]); + const out: Record<string, string> = {}; + for (const v of buildReportViews(root, { dialect, columnNamingStrategy: "literal" })) out[v.name] = v.sql; + return out; +} + +describe("Table D goldens (the bodies Task 6's canonical reports must produce)", () => { + test("postgres", async () => { + const v = await viewsOf(fitness(), "postgres"); + expect(v["v_program_roster"]).toBe(lines( + ` SELECT`, + ` p."id" AS "programKey",`, + ` p."title" AS "programTitle",`, + ` COUNT(w."id") AS "weeks",`, + ` CAST(SUM(w."durationMinutes") AS BIGINT) AS "totalMinutes",`, + ` COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero",`, + ` CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0) AS "longShare",`, + ` COALESCE(CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0), 0) AS "longShareOrZero"`, + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` GROUP BY p."id", p."title"`, + )); + expect(v["v_program_long_weeks"]).toBe(lines( + ` SELECT`, + ` p."id" AS "programKey",`, + ` COUNT(w."id") AS "weeks",`, + ` COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero"`, + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" AND w."durationMinutes" >= 60`, + ` GROUP BY p."id"`, + )); + expect(v["v_fitness_totals_filled"]).toBe(lines( + ` SELECT`, + ` COUNT(w."id") AS "weeks",`, + ` COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero",`, + ` COALESCE(CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0), 0) AS "longShareOrZero"`, + ` FROM "weeks" w`, + )); + }); + + test("sqlite, and d1 identical", async () => { + const v = await viewsOf(fitness(), "sqlite"); + expect(v["v_program_roster"]).toBe(lines( + ` SELECT`, + ` p."id" AS "programKey",`, + ` p."title" AS "programTitle",`, + ` COUNT(w."id") AS "weeks",`, + ` SUM(w."durationMinutes") AS "totalMinutes",`, + ` COALESCE(SUM(w."durationMinutes"), 0) AS "totalMinutesOrZero",`, + ` CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0) AS "longShare",`, + ` COALESCE(CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0), 0.0) AS "longShareOrZero"`, + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` GROUP BY p."id", p."title"`, + )); + expect(v["v_program_long_weeks"]).toBe(lines( + ` SELECT`, + ` p."id" AS "programKey",`, + ` COUNT(w."id") AS "weeks",`, + ` COALESCE(SUM(w."durationMinutes"), 0) AS "totalMinutesOrZero"`, + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" AND w."durationMinutes" >= 60`, + ` GROUP BY p."id"`, + )); + expect(v["v_fitness_totals_filled"]).toBe(lines( + ` SELECT`, + ` COUNT(w."id") AS "weeks",`, + ` COALESCE(SUM(w."durationMinutes"), 0) AS "totalMinutesOrZero",`, + ` COALESCE(CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0), 0.0) AS "longShareOrZero"`, + ` FROM "weeks" w`, + )); + expect(await viewsOf(fitness(), "d1")).toEqual(v); + }); + + test("mysql", async () => { + const v = await viewsOf(fitness(), "mysql"); + expect(v["v_program_roster"]).toBe(lines( + " SELECT", + " p.`id` AS `programKey`,", + " p.`title` AS `programTitle`,", + " COUNT(w.`id`) AS `weeks`,", + " CAST(SUM(w.`durationMinutes`) AS SIGNED) AS `totalMinutes`,", + " COALESCE(CAST(SUM(w.`durationMinutes`) AS SIGNED), 0) AS `totalMinutesOrZero`,", + " COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0) AS `longShare`,", + " COALESCE(COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0), 0) AS `longShareOrZero`", + " FROM `programs` p", + " LEFT OUTER JOIN `weeks` w ON p.`id` = w.`programId`", + " GROUP BY p.`id`, p.`title`", + )); + expect(v["v_program_long_weeks"]).toBe(lines( + " SELECT", + " p.`id` AS `programKey`,", + " COUNT(w.`id`) AS `weeks`,", + " COALESCE(CAST(SUM(w.`durationMinutes`) AS SIGNED), 0) AS `totalMinutesOrZero`", + " FROM `programs` p", + " LEFT OUTER JOIN `weeks` w ON p.`id` = w.`programId` AND w.`durationMinutes` >= 60", + " GROUP BY p.`id`", + )); + expect(v["v_fitness_totals_filled"]).toBe(lines( + " SELECT", + " COUNT(w.`id`) AS `weeks`,", + " COALESCE(CAST(SUM(w.`durationMinutes`) AS SIGNED), 0) AS `totalMinutesOrZero`,", + " COALESCE(COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0), 0) AS `longShareOrZero`", + " FROM `weeks` w", + )); + }); + + test("an onward join from the spine alias is LEFT OUTER even over a required reference", async () => { + const owner = ent("Owner", [tableSrc("owners"), field("long", "id"), field("string", "name"), idPk]); + const programRef = [ + field("long", "ownerId", { "@required": true }), + { "identity.reference": { name: "ownerRef", "@fields": "ownerId", "@references": "Owner" } }, + ]; + const dim = [{ "dimension.attribute": { name: "ownerName", "@of": "Owner.name", "@via": "Week.fkProgram.ownerRef" } }]; + const report = (name: string, spine: boolean): Json => rpt(name, { + "@from": "Week", ...(spine ? { "@spine": "Week.fkProgram" } : {}), + "@dimensions": ["programTitle", "ownerName"], "@measures": ["weeks"], + }, `v_${name}`); + const v = await viewsOf(fitness([owner, report("Spined", true), report("Plain", false)], programRef, dim), "postgres"); + expect(v["v_Spined"]).toBe(lines( + ` SELECT`, + ` p."title" AS "programTitle",`, + ` o."name" AS "ownerName",`, + ` COUNT(w."id") AS "weeks"`, + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` LEFT OUTER JOIN "owners" o ON o."id" = p."ownerId"`, + ` GROUP BY p."title", o."name"`, + )); + // The same report without @spine keeps #209: both required hops are INNER. + expect(v["v_Plain"]).toContain(lines( + ` FROM "weeks" w`, + ` INNER JOIN "programs" p ON p."id" = w."programId"`, + ` INNER JOIN "owners" o ON o."id" = p."ownerId"`, + )); + }); +}); + +describe("Table D: the spine chain, end to end", () => { + /** Program <- Week <- Session (Table D's two-hop case), each hop an identity.reference. */ + const sessions: Json[] = [ + ent("Program", [tableSrc("programs"), field("long", "id"), field("string", "title"), idPk]), + ent("Week", [ + tableSrc("weeks"), field("long", "id"), field("long", "programId"), idPk, + { "identity.reference": { name: "program", "@fields": "programId", "@references": "Program" } }, + ]), + ent("Session", [ + tableSrc("sessions"), field("long", "id"), field("long", "weekId"), field("int", "minutes"), idPk, + { "identity.reference": { name: "week", "@fields": "weekId", "@references": "Week" } }, + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "Session.week.program" } }, + { "measure.aggregate": { name: "sessions", "@agg": "count", "@of": "Session.id" } }, + ]), + rpt("SessionsByProgram", { + "@from": "Session", "@spine": "Session.week.program", "@dimensions": ["programTitle"], + "@measures": ["sessions"], "@filter": { minutes: { gte: 10 } }, + }, "v_sessions_by_program"), + ]; + + test("the two-hop chain, per dialect", async () => { + expect((await viewsOf(sessions, "postgres"))["v_sessions_by_program"]).toBe(lines( + ` SELECT`, + ` p."title" AS "programTitle",`, + ` COUNT(s."id") AS "sessions"`, + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` LEFT OUTER JOIN "sessions" s ON w."id" = s."weekId" AND s."minutes" >= 10`, + ` GROUP BY p."title"`, + )); + expect((await viewsOf(sessions, "sqlite"))["v_sessions_by_program"]).toContain(lines( + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` LEFT OUTER JOIN "sessions" s ON w."id" = s."weekId" AND s."minutes" >= 10`, + )); + expect((await viewsOf(sessions, "mysql"))["v_sessions_by_program"]).toContain(lines( + " FROM `programs` p", + " LEFT OUTER JOIN `weeks` w ON p.`id` = w.`programId`", + " LEFT OUTER JOIN `sessions` s ON w.`id` = s.`weekId` AND s.`minutes` >= 10", + )); + }); + + test("a one-to-one hop whose reference is held by the far entity, reversed", async () => { + // Account.profile is @cardinality one, but the foreign key is Profile.accountId. + const accounts: Json[] = [ + ent("Account", [ + tableSrc("accounts"), field("long", "id"), idPk, + { "relationship.association": { name: "profile", "@objectRef": "Profile", "@cardinality": "one" } }, + { "dimension.attribute": { name: "tier", "@of": "Profile.tier", "@via": "Account.profile" } }, + { "measure.aggregate": { name: "accounts", "@agg": "count", "@of": "Account.id" } }, + ]), + ent("Profile", [ + tableSrc("profiles"), field("long", "id"), field("long", "accountId"), field("string", "tier"), idPk, + { "identity.reference": { name: "accountRef", "@fields": "accountId", "@references": "Account" } }, + ]), + rpt("ByTier", { "@from": "Account", "@spine": "Account.profile", "@dimensions": ["tier"], "@measures": ["accounts"] }, "v_by_tier"), + rpt("ByTierPlain", { "@from": "Account", "@dimensions": ["tier"], "@measures": ["accounts"] }, "v_by_tier_plain"), + ]; + const v = await viewsOf(accounts, "postgres"); + expect(v["v_by_tier"]).toBe(lines( + ` SELECT`, + ` p."tier" AS "tier",`, + ` COUNT(a."id") AS "accounts"`, + ` FROM "profiles" p`, + ` LEFT OUTER JOIN "accounts" a ON p."accountId" = a."id"`, + ` GROUP BY p."tier"`, + )); + // The forward join renders the same predicate: reversing the hop moves no column. + expect(v["v_by_tier_plain"]).toContain(` LEFT OUTER JOIN "profiles" p ON p."accountId" = a."id"`); + }); +}); From 0d66724826d61fc44b1ce21867c7a2f779bf5b3f Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 20:08:31 -0400 Subject: [PATCH 09/18] test(persistence): canonical @spine and @default reports, value and idempotence tests --- .../canonical/meta.fitness.json | 15 +- .../canonical/schema.postgres.sql | 32 +++ .../report-shapes.json | 112 +++++++++ .../test/codegen-compile-conformance.test.ts | 3 +- .../test/report-shapes-artifact.test.ts | 32 ++- .../test/report-views-mysql.test.ts | 51 ++++- .../test/report-views-pg.test.ts | 213 +++++++++++++++++- .../test/report-views-sqlite.test.ts | 213 +++++++++++++++++- 8 files changed, 659 insertions(+), 12 deletions(-) diff --git a/fixtures/persistence-conformance/canonical/meta.fitness.json b/fixtures/persistence-conformance/canonical/meta.fitness.json index d743dbd67..8a8b0cf2d 100644 --- a/fixtures/persistence-conformance/canonical/meta.fitness.json +++ b/fixtures/persistence-conformance/canonical/meta.fitness.json @@ -44,7 +44,10 @@ { "measure.aggregate": { "name": "avgMinutes", "@agg": "avg", "@of": "Week.durationMinutes" } }, { "measure.aggregate": { "name": "minMinutes", "@agg": "min", "@of": "Week.durationMinutes" } }, { "measure.aggregate": { "name": "maxMinutes", "@agg": "max", "@of": "Week.durationMinutes" } }, - { "measure.ratio": { "name": "longShare", "@numerator": "longWeeks", "@denominator": "weeks" } } + { "measure.ratio": { "name": "longShare", "@numerator": "longWeeks", "@denominator": "weeks" } }, + { "dimension.attribute": { "name": "programKey", "@of": "Program.id", "@via": "Week.fkProgram" } }, + { "measure.aggregate": { "name": "totalMinutesOrZero", "@agg": "sum", "@of": "Week.durationMinutes", "@default": 0 } }, + { "measure.ratio": { "name": "longShareOrZero", "@numerator": "longWeeks", "@denominator": "weeks", "@default": 0 } } ] }}, { "object.entity": { @@ -340,6 +343,16 @@ { "object.report": { "name": "AssetActivity", "@from": "Asset", "@dimensions": ["recordedAt:hour", "asOfDate:week"], "@measures": ["assets"], "children": [ { "source.rdb": { "@kind": "view", "@view": "v_asset_activity" } } ] } }, + { "object.report": { "name": "ProgramRoster", "@from": "Week", "@spine": "Week.fkProgram", + "@dimensions": ["programKey", "programTitle"], + "@measures": ["weeks", "totalMinutes", "totalMinutesOrZero", "longShare", "longShareOrZero"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_roster" } } ] } }, + { "object.report": { "name": "ProgramLongWeeks", "@from": "Week", "@spine": "Week.fkProgram", + "@dimensions": ["programKey"], "@measures": ["weeks", "totalMinutesOrZero"], "@segment": "long", + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_long_weeks" } } ] } }, + { "object.report": { "name": "FitnessTotalsFilled", "@from": "Week", + "@measures": ["weeks", "totalMinutesOrZero", "longShareOrZero"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_fitness_totals_filled" } } ] } }, { "template.prompt": { "name": "coachNote", diff --git a/fixtures/persistence-conformance/canonical/schema.postgres.sql b/fixtures/persistence-conformance/canonical/schema.postgres.sql index 27f33a8a9..182a0fccd 100644 --- a/fixtures/persistence-conformance/canonical/schema.postgres.sql +++ b/fixtures/persistence-conformance/canonical/schema.postgres.sql @@ -236,3 +236,35 @@ CREATE VIEW "v_asset_activity" AS FROM "assets" a GROUP BY date_trunc('hour', a."recordedAt", 'UTC'), CAST(date_trunc('week', CAST(a."asOfDate" AS TIMESTAMP)) AS DATE); COMMENT ON VIEW "v_asset_activity" IS 'metaobjects:v1:sha256:02b68a9ec0c67e11c6e505a47591c24e8dece737a46161549167c2cafece0a26'; + +CREATE VIEW "v_program_roster" AS + SELECT + p."id" AS "programKey", + p."title" AS "programTitle", + COUNT(w."id") AS "weeks", + CAST(SUM(w."durationMinutes") AS BIGINT) AS "totalMinutes", + COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero", + CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0) AS "longShare", + COALESCE(CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0), 0) AS "longShareOrZero" + FROM "programs" p + LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" + GROUP BY p."id", p."title"; +COMMENT ON VIEW "v_program_roster" IS 'metaobjects:v1:sha256:7f6860157cb86898940ea877ae4874138a0053f1aa1f6e848e2b853ad6478b47'; + +CREATE VIEW "v_program_long_weeks" AS + SELECT + p."id" AS "programKey", + COUNT(w."id") AS "weeks", + COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero" + FROM "programs" p + LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" AND w."durationMinutes" >= 60 + GROUP BY p."id"; +COMMENT ON VIEW "v_program_long_weeks" IS 'metaobjects:v1:sha256:f9eaf671509f579d4d4f0345947ff383c8192707d66b1fb24a2ece5580aaa86e'; + +CREATE VIEW "v_fitness_totals_filled" AS + SELECT + COUNT(w."id") AS "weeks", + COALESCE(CAST(SUM(w."durationMinutes") AS BIGINT), 0) AS "totalMinutesOrZero", + COALESCE(CAST(COUNT(w."id") FILTER (WHERE w."durationMinutes" >= 60) AS NUMERIC) / NULLIF(COUNT(w."id"), 0), 0) AS "longShareOrZero" + FROM "weeks" w; +COMMENT ON VIEW "v_fitness_totals_filled" IS 'metaobjects:v1:sha256:b3ea9f392eb506a9d70ffa39f7b66e1e7f987b5913b0f3e74d0b84a4d6c299a9'; diff --git a/fixtures/persistence-conformance/report-shapes.json b/fixtures/persistence-conformance/report-shapes.json index 16dfe26aa..bb2cdf3d2 100644 --- a/fixtures/persistence-conformance/report-shapes.json +++ b/fixtures/persistence-conformance/report-shapes.json @@ -209,6 +209,118 @@ "typeSource": null } ] + }, + { + "report": "fitness::ProgramRoster", + "from": "fitness::Week", + "view": "v_program_roster", + "fields": [ + { + "name": "programKey", + "role": "dimension", + "subType": "long", + "required": true, + "typeSource": "fitness::Program.id" + }, + { + "name": "programTitle", + "role": "dimension", + "subType": "string", + "required": true, + "typeSource": "fitness::Program.title" + }, + { + "name": "weeks", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "totalMinutes", + "role": "measure", + "subType": "long", + "required": false, + "typeSource": null + }, + { + "name": "totalMinutesOrZero", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "longShare", + "role": "measure", + "subType": "decimal", + "required": false, + "typeSource": null + }, + { + "name": "longShareOrZero", + "role": "measure", + "subType": "decimal", + "required": true, + "typeSource": null + } + ] + }, + { + "report": "fitness::ProgramLongWeeks", + "from": "fitness::Week", + "view": "v_program_long_weeks", + "fields": [ + { + "name": "programKey", + "role": "dimension", + "subType": "long", + "required": true, + "typeSource": "fitness::Program.id" + }, + { + "name": "weeks", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "totalMinutesOrZero", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + } + ] + }, + { + "report": "fitness::FitnessTotalsFilled", + "from": "fitness::Week", + "view": "v_fitness_totals_filled", + "fields": [ + { + "name": "weeks", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "totalMinutesOrZero", + "role": "measure", + "subType": "long", + "required": true, + "typeSource": null + }, + { + "name": "longShareOrZero", + "role": "measure", + "subType": "decimal", + "required": true, + "typeSource": null + } + ] } ] } diff --git a/server/typescript/packages/codegen-ts/test/codegen-compile-conformance.test.ts b/server/typescript/packages/codegen-ts/test/codegen-compile-conformance.test.ts index 9da163b97..f587c0517 100644 --- a/server/typescript/packages/codegen-ts/test/codegen-compile-conformance.test.ts +++ b/server/typescript/packages/codegen-ts/test/codegen-compile-conformance.test.ts @@ -114,6 +114,7 @@ const DEFAULT_OPTIONS: ts.CompilerOptions = { /** The corpus's view-backed reports (FR-044). */ const SERVED_REPORTS = [ "ProgramMinutes", "FitnessTotals", "ProgramsByMonth", "ProgramsByWeek", "RecentPrograms", "AssetActivity", + "ProgramRoster", "ProgramLongWeeks", "FitnessTotalsFilled", ] as const; const PROFILES: ReadonlyArray<readonly [string, ts.CompilerOptions]> = [ @@ -197,7 +198,7 @@ describe("codegen-compile conformance — the shared fitness corpus", () => { "ProgramBrief.ts", "ProgramVerdict.ts", "WeekLabel.ts", - // FR-044 Plan 3: the six view-backed reports' entity, names and queries files. + // FR-044 Plan 3: the nine view-backed reports' entity, names and queries files. ...SERVED_REPORTS.flatMap((r) => [`${r}.ts`, `${r}.names.ts`, `${r}.queries.ts`]), ]) { expect([...emitted]).toContain(expected); diff --git a/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts b/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts index 39c09f30b..126a590f7 100644 --- a/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts +++ b/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts @@ -36,7 +36,7 @@ describe("canonical report-shapes artifact (report-shapes.json)", () => { expect(generated).toBe(committed); }); - test("the six canonical reports appear in declaration order, with the contract's byte format", () => { + test("the nine canonical reports appear in declaration order, with the contract's byte format", () => { const committed = readReportShapesJson(); expect(committed.endsWith("}\n")).toBe(true); expect(committed.startsWith('{\n "reports": [\n {\n "report": "fitness::ProgramMinutes"')).toBe(true); @@ -48,6 +48,36 @@ describe("canonical report-shapes artifact (report-shapes.json)", () => { ["fitness::ProgramsByWeek", "v_programs_by_week"], ["fitness::RecentPrograms", "v_recent_programs"], ["fitness::AssetActivity", "v_asset_activity"], + ["fitness::ProgramRoster", "v_program_roster"], + ["fitness::ProgramLongWeeks", "v_program_long_weeks"], + ["fitness::FitnessTotalsFilled", "v_fitness_totals_filled"], + ]); + }); + + test("@spine and @default (Table C): a spine key and a defaulted measure are required, their undefaulted twins are not", () => { + type Field = { name: string; role: string; subType: string; required: boolean }; + const parsed = JSON.parse(readReportShapesJson()) as { reports: { report: string; fields: Field[] }[] }; + const fields = (report: string): [string, string, string, boolean][] => + (parsed.reports.find((r) => r.report === report)?.fields ?? []).map((f) => [f.name, f.role, f.subType, f.required]); + + expect(fields("fitness::ProgramRoster")).toEqual([ + ["programKey", "dimension", "long", true], + ["programTitle", "dimension", "string", true], + ["weeks", "measure", "long", true], + ["totalMinutes", "measure", "long", false], + ["totalMinutesOrZero", "measure", "long", true], + ["longShare", "measure", "decimal", false], + ["longShareOrZero", "measure", "decimal", true], + ]); + expect(fields("fitness::ProgramLongWeeks")).toEqual([ + ["programKey", "dimension", "long", true], + ["weeks", "measure", "long", true], + ["totalMinutesOrZero", "measure", "long", true], + ]); + expect(fields("fitness::FitnessTotalsFilled")).toEqual([ + ["weeks", "measure", "long", true], + ["totalMinutesOrZero", "measure", "long", true], + ["longShareOrZero", "measure", "decimal", true], ]); }); diff --git a/server/typescript/packages/integration-tests/test/report-views-mysql.test.ts b/server/typescript/packages/integration-tests/test/report-views-mysql.test.ts index caa584c28..a37fa83f5 100644 --- a/server/typescript/packages/integration-tests/test/report-views-mysql.test.ts +++ b/server/typescript/packages/integration-tests/test/report-views-mysql.test.ts @@ -46,6 +46,7 @@ let canonical: MetaRoot; const CANONICAL_VIEWS = [ "v_program_minutes", "v_fitness_totals", "v_programs_by_month", "v_programs_by_week", "v_recent_programs", "v_asset_activity", + "v_program_roster", "v_program_long_weeks", "v_fitness_totals_filled", ]; /** The adopter's hand-written DDL (MySQL schema is not MetaObjects'). */ @@ -111,6 +112,11 @@ const SEED_PROGRAMS_AND_WEEKS = ` (13, 1, NULL, 60), (20, 2, 'Solo', 45);`; +/** The same, plus program 3 with no weeks: the row a `@spine` report keeps and a plain one drops. */ +const SEED_ROSTER = `${SEED_PROGRAMS_AND_WEEKS} + INSERT INTO programs (id, title, priceCents, status, created_ts) VALUES + (3, 'Mobility', 1000, 'DRAFT', '2026-06-01T00:00:00');`; + const SEED_PROGRAMS_BY_TIME = ` INSERT INTO programs (id, title, priceCents, status, created_ts) VALUES (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), @@ -178,7 +184,7 @@ const RELATIVE_MODEL = JSON.stringify({ "metadata.root": { package: "acme", chil { "source.rdb": { "@kind": "view", "@view": "v_up_to_tomorrow" } } ] } }, ]}}); -/** Every view and table this file creates: the canonical six, the inline models' and the recipe's. */ +/** Every view and table this file creates: the canonical nine, the inline models' and the recipe's. */ const OWN_VIEWS = [ ...CANONICAL_VIEWS, "v_events_by_grain", "v_last_twelve_hours", "v_last_two_weeks", "v_up_to_tomorrow", @@ -337,6 +343,49 @@ describe("report views — canonical model on real MySQL 8.4", () => { (2, 'Stale', 100, 'PUBLISHED', UTC_TIMESTAMP(3) - INTERVAL 60 DAY);`); expect(await select("SELECT * FROM `v_recent_programs`")).toEqual([{ programs: "1" }]); }); + + test("@spine and @default: v_program_roster through buildReportViews has a row for program 3, which has no weeks", async () => { + await exec(SEED_ROSTER); + // Four fractional digits on every decimal, the defaulted 0 included (Review Focus 5). + expect(await select("SELECT * FROM `v_program_roster` ORDER BY `programKey`")).toEqual([ + { programKey: "1", programTitle: "Foundations", weeks: "4", totalMinutes: "240", totalMinutesOrZero: "240", + longShare: "0.7500", longShareOrZero: "0.7500" }, + { programKey: "2", programTitle: "Strength", weeks: "1", totalMinutes: "45", totalMinutesOrZero: "45", + longShare: "0.0000", longShareOrZero: "0.0000" }, + { programKey: "3", programTitle: "Mobility", weeks: "0", totalMinutes: null, totalMinutesOrZero: "0", + longShare: null, longShareOrZero: "0.0000" }, + ]); + expect(await select("SELECT * FROM `v_program_long_weeks` ORDER BY `programKey`")).toEqual([ + { programKey: "1", weeks: "3", totalMinutesOrZero: "210" }, + { programKey: "2", weeks: "0", totalMinutesOrZero: "0" }, + { programKey: "3", weeks: "0", totalMinutesOrZero: "0" }, + ]); + expect((await select("SELECT count(*) AS n FROM `v_program_roster`"))[0]?.n).toBe("3"); + }); + + test("@default over an empty weeks table: v_fitness_totals_filled is (0, 0, 0)", async () => { + expect(await select("SELECT * FROM `v_fitness_totals_filled`")) + .toEqual([{ weeks: "0", totalMinutesOrZero: "0", longShareOrZero: "0.0000" }]); + }); + + test("COLUMN TYPES: every defaulted column is NOT NULL in information_schema, its undefaulted twin nullable", async () => { + const cols = await select( + `SELECT table_name AS v, column_name AS c, data_type AS t, is_nullable AS n FROM information_schema.columns + WHERE table_schema = DATABASE() + AND table_name IN ('v_program_roster', 'v_program_long_weeks', 'v_fitness_totals_filled') + AND column_name IN ('totalMinutes', 'totalMinutesOrZero', 'longShare', 'longShareOrZero') + ORDER BY table_name, ordinal_position`, + ); + expect(cols).toEqual([ + { v: "v_fitness_totals_filled", c: "totalMinutesOrZero", t: "bigint", n: "NO" }, + { v: "v_fitness_totals_filled", c: "longShareOrZero", t: "decimal", n: "NO" }, + { v: "v_program_long_weeks", c: "totalMinutesOrZero", t: "bigint", n: "NO" }, + { v: "v_program_roster", c: "totalMinutes", t: "bigint", n: "YES" }, + { v: "v_program_roster", c: "totalMinutesOrZero", t: "bigint", n: "NO" }, + { v: "v_program_roster", c: "longShare", t: "decimal", n: "YES" }, + { v: "v_program_roster", c: "longShareOrZero", t: "decimal", n: "NO" }, + ]); + }); }); }); diff --git a/server/typescript/packages/integration-tests/test/report-views-pg.test.ts b/server/typescript/packages/integration-tests/test/report-views-pg.test.ts index 745cb9eb8..f34d1d7fb 100644 --- a/server/typescript/packages/integration-tests/test/report-views-pg.test.ts +++ b/server/typescript/packages/integration-tests/test/report-views-pg.test.ts @@ -145,6 +145,11 @@ const SEED_PROGRAMS_AND_WEEKS = ` (13, 1, NULL, 60), (20, 2, 'Solo', 45);`; +/** The same, plus program 3 with no weeks: the row a `@spine` report keeps and a plain one drops. */ +const SEED_ROSTER = `${SEED_PROGRAMS_AND_WEEKS} + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (3, 'Mobility', 1000, 'DRAFT', '2026-06-01T00:00:00');`; + const SEED_PROGRAMS_BY_TIME = ` INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), @@ -166,12 +171,13 @@ describe("report views — canonical model on real Postgres", () => { // Convergence (spec §7: emit -> apply -> re-diff empty). // ------------------------------------------------------------------------- - test("CONVERGENCE: the six canonical report views migrate from empty, then a second and third migrate propose nothing", async () => { + test("CONVERGENCE: the nine canonical report views migrate from empty, then a second and third migrate propose nothing", async () => { const first = await migrate(canonical); for (const view of [ "v_program_minutes", "v_fitness_totals", "v_programs_by_month", "v_programs_by_week", "v_recent_programs", "v_asset_activity", + "v_program_roster", "v_program_long_weeks", "v_fitness_totals_filled", ]) { expect(first.up).toContain(`CREATE VIEW "${view}" AS`); const r = await sql.raw(`SELECT to_regclass('public.${view}') IS NOT NULL AS ok`).execute(k); @@ -299,6 +305,69 @@ describe("report views — canonical model on real Postgres", () => { (2, 'Stale', 100, 'PUBLISHED', (now() AT TIME ZONE 'UTC') - INTERVAL '60 days');`); expect(await select(`SELECT * FROM "v_recent_programs"`)).toEqual([{ programs: "1" }]); }); + + // ----------------------------------------------------------------------- + // @spine and @default (FR-044 R8 / R9, plan Tables D and E). + // ----------------------------------------------------------------------- + + test("v_program_roster: a row for program 3, which has no weeks; a default reads 0 where its twin reads null", async () => { + await applyRaw(SEED_ROSTER); + const rows = await select(`SELECT * FROM "v_program_roster" ORDER BY "programKey"`); + expect(rows.map((r): Record<string, unknown> => ({ + ...r, longShare: canonicalDecimal(r.longShare), longShareOrZero: canonicalDecimal(r.longShareOrZero), + }))).toEqual([ + { programKey: "1", programTitle: "Foundations", weeks: "4", totalMinutes: "240", totalMinutesOrZero: "240", + longShare: "0.75", longShareOrZero: "0.75" }, + { programKey: "2", programTitle: "Strength", weeks: "1", totalMinutes: "45", totalMinutesOrZero: "45", + longShare: "0", longShareOrZero: "0" }, + { programKey: "3", programTitle: "Mobility", weeks: "0", totalMinutes: null, totalMinutesOrZero: "0", + longShare: null, longShareOrZero: "0" }, + ]); + }); + + test("v_program_long_weeks: the report @segment is in the join, so a program whose weeks it scopes out still has a row", async () => { + await applyRaw(SEED_ROSTER); + expect(await select(`SELECT * FROM "v_program_long_weeks" ORDER BY "programKey"`)).toEqual([ + { programKey: "1", weeks: "3", totalMinutesOrZero: "210" }, + { programKey: "2", weeks: "0", totalMinutesOrZero: "0" }, + { programKey: "3", weeks: "0", totalMinutesOrZero: "0" }, + ]); + }); + + test("v_fitness_totals_filled: no @spine, a default alone; over an empty weeks table it is (0, 0, 0)", async () => { + expect((await select(`SELECT count(*) AS n FROM "weeks"`))[0]?.n).toBe("0"); + const empty = await select(`SELECT * FROM "v_fitness_totals_filled"`); + expect(empty.map((r): Record<string, unknown> => ({ ...r, longShareOrZero: canonicalDecimal(r.longShareOrZero) }))) + .toEqual([{ weeks: "0", totalMinutesOrZero: "0", longShareOrZero: "0" }]); + // With rows, the default does not touch the value: it is v_fitness_totals' row. + await applyRaw(SEED_ROSTER); + const seeded = await select(`SELECT * FROM "v_fitness_totals_filled"`); + expect(seeded.map((r): Record<string, unknown> => ({ ...r, longShareOrZero: canonicalDecimal(r.longShareOrZero) }))) + .toEqual([{ weeks: "5", totalMinutesOrZero: "285", longShareOrZero: "0.6" }]); + }); + + test("the view has one row per program", async () => { + await applyRaw(SEED_ROSTER); + const [programs] = await select(`SELECT count(*) AS n FROM "programs"`); + expect(programs?.n).toBe("3"); + for (const view of ["v_program_roster", "v_program_long_weeks"]) { + expect((await select(`SELECT count(*) AS n FROM "${view}"`))[0]?.n).toBe(programs?.n); + } + // The plain report over the same rows drops program 3: that is what @spine adds. + expect((await select(`SELECT count(*) AS n FROM "v_program_minutes"`))[0]?.n).toBe("2"); + }); + + test("COLUMN TYPES: a defaulted integral sum stays bigint and a defaulted ratio numeric", async () => { + const types = async (view: string) => Object.fromEntries((await sql.raw( + `SELECT column_name, data_type FROM information_schema.columns WHERE table_name = '${view}' ORDER BY ordinal_position`, + ).execute(k)).rows.map((r) => [(r as { column_name: string }).column_name, (r as { data_type: string }).data_type])); + expect(await types("v_program_roster")).toEqual({ + programKey: "bigint", programTitle: "character varying", weeks: "bigint", + totalMinutes: "bigint", totalMinutesOrZero: "bigint", longShare: "numeric", longShareOrZero: "numeric", + }); + expect(await types("v_program_long_weeks")).toEqual({ programKey: "bigint", weeks: "bigint", totalMinutesOrZero: "bigint" }); + expect(await types("v_fitness_totals_filled")).toEqual({ weeks: "bigint", totalMinutesOrZero: "bigint", longShareOrZero: "numeric" }); + }); }); }); @@ -322,8 +391,11 @@ const EVENT_MODEL = JSON.stringify({ "metadata.root": { package: "acme", childre { "source.rdb": { "@kind": "view", "@view": "v_events_by_hour" } } ] } }, ]}}); -/** A Fact whose program reference is NULLABLE, grouped by the referenced program's title. */ -const FACT_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ +/** + * A Fact whose program reference is NULLABLE, grouped by the referenced program's title. + * `report` adds attributes to the report (the `@spine` twin of the NULL-group test). + */ +const factModel = (report: Record<string, unknown> = {}): string => JSON.stringify({ "metadata.root": { package: "acme", children: [ { "object.entity": { name: "Prog", children: [ { "source.rdb": { "@table": "progs" } }, { "field.long": { name: "id" } }, @@ -339,9 +411,91 @@ const FACT_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children { "dimension.attribute": { name: "progTitle", "@of": "Prog.title", "@via": "Fact.fkProg" } }, { "measure.aggregate": { name: "facts", "@agg": "count", "@of": "Fact.id" } }, ] } }, - { "object.report": { name: "FactsByProg", "@from": "Fact", "@dimensions": ["progTitle"], "@measures": ["facts"], children: [ + { "object.report": { name: "FactsByProg", "@from": "Fact", "@dimensions": ["progTitle"], "@measures": ["facts"], ...report, children: [ { "source.rdb": { "@kind": "view", "@view": "v_facts_by_prog" } } ] } }, ]}}); +const FACT_MODEL = factModel(); + +/** The facts of the NULL-group test: two on program 1, three with no program. */ +const FACT_ROWS = ` + INSERT INTO "progs" ("id","title") VALUES (1, 'Alpha'); + INSERT INTO "facts" ("id","progId") VALUES (1, 1), (2, 1), (3, NULL), (4, NULL), (5, NULL);`; + +/** + * Program <- Week, where the week's label is nullable: a `count` and a `sum` scoped to the + * weeks with NO label, in a `@spine` report. A program with no weeks gets one + * null-extended row whose `label` is null too, and that row must not be counted. + */ +const UNLABELLED_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Program", children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "Week", children: [ + { "source.rdb": { "@table": "weeks" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "programId", "@required": true } }, + { "field.string": { name: "label" } }, + { "field.int": { name: "durationMinutes", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { name: "fkProgram", "@fields": "programId", "@references": "Program" } }, + { "dimension.attribute": { name: "programKey", "@of": "Program.id", "@via": "Week.fkProgram" } }, + { "measure.aggregate": { name: "unlabelled", "@agg": "count", "@of": "Week.id", "@filter": { label: { isNull: true } } } }, + { "measure.aggregate": { name: "unlabelledMinutes", "@agg": "sum", "@of": "Week.durationMinutes", "@filter": { label: { isNull: true } } } }, + ] } }, + { "object.report": { name: "UnlabelledWeeks", "@from": "Week", "@spine": "Week.fkProgram", + "@dimensions": ["programKey"], "@measures": ["unlabelled", "unlabelledMinutes"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_unlabelled_weeks" } } ] } }, +]}}); + +/** Program 1 has one unlabelled week of 60 minutes, program 2 only a labelled one, program 3 none. */ +const UNLABELLED_ROWS = ` + INSERT INTO "programs" ("id","title") VALUES (1, 'Foundations'), (2, 'Strength'), (3, 'Mobility'); + INSERT INTO "weeks" ("id","programId","label","durationMinutes") VALUES + (10, 1, 'Week 1', 30), (11, 1, 'Week 2', 60), (12, 1, 'Week 2', 90), (13, 1, NULL, 60), + (20, 2, 'Solo', 45);`; + +/** Program <- Week <- Session, spine `Session.week.program` (plan Table D's two-hop case). */ +const SESSION_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Program", children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "Week", children: [ + { "source.rdb": { "@table": "weeks" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "programId", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { name: "program", "@fields": "programId", "@references": "Program" } }, + ] } }, + { "object.entity": { name: "Session", children: [ + { "source.rdb": { "@table": "sessions" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "weekId", "@required": true } }, + { "field.int": { name: "minutes", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { name: "week", "@fields": "weekId", "@references": "Week" } }, + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "Session.week.program" } }, + { "measure.aggregate": { name: "sessions", "@agg": "count", "@of": "Session.id" } }, + { "measure.aggregate": { name: "totalMinutes", "@agg": "sum", "@of": "Session.minutes", "@default": 0 } }, + ] } }, + { "object.report": { name: "SessionsByProgram", "@from": "Session", "@spine": "Session.week.program", + "@dimensions": ["programTitle"], "@measures": ["sessions", "totalMinutes"], "@filter": { minutes: { gte: 10 } }, children: [ + { "source.rdb": { "@kind": "view", "@view": "v_sessions_by_program" } } ] } }, +]}}); + +/** + * Alpha: two weeks, sessions of 30 and 5 minutes in the first (the 5 is scoped out) and none + * in the second. Beta: a week whose only session is scoped out. Gamma: no weeks at all. + */ +const SESSION_ROWS = ` + INSERT INTO "programs" ("id","title") VALUES (1, 'Alpha'), (2, 'Beta'), (3, 'Gamma'); + INSERT INTO "weeks" ("id","programId") VALUES (10, 1), (11, 1), (20, 2); + INSERT INTO "sessions" ("id","weekId","minutes") VALUES (100, 10, 30), (101, 10, 5), (200, 20, 9);`; /** A Metric table with one measure, optionally a second; the report lists what it has. */ function metricModel(measures: string[]): string { @@ -430,9 +584,7 @@ describe("report views — inline models on real Postgres", () => { const { expected, unmanagedNames } = await migrate(root); await assertConverged(expected, unmanagedNames); - await applyRaw(` - INSERT INTO "progs" ("id","title") VALUES (1, 'Alpha'); - INSERT INTO "facts" ("id","progId") VALUES (1, 1), (2, 1), (3, NULL), (4, NULL), (5, NULL);`); + await applyRaw(FACT_ROWS); const rows = await select(`SELECT * FROM "v_facts_by_prog" ORDER BY "progTitle" NULLS LAST`); expect(rows).toEqual([ { progTitle: "Alpha", facts: "2" }, @@ -440,6 +592,53 @@ describe("report views — inline models on real Postgres", () => { ]); }, 60_000); + test("with @spine a fact whose reference is null is in no row", async () => { + // The twin of the NULL-group test above: the same model and rows, with @spine added. + const root = await loadInline(factModel({ "@spine": "Fact.fkProg" })); + expect(viewSql(root, "v_facts_by_prog")).toContain(`FROM "progs"`); + const { expected, unmanagedNames } = await migrate(root); + await assertConverged(expected, unmanagedNames); + + await applyRaw(FACT_ROWS); + // One row per prog; the three facts with no prog belong to none of them. + expect(await select(`SELECT * FROM "v_facts_by_prog" ORDER BY "progTitle"`)).toEqual([ + { progTitle: "Alpha", facts: "2" }, + ]); + }, 60_000); + + test("an isNull condition does not count the empty row", async () => { + const root = await loadInline(UNLABELLED_MODEL); + const { expected, unmanagedNames } = await migrate(root); + await assertConverged(expected, unmanagedNames); + + await applyRaw(UNLABELLED_ROWS); + // Program 3's null-extended row has a null label, and the count still reads 0: it counts + // the fact's id, which is null there too. The sum of nothing is null (no @default). + expect(await select(`SELECT * FROM "v_unlabelled_weeks" ORDER BY "programKey"`)).toEqual([ + { programKey: "1", unlabelled: "1", unlabelledMinutes: "60" }, + { programKey: "2", unlabelled: "0", unlabelledMinutes: null }, + { programKey: "3", unlabelled: "0", unlabelledMinutes: null }, + ]); + }, 60_000); + + test("a two-hop spine: every program has a row, whether it has no weeks or only scoped-out sessions", async () => { + const root = await loadInline(SESSION_MODEL); + expect(viewSql(root, "v_sessions_by_program")).toContain([ + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` LEFT OUTER JOIN "sessions" s ON w."id" = s."weekId" AND s."minutes" >= 10`, + ].join("\n")); + const { expected, unmanagedNames } = await migrate(root); + await assertConverged(expected, unmanagedNames); + + await applyRaw(SESSION_ROWS); + expect(await select(`SELECT * FROM "v_sessions_by_program" ORDER BY "programTitle"`)).toEqual([ + { programTitle: "Alpha", sessions: "1", totalMinutes: "30" }, + { programTitle: "Beta", sessions: "0", totalMinutes: "0" }, + { programTitle: "Gamma", sessions: "0", totalMinutes: "0" }, + ]); + }, 60_000); + test("CHANGE A REPORT: adding a measure is a drop and a create of that view, and then converges", async () => { const before = await loadInline(metricModel(["samples"])); const first = await migrate(before); diff --git a/server/typescript/packages/integration-tests/test/report-views-sqlite.test.ts b/server/typescript/packages/integration-tests/test/report-views-sqlite.test.ts index 015969ba1..41fb66355 100644 --- a/server/typescript/packages/integration-tests/test/report-views-sqlite.test.ts +++ b/server/typescript/packages/integration-tests/test/report-views-sqlite.test.ts @@ -129,6 +129,11 @@ const SEED_PROGRAMS_AND_WEEKS = ` (13, 1, NULL, 60), (20, 2, 'Solo', 45)`; +/** The same, plus program 3 with no weeks: the row a `@spine` report keeps and a plain one drops. */ +const SEED_ROSTER = `${SEED_PROGRAMS_AND_WEEKS}; + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (3, 'Mobility', 1000, 'DRAFT', '2026-06-01T00:00:00')`; + const SEED_PROGRAMS_BY_TIME = ` INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), @@ -148,11 +153,12 @@ describe("report views — canonical model on real SQLite", () => { ({ expected } = await migrate(canonical)); }); - test("CONVERGENCE: the six canonical views apply, then a second and third migrate propose nothing (the emitter is deterministic)", async () => { + test("CONVERGENCE: the nine canonical views apply, then a second and third migrate propose nothing (the emitter is deterministic)", async () => { const views = (await select(`SELECT name FROM sqlite_master WHERE type = 'view' ORDER BY name`)).map((r) => r.name); for (const v of [ "v_program_minutes", "v_fitness_totals", "v_programs_by_month", "v_programs_by_week", "v_recent_programs", "v_asset_activity", + "v_program_roster", "v_program_long_weeks", "v_fitness_totals_filled", ]) { expect(views).toContain(v); } @@ -265,6 +271,73 @@ describe("report views — canonical model on real SQLite", () => { (2, 'Stale', 100, 'PUBLISHED', strftime('%Y-%m-%dT%H:%M:%f','now','-60 days'))`); expect(await select(`SELECT * FROM "v_recent_programs"`)).toEqual([{ programs: 1 }]); }); + + // ------------------------------------------------------------------------- + // @spine and @default (FR-044 R8 / R9, plan Tables D and E). + // ------------------------------------------------------------------------- + + test("v_program_roster: a row for program 3, which has no weeks; a default reads 0 where its twin reads null", async () => { + await applyRaw(SEED_ROSTER); + expect(await select(`SELECT * FROM "v_program_roster" ORDER BY "programKey"`)).toEqual([ + { programKey: 1, programTitle: "Foundations", weeks: 4, totalMinutes: 240, totalMinutesOrZero: 240, + longShare: 0.75, longShareOrZero: 0.75 }, + { programKey: 2, programTitle: "Strength", weeks: 1, totalMinutes: 45, totalMinutesOrZero: 45, + longShare: 0, longShareOrZero: 0 }, + { programKey: 3, programTitle: "Mobility", weeks: 0, totalMinutes: null, totalMinutesOrZero: 0, + longShare: null, longShareOrZero: 0 }, + ]); + }); + + test("v_program_long_weeks: the report @segment is in the join, so a program whose weeks it scopes out still has a row", async () => { + await applyRaw(SEED_ROSTER); + expect(await select(`SELECT * FROM "v_program_long_weeks" ORDER BY "programKey"`)).toEqual([ + { programKey: 1, weeks: 3, totalMinutesOrZero: 210 }, + { programKey: 2, weeks: 0, totalMinutesOrZero: 0 }, + { programKey: 3, weeks: 0, totalMinutesOrZero: 0 }, + ]); + }); + + test("v_fitness_totals_filled: no @spine, a default alone; over an empty weeks table it is (0, 0, 0)", async () => { + expect(await select(`SELECT count(*) AS n FROM "weeks"`)).toEqual([{ n: 0 }]); + expect(await select(`SELECT * FROM "v_fitness_totals_filled"`)) + .toEqual([{ weeks: 0, totalMinutesOrZero: 0, longShareOrZero: 0 }]); + // With rows, the default does not touch the value: it is v_fitness_totals' row. + await applyRaw(SEED_ROSTER); + expect(await select(`SELECT * FROM "v_fitness_totals_filled"`)) + .toEqual([{ weeks: 5, totalMinutesOrZero: 285, longShareOrZero: 0.6 }]); + }); + + test("the view has one row per program", async () => { + await applyRaw(SEED_ROSTER); + const [programs] = await select(`SELECT count(*) AS n FROM "programs"`); + expect(programs?.n).toBe(3); + for (const view of ["v_program_roster", "v_program_long_weeks"]) { + expect((await select(`SELECT count(*) AS n FROM "${view}"`))[0]?.n).toBe(programs?.n); + } + // The plain report over the same rows drops program 3: that is what @spine adds. + expect((await select(`SELECT count(*) AS n FROM "v_program_minutes"`))[0]?.n).toBe(2); + }); + + test("STORAGE CLASSES: a defaulted ratio is REAL in every row, the defaulted ones included (the 0.0 literal)", async () => { + await applyRaw(SEED_ROSTER); + expect(await select( + `SELECT typeof("totalMinutes") AS "totalMinutes", typeof("totalMinutesOrZero") AS "totalMinutesOrZero", + typeof("longShare") AS "longShare", typeof("longShareOrZero") AS "longShareOrZero" + FROM "v_program_roster" ORDER BY "programKey"`, + )).toEqual([ + { totalMinutes: "integer", totalMinutesOrZero: "integer", longShare: "real", longShareOrZero: "real" }, + { totalMinutes: "integer", totalMinutesOrZero: "integer", longShare: "real", longShareOrZero: "real" }, + { totalMinutes: "null", totalMinutesOrZero: "integer", longShare: "null", longShareOrZero: "real" }, + ]); + expect(await select( + `SELECT typeof("totalMinutesOrZero") AS "totalMinutesOrZero" FROM "v_program_long_weeks" ORDER BY "programKey"`, + )).toEqual([{ totalMinutesOrZero: "integer" }, { totalMinutesOrZero: "integer" }, { totalMinutesOrZero: "integer" }]); + await applyRaw(`DELETE FROM "weeks"`); + expect(await select( + `SELECT typeof("totalMinutesOrZero") AS "totalMinutesOrZero", typeof("longShareOrZero") AS "longShareOrZero" + FROM "v_fitness_totals_filled"`, + )).toEqual([{ totalMinutesOrZero: "integer", longShareOrZero: "real" }]); + }); }); /** @@ -301,6 +374,87 @@ const ENGAGEMENT_ROWS = ` (1, 'b@x.test', 1, 1), (1, 'c@x.test', 1, 1), (1, 'a@x.test', 1, 1)`; +/** A Fact whose prog reference is NULLABLE, in a `@spine` report grouped by the prog's title. */ +const FACT_SPINE_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Prog", children: [ + { "source.rdb": { "@table": "progs" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "Fact", children: [ + { "source.rdb": { "@table": "facts" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "progId" } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { name: "fkProg", "@fields": "progId", "@references": "Prog" } }, + { "dimension.attribute": { name: "progTitle", "@of": "Prog.title", "@via": "Fact.fkProg" } }, + { "measure.aggregate": { name: "facts", "@agg": "count", "@of": "Fact.id" } }, + ] } }, + { "object.report": { name: "FactsByProg", "@from": "Fact", "@spine": "Fact.fkProg", "@dimensions": ["progTitle"], "@measures": ["facts"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_facts_by_prog" } } ] } }, +]}}); + +/** + * Program <- Week, where the week's label is nullable: a `count` and a `sum` scoped to the + * weeks with NO label, in a `@spine` report. A program with no weeks gets one + * null-extended row whose `label` is null too, and that row must not be counted. + */ +const UNLABELLED_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Program", children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "Week", children: [ + { "source.rdb": { "@table": "weeks" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "programId", "@required": true } }, + { "field.string": { name: "label" } }, + { "field.int": { name: "durationMinutes", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { name: "fkProgram", "@fields": "programId", "@references": "Program" } }, + { "dimension.attribute": { name: "programKey", "@of": "Program.id", "@via": "Week.fkProgram" } }, + { "measure.aggregate": { name: "unlabelled", "@agg": "count", "@of": "Week.id", "@filter": { label: { isNull: true } } } }, + { "measure.aggregate": { name: "unlabelledMinutes", "@agg": "sum", "@of": "Week.durationMinutes", "@filter": { label: { isNull: true } } } }, + ] } }, + { "object.report": { name: "UnlabelledWeeks", "@from": "Week", "@spine": "Week.fkProgram", + "@dimensions": ["programKey"], "@measures": ["unlabelled", "unlabelledMinutes"], children: [ + { "source.rdb": { "@kind": "view", "@view": "v_unlabelled_weeks" } } ] } }, +]}}); + +/** Program <- Week <- Session, spine `Session.week.program` (plan Table D's two-hop case). */ +const SESSION_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ + { "object.entity": { name: "Program", children: [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { name: "id" } }, + { "field.string": { name: "title", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + ] } }, + { "object.entity": { name: "Week", children: [ + { "source.rdb": { "@table": "weeks" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "programId", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { name: "program", "@fields": "programId", "@references": "Program" } }, + ] } }, + { "object.entity": { name: "Session", children: [ + { "source.rdb": { "@table": "sessions" } }, + { "field.long": { name: "id" } }, + { "field.long": { name: "weekId", "@required": true } }, + { "field.int": { name: "minutes", "@required": true } }, + { "identity.primary": { name: "id", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { name: "week", "@fields": "weekId", "@references": "Week" } }, + { "dimension.attribute": { name: "programTitle", "@of": "Program.title", "@via": "Session.week.program" } }, + { "measure.aggregate": { name: "sessions", "@agg": "count", "@of": "Session.id" } }, + { "measure.aggregate": { name: "totalMinutes", "@agg": "sum", "@of": "Session.minutes", "@default": 0 } }, + ] } }, + { "object.report": { name: "SessionsByProgram", "@from": "Session", "@spine": "Session.week.program", + "@dimensions": ["programTitle"], "@measures": ["sessions", "totalMinutes"], "@filter": { minutes: { gte: 10 } }, children: [ + { "source.rdb": { "@kind": "view", "@view": "v_sessions_by_program" } } ] } }, +]}}); + describe("report views — inline model on real SQLite", () => { /** A Stamp table with date and naive-timestamp columns for the quarter / year grains. */ const STAMP_MODEL = JSON.stringify({ "metadata.root": { package: "acme", children: [ @@ -365,6 +519,63 @@ describe("report views — inline model on real SQLite", () => { expect(await select(`SELECT * FROM "v_pair_totals"`)).toEqual([{ combos: 2 }]); }); + test("with @spine a fact whose reference is null is in no row", async () => { + const root = await loadInline(FACT_SPINE_MODEL); + expect(viewSql(root, "v_facts_by_prog")).toContain(`FROM "progs"`); + const { expected } = await migrate(root); + await assertConverged(expected); + + await applyRaw(` + INSERT INTO "progs" ("id","title") VALUES (1, 'Alpha'); + INSERT INTO "facts" ("id","progId") VALUES (1, 1), (2, 1), (3, NULL), (4, NULL), (5, NULL)`); + // One row per prog; the three facts with no prog belong to none of them. + expect(await select(`SELECT * FROM "v_facts_by_prog" ORDER BY "progTitle"`)).toEqual([ + { progTitle: "Alpha", facts: 2 }, + ]); + }); + + test("an isNull condition does not count the empty row", async () => { + const root = await loadInline(UNLABELLED_MODEL); + const { expected } = await migrate(root); + await assertConverged(expected); + + await applyRaw(` + INSERT INTO "programs" ("id","title") VALUES (1, 'Foundations'), (2, 'Strength'), (3, 'Mobility'); + INSERT INTO "weeks" ("id","programId","label","durationMinutes") VALUES + (10, 1, 'Week 1', 30), (11, 1, 'Week 2', 60), (12, 1, 'Week 2', 90), (13, 1, NULL, 60), + (20, 2, 'Solo', 45)`); + // Program 3's null-extended row has a null label, and the count still reads 0: it counts + // the fact's id, which is null there too. The sum of nothing is null (no @default). + expect(await select(`SELECT * FROM "v_unlabelled_weeks" ORDER BY "programKey"`)).toEqual([ + { programKey: 1, unlabelled: 1, unlabelledMinutes: 60 }, + { programKey: 2, unlabelled: 0, unlabelledMinutes: null }, + { programKey: 3, unlabelled: 0, unlabelledMinutes: null }, + ]); + }); + + test("a two-hop spine: every program has a row, whether it has no weeks or only scoped-out sessions", async () => { + const root = await loadInline(SESSION_MODEL); + expect(viewSql(root, "v_sessions_by_program")).toContain([ + ` FROM "programs" p`, + ` LEFT OUTER JOIN "weeks" w ON p."id" = w."programId"`, + ` LEFT OUTER JOIN "sessions" s ON w."id" = s."weekId" AND s."minutes" >= 10`, + ].join("\n")); + const { expected } = await migrate(root); + await assertConverged(expected); + + // Alpha: two weeks, sessions of 30 and 5 minutes in the first (the 5 is scoped out) and + // none in the second. Beta: a week whose only session is scoped out. Gamma: no weeks. + await applyRaw(` + INSERT INTO "programs" ("id","title") VALUES (1, 'Alpha'), (2, 'Beta'), (3, 'Gamma'); + INSERT INTO "weeks" ("id","programId") VALUES (10, 1), (11, 1), (20, 2); + INSERT INTO "sessions" ("id","weekId","minutes") VALUES (100, 10, 30), (101, 10, 5), (200, 20, 9)`); + expect(await select(`SELECT * FROM "v_sessions_by_program" ORDER BY "programTitle"`)).toEqual([ + { programTitle: "Alpha", sessions: 1, totalMinutes: 30 }, + { programTitle: "Beta", sessions: 0, totalMinutes: 0 }, + { programTitle: "Gamma", sessions: 0, totalMinutes: 0 }, + ]); + }); + test("QUARTER and YEAR grains: every month of a quarter lands on its first day, and the view converges", async () => { const root = await loadInline(STAMP_MODEL); expect(viewSql(root, "v_stamps_by_quarter")).toContain( From 2d6f8eab3348bdbec31852a629f07f703d5bad85 Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 20:25:32 -0400 Subject: [PATCH 10/18] test(persistence): @spine and @default read scenarios; describe them in meta docs --- fixtures/codegen-noop/reporting/README.md | 20 ++++--- .../reporting/with/meta.shop.json | 28 ++++++++- fixtures/persistence-conformance/README.md | 11 +++- .../queries/report-default-empty.yaml | 14 +++++ .../queries/report-spine-scoped.yaml | 42 +++++++++++++ .../queries/report-spine-zero-rows.yaml | 58 ++++++++++++++++++ .../cli/test/unit/reporting-inert.test.ts | 12 ++-- .../codegen-ts/src/generators/report-doc.ts | 14 +++-- .../codegen-ts/test/reporting-docs.test.ts | 35 +++++++++-- .../docs-site/src/builders/report-data.ts | 15 +++-- .../docs-site/templates/object.html.mustache | 2 +- .../packages/docs-site/test/coverage.test.ts | 5 +- .../docs-site/test/reporting-site.test.ts | 28 ++++++++- .../src/core/reporting/report-describe.ts | 59 +++++++++++++++++-- .../src/core/reporting/report-shape.ts | 29 +++++++++ .../typescript/packages/metadata/src/index.ts | 3 + .../metadata/test/report-describe.test.ts | 55 ++++++++++++++++- .../metadata/test/report-shape.test.ts | 31 ++++++++++ 18 files changed, 420 insertions(+), 41 deletions(-) create mode 100644 fixtures/persistence-conformance/queries/report-default-empty.yaml create mode 100644 fixtures/persistence-conformance/queries/report-spine-scoped.yaml create mode 100644 fixtures/persistence-conformance/queries/report-spine-zero-rows.yaml diff --git a/fixtures/codegen-noop/reporting/README.md b/fixtures/codegen-noop/reporting/README.md index ffffd658a..777ae096c 100644 --- a/fixtures/codegen-noop/reporting/README.md +++ b/fixtures/codegen-noop/reporting/README.md @@ -4,11 +4,16 @@ Two models that differ ONLY by the FR-044 reporting vocabulary: - `without/meta.shop.json` — three entities (`Program`, `Purchase`, `WorkoutEvent`). - `with/meta.shop.json` — the same entities plus their `dimension.*`, `measure.*` and - `segment.filter` children and three `object.report` nodes. It is the positive - conformance fixture `fixtures/conformance/reporting-vocabulary/` with one addition: - `StoreTotals` declares a read-only `source.rdb @kind: view` (rule R5 allows one), because - a view-backed report passes every source-keyed codegen gate and is the shape most likely - to leak output. + `segment.filter` children and four `object.report` nodes. It is the positive + conformance fixture `fixtures/conformance/reporting-vocabulary/` with these additions: + - `StoreTotals` declares a read-only `source.rdb @kind: view` (rule R5 allows one), because + a view-backed report passes every source-keyed codegen gate and is the shape most likely + to leak output. + - `ProgramCatalogue` declares `@spine: "Purchase.program"` and a report `@filter`, and lists + `programKey`, a dimension over `Program.id` reached by `@via: "Purchase.program"`. It + declares no source, so it is inert like the other sourceless reports. + - The ratio `avgDaysPerStarter` declares `@default: 0`. Only the sourceless + `ProgramEngagement` lists it, so no served output moves. What is lowered (FR-044 Plan 2): a report that declares a read-only `source.rdb @kind: view` becomes that view. What is served (FR-044 Plan 3): that same report gets a keyless read-only @@ -37,12 +42,13 @@ these places and no others: - Python codegen (`metaobjects gen`) writes four extra files, one per generator: `StoreTotals.py` (the Pydantic row model), `store_totals_filter_allowlist.py`, `store_totals_router.py` and `store_totals_names.py`. -- Documentation: `meta docs` writes a model page per report (all three, the two sourceless +- Documentation: `meta docs` writes a model page per report (all four, the three sourceless ones marked "Not served"), lists them under `## Reports` on the model index, adds a "Reporting" section to the pages of `Purchase` and `WorkoutEvent`, and writes one API page, for `StoreTotals`. Every other port's api-docs builder gains exactly the `StoreTotals` page. -What stays inert: a report with no read-only source (`ProgramEngagement`, `DailyRevenue`), in +What stays inert: a report with no read-only source (`ProgramEngagement`, `DailyRevenue`, +`ProgramCatalogue`), in every generator, migration and runtime (its only output is the model page above); the client UI tier for every unserved report, and every port's client tier but the TypeScript list hook (no other port has one). No port but TypeScript emits SQL for a report diff --git a/fixtures/codegen-noop/reporting/with/meta.shop.json b/fixtures/codegen-noop/reporting/with/meta.shop.json index b55b953ac..1e2007f03 100644 --- a/fixtures/codegen-noop/reporting/with/meta.shop.json +++ b/fixtures/codegen-noop/reporting/with/meta.shop.json @@ -154,6 +154,13 @@ ] } }, + { + "dimension.attribute": { + "name": "programKey", + "@of": "Program.id", + "@via": "Purchase.program" + } + }, { "measure.aggregate": { "name": "refundedPurchases", @@ -290,7 +297,8 @@ "measure.ratio": { "name": "avgDaysPerStarter", "@numerator": "daysEngaged", - "@denominator": "starters" + "@denominator": "starters", + "@default": 0 } }, { @@ -358,6 +366,24 @@ } ] } + }, + { + "object.report": { + "name": "ProgramCatalogue", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programKey", + "programTitle" + ], + "@measures": [ + "purchases", + "revenue" + ], + "@filter": { + "refunded": false + } + } } ] } diff --git a/fixtures/persistence-conformance/README.md b/fixtures/persistence-conformance/README.md index 52555d156..9b1948c09 100644 --- a/fixtures/persistence-conformance/README.md +++ b/fixtures/persistence-conformance/README.md @@ -316,7 +316,7 @@ Because M:N membership is a **set**, the runner compares `relate` results ### Report scenarios (FR-044) -Six `queries/report-*.yaml` scenarios read a view-backed `object.report` through the +Nine `queries/report-*.yaml` scenarios read a view-backed `object.report` through the port's runtime. They use only `op: list` and `op: count`, single-key `sort`, `filter` and `limit`: a report has no primary key, so there is no `op: get` and no write. The schema is still the committed `canonical/schema.postgres.sql`, which creates each report's view, so a @@ -333,12 +333,19 @@ then measures in `@measures` order; a time dimension is named `<name><Grain>`, f | `report-time-grains` | `month` grain beside an enum dimension; a segment-filtered `sum` that is `null`; the ISO Monday week boundary; a report-level `@segment` | | `report-time-hour-and-date` | `hour` on an instant, `week` on a `field.date` | | `report-relative-date` | a `{ now: "-P30D" }` filter, with a seed relative to the database clock | +| `report-spine-zero-rows` | `@spine`: a program with no weeks has a row (`weeks` `0`, `totalMinutes` and `longShare` null, the `@default: 0` measures `0`); `count` is the number of programs; a measure filter drops the empty rows; filter and sort on a defaulted measure | +| `report-spine-scoped` | `@spine` with a report `@segment`: a program whose weeks are all out of scope keeps its row, with `weeks` `0` | +| `report-default-empty` | `@default: 0` with no `@spine`, over an empty table: the twin of `report-totals-empty`, the sum and the ratio reading `0` | Wire shapes follow the view's column types and [`normalization.md`](./normalization.md): `count` and the integral `sum` are `BIGINT` (string), `min`/`max` of an int are `INTEGER` (number), `avg` and a ratio are `NUMERIC` (canonical decimal string, so `"60"`, `"0.75"` and `"0"`), a `date`-grain bucket is a `DATE` string, and an instant `hour` bucket is a -`TIMESTAMPTZ` string in UTC (`"2026-05-04T03:00:00Z"`). +`TIMESTAMPTZ` string in UTC (`"2026-05-04T03:00:00Z"`). A measure with `@default` keeps its +column type, so a defaulted integral `sum` is still a `BIGINT` string and a defaulted ratio a +canonical decimal string. No scenario sorts a measure that can be null across a row where it +is null: engines disagree on where nulls sort (Postgres last ascending, SQLite and MySQL +first), while a defaulted measure sorts as its default everywhere. ### Filter operators diff --git a/fixtures/persistence-conformance/queries/report-default-empty.yaml b/fixtures/persistence-conformance/queries/report-default-empty.yaml new file mode 100644 index 000000000..c1f6bc009 --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-default-empty.yaml @@ -0,0 +1,14 @@ +name: report-default-empty +description: | + FitnessTotalsFilled over an EMPTY weeks table: the twin of report-totals-empty, with + @default: 0 on the sum and the ratio and no @spine. A report with no dimensions still + returns exactly one row; the count is 0 as before, and where report-totals-empty + reads null (a sum of nothing, a ratio over a zero denominator) the defaulted measures + read their default, 0. The sum stays BIGINT (string) and the ratio NUMERIC + (canonical decimal string). +queries: + - name: one-row-over-an-empty-table + op: list + entity: FitnessTotalsFilled + expect: + - { weeks: "0", totalMinutesOrZero: "0", longShareOrZero: "0" } diff --git a/fixtures/persistence-conformance/queries/report-spine-scoped.yaml b/fixtures/persistence-conformance/queries/report-spine-scoped.yaml new file mode 100644 index 000000000..5aab5c401 --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-spine-scoped.yaml @@ -0,0 +1,42 @@ +name: report-spine-scoped +description: | + ProgramLongWeeks is ProgramRoster's twin with a report-level @segment: long + (durationMinutes >= 60). Under @spine the report's row scope decides which weeks are + aggregated and never removes a program: the scope sits in the join condition, not in + a WHERE that would turn the outer join back into an inner one. + + Program 1's long weeks are 60, 90 and 60 (its 30-minute week is out of scope). + Program 2's only week is 45 minutes, so it keeps its row with weeks 0 and the + defaulted totalMinutesOrZero 0, exactly like program 3, which has no weeks at all. +seed-data: | + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'), + (3, 'Mobility', 1500, 'DRAFT', '2026-05-20T08:00:00'); + INSERT INTO "weeks" ("id","programId","label","durationMinutes") VALUES + (10, 1, 'Week 1', 30), + (11, 1, 'Week 2', 60), + (12, 1, 'Week 2', 90), + (13, 1, NULL, 60), + (20, 2, 'Solo', 45); +queries: + - name: a-program-whose-weeks-are-all-out-of-scope-keeps-its-row + op: list + entity: ProgramLongWeeks + sort: [{ field: programKey, dir: asc }] + expect: + - { programKey: "1", weeks: "3", totalMinutesOrZero: "210" } + - { programKey: "2", weeks: "0", totalMinutesOrZero: "0" } + - { programKey: "3", weeks: "0", totalMinutesOrZero: "0" } + - name: count-is-the-number-of-programs + op: count + entity: ProgramLongWeeks + expect: 3 + - name: the-empty-rows + op: list + entity: ProgramLongWeeks + filter: { weeks: { eq: 0 } } + sort: [{ field: programKey, dir: asc }] + expect: + - { programKey: "2", weeks: "0", totalMinutesOrZero: "0" } + - { programKey: "3", weeks: "0", totalMinutesOrZero: "0" } diff --git a/fixtures/persistence-conformance/queries/report-spine-zero-rows.yaml b/fixtures/persistence-conformance/queries/report-spine-zero-rows.yaml new file mode 100644 index 000000000..9da02cd9c --- /dev/null +++ b/fixtures/persistence-conformance/queries/report-spine-zero-rows.yaml @@ -0,0 +1,58 @@ +name: report-spine-zero-rows +description: | + ProgramRoster is an object.report over Week with @spine: "Week.fkProgram": its rows + come from the programs, not from the weeks, so a program with no weeks still has a + row. Program 3 (Mobility) has none: weeks is 0 (a count), totalMinutes and longShare + are null (no @default), and totalMinutesOrZero and longShareOrZero read their + declared @default, 0. + + Wire shapes follow the view's column types: programKey (Program.id), the count and + the integral sums are BIGINT (string), the ratios are NUMERIC (canonical decimal + string). A defaulted measure keeps its column type. + + A defaulted measure is filterable and sortable by its default: eq 0 matches the empty + row, and an ascending sort puts it first on every engine. No query here sorts + totalMinutes or longShare, which are null in that row and sort where the engine puts + nulls. +seed-data: | + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01T10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17T23:30:00'), + (3, 'Mobility', 1500, 'DRAFT', '2026-05-20T08:00:00'); + INSERT INTO "weeks" ("id","programId","label","durationMinutes") VALUES + (10, 1, 'Week 1', 30), + (11, 1, 'Week 2', 60), + (12, 1, 'Week 2', 90), + (13, 1, NULL, 60), + (20, 2, 'Solo', 45); +queries: + - name: every-program-has-a-row + op: list + entity: ProgramRoster + sort: [{ field: programKey, dir: asc }] + expect: + - { programKey: "1", programTitle: "Foundations", weeks: "4", totalMinutes: "240", totalMinutesOrZero: "240", longShare: "0.75", longShareOrZero: "0.75" } + - { programKey: "2", programTitle: "Strength", weeks: "1", totalMinutes: "45", totalMinutesOrZero: "45", longShare: "0", longShareOrZero: "0" } + - { programKey: "3", programTitle: "Mobility", weeks: "0", totalMinutes: null, totalMinutesOrZero: "0", longShare: null, longShareOrZero: "0" } + - name: count-is-the-number-of-programs + op: count + entity: ProgramRoster + expect: 3 + - name: the-default-is-filterable + op: list + entity: ProgramRoster + filter: { totalMinutesOrZero: { eq: 0 } } + expect: + - { programKey: "3", programTitle: "Mobility", weeks: "0", totalMinutes: null, totalMinutesOrZero: "0", longShare: null, longShareOrZero: "0" } + - name: a-count-filter-drops-the-empty-rows + op: count + entity: ProgramRoster + filter: { weeks: { gt: 0 } } + expect: 2 + - name: sort-on-the-defaulted-measure + op: list + entity: ProgramRoster + sort: [{ field: totalMinutesOrZero, dir: asc }] + limit: 1 + expect: + - { programKey: "3", programTitle: "Mobility", weeks: "0", totalMinutes: null, totalMinutesOrZero: "0", longShare: null, longShareOrZero: "0" } diff --git a/server/typescript/packages/cli/test/unit/reporting-inert.test.ts b/server/typescript/packages/cli/test/unit/reporting-inert.test.ts index 821bb8694..7a2518934 100644 --- a/server/typescript/packages/cli/test/unit/reporting-inert.test.ts +++ b/server/typescript/packages/cli/test/unit/reporting-inert.test.ts @@ -124,7 +124,9 @@ const BARREL = `${OUT}/index.ts`; /** The client UI generators that emit NOTHING for a report: a report has a list hook and no * grid, grid hook or form. */ const UI_TIER = ["grid", "grid-hook", "form"] as const; -const SOURCELESS_REPORTS = ["ProgramEngagement", "DailyRevenue"] as const; +/** `ProgramCatalogue` declares `@spine` and lists `programKey`, a dimension reached by + * `@via`; `avgDaysPerStarter` declares `@default`. Sourceless, so still inert. */ +const SOURCELESS_REPORTS = ["ProgramEngagement", "DailyRevenue", "ProgramCatalogue"] as const; /** Assert `actual` is `expected` plus exactly `added`, with every shared file * byte-identical except the barrel. */ @@ -148,7 +150,7 @@ function expectOnlyAdds( describe("FR-044 a sourceless report is inert; a served report emits exactly its read-only files", () => { test("the with-model really carries the vocabulary (else every check below is vacuous)", () => { const reports = withReporting.objects().filter((o) => o.subType === OBJECT_SUBTYPE_REPORT); - expect(reports.map((o) => o.name).sort()).toEqual(["DailyRevenue", "ProgramEngagement", "StoreTotals"]); + expect(reports.map((o) => o.name).sort()).toEqual(["DailyRevenue", "ProgramCatalogue", "ProgramEngagement", "StoreTotals"]); expect(withoutReporting.objects().some((o) => o.subType === OBJECT_SUBTYPE_REPORT)).toBe(false); }); @@ -261,7 +263,7 @@ describe("FR-044 a selection of only reports", () => { }); test("a served report among them generates, and only for itself", async () => { - const result = await run(["DailyRevenue", "ProgramEngagement", "StoreTotals"]); + const result = await run(["DailyRevenue", "ProgramCatalogue", "ProgramEngagement", "StoreTotals"]); expect(result.warnings.some((w) => w.startsWith("No entities to generate"))).toBe(false); const names = result.files.map((f) => f.path.split(sep).pop()!); for (const file of Object.values(SERVED_REPORT_FILES).flat()) { @@ -328,7 +330,7 @@ describe("FR-044 meta docs differs by exactly the report pages, the Reporting se } } - const REPORTS = ["DailyRevenue", "ProgramEngagement", "StoreTotals"] as const; + const REPORTS = ["DailyRevenue", "ProgramCatalogue", "ProgramEngagement", "StoreTotals"] as const; /** The entities that declare dimensions, measures or segments in the with-model. */ const REPORTING_ENTITIES = ["Purchase", "WorkoutEvent"] as const; const SITE = "out--site/site"; @@ -406,7 +408,7 @@ describe("FR-044 meta docs differs by exactly the report pages, the Reporting se expect(actual[`${SITE_PKG}/${name}.html`]).toContain('id="s-reporting"'); expect(expected[`${SITE_PKG}/${name}.html`]).not.toContain('id="s-reporting"'); } - // An entity with no reporting nodes changes by its sidebar alone: the three report + // An entity with no reporting nodes changes by its sidebar alone: the four report // links, in the package it shares with them. const added = insertedLines(expected[`${SITE_PKG}/Program.html`]!, actual[`${SITE_PKG}/Program.html`]!); expect(added.length).toBe(REPORTS.length); diff --git a/server/typescript/packages/codegen-ts/src/generators/report-doc.ts b/server/typescript/packages/codegen-ts/src/generators/report-doc.ts index 85c962a30..b1c68575e 100644 --- a/server/typescript/packages/codegen-ts/src/generators/report-doc.ts +++ b/server/typescript/packages/codegen-ts/src/generators/report-doc.ts @@ -14,15 +14,14 @@ import { type MetaObject, type MetaRoot, DIMENSION_SUBTYPE_TIME, - OBJECT_REPORT_ATTR_FILTER, - OBJECT_REPORT_ATTR_SEGMENT, TYPE_DIMENSION, TYPE_MEASURE, TYPE_SEGMENT, describeDimension, describeMeasure, describeReportField, - describeRowScope, + describeReportRowScope, + describeReportRows, describeSegment, isMetaObject, reportFieldTypeName, @@ -48,7 +47,8 @@ function cell(text: string): string { /** * The body of a report page's "Report" section: `@from` (linked), the view or the - * not-served line, the row scope, and the column table. + * not-served line, where a `@spine` report's rows come from, the row scope, and the column + * table. * * Built from `reportShape`, which resolves for a report with no source, and not from the * read model, which exists only to give a SERVED report ordinary fields. @@ -60,8 +60,10 @@ export function buildReportBlock(report: MetaObject, root: MetaRoot, layout: Out lines.push(`**From:** [${shape.from.name}](${fromHref})`); const notServed = reportNotServedReason(report); lines.push(`**View:** ${notServed ?? tick(reportReadSource(report)?.physicalName ?? "")}`); - // ADR-0039: resolving. - const scope = describeRowScope(report.attr(OBJECT_REPORT_ATTR_SEGMENT), report.attr(OBJECT_REPORT_ATTR_FILTER)); + // Only a report with `@spine` has a Rows line; without one the page is as it was. + const rows = describeReportRows(shape, root); + if (rows !== undefined) lines.push(`**Rows:** ${rows}`); + const scope = describeReportRowScope(report); if (scope !== undefined) lines.push(`**Row scope:** ${scope}`); if (shape.fields.length > 0) { lines.push("", "| Column | Type | Nullable | Role | Definition |", "|---|---|---|---|---|"); diff --git a/server/typescript/packages/codegen-ts/test/reporting-docs.test.ts b/server/typescript/packages/codegen-ts/test/reporting-docs.test.ts index eba6c3cc4..08e8bdd8c 100644 --- a/server/typescript/packages/codegen-ts/test/reporting-docs.test.ts +++ b/server/typescript/packages/codegen-ts/test/reporting-docs.test.ts @@ -115,12 +115,13 @@ describe("FR-044 no-churn: a model with no report renders every docs page as bef describe("FR-044 model surface: a page for every report", () => { test("each report has a page, linked from the index under Reports", async () => { const pages = await modelPages(withReporting); - for (const name of ["StoreTotals", "ProgramEngagement", "DailyRevenue"]) { + for (const name of ["StoreTotals", "ProgramEngagement", "DailyRevenue", "ProgramCatalogue"]) { expect(Object.keys(pages)).toContain(`${name}.md`); } const index = pages["README.md"]!; expect(index).toContain( - "## Reports\n\n- [DailyRevenue](./DailyRevenue.md)\n- [ProgramEngagement](./ProgramEngagement.md)\n- [StoreTotals](./StoreTotals.md)\n", + "## Reports\n\n- [DailyRevenue](./DailyRevenue.md)\n- [ProgramCatalogue](./ProgramCatalogue.md)\n" + + "- [ProgramEngagement](./ProgramEngagement.md)\n- [StoreTotals](./StoreTotals.md)\n", ); // A report is not an entity: the entity list is the one the model without reports has. const entities = (md: string): string => { @@ -158,8 +159,9 @@ describe("FR-044 model surface: a page for every report", () => { expect(engagement).toContain( "| `daysEngaged` | `long` | no | measure | count of distinct (`WorkoutEvent.programId`, `WorkoutEvent.customerEmail`, `WorkoutEvent.weekNumber`, `WorkoutEvent.dayNumber`) |", ); + // `@default: 0`: never null, so the ratio's null rule gives way to the default. expect(engagement).toContain( - "| `avgDaysPerStarter` | `decimal` | yes | measure | `daysEngaged` / `starters`, null when the denominator is 0 |", + "| `avgDaysPerStarter` | `decimal` | no | measure | `daysEngaged` / `starters`; `0` when there is nothing to aggregate |", ); expect(engagement).toContain("| `lastActivityAt` | `timestamp` | yes | measure | max of `WorkoutEvent.occurredAt` |"); // No api unit exists for it, so nothing links to one. @@ -171,6 +173,27 @@ describe("FR-044 model surface: a page for every report", () => { expect(daily).toContain('**Row scope:** filter `{"purchasedAt":{"gte":{"now":"-P90D"}}}`'); expect(daily).toContain("| `purchasedAtDay` | `date` | yes | dimension | `Purchase.purchasedAt` truncated to day, UTC |"); expect(daily).not.toContain("API reference"); + // Neither declares @spine: no Rows line, and a row scope reads as it always did. + for (const page of [engagement, daily]) expect(page).not.toContain("**Rows:**"); + }); + + test("a @spine report says where its rows come from, and its row scope aggregates only", async () => { + const page = (await modelPages(withReporting))["ProgramCatalogue.md"]!; + const block = page.slice(page.indexOf("## Report\n")); + expect(block.slice(0, block.indexOf("\n\n| Column"))).toBe( + "## Report\n\n" + + "**From:** [Purchase](./Purchase.md)\n" + + "**View:** Not served: declares no view source\n" + + "**Rows:** one row per distinct dimension tuple among the rows of `Program`, reached by `Purchase.program`, " + + "including those no `Purchase` refers to\n" + + '**Row scope:** aggregating only filter `{"refunded":{"eq":false}}`', + ); + // A dimension over the spine entity's primary key is never null; one over another of + // its columns follows that column. + expect(page).toContain("| `programKey` | `long` | no | dimension | `Program.id` via `Purchase.program` |"); + expect(page).toContain("| `programTitle` | `string` | yes | dimension | `Program.title` via `Purchase.program` |"); + expect(page).toContain("| `revenue` | `currency` | yes | measure | sum of `Purchase.amountCents` where segment `active` |"); + expect(page).not.toContain("API reference"); }); test("the @from entity's page has a Reporting section: dimensions, measures, segments, reports", async () => { @@ -182,7 +205,8 @@ describe("FR-044 model surface: a page for every report", () => { "- `program` — `Purchase.programId`\n" + "- `programTitle` — `Program.title` via `Purchase.program`\n" + "- `purchasedAt` (time) — `Purchase.purchasedAt`; grains: day, week, month, quarter, year\n" + - "- `programCreatedAt` (time) — `Program.createdAt` via `Purchase.program`; grains: month, year\n\n" + + "- `programCreatedAt` (time) — `Program.createdAt` via `Purchase.program`; grains: month, year\n" + + "- `programKey` — `Program.id` via `Purchase.program`\n\n" + "**Measures**\n\n" + "- `refundedPurchases` — count of `Purchase.id` where filter `{\"refunded\":{\"eq\":true}}`\n" + "- `purchases` — count of `Purchase.id` where segment `active`\n" + @@ -192,6 +216,7 @@ describe("FR-044 model surface: a page for every report", () => { "- `active` — `{\"status\":{\"eq\":\"active\"}}`\n\n" + "**Reports**\n\n" + "- [DailyRevenue](./DailyRevenue.md)\n" + + "- [ProgramCatalogue](./ProgramCatalogue.md)\n" + "- [StoreTotals](./StoreTotals.md)\n", ); // Everything above the section is the page the model without reporting nodes has. @@ -255,6 +280,7 @@ describe("FR-044 api surface: one unit for a served report, none for the rest", expect(names).toContain("StoreTotals"); expect(names).not.toContain("ProgramEngagement"); expect(names).not.toContain("DailyRevenue"); + expect(names).not.toContain("ProgramCatalogue"); }); test("the unit is the row model, the list query and GET <served path>, per route surface", () => { @@ -330,6 +356,7 @@ describe("FR-044 api surface: one unit for a served report, none for the rest", expect(after[path]).toContain("StoreTotals"); expect(after[path]).not.toContain("ProgramEngagement"); expect(after[path]).not.toContain("DailyRevenue"); + expect(after[path]).not.toContain("ProgramCatalogue"); continue; } expect({ path, content: after[path] }).toEqual({ path, content }); diff --git a/server/typescript/packages/docs-site/src/builders/report-data.ts b/server/typescript/packages/docs-site/src/builders/report-data.ts index 244aa73b6..c80b08d22 100644 --- a/server/typescript/packages/docs-site/src/builders/report-data.ts +++ b/server/typescript/packages/docs-site/src/builders/report-data.ts @@ -1,8 +1,8 @@ // FR-044 on the HTML site (Table G of the Plan 3 spec): // // • an `object.report`'s page gets a "Report" section: its `@from` (linked), the view it -// is read from or why it is not served, its row scope, and one row per derived column -// from `reportShape`; +// is read from or why it is not served, where a `@spine` report's rows come from, its +// row scope, and one row per derived column from `reportShape`; // • the page of an entity that declares dimensions, measures or segments, or that a // report names as its `@from`, gets a "Reporting" section. // @@ -18,6 +18,7 @@ import { OBJECT_REPORT_ATTR_FROM, OBJECT_REPORT_ATTR_MEASURES, OBJECT_REPORT_ATTR_SEGMENT, + OBJECT_REPORT_ATTR_SPINE, OBJECT_SUBTYPE_REPORT, TYPE_DIMENSION, TYPE_MEASURE, @@ -26,7 +27,8 @@ import { describeDimension, describeMeasure, describeReportField, - describeRowScope, + describeReportRowScope, + describeReportRows, describeSegment, isMetaObject, reportFieldTypeName, @@ -47,6 +49,7 @@ export const REPORT_RENDERED_ATTRS: ReadonlySet<string> = new Set([ OBJECT_REPORT_ATTR_MEASURES, OBJECT_REPORT_ATTR_SEGMENT, OBJECT_REPORT_ATTR_FILTER, + OBJECT_REPORT_ATTR_SPINE, ]); export function isReportNode(node: MetaData): boolean { @@ -80,6 +83,8 @@ export interface ReportSection { viewName?: string | undefined; /** "Not served: …" (Table A). Absent for a served report. */ notServed?: string | undefined; + /** Where a `@spine` report's rows come from. Absent without `@spine`. */ + rowsHtml?: string | undefined; scopeHtml?: string | undefined; columns: ReportColumnRow[]; } @@ -92,12 +97,14 @@ export function buildReportSection(dn: DocNode, root: MetaRoot, g: LinkGraph, co const shape = reportShape(report, root); const from = g.byFqn(fqnOf(shape.from)); const notServed = reportNotServedReason(report); - const scope = describeRowScope(report.attr(OBJECT_REPORT_ATTR_SEGMENT), report.attr(OBJECT_REPORT_ATTR_FILTER)); + const rows = describeReportRows(shape, root); + const scope = describeReportRowScope(report); return { fromName: shape.from.name, fromHref: from ? g.relHref(dn.href, from.href) : "", viewName: notServed === undefined ? reportReadSource(report)?.physicalName : undefined, notServed, + rowsHtml: rows !== undefined ? html(rows) : undefined, scopeHtml: scope !== undefined ? html(scope) : undefined, columns: shape.fields.map((f) => ({ name: f.name, diff --git a/server/typescript/packages/docs-site/templates/object.html.mustache b/server/typescript/packages/docs-site/templates/object.html.mustache index 65c1e16c2..4f6ee3eb4 100644 --- a/server/typescript/packages/docs-site/templates/object.html.mustache +++ b/server/typescript/packages/docs-site/templates/object.html.mustache @@ -10,7 +10,7 @@ {{#objectAttrs.length}}<section id="s-attributes" class="mt-4 text-xs"><span class="opacity-50 mr-1">attributes</span>{{#objectAttrs}}<span class="badge badge-soft badge-ghost badge-xs mr-1 font-mono">@{{name}}={{value}}</span>{{/objectAttrs}}</section>{{/objectAttrs.length}} {{#report}} <section id="s-report" class="mt-6"><h2 class="text-lg font-semibold">Report</h2> -<div class="font-mono text-xs opacity-70 mt-1">from <a class="link font-mono" href="{{fromHref}}">{{fromName}}</a>{{#viewName}} · view <code>{{viewName}}</code>{{/viewName}}{{#notServed}} · <span class="badge badge-soft badge-warning badge-xs">{{notServed}}</span>{{/notServed}}{{#scopeHtml}} · row scope {{{scopeHtml}}}{{/scopeHtml}}</div> +<div class="font-mono text-xs opacity-70 mt-1">from <a class="link font-mono" href="{{fromHref}}">{{fromName}}</a>{{#viewName}} · view <code>{{viewName}}</code>{{/viewName}}{{#notServed}} · <span class="badge badge-soft badge-warning badge-xs">{{notServed}}</span>{{/notServed}}{{#rowsHtml}} · rows: {{{rowsHtml}}}{{/rowsHtml}}{{#scopeHtml}} · row scope {{{scopeHtml}}}{{/scopeHtml}}</div> <div class="overflow-x-auto"><table class="table table-xs"><thead><tr><th>Column</th><th>Type</th><th>Nullable</th><th>Role</th><th>Definition</th></tr></thead><tbody> {{#columns}}<tr><td class="font-mono">{{name}}</td><td class="opacity-60">{{type}}</td><td class="opacity-60">{{nullable}}</td><td class="opacity-60">{{role}}</td><td class="text-xs">{{{definitionHtml}}}</td></tr> {{/columns}} diff --git a/server/typescript/packages/docs-site/test/coverage.test.ts b/server/typescript/packages/docs-site/test/coverage.test.ts index 572bfcbf6..a5a4346a1 100644 --- a/server/typescript/packages/docs-site/test/coverage.test.ts +++ b/server/typescript/packages/docs-site/test/coverage.test.ts @@ -41,6 +41,9 @@ test("FR-044 reporting vocabulary is audited like any other kind: unrendered mea "dimension.attribute", "dimension.time", "measure.aggregate", "measure.ratio", "object.report", "segment.filter", ]); expect(reportingKinds.every((r) => !r.consumed)).toBe(true); - expect(rep.warnings).toContain("coverage: object.report (3) not rendered by any page"); + expect(rep.warnings).toContain("coverage: object.report (4) not rendered by any page"); + // Unrendered too: the @spine of a report and the @default of a measure. + expect(rep.attrs.find((r) => r.key === "object:@spine")).toEqual({ key: "object:@spine", count: 1, consumed: false }); + expect(rep.attrs.find((r) => r.key === "measure:@default")).toEqual({ key: "measure:@default", count: 1, consumed: false }); expect(rep.warnings.some((w) => w.includes("deferred"))).toBe(false); }); diff --git a/server/typescript/packages/docs-site/test/reporting-site.test.ts b/server/typescript/packages/docs-site/test/reporting-site.test.ts index fc3a27e89..bf28ff37b 100644 --- a/server/typescript/packages/docs-site/test/reporting-site.test.ts +++ b/server/typescript/packages/docs-site/test/reporting-site.test.ts @@ -74,7 +74,7 @@ describe("FR-044 the site renders reports", () => { test("each report has a page and a row in its package's object index", () => { const index = withSite.files[`${SHOP}/index.html`]!; - for (const name of ["StoreTotals", "ProgramEngagement", "DailyRevenue"]) { + for (const name of ["StoreTotals", "ProgramEngagement", "DailyRevenue", "ProgramCatalogue"]) { expect(Object.keys(withSite.files)).toContain(`${SHOP}/${name}.html`); expect(index).toContain(`href="${name}.html"`); } @@ -104,13 +104,31 @@ describe("FR-044 the site renders reports", () => { expect(engagement).toContain( "daysEngaged long no measure count of distinct (WorkoutEvent.programId, WorkoutEvent.customerEmail, WorkoutEvent.weekNumber, WorkoutEvent.dayNumber)", ); - expect(engagement).toContain("avgDaysPerStarter decimal yes measure daysEngaged / starters, null when the denominator is 0"); + // `@default: 0`: never null, so the ratio's null rule gives way to the default. + expect(engagement).toContain("avgDaysPerStarter decimal no measure daysEngaged / starters; 0 when there is nothing to aggregate"); expect(engagement).toContain("lastActivityAt timestamp yes measure max of WorkoutEvent.occurredAt"); const daily = text(withSite.files[`${SHOP}/DailyRevenue.html`]!); expect(daily).toContain("Not served: declares no view source"); expect(daily).toContain('row scope filter {"purchasedAt":{"gte":{"now":"-P90D"}}}'); expect(daily).toContain("purchasedAtDay date yes dimension Purchase.purchasedAt truncated to day, UTC"); + // Neither declares @spine: no rows sentence. + for (const page of [engagement, daily]) expect(page).not.toContain("one row per distinct dimension tuple"); + }); + + test("a @spine report says where its rows come from, and its row scope aggregates only", () => { + const html = withSite.files[`${SHOP}/ProgramCatalogue.html`]!; + const page = text(html); + expect(page).toContain("Not served: declares no view source"); + expect(page).toContain( + "rows: one row per distinct dimension tuple among the rows of Program, reached by Purchase.program, " + + "including those no Purchase refers to", + ); + expect(page).toContain('row scope aggregating only filter {"refunded":{"eq":false}}'); + expect(page).toContain("programKey long no dimension Program.id via Purchase.program"); + expect(page).toContain("programTitle string yes dimension Program.title via Purchase.program"); + // The Report section prints @spine in words, so it is not repeated as a raw attribute badge. + expect(html).not.toContain("@spine="); }); test("the @from entity's page has a Reporting section: dimensions, measures, segments, reports", () => { @@ -123,7 +141,11 @@ describe("FR-044 the site renders reports", () => { expect(section).toContain("purchasedAt time Purchase.purchasedAt; grains: day, week, month, quarter, year"); expect(section).toContain('refundedPurchases count of Purchase.id where filter {"refunded":{"eq":true}}'); expect(section).toContain("revenue sum of Purchase.amountCents where segment active"); + expect(section).toContain("programKey Program.id via Purchase.program"); expect(section).toContain('active {"status":{"eq":"active"}}'); + expect(text(withSite.files[`${SHOP}/WorkoutEvent.html`]!)).toContain( + "avgDaysPerStarter daysEngaged / starters; 0 when there is nothing to aggregate", + ); expect(html).toContain('<a class="link font-mono text-xs" href="DailyRevenue.html">DailyRevenue</a>'); expect(html).toContain('<a class="link font-mono text-xs" href="StoreTotals.html">StoreTotals</a>'); // An entity with no reporting nodes has no such section. @@ -165,7 +187,7 @@ describe("FR-044 the site renders reports", () => { test("a report is not an orphan: it is linked to the entity it reads from", () => { const orphans = withSite.result.anomalies.filter((a) => a.kind === "orphan").map((a) => a.subject); - for (const name of ["StoreTotals", "ProgramEngagement", "DailyRevenue"]) expect(orphans).not.toContain(name); + for (const name of ["StoreTotals", "ProgramEngagement", "DailyRevenue", "ProgramCatalogue"]) expect(orphans).not.toContain(name); // The entity shows its reports among what references it. expect(text(withSite.files[`${SHOP}/WorkoutEvent.html`]!)).toContain("referenced by ProgramEngagement"); }); diff --git a/server/typescript/packages/metadata/src/core/reporting/report-describe.ts b/server/typescript/packages/metadata/src/core/reporting/report-describe.ts index 965a4d731..516f64f90 100644 --- a/server/typescript/packages/metadata/src/core/reporting/report-describe.ts +++ b/server/typescript/packages/metadata/src/core/reporting/report-describe.ts @@ -13,13 +13,16 @@ // Browser-safe: no Node-only import. import type { MetaData } from "../../shared/meta-data.js"; +import type { MetaRoot } from "../../shared/meta-root.js"; import { TYPE_DIMENSION, TYPE_MEASURE, TYPE_SEGMENT } from "../../shared/base-types.js"; import { SOURCE_KIND_VIEW } from "../../persistence/source/source-constants.js"; import type { MetaObject } from "../object/meta-object.js"; +import { OBJECT_REPORT_ATTR_FILTER, OBJECT_REPORT_ATTR_SEGMENT } from "../object/object-constants.js"; import { DIMENSION_SUBTYPE_TIME, MEASURE_SUBTYPE_RATIO, REPORTING_ATTR_AGG, + REPORTING_ATTR_DEFAULT, REPORTING_ATTR_DENOMINATOR, REPORTING_ATTR_DISTINCT, REPORTING_ATTR_FILTER, @@ -29,8 +32,9 @@ import { REPORTING_ATTR_SEGMENT, REPORTING_ATTR_VIA, } from "./reporting-constants.js"; +import { reportSpine } from "./report-accessors.js"; import { reportReadSource } from "./report-read-model.js"; -import type { ReportField } from "./report-shape.js"; +import { reportSpineEntity, type ReportField, type ReportShape } from "./report-shape.js"; /** Inline code. A backtick cannot sit inside a single-backtick span, so it is dropped. */ function tick(text: string): string { @@ -88,18 +92,33 @@ export function describeDimensionColumn(dim: MetaData, grain?: string): string { return `${column}${joiner} truncated to ${grain}, UTC`; } -/** A measure in words: the aggregate and its row scope, or the ratio and its null rule. */ +/** + * "; `0` when there is nothing to aggregate": the clause a measure's `@default` adds (R9), or + * "" when it declares none. The declared integer is printed as written. + */ +function defaultClause(measure: MetaData): string { + const declared = measure.attr(REPORTING_ATTR_DEFAULT); + return typeof declared === "number" ? `; ${tick(String(declared))} when there is nothing to aggregate` : ""; +} + +/** + * A measure in words: the aggregate and its row scope, or the ratio and its null rule. A + * `@default` ends either with the value it reads when there is nothing to aggregate; a + * defaulted ratio is never null, so its null rule gives way to that clause. + */ export function describeMeasure(measure: MetaData): string { + const fallback = defaultClause(measure); if (measure.subType === MEASURE_SUBTYPE_RATIO) { const numerator = measure.attr(REPORTING_ATTR_NUMERATOR); const denominator = measure.attr(REPORTING_ATTR_DENOMINATOR); - return `${tick(String(numerator ?? ""))} / ${tick(String(denominator ?? ""))}, null when the denominator is 0`; + const quotient = `${tick(String(numerator ?? ""))} / ${tick(String(denominator ?? ""))}`; + return fallback !== "" ? `${quotient}${fallback}` : `${quotient}, null when the denominator is 0`; } const columns = stringList(measure.attr(REPORTING_ATTR_OF)).map(tick); const of = columns.length === 1 ? columns[0]! : `(${columns.join(", ")})`; const distinct = measure.attr(REPORTING_ATTR_DISTINCT) === true ? "distinct " : ""; const scope = describeRowScope(measure.attr(REPORTING_ATTR_SEGMENT), measure.attr(REPORTING_ATTR_FILTER)); - return `${String(measure.attr(REPORTING_ATTR_AGG) ?? "")} of ${distinct}${of}${scope !== undefined ? ` where ${scope}` : ""}`; + return `${String(measure.attr(REPORTING_ATTR_AGG) ?? "")} of ${distinct}${of}${scope !== undefined ? ` where ${scope}` : ""}${fallback}`; } /** A segment in words: its filter. */ @@ -113,6 +132,35 @@ export function describeReportField(field: ReportField): string { return field.measure !== undefined ? describeMeasure(field.measure) : ""; } +/** + * Where a report with `@spine` takes its rows (R8): "one row per distinct dimension tuple + * among the rows of `Program`, reached by `Purchase.program`, including those no `Purchase` + * refers to". Undefined for a report without `@spine`, whose rows are the groups of its + * `@from` rows and need no sentence. The spine is printed as declared. + */ +export function describeReportRows(shape: ReportShape, root: MetaRoot): string | undefined { + const spine = reportSpine(shape.report); + const entity = reportSpineEntity(shape.report, shape.from, root); + if (spine === undefined || entity === undefined) return undefined; + return ( + `one row per distinct dimension tuple among the rows of ${tick(entity.name)}, reached by ${tick(spine)}, ` + + `including those no ${tick(shape.from.name)} refers to` + ); +} + +/** + * A report's row scope ({@link describeRowScope} over its `@segment` and `@filter`), or + * undefined when it declares neither. Under `@spine` the scope chooses which rows of `@from` + * are aggregated and never removes a row of the report, so it reads "aggregating only + * segment `paid`". + */ +export function describeReportRowScope(report: MetaObject): string | undefined { + // ADR-0039: resolving. + const scope = describeRowScope(report.attr(OBJECT_REPORT_ATTR_SEGMENT), report.attr(OBJECT_REPORT_ATTR_FILTER)); + if (scope === undefined) return undefined; + return reportSpine(report) !== undefined ? `aggregating only ${scope}` : scope; +} + /** The neutral logical type of a derived column: its Table B subtype, `[]` for an array. */ export function reportFieldTypeName(field: ReportField): string { // ADR-0039: resolvedIsArray() is the resolving read of the native array flag. @@ -137,8 +185,9 @@ const DIMENSION_ATTRS = [REPORTING_ATTR_OF, REPORTING_ATTR_VIA] as const; const TIME_DIMENSION_ATTRS = [...DIMENSION_ATTRS, REPORTING_ATTR_GRAINS] as const; const AGGREGATE_ATTRS = [ REPORTING_ATTR_AGG, REPORTING_ATTR_OF, REPORTING_ATTR_DISTINCT, REPORTING_ATTR_SEGMENT, REPORTING_ATTR_FILTER, + REPORTING_ATTR_DEFAULT, ] as const; -const RATIO_ATTRS = [REPORTING_ATTR_NUMERATOR, REPORTING_ATTR_DENOMINATOR] as const; +const RATIO_ATTRS = [REPORTING_ATTR_NUMERATOR, REPORTING_ATTR_DENOMINATOR, REPORTING_ATTR_DEFAULT] as const; const SEGMENT_ATTRS = [REPORTING_ATTR_FILTER] as const; /** diff --git a/server/typescript/packages/metadata/src/core/reporting/report-shape.ts b/server/typescript/packages/metadata/src/core/reporting/report-shape.ts index 582adcb6c..7935afd36 100644 --- a/server/typescript/packages/metadata/src/core/reporting/report-shape.ts +++ b/server/typescript/packages/metadata/src/core/reporting/report-shape.ts @@ -22,6 +22,9 @@ import { FIELD_SUBTYPE_LONG, FIELD_SUBTYPE_TIMESTAMP, } from "../field/field-constants.js"; +import { TYPE_IDENTITY, TYPE_RELATIONSHIP } from "../../shared/base-types.js"; +import { IDENTITY_REFERENCE_ATTR_REFERENCES, IDENTITY_SUBTYPE_REFERENCE } from "../identity/identity-constants.js"; +import { RELATIONSHIP_ATTR_OBJECT_REF } from "../relationship/relationship-constants.js"; import { MetaDimension } from "./meta-dimension.js"; import { MetaMeasure } from "./meta-measure.js"; import { identityEffectiveFields } from "../identity/validate-identity-passthrough.js"; @@ -185,6 +188,32 @@ export function reportSpineHops(report: MetaObject, from: MetaObject, root: Meta return hops; } +/** + * The spine entity: the object at the end of a report's `@spine`, whose rows are the + * report's rows. Walked hop by hop from `from` as the loader's rule R8 walks it: each hop is + * a `relationship.*` (its `@objectRef`) or an `identity.reference` (its `@references`) of the + * entity reached so far, and its target resolves in that entity's package. Undefined when the + * report declares no `@spine`; throws, as {@link reportSpineHops} does, when one does not + * resolve (a report that passed `validateReporting` always resolves). + */ +export function reportSpineEntity(report: MetaObject, from: MetaObject, root: MetaRoot): MetaObject | undefined { + const hops = reportSpineHops(report, from, root); + if (hops === undefined) return undefined; + let current: MetaObject = from; + for (const hop of hops) { + // ADR-0039: resolving children(), so a relationship or reference inherited through + // extends is a hop, as the loader's walk finds it. + const node = + current.children().find((c) => c.type === TYPE_RELATIONSHIP && c.name === hop) ?? + current.children().find((c) => c.type === TYPE_IDENTITY && c.subType === IDENTITY_SUBTYPE_REFERENCE && c.name === hop); + const ref = node?.attr(node.type === TYPE_IDENTITY ? IDENTITY_REFERENCE_ATTR_REFERENCES : RELATIONSHIP_ATTR_OBJECT_REF); + const target = typeof ref === "string" ? resolveObjectRef(root, ref, packageOfKey(current.resolutionKey())).node : undefined; + if (!isMetaObject(target)) throw unresolved(report.name, `@spine '${reportSpine(report) ?? ""}' hop '${hop}'`); + current = target; + } + return current; +} + /** True when `field` is one of the `@fields` of `entity`'s `identity.primary`. */ function isPrimaryKeyField(entity: MetaObject, field: MetaField): boolean { // ADR-0039: resolving — an identity.primary (and its @fields) inherited from an diff --git a/server/typescript/packages/metadata/src/index.ts b/server/typescript/packages/metadata/src/index.ts index 63d4f39bd..afcf69d0d 100644 --- a/server/typescript/packages/metadata/src/index.ts +++ b/server/typescript/packages/metadata/src/index.ts @@ -59,6 +59,7 @@ export { export { measureDerivedSubType, reportShape, + reportSpineEntity, reportSpineHops, reportingMemberOwner, reportingViaHops, @@ -74,6 +75,8 @@ export { describeFilter, describeMeasure, describeReportField, + describeReportRowScope, + describeReportRows, describeRowScope, describeSegment, reportFieldTypeName, diff --git a/server/typescript/packages/metadata/test/report-describe.test.ts b/server/typescript/packages/metadata/test/report-describe.test.ts index ac81807d3..acf16c1d2 100644 --- a/server/typescript/packages/metadata/test/report-describe.test.ts +++ b/server/typescript/packages/metadata/test/report-describe.test.ts @@ -10,6 +10,8 @@ import { describeFilter, describeMeasure, describeReportField, + describeReportRowScope, + describeReportRows, describeRowScope, describeSegment, reportFieldTypeName, @@ -58,6 +60,8 @@ const MODEL = { { "measure.aggregate": { name: "paidLiveCents", "@agg": "sum", "@of": "Invoice.amountCents", "@segment": "paid", "@filter": { voided: false } } }, { "measure.ratio": { name: "paidShare", "@numerator": "paidInvoices", "@denominator": "invoices" } }, { "segment.filter": { name: "paid", "@filter": { status: "paid" } } }, + { "measure.aggregate": { name: "liveCentsOrZero", "@agg": "sum", "@of": "Invoice.amountCents", "@filter": { voided: false }, "@default": 0 } }, + { "measure.ratio": { name: "paidShareOrNone", "@numerator": "paidInvoices", "@denominator": "invoices", "@default": -1 } }, ] } }, { "object.report": { name: "ByMonth", "@from": "Invoice", "@dimensions": ["issuedOn:month", "customerJoinedAt:month", "region", "tags"], @@ -67,6 +71,11 @@ const MODEL = { children: [view("mv_invoices", "materializedView")] } }, { "object.report": { name: "AbstractReport", abstract: true, "@from": "Invoice", "@measures": ["invoices"], children: [view("v_abstract")] } }, + { "object.report": { name: "PaidTotals", "@from": "Invoice", "@measures": ["invoices"], "@segment": "paid" } }, + { "object.report": { name: "PaidByRegion", "@from": "Invoice", "@spine": "Invoice.customer", + "@dimensions": ["region"], "@measures": ["invoices", "liveCentsOrZero"], "@segment": "paid" } }, + { "object.report": { name: "CustomerRoster", "@from": "Invoice", "@spine": "Invoice.customer", + "@dimensions": ["region"], "@measures": ["invoices"] } }, ], }, }; @@ -166,6 +175,48 @@ describe("describing a report's columns", () => { }); }); +describe("describing a measure @default and a report @spine", () => { + test("a defaulted aggregate keeps its sentence and ends with the declared integer", async () => { + const root = await load(); + expect(describeMeasure(member(root, "measure", "liveCentsOrZero"))) + .toBe('sum of `Invoice.amountCents` where filter `{"voided":{"eq":false}}`; `0` when there is nothing to aggregate'); + }); + + test("a defaulted ratio is never null, so its null rule gives way to the default", async () => { + const root = await load(); + expect(describeMeasure(member(root, "measure", "paidShareOrNone"))) + .toBe("`paidInvoices` / `invoices`; `-1` when there is nothing to aggregate"); + }); + + test("a report with @spine: its rows come from the spine entity, and its row scope aggregates only", async () => { + const root = await load(); + const report = object(root, "PaidByRegion"); + expect(describeReportRows(reportShape(report, root), root)).toBe( + "one row per distinct dimension tuple among the rows of `Customer`, reached by `Invoice.customer`, " + + "including those no `Invoice` refers to", + ); + expect(describeReportRowScope(report)).toBe("aggregating only segment `paid`"); + // The column table describes a defaulted measure as the entity's page does. + const column = reportShape(report, root).fields.find((f) => f.name === "liveCentsOrZero")!; + expect(describeReportField(column)).toBe(describeMeasure(member(root, "measure", "liveCentsOrZero"))); + }); + + test("a @spine report with no row scope has none to describe", async () => { + const root = await load(); + const report = object(root, "CustomerRoster"); + expect(describeReportRows(reportShape(report, root), root)).toContain("among the rows of `Customer`"); + expect(describeReportRowScope(report)).toBeUndefined(); + }); + + test("a report without @spine: no rows sentence, and the row scope reads as it always did", async () => { + const root = await load(); + expect(describeReportRows(reportShape(object(root, "ByMonth"), root), root)).toBeUndefined(); + expect(describeReportRows(reportShape(object(root, "PaidTotals"), root), root)).toBeUndefined(); + expect(describeReportRowScope(object(root, "PaidTotals"))).toBe("segment `paid`"); + expect(describeReportRowScope(object(root, "ByMonth"))).toBeUndefined(); + }); +}); + describe("why a report is not served (Plan 3 Table A)", () => { test("a view-backed report is served", async () => { const root = await load(); @@ -187,8 +238,8 @@ describe("the attrs a describer reads", () => { const root = await load(); expect(reportingDescribedAttrs(member(root, "dimension", "status"))).toEqual(["of", "via"]); expect(reportingDescribedAttrs(member(root, "dimension", "issuedOn"))).toEqual(["of", "via", "grains"]); - expect(reportingDescribedAttrs(member(root, "measure", "invoices"))).toEqual(["agg", "of", "distinct", "segment", "filter"]); - expect(reportingDescribedAttrs(member(root, "measure", "paidShare"))).toEqual(["numerator", "denominator"]); + expect(reportingDescribedAttrs(member(root, "measure", "invoices"))).toEqual(["agg", "of", "distinct", "segment", "filter", "default"]); + expect(reportingDescribedAttrs(member(root, "measure", "paidShare"))).toEqual(["numerator", "denominator", "default"]); expect(reportingDescribedAttrs(member(root, "segment", "paid"))).toEqual(["filter"]); expect(reportingDescribedAttrs(object(root, "Invoice"))).toEqual([]); }); diff --git a/server/typescript/packages/metadata/test/report-shape.test.ts b/server/typescript/packages/metadata/test/report-shape.test.ts index f24749094..0e415804e 100644 --- a/server/typescript/packages/metadata/test/report-shape.test.ts +++ b/server/typescript/packages/metadata/test/report-shape.test.ts @@ -13,6 +13,7 @@ import { measureDerivedSubType, reportMeasureItemName, reportShape, + reportSpineEntity, reportSpineHops, type MetaObject, type MetaRoot, @@ -476,6 +477,36 @@ describe("reportSpineHops", () => { }); }); +describe("reportSpineEntity", () => { + test("undefined without @spine; the entity at the end of the path with one", async () => { + const root = await salesModel(); + const from = report(root, "Purchase"); + expect(reportSpineEntity(report(root, "PlainSales"), from, root)).toBeUndefined(); + expect(reportSpineEntity(report(root, "SpineSales"), from, root)).toBe(report(root, "Program")); + expect(reportSpineEntity(report(root, "CatalogSales"), from, root)).toBe(report(root, "Catalog")); + }); + + test("an identity.reference is a hop as well as a relationship (a tree built in code)", async () => { + const root = await salesModel(); + const byReference = withAttr(report(root, "SpineSales"), OBJECT_REPORT_ATTR_SPINE, "Purchase.fkProgram"); + expect(reportSpineEntity(byReference, report(root, "Purchase"), root)).toBe(report(root, "Program")); + }); + + test("a hop inherited from an abstract base is followed (ADR-0039, the inherited fixture)", async () => { + const fixture = join(REPO_ROOT, "fixtures", "conformance", "reporting-spine-inherited", "input", "meta.shop.json"); + const { root } = await loadUris([pathToFileURL(fixture).href]); + expect(reportSpineEntity(report(root, "ProgramMinutes"), report(root, "WorkoutEvent"), root)?.name).toBe("Program"); + }); + + test("a hop that names nothing throws, naming the report, the spine and the hop (a tree built in code)", async () => { + const root = await salesModel(); + const r = withAttr(report(root, "SpineSales"), OBJECT_REPORT_ATTR_SPINE, "Purchase.nope"); + expect(() => reportSpineEntity(r, report(root, "Purchase"), root)).toThrow( + "report 'SpineSales': @spine 'Purchase.nope' hop 'nope' does not resolve.", + ); + }); +}); + describe("measureDerivedSubType (Table B for one measure, listed in a report or not)", () => { test("each aggregate and the ratio", async () => { const root = await salesModel(); From 27da71f02530787ba64be0258835b0b3f2492b1e Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 20:41:23 -0400 Subject: [PATCH 11/18] test(api-contract): a @spine report with a defaulted measure in the report/ sub-corpus --- .../api-contract-conformance/report/README.md | 50 ++++-- .../api-contract-conformance/report/meta.json | 33 ++++ .../filter-on-defaulted-measure.yaml | 33 ++++ .../report/scenarios/list-spine.yaml | 26 +++ .../scenarios/sort-on-defaulted-measure.yaml | 23 +++ .../report/schema.postgres.sql | 27 +++ .../api-contract-conformance/report/seed.json | 15 ++ .../api-contract-report-generated-server.ts | 157 ++++++++++++++---- .../src/api-contract-report-sqlite-server.ts | 60 +++++-- .../test/api-contract-report-corpus.test.ts | 53 ++++-- .../test/api-contract-report-sqlite.test.ts | 3 + .../test/api-contract-report.test.ts | 1 + 12 files changed, 401 insertions(+), 80 deletions(-) create mode 100644 fixtures/api-contract-conformance/report/scenarios/filter-on-defaulted-measure.yaml create mode 100644 fixtures/api-contract-conformance/report/scenarios/list-spine.yaml create mode 100644 fixtures/api-contract-conformance/report/scenarios/sort-on-defaulted-measure.yaml diff --git a/fixtures/api-contract-conformance/report/README.md b/fixtures/api-contract-conformance/report/README.md index 9f40ecade..3e220ad34 100644 --- a/fixtures/api-contract-conformance/report/README.md +++ b/fixtures/api-contract-conformance/report/README.md @@ -2,7 +2,8 @@ Cross-port REST contract for an **`object.report` that declares a read-only view** (`source.rdb @kind:view @view:v_invoice_status_totals`), over the writable `Invoice` -entity it reads from. +entity it reads from, and for a **`@spine` report** (`ProductRevenue`) whose rows are the +`Product`s, including one with no `Sale`. ## What this gates @@ -32,6 +33,15 @@ object uses (`InvoicesByMonth` is `/api/invoices_by_months`). `date`, spelled `YYYY-MM-DD` (`issuedOnMonth`). - **A sum scoped by a segment is nullable**: the key is present and its value is `null` for a group with no row in the segment. +- **A `@spine` report has a row for every spine row, facts or not.** `ProductRevenue` + declares `@spine: "Sale.fkProduct"`, so `Cinder`, a product with no sale, is listed and + counted by `withCount`. On that row a count is `0`, a sum without `@default` + (`revenueCents`) is `null` with the key present, and the sum with `@default: 0` + (`revenueOrZero`) is `0`. `?filter[sales][gt]=0` removes the empty row at request time. +- **A defaulted measure filters and sorts as its default.** `?filter[revenueOrZero][eq]=0` + matches the empty row, and `?sort=revenueOrZero:asc` puts it first on every engine. The + corpus never sorts a measure without a `@default` across a null row: where a null sorts is + the engine's choice (Postgres last ascending, SQLite and MySQL first). - **No grid or form is generated for a report in any port.** The read route and its row type are the surface in every port; TypeScript, the one port with a client tier, also generates the list hook. The corpus gates the route, not the hook. @@ -52,7 +62,8 @@ object uses (`InvoicesByMonth` is `/api/invoices_by_months`). ## The model -One fact entity and four reports, three served and one sourceless on purpose. +Three entities (`Invoice`; `Product` and its child `Sale`) and five reports, four served +and one sourceless on purpose. | Report | Route | View | Derived fields | |---|---|---|---| @@ -60,6 +71,11 @@ One fact entity and four reports, three served and one sourceless on purpose. | `InvoicesByMonth` | `/api/invoices_by_months` | `v_invoices_by_month` | `issuedOnMonth`, `invoices`, `totalCents` | | `InvoiceTotals` | `/api/invoice_totals` | `v_invoice_totals` | `invoices`, `totalCents`, `paidShare` | | `InvoiceDays` | none | none | `issuedOnDay`, `invoices` | +| `ProductRevenue` | `/api/product_revenues` | `v_product_revenue` | `productId`, `productName`, `sales`, `revenueCents`, `revenueOrZero` | + +`ProductRevenue` is `@from: Sale` with `@spine: "Sale.fkProduct"`: its view reads FROM +`products` and LEFT JOINs `sales`, so its spine key `productId` and `productName` are never +null, and neither is `revenueOrZero` (`@default: 0`, a `COALESCE`). `InvoiceDays` declares no `source.rdb`. **A report is served only when it declares a view**, so a sourceless one must stay inert in every generator and mount nothing. It @@ -71,9 +87,9 @@ sits in the model so that a port which serves every report it finds fails this c ``` report/ ├── README.md # this file -├── meta.json # Invoice + four object.report nodes (three served, one sourceless) -├── seed.json # `invoices` (the base table) and `reports` (what the views return) -├── schema.postgres.sql # TypeScript-produced: the table and the three views +├── meta.json # Invoice, Product, Sale + five object.report nodes (four served, one sourceless) +├── seed.json # the base tables (`invoices`, `products`, `sales`) and `reports` (what the views return) +├── schema.postgres.sql # TypeScript-produced: the three tables and the four views └── scenarios/ ├── list.yaml # GET list, dimension + segment-scoped sum ├── list-time-grain.yaml # a time dimension at a grain @@ -87,18 +103,24 @@ report/ ├── sort-invalid.yaml # 400 envelope, naming the field ├── pagination.yaml # limit / offset / withCount count groups ├── write-verbs-405.yaml # POST on the collection -> 405 + envelope - └── no-item-route.yaml # GET/PATCH/PUT/DELETE on /{id} -> 404, no body assertion + ├── no-item-route.yaml # GET/PATCH/PUT/DELETE on /{id} -> 404, no body assertion + ├── list-spine.yaml # @spine: the product with no sale has a row; withCount counts it + ├── filter-on-defaulted-measure.yaml # eq=0 on a defaulted sum, isNull on the plain sum, gt=0 on the count + └── sort-on-defaulted-measure.yaml # a defaulted sum sorts its empty row as 0, both directions ``` ### Why `seed.json` has two halves -`invoices` is the base table. The full-stack lanes (TypeScript, C#) insert it and let -the real view derive the report rows from it. `reports` is what the three views return -for those invoices, and it is what the **seam lanes** (Java, Kotlin, Python) serve: -their in-memory repository or H2 table stands in for the view behind the consumer seam. -A TypeScript test holds the two halves together, so the seam lanes cannot drift from -what the SQL returns. `paidShare` is a string in the seed so a seam lane can build its -own decimal from it without a float in between. +Every top-level key but `reports` is a **base table**, named by its table name and listed +**parents first** (`products` before `sales`, which references it): a lane inserts the +keys in file order and every foreign key holds. The full-stack lanes (TypeScript, C#) +insert them and let the real views derive the report rows. `reports` is what the four +views return for those rows, and it is what the **seam lanes** (Java, Kotlin, Python) +serve: their in-memory repository or H2 table stands in for the view behind the consumer +seam. A TypeScript test holds the two halves together, so the seam lanes cannot drift +from what the SQL returns, and another holds the base tables to the model and to that +order. `paidShare` is a string in the seed so a seam lane can build its own decimal from +it without a float in between. ### Why the view SQL is a committed artifact @@ -120,7 +142,7 @@ hand-rolled reference server would answer every scenario by construction. | Port | Generated lane | Note | |---|---|---| -| TypeScript | wired | `server/typescript/packages/integration-tests/test/api-contract-report.test.ts` (13 scenarios + a seed-vs-view check). Full stack: generated Fastify routes over the real views on Testcontainers Postgres | +| TypeScript | wired | `server/typescript/packages/integration-tests/test/api-contract-report.test.ts` (16 scenarios + a seed-vs-view check). Full stack: generated Fastify routes over the real views on Testcontainers Postgres. `api-contract-report-sqlite.test.ts` runs the same 16 on SQLite | | C# | wired | `server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs`. Full stack: generated routes and EF Core over `schema.postgres.sql` on Testcontainers Postgres | | Java | wired | `server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java` (13 scenarios + a scenario-count check). Generated controllers behind an in-memory repository seeded from `reports` | | Kotlin | wired | `server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt` (13 scenarios, the count check, and a check that the sourceless report generated nothing). Generated controllers over the generated Exposed table objects, seeded from `reports` | diff --git a/fixtures/api-contract-conformance/report/meta.json b/fixtures/api-contract-conformance/report/meta.json index 66bc199b2..10e48b85e 100644 --- a/fixtures/api-contract-conformance/report/meta.json +++ b/fixtures/api-contract-conformance/report/meta.json @@ -47,6 +47,39 @@ "@from": "Invoice", "@dimensions": ["issuedOn:day"], "@measures": ["invoices"] + }}, + { "object.entity": { + "name": "Product", + "children": [ + { "source.rdb": { "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name", "@required": true } }, + { "identity.primary": { "name": "pk", "@fields": "id", "@generation": "increment" } } + ] + }}, + { "object.entity": { + "name": "Sale", + "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "productId", "@required": true } }, + { "field.long": { "name": "amountCents", "@required": true } }, + { "identity.primary": { "name": "pk", "@fields": "id", "@generation": "increment" } }, + { "identity.reference": { "name": "fkProduct", "@fields": "productId", "@references": "Product" } }, + { "dimension.attribute": { "name": "productId", "@of": "Product.id", "@via": "Sale.fkProduct" } }, + { "dimension.attribute": { "name": "productName", "@of": "Product.name", "@via": "Sale.fkProduct" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } }, + { "measure.aggregate": { "name": "revenueCents", "@agg": "sum", "@of": "Sale.amountCents" } }, + { "measure.aggregate": { "name": "revenueOrZero", "@agg": "sum", "@of": "Sale.amountCents", "@default": 0 } } + ] + }}, + { "object.report": { + "name": "ProductRevenue", + "@from": "Sale", + "@spine": "Sale.fkProduct", + "@dimensions": ["productId", "productName"], + "@measures": ["sales", "revenueCents", "revenueOrZero"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_product_revenue" } } ] }} ] } diff --git a/fixtures/api-contract-conformance/report/scenarios/filter-on-defaulted-measure.yaml b/fixtures/api-contract-conformance/report/scenarios/filter-on-defaulted-measure.yaml new file mode 100644 index 000000000..9eab3028c --- /dev/null +++ b/fixtures/api-contract-conformance/report/scenarios/filter-on-defaulted-measure.yaml @@ -0,0 +1,33 @@ +name: report-filter-on-defaulted-measure +description: > + A filter on a @spine report's measures. r1: revenueOrZero carries @default 0, so the + default is the value and eq=0 matches the product with no sale. r2: revenueCents has no + @default, so the same row is null there and isNull=true matches it. r3: a filter on the + count removes the empty row at request time, so one report serves the page that shows + it and the page that does not. +requests: + - id: r1 + method: GET + path: /api/product_revenues?filter[revenueOrZero][eq]=0 + expect: + status: 200 + body: + equals: + - { productId: 3, productName: "Cinder", sales: 0, revenueCents: null, revenueOrZero: 0 } + - id: r2 + method: GET + path: /api/product_revenues?filter[revenueCents][isNull]=true + expect: + status: 200 + body: + equals: + - { productId: 3, productName: "Cinder", sales: 0, revenueCents: null, revenueOrZero: 0 } + - id: r3 + method: GET + path: /api/product_revenues?filter[sales][gt]=0&sort=productId:asc + expect: + status: 200 + body: + equals: + - { productId: 1, productName: "Atlas", sales: 2, revenueCents: 30000, revenueOrZero: 30000 } + - { productId: 2, productName: "Beacon", sales: 1, revenueCents: 4500, revenueOrZero: 4500 } diff --git a/fixtures/api-contract-conformance/report/scenarios/list-spine.yaml b/fixtures/api-contract-conformance/report/scenarios/list-spine.yaml new file mode 100644 index 000000000..1e2fdeb24 --- /dev/null +++ b/fixtures/api-contract-conformance/report/scenarios/list-spine.yaml @@ -0,0 +1,26 @@ +name: report-list-spine +description: > + GET /api/product_revenues — ProductRevenue declares @spine Sale.fkProduct, so its rows + are the products, not the sales: Cinder has no sale and still has a row. On that row a + count is 0, a sum without a @default (revenueCents) is null with the key present, and + the sum with @default 0 (revenueOrZero) is 0. withCount=1 counts the empty row too. +requests: + - id: r1 + method: GET + path: /api/product_revenues?sort=productId:asc + expect: + status: 200 + body: + equals: + - { productId: 1, productName: "Atlas", sales: 2, revenueCents: 30000, revenueOrZero: 30000 } + - { productId: 2, productName: "Beacon", sales: 1, revenueCents: 4500, revenueOrZero: 4500 } + - { productId: 3, productName: "Cinder", sales: 0, revenueCents: null, revenueOrZero: 0 } + - id: r2 + method: GET + path: /api/product_revenues?sort=productId:asc&withCount=1 + expect: + status: 200 + body: + envelope: true + rowsLength: 3 + total: 3 diff --git a/fixtures/api-contract-conformance/report/scenarios/sort-on-defaulted-measure.yaml b/fixtures/api-contract-conformance/report/scenarios/sort-on-defaulted-measure.yaml new file mode 100644 index 000000000..a116c40d6 --- /dev/null +++ b/fixtures/api-contract-conformance/report/scenarios/sort-on-defaulted-measure.yaml @@ -0,0 +1,23 @@ +name: report-sort-on-defaulted-measure +description: > + A sort on a defaulted measure is deterministic on every engine: the product with no sale + sorts as its default, 0, never as a null whose place the engine chooses. The corpus never + sorts revenueCents (no @default) across that row, because Postgres puts a null last + ascending and SQLite and MySQL put it first. +requests: + - id: asc + method: GET + path: /api/product_revenues?sort=revenueOrZero:asc&limit=1 + expect: + status: 200 + body: + equals: + - { productId: 3, productName: "Cinder", sales: 0, revenueCents: null, revenueOrZero: 0 } + - id: desc + method: GET + path: /api/product_revenues?sort=revenueOrZero:desc&limit=1 + expect: + status: 200 + body: + equals: + - { productId: 1, productName: "Atlas", sales: 2, revenueCents: 30000, revenueOrZero: 30000 } diff --git a/fixtures/api-contract-conformance/report/schema.postgres.sql b/fixtures/api-contract-conformance/report/schema.postgres.sql index c1f94e6e8..7d67f523d 100644 --- a/fixtures/api-contract-conformance/report/schema.postgres.sql +++ b/fixtures/api-contract-conformance/report/schema.postgres.sql @@ -13,6 +13,21 @@ CREATE TABLE "invoices" ( CONSTRAINT "invoices_status_chk" CHECK ("status" IN ('OPEN', 'PAID', 'VOID')) ); +CREATE TABLE "products" ( + "id" BIGINT GENERATED BY DEFAULT AS IDENTITY NOT NULL, + "name" TEXT NOT NULL, + CONSTRAINT "products_pkey" PRIMARY KEY ("id") +); + +CREATE TABLE "sales" ( + "id" BIGINT GENERATED BY DEFAULT AS IDENTITY NOT NULL, + "productId" BIGINT NOT NULL, + "amountCents" BIGINT NOT NULL, + CONSTRAINT "sales_pkey" PRIMARY KEY ("id") +); + +ALTER TABLE "sales" ADD CONSTRAINT "sales_productId_fk" FOREIGN KEY ("productId") REFERENCES "products" ("id"); + CREATE VIEW "v_invoice_status_totals" AS SELECT i."status" AS "status", @@ -39,3 +54,15 @@ CREATE VIEW "v_invoice_totals" AS CAST(COUNT(i."id") FILTER (WHERE i."status" = 'PAID') AS NUMERIC) / NULLIF(COUNT(i."id"), 0) AS "paidShare" FROM "invoices" i; COMMENT ON VIEW "v_invoice_totals" IS 'metaobjects:v1:sha256:d99e43a93181f09bf2af44297ae4bf3f0f52dc5fc1dd73279b2ab76fab12a5fb'; + +CREATE VIEW "v_product_revenue" AS + SELECT + p."id" AS "productId", + p."name" AS "productName", + COUNT(s."id") AS "sales", + CAST(SUM(s."amountCents") AS BIGINT) AS "revenueCents", + COALESCE(CAST(SUM(s."amountCents") AS BIGINT), 0) AS "revenueOrZero" + FROM "products" p + LEFT OUTER JOIN "sales" s ON p."id" = s."productId" + GROUP BY p."id", p."name"; +COMMENT ON VIEW "v_product_revenue" IS 'metaobjects:v1:sha256:4591ac33c76e8bd7078de8aa46d3f790deec87c7653d3d52d1b8ce93ee740949'; diff --git a/fixtures/api-contract-conformance/report/seed.json b/fixtures/api-contract-conformance/report/seed.json index 9d389baee..a2c5e8a53 100644 --- a/fixtures/api-contract-conformance/report/seed.json +++ b/fixtures/api-contract-conformance/report/seed.json @@ -6,6 +6,16 @@ { "id": 4, "reference": "INV-1004", "status": "VOID", "amountCents": 0, "issuedOn": "2026-05-31" }, { "id": 5, "reference": "INV-1005", "status": "PAID", "amountCents": 60000, "issuedOn": "2026-06-01" } ], + "products": [ + { "id": 1, "name": "Atlas" }, + { "id": 2, "name": "Beacon" }, + { "id": 3, "name": "Cinder" } + ], + "sales": [ + { "id": 1, "productId": 1, "amountCents": 12000 }, + { "id": 2, "productId": 1, "amountCents": 18000 }, + { "id": 3, "productId": 2, "amountCents": 4500 } + ], "reports": { "InvoiceStatusTotals": [ { "status": "OPEN", "invoices": 2, "totalCents": 130500, "paidCents": null }, @@ -19,6 +29,11 @@ ], "InvoiceTotals": [ { "invoices": 5, "totalCents": 315500, "paidShare": "0.4" } + ], + "ProductRevenue": [ + { "productId": 1, "productName": "Atlas", "sales": 2, "revenueCents": 30000, "revenueOrZero": 30000 }, + { "productId": 2, "productName": "Beacon", "sales": 1, "revenueCents": 4500, "revenueOrZero": 4500 }, + { "productId": 3, "productName": "Cinder", "sales": 0, "revenueCents": null, "revenueOrZero": 0 } ] } } diff --git a/server/typescript/packages/integration-tests/src/api-contract-report-generated-server.ts b/server/typescript/packages/integration-tests/src/api-contract-report-generated-server.ts index a5b5fd307..112cb6a7b 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-report-generated-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-report-generated-server.ts @@ -1,15 +1,15 @@ // api-contract-report-generated-server.ts — boots the GENERATED report routes (the // deployed artifact) over HTTP and drives them against the FR-044 report corpus. // -// The model is one writable `Invoice` entity and four `object.report` nodes; three -// declare `source.rdb @kind:view` and are served, `InvoiceDays` declares no source and -// must emit nothing. It: +// The model is three writable entities (`Invoice`; `Product` and `Sale`, a parent and its +// child) and five `object.report` nodes; four declare `source.rdb @kind:view` and are +// served, `InvoiceDays` declares no source and must emit nothing. It: // 1. runs the real codegen (runGen) over report/meta.json into a temp dir and fails // if a routes file was emitted for the sourceless report; -// 2. provisions `invoices` by hand in the EMITTED snake_case spelling, then creates -// each view from buildReportViews (the real lowering, the real view SQL) under -// the emitted names, so the route reads through the real view; -// 3. imports the three EMITTED <Report>.routes.ts files unmodified and mounts them. +// 2. provisions the base tables by hand in the EMITTED snake_case spelling, then +// creates each view from buildReportViews (the real lowering, the real view SQL) +// under the emitted names, so the route reads through the real view; +// 3. imports the four EMITTED <Report>.routes.ts files unmodified and mounts them. // // Generated lane only, and on every port — see the corpus README. @@ -18,23 +18,108 @@ import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node: import { dirname, join } from "node:path"; import { fileURLToPath, pathToFileURL } from "node:url"; import { runGen, defineConfig, buildReportViews } from "@metaobjectsdev/codegen-ts"; -import { DEFAULT_COLUMN_NAMING_STRATEGY } from "@metaobjectsdev/metadata"; +import { + DEFAULT_COLUMN_NAMING_STRATEGY, resolveColumnName, resolveTableName, type MetaRoot, +} from "@metaobjectsdev/metadata"; import { entityFile, routesFile } from "@metaobjectsdev/test-generators"; import pg from "pg"; import { executeSql } from "./postgres-sql.ts"; import { loadMetadataFile } from "./load-metadata.ts"; +/** One value of a base-table row in seed.json. */ +export type SeedValue = string | number | boolean | null; + +/** A base-table row of seed.json, keyed by the model's field names. */ +export type SeedRow = Record<string, SeedValue>; + +/** + * report/seed.json. `reports` is what the views return: the seam lanes serve it, and it + * is not a table. Every other top-level key is a base table, named by its physical table + * name and listed parents first, so inserting the tables in file order satisfies every + * foreign key. + */ export interface ReportSeed { - invoices: Array<{ - id: number; - reference: string; - status: string; - amountCents: number; - issuedOn: string; - }>; reports?: Record<string, Array<Record<string, unknown>>>; + [table: string]: SeedRow[] | Record<string, Array<Record<string, unknown>>> | undefined; } +/** The one top-level key of seed.json that is not a base table. */ +export const SEED_REPORTS_KEY = "reports"; + +/** One row to insert, in the EMITTED column spelling. */ +export interface SeedInsert { + table: string; + columns: string[]; + values: SeedValue[]; +} + +/** + * The rows of every base table of `seed`, in file order (parents first). A key must be the + * table of an entity of `root` and a row key one of its fields; the column is the field's + * emitted spelling, `resolveColumnName` under the default naming strategy, as the codegen + * spells it. Anything else throws, so a seed the model cannot hold fails loudly. + */ +export function seedInserts(root: MetaRoot, seed: ReportSeed): SeedInsert[] { + const out: SeedInsert[] = []; + for (const [table, rows] of Object.entries(seed)) { + if (table === SEED_REPORTS_KEY) continue; + if (!Array.isArray(rows)) throw new Error(`seed.json: '${table}' is not an array of rows`); + const entity = root.objects().find((o) => o.isEntity() && resolveTableName(o) === table); + if (entity === undefined) throw new Error(`seed.json: '${table}' is not the table of an entity of the model`); + const fields = new Map(entity.fields().map((f) => [f.name, f])); + for (const row of rows) { + const columns: string[] = []; + const values: SeedValue[] = []; + for (const [name, value] of Object.entries(row)) { + const field = fields.get(name); + if (field === undefined) throw new Error(`seed.json: '${table}' row names '${name}', not a field of ${entity.name}`); + if (value !== null && !["string", "number", "boolean"].includes(typeof value)) { + throw new Error(`seed.json: '${table}.${name}' is not a scalar: ${JSON.stringify(value)}`); + } + columns.push(resolveColumnName(field, DEFAULT_COLUMN_NAMING_STRATEGY)); + values.push(value); + } + out.push({ table, columns, values }); + } + } + return out; +} + +/** + * The base tables, parents first, in the EMITTED snake_case spelling. The routes read the + * views and the views read these, so they are provisioned by hand rather than generated. + */ +const BASE_TABLES: ReadonlyArray<{ name: string; ddl: string }> = [ + { + name: "invoices", + ddl: ` + CREATE TABLE IF NOT EXISTS "invoices" ( + "id" bigserial PRIMARY KEY, + "reference" varchar(40) NOT NULL, + "status" varchar(20) NOT NULL, + "amount_cents" bigint NOT NULL, + "issued_on" date NOT NULL + );`, + }, + { + name: "products", + ddl: ` + CREATE TABLE IF NOT EXISTS "products" ( + "id" bigserial PRIMARY KEY, + "name" text NOT NULL + );`, + }, + { + name: "sales", + ddl: ` + CREATE TABLE IF NOT EXISTS "sales" ( + "id" bigserial PRIMARY KEY, + "product_id" bigint NOT NULL REFERENCES "products" ("id"), + "amount_cents" bigint NOT NULL + );`, + }, +]; + export interface GeneratedReportServerHandle { baseUrl: string; applySeed(seed: ReportSeed): Promise<void>; @@ -46,6 +131,7 @@ const SERVED_REPORTS = [ { name: "InvoiceStatusTotals", registrar: "invoiceStatusTotalsRoutes" }, { name: "InvoicesByMonth", registrar: "invoicesByMonthRoutes" }, { name: "InvoiceTotals", registrar: "invoiceTotalsRoutes" }, + { name: "ProductRevenue", registrar: "productRevenueRoutes" }, ] as const; export async function startGeneratedReportServer( @@ -89,7 +175,7 @@ export const db = drizzle(pool); `; writeFileSync(join(tmp, "db.ts"), dbModule, "utf8"); - // 3. Provision the base table (snake_case, the EMITTED Drizzle table's spelling) and + // 3. Provision the base tables (snake_case, the EMITTED Drizzle tables' spelling) and // each view from the real lowering under the same naming strategy. const views = buildReportViews(root, { dialect: "postgres", @@ -102,17 +188,11 @@ export const db = drizzle(pool); }) .join("\n"); await executeSql(connectionUri, ` - CREATE TABLE IF NOT EXISTS "invoices" ( - "id" bigserial PRIMARY KEY, - "reference" varchar(40) NOT NULL, - "status" varchar(20) NOT NULL, - "amount_cents" bigint NOT NULL, - "issued_on" date NOT NULL - ); + ${BASE_TABLES.map((t) => t.ddl).join("\n")} ${createViews} `); - // 4. Import the EMITTED route files unmodified and mount all three. + // 4. Import the EMITTED route files unmodified and mount every served report. const fastify = Fastify(); for (const r of SERVED_REPORTS) { const mod = (await import(pathToFileURL(join(tmp, `${r.name}.routes.ts`)).href)) as Record< @@ -132,7 +212,7 @@ export const db = drizzle(pool); return { baseUrl, applySeed: async (seed: ReportSeed) => { - await seedReport(connectionUri, seed); + await seedReport(connectionUri, root, seed); }, close: async () => { await fastify.close(); @@ -142,18 +222,23 @@ export const db = drizzle(pool); }; } -/** Truncate + insert the base table. Only `invoices` is seeded; the views derive on read. */ -export async function seedReport(connectionUri: string, seed: ReportSeed): Promise<void> { - await executeSql(connectionUri, `TRUNCATE TABLE "invoices" RESTART IDENTITY CASCADE;`); - for (const i of seed.invoices) { - await executeSql( - connectionUri, - `INSERT INTO "invoices" ("id","reference","status","amount_cents","issued_on") - VALUES (${i.id}, ${str(i.reference)}, ${str(i.status)}, ${i.amountCents}, ${str(i.issuedOn)})`, - ); - } +/** + * Truncate every base table, then insert each base table of the seed in file order + * (parents first). `reports` is not inserted: the views derive it on read. + */ +export async function seedReport(connectionUri: string, root: MetaRoot, seed: ReportSeed): Promise<void> { + const truncate = `TRUNCATE TABLE ${BASE_TABLES.map((t) => `"${t.name}"`).join(", ")} RESTART IDENTITY CASCADE;`; + const inserts = seedInserts(root, seed).map( + (r) => + `INSERT INTO "${r.table}" (${r.columns.map((c) => `"${c}"`).join(",")}) ` + + `VALUES (${r.values.map(sqlLiteral).join(", ")});`, + ); + await executeSql(connectionUri, [truncate, ...inserts].join("\n")); } -function str(v: string): string { +function sqlLiteral(v: SeedValue): string { + if (v === null) return "NULL"; + if (typeof v === "number") return String(v); + if (typeof v === "boolean") return v ? "TRUE" : "FALSE"; return `'${v.replace(/'/g, "''")}'`; } diff --git a/server/typescript/packages/integration-tests/src/api-contract-report-sqlite-server.ts b/server/typescript/packages/integration-tests/src/api-contract-report-sqlite-server.ts index cc5df6e7b..90faa5a24 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-report-sqlite-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-report-sqlite-server.ts @@ -6,9 +6,9 @@ // ratio there is gated by test/api-contract-report-sqlite.test.ts. // // Same shape as the Postgres server: run the real codegen over report/meta.json -// (dialect sqlite), provision `invoices` by hand in the EMITTED snake_case spelling, -// create each view from buildReportViews (the real lowering), import the EMITTED -// <Report>.routes.ts files unmodified and mount them. +// (dialect sqlite), provision the base tables by hand in the EMITTED snake_case +// spelling, create each view from buildReportViews (the real lowering), import the +// EMITTED <Report>.routes.ts files unmodified and mount them. import Fastify, { type FastifyInstance } from "fastify"; import type { Database } from "bun:sqlite"; @@ -19,7 +19,7 @@ import { runGen, defineConfig, buildReportViews } from "@metaobjectsdev/codegen- import { DEFAULT_COLUMN_NAMING_STRATEGY } from "@metaobjectsdev/metadata"; import { entityFile, routesFile } from "@metaobjectsdev/test-generators"; import { loadMetadataFile } from "./load-metadata.ts"; -import type { ReportSeed } from "./api-contract-report-generated-server.ts"; +import { seedInserts, type ReportSeed } from "./api-contract-report-generated-server.ts"; export interface SqliteReportServerHandle { baseUrl: string; @@ -31,8 +31,41 @@ const SERVED_REPORTS = [ { name: "InvoiceStatusTotals", registrar: "invoiceStatusTotalsRoutes" }, { name: "InvoicesByMonth", registrar: "invoicesByMonthRoutes" }, { name: "InvoiceTotals", registrar: "invoiceTotalsRoutes" }, + { name: "ProductRevenue", registrar: "productRevenueRoutes" }, ] as const; +/** The base tables, parents first, in the EMITTED snake_case spelling. */ +const BASE_TABLES: ReadonlyArray<{ name: string; ddl: string }> = [ + { + name: "invoices", + ddl: ` + CREATE TABLE "invoices" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT, + "reference" TEXT NOT NULL, + "status" TEXT NOT NULL, + "amount_cents" INTEGER NOT NULL, + "issued_on" TEXT NOT NULL + )`, + }, + { + name: "products", + ddl: ` + CREATE TABLE "products" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT, + "name" TEXT NOT NULL + )`, + }, + { + name: "sales", + ddl: ` + CREATE TABLE "sales" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT, + "product_id" INTEGER NOT NULL REFERENCES "products" ("id"), + "amount_cents" INTEGER NOT NULL + )`, + }, +]; + export async function startSqliteReportServer(metaPath: string): Promise<SqliteReportServerHandle> { const here = dirname(fileURLToPath(import.meta.url)); const genTmpRoot = join(here, "..", ".gen-tmp"); @@ -75,14 +108,7 @@ export const db = drizzle(client); dialect: "sqlite", columnNamingStrategy: DEFAULT_COLUMN_NAMING_STRATEGY, }); - dbMod.client.run(` - CREATE TABLE "invoices" ( - "id" INTEGER PRIMARY KEY AUTOINCREMENT, - "reference" TEXT NOT NULL, - "status" TEXT NOT NULL, - "amount_cents" INTEGER NOT NULL, - "issued_on" TEXT NOT NULL - )`); + for (const t of BASE_TABLES) dbMod.client.run(t.ddl); for (const v of views) { if (v.sql === undefined) throw new Error(`view ${v.name} has no SQL`); dbMod.client.run(`CREATE VIEW "${v.name}" AS ${v.sql}`); @@ -104,11 +130,13 @@ export const db = drizzle(client); return { baseUrl, applySeed: async (seed: ReportSeed) => { - dbMod.client.run(`DELETE FROM "invoices"`); - for (const i of seed.invoices) { + // Children before parents on the way out; the seed's file order (parents first) on the way in. + for (const t of [...BASE_TABLES].reverse()) dbMod.client.run(`DELETE FROM "${t.name}"`); + for (const r of seedInserts(root, seed)) { dbMod.client.run( - `INSERT INTO "invoices" ("id","reference","status","amount_cents","issued_on") VALUES (?,?,?,?,?)`, - [i.id, i.reference, i.status, i.amountCents, i.issuedOn], + `INSERT INTO "${r.table}" (${r.columns.map((c) => `"${c}"`).join(",")}) ` + + `VALUES (${r.columns.map(() => "?").join(",")})`, + r.values, ); } }, diff --git a/server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts b/server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts index 5fb65a2d4..09ba78a6b 100644 --- a/server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts +++ b/server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts @@ -2,52 +2,77 @@ import { describe, expect, test } from "bun:test"; import { readFileSync } from "node:fs"; import { join } from "node:path"; import { - OBJECT_SUBTYPE_REPORT, pluralize, reportReadSource, reportShape, toSnakeCase, + IDENTITY_REFERENCE_ATTR_REFERENCES, IDENTITY_SUBTYPE_REFERENCE, OBJECT_SUBTYPE_REPORT, TYPE_IDENTITY, + pluralize, reportReadSource, reportShape, resolveTableName, toSnakeCase, } from "@metaobjectsdev/metadata"; import { loadMetadataFile } from "../src/load-metadata.ts"; import { loadScenarios } from "../src/api-contract-scenario.ts"; import { generateReportApiSchemaSql } from "../src/api-contract-report-schema.ts"; +import { SEED_REPORTS_KEY, seedInserts, type ReportSeed } from "../src/api-contract-report-generated-server.ts"; import { API_CONTRACT_REPORT_DIR, API_CONTRACT_REPORT_SCENARIOS_DIR, API_CONTRACT_REPORT_SCHEMA_SQL_PATH, } from "../src/paths.ts"; -const seed = JSON.parse(readFileSync(join(API_CONTRACT_REPORT_DIR, "seed.json"), "utf8")) as { - invoices: Array<Record<string, unknown>>; - reports: Record<string, Array<Record<string, unknown>>>; +const seed = JSON.parse(readFileSync(join(API_CONTRACT_REPORT_DIR, "seed.json"), "utf8")) as ReportSeed; +const seedReports = (): Record<string, Array<Record<string, unknown>>> => { + if (seed.reports === undefined) throw new Error("seed.json has no reports"); + return seed.reports; }; describe("api-contract report corpus", () => { - test("the model loads and three of its four reports are view-backed", async () => { + test("the model loads and four of its five reports are view-backed", async () => { const root = await loadMetadataFile(join(API_CONTRACT_REPORT_DIR, "meta.json")); const reports = root.objects().filter((o) => o.subType === OBJECT_SUBTYPE_REPORT); expect(reports.map((r) => r.name)).toEqual( - ["InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals", "InvoiceDays"]); + ["InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals", "InvoiceDays", "ProductRevenue"]); expect(reports.filter((r) => reportReadSource(r) !== undefined).map((r) => r.name)).toEqual( - ["InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals"]); + ["InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals", "ProductRevenue"]); }); test("the route segments are the ones the scenarios call", async () => { const root = await loadMetadataFile(join(API_CONTRACT_REPORT_DIR, "meta.json")); const segment = (name: string): string => pluralize(toSnakeCase(name)); - expect(["InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals"].map(segment)).toEqual( - ["invoice_status_totals", "invoices_by_months", "invoice_totals"]); - expect(root.objects().length).toBe(5); + expect(["InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals", "ProductRevenue"].map(segment)).toEqual( + ["invoice_status_totals", "invoices_by_months", "invoice_totals", "product_revenues"]); + expect(root.objects().length).toBe(8); }); test("each seeded report row has exactly the report's derived fields", async () => { const root = await loadMetadataFile(join(API_CONTRACT_REPORT_DIR, "meta.json")); - for (const [name, rows] of Object.entries(seed.reports)) { + for (const [name, rows] of Object.entries(seedReports())) { const report = root.objects().find((o) => o.name === name); if (report === undefined) throw new Error(`seed.json names a report the model lacks: ${name}`); const fields = reportShape(report, root).fields.map((f) => f.name); for (const row of rows) expect(Object.keys(row)).toEqual(fields); } - expect(Object.keys(seed.reports)).toEqual(["InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals"]); + expect(Object.keys(seedReports())).toEqual( + ["InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals", "ProductRevenue"]); }); - test("every scenario parses, and there are thirteen", () => { + // A lane inserts every top-level key but `reports` in file order, so the order is the + // contract: a table that references another comes after it. + test("the base tables of seed.json are the model's tables, listed parents first", async () => { + const root = await loadMetadataFile(join(API_CONTRACT_REPORT_DIR, "meta.json")); + const tables = Object.keys(seed).filter((k) => k !== SEED_REPORTS_KEY); + expect(tables).toEqual(["invoices", "products", "sales"]); + // Every key is an entity's table and every row key one of its fields (seedInserts throws otherwise). + expect(seedInserts(root, seed).length).toBe(5 + 3 + 3); + for (const [i, table] of tables.entries()) { + const entity = root.objects().find((o) => o.isEntity() && resolveTableName(o) === table); + if (entity === undefined) throw new Error(`no entity has table ${table}`); + for (const ref of entity.children()) { + if (ref.type !== TYPE_IDENTITY || ref.subType !== IDENTITY_SUBTYPE_REFERENCE) continue; + const target = root.objects().find((o) => o.name === ref.attr(IDENTITY_REFERENCE_ATTR_REFERENCES)); + if (target === undefined) throw new Error(`${entity.name}.${ref.name} references no object`); + expect(tables.indexOf(resolveTableName(target))).toBeLessThan(i); + expect(tables.indexOf(resolveTableName(target))).toBeGreaterThanOrEqual(0); + } + } + }); + + test("every scenario parses, and there are sixteen", () => { const scenarios = loadScenarios(API_CONTRACT_REPORT_SCENARIOS_DIR); - expect(scenarios.length).toBe(13); + expect(scenarios.length).toBe(16); for (const s of scenarios) expect(s.requests.length).toBeGreaterThan(0); }); diff --git a/server/typescript/packages/integration-tests/test/api-contract-report-sqlite.test.ts b/server/typescript/packages/integration-tests/test/api-contract-report-sqlite.test.ts index 52570ef86..5f7cbc963 100644 --- a/server/typescript/packages/integration-tests/test/api-contract-report-sqlite.test.ts +++ b/server/typescript/packages/integration-tests/test/api-contract-report-sqlite.test.ts @@ -78,6 +78,9 @@ describe("api contract report (FR-044) — GENERATED routes on SQLite", () => { expect(await getJson(server, "/api/invoices_by_months?sort=issuedOnMonth:asc")).toEqual( SEED.reports!["InvoicesByMonth"], ); + expect(await getJson(server, "/api/product_revenues?sort=productId:asc")).toEqual( + SEED.reports!["ProductRevenue"], + ); }); }); diff --git a/server/typescript/packages/integration-tests/test/api-contract-report.test.ts b/server/typescript/packages/integration-tests/test/api-contract-report.test.ts index 05aa8d903..f05ddbe04 100644 --- a/server/typescript/packages/integration-tests/test/api-contract-report.test.ts +++ b/server/typescript/packages/integration-tests/test/api-contract-report.test.ts @@ -26,6 +26,7 @@ const SERVED_PATHS: Array<[string, string]> = [ ["InvoiceStatusTotals", "/api/invoice_status_totals"], ["InvoicesByMonth", "/api/invoices_by_months"], ["InvoiceTotals", "/api/invoice_totals"], + ["ProductRevenue", "/api/product_revenues"], ]; describe("api contract report (FR-044) — GENERATED routes lane", () => { From 6c03ee197902300323bf6a04a25acad982f5084e Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 21:24:38 -0400 Subject: [PATCH 12/18] feat(csharp): @spine and measure @default FR-044 zero rows and measure defaults, the C# port of the TypeScript reference (Tables A, B and C). - Register @spine (string) on object.report and @default (int) on measure.aggregate and measure.ratio; byte-copy the two spec files. MetaMeasure.DefaultValue() and ReportAccessors.ReportSpine(). - Loader rules R8 (the spine is D2's to-one walk, started at @from), R9 (every listed dimension is reached through the spine, hop names compared as written), M7 (@default on a count) and M8 (@default on min/max of a non-numeric field), with the TypeScript message texts. WalkToOneVia takes a ToOneWalk (D2Walk / SpineWalk); every D2 message is unchanged. A fractional @default is already refused by this port's generic attr.int type check (ERR_BAD_ATTR_VALUE on the measure), so no second error is added; it only skips M7/M8. - ReportShape Table C: under @spine a dimension on the spine is required when its @of is @required or a primary-key column; a defaulted measure is required. New ReportingViaHops / ReportSpineHops. - No generator change: the keyless row types a defaulted measure as long / decimal and the spine key as non-nullable. Regenerated the integration fixtures (three new rows, AppDbContext additions). - The report REST lane seeds every base table of seed.json in file order and expects ProductRevenue (and the Product / Sale entities) routed. Corpus counts: nine canonical reports, four inert reports. --- .../CodegenCompileConformanceTests.cs | 7 +- .../ReportRowCodegenTests.cs | 117 ++++- .../ReportingInertTests.cs | 20 +- .../ReportShapeTests.cs | 295 ++++++++++- .../ReportingAccessorsTests.cs | 60 +++ .../ReportingValidationTests.cs | 468 +++++++++++++++++- .../Api/ApiContractReportConformanceTest.cs | 3 +- .../Api/ReportFixture.cs | 88 ++-- .../Api/ReportGeneratedServerFactory.cs | 33 +- .../Generated/AppDbContext.g.cs | 6 + .../Generated/FitnessTotalsFilled.g.cs | 19 + .../Generated/ProgramLongWeeks.g.cs | 19 + .../Generated/ProgramRoster.g.cs | 30 ++ .../Core/Object/ObjectConstants.cs | 3 + .../Core/Reporting/ReportAccessors.cs | 4 + .../MetaObjects/Core/Reporting/ReportShape.cs | 122 ++++- .../Core/Reporting/ReportingConstants.cs | 3 + .../Core/Reporting/ReportingSchema.cs | 13 +- .../Loader/ValidationPasses.Reporting.cs | 205 ++++++-- .../MetaObjects/Loader/ValidationPasses.cs | 4 +- server/csharp/MetaObjects/Meta/MetaMeasure.cs | 15 + .../MetaObjects/SpecMetamodel/object.json | 3 +- .../MetaObjects/SpecMetamodel/reporting.json | 6 +- 23 files changed, 1410 insertions(+), 133 deletions(-) create mode 100644 server/csharp/MetaObjects.IntegrationTests/Generated/FitnessTotalsFilled.g.cs create mode 100644 server/csharp/MetaObjects.IntegrationTests/Generated/ProgramLongWeeks.g.cs create mode 100644 server/csharp/MetaObjects.IntegrationTests/Generated/ProgramRoster.g.cs diff --git a/server/csharp/MetaObjects.Codegen.Tests/CodegenCompileConformanceTests.cs b/server/csharp/MetaObjects.Codegen.Tests/CodegenCompileConformanceTests.cs index 6c904aa02..8325875b6 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/CodegenCompileConformanceTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/CodegenCompileConformanceTests.cs @@ -146,14 +146,15 @@ public void Every_generated_file_compiles_with_zero_errors(string selection, boo var files = generators.SelectMany(g => g.Generate(ctx)).ToList(); Assert.True(files.Count > 0, $"selection '{selection}' generated no files at all"); - // FR-044 — the corpus's six view-backed reports each generate a keyless row class and + // FR-044 — the corpus's nine view-backed reports each generate a keyless row class and // a filter allowlist (their routes file is in the excluded framework tier), and no // names artifact. Named, because a compile gate passes trivially over a file that - // was never emitted. + // was never emitted. The last three carry @spine / a measure @default. foreach (var report in new[] { "ProgramMinutes", "FitnessTotals", "ProgramsByMonth", "ProgramsByWeek", - "RecentPrograms", "AssetActivity", + "RecentPrograms", "AssetActivity", "ProgramRoster", "ProgramLongWeeks", + "FitnessTotalsFilled", }) { Assert.True(files.Any(f => f.Path == report + ".g.cs"), $"no row class was generated for report {report}"); diff --git a/server/csharp/MetaObjects.Codegen.Tests/ReportRowCodegenTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ReportRowCodegenTests.cs index 42cbd1bd1..2d5a29ecb 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ReportRowCodegenTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ReportRowCodegenTests.cs @@ -203,7 +203,8 @@ public void A_context_built_from_the_unfiltered_root_generates_the_same_files() // --------------------------------------------------------------------- /// <summary>fixtures/codegen-noop/reporting/with: `StoreTotals` is view-backed, - /// `ProgramEngagement` and `DailyRevenue` are sourceless.</summary> + /// `ProgramEngagement`, `DailyRevenue` and `ProgramCatalogue` (a `@spine` report) are + /// sourceless.</summary> private static MetaRoot LoadWithModel() { string path = Path.Combine( @@ -271,7 +272,7 @@ public void A_served_report_gets_a_filter_allowlist_naming_every_derived_field() public void A_sourceless_report_gets_no_routes_file_and_no_allowlist() { var files = Emit(RunnerContext(LoadWithModel()), new RoutesGenerator(), new FilterAllowlistGenerator()); - foreach (var sourceless in new[] { "ProgramEngagement", "DailyRevenue" }) + foreach (var sourceless in new[] { "ProgramEngagement", "DailyRevenue", "ProgramCatalogue" }) { Assert.DoesNotContain(files.Keys, k => k.Contains(sourceless, StringComparison.Ordinal)); Assert.DoesNotContain(files.Values, c => c.Contains(sourceless, StringComparison.Ordinal)); @@ -307,7 +308,7 @@ public void The_applies_to_predicates_answer_for_a_declared_report_node() Assert.True(RoutesGenerator.AppliesTo(Named("StoreTotals"), root)); Assert.True(FilterAllowlistGenerator.AppliesTo(Named("StoreTotals"))); - foreach (var sourceless in new[] { "ProgramEngagement", "DailyRevenue" }) + foreach (var sourceless in new[] { "ProgramEngagement", "DailyRevenue", "ProgramCatalogue" }) { Assert.False(RoutesGenerator.AppliesTo(Named(sourceless), root)); Assert.False(FilterAllowlistGenerator.AppliesTo(Named(sourceless))); @@ -439,6 +440,116 @@ public void A_derived_field_has_the_type_and_nullability_of_its_Table_B_row(stri Assert.Contains(" " + property + "\n", RowOf(Cube()).ReplaceLineEndings("\n")); } + // --------------------------------------------------------------------- + // Table C / Table F (FR-044 @spine and measure @default): no generator change. The + // shape marks a spine key and a defaulted measure required, and the row types a + // required field exactly as it types a count today: `long`, not `long?`. + // --------------------------------------------------------------------- + + // Store.id is the key and is NOT @required: under @spine it is not nullable because it + // is the spine entity's key (Table C's identity clause). + private const string SpineModel = + """ + { "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Store", "children": [ + { "source.rdb": { "@table": "stores" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "region" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ] } }, + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id", "@required": true } }, + { "field.long": { "name": "storeId" } }, + { "field.int": { "name": "units", "@required": true } }, + { "field.currency": { "name": "amountCents", "@required": true, "@currency": "USD" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "storeRef", "@references": "Store", "@fields": ["storeId"] } }, + { "relationship.association": { "name": "store", "@objectRef": "Store", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "storeKey", "@of": "Store.id", "@via": "Sale.store" } }, + { "dimension.attribute": { "name": "storeRegion", "@of": "Store.region", "@via": "Sale.store" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } }, + { "measure.aggregate": { "name": "unitsSold", "@agg": "sum", "@of": "Sale.units" } }, + { "measure.aggregate": { "name": "unitsSoldOrZero", "@agg": "sum", "@of": "Sale.units", "@default": 0 } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Sale.amountCents" } }, + { "measure.aggregate": { "name": "revenueOrZero", "@agg": "sum", "@of": "Sale.amountCents", "@default": 0 } }, + { "measure.aggregate": { "name": "avgUnitsOrZero", "@agg": "avg", "@of": "Sale.units", "@default": 0 } }, + { "measure.ratio": { "name": "unitsPerSale", "@numerator": "unitsSold", "@denominator": "sales" } }, + { "measure.ratio": { "name": "unitsPerSaleOrZero", "@numerator": "unitsSold", "@denominator": "sales", "@default": 0 } } + ] } }, + { "object.report": { "name": "StoreSales", "@from": "Sale", "@spine": "Sale.store", + "@dimensions": ["storeKey", "storeRegion"], + "@measures": ["sales", "unitsSold", "unitsSoldOrZero", "revenue", "revenueOrZero", "avgUnitsOrZero", "unitsPerSale", "unitsPerSaleOrZero"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_store_sales" } } ] } }, + { "object.report": { "name": "SalesByStore", "@from": "Sale", + "@dimensions": ["storeKey", "storeRegion"], + "@measures": ["sales", "unitsSold", "unitsSoldOrZero", "unitsPerSale", "unitsPerSaleOrZero"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_sales_by_store" } } ] } } + ] } } + """; + + private static MetaRoot LoadSpineModel() + { + var result = new MetaDataLoader().Load([new InMemoryStringSource(SpineModel, id: "meta.shop.json")]); + Assert.True(result.Errors.Count == 0, + "model did not load:\n" + string.Join("\n", result.Errors.Select(e => $" {e.Code}: {e.Message}"))); + return result.Root; + } + + private static string SpineRow(string report) => + Emit(RunnerContext(LoadSpineModel()), new EntityGenerator())[report + ".g.cs"].ReplaceLineEndings("\n"); + + [Theory] + [InlineData("public long StoreKey { get; set; }")] // the spine key: Store.id, the spine entity's key + [InlineData("public string? StoreRegion { get; set; }")] // on the spine, but optional + [InlineData("public long Sales { get; set; }")] // a count, as before + [InlineData("public long? UnitsSold { get; set; }")] // a sum without @default stays nullable + [InlineData("public long UnitsSoldOrZero { get; set; }")] // a defaulted sum is long, not long? + [InlineData("public long? Revenue { get; set; }")] + [InlineData("public long RevenueOrZero { get; set; }")] // a defaulted sum of a currency + [InlineData("public decimal AvgUnitsOrZero { get; set; }")] // a defaulted avg + [InlineData("public decimal? UnitsPerSale { get; set; }")] // a ratio without @default stays nullable + [InlineData("public decimal UnitsPerSaleOrZero { get; set; }")] // a defaulted ratio is decimal + public void A_spine_report_row_types_the_spine_key_and_a_defaulted_measure_as_not_nullable(string property) + { + Assert.Contains(" " + property + "\n", SpineRow("StoreSales")); + } + + [Theory] + [InlineData("public long? StoreKey { get; set; }")] // reached by @via, no @spine: nullable as today + [InlineData("public long UnitsSoldOrZero { get; set; }")] // @default alone, no @spine + [InlineData("public long? UnitsSold { get; set; }")] + [InlineData("public decimal UnitsPerSaleOrZero { get; set; }")] + [InlineData("public decimal? UnitsPerSale { get; set; }")] + public void Without_spine_a_key_reached_by_via_stays_nullable_and_a_defaulted_measure_is_not(string property) + { + Assert.Contains(" " + property + "\n", SpineRow("SalesByStore")); + } + + [Fact] + public void The_spine_report_rows_and_their_mapping_compile() + { + var ctx = RunnerContext(LoadSpineModel(), Config(includeNames: true)); + var files = new IGenerator[] + { + new EntityGenerator(), new DbContextGenerator(), new FilterAllowlistGenerator(), new NamesGenerator(), + } + .SelectMany(g => g.Generate(ctx)).ToList(); + Assert.Contains(files, f => f.Path == "StoreSales.g.cs"); + var trees = files + .Select(f => CSharpSyntaxTree.ParseText(f.Content, new CSharpParseOptions(LanguageVersion.CSharp12), path: f.Path)) + .ToList(); + var comp = CSharpCompilation.Create( + "report_spine_rows_" + Guid.NewGuid().ToString("N"), trees, + DbContextCompileTests.BuildReferences(), + new CSharpCompilationOptions(OutputKind.DynamicallyLinkedLibrary)); + var errors = comp.GetDiagnostics() + .Where(d => d.Severity == DiagnosticSeverity.Error) + .Select(d => $"{d.Location.GetLineSpan().Path}: {d.Id}: {d.GetMessage()}") + .ToList(); + Assert.True(errors.Count == 0, string.Join("\n", errors)); + } + [Fact] public void Fields_are_emitted_in_Table_B_order() { diff --git a/server/csharp/MetaObjects.Codegen.Tests/ReportingInertTests.cs b/server/csharp/MetaObjects.Codegen.Tests/ReportingInertTests.cs index e45a20920..1aed3546e 100644 --- a/server/csharp/MetaObjects.Codegen.Tests/ReportingInertTests.cs +++ b/server/csharp/MetaObjects.Codegen.Tests/ReportingInertTests.cs @@ -6,8 +6,10 @@ // `HasNoKey().ToView(...)` mapping plus a DbSet (Plan 2), and serves it through a filter // allowlist and a read-only routes file (Plan 3). So: // -// - a SOURCELESS report (`ProgramEngagement`, `DailyRevenue`) stays inert in every -// registered generator and in the api docs; +// - a SOURCELESS report (`ProgramEngagement`, `DailyRevenue`, and `ProgramCatalogue`, +// which declares `@spine` and lists a dimension reached by `@via`) stays inert in every +// registered generator and in the api docs, as does a measure `@default` +// (`avgDaysPerStarter`); // - the VIEW-BACKED report (`StoreTotals`) adds exactly three files (its row class, its // filter allowlist, its routes file), exactly its lines in AppDbContext.g.cs, one unit // in the api docs, and nothing in the names tier or in any other generator. @@ -28,6 +30,9 @@ namespace MetaObjects.Codegen.Tests; public class ReportingInertTests { + /// <summary>The with-model's reports that declare no source: inert everywhere.</summary> + private static readonly string[] SourcelessReports = ["ProgramEngagement", "DailyRevenue", "ProgramCatalogue"]; + private static string Model(string variant) => Path.Combine(CorpusPaths.RepoRoot(), "fixtures", "codegen-noop", "reporting", variant, "meta.shop.json"); @@ -89,7 +94,7 @@ public void The_with_model_really_carries_the_vocabulary() { // Else every comparison below is vacuously green. var reports = Load("with").Objects().Where(o => o.IsReport()).Select(o => o.Name).OrderBy(n => n).ToList(); - Assert.Equal(["DailyRevenue", "ProgramEngagement", "StoreTotals"], reports); + Assert.Equal(["DailyRevenue", "ProgramCatalogue", "ProgramEngagement", "StoreTotals"], reports); Assert.DoesNotContain(Load("without").Objects(), o => o.IsReport()); } @@ -293,7 +298,7 @@ public void A_report_reaches_no_tier_but_its_row_its_mapping_its_allowlist_and_i [RowFile, AllowlistFile, RoutesFile], files.Keys.Where(k => k.Contains("StoreTotals", StringComparison.Ordinal)).ToList()); // A sourceless report: no file at all, and no mention in any file. - foreach (var sourceless in new[] { "ProgramEngagement", "DailyRevenue" }) + foreach (var sourceless in SourcelessReports) { Assert.DoesNotContain(files.Keys, k => k.Contains(sourceless, StringComparison.Ordinal)); Assert.DoesNotContain(files.Values, c => c.Contains(sourceless, StringComparison.Ordinal)); @@ -343,14 +348,13 @@ public void Api_docs_add_one_page_for_the_served_report_and_change_nothing_else( // The index and the agent page gain lines for the report and lose none. var added = AddedLines(content, actual[path]); Assert.NotEmpty(added); - Assert.DoesNotContain(added, l => l.Contains("ProgramEngagement", StringComparison.Ordinal) - || l.Contains("DailyRevenue", StringComparison.Ordinal)); + Assert.DoesNotContain(added, l => SourcelessReports.Any(r => l.Contains(r, StringComparison.Ordinal))); continue; } Assert.True(content == actual[path], $"{path} differs once reporting nodes are declared"); } // A sourceless report is mentioned nowhere. - foreach (var sourceless in new[] { "ProgramEngagement", "DailyRevenue" }) + foreach (var sourceless in SourcelessReports) Assert.DoesNotContain(actual.Values, c => c.Contains(sourceless, StringComparison.Ordinal)); } @@ -360,7 +364,7 @@ public void The_served_report_is_documented_as_its_row_its_DbSet_the_list_GET_an var config = new GenConfig { OutDir = "/unused", Namespace = "Shop" }; var units = new CSharpApiModelBuilder(config).Build(Load("with"), "shop").Units; - Assert.DoesNotContain(units, u => u.Node is "ProgramEngagement" or "DailyRevenue"); + Assert.DoesNotContain(units, u => SourcelessReports.Contains(u.Node)); var unit = Assert.Single(units, u => u.Node == "StoreTotals"); Assert.Equal("report", unit.Kind); Assert.Equal("acme::shop", unit.Package); diff --git a/server/csharp/MetaObjects.Conformance.Tests/ReportShapeTests.cs b/server/csharp/MetaObjects.Conformance.Tests/ReportShapeTests.cs index f2184620d..385aec4b8 100644 --- a/server/csharp/MetaObjects.Conformance.Tests/ReportShapeTests.cs +++ b/server/csharp/MetaObjects.Conformance.Tests/ReportShapeTests.cs @@ -4,7 +4,10 @@ // // Container-free: pure metadata in, JSON out. The same artifact gates the Java, Kotlin // and Python derivations, so the five ports cannot drift on a derived field's name, -// subtype, nullability or type source. +// subtype, nullability or type source. `required` follows Table C of +// docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md (a @spine +// report's spine key and a measure with @default are not nullable); the Table C cases +// below are the TypeScript report-shape.test.ts cases. using System.IO; using System.Linq; @@ -45,15 +48,31 @@ public void Derived_shapes_byte_match_the_committed_artifact() } [Fact] - public void The_artifact_covers_the_six_canonical_reports() + public void The_artifact_covers_the_nine_canonical_reports() { // Else the byte comparison could pass over an empty list on both sides. var reports = LoadCanonical().Objects().Where(o => o.IsReport()).Select(o => o.Name).ToList(); Assert.Equal( - ["ProgramMinutes", "FitnessTotals", "ProgramsByMonth", "ProgramsByWeek", "RecentPrograms", "AssetActivity"], + ["ProgramMinutes", "FitnessTotals", "ProgramsByMonth", "ProgramsByWeek", "RecentPrograms", "AssetActivity", + "ProgramRoster", "ProgramLongWeeks", "FitnessTotalsFilled"], reports); } + [Fact] + public void The_canonical_spine_report_has_a_required_spine_key_and_defaulted_measures() + { + var fields = Shape(LoadCanonical(), "ProgramRoster").Fields.ToDictionary(f => f.Name); + Assert.True(fields["programKey"].Required); // Program.id, the spine entity's key + Assert.True(fields["programTitle"].Required); // Program.title, @required, on the spine + Assert.True(fields["weeks"].Required); // count + Assert.False(fields["totalMinutes"].Required); // no @default + Assert.True(fields["totalMinutesOrZero"].Required); // @default: 0 + Assert.False(fields["longShare"].Required); + Assert.True(fields["longShareOrZero"].Required); + Assert.Equal(FIELD_SUBTYPE_LONG, fields["totalMinutesOrZero"].SubType); + Assert.Equal(FIELD_SUBTYPE_DECIMAL, fields["longShareOrZero"].SubType); + } + [Fact] public void Dimensions_come_first_in_listed_order_then_measures() { @@ -236,4 +255,274 @@ public void A_time_dimension_item_with_no_grain_or_a_grain_outside_the_closed_se ReportShapes.Of(Stray("fitness", "Program", OBJECT_REPORT_ATTR_DIMENSIONS, "createdAt:fortnight"), root)); Assert.Equal("report 'Stray': time dimension 'createdAt' grain 'fortnight' does not resolve.", ex.Message); } + + // ----------------------------------------------------------------------- + // Table C (FR-044 @spine and measure @default): under @spine a column of the spine + // entity is not nullable when it is @required or a primary-key column; a dimension + // beyond the spine is; a measure with @default is not. + // ----------------------------------------------------------------------- + + private const string Catalog = + """ + { "object.entity": { "name": "Catalog", "children": [ + { "source.rdb": { "@table": "catalogs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name", "@required": true } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } ] } } + """; + + // No @required on Program.id: a key column is not nullable because it is the key. + private const string ProgramFields = + """ + { "source.rdb": { "@table": "programs" } }, + { "field.string": { "name": "title", "@required": true } }, + { "field.string": { "name": "subtitle" } }, + { "field.timestamp": { "name": "publishedAt", "@required": true } }, + { "field.long": { "name": "catalogId" } }, + { "identity.reference": { "name": "fkCatalog", "@references": "Catalog", "@fields": ["catalogId"] } }, + { "relationship.association": { "name": "catalog", "@objectRef": "Catalog", "@cardinality": "one" } } + """; + + private const string Program = + """ + { "object.entity": { "name": "Program", "children": [ + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + """ + ProgramFields + " ] } }"; + + // `<<COUNT>>` is replaced by extra attrs on the `purchases` count (a @default past the loader). + private const string PurchaseTemplate = + """ + { "object.entity": { "name": "Purchase", "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "programId" } }, + { "field.int": { "name": "minutes", "@required": true } }, + { "field.currency": { "name": "amountCents", "@currency": "USD" } }, + { "field.double": { "name": "score" } }, + { "field.timestamp": { "name": "purchasedAt" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@references": "Program", "@fields": ["programId"] } }, + { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programSubtitle", "@of": "Program.subtitle", "@via": "<<SUBTITLE_VIA>>" } }, + { "dimension.time": { "name": "publishedAt", "@of": "Program.publishedAt", "@via": "Purchase.program", "@grains": ["hour", "month"] } }, + { "dimension.attribute": { "name": "catalogId", "@of": "Catalog.id", "@via": "Purchase.program.catalog" } }, + { "dimension.attribute": { "name": "catalogName", "@of": "Catalog.name", "@via": "Purchase.program.catalog" } }, + { "dimension.attribute": { "name": "minutes", "@of": "Purchase.minutes" } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id"<<COUNT>> } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } }, + { "measure.aggregate": { "name": "revenueRaw", "@agg": "sum", "@of": "Purchase.amountCents" } }, + { "measure.aggregate": { "name": "avgMinutes", "@agg": "avg", "@of": "Purchase.minutes", "@default": 0 } }, + { "measure.aggregate": { "name": "avgMinutesRaw", "@agg": "avg", "@of": "Purchase.minutes" } }, + { "measure.aggregate": { "name": "minMinutes", "@agg": "min", "@of": "Purchase.minutes", "@default": -1 } }, + { "measure.aggregate": { "name": "minMinutesRaw", "@agg": "min", "@of": "Purchase.minutes" } }, + { "measure.ratio": { "name": "share", "@numerator": "revenue", "@denominator": "purchases", "@default": 0 } }, + { "measure.ratio": { "name": "shareRaw", "@numerator": "revenue", "@denominator": "purchases" } } ] } } + """; + + private static string Purchase(string count = "", string subtitleVia = "Purchase.program") => + PurchaseTemplate.Replace("<<COUNT>>", count).Replace("<<SUBTITLE_VIA>>", subtitleVia); + + private const string AllMeasures = + """ + "purchases", "revenue", "revenueRaw", "avgMinutes", "avgMinutesRaw", "minMinutes", "minMinutesRaw", "share", "shareRaw" + """; + + private const string SpineSales = + """ + { "object.report": { "name": "SpineSales", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle", "programSubtitle", "publishedAt:hour", "publishedAt:month", "catalogId", "catalogName"], + "@measures": [ + """ + AllMeasures + "] } }"; + + private const string PlainSales = + """ + { "object.report": { "name": "PlainSales", "@from": "Purchase", + "@dimensions": ["programId", "programTitle", "programSubtitle", "catalogId", "catalogName", "minutes"], + "@measures": [ + """ + AllMeasures + "] } }"; + + private const string CatalogSales = + """ + { "object.report": { "name": "CatalogSales", "@from": "Purchase", "@spine": "Purchase.program.catalog", + "@dimensions": ["catalogId", "catalogName"], "@measures": ["purchases", "revenue"] } } + """; + + private static string AcmeFile(params string[] children) => + "{ \"metadata.root\": { \"package\": \"acme\", \"children\": [" + string.Join(", ", children) + "] } }"; + + private static MetaRoot SalesModel() => + LoadInline(AcmeFile(Catalog, Program, Purchase(), SpineSales, PlainSales, CatalogSales)); + + private static Dictionary<string, bool> RequiredOf(MetaRoot root, string report) => + Shape(root, report).Fields.ToDictionary(f => f.Name, f => f.Required); + + private static MetaObject Obj(MetaRoot root, string name) => root.Objects().Single(o => o.Name == name); + + private static MetaRoot LoadSpineInheritedFixture() + { + var result = new MetaDataLoader().Load( + [new FileSource(Path.Combine(CorpusRoot.Path, "reporting-spine-inherited", "input", "meta.shop.json"))]); + Assert.Empty(result.Errors); + return result.Root; + } + + [Fact] + public void Under_spine_a_column_of_the_spine_entity_is_required_when_it_is_a_key_or_required() + { + var required = RequiredOf(SalesModel(), "SpineSales"); + Assert.True(required["programId"]); // Program.id: no @required, in identity.primary + Assert.True(required["programTitle"]); // Program.title: @required + Assert.False(required["programSubtitle"]); // Program.subtitle: optional + // A time dimension over a @required spine column, at either grain path (timestamp / date). + Assert.True(required["publishedAtHour"]); + Assert.True(required["publishedAtMonth"]); + } + + [Fact] + public void Under_spine_a_dimension_one_hop_beyond_the_spine_is_not_required_even_over_a_key_or_a_required_field() + { + var required = RequiredOf(SalesModel(), "SpineSales"); + Assert.False(required["catalogId"]); // Catalog.id: a key, beyond the spine + Assert.False(required["catalogName"]); // Catalog.name: @required, beyond the spine + } + + [Fact] + public void A_two_hop_spine_a_dimension_whose_via_equals_the_whole_spine_is_on_it() + { + var required = RequiredOf(SalesModel(), "CatalogSales"); + Assert.True(required["catalogId"]); + Assert.True(required["catalogName"]); + } + + [Fact] + public void The_same_dimensions_in_a_report_without_spine_keep_todays_values() + { + var required = RequiredOf(SalesModel(), "PlainSales"); + Assert.False(required["programId"]); + Assert.False(required["programTitle"]); + Assert.False(required["programSubtitle"]); + Assert.False(required["catalogId"]); + Assert.False(required["catalogName"]); + Assert.True(required["minutes"]); // no @via, @of @required: unchanged + } + + [Theory] + [InlineData("SpineSales")] + [InlineData("PlainSales")] + public void A_sum_an_avg_a_min_and_a_ratio_are_required_with_default_and_not_without_a_count_always_is(string report) + { + var required = RequiredOf(SalesModel(), report); + Assert.True(required["purchases"]); + Assert.True(required["revenue"]); + Assert.False(required["revenueRaw"]); + Assert.True(required["avgMinutes"]); + Assert.False(required["avgMinutesRaw"]); + Assert.True(required["minMinutes"]); + Assert.False(required["minMinutesRaw"]); + Assert.True(required["share"]); + Assert.False(required["shareRaw"]); + } + + [Fact] + public void A_count_with_a_default_past_the_loader_which_refuses_it_is_still_required() + { + var result = new MetaDataLoader().Load([new InMemoryStringSource( + AcmeFile(Catalog, Program, Purchase(count: ", \"@default\": 5"), PlainSales), id: "meta.inline.json")]); + Assert.Equal([true], result.Errors.Select(e => e.Message.Contains("@default cannot apply to @agg: count"))); + Assert.True(RequiredOf(result.Root, "PlainSales")["purchases"]); + } + + [Fact] + public void Subtypes_and_type_sources_are_untouched_by_default() + { + var shape = Shape(SalesModel(), "SpineSales").Fields.Select(f => $"{f.Name} {f.SubType} {f.TypeSource?.Name ?? "-"}"); + Assert.Equal( + [ + "programId long id", "programTitle string title", "programSubtitle string subtitle", + "publishedAtHour timestamp publishedAt", "publishedAtMonth date -", "catalogId long id", + "catalogName string name", "purchases long -", "revenue currency amountCents", + "revenueRaw currency amountCents", "avgMinutes decimal -", "avgMinutesRaw decimal -", + "minMinutes int minutes", "minMinutesRaw int minutes", "share decimal -", "shareRaw decimal -", + ], + shape); + } + + [Fact] + public void An_identity_primary_inherited_from_an_abstract_base_makes_the_key_column_required() + { + // ADR-0039: resolving — Program's key is declared on the abstract Keyed. + const string keyed = + """ + { "object.entity": { "name": "Keyed", "abstract": true, "children": [ + { "field.long": { "name": "id" } }, { "identity.primary": { "name": "pk", "@fields": ["id"] } } ] } } + """; + string inheritedProgram = + """ + { "object.entity": { "name": "Program", "extends": "Keyed", "children": [ + """ + ProgramFields + " ] } }"; + var root = LoadInline(AcmeFile(keyed, Catalog, inheritedProgram, Purchase(), SpineSales)); + Assert.True(RequiredOf(root, "SpineSales")["programId"]); + } + + [Fact] + public void Under_spine_a_dimension_whose_via_does_not_resolve_throws_naming_the_report() + { + // An owner that is not @from (or an entity it extends): ReportingViaHops does not + // resolve it. Past the loader, which refuses the @via under rule D2. + var result = new MetaDataLoader().Load([new InMemoryStringSource( + AcmeFile(Catalog, Program, Purchase(subtitleVia: "Program.catalog"), SpineSales), id: "meta.inline.json")]); + Assert.Single(result.Errors); + var ex = Assert.Throws<InvalidOperationException>(() => Shape(result.Root, "SpineSales")); + Assert.Equal("report 'SpineSales': dimension 'programSubtitle' @via 'Program.catalog' does not resolve.", ex.Message); + } + + [Fact] + public void A_spine_written_with_an_abstract_base_as_its_owner_over_a_concrete_from() + { + var root = LoadSpineInheritedFixture(); + Assert.Equal( + ["programId True", "programTitle True", "totalMinutes True"], + Shape(root, "ProgramMinutes").Fields.Select(f => $"{f.Name} {f.Required}").ToList()); + } + + [Fact] + public void ReportSpineHops_is_null_without_spine_and_the_hop_names_with_one() + { + var root = SalesModel(); + var from = Obj(root, "Purchase"); + Assert.Null(ReportShapes.ReportSpineHops(Obj(root, "PlainSales"), from, root)); + Assert.Equal(["program"], ReportShapes.ReportSpineHops(Obj(root, "SpineSales"), from, root)); + Assert.Equal(["program", "catalog"], ReportShapes.ReportSpineHops(Obj(root, "CatalogSales"), from, root)); + } + + [Fact] + public void ReportSpineHops_owner_may_be_an_entity_from_extends_resolved_in_the_reports_package() + { + var root = LoadSpineInheritedFixture(); + Assert.Equal(["program"], ReportShapes.ReportSpineHops(Obj(root, "ProgramMinutes"), Obj(root, "WorkoutEvent"), root)); + } + + [Theory] + [InlineData("Program.catalog")] + [InlineData("Purchase")] + [InlineData("Purchase..program")] + public void A_spine_that_does_not_resolve_throws_naming_the_report(string spine) + { + // A report built in code (never added to the root), past the loader, which refuses + // these under rule R8. A loaded tree is frozen, so it cannot be edited instead. + var root = SalesModel(); + var report = new MetaObject(new TypeId(TYPE_OBJECT, OBJECT_SUBTYPE_REPORT), "SpineSales"); + report.SetPackage("acme"); + report.SetAttr(OBJECT_REPORT_ATTR_FROM, "Purchase"); + report.SetAttr(OBJECT_REPORT_ATTR_SPINE, spine); + report.SetAttr(OBJECT_REPORT_ATTR_DIMENSIONS, new List<object?> { "programId" }); + report.SetAttr(OBJECT_REPORT_ATTR_MEASURES, new List<object?> { "purchases" }); + var ex = Assert.Throws<InvalidOperationException>(() => ReportShapes.ReportSpineHops(report, Obj(root, "Purchase"), root)); + Assert.Equal($"report 'SpineSales': @spine '{spine}' does not resolve.", ex.Message); + ex = Assert.Throws<InvalidOperationException>(() => ReportShapes.Of(report, root)); + Assert.Equal($"report 'SpineSales': @spine '{spine}' does not resolve.", ex.Message); + } } diff --git a/server/csharp/MetaObjects.Conformance.Tests/ReportingAccessorsTests.cs b/server/csharp/MetaObjects.Conformance.Tests/ReportingAccessorsTests.cs index de9524756..41fbfd086 100644 --- a/server/csharp/MetaObjects.Conformance.Tests/ReportingAccessorsTests.cs +++ b/server/csharp/MetaObjects.Conformance.Tests/ReportingAccessorsTests.cs @@ -27,6 +27,17 @@ private static LoadResult LoadPositiveFixture() private static MetaData Root(LoadResult r, string name) => r.Root.Children().Single(c => c.Name == name); + private static LoadResult LoadFixture(string fixture) + { + string path = Path.Combine(CorpusRoot.Path, fixture, "input", "meta.shop.json"); + var result = new MetaDataLoader().Load([new InMemoryStringSource(File.ReadAllText(path), id: "meta.shop.json")]); + Assert.Empty(result.Errors); + return result; + } + + private static MetaMeasure MeasureOf(LoadResult r, string entity, string name) => + Assert.IsType<MetaMeasure>(Root(r, entity).ChildByTypeAndName(TYPE_MEASURE, name)); + [Fact] public void ReportDimensionItems_parses_name_and_name_colon_grain() { @@ -115,6 +126,55 @@ public void Segment_filter_reads_the_desugared_row_scope() Assert.Equal("active", clause[FILTER_OP_EQ]); } + [Fact] + public void MeasureDefaultValue_is_the_declared_integer_zero_and_negatives_included_else_null() + { + // fixtures/conformance/reporting-spine-and-default: revenue @default 0, smallest + // @default -1, the ratio revenuePerPurchase @default 0, the count purchases none. + var r = LoadFixture("reporting-spine-and-default"); + Assert.Equal(0L, MeasureOf(r, "Purchase", "revenue").DefaultValue()); + Assert.Equal(-1L, MeasureOf(r, "Purchase", "smallest").DefaultValue()); + var ratio = MeasureOf(r, "Purchase", "revenuePerPurchase"); + Assert.True(ratio.IsRatio()); + Assert.Equal(0L, ratio.DefaultValue()); + Assert.Null(MeasureOf(r, "Purchase", "purchases").DefaultValue()); + Assert.Null(MeasureOf(r, "Purchase", "lastPurchaseAt").DefaultValue()); + } + + [Fact] + public void MeasureDefaultValue_of_a_measure_inherited_from_an_abstract_base_is_its_default() + { + // ADR-0039: resolving — WorkoutEvent reaches BaseEvent's totalMinutes through extends. + var r = LoadFixture("reporting-spine-inherited"); + var inherited = Root(r, "WorkoutEvent").Children() + .OfType<MetaMeasure>().Single(m => m.Name == "totalMinutes"); + Assert.Equal(0L, inherited.DefaultValue()); + } + + [Theory] + // A tree built in code, past the loader (whose attr.int check refuses every double). + [InlineData(0.0, 0L)] + [InlineData(-3.0, -3L)] + [InlineData(0.5, null)] + [InlineData(-1.5, null)] + public void MeasureDefaultValue_reads_an_integral_number_as_its_integer_and_a_fraction_as_null(double value, long? expected) + { + var measure = new MetaMeasure(new TypeId(TYPE_MEASURE, MEASURE_SUBTYPE_AGGREGATE), "m"); + measure.SetAttr(REPORTING_ATTR_DEFAULT, value); + Assert.Equal(expected, measure.DefaultValue()); + measure.SetAttr(REPORTING_ATTR_DEFAULT, "zero"); + Assert.Null(measure.DefaultValue()); + } + + [Fact] + public void ReportSpine_is_the_declared_path_and_null_when_the_report_declares_none() + { + var r = LoadFixture("reporting-spine-and-default"); + Assert.Equal("Purchase.program", ReportAccessors.ReportSpine(Root(r, "ProgramSales"))); + Assert.Equal("Purchase.program.catalog", ReportAccessors.ReportSpine(Root(r, "CatalogSales"))); + Assert.Null(ReportAccessors.ReportSpine(Root(r, "SalesByDay"))); + } + [Fact] public void Iso_duration_pattern_accepts_durations_and_refuses_the_degenerate_forms() { diff --git a/server/csharp/MetaObjects.Conformance.Tests/ReportingValidationTests.cs b/server/csharp/MetaObjects.Conformance.Tests/ReportingValidationTests.cs index a052b8857..de968741a 100644 --- a/server/csharp/MetaObjects.Conformance.Tests/ReportingValidationTests.cs +++ b/server/csharp/MetaObjects.Conformance.Tests/ReportingValidationTests.cs @@ -5,14 +5,19 @@ // // C# parity port of selected cases from the TS reference suite // server/typescript/packages/metadata/test/reporting-validation.test.ts -// (the "relative dates" and "inherited members" blocks). Every expected message is the -// TS text verbatim — the ports share one message contract. +// (the "relative dates", "inherited members", "@spine (R8, R9)" and "measure @default +// (M7, M8)" blocks). Every expected message is the TS text verbatim — the ports share one +// message contract. The one exception is a fractional measure @default: this port's +// generic attr.int type check already refuses it with ERR_BAD_ATTR_VALUE (its own text), +// so no second, reporting-specific error is added; those cases pin the code, the node and +// that it is the only error. using System.IO; using System.Linq; using System.Text.Json.Nodes; using MetaObjects.Loader; using MetaObjects.Meta; +using MetaObjects.Source; using Xunit; namespace MetaObjects.Conformance.Tests; @@ -57,6 +62,465 @@ private static string Single(LoadResult r, ErrorCode code) private static JsonNode Positive() => Fixture("reporting-vocabulary"); + private static JsonNode J(string json) => JsonNode.Parse(json)!; + + private static JsonNode Field(string subType, string name) => + Wrap($"field.{subType}", $$"""{ "name": "{{name}}" }"""); + + private static JsonNode Primary() => Wrap("identity.primary", """{ "name": "id", "@fields": ["id"] }"""); + + /// <summary>Set (or, with a null value, delete) attrs on a JSON body.</summary> + private static void Patch(JsonObject body, (string Key, JsonNode? Value)[] attrs) + { + foreach (var (key, value) in attrs) + { + if (value is null) body.Remove(key); + else body[key] = value; + } + } + + /// <summary>Patch the attrs of the root object (a report, say) named <paramref name="objName"/>.</summary> + private static void PatchObject(JsonNode doc, string objName, params (string Key, JsonNode? Value)[] attrs) => + Patch(ObjectBody(doc, objName), attrs); + + /// <summary>Patch the attrs of the dimension / measure / segment <paramref name="childName"/> of <paramref name="objName"/>.</summary> + private static void PatchMember(JsonNode doc, string objName, string childName, params (string Key, JsonNode? Value)[] attrs) + { + string[] memberTypes = [TYPE_DIMENSION, TYPE_MEASURE, TYPE_SEGMENT]; + var body = ChildrenOf(doc, objName) + .Select(w => w!.AsObject().First()) + .Single(kv => memberTypes.Contains(kv.Key.Split('.')[0]) && (string?)kv.Value!["name"] == childName) + .Value!.AsObject(); + Patch(body, attrs); + } + + private static string? JsonPathOf(MetaError e) => (e.Envelope as JsonSource)?.JsonPath; + + // ------------------------------------------------------------------------- + // R8 / R9 — a report's @spine (Table B of + // docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md) + // ------------------------------------------------------------------------- + + /// <summary>Append report <c>ProgramPurchases</c> over Purchase with <c>@spine: Purchase.program</c>; + /// <paramref name="attrs"/> override (a null value deletes).</summary> + private static void AddSpineReport(JsonNode doc, params (string Key, JsonNode? Value)[] attrs) + { + RootChildren(doc).Add(Wrap("object.report", """ + { "name": "ProgramPurchases", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programTitle"], "@measures": ["purchases", "revenue"] } + """)); + PatchObject(doc, "ProgramPurchases", attrs); + } + + /// <summary>Program -> Coach -> Agency, both to-one, plus Purchase dimensions at the end of each hop.</summary> + private static void AddCoachChain(JsonNode doc) + { + var program = ChildrenOf(doc, "Program"); + program.Add(Field("long", "coachId")); + program.Add(Wrap("identity.reference", """{ "name": "coachRef", "@references": "Coach", "@fields": ["coachId"] }""")); + program.Add(Wrap("relationship.association", """{ "name": "coach", "@objectRef": "Coach", "@cardinality": "one" }""")); + RootChildren(doc).Add(Wrap("object.entity", """ + { "name": "Coach", "children": [ + { "source.rdb": { "@table": "coaches" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "field.long": { "name": "agencyId" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "agencyRef", "@references": "Agency", "@fields": ["agencyId"] } }, + { "relationship.association": { "name": "agency", "@objectRef": "Agency", "@cardinality": "one" } } ] } + """)); + RootChildren(doc).Add(Wrap("object.entity", """ + { "name": "Agency", "children": [ + { "source.rdb": { "@table": "agencies" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } ] } + """)); + var purchase = ChildrenOf(doc, "Purchase"); + purchase.Add(Wrap("dimension.attribute", + """{ "name": "coachName", "@of": "Coach.name", "@via": "Purchase.program.coach" }""")); + purchase.Add(Wrap("dimension.attribute", + """{ "name": "agencyName", "@of": "Agency.name", "@via": "Purchase.program.coach.agency" }""")); + } + + /// <summary>A dimension of Purchase reaching Program.title through the identity.reference, not the relationship.</summary> + private static JsonNode ProgramTitleByRef() => + Wrap("dimension.attribute", """{ "name": "programTitleByRef", "@of": "Program.title", "@via": "Purchase.programRef" }"""); + + [Fact] + public void R8_a_spine_report_whose_dimensions_are_all_reached_through_the_spine_loads_clean() + { + var m = Positive(); + AddSpineReport(m); + Assert.Empty(Load(m).Errors); + } + + [Fact] + public void R8_a_to_many_spine_hop_is_refused_naming_the_report_and_the_hops_entity() + { + var m = Positive(); + ChildrenOf(m, "Program").Add(Wrap("measure.aggregate", + """{ "name": "programs", "@agg": "count", "@of": "Program.id" }""")); + RootChildren(m).Add(Wrap("object.report", """ + { "name": "ProgramReach", "@from": "Program", "@spine": "Program.purchases", "@measures": ["programs"] } + """)); + Assert.Equal( + "report 'acme::shop::ProgramReach': @spine 'Program.purchases' crosses relationship 'purchases' on " + + "'acme::shop::Program', which is not to-one. A @spine follows only @cardinality: one relationships " + + "and identity.reference hops, so each fact row joins at most one row of the spine entity and is never " + + "counted twice.", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void R8_a_spine_whose_owner_is_another_entity_is_refused() + { + var m = Positive(); + AddSpineReport(m, ("@spine", "Program.purchases")); + Assert.Equal( + "report 'acme::shop::ProgramPurchases': @spine 'Program.purchases' must start at @from 'acme::shop::Purchase'.", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void R8_a_spine_hop_that_names_nothing_is_refused_naming_from_as_the_hops_entity() + { + var m = Positive(); + AddSpineReport(m, ("@spine", "Purchase.nope")); + Assert.Equal( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase.nope' names 'nope', which is not a relationship " + + "or identity.reference of 'acme::shop::Purchase'.", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void R8_a_spine_with_no_hop_is_refused() + { + var m = Positive(); + AddSpineReport(m, ("@spine", "Purchase")); + Assert.Equal( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase' must be Owner.hop[.hop...], starting at " + + "@from 'acme::shop::Purchase'.", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void D2s_to_many_wording_is_unchanged_by_the_walks_wording_argument() + { + var m = Positive(); + ChildrenOf(m, "Program").Add(Wrap("dimension.attribute", + """{ "name": "buyerEmail", "@of": "Purchase.customerEmail", "@via": "Program.purchases" }""")); + Assert.Equal( + "dimension 'buyerEmail' on entity 'acme::shop::Program': @via 'Program.purchases' crosses relationship " + + "'purchases' on 'acme::shop::Program', which is not to-one. A dimension follows only @cardinality: one " + + "relationships and identity.reference hops, so grouping can never multiply the measured rows.", + Single(Load(m), ErrorCode.ERR_INVALID_DIMENSION)); + } + + [Theory] + [InlineData("Purchase", "@via 'Purchase' must be Owner.hop[.hop...], starting at the owning entity.")] + [InlineData("WorkoutEvent.program", "@via 'WorkoutEvent.program' must start at the owning entity 'acme::shop::Purchase'.")] + [InlineData("Purchase.nope", + "@via 'Purchase.nope' names 'nope', which is not a relationship or identity.reference of 'acme::shop::Purchase'.")] + public void D2s_own_wording_is_unchanged_by_the_walks_wording_argument(string via, string body) + { + var m = Positive(); + PatchMember(m, "Purchase", "programTitle", ("@via", via)); + Assert.Equal( + "dimension 'programTitle' on entity 'acme::shop::Purchase': " + body, + Single(Load(m), ErrorCode.ERR_INVALID_DIMENSION)); + } + + [Fact] + public void R8_the_error_source_is_the_report_node() + { + var m = Positive(); + AddSpineReport(m, ("@spine", "Purchase.nope")); + var errors = Load(m).Errors; + Assert.EndsWith("['object.report']", JsonPathOf(Assert.Single(errors))); + } + + [Fact] + public void R8_failing_skips_R9_a_broken_spine_with_an_off_spine_dimension_is_one_error() + { + var m = Positive(); + AddSpineReport(m, ("@spine", "Purchase.nope"), ("@dimensions", J("""["purchasedAt:day"]"""))); + Assert.Contains("@spine 'Purchase.nope' names 'nope'", Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void R8_an_inherited_spine_owner_written_as_the_abstract_base_loads_clean() + { + var m = Inherited(); + RootChildren(m).Add(Wrap("object.entity", """ + { "name": "Program", "children": [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } ] } + """)); + var baseEvent = ChildrenOf(m, "BaseEvent"); + baseEvent.Add(Field("long", "programId")); + baseEvent.Add(Wrap("identity.reference", """{ "name": "programRef", "@references": "Program", "@fields": ["programId"] }""")); + baseEvent.Add(Wrap("relationship.association", """{ "name": "program", "@objectRef": "Program", "@cardinality": "one" }""")); + baseEvent.Add(Wrap("dimension.attribute", """{ "name": "programTitle", "@of": "Program.title", "@via": "BaseEvent.program" }""")); + RootChildren(m).Add(Wrap("object.report", """ + { "name": "ProgramEvents", "@from": "WorkoutEvent", "@spine": "BaseEvent.program", + "@dimensions": ["programTitle"], "@measures": ["events"] } + """)); + Assert.Empty(Load(m).Errors); + } + + [Fact] + public void R9_a_spine_report_with_no_dimensions_is_refused() + { + var m = Positive(); + AddSpineReport(m, ("@dimensions", null)); + Assert.Equal( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase.program' needs at least one dimension. The " + + "report's rows are the dimension tuples of 'acme::shop::Program'; with no dimension it would be one " + + "totals row.", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void R9_a_dimension_with_no_via_read_from_the_fact_row_is_refused() + { + var m = Positive(); + AddSpineReport(m, ("@dimensions", J("""["programTitle", "program"]"""))); + Assert.Equal( + "report 'acme::shop::ProgramPurchases': dimension 'program' is read from @from 'acme::shop::Purchase', so " + + "it has no value in a row that has no facts. With @spine 'Purchase.program' every dimension must be " + + "reached through it: declare the dimension over a field of 'acme::shop::Program' (or an entity to-one " + + "from it) with an @via that begins 'Purchase.program'.", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void R9_a_time_dimension_over_a_fact_column_is_refused() + { + var m = Positive(); + AddSpineReport(m, ("@dimensions", J("""["programTitle", "purchasedAt:day"]"""))); + Assert.Contains( + "report 'acme::shop::ProgramPurchases': dimension 'purchasedAt' is read from @from", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void R9_each_offending_dimension_is_reported_once_even_when_listed_at_two_grains() + { + var m = Positive(); + AddSpineReport(m, ("@dimensions", J("""["program", "purchasedAt:day", "purchasedAt:week", "programTitle"]"""))); + var errors = Load(m).Errors; + Assert.Equal([ErrorCode.ERR_INVALID_REPORT, ErrorCode.ERR_INVALID_REPORT], errors.Select(e => e.Code)); + Assert.Contains("dimension 'program' is read from @from", errors[0].Message); + Assert.Contains("dimension 'purchasedAt' is read from @from", errors[1].Message); + } + + [Fact] + public void R9_a_dimension_through_a_second_reference_to_the_same_entity_is_refused() + { + var m = Positive(); + var purchase = ChildrenOf(m, "Purchase"); + purchase.Add(Field("long", "giftProgramId")); + purchase.Add(Wrap("identity.reference", + """{ "name": "giftProgramRef", "@references": "Program", "@fields": ["giftProgramId"] }""")); + purchase.Add(ProgramTitleByRef()); + purchase.Add(Wrap("dimension.attribute", + """{ "name": "giftProgramTitle", "@of": "Program.title", "@via": "Purchase.giftProgramRef" }""")); + AddSpineReport(m, ("@spine", "Purchase.programRef"), + ("@dimensions", J("""["programTitleByRef", "giftProgramTitle"]"""))); + Assert.Equal( + "report 'acme::shop::ProgramPurchases': dimension 'giftProgramTitle' is reached by @via " + + "'Purchase.giftProgramRef', which does not begin with the hops of @spine 'Purchase.programRef'. Hop names " + + "are compared as written: if both name the same join, write the same hops; otherwise the dimension is " + + "not reached through the spine.", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + [Fact] + public void R9_a_dimension_whose_own_via_fails_D2_is_reported_once_by_D2_not_also_by_R9() + { + var m = Positive(); + PatchMember(m, "Purchase", "programTitle", ("@via", "Purchase.programm")); + AddSpineReport(m); + Assert.Contains("@via 'Purchase.programm' names 'programm'", Single(Load(m), ErrorCode.ERR_INVALID_DIMENSION)); + } + + [Fact] + public void R9_the_same_join_named_by_the_relationship_in_spine_and_by_the_reference_in_via_is_refused() + { + var m = Positive(); + ChildrenOf(m, "Purchase").Add(ProgramTitleByRef()); + AddSpineReport(m, ("@spine", "Purchase.program"), ("@dimensions", J("""["programTitleByRef"]"""))); + string msg = Single(Load(m), ErrorCode.ERR_INVALID_REPORT); + Assert.Contains("dimension 'programTitleByRef' is reached by @via 'Purchase.programRef'", msg); + Assert.Contains("@spine 'Purchase.program'", msg); + } + + [Fact] + public void R9_the_owner_segment_is_not_compared_an_fqn_spine_owner_and_a_bare_dimension_via() + { + var m = Positive(); + AddSpineReport(m, ("@spine", "acme::shop::Purchase.program")); + Assert.Empty(Load(m).Errors); + } + + [Fact] + public void R9_a_time_dimension_over_a_column_of_the_spine_entity_is_legal() + { + var m = Positive(); + ChildrenOf(m, "Program").Add(Field("timestamp", "publishedAt")); + ChildrenOf(m, "Purchase").Add(Wrap("dimension.time", """ + { "name": "programPublishedAt", "@of": "Program.publishedAt", "@via": "Purchase.program", "@grains": ["month"] } + """)); + AddSpineReport(m, ("@dimensions", J("""["programTitle", "programPublishedAt:month"]"""))); + Assert.Empty(Load(m).Errors); + } + + [Fact] + public void R9_a_two_hop_spine_with_a_dimension_at_it_and_one_beyond_it_is_legal() + { + var m = Positive(); + AddCoachChain(m); + AddSpineReport(m, ("@spine", "Purchase.program.coach"), ("@dimensions", J("""["coachName", "agencyName"]"""))); + Assert.Empty(Load(m).Errors); + } + + [Fact] + public void R9_a_dimension_that_stops_short_of_a_two_hop_spine_is_refused() + { + var m = Positive(); + AddCoachChain(m); + AddSpineReport(m, ("@spine", "Purchase.program.coach"), ("@dimensions", J("""["coachName", "programTitle"]"""))); + Assert.Contains( + "dimension 'programTitle' is reached by @via 'Purchase.program'", + Single(Load(m), ErrorCode.ERR_INVALID_REPORT)); + } + + // ------------------------------------------------------------------------- + // M7 / M8 — where a measure's @default can apply (Table B) + // ------------------------------------------------------------------------- + + [Fact] + public void M7_default_on_a_count_is_refused_on_the_measure_node() + { + var m = Positive(); + PatchMember(m, "Purchase", "purchases", ("@default", 0)); + var errors = Load(m).Errors; + var e = Assert.Single(errors); + Assert.Equal(ErrorCode.ERR_INVALID_MEASURE, e.Code); + Assert.Equal( + "measure 'purchases' on entity 'acme::shop::Purchase': @default cannot apply to @agg: count. A count is " + + "never null (it is 0 when nothing matches); remove @default.", + e.Message); + Assert.Contains("['measure.aggregate']", JsonPathOf(e)); + } + + [Fact] + public void M7_default_on_a_distinct_count_of_a_tuple_is_refused() + { + var m = Positive(); + PatchMember(m, "WorkoutEvent", "daysEngaged", ("@default", 0)); + string msg = Single(Load(m), ErrorCode.ERR_INVALID_MEASURE); + Assert.Contains("measure 'daysEngaged'", msg); + Assert.Contains("@default cannot apply to @agg: count", msg); + } + + [Fact] + public void M8_default_on_a_max_of_a_timestamp_is_refused() + { + var m = Positive(); + PatchMember(m, "WorkoutEvent", "lastActivityAt", ("@default", 0)); + Assert.Equal( + "measure 'lastActivityAt' on entity 'acme::shop::WorkoutEvent': @default is a number, but " + + "'WorkoutEvent.occurredAt', the @of of @agg 'max', is a field.timestamp. A default is supported on " + + "numeric measures only.", + Single(Load(m), ErrorCode.ERR_INVALID_MEASURE)); + } + + [Fact] + public void A_non_integer_default_on_a_count_reports_M7_and_the_attribute_type_error() + { + var m = Positive(); + PatchMember(m, "Purchase", "purchases", ("@default", "zero")); + var errors = Load(m).Errors; + Assert.Equal( + [ErrorCode.ERR_BAD_ATTR_VALUE, ErrorCode.ERR_INVALID_MEASURE], + errors.Select(e => e.Code).OrderBy(c => c.ToString(), System.StringComparer.Ordinal)); + Assert.Contains( + "measure 'purchases' on entity 'acme::shop::Purchase': @default cannot apply to @agg: count.", + errors.Single(e => e.Code == ErrorCode.ERR_INVALID_MEASURE).Message); + } + + [Theory] + // entity, measure, the fraction, the measure's wrapper key + [InlineData("Purchase", "revenue", 0.5, "measure.aggregate")] // a sum + [InlineData("WorkoutEvent", "avgDaysPerStarter", 0.5, "measure.ratio")] // a ratio + [InlineData("Purchase", "purchases", 0.5, "measure.aggregate")] // a count: M7 is skipped + [InlineData("WorkoutEvent", "lastActivityAt", -1.5, "measure.aggregate")] // a max of a timestamp: M8 is skipped + public void A_fractional_default_is_ERR_BAD_ATTR_VALUE_on_the_measure_node_and_the_only_error( + string entity, string measure, double value, string wrapper) + { + var m = Positive(); + PatchMember(m, entity, measure, ("@default", value)); + var e = Assert.Single(Load(m).Errors); + Assert.Equal(ErrorCode.ERR_BAD_ATTR_VALUE, e.Code); + Assert.Contains("'@default'", e.Message); + Assert.Contains($"['{wrapper}']", JsonPathOf(e)); + } + + [Fact] + public void A_fractional_default_on_a_sum_declared_on_an_abstract_base_is_reported_once() + { + var m = Inherited(); + ReplaceChild(m, "BaseEvent", "measure.aggregate", "events", + """{ "name": "events", "@agg": "sum", "@of": "BaseEvent.id", "@default": 0.5 }"""); + Assert.Contains("'@default'", Single(Load(m), ErrorCode.ERR_BAD_ATTR_VALUE)); + } + + [Fact] + public void Default_on_a_sum_an_avg_a_min_of_an_int_and_a_ratio_is_fine() + { + var m = Positive(); + PatchMember(m, "Purchase", "revenue", ("@default", 0)); + ChildrenOf(m, "Purchase").Add(Wrap("measure.aggregate", + """{ "name": "avgRevenue", "@agg": "avg", "@of": "Purchase.amountCents", "@default": 0 }""")); + ChildrenOf(m, "WorkoutEvent").Add(Wrap("measure.aggregate", + """{ "name": "firstDay", "@agg": "min", "@of": "WorkoutEvent.dayNumber", "@default": 1 }""")); + PatchMember(m, "WorkoutEvent", "avgDaysPerStarter", ("@default", 0)); + Assert.Empty(Load(m).Errors); + } + + [Fact] + public void A_measure_that_breaks_M4_and_declares_default_reports_M4_only() + { + var m = Positive(); + PatchMember(m, "Purchase", "revenue", ("@of", "Purchase.status"), ("@default", 0)); + string msg = Single(Load(m), ErrorCode.ERR_INVALID_MEASURE); + Assert.Contains("field.string", msg); + Assert.DoesNotContain("@default", msg); + } + + [Fact] + public void A_measure_that_breaks_M1_and_declares_default_on_a_count_reports_M1_only() + { + var m = Positive(); + PatchMember(m, "Purchase", "purchases", ("@of", "Purchase.nope"), ("@default", 0)); + string msg = Single(Load(m), ErrorCode.ERR_INVALID_MEASURE); + Assert.Contains("names no field 'nope'", msg); + Assert.DoesNotContain("@default", msg); + } + + [Fact] + public void M7_on_a_count_declared_on_an_abstract_base_is_reported_once() + { + var m = Inherited(); + ReplaceChild(m, "BaseEvent", "measure.aggregate", "events", + """{ "name": "events", "@agg": "count", "@of": "BaseEvent.id", "@default": 0 }"""); + Assert.Contains("measure 'events' on entity 'acme::shop::BaseEvent'", Single(Load(m), ErrorCode.ERR_INVALID_MEASURE)); + } + private static JsonNode Inherited() => Fixture("reporting-inherited-members"); // ------------------------------------------------------------------------- diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs index 8005db77e..57460544c 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs @@ -3,7 +3,8 @@ // Drives the fixtures/api-contract-conformance/report/ scenarios over HTTP against the // GENERATED report routes (the deployed artifact): the emitted <Report>Routes booted // unmodified on Kestrel against a Postgres testcontainer in which the committed -// TypeScript-produced schema has created the `invoices` table and the three views. +// TypeScript-produced schema has created the base tables (`invoices`, `products`, `sales`) +// and the four views. // // Generated lane ONLY, on purpose and on every port (see the subcorpus README). What is // under test is whether the port's GENERATOR emits a read route for a view-backed report diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs index 609786a8f..6643de74e 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs @@ -3,15 +3,17 @@ // // The schema is NOT written here. View SQL is produced by TypeScript only (ADR-0015), so // this lane executes the committed, TypeScript-produced, drift-checked artifact -// `report/schema.postgres.sql` verbatim: the `invoices` table and the three views the -// served reports read. That artifact carries LITERAL column naming (amountCents, -// issuedOnMonth), which is the strategy this lane generates with. +// `report/schema.postgres.sql` verbatim: the base tables (`invoices`, `products`, `sales`) +// and the views the served reports read. That artifact carries LITERAL column naming +// (amountCents, issuedOnMonth), which is the strategy this lane generates with, so a seed +// row's keys are its column names. // -// The seed is `report/seed.json`. Only its `invoices` half is inserted: the views derive -// the report rows from it. Its `reports` half is what the seam lanes (Java, Kotlin, Python) -// serve in place of the views, and is not used here. +// The seed is `report/seed.json`. Every top-level key but `reports` is a base table, listed +// parents first (`products` before `sales`, which references it), and is inserted in file +// order so every foreign key holds: the views derive the report rows from them. `reports` +// is what the seam lanes (Java, Kotlin, Python) serve in place of the views, and is not +// used here. -using System.Globalization; using System.Text.Json; // JsonValueKind using System.Text.Json.Nodes; // JsonNode / JsonObject / JsonArray using Npgsql; @@ -20,10 +22,8 @@ namespace MetaObjects.IntegrationTests.Api; internal static class ReportFixture { - private static readonly string[] InvoiceCols = { "id", "reference", "status", "amountCents", "issuedOn" }; - - // The one DATE column: bound as a DateOnly so Npgsql sends a date, not text. - private const string DateCol = "issuedOn"; + /// <summary>The one top-level key of seed.json that is not a base table.</summary> + private const string SeedReportsKey = "reports"; /// <summary>Execute the committed TypeScript-produced schema on a fresh container.</summary> public static async Task ProvisionSchemaAsync(string connString) @@ -36,43 +36,61 @@ public static async Task ProvisionSchemaAsync(string connString) } /// <summary> - /// Insert the seed's `invoices`. The views are never seeded — they derive from - /// `invoices`, which is what makes this a full-stack lane. + /// Insert every base table of the seed, in file order (parents first). The views are + /// never seeded — they derive from the base tables, which is what makes this a + /// full-stack lane. Values are SQL literals, as the TypeScript lane writes them, so + /// Postgres types each one by its column (`'2026-04-30'` into a DATE). /// </summary> public static async Task ApplySeedAsync(string connString, string seedPath) { var root = JsonNode.Parse(File.ReadAllText(seedPath)) as JsonObject ?? throw new InvalidOperationException($"{seedPath}: top-level must be an object"); - if (root["invoices"] is not JsonArray rows || rows.Count == 0) - throw new InvalidOperationException($"{seedPath}: no `invoices` rows to seed"); + var tables = root.Where(kv => kv.Key != SeedReportsKey).ToList(); + if (tables.Count == 0) + throw new InvalidOperationException($"{seedPath}: no base-table rows to seed"); await using var c = new NpgsqlConnection(connString); await c.OpenAsync(); - var colList = string.Join(", ", InvoiceCols.Select(col => "\"" + col + "\"")); - var paramList = string.Join(", ", InvoiceCols.Select((_, i) => "@p" + i)); - foreach (var rowNode in rows) + foreach (var (table, rowsNode) in tables) { - if (rowNode is not JsonObject row) continue; - await using var ins = c.CreateCommand(); - ins.CommandText = $"INSERT INTO \"invoices\" ({colList}) VALUES ({paramList})"; - for (int i = 0; i < InvoiceCols.Length; i++) + if (rowsNode is not JsonArray rows) + throw new InvalidOperationException($"{seedPath}: '{table}' is not an array of rows"); + foreach (var rowNode in rows) + { + if (rowNode is not JsonObject row) + throw new InvalidOperationException($"{seedPath}: '{table}' has a row that is not an object"); + var colList = string.Join(", ", row.Select(kv => "\"" + kv.Key + "\"")); + var valueList = string.Join(", ", row.Select(kv => SqlLiteral(seedPath, table, kv.Key, kv.Value))); + await using var ins = c.CreateCommand(); + ins.CommandText = $"INSERT INTO \"{table}\" ({colList}) VALUES ({valueList})"; + await ins.ExecuteNonQueryAsync(); + } + + // The seed writes explicit ids; move each identity past them. + if (rows.Any(r => r is JsonObject o && o.ContainsKey("id"))) { - var v = row[InvoiceCols[i]]; - object val = v is null ? DBNull.Value - : InvoiceCols[i] == DateCol - ? DateOnly.ParseExact(v.GetValue<string>(), "yyyy-MM-dd", CultureInfo.InvariantCulture) - : v.GetValueKind() == JsonValueKind.Number ? v.GetValue<long>() - : v.GetValue<string>(); - ins.Parameters.AddWithValue("@p" + i, val); + await using var bump = c.CreateCommand(); + bump.CommandText = + $"SELECT setval(pg_get_serial_sequence('{table}', 'id'), " + + $"COALESCE((SELECT MAX(id) FROM \"{table}\"), 1))"; + await bump.ExecuteScalarAsync(); } - await ins.ExecuteNonQueryAsync(); } + } - await using var bump = c.CreateCommand(); - bump.CommandText = - "SELECT setval(pg_get_serial_sequence('invoices', 'id'), " + - "COALESCE((SELECT MAX(id) FROM \"invoices\"), 1))"; - await bump.ExecuteScalarAsync(); + /// <summary>A seed value as a SQL literal: NULL, a JSON number as written, TRUE/FALSE, or a quoted string.</summary> + private static string SqlLiteral(string seedPath, string table, string column, JsonNode? v) + { + if (v is null) return "NULL"; + return v.GetValueKind() switch + { + JsonValueKind.Number => v.ToJsonString(), + JsonValueKind.True => "TRUE", + JsonValueKind.False => "FALSE", + JsonValueKind.String => "'" + v.GetValue<string>().Replace("'", "''") + "'", + _ => throw new InvalidOperationException( + $"{seedPath}: '{table}.{column}' is not a scalar: {v.ToJsonString()}"), + }; } } diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ReportGeneratedServerFactory.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ReportGeneratedServerFactory.cs index 57edb9f40..5606beff3 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ReportGeneratedServerFactory.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ReportGeneratedServerFactory.cs @@ -2,16 +2,17 @@ // corpus. // // Runs the real MetaObjects.Codegen generators (Entity + DbContext + FilterAllowlist -// + Routes + Names) on the report model (the Invoice table, three view-backed reports and -// one sourceless report), Roslyn-compiles the emitted sources in-memory, and hosts them on -// Kestrel against Testcontainers Postgres with the three views present. A failing scenario -// is a real generator bug (RoutesGenerator / DbContextGenerator / FilterAllowlistGenerator -// / ReportRows), never something fixed by hand-editing emitted code. +// + Routes + Names) on the report model (the Invoice, Product and Sale tables, four +// view-backed reports — one of them, ProductRevenue, a @spine report with a defaulted +// measure — and one sourceless report), Roslyn-compiles the emitted sources in-memory, and +// hosts them on Kestrel against Testcontainers Postgres with the four views present. A +// failing scenario is a real generator bug (RoutesGenerator / DbContextGenerator / +// FilterAllowlistGenerator / ReportRows), never something fixed by hand-editing emitted code. // // Mirrors ProjectionGeneratedServerFactory. The routes to mount are chosen by -// RoutesGenerator.AppliesTo over the DECLARED nodes, and must be exactly Invoice and the -// three served reports: the sourceless InvoiceDays sits in the model so that a generator -// which serves every report it finds fails here. +// RoutesGenerator.AppliesTo over the DECLARED nodes, and must be exactly the three entities +// and the four served reports: the sourceless InvoiceDays sits in the model so that a +// generator which serves every report it finds fails here. // // No JSON options are configured on the host. The date wire format under test // (`issuedOnMonth` as YYYY-MM-DD) is what the generated row's DateOnly property gives by @@ -39,7 +40,7 @@ internal sealed class ReportGeneratedServerFactory : IAsyncDisposable // Ordinal order. InvoiceDays is in the model and declares no source. private static readonly string[] ExpectedRoutedNames = - ["Invoice", "InvoiceStatusTotals", "InvoiceTotals", "InvoicesByMonth"]; + ["Invoice", "InvoiceStatusTotals", "InvoiceTotals", "InvoicesByMonth", "Product", "ProductRevenue", "Sale"]; private const string SourcelessReport = "InvoiceDays"; private readonly PostgresContainer _pg; @@ -78,9 +79,9 @@ public static async Task<ReportGeneratedServerFactory> StartAsync(PostgresContai var app = builder.Build(); - // Mount every generated Map<Name>Routes(app, "/api") — Invoice's writable set and - // the three served reports' read-only ones. Invoice is mounted so the lane also - // proves the two coexist; only the reports' routes carry scenarios. + // Mount every generated Map<Name>Routes(app, "/api") — the three entities' writable + // sets and the four served reports' read-only ones. The entities are mounted so the + // lane also proves the two coexist; only the reports' routes carry scenarios. foreach (var name in routedNames) { var routesType = assembly.GetType($"{GeneratedNamespace}.{name}Routes") @@ -112,9 +113,9 @@ private static (Assembly Assembly, IReadOnlyList<string> RoutedNames) CompileGen string.Join("; ", loadResult.Errors.Select(e => e.ToString()))); var root = loadResult.Root; - // Asked of the DECLARED nodes: Invoice and the three served reports, and not the - // sourceless InvoiceDays. Exactly these, so a generator that serves every report it - // finds (or none) fails here by name instead of as twelve unexplained 404s. + // Asked of the DECLARED nodes: the three entities and the four served reports, and not + // the sourceless InvoiceDays. Exactly these, so a generator that serves every report it + // finds (or none) fails here by name instead of as a run of unexplained 404s. var routedNames = root.Objects() .Where(o => RoutesGenerator.AppliesTo(o, root)) .Select(o => CSharpNaming.Pascal(o.Name)) @@ -135,7 +136,7 @@ private static (Assembly Assembly, IReadOnlyList<string> RoutedNames) CompileGen Namespace = GeneratedNamespace, ColumnNamingStrategy = ColumnNamingStrategy.Literal, EmitAbstractShapes = false, - // Invoice binds through its names artifact. A report has none (it binds its + // The entities bind through their names artifacts. A report has none (it binds its // view and columns by literal), so this also proves no generated report file // references a <Report>Names class that was never emitted. IncludeNames = true, diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs index 414ad1e5a..5acfa3fec 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/AppDbContext.g.cs @@ -14,6 +14,7 @@ public AppDbContext(DbContextOptions<AppDbContext> options) : base(options) { } public DbSet<AssetActivity> AssetActivities { get; set; } = default!; public DbSet<Auth> Auths { get; set; } = default!; public DbSet<FitnessTotals> FitnessTotals { get; set; } = default!; + public DbSet<FitnessTotalsFilled> FitnessTotalsFilleds { get; set; } = default!; public DbSet<Follow> Follows { get; set; } = default!; public DbSet<Friendship> Friendships { get; set; } = default!; public DbSet<Measurement> Measurements { get; set; } = default!; @@ -23,7 +24,9 @@ public AppDbContext(DbContextOptions<AppDbContext> options) : base(options) { } public DbSet<PostReferral> PostReferrals { get; set; } = default!; public DbSet<PostTag> PostTags { get; set; } = default!; public DbSet<Program> Programs { get; set; } = default!; + public DbSet<ProgramLongWeeks> ProgramLongWeeks { get; set; } = default!; public DbSet<ProgramMinutes> ProgramMinutes { get; set; } = default!; + public DbSet<ProgramRoster> ProgramRosters { get; set; } = default!; public DbSet<ProgramStat> ProgramStats { get; set; } = default!; public DbSet<ProgramView> ProgramViews { get; set; } = default!; public DbSet<ProgramsByMonth> ProgramsByMonths { get; set; } = default!; @@ -36,7 +39,10 @@ protected override void OnModelCreating(ModelBuilder modelBuilder) { modelBuilder.Entity<AssetActivity>().HasNoKey().ToView("v_asset_activity"); modelBuilder.Entity<FitnessTotals>().HasNoKey().ToView("v_fitness_totals"); + modelBuilder.Entity<FitnessTotalsFilled>().HasNoKey().ToView("v_fitness_totals_filled"); + modelBuilder.Entity<ProgramLongWeeks>().HasNoKey().ToView("v_program_long_weeks"); modelBuilder.Entity<ProgramMinutes>().HasNoKey().ToView("v_program_minutes"); + modelBuilder.Entity<ProgramRoster>().HasNoKey().ToView("v_program_roster"); modelBuilder.Entity<ProgramStat>().ToView(ProgramStatNames.SourcePrimaryView); modelBuilder.Entity<ProgramView>().ToView(ProgramViewNames.SourcePrimaryView); modelBuilder.Entity<ProgramView>().Property(x => x.Status).HasConversion<string>(); diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/FitnessTotalsFilled.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/FitnessTotalsFilled.g.cs new file mode 100644 index 000000000..3f9d9007f --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/FitnessTotalsFilled.g.cs @@ -0,0 +1,19 @@ +// <auto-generated/> +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class FitnessTotalsFilled +{ + [Column("weeks")] + public long Weeks { get; set; } + [Column("totalMinutesOrZero")] + public long TotalMinutesOrZero { get; set; } + [Column("longShareOrZero")] + public decimal LongShareOrZero { get; set; } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramLongWeeks.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramLongWeeks.g.cs new file mode 100644 index 000000000..750a0ae89 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramLongWeeks.g.cs @@ -0,0 +1,19 @@ +// <auto-generated/> +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class ProgramLongWeeks +{ + [Column("programKey")] + public long ProgramKey { get; set; } + [Column("weeks")] + public long Weeks { get; set; } + [Column("totalMinutesOrZero")] + public long TotalMinutesOrZero { get; set; } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramRoster.g.cs b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramRoster.g.cs new file mode 100644 index 000000000..615a97010 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Generated/ProgramRoster.g.cs @@ -0,0 +1,30 @@ +// <auto-generated/> +// Generated by MetaObjects entity-generator. Do not edit by hand. +#nullable enable +using System; +using System.Collections.Generic; +using System.ComponentModel.DataAnnotations; +using System.ComponentModel.DataAnnotations.Schema; + +namespace MetaObjects.IntegrationTests.Generated; + +public class ProgramRoster +{ + [Column("programKey")] + public long ProgramKey { get; set; } + [Column("programTitle")] + [Required(AllowEmptyStrings = true)] + [MaxLength(200)] + [MinLength(1)] + public string ProgramTitle { get; set; } = default!; + [Column("weeks")] + public long Weeks { get; set; } + [Column("totalMinutes")] + public long? TotalMinutes { get; set; } + [Column("totalMinutesOrZero")] + public long TotalMinutesOrZero { get; set; } + [Column("longShare")] + public decimal? LongShare { get; set; } + [Column("longShareOrZero")] + public decimal LongShareOrZero { get; set; } +} diff --git a/server/csharp/MetaObjects/Core/Object/ObjectConstants.cs b/server/csharp/MetaObjects/Core/Object/ObjectConstants.cs index d32203d23..1bfe38489 100644 --- a/server/csharp/MetaObjects/Core/Object/ObjectConstants.cs +++ b/server/csharp/MetaObjects/Core/Object/ObjectConstants.cs @@ -79,6 +79,9 @@ public static class ObjectConstants /// <summary>Optional row scope over @from's fields (an attr.filter).</summary> public const string OBJECT_REPORT_ATTR_FILTER = "filter"; + /// <summary>Optional to-one path from @from to the entity whose rows supply the report's rows (R8).</summary> + public const string OBJECT_REPORT_ATTR_SPINE = "spine"; + public static readonly string[] OBJECT_SUBTYPES = [ BaseTypes.SUBTYPE_BASE, diff --git a/server/csharp/MetaObjects/Core/Reporting/ReportAccessors.cs b/server/csharp/MetaObjects/Core/Reporting/ReportAccessors.cs index 8226e81f5..72ae8eeea 100644 --- a/server/csharp/MetaObjects/Core/Reporting/ReportAccessors.cs +++ b/server/csharp/MetaObjects/Core/Reporting/ReportAccessors.cs @@ -35,6 +35,10 @@ public static class ReportAccessors /// <summary>The <c>@from</c> entity name of a report.</summary> public static string? ReportFrom(MetaData obj) => obj.Attr(OBJECT_REPORT_ATTR_FROM) as string; + /// <summary><c>@spine</c>: the to-one path to the entity whose rows supply the report's rows; null when absent or empty.</summary> + public static string? ReportSpine(MetaData obj) => + obj.Attr(OBJECT_REPORT_ATTR_SPINE) is string s && s != "" ? s : null; + /// <summary>The <c>@dimensions</c> items, each <c>name</c> or <c>name:grain</c> (split at the first <c>:</c>).</summary> public static IReadOnlyList<ReportDimensionItem> ReportDimensionItems(MetaData obj) => ReportingValues.StringList(obj.Attr(OBJECT_REPORT_ATTR_DIMENSIONS)) diff --git a/server/csharp/MetaObjects/Core/Reporting/ReportShape.cs b/server/csharp/MetaObjects/Core/Reporting/ReportShape.cs index a5ebf7300..901d00c4c 100644 --- a/server/csharp/MetaObjects/Core/Reporting/ReportShape.cs +++ b/server/csharp/MetaObjects/Core/Reporting/ReportShape.cs @@ -1,5 +1,7 @@ // A report's derived fields (FR-044, contract Table B): one field per `@dimensions` item -// in listed order, then one per `@measures` item in listed order. +// in listed order, then one per `@measures` item in listed order. `required` follows +// Table C of docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md, +// which amends Table B for a report with `@spine` and a measure with `@default`. // // Ported rule for rule from // server/typescript/packages/metadata/src/core/reporting/report-shape.ts, and gated @@ -80,7 +82,16 @@ public static class ReportShapes /// Null when any step fails. /// </summary> public static MetaField? ResolveReportingFieldRef( - string reference, MetaData declaring, MetaRoot root, MetaObject? host = null) + string reference, MetaData declaring, MetaRoot root, MetaObject? host = null) => + ResolveReportingFieldRefNamed(reference, declaring, root, host)?.Field; + + /// <summary> + /// <see cref="ResolveReportingFieldRef"/>, also returning <c>Named</c>: the entity the + /// reference's entity half names (which, with <paramref name="host"/>, may be an ancestor + /// of the entity the field is read from). Table C reads <c>Named</c>'s primary identity. + /// </summary> + private static (MetaObject Named, MetaField Field)? ResolveReportingFieldRefNamed( + string reference, MetaData declaring, MetaRoot root, MetaObject? host) { // `Entity.field`; a package qualifier uses `::`, so the member separator is the LAST dot. int dot = reference.LastIndexOf(CHILD_REF_SEPARATOR, StringComparison.Ordinal); @@ -89,26 +100,98 @@ public static class ReportShapes is not MetaObject named) return null; if (host is not null && !ValidationPasses.IsSelfOrAncestor(named, host)) return null; // ADR-0039: resolving, so a field inherited through extends is found. - return (host ?? named).FindField(reference[(dot + CHILD_REF_SEPARATOR.Length)..]); + var field = (host ?? named).FindField(reference[(dot + CHILD_REF_SEPARATOR.Length)..]); + return field is null ? null : (named, field); + } + + /// <summary> + /// The hop names of a dimension's <c>@via</c> (<c>Owner.hop[.hop...]</c>), read as the + /// loader reads it (<c>ValidateReporting</c> rule D2): <c>Owner</c> resolves in the package + /// of <paramref name="declaring"/> (<see cref="ReportingMemberOwner"/>) and must be + /// <paramref name="from"/> or an entity <paramref name="from"/> extends. The walk itself + /// then starts AT <paramref name="from"/>, whichever of the two <c>Owner</c> named. Null + /// when the reference has no owner, no hop, or an owner that is not <paramref name="from"/> + /// or an ancestor of it. + /// </summary> + public static string[]? ReportingViaHops(string via, MetaData declaring, MetaObject from, MetaRoot root) + { + // The owner ends at the first `.` after the last `::` (a package qualifier has no `.`). + int lastSep = via.LastIndexOf(PACKAGE_SEPARATOR, StringComparison.Ordinal); + int segStart = lastSep == -1 ? 0 : lastSep + PACKAGE_SEPARATOR.Length; + int dot = via.IndexOf(CHILD_REF_SEPARATOR, segStart, StringComparison.Ordinal); + if (dot <= segStart) return null; + string[] hops = via[(dot + CHILD_REF_SEPARATOR.Length)..].Split(CHILD_REF_SEPARATOR); + if (hops.Any(h => h == "")) return null; + var owner = NamingRefs.ResolveObjectRef(root, via[..dot], NamingRefs.EffectivePackage(declaring)); + if (owner is null || !ValidationPasses.IsSelfOrAncestor(owner, from)) return null; + return hops; } private static InvalidOperationException Unresolved(string reportName, string what) => new($"report '{reportName}': {what} does not resolve."); + /// <summary> + /// The hop names of a report's <c>@spine</c> (<c>Owner.hop[.hop...]</c>), read as + /// <see cref="ReportingViaHops"/> reads a <c>@via</c>, the owner resolving in the REPORT's + /// package (loader rule R8). Null when the report declares no <c>@spine</c>. Throws + /// <see cref="InvalidOperationException"/> naming the report when a declared <c>@spine</c> + /// does not resolve (a report that passed <c>ValidateReporting</c> always resolves): + /// reading it as "no spine" would claim a column non-null that a spine row with no facts + /// leaves null. + /// </summary> + public static string[]? ReportSpineHops(MetaObject report, MetaObject from, MetaRoot root) + { + string? spine = ReportAccessors.ReportSpine(report); + if (spine is null) return null; + return ReportingViaHops(spine, report, from, root) ?? throw Unresolved(report.Name, $"@spine '{spine}'"); + } + + /// <summary>True when <paramref name="field"/> is one of the <c>@fields</c> of <paramref name="entity"/>'s <c>identity.primary</c>.</summary> + private static bool IsPrimaryKeyField(MetaObject entity, MetaField field) + { + // ADR-0039: resolving — an identity.primary (and its @fields) inherited from an + // abstract base counts. + var pk = entity.PrimaryIdentity(); + return pk is not null && (ValidationPasses.IdentityEffectiveFields(pk) ?? []).Contains(field.Name); + } + + /// <summary> + /// Table C (<c>required</c> of a dimension). Without <c>@spine</c>: only a dimension with + /// no <c>@via</c> over an <c>@of</c> field whose effective <c>@required</c> is true. With + /// <c>@spine</c>: only a dimension whose <c>@via</c> hops equal the spine's (a column of + /// the spine entity itself, whose rows are the report's rows) over an <c>@of</c> field that + /// is <c>@required</c> or a primary-key column of the entity <c>@of</c> names. A dimension + /// beyond the spine is reached by a LEFT OUTER join. + /// </summary> + private static bool DimensionRequired( + MetaDimension dim, MetaObject named, MetaField of, MetaObject from, MetaRoot root, + string[]? spine, string reportName) + { + string? via = dim.Via(); + // The @required ATTR only, read resolving (ADR-0039); a validator.required child does not count. + bool ofRequired = of.Attr(FIELD_ATTR_REQUIRED) is true; + if (spine is null) return via is null && ofRequired; + if (via is null) return false; // a column of @from: null in a spine row with no facts + var hops = ReportingViaHops(via, ReportingMemberOwner(dim, from), from, root) + ?? throw Unresolved(reportName, $"dimension '{dim.Name}' @via '{via}'"); + bool onSpine = hops.SequenceEqual(spine, StringComparer.Ordinal); + return onSpine && (ofRequired || IsPrimaryKeyField(named, of)); + } + private static T? DeclaredMember<T>(MetaObject from, string type, string name) where T : MetaData => // ADR-0039: resolving Children(), so a member declared on an abstract base is found. from.Children().OfType<T>().FirstOrDefault(c => c.Type == type && c.Name == name); - private static ReportField DimensionField(ReportDimensionItem item, MetaObject from, MetaRoot root, string reportName) + private static ReportField DimensionField( + ReportDimensionItem item, MetaObject from, MetaRoot root, string reportName, string[]? spine) { var dim = DeclaredMember<MetaDimension>(from, TYPE_DIMENSION, item.Name) ?? throw Unresolved(reportName, $"dimension '{item.Name}' on '{from.Name}'"); bool vialess = dim.Via() is null; - var of = ResolveReportingFieldRef(dim.Of() ?? "", ReportingMemberOwner(dim, from), root, vialess ? from : null) + var (named, of) = ResolveReportingFieldRefNamed(dim.Of() ?? "", ReportingMemberOwner(dim, from), root, vialess ? from : null) ?? throw Unresolved(reportName, $"dimension '{item.Name}' @of"); string name = ReportAccessors.ReportDerivedFieldName(item); - // The @required ATTR only, read resolving (ADR-0039); a validator.required child does not count. - bool required = vialess && of.Attr(FIELD_ATTR_REQUIRED) is true; + bool required = DimensionRequired(dim, named, of, from, root, spine, reportName); if (dim.IsTime()) { // Loader rule R2 guarantees a grain from the closed set; a tree built in code does not. @@ -140,45 +223,48 @@ private static ReportField MeasureField(string item, MetaObject report, MetaObje } var m = DeclaredMember<MetaMeasure>(from, TYPE_MEASURE, name) ?? throw Unresolved(reportName, $"measure '{item}' on '{from.Name}'"); - if (m.IsRatio()) - return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_DECIMAL, false, Measure: m); string? agg = m.Agg(); + // Table C: a count is never null; any other measure is not null when it has a @default. + bool required = (!m.IsRatio() && agg == AGG_COUNT) || m.DefaultValue() is not null; + if (m.IsRatio()) + return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_DECIMAL, required, Measure: m); if (agg == AGG_COUNT) - return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_LONG, true, Measure: m); + return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_LONG, required, Measure: m); var of = ResolveReportingFieldRef(m.OfColumns().FirstOrDefault() ?? "", ReportingMemberOwner(m, from), root, from) ?? throw Unresolved(reportName, $"measure '{name}' @of"); string src = of.SubType; if (agg == AGG_SUM) { if (src == FIELD_SUBTYPE_CURRENCY) - return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_CURRENCY, false, of, Measure: m); + return new ReportField(name, ReportFieldRole.Measure, FIELD_SUBTYPE_CURRENCY, required, of, Measure: m); string sumType = SumLong.Contains(src) ? FIELD_SUBTYPE_LONG : Floating.Contains(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL; - return new ReportField(name, ReportFieldRole.Measure, sumType, false, Measure: m); + return new ReportField(name, ReportFieldRole.Measure, sumType, required, Measure: m); } if (agg == AGG_AVG) { string avgType = Floating.Contains(src) ? FIELD_SUBTYPE_DOUBLE : FIELD_SUBTYPE_DECIMAL; - return new ReportField(name, ReportFieldRole.Measure, avgType, false, Measure: m); + return new ReportField(name, ReportFieldRole.Measure, avgType, required, Measure: m); } // min / max keep the source field's type. - return new ReportField(name, ReportFieldRole.Measure, src, false, of, Measure: m); + return new ReportField(name, ReportFieldRole.Measure, src, required, of, Measure: m); } /// <summary> - /// Table B. Throws <see cref="InvalidOperationException"/> naming the report when a - /// reference does not resolve (a report that passed the loader's report validation - /// always resolves). + /// Table B, with Table C's <c>required</c>. Throws <see cref="InvalidOperationException"/> + /// naming the report when a reference does not resolve (a report that passed the loader's + /// report validation always resolves). /// </summary> public static ReportShape Of(MetaObject report, MetaRoot root) { string fromName = ReportAccessors.ReportFrom(report) ?? throw Unresolved(report.Name, "@from"); var from = NamingRefs.ResolveObjectRef(root, fromName, NamingRefs.EffectivePackage(report)) as MetaObject ?? throw Unresolved(report.Name, $"@from '{fromName}'"); + string[]? spine = ReportSpineHops(report, from, root); var fields = new List<ReportField>(); foreach (var item in ReportAccessors.ReportDimensionItems(report)) - fields.Add(DimensionField(item, from, root, report.Name)); + fields.Add(DimensionField(item, from, root, report.Name, spine)); foreach (string item in ReportAccessors.ReportMeasureNames(report)) fields.Add(MeasureField(item, report, from, root)); return new ReportShape(report, from, fields.AsReadOnly()); diff --git a/server/csharp/MetaObjects/Core/Reporting/ReportingConstants.cs b/server/csharp/MetaObjects/Core/Reporting/ReportingConstants.cs index ce1671358..bf7a645fd 100644 --- a/server/csharp/MetaObjects/Core/Reporting/ReportingConstants.cs +++ b/server/csharp/MetaObjects/Core/Reporting/ReportingConstants.cs @@ -79,6 +79,9 @@ public static class ReportingConstants public const string REPORTING_ATTR_DENOMINATOR = "denominator"; + /// <summary>Integer a measure reads when it would otherwise be null (<c>measure.aggregate</c> / <c>measure.ratio</c>).</summary> + public const string REPORTING_ATTR_DEFAULT = "default"; + // ----------------------------------------------------------------------- // Closed sets — mirrored by `allowedValues` in spec/metamodel/reporting.json. // Order is part of the contract (it is the order the registry manifest records). diff --git a/server/csharp/MetaObjects/Core/Reporting/ReportingSchema.cs b/server/csharp/MetaObjects/Core/Reporting/ReportingSchema.cs index e7702276b..8d27cd256 100644 --- a/server/csharp/MetaObjects/Core/Reporting/ReportingSchema.cs +++ b/server/csharp/MetaObjects/Core/Reporting/ReportingSchema.cs @@ -24,6 +24,10 @@ private static AttrSchema Str(string name, bool required, bool isArray = false, private static AttrSchema Filter(string name, bool required) => new(Name: name, ValueType: AttrConstants.ATTR_SUBTYPE_FILTER, Required: required); + /// <summary>@default on measure.aggregate and measure.ratio — an optional integer (FR-044 Table A).</summary> + private static AttrSchema MeasureDefault() => + new(Name: ReportingConstants.REPORTING_ATTR_DEFAULT, ValueType: AttrConstants.ATTR_SUBTYPE_INT, Required: false); + /// <summary>dimension.attribute — @of (required), @via (optional).</summary> private static readonly IReadOnlyList<AttrSchema> DimensionAttributeAttrs = [ @@ -41,7 +45,7 @@ private static AttrSchema Filter(string name, bool required) => ]; /// <summary>measure.aggregate — @agg (closed set), @of (isArray; a bare string is one column), - /// @distinct, @filter, @segment.</summary> + /// @distinct, @filter, @segment, @default.</summary> private static readonly IReadOnlyList<AttrSchema> MeasureAggregateAttrs = [ Str(ReportingConstants.REPORTING_ATTR_AGG, required: true, @@ -51,13 +55,15 @@ private static AttrSchema Filter(string name, bool required) => Required: false), Filter(ReportingConstants.REPORTING_ATTR_FILTER, required: false), Str(ReportingConstants.REPORTING_ATTR_SEGMENT, required: false), + MeasureDefault(), ]; - /// <summary>measure.ratio — @numerator, @denominator (both required).</summary> + /// <summary>measure.ratio — @numerator, @denominator (both required), @default.</summary> private static readonly IReadOnlyList<AttrSchema> MeasureRatioAttrs = [ Str(ReportingConstants.REPORTING_ATTR_NUMERATOR, required: true), Str(ReportingConstants.REPORTING_ATTR_DENOMINATOR, required: true), + MeasureDefault(), ]; /// <summary>segment.filter — the required @filter.</summary> @@ -89,7 +95,7 @@ private static AttrSchema Filter(string name, bool required) => [ReportingConstants.SEGMENT_SUBTYPE_FILTER] = SegmentFilterAttrs, }; - /// <summary>object.report — @from, @dimensions, @measures, @segment, @filter.</summary> + /// <summary>object.report — @from, @dimensions, @measures, @segment, @filter, @spine.</summary> public static readonly IReadOnlyList<AttrSchema> ReportAttrs = [ Str(ObjectConstants.OBJECT_REPORT_ATTR_FROM, required: true), @@ -97,5 +103,6 @@ private static AttrSchema Filter(string name, bool required) => Str(ObjectConstants.OBJECT_REPORT_ATTR_MEASURES, required: true, isArray: true), Str(ObjectConstants.OBJECT_REPORT_ATTR_SEGMENT, required: false), Filter(ObjectConstants.OBJECT_REPORT_ATTR_FILTER, required: false), + Str(ObjectConstants.OBJECT_REPORT_ATTR_SPINE, required: false), ]; } diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs index 281e00e45..0acbcf8d3 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs @@ -2,14 +2,23 @@ // // Ported rule-for-rule, WITH THE SAME MESSAGE TEXT, from the TS reference // server/typescript/packages/metadata/src/loader/reporting-validation.ts. The rule ids -// (D1…F2) match the rule table in the FR-044 plan and the error fixtures in -// fixtures/conformance/error-*; the fixtures are the contract. +// (D1…F2) match the rule table in the FR-044 plan (R8, R9, M7 and M8: Table B of +// docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md) and the error +// fixtures in fixtures/conformance/error-*; the fixtures are the contract. // // Two design rules hold throughout, so one broken rule yields exactly one error: // - No cascades. A member that fails a structural rule is not checked further -// (a dimension whose @via fails D2 skips D1/D3/D4; a report whose @from fails -// R1 skips R2/R3/R6/R7 and its @filter; an invalid @dimensions/@measures item -// derives no report field for R6). +// (a dimension whose @via fails D2 skips D1/D3/D4; a measure that fails one +// of M1–M4, or whose @default is a fraction, skips M7/M8; a report whose +// @from fails R1 skips R2/R3/R6/R7, R8/R9 and its @filter; a report whose +// @spine fails R8 skips R9, and R9 skips a dimension whose @via fails D2; an +// invalid @dimensions/@measures item derives no report field for R6). +// +// A fractional measure @default: the TypeScript reference refuses it here, in +// CheckMeasure, because its generic attr.int check lets a fraction through. This port's +// generic attr-type check (ValidateAttrSchema, ValueMatchesType) already refuses it with +// ERR_BAD_ATTR_VALUE on the measure node, so no second error is added here; CheckMeasure +// only skips M7/M8 for it, as the reference does. // - Each error's source is the offending node (the dimension / measure / segment / // report, or for R4/R5 the declared child), so a conformance fixture's jsonPath // points at it. @@ -308,7 +317,7 @@ void Err(string message) => string? via = dim.Via(); if (via is not null) { - var terminal = WalkToOneVia(ctx, via, Err); + var terminal = WalkToOneVia(ctx.Root, ctx.Host, via, Err, D2Walk(ctx.Declaring)); if (terminal is null) return; ofEntity = terminal; } @@ -359,25 +368,70 @@ void Err(string message) => } /// <summary> - /// D2 — walk <c>Owner.hop[.hop...]</c>: Owner is the owning entity, and every hop is a - /// to-one <c>relationship.*</c> or an <c>identity.reference</c>. Returns the terminal - /// entity, or null after reporting the first failure. + /// What a to-one walk resolves against and how its messages name it. D2 walks a + /// dimension's @via (<see cref="D2Walk"/>); R8 walks a report's @spine from @from + /// (<see cref="SpineWalk"/>). Every difference between the two is a field here, so each + /// port copies one explicit rule. /// </summary> - private static MetaData? WalkToOneVia(MemberCtx ctx, string via, Action<string> err) + /// <param name="Attr">The attribute holding the path: <c>via</c> (D2) or <c>spine</c> (R8).</param> + /// <param name="OwnerPkg">The package Owner resolves in (ADR-0042): the declaring entity's (D2) or the report's (R8).</param> + /// <param name="HostName">The FQN named for the walk's first entity: the declaring entity (D2) or @from (R8).</param> + /// <param name="Start">What Owner must be: <c>the owning entity '<FQN>'</c> (D2) or <c>@from '<FQN>'</c> (R8).</param> + /// <param name="StartShort">The same, in the malformed-path message: <c>the owning entity</c> (D2) or <c>@from '<FQN>'</c> (R8).</param> + /// <param name="ToOneReason">The sentence that ends the to-many message: why only to-one hops are followed.</param> + private sealed record ToOneWalk( + string Attr, string OwnerPkg, string HostName, string Start, string StartShort, string ToOneReason); + + /// <summary>D2 — a dimension's @via, declared on <paramref name="declaring"/>. Reproduces D2's messages exactly.</summary> + private static ToOneWalk D2Walk(MetaData declaring) { + string declaringKey = declaring.ResolutionKey(); + return new ToOneWalk( + Attr: REPORTING_ATTR_VIA, + OwnerPkg: NamingRefs.EffectivePackage(declaring), + HostName: declaringKey, + Start: $"the owning entity '{declaringKey}'", + StartShort: "the owning entity", + ToOneReason: + "A dimension follows only @cardinality: one relationships and identity.reference hops, so grouping " + + "can never multiply the measured rows."); + } + + /// <summary>R8 — a report's @spine, started at @from (<paramref name="fromKey"/>); Owner resolves in the report's package.</summary> + private static ToOneWalk SpineWalk(MetaData report, string fromKey) => + new( + Attr: OBJECT_REPORT_ATTR_SPINE, + OwnerPkg: NamingRefs.EffectivePackage(report), + HostName: fromKey, + Start: $"@from '{fromKey}'", + StartShort: $"@from '{fromKey}'", + ToOneReason: + "A @spine follows only @cardinality: one relationships and identity.reference hops, so each fact row " + + "joins at most one row of the spine entity and is never counted twice."); + + /// <summary> + /// Walk <c>Owner.hop[.hop...]</c> from <paramref name="host"/>: Owner is the host or an + /// entity it extends, and every hop is a to-one <c>relationship.*</c> or an + /// <c>identity.reference</c>. Returns the terminal entity, or null after reporting the + /// first failure. D2 and R8 both run it; <paramref name="walk"/> says which. + /// </summary> + private static MetaData? WalkToOneVia(MetaData root, MetaData host, string via, Action<string> err, ToOneWalk walk) + { + string named = $"@{walk.Attr} '{via}'"; + string NameOf(MetaData entity) => ReferenceEquals(entity, host) ? walk.HostName : entity.ResolutionKey(); var parts = ReportingSplitDotted(via); if (parts is null) { - err($"@via '{via}' must be Owner.hop[.hop...], starting at the owning entity."); + err($"{named} must be Owner.hop[.hop...], starting at {walk.StartShort}."); return null; } - var owner = NamingRefs.ResolveObjectRef(ctx.Root, parts.Value.Owner, NamingRefs.EffectivePackage(ctx.Declaring)); - if (!IsSelfOrAncestor(owner, ctx.Host)) + var owner = NamingRefs.ResolveObjectRef(root, parts.Value.Owner, walk.OwnerPkg); + if (!IsSelfOrAncestor(owner, host)) { - err($"@via '{via}' must start at the owning entity '{ctx.Declaring.ResolutionKey()}'."); + err($"{named} must start at {walk.Start}."); return null; } - MetaData current = ctx.Host; + MetaData current = host; foreach (string hopName in parts.Value.Path) { var hop = @@ -387,28 +441,27 @@ void Err(string message) => c.Type == TYPE_IDENTITY && c.SubType == IDENTITY_SUBTYPE_REFERENCE && c.Name == hopName); if (hop is null) { - err($"@via '{via}' names '{hopName}', which is not a relationship or identity.reference of " + - $"'{Shown(ctx, current)}'."); + err($"{named} names '{hopName}', which is not a relationship or identity.reference of " + + $"'{NameOf(current)}'."); return null; } bool isReference = hop.Type == TYPE_IDENTITY; // ADR-0039: resolving — @cardinality may be inherited via extends. if (!isReference && !Equals(hop.Attr(RELATIONSHIP_ATTR_CARDINALITY), CARDINALITY_ONE)) { - err($"@via '{via}' crosses relationship '{hopName}' on '{Shown(ctx, current)}', which is not to-one. " + - "A dimension follows only @cardinality: one relationships and identity.reference hops, so grouping " + - "can never multiply the measured rows."); + err($"{named} crosses relationship '{hopName}' on '{NameOf(current)}', which is not to-one. " + + walk.ToOneReason); return null; } // ADR-0039: resolving — the hop target attr may be inherited via extends. var targetRef = hop.Attr(isReference ? IDENTITY_REFERENCE_ATTR_REFERENCES : RELATIONSHIP_ATTR_OBJECT_REF); // ADR-0042 — a hop target resolves in the package of the entity declaring the hop. var target = targetRef is string tr - ? NamingRefs.ResolveObjectRef(ctx.Root, tr, NamingRefs.EffectivePackage(current)) + ? NamingRefs.ResolveObjectRef(root, tr, NamingRefs.EffectivePackage(current)) : null; if (target is null) { - err($"@via '{via}' hop '{hopName}' on '{Shown(ctx, current)}' targets no object."); + err($"{named} hop '{hopName}' on '{NameOf(current)}' targets no object."); return null; } current = target; @@ -421,6 +474,13 @@ private static void CheckMeasure(MemberCtx ctx, MetaMeasure measure) void Err(string message) => ctx.Sink.Push(measure, ErrorCode.ERR_INVALID_MEASURE, ctx.Label, $": {message}", ctx.Suffix); + // The type rule — a measure's @default is a whole number. ValidateAttrSchema already + // refuses a fraction (and a non-number) with ERR_BAD_ATTR_VALUE on this node (see the + // file header), so nothing is reported here. A fraction only skips M7/M8: one mistake, + // one error. ADR-0039: resolving — the raw attribute, inherited or not. + var declaredDefault = measure.Attr(REPORTING_ATTR_DEFAULT); + bool fractionalDefault = declaredDefault is double d && double.IsFinite(d) && d != Math.Floor(d); + if (measure.IsRatio()) { CheckRatioOperands(ctx, measure, Err); @@ -428,7 +488,27 @@ void Err(string message) => } if (measure.SubType != MEASURE_SUBTYPE_AGGREGATE) return; - CheckAggregateColumns(ctx, measure, Err); + bool clean = CheckAggregateColumns(ctx, measure, Err, out var ofField); + + // M7 / M8 — where a @default can apply. Only when M1–M4 passed and the @default is + // not a fraction: one mistake, one error. The presence test reads the raw attribute, + // so a non-number value on a count still reports M7 beside the attribute type error. + if (clean && !fractionalDefault && declaredDefault is not null) + { + string? agg = measure.Agg(); + if (agg == AGG_COUNT) + { + Err("@default cannot apply to @agg: count. A count is never null (it is 0 when nothing matches); " + + "remove @default."); + } + else if ((agg == AGG_MIN || agg == AGG_MAX) && + ofField is not null && + !ReportingNumericFieldSubtypes.Contains(ofField.SubType)) + { + Err($"@default is a number, but '{measure.OfColumns()[0]}', the @of of @agg '{agg}', is a " + + $"field.{ofField.SubType}. A default is supported on numeric measures only."); + } + } // M5 — @segment names a segment of the owning entity. string? segment = measure.SegmentName(); @@ -445,9 +525,14 @@ void Err(string message) => } } - /// <summary>M1–M4, in order; the first failure stops the chain (no M2+M3 double report).</summary> - private static void CheckAggregateColumns(MemberCtx ctx, MetaMeasure measure, Action<string> err) + /// <summary> + /// M1–M4, in order; the first failure stops the chain (no M2+M3 double report). + /// Returns false when one of them fired; otherwise <paramref name="field"/> is the single + /// resolved @of field (null for a tuple), for M7/M8. + /// </summary> + private static bool CheckAggregateColumns(MemberCtx ctx, MetaMeasure measure, Action<string> err, out MetaData? field) { + field = null; string? agg = measure.Agg(); var columns = measure.OfColumns(); @@ -459,22 +544,22 @@ private static void CheckAggregateColumns(MemberCtx ctx, MetaMeasure measure, Ac if (parts is null || parts.Value.Path.Length != 1) { err($"@of '{item}' must be Entity.field."); - return; + return false; } var named = NamingRefs.ResolveObjectRef(ctx.Root, parts.Value.Owner, NamingRefs.EffectivePackage(ctx.Declaring)); if (!IsSelfOrAncestor(named, ctx.Host)) { err($"@of '{item}' must name a field of the owning entity '{ctx.Declaring.ResolutionKey()}'. " + "A measure aggregates its own entity's rows; declare it on the entity that owns the column."); - return; + return false; } - var field = ReportingFieldOf(ctx.Host, parts.Value.Path[0]); - if (field is null) + var ofField = ReportingFieldOf(ctx.Host, parts.Value.Path[0]); + if (ofField is null) { err($"@of '{item}' names no field '{parts.Value.Path[0]}' on '{ctx.Declaring.ResolutionKey()}'."); - return; + return false; } - fields.Add(field); + fields.Add(ofField); } // M2 — a tuple is a distinct count only. @@ -482,30 +567,33 @@ private static void CheckAggregateColumns(MemberCtx ctx, MetaMeasure measure, Ac { err($"@of lists {columns.Count} columns; a tuple is legal only with @agg: count and @distinct: true " + "(a distinct count of the tuple)."); - return; + return false; } // M3 — @distinct is a count modifier. if (measure.Distinct() && agg is not null && agg != AGG_COUNT) { err($"@distinct: true requires @agg: count, not '{agg}'."); - return; + return false; } // M4 — the aggregate must be meaningful for the column's type. var single = fields.Count == 1 ? fields[0] : null; - if (single is null || agg is null) return; + field = single; + if (single is null || agg is null) return true; string first = columns[0]; if ((agg == AGG_SUM || agg == AGG_AVG) && !ReportingNumericFieldSubtypes.Contains(single.SubType)) { err($"@agg '{agg}' needs a numeric field (field.int, long, double, float, decimal or currency), " + $"but '{first}' is field.{single.SubType}."); - return; + return false; } if ((agg == AGG_MIN || agg == AGG_MAX) && ReportingUnorderedFieldSubtypes.Contains(single.SubType)) { err($"@agg '{agg}' cannot order '{first}', a field.{single.SubType}."); + return false; } + return true; } /// <summary>M6 — each operand names a measure.aggregate of the same entity.</summary> @@ -659,8 +747,8 @@ void Err(string message, MetaData? node = null, ErrorCode code = ErrorCode.ERR_I } } - // R1 — @from resolves to an object.entity. Without it, R2/R3/R6/R7 and the - // @filter have nothing to resolve against, so they are skipped. + // R1 — @from resolves to an object.entity. Without it, R2/R3/R6/R7, R8/R9 and + // the @filter have nothing to resolve against, so they are skipped. string? fromRef = ReportAccessors.ReportFrom(report); if (fromRef is null) return; // missing @from is ERR_MISSING_REQUIRED_ATTR var from = NamingRefs.ResolveObjectRef(root, fromRef, NamingRefs.EffectivePackage(report)); @@ -750,6 +838,49 @@ void Claim(string fieldName, string what) Err($": @segment '{segment}' names no segment of @from '{fromKey}'."); } + // R8 — @spine is a to-one path from @from: rule D2's walk, started at @from. + string? spine = ReportAccessors.ReportSpine(report); + if (spine is not null) + { + var terminal = WalkToOneVia(root, from, spine, m => Err($": {m}"), SpineWalk(report, fromKey)); + // R9 — every listed dimension is reached through the spine. Skipped when R8 failed. + if (terminal is not null) + { + string[] spineHops = ReportingSplitDotted(spine)?.Path ?? []; + var items = ReportAccessors.ReportDimensionItems(report); + if (items.Count == 0) + { + Err($": @spine '{spine}' needs at least one dimension. The report's rows are the dimension tuples of " + + $"'{terminal.ResolutionKey()}'; with no dimension it would be one totals row."); + } + // One verdict per dimension, however many grains list it. + var checkedDimensions = new HashSet<string>(StringComparer.Ordinal); + foreach (var item in items) + { + if (!checkedDimensions.Add(item.Name)) continue; + if (ReportingChildOfType(from, TYPE_DIMENSION, item.Name) is not MetaDimension dim) continue; // R2 already reported it + string? via = dim.Via(); + if (via is null) + { + Err($": dimension '{item.Name}' is read from @from '{fromKey}', so it has no value in a row that has " + + $"no facts. With @spine '{spine}' every dimension must be reached through it: declare the " + + $"dimension over a field of '{terminal.ResolutionKey()}' (or an entity to-one from it) with an " + + $"@via that begins '{spine}'."); + continue; + } + // A @via that does not walk is D2's error, on the dimension; R9 does not report it again. + if (WalkToOneVia(root, from, via, _ => { }, D2Walk(dim.Parent ?? from)) is null) continue; + // Hop names are compared as written; the owner segment is not compared. + string[] hops = ReportingSplitDotted(via)?.Path ?? []; + if (hops.Length >= spineHops.Length && + hops.Take(spineHops.Length).SequenceEqual(spineHops, StringComparer.Ordinal)) continue; + Err($": dimension '{item.Name}' is reached by @via '{via}', which does not begin with the hops of " + + $"@spine '{spine}'. Hop names are compared as written: if both name the same join, write the same " + + "hops; otherwise the dimension is not reached through the spine."); + } + } + } + // S1 / F2 — the report's row scope over @from. // ADR-0039: resolving — a report may inherit @filter via extends. if (AsPlainObject(report.Attr(OBJECT_REPORT_ATTR_FILTER)) is { } filter) diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.cs index 038ccc299..032427cae 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.cs @@ -1504,7 +1504,9 @@ public static IReadOnlyList<MetaError> ValidateIdentityPassthrough(MetaData root private static List<string>? IdentityOwnFields(MetaData identity) => NormalizeIdentityFields(identity.OwnAttr(IDENTITY_ATTR_FIELDS)); - private static List<string>? IdentityEffectiveFields(MetaData identity) + // Internal: the report shape (ReportShapes, FR-044 Table C) reads a primary key's + // effective @fields with the same rule, so it uses this one. + internal static List<string>? IdentityEffectiveFields(MetaData identity) => NormalizeIdentityFields(identity.Attr(IDENTITY_ATTR_FIELDS)); internal static (MetaData Entity, List<string> ComputedFields, List<string> Missing)? diff --git a/server/csharp/MetaObjects/Meta/MetaMeasure.cs b/server/csharp/MetaObjects/Meta/MetaMeasure.cs index 9f805990e..e750cf6ec 100644 --- a/server/csharp/MetaObjects/Meta/MetaMeasure.cs +++ b/server/csharp/MetaObjects/Meta/MetaMeasure.cs @@ -36,6 +36,21 @@ public class MetaMeasure(TypeId typeId, string name) : MetaData(typeId, name) public IReadOnlyDictionary<string, object?>? Filter() => Attr(REPORTING_ATTR_FILTER) as IReadOnlyDictionary<string, object?>; + /// <summary> + /// <c>@default</c>: the integer this measure reads when it would otherwise be null (rules + /// M7/M8 decide where it is legal). Null when none is declared or the value is not an + /// integer. ADR-0039: resolving, so an inherited measure keeps it. + /// </summary> + public long? DefaultValue() => Attr(REPORTING_ATTR_DEFAULT) switch + { + long l => l, + int i => i, + // A JSON number written with a fraction part (`0.0`) parses as a double; TypeScript + // reads the same text as the integer 0. + double d when double.IsFinite(d) && d == Math.Floor(d) && Math.Abs(d) <= 9007199254740991d => (long)d, + _ => null, + }; + /// <summary>Name of the <c>measure.aggregate</c> sibling used as the numerator (<c>measure.ratio</c> only).</summary> public string? Numerator() => Attr(REPORTING_ATTR_NUMERATOR) as string; diff --git a/server/csharp/MetaObjects/SpecMetamodel/object.json b/server/csharp/MetaObjects/SpecMetamodel/object.json index cb5f9946f..76187a94a 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/object.json +++ b/server/csharp/MetaObjects/SpecMetamodel/object.json @@ -64,7 +64,8 @@ { "type": "attr", "subType": "string", "name": "dimensions", "isArray": true, "min": 0, "max": 1, "description": "Dimension names of @from; a time dimension is written 'name:grain'. Absent means one global row." }, { "type": "attr", "subType": "string", "name": "measures", "isArray": true, "min": 1, "max": 1, "description": "Measure names of @from." }, { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional segment of @from scoping the rows." }, - { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope over @from's fields; may use relative-date values." } + { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope over @from's fields; may use relative-date values." }, + { "type": "attr", "subType": "string", "name": "spine", "min": 0, "max": 1, "description": "Optional to-one path from @from to the entity whose rows supply the report's rows (e.g. 'Purchase.program'), written like a dimension's @via. With it the report has one row per distinct dimension tuple among THAT entity's rows, including the ones no row of @from refers to: a count there is 0 and any other measure is null unless it declares @default. Every listed dimension must be reached through this path. @segment and @filter still scope the rows of @from and never remove a row. A row of @from whose reference is null or matches nothing is in no row of the report." } ] } ] diff --git a/server/csharp/MetaObjects/SpecMetamodel/reporting.json b/server/csharp/MetaObjects/SpecMetamodel/reporting.json index 024196d63..019dda238 100644 --- a/server/csharp/MetaObjects/SpecMetamodel/reporting.json +++ b/server/csharp/MetaObjects/SpecMetamodel/reporting.json @@ -24,14 +24,16 @@ { "type": "attr", "subType": "string", "name": "of", "isArray": true, "min": 1, "max": 1, "description": "Dotted Entity.field reference(s) on the declaring entity. A bare string is one column; more than one requires @agg: count and @distinct: true." }, { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Count distinct values. Legal only with @agg: count." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope (a portable attr.filter over the declaring entity's fields). May use relative-date values ({ now: \"-P7D\" }). Combines with @segment by AND." }, - { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional name of a segment declared on the same entity. Combines with @filter by AND." } + { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional name of a segment declared on the same entity. Combines with @filter by AND." }, + { "type": "attr", "subType": "int", "name": "default", "min": 0, "max": 1, "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } ] }, { "type": "measure", "subType": "ratio", "description": "A named quotient of two measure.aggregate siblings (FR-044), lowered as numerator / NULLIF(denominator, 0) and typed decimal. A zero denominator yields null.", "whenToUse": "Averages per unit that are not a plain avg: average days engaged per starter.", "children": [ { "type": "attr", "subType": "string", "name": "numerator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." }, - { "type": "attr", "subType": "string", "name": "denominator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." } + { "type": "attr", "subType": "string", "name": "denominator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." }, + { "type": "attr", "subType": "int", "name": "default", "min": 0, "max": 1, "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } ] }, { "type": "segment", "subType": "filter", "description": "A named, reusable row filter on the declaring entity (FR-044). Measures and reports reference it by name; exporters emit it as a named segment.", From cf437b4873d6a6bb0f9d87bede3313e5e1b95c15 Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 21:53:04 -0400 Subject: [PATCH 13/18] feat(java): @spine and measure @default FR-044 zero rows and measure defaults, the Java port of the TypeScript reference (Tables A, B and C). Kotlin rides on the loader and ReportShape. - Register @spine (string) on object.report and @default (int) on measure.aggregate and measure.ratio. MetaMeasure.getDefaultValue() and ReportAccessors.reportSpine(). The registry manifest's null value type for an attr named default is narrowed to field registrations, so a measure's default prints int. - Loader rules R8 (the spine is D2's to-one walk, started at @from), R9 (every listed dimension is reached through the spine, hop names compared as written), M7 (@default on a count) and M8 (@default on min/max of a non-numeric field), with the TypeScript message texts. walkToOneVia takes a ToOneWalk (d2Walk / spineWalk); every D2 message is unchanged. A fractional @default is already refused by this port's generic attr.int parse (ERR_BAD_ATTR_VALUE on the measure, and the load stops there), so no second error is added; a non-integer value only skips M7/M8. - ReportShape Table C: under @spine a dimension on the spine is required when its @of is @required or a primary-key column; a defaulted measure is required. New reportingViaHops / reportSpineHops. - No generator change: the report DTO marks the spine key and a defaulted measure @NotNull, as it does a count. OMDB reads a @spine view's empty row. - The report REST seam lane serves ProductRevenue; corpus counts: nine canonical reports, four inert reports, sixteen REST scenarios. --- .../api-contract-conformance/report/README.md | 2 +- .../spring/CodegenCompileConformanceTest.java | 8 +- .../generator/spring/ReportingInertTest.java | 17 +- .../spring/SpringReportRestSurfaceTest.java | 30 ++ .../integration/api/ReportCorpus.java | 17 +- ...rtGeneratedApiContractConformanceTest.java | 8 +- .../loader/ReportingValidation.java | 225 ++++++++-- .../com/metaobjects/object/MetaObject.java | 9 + .../registry/RegistryManifest.java | 24 +- .../reporting/AggregateMeasure.java | 3 + .../metaobjects/reporting/MetaMeasure.java | 14 + .../metaobjects/reporting/RatioMeasure.java | 3 + .../reporting/ReportAccessors.java | 9 + .../metaobjects/reporting/ReportShape.java | 112 ++++- .../reporting/ReportingConstants.java | 6 + .../loader/ReportingValidationTest.java | 411 ++++++++++++++++++ .../reporting/ReportReadModelTest.java | 24 +- .../reporting/ReportShapeTest.java | 290 ++++++++++++ .../metaobjects/reporting/ReportingTest.java | 30 ++ .../manager/db/ReportReadTest.java | 35 ++ .../omdb/src/test/resources/meta.report.json | 24 + 21 files changed, 1223 insertions(+), 78 deletions(-) diff --git a/fixtures/api-contract-conformance/report/README.md b/fixtures/api-contract-conformance/report/README.md index 3e220ad34..e79e70287 100644 --- a/fixtures/api-contract-conformance/report/README.md +++ b/fixtures/api-contract-conformance/report/README.md @@ -144,7 +144,7 @@ hand-rolled reference server would answer every scenario by construction. |---|---|---| | TypeScript | wired | `server/typescript/packages/integration-tests/test/api-contract-report.test.ts` (16 scenarios + a seed-vs-view check). Full stack: generated Fastify routes over the real views on Testcontainers Postgres. `api-contract-report-sqlite.test.ts` runs the same 16 on SQLite | | C# | wired | `server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs`. Full stack: generated routes and EF Core over `schema.postgres.sql` on Testcontainers Postgres | -| Java | wired | `server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java` (13 scenarios + a scenario-count check). Generated controllers behind an in-memory repository seeded from `reports` | +| Java | wired | `server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java` (16 scenarios + a scenario-count check). Generated controllers behind an in-memory repository seeded from `reports` | | Kotlin | wired | `server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt` (13 scenarios, the count check, and a check that the sourceless report generated nothing). Generated controllers over the generated Exposed table objects, seeded from `reports` | | Python | wired | `server/python/tests/integration/test_api_contract_report.py` (13 scenarios, a check that exactly the served reports are generated, and `/api/invoice_days` is `404`). Generated routers behind in-memory repositories seeded from `reports` | diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/CodegenCompileConformanceTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/CodegenCompileConformanceTest.java index 40d61634b..fe60b14b7 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/CodegenCompileConformanceTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/CodegenCompileConformanceTest.java @@ -199,13 +199,15 @@ public void theSpringWebTierCompiles() throws Exception { selection.put("repository", PLAIN); selection.put("filter-allowlist", PLAIN); selection.put("names", PLAIN); - // FR-044 Plan 3: the corpus's six view-backed reports are served, so their row + // FR-044 Plan 3: the corpus's nine view-backed reports are served, so their row // DTOs, repository seams and filter allowlists are in this compile. Named, so a - // gate that stops admitting a report fails here instead of compiling less. + // gate that stops admitting a report fails here instead of compiling less. The + // last three carry @spine / a measure @default. // ProgramMinutes derives eleven filterable fields, one past Map.of's ten pairs. generateAndCompile("web", selection, 17, "ProgramMinutesDto", "FitnessTotalsDto", "ProgramsByMonthDto", "ProgramsByWeekDto", - "RecentProgramsDto", "AssetActivityDto", + "RecentProgramsDto", "AssetActivityDto", "ProgramRosterDto", "ProgramLongWeeksDto", + "FitnessTotalsFilledDto", "ProgramMinutesRepository", "ProgramMinutesFilterAllowlist"); } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/ReportingInertTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/ReportingInertTest.java index f9033ac94..da873f6d7 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/ReportingInertTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/ReportingInertTest.java @@ -44,7 +44,9 @@ * SERVED: the REST-surface generators emit its read-only files and nothing else changes. * {@code StoreTotals} is that report, so {@code with/} differs from {@code without/} in * exactly {@link #SERVED_REPORT_FILES} (one file per generator) and one api-docs unit. - * {@code DailyRevenue} and {@code ProgramEngagement} declare no source and stay inert. + * {@code DailyRevenue}, {@code ProgramEngagement} and {@code ProgramCatalogue} (which declares + * {@code @spine} and lists a dimension reached by {@code @via}) declare no source and stay + * inert, as does a measure {@code @default} ({@code avgDaysPerStarter}). * * <p>The model pair is {@code fixtures/codegen-noop/reporting/{with,without}}, shared with the * other four ports' copies of this test. @@ -56,6 +58,10 @@ public class ReportingInertTest extends SharedRegistryTestBase { private static final String THREW = "<threw>"; + /** The with-model's reports that declare no source: inert everywhere. */ + private static final List<String> SOURCELESS_REPORTS = + List.of("DailyRevenue", "ProgramEngagement", "ProgramCatalogue"); + /** * The one file each REST-surface generator adds for the served report {@code StoreTotals} * (contract Table E), by generator stable name. Every generator not named here must emit @@ -155,7 +161,7 @@ public void theWithModelReallyCarriesTheVocabulary() throws Exception { if (MetaObject.SUBTYPE_REPORT.equals(mo.getSubType())) reports.add(mo.getShortName()); } reports.sort(null); - assertEquals(List.of("DailyRevenue", "ProgramEngagement", "StoreTotals"), reports); + assertEquals(List.of("DailyRevenue", "ProgramCatalogue", "ProgramEngagement", "StoreTotals"), reports); for (MetaObject mo : load("without").getMetaObjects()) { assertFalse("without-model declares a report", MetaObject.SUBTYPE_REPORT.equals(mo.getSubType())); } @@ -188,7 +194,7 @@ public void noGeneratorEmitsAnythingForASourcelessReport() throws Exception { for (GeneratorInfo info : GeneratorRegistry.list().values()) { for (String path : emit("with", List.of(info)).keySet()) { assertFalse(info.stableName() + " emitted " + path, - path.contains("DailyRevenue") || path.contains("ProgramEngagement")); + SOURCELESS_REPORTS.stream().anyMatch(path::contains)); } } } @@ -277,8 +283,9 @@ public void apiDocsGainOneUnitForTheServedReportAndNothingElseChanges() throws E assertTrue(listing + " lists the served report", actual.get(listing).contains("StoreTotals")); } for (String page : actual.values()) { - assertFalse("a sourceless report is documented", page.contains("DailyRevenue")); - assertFalse("a sourceless report is documented", page.contains("ProgramEngagement")); + for (String sourceless : SOURCELESS_REPORTS) { + assertFalse("a sourceless report is documented: " + sourceless, page.contains(sourceless)); + } } } } diff --git a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringReportRestSurfaceTest.java b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringReportRestSurfaceTest.java index f2387e624..dbfe44e78 100644 --- a/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringReportRestSurfaceTest.java +++ b/server/java/codegen-spring/src/test/java/com/metaobjects/generator/spring/SpringReportRestSurfaceTest.java @@ -207,6 +207,36 @@ public void aDerivedEnumFieldIsTypedByAnEnumNestedInTheReportsOwnDto() throws Ex assertTrue("a time dimension at a date grain is a LocalDate", src.contains("java.time.LocalDate createdAtMonth")); } + /** + * Table F of {@code docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md} + * (the Java row): no generator is edited; the shape's {@code required} (Table C) reaches + * the DTO as the same {@code @NotNull} a count carries. {@code ProgramRoster} declares + * {@code @spine: "Week.fkProgram"}; {@code totalMinutesOrZero} / {@code longShareOrZero} + * are {@code totalMinutes} / {@code longShare} with {@code @default: 0}. + */ + @Test + public void aSpineKeyAndADefaultedMeasureAreNotNullAndTheSameMeasureWithoutADefaultIsNullable() throws Exception { + String fitness = Files.readString( + SpringTestFixtures.findCorpusRoot().resolve("canonical/meta.fitness.json"), StandardCharsets.UTF_8); + Path gen = generateAll("spine", fitness); + Path dto; + try (Stream<Path> s = Files.walk(gen)) { + dto = s.filter(p -> p.getFileName().toString().equals("ProgramRosterDto.java")).findFirst().orElseThrow(); + } + String src = Files.readString(dto); + assertTrue("the spine entity's key", src.contains("@NotNull Long programKey")); + assertTrue("a @required column of the spine entity", + src.contains("@NotNull @Size(min = 1, max = 200) String programTitle")); + assertTrue("a count, as before", src.contains("@NotNull Long weeks")); + assertTrue("a sum with @default", src.contains("@NotNull Long totalMinutesOrZero")); + assertTrue("a ratio with @default", src.contains("@NotNull java.math.BigDecimal longShareOrZero")); + assertTrue(src, src.contains(" Long totalMinutes,")); + assertFalse("the same sum without @default stays nullable", src.contains("@NotNull Long totalMinutes,")); + assertTrue(src, src.contains(" java.math.BigDecimal longShare,")); + assertFalse("the same ratio without @default stays nullable", + src.contains("@NotNull java.math.BigDecimal longShare,")); + } + @Test public void theRepositoryListsAndCountsAndHasNoIdentity() throws Exception { String src = Files.readString(generateAll("seam", withModel()).resolve("acme/shop/StoreTotalsRepository.java")); diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportCorpus.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportCorpus.java index c65ee4695..6187d75e9 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportCorpus.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportCorpus.java @@ -14,19 +14,22 @@ * Locates the FR-044 view-backed-report sub-corpus * ({@code fixtures/api-contract-conformance/report/}) and loads its seed rows. * - * <p>{@code seed.json} has two halves. {@code invoices} is the base table, used by the - * full-stack lanes. This lane is a SEAM lane: its in-memory repository stands in for the - * view, so it is seeded from {@code reports}, which is what the three views return for - * those invoices. No SQL for a report is produced or run here (ADR-0015: view SQL is - * TypeScript's). A TypeScript test holds the two halves together.</p> + * <p>{@code seed.json} has two halves. Every top-level key but {@code reports} is a base + * table ({@code invoices}, {@code products}, {@code sales}), used by the full-stack lanes. + * This lane is a SEAM lane: its in-memory repository stands in for the view, so it is + * seeded from {@code reports} alone, which is what the four views return for those rows. + * No SQL for a report is produced or run here (ADR-0015: view SQL is TypeScript's). A + * TypeScript test holds the two halves together.</p> */ final class ReportCorpus { private ReportCorpus() {} private static final ObjectMapper MAPPER = new ObjectMapper(); - /** The three served reports, in declaration order. */ - static final List<String> SERVED = List.of("InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals"); + /** The four served reports, in declaration order. {@code ProductRevenue} declares + * {@code @spine} and a measure with {@code @default}. */ + static final List<String> SERVED = + List.of("InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals", "ProductRevenue"); /** The report that declares no view and must generate nothing. */ static final String SOURCELESS = "InvoiceDays"; diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java index 3f3519ee7..8ca551be7 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java @@ -23,9 +23,9 @@ /** * FR-044 Plan 3 — the Java GENERATED-controller lane for the view-backed-report - * api-contract sub-corpus. Boots the three GENERATED Spring report controllers on one + * api-contract sub-corpus. Boots the four GENERATED Spring report controllers on one * embedded Tomcat, each behind an in-memory repository seam seeded with what its view - * returns, and drives all thirteen scenarios. + * returns, and drives all sixteen scenarios. * * <p>Generated lane ONLY, on purpose and on every port (see the sub-corpus README). What * is under test is whether the port's GENERATOR emits a read route for a served report, @@ -61,9 +61,9 @@ static Stream<Arguments> scenarios() { } @Test - void theCorpusCarriesItsThirteenScenarios() { + void theCorpusCarriesItsSixteenScenarios() { // A scenarios directory that resolved to nothing would leave this lane green and empty. - assertEquals(13, SCENARIOS.size()); + assertEquals(16, SCENARIOS.size()); } @ParameterizedTest(name = "{0}") diff --git a/server/java/metadata/src/main/java/com/metaobjects/loader/ReportingValidation.java b/server/java/metadata/src/main/java/com/metaobjects/loader/ReportingValidation.java index 34b7756a1..144d0aa52 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/loader/ReportingValidation.java +++ b/server/java/metadata/src/main/java/com/metaobjects/loader/ReportingValidation.java @@ -61,17 +61,21 @@ /** * FR-044 — cross-node rules for the reporting vocabulary. The rule ids (D1…F2) match the - * rule table in the FR-044 Plan 1 document and the error fixtures in - * {@code fixtures/conformance/error-*}. Every port implements the same table WITH THE SAME - * MESSAGE TEXT; the TypeScript {@code reporting-validation.ts} is the reference and the - * fixtures are the contract. + * rule table in the FR-044 Plan 1 document (R8, R9, M7 and M8: Table B of + * {@code docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md}) and + * the error fixtures in {@code fixtures/conformance/error-*}. Every port implements the same + * table WITH THE SAME MESSAGE TEXT; the TypeScript {@code reporting-validation.ts} is the + * reference and the fixtures are the contract. * * <p>Two design rules hold throughout, so one broken rule yields exactly one error:</p> * <ul> * <li>No cascades. A member that fails a structural rule is not checked further (a - * dimension whose {@code @via} fails D2 skips D1/D3/D4; a report whose {@code @from} - * fails R1 skips R2/R3/R6/R7 and its {@code @filter}; an invalid - * {@code @dimensions}/{@code @measures} item derives no report field for R6).</li> + * dimension whose {@code @via} fails D2 skips D1/D3/D4; a measure that fails one of + * M1–M4, or whose {@code @default} is a fraction, skips M7/M8; a report whose + * {@code @from} fails R1 skips R2/R3/R6/R7, R8/R9 and its {@code @filter}; a report + * whose {@code @spine} fails R8 skips R9, and R9 skips a dimension whose {@code @via} + * fails D2; an invalid {@code @dimensions}/{@code @measures} item derives no report + * field for R6).</li> * <li>Each error's source is the offending node (the dimension / measure / segment / * report, or for R4/R5 the declared child), so a conformance fixture's jsonPath * points at it.</li> @@ -87,6 +91,13 @@ * <p>Java addition: {@code .withEnum} is decorative on attr children in this port (see * {@code validateRequirementStatus}), so the closed {@code @agg} / {@code @grains} sets the * TS registry enforces generically are checked here as {@code ERR_BAD_ATTR_VALUE}.</p> + * + * <p>A fractional measure {@code @default}: the TypeScript reference refuses it here, in + * {@code checkMeasure}, because its generic attr.int check lets a fraction through. This + * port's generic attr.int parse ({@code IntAttribute.setValueAsString}, reached from + * {@code BaseMetaDataParser.parseInlineAttribute}) already refuses it with + * {@code ERR_BAD_ATTR_VALUE} on the measure node, so no second error is added here; + * {@code checkMeasure} only skips M7/M8 for a non-integer value, as the reference does.</p> */ final class ReportingValidation { @@ -403,7 +414,7 @@ private static void checkDimension(MemberCtx ctx, MetaDimension dim) { MetaData ofEntity = ctx.host(); String via = dim.getVia(); if (via != null) { - MetaData terminal = walkToOneVia(ctx, via, err); + MetaData terminal = walkToOneVia(ctx.root(), ctx.host(), via, err, d2Walk(ctx.declaring())); if (terminal == null) return; ofEntity = terminal; } @@ -449,22 +460,75 @@ private static void checkDimension(MemberCtx ctx, MetaDimension dim) { } /** - * D2 — walk {@code Owner.hop[.hop...]}: Owner is the owning entity, and every hop is a - * to-one {@code relationship.*} or an {@code identity.reference}. Returns the terminal - * entity, or {@code null} after reporting the first failure. + * What a to-one walk resolves against and how its messages name it. D2 walks a + * dimension's {@code @via} ({@link #d2Walk}); R8 walks a report's {@code @spine} from + * {@code @from} ({@link #spineWalk}). Every difference between the two is a field here, + * so each port copies one explicit rule. + * + * @param attr the attribute holding the path: {@code via} (D2) or {@code spine} (R8) + * @param ownerPkg the package Owner resolves in (ADR-0042): the declaring entity's (D2) + * or the report's (R8) + * @param hostName the FQN named for the walk's first entity: the declaring entity (D2) + * or {@code @from} (R8) + * @param start what Owner must be: {@code the owning entity '<FQN>'} (D2) or + * {@code @from '<FQN>'} (R8) + * @param startShort the same, in the malformed-path message: {@code the owning entity} + * (D2) or {@code @from '<FQN>'} (R8) + * @param toOneReason the sentence that ends the to-many message: why only to-one hops + * are followed */ - private static MetaData walkToOneVia(MemberCtx ctx, String via, Consumer<String> err) { + private record ToOneWalk(String attr, String ownerPkg, String hostName, String start, String startShort, + String toOneReason) { + } + + /** D2 — a dimension's {@code @via}, declared on {@code declaring}. Reproduces D2's messages exactly. */ + private static ToOneWalk d2Walk(MetaData declaring) { + String declaringKey = declaring.getName(); + return new ToOneWalk( + ReportingConstants.ATTR_VIA, + pkgOf(declaring), + declaringKey, + "the owning entity '" + declaringKey + "'", + "the owning entity", + "A dimension follows only @cardinality: one relationships and identity.reference hops, so grouping " + + "can never multiply the measured rows."); + } + + /** R8 — a report's {@code @spine}, started at {@code @from} ({@code fromKey}); Owner resolves + * in the report's package. */ + private static ToOneWalk spineWalk(MetaData report, String fromKey) { + return new ToOneWalk( + MetaObject.ATTR_REPORT_SPINE, + pkgOf(report), + fromKey, + "@from '" + fromKey + "'", + "@from '" + fromKey + "'", + "A @spine follows only @cardinality: one relationships and identity.reference hops, so each fact row " + + "joins at most one row of the spine entity and is never counted twice."); + } + + /** + * Walk {@code Owner.hop[.hop...]} from {@code host}: Owner is {@code host} or an entity it + * extends, and every hop is a to-one {@code relationship.*} or an {@code identity.reference}. + * Returns the terminal entity, or {@code null} after reporting the first failure. D2 and R8 + * both run it; {@code walk} says which. + */ + private static MetaData walkToOneVia(MetaRoot root, MetaData host, String via, Consumer<String> err, + ToOneWalk walk) { + String named = "@" + walk.attr() + " '" + via + "'"; + java.util.function.Function<MetaData, String> nameOf = + entity -> entity == host ? walk.hostName() : entity.getName(); Dotted parts = splitDotted(via); if (parts == null) { - err.accept("@via '" + via + "' must be Owner.hop[.hop...], starting at the owning entity."); + err.accept(named + " must be Owner.hop[.hop...], starting at " + walk.startShort() + "."); return null; } - MetaData owner = ValidationPhase.resolveRootObject(ctx.root(), parts.owner(), pkgOf(ctx.declaring())); - if (!isSelfOrAncestor(owner, ctx.host())) { - err.accept("@via '" + via + "' must start at the owning entity '" + ctx.declaring().getName() + "'."); + MetaData owner = ValidationPhase.resolveRootObject(root, parts.owner(), walk.ownerPkg()); + if (!isSelfOrAncestor(owner, host)) { + err.accept(named + " must start at " + walk.start() + "."); return null; } - MetaData current = ctx.host(); + MetaData current = host; for (String hopName : parts.path()) { MetaData hop = childOfType(current, MetaRelationship.TYPE_RELATIONSHIP, hopName); if (hop == null) { @@ -479,23 +543,22 @@ private static MetaData walkToOneVia(MemberCtx ctx, String via, Consumer<String> } } if (hop == null) { - err.accept("@via '" + via + "' names '" + hopName - + "', which is not a relationship or identity.reference of '" + shown(ctx, current) + "'."); + err.accept(named + " names '" + hopName + + "', which is not a relationship or identity.reference of '" + nameOf.apply(current) + "'."); return null; } boolean isReference = MetaIdentity.TYPE_IDENTITY.equals(hop.getType()); if (!isReference && !MetaRelationship.CARDINALITY_ONE.equals(stringAttr(hop, MetaRelationship.ATTR_CARDINALITY))) { - err.accept("@via '" + via + "' crosses relationship '" + hopName + "' on '" + shown(ctx, current) - + "', which is not to-one. A dimension follows only @cardinality: one relationships and " - + "identity.reference hops, so grouping can never multiply the measured rows."); + err.accept(named + " crosses relationship '" + hopName + "' on '" + nameOf.apply(current) + + "', which is not to-one. " + walk.toOneReason()); return null; } String targetRef = stringAttr(hop, isReference ? MetaIdentity.ATTR_REFERENCES : MetaRelationship.ATTR_OBJECT_REF); // ADR-0042 — a hop target resolves in the package of the entity declaring the hop. MetaData target = targetRef == null ? null - : ValidationPhase.resolveRootObject(ctx.root(), targetRef, pkgOf(current)); + : ValidationPhase.resolveRootObject(root, targetRef, pkgOf(current)); if (target == null) { - err.accept("@via '" + via + "' hop '" + hopName + "' on '" + shown(ctx, current) + "' targets no object."); + err.accept(named + " hop '" + hopName + "' on '" + nameOf.apply(current) + "' targets no object."); return null; } current = target; @@ -514,13 +577,43 @@ private static void checkMeasure(MemberCtx ctx, MetaMeasure measure) { Consumer<String> err = message -> ctx.sink().push(measure, ErrorCode.ERR_INVALID_MEASURE, ctx.label(), ": " + message, ctx.suffix()); + // The type rule — a measure's @default is a whole number. This port's generic attr.int + // parse already refuses a fraction (and a non-number) with ERR_BAD_ATTR_VALUE on this + // node (see the class comment), so nothing is reported here; a strict load stops at + // that parse error before this pass runs. A numeric value that is not an integer (a + // tree built in code) only skips M7/M8: one mistake, one error. ADR-0039: resolving — + // the raw attribute, inherited or not. + boolean declaresDefault = measure.hasMetaAttr(ReportingConstants.ATTR_DEFAULT); + Object declaredDefault = declaresDefault + ? measure.getMetaAttr(ReportingConstants.ATTR_DEFAULT).getValue() : null; + boolean fractionalDefault = declaredDefault instanceof Number + && measure.getDefaultValue() == null; + if (measure.isRatio()) { checkRatioOperands(ctx, measure, err); return; } if (!ReportingConstants.MEASURE_SUBTYPE_AGGREGATE.equals(measure.getSubType())) return; - checkAggregateColumns(ctx, measure, err); + AggregateColumns checked = checkAggregateColumns(ctx, measure, err); + + // M7 / M8 — where a @default can apply. Only when M1–M4 passed and the @default is + // not a fraction: one mistake, one error. The presence test reads the raw attribute, + // as the reference does (in a lenient load an unparsed value is still a declared one). + if (checked != null && !fractionalDefault && declaresDefault) { + String agg = measure.getAgg(); + MetaData ofField = checked.field(); + if (ReportingConstants.AGG_COUNT.equals(agg)) { + err.accept("@default cannot apply to @agg: count. A count is never null (it is 0 when nothing " + + "matches); remove @default."); + } else if ((ReportingConstants.AGG_MIN.equals(agg) || ReportingConstants.AGG_MAX.equals(agg)) + && ofField != null + && !NUMERIC_FIELD_SUBTYPES.contains(ofField.getSubType())) { + err.accept("@default is a number, but '" + measure.getOfColumns().get(0) + "', the @of of @agg '" + + agg + "', is a field." + ofField.getSubType() + + ". A default is supported on numeric measures only."); + } + } // M5 — @segment names a segment of the owning entity. String segment = measure.getSegmentName(); @@ -535,8 +628,16 @@ private static void checkMeasure(MemberCtx ctx, MetaMeasure measure) { } } - /** M1–M4, in order; the first failure stops the chain (no M2+M3 double report). */ - private static void checkAggregateColumns(MemberCtx ctx, MetaMeasure measure, Consumer<String> err) { + /** What M1–M4 resolved: the single {@code @of} field, or {@code null} for a tuple. */ + private record AggregateColumns(MetaData field) { + } + + /** + * M1–M4, in order; the first failure stops the chain (no M2+M3 double report). Returns + * {@code null} when one of them fired; otherwise its {@code field} is the single resolved + * {@code @of} field ({@code null} for a tuple), for M7/M8. + */ + private static AggregateColumns checkAggregateColumns(MemberCtx ctx, MetaMeasure measure, Consumer<String> err) { String agg = measure.getAgg(); List<String> columns = measure.getOfColumns(); @@ -546,19 +647,19 @@ private static void checkAggregateColumns(MemberCtx ctx, MetaMeasure measure, Co Dotted parts = splitDotted(item); if (parts == null || parts.path().size() != 1) { err.accept("@of '" + item + "' must be Entity.field."); - return; + return null; } MetaData named = ValidationPhase.resolveRootObject(ctx.root(), parts.owner(), pkgOf(ctx.declaring())); if (!isSelfOrAncestor(named, ctx.host())) { err.accept("@of '" + item + "' must name a field of the owning entity '" + ctx.declaring().getName() + "'. A measure aggregates its own entity's rows; declare it on the entity that owns the column."); - return; + return null; } MetaData field = fieldOf(ctx.host(), parts.path().get(0)); if (field == null) { err.accept("@of '" + item + "' names no field '" + parts.path().get(0) + "' on '" + ctx.declaring().getName() + "'."); - return; + return null; } fields.add(field); } @@ -567,29 +668,31 @@ private static void checkAggregateColumns(MemberCtx ctx, MetaMeasure measure, Co if (columns.size() > 1 && (!ReportingConstants.AGG_COUNT.equals(agg) || !measure.isDistinct())) { err.accept("@of lists " + columns.size() + " columns; a tuple is legal only with @agg: count and " + "@distinct: true (a distinct count of the tuple)."); - return; + return null; } // M3 — @distinct is a count modifier. if (measure.isDistinct() && agg != null && !ReportingConstants.AGG_COUNT.equals(agg)) { err.accept("@distinct: true requires @agg: count, not '" + agg + "'."); - return; + return null; } // M4 — the aggregate must be meaningful for the column's type. MetaData field = fields.size() == 1 ? fields.get(0) : null; - if (field == null || agg == null) return; + if (field == null || agg == null) return new AggregateColumns(field); String item = columns.get(0); if ((ReportingConstants.AGG_SUM.equals(agg) || ReportingConstants.AGG_AVG.equals(agg)) && !NUMERIC_FIELD_SUBTYPES.contains(field.getSubType())) { err.accept("@agg '" + agg + "' needs a numeric field (field.int, long, double, float, decimal or currency), " + "but '" + item + "' is field." + field.getSubType() + "."); - return; + return null; } if ((ReportingConstants.AGG_MIN.equals(agg) || ReportingConstants.AGG_MAX.equals(agg)) && UNORDERED_FIELD_SUBTYPES.contains(field.getSubType())) { err.accept("@agg '" + agg + "' cannot order '" + item + "', a field." + field.getSubType() + "."); + return null; } + return new AggregateColumns(field); } /** M6 — each operand names a measure.aggregate of the same entity. */ @@ -717,8 +820,8 @@ private static void checkReport(MetaRoot root, MetaObject report, ErrorSink sink } } - // R1 — @from resolves to an object.entity. Without it, R2/R3/R6/R7 and the - // @filter have nothing to resolve against, so they are skipped. + // R1 — @from resolves to an object.entity. Without it, R2/R3/R6/R7, R8/R9 and + // the @filter have nothing to resolve against, so they are skipped. String fromRef = ReportAccessors.reportFrom(report); if (fromRef == null) return; // missing @from is ERR_MISSING_REQUIRED_ATTR MetaObject from = ValidationPhase.resolveRootObject(root, fromRef, pkgOf(report)); @@ -804,6 +907,12 @@ private static void checkReport(MetaRoot root, MetaObject report, ErrorSink sink + "' names no segment of @from '" + fromKey + "'."); } + // R8 — @spine is a to-one path from @from: rule D2's walk, started at @from. + String spine = ReportAccessors.reportSpine(report); + if (spine != null) { + checkSpine(root, report, from, spine, label, sink); + } + // S1 / F2 — the report's row scope over @from. // ADR-0039: resolving — a report that extends another inherits its @filter. if (report.hasMetaAttr(MetaObject.ATTR_FILTER)) { @@ -812,6 +921,50 @@ private static void checkReport(MetaRoot root, MetaObject report, ErrorSink sink } } + /** R8, then R9 (skipped when R8 failed): the report's {@code @spine} and its dimensions. */ + private static void checkSpine(MetaRoot root, MetaObject report, MetaObject from, String spine, String label, + ErrorSink sink) { + String fromKey = from.getName(); + Consumer<String> err = message -> sink.push(report, ErrorCode.ERR_INVALID_REPORT, label, message); + MetaData terminal = walkToOneVia(root, from, spine, message -> err.accept(": " + message), + spineWalk(report, fromKey)); + // R9 — every listed dimension is reached through the spine. Skipped when R8 failed. + if (terminal == null) return; + Dotted spineParts = splitDotted(spine); + List<String> spineHops = spineParts == null ? List.of() : spineParts.path(); + List<ReportDimensionItem> items = ReportAccessors.reportDimensionItems(report); + if (items.isEmpty()) { + err.accept(": @spine '" + spine + "' needs at least one dimension. The report's rows are the dimension " + + "tuples of '" + terminal.getName() + "'; with no dimension it would be one totals row."); + } + // One verdict per dimension, however many grains list it. + Set<String> checkedDimensions = new HashSet<>(); + for (ReportDimensionItem item : items) { + if (!checkedDimensions.add(item.name())) continue; + MetaData dimNode = childOfType(from, ReportingConstants.TYPE_DIMENSION, item.name()); + if (!(dimNode instanceof MetaDimension)) continue; // R2 already reported it + MetaDimension dim = (MetaDimension) dimNode; + String via = dim.getVia(); + if (via == null) { + err.accept(": dimension '" + item.name() + "' is read from @from '" + fromKey + "', so it has no " + + "value in a row that has no facts. With @spine '" + spine + "' every dimension must be " + + "reached through it: declare the dimension over a field of '" + terminal.getName() + + "' (or an entity to-one from it) with an @via that begins '" + spine + "'."); + continue; + } + // A @via that does not walk is D2's error, on the dimension; R9 does not report it again. + MetaData declaring = dim.getParent() != null ? dim.getParent() : from; + if (walkToOneVia(root, from, via, message -> { }, d2Walk(declaring)) == null) continue; + // Hop names are compared as written; the owner segment is not compared. + Dotted viaParts = splitDotted(via); + List<String> hops = viaParts == null ? List.of() : viaParts.path(); + if (hops.size() >= spineHops.size() && hops.subList(0, spineHops.size()).equals(spineHops)) continue; + err.accept(": dimension '" + item.name() + "' is reached by @via '" + via + "', which does not begin " + + "with the hops of @spine '" + spine + "'. Hop names are compared as written: if both name the " + + "same join, write the same hops; otherwise the dimension is not reached through the spine."); + } + } + /** * R3 for one {@code @measures} item (bare {@code name} or dotted {@code Entity.name}). * Returns the measure's name when it is a measure of {@code @from} (for R6), else reports diff --git a/server/java/metadata/src/main/java/com/metaobjects/object/MetaObject.java b/server/java/metadata/src/main/java/com/metaobjects/object/MetaObject.java index 791b9961c..78ff24bd1 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/object/MetaObject.java +++ b/server/java/metadata/src/main/java/com/metaobjects/object/MetaObject.java @@ -107,6 +107,13 @@ public abstract class MetaObject extends MetaData { /** FR-044 — {@code object.report}: optional segment of {@code @from} scoping the rows. */ public static final String ATTR_REPORT_SEGMENT = "segment"; + /** + * FR-044 — {@code object.report}: optional to-one path from {@code @from} to the entity + * whose rows supply the report's rows ({@code Purchase.program}), written like a + * dimension's {@code @via}. + */ + public static final String ATTR_REPORT_SPINE = "spine"; + /** * Register MetaObject type and constraints with registry. * Called by ObjectTypesMetaDataProvider during service discovery. @@ -261,6 +268,8 @@ public static void registerEntityValueTypes(MetaDataRegistry registry) { .ofType(StringAttribute.SUBTYPE_STRING).asSingle(); def.optionalAttributeWithConstraints(ATTR_FILTER) .ofType(FilterAttribute.SUBTYPE_FILTER).asSingle(); + def.optionalAttributeWithConstraints(ATTR_REPORT_SPINE) + .ofType(StringAttribute.SUBTYPE_STRING).asSingle(); }); // ADR-0006 Rule 1 — bare `object:` YAML key fuses to `object.entity`. diff --git a/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java b/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java index c2d7a8b43..d73f9b54b 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java +++ b/server/java/metadata/src/main/java/com/metaobjects/registry/RegistryManifest.java @@ -261,8 +261,10 @@ public static List<MetaDataTypeProvider> metamodelProviders() { /** * The polymorphic / untyped attr whose value-type follows its owning field's * subtype. The manifest renders its {@code valueType} as an explicit - * {@code null} literal (mirrors the TS reference, where {@code @default} has - * no fixed value-type). + * {@code null} literal (mirrors the TS reference, where a field's {@code @default} + * has no fixed value-type). FIELD registrations only ({@link #isPolymorphicDefault}): + * a measure's {@code @default} (FR-044) shares the name but is an attr.int and + * prints {@code int}. */ private static final String POLYMORPHIC_DEFAULT_ATTR = MetaField.ATTR_DEFAULT; @@ -464,7 +466,7 @@ private static List<ManifestAttr> attrsOf(MetaDataRegistry registry, String type if (classifyPerTypeAttr(name) != ExclusionReason.INCLUDED) { continue; // carved out (structural keyword / native binding / description dup) — see classifyPerTypeAttr } - String valueType = valueTypeOf(name, req.getExpectedSubType()); + String valueType = valueTypeOf(type, name, req.getExpectedSubType()); // Array-ness is the orthogonal axis: a StringAttribute requirement // marked .asArray() emits valueType "string" + isArray true (the // retired stringarray subtype). Detected via the array-constraint set. @@ -624,7 +626,7 @@ private static boolean isArrayAttr(MetaDataRegistry registry, String type, Strin /** * Map a Java attr requirement to the cross-port manifest value-type vocabulary. - * The polymorphic {@code @default} attr is rendered as {@code null}; every + * A field's polymorphic {@code @default} attr is rendered as {@code null}; every * other attr carries its {@code expectedSubType} verbatim (the attr subtype * names — {@code string}, {@code int}, {@code boolean}, {@code long}, * {@code double}, {@code properties}, {@code filter} — are already the @@ -632,8 +634,8 @@ private static boolean isArrayAttr(MetaDataRegistry registry, String type, Strin * {@code string}, with array-ness carried by the separate {@code isArray} * flag). A wildcard subtype is rendered as {@code null} (untyped). */ - private static String valueTypeOf(String attrName, String expectedSubType) { - if (POLYMORPHIC_DEFAULT_ATTR.equals(attrName)) { + private static String valueTypeOf(String ownerType, String attrName, String expectedSubType) { + if (isPolymorphicDefault(ownerType, attrName)) { return null; } if (expectedSubType == null || WILDCARD.equals(expectedSubType)) { @@ -642,6 +644,16 @@ private static String valueTypeOf(String attrName, String expectedSubType) { return expectedSubType; } + /** + * True for a FIELD's {@code @default}: registered on {@code field.base} as a string and + * re-read by the field's own subtype, so it has no fixed value-type. Keyed by owner type + * AND name, never by name alone: {@code measure.aggregate} / {@code measure.ratio} + * register an attr.int of the same name (FR-044), which keeps its {@code int}. + */ + private static boolean isPolymorphicDefault(String ownerType, String attrName) { + return MetaField.TYPE_FIELD.equals(ownerType) && POLYMORPHIC_DEFAULT_ATTR.equals(attrName); + } + // ------------------------------------------------------------------ // Emit // ------------------------------------------------------------------ diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/AggregateMeasure.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/AggregateMeasure.java index cd2fd0520..b41ea7541 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/reporting/AggregateMeasure.java +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/AggregateMeasure.java @@ -18,6 +18,7 @@ import com.metaobjects.MetaData; import com.metaobjects.attr.BooleanAttribute; import com.metaobjects.attr.FilterAttribute; +import com.metaobjects.attr.IntAttribute; import com.metaobjects.attr.StringAttribute; import com.metaobjects.registry.MetaDataRegistry; @@ -52,6 +53,8 @@ public static void registerTypes(MetaDataRegistry registry) { .ofType(FilterAttribute.SUBTYPE_FILTER).asSingle(); def.optionalAttributeWithConstraints(ReportingConstants.ATTR_SEGMENT) .ofType(StringAttribute.SUBTYPE_STRING).asSingle(); + def.optionalAttributeWithConstraints(ReportingConstants.ATTR_DEFAULT) + .ofType(IntAttribute.SUBTYPE_INT).asSingle(); }); } } diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/MetaMeasure.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/MetaMeasure.java index 03715d13f..9714dc262 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/reporting/MetaMeasure.java +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/MetaMeasure.java @@ -65,6 +65,20 @@ public Map<String, Object> getFilter() { return ReportingAttrs.filterMap(this, ReportingConstants.ATTR_FILTER); } + /** + * {@code @default}: the integer this measure reads when it would otherwise be null + * (loader rules M7/M8 decide where it is legal). {@code null} when none is declared or + * the value is not an integer. Resolving (ADR-0039), so an inherited measure keeps it. + */ + public Long getDefaultValue() { + if (!hasMetaAttr(ReportingConstants.ATTR_DEFAULT)) return null; + Object v = getMetaAttr(ReportingConstants.ATTR_DEFAULT).getValue(); + if (v instanceof Integer || v instanceof Long || v instanceof Short || v instanceof Byte) { + return ((Number) v).longValue(); + } + return null; + } + /** Name of the {@code measure.aggregate} sibling used as the numerator ({@code measure.ratio} only). */ public String getNumerator() { return ReportingAttrs.optionalString(this, ReportingConstants.ATTR_NUMERATOR); diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/RatioMeasure.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/RatioMeasure.java index 50ba8f8b6..ee1ce9162 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/reporting/RatioMeasure.java +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/RatioMeasure.java @@ -16,6 +16,7 @@ package com.metaobjects.reporting; import com.metaobjects.MetaData; +import com.metaobjects.attr.IntAttribute; import com.metaobjects.attr.StringAttribute; import com.metaobjects.registry.MetaDataRegistry; @@ -40,6 +41,8 @@ public static void registerTypes(MetaDataRegistry registry) { .ofType(StringAttribute.SUBTYPE_STRING).asSingle(); def.requiredAttributeWithConstraints(ReportingConstants.ATTR_DENOMINATOR) .ofType(StringAttribute.SUBTYPE_STRING).asSingle(); + def.optionalAttributeWithConstraints(ReportingConstants.ATTR_DEFAULT) + .ofType(IntAttribute.SUBTYPE_INT).asSingle(); }); } } diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportAccessors.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportAccessors.java index b86065302..9c2e15a08 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportAccessors.java +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportAccessors.java @@ -44,6 +44,15 @@ public static String reportFrom(MetaData report) { return ReportingAttrs.optionalString(report, MetaObject.ATTR_REPORT_FROM); } + /** + * {@code @spine}: the to-one path from {@code @from} to the entity whose rows supply the + * report's rows, or {@code null} when absent or empty. + */ + public static String reportSpine(MetaData report) { + String v = ReportingAttrs.optionalString(report, MetaObject.ATTR_REPORT_SPINE); + return v == null || v.isEmpty() ? null : v; + } + /** The {@code @dimensions} items, each {@code name} or {@code name:grain} (split at the first separator). */ public static List<ReportDimensionItem> reportDimensionItems(MetaData report) { List<ReportDimensionItem> out = new ArrayList<>(); diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportShape.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportShape.java index a0c6f66bc..3bea93ec9 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportShape.java +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportShape.java @@ -27,6 +27,7 @@ import com.metaobjects.field.LongField; import com.metaobjects.field.MetaField; import com.metaobjects.field.TimestampField; +import com.metaobjects.identity.PrimaryIdentity; import com.metaobjects.loader.ValidationPhase; import com.metaobjects.object.MetaObject; import com.metaobjects.reporting.ReportAccessors.ReportDimensionItem; @@ -42,7 +43,9 @@ * A report's derived fields (FR-044, contract Table B): the read shape of an * {@code object.report}, which declares no fields of its own. One field per * {@code @dimensions} item in listed order, then one per {@code @measures} item in - * listed order. + * listed order. {@code required} follows Table C of + * {@code docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md}, which + * amends Table B for a report with {@code @spine} and a measure with {@code @default}. * * <p>The single definition in the JVM ports — the Kotlin generators consume this class * rather than restating the table. Rule-for-rule the TypeScript {@code report-shape.ts}; @@ -112,6 +115,9 @@ public String typeSourceKey() { /** The member separator of an {@code Entity.field} reference. */ private static final String SEP = "."; + /** The package separator of a qualified entity name. */ + private static final String PKG_SEP = MetaData.PKG_SEPARATOR; + private static final Set<String> SUM_LONG = Set.of(IntegerField.SUBTYPE_INT, LongField.SUBTYPE_LONG); private static final Set<String> FLOATING = Set.of(DoubleField.SUBTYPE_DOUBLE, FloatField.SUBTYPE_FLOAT); @@ -201,7 +207,7 @@ public static ReportShape of(MetaObject report) { } /** - * Table B. + * Table B, with Table C's {@code required}. * * @param report an {@code object.report} * @param root the model its references resolve in @@ -214,9 +220,10 @@ public static ReportShape of(MetaObject report, MetaRoot root) { MetaObject from = ValidationPhase.resolveRootObject(root, fromName, packageOf(report)); if (from == null) throw unresolved(report, "@from '" + fromName + "'"); + List<String> spine = reportSpineHops(report, from, root); List<Field> fields = new ArrayList<>(); for (ReportDimensionItem item : ReportAccessors.reportDimensionItems(report)) { - fields.add(dimensionField(item, from, root, report)); + fields.add(dimensionField(item, from, root, report, spine)); } for (String item : ReportAccessors.reportMeasureNames(report)) { fields.add(measureField(item, from, root, report)); @@ -279,6 +286,75 @@ public static MetaObject resolveFieldRefEntity(String ref, MetaData declaring, M return ValidationPhase.resolveRootObject(root, ref.substring(0, dot), packageOf(declaring)); } + /** + * The hop names of a dimension's {@code @via} ({@code Owner.hop[.hop...]}), read as the + * loader reads it (reporting validation rule D2): {@code Owner} resolves in the package of + * {@code declaring} ({@link #memberOwner}) and must be {@code from} or an entity + * {@code from} extends. The walk itself then starts AT {@code from}, whichever of the two + * {@code Owner} named. {@code null} when the reference has no owner, no hop, or an owner + * that is not {@code from} or an ancestor of it. + */ + public static List<String> reportingViaHops(String via, MetaData declaring, MetaObject from, MetaRoot root) { + // The owner ends at the first `.` after the last `::` (a package qualifier has no `.`). + int lastSep = via.lastIndexOf(PKG_SEP); + int segStart = lastSep == -1 ? 0 : lastSep + PKG_SEP.length(); + int dot = via.indexOf(SEP, segStart); + if (dot <= segStart) return null; + List<String> hops = List.of(via.substring(dot + SEP.length()).split(java.util.regex.Pattern.quote(SEP), -1)); + for (String h : hops) { + if (h.isEmpty()) return null; + } + MetaObject owner = ValidationPhase.resolveRootObject(root, via.substring(0, dot), packageOf(declaring)); + if (owner == null || !isSelfOrAncestor(owner, from)) return null; + return hops; + } + + /** + * The hop names of a report's {@code @spine} ({@code Owner.hop[.hop...]}), read as + * {@link #reportingViaHops} reads a {@code @via}, the owner resolving in the REPORT's + * package (loader rule R8). {@code null} when the report declares no {@code @spine}. + * + * @throws MetaDataException naming the report, when a declared {@code @spine} does not + * resolve (a report that passed the loader's reporting validation + * always resolves): reading it as "no spine" would claim a column + * non-null that a spine row with no facts leaves null + */ + public static List<String> reportSpineHops(MetaObject report, MetaObject from, MetaRoot root) { + String spine = ReportAccessors.reportSpine(report); + if (spine == null) return null; + List<String> hops = reportingViaHops(spine, report, from, root); + if (hops == null) throw unresolved(report, "@spine '" + spine + "'"); + return hops; + } + + /** True when {@code field} is one of the {@code @fields} of {@code entity}'s {@code identity.primary}. */ + private static boolean isPrimaryKeyField(MetaObject entity, MetaField<?> field) { + // ADR-0039: resolving — an identity.primary (and its @fields) inherited from an + // abstract base counts (getPrimaryIdentity and getFields both read through extends). + PrimaryIdentity pk = entity.getPrimaryIdentity(); + return pk != null && pk.getFields().contains(field.getName()); + } + + /** + * Table C ({@code required} of a dimension). Without {@code @spine}: only a dimension with + * no {@code @via} over an {@code @of} field whose effective {@code @required} is true. With + * {@code @spine}: only a dimension whose {@code @via} hops equal the spine's (a column of + * the spine entity itself, whose rows are the report's rows) over an {@code @of} field that + * is {@code @required} or a primary-key column of the entity {@code @of} names. A dimension + * beyond the spine is reached by a LEFT OUTER join. + */ + private static boolean dimensionRequired(MetaDimension dim, MetaObject named, MetaField<?> of, MetaObject from, + MetaRoot root, List<String> spine, MetaObject report) { + String via = dim.getVia(); + // Attr only: a validator.required child does not make the column non-null. + boolean ofRequired = ReportingAttrs.isTrue(of, MetaField.ATTR_REQUIRED); + if (spine == null) return via == null && ofRequired; + if (via == null) return false; // a column of @from: null in a spine row with no facts + List<String> hops = reportingViaHops(via, memberOwner(dim, from), from, root); + if (hops == null) throw unresolved(report, "dimension '" + dim.getShortName() + "' @via '" + via + "'"); + return hops.equals(spine) && (ofRequired || isPrimaryKeyField(named, of)); + } + /** True when {@code candidate} is {@code entity} or an entity it extends (the super chain). */ private static boolean isSelfOrAncestor(MetaData candidate, MetaData entity) { Set<MetaData> visited = Collections.newSetFromMap(new IdentityHashMap<>()); @@ -289,17 +365,21 @@ private static boolean isSelfOrAncestor(MetaData candidate, MetaData entity) { return false; } - private static Field dimensionField(ReportDimensionItem item, MetaObject from, MetaRoot root, MetaObject report) { + private static Field dimensionField(ReportDimensionItem item, MetaObject from, MetaRoot root, MetaObject report, + List<String> spine) { MetaDimension dim = declaredMember(from, MetaDimension.class, item.name()); if (dim == null) throw unresolved(report, "dimension '" + item.name() + "' on '" + from.getShortName() + "'"); boolean vialess = dim.getVia() == null; + MetaData declaring = memberOwner(dim, from); MetaField<?> of = dim.getOf() == null ? null - : resolveFieldRef(dim.getOf(), memberOwner(dim, from), root, vialess ? from : null); + : resolveFieldRef(dim.getOf(), declaring, root, vialess ? from : null); if (of == null) throw unresolved(report, "dimension '" + item.name() + "' @of"); + // The entity @of names (with a host, it may be an ancestor of the entity the field is + // read from); Table C reads its primary identity. Non-null: resolveFieldRef resolved it. + MetaObject named = resolveFieldRefEntity(dim.getOf(), declaring, root); String name = ReportAccessors.reportDerivedFieldName(item); - // Attr only: a validator.required child does not make the column non-null. - boolean required = vialess && ReportingAttrs.isTrue(of, MetaField.ATTR_REQUIRED); + boolean required = dimensionRequired(dim, named, of, from, root, spine, report); if (dim.isTime()) { // Loader rule R2 guarantees a grain from the closed set; a tree built in code does not. String grain = item.grain(); @@ -332,13 +412,15 @@ private static Field measureField(String item, MetaObject from, MetaRoot root, M } MetaMeasure m = declaredMember(from, MetaMeasure.class, name); if (m == null) throw unresolved(report, "measure '" + item + "' on '" + from.getShortName() + "'"); + String agg = m.getAgg(); + // Table C: a count is never null (with or without @distinct); any other measure is + // not null when it has a @default. + boolean required = (!m.isRatio() && ReportingConstants.AGG_COUNT.equals(agg)) || m.getDefaultValue() != null; if (m.isRatio()) { - return new Field(name, Role.MEASURE, DecimalField.SUBTYPE_DECIMAL, false, null, null, null, m); + return new Field(name, Role.MEASURE, DecimalField.SUBTYPE_DECIMAL, required, null, null, null, m); } - String agg = m.getAgg(); if (ReportingConstants.AGG_COUNT.equals(agg)) { - // A count is never null, with or without @distinct. - return new Field(name, Role.MEASURE, LongField.SUBTYPE_LONG, true, null, null, null, m); + return new Field(name, Role.MEASURE, LongField.SUBTYPE_LONG, required, null, null, null, m); } List<String> columns = m.getOfColumns(); MetaField<?> of = columns.isEmpty() ? null @@ -347,19 +429,19 @@ private static Field measureField(String item, MetaObject from, MetaRoot root, M String src = of.getSubType(); if (ReportingConstants.AGG_SUM.equals(agg)) { if (CurrencyField.SUBTYPE_CURRENCY.equals(src)) { - return new Field(name, Role.MEASURE, CurrencyField.SUBTYPE_CURRENCY, false, of, null, null, m); + return new Field(name, Role.MEASURE, CurrencyField.SUBTYPE_CURRENCY, required, of, null, null, m); } String subType = SUM_LONG.contains(src) ? LongField.SUBTYPE_LONG : FLOATING.contains(src) ? DoubleField.SUBTYPE_DOUBLE : DecimalField.SUBTYPE_DECIMAL; - return new Field(name, Role.MEASURE, subType, false, null, null, null, m); + return new Field(name, Role.MEASURE, subType, required, null, null, null, m); } if (ReportingConstants.AGG_AVG.equals(agg)) { String subType = FLOATING.contains(src) ? DoubleField.SUBTYPE_DOUBLE : DecimalField.SUBTYPE_DECIMAL; - return new Field(name, Role.MEASURE, subType, false, null, null, null, m); + return new Field(name, Role.MEASURE, subType, required, null, null, null, m); } // min / max keep the source field's type. - return new Field(name, Role.MEASURE, src, false, of, null, null, m); + return new Field(name, Role.MEASURE, src, required, of, null, null, m); } private static <T extends MetaData> T declaredMember(MetaObject from, Class<T> type, String name) { diff --git a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportingConstants.java b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportingConstants.java index f8d9be08a..708558238 100644 --- a/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportingConstants.java +++ b/server/java/metadata/src/main/java/com/metaobjects/reporting/ReportingConstants.java @@ -79,6 +79,12 @@ private ReportingConstants() { /** {@code measure.ratio} operand names. */ public static final String ATTR_NUMERATOR = "numerator"; public static final String ATTR_DENOMINATOR = "denominator"; + /** + * The integer a {@code measure.aggregate} / {@code measure.ratio} reads when it would + * otherwise be null (an attr.int). Same name as a field's {@code @default}, a different + * registration: a field's is an untyped string re-read by the field's subtype. + */ + public static final String ATTR_DEFAULT = "default"; // ------------------------------------------------------------------ // Closed sets — mirrored by allowedValues in spec/metamodel/reporting.json. diff --git a/server/java/metadata/src/test/java/com/metaobjects/loader/ReportingValidationTest.java b/server/java/metadata/src/test/java/com/metaobjects/loader/ReportingValidationTest.java index 558412406..aafaa562b 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/loader/ReportingValidationTest.java +++ b/server/java/metadata/src/test/java/com/metaobjects/loader/ReportingValidationTest.java @@ -39,6 +39,7 @@ import java.util.stream.Stream; import static org.junit.Assert.assertEquals; +import static org.junit.Assert.assertFalse; import static org.junit.Assert.assertNotNull; import static org.junit.Assert.assertTrue; @@ -185,6 +186,23 @@ public void everyErrorFixtureYieldsTheReferenceCodeAndMessage() throws IOExcepti "report 'acme::shop::StoreTotals': source.rdb is writable; a report is read-only, so its source must declare a read-only @kind (view, materializedView, storedProc or tableFunction)."}); expect.put("error-segment-filter-bad-field", new String[]{"ERR_BAD_ATTR_FILTER", "segment 'active' on entity 'acme::shop::Purchase': @filter names 'state', which is not a field of 'acme::shop::Purchase'."}); + // R8 / R9 / M7 / M8 (FR-044 zero rows and measure defaults). error-measure-default-not-integer + // is not here: this port refuses a fractional @default in the generic attr.int parse, so the + // load stops before this pass runs (aFractionalMeasureDefaultIsOneBadAttrValueOnTheMeasure). + expect.put("error-report-spine-to-many", new String[]{"ERR_INVALID_REPORT", + "report 'acme::shop::ProgramSales': @spine 'Purchase.program.purchases' crosses relationship 'purchases' on 'acme::shop::Program', which is not to-one. A @spine follows only @cardinality: one relationships and identity.reference hops, so each fact row joins at most one row of the spine entity and is never counted twice."}); + expect.put("error-report-spine-not-from", new String[]{"ERR_INVALID_REPORT", + "report 'acme::shop::ProgramSales': @spine 'Program.catalog' must start at @from 'acme::shop::Purchase'."}); + expect.put("error-report-spine-no-dimensions", new String[]{"ERR_INVALID_REPORT", + "report 'acme::shop::CatalogSales': @spine 'Purchase.program.catalog' needs at least one dimension. The report's rows are the dimension tuples of 'acme::shop::Catalog'; with no dimension it would be one totals row."}); + expect.put("error-report-spine-dimension-off-spine", new String[]{"ERR_INVALID_REPORT", + "report 'acme::shop::ProgramSales': dimension 'purchasedAt' is read from @from 'acme::shop::Purchase', so it has no value in a row that has no facts. With @spine 'Purchase.program' every dimension must be reached through it: declare the dimension over a field of 'acme::shop::Program' (or an entity to-one from it) with an @via that begins 'Purchase.program'."}); + expect.put("error-report-spine-dimension-other-path", new String[]{"ERR_INVALID_REPORT", + "report 'acme::shop::ProgramSales': dimension 'programByRef' is reached by @via 'Purchase.fkProgram', which does not begin with the hops of @spine 'Purchase.program'. Hop names are compared as written: if both name the same join, write the same hops; otherwise the dimension is not reached through the spine."}); + expect.put("error-measure-default-on-count", new String[]{"ERR_INVALID_MEASURE", + "measure 'purchases' on entity 'acme::shop::Purchase': @default cannot apply to @agg: count. A count is never null (it is 0 when nothing matches); remove @default."}); + expect.put("error-measure-default-non-numeric", new String[]{"ERR_INVALID_MEASURE", + "measure 'lastPurchaseAt' on entity 'acme::shop::Purchase': @default is a number, but 'Purchase.purchasedAt', the @of of @agg 'max', is a field.timestamp. A default is supported on numeric measures only."}); List<String> failures = new ArrayList<>(); for (Map.Entry<String, String[]> e : expect.entrySet()) { @@ -215,6 +233,8 @@ public void everyErrorFixtureYieldsTheReferenceCodeAndMessage() throws IOExcepti public void positiveFixturesAreClean() throws IOException { assertEquals(List.of(), ValidationPhase.validateReporting(loadJson(cleanModel()))); assertEquals(List.of(), ValidationPhase.validateReporting(loadJson(fixtureDoc("reporting-inherited-members")))); + assertEquals(List.of(), ValidationPhase.validateReporting(loadJson(fixtureDoc("reporting-spine-and-default")))); + assertEquals(List.of(), ValidationPhase.validateReporting(loadJson(fixtureDoc("reporting-spine-inherited")))); } // --------------------------------------------------------------------------- @@ -456,4 +476,395 @@ public void aBareStringOfWithACommaIsOneInvalidItem() throws IOException { assertEquals("measure 'revenue' on entity 'acme::shop::Purchase': @of 'Purchase.amountCents,Purchase.id' " + "must be Entity.field.", got.get(0).getMessage()); } + + // --------------------------------------------------------------------------- + // R8 / R9 — a report's @spine (Table B of + // docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md). + // The same cases as the TypeScript reporting-validation.test.ts, message for message. + // --------------------------------------------------------------------------- + + private static JsonObject wrap(String typeKey, String body) { + JsonObject w = new JsonObject(); + w.add(typeKey, json(body)); + return w; + } + + private static JsonArray childrenOf(JsonObject doc, String entity) { + return rootNode(doc, "object.entity", entity).getAsJsonArray("children"); + } + + /** Append report {@code ProgramPurchases} over Purchase with {@code @spine: Purchase.program}; + * {@code attrs} override (a JSON null deletes). */ + private static void addSpineReport(JsonObject doc, String attrs) { + JsonObject report = json("{\"name\":\"ProgramPurchases\",\"@from\":\"Purchase\",\"@spine\":\"Purchase.program\"," + + "\"@dimensions\":[\"programTitle\"],\"@measures\":[\"purchases\",\"revenue\"]}"); + for (Map.Entry<String, JsonElement> e : json(attrs).entrySet()) { + if (e.getValue().isJsonNull()) report.remove(e.getKey()); + else report.add(e.getKey(), e.getValue()); + } + JsonObject wrapper = new JsonObject(); + wrapper.add("object.report", report); + rootChildren(doc).add(wrapper); + } + + /** Program -> Coach -> Agency, both to-one, plus Purchase dimensions at the end of each hop. */ + private static void addCoachChain(JsonObject doc) { + JsonArray program = childrenOf(doc, "Program"); + program.add(wrap("field.long", "{\"name\":\"coachId\"}")); + program.add(wrap("identity.reference", "{\"name\":\"coachRef\",\"@references\":\"Coach\",\"@fields\":[\"coachId\"]}")); + program.add(wrap("relationship.association", "{\"name\":\"coach\",\"@objectRef\":\"Coach\",\"@cardinality\":\"one\"}")); + rootChildren(doc).add(wrap("object.entity", "{\"name\":\"Coach\",\"children\":[" + + "{\"source.rdb\":{\"@table\":\"coaches\"}}," + + "{\"field.long\":{\"name\":\"id\"}}," + + "{\"field.string\":{\"name\":\"name\"}}," + + "{\"field.long\":{\"name\":\"agencyId\"}}," + + "{\"identity.primary\":{\"name\":\"id\",\"@fields\":[\"id\"]}}," + + "{\"identity.reference\":{\"name\":\"agencyRef\",\"@references\":\"Agency\",\"@fields\":[\"agencyId\"]}}," + + "{\"relationship.association\":{\"name\":\"agency\",\"@objectRef\":\"Agency\",\"@cardinality\":\"one\"}}]}")); + rootChildren(doc).add(wrap("object.entity", "{\"name\":\"Agency\",\"children\":[" + + "{\"source.rdb\":{\"@table\":\"agencies\"}}," + + "{\"field.long\":{\"name\":\"id\"}}," + + "{\"field.string\":{\"name\":\"name\"}}," + + "{\"identity.primary\":{\"name\":\"id\",\"@fields\":[\"id\"]}}]}")); + JsonArray purchase = childrenOf(doc, "Purchase"); + purchase.add(wrap("dimension.attribute", + "{\"name\":\"coachName\",\"@of\":\"Coach.name\",\"@via\":\"Purchase.program.coach\"}")); + purchase.add(wrap("dimension.attribute", + "{\"name\":\"agencyName\",\"@of\":\"Agency.name\",\"@via\":\"Purchase.program.coach.agency\"}")); + } + + /** A dimension of Purchase reaching Program.title through the identity.reference, not the relationship. */ + private static JsonObject programTitleByRef() { + return wrap("dimension.attribute", + "{\"name\":\"programTitleByRef\",\"@of\":\"Program.title\",\"@via\":\"Purchase.programRef\"}"); + } + + /** The one finding of the reporting pass over {@code doc}, asserting its code. */ + private static String single(JsonObject doc, String code) { + List<MetaDataException> got = ValidationPhase.validateReporting(loadJson(doc)); + assertEquals(got.toString(), List.of(code), codes(got)); + return got.get(0).getMessage(); + } + + @Test + public void spineReportWhoseDimensionsAreAllReachedThroughTheSpineLoadsClean() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{}"); + assertEquals(List.of(), loadErrors(doc)); + } + + @Test + public void r8ToManySpineHopIsRefusedNamingTheReportAndTheHopsEntity() throws IOException { + JsonObject doc = cleanModel(); + childrenOf(doc, "Program").add(wrap("measure.aggregate", + "{\"name\":\"programs\",\"@agg\":\"count\",\"@of\":\"Program.id\"}")); + rootChildren(doc).add(wrap("object.report", "{\"name\":\"ProgramReach\",\"@from\":\"Program\"," + + "\"@spine\":\"Program.purchases\",\"@measures\":[\"programs\"]}")); + assertEquals("report 'acme::shop::ProgramReach': @spine 'Program.purchases' crosses relationship 'purchases' on " + + "'acme::shop::Program', which is not to-one. A @spine follows only @cardinality: one relationships " + + "and identity.reference hops, so each fact row joins at most one row of the spine entity and is never " + + "counted twice.", single(doc, "ERR_INVALID_REPORT")); + } + + @Test + public void r8SpineWhoseOwnerIsAnotherEntityIsRefused() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@spine\":\"Program.purchases\"}"); + assertEquals("report 'acme::shop::ProgramPurchases': @spine 'Program.purchases' must start at @from " + + "'acme::shop::Purchase'.", single(doc, "ERR_INVALID_REPORT")); + } + + @Test + public void r8SpineHopThatNamesNothingNamesFromAsTheHopsEntity() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@spine\":\"Purchase.nope\"}"); + assertEquals("report 'acme::shop::ProgramPurchases': @spine 'Purchase.nope' names 'nope', which is not a " + + "relationship or identity.reference of 'acme::shop::Purchase'.", single(doc, "ERR_INVALID_REPORT")); + } + + @Test + public void r8SpineWithNoHopIsRefused() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@spine\":\"Purchase\"}"); + assertEquals("report 'acme::shop::ProgramPurchases': @spine 'Purchase' must be Owner.hop[.hop...], starting at " + + "@from 'acme::shop::Purchase'.", single(doc, "ERR_INVALID_REPORT")); + } + + @Test + public void d2WordingIsUnchangedByTheWalksWordingArgument() throws IOException { + String head = "dimension 'programTitle' on entity 'acme::shop::Purchase': "; + String[][] cases = { + {"Purchase", head + "@via 'Purchase' must be Owner.hop[.hop...], starting at the owning entity."}, + {"WorkoutEvent.program", head + "@via 'WorkoutEvent.program' must start at the owning entity " + + "'acme::shop::Purchase'."}, + {"Purchase.nope", head + "@via 'Purchase.nope' names 'nope', which is not a relationship or " + + "identity.reference of 'acme::shop::Purchase'."}, + }; + for (String[] c : cases) { + JsonObject doc = cleanModel(); + member(doc, "Purchase", "dimension.attribute", "programTitle").addProperty("@via", c[0]); + assertEquals(c[1], single(doc, "ERR_INVALID_DIMENSION")); + } + } + + @Test + public void r8ErrorSourceIsTheReportNode() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@spine\":\"Purchase.nope\"}"); + List<MetaDataException> got = loadErrors(doc); + assertEquals(List.of("ERR_INVALID_REPORT"), codes(got)); + String path = String.valueOf(got.get(0).getEnvelope().orElse(null)); + assertTrue(path, path.contains("['object.report']")); + assertTrue(path, path.endsWith("['object.report'], yamlPosition=null]")); + } + + @Test + public void r8FailingSkipsR9() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@spine\":\"Purchase.nope\",\"@dimensions\":[\"purchasedAt:day\"]}"); + assertTrue(single(doc, "ERR_INVALID_REPORT").contains("@spine 'Purchase.nope' names 'nope'")); + } + + @Test + public void r8InheritedSpineWithTheAbstractBaseAsOwnerLoadsClean() throws IOException { + JsonObject doc = fixtureDoc("reporting-inherited-members"); + rootChildren(doc).add(wrap("object.entity", "{\"name\":\"Program\",\"children\":[" + + "{\"source.rdb\":{\"@table\":\"programs\"}}," + + "{\"field.long\":{\"name\":\"id\"}}," + + "{\"field.string\":{\"name\":\"title\"}}," + + "{\"identity.primary\":{\"name\":\"id\",\"@fields\":[\"id\"]}}]}")); + JsonArray base = childrenOf(doc, "BaseEvent"); + base.add(wrap("field.long", "{\"name\":\"programId\"}")); + base.add(wrap("identity.reference", "{\"name\":\"programRef\",\"@references\":\"Program\",\"@fields\":[\"programId\"]}")); + base.add(wrap("relationship.association", "{\"name\":\"program\",\"@objectRef\":\"Program\",\"@cardinality\":\"one\"}")); + base.add(wrap("dimension.attribute", + "{\"name\":\"programTitle\",\"@of\":\"Program.title\",\"@via\":\"BaseEvent.program\"}")); + rootChildren(doc).add(wrap("object.report", "{\"name\":\"ProgramEvents\",\"@from\":\"WorkoutEvent\"," + + "\"@spine\":\"BaseEvent.program\",\"@dimensions\":[\"programTitle\"],\"@measures\":[\"events\"]}")); + assertEquals(List.of(), loadErrors(doc)); + } + + @Test + public void r9SpineReportWithNoDimensionsIsRefused() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@dimensions\":null}"); + assertEquals("report 'acme::shop::ProgramPurchases': @spine 'Purchase.program' needs at least one dimension. The " + + "report's rows are the dimension tuples of 'acme::shop::Program'; with no dimension it would be one " + + "totals row.", single(doc, "ERR_INVALID_REPORT")); + } + + @Test + public void r9DimensionWithNoViaIsRefused() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@dimensions\":[\"programTitle\",\"program\"]}"); + assertEquals("report 'acme::shop::ProgramPurchases': dimension 'program' is read from @from 'acme::shop::Purchase', so " + + "it has no value in a row that has no facts. With @spine 'Purchase.program' every dimension must be " + + "reached through it: declare the dimension over a field of 'acme::shop::Program' (or an entity to-one " + + "from it) with an @via that begins 'Purchase.program'.", single(doc, "ERR_INVALID_REPORT")); + } + + @Test + public void r9TimeDimensionOverAFactColumnIsRefused() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@dimensions\":[\"programTitle\",\"purchasedAt:day\"]}"); + assertTrue(single(doc, "ERR_INVALID_REPORT").contains( + "report 'acme::shop::ProgramPurchases': dimension 'purchasedAt' is read from @from")); + } + + @Test + public void r9EachOffendingDimensionIsReportedOnceEvenWhenListedAtTwoGrains() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@dimensions\":[\"program\",\"purchasedAt:day\",\"purchasedAt:week\",\"programTitle\"]}"); + List<MetaDataException> got = loadErrors(doc); + assertEquals(List.of("ERR_INVALID_REPORT", "ERR_INVALID_REPORT"), codes(got)); + assertTrue(got.get(0).getMessage().contains("dimension 'program' is read from @from")); + assertTrue(got.get(1).getMessage().contains("dimension 'purchasedAt' is read from @from")); + } + + @Test + public void r9DimensionThroughASecondReferenceToTheSameEntityIsRefused() throws IOException { + JsonObject doc = cleanModel(); + JsonArray purchase = childrenOf(doc, "Purchase"); + purchase.add(wrap("field.long", "{\"name\":\"giftProgramId\"}")); + purchase.add(wrap("identity.reference", + "{\"name\":\"giftProgramRef\",\"@references\":\"Program\",\"@fields\":[\"giftProgramId\"]}")); + purchase.add(programTitleByRef()); + purchase.add(wrap("dimension.attribute", + "{\"name\":\"giftProgramTitle\",\"@of\":\"Program.title\",\"@via\":\"Purchase.giftProgramRef\"}")); + addSpineReport(doc, "{\"@spine\":\"Purchase.programRef\",\"@dimensions\":[\"programTitleByRef\",\"giftProgramTitle\"]}"); + assertEquals("report 'acme::shop::ProgramPurchases': dimension 'giftProgramTitle' is reached by @via " + + "'Purchase.giftProgramRef', which does not begin with the hops of @spine 'Purchase.programRef'. Hop names " + + "are compared as written: if both name the same join, write the same hops; otherwise the dimension is " + + "not reached through the spine.", single(doc, "ERR_INVALID_REPORT")); + } + + @Test + public void r9DimensionWhoseOwnViaFailsD2IsReportedOnceByD2() throws IOException { + JsonObject doc = cleanModel(); + member(doc, "Purchase", "dimension.attribute", "programTitle").addProperty("@via", "Purchase.programm"); + addSpineReport(doc, "{}"); + assertTrue(single(doc, "ERR_INVALID_DIMENSION").contains("@via 'Purchase.programm' names 'programm'")); + } + + @Test + public void r9SameJoinNamedByTheRelationshipInSpineAndTheReferenceInViaIsRefused() throws IOException { + JsonObject doc = cleanModel(); + childrenOf(doc, "Purchase").add(programTitleByRef()); + addSpineReport(doc, "{\"@spine\":\"Purchase.program\",\"@dimensions\":[\"programTitleByRef\"]}"); + String msg = single(doc, "ERR_INVALID_REPORT"); + assertTrue(msg, msg.contains("dimension 'programTitleByRef' is reached by @via 'Purchase.programRef'")); + assertTrue(msg, msg.contains("@spine 'Purchase.program'")); + } + + @Test + public void r9OwnerSegmentIsNotCompared() throws IOException { + JsonObject doc = cleanModel(); + addSpineReport(doc, "{\"@spine\":\"acme::shop::Purchase.program\"}"); + assertEquals(List.of(), loadErrors(doc)); + } + + @Test + public void r9TimeDimensionOverAColumnOfTheSpineEntityIsLegal() throws IOException { + JsonObject doc = cleanModel(); + childrenOf(doc, "Program").add(wrap("field.timestamp", "{\"name\":\"publishedAt\"}")); + childrenOf(doc, "Purchase").add(wrap("dimension.time", "{\"name\":\"programPublishedAt\"," + + "\"@of\":\"Program.publishedAt\",\"@via\":\"Purchase.program\",\"@grains\":[\"month\"]}")); + addSpineReport(doc, "{\"@dimensions\":[\"programTitle\",\"programPublishedAt:month\"]}"); + assertEquals(List.of(), loadErrors(doc)); + } + + @Test + public void r9TwoHopSpineWithADimensionAtItAndOneBeyondItIsLegal() throws IOException { + JsonObject doc = cleanModel(); + addCoachChain(doc); + addSpineReport(doc, "{\"@spine\":\"Purchase.program.coach\",\"@dimensions\":[\"coachName\",\"agencyName\"]}"); + assertEquals(List.of(), loadErrors(doc)); + } + + @Test + public void r9DimensionThatStopsShortOfATwoHopSpineIsRefused() throws IOException { + JsonObject doc = cleanModel(); + addCoachChain(doc); + addSpineReport(doc, "{\"@spine\":\"Purchase.program.coach\",\"@dimensions\":[\"coachName\",\"programTitle\"]}"); + assertTrue(single(doc, "ERR_INVALID_REPORT").contains("dimension 'programTitle' is reached by @via 'Purchase.program'")); + } + + // --------------------------------------------------------------------------- + // M7 / M8 — where a measure's @default can apply (Table B) + // --------------------------------------------------------------------------- + + @Test + public void m7DefaultOnACountIsRefusedOnTheMeasureNode() throws IOException { + JsonObject doc = cleanModel(); + member(doc, "Purchase", "measure.aggregate", "purchases").addProperty("@default", 0); + List<MetaDataException> got = loadErrors(doc); + assertEquals(List.of("ERR_INVALID_MEASURE"), codes(got)); + assertEquals("measure 'purchases' on entity 'acme::shop::Purchase': @default cannot apply to @agg: count. A count is " + + "never null (it is 0 when nothing matches); remove @default.", got.get(0).getMessage()); + String path = String.valueOf(got.get(0).getEnvelope().orElse(null)); + assertTrue(path, path.contains("['measure.aggregate']")); + } + + @Test + public void m7DefaultOnADistinctCountOfATupleIsRefused() throws IOException { + JsonObject doc = cleanModel(); + member(doc, "WorkoutEvent", "measure.aggregate", "daysEngaged").addProperty("@default", 0); + String msg = single(doc, "ERR_INVALID_MEASURE"); + assertTrue(msg, msg.contains("measure 'daysEngaged'")); + assertTrue(msg, msg.contains("@default cannot apply to @agg: count")); + } + + @Test + public void m8DefaultOnAMaxOfATimestampIsRefused() throws IOException { + JsonObject doc = cleanModel(); + member(doc, "WorkoutEvent", "measure.aggregate", "lastActivityAt").addProperty("@default", 0); + assertEquals("measure 'lastActivityAt' on entity 'acme::shop::WorkoutEvent': @default is a number, but " + + "'WorkoutEvent.occurredAt', the @of of @agg 'max', is a field.timestamp. A default is supported on " + + "numeric measures only.", single(doc, "ERR_INVALID_MEASURE")); + } + + /** + * The TypeScript reference refuses a fractional {@code @default} in its reporting pass with + * its own text. This port's generic attr.int parse already refuses it: one + * {@code ERR_BAD_ATTR_VALUE}, on the measure node, and the load stops there (so M7/M8 never + * run for it). No second, reporting-specific error is added. + */ + @Test + public void aFractionalMeasureDefaultIsOneBadAttrValueOnTheMeasure() throws IOException { + String[][] cases = { + // fixture or member edit, the measure's type key, the value the message names + {"Purchase", "revenue", "measure.aggregate", "0.5"}, + {"WorkoutEvent", "avgDaysPerStarter", "measure.ratio", "0.5"}, + {"Purchase", "purchases", "measure.aggregate", "0.5"}, // a count: M7 is skipped + {"WorkoutEvent", "lastActivityAt", "measure.aggregate", "-1.5"}, // a max of a timestamp: M8 is skipped + }; + for (String[] c : cases) { + JsonObject doc = cleanModel(); + member(doc, c[0], c[2], c[1]).add("@default", JsonParser.parseString(c[3])); + List<MetaDataException> got = loadErrors(doc); + assertEquals(c[1] + ": " + got, List.of("ERR_BAD_ATTR_VALUE"), codes(got)); + String path = String.valueOf(got.get(0).getEnvelope().orElse(null)); + assertTrue(path, path.contains("['" + c[2] + "']")); + assertTrue(got.get(0).getMessage(), got.get(0).getMessage().contains("[default]")); + assertTrue(got.get(0).getMessage(), got.get(0).getMessage().contains(c[3])); + } + // The shared fixture: exactly one error, on the measure the expected envelope names. + Path input = corpusRoot().resolve("error-measure-default-not-integer").resolve("input"); + JsonObject doc = JsonParser.parseString(Files.readString(input.resolve("meta.shop.json"), StandardCharsets.UTF_8)) + .getAsJsonObject(); + List<MetaDataException> got = loadErrors(doc); + assertEquals(got.toString(), List.of("ERR_BAD_ATTR_VALUE"), codes(got)); + assertTrue(got.get(0).getMessage(), got.get(0).getMessage().contains("[measure:aggregate:revenue]")); + } + + @Test + public void aFractionalDefaultOnAMeasureOfAnAbstractBaseIsReportedOnce() throws IOException { + JsonObject doc = fixtureDoc("reporting-inherited-members"); + JsonArray base = childrenOf(doc, "BaseEvent"); + base.set(3, wrap("measure.aggregate", + "{\"name\":\"events\",\"@agg\":\"sum\",\"@of\":\"BaseEvent.id\",\"@default\":0.5}")); + assertEquals(List.of("ERR_BAD_ATTR_VALUE"), codes(loadErrors(doc))); + } + + @Test + public void defaultOnASumAnAvgAMinOfAnIntAndARatioIsFine() throws IOException { + JsonObject doc = cleanModel(); + member(doc, "Purchase", "measure.aggregate", "revenue").addProperty("@default", 0); + childrenOf(doc, "Purchase").add(wrap("measure.aggregate", + "{\"name\":\"avgRevenue\",\"@agg\":\"avg\",\"@of\":\"Purchase.amountCents\",\"@default\":0}")); + childrenOf(doc, "WorkoutEvent").add(wrap("measure.aggregate", + "{\"name\":\"firstDay\",\"@agg\":\"min\",\"@of\":\"WorkoutEvent.dayNumber\",\"@default\":1}")); + member(doc, "WorkoutEvent", "measure.ratio", "avgDaysPerStarter").addProperty("@default", 0); + assertEquals(List.of(), loadErrors(doc)); + } + + @Test + public void aMeasureThatBreaksM4AndDeclaresDefaultReportsM4Only() throws IOException { + JsonObject doc = cleanModel(); + JsonObject revenue = member(doc, "Purchase", "measure.aggregate", "revenue"); + revenue.addProperty("@of", "Purchase.status"); + revenue.addProperty("@default", 0); + String msg = single(doc, "ERR_INVALID_MEASURE"); + assertTrue(msg, msg.contains("field.string")); + assertFalse(msg, msg.contains("@default")); + } + + @Test + public void aMeasureThatBreaksM1AndDeclaresDefaultOnACountReportsM1Only() throws IOException { + JsonObject doc = cleanModel(); + JsonObject purchases = member(doc, "Purchase", "measure.aggregate", "purchases"); + purchases.addProperty("@of", "Purchase.nope"); + purchases.addProperty("@default", 0); + String msg = single(doc, "ERR_INVALID_MEASURE"); + assertTrue(msg, msg.contains("names no field 'nope'")); + assertFalse(msg, msg.contains("@default")); + } + + @Test + public void m7OnACountDeclaredOnAnAbstractBaseIsReportedOnce() throws IOException { + JsonObject doc = fixtureDoc("reporting-inherited-members"); + childrenOf(doc, "BaseEvent").set(3, wrap("measure.aggregate", + "{\"name\":\"events\",\"@agg\":\"count\",\"@of\":\"BaseEvent.id\",\"@default\":0}")); + assertTrue(single(doc, "ERR_INVALID_MEASURE").contains("measure 'events' on entity 'acme::shop::BaseEvent'")); + } } diff --git a/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportReadModelTest.java b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportReadModelTest.java index 244dbb42d..c09b7cf21 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportReadModelTest.java +++ b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportReadModelTest.java @@ -301,6 +301,28 @@ public void anUnmanagedSourceIsCopiedLikeAnyOther() { assertTrue(model.getSources().iterator().next().isUnmanaged()); } + @Test + public void aMeasureWithDefaultYieldsADetachedFieldWithRequiredTrue() { + // Table C of docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md. + MetaRoot root = loadJson(""" + { "metadata.root": { "package": "acme", "children": [ + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "field.currency": { "name": "amountCents", "@currency": "EUR" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "increment" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Sale.amountCents", "@default": 0 } }, + { "measure.aggregate": { "name": "revenueRaw", "@agg": "sum", "@of": "Sale.amountCents" } } + ] } }, + { "object.report": { "name": "Revenue", "@from": "Sale", "@measures": ["revenue", "revenueRaw"] } } + ] } } + """); + ReportReadModel model = ReportReadModel.of(object(root, "Revenue"), root); + assertEquals(List.of("revenue", "revenueRaw"), fieldNames(model)); + assertTrue(required(model.getMetaField("revenue"))); + assertFalse(required(model.getMetaField("revenueRaw"))); + } + @Test public void aSourcelessReportHasAShapeAndNoView() { MetaRoot root = loadJson(SALES_MODEL); @@ -326,7 +348,7 @@ public void isDetachedAndLeavesTheLoadedModelUntouched() { for (MetaObject o : canonical.getChildren(MetaObject.class, false)) { if (ReportReadModel.isReport(o)) models.add(ReportReadModel.of(o, canonical)); } - assertEquals(6, models.size()); + assertEquals(9, models.size()); for (ReportReadModel model : models) { assertNull("never attached", model.getParent()); diff --git a/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportShapeTest.java b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportShapeTest.java index ddd8bf1b5..2946f2f8a 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportShapeTest.java +++ b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportShapeTest.java @@ -444,4 +444,294 @@ public void aTimeDimensionItemWithNoGrainOrAGrainOutsideTheClosedSetDoesNotResol assertUnresolved("report 'Stray': time dimension 'createdAt' grain 'fortnight' does not resolve.", stray("fitness::Stray", "Program", MetaObject.ATTR_REPORT_DIMENSIONS, "createdAt:fortnight"), canonical); } + + // --------------------------------------------------------------------------- + // Table C of docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md: + // under @spine a column of the spine entity is not nullable when it is @required or a + // primary-key column; a dimension beyond the spine is; a measure with @default is not. + // The same cases as the TypeScript report-shape.test.ts. + // --------------------------------------------------------------------------- + + private static final String CATALOG = """ + { "object.entity": { "name": "Catalog", "children": [ + { "source.rdb": { "@table": "catalogs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name", "@required": true } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ] } }, + """; + + /** Program's members after its key (Program declares {@code id} + {@code pk} itself, or inherits them). */ + private static final String PROGRAM_MEMBERS = """ + { "field.string": { "name": "title", "@required": true } }, + { "field.string": { "name": "subtitle" } }, + { "field.timestamp": { "name": "publishedAt", "@required": true } }, + { "field.long": { "name": "catalogId" } }, + { "identity.reference": { "name": "fkCatalog", "@references": "Catalog", "@fields": ["catalogId"] } }, + { "relationship.association": { "name": "catalog", "@objectRef": "Catalog", "@cardinality": "one" } } + """; + + private static final String PROGRAM = """ + { "object.entity": { "name": "Program", "children": [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + """ + PROGRAM_MEMBERS + """ + ] } }, + """; + + /** Purchase, with {@code countDefault} spliced into its count measure (empty for none). */ + private static String purchase(String countDefault, String subtitleVia) { + return """ + { "object.entity": { "name": "Purchase", "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "programId" } }, + { "field.int": { "name": "minutes", "@required": true } }, + { "field.currency": { "name": "amountCents", "@currency": "USD" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@references": "Program", "@fields": ["programId"] } }, + { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programSubtitle", "@of": "Program.subtitle", "@via": "SUBTITLE_VIA" } }, + { "dimension.time": { "name": "publishedAt", "@of": "Program.publishedAt", "@via": "Purchase.program", + "@grains": ["hour", "month"] } }, + { "dimension.attribute": { "name": "catalogId", "@of": "Catalog.id", "@via": "Purchase.program.catalog" } }, + { "dimension.attribute": { "name": "catalogName", "@of": "Catalog.name", "@via": "Purchase.program.catalog" } }, + { "dimension.attribute": { "name": "minutes", "@of": "Purchase.minutes" } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id"COUNT_DEFAULT } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } }, + { "measure.aggregate": { "name": "revenueRaw", "@agg": "sum", "@of": "Purchase.amountCents" } }, + { "measure.aggregate": { "name": "avgMinutes", "@agg": "avg", "@of": "Purchase.minutes", "@default": 0 } }, + { "measure.aggregate": { "name": "avgMinutesRaw", "@agg": "avg", "@of": "Purchase.minutes" } }, + { "measure.aggregate": { "name": "minMinutes", "@agg": "min", "@of": "Purchase.minutes", "@default": -1 } }, + { "measure.aggregate": { "name": "minMinutesRaw", "@agg": "min", "@of": "Purchase.minutes" } }, + { "measure.ratio": { "name": "share", "@numerator": "revenue", "@denominator": "purchases", "@default": 0 } }, + { "measure.ratio": { "name": "shareRaw", "@numerator": "revenue", "@denominator": "purchases" } } + ] } }, + """.replace("COUNT_DEFAULT", countDefault).replace("SUBTITLE_VIA", subtitleVia); + } + + private static final String ALL_MEASURES = """ + ["purchases", "revenue", "revenueRaw", "avgMinutes", "avgMinutesRaw", "minMinutes", "minMinutesRaw", + "share", "shareRaw"]"""; + + private static final String SPINE_SALES = """ + { "object.report": { "name": "SpineSales", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle", "programSubtitle", "publishedAt:hour", "publishedAt:month", + "catalogId", "catalogName"], + "@measures": """ + ALL_MEASURES + " } }"; + + private static final String PLAIN_SALES = """ + { "object.report": { "name": "PlainSales", "@from": "Purchase", + "@dimensions": ["programId", "programTitle", "programSubtitle", "catalogId", "catalogName", "minutes"], + "@measures": """ + ALL_MEASURES + " } }"; + + private static final String CATALOG_SALES = """ + { "object.report": { "name": "CatalogSales", "@from": "Purchase", "@spine": "Purchase.program.catalog", + "@dimensions": ["catalogId", "catalogName"], "@measures": ["purchases", "revenue"] } }"""; + + private static String acme(String children) { + return "{ \"metadata.root\": { \"package\": \"acme\", \"children\": [" + children + "] } }"; + } + + private static MetaRoot salesModel() { + return loadJson(acme(CATALOG + PROGRAM + purchase("", "Purchase.program") + + SPINE_SALES + "," + PLAIN_SALES + "," + CATALOG_SALES)); + } + + /** Loads a model the loader refuses (a rule broken on purpose); returns the built tree and the errors. */ + private static MetaRoot loadPastTheLoader(List<MetaDataException> errors, String... files) { + MetaDataLoader loader = new MetaDataLoader( + LoaderOptions.create(false, false, true), MetaDataLoader.SUBTYPE_MANUAL, "report-shape-past-loader"); + loader.setSourceURIs(java.util.Collections.emptyList()); + loader.init(); + List<com.metaobjects.loader.MetaDataSource> sources = new java.util.ArrayList<>(); + for (int i = 0; i < files.length; i++) { + sources.add(new InMemoryStringSource(files[i], "meta.inline" + i + ".json")); + } + try { + loader.load(sources); + } catch (MetaDataException thrown) { + errors.addAll(loader.getErrors()); + errors.add(thrown); + } + return loader.getRoot(); + } + + private static java.util.Map<String, Boolean> requiredOf(MetaRoot root, String report) { + java.util.Map<String, Boolean> out = new java.util.LinkedHashMap<>(); + for (ReportShape.Field f : ReportShape.of(object(root, report), root).fields()) out.put(f.name(), f.required()); + return out; + } + + private static MetaRoot inheritedSpineFixture() { + Path dir = corpusDir().getParent().resolve("conformance/reporting-spine-inherited/input"); + try { + return loadJson(Files.readString(dir.resolve("meta.shop.json"), StandardCharsets.UTF_8)); + } catch (IOException e) { + throw new AssertionError(e); + } + } + + @Test + public void underSpineAColumnOfTheSpineEntityIsRequiredWhenItIsAKeyOrRequired() { + java.util.Map<String, Boolean> required = requiredOf(salesModel(), "SpineSales"); + assertEquals("Program.id: no @required, in identity.primary", true, required.get("programId")); + assertEquals("Program.title: @required", true, required.get("programTitle")); + assertEquals("Program.subtitle: optional", false, required.get("programSubtitle")); + // A time dimension over a @required spine column, at either grain path (timestamp / date). + assertEquals(true, required.get("publishedAtHour")); + assertEquals(true, required.get("publishedAtMonth")); + } + + @Test + public void underSpineADimensionOneHopBeyondTheSpineIsNotRequiredEvenOverAKeyOrARequiredField() { + java.util.Map<String, Boolean> required = requiredOf(salesModel(), "SpineSales"); + assertEquals("Catalog.id: a key, beyond the spine", false, required.get("catalogId")); + assertEquals("Catalog.name: @required, beyond the spine", false, required.get("catalogName")); + } + + @Test + public void aTwoHopSpineMakesADimensionWhoseViaEqualsTheWholeSpineRequired() { + java.util.Map<String, Boolean> required = requiredOf(salesModel(), "CatalogSales"); + assertEquals(true, required.get("catalogId")); + assertEquals(true, required.get("catalogName")); + } + + @Test + public void theSameDimensionsWithoutSpineKeepTodaysValues() { + java.util.Map<String, Boolean> required = requiredOf(salesModel(), "PlainSales"); + assertEquals(false, required.get("programId")); + assertEquals(false, required.get("programTitle")); + assertEquals(false, required.get("programSubtitle")); + assertEquals(false, required.get("catalogId")); + assertEquals(false, required.get("catalogName")); + assertEquals("no @via, @of @required: unchanged", true, required.get("minutes")); + } + + @Test + public void aMeasureIsRequiredWithDefaultAndNotWithoutACountAlwaysIs() { + MetaRoot root = salesModel(); + for (String report : List.of("SpineSales", "PlainSales")) { + java.util.Map<String, Boolean> required = requiredOf(root, report); + java.util.Map<String, Boolean> measures = new java.util.LinkedHashMap<>(); + for (String m : List.of("purchases", "revenue", "revenueRaw", "avgMinutes", "avgMinutesRaw", + "minMinutes", "minMinutesRaw", "share", "shareRaw")) { + measures.put(m, required.get(m)); + } + assertEquals(report, "{purchases=true, revenue=true, revenueRaw=false, avgMinutes=true, " + + "avgMinutesRaw=false, minMinutes=true, minMinutesRaw=false, share=true, shareRaw=false}", + measures.toString()); + } + } + + @Test + public void aCountWithADefaultPastTheLoaderIsStillRequired() { + List<MetaDataException> errors = new java.util.ArrayList<>(); + MetaRoot root = loadPastTheLoader(errors, + acme(CATALOG + PROGRAM + purchase(", \"@default\": 5", "Purchase.program") + PLAIN_SALES)); + assertEquals(1, errors.size()); + assertTrue(errors.get(0).getMessage(), errors.get(0).getMessage().contains("@default cannot apply to @agg: count")); + assertEquals(true, requiredOf(root, "PlainSales").get("purchases")); + } + + @Test + public void subtypesAndTypeSourcesAreUntouchedByDefault() { + MetaRoot root = salesModel(); + List<String> typed = ReportShape.of(object(root, "SpineSales"), root).fields().stream() + .map(f -> f.name() + " " + f.subType() + " " + (f.typeSource() == null ? null : f.typeSource().getName())) + .collect(Collectors.toList()); + assertEquals(List.of( + "programId long id", + "programTitle string title", + "programSubtitle string subtitle", + "publishedAtHour timestamp publishedAt", + "publishedAtMonth date null", + "catalogId long id", + "catalogName string name", + "purchases long null", + "revenue currency amountCents", + "revenueRaw currency amountCents", + "avgMinutes decimal null", + "avgMinutesRaw decimal null", + "minMinutes int minutes", + "minMinutesRaw int minutes", + "share decimal null", + "shareRaw decimal null"), typed); + } + + @Test + public void anIdentityPrimaryInheritedFromAnAbstractBaseMakesTheKeyColumnRequired() { + String keyed = """ + { "object.entity": { "name": "Keyed", "abstract": true, "children": [ + { "field.long": { "name": "id" } }, + { "identity.primary": { "name": "pk", "@fields": ["id"] } } + ] } }, + """; + String inheritedProgram = """ + { "object.entity": { "name": "Program", "extends": "Keyed", "children": [ + { "source.rdb": { "@table": "programs" } }, + """ + PROGRAM_MEMBERS + """ + ] } }, + """; + MetaRoot root = loadJson(acme(keyed + CATALOG + inheritedProgram + purchase("", "Purchase.program") + SPINE_SALES)); + assertEquals(true, requiredOf(root, "SpineSales").get("programId")); + } + + @Test + public void underSpineADimensionWhoseViaDoesNotResolveThrowsNamingTheReport() { + // An owner that is not @from (or an entity it extends): reportingViaHops does not resolve it. + // Past the loader, which refuses the @via under rule D2. + List<MetaDataException> errors = new java.util.ArrayList<>(); + MetaRoot root = loadPastTheLoader(errors, + acme(CATALOG + PROGRAM + purchase("", "Program.catalog") + SPINE_SALES)); + assertEquals(1, errors.size()); + assertUnresolved("report 'SpineSales': dimension 'programSubtitle' @via 'Program.catalog' does not resolve.", + object(root, "SpineSales"), root); + } + + @Test + public void aSpineWrittenWithAnAbstractBaseAsItsOwnerOverAConcreteFrom() { + MetaRoot root = inheritedSpineFixture(); + assertEquals("{programId=true, programTitle=true, totalMinutes=true}", + requiredOf(root, "ProgramMinutes").toString()); + } + + @Test + public void reportSpineHopsIsNullWithoutSpineAndTheHopNamesWithOne() { + MetaRoot root = salesModel(); + MetaObject from = object(root, "Purchase"); + assertNull(ReportShape.reportSpineHops(object(root, "PlainSales"), from, root)); + assertEquals(List.of("program"), ReportShape.reportSpineHops(object(root, "SpineSales"), from, root)); + assertEquals(List.of("program", "catalog"), ReportShape.reportSpineHops(object(root, "CatalogSales"), from, root)); + } + + @Test + public void reportSpineHopsOwnerMayBeAnEntityFromExtendsResolvedInTheReportsPackage() { + MetaRoot root = inheritedSpineFixture(); + assertEquals(List.of("program"), + ReportShape.reportSpineHops(object(root, "ProgramMinutes"), object(root, "WorkoutEvent"), root)); + } + + @Test + public void aSpineThatDoesNotResolveThrowsNamingTheReport() { + MetaRoot root = salesModel(); + MetaObject from = object(root, "Purchase"); + // Past the loader, which refuses these under rule R8: a report built in code. + for (String spine : List.of("Program.catalog", "Purchase", "Purchase..program")) { + com.metaobjects.object.ReportMetaObject report = + stray("acme::SpineSales", "Purchase", MetaObject.ATTR_REPORT_MEASURES, "purchases"); + report.addMetaAttr(com.metaobjects.attr.StringAttribute.create(MetaObject.ATTR_REPORT_SPINE, spine)); + String expected = "report 'SpineSales': @spine '" + spine + "' does not resolve."; + try { + ReportShape.reportSpineHops(report, from, root); + fail("expected: " + expected); + } catch (MetaDataException e) { + assertEquals(expected, e.getMessage()); + } + assertUnresolved(expected, report, root); + } + } } diff --git a/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportingTest.java b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportingTest.java index 41b1cac53..56dea23b5 100644 --- a/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportingTest.java +++ b/server/java/metadata/src/test/java/com/metaobjects/reporting/ReportingTest.java @@ -230,6 +230,36 @@ public void reportAccessors() throws IOException { ReportAccessors.reportDimensionItems(object(root, "ProgramEngagement"))); } + @Test + public void reportSpineIsTheDeclaredPathAndNullWhenNoneIsDeclared() throws IOException { + MetaRoot root = loadFixture("reporting-spine-and-default"); + assertEquals("Purchase.program", ReportAccessors.reportSpine(object(root, "ProgramSales"))); + assertEquals("Purchase.program.catalog", ReportAccessors.reportSpine(object(root, "CatalogSales"))); + assertNull(ReportAccessors.reportSpine(object(root, "SalesByDay"))); + } + + @Test + public void measureDefaultValueIsTheDeclaredIntegerZeroAndNegativesIncluded() throws IOException { + MetaRoot root = loadFixture("reporting-spine-and-default"); + MetaObject purchase = object(root, "Purchase"); + assertEquals(Long.valueOf(0), member(purchase, MetaMeasure.class, "revenue").getDefaultValue()); + assertEquals(Long.valueOf(0), member(purchase, MetaMeasure.class, "avgAmount").getDefaultValue()); + assertEquals(Long.valueOf(-1), member(purchase, MetaMeasure.class, "smallest").getDefaultValue()); + MetaMeasure ratio = member(purchase, MetaMeasure.class, "revenuePerPurchase"); + assertTrue(ratio.isRatio()); + assertEquals(Long.valueOf(0), ratio.getDefaultValue()); + assertNull(member(purchase, MetaMeasure.class, "purchases").getDefaultValue()); + assertNull(member(purchase, MetaMeasure.class, "lastPurchaseAt").getDefaultValue()); + } + + @Test + public void aMeasureInheritedFromAnAbstractBaseKeepsItsDefault() throws IOException { + MetaRoot root = loadFixture("reporting-spine-inherited"); + // ADR-0039: the measure is declared on BaseEvent and read through WorkoutEvent. + MetaMeasure total = member(object(root, "WorkoutEvent"), MetaMeasure.class, "totalMinutes"); + assertEquals(Long.valueOf(0), total.getDefaultValue()); + } + @Test public void derivedFieldName() { assertEquals("purchasedAtDay", ReportAccessors.reportDerivedFieldName(new ReportDimensionItem("purchasedAt", "day"))); diff --git a/server/java/omdb/src/test/java/com/metaobjects/manager/db/ReportReadTest.java b/server/java/omdb/src/test/java/com/metaobjects/manager/db/ReportReadTest.java index d08bf909a..f4e120b79 100644 --- a/server/java/omdb/src/test/java/com/metaobjects/manager/db/ReportReadTest.java +++ b/server/java/omdb/src/test/java/com/metaobjects/manager/db/ReportReadTest.java @@ -137,6 +137,18 @@ protected MetaObject findResultClass(String className) { s.execute("CREATE VIEW RPT_V_SALE_REGIONS (id, region) AS SELECT id, region FROM RPT_SALES"); s.execute("CREATE TABLE INERT_SALES (sales BIGINT)"); s.execute("INSERT INTO INERT_SALES VALUES (999)"); + // A @spine report (FR-044): one row per territory, the one no visit names + // included. The view is written by hand, as the TypeScript toolchain lowers it + // (ADR-0015: no other port emits SQL): the spine entity LEFT JOINed to the facts, + // and a defaulted measure wrapped in COALESCE. + s.execute("CREATE TABLE RPT_TERRITORIES (id BIGINT PRIMARY KEY, name VARCHAR(16) NOT NULL)"); + s.execute("INSERT INTO RPT_TERRITORIES VALUES (10, 'north'), (20, 'south'), (30, 'quiet')"); + s.execute("CREATE TABLE RPT_VISITS (id BIGINT PRIMARY KEY, territoryId BIGINT, minutes BIGINT NOT NULL)"); + s.execute("INSERT INTO RPT_VISITS VALUES (1, 10, 30), (2, 10, 15), (3, 20, 5)"); + s.execute("CREATE VIEW RPT_V_VISITS_BY_TERRITORY" + + " (territoryKey, territoryName, visits, totalMinutes, totalMinutesOrZero) AS" + + " SELECT t.id, t.name, COUNT(v.id), SUM(v.minutes), COALESCE(SUM(v.minutes), 0)" + + " FROM RPT_TERRITORIES t LEFT JOIN RPT_VISITS v ON v.territoryId = t.id GROUP BY t.id, t.name"); } } @@ -242,6 +254,29 @@ public void getObjectsReturnsTheViewsRowsKeyedByDerivedFieldName() { row("region", "west", "sales", 1L, "revenue", 50L, "minAmount", 50L)), rows); } + @Test + public void aSpineReportReadsTheEmptyRowWithItsDefaultedMeasure() { + List<Map<String, Object>> rows = read("VisitsByTerritory", sortedBy("territoryKey", SortOrder.ASC)); + assertEquals(List.of( + row("territoryKey", 10L, "territoryName", "north", "visits", 2L, "totalMinutes", 45L, + "totalMinutesOrZero", 45L), + row("territoryKey", 20L, "territoryName", "south", "visits", 1L, "totalMinutes", 5L, + "totalMinutesOrZero", 5L), + // The territory no visit names: a count is 0, a measure without @default is null, + // the defaulted one reads its default. + row("territoryKey", 30L, "territoryName", "quiet", "visits", 0L, "totalMinutes", null, + "totalMinutesOrZero", 0L)), rows); + // A filter on the defaulted measure matches the empty row by its default (Table G). + assertEquals(1L, count("VisitsByTerritory", new Expression("totalMinutesOrZero", 0L))); + // The read model says which columns can never be null (Table C). + Map<String, Boolean> required = new LinkedHashMap<>(); + for (MetaField<?> f : omdb.readObjectFor(object("VisitsByTerritory")).getMetaFields()) { + required.put(f.getName(), Boolean.TRUE.equals(f.getMetaAttr(MetaField.ATTR_REQUIRED).getValue())); + } + assertEquals("{territoryKey=true, territoryName=true, visits=true, totalMinutes=false, totalMinutesOrZero=true}", + required.toString()); + } + @Test public void rowsAreInstancesOfTheReadModelNotOfTheDeclaredNode() { MetaObject declared = object("SalesByRegion"); diff --git a/server/java/omdb/src/test/resources/meta.report.json b/server/java/omdb/src/test/resources/meta.report.json index 8af329aaf..d83a52da9 100644 --- a/server/java/omdb/src/test/resources/meta.report.json +++ b/server/java/omdb/src/test/resources/meta.report.json @@ -32,6 +32,30 @@ { "source.rdb": { "name": "main", "@kind": "view", "@view": "RPT_V_PRIMARY" } } ] } }, { "object.report": { "name": "InertSales", "@from": "Sale", "@measures": ["sales"] } }, + { "object.entity": { "name": "Territory", "children": [ + { "source.rdb": { "@table": "RPT_TERRITORIES" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name", "@required": true, "@maxLength": 16 } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "assigned" } } + ] } }, + { "object.entity": { "name": "Visit", "children": [ + { "source.rdb": { "@table": "RPT_VISITS" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "territoryId" } }, + { "field.long": { "name": "minutes", "@required": true } }, + { "identity.primary": { "name": "pk", "@fields": ["id"], "@generation": "assigned" } }, + { "identity.reference": { "name": "fkTerritory", "@references": "Territory", "@fields": ["territoryId"] } }, + { "dimension.attribute": { "name": "territoryKey", "@of": "Territory.id", "@via": "Visit.fkTerritory" } }, + { "dimension.attribute": { "name": "territoryName", "@of": "Territory.name", "@via": "Visit.fkTerritory" } }, + { "measure.aggregate": { "name": "visits", "@agg": "count", "@of": "Visit.id" } }, + { "measure.aggregate": { "name": "totalMinutes", "@agg": "sum", "@of": "Visit.minutes" } }, + { "measure.aggregate": { "name": "totalMinutesOrZero", "@agg": "sum", "@of": "Visit.minutes", "@default": 0 } } + ] } }, + { "object.report": { "name": "VisitsByTerritory", "@from": "Visit", "@spine": "Visit.fkTerritory", + "@dimensions": ["territoryKey", "territoryName"], + "@measures": ["visits", "totalMinutes", "totalMinutesOrZero"], "children": [ + { "source.rdb": { "@kind": "view", "@view": "RPT_V_VISITS_BY_TERRITORY" } } + ] } }, { "object.projection": { "name": "SaleRegionView", "children": [ { "source.rdb": { "@kind": "view", "@view": "RPT_V_SALE_REGIONS" } }, { "field.long": { "name": "id", "extends": "reporttest::Sale.id" } }, From b32aa0a8913dcd36ebaa5a80c171291a7b062603 Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 22:11:30 -0400 Subject: [PATCH 14/18] test(kotlin): @spine and measure @default through Exposed FR-044 zero rows and measure defaults, the Kotlin side. Kotlin reads the loader and ReportShape from Java, so no Kotlin generator changes: a spine key and a defaulted measure are required in the shape, and a required field is already a non-null Exposed column and data class property. - KotlinReportTableGeneratorTest pins Table C/F: ProgramRoster's table text; a defaulted sum, min and ratio are non-null in the table and the row while the same measure without @default stays nullable; under @spine the spine entity's key is non-null and without @spine the same dimension is nullable. - Three hand-written reference tables (ProgramRoster, ProgramLongWeeks, FitnessTotalsFilled) for the persistence lane, which now runs the three new report scenarios. KotlinCodegenMatchesReferenceTest also checks every report reference table against the same expectations (view, columns in order, nullability), so a reference cannot drift from the generator. - The report/ REST seam lane serves ProductRevenue (sixteen scenarios); the inert test counts four reports, ProgramCatalogue sourceless. --- .../api-contract-conformance/report/README.md | 2 +- .../kotlin/KotlinReportRestSurfaceTest.kt | 8 +- .../kotlin/KotlinReportTableGeneratorTest.kt | 144 +++++++++++++++++- .../generator/kotlin/ReportingInertTest.kt | 15 +- .../KotlinCodegenMatchesReferenceTest.kt | 75 ++++++++- .../integration/kotlin/QueryScenarioRunner.kt | 7 + ...portGeneratedApiContractConformanceTest.kt | 13 +- .../GeneratedReportControllerHarness.kt | 16 +- .../kotlin/tables/FitnessTotalsFilledView.kt | 28 ++++ .../kotlin/tables/ProgramLongWeeksView.kt | 28 ++++ .../kotlin/tables/ProgramRosterView.kt | 36 +++++ 11 files changed, 349 insertions(+), 23 deletions(-) create mode 100644 server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/FitnessTotalsFilledView.kt create mode 100644 server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramLongWeeksView.kt create mode 100644 server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramRosterView.kt diff --git a/fixtures/api-contract-conformance/report/README.md b/fixtures/api-contract-conformance/report/README.md index e79e70287..5783b9256 100644 --- a/fixtures/api-contract-conformance/report/README.md +++ b/fixtures/api-contract-conformance/report/README.md @@ -145,7 +145,7 @@ hand-rolled reference server would answer every scenario by construction. | TypeScript | wired | `server/typescript/packages/integration-tests/test/api-contract-report.test.ts` (16 scenarios + a seed-vs-view check). Full stack: generated Fastify routes over the real views on Testcontainers Postgres. `api-contract-report-sqlite.test.ts` runs the same 16 on SQLite | | C# | wired | `server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs`. Full stack: generated routes and EF Core over `schema.postgres.sql` on Testcontainers Postgres | | Java | wired | `server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java` (16 scenarios + a scenario-count check). Generated controllers behind an in-memory repository seeded from `reports` | -| Kotlin | wired | `server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt` (13 scenarios, the count check, and a check that the sourceless report generated nothing). Generated controllers over the generated Exposed table objects, seeded from `reports` | +| Kotlin | wired | `server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt` (16 scenarios, the count check, and a check that the sourceless report generated nothing). Generated controllers over the generated Exposed table objects, seeded from `reports` | | Python | wired | `server/python/tests/integration/test_api_contract_report.py` (13 scenarios, a check that exactly the served reports are generated, and `/api/invoice_days` is `404`). Generated routers behind in-memory repositories seeded from `reports` | The scenarios use only assertion keys every runner already has (`equals`, diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportRestSurfaceTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportRestSurfaceTest.kt index 9bd2d0de6..4d139c61e 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportRestSurfaceTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportRestSurfaceTest.kt @@ -110,7 +110,8 @@ class KotlinReportRestSurfaceTest { @Test fun `a sourceless report emits nothing`() { val files = emit(withModel()) - for (report in listOf("ProgramEngagement", "DailyRevenue")) { + // ProgramCatalogue declares @spine; a sourceless report is inert with it too. + for (report in listOf("ProgramEngagement", "DailyRevenue", "ProgramCatalogue")) { val hits = files.filter { (path, text) -> report in path || report in text }.keys assertTrue(hits.isEmpty(), "$report leaked into $hits") } @@ -362,10 +363,11 @@ class KotlinReportRestSurfaceTest { } @Test - fun `the six canonical reports' rows, tables and allowlists compile beside their entities`() { + fun `the nine canonical reports' rows, tables and allowlists compile beside their entities`() { val files = emit(canonical(), args = mapOf("columnNaming" to "literal")) for (report in listOf( "ProgramMinutes", "FitnessTotals", "ProgramsByMonth", "ProgramsByWeek", "RecentPrograms", "AssetActivity", + "ProgramRoster", "ProgramLongWeeks", "FitnessTotalsFilled", )) { for (suffix in listOf("", "Table", "FilterAllowlist", "Controller")) { assertTrue("fitness/$report$suffix.kt" in files, "$report$suffix missing from ${files.keys}") @@ -390,7 +392,7 @@ class KotlinReportRestSurfaceTest { ), unit.symbols.map { it.kind to it.signature }, ) - assertFalse(model.units.any { it.node == "DailyRevenue" || it.node == "ProgramEngagement" }) + assertFalse(model.units.any { it.node in setOf("DailyRevenue", "ProgramEngagement", "ProgramCatalogue") }) } private companion object { diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt index 95a79ae51..94cae0746 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/KotlinReportTableGeneratorTest.kt @@ -24,9 +24,14 @@ import kotlin.test.assertTrue * `object.report`, with one column per derived field (contract Table B), and nothing for a * report that has no view. * - * The six canonical reports are generated from the shared persistence corpus, the same + * The nine canonical reports are generated from the shared persistence corpus, the same * model the hand-written reference tables in `integration-tests-kotlin` map and the * persistence lane reads. + * + * A report's `@spine` and a measure's `@default` change only which columns are nullable + * (contract Table C). No Kotlin generator reads either attribute: the shape Java's + * `ReportShape` derives marks the field required, and a required field is already a + * non-null column and a non-null property. */ @OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) class KotlinReportTableGeneratorTest { @@ -114,6 +119,8 @@ class KotlinReportTableGeneratorTest { "ProgramMinutes" to "v_program_minutes", "FitnessTotals" to "v_fitness_totals", "ProgramsByMonth" to "v_programs_by_month", "ProgramsByWeek" to "v_programs_by_week", "RecentPrograms" to "v_recent_programs", "AssetActivity" to "v_asset_activity", + "ProgramRoster" to "v_program_roster", "ProgramLongWeeks" to "v_program_long_weeks", + "FitnessTotalsFilled" to "v_fitness_totals_filled", ) for ((report, view) in reports) { val src = files.getValue("fitness/${report}Table.kt") @@ -186,6 +193,141 @@ class KotlinReportTableGeneratorTest { assertFalse(files.keys.any { it.endsWith("ProgramMinutesNames.kt") }, files.keys.toString()) } + // --- @spine and @default: what is nullable (Table C) ----------------------------------- + + @Test + fun `a spine report's key and defaulted measures are non-null columns, the rest stay nullable`() { + // ProgramRoster: @spine Week.fkProgram. Its two dimensions are columns of the spine + // entity (Program.id, a primary key, and Program.title, @required); the pairs + // totalMinutes/totalMinutesOrZero and longShare/longShareOrZero are the same measure + // without and with @default 0. + assertEquals( + """ + |package fitness + | + |import org.jetbrains.exposed.sql.Table + | + |/** READ-ONLY VIEW — generated from view metadata; do not insert/update/delete directly. */ + |/** GENERATED — do not hand-edit. Regenerated from metadata. */ + |object ProgramRosterTable : Table("v_program_roster") { + | val programKey = long("programKey") + | val programTitle = varchar("programTitle", 200) + | val weeks = long("weeks") + | val totalMinutes = long("totalMinutes").nullable() + | val totalMinutesOrZero = long("totalMinutesOrZero") + | val longShare = decimal("longShare", 38, 18).nullable() + | val longShareOrZero = decimal("longShareOrZero", 38, 18) + |} + |""".trimMargin(), + canonical().getValue("fitness/ProgramRosterTable.kt"), + ) + // The same Program.title dimension in ProgramMinutes, which has no @spine, stays + // nullable: without @spine Table C changes nothing, and a dimension reached by @via + // is never required there. + assertTrue( + " val programTitle = varchar(\"programTitle\", 200).nullable()\n" in + canonical().getValue("fitness/ProgramMinutesTable.kt"), + ) + // With no dimensions and no @spine, a defaulted sum and ratio are still non-null. + val filled = canonical().getValue("fitness/FitnessTotalsFilledTable.kt") + assertTrue(" val totalMinutesOrZero = long(\"totalMinutesOrZero\")\n" in filled, filled) + assertTrue(" val longShareOrZero = decimal(\"longShareOrZero\", 38, 18)\n" in filled, filled) + assertFalse(".nullable()" in filled, filled) + } + + /** A `measure.aggregate` on Sale: its name to its node. */ + private fun aggregate(name: String, attrs: String) = + name to """{ "measure.aggregate": { "name": "$name", $attrs } }""" + + /** A `measure.ratio` on Sale: its name to its node. */ + private fun ratio(name: String, attrs: String) = + name to """{ "measure.ratio": { "name": "$name", $attrs } }""" + + /** + * `Product` and `Sale` (a to-one `fkProduct` reference), and `ProductRevenue` over Sale + * with [reportAttrs] (a `@spine`, or nothing). [measures] are declared on Sale beside a + * `sales` count; the report lists the product key, `sales` and every one of them. + */ + private fun productRevenue(reportAttrs: String, measures: List<Pair<String, String>>): String = """{ + "metadata.root": { "package": "acme::shop", "children": [ + { "object.entity": { "name": "Product", "children": [ + { "source.rdb": { "@table": "products" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "name", "@maxLength": 80 } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } + ] } }, + { "object.entity": { "name": "Sale", "children": [ + { "source.rdb": { "@table": "sales" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "productId", "@required": true } }, + { "field.long": { "name": "amountCents" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProduct", "@fields": ["productId"], "@references": "Product" } }, + { "dimension.attribute": { "name": "productId", "@of": "Product.id", "@via": "Sale.fkProduct" } }, + { "measure.aggregate": { "name": "sales", "@agg": "count", "@of": "Sale.id" } }, + ${measures.joinToString(",\n ") { it.second }} + ] } }, + { "object.report": { "name": "ProductRevenue", "@from": "Sale", $reportAttrs + "@dimensions": ["productId"], + "@measures": ["sales", ${measures.joinToString(", ") { "\"${it.first}\"" }}], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_product_revenue" } } ] } } + ] } + }""" + + /** The table and the row data class of `ProductRevenue`. */ + private fun productRevenueFiles(json: String): Pair<String, String> { + val files = emit( + loadString("report-table-spine-default", json), + mapOf("columnNaming" to "literal"), + listOf(KotlinEntityGenerator(), KotlinExposedTableGenerator()), + ) + assertCompiles(files) + return files.getValue("acme/shop/ProductRevenueTable.kt") to files.getValue("acme/shop/ProductRevenue.kt") + } + + @Test + fun `a defaulted measure is a non-null column and property, and the same measure without default stays nullable`() { + val (table, row) = productRevenueFiles(productRevenue( + """"@spine": "Sale.fkProduct",""", + listOf( + aggregate("revenue", """"@agg": "sum", "@of": "Sale.amountCents""""), + aggregate("revenueOrZero", """"@agg": "sum", "@of": "Sale.amountCents", "@default": 0"""), + aggregate("smallestOrZero", """"@agg": "min", "@of": "Sale.amountCents", "@default": 0"""), + ratio("perSale", """"@numerator": "revenue", "@denominator": "sales""""), + ratio("perSaleOrZero", """"@numerator": "revenue", "@denominator": "sales", "@default": 0"""), + ), + )) + // The Exposed columns. + assertTrue(" val revenueOrZero = long(\"revenueOrZero\")\n" in table, table) + assertTrue(" val smallestOrZero = long(\"smallestOrZero\")\n" in table, table) + assertTrue(" val perSaleOrZero = decimal(\"perSaleOrZero\", 38, 18)\n" in table, table) + assertTrue(" val revenue = long(\"revenue\").nullable()\n" in table, table) + assertTrue(" val perSale = decimal(\"perSale\", 38, 18).nullable()\n" in table, table) + // The row data class: a non-null property has no `?` and no `= null` default. + assertTrue(" public val revenueOrZero: Long,\n" in row, row) + assertTrue(" public val smallestOrZero: Long,\n" in row, row) + assertTrue(" public val perSaleOrZero: BigDecimal,\n" in row, row) + assertTrue(" public val revenue: Long? = null,\n" in row, row) + assertTrue(" public val perSale: BigDecimal? = null,\n" in row, row) + } + + @Test + fun `under spine the key of the spine entity is non-null, and without spine the same dimension is nullable`() { + val sum = listOf(aggregate("revenue", """"@agg": "sum", "@of": "Sale.amountCents"""")) + // Product.id is the spine entity's primary key and carries no @required. + val (spineTable, spineRow) = productRevenueFiles(productRevenue(""""@spine": "Sale.fkProduct",""", sum)) + assertTrue(" val productId = long(\"productId\")\n" in spineTable, spineTable) + assertTrue(" public val productId: Long,\n" in spineRow, spineRow) + // A count is non-null either way, and nothing else moved. + assertTrue(" val sales = long(\"sales\")\n" in spineTable, spineTable) + assertTrue(" val revenue = long(\"revenue\").nullable()\n" in spineTable, spineTable) + + val (plainTable, plainRow) = productRevenueFiles(productRevenue("", sum)) + assertTrue(" val productId = long(\"productId\").nullable()\n" in plainTable, plainTable) + assertTrue(" public val productId: Long? = null,\n" in plainRow, plainRow) + assertTrue(" val sales = long(\"sales\")\n" in plainTable, plainTable) + } + // --- Which reports emit (Table A) -------------------------------------------------- /** A `Sale` entity with [measures], and a `SaleTotals` report over them with [reportSource]. */ diff --git a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt index 815f0458b..7f72e3e90 100644 --- a/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt +++ b/server/java/codegen-kotlin/src/test/kotlin/com/metaobjects/generator/kotlin/ReportingInertTest.kt @@ -32,8 +32,10 @@ import kotlin.test.assertTrue * byte for byte, and those four emit nothing else new. * * The model pair is `fixtures/codegen-noop/reporting/{with,without}`, shared with the other - * four ports' copies of this test. `with/` carries two sourceless reports - * (`ProgramEngagement`, `DailyRevenue`) and one view-backed one (`StoreTotals`). + * four ports' copies of this test. `with/` carries three sourceless reports + * (`ProgramEngagement`, `DailyRevenue`, and `ProgramCatalogue`, which declares `@spine` and + * lists a dimension reached by `@via`) and one view-backed one (`StoreTotals`). A measure's + * `@default` (`avgDaysPerStarter`) generates nothing either. */ class ReportingInertTest { @@ -118,7 +120,7 @@ class ReportingInertTest { .filter { it.subType == MetaObject.SUBTYPE_REPORT } .map { it.shortName } .sorted() - assertEquals(listOf("DailyRevenue", "ProgramEngagement", "StoreTotals"), reports) + assertEquals(listOf("DailyRevenue", "ProgramCatalogue", "ProgramEngagement", "StoreTotals"), reports) assertFalse(load("without").metaObjects.any { it.subType == MetaObject.SUBTYPE_REPORT }) } @@ -154,7 +156,7 @@ class ReportingInertTest { fun `a sourceless report appears in no generated file`() { val files = emit("with", GENERATOR_REGISTRY.values.toList()) assertFalse(THREW in files, "the combined suite threw: ${files[THREW]}") - for (report in listOf("ProgramEngagement", "DailyRevenue")) { + for (report in SOURCELESS_REPORTS) { val hits = files.filter { (path, text) -> report in path || report in text }.keys assertTrue(hits.isEmpty(), "$report leaked into $hits") } @@ -220,7 +222,7 @@ class ReportingInertTest { .filterNot { it.isBlank() || "StoreTotals" in it || "store_totals" in it } assertEquals(expected.getValue(page).lines().filterNot { it.isBlank() }, lines, page) } - for (report in listOf("ProgramEngagement", "DailyRevenue")) { + for (report in SOURCELESS_REPORTS) { assertFalse(actual.any { (path, text) -> report in path || report in text }, "$report is documented") } } @@ -233,6 +235,9 @@ class ReportingInertTest { const val STORE_TOTALS_TABLE_PATH = "acme/shop/StoreTotalsTable.kt" + /** The with-model's reports that declare no source: inert everywhere. */ + val SOURCELESS_REPORTS = listOf("DailyRevenue", "ProgramCatalogue", "ProgramEngagement") + /** Registry id of each generator that emits for a served report, to the one file it adds. */ val STORE_TOTALS_FILES = mapOf( EXPOSED_TABLE to STORE_TOTALS_TABLE_PATH, diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt index 73bf6a2a2..3d7fef8e5 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/KotlinCodegenMatchesReferenceTest.kt @@ -1,10 +1,21 @@ package com.metaobjects.integration.kotlin import com.metaobjects.generator.kotlin.KotlinExposedTableGenerator +import com.metaobjects.integration.kotlin.tables.AssetActivityView +import com.metaobjects.integration.kotlin.tables.FitnessTotalsFilledView +import com.metaobjects.integration.kotlin.tables.FitnessTotalsView +import com.metaobjects.integration.kotlin.tables.ProgramLongWeeksView +import com.metaobjects.integration.kotlin.tables.ProgramMinutesView +import com.metaobjects.integration.kotlin.tables.ProgramRosterView +import com.metaobjects.integration.kotlin.tables.ProgramsByMonthView +import com.metaobjects.integration.kotlin.tables.ProgramsByWeekView +import com.metaobjects.integration.kotlin.tables.RecentProgramsView import com.metaobjects.loader.MetaDataLoader +import org.jetbrains.exposed.sql.Table import org.junit.jupiter.api.Test import java.io.File import java.nio.file.Files +import kotlin.test.assertEquals import kotlin.test.assertTrue import kotlin.test.fail @@ -104,7 +115,7 @@ internal class KotlinCodegenMatchesReferenceTest { ExpectedColumn("recordedAt", families = setOf("instantWithTimeZone")), ), ), - // FR-044: the six view-backed reports. Each mirrors its hand-written reference + // FR-044: the nine view-backed reports. Each mirrors its hand-written reference // (`tables/<Report>View.kt`) column for column: the family is the view's real column // type, and a column is nullable exactly when the reference's is. A report has no // identity, so none carries a primary key. @@ -160,6 +171,37 @@ internal class KotlinCodegenMatchesReferenceTest { ), report = "v_asset_activity", ), + // `@spine` and a measure's `@default` (contract Table C): under @spine a column of the + // spine entity that is its key or `@required` is non-null, and a measure with + // `@default` is non-null; the same measure without it stays nullable. + "ProgramRoster" to EntityExpectation( + columns = listOf( + ExpectedColumn("programKey", families = setOf("long"), nullable = false), + ExpectedColumn("programTitle", families = setOf("varchar"), nullable = false), + ExpectedColumn("weeks", families = setOf("long"), nullable = false), + ExpectedColumn("totalMinutes", families = setOf("long"), nullable = true), + ExpectedColumn("totalMinutesOrZero", families = setOf("long"), nullable = false), + ExpectedColumn("longShare", families = setOf("decimal"), nullable = true), + ExpectedColumn("longShareOrZero", families = setOf("decimal"), nullable = false), + ), + report = "v_program_roster", + ), + "ProgramLongWeeks" to EntityExpectation( + columns = listOf( + ExpectedColumn("programKey", families = setOf("long"), nullable = false), + ExpectedColumn("weeks", families = setOf("long"), nullable = false), + ExpectedColumn("totalMinutesOrZero", families = setOf("long"), nullable = false), + ), + report = "v_program_long_weeks", + ), + "FitnessTotalsFilled" to EntityExpectation( + columns = listOf( + ExpectedColumn("weeks", families = setOf("long"), nullable = false), + ExpectedColumn("totalMinutesOrZero", families = setOf("long"), nullable = false), + ExpectedColumn("longShareOrZero", families = setOf("decimal"), nullable = false), + ), + report = "v_fitness_totals_filled", + ), ) @Test @@ -187,6 +229,37 @@ internal class KotlinCodegenMatchesReferenceTest { } } + /** + * The hand-written reference tables the persistence lane reads the views through hold the + * same expectations: the view, the columns in order, and which are nullable. Without this + * a reference table could drift from the generator and the lane would only notice when a + * value happened to be null. + */ + @Test + fun `the hand-written report reference tables match the same expectations`() { + val references: Map<String, Table> = mapOf( + "ProgramMinutes" to ProgramMinutesView, + "FitnessTotals" to FitnessTotalsView, + "ProgramsByMonth" to ProgramsByMonthView, + "ProgramsByWeek" to ProgramsByWeekView, + "RecentPrograms" to RecentProgramsView, + "AssetActivity" to AssetActivityView, + "ProgramRoster" to ProgramRosterView, + "ProgramLongWeeks" to ProgramLongWeeksView, + "FitnessTotalsFilled" to FitnessTotalsFilledView, + ) + assertEquals(expectations.filterValues { it.report != null }.keys, references.keys) + for ((report, table) in references) { + val expected = expectations.getValue(report) + assertEquals(expected.report, table.tableName, report) + assertEquals( + expected.columns.map { it.name to it.nullable }, + table.columns.map { it.name to it.columnType.nullable }, + "$report: the reference table's columns (name to nullable)", + ) + } + } + private fun findEmittedTableFile(root: File, filename: String): File? = root.walkTopDown().firstOrNull { it.isFile && it.name == filename } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt index 149c5f8c7..3731906a4 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt @@ -17,8 +17,11 @@ import com.metaobjects.integration.kotlin.tables.ProgramTable import com.metaobjects.integration.kotlin.tables.ProgramView import com.metaobjects.integration.kotlin.tables.WeekTable import com.metaobjects.integration.kotlin.tables.AssetActivityView +import com.metaobjects.integration.kotlin.tables.FitnessTotalsFilledView import com.metaobjects.integration.kotlin.tables.FitnessTotalsView +import com.metaobjects.integration.kotlin.tables.ProgramLongWeeksView import com.metaobjects.integration.kotlin.tables.ProgramMinutesView +import com.metaobjects.integration.kotlin.tables.ProgramRosterView import com.metaobjects.integration.kotlin.tables.ProgramsByMonthView import com.metaobjects.integration.kotlin.tables.ProgramsByWeekView import com.metaobjects.integration.kotlin.tables.RecentProgramsView @@ -498,6 +501,10 @@ object QueryScenarioRunner { "ProgramsByWeek" -> ProgramsByWeekView "RecentPrograms" -> RecentProgramsView "AssetActivity" -> AssetActivityView + // A @spine report (rows from the spine entity) and a report of defaulted measures. + "ProgramRoster" -> ProgramRosterView + "ProgramLongWeeks" -> ProgramLongWeeksView + "FitnessTotalsFilled" -> FitnessTotalsFilledView "Asset" -> AssetTable "AllTypes" -> AllTypesTable // FR-017 TPH: the discriminator base + all its subtypes share the single `auths` table. diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt index 94a65e7f9..d13047948 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt @@ -21,7 +21,7 @@ import java.util.stream.Stream /** * FR-044 Plan 3: the Kotlin GENERATED-controller lane for the view-backed-report - * api-contract sub-corpus. Drives the three generated read-only report controllers over an + * api-contract sub-corpus. Drives the four generated read-only report controllers over an * embedded Tomcat: GET list with the FR-009 filter and sort allowlists over the report's * DERIVED fields, `POST` answering `405 {"error": "method_not_allowed"}`, and no `/{id}`. * @@ -46,9 +46,9 @@ internal class ReportGeneratedApiContractConformanceTest { } @Test - fun `the corpus has its thirteen scenarios`() { + fun `the corpus has its sixteen scenarios`() { // A lane that silently ran fewer would still be green. - assertEquals(13, scenarios().count()) + assertEquals(16, scenarios().count()) } @Test @@ -68,9 +68,10 @@ internal class ReportGeneratedApiContractConformanceTest { private val MAPPER = ObjectMapper() /** - * The `reports` half of the seed: what the three views return for the seeded - * invoices. This lane's H2 tables stand in for the views (no view SQL is written in - * Kotlin), so they are seeded with the views' rows, not with the base table's. + * The `reports` half of the seed: what the four views return for the seeded base + * tables. This lane's H2 tables stand in for the views (no view SQL is written in + * Kotlin), so they are seeded with the views' rows; every other top-level key + * (`invoices`, `products`, `sales`) is a base table this lane never creates. */ @Suppress("UNCHECKED_CAST") private val SEED: Map<String, List<Map<String, Any?>>> by lazy { diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/generated/GeneratedReportControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/generated/GeneratedReportControllerHarness.kt index e97b70939..d1e4b31f4 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/generated/GeneratedReportControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/generated/GeneratedReportControllerHarness.kt @@ -26,7 +26,7 @@ import kotlin.io.path.isRegularFile import kotlin.io.path.readText /** - * FR-044 Plan 3: host the GENERATED Kotlin Spring controllers of the three served reports + * FR-044 Plan 3: host the GENERATED Kotlin Spring controllers of the four served reports * of the `report/` corpus over real HTTP (an embedded Tomcat). Mirrors * [com.metaobjects.integration.kotlin.api.projection.generated.GeneratedProjectionControllerHarness]. * @@ -38,12 +38,13 @@ import kotlin.io.path.readText * report controller COMPILES against its row, table and allowlist: the codegen-compile * gate excludes the framework-bound route tier in every port by design. * 4. Per scenario: a fresh in-memory H2 (PostgreSQL mode), `SchemaUtils.create(...)` on the - * three generated report table objects, then the `reports` half of `seed.json`. + * four generated report table objects, then the `reports` half of `seed.json`. * 5. Serve the three controllers from one embedded Tomcat ([TomcatHost]). * * No view SQL is written here (ADR-0015: TypeScript owns it). H2 creates each generated * `<R>Table` as a plain table that STANDS IN for the view, and the seed's `reports` half is - * what the three views return for the seeded invoices. What this lane proves is the + * what the four views return for the seeded base tables. The base-table keys (`invoices`, + * `products`, `sales`) are the full-stack lanes' input; this seam lane does not read them. What this lane proves is the * generated route over the generated binding, not the lowering. */ @OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) @@ -114,7 +115,7 @@ class GeneratedReportControllerHarness( } } - /** Rebuild a fresh in-memory H2, the three stand-in tables, the seed, and Tomcat. */ + /** Rebuild a fresh in-memory H2, the four stand-in tables, the seed, and Tomcat. */ fun reset() { val dbName = "report_invoice_${dbSeq.incrementAndGet()}" val db = Database.connect("jdbc:h2:mem:$dbName;DB_CLOSE_DELAY=-1;MODE=PostgreSQL", driver = "org.h2.Driver") @@ -161,8 +162,11 @@ class GeneratedReportControllerHarness( companion object { const val ENTITY_PKG = "acme.sales" - /** The three reports of the corpus that declare a view. `InvoiceDays` declares none. */ - val SERVED = listOf("InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals") + /** + * The four reports of the corpus that declare a view. `InvoiceDays` declares none. + * `ProductRevenue` declares `@spine` and a measure with `@default`. + */ + val SERVED = listOf("InvoiceStatusTotals", "InvoicesByMonth", "InvoiceTotals", "ProductRevenue") private fun snakeCase(s: String): String = buildString { for (c in s) { diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/FitnessTotalsFilledView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/FitnessTotalsFilledView.kt new file mode 100644 index 000000000..c1d24c833 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/FitnessTotalsFilledView.kt @@ -0,0 +1,28 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table + +/** + * Hand-written reference Exposed Table mapping the `FitnessTotalsFilled` report (FR-044) + * from `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_fitness_totals_filled`, created by the committed canonical + * DDL (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is + * purely the read-only query mapping, as [ProgramStatView] is for a projection. + * + * [FitnessTotalsView] with `@default: 0` on its sum and its ratio. No dimensions, so the view + * returns exactly one row, over an empty table too, and then every column is 0, never NULL. + * No column is nullable (`fixtures/persistence-conformance/report-shapes.json` says + * `required: true` for all three): + * - `weeks` = a count → BIGINT → `long` + * - `totalMinutesOrZero` = a sum with `@default: 0` → BIGINT → `long` + * - `longShareOrZero` = a ratio with `@default: 0` → NUMERIC → `decimal` + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object FitnessTotalsFilledView : Table("v_fitness_totals_filled") { + val weeks = long("weeks") + val totalMinutesOrZero = long("totalMinutesOrZero") + val longShareOrZero = decimal("longShareOrZero", 38, 18) +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramLongWeeksView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramLongWeeksView.kt new file mode 100644 index 000000000..dcfee4106 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramLongWeeksView.kt @@ -0,0 +1,28 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table + +/** + * Hand-written reference Exposed Table mapping the `ProgramLongWeeks` report (FR-044) from + * `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_program_long_weeks`, created by the committed canonical DDL + * (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is purely + * the read-only query mapping, as [ProgramStatView] is for a projection. + * + * The report declares `@spine: "Week.fkProgram"` and `@segment: long`. The segment sits in + * the LEFT OUTER JOIN's ON clause, so a program with no long week keeps its row. No column + * is nullable (`fixtures/persistence-conformance/report-shapes.json` says `required: true` + * for all three): + * - `programKey` = Program.id, the spine entity's key → BIGINT → `long` + * - `weeks` = a count, 0 on the empty row → BIGINT → `long` + * - `totalMinutesOrZero` = a sum with `@default: 0` → BIGINT → `long` + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object ProgramLongWeeksView : Table("v_program_long_weeks") { + val programKey = long("programKey") + val weeks = long("weeks") + val totalMinutesOrZero = long("totalMinutesOrZero") +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramRosterView.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramRosterView.kt new file mode 100644 index 000000000..47f75669a --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/tables/ProgramRosterView.kt @@ -0,0 +1,36 @@ +package com.metaobjects.integration.kotlin.tables + +import org.jetbrains.exposed.sql.Table + +/** + * Hand-written reference Exposed Table mapping the `ProgramRoster` report (FR-044) from + * `fixtures/persistence-conformance/canonical/meta.fitness.json`. + * + * Backed by the Postgres VIEW `v_program_roster`, created by the committed canonical DDL + * (`fixtures/persistence-conformance/canonical/schema.postgres.sql`); this object is purely + * the read-only query mapping, as [ProgramStatView] is for a projection. + * + * The report declares `@spine: "Week.fkProgram"`, so the view starts from `programs` and + * LEFT OUTER JOINs `weeks`: a program with no week still has a row. Nullable exactly where + * `fixtures/persistence-conformance/report-shapes.json` says `required: false`: + * - `programKey` = Program.id, the spine entity's key → BIGINT → `long` + * - `programTitle` = Program.title, a `@required` column of + * the spine entity → VARCHAR → `varchar` + * - `weeks` = a count, 0 on the empty row → BIGINT → `long` + * - `totalMinutes` = a sum, NULL on the empty row → BIGINT → `long`, nullable + * - `totalMinutesOrZero` = the same sum with `@default: 0` → BIGINT → `long` + * - `longShare` = a ratio, NULL on the empty row → NUMERIC → `decimal`, nullable + * - `longShareOrZero` = the same ratio with `@default: 0` → NUMERIC → `decimal` + * + * A report has no identity, so there is no `primaryKey`: it is listed and counted, never + * fetched by id. Column names are the derived field names (the corpus's `literal` naming). + */ +object ProgramRosterView : Table("v_program_roster") { + val programKey = long("programKey") + val programTitle = varchar("programTitle", 200) + val weeks = long("weeks") + val totalMinutes = long("totalMinutes").nullable() + val totalMinutesOrZero = long("totalMinutesOrZero") + val longShare = decimal("longShare", 38, 18).nullable() + val longShareOrZero = decimal("longShareOrZero", 38, 18) +} From c80e82f9fe274e0f2e1f9e701365e725d0952b93 Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 22:32:47 -0400 Subject: [PATCH 15/18] feat(python): @spine and measure @default FR-044 zero rows and measure defaults, the Python port of the TypeScript reference (Tables A, B and C). - Register @spine (string) on object.report and @default (int) on measure.aggregate and measure.ratio; byte-copy the two spec files. MetaMeasure.default_value() (refuses bool) and report_spine(). - Loader rules R8 (the spine is D2's to-one walk, started at @from), R9 (every listed dimension is reached through the spine, hop names compared as written), M7 (@default on a count) and M8 (@default on min/max of a non-numeric field), with the TypeScript message texts. _walk_to_one_via takes a _ToOneWalk (_d2_walk / _spine_walk); every D2 message is unchanged. A fractional @default is already refused by this port's generic attr.int type check (ERR_BAD_ATTR_VALUE on the measure), so no second error is added; it only skips M7/M8. - report_shape Table C: under @spine a dimension on the spine is required when its @of is @required or a primary-key column; a defaulted measure is required. New reporting_via_hops / report_spine_hops. - No generator or runtime change: the Pydantic row model types the spine key and a defaulted measure without "| None", as it does a count, and the ObjectManager reads a @spine view's empty row. - The report REST seam lane serves ProductRevenue; corpus counts: nine canonical reports, four inert reports, sixteen REST scenarios. A new unit file ports the TypeScript exact-text assertions for R8/R9/M7/M8. --- .../api-contract-conformance/report/README.md | 2 +- server/python/src/metaobjects/core_types.py | 7 + .../metaobjects/loader/validate_reporting.py | 252 +++++- .../meta/core/object/object_constants.py | 2 + .../meta/core/reporting/meta_measure.py | 9 + .../meta/core/reporting/report_accessors.py | 8 + .../meta/core/reporting/report_shape.py | 138 ++- .../core/reporting/reporting_constants.py | 2 + .../metaobjects/spec_metamodel/object.json | 3 +- .../metaobjects/spec_metamodel/reporting.json | 6 +- .../tests/codegen/test_report_router.py | 43 +- .../tests/integration/generated_report_app.py | 9 +- .../integration/test_api_contract_report.py | 10 +- .../runtime/test_object_manager_report.py | 39 +- server/python/tests/test_report_read_model.py | 37 + server/python/tests/test_report_shape.py | 306 ++++++- server/python/tests/test_reporting_inert.py | 26 +- .../tests/unit/test_reporting_accessors.py | 55 ++ ...test_reporting_validation_spine_default.py | 827 ++++++++++++++++++ 19 files changed, 1700 insertions(+), 81 deletions(-) create mode 100644 server/python/tests/unit/test_reporting_validation_spine_default.py diff --git a/fixtures/api-contract-conformance/report/README.md b/fixtures/api-contract-conformance/report/README.md index 5783b9256..af3a68098 100644 --- a/fixtures/api-contract-conformance/report/README.md +++ b/fixtures/api-contract-conformance/report/README.md @@ -146,7 +146,7 @@ hand-rolled reference server would answer every scenario by construction. | C# | wired | `server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs`. Full stack: generated routes and EF Core over `schema.postgres.sql` on Testcontainers Postgres | | Java | wired | `server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java` (16 scenarios + a scenario-count check). Generated controllers behind an in-memory repository seeded from `reports` | | Kotlin | wired | `server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt` (16 scenarios, the count check, and a check that the sourceless report generated nothing). Generated controllers over the generated Exposed table objects, seeded from `reports` | -| Python | wired | `server/python/tests/integration/test_api_contract_report.py` (13 scenarios, a check that exactly the served reports are generated, and `/api/invoice_days` is `404`). Generated routers behind in-memory repositories seeded from `reports` | +| Python | wired | `server/python/tests/integration/test_api_contract_report.py` (16 scenarios, a check that exactly the served reports are generated, and `/api/invoice_days` is `404`). Generated routers behind in-memory repositories seeded from `reports` | The scenarios use only assertion keys every runner already has (`equals`, `length`, `envelope`, `error` with `field`, and a status with no `body`). diff --git a/server/python/src/metaobjects/core_types.py b/server/python/src/metaobjects/core_types.py index 483c6a537..8a0d1fbb3 100644 --- a/server/python/src/metaobjects/core_types.py +++ b/server/python/src/metaobjects/core_types.py @@ -65,6 +65,7 @@ OBJECT_REPORT_ATTR_FROM, OBJECT_REPORT_ATTR_MEASURES, OBJECT_REPORT_ATTR_SEGMENT, + OBJECT_REPORT_ATTR_SPINE, OBJECT_SUBTYPE_ENTITY, OBJECT_SUBTYPE_PROJECTION, OBJECT_SUBTYPE_REPORT, @@ -81,6 +82,7 @@ MEASURE_SUBTYPE_AGGREGATE, MEASURE_SUBTYPE_RATIO, REPORTING_ATTR_AGG, + REPORTING_ATTR_DEFAULT, REPORTING_ATTR_DENOMINATOR, REPORTING_ATTR_DISTINCT, REPORTING_ATTR_FILTER, @@ -360,6 +362,8 @@ def _register_subtypes( ), AttrSchema(name=OBJECT_REPORT_ATTR_SEGMENT, value_type=ATTR_SUBTYPE_STRING, required=False), AttrSchema(name=OBJECT_REPORT_ATTR_FILTER, value_type=ATTR_SUBTYPE_FILTER, required=False), + # R8: the to-one path from @from whose entity supplies the report's rows. + AttrSchema(name=OBJECT_REPORT_ATTR_SPINE, value_type=ATTR_SUBTYPE_STRING, required=False), ] ) break @@ -427,6 +431,8 @@ def _register_subtypes( AttrSchema(name=REPORTING_ATTR_DISTINCT, value_type=ATTR_SUBTYPE_BOOLEAN, required=False), AttrSchema(name=REPORTING_ATTR_FILTER, value_type=ATTR_SUBTYPE_FILTER, required=False), AttrSchema(name=REPORTING_ATTR_SEGMENT, value_type=ATTR_SUBTYPE_STRING, required=False), + # What the measure reads when it would otherwise be null; M7/M8 decide where it is legal. + AttrSchema(name=REPORTING_ATTR_DEFAULT, value_type=ATTR_SUBTYPE_INT, required=False), ], child_rules=[ChildRule(TYPE_ATTR, "*")], ) @@ -439,6 +445,7 @@ def _register_subtypes( attrs=[ AttrSchema(name=REPORTING_ATTR_NUMERATOR, value_type=ATTR_SUBTYPE_STRING, required=True), AttrSchema(name=REPORTING_ATTR_DENOMINATOR, value_type=ATTR_SUBTYPE_STRING, required=True), + AttrSchema(name=REPORTING_ATTR_DEFAULT, value_type=ATTR_SUBTYPE_INT, required=False), ], child_rules=[ChildRule(TYPE_ATTR, "*")], ) diff --git a/server/python/src/metaobjects/loader/validate_reporting.py b/server/python/src/metaobjects/loader/validate_reporting.py index a303aeb70..ae1582b97 100644 --- a/server/python/src/metaobjects/loader/validate_reporting.py +++ b/server/python/src/metaobjects/loader/validate_reporting.py @@ -2,16 +2,19 @@ A rule-for-rule port of the TypeScript reference ``server/typescript/packages/metadata/src/loader/reporting-validation.ts``. The rule -ids (D1…F2) match the shared rule table and the error fixtures in -``fixtures/conformance/error-*``. Every port implements the same table WITH THE SAME -MESSAGE TEXT; the fixtures are the contract. +ids (D1…F2) match the shared rule table (R8, R9, M7 and M8: Table B of +docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md) and the +error fixtures in ``fixtures/conformance/error-*``. Every port implements the same +table WITH THE SAME MESSAGE TEXT; the fixtures are the contract. Two design rules hold throughout, so one broken rule yields exactly one error: - No cascades. A member that fails a structural rule is not checked further (a - dimension whose @via fails D2 skips D1/D3/D4; a report whose @from fails R1 skips - R2/R3/R6/R7 and its @filter; an invalid @dimensions/@measures item derives no - report field for R6). + dimension whose @via fails D2 skips D1/D3/D4; a measure that fails one of M1–M4, or + whose @default is a fraction, skips M7/M8; a report whose @from fails R1 skips + R2/R3/R6/R7, R8/R9 and its @filter; a report whose @spine fails R8 skips R9, and R9 + skips a dimension whose @via fails D2; an invalid @dimensions/@measures item derives + no report field for R6). - Each error's ``envelope`` is the offending node's source (the dimension / measure / segment / report, or for R4/R5 the declared child), so a conformance fixture's jsonPath points at it. @@ -23,6 +26,12 @@ repeated, so a broken base member is reported ONCE, and a failure that only an inheritor exposes carries `` (inherited by '<entity>')``. +A fractional measure @default: the TypeScript reference refuses it in its measure check, +because its generic attr.int check lets a fraction through. This port's generic attr-type +check (``validation_passes._type_ok``) already refuses it with ERR_BAD_ATTR_VALUE on the +measure node, so no second error is added here; ``_check_measure`` only skips M7/M8 for +it, as the reference does. + ADR-0039 NAMING INVERSION: Python ``attr()`` is OWN-ONLY. Every place the TS reference calls the resolving ``attr()`` / ``children()``, this module calls ``get_meta_attr()`` / ``children()``. The two ``own*`` uses (R4 and the F1 tree walk) carry their sanctioned @@ -56,6 +65,7 @@ from ..meta.core.object.object_constants import ( OBJECT_REPORT_ATTR_FILTER, OBJECT_REPORT_ATTR_SEGMENT, + OBJECT_REPORT_ATTR_SPINE, OBJECT_SUBTYPE_ENTITY, OBJECT_SUBTYPE_REPORT, ) @@ -72,6 +82,7 @@ report_dimension_items, report_from, report_measure_names, + report_spine, ) from ..meta.core.reporting.reporting_constants import ( AGG_AVG, @@ -84,8 +95,10 @@ ISO_DURATION_RE, MEASURE_SUBTYPE_AGGREGATE, REPORT_DIMENSION_GRAIN_SEPARATOR, + REPORTING_ATTR_DEFAULT, REPORTING_ATTR_DENOMINATOR, REPORTING_ATTR_NUMERATOR, + REPORTING_ATTR_VIA, ) from ..meta.meta_data import MetaData from ..meta.persistence.source.meta_source import MetaSource @@ -419,7 +432,7 @@ def err(message: str) -> None: of_entity = ctx.host via = dim.via() if via is not None: - terminal = _walk_to_one_via(ctx, via, err) + terminal = _walk_to_one_via(ctx.root, ctx.host, via, err, _d2_walk(ctx.declaring)) if terminal is None: return of_entity = terminal @@ -462,19 +475,83 @@ def err(message: str) -> None: err(f"grain 'hour' is impossible on '{of}', a field.date (a date has no hour). Remove 'hour' from @grains.") -def _walk_to_one_via(ctx: _MemberCtx, via: str, err: Callable[[str], None]) -> MetaData | None: - """D2 — walk ``Owner.hop[.hop...]``: Owner is the owning entity, and every hop is a - to-one ``relationship.*`` or an ``identity.reference``. Returns the terminal - entity, or ``None`` after reporting the first failure.""" +@dataclass(frozen=True) +class _ToOneWalk: + """What a to-one walk resolves against and how its messages name it. D2 walks a + dimension's @via (:func:`_d2_walk`); R8 walks a report's @spine from @from + (:func:`_spine_walk`). Every difference between the two is a field here, so each port + copies one explicit rule (TS ``ToOneWalk``).""" + + #: The attribute holding the path: ``via`` (D2) or ``spine`` (R8). + attr: str + #: The package Owner resolves in (ADR-0042): the declaring entity's (D2) or the report's (R8). + owner_pkg: str + #: The FQN named for the walk's first entity: the declaring entity (D2) or @from (R8). + host_name: str + #: What Owner must be: ``the owning entity '<FQN>'`` (D2) or ``@from '<FQN>'`` (R8). + start: str + #: The same, in the malformed-path message: ``the owning entity`` (D2) or ``@from '<FQN>'`` (R8). + start_short: str + #: The sentence that ends the to-many message: why only to-one hops are followed. + to_one_reason: str + + +def _d2_walk(declaring: MetaData) -> _ToOneWalk: + """D2 — a dimension's @via, declared on ``declaring``. Reproduces D2's messages exactly.""" + declaring_key = declaring.resolution_key() + return _ToOneWalk( + attr=REPORTING_ATTR_VIA, + owner_pkg=_pkg_of(declaring), + host_name=declaring_key, + start=f"the owning entity '{declaring_key}'", + start_short="the owning entity", + to_one_reason=( + "A dimension follows only @cardinality: one relationships and identity.reference hops, so grouping " + "can never multiply the measured rows." + ), + ) + + +def _spine_walk(report: MetaData, from_key: str) -> _ToOneWalk: + """R8 — a report's @spine, started at @from (``from_key``); Owner resolves in the report's package.""" + return _ToOneWalk( + attr=OBJECT_REPORT_ATTR_SPINE, + owner_pkg=_pkg_of(report), + host_name=from_key, + start=f"@from '{from_key}'", + start_short=f"@from '{from_key}'", + to_one_reason=( + "A @spine follows only @cardinality: one relationships and identity.reference hops, so each fact row " + "joins at most one row of the spine entity and is never counted twice." + ), + ) + + +def _walk_to_one_via( + root: MetaData, + host: MetaData, + via: str, + err: Callable[[str], None], + walk: _ToOneWalk, +) -> MetaData | None: + """Walk ``Owner.hop[.hop...]`` from ``host``: Owner is ``host`` or an entity it + extends, and every hop is a to-one ``relationship.*`` or an ``identity.reference``. + Returns the terminal entity, or ``None`` after reporting the first failure. D2 and R8 + both run it; ``walk`` says which.""" + named = f"@{walk.attr} '{via}'" + + def name_of(entity: MetaData) -> str: + return walk.host_name if entity is host else entity.resolution_key() + parts = _split_dotted(via) if parts is None: - err(f"@via '{via}' must be Owner.hop[.hop...], starting at the owning entity.") + err(f"{named} must be Owner.hop[.hop...], starting at {walk.start_short}.") return None - owner = resolve_object_ref(ctx.root, parts.owner, _pkg_of(ctx.declaring)) - if not _is_self_or_ancestor(owner, ctx.host): - err(f"@via '{via}' must start at the owning entity '{ctx.declaring.resolution_key()}'.") + owner = resolve_object_ref(root, parts.owner, walk.owner_pkg) + if not _is_self_or_ancestor(owner, host): + err(f"{named} must start at {walk.start}.") return None - current = ctx.host + current = host for hop_name in parts.path: hop = _child_of_type(current, TYPE_RELATIONSHIP, hop_name) if hop is None: @@ -489,17 +566,16 @@ def _walk_to_one_via(ctx: _MemberCtx, via: str, err: Callable[[str], None]) -> M ) if hop is None: err( - f"@via '{via}' names '{hop_name}', which is not a relationship or identity.reference of " - f"'{_shown(ctx, current)}'." + f"{named} names '{hop_name}', which is not a relationship or identity.reference of " + f"'{name_of(current)}'." ) return None is_reference = hop.type == TYPE_IDENTITY # ADR-0039: resolving (get_meta_attr) — @cardinality may be inherited via extends. if not is_reference and hop.get_meta_attr(RELATIONSHIP_ATTR_CARDINALITY) != CARDINALITY_ONE: err( - f"@via '{via}' crosses relationship '{hop_name}' on '{_shown(ctx, current)}', which is not to-one. " - f"A dimension follows only @cardinality: one relationships and identity.reference hops, so grouping " - f"can never multiply the measured rows." + f"{named} crosses relationship '{hop_name}' on '{name_of(current)}', which is not to-one. " + f"{walk.to_one_reason}" ) return None # ADR-0039: resolving. @@ -507,11 +583,9 @@ def _walk_to_one_via(ctx: _MemberCtx, via: str, err: Callable[[str], None]) -> M IDENTITY_REFERENCE_ATTR_REFERENCES if is_reference else RELATIONSHIP_ATTR_OBJECT_REF ) # ADR-0042 — a hop target resolves in the package of the entity declaring the hop. - target = ( - resolve_object_ref(ctx.root, target_ref, _pkg_of(current)) if isinstance(target_ref, str) else None - ) + target = resolve_object_ref(root, target_ref, _pkg_of(current)) if isinstance(target_ref, str) else None if target is None: - err(f"@via '{via}' hop '{hop_name}' on '{_shown(ctx, current)}' targets no object.") + err(f"{named} hop '{hop_name}' on '{name_of(current)}' targets no object.") return None current = target return current @@ -521,13 +595,39 @@ def _check_measure(ctx: _MemberCtx, measure: MetaMeasure) -> None: def err(message: str) -> None: ctx.sink.push(measure, _ERR_INVALID_MEASURE, ctx.label, f": {message}", ctx.suffix) + # The type rule — a measure's @default is a whole number. The attr-schema pass already + # refuses a fraction (and a non-number) with ERR_BAD_ATTR_VALUE on this node (see the + # module docstring), so nothing is reported here. A fraction only skips M7/M8: one + # mistake, one error. ``bool`` is not a number here, as in the reference (it is the + # type error, and M7/M8 still read the attribute as present). + # ADR-0039: resolving (get_meta_attr) — the raw attribute, inherited or not. + declared_default = measure.get_meta_attr(REPORTING_ATTR_DEFAULT) + fractional_default = isinstance(declared_default, float) and not declared_default.is_integer() + if measure.is_ratio(): _check_ratio_operands(ctx, measure, err) return if measure.sub_type != MEASURE_SUBTYPE_AGGREGATE: return - _check_aggregate_columns(ctx, measure, err) + checked = _check_aggregate_columns(ctx, measure, err) + + # M7 / M8 — where a @default can apply. Only when M1–M4 passed and the @default is + # not a fraction: one mistake, one error. The presence test reads the raw attribute, + # so a non-number value on a count still reports M7 beside the attribute type error. + if checked is not None and not fractional_default and declared_default is not None: + agg = measure.agg() + of_field = checked.field + if agg == AGG_COUNT: + err( + "@default cannot apply to @agg: count. A count is never null (it is 0 when nothing matches); " + "remove @default." + ) + elif agg in (AGG_MIN, AGG_MAX) and of_field is not None and of_field.sub_type not in _NUMERIC_FIELD_SUBTYPES: + err( + f"@default is a number, but '{measure.of_columns()[0]}', the @of of @agg '{agg}', is a " + f"field.{of_field.sub_type}. A default is supported on numeric measures only." + ) # M5 — @segment names a segment of the owning entity. segment = measure.segment_name() @@ -542,8 +642,18 @@ def err(message: str) -> None: ) -def _check_aggregate_columns(ctx: _MemberCtx, measure: MetaMeasure, err: Callable[[str], None]) -> None: - """M1–M4, in order; the first failure stops the chain (no M2+M3 double report).""" +@dataclass(frozen=True) +class _CheckedColumns: + """M1–M4 passed. ``field`` is the single resolved @of field (``None`` for a tuple), for M7/M8.""" + + field: MetaData | None + + +def _check_aggregate_columns( + ctx: _MemberCtx, measure: MetaMeasure, err: Callable[[str], None] +) -> _CheckedColumns | None: + """M1–M4, in order; the first failure stops the chain (no M2+M3 double report). + Returns ``None`` when one of them fired.""" agg = measure.agg() columns = measure.of_columns() @@ -553,18 +663,18 @@ def _check_aggregate_columns(ctx: _MemberCtx, measure: MetaMeasure, err: Callabl parts = _split_dotted(item) if parts is None or len(parts.path) != 1: err(f"@of '{item}' must be Entity.field.") - return + return None named = resolve_object_ref(ctx.root, parts.owner, _pkg_of(ctx.declaring)) if not _is_self_or_ancestor(named, ctx.host): err( f"@of '{item}' must name a field of the owning entity '{ctx.declaring.resolution_key()}'. " f"A measure aggregates its own entity's rows; declare it on the entity that owns the column." ) - return + return None field = _field_of(ctx.host, parts.path[0]) if field is None: err(f"@of '{item}' names no field '{parts.path[0]}' on '{ctx.declaring.resolution_key()}'.") - return + return None fields.append(field) # M2 — a tuple is a distinct count only. @@ -573,26 +683,28 @@ def _check_aggregate_columns(ctx: _MemberCtx, measure: MetaMeasure, err: Callabl f"@of lists {len(columns)} columns; a tuple is legal only with @agg: count and @distinct: true " f"(a distinct count of the tuple)." ) - return + return None # M3 — @distinct is a count modifier. if measure.distinct() and agg is not None and agg != AGG_COUNT: err(f"@distinct: true requires @agg: count, not '{agg}'.") - return + return None # M4 — the aggregate must be meaningful for the column's type. field = fields[0] if len(fields) == 1 else None if field is None or agg is None: - return + return _CheckedColumns(field) item = columns[0] if agg in (AGG_SUM, AGG_AVG) and field.sub_type not in _NUMERIC_FIELD_SUBTYPES: err( f"@agg '{agg}' needs a numeric field (field.int, long, double, float, decimal or currency), " f"but '{item}' is field.{field.sub_type}." ) - return + return None if agg in (AGG_MIN, AGG_MAX) and field.sub_type in _UNORDERED_FIELD_SUBTYPES: err(f"@agg '{agg}' cannot order '{item}', a field.{field.sub_type}.") + return None + return _CheckedColumns(field) def _check_ratio_operands(ctx: _MemberCtx, ratio: MetaMeasure, err: Callable[[str], None]) -> None: @@ -692,7 +804,7 @@ def err(message: str) -> None: # --------------------------------------------------------------------------- -# R1–R7 — object.report +# R1–R9 — object.report # --------------------------------------------------------------------------- @@ -722,8 +834,8 @@ def err(message: str, node: MetaData = report, code: ErrorCode = _ERR_INVALID_RE source, ) - # R1 — @from resolves to an object.entity. Without it, R2/R3/R6/R7 and the - # @filter have nothing to resolve against, so they are skipped. + # R1 — @from resolves to an object.entity. Without it, R2/R3/R6/R7, R8/R9 and + # the @filter have nothing to resolve against, so they are skipped. from_ref = report_from(report) if from_ref is None: return # missing @from is ERR_MISSING_REQUIRED_ATTR @@ -799,13 +911,77 @@ def claim(field_name: str, what: str) -> None: if isinstance(segment, str) and _child_of_type(from_, TYPE_SEGMENT, segment) is None: err(f": @segment '{segment}' names no segment of @from '{from_key}'.") + # R8 — @spine is a to-one path from @from: rule D2's walk, started at @from. + spine = report_spine(report) + if spine is not None: + terminal = _walk_to_one_via( + root, from_, spine, lambda message: err(f": {message}"), _spine_walk(report, from_key) + ) + # R9 — every listed dimension is reached through the spine. Skipped when R8 failed. + if terminal is not None: + _check_spine_dimensions(root, report, from_, spine, terminal, err) + # S1 / F2 — the report's row scope over @from. + # ADR-0039: resolving. filter_value = report.get_meta_attr(OBJECT_REPORT_ATTR_FILTER) if isinstance(filter_value, dict): _check_filter(filter_value, from_, from_key, label, report, "", sink) +def _check_spine_dimensions( + root: MetaData, + report: MetaData, + from_: MetaData, + spine: str, + terminal: MetaData, + err: Callable[[str], None], +) -> None: + """R9 — every listed dimension is reached through the spine (``terminal`` is the + entity R8's walk reached). ``err`` takes the message tail after the report label.""" + from_key = from_.resolution_key() + spine_parts = _split_dotted(spine) + spine_hops = spine_parts.path if spine_parts is not None else [] + items = report_dimension_items(report) + if len(items) == 0: + err( + f": @spine '{spine}' needs at least one dimension. The report's rows are the dimension tuples of " + f"'{terminal.resolution_key()}'; with no dimension it would be one totals row." + ) + # One verdict per dimension, however many grains list it. + checked_dimensions: set[str] = set() + for item in items: + if item.name in checked_dimensions: + continue + checked_dimensions.add(item.name) + dim = _child_of_type(from_, TYPE_DIMENSION, item.name) + if not isinstance(dim, MetaDimension): + continue # R2 already reported it + via = dim.via() + if via is None: + err( + f": dimension '{item.name}' is read from @from '{from_key}', so it has no value in a row that has " + f"no facts. With @spine '{spine}' every dimension must be reached through it: declare the " + f"dimension over a field of '{terminal.resolution_key()}' (or an entity to-one from it) with an " + f"@via that begins '{spine}'." + ) + continue + # A @via that does not walk is D2's error, on the dimension; R9 does not report it again. + declaring = dim.parent if dim.parent is not None else from_ + if _walk_to_one_via(root, from_, via, lambda _message: None, _d2_walk(declaring)) is None: + continue + # Hop names are compared as written; the owner segment is not compared. + via_parts = _split_dotted(via) + hops = via_parts.path if via_parts is not None else [] + if hops[: len(spine_hops)] == spine_hops: + continue + err( + f": dimension '{item.name}' is reached by @via '{via}', which does not begin with the hops of " + f"@spine '{spine}'. Hop names are compared as written: if both name the same join, write the same " + f"hops; otherwise the dimension is not reached through the spine." + ) + + def _check_report_measure( root: MetaData, report: MetaData, diff --git a/server/python/src/metaobjects/meta/core/object/object_constants.py b/server/python/src/metaobjects/meta/core/object/object_constants.py index c87daebfe..1baeef43b 100644 --- a/server/python/src/metaobjects/meta/core/object/object_constants.py +++ b/server/python/src/metaobjects/meta/core/object/object_constants.py @@ -41,3 +41,5 @@ OBJECT_REPORT_ATTR_SEGMENT = "segment" #: Optional row scope over @from's fields (an attr.filter). OBJECT_REPORT_ATTR_FILTER = "filter" +#: Optional to-one path from @from to the entity whose rows supply the report's rows (R8). +OBJECT_REPORT_ATTR_SPINE = "spine" diff --git a/server/python/src/metaobjects/meta/core/reporting/meta_measure.py b/server/python/src/metaobjects/meta/core/reporting/meta_measure.py index d0614878c..f58c7017a 100644 --- a/server/python/src/metaobjects/meta/core/reporting/meta_measure.py +++ b/server/python/src/metaobjects/meta/core/reporting/meta_measure.py @@ -11,6 +11,7 @@ MEASURE_AGGS, MEASURE_SUBTYPE_RATIO, REPORTING_ATTR_AGG, + REPORTING_ATTR_DEFAULT, REPORTING_ATTR_DENOMINATOR, REPORTING_ATTR_DISTINCT, REPORTING_ATTR_FILTER, @@ -56,6 +57,14 @@ def filter(self) -> dict[str, object] | None: v = self.get_meta_attr(REPORTING_ATTR_FILTER) return v if isinstance(v, dict) else None + def default_value(self) -> int | None: + """ADR-0039: resolving, so an inherited measure keeps it. ``@default``: the integer + this measure reads when it would otherwise be null (rules M7/M8 decide where it is + legal). ``None`` for anything that is not an integer: ``bool`` is a subclass of + ``int`` in Python, and ``True`` is not a default.""" + v = self.get_meta_attr(REPORTING_ATTR_DEFAULT) + return v if isinstance(v, int) and not isinstance(v, bool) else None + def numerator(self) -> str | None: """ADR-0039: resolving. Name of the ``measure.aggregate`` sibling used as the numerator (``measure.ratio`` only).""" diff --git a/server/python/src/metaobjects/meta/core/reporting/report_accessors.py b/server/python/src/metaobjects/meta/core/reporting/report_accessors.py index 1cc4335c8..f4a7a668a 100644 --- a/server/python/src/metaobjects/meta/core/reporting/report_accessors.py +++ b/server/python/src/metaobjects/meta/core/reporting/report_accessors.py @@ -15,6 +15,7 @@ OBJECT_REPORT_ATTR_DIMENSIONS, OBJECT_REPORT_ATTR_FROM, OBJECT_REPORT_ATTR_MEASURES, + OBJECT_REPORT_ATTR_SPINE, ) from .reporting_constants import REPORT_DIMENSION_GRAIN_SEPARATOR @@ -39,6 +40,13 @@ def report_from(obj: MetaData) -> str | None: return v if isinstance(v, str) else None +def report_spine(obj: MetaData) -> str | None: + """ADR-0039: resolving. ``@spine``: the to-one path to the entity whose rows supply + the report's rows (``None`` when absent or empty).""" + v = obj.get_meta_attr(OBJECT_REPORT_ATTR_SPINE) + return v if isinstance(v, str) and v != "" else None + + def report_dimension_items(obj: MetaData) -> list[ReportDimensionItem]: """ADR-0039: resolving. The ``@dimensions`` items, each ``name`` or ``name:grain`` (split at the FIRST separator).""" diff --git a/server/python/src/metaobjects/meta/core/reporting/report_shape.py b/server/python/src/metaobjects/meta/core/reporting/report_shape.py index 0cbfd268b..d6a19a66b 100644 --- a/server/python/src/metaobjects/meta/core/reporting/report_shape.py +++ b/server/python/src/metaobjects/meta/core/reporting/report_shape.py @@ -1,6 +1,8 @@ """Table B of the FR-044 Plan 2 contract: a report's derived fields. The single Python definition; every port has a rule-for-rule copy, gated by -``fixtures/persistence-conformance/report-shapes.json``. +``fixtures/persistence-conformance/report-shapes.json``. ``required`` follows Table C of +docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md, which amends +Table B for a report with ``@spine`` and a measure with ``@default``. ADR-0039: every read is RESOLVING. Python naming inversion: ``attr()`` is OWN-ONLY here, so the TypeScript reference's ``attr()`` is ``get_meta_attr()`` @@ -27,6 +29,7 @@ FIELD_SUBTYPE_TIMESTAMP, ) from ..field.meta_field import MetaField +from ..identity.identity_constants import IDENTITY_ATTR_FIELDS from ..object.meta_object import MetaObject from .meta_dimension import MetaDimension from .meta_measure import MetaMeasure @@ -38,6 +41,7 @@ report_measure_item_name, report_measure_item_owner, report_measure_names, + report_spine, ) from .reporting_constants import ( AGG_AVG, @@ -122,6 +126,16 @@ def resolve_reporting_field_ref( ``None`` when any step fails. """ + named = _resolve_reporting_field_ref_named(ref, declaring, root, host) + return None if named is None else named[1] + + +def _resolve_reporting_field_ref_named( + ref: str, declaring: MetaData, root: MetaRoot, host: MetaObject | None = None +) -> tuple[MetaObject, MetaField] | None: + """:func:`resolve_reporting_field_ref`, also returning the entity the reference's + entity half NAMES (which, with ``host``, may be an ancestor of the entity the field is + read from). Table C reads that entity's primary identity.""" # ``Entity.field``; a package qualifier uses ``::``, so the member separator is the LAST dot. dot = ref.rfind(CHILD_REF_SEP) if dot <= 0: @@ -133,13 +147,67 @@ def resolve_reporting_field_ref( return None # ADR-0039: resolving fields(), so a field inherited through extends is found. member = ref[dot + len(CHILD_REF_SEP):] - return next((f for f in (host if host is not None else named).fields() if f.name == member), None) + field = next((f for f in (host if host is not None else named).fields() if f.name == member), None) + return None if field is None else (named, field) + + +def reporting_via_hops(via: str, declaring: MetaData, from_: MetaObject, root: MetaRoot) -> list[str] | None: + """The hop names of a dimension's ``@via`` (``Owner.hop[.hop...]``), the owner resolving + in the package of ``declaring`` and required to be ``from_`` or an entity it extends, + as the loader's walk requires (TS ``reportingViaHops``). ``None`` when the path is + malformed or its owner does not resolve so.""" + # The owner ends at the first ``.`` after the last ``::`` (a package qualifier has no ``.``). + last_sep = via.rfind(PACKAGE_SEP) + seg_start = 0 if last_sep == -1 else last_sep + len(PACKAGE_SEP) + dot = via.find(CHILD_REF_SEP, seg_start) + if dot <= seg_start: + return None + hops = via[dot + len(CHILD_REF_SEP):].split(CHILD_REF_SEP) + if any(h == "" for h in hops): + return None + owner = resolve_object_ref(root, via[:dot], _package_of_key(declaring.resolution_key())) + if owner is None or not _is_self_or_ancestor(owner, from_): + return None + return hops def _unresolved(report_name: str, what: str) -> ValueError: return ValueError(f"report '{report_name}': {what} does not resolve.") +def report_spine_hops(report: MetaObject, from_: MetaObject, root: MetaRoot) -> list[str] | None: + """The hop names of a report's ``@spine`` (``Owner.hop[.hop...]``), read as + :func:`reporting_via_hops` reads a ``@via``, the owner resolving in the REPORT's package + (loader rule R8). ``None`` when the report declares no ``@spine``. Raises a + ``ValueError`` naming the report when a declared ``@spine`` does not resolve (a report + that passed ``validate_reporting`` always resolves): reading it as "no spine" would + claim a column non-null that a spine row with no facts leaves null.""" + spine = report_spine(report) + if spine is None: + return None + hops = reporting_via_hops(spine, report, from_, root) + if hops is None: + raise _unresolved(report.name, f"@spine '{spine}'") + return hops + + +def _is_primary_key_field(entity: MetaObject, field: MetaField) -> bool: + """True when ``field`` is one of the ``@fields`` of ``entity``'s ``identity.primary``.""" + # ADR-0039: resolving — an identity.primary (and its @fields) inherited from an + # abstract base counts (primary_identity() reads children(); get_meta_attr resolves). + pk = entity.primary_identity() + if pk is None: + return False + raw = pk.get_meta_attr(IDENTITY_ATTR_FIELDS) + if isinstance(raw, list): + names = [str(v).strip() for v in raw] + elif isinstance(raw, str): + names = [s.strip() for s in raw.split(",") if s.strip()] + else: + names = [] + return field.name in names + + def _declared_member(from_: MetaObject, type_: str, name: str, cls: type[MetaData]) -> MetaData | None: # ADR-0039: resolving children(), so a member declared on an abstract base is found. return next( @@ -148,22 +216,54 @@ def _declared_member(from_: MetaObject, type_: str, name: str, cls: type[MetaDat ) +def _dimension_required( + dim: MetaDimension, + named: MetaObject, + of: MetaField, + from_: MetaObject, + root: MetaRoot, + spine: list[str] | None, + report_name: str, +) -> bool: + """Table C (``required`` of a dimension). Without ``@spine``: only a dimension with no + ``@via`` over an ``@of`` field whose effective ``@required`` is true. With ``@spine``: + only a dimension whose ``@via`` hops equal the spine's (a column of the spine entity + itself, whose rows are the report's rows) over an ``@of`` field that is ``@required`` or + a primary-key column of the entity ``@of`` names. A dimension beyond the spine is + reached by a LEFT OUTER join.""" + via = dim.via() + # ADR-0039 resolving: the @of field's effective @required (the attr only; a + # validator.required child does not count). + of_required = of.get_meta_attr(FIELD_ATTR_REQUIRED) is True + if spine is None: + return via is None and of_required + if via is None: + return False # a column of @from: null in a spine row with no facts + hops = reporting_via_hops(via, reporting_member_owner(dim, from_), from_, root) + if hops is None: + raise _unresolved(report_name, f"dimension '{dim.name}' @via '{via}'") + return hops == spine and (of_required or _is_primary_key_field(named, of)) + + def _dimension_field( - item: ReportDimensionItem, from_: MetaObject, root: MetaRoot, report_name: str + item: ReportDimensionItem, + from_: MetaObject, + root: MetaRoot, + report_name: str, + spine: list[str] | None, ) -> ReportField: dim = _declared_member(from_, TYPE_DIMENSION, item.name, MetaDimension) if not isinstance(dim, MetaDimension): raise _unresolved(report_name, f"dimension '{item.name}' on '{from_.name}'") vialess = dim.via() is None - of = resolve_reporting_field_ref( + ref = _resolve_reporting_field_ref_named( dim.of() or "", reporting_member_owner(dim, from_), root, from_ if vialess else None ) - if of is None: + if ref is None: raise _unresolved(report_name, f"dimension '{item.name}' @of") + named, of = ref name = report_derived_field_name(item) - # ADR-0039 resolving: the @of field's effective @required (the attr only; a - # validator.required child does not count). - required = vialess and of.get_meta_attr(FIELD_ATTR_REQUIRED) is True + required = _dimension_required(dim, named, of, from_, root, spine, report_name) if dim.is_time(): # Loader rule R2 guarantees a grain from the closed set; a tree built in code does not. grain = item.grain @@ -191,11 +291,13 @@ def _measure_field(item: str, report: MetaObject, from_: MetaObject, root: MetaR m = _declared_member(from_, TYPE_MEASURE, name, MetaMeasure) if not isinstance(m, MetaMeasure): raise _unresolved(report_name, f"measure '{item}' on '{from_.name}'") + # Table C: a count is never null; any other measure is not null when it has a @default. + required = (not m.is_ratio() and m.agg() == AGG_COUNT) or m.default_value() is not None if m.is_ratio(): - return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_DECIMAL, False, measure=m) + return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_DECIMAL, required, measure=m) agg = m.agg() if agg == AGG_COUNT: - return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_LONG, True, measure=m) + return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_LONG, required, measure=m) cols = m.of_columns() of = resolve_reporting_field_ref(cols[0] if cols else "", reporting_member_owner(m, from_), root, from_) if of is None: @@ -203,7 +305,7 @@ def _measure_field(item: str, report: MetaObject, from_: MetaObject, root: MetaR src = of.sub_type if agg == AGG_SUM: if src == FIELD_SUBTYPE_CURRENCY: - return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_CURRENCY, False, of, measure=m) + return ReportField(name, ROLE_MEASURE, FIELD_SUBTYPE_CURRENCY, required, of, measure=m) sub_type = ( FIELD_SUBTYPE_LONG if src in _SUM_LONG @@ -211,25 +313,27 @@ def _measure_field(item: str, report: MetaObject, from_: MetaObject, root: MetaR if src in _FLOATING else FIELD_SUBTYPE_DECIMAL ) - return ReportField(name, ROLE_MEASURE, sub_type, False, measure=m) + return ReportField(name, ROLE_MEASURE, sub_type, required, measure=m) if agg == AGG_AVG: sub_type = FIELD_SUBTYPE_DOUBLE if src in _FLOATING else FIELD_SUBTYPE_DECIMAL - return ReportField(name, ROLE_MEASURE, sub_type, False, measure=m) + return ReportField(name, ROLE_MEASURE, sub_type, required, measure=m) # min / max keep the source field's type. - return ReportField(name, ROLE_MEASURE, src, False, of, measure=m) + return ReportField(name, ROLE_MEASURE, src, required, of, measure=m) def report_shape(report: MetaObject, root: MetaRoot) -> ReportShape: - """Table B. Raises a ``ValueError`` naming the report when a reference does not - resolve (a report that passed ``validate_reporting`` always resolves).""" + """Table B, with Table C's ``required``. Raises a ``ValueError`` naming the report when + a reference does not resolve (a report that passed ``validate_reporting`` always + resolves).""" from_name = report_from(report) if from_name is None: raise _unresolved(report.name, "@from") from_ = resolve_object_ref(root, from_name, _package_of_key(report.resolution_key())) if not isinstance(from_, MetaObject): raise _unresolved(report.name, f"@from '{from_name}'") + spine = report_spine_hops(report, from_, root) fields = ( - *(_dimension_field(item, from_, root, report.name) for item in report_dimension_items(report)), + *(_dimension_field(item, from_, root, report.name, spine) for item in report_dimension_items(report)), *(_measure_field(item, report, from_, root) for item in report_measure_names(report)), ) return ReportShape(report, from_, tuple(fields)) diff --git a/server/python/src/metaobjects/meta/core/reporting/reporting_constants.py b/server/python/src/metaobjects/meta/core/reporting/reporting_constants.py index d1fcf090e..89ce512e1 100644 --- a/server/python/src/metaobjects/meta/core/reporting/reporting_constants.py +++ b/server/python/src/metaobjects/meta/core/reporting/reporting_constants.py @@ -61,6 +61,8 @@ #: ``measure.ratio`` operand names. REPORTING_ATTR_NUMERATOR = "numerator" REPORTING_ATTR_DENOMINATOR = "denominator" +#: Integer a measure reads when it would otherwise be null (``measure.aggregate`` / ``measure.ratio``). +REPORTING_ATTR_DEFAULT = "default" # --------------------------------------------------------------------------- # Closed sets — mirrored by ``allowedValues`` in spec/metamodel/reporting.json. diff --git a/server/python/src/metaobjects/spec_metamodel/object.json b/server/python/src/metaobjects/spec_metamodel/object.json index cb5f9946f..76187a94a 100644 --- a/server/python/src/metaobjects/spec_metamodel/object.json +++ b/server/python/src/metaobjects/spec_metamodel/object.json @@ -64,7 +64,8 @@ { "type": "attr", "subType": "string", "name": "dimensions", "isArray": true, "min": 0, "max": 1, "description": "Dimension names of @from; a time dimension is written 'name:grain'. Absent means one global row." }, { "type": "attr", "subType": "string", "name": "measures", "isArray": true, "min": 1, "max": 1, "description": "Measure names of @from." }, { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional segment of @from scoping the rows." }, - { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope over @from's fields; may use relative-date values." } + { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope over @from's fields; may use relative-date values." }, + { "type": "attr", "subType": "string", "name": "spine", "min": 0, "max": 1, "description": "Optional to-one path from @from to the entity whose rows supply the report's rows (e.g. 'Purchase.program'), written like a dimension's @via. With it the report has one row per distinct dimension tuple among THAT entity's rows, including the ones no row of @from refers to: a count there is 0 and any other measure is null unless it declares @default. Every listed dimension must be reached through this path. @segment and @filter still scope the rows of @from and never remove a row. A row of @from whose reference is null or matches nothing is in no row of the report." } ] } ] diff --git a/server/python/src/metaobjects/spec_metamodel/reporting.json b/server/python/src/metaobjects/spec_metamodel/reporting.json index 024196d63..019dda238 100644 --- a/server/python/src/metaobjects/spec_metamodel/reporting.json +++ b/server/python/src/metaobjects/spec_metamodel/reporting.json @@ -24,14 +24,16 @@ { "type": "attr", "subType": "string", "name": "of", "isArray": true, "min": 1, "max": 1, "description": "Dotted Entity.field reference(s) on the declaring entity. A bare string is one column; more than one requires @agg: count and @distinct: true." }, { "type": "attr", "subType": "boolean", "name": "distinct", "min": 0, "max": 1, "description": "Count distinct values. Legal only with @agg: count." }, { "type": "attr", "subType": "filter", "name": "filter", "min": 0, "max": 1, "description": "Optional row scope (a portable attr.filter over the declaring entity's fields). May use relative-date values ({ now: \"-P7D\" }). Combines with @segment by AND." }, - { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional name of a segment declared on the same entity. Combines with @filter by AND." } + { "type": "attr", "subType": "string", "name": "segment", "min": 0, "max": 1, "description": "Optional name of a segment declared on the same entity. Combines with @filter by AND." }, + { "type": "attr", "subType": "int", "name": "default", "min": 0, "max": 1, "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } ] }, { "type": "measure", "subType": "ratio", "description": "A named quotient of two measure.aggregate siblings (FR-044), lowered as numerator / NULLIF(denominator, 0) and typed decimal. A zero denominator yields null.", "whenToUse": "Averages per unit that are not a plain avg: average days engaged per starter.", "children": [ { "type": "attr", "subType": "string", "name": "numerator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." }, - { "type": "attr", "subType": "string", "name": "denominator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." } + { "type": "attr", "subType": "string", "name": "denominator", "min": 1, "max": 1, "description": "Name of a measure.aggregate on the same entity." }, + { "type": "attr", "subType": "int", "name": "default", "min": 0, "max": 1, "description": "Optional integer the measure reads when it would otherwise be null: nothing matched, every matched value is null, or (a ratio) the denominator is zero or null. The derived report field is then never null. Refused on @agg: count (a count is never null) and on min/max over a field that is not numeric. A ratio's operand carries its own @default into the ratio." } ] }, { "type": "segment", "subType": "filter", "description": "A named, reusable row filter on the declaring entity (FR-044). Measures and reports reference it by name; exporters emit it as a named segment.", diff --git a/server/python/tests/codegen/test_report_router.py b/server/python/tests/codegen/test_report_router.py index 6b9fe05a0..990c67bac 100644 --- a/server/python/tests/codegen/test_report_router.py +++ b/server/python/tests/codegen/test_report_router.py @@ -1,8 +1,9 @@ """FR-044 Plan 3 — the Python port generates a keyless read-only surface for a served report (Table A/B/C/E) and no item routes for ANY keyless object (open question 4). -The model is the shared ``fixtures/api-contract-conformance/report`` corpus: three -view-backed reports and one sourceless one (``InvoiceDays``), over a writable ``Invoice``. +The model is the shared ``fixtures/api-contract-conformance/report`` corpus: four +view-backed reports and one sourceless one (``InvoiceDays``), over a writable ``Invoice`` +and, for the ``@spine`` report ``ProductRevenue``, ``Product`` and its child ``Sale``. """ from __future__ import annotations @@ -15,6 +16,7 @@ from metaobjects import MetaDataLoader from metaobjects.codegen.config import GenConfig from metaobjects.codegen.generator_registry import GeneratorBuildContext, list_generators +from metaobjects.codegen.generators.entity_model import render_entity_model from metaobjects.codegen.generators.filter_allowlist_generator import render_filter_allowlist from metaobjects.codegen.generators.router_generator import render_router from metaobjects.codegen.instance_artifacts import has_item_route, is_served_report @@ -42,6 +44,7 @@ def test_served_report_predicate_follows_table_a() -> None: root = _load() assert is_served_report(_obj(root, "InvoiceStatusTotals")) assert is_served_report(_obj(root, "InvoiceTotals")) + assert is_served_report(_obj(root, "ProductRevenue")) assert not is_served_report(_obj(root, "InvoiceDays")) # sourceless assert not is_served_report(_obj(root, "Invoice")) # an entity # the read model keeps the report subtype and a copy of the source: also served @@ -181,7 +184,7 @@ def test_report_allowlist_names_the_report_s_own_derived_fields() -> None: assert '"reference"' not in src -def test_run_gen_serves_three_reports_and_nothing_for_the_sourceless_one(tmp_path: Path) -> None: +def test_run_gen_serves_four_reports_and_nothing_for_the_sourceless_one(tmp_path: Path) -> None: root = _load() templates = tmp_path / "t" templates.mkdir() @@ -193,9 +196,41 @@ def test_run_gen_serves_three_reports_and_nothing_for_the_sourceless_one(tmp_pat out = tmp_path / "out" run_gen(GenConfig(out_dir=str(out)), root, generators=gens) files = {p.name for p in out.rglob("*.py")} - for snake in ("invoice_status_totals", "invoices_by_month", "invoice_totals"): + for snake in ("invoice_status_totals", "invoices_by_month", "invoice_totals", "product_revenue"): assert f"{snake}_router.py" in files assert f"{snake}_filter_allowlist.py" in files assert f"{snake}_names.py" in files assert not any(f.startswith("invoice_days") for f in files) assert not any("invoice_days" in f for f in files) + + +def _row_fields(report: str) -> list[str]: + """The field lines of a served report's generated Pydantic row model.""" + root = _load() + src = render_entity_model(report_read_model(_obj(root, report), root)) + body = src.split(f"class {report}(BaseModel):\n", 1)[1] + return [ln.strip() for ln in body.splitlines() if ln.startswith(" ") and ln.strip()] + + +def test_a_count_is_not_optional_and_a_plain_sum_is() -> None: + """The spelling a required read-model field takes today: the bare type, no default.""" + assert _row_fields("InvoiceStatusTotals") == [ + 'status: Literal["OPEN", "PAID", "VOID"]', + "invoices: int", + "totalCents: int | None = None", + "paidCents: int | None = None", + ] + + +def test_the_spine_key_and_a_defaulted_measure_are_not_optional() -> None: + """Table F: under ``@spine`` the spine entity's key (``Product.id``, no ``@required``) + and a ``@required`` spine column are the bare type; a sum with ``@default: 0`` is the + bare type; the same sum without ``@default`` keeps ``| None``. No generator changed: + the read model's ``@required`` (Table C) is what the row model reads.""" + assert _row_fields("ProductRevenue") == [ + "productId: int", + "productName: str", + "sales: int", + "revenueCents: int | None = None", + "revenueOrZero: int", + ] diff --git a/server/python/tests/integration/generated_report_app.py b/server/python/tests/integration/generated_report_app.py index 8f8a36181..65b6883a7 100644 --- a/server/python/tests/integration/generated_report_app.py +++ b/server/python/tests/integration/generated_report_app.py @@ -2,7 +2,7 @@ Peer of ``generated_projection_app.py``, for the ``report/`` api-contract corpus (``fixtures/api-contract-conformance/report/``). Runs the REAL generation path -(``run_gen``, the one ``metaobjects gen`` takes) for the three served reports, writes the +(``run_gen``, the one ``metaobjects gen`` takes) for the four served reports, writes the emitted package to a temp dir, imports each generated router UNMODIFIED, and mounts it behind an in-memory repository. @@ -11,7 +11,8 @@ Python emits no SQL for a report (ADR-0015: view SQL is TypeScript-only), so the in-memory repository stands in for the view. It is seeded with the ``reports`` half of the corpus -``seed.json``, which is what the three views return for the ``invoices`` rows. Only +``seed.json``, which is what the four views return for the base-table rows (the other +top-level keys, which this lane does not load: it has no tables). Only ``list`` / ``count`` exist, matching the generated ``Protocol``: a report has no ``find_by_id``. """ @@ -35,11 +36,13 @@ from metaobjects.codegen.runner import run_gen from metaobjects.codegen.runtime.filter_parser import FilterPredicate -#: The three served reports, as (report name, generated module stem). +#: The four served reports, as (report name, generated module stem). ``ProductRevenue`` +#: declares ``@spine``: its seeded rows include the product with no sale. SERVED_REPORTS: dict[str, str] = { "InvoiceStatusTotals": "invoice_status_totals", "InvoicesByMonth": "invoices_by_month", "InvoiceTotals": "invoice_totals", + "ProductRevenue": "product_revenue", } #: The sourceless report: generated nowhere, mounted nowhere. UNSERVED_REPORT = "InvoiceDays" diff --git a/server/python/tests/integration/test_api_contract_report.py b/server/python/tests/integration/test_api_contract_report.py index c7fac4fb6..735ebdaf6 100644 --- a/server/python/tests/integration/test_api_contract_report.py +++ b/server/python/tests/integration/test_api_contract_report.py @@ -1,7 +1,7 @@ """FR-044 — cross-port API contract conformance for a VIEW-BACKED ``object.report``. Drives ``fixtures/api-contract-conformance/report/`` against the GENERATED routers for the -three served reports (the deployed artifact), with a read-only in-memory repo behind each +four served reports (the deployed artifact), with a read-only in-memory repo behind each generated consumer seam, seeded from ``seed.json``'s ``reports`` half. Generated lane only, by design — see the corpus README. The thing under test is whether @@ -60,7 +60,7 @@ def _load_scenarios() -> list[tuple[str, dict[str, Any]]]: def _load_seed_reports() -> dict[str, list[dict[str, Any]]]: - """The seam lane's half of the seed: what the three views return for the base rows.""" + """The seam lane's half of the seed: what the four views return for the base rows.""" parsed = json.loads((_CORPUS / "seed.json").read_text()) reports = parsed.get("reports") if not isinstance(reports, dict): @@ -79,13 +79,15 @@ def test_exactly_the_served_reports_are_generated() -> None: """The sourceless report generates nothing; each served one generates its row model, allowlist and router (and the names module is not a route concern, so it is not asked for here: only the three generators the lane runs).""" + assert len(SERVED_REPORTS) == 4 for report, stem in SERVED_REPORTS.items(): assert f"{report}.py" in _EMITTED assert f"{stem}_filter_allowlist.py" in _EMITTED assert f"{stem}_router.py" in _EMITTED assert not any(UNSERVED_REPORT in f or "invoice_days" in f for f in _EMITTED), _EMITTED - # The base entity was not asked for, so no writable router sits beside the reports. - assert "invoice_router.py" not in _EMITTED + # The base entities were not asked for, so no writable router sits beside the reports. + for base in ("invoice", "product", "sale"): + assert f"{base}_router.py" not in _EMITTED def test_the_unserved_report_mounts_no_route() -> None: diff --git a/server/python/tests/runtime/test_object_manager_report.py b/server/python/tests/runtime/test_object_manager_report.py index 26d9886f4..8b6508b16 100644 --- a/server/python/tests/runtime/test_object_manager_report.py +++ b/server/python/tests/runtime/test_object_manager_report.py @@ -5,11 +5,12 @@ ``_require_entity``, so the column list, filter/sort resolution, read coercion and table resolution all see an ordinary view-backed object. These tests use a recording driver (no database): they pin the SQL, the refusals, and that the loaded tree is -never touched. The rows themselves are proven by the six shared ``report-*`` persistence +never touched. The rows themselves are proven by the nine shared ``report-*`` persistence scenarios against Postgres (``tests/integration/test_query_scenarios.py``). """ from __future__ import annotations +from decimal import Decimal from pathlib import Path from typing import Any @@ -103,6 +104,42 @@ def test_find_many_selects_the_derived_columns_from_the_view() -> None: ) +class CannedDriver(RecordingDriver): + """A ``RecordingDriver`` whose ``select`` answers with the given rows.""" + + def __init__(self, rows: list[dict[str, Any]]) -> None: + super().__init__() + self._rows = rows + + def select(self, sql: str, params: tuple[Any, ...] = ()) -> SelectResult: + super().select(sql, params) + return SelectResult([dict(r) for r in self._rows], {}) + + +def test_a_spine_view_with_an_empty_row_is_read() -> None: + """``ProgramRoster`` (``@spine: Week.fkProgram``): a program with no weeks is a row of + the view. The runtime reads it as it reads any row: the count is 0, a measure with no + ``@default`` is ``None`` and a defaulted one is its default. Nothing in the read path + special-cases a spine report; the shape and the registration are the whole change.""" + empty = { + "programKey": 3, + "programTitle": "Mobility", + "weeks": 0, + "totalMinutes": None, + "totalMinutesOrZero": 0, + "longShare": None, + "longShareOrZero": Decimal("0"), + } + drv = CannedDriver([empty]) + rows = _om(_canonical_root(), drv).find_many("ProgramRoster", {"totalMinutesOrZero": {"eq": 0}}) + assert drv.sql[0] == ( + 'SELECT "programKey", "programTitle", "weeks", "totalMinutes", "totalMinutesOrZero", "longShare", ' + '"longShareOrZero" FROM "v_program_roster" WHERE "totalMinutesOrZero" = %s' + ) + assert drv.params[0] == (0,) + assert rows == [empty] + + def test_filter_sort_limit_on_derived_fields() -> None: root = _canonical_root() drv = RecordingDriver() diff --git a/server/python/tests/test_report_read_model.py b/server/python/tests/test_report_read_model.py index 36a8b5f24..cd37a93f8 100644 --- a/server/python/tests/test_report_read_model.py +++ b/server/python/tests/test_report_read_model.py @@ -178,3 +178,40 @@ def test_every_derived_field_with_a_filter_band_is_filterable() -> None: "code", "ref", "tags", "status", "bookedAtHour", "bookedAtDay", "sales", "revenue", "minAmount", "totalWeight", "maxWeight", } + + +def test_a_measure_with_default_yields_a_detached_field_with_required_true() -> None: + """Table C: a defaulted measure is never null, so its derived field is ``@required``; the + same measure without ``@default`` is not. The measure's ``@default`` itself is never + carried onto the field (a field ``@default`` is an insert default, not this).""" + model = { + "metadata.root": { + "package": "acme", + "children": [ + { + "object.entity": { + "name": "Sale", + "children": [ + {"source.rdb": {"@table": "sales"}}, + {"field.long": {"name": "id"}}, + {"field.currency": {"name": "amountCents", "@currency": "EUR"}}, + {"identity.primary": {"name": "pk", "@fields": "id", "@generation": "increment"}}, + {"measure.aggregate": {"name": "revenue", "@agg": "sum", "@of": "Sale.amountCents", + "@default": 0}}, + {"measure.aggregate": {"name": "revenueRaw", "@agg": "sum", "@of": "Sale.amountCents"}}, + ], + } + }, + {"object.report": {"name": "Revenue", "@from": "Sale", "@measures": ["revenue", "revenueRaw"]}}, + ], + } + } + result = MetaDataLoader().load([InMemoryStringSource(json.dumps(model), "meta.acme.json")]) + assert result.errors == [], [e.message for e in result.errors] + report = next(c for c in result.root.children() if c.type == TYPE_OBJECT and c.name == "Revenue") + fields = report_read_model(report, result.root).fields() + assert [(f.name, f.get_meta_attr(FIELD_ATTR_REQUIRED)) for f in fields] == [ + ("revenue", True), + ("revenueRaw", False), + ] + assert all(f.get_meta_attr(FIELD_ATTR_DEFAULT) is None for f in fields) diff --git a/server/python/tests/test_report_shape.py b/server/python/tests/test_report_shape.py index 97f3fe349..3ea208463 100644 --- a/server/python/tests/test_report_shape.py +++ b/server/python/tests/test_report_shape.py @@ -10,8 +10,10 @@ """ from __future__ import annotations +import copy import json from pathlib import Path +from typing import Any, cast import pytest @@ -24,6 +26,7 @@ OBJECT_REPORT_ATTR_DIMENSIONS, OBJECT_REPORT_ATTR_FROM, OBJECT_REPORT_ATTR_MEASURES, + OBJECT_REPORT_ATTR_SPINE, OBJECT_SUBTYPE_REPORT, ) from metaobjects.meta.core.reporting.report_accessors import ( @@ -31,7 +34,8 @@ report_measure_item_owner, ) from metaobjects.meta.core.reporting.report_read_model import report_read_source -from metaobjects.meta.core.reporting.report_shape import report_shape +from metaobjects.meta.core.reporting.report_shape import report_shape, report_spine_hops +from metaobjects.meta.meta_root import MetaRoot from metaobjects.shared.base_types import TYPE_OBJECT CORPUS = Path(__file__).parents[3] / "fixtures" / "persistence-conformance" @@ -118,9 +122,9 @@ def test_derived_shapes_byte_match_the_committed_artifact() -> None: assert generate_report_shapes_json(_root()) == expected -def test_the_canonical_model_has_six_reports() -> None: +def test_the_canonical_model_has_nine_reports() -> None: reports = [c for c in _root().children() if c.type == TYPE_OBJECT and c.sub_type == OBJECT_SUBTYPE_REPORT] - assert len(reports) == 6 + assert len(reports) == 9 assert all(isinstance(r, MetaObject) for r in reports) @@ -356,3 +360,299 @@ def row(name: str) -> tuple[str, bool, str | None]: def test_a_sourceless_report_has_a_shape_and_no_view() -> None: root = _load(_file("shop", [_CUBE_BASE, _CUBE, _CUBE_REPORT])) assert json.loads(generate_report_shapes_json(root))["reports"][0]["view"] is None + + +# --------------------------------------------------------------------------- +# Table C of docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md: +# under @spine a column of the spine entity is not nullable when it is @required or a +# primary-key column; a dimension beyond the spine is; a measure with @default is not. +# The same cases as the TypeScript report-shape.test.ts. +# --------------------------------------------------------------------------- + +_CATALOG: dict[str, Any] = { + "object.entity": { + "name": "Catalog", + "children": [ + {"source.rdb": {"@table": "catalogs"}}, + {"field.long": {"name": "id"}}, + {"field.string": {"name": "name", "@required": True}}, + {"identity.primary": {"name": "pk", "@fields": ["id"]}}, + ], + } +} +_PROGRAM: dict[str, Any] = { + "object.entity": { + "name": "Program", + "children": [ + {"source.rdb": {"@table": "programs"}}, + # No @required: a key column is not nullable because it is the key. + {"field.long": {"name": "id"}}, + {"field.string": {"name": "title", "@required": True}}, + {"field.string": {"name": "subtitle"}}, + {"field.timestamp": {"name": "publishedAt", "@required": True}}, + {"field.long": {"name": "catalogId"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"]}}, + {"identity.reference": {"name": "fkCatalog", "@references": "Catalog", "@fields": ["catalogId"]}}, + {"relationship.association": {"name": "catalog", "@objectRef": "Catalog", "@cardinality": "one"}}, + ], + } +} +_PURCHASE: dict[str, Any] = { + "object.entity": { + "name": "Purchase", + "children": [ + {"source.rdb": {"@table": "purchases"}}, + {"field.long": {"name": "id"}}, + {"field.long": {"name": "programId"}}, + {"field.int": {"name": "minutes", "@required": True}}, + {"field.currency": {"name": "amountCents", "@currency": "USD"}}, + {"field.double": {"name": "score"}}, + {"field.timestamp": {"name": "purchasedAt"}}, + {"identity.primary": {"name": "pk", "@fields": ["id"]}}, + {"identity.reference": {"name": "fkProgram", "@references": "Program", "@fields": ["programId"]}}, + {"relationship.association": {"name": "program", "@objectRef": "Program", "@cardinality": "one"}}, + {"dimension.attribute": {"name": "programId", "@of": "Program.id", "@via": "Purchase.program"}}, + {"dimension.attribute": {"name": "programTitle", "@of": "Program.title", "@via": "Purchase.program"}}, + {"dimension.attribute": {"name": "programSubtitle", "@of": "Program.subtitle", "@via": "Purchase.program"}}, + { + "dimension.time": { + "name": "publishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": ["hour", "month"], + } + }, + {"dimension.attribute": {"name": "catalogId", "@of": "Catalog.id", "@via": "Purchase.program.catalog"}}, + # Catalog.name is @required, but one hop beyond a Purchase.program spine its join is LEFT OUTER. + {"dimension.attribute": {"name": "catalogName", "@of": "Catalog.name", "@via": "Purchase.program.catalog"}}, + {"dimension.attribute": {"name": "minutes", "@of": "Purchase.minutes"}}, + {"measure.aggregate": {"name": "purchases", "@agg": "count", "@of": "Purchase.id"}}, + {"measure.aggregate": {"name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0}}, + {"measure.aggregate": {"name": "revenueRaw", "@agg": "sum", "@of": "Purchase.amountCents"}}, + {"measure.aggregate": {"name": "avgMinutes", "@agg": "avg", "@of": "Purchase.minutes", "@default": 0}}, + {"measure.aggregate": {"name": "avgMinutesRaw", "@agg": "avg", "@of": "Purchase.minutes"}}, + {"measure.aggregate": {"name": "minMinutes", "@agg": "min", "@of": "Purchase.minutes", "@default": -1}}, + {"measure.aggregate": {"name": "minMinutesRaw", "@agg": "min", "@of": "Purchase.minutes"}}, + {"measure.aggregate": {"name": "totalScore", "@agg": "sum", "@of": "Purchase.score"}}, + {"measure.aggregate": {"name": "avgScore", "@agg": "avg", "@of": "Purchase.score"}}, + {"measure.aggregate": {"name": "lastAt", "@agg": "max", "@of": "Purchase.purchasedAt"}}, + {"measure.ratio": {"name": "share", "@numerator": "revenue", "@denominator": "purchases", "@default": 0}}, + {"measure.ratio": {"name": "shareRaw", "@numerator": "revenue", "@denominator": "purchases"}}, + ], + } +} +_ALL_MEASURES = [ + "purchases", "revenue", "revenueRaw", "avgMinutes", "avgMinutesRaw", + "minMinutes", "minMinutesRaw", "share", "shareRaw", +] +_SPINE_SALES: dict[str, Any] = { + "object.report": { + "name": "SpineSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": [ + "programId", "programTitle", "programSubtitle", "publishedAt:hour", "publishedAt:month", + "catalogId", "catalogName", + ], + "@measures": _ALL_MEASURES, + } +} +_PLAIN_SALES: dict[str, Any] = { + "object.report": { + "name": "PlainSales", + "@from": "Purchase", + "@dimensions": ["programId", "programTitle", "programSubtitle", "catalogId", "catalogName", "minutes"], + "@measures": _ALL_MEASURES, + } +} +_CATALOG_SALES: dict[str, Any] = { + "object.report": { + "name": "CatalogSales", + "@from": "Purchase", + "@spine": "Purchase.program.catalog", + "@dimensions": ["catalogId", "catalogName"], + "@measures": ["purchases", "revenue"], + } +} + + +def _sales_model() -> MetaRoot: + root = _load(_file("acme", [_CATALOG, _PROGRAM, _PURCHASE, _SPINE_SALES, _PLAIN_SALES, _CATALOG_SALES])) + return cast(MetaRoot, root) + + +def _required_of(root: MetaRoot, name: str) -> dict[str, bool]: + return {f.name: f.required for f in report_shape(_object(root, name), root).fields} + + +def _purchase_member(entity: dict[str, Any], wrapper: str, name: str) -> dict[str, Any]: + children: list[dict[str, Any]] = entity["object.entity"]["children"] + member: dict[str, Any] = next(c[wrapper] for c in children if wrapper in c and c[wrapper]["name"] == name) + return member + + +def _inherited_spine_root() -> MetaRoot: + fixture = CORPUS.parent / "conformance" / "reporting-spine-inherited" / "input" / "meta.shop.json" + result = MetaDataLoader().load([InMemoryStringSource(fixture.read_text(encoding="utf-8"), "meta.shop.json")]) + assert result.errors == [], [e.message for e in result.errors] + return cast(MetaRoot, result.root) + + +def test_under_spine_a_column_of_the_spine_entity_is_required_when_it_is_a_key_or_required() -> None: + required = _required_of(_sales_model(), "SpineSales") + assert required["programId"] is True # Program.id: no @required, in identity.primary + assert required["programTitle"] is True # Program.title: @required + assert required["programSubtitle"] is False # Program.subtitle: optional + # A time dimension over a @required spine column, at either grain path (timestamp / date). + assert required["publishedAtHour"] is True + assert required["publishedAtMonth"] is True + + +def test_under_spine_a_dimension_one_hop_beyond_the_spine_is_not_required() -> None: + required = _required_of(_sales_model(), "SpineSales") + assert required["catalogId"] is False # Catalog.id: a key, beyond the spine + assert required["catalogName"] is False # Catalog.name: @required, beyond the spine + + +def test_a_two_hop_spine_a_dimension_whose_via_equals_the_whole_spine_is_on_it() -> None: + required = _required_of(_sales_model(), "CatalogSales") + assert required["catalogId"] is True + assert required["catalogName"] is True + + +def test_the_same_dimensions_without_spine_keep_todays_values() -> None: + required = _required_of(_sales_model(), "PlainSales") + assert required["programId"] is False + assert required["programTitle"] is False + assert required["programSubtitle"] is False + assert required["catalogId"] is False + assert required["catalogName"] is False + assert required["minutes"] is True # no @via, @of @required: unchanged + + +def test_a_sum_avg_min_and_ratio_are_required_with_default_and_not_without_a_count_always_is() -> None: + root = _sales_model() + for name in ("SpineSales", "PlainSales"): + required = _required_of(root, name) + assert {m: required[m] for m in _ALL_MEASURES} == { + "purchases": True, + "revenue": True, + "revenueRaw": False, + "avgMinutes": True, + "avgMinutesRaw": False, + "minMinutes": True, + "minMinutesRaw": False, + "share": True, + "shareRaw": False, + }, name + + +def test_a_count_with_a_default_past_the_loader_is_still_required() -> None: + with_count_default = copy.deepcopy(_PURCHASE) + _purchase_member(with_count_default, "measure.aggregate", "purchases")["@default"] = 5 + result = MetaDataLoader().load([_file("acme", [_CATALOG, _PROGRAM, with_count_default, _PLAIN_SALES])]) + assert ["@default cannot apply to @agg: count" in e.message for e in result.errors] == [True] + assert _required_of(cast(MetaRoot, result.root), "PlainSales")["purchases"] is True + + +def test_subtypes_and_type_sources_are_untouched_by_default() -> None: + root = _sales_model() + shape = report_shape(_object(root, "SpineSales"), root).fields + assert [(f.name, f.sub_type, None if f.type_source is None else f.type_source.name) for f in shape] == [ + ("programId", "long", "id"), + ("programTitle", "string", "title"), + ("programSubtitle", "string", "subtitle"), + ("publishedAtHour", "timestamp", "publishedAt"), + ("publishedAtMonth", "date", None), + ("catalogId", "long", "id"), + ("catalogName", "string", "name"), + ("purchases", "long", None), + ("revenue", "currency", "amountCents"), + ("revenueRaw", "currency", "amountCents"), + ("avgMinutes", "decimal", None), + ("avgMinutesRaw", "decimal", None), + ("minMinutes", "int", "minutes"), + ("minMinutesRaw", "int", "minutes"), + ("share", "decimal", None), + ("shareRaw", "decimal", None), + ] + + +def test_an_identity_primary_inherited_from_an_abstract_base_makes_the_key_column_required() -> None: + """ADR-0039: the spine entity's key is read resolving.""" + keyed = { + "object.entity": { + "name": "Keyed", + "abstract": True, + "children": [{"field.long": {"name": "id"}}, {"identity.primary": {"name": "pk", "@fields": ["id"]}}], + } + } + program_children: list[dict[str, Any]] = _PROGRAM["object.entity"]["children"] + inherited_program = { + "object.entity": { + "name": "Program", + "extends": "Keyed", + "children": [ + c + for c in program_children + if "identity.primary" not in c and not ("field.long" in c and c["field.long"]["name"] == "id") + ], + } + } + root = cast(MetaRoot, _load(_file("acme", [keyed, _CATALOG, inherited_program, _PURCHASE, _SPINE_SALES]))) + assert _required_of(root, "SpineSales")["programId"] is True + + +def test_under_spine_a_dimension_whose_via_does_not_resolve_raises_naming_the_report() -> None: + broken = copy.deepcopy(_PURCHASE) + # An owner that is not @from (or an entity it extends): the hop split does not resolve it. + _purchase_member(broken, "dimension.attribute", "programSubtitle")["@via"] = "Program.catalog" + # Past the loader, which refuses the @via under rule D2. + result = MetaDataLoader().load([_file("acme", [_CATALOG, _PROGRAM, broken, _SPINE_SALES])]) + assert len(result.errors) == 1 + root = cast(MetaRoot, result.root) + with pytest.raises(ValueError) as e: + report_shape(_object(root, "SpineSales"), root) + assert str(e.value) == "report 'SpineSales': dimension 'programSubtitle' @via 'Program.catalog' does not resolve." + + +def test_a_spine_written_with_an_abstract_base_as_its_owner_over_a_concrete_from() -> None: + root = _inherited_spine_root() + fields = report_shape(_object(root, "ProgramMinutes"), root).fields + assert [(f.name, f.required) for f in fields] == [ + ("programId", True), + ("programTitle", True), + ("totalMinutes", True), + ] + + +def test_report_spine_hops_none_without_spine_the_hop_names_with_one() -> None: + root = _sales_model() + from_ = _object(root, "Purchase") + assert report_spine_hops(_object(root, "PlainSales"), from_, root) is None + assert report_spine_hops(_object(root, "SpineSales"), from_, root) == ["program"] + assert report_spine_hops(_object(root, "CatalogSales"), from_, root) == ["program", "catalog"] + + +def test_report_spine_hops_owner_may_be_an_entity_from_extends_resolved_in_the_reports_package() -> None: + root = _inherited_spine_root() + hops = report_spine_hops(_object(root, "ProgramMinutes"), _object(root, "WorkoutEvent"), root) + assert hops == ["program"] + + +@pytest.mark.parametrize("spine", ["Program.catalog", "Purchase", "Purchase..program"]) +def test_a_spine_that_does_not_resolve_raises_naming_the_report(spine: str) -> None: + """A report built in code (never added to the root): the loader refuses these under R8.""" + root = _sales_model() + report = MetaObject(TYPE_OBJECT, OBJECT_SUBTYPE_REPORT, "SpineSales") + report.package = "acme" + report.set_attr(OBJECT_REPORT_ATTR_FROM, "Purchase") + report.set_attr(OBJECT_REPORT_ATTR_SPINE, spine) + report.set_attr(OBJECT_REPORT_ATTR_DIMENSIONS, ["programTitle"]) + report.set_attr(OBJECT_REPORT_ATTR_MEASURES, ["purchases"]) + with pytest.raises(ValueError) as e: + report_spine_hops(report, _object(root, "Purchase"), root) + assert str(e.value) == f"report 'SpineSales': @spine '{spine}' does not resolve." + with pytest.raises(ValueError) as e: + report_shape(report, root) + assert str(e.value) == f"report 'SpineSales': @spine '{spine}' does not resolve." diff --git a/server/python/tests/test_reporting_inert.py b/server/python/tests/test_reporting_inert.py index 6caa34f22..ab334df5c 100644 --- a/server/python/tests/test_reporting_inert.py +++ b/server/python/tests/test_reporting_inert.py @@ -5,8 +5,10 @@ validates them at load. A model that USES the vocabulary must generate exactly what the same model without it generates, byte for byte, through every registered generator, with ONE exception: a report that declares a read-only ``source.rdb @kind: view`` is SERVED -(Plan 3, Table A) and gains exactly the files below, all for ``StoreTotals``. The other two -reports (``DailyRevenue``, ``ProgramEngagement``) declare no view and generate nothing. +(Plan 3, Table A) and gains exactly the files below, all for ``StoreTotals``. The other three +reports (``DailyRevenue``, ``ProgramEngagement`` and ``ProgramCatalogue``, which declares +``@spine`` and lists a dimension reached by ``@via``) declare no view and generate nothing, +and a measure ``@default`` (``avgDaysPerStarter``) changes nothing either. The model pair is ``fixtures/codegen-noop/reporting/{with,without}``, shared with the other four ports' copies of this test. ``with/`` carries a report that declares a read-only @@ -36,7 +38,9 @@ ) from metaobjects.codegen.runner import run_gen from metaobjects.meta.core.object.object_constants import OBJECT_SUBTYPE_REPORT -from metaobjects.shared.base_types import TYPE_OBJECT +from metaobjects.meta.core.reporting.meta_measure import MetaMeasure +from metaobjects.meta.core.reporting.report_accessors import report_spine +from metaobjects.shared.base_types import TYPE_MEASURE, TYPE_OBJECT MODELS = Path(__file__).parents[3] / "fixtures" / "codegen-noop" / "reporting" THREW = "<threw>" @@ -95,7 +99,15 @@ def _emit(variant: str, entries: list[GeneratorEntry], tmp: Path) -> dict[str, s def test_the_with_model_really_carries_the_vocabulary() -> None: # Else every comparison below is vacuously green. reports = sorted(o.name for o in _objects("with") if o.sub_type == OBJECT_SUBTYPE_REPORT) - assert reports == ["DailyRevenue", "ProgramEngagement", "StoreTotals"] + assert reports == ["DailyRevenue", "ProgramCatalogue", "ProgramEngagement", "StoreTotals"] + with_objects = {o.name: o for o in _objects("with")} + assert report_spine(with_objects["ProgramCatalogue"]) is not None + # ADR-0039: resolving children() — the measure is the event entity's own member. + defaulted = [ + m for o in with_objects.values() for m in o.children() + if m.type == TYPE_MEASURE and isinstance(m, MetaMeasure) and m.default_value() is not None + ] + assert [m.name for m in defaulted] == ["avgDaysPerStarter"] assert not any(o.sub_type == OBJECT_SUBTYPE_REPORT for o in _objects("without")) @@ -154,7 +166,7 @@ def test_api_docs_gain_only_the_served_report() -> None: expected = _api_docs("without") assert len(expected) > 3, f"only {len(expected)} pages — the docs barely ran" actual = _api_docs("with") - # One new unit page, for the served report; the two sourceless reports get none. + # One new unit page, for the served report; the three sourceless reports get none. assert sorted(set(actual) - set(expected)) == ["acme/shop/StoreTotals.md"] assert set(expected) <= set(actual) page = actual["acme/shop/StoreTotals.md"] @@ -186,7 +198,7 @@ def test_a_selection_of_only_unserved_reports_warns_that_there_is_nothing_to_gen GenConfig(out_dir=str(tmp_path / "out")), _load("with"), generators=[_build(e, tmp_path) for e in list_generators()], - entity_filter=["DailyRevenue", "ProgramEngagement"], + entity_filter=["DailyRevenue", "ProgramEngagement", "ProgramCatalogue"], ) assert result.files == [] assert any( @@ -199,7 +211,7 @@ def test_a_selection_of_only_reports_generates_only_the_served_one(tmp_path: Pat GenConfig(out_dir=str(tmp_path / "out")), _load("with"), generators=[_build(e, tmp_path) for e in list_generators()], - entity_filter=["DailyRevenue", "ProgramEngagement", "StoreTotals"], + entity_filter=["DailyRevenue", "ProgramEngagement", "ProgramCatalogue", "StoreTotals"], ) names = sorted(Path(path).name for path, _ in result.files) assert names == sorted( diff --git a/server/python/tests/unit/test_reporting_accessors.py b/server/python/tests/unit/test_reporting_accessors.py index 503932078..c8b7b646a 100644 --- a/server/python/tests/unit/test_reporting_accessors.py +++ b/server/python/tests/unit/test_reporting_accessors.py @@ -24,11 +24,14 @@ report_dimension_items, report_from, report_measure_names, + report_spine, ) from metaobjects.meta.core.reporting.reporting_constants import ( FILTER_RELATIVE_NOW, ISO_DURATION_RE, MEASURE_AGGS, + MEASURE_SUBTYPE_AGGREGATE, + REPORTING_ATTR_DEFAULT, TIME_GRAINS, ) from metaobjects.meta.meta_data import MetaData @@ -195,6 +198,58 @@ def test_report_accessors(vocab: MetaData) -> None: assert report_measure_names(totals) == ["purchases", "buyers", "revenue"] +def test_measure_default_value_is_the_declared_integer_else_none() -> None: + """``default_value()``: 0 and negatives included; ``None`` when undeclared. Resolving + (ADR-0039), so an inherited measure keeps it.""" + doc = copy.deepcopy(_fixture_doc("reporting-vocabulary")) + _member_body(doc, "Purchase", "measure.aggregate", "revenue")["@default"] = 0 + _body(doc, "Purchase")["children"].append( + {"measure.aggregate": {"name": "net", "@agg": "avg", "@of": "Purchase.amountCents", "@default": -1}} + ) + _member_body(doc, "WorkoutEvent", "measure.ratio", "avgDaysPerStarter")["@default"] = 0 + result = _load(doc) + assert result.errors == [], [e.message for e in result.errors] + + def measure(entity: str, name: str) -> MetaMeasure: + m = _child(_object(result.root, entity), TYPE_MEASURE, name) + assert isinstance(m, MetaMeasure) + return m + + assert measure("Purchase", "revenue").default_value() == 0 + assert measure("Purchase", "net").default_value() == -1 + assert measure("WorkoutEvent", "avgDaysPerStarter").default_value() == 0 + assert measure("WorkoutEvent", "avgDaysPerStarter").is_ratio() + assert measure("Purchase", "purchases").default_value() is None + + +@pytest.mark.parametrize("value", [True, False, 0.5, -1.5, "0", None]) +def test_measure_default_value_refuses_a_non_integer(value: object) -> None: + """``bool`` is a subclass of ``int`` in Python: ``True`` is not the integer 1 here.""" + measure = MetaMeasure(TYPE_MEASURE, MEASURE_SUBTYPE_AGGREGATE, "m") + if value is not None: + measure.set_attr(REPORTING_ATTR_DEFAULT, value) + assert measure.default_value() is None + + +def test_report_spine_is_the_declared_path_else_none(vocab: MetaData) -> None: + doc = copy.deepcopy(_fixture_doc("reporting-vocabulary")) + doc["metadata.root"]["children"].append( + { + "object.report": { + "name": "ProgramPurchases", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": ["programTitle"], + "@measures": ["purchases", "revenue"], + } + } + ) + result = _load(doc) + assert result.errors == [], [e.message for e in result.errors] + assert report_spine(_object(result.root, "ProgramPurchases")) == "Purchase.program" + assert report_spine(_object(vocab, "DailyRevenue")) is None + + def test_report_derived_field_name() -> None: assert report_derived_field_name(ReportDimensionItem("program")) == "program" assert report_derived_field_name(ReportDimensionItem("purchasedAt", "day")) == "purchasedAtDay" diff --git a/server/python/tests/unit/test_reporting_validation_spine_default.py b/server/python/tests/unit/test_reporting_validation_spine_default.py new file mode 100644 index 000000000..262037ed1 --- /dev/null +++ b/server/python/tests/unit/test_reporting_validation_spine_default.py @@ -0,0 +1,827 @@ +"""FR-044 — ``validate_reporting`` for a report's ``@spine`` (R8, R9) and a measure's +``@default`` (M7, M8): Table B of +docs/superpowers/plans/2026-10-09-fr-044-zero-rows-and-measure-defaults.md. + +A Python port of the "@spine (R8, R9)" and "measure @default (M7, M8)" blocks of the +TypeScript reference suite +``server/typescript/packages/metadata/test/reporting-validation.test.ts``. Every expected +message is the TypeScript text verbatim: the ports share one message contract, and the +shared conformance fixtures gate only the codes and the sources. + +The one exception is a fractional measure ``@default``. TypeScript refuses it in its +reporting pass because its generic attr.int check lets a fraction through; this port's +generic attr-type check already refuses it with ERR_BAD_ATTR_VALUE (its own text) on the +measure node, so no second, reporting-specific error is added. Those cases pin the code, +the node and that it is the only error. + +Every broken model is the clean spec §4 model (Purchase / Program / WorkoutEvent and three +reports) with exactly one change, and every test asserts the FULL error-code list: one +broken rule produces exactly one error, never a cascade. +""" +from __future__ import annotations + +import json +from typing import Any + +from metaobjects.errors import MetaError +from metaobjects.loader.meta_data_loader import MetaDataLoader + +Body = dict[str, Any] +Wrapper = dict[str, Body] +Model = dict[str, Any] + +_PKG = "acme::shop" +_MEMBER_TYPES = ("dimension", "measure", "segment") + + +# --------------------------------------------------------------------------- +# Model builders (the TypeScript suite's, rule for rule) +# --------------------------------------------------------------------------- + + +def _field(sub_type: str, name: str, **extra: Any) -> Wrapper: + return {f"field.{sub_type}": {"name": name, **extra}} + + +def _primary() -> Wrapper: + return {"identity.primary": {"name": "id", "@fields": ["id"]}} + + +def _full_reporting_model() -> Model: + return { + "metadata.root": { + "package": _PKG, + "children": [ + { + "object.entity": { + "name": "Program", + "children": [ + {"source.rdb": {"@table": "programs"}}, + _field("long", "id"), + _field("string", "title"), + _primary(), + { + "relationship.association": { + "name": "purchases", + "@objectRef": "Purchase", + "@cardinality": "many", + } + }, + ], + } + }, + { + "object.entity": { + "name": "Purchase", + "children": [ + {"source.rdb": {"@table": "purchases"}}, + _field("long", "id"), + _field("long", "programId"), + _field("string", "customerEmail"), + _field("currency", "amountCents"), + _field("string", "status"), + _field("boolean", "refunded"), + _field("timestamp", "purchasedAt"), + _field("date", "purchasedOn"), + _primary(), + { + "identity.reference": { + "name": "programRef", + "@references": "Program", + "@fields": ["programId"], + } + }, + { + "relationship.association": { + "name": "program", + "@objectRef": "Program", + "@cardinality": "one", + } + }, + {"dimension.attribute": {"name": "program", "@of": "Purchase.programId"}}, + { + "dimension.attribute": { + "name": "programTitle", + "@of": "Program.title", + "@via": "Purchase.program", + } + }, + { + "dimension.time": { + "name": "purchasedAt", + "@of": "Purchase.purchasedAt", + "@grains": ["day", "week", "month", "quarter", "year"], + } + }, + { + "measure.aggregate": { + "name": "purchases", + "@agg": "count", + "@of": "Purchase.id", + "@segment": "active", + } + }, + { + "measure.aggregate": { + "name": "buyers", + "@agg": "count", + "@distinct": True, + "@of": "Purchase.customerEmail", + "@segment": "active", + } + }, + { + "measure.aggregate": { + "name": "revenue", + "@agg": "sum", + "@of": "Purchase.amountCents", + "@segment": "active", + } + }, + {"segment.filter": {"name": "active", "@filter": {"status": "active"}}}, + ], + } + }, + { + "object.entity": { + "name": "WorkoutEvent", + "children": [ + {"source.rdb": {"@table": "workout_events"}}, + _field("long", "id"), + _field("long", "programId"), + _field("string", "customerEmail"), + _field("int", "weekNumber"), + _field("int", "dayNumber"), + _field("string", "eventType"), + _field("timestamp", "occurredAt"), + _primary(), + {"dimension.attribute": {"name": "program", "@of": "WorkoutEvent.programId"}}, + { + "measure.aggregate": { + "name": "starters", + "@agg": "count", + "@distinct": True, + "@of": "WorkoutEvent.customerEmail", + } + }, + { + "measure.aggregate": { + "name": "daysEngaged", + "@agg": "count", + "@distinct": True, + "@of": [ + "WorkoutEvent.programId", + "WorkoutEvent.customerEmail", + "WorkoutEvent.weekNumber", + "WorkoutEvent.dayNumber", + ], + } + }, + { + "measure.aggregate": { + "name": "lastActivityAt", + "@agg": "max", + "@of": "WorkoutEvent.occurredAt", + } + }, + { + "measure.ratio": { + "name": "avgDaysPerStarter", + "@numerator": "daysEngaged", + "@denominator": "starters", + } + }, + { + "segment.filter": { + "name": "completions", + "@filter": {"eventType": "exercise_complete"}, + } + }, + ], + } + }, + { + "object.report": { + "name": "ProgramEngagement", + "@from": "WorkoutEvent", + "@dimensions": ["program"], + "@measures": ["starters", "daysEngaged", "avgDaysPerStarter", "lastActivityAt"], + "@segment": "completions", + } + }, + { + "object.report": { + "name": "DailyRevenue", + "@from": "Purchase", + "@dimensions": ["purchasedAt:day"], + "@measures": ["purchases", "revenue"], + "@filter": {"purchasedAt": {"gte": {"now": "-P90D"}}}, + } + }, + { + "object.report": { + "name": "StoreTotals", + "@from": "Purchase", + "@measures": ["purchases", "buyers", "revenue"], + } + }, + ], + } + } + + +def _inherited_model() -> Model: + return { + "metadata.root": { + "package": _PKG, + "children": [ + { + "object.entity": { + "name": "BaseEvent", + "abstract": True, + "children": [ + _field("long", "id"), + _field("timestamp", "occurredAt"), + { + "dimension.time": { + "name": "occurredAt", + "@of": "BaseEvent.occurredAt", + "@grains": ["day", "week"], + } + }, + {"measure.aggregate": {"name": "events", "@agg": "count", "@of": "BaseEvent.id"}}, + ], + } + }, + { + "object.entity": { + "name": "WorkoutEvent", + "extends": "BaseEvent", + "children": [ + {"source.rdb": {"@table": "workout_events"}}, + _field("string", "eventType"), + _primary(), + ], + } + }, + { + "object.report": { + "name": "DailyEvents", + "@from": "WorkoutEvent", + "@dimensions": ["occurredAt:day"], + "@measures": ["events"], + } + }, + ], + } + } + + +def _object_body(m: Model, name: str) -> Body: + for w in m["metadata.root"]["children"]: + body: Body = next(iter(w.values())) + if body.get("name") == name: + return body + raise KeyError(f"no object {name}") + + +def _children_of(m: Model, obj_name: str) -> list[Wrapper]: + body = _object_body(m, obj_name) + kids: list[Wrapper] = body.setdefault("children", []) + return kids + + +def _wrapper_type(w: Wrapper) -> str: + return next(iter(w)).split(".")[0] + + +def _set_child(m: Model, obj_name: str, child_name: str, w: Wrapper) -> None: + """Replace the child of ``obj_name`` with the same base type as ``w`` and named + ``child_name``, or append ``w``.""" + kids = _children_of(m, obj_name) + for i, k in enumerate(kids): + if _wrapper_type(k) == _wrapper_type(w) and next(iter(k.values())).get("name") == child_name: + kids[i] = w + return + kids.append(w) + + +def _patch(body: Body, attrs: Body) -> None: + """Shallow-merge ``attrs``; a ``None`` value deletes the key.""" + for k, v in attrs.items(): + if v is None: + body.pop(k, None) + else: + body[k] = v + + +def _patch_object(m: Model, obj_name: str, attrs: Body) -> None: + _patch(_object_body(m, obj_name), attrs) + + +def _patch_member(m: Model, obj_name: str, child_name: str, attrs: Body) -> None: + for k in _children_of(m, obj_name): + if _wrapper_type(k) in _MEMBER_TYPES and next(iter(k.values())).get("name") == child_name: + _patch(next(iter(k.values())), attrs) + return + raise KeyError(f"no member {obj_name}.{child_name}") + + +def _edit(fn: Any) -> Model: + m = _full_reporting_model() + fn(m) + return m + + +def _errors(doc: Model) -> list[MetaError]: + return MetaDataLoader.from_string(json.dumps(doc), strict=True).errors + + +def _codes(doc: Model) -> list[str]: + return [e.code.value for e in _errors(doc)] + + +def _single(doc: Model, code: str) -> str: + errors = _errors(doc) + assert [e.code.value for e in errors] == [code], [e.message for e in errors] + return errors[0].message + + +def _json_path(error: MetaError) -> str: + path = getattr(error.envelope, "json_path", None) + assert isinstance(path, str), error + return path + + +# --------------------------------------------------------------------------- +# R8 / R9 — a report's @spine +# --------------------------------------------------------------------------- + + +def _add_spine_report(m: Model, attrs: Body | None = None) -> None: + """Append report ``ProgramPurchases`` over Purchase with ``@spine: Purchase.program``; + ``attrs`` override (``None`` deletes).""" + m["metadata.root"]["children"].append( + { + "object.report": { + "name": "ProgramPurchases", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": ["programTitle"], + "@measures": ["purchases", "revenue"], + } + } + ) + _patch_object(m, "ProgramPurchases", attrs or {}) + + +def _add_coach_chain(m: Model) -> None: + """Program -> Coach -> Agency, both to-one, plus Purchase dimensions at the end of each hop.""" + _children_of(m, "Program").extend( + [ + _field("long", "coachId"), + {"identity.reference": {"name": "coachRef", "@references": "Coach", "@fields": ["coachId"]}}, + {"relationship.association": {"name": "coach", "@objectRef": "Coach", "@cardinality": "one"}}, + ] + ) + m["metadata.root"]["children"].extend( + [ + { + "object.entity": { + "name": "Coach", + "children": [ + {"source.rdb": {"@table": "coaches"}}, + _field("long", "id"), + _field("string", "name"), + _field("long", "agencyId"), + _primary(), + {"identity.reference": {"name": "agencyRef", "@references": "Agency", "@fields": ["agencyId"]}}, + {"relationship.association": {"name": "agency", "@objectRef": "Agency", "@cardinality": "one"}}, + ], + } + }, + { + "object.entity": { + "name": "Agency", + "children": [ + {"source.rdb": {"@table": "agencies"}}, + _field("long", "id"), + _field("string", "name"), + _primary(), + ], + } + }, + ] + ) + _children_of(m, "Purchase").extend( + [ + {"dimension.attribute": {"name": "coachName", "@of": "Coach.name", "@via": "Purchase.program.coach"}}, + { + "dimension.attribute": { + "name": "agencyName", + "@of": "Agency.name", + "@via": "Purchase.program.coach.agency", + } + }, + ] + ) + + +def _program_title_by_ref() -> Wrapper: + """A dimension of Purchase reaching Program.title through the identity.reference, not the relationship.""" + return { + "dimension.attribute": {"name": "programTitleByRef", "@of": "Program.title", "@via": "Purchase.programRef"} + } + + +def test_a_spine_report_whose_dimensions_are_all_reached_through_the_spine_loads_clean() -> None: + assert _codes(_edit(_add_spine_report)) == [] + + +def test_r8_a_to_many_spine_hop_is_refused_naming_the_report_and_the_hops_entity() -> None: + def fn(x: Model) -> None: + _set_child( + x, "Program", "programs", {"measure.aggregate": {"name": "programs", "@agg": "count", "@of": "Program.id"}} + ) + x["metadata.root"]["children"].append( + { + "object.report": { + "name": "ProgramReach", + "@from": "Program", + "@spine": "Program.purchases", + "@measures": ["programs"], + } + } + ) + + assert _single(_edit(fn), "ERR_INVALID_REPORT") == ( + "report 'acme::shop::ProgramReach': @spine 'Program.purchases' crosses relationship 'purchases' on " + "'acme::shop::Program', which is not to-one. A @spine follows only @cardinality: one relationships " + "and identity.reference hops, so each fact row joins at most one row of the spine entity and is never " + "counted twice." + ) + + +def test_r8_a_spine_whose_owner_is_another_entity_is_refused() -> None: + m = _edit(lambda x: _add_spine_report(x, {"@spine": "Program.purchases"})) + assert _single(m, "ERR_INVALID_REPORT") == ( + "report 'acme::shop::ProgramPurchases': @spine 'Program.purchases' must start at @from 'acme::shop::Purchase'." + ) + + +def test_r8_a_spine_hop_that_names_nothing_is_refused_naming_from_as_the_hops_entity() -> None: + m = _edit(lambda x: _add_spine_report(x, {"@spine": "Purchase.nope"})) + assert _single(m, "ERR_INVALID_REPORT") == ( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase.nope' names 'nope', which is not a relationship " + "or identity.reference of 'acme::shop::Purchase'." + ) + + +def test_r8_a_spine_with_no_hop_is_refused() -> None: + m = _edit(lambda x: _add_spine_report(x, {"@spine": "Purchase"})) + assert _single(m, "ERR_INVALID_REPORT") == ( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase' must be Owner.hop[.hop...], starting at " + "@from 'acme::shop::Purchase'." + ) + + +def test_d2s_to_many_wording_is_unchanged_by_the_walks_wording_argument() -> None: + m = _edit( + lambda x: _set_child( + x, + "Program", + "buyerEmail", + { + "dimension.attribute": { + "name": "buyerEmail", + "@of": "Purchase.customerEmail", + "@via": "Program.purchases", + } + }, + ) + ) + assert _single(m, "ERR_INVALID_DIMENSION") == ( + "dimension 'buyerEmail' on entity 'acme::shop::Program': @via 'Program.purchases' crosses relationship " + "'purchases' on 'acme::shop::Program', which is not to-one. A dimension follows only @cardinality: one " + "relationships and identity.reference hops, so grouping can never multiply the measured rows." + ) + + +def test_d2s_own_wording_is_unchanged_by_the_walks_wording_argument() -> None: + def d2(via: str) -> str: + return _single( + _edit(lambda x: _patch_member(x, "Purchase", "programTitle", {"@via": via})), "ERR_INVALID_DIMENSION" + ) + + head = "dimension 'programTitle' on entity 'acme::shop::Purchase': " + assert d2("Purchase") == f"{head}@via 'Purchase' must be Owner.hop[.hop...], starting at the owning entity." + assert d2("WorkoutEvent.program") == ( + f"{head}@via 'WorkoutEvent.program' must start at the owning entity 'acme::shop::Purchase'." + ) + assert d2("Purchase.nope") == ( + f"{head}@via 'Purchase.nope' names 'nope', which is not a relationship or identity.reference of " + "'acme::shop::Purchase'." + ) + + +def test_r8_the_error_source_is_the_report_node() -> None: + errors = _errors(_edit(lambda x: _add_spine_report(x, {"@spine": "Purchase.nope"}))) + assert _json_path(errors[0]).endswith("['object.report']") + + +def test_r8_failing_skips_r9_a_broken_spine_with_an_off_spine_dimension_is_one_error() -> None: + m = _edit(lambda x: _add_spine_report(x, {"@spine": "Purchase.nope", "@dimensions": ["purchasedAt:day"]})) + assert "@spine 'Purchase.nope' names 'nope'" in _single(m, "ERR_INVALID_REPORT") + + +def test_r8_an_inherited_spine_owner_written_as_the_abstract_base_loads_clean() -> None: + m = _inherited_model() + m["metadata.root"]["children"].append( + { + "object.entity": { + "name": "Program", + "children": [ + {"source.rdb": {"@table": "programs"}}, + _field("long", "id"), + _field("string", "title"), + _primary(), + ], + } + } + ) + _children_of(m, "BaseEvent").extend( + [ + _field("long", "programId"), + {"identity.reference": {"name": "programRef", "@references": "Program", "@fields": ["programId"]}}, + {"relationship.association": {"name": "program", "@objectRef": "Program", "@cardinality": "one"}}, + {"dimension.attribute": {"name": "programTitle", "@of": "Program.title", "@via": "BaseEvent.program"}}, + ] + ) + m["metadata.root"]["children"].append( + { + "object.report": { + "name": "ProgramEvents", + "@from": "WorkoutEvent", + "@spine": "BaseEvent.program", + "@dimensions": ["programTitle"], + "@measures": ["events"], + } + } + ) + assert _codes(m) == [] + + +def test_r9_a_spine_report_with_no_dimensions_is_refused() -> None: + m = _edit(lambda x: _add_spine_report(x, {"@dimensions": None})) + assert _single(m, "ERR_INVALID_REPORT") == ( + "report 'acme::shop::ProgramPurchases': @spine 'Purchase.program' needs at least one dimension. The " + "report's rows are the dimension tuples of 'acme::shop::Program'; with no dimension it would be one " + "totals row." + ) + + +def test_r9_a_dimension_with_no_via_read_from_the_fact_row_is_refused() -> None: + m = _edit(lambda x: _add_spine_report(x, {"@dimensions": ["programTitle", "program"]})) + assert _single(m, "ERR_INVALID_REPORT") == ( + "report 'acme::shop::ProgramPurchases': dimension 'program' is read from @from 'acme::shop::Purchase', so " + "it has no value in a row that has no facts. With @spine 'Purchase.program' every dimension must be " + "reached through it: declare the dimension over a field of 'acme::shop::Program' (or an entity to-one " + "from it) with an @via that begins 'Purchase.program'." + ) + + +def test_r9_a_time_dimension_over_a_fact_column_is_refused() -> None: + m = _edit(lambda x: _add_spine_report(x, {"@dimensions": ["programTitle", "purchasedAt:day"]})) + assert "report 'acme::shop::ProgramPurchases': dimension 'purchasedAt' is read from @from" in _single( + m, "ERR_INVALID_REPORT" + ) + + +def test_r9_each_offending_dimension_is_reported_once_even_when_listed_at_two_grains() -> None: + m = _edit( + lambda x: _add_spine_report( + x, {"@dimensions": ["program", "purchasedAt:day", "purchasedAt:week", "programTitle"]} + ) + ) + errors = _errors(m) + assert [e.code.value for e in errors] == ["ERR_INVALID_REPORT", "ERR_INVALID_REPORT"] + assert "dimension 'program' is read from @from" in errors[0].message + assert "dimension 'purchasedAt' is read from @from" in errors[1].message + + +def test_r9_a_dimension_through_a_second_reference_to_the_same_entity_is_refused() -> None: + def fn(x: Model) -> None: + _children_of(x, "Purchase").extend( + [ + _field("long", "giftProgramId"), + { + "identity.reference": { + "name": "giftProgramRef", + "@references": "Program", + "@fields": ["giftProgramId"], + } + }, + _program_title_by_ref(), + { + "dimension.attribute": { + "name": "giftProgramTitle", + "@of": "Program.title", + "@via": "Purchase.giftProgramRef", + } + }, + ] + ) + _add_spine_report( + x, {"@spine": "Purchase.programRef", "@dimensions": ["programTitleByRef", "giftProgramTitle"]} + ) + + assert _single(_edit(fn), "ERR_INVALID_REPORT") == ( + "report 'acme::shop::ProgramPurchases': dimension 'giftProgramTitle' is reached by @via " + "'Purchase.giftProgramRef', which does not begin with the hops of @spine 'Purchase.programRef'. Hop names " + "are compared as written: if both name the same join, write the same hops; otherwise the dimension is " + "not reached through the spine." + ) + + +def test_r9_a_dimension_whose_own_via_fails_d2_is_reported_once_by_d2_not_also_by_r9() -> None: + def fn(x: Model) -> None: + _patch_member(x, "Purchase", "programTitle", {"@via": "Purchase.programm"}) + _add_spine_report(x) + + assert "@via 'Purchase.programm' names 'programm'" in _single(_edit(fn), "ERR_INVALID_DIMENSION") + + +def test_r9_the_same_join_named_by_the_relationship_in_spine_and_the_reference_in_via_is_refused() -> None: + def fn(x: Model) -> None: + _children_of(x, "Purchase").append(_program_title_by_ref()) + _add_spine_report(x, {"@spine": "Purchase.program", "@dimensions": ["programTitleByRef"]}) + + msg = _single(_edit(fn), "ERR_INVALID_REPORT") + assert "dimension 'programTitleByRef' is reached by @via 'Purchase.programRef'" in msg + assert "@spine 'Purchase.program'" in msg + + +def test_r9_the_owner_segment_is_not_compared() -> None: + """An FQN spine owner, a bare dimension @via.""" + assert _codes(_edit(lambda x: _add_spine_report(x, {"@spine": "acme::shop::Purchase.program"}))) == [] + + +def test_r9_a_time_dimension_over_a_column_of_the_spine_entity_is_legal() -> None: + def fn(x: Model) -> None: + _children_of(x, "Program").append(_field("timestamp", "publishedAt")) + _children_of(x, "Purchase").append( + { + "dimension.time": { + "name": "programPublishedAt", + "@of": "Program.publishedAt", + "@via": "Purchase.program", + "@grains": ["month"], + } + } + ) + _add_spine_report(x, {"@dimensions": ["programTitle", "programPublishedAt:month"]}) + + assert _codes(_edit(fn)) == [] + + +def test_r9_a_two_hop_spine_with_a_dimension_at_it_and_one_beyond_it_is_legal() -> None: + def fn(x: Model) -> None: + _add_coach_chain(x) + _add_spine_report(x, {"@spine": "Purchase.program.coach", "@dimensions": ["coachName", "agencyName"]}) + + assert _codes(_edit(fn)) == [] + + +def test_r9_a_dimension_that_stops_short_of_a_two_hop_spine_is_refused() -> None: + def fn(x: Model) -> None: + _add_coach_chain(x) + _add_spine_report(x, {"@spine": "Purchase.program.coach", "@dimensions": ["coachName", "programTitle"]}) + + assert "dimension 'programTitle' is reached by @via 'Purchase.program'" in _single(_edit(fn), "ERR_INVALID_REPORT") + + +# --------------------------------------------------------------------------- +# M7 / M8 — where a measure's @default can apply +# --------------------------------------------------------------------------- + + +def test_m7_default_on_a_count_is_refused_on_the_measure_node() -> None: + errors = _errors(_edit(lambda x: _patch_member(x, "Purchase", "purchases", {"@default": 0}))) + assert [e.code.value for e in errors] == ["ERR_INVALID_MEASURE"] + assert errors[0].message == ( + "measure 'purchases' on entity 'acme::shop::Purchase': @default cannot apply to @agg: count. A count is " + "never null (it is 0 when nothing matches); remove @default." + ) + assert "['measure.aggregate']" in _json_path(errors[0]) + + +def test_m7_default_on_a_distinct_count_of_a_tuple_is_refused() -> None: + msg = _single( + _edit(lambda x: _patch_member(x, "WorkoutEvent", "daysEngaged", {"@default": 0})), "ERR_INVALID_MEASURE" + ) + assert "measure 'daysEngaged'" in msg + assert "@default cannot apply to @agg: count" in msg + + +def test_m8_default_on_a_max_of_a_timestamp_is_refused() -> None: + msg = _single( + _edit(lambda x: _patch_member(x, "WorkoutEvent", "lastActivityAt", {"@default": 0})), "ERR_INVALID_MEASURE" + ) + assert msg == ( + "measure 'lastActivityAt' on entity 'acme::shop::WorkoutEvent': @default is a number, but " + "'WorkoutEvent.occurredAt', the @of of @agg 'max', is a field.timestamp. A default is supported on " + "numeric measures only." + ) + + +def test_a_non_integer_default_on_a_count_reports_m7_and_the_attribute_type_error() -> None: + errors = _errors(_edit(lambda x: _patch_member(x, "Purchase", "purchases", {"@default": "zero"}))) + assert sorted(e.code.value for e in errors) == ["ERR_BAD_ATTR_VALUE", "ERR_INVALID_MEASURE"] + m7 = next(e for e in errors if e.code.value == "ERR_INVALID_MEASURE") + assert "measure 'purchases' on entity 'acme::shop::Purchase': @default cannot apply to @agg: count." in m7.message + + +def test_a_boolean_default_on_a_count_reports_m7_and_the_attribute_type_error() -> None: + """``bool`` is a subclass of ``int`` in Python, so ``true`` must not pass for the + integer 1: it is the attribute type error, not a fraction, so M7 still reads the + attribute as present (the "zero" case above, for this port's bool trap).""" + errors = _errors(_edit(lambda x: _patch_member(x, "Purchase", "purchases", {"@default": True}))) + assert sorted(e.code.value for e in errors) == ["ERR_BAD_ATTR_VALUE", "ERR_INVALID_MEASURE"] + + +def _fractional(entity: str, measure: str, value: float) -> MetaError: + """The ONE error a fractional @default yields: this port's generic attr.int check.""" + errors = _errors(_edit(lambda x: _patch_member(x, entity, measure, {"@default": value}))) + assert [e.code.value for e in errors] == ["ERR_BAD_ATTR_VALUE"], [e.message for e in errors] + assert "'@default'" in errors[0].message + return errors[0] + + +def test_a_fractional_default_on_a_sum_is_err_bad_attr_value_on_the_measure_node_once() -> None: + assert "['measure.aggregate']" in _json_path(_fractional("Purchase", "revenue", 0.5)) + + +def test_a_fractional_default_on_a_ratio_is_err_bad_attr_value_on_the_ratio() -> None: + assert "['measure.ratio']" in _json_path(_fractional("WorkoutEvent", "avgDaysPerStarter", 0.5)) + + +def test_the_integer_check_runs_first_a_fractional_default_on_a_count_skips_m7() -> None: + _fractional("Purchase", "purchases", 0.5) + + +def test_the_integer_check_runs_first_a_fractional_default_on_a_max_of_a_timestamp_skips_m8() -> None: + _fractional("WorkoutEvent", "lastActivityAt", -1.5) + + +def test_a_fractional_default_on_a_sum_declared_on_an_abstract_base_is_reported_once() -> None: + m = _inherited_model() + _children_of(m, "BaseEvent")[3] = { + "measure.aggregate": {"name": "events", "@agg": "sum", "@of": "BaseEvent.id", "@default": 0.5} + } + assert "'@default'" in _single(m, "ERR_BAD_ATTR_VALUE") + + +def test_default_on_a_sum_an_avg_a_min_of_an_int_and_a_ratio_is_fine() -> None: + def fn(x: Model) -> None: + _patch_member(x, "Purchase", "revenue", {"@default": 0}) + _set_child( + x, + "Purchase", + "avgRevenue", + {"measure.aggregate": {"name": "avgRevenue", "@agg": "avg", "@of": "Purchase.amountCents", "@default": 0}}, + ) + _set_child( + x, + "WorkoutEvent", + "firstDay", + {"measure.aggregate": {"name": "firstDay", "@agg": "min", "@of": "WorkoutEvent.dayNumber", "@default": 1}}, + ) + _patch_member(x, "WorkoutEvent", "avgDaysPerStarter", {"@default": 0}) + + assert _codes(_edit(fn)) == [] + + +def test_a_measure_that_breaks_m4_and_declares_default_reports_m4_only() -> None: + msg = _single( + _edit(lambda x: _patch_member(x, "Purchase", "revenue", {"@of": "Purchase.status", "@default": 0})), + "ERR_INVALID_MEASURE", + ) + assert "field.string" in msg + assert "@default" not in msg + + +def test_a_measure_that_breaks_m1_and_declares_default_on_a_count_reports_m1_only() -> None: + msg = _single( + _edit(lambda x: _patch_member(x, "Purchase", "purchases", {"@of": "Purchase.nope", "@default": 0})), + "ERR_INVALID_MEASURE", + ) + assert "names no field 'nope'" in msg + assert "@default" not in msg + + +def test_m7_on_a_count_declared_on_an_abstract_base_is_reported_once() -> None: + m = _inherited_model() + _children_of(m, "BaseEvent")[3] = { + "measure.aggregate": {"name": "events", "@agg": "count", "@of": "BaseEvent.id", "@default": 0} + } + assert "measure 'events' on entity 'acme::shop::BaseEvent'" in _single(m, "ERR_INVALID_MEASURE") From 9e47468ebc06214c11fd939a386831ff66154a6b Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 22:47:42 -0400 Subject: [PATCH 16/18] docs(reporting): @spine and measure @default --- .claude/rules/cross-language-porting.md | 2 +- CHANGELOG.md | 43 +++ .../skills/metaobjects-authoring/SKILL.md | 16 +- .../references/reporting.md | 69 ++++- .../references/typescript-mysql.md | 5 +- docs/CONFORMANCE.md | 19 +- docs/features/api-contract.md | 12 +- docs/features/reporting.md | 251 ++++++++++++++++-- docs/ports/csharp.md | 6 +- docs/ports/java.md | 5 +- docs/ports/kotlin.md | 5 +- docs/ports/python.md | 3 + docs/ports/typescript.md | 5 +- docs/recipes/mysql.md | 5 +- .../skills/metaobjects-authoring/SKILL.md | 18 +- .../references/reporting.md | 69 ++++- .../skills/metaobjects-authoring/SKILL.md | 18 +- .../references/reporting.md | 69 ++++- .../skills/metaobjects-authoring/SKILL.md | 18 +- .../references/reporting.md | 69 ++++- .../skills/metaobjects-authoring/SKILL.md | 18 +- .../references/reporting.md | 69 ++++- .../references/typescript-mysql.md | 5 +- .../skills/metaobjects-authoring/SKILL.md | 18 +- .../references/reporting.md | 69 ++++- .../references/typescript-mysql.md | 5 +- fixtures/api-contract-conformance/README.md | 6 +- 27 files changed, 802 insertions(+), 95 deletions(-) diff --git a/.claude/rules/cross-language-porting.md b/.claude/rules/cross-language-porting.md index 8a7206904..dd3cb0ffc 100644 --- a/.claude/rules/cross-language-porting.md +++ b/.claude/rules/cross-language-porting.md @@ -17,7 +17,7 @@ Preserve the following contracts exactly across all language ports: **Metamodel subtype vocabularies (must be identical across languages):** the `registry-conformance` gate (`fixtures/registry-conformance/`) is the structural enforcer of this rule — each port emits its registry as a canonical manifest byte-matched to `expected-registry.json`. **All five ports (TS / C# / Java / Kotlin / Python) are live + green** (SP-G Java/Kotlin reconciliation complete; the JVM runners compose from the defined metamodel provider set so codegen-base/om classpath SPI does not pollute the measured vocabulary). See `fixtures/registry-conformance/README.md`. - Filter operators: `eq`, `ne`, `gt`, `gte`, `lt`, `lte`, `in`, `like`, `isNull` -- Object subtypes: `entity` (owns data: own identity, writable sources, lifecycle), `value` (pure shape: NO identity, NO source, ever; constructed — by caller/embedding — never populated; may `extends` entity fields for shape; a value-hosted field may carry `origin.passthrough` but never an assembly origin), `projection` (derived read-only representation: fields `extends`-bound / origin-derived / self-declared-under-external-assembly, all read-only at subtype level; identity optional and MUST extend an entity identity; sources restricted to read-only `@kind`s; the declared field set IS the exposure — inclusive list, fail-closed). A field carrying `origin.*` is derived ⇒ read-only wherever it lives (incl. on entities). An entity's primary source must be a writable `@kind` (read-only kinds only in read role). See [ADR-0028](spec/decisions/ADR-0028-object-taxonomy-projection-value-purity.md). (FR-024 Phase E — `object.projection`/`value` are registered in `expected-registry.json` and the projection/value validation passes [identity pass-through, value-purity, projection-licensing, `@via` inference/cardinality, extends/origin agreement, derived-field providability] are enforced cross-port in all 5 ports. The **B4b** entity-primary-source-readonly cutover [the "writable `@kind`" clause above — `ERR_ENTITY_PRIMARY_SOURCE_READONLY`] + the projection codegen fan-out (read-only DTOs for view-kind projections; FR-015 proc-callables for proc-kind projections in TypeScript, C# and Kotlin ONLY — Java and Python ship no callable generator at all, so the cross-port claim does NOT cover that clause; api-docs label `object.projection` units as `projection` and document their generated `<Name>Dto`) are now shipped cross-port; the remaining FR-024 work is the declared-API surface — tracked in #10.) **`report` (FR-044 Plan 1)** is a root object subtype registered in all five ports, with the `dimension.attribute` / `dimension.time` / `measure.aggregate` / `measure.ratio` / `segment.filter` children on `object.entity` and the relative-date filter value — loader-validated (`ERR_INVALID_DIMENSION` / `ERR_INVALID_MEASURE` / `ERR_INVALID_REPORT` / `ERR_REPORT_FOREIGN_MEASURE`, plus `ERR_BAD_ATTR_FILTER` for relative dates off a reporting host) and, since FR-044 Plan 2, **lowered only when it declares a read-only `source.rdb` of `@kind: view`**: `meta migrate` creates that view (TypeScript only, ADR-0015), every port reads it (persistence corpus, `report-shapes.json`), C# generates its typed row and Kotlin its Exposed table object, and, since FR-044 Plan 3, every port's generators **serve it as a keyless read-only projection** (GET list with filter/sort/paging on every derived field that has a filter band, `POST` 405, no `/{id}`; row type + filter allowlist + route/controller per port, gated by the api-contract `report/` sub-corpus in the generated lane). The client UI tier (hooks, grids, forms) and a report with no view source at all stay inert, gated by the 27 `reporting` conformance fixtures and the `codegen-noop` corpus. A served report is the predicate `servedReport` (TS) / `ReportRows.IsViewBacked` (C#) / `RestSurfaceGate.isServedReport` (Java, Kotlin) / `is_served_report` (Python): concrete, read source `@kind: view`. See [docs/features/reporting.md](docs/features/reporting.md). +- Object subtypes: `entity` (owns data: own identity, writable sources, lifecycle), `value` (pure shape: NO identity, NO source, ever; constructed — by caller/embedding — never populated; may `extends` entity fields for shape; a value-hosted field may carry `origin.passthrough` but never an assembly origin), `projection` (derived read-only representation: fields `extends`-bound / origin-derived / self-declared-under-external-assembly, all read-only at subtype level; identity optional and MUST extend an entity identity; sources restricted to read-only `@kind`s; the declared field set IS the exposure — inclusive list, fail-closed). A field carrying `origin.*` is derived ⇒ read-only wherever it lives (incl. on entities). An entity's primary source must be a writable `@kind` (read-only kinds only in read role). See [ADR-0028](spec/decisions/ADR-0028-object-taxonomy-projection-value-purity.md). (FR-024 Phase E — `object.projection`/`value` are registered in `expected-registry.json` and the projection/value validation passes [identity pass-through, value-purity, projection-licensing, `@via` inference/cardinality, extends/origin agreement, derived-field providability] are enforced cross-port in all 5 ports. The **B4b** entity-primary-source-readonly cutover [the "writable `@kind`" clause above — `ERR_ENTITY_PRIMARY_SOURCE_READONLY`] + the projection codegen fan-out (read-only DTOs for view-kind projections; FR-015 proc-callables for proc-kind projections in TypeScript, C# and Kotlin ONLY — Java and Python ship no callable generator at all, so the cross-port claim does NOT cover that clause; api-docs label `object.projection` units as `projection` and document their generated `<Name>Dto`) are now shipped cross-port; the remaining FR-024 work is the declared-API surface — tracked in #10.) **`report` (FR-044 Plan 1)** is a root object subtype registered in all five ports, with the `dimension.attribute` / `dimension.time` / `measure.aggregate` / `measure.ratio` / `segment.filter` children on `object.entity` and the relative-date filter value — loader-validated (`ERR_INVALID_DIMENSION` / `ERR_INVALID_MEASURE` / `ERR_INVALID_REPORT` / `ERR_REPORT_FOREIGN_MEASURE`, plus `ERR_BAD_ATTR_FILTER` for relative dates off a reporting host) and, since FR-044 Plan 2, **lowered only when it declares a read-only `source.rdb` of `@kind: view`**: `meta migrate` creates that view (TypeScript only, ADR-0015), every port reads it (persistence corpus, `report-shapes.json`), C# generates its typed row and Kotlin its Exposed table object, and, since FR-044 Plan 3, every port's generators **serve it as a keyless read-only projection** (GET list with filter/sort/paging on every derived field that has a filter band, `POST` 405, no `/{id}`; row type + filter allowlist + route/controller per port, gated by the api-contract `report/` sub-corpus in the generated lane). The client UI tier (hooks, grids, forms) and a report with no view source at all stay inert, gated by the 37 `reporting` conformance fixtures and the `codegen-noop` corpus. A served report is the predicate `servedReport` (TS) / `ReportRows.IsViewBacked` (C#) / `RestSurfaceGate.isServedReport` (Java, Kotlin) / `is_served_report` (Python): concrete, read source `@kind: view`. See [docs/features/reporting.md](docs/features/reporting.md). - Source subtypes: `rdb` (paradigm; ADR-0007). The pre-v2 `dbTable`/`dbView` subtypes are RETIRED — `source.rdb` + `@kind: table|view|materializedView|storedProc|tableFunction` is the form, with read-only-ness derived from `@kind`. Multi-source via `@role` (exactly one `primary` per object). Source physical name = `@table` (NOT `@name`); field physical name = `@column` (renamed from `@dbColumn`). Referential actions on relationships: `@onDelete` / `@onUpdate`. - Origin subtypes: `passthrough`, `aggregate`, `collection`, `computed`, `first` (concrete; `base` is the abstract root). `passthrough` is legal on an `object.value`-hosted field (FR-015 parameter lineage); the four assembly origins (`aggregate`/`computed`/`collection`/`first`) live on `object.projection` only — a value-hosted assembly origin is `ERR_SUBTYPE_RULE_VIOLATION` (#210). - Relationship subtypes: `association`, `aggregation`, `composition`. Cardinality via `@cardinality: one|many`; target via `@objectRef`. **M:N (FR-018) slim vocabulary:** `@cardinality: "many"` + `@objectRef` (target) + `@through` (the junction/through entity — a third entity that MUST declare two `identity.reference` children, one per FK side). The relationship's FK fields are **derived** from those references (the `identity.reference` SSOT for FK direction), never restated. `@sourceRefField` (optional) disambiguates a *directed* self-join by naming the source-side FK field on the junction (the other reference is the target side); on a `@cardinality: one` relationship it instead names which of several `identity.reference` nodes onto the same target this relationship navigates, short-circuiting the unique-candidate/`@sourceRefField`/name-pairing ladder (#368, [ADR-0029](spec/decisions/ADR-0029-entity-child-extends-and-via-inference.md) Amendment 1) — an unresolvable 1:N reference set is `ERR_INVALID_RELATIONSHIP` at load. `@symmetric` (optional boolean) marks an *undirected* self-join (union-on-read) — valid only when `@objectRef` == the declaring entity, and mutually exclusive with `@sourceRefField`. The pre-FR-018 `@joinEntity`/`@joinFields` attrs are REMOVED. Validation errors: symmetric-on-hetero / symmetric+sourceRefField → `ERR_BAD_ATTR_VALUE`; junction-missing-two-references / sourceRefField-not-matching / M:N-attr-on-1:N / **junction-unpairable** → `ERR_INVALID_RELATIONSHIP`. **Unpairable means the junction declares its two references but neither resolves to the navigating entity, or neither to the `@objectRef` target** — declaring two references is NOT enough, and the loader now checks WHAT they point at (owner ruling 2026-09-20). It does so by running the real FK derivation and converting its failure, never a parallel re-implementation, so the loader and the derivation cannot drift; scope mirrors codegen's own iteration exactly — every CONCRETE, non-projection object crossed with its EFFECTIVE relationships, NOT deduped by declaration, because pairing is a property of the navigating entity and an inherited M:N can pair from one subtype and not another. Before the ruling this loaded clean and then diverged: TS and C# warned and emitted no traversal route (a silent 404), while Java, Kotlin and Python failed the build. Gated by `fixtures/conformance/error-relationship-m2m-junction-unpairable/`. diff --git a/CHANGELOG.md b/CHANGELOG.md index 807ec81ce..5658b8895 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -227,6 +227,49 @@ it until 1.1 ships._ else breaks: an owned `grid`, `grid-hook` or `form` copy needs no change, and a hand-written generator that gates on `servesReadApi` receives a served report's read model and emits its list hook, which is what a hook generator wants; gate a grid or form on `servesClientTier`. +- **Rows with no facts (`@spine`) and a default for an empty measure (`@default`), in all + five ports (FR-044).** Two optional attributes join the unreleased 1.1 vocabulary, so + `metamodelVersion` stays `1.1`. `@spine` on `object.report` is a to-one path from `@from`, + written like `@via` (`"Purchase.program"`): the report's rows then come from the entity at + its end, so a program nobody bought still has a row, with a `count` of `0`. `@default` on + `measure.aggregate` and `measure.ratio` is an integer the measure reads when it would be null + (nothing matched, every value null, or a ratio's denominator zero or null), and its column is + then never null in the view or in any port's row type. **A model that uses neither generates + exactly as before**, in every port. Merging this rules on four things that would be costly to + change after 1.1.0: (1) the grain does not change: a report is still one row per distinct + dimension tuple, now among the spine entity's rows, so list the spine entity's key to get one + row per spine row; (2) `@segment` and `@filter` on a `@spine` report scope the facts and never + remove a spine row (they sit in the join condition; there is no `WHERE`); (3) a fact row whose + reference is null or matches nothing is in no row of a `@spine` report (without `@spine` such + rows still form a null group); (4) a ratio's operand carries its own `@default` into the ratio, + so `revenue` defaulted to `0` over `buyers` reads `0`, not null, and the ratio's own `@default` + then covers a zero denominator. New load rules, gated by 10 new shared conformance fixtures + (37 for reporting, 374 in all): a `@spine` that is not a to-one path from `@from`, a `@spine` + report with no dimension, and a dimension not reached through the spine (hop names compared as + written) are `ERR_INVALID_REPORT`; a `@default` on a `count`, or on a `min`/`max` over a field + that is not numeric, is `ERR_INVALID_MEASURE`. A fractional `@default` (`0.5`) is + `ERR_BAD_ATTR_VALUE` on the measure in every port: TypeScript says "@default '0.5' is not an + integer. A measure's @default is a whole number (for example 0).", while C#, Java and Python + refuse it through their existing integer-attribute check with that check's message. No port + changed its general `attr.int` check, so a fraction in a validator's `@min` or `@max` loads + where it did. Two edge cases have no fixture: `@default: 0.0` (or `1.0`) is read as an integer + by TypeScript and refused by C#, Java and Python, and `@default: "zero"` on a `count` gives two + errors in TypeScript, C# and Python and one in Java. Write a whole number with no decimal + point. TypeScript lowers it: the view selects `FROM` the spine entity's table with `LEFT OUTER + JOIN`s back to `@from`, and a defaulted measure is `COALESCE(…, n)` (SQLite writes `n.0` for a + `decimal`, `double` or `float` measure and for a ratio, so the column keeps one storage class; + MySQL reads a defaulted ratio of zero as `0.0000`). `meta migrate` refuses a derived `@spine` + report, by name, when the spine entity or an entity on the way has no table, is a TPH subtype, + or a hop has no `identity.reference`. In `meta docs` a defaulted measure's definition ends + "; `n` when there is nothing to aggregate" (a defaulted ratio no longer says "null when the + denominator is 0"), and a `@spine` report's page gains a **Rows** line and a row scope that + reads "aggregating only …". Gated by three new persistence scenarios (42 in all, nine for + reports, through every port's runtime) and three new REST scenarios in the `report/` + sub-corpus (16; the api-contract corpus is 82), run in every port's generated lane and on + SQLite in TypeScript. **No generator an adopter can own changed** (the SQL and the `meta + docs` wording come from the packages, and each port's existing generators already type a + required field as not nullable), so an owned generator needs no resync for this. See + [docs/features/reporting.md](docs/features/reporting.md#rows-from-a-dimensions-entity). ### Changed diff --git a/agent-context/skills/metaobjects-authoring/SKILL.md b/agent-context/skills/metaobjects-authoring/SKILL.md index 1f0c1b64a..304bae183 100644 --- a/agent-context/skills/metaobjects-authoring/SKILL.md +++ b/agent-context/skills/metaobjects-authoring/SKILL.md @@ -29,7 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | -| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, rows with no facts (`@spine`), defaults (`@default`), engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -740,6 +740,20 @@ Three rules an author trips on: measure (a time dimension at a grain is `<name><Grain>`, so `purchasedAt:day` is `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. +Two questions to answer for every report: + +- **Do I want the rows that have no facts?** A report's rows come from the facts, so a + program nobody bought has no row. If the page should list it, add `"@spine": + "Purchase.program"` to the report (a to-one path from `@from`, written like `@via`), and + **reach every listed dimension through it**: each needs an `@via` that begins with the + spine's hops (`programId` as `Program.id` via `Purchase.program`). A dimension on the fact + row is a load error under `@spine`. List the spine entity's key to get one row per program. +- **Is zero true here, or is null?** A `sum`, `avg`, `min`, `max` or ratio with nothing to + aggregate is null. If zero is the true answer (revenue of a program nobody bought), declare + `"@default": 0` on the measure; if the value is unknown (an average rating with no + ratings), leave it null. **Never on a `count`**: it is already `0`, and `@default` there is a + load error. Write `0`, never `0.0`. + **A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration is what makes `meta migrate` create the view (Postgres, SQLite, D1), what every port's runtime reads, and what makes every port's generators serve it: one read-only list route at diff --git a/agent-context/skills/metaobjects-authoring/references/reporting.md b/agent-context/skills/metaobjects-authoring/references/reporting.md index be0bc0003..47c54d6d8 100644 --- a/agent-context/skills/metaobjects-authoring/references/reporting.md +++ b/agent-context/skills/metaobjects-authoring/references/reporting.md @@ -1,6 +1,6 @@ # Reporting: the columns a report gets, and how it behaves -> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, rows with no facts (`@spine`), defaults for empty measures (`@default`), what differs between databases, and what it leaves out. ## A report is served only with a view source @@ -34,8 +34,10 @@ A report declares no fields. Its columns are one per `@dimensions` item in liste | `avg` of `double` / `float` | the measure's name | `double` | no | | `min` / `max` | the measure's name | the `@of` field's type | no | | `measure.ratio` | the measure's name | `decimal` | no | +| any measure above except `count`, declaring `@default` | the measure's name | as above | yes: it reads the default instead of null | +| in a `@spine` report, a dimension over a column of the spine entity | the dimension's name | the `@of` field's type | yes when the field is `@required` or in that entity's `identity.primary`; no for a dimension beyond the spine entity | -A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. +Only those last two rows make a measure other than `count`, or a dimension reached by `@via`, never null; every port's row type follows (no `| null`, no `?`, no `Optional`). A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no field `@default`, no validators. ## Time grains @@ -50,13 +52,64 @@ A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 dur ## Nulls and zeros - A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. -- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. So are `avg`, `min` and `max`. - A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. - A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). +**Ask: is zero true here, or is null?** If "nothing to aggregate" means zero (revenue of a program nobody bought is `0`), declare `"@default": 0` on the measure (`revenue` in the example under "Rows with no facts"). If it means unknown (the average rating of a product with no ratings is not `0`), leave it null. Do not default in the client what the model can state once. + +- `@default` is an integer. Write `0`, not `0.0`: a fraction (`0.5`) is refused in every port, and `0.0` loads in TypeScript but is refused by C#, Java and Python. +- **Never on a `count`** (it is never null, so the default could never apply) and never on a `min` / `max` over a field that is not numeric (a date or a string): both are load errors. +- It applies whenever the measure would be null: nothing matched, every matched value was null, or a ratio's denominator is zero or null. The column is then never null in any port. +- **A ratio's operand carries its own `@default` into the ratio.** `revenue` defaulted to `0` over `buyers` reads `0` for a group with buyers and no revenue. The ratio's own `@default` covers a zero denominator. +- A filter on a defaulted measure sees the default: `?filter[revenue][eq]=0` matches the empty rows, `isNull=true` matches nothing. A sort on it is the same on every engine. + +## Rows with no facts: `@spine` + +A report's rows are the dimension tuples its `@from` rows have, so a per-program report has no row for a program nobody bought. **Ask: do I want the rows that have no facts?** If the page lists every program (a catalogue, a chart whose empty bars matter), add `@spine`: a to-one path from `@from`, written like `@via`, to the entity whose rows the report should have. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Program", "children": [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } ] } }, + { "object.entity": { "name": "Purchase", "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "programId" } }, + { "field.currency": { "name": "amountCents" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }, + { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } } ] } }, + { "object.report": { "name": "ProgramSales", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle"], "@measures": ["purchases", "revenue"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_sales" } } ] } } + ] +}} +``` + +A program with no purchases has a row: `purchases` `0`, `revenue` `0`. + +- **Every dimension goes through the spine.** Each listed dimension needs an `@via` that begins with the spine's hops (`programId` is `Program.id` via `Purchase.program`). A dimension on the fact row (`Purchase.programId` with no `@via`, the purchase month) is a load error: a program with no purchases has no value for it. +- **Write the same hops as the spine.** `Purchase.program` (relationship) and `Purchase.fkProgram` (reference) are the same join, but a spine written one way and a dimension the other is refused. +- List at least one dimension, and list the spine entity's key (`Program.id`) to get exactly one row per program; two programs listed only by a shared title are one row. +- On a row with no facts, a `count` is `0` and every other measure is null unless it declares `@default`. +- The report's `@segment` and `@filter` still scope the facts and never remove a program's row. They name `@from`'s fields only; to show only some programs, list the column as a dimension and filter it on the request. +- A fact whose reference is null, or matches nothing, is in no row of a `@spine` report. +- Every hop needs an `identity.reference`, and the spine entity (and every entity on the way) needs a table of its own: an abstract entity or a TPH subtype fails `meta migrate`. +- The empty rows are listed and counted on the route; `?filter[purchases][gt]=0` hides them on request, so one report serves both pages. + ## Joins: a dimension through a required reference drops rows -A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. In a `@spine` report every join is `LEFT OUTER`. **Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. @@ -67,6 +120,7 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t | View created by | `meta migrate` | `meta migrate` | you: see below | | A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | | `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| A ratio reading its `@default: 0` | `0` | `0.0` (the default is written `0.0` so the `REAL` column keeps one storage class) | `0.0000` | | Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | **MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. @@ -77,6 +131,11 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t - **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. - **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, is not served by any port and gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). - **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Java, Kotlin and C# `gen`; only TypeScript and Python read and serve it (as parsed JSON, neither filterable nor sortable). Group by a scalar field. +- **One `@spine` per report.** Two independent spines (every program against every customer) cannot be written. Entities to-one beyond the spine are ordinary dimensions. +- **No row scope on the spine entity** (only published programs): a report `@filter` names `@from`'s fields. List the column as a dimension and filter it on the request. +- **No fact-row dimension beside a spine** (every program, split by purchase month), and no dimension over the reference column itself (`Purchase.programId`). Declare it over the spine entity's key instead. Zero-filled time buckets are not available yet. +- **No join on a natural key, no anti-join, no report over two fact tables.** A `@spine` reaches its facts through a declared reference only: give the fact a reference to the spine entity. +- **`@default` is a whole number.** A fraction is `ERR_BAD_ATTR_VALUE` on the measure in every port (TypeScript's message says it is not an integer; C#, Java and Python use their integer-attribute message). `0.0` or `1.0` loads in TypeScript and is refused elsewhere, and `"zero"` on a `count` is two errors in TypeScript, C# and Python but one in Java. Write `0`. ## How a report is served @@ -90,7 +149,7 @@ A concrete report whose read source is `@kind: view` is served by every port's g - `<segment>` is the report name, snake_cased then pluralized, like any object: `InvoicesByMonth` is `/invoices_by_months`, `StoreTotals` is `/store_totals`. There is no report-specific spelling. - **Every derived field with filter operators is filterable and sortable**, dimension and measure alike, with its type's own operators. You cannot narrow that set: a report has no field to put `@filterable` on. A field of the `@from` entity that the report does not expose is refused with a `400`. -- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present; `currency` is integer minor units. +- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present (a measure with `@default` is never null); `currency` is integer minor units. - **A decimal (`avg`, a ratio, a `sum` of a decimal) has no agreed JSON spelling**: each port sends its own (TypeScript a string). Do not compare one across backends as text. - `meta docs` writes a model page for every report (a sourceless one is marked "Not served") and an API page for a served one. diff --git a/agent-context/skills/metaobjects-codegen/references/typescript-mysql.md b/agent-context/skills/metaobjects-codegen/references/typescript-mysql.md index eb54c6be2..8392d6051 100644 --- a/agent-context/skills/metaobjects-codegen/references/typescript-mysql.md +++ b/agent-context/skills/metaobjects-codegen/references/typescript-mysql.md @@ -94,11 +94,14 @@ migration); the loop will not. Pass `columnNamingStrategy` to match your tables' column names (the default is `snake_case`). The bodies are valid under MySQL's default `sql_mode`, `ONLY_FULL_GROUP_BY` included, and a change to a report means a new `CREATE OR REPLACE VIEW` (or `DROP` and `CREATE`) in your -migrations; nothing diffs the live view for you. Two things differ from Postgres and SQLite: +migrations; nothing diffs the live view for you. A `@spine` report's body is valid as +emitted too, and a measure with `@default` is a `COALESCE` whose column MySQL reports +`NOT NULL`. Two things differ from Postgres and SQLite: - **Ratios and averages have four fractional digits by default.** MySQL divides to `div_precision_increment` digits, so a ratio of 2 to 3 is `0.6667` (Postgres returns `0.66666666666666666667`, SQLite `0.6666666666666666`), and a ratio of 3 to 4 is `0.7500`. + A ratio that reads its `@default: 0` is `0.0000`. - **`DATETIME` values are read as the UTC wall clock.** A `DATETIME(3)` column carries no zone, so every time grain and every relative-date window (`{ "now": "-P30D" }`, evaluated with `UTC_TIMESTAMP(3)` when the view is queried) treats the stored value as UTC. That matches diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 5cd772b91..41dfec3fa 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -33,8 +33,8 @@ regenerate with `ls -d fixtures/<corpus>/*/ | wc -l` for directory-shaped corpor | [`fixtures/render-conformance/`](../fixtures/render-conformance/) | 15 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 48 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | -| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 39 (33 query + 6 migration) | all 39 | 33 query (migrations TS-only, ADR-0015) | 33 query (via Exposed) | 33 query | 33 query | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 78 (31 core + 10 tph + 9 m2m + 2 jsonb + 2 write-through + 11 projection + 13 report) | ✓ (Fastify reference + generated lane) | ✓ (embedded HTTP + JDBC) | ✓ (embedded HTTP + Exposed) | ✓ (HttpListener + Npgsql) | ✓ (FastAPI + pg8000) | +| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 42 (36 query + 6 migration) | all 42 | 36 query (migrations TS-only, ADR-0015) | 36 query (via Exposed) | 36 query | 36 query | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 82 (31 core + 10 tph + 9 m2m + 2 jsonb + 2 write-through + 12 projection + 16 report) | ✓ (Fastify reference + generated lane) | ✓ (embedded HTTP + JDBC) | ✓ (embedded HTTP + Exposed) | ✓ (HttpListener + Npgsql) | ✓ (FastAPI + pg8000) | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | | [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/object-model-conformance/`](../fixtures/object-model-conformance/) | 1 shared metadata fixture (per-port scenarios) | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -269,14 +269,14 @@ trailing-newline preservation, and unicode multibyte handling. All 31 fixtures → [features/migrations-and-drift.md](features/migrations-and-drift.md) (template drift section — `Renderer.verify`). -### `fixtures/persistence-conformance/` (39 — 33 query + 6 migration) +### `fixtures/persistence-conformance/` (42 — 36 query + 6 migration) - `migrations/*` (6) → [features/migrations-and-drift.md](features/migrations-and-drift.md) (schema migration section) -- `queries/*` (33) → [features/source-kinds.md](features/source-kinds.md) (query semantics against `source.rdb`) +- `queries/*` (36) → [features/source-kinds.md](features/source-kinds.md) (query semantics against `source.rdb`) -### `fixtures/api-contract-conformance/` (78) +### `fixtures/api-contract-conformance/` (82) -All 78 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port +All 82 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port REST API URL grammar + JSON wire format). Verifies every backend's emitted CRUD routes answer identically over HTTP — list / get / create / patch+put / delete, plus pagination (`limit`/`offset`), sort (`sort=field:dir`), the `withCount=1` @@ -294,9 +294,10 @@ each corpus alone could not reach, and 1 pinning the collection-URL spelling), `write-through/` (2, table-write + view-read entities), `projection/` (12, a read-only view answers reads and refuses writes with 405; a projection with no declared identity has no item route; a projection keyed on a field not named `id` is addressed by -it, whether its identity names `@fields` or derives them; decimal and float fields filter) and `report/` (13, +it, whether its identity names `@fields` or derives them; decimal and float fields filter) and `report/` (16, FR-044: a view-backed `object.report` is listed, filtered, sorted and paged on -its derived fields, answers `POST` with 405 and mounts no `/{id}`). All 5 ports — TS, Java, +its derived fields, answers `POST` with 405 and mounts no `/{id}`; a `@spine` report keeps +a row with no facts, and a measure with `@default` filters and sorts as its default). All 5 ports — TS, Java, Kotlin, C#, Python — run it in BOTH lanes: a hand-rolled reference server and the port's own GENERATED API artifact booted over HTTP. `write-through/`, `projection/` and `report/` run the generated lane only, on all five ports: what @@ -460,7 +461,7 @@ Kotlin's identity function is the Java one; the names its generator emits are as ## Orphaned fixtures (tested but not yet documented) The fixtures in the nine corpora mapped above (metamodel 374 + yaml 16 + verify 31 -+ render 15 + persistence 39 + api-contract 78 + source-resolution 25 + scope 10 + ++ render 15 + persistence 42 + api-contract 82 + source-resolution 25 + scope 10 + dependency 23) each map to a feature doc, and so do the two requirement corpora, whose case lists live in their own READMEs. None are orphaned today. The remaining corpora in the totals table gate tooling contracts (registry manifests, provider diff --git a/docs/features/api-contract.md b/docs/features/api-contract.md index bb10d037a..5e1caedd2 100644 --- a/docs/features/api-contract.md +++ b/docs/features/api-contract.md @@ -470,11 +470,17 @@ report with no source, an abstract one, and one over a `materializedView`, `stor Every derived field whose type has filter operators is filterable and sortable, dimension and measure alike; the allowlists are the report's own derived fields, never the `@from` -entity's. No request parameter picks dimensions, measures or a grain. No port generates a -client hook, grid or form for a report yet. +entity's. No request parameter picks dimensions, measures or a grain. TypeScript generates a +list hook for a report; no port generates a grid or form for one. + +A report with `@spine` lists the rows that have no facts too (a count of `0` there), and +`withCount=1` counts them; `?filter[<count>][gt]=0` removes them on request. A measure with +`@default` reads its default on such a row and is never `null`: `?filter[<m>][eq]=0` matches +the empty rows, `isNull=true` matches nothing, and a sort on it orders the same on every +engine. Gated by [`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/) -(13 scenarios), **generated lane only, on all five ports**, for the reason `projection/` +(16 scenarios), **generated lane only, on all five ports**, for the reason `projection/` gives. The columns, their types and the per-port generated files are in [reporting.md](reporting.md#how-a-report-is-served). diff --git a/docs/features/reporting.md b/docs/features/reporting.md index ab4455eff..567262129 100644 --- a/docs/features/reporting.md +++ b/docs/features/reporting.md @@ -16,8 +16,9 @@ the standard filter, sort and paging on the derived fields. stays inert: it is a checked statement of intent. No generator, migration or runtime acts on it, and the one thing written about it is a `meta docs` model page marked "not served". -**What does not exist yet.** No typed client hook, grid, form or other UI-tier output is -generated for a report in any port (a later plan of FR-044). There is no `measure.derived`, +**What does not exist yet.** No grid, form or other UI-tier output is generated for a report +in any port; TypeScript's list hook is the one client piece (a later plan of FR-044 adds the +rest). There is no `measure.derived`, no query-time choice of dimensions or measures (a report is a fixed combination, compiled once), and no time-zone vocabulary: time grains and relative dates are UTC. @@ -47,10 +48,10 @@ FR-037 R5's arithmetic wave and will need its own agreement): |---|---|---| | `dimension.attribute` | `object.entity` | `@of` (required), `@via` | | `dimension.time` | `object.entity` | `@of` (required), `@via`, `@grains` (required, array) | -| `measure.aggregate` | `object.entity` | `@agg` (required), `@of` (required, array), `@distinct`, `@filter`, `@segment` | -| `measure.ratio` | `object.entity` | `@numerator` (required), `@denominator` (required) | +| `measure.aggregate` | `object.entity` | `@agg` (required), `@of` (required, array), `@distinct`, `@filter`, `@segment`, `@default` | +| `measure.ratio` | `object.entity` | `@numerator` (required), `@denominator` (required), `@default` | | `segment.filter` | `object.entity` | `@filter` (required) | -| `object.report` | root | `@from` (required), `@dimensions` (array), `@measures` (required, array), `@segment`, `@filter` | +| `object.report` | root | `@from` (required), `@dimensions` (array), `@measures` (required, array), `@segment`, `@filter`, `@spine` | Closed sets: @@ -58,6 +59,8 @@ Closed sets: in every lowering. - `@agg`: `count, sum, avg, min, max`. +`@spine` is a path written like a dimension's `@via`. A measure's `@default` is an integer. + Plus one new value form inside the existing portable filter grammar: the **relative date** `{ "now": "<ISO-8601 duration>" }` (see [Relative dates](#relative-dates-in-filters)). @@ -96,6 +99,8 @@ A report is a top-level object that names an entity as its `@from`: "@cardinality": "one" } }, { "segment.filter": { "name": "active", "@filter": { "status": "active" } } }, { "dimension.attribute": { "name": "program", "@of": "Purchase.programId" } }, + { "dimension.attribute": { "name": "programId", + "@of": "Program.id", "@via": "Purchase.program" } }, { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, { "dimension.time": { "name": "purchasedAt", "@of": "Purchase.purchasedAt", @@ -105,7 +110,7 @@ A report is a top-level object that names an entity as its `@from`: { "measure.aggregate": { "name": "buyers", "@agg": "count", "@distinct": true, "@of": "Purchase.customerEmail" } }, { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", - "@segment": "active" } }, + "@segment": "active", "@default": 0 } }, { "measure.ratio": { "name": "revenuePerBuyer", "@numerator": "revenue", "@denominator": "buyers" } } ] @@ -124,6 +129,16 @@ A report is a top-level object that names an entity as its `@from`: "children": [ { "source.rdb": { "@kind": "view", "@view": "v_store_totals" } } ] + }}, + { "object.report": { + "name": "ProgramSales", + "@from": "Purchase", + "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle"], + "@measures": ["purchases", "revenue", "revenuePerBuyer"], + "children": [ + { "source.rdb": { "@kind": "view", "@view": "v_program_sales" } } + ] }} ] }} @@ -144,14 +159,20 @@ What each piece means: aggregates its own entity's rows and a dimension reaches only to-one paths, so no join inflates it. `@filter` and `@segment` scope the rows and combine by AND. - **`measure.ratio`** is `numerator / NULLIF(denominator, 0)`, typed decimal. A zero - denominator yields null. Both operands are `measure.aggregate` siblings (a ratio of ratios - is not supported). + denominator yields null, unless the ratio declares `@default`. Both operands are + `measure.aggregate` siblings (a ratio of ratios is not supported). - **`segment.filter`** is a named, reusable `attr.filter`. Use it when the same row rule ("active purchase") would otherwise be repeated across measures and reports. An inline `@filter` on a measure stays legal. - **`object.report`** is a fixed combination of dimensions and measures of ONE entity. One row per distinct dimension tuple; **no dimensions means exactly one row** (the totals case). Its `@filter` and `@segment` scope rows before grouping and combine by AND. +- **`@spine`** on a report takes its rows from the entity at the end of a to-one path, so + `ProgramSales` has a row for a program nobody bought. See + [Rows from a dimension's entity](#rows-from-a-dimensions-entity). +- **`@default`** on a measure is the integer it reads when it would be null: `revenue` reads + `0`, not null, for a group with no active purchase. See + [A default for an empty measure](#a-default-for-an-empty-measure). ### A report's fields are derived, not declared @@ -169,9 +190,9 @@ colon, so it cannot collide with the `::` package separator). A `@measures` item name, or `Entity.name` where `Entity` is the `@from` entity or one it extends; both forms name the same measure and derive the same field. -`StoreTotals` above declares the source that makes it **served**; `DailyRevenue` declares -none, so it is checked at load and nothing more. A report is served only when it declares a -`source.rdb` with `@kind: view` (the next section says what that does). +`StoreTotals` and `ProgramSales` above declare the source that makes them **served**; +`DailyRevenue` declares none, so it is checked at load and nothing more. A report is served +only when it declares a `source.rdb` with `@kind: view` (the next section says what that does). ## What a report lowers to @@ -206,6 +227,9 @@ A derived report view is refused in three more cases, each with an error naming `identity.reference` it needs. - **a filter's `in` list is empty**, which no database accepts as SQL. +A `@spine` report has three refusals of its own, listed under +[Rows from a dimension's entity](#rows-from-a-dimensions-entity). + ### The columns you get A report has no primary key and declares no fields; its read shape is derived. One column @@ -227,11 +251,18 @@ physical column name is your naming strategy applied to the **derived field name | `avg` of `double` or `float` | the measure's name | `double` | no | | `min` / `max` | the measure's name | the `@of` field's type | no | | `measure.ratio` | the measure's name | `decimal` | no | +| any measure above except `count`, declaring `@default` | the measure's name | as above | yes: it reads its `@default` instead of null | +| in a `@spine` report, a dimension over a column of the spine entity | the dimension's name | the `@of` field's type | yes when the `@of` field is `@required` or one of the spine entity's `identity.primary` `@fields`; no for a dimension reached beyond the spine entity | + +The last two rows are the only ways a measure other than `count`, or a dimension reached +through `@via`, is never null. The key clause applies only under `@spine`, where the key is +the row's own; without `@spine` a dimension reached through `@via` stays nullable. A derived column carries the type-shaping attributes of its `@of` field where they apply (`@currency`, `@values`, `@intValueMap`, `@maxLength`, `@precision`, `@scale`, `@localTime`, `@objectRef`, `@storage`, `@dbColumnType`, `isArray`) and nothing else: no `@column`, no -`@required` beyond the rule above, no `@default`, no validators. +`@required` beyond the rules above, no `@default` (a measure's `@default` is applied in the +view, not carried), no validators. ### Measures @@ -243,16 +274,17 @@ applied. The aggregates: items) counts distinct tuples, and a tuple with any null component is not counted, on every engine. - **`sum`** of nothing is **null**, not zero: a report with no matching rows, or a filtered - measure that matched none of a group's rows, shows null. A `sum` of an integer type is a + measure that matched none of a group's rows, shows null, unless the measure declares + [`@default`](#a-default-for-an-empty-measure). A `sum` of an integer type is a 64-bit integer on every engine (Postgres casts it to `BIGINT`, MySQL to `SIGNED`, and SQLite's integer `SUM` already is one). - **`avg`, `min`, `max`** are the engine's own. -- **`measure.ratio`** is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**, - never an error. Each operand is repeated inline with its own conditions, so an operand need - not be listed in `@measures`. +- **`measure.ratio`** is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null** + (or the ratio's `@default`), never an error. Each operand is repeated inline with its own + conditions and its own `@default`, so an operand need not be listed in `@measures`. A report with no dimensions is one row over the whole table. Over an **empty** table that row -still exists: counts are `0`, sums and ratios are null. +still exists: counts are `0`, sums and ratios are null, or their `@default`. ### Dimensions, time grains and joins @@ -263,7 +295,7 @@ still exists: counts are `0`, sums and ratios are null. `LEFT OUTER`, and an `INNER` survives only when every join above it is `INNER`. The consequence to know: **a dimension reached through a required reference drops a fact row whose reference matches no row, from that report.** A dimension that is not listed in - `@dimensions` adds no join. + `@dimensions` adds no join. In a `@spine` report every join is `LEFT OUTER`. - **Grains** are `hour, day, week, month, quarter, year`. A bucket is the first instant (for `hour`) or first day (for the rest) of the period. **Weeks start on Monday (ISO-8601)** on every engine: the week of Sunday 2026-05-17 starts 2026-05-11, and Monday 2026-06-01 starts @@ -272,10 +304,112 @@ still exists: counts are `0`, sums and ratios are null. time zone is, so two readers get the same buckets. A `field.timestamp` with `@localTime` and a `field.date` are bucketed as stored. There is no vocabulary for another zone. - `GROUP BY` is every listed dimension, in `@dimensions` order. The report's `@segment` and - `@filter` are the `WHERE`: rows are scoped before grouping, and there is no `HAVING`. + `@filter` are the `WHERE`: rows are scoped before grouping, and there is no `HAVING`. In a + `@spine` report they sit in a join condition instead, and there is no `WHERE` (next + section). - **Relative dates** in a view are evaluated when the view is **queried**, against the UTC clock. A naive (`@localTime`) timestamp is compared with the UTC wall clock. +### Rows from a dimension's entity + +A report's rows are the dimension tuples its `@from` rows have, so a per-program report has +no row for a program nobody bought. `@spine` fixes that. It is a to-one path from `@from`, +written like `@via` (`"Purchase.program"`), and the entity at its end, the **spine entity**, +supplies the rows: one row per distinct dimension tuple among the spine entity's rows, +including the ones no row of `@from` refers to. `ProgramSales` above has a row for every +program. + +- **Every listed dimension is reached through the spine.** Its `@via` begins with the spine's + hops, so it is a column of the spine entity or of an entity to-one from it. A dimension read + from the fact row (no `@via`, or a `@via` through another reference) is refused at load: it + has no value in a row with no facts. A time dimension follows the same rule: the month a + program was published is legal, the month of a purchase is not. +- **Hop names are compared as written.** A spine written with the relationship's name + (`Purchase.program`) and a dimension written with the reference's name (`Purchase.fkProgram`) + name the same join, and are still refused. Write the same hops in both. +- **At least one dimension.** With none, the report would be one totals row. +- **The grain does not change.** A row is still one distinct dimension tuple. To get exactly one + row per spine row, list a dimension over the spine entity's key (`programId` above). List only + the title, and two programs with the same title are one row. +- **`@segment` and `@filter` scope the facts, never the spine rows.** They choose which rows of + `@from` are aggregated. A program whose purchases are all out of scope keeps its row, with a + count of `0`. Both still name fields of `@from`. +- **A fact row with no spine row is in no row.** A purchase whose `programId` is null, or + matches no program, has no spine row to sit in. Without `@spine`, those rows form a null + group. +- **A row with no facts** reads `0` for a `count`, and null for a `sum`, `avg`, `min`, `max` + and a ratio, unless the measure declares `@default` (next section). This holds even for a + measure whose `@filter` is `isNull: true` on a fact column: the empty row's `@of` column is + null, and an aggregate ignores a null. + +**The view.** The spine entity's table is the `FROM`. The spine is walked back to `@from` with +one `LEFT OUTER JOIN` per hop, and the report's `@segment` and `@filter` are ANDed onto the `ON` +of the join that brings in `@from`'s table. There is no `WHERE`: a `WHERE` on a fact column +would turn the outer join back into an inner one, and the empty rows would vanish with no error. +Every other join of a `@spine` report is `LEFT OUTER` too, so no join can drop a spine row. +`ProgramSales` lowers on Postgres (`literal` column naming) to: + +```sql + SELECT + p0."id" AS "programId", + p0."title" AS "programTitle", + COUNT(p."id") FILTER (WHERE p."status" = 'active') AS "purchases", + COALESCE(CAST(SUM(p."amountCents") FILTER (WHERE p."status" = 'active') AS BIGINT), 0) AS "revenue", + CAST(COALESCE(CAST(SUM(p."amountCents") FILTER (WHERE p."status" = 'active') AS BIGINT), 0) AS NUMERIC) / NULLIF(COUNT(DISTINCT p."customerEmail"), 0) AS "revenuePerBuyer" + FROM "programs" p0 + LEFT OUTER JOIN "purchases" p ON p0."id" = p."programId" + GROUP BY p0."id", p0."title" +``` + +With `"@segment": "active"` on the report, the join would read +`LEFT OUTER JOIN "purchases" p ON p0."id" = p."programId" AND p."status" = 'active'`. The +aliases are the ones the same report would get without `@spine`. + +`meta migrate` refuses a derived `@spine` report, naming the report and the spine, when: + +- the spine entity, or an entity on the way to it, has no table (it is abstract, or declares + no writable `source.rdb`); +- one of them is a TPH subtype: it shares its base's table with every other subtype, so the + report would have a row for each row of all of them; +- a hop has no `identity.reference` behind it (the same error as for a `@via` hop). + +**Serving it.** The empty rows are listed, and `withCount=1` counts them. +`?filter[purchases][gt]=0` removes them at request time, so one report serves the page that +shows them and the page that does not. To keep only some spine rows (published programs), +list the column as a dimension and filter it on the request: a report `@filter` names +`@from`'s fields only. + +### A default for an empty measure + +`"@default": 0` on a `measure.aggregate` or a `measure.ratio` is the value the measure reads +when it would otherwise be null: nothing matched (an empty table, a `@spine` row with no facts, +a measure `@filter` or `@segment` that matched none of a group's rows), every matched value was +null, or, for a ratio, the denominator is zero or null. The column is then never null, in the +database and in every port's row type. + +Declare it only where zero is true. A program with no purchases has revenue `0`. An average +rating with no ratings is not `0`; null is the honest answer there. + +- **It is an integer.** Every case found is zero; a sentinel such as `-1` is legal too. Write + a whole number with no decimal point (see [Known limits](#known-limits)). +- **Where it is legal.** A `sum`, an `avg`, a `min` or `max` over a numeric field, and a ratio. + It is refused on a `count`, which is never null, and on a `min` or `max` over a field that is + not numeric: a made-up date would be a wrong answer. +- **A ratio's operand carries its own `@default` into the ratio.** `revenue` declares + `@default: 0`, so `revenuePerBuyer` reads `0`, not null, for a program with buyers and no + active revenue. The ratio's own `@default` then covers a zero or null denominator: without + one, `revenuePerBuyer` is still null for a program with no buyers. +- **In SQL it is `COALESCE(<the measure's expression>, n)`**, as `revenue` shows above. The + column keeps its type: a defaulted integral `sum` is still a 64-bit integer, a defaulted + `avg` or ratio still a decimal. SQLite writes the value as `n.0` for a `decimal`, `double` + or `float` measure and for a ratio, so a `REAL` column has one storage class in every row. + +**Filter and sort.** The default is the value. `?filter[revenue][eq]=0` matches the empty +rows, `isNull=true` matches nothing and `isNull=false` every row. A sort on a defaulted +measure is the same on every engine, since the empty rows sort as their default. A measure +without one sorts its null rows where the engine puts them: Postgres last ascending, SQLite +and MySQL first. + ### What the runtime does with it | Port | Read side | @@ -367,7 +501,12 @@ not core: copy and own them with your port's `eject`. **`meta docs`.** Every report gets a model page (kind `report`, its `@from`, its view or "Not served" with the reason, its row scope, and a column table with a definition per -column), listed under `## Reports` on the model index. An entity that declares dimensions, +column), listed under `## Reports` on the model index. A measure with `@default` ends its +definition with "; `0` when there is nothing to aggregate", and a defaulted ratio's +definition drops "null when the denominator is 0". A `@spine` report's page gains a **Rows** +line ("one row per distinct dimension tuple among the rows of `Program`, reached by +`Purchase.program`, including those no `Purchase` refers to"), and its row scope reads +"aggregating only …", since the scope never removes a row. An entity that declares dimensions, measures or segments, or that a report names as its `@from`, gains a "Reporting" section. A served report gets one API page: its row model, `GET <served path>` and the list query (in a port with no query function, the repository seam). A report that is not served gets no API @@ -382,6 +521,8 @@ builder documents a served report the same way. | Created by | `meta migrate` | `meta migrate` | you (see below) | | `avg` and ratio of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | | `decimal` | `NUMERIC` | none: SQLite has no decimal, so `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| A ratio with `@default: 0` and nothing to aggregate | `0` | `0.0`: the default is written `0.0` (also for a defaulted `decimal`, `double` or `float` measure), so the `REAL` column has one storage class in every row | `0.0000` | +| A defaulted column's type | unchanged (`BIGINT`, `NUMERIC`) | unchanged | unchanged (`BIGINT`, `DECIMAL`), and reported `NOT NULL` | | Instants and dates | `TIMESTAMPTZ`, `DATE` | ISO-8601 text (an hour bucket is `...:00:00.000Z`) | `DATETIME(3)` read as the UTC wall clock | A changed report is dropped and re-created by `meta migrate` (it does not `CREATE OR @@ -393,7 +534,8 @@ REPLACE`, since the diff does not know the old column list). `buildReportViews(root, { dialect: "mysql" })` from `@metaobjectsdev/codegen-ts` returns the body of each view-backed report; the recipe in [`docs/recipes/mysql.md`](../recipes/mysql.md) ("Reports") shows the loop and its caveats. It skips a report whose source is `@unmanaged`, and -the bodies are valid under MySQL's default `ONLY_FULL_GROUP_BY`. +the bodies are valid under MySQL's default `ONLY_FULL_GROUP_BY`, a `@spine` report's as +emitted. ### Known limits @@ -439,25 +581,58 @@ the bodies are valid under MySQL's default `ONLY_FULL_GROUP_BY`. subtype that declares the field you mean. Without `@via` the field is read from the `@from` entity itself. +Each of the next five is refused at load today, so admitting it later is additive: + +- **One spine per report.** Two independent spines (every program against every customer) + would be a cross join the size of both tables multiplied. Entities to-one beyond the spine + are ordinary dimensions, which covers most charts (a workout's week, the week's program). +- **No row scope on the spine entity** (only published programs). A report `@filter` names + `@from`'s fields. List the column as a dimension and filter it on the request. +- **No fact-row dimension beside a spine** (every program, split by purchase month). The row + for a program with no purchases would carry a null month. Zero-filled time buckets wait for + the `reporting` library's calendar spine. +- **No dimension over the reference column itself** (`Purchase.programId`) in a `@spine` + report. Declare the dimension over the spine entity's key (`Program.id` via + `Purchase.program`) instead. +- **`@default` is an integer, and a whole number is the only portable spelling.** A + fractional `@default` (`0.5`) is refused with `ERR_BAD_ATTR_VALUE` on the measure in every + port. TypeScript says "@default '0.5' is not an integer. A measure's @default is a whole + number (for example 0)."; C#, Java and Python refuse it through their existing integer + attribute check, with that check's own message. Two edge cases have no fixture and differ + by port: `@default: 0.0` (or `1.0`) loads as an integer in TypeScript and is refused by C#, + Java and Python; and `@default: "zero"` on a `count` gives two errors in TypeScript, C# and + Python (the attribute type error and the count rule) and one in Java. Write `0`, not `0.0`. + +A `@spine` row's facts are reached through a declared reference only. A join on a natural key +(events matched to a schedule on several columns), an anti-join ("bought, never started") and +a report over two fact tables stay out: give the fact a reference to the spine entity, write +the anti-join by hand, and make two reports. + ### What the corpus gates -Six shared scenarios under `fixtures/persistence-conformance/queries/report-*.yaml` read the +Nine shared scenarios under `fixtures/persistence-conformance/queries/report-*.yaml` read the canonical reports through every port's runtime (list and count, filter, sort, an empty table, -the Monday boundary, an hour bucket, a relative window). The derived columns are pinned by +the Monday boundary, an hour bucket, a relative window, a `@spine` report with a program that +has no weeks, a `@spine` report whose segment leaves a program no facts, and defaulted measures +over an empty table). The derived columns are pinned by `fixtures/persistence-conformance/report-shapes.json`, produced by TypeScript and byte-matched by every port. The SQL is produced by TypeScript only, so the other ports read the view the -TypeScript migrate engine produced and never lower a report themselves. +TypeScript migrate engine produced and never lower a report themselves. TypeScript's own value +tests read the `@spine` and `@default` views and their column types on Postgres, SQLite and +MySQL, and on Postgres and SQLite also a fact whose reference is null, an `isNull` condition +and a two-hop spine. -The REST surface is gated by thirteen scenarios under +The REST surface is gated by sixteen scenarios under [`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/), run in the **generated lane on all five ports**: list (a dimension with a segment-scoped sum that is null for one group), a time dimension at a grain (`YYYY-MM-DD`), a no-dimension totals report with the `withCount` envelope, a filter on a dimension and on a measure, a sort on a measure and an enum dimension, paging over groups, the three field-naming `400` envelopes, -`405` on `POST`, and `404` on every verb at `/{id}`. The corpus model carries one sourceless -report, so a port that serves every report it finds fails. No scenario asserts a decimal's +`405` on `POST`, `404` on every verb at `/{id}`, and a `@spine` report whose product with no +sale keeps its row, with a filter and a sort on its defaulted measure. The corpus model +carries one sourceless report, so a port that serves every report it finds fails. No scenario asserts a decimal's spelling or a timestamp literal. TypeScript and C# run the scenarios against the real views on -Postgres, and TypeScript runs the same thirteen against SQLite as well +Postgres, and TypeScript runs the same sixteen against SQLite as well (`api-contract-report-sqlite.test.ts`), where it also holds that a ratio reaches the wire as a string, as it does on Postgres; Java, Kotlin and Python serve seeded rows behind their repository seam, and a TypeScript test holds those rows equal to what the views return. @@ -496,9 +671,15 @@ row, and a `SUM` over the result would double-count. A to-one hop cannot fan out | **M4** | `sum` or `avg` over a non-numeric field (numeric is `int, long, double, float, decimal, currency`); `min` or `max` over a `boolean`, `object` or `map` field. | | **M5** | An `@segment` that names no `segment` child of the owning entity. | | **M6** | A `measure.ratio` whose `@numerator` or `@denominator` names no measure of the entity, or names one that is not a `measure.aggregate`. | +| **M7** | A `@default` on an `@agg: count`. A count is never null, so the default could never apply. | +| **M8** | A `@default` on an `@agg: min` or `max` over a field that is not numeric (M4's set). | A measure that breaks several of M1 to M4 reports only the first, in that order, so one -mistake gives one error. +mistake gives one error. M7 and M8 run only when none of M1 to M4 fired. + +**A `@default` must be a whole number** (`ERR_BAD_ATTR_VALUE`, on the measure). A fraction is +refused in every port, and then M7 and M8 are not checked. A ratio's `@default` is checked for +this only. The message and two edge cases differ by port: see [Known limits](#known-limits). ### Segments and filters (`ERR_BAD_ATTR_FILTER`) @@ -517,10 +698,15 @@ mistake gives one error. | **R5** | `ERR_INVALID_REPORT` | A report `source.*` that is writable. A report's source, if present, must have a read-only `@kind` (`view`, `materializedView`, `storedProc`, `tableFunction`); a writable `@kind: table` is refused. | | **R6** | `ERR_INVALID_REPORT` | Two derived field names that collide: a dimension `revenue` with a measure `revenue`, or dimension `purchasedAt` at grain `day` with a measure named `purchasedAtDay`. A measure listed twice gets the same code with its own message ("lists 'x' more than once"). A repeated dimension item is reported under R2, not R6. | | **R7** | `ERR_INVALID_REPORT` | A report `@segment` that names no segment of `@from`. | +| **R8** | `ERR_INVALID_REPORT` | A `@spine` that is not `Owner.hop[.hop...]` with `Owner` resolving to `@from` or an entity it extends (in the report's package), or that crosses a hop which is not a `relationship.*` with `@cardinality: one` or an `identity.reference`, or a hop whose target resolves to no object. This is D2's walk, started at `@from`. | +| **R9** | `ERR_INVALID_REPORT` | A `@spine` report with no dimension, or a listed dimension whose `@via` does not begin with the spine's hops (no `@via` at all included). Hop names are compared as written; the owner segment is not. | `purchasedAt:day` and `purchasedAt:week` together are legal, since they derive different fields (`purchasedAtDay`, `purchasedAtWeek`). +R8 and R9 run only when `@from` resolved. R9 is skipped when R8 failed, reports each offending +dimension once, and leaves a dimension whose own `@via` fails D2 to D2. + **Why every measure comes from `@from`.** Combining measures of two fact tables in one report is the "chasm trap" that every surveyed BI tool has bugs around: join the two tables and each side's rows multiply the other's. v1 avoids it entirely. Two fact tables are two @@ -565,6 +751,13 @@ where the same change fixed Java and Python parsing bugs, each toward what TypeS did; the [CHANGELOG](../../CHANGELOG.md) lists them and the models they affect. Until 1.1 ships, `main` carries `metamodelVersion` 1.1, so no 1.0.x PATCH is cut from it. +`@spine` and a measure's `@default` joined the same 1.1 change before it shipped, so +`metamodelVersion` stays 1.1. A model that uses neither generates exactly as before. No +generator you can own changed for them (the view SQL and the `meta docs` wording come from the +packages), so an owned generator needs no resync. A report that declares +`@spine`, or lists a measure with `@default`, lowers to different SQL and a tighter row type +than the same report without it. + Serving reports added no vocabulary (`metamodelVersion` stays 1.1). The same change corrected generated output for models that declare no report. Each is in the [CHANGELOG](../../CHANGELOG.md) with the shape it affects: diff --git a/docs/ports/csharp.md b/docs/ports/csharp.md index 787977abc..ae5ed8d54 100644 --- a/docs/ports/csharp.md +++ b/docs/ports/csharp.md @@ -113,8 +113,10 @@ The codegen emits: (`MapGet` list, `MapPost` answering 405, no `{id}` route) and `<Report>FilterAllowlist.g.cs`, with every derived field that has filter operators filterable and sortable (on a report an enum dimension sorts too; an entity's enum field - does not). No names artifact. An enum dimension reaches the wire as its string symbol only when - the host serializes enums as strings (`ConfigureHttpJsonOptions` with a + does not). A derived field that can be null is a nullable member (`long?`, `decimal?`); a + measure with `@default`, and in a `@spine` report a key or `@required` column of the spine + entity, is not (`long`, `decimal`). No names artifact. An enum dimension reaches the wire + as its string symbol only when the host serializes enums as strings (`ConfigureHttpJsonOptions` with a `JsonStringEnumConverter`): the routes return the row object and the host owns serialization. See [reporting](../features/reporting.md#how-a-report-is-served). - `AppDbContext.g.cs` — `DbSet<Author>`, projection `.ToView()`, `@storage` owned diff --git a/docs/ports/java.md b/docs/ports/java.md index 62f848051..d03257fb9 100644 --- a/docs/ports/java.md +++ b/docs/ports/java.md @@ -308,7 +308,10 @@ like a keyless read-only projection. Four generators write one file each for a r `<R>Repository` (`list` and `count` only, which you implement against the view), `filter-allowlist` writes `<R>FilterAllowlist`, and `routes` writes `<R>Controller` (one `@GetMapping`, a `@PostMapping` answering 405, no `/{id}` mapping). Every derived field with -filter operators is filterable and sortable. No other generator emits for a report: no names +filter operators is filterable and sortable. A DTO component that can never be null carries +`@NotNull`, as any required field does: a `count`, a measure with `@default`, and in a +`@spine` report a key or `@required` column of the spine entity (`@NotNull Long programKey`). +No other generator emits for a report: no names class, no entity class. `gen` fails, naming the report and the dimension or measure, when a derived field reads a `field.object`. A report with no view source, or an abstract one, generates nothing. The contract is in diff --git a/docs/ports/kotlin.md b/docs/ports/kotlin.md index 0a715276c..304897e38 100644 --- a/docs/ports/kotlin.md +++ b/docs/ports/kotlin.md @@ -406,7 +406,10 @@ object ProgramMinutesTable : Table("v_program_minutes") { - A report has no identity, so the object has no `primaryKey`. List it and count it; there is no by-id read and no write. - A column is nullable exactly when the derived field can be null: a `sum`, `avg`, `min`, `max` - or ratio, and a dimension reached through `@via`. + or ratio, and a dimension reached through `@via`. Two exceptions: a measure that declares + `@default` is never null, and in a `@spine` report neither is a dimension over a `@required` + or primary-key column of the spine entity. Each gets a column with no `.nullable()` and a + property with no `?` and no `= null`. - A derived decimal with no declared precision (an `avg`, a ratio, a `sum` of a decimal) is read as `decimal(name, 38, 18)`. Exposed rounds a decimal to the column's scale when it reads it, so the value is exact to 18 places. The object maps a view, so those numbers never reach diff --git a/docs/ports/python.md b/docs/ports/python.md index d81effc19..f55000bb8 100644 --- a/docs/ports/python.md +++ b/docs/ports/python.md @@ -141,6 +141,9 @@ like a keyless read-only projection. Four generators write one file each for a r (`GET` list, `POST` answering 405, no `/{id}` route, and a repository Protocol with `list` and `count` only, which you implement against the view), and `names` writes `<snake>_names.py`. Every derived field with filter operators is filterable and sortable. A +row-model field that can be null is `T | None = None`; a measure with `@default`, and in a +`@spine` report a key or `@required` column of the spine entity, is the bare type +(`revenueOrZero: int` beside `revenueCents: int | None = None`). A report with no view source, or an abstract one, generates nothing. The contract is in [reporting](../features/reporting.md#how-a-report-is-served). diff --git a/docs/ports/typescript.md b/docs/ports/typescript.md index f854c81fe..00bb957a7 100644 --- a/docs/ports/typescript.md +++ b/docs/ports/typescript.md @@ -314,7 +314,10 @@ like a keyless read-only projection. For a report `<R>` the generators you have | `tanstackQuery()` | `<R>.hooks.ts`, `<R>.meta.ts` | the list hook (`use<R>List`, with the report's filter type) and its descriptor. No detail hook, no mutation | | `barrel()` | `index.ts` | one export | -Every derived field with filter operators is filterable and sortable. A decimal column (an +Every derived field with filter operators is filterable and sortable. A derived field that can +be null is `.nullable()` in the Zod read schema and `T | null` in the row type. A measure with +`@default`, and in a `@spine` report a key or `@required` column of the spine entity, can never +be null: no `.nullable()`, and the row type is `T`. A decimal column (an `avg`, a ratio) is a `string` in the read schema and on the wire, on SQLite too: SQLite computes it as a REAL, so on that dialect the generated route passes the decimal field names to the mount as `decimalColumns` and the mount sends each as its string. The UI tier writes the diff --git a/docs/recipes/mysql.md b/docs/recipes/mysql.md index d67dae1be..83f133c7c 100644 --- a/docs/recipes/mysql.md +++ b/docs/recipes/mysql.md @@ -112,11 +112,14 @@ migration); the loop will not. Pass `columnNamingStrategy` to match your tables' column names (the default is `snake_case`). The bodies are valid under MySQL's default `sql_mode`, `ONLY_FULL_GROUP_BY` included, and a change to a report means a new `CREATE OR REPLACE VIEW` (or `DROP` and `CREATE`) in your -migrations; nothing diffs the live view for you. Two things differ from Postgres and SQLite: +migrations; nothing diffs the live view for you. A `@spine` report's body is valid as +emitted too, and a measure with `@default` is a `COALESCE` whose column MySQL reports +`NOT NULL`. Two things differ from Postgres and SQLite: - **Ratios and averages have four fractional digits by default.** MySQL divides to `div_precision_increment` digits, so a ratio of 2 to 3 is `0.6667` (Postgres returns `0.66666666666666666667`, SQLite `0.6666666666666666`), and a ratio of 3 to 4 is `0.7500`. + A ratio that reads its `@default: 0` is `0.0000`. - **`DATETIME` values are read as the UTC wall clock.** A `DATETIME(3)` column carries no zone, so every time grain and every relative-date window (`{ "now": "-P30D" }`, evaluated with `UTC_TIMESTAMP(3)` when the view is queried) treats the stored value as UTC. That matches diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index a7ed27bf9..304bae183 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,7 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | -| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, rows with no facts (`@spine`), defaults (`@default`), engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -740,6 +740,20 @@ Three rules an author trips on: measure (a time dimension at a grain is `<name><Grain>`, so `purchasedAt:day` is `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. +Two questions to answer for every report: + +- **Do I want the rows that have no facts?** A report's rows come from the facts, so a + program nobody bought has no row. If the page should list it, add `"@spine": + "Purchase.program"` to the report (a to-one path from `@from`, written like `@via`), and + **reach every listed dimension through it**: each needs an `@via` that begins with the + spine's hops (`programId` as `Program.id` via `Purchase.program`). A dimension on the fact + row is a load error under `@spine`. List the spine entity's key to get one row per program. +- **Is zero true here, or is null?** A `sum`, `avg`, `min`, `max` or ratio with nothing to + aggregate is null. If zero is the true answer (revenue of a program nobody bought), declare + `"@default": 0` on the measure; if the value is unknown (an average rating with no + ratings), leave it null. **Never on a `count`**: it is already `0`, and `@default` there is a + load error. Write `0`, never `0.0`. + **A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration is what makes `meta migrate` create the view (Postgres, SQLite, D1), what every port's runtime reads, and what makes every port's generators serve it: one read-only list route at @@ -752,7 +766,7 @@ no `source.*` is checked at load and generates nothing. A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its base's table): declare it `@from` the base with an `@filter` on the discriminator field. -What does not exist: no client hook, grid or form for a report yet, no way to narrow which +What does not exist: no grid or form for a report (TypeScript generates a list hook; no other port has a client tier), no way to narrow which derived fields are filterable, no `measure.derived` (arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and diff --git a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md index be0bc0003..47c54d6d8 100644 --- a/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md +++ b/fixtures/agent-context-conformance/java-kotlin-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -1,6 +1,6 @@ # Reporting: the columns a report gets, and how it behaves -> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, rows with no facts (`@spine`), defaults for empty measures (`@default`), what differs between databases, and what it leaves out. ## A report is served only with a view source @@ -34,8 +34,10 @@ A report declares no fields. Its columns are one per `@dimensions` item in liste | `avg` of `double` / `float` | the measure's name | `double` | no | | `min` / `max` | the measure's name | the `@of` field's type | no | | `measure.ratio` | the measure's name | `decimal` | no | +| any measure above except `count`, declaring `@default` | the measure's name | as above | yes: it reads the default instead of null | +| in a `@spine` report, a dimension over a column of the spine entity | the dimension's name | the `@of` field's type | yes when the field is `@required` or in that entity's `identity.primary`; no for a dimension beyond the spine entity | -A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. +Only those last two rows make a measure other than `count`, or a dimension reached by `@via`, never null; every port's row type follows (no `| null`, no `?`, no `Optional`). A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no field `@default`, no validators. ## Time grains @@ -50,13 +52,64 @@ A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 dur ## Nulls and zeros - A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. -- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. So are `avg`, `min` and `max`. - A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. - A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). +**Ask: is zero true here, or is null?** If "nothing to aggregate" means zero (revenue of a program nobody bought is `0`), declare `"@default": 0` on the measure (`revenue` in the example under "Rows with no facts"). If it means unknown (the average rating of a product with no ratings is not `0`), leave it null. Do not default in the client what the model can state once. + +- `@default` is an integer. Write `0`, not `0.0`: a fraction (`0.5`) is refused in every port, and `0.0` loads in TypeScript but is refused by C#, Java and Python. +- **Never on a `count`** (it is never null, so the default could never apply) and never on a `min` / `max` over a field that is not numeric (a date or a string): both are load errors. +- It applies whenever the measure would be null: nothing matched, every matched value was null, or a ratio's denominator is zero or null. The column is then never null in any port. +- **A ratio's operand carries its own `@default` into the ratio.** `revenue` defaulted to `0` over `buyers` reads `0` for a group with buyers and no revenue. The ratio's own `@default` covers a zero denominator. +- A filter on a defaulted measure sees the default: `?filter[revenue][eq]=0` matches the empty rows, `isNull=true` matches nothing. A sort on it is the same on every engine. + +## Rows with no facts: `@spine` + +A report's rows are the dimension tuples its `@from` rows have, so a per-program report has no row for a program nobody bought. **Ask: do I want the rows that have no facts?** If the page lists every program (a catalogue, a chart whose empty bars matter), add `@spine`: a to-one path from `@from`, written like `@via`, to the entity whose rows the report should have. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Program", "children": [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } ] } }, + { "object.entity": { "name": "Purchase", "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "programId" } }, + { "field.currency": { "name": "amountCents" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }, + { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } } ] } }, + { "object.report": { "name": "ProgramSales", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle"], "@measures": ["purchases", "revenue"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_sales" } } ] } } + ] +}} +``` + +A program with no purchases has a row: `purchases` `0`, `revenue` `0`. + +- **Every dimension goes through the spine.** Each listed dimension needs an `@via` that begins with the spine's hops (`programId` is `Program.id` via `Purchase.program`). A dimension on the fact row (`Purchase.programId` with no `@via`, the purchase month) is a load error: a program with no purchases has no value for it. +- **Write the same hops as the spine.** `Purchase.program` (relationship) and `Purchase.fkProgram` (reference) are the same join, but a spine written one way and a dimension the other is refused. +- List at least one dimension, and list the spine entity's key (`Program.id`) to get exactly one row per program; two programs listed only by a shared title are one row. +- On a row with no facts, a `count` is `0` and every other measure is null unless it declares `@default`. +- The report's `@segment` and `@filter` still scope the facts and never remove a program's row. They name `@from`'s fields only; to show only some programs, list the column as a dimension and filter it on the request. +- A fact whose reference is null, or matches nothing, is in no row of a `@spine` report. +- Every hop needs an `identity.reference`, and the spine entity (and every entity on the way) needs a table of its own: an abstract entity or a TPH subtype fails `meta migrate`. +- The empty rows are listed and counted on the route; `?filter[purchases][gt]=0` hides them on request, so one report serves both pages. + ## Joins: a dimension through a required reference drops rows -A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. In a `@spine` report every join is `LEFT OUTER`. **Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. @@ -67,6 +120,7 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t | View created by | `meta migrate` | `meta migrate` | you: see below | | A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | | `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| A ratio reading its `@default: 0` | `0` | `0.0` (the default is written `0.0` so the `REAL` column keeps one storage class) | `0.0000` | | Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | **MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. @@ -77,6 +131,11 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t - **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. - **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, is not served by any port and gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). - **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Java, Kotlin and C# `gen`; only TypeScript and Python read and serve it (as parsed JSON, neither filterable nor sortable). Group by a scalar field. +- **One `@spine` per report.** Two independent spines (every program against every customer) cannot be written. Entities to-one beyond the spine are ordinary dimensions. +- **No row scope on the spine entity** (only published programs): a report `@filter` names `@from`'s fields. List the column as a dimension and filter it on the request. +- **No fact-row dimension beside a spine** (every program, split by purchase month), and no dimension over the reference column itself (`Purchase.programId`). Declare it over the spine entity's key instead. Zero-filled time buckets are not available yet. +- **No join on a natural key, no anti-join, no report over two fact tables.** A `@spine` reaches its facts through a declared reference only: give the fact a reference to the spine entity. +- **`@default` is a whole number.** A fraction is `ERR_BAD_ATTR_VALUE` on the measure in every port (TypeScript's message says it is not an integer; C#, Java and Python use their integer-attribute message). `0.0` or `1.0` loads in TypeScript and is refused elsewhere, and `"zero"` on a `count` is two errors in TypeScript, C# and Python but one in Java. Write `0`. ## How a report is served @@ -90,7 +149,7 @@ A concrete report whose read source is `@kind: view` is served by every port's g - `<segment>` is the report name, snake_cased then pluralized, like any object: `InvoicesByMonth` is `/invoices_by_months`, `StoreTotals` is `/store_totals`. There is no report-specific spelling. - **Every derived field with filter operators is filterable and sortable**, dimension and measure alike, with its type's own operators. You cannot narrow that set: a report has no field to put `@filterable` on. A field of the `@from` entity that the report does not expose is refused with a `400`. -- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present; `currency` is integer minor units. +- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present (a measure with `@default` is never null); `currency` is integer minor units. - **A decimal (`avg`, a ratio, a `sum` of a decimal) has no agreed JSON spelling**: each port sends its own (TypeScript a string). Do not compare one across backends as text. - `meta docs` writes a model page for every report (a sourceless one is marked "Not served") and an API page for a served one. diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md index a7ed27bf9..304bae183 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,7 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | -| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, rows with no facts (`@spine`), defaults (`@default`), engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -740,6 +740,20 @@ Three rules an author trips on: measure (a time dimension at a grain is `<name><Grain>`, so `purchasedAt:day` is `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. +Two questions to answer for every report: + +- **Do I want the rows that have no facts?** A report's rows come from the facts, so a + program nobody bought has no row. If the page should list it, add `"@spine": + "Purchase.program"` to the report (a to-one path from `@from`, written like `@via`), and + **reach every listed dimension through it**: each needs an `@via` that begins with the + spine's hops (`programId` as `Program.id` via `Purchase.program`). A dimension on the fact + row is a load error under `@spine`. List the spine entity's key to get one row per program. +- **Is zero true here, or is null?** A `sum`, `avg`, `min`, `max` or ratio with nothing to + aggregate is null. If zero is the true answer (revenue of a program nobody bought), declare + `"@default": 0` on the measure; if the value is unknown (an average rating with no + ratings), leave it null. **Never on a `count`**: it is already `0`, and `@default` there is a + load error. Write `0`, never `0.0`. + **A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration is what makes `meta migrate` create the view (Postgres, SQLite, D1), what every port's runtime reads, and what makes every port's generators serve it: one read-only list route at @@ -752,7 +766,7 @@ no `source.*` is checked at load and generates nothing. A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its base's table): declare it `@from` the base with an `@filter` on the discriminator field. -What does not exist: no client hook, grid or form for a report yet, no way to narrow which +What does not exist: no grid or form for a report (TypeScript generates a list hook; no other port has a client tier), no way to narrow which derived fields are filterable, no `measure.derived` (arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and diff --git a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/reporting.md index be0bc0003..47c54d6d8 100644 --- a/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/reporting.md +++ b/fixtures/agent-context-conformance/java-react/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -1,6 +1,6 @@ # Reporting: the columns a report gets, and how it behaves -> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, rows with no facts (`@spine`), defaults for empty measures (`@default`), what differs between databases, and what it leaves out. ## A report is served only with a view source @@ -34,8 +34,10 @@ A report declares no fields. Its columns are one per `@dimensions` item in liste | `avg` of `double` / `float` | the measure's name | `double` | no | | `min` / `max` | the measure's name | the `@of` field's type | no | | `measure.ratio` | the measure's name | `decimal` | no | +| any measure above except `count`, declaring `@default` | the measure's name | as above | yes: it reads the default instead of null | +| in a `@spine` report, a dimension over a column of the spine entity | the dimension's name | the `@of` field's type | yes when the field is `@required` or in that entity's `identity.primary`; no for a dimension beyond the spine entity | -A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. +Only those last two rows make a measure other than `count`, or a dimension reached by `@via`, never null; every port's row type follows (no `| null`, no `?`, no `Optional`). A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no field `@default`, no validators. ## Time grains @@ -50,13 +52,64 @@ A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 dur ## Nulls and zeros - A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. -- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. So are `avg`, `min` and `max`. - A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. - A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). +**Ask: is zero true here, or is null?** If "nothing to aggregate" means zero (revenue of a program nobody bought is `0`), declare `"@default": 0` on the measure (`revenue` in the example under "Rows with no facts"). If it means unknown (the average rating of a product with no ratings is not `0`), leave it null. Do not default in the client what the model can state once. + +- `@default` is an integer. Write `0`, not `0.0`: a fraction (`0.5`) is refused in every port, and `0.0` loads in TypeScript but is refused by C#, Java and Python. +- **Never on a `count`** (it is never null, so the default could never apply) and never on a `min` / `max` over a field that is not numeric (a date or a string): both are load errors. +- It applies whenever the measure would be null: nothing matched, every matched value was null, or a ratio's denominator is zero or null. The column is then never null in any port. +- **A ratio's operand carries its own `@default` into the ratio.** `revenue` defaulted to `0` over `buyers` reads `0` for a group with buyers and no revenue. The ratio's own `@default` covers a zero denominator. +- A filter on a defaulted measure sees the default: `?filter[revenue][eq]=0` matches the empty rows, `isNull=true` matches nothing. A sort on it is the same on every engine. + +## Rows with no facts: `@spine` + +A report's rows are the dimension tuples its `@from` rows have, so a per-program report has no row for a program nobody bought. **Ask: do I want the rows that have no facts?** If the page lists every program (a catalogue, a chart whose empty bars matter), add `@spine`: a to-one path from `@from`, written like `@via`, to the entity whose rows the report should have. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Program", "children": [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } ] } }, + { "object.entity": { "name": "Purchase", "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "programId" } }, + { "field.currency": { "name": "amountCents" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }, + { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } } ] } }, + { "object.report": { "name": "ProgramSales", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle"], "@measures": ["purchases", "revenue"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_sales" } } ] } } + ] +}} +``` + +A program with no purchases has a row: `purchases` `0`, `revenue` `0`. + +- **Every dimension goes through the spine.** Each listed dimension needs an `@via` that begins with the spine's hops (`programId` is `Program.id` via `Purchase.program`). A dimension on the fact row (`Purchase.programId` with no `@via`, the purchase month) is a load error: a program with no purchases has no value for it. +- **Write the same hops as the spine.** `Purchase.program` (relationship) and `Purchase.fkProgram` (reference) are the same join, but a spine written one way and a dimension the other is refused. +- List at least one dimension, and list the spine entity's key (`Program.id`) to get exactly one row per program; two programs listed only by a shared title are one row. +- On a row with no facts, a `count` is `0` and every other measure is null unless it declares `@default`. +- The report's `@segment` and `@filter` still scope the facts and never remove a program's row. They name `@from`'s fields only; to show only some programs, list the column as a dimension and filter it on the request. +- A fact whose reference is null, or matches nothing, is in no row of a `@spine` report. +- Every hop needs an `identity.reference`, and the spine entity (and every entity on the way) needs a table of its own: an abstract entity or a TPH subtype fails `meta migrate`. +- The empty rows are listed and counted on the route; `?filter[purchases][gt]=0` hides them on request, so one report serves both pages. + ## Joins: a dimension through a required reference drops rows -A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. In a `@spine` report every join is `LEFT OUTER`. **Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. @@ -67,6 +120,7 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t | View created by | `meta migrate` | `meta migrate` | you: see below | | A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | | `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| A ratio reading its `@default: 0` | `0` | `0.0` (the default is written `0.0` so the `REAL` column keeps one storage class) | `0.0000` | | Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | **MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. @@ -77,6 +131,11 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t - **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. - **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, is not served by any port and gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). - **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Java, Kotlin and C# `gen`; only TypeScript and Python read and serve it (as parsed JSON, neither filterable nor sortable). Group by a scalar field. +- **One `@spine` per report.** Two independent spines (every program against every customer) cannot be written. Entities to-one beyond the spine are ordinary dimensions. +- **No row scope on the spine entity** (only published programs): a report `@filter` names `@from`'s fields. List the column as a dimension and filter it on the request. +- **No fact-row dimension beside a spine** (every program, split by purchase month), and no dimension over the reference column itself (`Purchase.programId`). Declare it over the spine entity's key instead. Zero-filled time buckets are not available yet. +- **No join on a natural key, no anti-join, no report over two fact tables.** A `@spine` reaches its facts through a declared reference only: give the fact a reference to the spine entity. +- **`@default` is a whole number.** A fraction is `ERR_BAD_ATTR_VALUE` on the measure in every port (TypeScript's message says it is not an integer; C#, Java and Python use their integer-attribute message). `0.0` or `1.0` loads in TypeScript and is refused elsewhere, and `"zero"` on a `count` is two errors in TypeScript, C# and Python but one in Java. Write `0`. ## How a report is served @@ -90,7 +149,7 @@ A concrete report whose read source is `@kind: view` is served by every port's g - `<segment>` is the report name, snake_cased then pluralized, like any object: `InvoicesByMonth` is `/invoices_by_months`, `StoreTotals` is `/store_totals`. There is no report-specific spelling. - **Every derived field with filter operators is filterable and sortable**, dimension and measure alike, with its type's own operators. You cannot narrow that set: a report has no field to put `@filterable` on. A field of the `@from` entity that the report does not expose is refused with a `400`. -- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present; `currency` is integer minor units. +- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present (a measure with `@default` is never null); `currency` is integer minor units. - **A decimal (`avg`, a ratio, a `sum` of a decimal) has no agreed JSON spelling**: each port sends its own (TypeScript a string). Do not compare one across backends as text. - `meta docs` writes a model page for every report (a sourceless one is marked "Not served") and an API page for a served one. diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md index a7ed27bf9..304bae183 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,7 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | -| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, rows with no facts (`@spine`), defaults (`@default`), engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -740,6 +740,20 @@ Three rules an author trips on: measure (a time dimension at a grain is `<name><Grain>`, so `purchasedAt:day` is `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. +Two questions to answer for every report: + +- **Do I want the rows that have no facts?** A report's rows come from the facts, so a + program nobody bought has no row. If the page should list it, add `"@spine": + "Purchase.program"` to the report (a to-one path from `@from`, written like `@via`), and + **reach every listed dimension through it**: each needs an `@via` that begins with the + spine's hops (`programId` as `Program.id` via `Purchase.program`). A dimension on the fact + row is a load error under `@spine`. List the spine entity's key to get one row per program. +- **Is zero true here, or is null?** A `sum`, `avg`, `min`, `max` or ratio with nothing to + aggregate is null. If zero is the true answer (revenue of a program nobody bought), declare + `"@default": 0` on the measure; if the value is unknown (an average rating with no + ratings), leave it null. **Never on a `count`**: it is already `0`, and `@default` there is a + load error. Write `0`, never `0.0`. + **A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration is what makes `meta migrate` create the view (Postgres, SQLite, D1), what every port's runtime reads, and what makes every port's generators serve it: one read-only list route at @@ -752,7 +766,7 @@ no `source.*` is checked at load and generates nothing. A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its base's table): declare it `@from` the base with an `@filter` on the discriminator field. -What does not exist: no client hook, grid or form for a report yet, no way to narrow which +What does not exist: no grid or form for a report (TypeScript generates a list hook; no other port has a client tier), no way to narrow which derived fields are filterable, no `measure.derived` (arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and diff --git a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/reporting.md index be0bc0003..47c54d6d8 100644 --- a/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/reporting.md +++ b/fixtures/agent-context-conformance/python/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -1,6 +1,6 @@ # Reporting: the columns a report gets, and how it behaves -> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, rows with no facts (`@spine`), defaults for empty measures (`@default`), what differs between databases, and what it leaves out. ## A report is served only with a view source @@ -34,8 +34,10 @@ A report declares no fields. Its columns are one per `@dimensions` item in liste | `avg` of `double` / `float` | the measure's name | `double` | no | | `min` / `max` | the measure's name | the `@of` field's type | no | | `measure.ratio` | the measure's name | `decimal` | no | +| any measure above except `count`, declaring `@default` | the measure's name | as above | yes: it reads the default instead of null | +| in a `@spine` report, a dimension over a column of the spine entity | the dimension's name | the `@of` field's type | yes when the field is `@required` or in that entity's `identity.primary`; no for a dimension beyond the spine entity | -A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. +Only those last two rows make a measure other than `count`, or a dimension reached by `@via`, never null; every port's row type follows (no `| null`, no `?`, no `Optional`). A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no field `@default`, no validators. ## Time grains @@ -50,13 +52,64 @@ A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 dur ## Nulls and zeros - A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. -- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. So are `avg`, `min` and `max`. - A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. - A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). +**Ask: is zero true here, or is null?** If "nothing to aggregate" means zero (revenue of a program nobody bought is `0`), declare `"@default": 0` on the measure (`revenue` in the example under "Rows with no facts"). If it means unknown (the average rating of a product with no ratings is not `0`), leave it null. Do not default in the client what the model can state once. + +- `@default` is an integer. Write `0`, not `0.0`: a fraction (`0.5`) is refused in every port, and `0.0` loads in TypeScript but is refused by C#, Java and Python. +- **Never on a `count`** (it is never null, so the default could never apply) and never on a `min` / `max` over a field that is not numeric (a date or a string): both are load errors. +- It applies whenever the measure would be null: nothing matched, every matched value was null, or a ratio's denominator is zero or null. The column is then never null in any port. +- **A ratio's operand carries its own `@default` into the ratio.** `revenue` defaulted to `0` over `buyers` reads `0` for a group with buyers and no revenue. The ratio's own `@default` covers a zero denominator. +- A filter on a defaulted measure sees the default: `?filter[revenue][eq]=0` matches the empty rows, `isNull=true` matches nothing. A sort on it is the same on every engine. + +## Rows with no facts: `@spine` + +A report's rows are the dimension tuples its `@from` rows have, so a per-program report has no row for a program nobody bought. **Ask: do I want the rows that have no facts?** If the page lists every program (a catalogue, a chart whose empty bars matter), add `@spine`: a to-one path from `@from`, written like `@via`, to the entity whose rows the report should have. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Program", "children": [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } ] } }, + { "object.entity": { "name": "Purchase", "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "programId" } }, + { "field.currency": { "name": "amountCents" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }, + { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } } ] } }, + { "object.report": { "name": "ProgramSales", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle"], "@measures": ["purchases", "revenue"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_sales" } } ] } } + ] +}} +``` + +A program with no purchases has a row: `purchases` `0`, `revenue` `0`. + +- **Every dimension goes through the spine.** Each listed dimension needs an `@via` that begins with the spine's hops (`programId` is `Program.id` via `Purchase.program`). A dimension on the fact row (`Purchase.programId` with no `@via`, the purchase month) is a load error: a program with no purchases has no value for it. +- **Write the same hops as the spine.** `Purchase.program` (relationship) and `Purchase.fkProgram` (reference) are the same join, but a spine written one way and a dimension the other is refused. +- List at least one dimension, and list the spine entity's key (`Program.id`) to get exactly one row per program; two programs listed only by a shared title are one row. +- On a row with no facts, a `count` is `0` and every other measure is null unless it declares `@default`. +- The report's `@segment` and `@filter` still scope the facts and never remove a program's row. They name `@from`'s fields only; to show only some programs, list the column as a dimension and filter it on the request. +- A fact whose reference is null, or matches nothing, is in no row of a `@spine` report. +- Every hop needs an `identity.reference`, and the spine entity (and every entity on the way) needs a table of its own: an abstract entity or a TPH subtype fails `meta migrate`. +- The empty rows are listed and counted on the route; `?filter[purchases][gt]=0` hides them on request, so one report serves both pages. + ## Joins: a dimension through a required reference drops rows -A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. In a `@spine` report every join is `LEFT OUTER`. **Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. @@ -67,6 +120,7 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t | View created by | `meta migrate` | `meta migrate` | you: see below | | A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | | `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| A ratio reading its `@default: 0` | `0` | `0.0` (the default is written `0.0` so the `REAL` column keeps one storage class) | `0.0000` | | Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | **MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. @@ -77,6 +131,11 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t - **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. - **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, is not served by any port and gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). - **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Java, Kotlin and C# `gen`; only TypeScript and Python read and serve it (as parsed JSON, neither filterable nor sortable). Group by a scalar field. +- **One `@spine` per report.** Two independent spines (every program against every customer) cannot be written. Entities to-one beyond the spine are ordinary dimensions. +- **No row scope on the spine entity** (only published programs): a report `@filter` names `@from`'s fields. List the column as a dimension and filter it on the request. +- **No fact-row dimension beside a spine** (every program, split by purchase month), and no dimension over the reference column itself (`Purchase.programId`). Declare it over the spine entity's key instead. Zero-filled time buckets are not available yet. +- **No join on a natural key, no anti-join, no report over two fact tables.** A `@spine` reaches its facts through a declared reference only: give the fact a reference to the spine entity. +- **`@default` is a whole number.** A fraction is `ERR_BAD_ATTR_VALUE` on the measure in every port (TypeScript's message says it is not an integer; C#, Java and Python use their integer-attribute message). `0.0` or `1.0` loads in TypeScript and is refused elsewhere, and `"zero"` on a `count` is two errors in TypeScript, C# and Python but one in Java. Write `0`. ## How a report is served @@ -90,7 +149,7 @@ A concrete report whose read source is `@kind: view` is served by every port's g - `<segment>` is the report name, snake_cased then pluralized, like any object: `InvoicesByMonth` is `/invoices_by_months`, `StoreTotals` is `/store_totals`. There is no report-specific spelling. - **Every derived field with filter operators is filterable and sortable**, dimension and measure alike, with its type's own operators. You cannot narrow that set: a report has no field to put `@filterable` on. A field of the `@from` entity that the report does not expose is refused with a `400`. -- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present; `currency` is integer minor units. +- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present (a measure with `@default` is never null); `currency` is integer minor units. - **A decimal (`avg`, a ratio, a `sum` of a decimal) has no agreed JSON spelling**: each port sends its own (TypeScript a string). Do not compare one across backends as text. - `meta docs` writes a model page for every report (a sourceless one is marked "Not served") and an API page for a served one. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md index a7ed27bf9..304bae183 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,7 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | -| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, rows with no facts (`@spine`), defaults (`@default`), engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -740,6 +740,20 @@ Three rules an author trips on: measure (a time dimension at a grain is `<name><Grain>`, so `purchasedAt:day` is `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. +Two questions to answer for every report: + +- **Do I want the rows that have no facts?** A report's rows come from the facts, so a + program nobody bought has no row. If the page should list it, add `"@spine": + "Purchase.program"` to the report (a to-one path from `@from`, written like `@via`), and + **reach every listed dimension through it**: each needs an `@via` that begins with the + spine's hops (`programId` as `Program.id` via `Purchase.program`). A dimension on the fact + row is a load error under `@spine`. List the spine entity's key to get one row per program. +- **Is zero true here, or is null?** A `sum`, `avg`, `min`, `max` or ratio with nothing to + aggregate is null. If zero is the true answer (revenue of a program nobody bought), declare + `"@default": 0` on the measure; if the value is unknown (an average rating with no + ratings), leave it null. **Never on a `count`**: it is already `0`, and `@default` there is a + load error. Write `0`, never `0.0`. + **A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration is what makes `meta migrate` create the view (Postgres, SQLite, D1), what every port's runtime reads, and what makes every port's generators serve it: one read-only list route at @@ -752,7 +766,7 @@ no `source.*` is checked at load and generates nothing. A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its base's table): declare it `@from` the base with an `@filter` on the discriminator field. -What does not exist: no client hook, grid or form for a report yet, no way to narrow which +What does not exist: no grid or form for a report (TypeScript generates a list hook; no other port has a client tier), no way to narrow which derived fields are filterable, no `measure.derived` (arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md index be0bc0003..47c54d6d8 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -1,6 +1,6 @@ # Reporting: the columns a report gets, and how it behaves -> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, rows with no facts (`@spine`), defaults for empty measures (`@default`), what differs between databases, and what it leaves out. ## A report is served only with a view source @@ -34,8 +34,10 @@ A report declares no fields. Its columns are one per `@dimensions` item in liste | `avg` of `double` / `float` | the measure's name | `double` | no | | `min` / `max` | the measure's name | the `@of` field's type | no | | `measure.ratio` | the measure's name | `decimal` | no | +| any measure above except `count`, declaring `@default` | the measure's name | as above | yes: it reads the default instead of null | +| in a `@spine` report, a dimension over a column of the spine entity | the dimension's name | the `@of` field's type | yes when the field is `@required` or in that entity's `identity.primary`; no for a dimension beyond the spine entity | -A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. +Only those last two rows make a measure other than `count`, or a dimension reached by `@via`, never null; every port's row type follows (no `| null`, no `?`, no `Optional`). A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no field `@default`, no validators. ## Time grains @@ -50,13 +52,64 @@ A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 dur ## Nulls and zeros - A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. -- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. So are `avg`, `min` and `max`. - A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. - A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). +**Ask: is zero true here, or is null?** If "nothing to aggregate" means zero (revenue of a program nobody bought is `0`), declare `"@default": 0` on the measure (`revenue` in the example under "Rows with no facts"). If it means unknown (the average rating of a product with no ratings is not `0`), leave it null. Do not default in the client what the model can state once. + +- `@default` is an integer. Write `0`, not `0.0`: a fraction (`0.5`) is refused in every port, and `0.0` loads in TypeScript but is refused by C#, Java and Python. +- **Never on a `count`** (it is never null, so the default could never apply) and never on a `min` / `max` over a field that is not numeric (a date or a string): both are load errors. +- It applies whenever the measure would be null: nothing matched, every matched value was null, or a ratio's denominator is zero or null. The column is then never null in any port. +- **A ratio's operand carries its own `@default` into the ratio.** `revenue` defaulted to `0` over `buyers` reads `0` for a group with buyers and no revenue. The ratio's own `@default` covers a zero denominator. +- A filter on a defaulted measure sees the default: `?filter[revenue][eq]=0` matches the empty rows, `isNull=true` matches nothing. A sort on it is the same on every engine. + +## Rows with no facts: `@spine` + +A report's rows are the dimension tuples its `@from` rows have, so a per-program report has no row for a program nobody bought. **Ask: do I want the rows that have no facts?** If the page lists every program (a catalogue, a chart whose empty bars matter), add `@spine`: a to-one path from `@from`, written like `@via`, to the entity whose rows the report should have. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Program", "children": [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } ] } }, + { "object.entity": { "name": "Purchase", "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "programId" } }, + { "field.currency": { "name": "amountCents" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }, + { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } } ] } }, + { "object.report": { "name": "ProgramSales", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle"], "@measures": ["purchases", "revenue"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_sales" } } ] } } + ] +}} +``` + +A program with no purchases has a row: `purchases` `0`, `revenue` `0`. + +- **Every dimension goes through the spine.** Each listed dimension needs an `@via` that begins with the spine's hops (`programId` is `Program.id` via `Purchase.program`). A dimension on the fact row (`Purchase.programId` with no `@via`, the purchase month) is a load error: a program with no purchases has no value for it. +- **Write the same hops as the spine.** `Purchase.program` (relationship) and `Purchase.fkProgram` (reference) are the same join, but a spine written one way and a dimension the other is refused. +- List at least one dimension, and list the spine entity's key (`Program.id`) to get exactly one row per program; two programs listed only by a shared title are one row. +- On a row with no facts, a `count` is `0` and every other measure is null unless it declares `@default`. +- The report's `@segment` and `@filter` still scope the facts and never remove a program's row. They name `@from`'s fields only; to show only some programs, list the column as a dimension and filter it on the request. +- A fact whose reference is null, or matches nothing, is in no row of a `@spine` report. +- Every hop needs an `identity.reference`, and the spine entity (and every entity on the way) needs a table of its own: an abstract entity or a TPH subtype fails `meta migrate`. +- The empty rows are listed and counted on the route; `?filter[purchases][gt]=0` hides them on request, so one report serves both pages. + ## Joins: a dimension through a required reference drops rows -A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. In a `@spine` report every join is `LEFT OUTER`. **Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. @@ -67,6 +120,7 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t | View created by | `meta migrate` | `meta migrate` | you: see below | | A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | | `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| A ratio reading its `@default: 0` | `0` | `0.0` (the default is written `0.0` so the `REAL` column keeps one storage class) | `0.0000` | | Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | **MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. @@ -77,6 +131,11 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t - **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. - **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, is not served by any port and gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). - **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Java, Kotlin and C# `gen`; only TypeScript and Python read and serve it (as parsed JSON, neither filterable nor sortable). Group by a scalar field. +- **One `@spine` per report.** Two independent spines (every program against every customer) cannot be written. Entities to-one beyond the spine are ordinary dimensions. +- **No row scope on the spine entity** (only published programs): a report `@filter` names `@from`'s fields. List the column as a dimension and filter it on the request. +- **No fact-row dimension beside a spine** (every program, split by purchase month), and no dimension over the reference column itself (`Purchase.programId`). Declare it over the spine entity's key instead. Zero-filled time buckets are not available yet. +- **No join on a natural key, no anti-join, no report over two fact tables.** A `@spine` reaches its facts through a declared reference only: give the fact a reference to the spine entity. +- **`@default` is a whole number.** A fraction is `ERR_BAD_ATTR_VALUE` on the measure in every port (TypeScript's message says it is not an integer; C#, Java and Python use their integer-attribute message). `0.0` or `1.0` loads in TypeScript and is refused elsewhere, and `"zero"` on a `count` is two errors in TypeScript, C# and Python but one in Java. Write `0`. ## How a report is served @@ -90,7 +149,7 @@ A concrete report whose read source is `@kind: view` is served by every port's g - `<segment>` is the report name, snake_cased then pluralized, like any object: `InvoicesByMonth` is `/invoices_by_months`, `StoreTotals` is `/store_totals`. There is no report-specific spelling. - **Every derived field with filter operators is filterable and sortable**, dimension and measure alike, with its type's own operators. You cannot narrow that set: a report has no field to put `@filterable` on. A field of the `@from` entity that the report does not expose is refused with a `400`. -- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present; `currency` is integer minor units. +- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present (a measure with `@default` is never null); `currency` is integer minor units. - **A decimal (`avg`, a ratio, a `sum` of a decimal) has no agreed JSON spelling**: each port sends its own (TypeScript a string). Do not compare one across backends as text. - `meta docs` writes a model page for every report (a sourceless one is marked "Not served") and an API page for a served one. diff --git a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md index eb54c6be2..8392d6051 100644 --- a/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md +++ b/fixtures/agent-context-conformance/ts-react-tanstack/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md @@ -94,11 +94,14 @@ migration); the loop will not. Pass `columnNamingStrategy` to match your tables' column names (the default is `snake_case`). The bodies are valid under MySQL's default `sql_mode`, `ONLY_FULL_GROUP_BY` included, and a change to a report means a new `CREATE OR REPLACE VIEW` (or `DROP` and `CREATE`) in your -migrations; nothing diffs the live view for you. Two things differ from Postgres and SQLite: +migrations; nothing diffs the live view for you. A `@spine` report's body is valid as +emitted too, and a measure with `@default` is a `COALESCE` whose column MySQL reports +`NOT NULL`. Two things differ from Postgres and SQLite: - **Ratios and averages have four fractional digits by default.** MySQL divides to `div_precision_increment` digits, so a ratio of 2 to 3 is `0.6667` (Postgres returns `0.66666666666666666667`, SQLite `0.6666666666666666`), and a ratio of 3 to 4 is `0.7500`. + A ratio that reads its `@default: 0` is `0.0000`. - **`DATETIME` values are read as the UTC wall clock.** A `DATETIME(3)` column carries no zone, so every time grain and every relative-date window (`{ "now": "-P30D" }`, evaluated with `UTC_TIMESTAMP(3)` when the view is queried) treats the stored value as UTC. That matches diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md index a7ed27bf9..304bae183 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/SKILL.md @@ -29,7 +29,7 @@ This file covers what almost every model needs. The topics below live in | `references/read-views-and-projections.md` | an `object.projection`, `origin.*` vocabulary, `@filter` / `@expr`, or an `@sql` / `@unmanaged` view | | `references/inheritance-tph.md` | several entities are variants of one thing sharing a single table (`@discriminator`) | | `references/metadata-dependencies.md` | the project builds on another package's metadata (`dependencies`, cross-package `overlay`) | -| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, engine differences | +| `references/reporting.md` | a dashboard number, count or total over one entity's rows: what columns a report gets, time grains, null rules, rows with no facts (`@spine`), defaults (`@default`), engine differences | | `references/requirements.md` | installed only when the project declares `requirement.*` nodes | ## The operating principle: model-first, generate-first @@ -740,6 +740,20 @@ Three rules an author trips on: measure (a time dimension at a grain is `<name><Grain>`, so `purchasedAt:day` is `purchasedAtDay`). A `field.*` or `identity.*` child on a report is an error. +Two questions to answer for every report: + +- **Do I want the rows that have no facts?** A report's rows come from the facts, so a + program nobody bought has no row. If the page should list it, add `"@spine": + "Purchase.program"` to the report (a to-one path from `@from`, written like `@via`), and + **reach every listed dimension through it**: each needs an `@via` that begins with the + spine's hops (`programId` as `Program.id` via `Purchase.program`). A dimension on the fact + row is a load error under `@spine`. List the spine entity's key to get one row per program. +- **Is zero true here, or is null?** A `sum`, `avg`, `min`, `max` or ratio with nothing to + aggregate is null. If zero is the true answer (revenue of a program nobody bought), declare + `"@default": 0` on the measure; if the value is unknown (an average rating with no + ratings), leave it null. **Never on a `count`**: it is already `0`, and `@default` there is a + load error. Write `0`, never `0.0`. + **A report is served only when it declares `source.rdb` with `@kind: view`.** That declaration is what makes `meta migrate` create the view (Postgres, SQLite, D1), what every port's runtime reads, and what makes every port's generators serve it: one read-only list route at @@ -752,7 +766,7 @@ no `source.*` is checked at load and generates nothing. A report `@from` a TPH subtype is refused when its view is derived (the subtype shares its base's table): declare it `@from` the base with an `@filter` on the discriminator field. -What does not exist: no client hook, grid or form for a report yet, no way to narrow which +What does not exist: no grid or form for a report (TypeScript generates a list hook; no other port has a client tier), no way to narrow which derived fields are filterable, no `measure.derived` (arithmetic between measures beyond `measure.ratio`), no query-time choice of dimensions or measures (a report is a fixed, compiled combination), and no time-zone vocabulary (grains and diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/reporting.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/reporting.md index be0bc0003..47c54d6d8 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/reporting.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-authoring/references/reporting.md @@ -1,6 +1,6 @@ # Reporting: the columns a report gets, and how it behaves -> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, what differs between databases, and what it leaves out. +> Part of the `metaobjects-authoring` skill. The skill covers declaring dimensions, measures, segments and a report. Read this when you need to know what a report returns: its column names and types, the time-grain and null rules, rows with no facts (`@spine`), defaults for empty measures (`@default`), what differs between databases, and what it leaves out. ## A report is served only with a view source @@ -34,8 +34,10 @@ A report declares no fields. Its columns are one per `@dimensions` item in liste | `avg` of `double` / `float` | the measure's name | `double` | no | | `min` / `max` | the measure's name | the `@of` field's type | no | | `measure.ratio` | the measure's name | `decimal` | no | +| any measure above except `count`, declaring `@default` | the measure's name | as above | yes: it reads the default instead of null | +| in a `@spine` report, a dimension over a column of the spine entity | the dimension's name | the `@of` field's type | yes when the field is `@required` or in that entity's `identity.primary`; no for a dimension beyond the spine entity | -A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no `@default`, no validators. +Only those last two rows make a measure other than `count`, or a dimension reached by `@via`, never null; every port's row type follows (no `| null`, no `?`, no `Optional`). A column carries its `@of` field's type-shaping attributes (`@currency`, `@values`, `@precision`, `@scale`, `@localTime`, ...) and nothing else: no field `@default`, no validators. ## Time grains @@ -50,13 +52,64 @@ A filter value `{ "now": "-P30D" }` (the current time plus a signed ISO-8601 dur ## Nulls and zeros - A `count` is `0` over nothing, never null. It counts rows whose `@of` column is not null; a tuple with any null component is not counted. -- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. +- A `sum` of nothing is **null**, not zero: no matching rows, or a filtered measure that matched none of a group's rows. So are `avg`, `min` and `max`. - A ratio is `numerator / NULLIF(denominator, 0)`: a zero denominator is **null**. - A report with no dimensions is one row for the whole table, and still one row over an empty table (counts `0`, sums and ratios null). +**Ask: is zero true here, or is null?** If "nothing to aggregate" means zero (revenue of a program nobody bought is `0`), declare `"@default": 0` on the measure (`revenue` in the example under "Rows with no facts"). If it means unknown (the average rating of a product with no ratings is not `0`), leave it null. Do not default in the client what the model can state once. + +- `@default` is an integer. Write `0`, not `0.0`: a fraction (`0.5`) is refused in every port, and `0.0` loads in TypeScript but is refused by C#, Java and Python. +- **Never on a `count`** (it is never null, so the default could never apply) and never on a `min` / `max` over a field that is not numeric (a date or a string): both are load errors. +- It applies whenever the measure would be null: nothing matched, every matched value was null, or a ratio's denominator is zero or null. The column is then never null in any port. +- **A ratio's operand carries its own `@default` into the ratio.** `revenue` defaulted to `0` over `buyers` reads `0` for a group with buyers and no revenue. The ratio's own `@default` covers a zero denominator. +- A filter on a defaulted measure sees the default: `?filter[revenue][eq]=0` matches the empty rows, `isNull=true` matches nothing. A sort on it is the same on every engine. + +## Rows with no facts: `@spine` + +A report's rows are the dimension tuples its `@from` rows have, so a per-program report has no row for a program nobody bought. **Ask: do I want the rows that have no facts?** If the page lists every program (a catalogue, a chart whose empty bars matter), add `@spine`: a to-one path from `@from`, written like `@via`, to the entity whose rows the report should have. + +```json +{ "metadata.root": { + "package": "acme::shop", + "children": [ + { "object.entity": { "name": "Program", "children": [ + { "source.rdb": { "@table": "programs" } }, + { "field.long": { "name": "id" } }, + { "field.string": { "name": "title" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } } ] } }, + { "object.entity": { "name": "Purchase", "children": [ + { "source.rdb": { "@table": "purchases" } }, + { "field.long": { "name": "id" } }, + { "field.long": { "name": "programId" } }, + { "field.currency": { "name": "amountCents" } }, + { "identity.primary": { "name": "id", "@fields": ["id"] } }, + { "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }, + { "relationship.association": { "name": "program", "@objectRef": "Program", "@cardinality": "one" } }, + { "dimension.attribute": { "name": "programId", "@of": "Program.id", "@via": "Purchase.program" } }, + { "dimension.attribute": { "name": "programTitle", "@of": "Program.title", "@via": "Purchase.program" } }, + { "measure.aggregate": { "name": "purchases", "@agg": "count", "@of": "Purchase.id" } }, + { "measure.aggregate": { "name": "revenue", "@agg": "sum", "@of": "Purchase.amountCents", "@default": 0 } } ] } }, + { "object.report": { "name": "ProgramSales", "@from": "Purchase", "@spine": "Purchase.program", + "@dimensions": ["programId", "programTitle"], "@measures": ["purchases", "revenue"], + "children": [ { "source.rdb": { "@kind": "view", "@view": "v_program_sales" } } ] } } + ] +}} +``` + +A program with no purchases has a row: `purchases` `0`, `revenue` `0`. + +- **Every dimension goes through the spine.** Each listed dimension needs an `@via` that begins with the spine's hops (`programId` is `Program.id` via `Purchase.program`). A dimension on the fact row (`Purchase.programId` with no `@via`, the purchase month) is a load error: a program with no purchases has no value for it. +- **Write the same hops as the spine.** `Purchase.program` (relationship) and `Purchase.fkProgram` (reference) are the same join, but a spine written one way and a dimension the other is refused. +- List at least one dimension, and list the spine entity's key (`Program.id`) to get exactly one row per program; two programs listed only by a shared title are one row. +- On a row with no facts, a `count` is `0` and every other measure is null unless it declares `@default`. +- The report's `@segment` and `@filter` still scope the facts and never remove a program's row. They name `@from`'s fields only; to show only some programs, list the column as a dimension and filter it on the request. +- A fact whose reference is null, or matches nothing, is in no row of a `@spine` report. +- Every hop needs an `identity.reference`, and the spine entity (and every entity on the way) needs a table of its own: an abstract entity or a TPH subtype fails `meta migrate`. +- The empty rows are listed and counted on the route; `?filter[purchases][gt]=0` hides them on request, so one report serves both pages. + ## Joins: a dimension through a required reference drops rows -A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. +A dimension reached by `@via` joins like a projection does: a required belongs-to foreign key joins `INNER`, anything else `LEFT OUTER`. So **a fact row whose required reference matches no row is left out of that report** (a dimension you do not list adds no join). That is the existing projection rule, not a reporting special case. In a `@spine` report every join is `LEFT OUTER`. **Each `@via` hop needs a foreign key the model declares**: an `identity.reference` between the two entities, for example `{ "identity.reference": { "name": "fkProgram", "@fields": ["programId"], "@references": "Program" } }` on the entity that holds `programId`. A `relationship.*` with `@cardinality: one` and no reference behind it loads, and then `meta migrate` fails with an error naming the hop. @@ -67,6 +120,7 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t | View created by | `meta migrate` | `meta migrate` | you: see below | | A ratio or `avg` of `2` over `3` | `0.66666666666666666667` | `0.6666666666666666` | `0.6667` | | `decimal` | `NUMERIC` | none: `avg`, a ratio and a `sum` of a decimal column are `REAL` | `DECIMAL` | +| A ratio reading its `@default: 0` | `0` | `0.0` (the default is written `0.0` so the `REAL` column keeps one storage class) | `0.0000` | | Instants | `TIMESTAMPTZ` | ISO-8601 text | `DATETIME(3)`, read as the UTC wall clock | **MySQL owns its own DDL.** `meta migrate` never targets MySQL, so you create the view yourself: `buildReportViews(root, { dialect: "mysql" })` (`@metaobjectsdev/codegen-ts`) returns each view-backed report's body. That function is in the TypeScript package, so the MySQL view SQL comes from a TypeScript toolchain whatever language your application is in; the recipe showing the loop ships as the MySQL guide in the `metaobjects-codegen` skill's TypeScript stacks only. It skips a report whose source is `@unmanaged`. @@ -77,6 +131,11 @@ A dimension reached by `@via` joins like a projection does: a required belongs-t - **An empty `in` list in a filter is refused** at `meta migrate`, naming the report and the field. - **An abstract view-backed report, or one whose source `@kind` is `materializedView`, `storedProc` or `tableFunction`, is not served by any port and gets no C# row class and no Kotlin table object.** The TypeScript, Java and Python runtimes still read whatever relation the source names (fine for a materialized view you created, a database error for a routine). - **Do not group by a `field.object`.** A dimension over one, or over a field carrying `@objectRef`, loads everywhere but is refused by name by Java OMDB on read and by Java, Kotlin and C# `gen`; only TypeScript and Python read and serve it (as parsed JSON, neither filterable nor sortable). Group by a scalar field. +- **One `@spine` per report.** Two independent spines (every program against every customer) cannot be written. Entities to-one beyond the spine are ordinary dimensions. +- **No row scope on the spine entity** (only published programs): a report `@filter` names `@from`'s fields. List the column as a dimension and filter it on the request. +- **No fact-row dimension beside a spine** (every program, split by purchase month), and no dimension over the reference column itself (`Purchase.programId`). Declare it over the spine entity's key instead. Zero-filled time buckets are not available yet. +- **No join on a natural key, no anti-join, no report over two fact tables.** A `@spine` reaches its facts through a declared reference only: give the fact a reference to the spine entity. +- **`@default` is a whole number.** A fraction is `ERR_BAD_ATTR_VALUE` on the measure in every port (TypeScript's message says it is not an integer; C#, Java and Python use their integer-attribute message). `0.0` or `1.0` loads in TypeScript and is refused elsewhere, and `"zero"` on a `count` is two errors in TypeScript, C# and Python but one in Java. Write `0`. ## How a report is served @@ -90,7 +149,7 @@ A concrete report whose read source is `@kind: view` is served by every port's g - `<segment>` is the report name, snake_cased then pluralized, like any object: `InvoicesByMonth` is `/invoices_by_months`, `StoreTotals` is `/store_totals`. There is no report-specific spelling. - **Every derived field with filter operators is filterable and sortable**, dimension and measure alike, with its type's own operators. You cannot narrow that set: a report has no field to put `@filterable` on. A field of the `@from` entity that the report does not expose is refused with a `400`. -- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present; `currency` is integer minor units. +- A time dimension at `day` or coarser is `YYYY-MM-DD`; a null measure is `null` with the key present (a measure with `@default` is never null); `currency` is integer minor units. - **A decimal (`avg`, a ratio, a `sum` of a decimal) has no agreed JSON spelling**: each port sends its own (TypeScript a string). Do not compare one across backends as text. - `meta docs` writes a model page for every report (a sourceless one is marked "Not served") and an API page for a served one. diff --git a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md index eb54c6be2..8392d6051 100644 --- a/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md +++ b/fixtures/agent-context-conformance/ts-requirements/expected/.claude/skills/metaobjects-codegen/references/typescript-mysql.md @@ -94,11 +94,14 @@ migration); the loop will not. Pass `columnNamingStrategy` to match your tables' column names (the default is `snake_case`). The bodies are valid under MySQL's default `sql_mode`, `ONLY_FULL_GROUP_BY` included, and a change to a report means a new `CREATE OR REPLACE VIEW` (or `DROP` and `CREATE`) in your -migrations; nothing diffs the live view for you. Two things differ from Postgres and SQLite: +migrations; nothing diffs the live view for you. A `@spine` report's body is valid as +emitted too, and a measure with `@default` is a `COALESCE` whose column MySQL reports +`NOT NULL`. Two things differ from Postgres and SQLite: - **Ratios and averages have four fractional digits by default.** MySQL divides to `div_precision_increment` digits, so a ratio of 2 to 3 is `0.6667` (Postgres returns `0.66666666666666666667`, SQLite `0.6666666666666666`), and a ratio of 3 to 4 is `0.7500`. + A ratio that reads its `@default: 0` is `0.0000`. - **`DATETIME` values are read as the UTC wall clock.** A `DATETIME(3)` column carries no zone, so every time grain and every relative-date window (`{ "now": "-P30D" }`, evaluated with `UTC_TIMESTAMP(3)` when the view is queried) treats the stored value as UTC. That matches diff --git a/fixtures/api-contract-conformance/README.md b/fixtures/api-contract-conformance/README.md index ed40981d3..9ee39956f 100644 --- a/fixtures/api-contract-conformance/README.md +++ b/fixtures/api-contract-conformance/README.md @@ -53,7 +53,7 @@ fixtures/api-contract-conformance/ ``` Beside the 31 core scenarios above sit six **sub-corpora**, each a directory with its -own `meta.json`, `seed.json`, `scenarios/` and `README.md` (78 scenarios in all): +own `meta.json`, `seed.json`, `scenarios/` and `README.md` (82 scenarios in all): | Directory | Scenarios | Gates | Lanes | |---|---|---|---| @@ -61,8 +61,8 @@ own `meta.json`, `seed.json`, `scenarios/` and `README.md` (78 scenarios in all) | `m2m/` | 9 | M:N traversal, and TPH x M:N together | both | | `jsonb/` | 2 | typed value-object columns | both | | `write-through/` | 2 | an entity that writes a table and reads a view | generated only | -| `projection/` | 11 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | -| `report/` | 13 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | +| `projection/` | 12 | a read-only `object.projection`: reads served, every write `405`; no declared identity means no `/{id}` route; a key not named `id` addresses the row; decimal and float filters. Its `docs-routes.json` pins the routes each port documents | generated only | +| `report/` | 16 | a view-backed `object.report` (FR-044): list, filter, sort and paging on derived fields, `POST` is `405`, no `/{id}` route; a `@spine` report keeps a row with no facts, and a measure with `@default` filters and sorts as its default. Also carries `schema.postgres.sql`, the TypeScript-produced table and views the C# lane executes | generated only | `meta.json` declares a single canonical `Author` entity in the `acme::blog` package: From cd91e2549b77a8ef26bb7a50303f07f89f2118eb Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Fri, 9 Oct 2026 23:59:49 -0400 Subject: [PATCH 17/18] =?UTF-8?q?fix(reporting):=20review=20fixes=20?= =?UTF-8?q?=E2=80=94=20null-group=20wording,=20@sql=20shape=20note,=20fail?= =?UTF-8?q?-closed=20spine=20check?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Without @spine a fact row with a null or unmatched reference still counts: it falls in a null group through a nullable (LEFT OUTER) reference and is dropped through a required (INNER) one. Corrected in the CHANGELOG, reporting.md and the spec's R8. reporting.md also notes that an @sql or @unmanaged report keeps the derived read shape, so a defaulted measure and a @spine key are non-null in every port. extractReportSpec now throws on an unresolvable spine-chain entity instead of skipping the no-table and TPH refusals. Comment-only tidy-ups in the Kotlin harness, the C# reporting passes and the Python validator. --- CHANGELOG.md | 3 ++- docs/features/reporting.md | 11 ++++++++--- .../specs/2026-10-02-fr-044-core-reporting-design.md | 6 ++++-- .../MetaObjects/Loader/ValidationPasses.Reporting.cs | 6 +++--- .../generated/GeneratedReportControllerHarness.kt | 6 +++--- .../src/metaobjects/loader/validate_reporting.py | 1 - .../codegen-ts/src/projection/extract-report-spec.ts | 9 ++++++++- 7 files changed, 28 insertions(+), 14 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 5658b8895..067b42218 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -241,7 +241,8 @@ it until 1.1 ships._ row per spine row; (2) `@segment` and `@filter` on a `@spine` report scope the facts and never remove a spine row (they sit in the join condition; there is no `WHERE`); (3) a fact row whose reference is null or matches nothing is in no row of a `@spine` report (without `@spine` such - rows still form a null group); (4) a ratio's operand carries its own `@default` into the ratio, + a row still counts: it falls in a null group through a nullable reference, a `LEFT OUTER` + join, and is dropped through a required one, an `INNER` join); (4) a ratio's operand carries its own `@default` into the ratio, so `revenue` defaulted to `0` over `buyers` reads `0`, not null, and the ratio's own `@default` then covers a zero denominator. New load rules, gated by 10 new shared conformance fixtures (37 for reporting, 374 in all): a `@spine` that is not a to-one path from `@from`, a `@spine` diff --git a/docs/features/reporting.md b/docs/features/reporting.md index 567262129..50250815b 100644 --- a/docs/features/reporting.md +++ b/docs/features/reporting.md @@ -211,6 +211,10 @@ and the one every runtime reads. | the same, plus `@unmanaged: true` | `meta migrate` never creates or drops it (you or a migration tool own the DDL), but the runtime still reads it through the shape below. | | `@kind: materializedView`, `storedProc` or `tableFunction` | `meta migrate` skips it, as for a projection. | +A report whose view body is your own SQL (`@sql`) or that you manage yourself (`@unmanaged`) +still gets the derived read shape below, so a defaulted measure and a `@spine` key column are +typed non-null in every port. Your SQL must keep them non-null (for example with `COALESCE`). + A **derived** report view (no `@sql`) whose `@from` entity has no table (it is abstract, or declares no writable `source.rdb`) fails `meta migrate` with an error naming the report and the entity, rather than emitting a view over a table that does not exist. A report with an `@sql` @@ -335,8 +339,9 @@ program. `@from` are aggregated. A program whose purchases are all out of scope keeps its row, with a count of `0`. Both still name fields of `@from`. - **A fact row with no spine row is in no row.** A purchase whose `programId` is null, or - matches no program, has no spine row to sit in. Without `@spine`, those rows form a null - group. + matches no program, has no spine row to sit in. Without `@spine`, such a row still counts: it + falls in a null group through a nullable reference (`LEFT OUTER` join) and is dropped through + a required one (`INNER` join; see [Dimensions, time grains and joins](#dimensions-time-grains-and-joins)). - **A row with no facts** reads `0` for a `count`, and null for a `sum`, `avg`, `min`, `max` and a ratio, unless the measure declares `@default` (next section). This holds even for a measure whose `@filter` is `isNull: true` on a fact column: the empty row's `@of` column is @@ -672,7 +677,7 @@ row, and a `SUM` over the result would double-count. A to-one hop cannot fan out | **M5** | An `@segment` that names no `segment` child of the owning entity. | | **M6** | A `measure.ratio` whose `@numerator` or `@denominator` names no measure of the entity, or names one that is not a `measure.aggregate`. | | **M7** | A `@default` on an `@agg: count`. A count is never null, so the default could never apply. | -| **M8** | A `@default` on an `@agg: min` or `max` over a field that is not numeric (M4's set). | +| **M8** | A `@default` on an `@agg: min` or `max` over a field that is not numeric (not one of `int, long, double, float, decimal, currency`). | A measure that breaks several of M1 to M4 reports only the first, in that order, so one mistake gives one error. M7 and M8 run only when none of M1 to M4 fired. diff --git a/docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md b/docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md index 141be1286..6dce7b05f 100644 --- a/docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md +++ b/docs/superpowers/specs/2026-10-02-fr-044-core-reporting-design.md @@ -405,8 +405,10 @@ tuple among **that entity's** rows, whether or not any row of `@from` refers to (unless the measure declares R9's `@default`), and a ratio is computed from its operands as they read. - **A fact row whose reference is null, or matches no spine row, is in no row of the - report.** Without `@spine` such rows form a null group; with it there is no spine row to - hold them. + report.** Without `@spine` such a row still counts: it falls in a null group through a + nullable reference (`LEFT OUTER` join) and is dropped through a required one (`INNER` join; + see "Dimensions, time grains and joins" in `docs/features/reporting.md`). With `@spine` there + is no spine row to hold it. - The grain is unchanged: one row per distinct dimension tuple. To get exactly one row per row of the spine entity, list a dimension over its identity (`programId` above). - One spine per report. diff --git a/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs b/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs index 0acbcf8d3..b08659fb8 100644 --- a/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs +++ b/server/csharp/MetaObjects/Loader/ValidationPasses.Reporting.cs @@ -13,15 +13,15 @@ // @from fails R1 skips R2/R3/R6/R7, R8/R9 and its @filter; a report whose // @spine fails R8 skips R9, and R9 skips a dimension whose @via fails D2; an // invalid @dimensions/@measures item derives no report field for R6). +// - Each error's source is the offending node (the dimension / measure / segment / +// report, or for R4/R5 the declared child), so a conformance fixture's jsonPath +// points at it. // // A fractional measure @default: the TypeScript reference refuses it here, in // CheckMeasure, because its generic attr.int check lets a fraction through. This port's // generic attr-type check (ValidateAttrSchema, ValueMatchesType) already refuses it with // ERR_BAD_ATTR_VALUE on the measure node, so no second error is added here; CheckMeasure // only skips M7/M8 for it, as the reference does. -// - Each error's source is the offending node (the dimension / measure / segment / -// report, or for R4/R5 the declared child), so a conformance fixture's jsonPath -// points at it. // // Inheritance (ADR-0039): an entity's members are read through Children(), so a member // declared on an abstract base is validated against every entity that inherits it. diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/generated/GeneratedReportControllerHarness.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/generated/GeneratedReportControllerHarness.kt index d1e4b31f4..1c79b67bd 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/generated/GeneratedReportControllerHarness.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/generated/GeneratedReportControllerHarness.kt @@ -39,13 +39,13 @@ import kotlin.io.path.readText * gate excludes the framework-bound route tier in every port by design. * 4. Per scenario: a fresh in-memory H2 (PostgreSQL mode), `SchemaUtils.create(...)` on the * four generated report table objects, then the `reports` half of `seed.json`. - * 5. Serve the three controllers from one embedded Tomcat ([TomcatHost]). + * 5. Serve the four controllers from one embedded Tomcat ([TomcatHost]). * * No view SQL is written here (ADR-0015: TypeScript owns it). H2 creates each generated * `<R>Table` as a plain table that STANDS IN for the view, and the seed's `reports` half is * what the four views return for the seeded base tables. The base-table keys (`invoices`, - * `products`, `sales`) are the full-stack lanes' input; this seam lane does not read them. What this lane proves is the - * generated route over the generated binding, not the lowering. + * `products`, `sales`) are the full-stack lanes' input; this seam lane does not read them. + * What this lane proves is the generated route over the generated binding, not the lowering. */ @OptIn(org.jetbrains.kotlin.compiler.plugin.ExperimentalCompilerApi::class) class GeneratedReportControllerHarness( diff --git a/server/python/src/metaobjects/loader/validate_reporting.py b/server/python/src/metaobjects/loader/validate_reporting.py index ae1582b97..e6255910e 100644 --- a/server/python/src/metaobjects/loader/validate_reporting.py +++ b/server/python/src/metaobjects/loader/validate_reporting.py @@ -922,7 +922,6 @@ def claim(field_name: str, what: str) -> None: _check_spine_dimensions(root, report, from_, spine, terminal, err) # S1 / F2 — the report's row scope over @from. - # ADR-0039: resolving. filter_value = report.get_meta_attr(OBJECT_REPORT_ATTR_FILTER) if isinstance(filter_value, dict): diff --git a/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts b/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts index 8b805b6bd..6bf45432a 100644 --- a/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts +++ b/server/typescript/packages/codegen-ts/src/projection/extract-report-spec.ts @@ -300,7 +300,14 @@ function spinePathOf( } for (const step of path) { const entity = stepTarget(step, root); - if (entity === undefined) continue; // the walk resolved it, so it is in the root + if (entity === undefined) { + // The walk resolved this hop, so it should be in the root. Skipping it would skip the no-table + // and TPH refusals below for that entity, so fail closed instead. + throw new Error( + `${where} @spine '${spine}' reaches '${step.targetEntity}', which is not an entity in the model, so ` + + `its table cannot be checked. Fix the @spine path or declare the entity.`, + ); + } const head = `${where} @spine '${spine}' reaches '${entity.name}'`; if (entity.isAbstract || !hasWritableRdbSource(entity)) { throw new Error( From a636faec6e381a68d45b1b37a13e813d7e686bc8 Mon Sep 17 00:00:00 2001 From: Doug Mealing <doug@dougmealing.com> Date: Sat, 10 Oct 2026 00:25:07 -0400 Subject: [PATCH 18/18] no-mistakes(document): Updated AGENTS.md FR-044 description to reflect shipped @spine and @default attributes. --- AGENTS.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/AGENTS.md b/AGENTS.md index 36cdac57a..864f9ce98 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -82,7 +82,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken. - YAML / verify corpora green across the ports that ship those layers. - **Codegen-compile gate** (all five ports; a GATE, not a corpus — it has no fixtures of its own and no row in the matrix). Every corpus above gates BEHAVIOUR; none asks whether the emitted code BUILDS, which is how four "generated code does not compile" defects shipped in 1.0.4 with the whole matrix green — `gen` exits 0 in all four cases and the adopter's build is the first thing that disagrees. Each port generates from `fixtures/persistence-conformance/canonical/meta.fitness.json` (reused deliberately: a second kitchen sink would drift from the one the other corpora already maintain) and compiles the emitted tree with its real compiler — `ts.createProgram` / Roslyn / `javac` / `KotlinCompilation` / (Python, having no static compiler) importing the generated package plus `ruff` F821. **Every port excludes its framework-bound route tier** (TS `routesFile`, C# `RoutesGenerator`, Java `SpringControllerGenerator`, Kotlin `KotlinSpringControllerGenerator`): those imports are not on an in-memory compile's classpath and stubbing them drowns the signal, so that tier is proven by the api-contract integration lane instead. One cross-port rule, not four local concessions. Found 5 further real defects on first run. Boundary detail: `docs/CONFORMANCE.md` → "Split coverage". -**Key cross-language features shipped:** FR5 family (a/b/c/d/e + WARN envelope-shape — actionable loader errors per ADR-0009); FR-003 (Java RDB runtime persistence + projections; schema migrations are TS-only — the Java migration engine was removed); FR-006 (template.output parser-on-receipt codegen per ADR-0010 in all 5 ports); FR-008 + FR-009 (cross-port REST API contract + the nine filter operators); FR-018 (M:N relationship codegen in all 5 ports — entity navigation + idiomatic ORM wiring [Drizzle m2m / EF Core `UsingEntity` / Spring repo+JPA / Exposed / Pydantic+route as the SQLAlchemy-secondary equivalent] + REST traversal `GET /<source-plural>/{id}/<relation>` + Tier-2 docs, gated by the shared api-contract m2m corpus in both lanes + persistence-conformance; the TanStack M:N client hook is a deferred client-ergonomics follow-up); SP-H (field-subtype end-to-end hardening: every concrete `field.*` subtype write+read round-trips cross-port via the persistence `op: roundtrip` gate; cut `field.byte`/`field.short`/`field.class` non-functional stubs; cross-port filter-op reconciliation for uuid/currency); source v2 paradigm (ADR-0007); metadata-ktx Kotlin facade; per-target output directories (TS codegen); FR-044 Plan 1 (the reporting vocabulary — `dimension.attribute`/`dimension.time`, `measure.aggregate`/`measure.ratio`, `segment.filter`, `object.report`, plus the relative-date filter value — registered and loader-validated in all five ports behind 27 conformance fixtures + the `codegen-noop` corpus; a report that declares a read-only `source.rdb @kind: view` is lowered to a SQL view by `meta migrate` and read by every port (Plan 2), and served by a generated read-only list route in every port, gated by the api-contract `report/` sub-corpus (Plan 3); no client hook, grid or form is generated for a report yet, see [docs/features/reporting.md](docs/features/reporting.md)). +**Key cross-language features shipped:** FR5 family (a/b/c/d/e + WARN envelope-shape — actionable loader errors per ADR-0009); FR-003 (Java RDB runtime persistence + projections; schema migrations are TS-only — the Java migration engine was removed); FR-006 (template.output parser-on-receipt codegen per ADR-0010 in all 5 ports); FR-008 + FR-009 (cross-port REST API contract + the nine filter operators); FR-018 (M:N relationship codegen in all 5 ports — entity navigation + idiomatic ORM wiring [Drizzle m2m / EF Core `UsingEntity` / Spring repo+JPA / Exposed / Pydantic+route as the SQLAlchemy-secondary equivalent] + REST traversal `GET /<source-plural>/{id}/<relation>` + Tier-2 docs, gated by the shared api-contract m2m corpus in both lanes + persistence-conformance; the TanStack M:N client hook is a deferred client-ergonomics follow-up); SP-H (field-subtype end-to-end hardening: every concrete `field.*` subtype write+read round-trips cross-port via the persistence `op: roundtrip` gate; cut `field.byte`/`field.short`/`field.class` non-functional stubs; cross-port filter-op reconciliation for uuid/currency); source v2 paradigm (ADR-0007); metadata-ktx Kotlin facade; per-target output directories (TS codegen); FR-044 (the reporting vocabulary — `dimension.attribute`/`dimension.time`, `measure.aggregate`/`measure.ratio`, `segment.filter`, `object.report`, `@spine` on a report to take rows from a to-one dimension entity, `@default` on a measure for empty aggregates, plus the relative-date filter value — registered and loader-validated in all five ports; a report that declares a read-only `source.rdb @kind: view` is lowered to a SQL view by `meta migrate` and read by every port, and served by a generated read-only list route in every port, gated by the api-contract `report/` sub-corpus; no client hook, grid or form is generated for a report yet, see [docs/features/reporting.md](docs/features/reporting.md)). **Latest release: 1.0.13** (2026-10-03) — npm `1.0.13`, PyPI `1.0.13`, NuGet `1.0.13`, Maven Central `8.0.13`. A PATCH: an already-plural entity name (`Stats`, `Settings`) no longer doubles in API-surface names (REST paths, hooks, finders, DbSets) in any port, while default physical table names stay frozen on the old rule; two entities that would share one API name are now a generation error; two Kotlin controller compile fixes (`field.inet` filter ops, `@dbColumnType: uuid` on a string field). Gated by a private `1.0.13-rc.1` build on the adopter estate (`rc-gate.sh` 7/7) and a full `--strict-toolchains` local CI run. The previous release, 1.0.12 (2026-10-02), added `fmt` in every CLI, a deprecated-reference `verify` advisory, the `onLocate` extract hook, and an Exposed 1.x Kotlin output mode.