diff --git a/AGENTS.md b/AGENTS.md index 6c1478e0a..21c1d119d 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -73,7 +73,7 @@ PyPI has had no product change since `0.25.0` — nothing is broken. **Cross-port conformance corpora** (every port runs the shared corpus): - Metamodel: `fixtures/conformance/` (374 fixtures; 29 shared corpora in total — per-corpus counts + the corpus x port matrix live in `docs/CONFORMANCE.md`). TS / C# / Java / Python all green. - Render: `fixtures/render-conformance/`. TS / C# / Java / Kotlin / Python byte-identical. -- Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) +- Persistence: `fixtures/persistence-conformance/`. **Query** scenarios run on every port (TS / C# / Java / Kotlin / Python), each provisioning its test DB by executing the committed, TS-produced `canonical/schema.postgres.sql` (Postgres only — Derby dropped for the cross-port query corpus, ADR-0015; the FR-044 `report-*` scenarios also run on SQLite, MySQL and D1 where a port has the engine, see [docs/CONFORMANCE.md](docs/CONFORMANCE.md)). The **migration** scenarios are exercised by **TS only** (TS owns schema migrations). **The corpus now gates WRITES, not just reads (SP-H):** an `op: roundtrip` scenario type INSERTs through each port's runtime/ORM write codec (NOT raw SQL), reads the row back, and asserts the wire-normalized value. The `AllTypes` entity (`roundtrip-all-types.yaml`) carries one field of **every** persistable `field.*` subtype — string/int/long/double/float/decimal/boolean/date/time/timestamp(+tz)/currency/enum/uuid/object — plus an **array-of-VO** `field.object @isArray @storage:jsonb` column (`labels`, written as 2-element / empty-`[]` / single-element arrays across the three rows) — so every subtype write+read (incl. the array-of-value-object jsonb codec) round-trips through every port against Testcontainers PG. (`field.byte`/`field.short`/`field.class` were cut as non-functional registration-only stubs — the matrix tracks only genuinely-supported subtypes; see `fixtures/registry-conformance/README.md` → "Per-subtype write-round-trip matrix".) - API-contract: `fixtures/api-contract-conformance/`. TS / C# / Java / Kotlin / Python all green — each port runs **two lanes**: a hand-rolled reference server AND its **generated** API artifact booted over HTTP (the deployed controller/routes; TS+C# full-stack vs Testcontainers PG, Java/Kotlin/Python generated controller + in-memory repo behind the consumer seam). The generated fan-out found 10 real deployment bugs golden snapshots missed. Three sub-corpora run the **generated lane only, on all five ports** — `write-through/`, `projection/` (F22: a view-only `object.projection` serves GET list + GET by id and answers every write verb with `405 {"error": "method_not_allowed"}`) and `report/` (FR-044: a view-backed `object.report` serves GET list, answers `POST` with that same `405`, and mounts no `/{id}` route). That is deliberate, not a gap: what is under test is whether a port's GENERATOR emits those routes, and a hand-rolled reference server would answer every scenario by construction. The `m2m/` sub-corpus also gates **TPH x M:N together** (base-declared, subtype-declared, abstract-mid-declared, a non-subtype source onto a subtype TARGET, and the cross-subtype source id answering `200 []`) — the two corpora were originally built disjoint (`tph/` had no relationships, `m2m/` no discriminators), which is precisely how that defect class survived. - YAML / verify corpora green across the ports that ship those layers. - **Codegen-compile gate** (all five ports; a GATE, not a corpus — it has no fixtures of its own and no row in the matrix). Every corpus above gates BEHAVIOUR; none asks whether the emitted code BUILDS, which is how four "generated code does not compile" defects shipped in 1.0.4 with the whole matrix green — `gen` exits 0 in all four cases and the adopter's build is the first thing that disagrees. Each port generates from `fixtures/persistence-conformance/canonical/meta.fitness.json` (reused deliberately: a second kitchen sink would drift from the one the other corpora already maintain) and compiles the emitted tree with its real compiler — `ts.createProgram` / Roslyn / `javac` / `KotlinCompilation` / (Python, having no static compiler) importing the generated package plus `ruff` F821. **Every port excludes its framework-bound route tier** (TS `routesFile`, C# `RoutesGenerator`, Java `SpringControllerGenerator`, Kotlin `KotlinSpringControllerGenerator`): those imports are not on an in-memory compile's classpath and stubbing them drowns the signal, so that tier is proven by the api-contract integration lane instead. One cross-port rule, not four local concessions. Found 5 further real defects on first run. Boundary detail: `docs/CONFORMANCE.md` → "Split coverage". diff --git a/bun.lock b/bun.lock index 1e6184935..38b3d598a 100644 --- a/bun.lock +++ b/bun.lock @@ -317,6 +317,7 @@ "@types/node": "^22.5.4", "@types/pg": "^8.11.6", "bun-types": "^1.1.27", + "miniflare": "4.20251210.0", "mysql2": "^3.11.0", "typescript": "^5.6.2", }, @@ -550,6 +551,18 @@ "@bramus/specificity": ["@bramus/specificity@2.4.3", "", { "dependencies": { "css-tree": "^3.2.1" }, "bin": { "specificity": "bin/cli.js" } }, "sha512-0GEz8qJWokkUVYQYRukmkSHIFNoBH8QTreeHrAFbvz1XFYc3Y+puhMs8PQ5x8GNugiC2l5jvDBXKLbBIByeSBw=="], + "@cloudflare/workerd-darwin-64": ["@cloudflare/workerd-darwin-64@1.20251210.0", "", { "os": "darwin", "cpu": "x64" }, "sha512-Nn9X1moUDERA9xtFdCQ2XpQXgAS9pOjiCxvOT8sVx9UJLAiBLkfSCGbpsYdarODGybXCpjRlc77Yppuolvt7oQ=="], + + "@cloudflare/workerd-darwin-arm64": ["@cloudflare/workerd-darwin-arm64@1.20251210.0", "", { "os": "darwin", "cpu": "arm64" }, "sha512-Mg8iYIZQFnbevq/ls9eW/eneWTk/EE13Pej1MwfkY5et0jVpdHnvOLywy/o+QtMJFef1AjsqXGULwAneYyBfHw=="], + + "@cloudflare/workerd-linux-64": ["@cloudflare/workerd-linux-64@1.20251210.0", "", { "os": "linux", "cpu": "x64" }, "sha512-kjC2fCZhZ2Gkm1biwk2qByAYpGguK5Gf5ic8owzSCUw0FOUfQxTZUT9Lp3gApxsfTLbbnLBrX/xzWjywH9QR4g=="], + + "@cloudflare/workerd-linux-arm64": ["@cloudflare/workerd-linux-arm64@1.20251210.0", "", { "os": "linux", "cpu": "arm64" }, "sha512-2IB37nXi7PZVQLa1OCuO7/6pNxqisRSO8DmCQ5x/3sezI5op1vwOxAcb1osAnuVsVN9bbvpw70HJvhKruFJTuA=="], + + "@cloudflare/workerd-windows-64": ["@cloudflare/workerd-windows-64@1.20251210.0", "", { "os": "win32", "cpu": "x64" }, "sha512-Uaz6/9XE+D6E7pCY4OvkCuJHu7HcSDzeGcCGY1HLhojXhHd7yL52c3yfiyJdS8hPatiAa0nn5qSI/42+aTdDSw=="], + + "@cspotcode/source-map-support": ["@cspotcode/source-map-support@0.8.1", "", { "dependencies": { "@jridgewell/trace-mapping": "0.3.9" } }, "sha512-IchNf6dN4tHoMFIn/7OE8LWZ19Y6q/67Bmf6vnGREv8RSbBVb9LPJxEcnwrcwX6ixSvaiGoomAUvu4YSxXrVgw=="], + "@csstools/color-helpers": ["@csstools/color-helpers@6.1.2", "", {}, "sha512-grhRy3OKmniaAEKXMjua5z/EODX0MSqBGjunw8+j/3HQjOnahs2AGhvEOIYVUWcU6ScApbhLhVrQTX8XqrMrow=="], "@csstools/css-calc": ["@csstools/css-calc@3.4.3", "", { "peerDependencies": { "@csstools/css-parser-algorithms": "^4.0.2", "@csstools/css-tokenizer": "^4.0.2" } }, "sha512-iex20d8CHVkyvg6B7UKV7uHnI2Bqo9g+EFfT9E0y+GvTvhZ/DwONJ+9aKb1dlqm0ZiGsL5RXjp0fCoJYnkeDjA=="], @@ -564,6 +577,8 @@ "@electric-sql/pglite": ["@electric-sql/pglite@0.5.8", "", {}, "sha512-n9tsbUOhwx2epK1V0ZG9Ar4SHWUju04dhmzZXiSBXwBoleOvIfals33NAaWgagQVAL4Rbvx/Ptsu3P+pA09f6Q=="], + "@emnapi/runtime": ["@emnapi/runtime@1.11.3", "", { "dependencies": { "tslib": "^2.4.0" } }, "sha512-Xz4Tpyki7XyrpbUK1jR1AhdAdaXyhhY4lZ3neLodmhpuWfy2PAQN5B46sAiU4liOXGLkHypn/qU+jvfWSCYYLA=="], + "@exodus/bytes": ["@exodus/bytes@1.16.0", "", { "peerDependencies": { "@noble/hashes": "^1.8.0 || ^2.0.0" }, "optionalPeers": ["@noble/hashes"] }, "sha512-IcpW84uEn3N7ETtNZMlxKhfl6Pec8rUNGOTBtWbK1FKhJxIFAptZyVrvVRVBimAJxJCgc3PxepxkdWWG4DVzfA=="], "@fastify/ajv-compiler": ["@fastify/ajv-compiler@4.0.6", "", { "dependencies": { "ajv": "^8.12.0", "ajv-formats": "^3.0.1", "fast-uri": "^4.0.0" } }, "sha512-NtuzM0SfaMJbGlnjr9LWQUN5LzgSrbB8tf/wRZNas+4E1O/Nmzl53e7ruT61HDZyRCJGC6FxIogmNZO1c5ETBA=="], @@ -588,6 +603,44 @@ "@iarna/toml": ["@iarna/toml@2.2.5", "", {}, "sha512-trnsAYxU3xnS1gPHPyU961coFyLkh4gAD/0zQ5mymY4yOZ+CYvsPqUbOFSw0aDM4y0tV7tiFxL/1XfXPNC6IPg=="], + "@img/sharp-darwin-arm64": ["@img/sharp-darwin-arm64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-arm64": "1.0.4" }, "os": "darwin", "cpu": "arm64" }, "sha512-UT4p+iz/2H4twwAoLCqfA9UH5pI6DggwKEGuaPy7nCVQ8ZsiY5PIcrRvD1DzuY3qYL07NtIQcWnBSY/heikIFQ=="], + + "@img/sharp-darwin-x64": ["@img/sharp-darwin-x64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-darwin-x64": "1.0.4" }, "os": "darwin", "cpu": "x64" }, "sha512-fyHac4jIc1ANYGRDxtiqelIbdWkIuQaI84Mv45KvGRRxSAa7o7d1ZKAOBaYbnepLC1WqxfpimdeWfvqqSGwR2Q=="], + + "@img/sharp-libvips-darwin-arm64": ["@img/sharp-libvips-darwin-arm64@1.0.4", "", { "os": "darwin", "cpu": "arm64" }, "sha512-XblONe153h0O2zuFfTAbQYAX2JhYmDHeWikp1LM9Hul9gVPjFY427k6dFEcOL72O01QxQsWi761svJ/ev9xEDg=="], + + "@img/sharp-libvips-darwin-x64": ["@img/sharp-libvips-darwin-x64@1.0.4", "", { "os": "darwin", "cpu": "x64" }, "sha512-xnGR8YuZYfJGmWPvmlunFaWJsb9T/AO2ykoP3Fz/0X5XV2aoYBPkX6xqCQvUTKKiLddarLaxpzNe+b1hjeWHAQ=="], + + "@img/sharp-libvips-linux-arm": ["@img/sharp-libvips-linux-arm@1.0.5", "", { "os": "linux", "cpu": "arm" }, "sha512-gvcC4ACAOPRNATg/ov8/MnbxFDJqf/pDePbBnuBDcjsI8PssmjoKMAz4LtLaVi+OnSb5FK/yIOamqDwGmXW32g=="], + + "@img/sharp-libvips-linux-arm64": ["@img/sharp-libvips-linux-arm64@1.0.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-9B+taZ8DlyyqzZQnoeIvDVR/2F4EbMepXMc/NdVbkzsJbzkUjhXv/70GQJ7tdLA4YJgNP25zukcxpX2/SueNrA=="], + + "@img/sharp-libvips-linux-s390x": ["@img/sharp-libvips-linux-s390x@1.0.4", "", { "os": "linux", "cpu": "s390x" }, "sha512-u7Wz6ntiSSgGSGcjZ55im6uvTrOxSIS8/dgoVMoiGE9I6JAfU50yH5BoDlYA1tcuGS7g/QNtetJnxA6QEsCVTA=="], + + "@img/sharp-libvips-linux-x64": ["@img/sharp-libvips-linux-x64@1.0.4", "", { "os": "linux", "cpu": "x64" }, "sha512-MmWmQ3iPFZr0Iev+BAgVMb3ZyC4KeFc3jFxnNbEPas60e1cIfevbtuyf9nDGIzOaW9PdnDciJm+wFFaTlj5xYw=="], + + "@img/sharp-libvips-linuxmusl-arm64": ["@img/sharp-libvips-linuxmusl-arm64@1.0.4", "", { "os": "linux", "cpu": "arm64" }, "sha512-9Ti+BbTYDcsbp4wfYib8Ctm1ilkugkA/uscUn6UXK1ldpC1JjiXbLfFZtRlBhjPZ5o1NCLiDbg8fhUPKStHoTA=="], + + "@img/sharp-libvips-linuxmusl-x64": ["@img/sharp-libvips-linuxmusl-x64@1.0.4", "", { "os": "linux", "cpu": "x64" }, "sha512-viYN1KX9m+/hGkJtvYYp+CCLgnJXwiQB39damAO7WMdKWlIhmYTfHjwSbQeUK/20vY154mwezd9HflVFM1wVSw=="], + + "@img/sharp-linux-arm": ["@img/sharp-linux-arm@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-arm": "1.0.5" }, "os": "linux", "cpu": "arm" }, "sha512-JTS1eldqZbJxjvKaAkxhZmBqPRGmxgu+qFKSInv8moZ2AmT5Yib3EQ1c6gp493HvrvV8QgdOXdyaIBrhvFhBMQ=="], + + "@img/sharp-linux-arm64": ["@img/sharp-linux-arm64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-arm64": "1.0.4" }, "os": "linux", "cpu": "arm64" }, "sha512-JMVv+AMRyGOHtO1RFBiJy/MBsgz0x4AWrT6QoEVVTyh1E39TrCUpTRI7mx9VksGX4awWASxqCYLCV4wBZHAYxA=="], + + "@img/sharp-linux-s390x": ["@img/sharp-linux-s390x@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-s390x": "1.0.4" }, "os": "linux", "cpu": "s390x" }, "sha512-y/5PCd+mP4CA/sPDKl2961b+C9d+vPAveS33s6Z3zfASk2j5upL6fXVPZi7ztePZ5CuH+1kW8JtvxgbuXHRa4Q=="], + + "@img/sharp-linux-x64": ["@img/sharp-linux-x64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linux-x64": "1.0.4" }, "os": "linux", "cpu": "x64" }, "sha512-opC+Ok5pRNAzuvq1AG0ar+1owsu842/Ab+4qvU879ippJBHvyY5n2mxF1izXqkPYlGuP/M556uh53jRLJmzTWA=="], + + "@img/sharp-linuxmusl-arm64": ["@img/sharp-linuxmusl-arm64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linuxmusl-arm64": "1.0.4" }, "os": "linux", "cpu": "arm64" }, "sha512-XrHMZwGQGvJg2V/oRSUfSAfjfPxO+4DkiRh6p2AFjLQztWUuY/o8Mq0eMQVIY7HJ1CDQUJlxGGZRw1a5bqmd1g=="], + + "@img/sharp-linuxmusl-x64": ["@img/sharp-linuxmusl-x64@0.33.5", "", { "optionalDependencies": { "@img/sharp-libvips-linuxmusl-x64": "1.0.4" }, "os": "linux", "cpu": "x64" }, "sha512-WT+d/cgqKkkKySYmqoZ8y3pxx7lx9vVejxW/W4DOFMYVSkErR+w7mf2u8m/y4+xHe7yY9DAXQMWQhpnMuFfScw=="], + + "@img/sharp-wasm32": ["@img/sharp-wasm32@0.33.5", "", { "dependencies": { "@emnapi/runtime": "^1.2.0" }, "cpu": "none" }, "sha512-ykUW4LVGaMcU9lu9thv85CbRMAwfeadCJHRsg2GmeRa/cJxsVY9Rbd57JcMxBkKHag5U/x7TSBpScF4U8ElVzg=="], + + "@img/sharp-win32-ia32": ["@img/sharp-win32-ia32@0.33.5", "", { "os": "win32", "cpu": "ia32" }, "sha512-T36PblLaTwuVJ/zw/LaH0PdZkRz5rd3SmMHX8GSmR7vtNSP5Z6bQkExdSK7xGWyxLw4sUknBuugTelgw2faBbQ=="], + + "@img/sharp-win32-x64": ["@img/sharp-win32-x64@0.33.5", "", { "os": "win32", "cpu": "x64" }, "sha512-MpY/o8/8kj+EcnxwvrP4aTJSWw/aZ7JIGR4aBeZkZw5B7/Jn+tY9/VNwtcoGmdT7GfggGIU4kygOMSbYnOrAbg=="], + "@inquirer/ansi": ["@inquirer/ansi@2.0.9", "", {}, "sha512-0Ywt3S8tqM6TWUcsRlBFQ1x+X3/o8kktX0ZiKAnByYb9GFDknnOhse1PVtfiD2BXPI8ar+BWx9PV9R0lQIcygw=="], "@inquirer/checkbox": ["@inquirer/checkbox@5.2.6", "", { "dependencies": { "@inquirer/ansi": "^2.0.9", "@inquirer/core": "^12.0.4", "@inquirer/figures": "^2.0.9", "@inquirer/type": "4.2.0" }, "peerDependencies": { "@types/node": ">=18" }, "optionalPeers": ["@types/node"] }, "sha512-XkOardciF3tow2NUlY6/Qjk1QqYa+6WpmVLTpxXl97Uv4erczDEZ+xg1j6LqBf0ekcRSUiICQA4nQXHHkVKQ9w=="], @@ -630,7 +683,7 @@ "@jridgewell/sourcemap-codec": ["@jridgewell/sourcemap-codec@1.6.0", "", {}, "sha512-T7jf+5zgsZHwNJ4lvQ7/aezbyk0nNX+zJVWpmHA7VYsEx7a7qr5Rg5IbtJFqkgze5Y2sruq1RUY8Q837Od7iFw=="], - "@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="], + "@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.9", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.0.3", "@jridgewell/sourcemap-codec": "^1.4.10" } }, "sha512-3Belt6tdc8bPgAtbcmdtNJlirVoTmEb5e2gC94PnkwEW9jI6CAHUeoG85tjWP5WquqfavoMtMwiG4P926ZKKuQ=="], "@js-sdsl/ordered-map": ["@js-sdsl/ordered-map@4.4.2", "", {}, "sha512-iUKgm52T8HOE/makSxjqoWhe95ZJA1/G1sYsGev2JDKUSS14KAgg1LHb+Ba+IPow0xflbnSkOsZcO08C7w1gYw=="], @@ -710,6 +763,12 @@ "@pkgjs/parseargs": ["@pkgjs/parseargs@0.11.0", "", {}, "sha512-+1VkjdD0QBLPodGrJUeqarH8VAIvQODIbwh9XpP5Syisf7YoQgsJKPNFoqqLQlu+VQ/tVSshMR6loPMn8U+dPg=="], + "@poppinss/colors": ["@poppinss/colors@4.1.6", "", { "dependencies": { "kleur": "^4.1.5" } }, "sha512-H9xkIdFswbS8n1d6vmRd8+c10t2Qe+rZITbbDHHkQixH5+2x1FDGmi/0K+WgWiqQFKPSlIYB7jlH6Kpfn6Fleg=="], + + "@poppinss/dumper": ["@poppinss/dumper@0.6.5", "", { "dependencies": { "@poppinss/colors": "^4.1.5", "@sindresorhus/is": "^7.0.2", "supports-color": "^10.0.0" } }, "sha512-NBdYIb90J7LfOI32dOewKI1r7wnkiH6m920puQ3qHUeZkxNkQiFnXVWoE6YtFSv6QOiPPf7ys6i+HWWecDz7sw=="], + + "@poppinss/exception": ["@poppinss/exception@1.2.3", "", {}, "sha512-dCED+QRChTVatE9ibtoaxc+WkdzOSjYTKi/+uacHWIsfodVfpsueo3+DKpgU5Px8qXjgmXkSvhXvSCz3fnP9lw=="], + "@protobufjs/aspromise": ["@protobufjs/aspromise@1.1.2", "", {}, "sha512-j+gKExEuLmKwvz3OgROXtrJ2UG2x8Ch2YZUxahh+s1F2HZ+wAceUNLkvy6zKCPVRkU++ZWQrdxsUeQXmcg4uoQ=="], "@protobufjs/base64": ["@protobufjs/base64@1.1.2", "", {}, "sha512-AZkcAA5vnN/v4PDqKyMR5lx7hZttPDgClv83E//FMNhR2TMcLUhfRUBHCmSl0oi9zMgDDqRUJkSxO3wm85+XLg=="], @@ -730,8 +789,12 @@ "@sec-ant/readable-stream": ["@sec-ant/readable-stream@0.4.1", "", {}, "sha512-831qok9r2t8AlxLko40y2ebgSDhenenCatLVeW/uBtnHPyhHOvG0C7TvfgecV+wHzIm5KUICgzmVpWS+IMEAeg=="], + "@sindresorhus/is": ["@sindresorhus/is@7.2.0", "", {}, "sha512-P1Cz1dWaFfR4IR+U13mqqiGsLFf1KbayybWwdd2vfctdV6hDpUkgCY0nKOLLTMSoRd/jJNjtbqzf13K8DCCXQw=="], + "@sindresorhus/merge-streams": ["@sindresorhus/merge-streams@4.0.0", "", {}, "sha512-tlqY9xq5ukxTUZBmoOp+m61cqwQD5pHJtFY3Mn8CA8ps6yghLH/Hw8UPdqg4OLmFW3IFlcXnQNmo/dh8HzXYIQ=="], + "@speed-highlight/core": ["@speed-highlight/core@1.2.24", "", {}, "sha512-qeW2e1l78afw8VhRPfPQ1Gjj+KU5XFQ/OFV5ti6eTa9bruO7mJyZtA4vw0ofqmA3tKCkROE9xLk3VZoeRc98nw=="], + "@stryker-mutator/api": ["@stryker-mutator/api@9.6.1", "", { "dependencies": { "mutation-testing-metrics": "3.7.3", "mutation-testing-report-schema": "3.7.3", "tslib": "~2.8.0", "typed-inject": "~5.0.0" } }, "sha512-g8VNoFWQWbx0pdal3Vt8jVCZW+v3sc3gi94iI0GVtVgUGTqphAjJF6EAruPTx0lqvtonsaAxn5TD36hcG1d6Wg=="], "@stryker-mutator/core": ["@stryker-mutator/core@9.6.1", "", { "dependencies": { "@inquirer/prompts": "^8.0.0", "@stryker-mutator/api": "9.6.1", "@stryker-mutator/instrumenter": "9.6.1", "@stryker-mutator/util": "9.6.1", "ajv": "~8.18.0", "chalk": "~5.6.0", "commander": "~14.0.0", "diff-match-patch": "1.0.5", "emoji-regex": "~10.6.0", "execa": "~9.6.0", "json-rpc-2.0": "^1.7.0", "lodash.groupby": "~4.6.0", "minimatch": "~10.2.4", "mutation-server-protocol": "~0.4.0", "mutation-testing-elements": "3.7.3", "mutation-testing-metrics": "3.7.3", "mutation-testing-report-schema": "3.7.3", "npm-run-path": "~6.0.0", "progress": "~2.0.3", "rxjs": "~7.8.1", "semver": "^7.6.3", "source-map": "~0.7.4", "tree-kill": "~1.2.2", "tslib": "2.8.1", "typed-inject": "~5.0.0", "typed-rest-client": "~2.3.0" }, "bin": { "stryker": "bin/stryker.js" } }, "sha512-WMgnvf+Wyh/yiruhNZwc8w8DlzmmjXhPjSn5MR8RhAXzlnWji8TQrUYgBUkHk9bEgSaIlB3KZHm37iiU5Q2cLQ=="], @@ -786,6 +849,10 @@ "abstract-logging": ["abstract-logging@2.0.1", "", {}, "sha512-2BjRTZxTPvheOvGbBslFSYOUkr+SjPtOnrLP33f+VIWLzezQpZcqVg7ja3L4dBXmzzgwT+a029jRx5PCi3JuiA=="], + "acorn": ["acorn@8.14.0", "", { "bin": { "acorn": "bin/acorn" } }, "sha512-cl669nCJTZBsL97OF4kUQm5g5hC2uihk0NxY3WENAC0TYdILVkAyHymAntgxGkl7K+t0cXIrH5siy5S4XkFycA=="], + + "acorn-walk": ["acorn-walk@8.3.2", "", {}, "sha512-cjkyv4OtNCIeqhHrfS81QWXoCBPExR/J62oyEqepVw8WaQeSqpW2uhuLPh1m9eWhDuOo/jUXVTlifvesOWp/4A=="], + "ajv": ["ajv@8.18.0", "", { "dependencies": { "fast-deep-equal": "^3.1.3", "fast-uri": "^3.0.1", "json-schema-traverse": "^1.0.0", "require-from-string": "^2.0.2" } }, "sha512-PlXPeEWMXMZ7sPYOHqmDyCJzcfNrUr3fGNKtezX14ykXOEIvyK81d+qydx89KY5O71FKMPaQ2vBfBFI5NHR63A=="], "ajv-formats": ["ajv-formats@3.0.1", "", { "dependencies": { "ajv": "^8.0.0" } }, "sha512-8iUql50EUR+uUcdRQ3HDqa6EVyo3docL8g5WJ3FNcWmu62IbkGUue/pEyLBW8VGKKucTPgqeks4fIU1DA4yowQ=="], @@ -872,10 +939,14 @@ "cliui": ["cliui@8.0.1", "", { "dependencies": { "string-width": "^4.2.0", "strip-ansi": "^6.0.1", "wrap-ansi": "^7.0.0" } }, "sha512-BSeNnyus75C4//NQ9gQt1/csTXyo/8Sb+afLAkzAptFuMsod9HFokGNudZpi/oQV73hnVK+sR+5PVRMd+Dr7YQ=="], + "color": ["color@4.2.3", "", { "dependencies": { "color-convert": "^2.0.1", "color-string": "^1.9.0" } }, "sha512-1rXeuUUiGGrykh+CeBdu5Ie7OJwinCgQY0bc7GCRxy5xVHy+moaqkpL/jqQq0MtQOeYcrqEz4abc5f0KtU7W4A=="], + "color-convert": ["color-convert@2.0.1", "", { "dependencies": { "color-name": "~1.1.4" } }, "sha512-RRECPsj7iu/xb5oKYcsFHSppFNnsj/52OVTRKb4zP5onXwVF3zVmmToNcOfGC+CRDpfK/U584fMg38ZHCaElKQ=="], "color-name": ["color-name@1.1.4", "", {}, "sha512-dOy+3AuW3a2wNbZHIuMZpTcgjGuLU/uBL/ubcZF9OXbDo8ff4O8yVp5Bf0efS8uEoYo5q4Fx7dY9OgQGXgAsQA=="], + "color-string": ["color-string@1.9.1", "", { "dependencies": { "color-name": "^1.0.0", "simple-swizzle": "^0.2.2" } }, "sha512-shrVawQFojnZv6xM40anx4CkoDP+fZsw/ZerEMsW/pyzsRbElpsL/DBVW7q3ExxwusdNXI3lXpuhEZkzs8p5Eg=="], + "commander": ["commander@14.0.3", "", {}, "sha512-H+y0Jo/T1RZ9qPP4Eh1pkcQcLRglraJaSLoyOtHxu6AapkjWVCy2Sit1QQ4x3Dng8qDlSsZEet7g5Pq06MvTgw=="], "compress-commons": ["compress-commons@6.0.2", "", { "dependencies": { "crc-32": "^1.2.0", "crc32-stream": "^6.0.0", "is-stream": "^2.0.1", "normalize-path": "^3.0.0", "readable-stream": "^4.0.0" } }, "sha512-6FqVXeETqWPoGcfzrXb37E50NP0LXT8kAMu5ooZayhWWdgEY4lBEEcbQNXtkuKQsGduxiIcI4gOTsxTmuq/bSg=="], @@ -942,6 +1013,8 @@ "entities": ["entities@8.1.0", "", {}, "sha512-kxL7msIffSuh9aaFAMD7rxAIuTRMAHMeBtgHW2yUdWw732ZNh4MehkF2gdjvtdmikkaIP9bFDDJOPlsvm7avrA=="], + "error-stack-parser-es": ["error-stack-parser-es@1.0.5", "", {}, "sha512-5qucVt2XcuGMcEGgWI7i+yZpmpByQ8J1lHhcL7PwqCwu9FPP3VUXzT4ltHe5i2z9dePwEHcDVOAfSnHsOlCXRA=="], + "es-define-property": ["es-define-property@1.0.1", "", {}, "sha512-e3nRfgfUZ4rNGL232gUgX06QNyyez04KdjFrF+LTRoOXmrOgFKDg4BCdsjW8EnT69eqdYGmRpJwiPVYNrCaW3g=="], "es-errors": ["es-errors@1.3.0", "", {}, "sha512-Zf5H2Kxt2xjTvbJvP2ZWLEICxA6j+hAmMzIlypy4xcBg1vKVnx89Wy0GbS+kf5cwCVFFzdCFh2XSCFNULS6csw=="], @@ -958,6 +1031,8 @@ "execa": ["execa@9.6.1", "", { "dependencies": { "@sindresorhus/merge-streams": "^4.0.0", "cross-spawn": "^7.0.6", "figures": "^6.1.0", "get-stream": "^9.0.0", "human-signals": "^8.0.1", "is-plain-obj": "^4.1.0", "is-stream": "^4.0.1", "npm-run-path": "^6.0.0", "pretty-ms": "^9.2.0", "signal-exit": "^4.1.0", "strip-final-newline": "^4.0.0", "yoctocolors": "^2.1.1" } }, "sha512-9Be3ZoN4LmYR90tUoVu2te2BsbzHfhJyfEiAVfz7N5/zv+jduIfLrV2xdQXOHbaD6KgpGdO9PRPM1Y4Q9QkPkA=="], + "exit-hook": ["exit-hook@2.2.1", "", {}, "sha512-eNTPlAD67BmP31LDINZ3U7HSF8l57TxOY2PmBJ1shpCvpnxBF93mWCE8YHBnXs8qiUZJc9WDcWIeC3a2HIAMfw=="], + "fast-decode-uri-component": ["fast-decode-uri-component@1.0.1", "", {}, "sha512-WKgKWg5eUxvRZGwW8FvfbaH7AXSh2cL+3j5fMGzUMCxWBJ3dV3a7Wz8y2f/uQ0e3B6WmodD3oS54jTQ9HVTIIg=="], "fast-deep-equal": ["fast-deep-equal@3.1.3", "", {}, "sha512-f3qQ9oQy9j2AhBe/H9VC91wLmKBCCU/gDOnKNAYG5hswO7BLKj09Hc5HYNz9cGI++xlpDCIgDaitVs03ATR84Q=="], @@ -1012,6 +1087,8 @@ "glob": ["glob@10.5.0", "", { "dependencies": { "foreground-child": "^3.1.0", "jackspeak": "^3.1.2", "minimatch": "^9.0.4", "minipass": "^7.1.2", "package-json-from-dist": "^1.0.0", "path-scurry": "^1.11.1" }, "bin": { "glob": "dist/esm/bin.mjs" } }, "sha512-DfXN8DfhJ7NH3Oe7cFmu3NCu1wKbkReJ8TorzSAFbSKrlNaQSKfIzqYqVY8zlbs2NLBbWpRiU52GX2PbaBVNkg=="], + "glob-to-regexp": ["glob-to-regexp@0.4.1", "", {}, "sha512-lkX1HJXwyMcprw/5YUZc2s7DrpAiHB21/V+E1rHUrVNokkvB6bqMzT0VfV6/86ZNabt1k14YOIaT7nDvOX3Iiw=="], + "gopd": ["gopd@1.2.0", "", {}, "sha512-ZUKRh6/kUFoAiTAtTYPZJ3hw9wNxx+BIBOijnlG9PnrJsCcSjs1wyyD6vJpaYtgnzDrKYRSqf3OO6Rfa93xsRg=="], "graceful-fs": ["graceful-fs@4.2.11", "", {}, "sha512-RbJ5/jmFcNNCcDV5o9eTnBLJ/HszWV0P73bc+Ff4nS/rJj+YaS6IGyiOL0VoBYX+l1Wrl3k63h/KrH+nhJ0XvQ=="], @@ -1040,6 +1117,8 @@ "ipaddr.js": ["ipaddr.js@2.5.0", "", {}, "sha512-aq+t5NAc+cS6rZQQVWC2x98CPqGtKKTMDd4Gaodv0wShnItdKg/51djkGJ1hqH+Oy0ivDftCbSLCQob8zso01w=="], + "is-arrayish": ["is-arrayish@0.3.4", "", {}, "sha512-m6UrgzFVUYawGBh1dUsWR5M2Clqic9RVXC/9f8ceNlv2IcO9j9J/z8UoCLPqtsPBFNzEpfR3xftohbfqDx8EQA=="], + "is-fullwidth-code-point": ["is-fullwidth-code-point@3.0.0", "", {}, "sha512-zymm5+u+sCsSWyD9qNaejV3DFvhCKclKdizYaJUuHA83RLjb7nSuGnddCHGv0hk+KY7BMAlsWeK4Ueg6EV6XQg=="], "is-plain-obj": ["is-plain-obj@4.1.0", "", {}, "sha512-+Pgi+vMuUNkJyExiMBt5IlFoMyKnr5zhJ4Uspz58WOhBF5QoIZkFyNHIbBAtHwzVAgk5RtndVNsDRN61/mmDqg=="], @@ -1084,6 +1163,8 @@ "jsonify": ["jsonify@0.0.1", "", {}, "sha512-2/Ki0GcmuqSrgFyelQq9M05y7PS0mEwuIzrf3f1fPqkVDVRvZrPZtVSMHxdgo8Aq0sxAOb/cr2aqqA3LeWHVPg=="], + "kleur": ["kleur@4.1.5", "", {}, "sha512-o+NO+8WrRiQEE4/7nwRJhN1HWpVmJm511pBHUxPLtp0BUISzlBplORYSmTclCnJvQq2tKu/sgl3xVpkc7ZWuQQ=="], + "kysely": ["kysely@0.27.6", "", {}, "sha512-FIyV/64EkKhJmjgC0g2hygpBv5RNWVPyNCqSAD7eTCv6eFWNIi4PN1UvdSJGicN/o35bnevgis4Y0UDC0qi8jQ=="], "lazystream": ["lazystream@1.0.1", "", { "dependencies": { "readable-stream": "^2.0.5" } }, "sha512-b94GiNHQNy6JNTrt5w6zNyffMrNkXZb3KTkCZJb2V1xaEGCk093vkZ2jk3tpaeP33/OiXC+WvK9AxUebnf5nbw=="], @@ -1112,6 +1193,8 @@ "mdn-data": ["mdn-data@2.27.1", "", {}, "sha512-9Yubnt3e8A0OKwxYSXyhLymGW4sCufcLG6VdiDdUGVkPhpqLxlvP5vl1983gQjJl3tqbrM731mjaZaP68AgosQ=="], + "miniflare": ["miniflare@4.20251210.0", "", { "dependencies": { "@cspotcode/source-map-support": "0.8.1", "acorn": "8.14.0", "acorn-walk": "8.3.2", "exit-hook": "2.2.1", "glob-to-regexp": "0.4.1", "sharp": "^0.33.5", "stoppable": "1.1.0", "undici": "7.14.0", "workerd": "1.20251210.0", "ws": "8.18.0", "youch": "4.1.0-beta.10", "zod": "3.22.3" }, "bin": { "miniflare": "bootstrap.js" } }, "sha512-k6kIoXwGVqlPZb0hcn+X7BmnK+8BjIIkusQPY22kCo2RaQJ/LzAjtxHQdGXerlHSnJyQivDQsL6BJHMpQfUFyw=="], + "minimalistic-assert": ["minimalistic-assert@1.0.1", "", {}, "sha512-UtJcAD4yEaGtjPezWuO9wC4nwUnVH/8/Im3yEHQP4b67cXlD/Qr9hdITCU1xDbSEXg2XKNaP8jsReV7vQd00/A=="], "minimatch": ["minimatch@10.2.6", "", { "dependencies": { "brace-expansion": "^5.0.8" } }, "sha512-vpLQEs+VLCr1nU0BXS07maYoFwlDAH0gngQuuttxIwutDFEMHq2blX+8vpgxDdK3J1PwjCJiep77OitTZ4Ll1A=="], @@ -1298,6 +1381,8 @@ "set-function-length": ["set-function-length@1.2.2", "", { "dependencies": { "define-data-property": "^1.1.4", "es-errors": "^1.3.0", "function-bind": "^1.1.2", "get-intrinsic": "^1.2.4", "gopd": "^1.0.1", "has-property-descriptors": "^1.0.2" } }, "sha512-pgRc4hJ4/sNjWCSS9AmnS40x3bNMDTknHgL5UaMBTMyJnU90EgWh1Rz+MC9eFu4BuN/UwZjKQuY/1v3rM7HMfg=="], + "sharp": ["sharp@0.33.5", "", { "dependencies": { "color": "^4.2.3", "detect-libc": "^2.0.3", "semver": "^7.6.3" }, "optionalDependencies": { "@img/sharp-darwin-arm64": "0.33.5", "@img/sharp-darwin-x64": "0.33.5", "@img/sharp-libvips-darwin-arm64": "1.0.4", "@img/sharp-libvips-darwin-x64": "1.0.4", "@img/sharp-libvips-linux-arm": "1.0.5", "@img/sharp-libvips-linux-arm64": "1.0.4", "@img/sharp-libvips-linux-s390x": "1.0.4", "@img/sharp-libvips-linux-x64": "1.0.4", "@img/sharp-libvips-linuxmusl-arm64": "1.0.4", "@img/sharp-libvips-linuxmusl-x64": "1.0.4", "@img/sharp-linux-arm": "0.33.5", "@img/sharp-linux-arm64": "0.33.5", "@img/sharp-linux-s390x": "0.33.5", "@img/sharp-linux-x64": "0.33.5", "@img/sharp-linuxmusl-arm64": "0.33.5", "@img/sharp-linuxmusl-x64": "0.33.5", "@img/sharp-wasm32": "0.33.5", "@img/sharp-win32-ia32": "0.33.5", "@img/sharp-win32-x64": "0.33.5" } }, "sha512-haPVm1EkS9pgvHrQ/F3Xy+hgcuMV0Wm9vfIBSiwZ05k+xgb0PkBQpGsAA/oWdDobNaZTH5ppvHtzCFbnSEwHVw=="], + "shebang-command": ["shebang-command@2.0.0", "", { "dependencies": { "shebang-regex": "^3.0.0" } }, "sha512-kHxr2zZpYtdmrN1qDjrrX/Z1rR1kG8Dx+gkpK1G4eXmvXswmcE1hTWBWYUzlraYw1/yZp6YuDY77YtvbN0dmDA=="], "shebang-regex": ["shebang-regex@3.0.0", "", {}, "sha512-7++dFhtcx3353uBaq8DDR4NuxBetBzC7ZQOhmTQInHEd6bSrXdiEyzCvG07Z44UYdLShWUyXt5M/yhz8ekcb1A=="], @@ -1312,6 +1397,8 @@ "signal-exit": ["signal-exit@3.0.7", "", {}, "sha512-wnD2ZE+l+SPC/uoS0vXeE9L1+0wuaMqKlfz9AMUo38JsyLSBWSFcHR1Rri62LZc12vLr1gb3jl7iwQhgwpAbGQ=="], + "simple-swizzle": ["simple-swizzle@0.2.4", "", { "dependencies": { "is-arrayish": "^0.3.1" } }, "sha512-nAu1WFPQSMNr2Zn9PGSZK9AGn4t/y97lEm+MXTtUDwfP0ksAIX4nO+6ruD9Jwut4C49SB1Ws+fbXsm/yScWOHw=="], + "sonic-boom": ["sonic-boom@4.2.1", "", { "dependencies": { "atomic-sleep": "^1.0.0" } }, "sha512-w6AxtubXa2wTXAUsZMMWERrsIRAdrK0Sc+FUytWvYAhBJLyuI4llrMIC1DtlNSdI99EI86KZum2MMq3EAZlF9Q=="], "source-map": ["source-map@0.7.6", "", {}, "sha512-i5uvt8C3ikiWeNZSVZNWcfZPItFQOsYTUAOkcUPGd8DqDy1uOUikjt5dG+uRlwyvR108Fb9DOd4GvXfT0N2/uQ=="], @@ -1328,6 +1415,8 @@ "ssh2": ["ssh2@1.17.0", "", { "dependencies": { "asn1": "^0.2.6", "bcrypt-pbkdf": "^1.0.2" }, "optionalDependencies": { "cpu-features": "~0.0.10", "nan": "^2.23.0" } }, "sha512-wPldCk3asibAjQ/kziWQQt1Wh3PgDFpC0XpwclzKcdT1vql6KeYxf5LIt4nlFkUeR8WuphYMKqUA56X4rjbfgQ=="], + "stoppable": ["stoppable@1.1.0", "", {}, "sha512-KXDYZ9dszj6bzvnEMRYvxgeTHU74QBFL54XKtP3nyMuJ81CFYtABZ3bAzL2EdFUaEwJOBOgENyFj3R7oTzDyyw=="], + "streamx": ["streamx@2.28.1", "", { "dependencies": { "events-universal": "^1.0.0", "fast-fifo": "^1.3.2", "text-decoder": "^1.1.0" } }, "sha512-zEzXb0s5Cds7tqMH6rhZ05lcJydCWiQPEwiNngVqzsxCc962vLY4Uw+mW7od8kDH258k2Uz/JrOkdIAAhSh9VA=="], "string-width": ["string-width@4.2.3", "", { "dependencies": { "emoji-regex": "^8.0.0", "is-fullwidth-code-point": "^3.0.0", "strip-ansi": "^6.0.1" } }, "sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g=="], @@ -1342,6 +1431,8 @@ "strip-final-newline": ["strip-final-newline@4.0.0", "", {}, "sha512-aulFJcD6YK8V1G7iRB5tigAP4TsHBZZrOV8pjV++zdUwmeV8uzbY7yn6h9MswN62adStNZFuCIx4haBnRuMDaw=="], + "supports-color": ["supports-color@10.2.2", "", {}, "sha512-SS+jx45GF1QjgEXQx4NJZV9ImqmO2NPz5FNsIHrsDjh2YsHnawpan7SNQ1o8NuhrbHZy9AZhIoCUiCeaW/C80g=="], + "symbol-tree": ["symbol-tree@3.2.4", "", {}, "sha512-9QNk5KwDF+Bvz+PyObkmSYjI5ksVUYtjW7AU22r2NKcfLJcXp96hkDWU3+XndOsUb+AQ9QhfzfCT2O+CNWT5Tw=="], "tar-fs": ["tar-fs@3.1.3", "", { "dependencies": { "pump": "^3.0.0", "tar-stream": "^3.1.5" }, "optionalDependencies": { "bare-fs": "^4.0.1", "bare-path": "^3.0.0" } }, "sha512-/hU4AXnIdZu+Gvl1pk0oI5f5HxWsCJRtY2aFaJdk9VvyL48DWU6iU5WAIPG+wIi1YvWA6eTJvIviP/tMAZZNwQ=="], @@ -1386,7 +1477,7 @@ "underscore": ["underscore@1.13.8", "", {}, "sha512-DXtD3ZtEQzc7M8m4cXotyHR+FAS18C64asBYY5vqZexfYryNNnDc02W4hKg3rdQuqOYas1jkseX0+nZXjTXnvQ=="], - "undici": ["undici@5.29.0", "", { "dependencies": { "@fastify/busboy": "^2.0.0" } }, "sha512-raqeBD6NQK4SkWhQzeYKd1KmIG6dllBOTt55Rmkt4HtI9mwdWtJljnrXjAFUBLTSN67HWrOIZ3EPF4kjUw80Bg=="], + "undici": ["undici@7.14.0", "", {}, "sha512-Vqs8HTzjpQXZeXdpsfChQTlafcMQaaIwnGwLam1wudSSjlJeQ3bw1j+TLPePgrCnCpUXx7Ba5Pdpf5OBih62NQ=="], "undici-types": ["undici-types@6.21.0", "", {}, "sha512-iwDZqg0QAGrg9Rav5H4n0M64c3mkR59cJ6wQp+7C4nI0gsmExaedaYLNO44eT4AtBBwjbTiGPMlt2Md0T9H9JQ=="], @@ -1412,13 +1503,15 @@ "which": ["which@2.0.2", "", { "dependencies": { "isexe": "^2.0.0" }, "bin": { "node-which": "./bin/node-which" } }, "sha512-BLI3Tl1TW3Pvl70l3yq3Y64i+awpwXqsGBYWkkqMtnbXgrMD+yj7rhW0kuEDxzJaYXGjEW5ogapKNMEKNMjibA=="], + "workerd": ["workerd@1.20251210.0", "", { "optionalDependencies": { "@cloudflare/workerd-darwin-64": "1.20251210.0", "@cloudflare/workerd-darwin-arm64": "1.20251210.0", "@cloudflare/workerd-linux-64": "1.20251210.0", "@cloudflare/workerd-linux-arm64": "1.20251210.0", "@cloudflare/workerd-windows-64": "1.20251210.0" }, "bin": { "workerd": "bin/workerd" } }, "sha512-9MUUneP1BnRE9XAYi94FXxHmiLGbO75EHQZsgWqSiOXjoXSqJCw8aQbIEPxCy19TclEl/kHUFYce8ST2W+Qpjw=="], + "wrap-ansi": ["wrap-ansi@7.0.0", "", { "dependencies": { "ansi-styles": "^4.0.0", "string-width": "^4.1.0", "strip-ansi": "^6.0.0" } }, "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q=="], "wrap-ansi-cjs": ["wrap-ansi@7.0.0", "", { "dependencies": { "ansi-styles": "^4.0.0", "string-width": "^4.1.0", "strip-ansi": "^6.0.0" } }, "sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q=="], "wrappy": ["wrappy@1.0.2", "", {}, "sha512-l4Sp/DRseor9wL6EvV2+TuQn63dMkPjZ/sp9XkghTEbV9KlPS1xUsZ3u7/IQO4wxtcFB4bgpQPRcR3QCvezPcQ=="], - "ws": ["ws@8.22.0", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-Ydggc987+RO0AnWtZ/7Wq9FtNvcrL1b/RO0ud9mWjUPgDrsAAwQSF51sm2hm1XofbU/4jkpGEsLFsZZxU+1DOg=="], + "ws": ["ws@8.18.0", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-8VbfWfHLbbwu3+N6OKsOMpBdT4kXPDDB9cJk2bJ6mh9ucxdlnNvH1e+roYkKmN9Nxw2yjz7VzeO9oOz2zJ04Pw=="], "xml-name-validator": ["xml-name-validator@5.0.0", "", {}, "sha512-EvGK8EJ3DhaHfbRlETOWAS5pO9MZITeauHKJyb8wyajUfQUenkIg2MvLDTZ4T/TgIcm3HU0TFBgWWboAZ30UHg=="], @@ -1438,6 +1531,10 @@ "yoctocolors": ["yoctocolors@2.2.0", "", {}, "sha512-xYqdZFUK/VYazNl/oCDYN+3WloWQwMfZxBoiNt6qNyk+xfOdi598muWE42rNZFp1kNOiqW936q5RhUdnpqElSg=="], + "youch": ["youch@4.1.0-beta.10", "", { "dependencies": { "@poppinss/colors": "^4.1.5", "@poppinss/dumper": "^0.6.4", "@speed-highlight/core": "^1.2.7", "cookie": "^1.0.2", "youch-core": "^0.3.3" } }, "sha512-rLfVLB4FgQneDr0dv1oddCVZmKjcJ6yX6mS4pU82Mq/Dt9a3cLZQ62pDBL4AUO+uVrCvtWz3ZFUL2HFAFJ/BXQ=="], + + "youch-core": ["youch-core@0.3.3", "", { "dependencies": { "@poppinss/exception": "^1.2.2", "error-stack-parser-es": "^1.0.5" } }, "sha512-ho7XuGjLaJ2hWHoK8yFnsUGy2Y5uDpqSTq1FkHLK4/oqKtyUU1AFbOOxY4IpC9f0fTLjwYbslUz0Po5BpD1wrA=="], + "zip-stream": ["zip-stream@6.0.1", "", { "dependencies": { "archiver-utils": "^5.0.0", "compress-commons": "^6.0.2", "readable-stream": "^4.0.0" } }, "sha512-zK7YHHz4ZXpW89AHXUPbQVGKI7uvkd3hzusTdotCg1UxyaVtg0zFJSTfW/Dq5f7OBBVnq6cZIaC8Ti4hb6dtCA=="], "zod": ["zod@4.6.5", "", {}, "sha512-v5l/aFXZQeai4awLbOpSoHecE9UiMrnfx75tEXLjNonXVARxQ5mOeipTjROUchszUNCqnE+hqAMujRsRHsut2Q=="], @@ -1446,6 +1543,8 @@ "@babel/core/semver": ["semver@6.3.1", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="], + "@babel/generator/@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="], + "@babel/helper-compilation-targets/lru-cache": ["lru-cache@5.1.1", "", { "dependencies": { "yallist": "^3.0.2" } }, "sha512-KpNARQA3Iwv+jTA0utUVVbrh+Jlrr1Fv0e56GGzAFOXN7dk/FviaDW8LHmK52DlcH4WP2n6gI8vN1aesBFgo9w=="], "@babel/helper-compilation-targets/semver": ["semver@6.3.1", "", { "bin": { "semver": "bin/semver.js" } }, "sha512-BR7VvDCVHO+q2xBEWskxS6DJE1qRnb7DxzUrogb71CWoSficBxYsiAGd+Kl0mmq/MprG9yArRkyrQxTO6XjMzA=="], @@ -1462,6 +1561,12 @@ "@isaacs/cliui/wrap-ansi": ["wrap-ansi@8.1.0", "", { "dependencies": { "ansi-styles": "^6.1.0", "string-width": "^5.0.1", "strip-ansi": "^7.0.1" } }, "sha512-si7QWI6zUMq56bESFvagtmzMdGOtoxfR+Sez11Mobfc7tm+VkUckk9bW2UeffTGVUbOksxmSw0AA2gs8g71NCQ=="], + "@jridgewell/gen-mapping/@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="], + + "@jridgewell/remapping/@jridgewell/trace-mapping": ["@jridgewell/trace-mapping@0.3.31", "", { "dependencies": { "@jridgewell/resolve-uri": "^3.1.0", "@jridgewell/sourcemap-codec": "^1.4.14" } }, "sha512-zzNR+SdQSDJzc8joaeP8QQoCQr8NuYx2dIIytl1QeBEZHJ9uW6hebsrYgbz8hJwUQao3TWCMtmfV8Nu1twOLAw=="], + + "@libsql/isomorphic-ws/ws": ["ws@8.22.0", "", { "peerDependencies": { "bufferutil": "^4.0.1", "utf-8-validate": ">=5.0.2" }, "optionalPeers": ["bufferutil", "utf-8-validate"] }, "sha512-Ydggc987+RO0AnWtZ/7Wq9FtNvcrL1b/RO0ud9mWjUPgDrsAAwQSF51sm2hm1XofbU/4jkpGEsLFsZZxU+1DOg=="], + "@libsql/kysely-libsql/@libsql/client": ["@libsql/client@0.8.1", "", { "dependencies": { "@libsql/core": "^0.8.1", "@libsql/hrana-client": "^0.6.2", "js-base64": "^3.7.5", "libsql": "^0.3.10", "promise-limit": "^2.7.0" } }, "sha512-xGg0F4iTDFpeBZ0r4pA6icGsYa5rG6RAG+i/iLDnpCAnSuTqEWMDdPlVseiq4Z/91lWI9jvvKKiKpovqJ1kZWA=="], "@metaobjectsdev/codegen-ts/drizzle-orm": ["drizzle-orm@0.38.4", "", { "peerDependencies": { "@aws-sdk/client-rds-data": ">=3", "@cloudflare/workers-types": ">=4", "@electric-sql/pglite": ">=0.2.0", "@libsql/client": ">=0.10.0", "@libsql/client-wasm": ">=0.10.0", "@neondatabase/serverless": ">=0.10.0", "@op-engineering/op-sqlite": ">=2", "@opentelemetry/api": "^1.4.1", "@planetscale/database": ">=1", "@prisma/client": "*", "@tidbcloud/serverless": "*", "@types/better-sqlite3": "*", "@types/pg": "*", "@types/react": ">=18", "@types/sql.js": "*", "@vercel/postgres": ">=0.8.0", "@xata.io/client": "*", "better-sqlite3": ">=7", "bun-types": "*", "expo-sqlite": ">=14.0.0", "knex": "*", "kysely": "*", "mysql2": ">=2", "pg": ">=8", "postgres": ">=3", "prisma": "*", "react": ">=18", "sql.js": ">=1", "sqlite3": ">=5" }, "optionalPeers": ["@aws-sdk/client-rds-data", "@cloudflare/workers-types", "@electric-sql/pglite", "@libsql/client", "@libsql/client-wasm", "@neondatabase/serverless", "@op-engineering/op-sqlite", "@opentelemetry/api", "@planetscale/database", "@prisma/client", "@tidbcloud/serverless", "@types/better-sqlite3", "@types/pg", "@types/react", "@types/sql.js", "@vercel/postgres", "@xata.io/client", "better-sqlite3", "bun-types", "expo-sqlite", "knex", "kysely", "mysql2", "pg", "postgres", "prisma", "react", "sql.js", "sqlite3"] }, "sha512-s7/5BpLKO+WJRHspvpqTydxFob8i1vo2rEx4pY6TGY7QSMuUfWUuzaY0DIpXCkgHOo37BaFC+SJQb99dDUXT3Q=="], @@ -1506,6 +1611,8 @@ "light-my-request/process-warning": ["process-warning@4.0.1", "", {}, "sha512-3c2LzQ3rY9d0hc1emcsHhfT9Jwz0cChib/QN89oME2R451w5fy3f0afAhERFZAwrbDU43wk12d0ORBpDVME50Q=="], + "miniflare/zod": ["zod@3.22.3", "", {}, "sha512-EjIevzuJRiRPbVH4mGc8nApb/lVLKVpmUhAaR5R5doKGfAnGJ6Gr3CViAVjP+4FWSxCsybeWQdcgCtbX+7oZug=="], + "nearley/commander": ["commander@2.20.3", "", {}, "sha512-GpVkmM8vF2vQUkj2LvZmD35JxeJOLCwJ9cUkugyk2nuhbv3+mJvpLYYt+0+USMxE+oj+ey/lJEnhZw75x/OMcQ=="], "path-scurry/lru-cache": ["lru-cache@10.4.3", "", {}, "sha512-JNAzZcXrCt42VGLuYz0zfAzDfAvJWW6AfYlDBQyDV5DClI2m5sAmK+OIO7s59XfsRsWHp02jAJrRadPRGTt6SQ=="], @@ -1514,10 +1621,14 @@ "readdir-glob/minimatch": ["minimatch@5.1.9", "", { "dependencies": { "brace-expansion": "^2.0.1" } }, "sha512-7o1wEA2RyMP7Iu7GNba9vc0RWWGACJOCZBJX2GJWip0ikV+wcOsgVuY9uE8CPiyQhkGFSlhuSkZPavN7u1c2Fw=="], + "sharp/detect-libc": ["detect-libc@2.1.2", "", {}, "sha512-Btj2BOOO83o3WyH59e8MgXsxEQVcarkUOpEYrubB0urwnN10yQ364rsiByU11nZlqWYZm05i/of7io4mzihBtQ=="], + "string-width/emoji-regex": ["emoji-regex@8.0.0", "", {}, "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A=="], "string-width-cjs/emoji-regex": ["emoji-regex@8.0.0", "", {}, "sha512-MSjYzcWNOA0ewAHpz0MxpYFvwg6yjy1NG3xteoqz644VCo/RPgnr1/GGt+ic3iJTzQ8Eu3TdM14SawnVUmGE6A=="], + "testcontainers/undici": ["undici@5.29.0", "", { "dependencies": { "@fastify/busboy": "^2.0.0" } }, "sha512-raqeBD6NQK4SkWhQzeYKd1KmIG6dllBOTt55Rmkt4HtI9mwdWtJljnrXjAFUBLTSN67HWrOIZ3EPF4kjUw80Bg=="], + "typed-rest-client/qs": ["qs@6.15.1", "", { "dependencies": { "side-channel": "^1.1.0" } }, "sha512-6YHEFRL9mfgcAvql/XhwTvf5jKcOiiupt2FiJxHkiX1z4j7WL8J/jRHYLluORvc1XxB5rV20KoeK00gVJamspg=="], "wrap-ansi/ansi-styles": ["ansi-styles@4.3.0", "", { "dependencies": { "color-convert": "^2.0.1" } }, "sha512-zbB9rCJAT1rbjiVDb2hqKFHNYLxgtk8NURxZ3IZwD3F6NtxbXZQCnnSi1Lkx+IDohdPlFp222wVALIheZJQSEg=="], diff --git a/docs/CONFORMANCE.md b/docs/CONFORMANCE.md index 3a2c51eff..d0bce8e3c 100644 --- a/docs/CONFORMANCE.md +++ b/docs/CONFORMANCE.md @@ -33,8 +33,8 @@ regenerate with `ls -d fixtures//*/ | wc -l` for directory-shaped corpor | [`fixtures/render-conformance/`](../fixtures/render-conformance/) | 15 | ✓ | ✓ | inherits via Java | ✓ | ✓ | | [`fixtures/extract-conformance/`](../fixtures/extract-conformance/) | 48 | ✓ | ✓ | inherits the shared JVM engine | ✓ | ✓ | | [`fixtures/output-prompt-conformance/`](../fixtures/output-prompt-conformance/) | 17 | ✓ | ✓ | ✓ | ✓ | ✓ | -| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 42 (36 query + 6 migration) | all 42 | 36 query (migrations TS-only, ADR-0015) | 36 query (via Exposed) | 36 query | 36 query | -| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 82 (31 core + 10 tph + 9 m2m + 2 jsonb + 2 write-through + 12 projection + 16 report) | ✓ (Fastify reference + generated lane) | ✓ (embedded HTTP + JDBC) | ✓ (embedded HTTP + Exposed) | ✓ (HttpListener + Npgsql) | ✓ (FastAPI + pg8000) | +| [`fixtures/persistence-conformance/`](../fixtures/persistence-conformance/) | 42 (36 query + 6 migration) | all 42 (the 9 `report-*` also on SQLite, MySQL and D1) | 36 query (migrations TS-only, ADR-0015); the 9 `report-*` also on MySQL | 36 query (via Exposed); the 9 `report-*` also on SQLite and MySQL | 36 query; the 9 `report-*` also on SQLite and MySQL | 36 query (Postgres only: pg8000) | +| [`fixtures/api-contract-conformance/`](../fixtures/api-contract-conformance/) | 82 (31 core + 10 tph + 9 m2m + 2 jsonb + 2 write-through + 12 projection + 16 report) | ✓ (Fastify reference + generated lane); `report/` also on SQLite, MySQL and D1 | ✓ (embedded HTTP + JDBC) | ✓ (embedded HTTP + Exposed) | ✓ (HttpListener + Npgsql); `report/` also on SQLite and MySQL | ✓ (FastAPI + pg8000) | | [`fixtures/validation-conformance/`](../fixtures/validation-conformance/) | 42 cases | ✓ (generated Zod + run-time `runValidators`) | ✓ | ✓ | ✓ | ✓ (generated Pydantic + run-time `run_validators`) | | [`fixtures/registry-conformance/`](../fixtures/registry-conformance/) | 1 canonical manifest | ✓ (reference emitter) | ✓ | ✓ | ✓ | ✓ | | [`fixtures/object-model-conformance/`](../fixtures/object-model-conformance/) | 1 shared metadata fixture (per-port scenarios) | ✓ | ✓ | ✓ | ✓ | ✓ | @@ -138,9 +138,15 @@ manifest contains without moving these boundaries. compile, the two MySQL cases included: compiling a model runs no SQL, so the lane's Postgres data source serves them too. For the nine canonical reports it compares the Cube query with the report view (a `@spine` report through its Cube view, its empty spine rows included), and for - the `escaping` and `measure-default` cases it reads the SQL back from Cube's `/v1/sql`. No - SQL of a MySQL case is executed; the goldens hold it. The lane runs Cube in development mode, - so Cube's production mode with a separate Cube Store is not gated. + the `escaping` and `measure-default` cases it reads the SQL back from Cube's `/v1/sql`. + A second live file (`cube-model-mysql.live.ts`) runs the MySQL path: the canonical golden + generated with `dialect: "mysql"` is loaded into the same pinned Cube over a real MySQL 8.4 + holding the report views `buildReportViews` lowers (`fixtures/cube-model/canonical/seed.mysql.sql`), + and each of the nine reports' Cube query is compared with `SELECT *` from its view, with the + pre-aggregation used asserted. That is the smaller of the two honest options: the two MySQL + *cases* (escaping, a quoted identifier) are still compiled, not executed, and the goldens hold + their SQL. The lane runs Cube in development mode, so Cube's production mode with a separate + Cube Store is not gated. - *The lane is a gate, not a second corpus.* Like the codegen-compile gate, it reuses `fixtures/persistence-conformance/canonical/meta.fitness.json` and the committed `schema.postgres.sql` rather than adding a model beside them; only the seed rows are its own @@ -297,6 +303,20 @@ All 31 fixtures → [features/migrations-and-drift.md](features/migrations-and-d - `migrations/*` (6) → [features/migrations-and-drift.md](features/migrations-and-drift.md) (schema migration section) - `queries/*` (36) → [features/source-kinds.md](features/source-kinds.md) (query semantics against `source.rdb`) +**Engines beyond Postgres (FR-044 reports).** The nine `queries/report-*.yaml` scenarios also +run on SQLite and MySQL 8.4 wherever a port's runtime has the engine, and on D1's local runtime +in TypeScript (no cloud account): TypeScript on all four, C# and Kotlin on SQLite and MySQL, Java +on MySQL. **Stated limitations, not gaps:** Java's OMDB has no SQLite driver, and Python's runtime +ships a Postgres driver only (pg8000), so Python runs the corpus on Postgres alone. Adding a +driver to run a view-reading test is not worth the surface. Each engine executes an artifact +TypeScript produced (`canonical/schema.sqlite.sql` from `meta migrate --dialect sqlite`; +`canonical/schema.mysql.sql` from the adopter-written tables plus `buildReportViews`), held +equal to its generator by a test, so no port writes view SQL. The `expect` blocks stay the +Postgres wire ones; engine spelling is normalised on the actual side only (a SQLite ratio is a +REAL, a MySQL `DECIMAL` keeps its scale, a MySQL `DATETIME` is the UTC wall clock), and +`report-relative-date` carries a per-engine seed (`seed-data-engine`). `op: relate` and the +write ops stay Postgres-only. See [features/reporting.md](features/reporting.md#what-the-corpus-gates). + ### `fixtures/api-contract-conformance/` (82) All 82 scenarios → [features/api-contract.md](features/api-contract.md) (cross-port @@ -324,7 +344,11 @@ a row with no facts, and a measure with `@default` filters and sorts as its defa Kotlin, C#, Python — run it in BOTH lanes: a hand-rolled reference server and the port's own GENERATED API artifact booted over HTTP. `write-through/`, `projection/` and `report/` run the generated lane only, on all five ports: what -they test is whether a port's generator emits the routes. +they test is whether a port's generator emits the routes. The `report/` sub-corpus is the one +that reads real views, so it also runs on SQLite and MySQL (`report/schema.sqlite.sql`, +`report/schema.mysql.sql`, both TypeScript-produced) in TypeScript and C#, and on D1's local +runtime in TypeScript. Java, Kotlin and Python serve that corpus from seeded rows behind their +repository seam, so no engine reaches them and the extra engines are not applicable there. ### `fixtures/source-resolution-conformance/` (25 cases) diff --git a/docs/features/cube-export.md b/docs/features/cube-export.md index 1eadb2564..2fc4cbdcd 100644 --- a/docs/features/cube-export.md +++ b/docs/features/cube-export.md @@ -580,7 +580,7 @@ development mode, with its embedded Cube Store, against a private `postgres:16-a holds the persistence-conformance schema and a seed (`fixtures/cube-model/canonical/seed.sql`). Each run owns a private Docker network and binds one ephemeral port on `127.0.0.1`; it never uses the shared Postgres sidecar, and every -container and the network are removed on every exit path. It checks six things: +container and the network are removed on every exit path. It checks six things on Postgres: 1. The generated canonical model is the reviewed golden. 2. Cube compiles it and lists every cube and member the files declare. @@ -605,6 +605,12 @@ container and the network are removed on every exit path. It checks six things: `@default` divides that numerator's `COALESCE(sum(…), 0)`, with and without the ratio's own default, as the view does. +The MySQL path is a second file in the same lane (`cube-model-mysql.live.ts`). It runs the same +Cube over a private MySQL 8.4 holding the report views `buildReportViews` lowers +(`fixtures/cube-model/canonical/seed.mysql.sql`), and for each served report it checks item 3: +the Cube query returns the rows of `SELECT * FROM `, from the rollup when the model has one. +Checks 4, 5 and 6 are Postgres only. + Run it with `scripts/ci-local.sh --only cube`. The full `scripts/ci-local.sh` runs it after the integration suite, and `--quick` and `--no-integration` drop it. The first run pulls the Cube image, which is about 1 GB. With Docker down, the lane records a SKIP behind a banner, never a @@ -627,9 +633,10 @@ the table names above are the development-mode form. refuses the model with `ERR_CUBE_AMBIGUOUS_PATH`, naming both. The lossless form needs a nested alias for each path. That is more machinery than a rare model is worth yet, so the exporter errors rather than guess. -- **The lane never executes MySQL SQL.** The corpus pins it byte for byte and the lane compiles - it in Cube, but no MySQL database runs it there. The one MySQL form chosen for its meaning, the - tuple distinct count, was compared with the view's on mysql:8.4 by hand when it was built. +- **The two MySQL mapping cases are compiled, never executed.** The corpus pins their SQL byte + for byte and Cube compiles them, but no MySQL database runs them. The MySQL path runs only the + served reports. The tuple distinct count's MySQL form was compared with the view's on mysql:8.4 + by hand when it was built. - **Some shapes are compile-checked, not query-checked.** Cube accepts the alias cubes, the TPH subtype's `sql`, one-to-one joins, the int-backed enum's `CASE` and a two-hop `@spine` chain (the corpus pass), but no live query of the lane crosses them. The two-hop chain's rows were diff --git a/docs/features/reporting.md b/docs/features/reporting.md index b42019979..4187d603b 100644 --- a/docs/features/reporting.md +++ b/docs/features/reporting.md @@ -633,6 +633,31 @@ tests read the `@spine` and `@default` views and their column types on Postgres, MySQL, and on Postgres and SQLite also a fact whose reference is null, an `isNull` condition and a two-hop spine. +The nine scenarios run on more than Postgres, wherever a port's runtime has the engine: + +| | Postgres | SQLite | MySQL 8.4 | D1 (local runtime) | +|---|---|---|---|---| +| TypeScript | ✓ | ✓ | ✓ | ✓ | +| C# (EF Core) | ✓ | ✓ | ✓ (Pomelo) | n/a | +| Kotlin (Exposed) | ✓ | ✓ | ✓ | n/a | +| Java (OMDB) | ✓ | not run: OMDB has no SQLite driver | ✓ | n/a | +| Python | ✓ | not run: the runtime ships a Postgres driver only (pg8000) | not run: same | n/a | + +Each engine reads the artifact TypeScript produced for it: `canonical/schema.sqlite.sql` is +the output of `meta migrate --dialect sqlite`, and `canonical/schema.mysql.sql` is the +adopter-written tables plus a `CREATE VIEW` per report from `buildReportViews`. No port +writes view SQL, and a test holds each artifact equal to its generator. The expectations +stay the Postgres wire ones, never loosened: the engine's own spelling (a REAL ratio on SQLite, +a scale-keeping `DECIMAL` and a zoneless `DATETIME` on MySQL) is mapped on the actual side by +each runner, and `report-relative-date` spells its seed per engine (`seed-data-engine`). D1 +has no separate corpus: it is SQLite behind Cloudflare's binding, so TypeScript runs the same +scenarios and the same sixteen REST scenarios through Miniflare's local D1 (`meta migrate +--dialect d1`), with no cloud account. + +A port without the engine is a stated limitation, not a gap to fill by building a driver: +Java's OMDB and Python's `ObjectManager` ship the drivers they ship, and a view-reading test +does not justify a new one. + The REST surface is gated by sixteen scenarios under [`fixtures/api-contract-conformance/report/`](../../fixtures/api-contract-conformance/report/), run in the **generated lane on all five ports**: list (a dimension with a segment-scoped sum @@ -643,10 +668,12 @@ on a measure and an enum dimension, paging over groups, the three field-naming ` sale keeps its row, with a filter and a sort on its defaulted measure. The corpus model carries one sourceless report, so a port that serves every report it finds fails. No scenario asserts a decimal's spelling or a timestamp literal. TypeScript and C# run the scenarios against the real views on -Postgres, and TypeScript runs the same sixteen against SQLite as well -(`api-contract-report-sqlite.test.ts`), where it also holds that a ratio reaches the wire as a -string, as it does on Postgres; Java, Kotlin and Python serve seeded rows behind their repository seam, and a -TypeScript test holds those rows equal to what the views return. +Postgres, SQLite and MySQL (`report/schema.sqlite.sql`, `report/schema.mysql.sql`), and +TypeScript also through local D1; TypeScript also holds that a ratio reaches the wire as a +string on SQLite, as it does on Postgres. Java, Kotlin and Python serve seeded rows behind +their repository seam, so they read no view and no engine reaches them: running that lane on +SQLite or MySQL would test nothing the seam does not already, and a TypeScript test holds +those rows equal to what the views return. ## Exporting to Cube diff --git a/docs/recipes/mysql.md b/docs/recipes/mysql.md index 83f133c7c..4a4103dbc 100644 --- a/docs/recipes/mysql.md +++ b/docs/recipes/mysql.md @@ -26,7 +26,11 @@ These tests exercise the whole list against a real MySQL 8.4 server: - `mysql-object-manager.test.ts`: both TypeScript ObjectManager drivers; - `MySqlObjectManagerTest`: the Java OMDB driver. -Python, C# and Kotlin have no MySQL-specific code, and nothing tests them on MySQL yet. +The FR-044 report views are read on MySQL by C# (EF Core with Pomelo), Kotlin (Exposed) and +Java (OMDB) as well, through the persistence `report-*` scenarios; C# also serves its generated +report routes over MySQL. Python ships a Postgres driver only (pg8000), so it is not run there. +The Cube queries for the served reports are executed against a real MySQL 8.4 by the `cube` lane +(see [features/cube-export.md](../features/cube-export.md)). ## TypeScript setup diff --git a/fixtures/api-contract-conformance/report/README.md b/fixtures/api-contract-conformance/report/README.md index af3a68098..db7728a86 100644 --- a/fixtures/api-contract-conformance/report/README.md +++ b/fixtures/api-contract-conformance/report/README.md @@ -90,6 +90,8 @@ report/ ├── meta.json # Invoice, Product, Sale + five object.report nodes (four served, one sourceless) ├── seed.json # the base tables (`invoices`, `products`, `sales`) and `reports` (what the views return) ├── schema.postgres.sql # TypeScript-produced: the three tables and the four views +├── schema.sqlite.sql # the same, for SQLite and D1 (`meta migrate --dialect sqlite`) +├── schema.mysql.sql # the same, for MySQL 8.4: the tables plus `buildReportViews` bodies └── scenarios/ ├── list.yaml # GET list, dimension + segment-scoped sum ├── list-time-grain.yaml # a time dimension at a grain @@ -142,8 +144,8 @@ hand-rolled reference server would answer every scenario by construction. | Port | Generated lane | Note | |---|---|---| -| TypeScript | wired | `server/typescript/packages/integration-tests/test/api-contract-report.test.ts` (16 scenarios + a seed-vs-view check). Full stack: generated Fastify routes over the real views on Testcontainers Postgres. `api-contract-report-sqlite.test.ts` runs the same 16 on SQLite | -| C# | wired | `server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs`. Full stack: generated routes and EF Core over `schema.postgres.sql` on Testcontainers Postgres | +| TypeScript | wired | `server/typescript/packages/integration-tests/test/api-contract-report.test.ts` (16 scenarios + a seed-vs-view check). Full stack: generated Fastify routes over the real views on Testcontainers Postgres. `api-contract-report-sqlite.test.ts` runs the same 16 on SQLite, `api-contract-report-mysql.test.ts` on MySQL 8.4, and the SQLite test file also runs them through local D1 (Miniflare) | +| C# | wired | `server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs`. Full stack: generated routes and EF Core over `schema.postgres.sql` on Testcontainers Postgres, over `schema.sqlite.sql` on SQLite, and over `schema.mysql.sql` on MySQL 8.4 | | Java | wired | `server/java/integration-tests/src/test/java/com/metaobjects/integration/api/ReportGeneratedApiContractConformanceTest.java` (16 scenarios + a scenario-count check). Generated controllers behind an in-memory repository seeded from `reports` | | Kotlin | wired | `server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/api/report/ReportGeneratedApiContractConformanceTest.kt` (16 scenarios, the count check, and a check that the sourceless report generated nothing). Generated controllers over the generated Exposed table objects, seeded from `reports` | | Python | wired | `server/python/tests/integration/test_api_contract_report.py` (16 scenarios, a check that exactly the served reports are generated, and `/api/invoice_days` is `404`). Generated routers behind in-memory repositories seeded from `reports` | diff --git a/fixtures/api-contract-conformance/report/schema.mysql.sql b/fixtures/api-contract-conformance/report/schema.mysql.sql new file mode 100644 index 000000000..fc15902b9 --- /dev/null +++ b/fixtures/api-contract-conformance/report/schema.mysql.sql @@ -0,0 +1,60 @@ +-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT. +-- api-contract report sub-corpus schema (MySQL 8.4, literal column naming). +-- Tables: the adopter's hand-written DDL (meta migrate does not own MySQL, ADR-0015). +-- Views: buildReportViews(root, { dialect: "mysql" }), produced by TypeScript. +-- Regenerate: `bun run gen:report-api-schema` in server/typescript/packages/integration-tests. + +CREATE TABLE `invoices` ( + `id` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + `reference` VARCHAR(40) NOT NULL, + `status` VARCHAR(20) NOT NULL, + `amountCents` BIGINT NOT NULL, + `issuedOn` DATE NOT NULL +); + +CREATE TABLE `products` ( + `id` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + `name` VARCHAR(200) NOT NULL +); + +CREATE TABLE `sales` ( + `id` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + `productId` BIGINT NOT NULL, + `amountCents` BIGINT NOT NULL, + FOREIGN KEY (`productId`) REFERENCES `products` (`id`) +); + +CREATE VIEW `v_invoice_status_totals` AS + SELECT + i.`status` AS `status`, + COUNT(i.`id`) AS `invoices`, + CAST(SUM(i.`amountCents`) AS SIGNED) AS `totalCents`, + CAST(SUM(CASE WHEN i.`status` = 'PAID' THEN i.`amountCents` END) AS SIGNED) AS `paidCents` + FROM `invoices` i + GROUP BY i.`status`; + +CREATE VIEW `v_invoices_by_month` AS + SELECT + DATE(DATE_FORMAT(i.`issuedOn`, '%Y-%m-01')) AS `issuedOnMonth`, + COUNT(i.`id`) AS `invoices`, + CAST(SUM(i.`amountCents`) AS SIGNED) AS `totalCents` + FROM `invoices` i + GROUP BY DATE(DATE_FORMAT(i.`issuedOn`, '%Y-%m-01')); + +CREATE VIEW `v_invoice_totals` AS + SELECT + COUNT(i.`id`) AS `invoices`, + CAST(SUM(i.`amountCents`) AS SIGNED) AS `totalCents`, + COUNT(CASE WHEN i.`status` = 'PAID' THEN i.`id` END) / NULLIF(COUNT(i.`id`), 0) AS `paidShare` + FROM `invoices` i; + +CREATE VIEW `v_product_revenue` AS + SELECT + p.`id` AS `productId`, + p.`name` AS `productName`, + COUNT(s.`id`) AS `sales`, + CAST(SUM(s.`amountCents`) AS SIGNED) AS `revenueCents`, + COALESCE(CAST(SUM(s.`amountCents`) AS SIGNED), 0) AS `revenueOrZero` + FROM `products` p + LEFT OUTER JOIN `sales` s ON p.`id` = s.`productId` + GROUP BY p.`id`, p.`name`; diff --git a/fixtures/api-contract-conformance/report/schema.sqlite.sql b/fixtures/api-contract-conformance/report/schema.sqlite.sql new file mode 100644 index 000000000..39c986ad8 --- /dev/null +++ b/fixtures/api-contract-conformance/report/schema.sqlite.sql @@ -0,0 +1,60 @@ +-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT. +-- api-contract report sub-corpus schema (SQLite, literal column naming). +-- Produced by TypeScript from report/meta.json; the C# generated lane executes it. +-- Regenerate: `bun run gen:report-api-schema` in this package. + +CREATE TABLE "invoices" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "reference" VARCHAR(40) NOT NULL, + "status" TEXT NOT NULL, + "amountCents" INTEGER NOT NULL, + "issuedOn" TEXT NOT NULL, + CONSTRAINT "invoices_status_chk" CHECK ("status" IN ('OPEN', 'PAID', 'VOID')) +); + +CREATE TABLE "products" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "name" TEXT NOT NULL +); + +CREATE TABLE "sales" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "productId" INTEGER NOT NULL, + "amountCents" INTEGER NOT NULL, + FOREIGN KEY ("productId") REFERENCES "products" ("id") +); + +CREATE VIEW "v_invoice_status_totals" AS + SELECT + i."status" AS "status", + COUNT(i."id") AS "invoices", + SUM(i."amountCents") AS "totalCents", + SUM(CASE WHEN i."status" = 'PAID' THEN i."amountCents" END) AS "paidCents" + FROM "invoices" i + GROUP BY i."status"; + +CREATE VIEW "v_invoices_by_month" AS + SELECT + date(i."issuedOn", 'start of month') AS "issuedOnMonth", + COUNT(i."id") AS "invoices", + SUM(i."amountCents") AS "totalCents" + FROM "invoices" i + GROUP BY date(i."issuedOn", 'start of month'); + +CREATE VIEW "v_invoice_totals" AS + SELECT + COUNT(i."id") AS "invoices", + SUM(i."amountCents") AS "totalCents", + CAST(COUNT(CASE WHEN i."status" = 'PAID' THEN i."id" END) AS REAL) / NULLIF(COUNT(i."id"), 0) AS "paidShare" + FROM "invoices" i; + +CREATE VIEW "v_product_revenue" AS + SELECT + p."id" AS "productId", + p."name" AS "productName", + COUNT(s."id") AS "sales", + SUM(s."amountCents") AS "revenueCents", + COALESCE(SUM(s."amountCents"), 0) AS "revenueOrZero" + FROM "products" p + LEFT OUTER JOIN "sales" s ON p."id" = s."productId" + GROUP BY p."id", p."name"; diff --git a/fixtures/cube-model/README.md b/fixtures/cube-model/README.md index c4d4f6980..576a67443 100644 --- a/fixtures/cube-model/README.md +++ b/fixtures/cube-model/README.md @@ -19,6 +19,9 @@ Run by `server/typescript/packages/codegen-ts/test/cube/cube-model-corpus.test.t canonical/expected/ Table H: the canonical persistence model, written by `bun run gen:cube-canonical` (codegen-ts) and drift-checked by test/cube/cube-model-canonical.test.ts +canonical/seed.sql the rows the live Cube check (Postgres) reads +canonical/seed.mysql.sql the same rows spelled for MySQL 8.4: the live MySQL check runs the + report views `buildReportViews` lowers over them (cube-model-mysql.live.ts) ``` A case holds exactly one of `expected/` and `expected-error.txt`. A file the generator writes diff --git a/fixtures/cube-model/canonical/seed.mysql.sql b/fixtures/cube-model/canonical/seed.mysql.sql new file mode 100644 index 000000000..5c7b84640 --- /dev/null +++ b/fixtures/cube-model/canonical/seed.mysql.sql @@ -0,0 +1,25 @@ +-- MySQL twin of seed.sql, for the MySQL cube-model live check (server/typescript/packages/ +-- integration-tests/cube-live/cube-model-mysql.live.ts). Applied after +-- fixtures/persistence-conformance/canonical/schema.mysql.sql. The rows are seed.sql's, spelled +-- for MySQL: DATETIME literals carry no zone designator (the session is UTC), and the two +-- programs created relative to the database clock use UTC_TIMESTAMP(3) so RecentPrograms keeps +-- exactly one of them whenever the lane runs. +INSERT INTO `programs` (`id`,`title`,`priceCents`,`status`,`created_ts`) VALUES + (1, 'Foundations', 4999, 'PUBLISHED', '2026-05-01 10:00:00'), + (2, 'Strength', 2500, 'PUBLISHED', '2026-05-17 23:30:00'), + (3, 'Mobility', 1000, 'DRAFT', '2026-06-01 00:00:00'), + (4, 'Legacy', 700, 'ARCHIVED', '2026-05-31 23:59:59'), + (5, 'Monday', 300, 'PUBLISHED', '2026-05-18 00:00:00'), + (6, 'Fresh', 1000, 'PUBLISHED', UTC_TIMESTAMP(3) - INTERVAL 3 DAY), + (7, 'Stale', 2000, 'PUBLISHED', UTC_TIMESTAMP(3) - INTERVAL 60 DAY); +INSERT INTO `weeks` (`id`,`programId`,`label`,`durationMinutes`) VALUES + (10, 1, 'Week 1', 30), + (11, 1, 'Week 2', 60), + (12, 1, 'Week 2', 90), + (13, 1, NULL, 60), + (20, 2, 'Solo', 45), + (30, 5, NULL, 20); +INSERT INTO `assets` (`id`,`ownerId`,`externalId`,`payload`,`recordedAt`,`observedAt`,`asOfDate`,`atTime`) VALUES + ('11111111-1111-4111-8111-111111111111','aaaaaaaa-aaaa-4aaa-8aaa-aaaaaaaaaaaa','22222222-2222-4222-8222-222222222222','{"k": 1}','2026-05-04 03:30:00','2026-05-04 03:30:00','2026-05-03','03:30:00'), + ('33333333-3333-4333-8333-333333333333','44444444-4444-4444-8444-444444444444','55555555-5555-4555-8555-555555555555','{"k": 2}','2026-05-04 03:45:00','2026-05-04 03:45:00','2026-05-03','03:45:00'), + ('66666666-6666-4666-8666-666666666666','77777777-7777-4777-8777-777777777777','88888888-8888-4888-8888-888888888888','{"k": 3}','2026-05-04 04:10:00','2026-05-04 04:10:00','2026-05-04','04:10:00'); diff --git a/fixtures/persistence-conformance/README.md b/fixtures/persistence-conformance/README.md index 9b1948c09..7776357a8 100644 --- a/fixtures/persistence-conformance/README.md +++ b/fixtures/persistence-conformance/README.md @@ -29,7 +29,9 @@ fixtures/persistence-conformance/ ├── report-shapes.json # TS-produced derived fields of each report; every port byte-matches it ├── canonical/ # SHARED "kitchen-sink" metadata + committed schema DDL │ ├── meta.*.json # the kitchen-sink metadata every query scenario reads -│ └── schema.postgres.sql # TS-produced canonical DDL every port executes to set up its DB +│ ├── schema.postgres.sql # TS-produced canonical DDL every port executes to set up its DB +│ ├── schema.sqlite.sql # the same for SQLite/D1 (`meta migrate --dialect sqlite`); report scenarios only +│ └── schema.mysql.sql # the same for MySQL 8.4 (adopter tables + `buildReportViews`); report scenarios only ├── migrations/ # per-scenario schema-evolution tests (TS-only) │ └── .yaml └── queries/ # query scenarios against the canonical schema (every port) @@ -321,6 +323,17 @@ port's runtime. They use only `op: list` and `op: count`, single-key `sort`, `fi `limit`: a report has no primary key, so there is no `op: get` and no write. The schema is still the committed `canonical/schema.postgres.sql`, which creates each report's view, so a port reads the view the TypeScript migrate engine produced and never lowers a report itself. + +The nine also run on SQLite and MySQL 8.4 (and TypeScript on D1's local runtime) in every +port whose runtime has the engine: they execute `canonical/schema.sqlite.sql` or +`canonical/schema.mysql.sql` instead, each TypeScript-produced and held equal to its generator +by a test. The `expect` blocks stay the Postgres wire ones; a runner maps the engine's own +spelling on the actual side (a SQLite ratio is a REAL, a MySQL `DECIMAL` keeps its scale, a +MySQL `DATETIME` is the UTC wall clock). A scenario whose seed cannot be shared (an instant +relative to the database clock) adds a `seed-data-engine:` map keyed by `sqlite` / `mysql`, +which wins over `seed-data` for that engine; the rest keep one Postgres-spelled `seed-data`, +which the MySQL runners translate mechanically (`"id"` to `` `id` ``, a UTC `Z` dropped). Java has +no SQLite driver and Python has only a Postgres one, so they do not run those engines. Rows are keyed by the report's **derived field names** (dimensions in `@dimensions` order, then measures in `@measures` order; a time dimension is named ``, for example `createdAtMonth`). diff --git a/fixtures/persistence-conformance/canonical/schema.mysql.sql b/fixtures/persistence-conformance/canonical/schema.mysql.sql new file mode 100644 index 000000000..c84254109 --- /dev/null +++ b/fixtures/persistence-conformance/canonical/schema.mysql.sql @@ -0,0 +1,117 @@ +-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT. +-- Canonical persistence-conformance schema (MySQL 8.4, literal column naming). +-- Tables: the adopter's hand-written DDL (meta migrate does not own MySQL, ADR-0015). +-- Views: buildReportViews(root, { dialect: "mysql" }), produced by TypeScript. +-- Only the tables the report-* scenarios read are declared. +-- Regenerate: `bun run gen:schema` in server/typescript/packages/integration-tests. + +CREATE TABLE `programs` ( + `id` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + `title` VARCHAR(200) NOT NULL, + `priceCents` BIGINT NOT NULL, + `status` VARCHAR(9) NOT NULL, + `created_ts` DATETIME(3) NOT NULL +); + +CREATE TABLE `weeks` ( + `id` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + `programId` BIGINT NOT NULL, + `label` VARCHAR(80), + `durationMinutes` INT NOT NULL, + FOREIGN KEY (`programId`) REFERENCES `programs` (`id`) +); + +CREATE TABLE `assets` ( + `id` VARCHAR(36) NOT NULL DEFAULT (UUID()) PRIMARY KEY, + `ownerId` VARCHAR(36) NOT NULL, + `externalId` VARCHAR(36), + `payload` JSON, + `recordedAt` DATETIME(3) NOT NULL, + `observedAt` DATETIME(3) NOT NULL, + `asOfDate` DATE NOT NULL, + `atTime` TIME(3) NOT NULL +); + +CREATE VIEW `v_program_minutes` AS + SELECT + w.`programId` AS `program`, + p.`title` AS `programTitle`, + COUNT(w.`id`) AS `weeks`, + COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) AS `longWeeks`, + COUNT(DISTINCT w.`label`) AS `labels`, + COUNT(DISTINCT w.`programId`, w.`durationMinutes`) AS `slots`, + CAST(SUM(w.`durationMinutes`) AS SIGNED) AS `totalMinutes`, + AVG(w.`durationMinutes`) AS `avgMinutes`, + MIN(w.`durationMinutes`) AS `minMinutes`, + MAX(w.`durationMinutes`) AS `maxMinutes`, + COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0) AS `longShare` + FROM `weeks` w + INNER JOIN `programs` p ON p.`id` = w.`programId` + GROUP BY w.`programId`, p.`title`; + +CREATE VIEW `v_fitness_totals` AS + SELECT + COUNT(w.`id`) AS `weeks`, + CAST(SUM(w.`durationMinutes`) AS SIGNED) AS `totalMinutes`, + COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0) AS `longShare` + FROM `weeks` w; + +CREATE VIEW `v_programs_by_month` AS + SELECT + DATE(DATE_FORMAT(p.`created_ts`, '%Y-%m-01')) AS `createdAtMonth`, + p.`status` AS `status`, + COUNT(p.`id`) AS `programs`, + CAST(SUM(CASE WHEN p.`status` = 'PUBLISHED' THEN p.`priceCents` END) AS SIGNED) AS `listValue` + FROM `programs` p + GROUP BY DATE(DATE_FORMAT(p.`created_ts`, '%Y-%m-01')), p.`status`; + +CREATE VIEW `v_programs_by_week` AS + SELECT + DATE(DATE_SUB(p.`created_ts`, INTERVAL WEEKDAY(p.`created_ts`) DAY)) AS `createdAtWeek`, + COUNT(p.`id`) AS `programs` + FROM `programs` p + WHERE p.`status` = 'PUBLISHED' + GROUP BY DATE(DATE_SUB(p.`created_ts`, INTERVAL WEEKDAY(p.`created_ts`) DAY)); + +CREATE VIEW `v_recent_programs` AS + SELECT + COUNT(p.`id`) AS `programs` + FROM `programs` p + WHERE p.`created_ts` >= (UTC_TIMESTAMP(3) - INTERVAL 30 DAY); + +CREATE VIEW `v_asset_activity` AS + SELECT + CAST(DATE_FORMAT(a.`recordedAt`, '%Y-%m-%d %H:00:00') AS DATETIME(3)) AS `recordedAtHour`, + DATE(DATE_SUB(a.`asOfDate`, INTERVAL WEEKDAY(a.`asOfDate`) DAY)) AS `asOfDateWeek`, + COUNT(a.`id`) AS `assets` + FROM `assets` a + GROUP BY CAST(DATE_FORMAT(a.`recordedAt`, '%Y-%m-%d %H:00:00') AS DATETIME(3)), DATE(DATE_SUB(a.`asOfDate`, INTERVAL WEEKDAY(a.`asOfDate`) DAY)); + +CREATE VIEW `v_program_roster` AS + SELECT + p.`id` AS `programKey`, + p.`title` AS `programTitle`, + COUNT(w.`id`) AS `weeks`, + CAST(SUM(w.`durationMinutes`) AS SIGNED) AS `totalMinutes`, + COALESCE(CAST(SUM(w.`durationMinutes`) AS SIGNED), 0) AS `totalMinutesOrZero`, + COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0) AS `longShare`, + COALESCE(COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0), 0) AS `longShareOrZero` + FROM `programs` p + LEFT OUTER JOIN `weeks` w ON p.`id` = w.`programId` + GROUP BY p.`id`, p.`title`; + +CREATE VIEW `v_program_long_weeks` AS + SELECT + p.`id` AS `programKey`, + COUNT(w.`id`) AS `weeks`, + COALESCE(CAST(SUM(w.`durationMinutes`) AS SIGNED), 0) AS `totalMinutesOrZero` + FROM `programs` p + LEFT OUTER JOIN `weeks` w ON p.`id` = w.`programId` AND w.`durationMinutes` >= 60 + GROUP BY p.`id`; + +CREATE VIEW `v_fitness_totals_filled` AS + SELECT + COUNT(w.`id`) AS `weeks`, + COALESCE(CAST(SUM(w.`durationMinutes`) AS SIGNED), 0) AS `totalMinutesOrZero`, + COALESCE(COUNT(CASE WHEN w.`durationMinutes` >= 60 THEN w.`id` END) / NULLIF(COUNT(w.`id`), 0), 0) AS `longShareOrZero` + FROM `weeks` w; diff --git a/fixtures/persistence-conformance/canonical/schema.sqlite.sql b/fixtures/persistence-conformance/canonical/schema.sqlite.sql new file mode 100644 index 000000000..463fc70c7 --- /dev/null +++ b/fixtures/persistence-conformance/canonical/schema.sqlite.sql @@ -0,0 +1,239 @@ +-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT. +-- Canonical persistence-conformance schema (SQLite). +-- Produced by TypeScript from canonical/meta.fitness.json; every port's +-- query-scenario runner executes this file verbatim to provision its test DB. +-- Regenerate: `bun run gen:schema` in this package. Drift-checked by +-- schema-artifact.test.ts (CI fails if this file diverges from metadata). + +CREATE TABLE "programs" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "title" VARCHAR(200) NOT NULL, + "priceCents" INTEGER NOT NULL, + "status" TEXT NOT NULL, + "created_ts" TEXT NOT NULL, + CONSTRAINT "programs_status_chk" CHECK ("status" IN ('DRAFT', 'PUBLISHED', 'ARCHIVED')) +); + +CREATE TABLE "weeks" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "programId" INTEGER NOT NULL, + "label" VARCHAR(80), + "durationMinutes" INTEGER NOT NULL, + FOREIGN KEY ("programId") REFERENCES "programs" ("id") ON DELETE CASCADE ON UPDATE CASCADE +); + +CREATE TABLE "nodes" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "label" VARCHAR(80) NOT NULL, + "parentId" INTEGER, + FOREIGN KEY ("parentId") REFERENCES "nodes" ("id") +); + +CREATE TABLE "measurements" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "tempC" REAL NOT NULL, + "massKg" REAL NOT NULL, + "preciseKg" NUMERIC(9,4) NOT NULL +); + +CREATE TABLE "assets" ( + "id" TEXT PRIMARY KEY NOT NULL DEFAULT (lower(hex(randomblob(16)))), + "ownerId" TEXT NOT NULL, + "externalId" TEXT, + "payload" TEXT, + "recordedAt" TEXT NOT NULL, + "observedAt" TEXT NOT NULL, + "asOfDate" TEXT NOT NULL, + "atTime" TEXT NOT NULL +); + +CREATE TABLE "posts" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "title" VARCHAR(200) NOT NULL +); + +CREATE TABLE "tags" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "name" VARCHAR(80) NOT NULL +); + +CREATE TABLE "post_tags" ( + "postId" INTEGER NOT NULL, + "tagId" INTEGER NOT NULL, + PRIMARY KEY ("postId", "tagId"), + FOREIGN KEY ("postId") REFERENCES "posts" ("id"), + FOREIGN KEY ("tagId") REFERENCES "tags" ("id") +); + +CREATE TABLE "people" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "name" VARCHAR(80) NOT NULL +); + +CREATE TABLE "follows" ( + "followerId" INTEGER NOT NULL, + "followeeId" INTEGER NOT NULL, + PRIMARY KEY ("followerId", "followeeId"), + FOREIGN KEY ("followerId") REFERENCES "people" ("id"), + FOREIGN KEY ("followeeId") REFERENCES "people" ("id") +); + +CREATE TABLE "friendships" ( + "personAId" INTEGER NOT NULL, + "personBId" INTEGER NOT NULL, + PRIMARY KEY ("personAId", "personBId"), + FOREIGN KEY ("personAId") REFERENCES "people" ("id"), + FOREIGN KEY ("personBId") REFERENCES "people" ("id") +); + +CREATE TABLE "auths" ( + "id" INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL, + "type" TEXT, + "reference" VARCHAR(80) NOT NULL, + "quantity" INTEGER, + "copayAmount" NUMERIC(10,2), + "approver" VARCHAR(80), + "urgency" VARCHAR(16), + CONSTRAINT "auths_type_chk" CHECK ("type" IN ('Bridge', 'Copay', 'PriorAuth', 'Referral')) +); + +CREATE TABLE "post_referrals" ( + "postId" INTEGER NOT NULL, + "referralId" INTEGER NOT NULL, + PRIMARY KEY ("postId", "referralId"), + FOREIGN KEY ("postId") REFERENCES "posts" ("id"), + FOREIGN KEY ("referralId") REFERENCES "auths" ("id") +); + +CREATE TABLE "all_types" ( + "id" TEXT PRIMARY KEY NOT NULL DEFAULT (lower(hex(randomblob(16)))), + "sVal" VARCHAR(200) NOT NULL, + "iVal" INTEGER NOT NULL, + "lVal" INTEGER NOT NULL, + "dVal" REAL NOT NULL, + "fVal" REAL NOT NULL, + "decVal" NUMERIC(18,6) NOT NULL, + "bVal" INTEGER NOT NULL, + "dateVal" TEXT NOT NULL, + "timeVal" TEXT NOT NULL, + "tsVal" TEXT NOT NULL, + "tsTzVal" TEXT NOT NULL, + "moneyVal" INTEGER NOT NULL, + "enumVal" TEXT NOT NULL, + "intEnumVal" INTEGER, + "uuidVal" TEXT NOT NULL, + "uriVal" TEXT NOT NULL, + "inetVal" TEXT NOT NULL, + "inet6Val" TEXT NOT NULL, + "settings" TEXT, + "labels" TEXT, + CONSTRAINT "all_types_enumVal_chk" CHECK ("enumVal" IN ('LOW', 'MEDIUM', 'HIGH')), + CONSTRAINT "all_types_intEnumVal_chk" CHECK ("intEnumVal" IN (0, 5, 9)) +); + +CREATE UNIQUE INDEX "byTitle" ON "programs" ("title"); + +CREATE INDEX "idx_programs_title_status" ON "programs" ("title", "status"); + +CREATE VIEW "v_program" AS + SELECT + p.id AS id, + p.title AS title, + p.status AS status + FROM programs p; + +CREATE VIEW "v_program_stat" AS + SELECT + p.id AS "programId", + COUNT(DISTINCT w.id) AS "weekCount", + SUM(w."durationMinutes") AS "totalMinutes", + AVG(w."durationMinutes") AS "avgMinutes", + MIN(w."durationMinutes") AS "minMinutes", + MAX(w."durationMinutes") AS "maxMinutes" + FROM programs p + LEFT OUTER JOIN weeks w ON w."programId" = p.id + GROUP BY p.id; + +CREATE VIEW "v_program_minutes" AS + SELECT + w."programId" AS "program", + p."title" AS "programTitle", + COUNT(w."id") AS "weeks", + COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS "longWeeks", + COUNT(DISTINCT w."label") AS "labels", + COUNT(DISTINCT CASE WHEN w."programId" IS NOT NULL AND w."durationMinutes" IS NOT NULL THEN json_array(w."programId", w."durationMinutes") END) AS "slots", + SUM(w."durationMinutes") AS "totalMinutes", + AVG(w."durationMinutes") AS "avgMinutes", + MIN(w."durationMinutes") AS "minMinutes", + MAX(w."durationMinutes") AS "maxMinutes", + CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0) AS "longShare" + FROM "weeks" w + INNER JOIN "programs" p ON p."id" = w."programId" + GROUP BY w."programId", p."title"; + +CREATE VIEW "v_fitness_totals" AS + SELECT + COUNT(w."id") AS "weeks", + SUM(w."durationMinutes") AS "totalMinutes", + CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0) AS "longShare" + FROM "weeks" w; + +CREATE VIEW "v_programs_by_month" AS + SELECT + date(p."created_ts", 'start of month') AS "createdAtMonth", + p."status" AS "status", + COUNT(p."id") AS "programs", + SUM(CASE WHEN p."status" = 'PUBLISHED' THEN p."priceCents" END) AS "listValue" + FROM "programs" p + GROUP BY date(p."created_ts", 'start of month'), p."status"; + +CREATE VIEW "v_programs_by_week" AS + SELECT + date(p."created_ts", 'weekday 0', '-6 days') AS "createdAtWeek", + COUNT(p."id") AS "programs" + FROM "programs" p + WHERE p."status" = 'PUBLISHED' + GROUP BY date(p."created_ts", 'weekday 0', '-6 days'); + +CREATE VIEW "v_recent_programs" AS + SELECT + COUNT(p."id") AS "programs" + FROM "programs" p + WHERE p."created_ts" >= strftime('%Y-%m-%dT%H:%M:%f', 'now', '-30 days'); + +CREATE VIEW "v_asset_activity" AS + SELECT + strftime('%Y-%m-%dT%H:00:00.000Z', a."recordedAt") AS "recordedAtHour", + date(a."asOfDate", 'weekday 0', '-6 days') AS "asOfDateWeek", + COUNT(a."id") AS "assets" + FROM "assets" a + GROUP BY strftime('%Y-%m-%dT%H:00:00.000Z', a."recordedAt"), date(a."asOfDate", 'weekday 0', '-6 days'); + +CREATE VIEW "v_program_roster" AS + SELECT + p."id" AS "programKey", + p."title" AS "programTitle", + COUNT(w."id") AS "weeks", + SUM(w."durationMinutes") AS "totalMinutes", + COALESCE(SUM(w."durationMinutes"), 0) AS "totalMinutesOrZero", + CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0) AS "longShare", + COALESCE(CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0), 0.0) AS "longShareOrZero" + FROM "programs" p + LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" + GROUP BY p."id", p."title"; + +CREATE VIEW "v_program_long_weeks" AS + SELECT + p."id" AS "programKey", + COUNT(w."id") AS "weeks", + COALESCE(SUM(w."durationMinutes"), 0) AS "totalMinutesOrZero" + FROM "programs" p + LEFT OUTER JOIN "weeks" w ON p."id" = w."programId" AND w."durationMinutes" >= 60 + GROUP BY p."id"; + +CREATE VIEW "v_fitness_totals_filled" AS + SELECT + COUNT(w."id") AS "weeks", + COALESCE(SUM(w."durationMinutes"), 0) AS "totalMinutesOrZero", + COALESCE(CAST(COUNT(CASE WHEN w."durationMinutes" >= 60 THEN w."id" END) AS REAL) / NULLIF(COUNT(w."id"), 0), 0.0) AS "longShareOrZero" + FROM "weeks" w; diff --git a/fixtures/persistence-conformance/queries/report-relative-date.yaml b/fixtures/persistence-conformance/queries/report-relative-date.yaml index 6669beac2..779074638 100644 --- a/fixtures/persistence-conformance/queries/report-relative-date.yaml +++ b/fixtures/persistence-conformance/queries/report-relative-date.yaml @@ -10,6 +10,15 @@ seed-data: | INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES (1, 'Fresh', 1000, 'PUBLISHED', (now() AT TIME ZONE 'UTC') - INTERVAL '3 days'), (2, 'Stale', 2000, 'PUBLISHED', (now() AT TIME ZONE 'UTC') - INTERVAL '60 days'); +seed-data-engine: + sqlite: | + INSERT INTO "programs" ("id","title","priceCents","status","created_ts") VALUES + (1, 'Fresh', 1000, 'PUBLISHED', strftime('%Y-%m-%dT%H:%M:%fZ','now','-3 days')), + (2, 'Stale', 2000, 'PUBLISHED', strftime('%Y-%m-%dT%H:%M:%fZ','now','-60 days')); + mysql: | + INSERT INTO `programs` (`id`,`title`,`priceCents`,`status`,`created_ts`) VALUES + (1, 'Fresh', 1000, 'PUBLISHED', UTC_TIMESTAMP(3) - INTERVAL 3 DAY), + (2, 'Stale', 2000, 'PUBLISHED', UTC_TIMESTAMP(3) - INTERVAL 60 DAY); queries: - name: only-the-recent-program-counts op: list diff --git a/scripts/integration-test.sh b/scripts/integration-test.sh index 0b7053cb0..3adae422c 100755 --- a/scripts/integration-test.sh +++ b/scripts/integration-test.sh @@ -5,7 +5,9 @@ # fixture corpus through every shipped runner: TypeScript (Bun), C# (dotnet), # Java (Maven), Python (uv + pytest), and Kotlin (Maven). Each runner # exercises that port's metaobjects persistence layer (codegen + runtime) -# against a real Postgres instance. +# against a real Postgres instance. The FR-044 report scenarios also run on MySQL 8.4 and +# SQLite where a port's runtime has the engine (see docs/CONFORMANCE.md); the lanes need no +# extra flags, the MySQL ones start their own mysql:8.4 container. # # The Java- and Kotlin-runner Maven modules are intentionally NOT in the # parent reactor — they require docker and we keep `mvn test` docker-free. @@ -24,7 +26,8 @@ # into a real Cube (cubejs/cube, pinned) over its own throwaway Postgres and compares each # report's Cube query with the report's view (FR-044). It owns its containers and a private # network and binds only an ephemeral 127.0.0.1 port. `all` leaves it out because the image -# is about 1 GB; scripts/ci-local.sh runs it as its own `cube` lane. +# is about 1 GB; scripts/ci-local.sh runs it as its own `cube` lane. It runs two files: the +# Postgres path and the MySQL path (Cube against a real MySQL 8.4 holding the report views). # # Pre-flight: docker daemon must be running. diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractCorpusPaths.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractCorpusPaths.cs index b7360598b..78903ca7c 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractCorpusPaths.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractCorpusPaths.cs @@ -61,6 +61,8 @@ internal static class ApiContractCorpusPaths public static readonly string ReportSeedFile = Path.Combine(ReportDir, "seed.json"); public static readonly string ReportMetaJson = Path.Combine(ReportDir, "meta.json"); public static readonly string ReportSchemaSql = Path.Combine(ReportDir, "schema.postgres.sql"); + public static readonly string ReportSchemaSqliteSql = Path.Combine(ReportDir, "schema.sqlite.sql"); + public static readonly string ReportSchemaMysqlSql = Path.Combine(ReportDir, "schema.mysql.sql"); // #214 write-through read-your-writes subcorpus. public static readonly string WriteThroughDir = Path.Combine(Corpus, "write-through"); diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs index 57460544c..3f33a8055 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ApiContractReportConformanceTest.cs @@ -18,7 +18,8 @@ namespace MetaObjects.IntegrationTests.Api; -public sealed class ApiContractReportConformanceTest +[Collection(MySqlCollection.Name)] +public sealed class ApiContractReportConformanceTest(MySqlServer mysqlServer) { [Theory] [MemberData(nameof(Scenarios))] @@ -31,6 +32,34 @@ public async Task Api_contract_report_generated(string scenarioPath) await RunAsync(scenario, server.BaseUrl); } + // The same generated routes, over SQLite: the schema is the TypeScript-produced + // report/schema.sqlite.sql, so the four views are the ones `meta migrate --dialect + // sqlite` creates. A ratio reads as a REAL (no decimal type), which the corpus's + // numeric comparison already treats as the number it is. + [Theory] + [MemberData(nameof(Scenarios))] + public async Task Api_contract_report_generated_on_sqlite(string scenarioPath) + { + var scenario = ApiContractScenarioLoader.LoadScenario(scenarioPath); + using var db = new SqliteTempDatabase("mo-report-api-"); + await using var server = await ReportGeneratedServerFactory.StartSqliteAsync(db.FilePath); + await server.ApplySeedAsync(); + await RunAsync(scenario, server.BaseUrl); + } + + // ...and over MySQL 8.4: the schema is report/schema.mysql.sql (the adopter's tables plus + // the views TypeScript lowers with buildReportViews). A ratio reads as a DECIMAL. + [Theory] + [MemberData(nameof(Scenarios))] + public async Task Api_contract_report_generated_on_mysql(string scenarioPath) + { + var scenario = ApiContractScenarioLoader.LoadScenario(scenarioPath); + await using var mysql = await mysqlServer.CreateDatabaseAsync(); + await using var server = await ReportGeneratedServerFactory.StartMySqlAsync(mysql); + await server.ApplySeedAsync(); + await RunAsync(scenario, server.BaseUrl); + } + private static async Task RunAsync(ApiScenario scenario, string baseUrl) { using var client = new HttpClient { BaseAddress = new Uri(baseUrl) }; diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs index 6643de74e..be9f6171d 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ReportFixture.cs @@ -16,7 +16,7 @@ using System.Text.Json; // JsonValueKind using System.Text.Json.Nodes; // JsonNode / JsonObject / JsonArray -using Npgsql; +using MetaObjects.IntegrationTests.Runner; namespace MetaObjects.IntegrationTests.Api; @@ -25,14 +25,16 @@ internal static class ReportFixture /// The one top-level key of seed.json that is not a base table. private const string SeedReportsKey = "reports"; - /// Execute the committed TypeScript-produced schema on a fresh container. - public static async Task ProvisionSchemaAsync(string connString) + /// Execute the committed TypeScript-produced schema for this engine's artifact. + public static async Task ProvisionSchemaAsync(string connString, ScenarioEngine engine = ScenarioEngine.Postgres) { - await using var c = new NpgsqlConnection(connString); - await c.OpenAsync(); - await using var cmd = c.CreateCommand(); - cmd.CommandText = await File.ReadAllTextAsync(ApiContractCorpusPaths.ReportSchemaSql); - await cmd.ExecuteNonQueryAsync(); + var path = engine switch + { + ScenarioEngine.Sqlite => ApiContractCorpusPaths.ReportSchemaSqliteSql, + ScenarioEngine.MySql => ApiContractCorpusPaths.ReportSchemaMysqlSql, + _ => ApiContractCorpusPaths.ReportSchemaSql, + }; + await engine.ExecuteScriptAsync(connString, await File.ReadAllTextAsync(path)); } /// @@ -41,7 +43,7 @@ public static async Task ProvisionSchemaAsync(string connString) /// full-stack lane. Values are SQL literals, as the TypeScript lane writes them, so /// Postgres types each one by its column (`'2026-04-30'` into a DATE). /// - public static async Task ApplySeedAsync(string connString, string seedPath) + public static async Task ApplySeedAsync(string connString, string seedPath, ScenarioEngine engine = ScenarioEngine.Postgres) { var root = JsonNode.Parse(File.ReadAllText(seedPath)) as JsonObject ?? throw new InvalidOperationException($"{seedPath}: top-level must be an object"); @@ -49,8 +51,7 @@ public static async Task ApplySeedAsync(string connString, string seedPath) if (tables.Count == 0) throw new InvalidOperationException($"{seedPath}: no base-table rows to seed"); - await using var c = new NpgsqlConnection(connString); - await c.OpenAsync(); + await using var c = await engine.OpenConnectionAsync(connString); foreach (var (table, rowsNode) in tables) { @@ -60,15 +61,17 @@ public static async Task ApplySeedAsync(string connString, string seedPath) { if (rowNode is not JsonObject row) throw new InvalidOperationException($"{seedPath}: '{table}' has a row that is not an object"); - var colList = string.Join(", ", row.Select(kv => "\"" + kv.Key + "\"")); + var quote = engine.Quote(); + var colList = string.Join(", ", row.Select(kv => quote + kv.Key + quote)); var valueList = string.Join(", ", row.Select(kv => SqlLiteral(seedPath, table, kv.Key, kv.Value))); await using var ins = c.CreateCommand(); - ins.CommandText = $"INSERT INTO \"{table}\" ({colList}) VALUES ({valueList})"; + ins.CommandText = $"INSERT INTO {quote}{table}{quote} ({colList}) VALUES ({valueList})"; await ins.ExecuteNonQueryAsync(); } - // The seed writes explicit ids; move each identity past them. - if (rows.Any(r => r is JsonObject o && o.ContainsKey("id"))) + // The seed writes explicit ids; move each identity past them. (SQLite's rowid + // alias already continues from the largest id.) + if (engine == ScenarioEngine.Postgres && rows.Any(r => r is JsonObject o && o.ContainsKey("id"))) { await using var bump = c.CreateCommand(); bump.CommandText = diff --git a/server/csharp/MetaObjects.IntegrationTests/Api/ReportGeneratedServerFactory.cs b/server/csharp/MetaObjects.IntegrationTests/Api/ReportGeneratedServerFactory.cs index 5606beff3..881f2a23a 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Api/ReportGeneratedServerFactory.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Api/ReportGeneratedServerFactory.cs @@ -5,7 +5,8 @@ // + Routes + Names) on the report model (the Invoice, Product and Sale tables, four // view-backed reports — one of them, ProductRevenue, a @spine report with a defaulted // measure — and one sourceless report), Roslyn-compiles the emitted sources in-memory, and -// hosts them on Kestrel against Testcontainers Postgres with the four views present. A +// hosts them on Kestrel against Testcontainers Postgres (or, for StartSqliteAsync / +// StartMySqlAsync, a SQLite file / a MySQL database) with the four views present. A // failing scenario is a real generator bug (RoutesGenerator / DbContextGenerator / // FilterAllowlistGenerator / ReportRows), never something fixed by hand-editing emitted code. // @@ -43,21 +44,34 @@ internal sealed class ReportGeneratedServerFactory : IAsyncDisposable ["Invoice", "InvoiceStatusTotals", "InvoiceTotals", "InvoicesByMonth", "Product", "ProductRevenue", "Sale"]; private const string SourcelessReport = "InvoiceDays"; - private readonly PostgresContainer _pg; + private readonly ScenarioEngine _engine; + private readonly string _connectionString; private readonly WebApplication _app; public string BaseUrl { get; } - private ReportGeneratedServerFactory(PostgresContainer pg, WebApplication app, string baseUrl) + private ReportGeneratedServerFactory(ScenarioEngine engine, string connectionString, WebApplication app, string baseUrl) { - _pg = pg; + _engine = engine; + _connectionString = connectionString; _app = app; BaseUrl = baseUrl; } - public static async Task StartAsync(PostgresContainer pg) + public static Task StartAsync(PostgresContainer pg) => + StartAsync(ScenarioEngine.Postgres, pg.ConnectionString); + + /// The same generated server over a SQLite database file the caller owns. + public static Task StartSqliteAsync(string databasePath) => + StartAsync(ScenarioEngine.Sqlite, $"Data Source={databasePath}"); + + /// The same generated server over a MySQL database (a fresh, empty one). + public static Task StartMySqlAsync(MySqlDatabase mysql) => + StartAsync(ScenarioEngine.MySql, mysql.ConnectionString); + + private static async Task StartAsync(ScenarioEngine engine, string connectionString) { - await ReportFixture.ProvisionSchemaAsync(pg.ConnectionString); + await ReportFixture.ProvisionSchemaAsync(connectionString, engine); var (assembly, routedNames) = CompileGeneratedServer(); var dbContextType = assembly.GetType($"{GeneratedNamespace}.AppDbContext") @@ -75,7 +89,7 @@ public static async Task StartAsync(PostgresContai // as the TPH lane does for its discriminator. builder.Services.ConfigureHttpJsonOptions(o => o.SerializerOptions.Converters.Add(new System.Text.Json.Serialization.JsonStringEnumConverter())); - RegisterGeneratedDbContext(builder.Services, dbContextType, pg.ConnectionString); + RegisterGeneratedDbContext(builder.Services, dbContextType, engine, connectionString); var app = builder.Build(); @@ -92,11 +106,11 @@ public static async Task StartAsync(PostgresContai } await app.StartAsync(); - return new ReportGeneratedServerFactory(pg, app, baseUrl); + return new ReportGeneratedServerFactory(engine, connectionString, app, baseUrl); } public async Task ApplySeedAsync() => - await ReportFixture.ApplySeedAsync(_pg.ConnectionString, ApiContractCorpusPaths.ReportSeedFile); + await ReportFixture.ApplySeedAsync(_connectionString, ApiContractCorpusPaths.ReportSeedFile, _engine); public async ValueTask DisposeAsync() { @@ -204,7 +218,7 @@ private static List BuildReferenceSet() } private static void RegisterGeneratedDbContext( - IServiceCollection services, Type dbContextType, string connString) + IServiceCollection services, Type dbContextType, ScenarioEngine engine, string connString) { var addDbContext = typeof(EntityFrameworkServiceCollectionExtensions) .GetMethods(BindingFlags.Public | BindingFlags.Static) @@ -214,7 +228,9 @@ private static void RegisterGeneratedDbContext( && m.GetParameters().Length == 4) .MakeGenericMethod(dbContextType); - Action configure = opts => opts.UseNpgsql(connString); + // Provider choice lives on the engine (ScenarioEngineExtensions.UseEngine) — the one + // MySqlServerVersion pin too, so the provider and the mysql:8.4 image cannot drift apart. + Action configure = opts => opts.UseEngine(engine, connString); addDbContext.Invoke(null, new object?[] { services, configure, ServiceLifetime.Scoped, ServiceLifetime.Scoped, diff --git a/server/csharp/MetaObjects.IntegrationTests/MetaObjects.IntegrationTests.csproj b/server/csharp/MetaObjects.IntegrationTests/MetaObjects.IntegrationTests.csproj index cce58acdc..55080c32c 100644 --- a/server/csharp/MetaObjects.IntegrationTests/MetaObjects.IntegrationTests.csproj +++ b/server/csharp/MetaObjects.IntegrationTests/MetaObjects.IntegrationTests.csproj @@ -35,5 +35,11 @@ + + + + + diff --git a/server/csharp/MetaObjects.IntegrationTests/QueryScenarioMySqlTests.cs b/server/csharp/MetaObjects.IntegrationTests/QueryScenarioMySqlTests.cs new file mode 100644 index 000000000..a095ee322 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/QueryScenarioMySqlTests.cs @@ -0,0 +1,29 @@ +// QueryScenarioMySqlTests — the FR-044 report scenarios (persistence-conformance +// queries/report-*.yaml) on MySQL 8.4, through the same generated EF Core AppDbContext +// (Pomelo provider). +// +// The schema is the committed canonical/schema.mysql.sql: the adopter's tables (MySQL is not +// owned by `meta migrate`, ADR-0015) plus the report views TypeScript lowers with +// buildReportViews; C# never writes view SQL. The expectations are the Postgres wire ones, +// unchanged: a MySQL DECIMAL keeps its scale and a DATETIME holds the UTC wall clock, and +// Normalization maps the engine spelling on the ACTUAL side only. + +using MetaObjects.IntegrationTests.Runner; +using Xunit; + +namespace MetaObjects.IntegrationTests; + +[Collection(MySqlCollection.Name)] +public sealed class QueryScenarioMySqlTests(MySqlServer server) +{ + [Theory] + [MemberData(nameof(Scenarios))] + public async Task Report_scenario_on_mysql(string scenarioPath) + { + var scenario = ScenarioLoader.LoadQuery(scenarioPath); + await using var mysql = await server.CreateDatabaseAsync(); + await QueryScenarioRunner.RunMySqlAsync(scenario, mysql.ConnectionString); + } + + public static IEnumerable Scenarios() => CorpusPaths.ReportQueryScenarios(); +} diff --git a/server/csharp/MetaObjects.IntegrationTests/QueryScenarioSqliteTests.cs b/server/csharp/MetaObjects.IntegrationTests/QueryScenarioSqliteTests.cs new file mode 100644 index 000000000..d5cc5d274 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/QueryScenarioSqliteTests.cs @@ -0,0 +1,27 @@ +// QueryScenarioSqliteTests — the FR-044 report scenarios (persistence-conformance +// queries/report-*.yaml) on SQLite, through the same generated EF Core AppDbContext. +// +// The schema is the TypeScript-produced canonical/schema.sqlite.sql (tables + the report +// views `meta migrate --dialect sqlite` creates); C# never writes view SQL (ADR-0015). +// The expectations are the Postgres wire ones, unchanged: SQLite has one INTEGER class +// and REAL ratios, and Normalization maps the engine spelling on the ACTUAL side only. +// No container: each scenario gets a throwaway database file. + +using MetaObjects.IntegrationTests.Runner; +using Xunit; + +namespace MetaObjects.IntegrationTests; + +public sealed class QueryScenarioSqliteTests +{ + [Theory] + [MemberData(nameof(Scenarios))] + public async Task Report_scenario_on_sqlite(string scenarioPath) + { + var scenario = ScenarioLoader.LoadQuery(scenarioPath); + using var db = new SqliteTempDatabase("mo-report-"); + await QueryScenarioRunner.RunSqliteAsync(scenario, db.FilePath); + } + + public static IEnumerable Scenarios() => CorpusPaths.ReportQueryScenarios(); +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Runner/CorpusPaths.cs b/server/csharp/MetaObjects.IntegrationTests/Runner/CorpusPaths.cs index c914e7007..7e10b5b35 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Runner/CorpusPaths.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Runner/CorpusPaths.cs @@ -15,6 +15,16 @@ internal static class CorpusPaths public static readonly string CanonicalDir = Path.Combine(Corpus, "canonical"); public static readonly string QueriesDir = Path.Combine(Corpus, "queries"); + /// + /// The report-* query scenarios as xUnit member data — the only corpus subset whose + /// tables exist in the SQLite and MySQL schema artifacts. One home for the name predicate, + /// so a dialect-gated scenario cannot be silently skipped by one lane's enumeration. + /// + public static IEnumerable ReportQueryScenarios() => + Directory.EnumerateFiles(QueriesDir, "report-*.yaml", SearchOption.TopDirectoryOnly) + .OrderBy(p => p, StringComparer.Ordinal) + .Select(p => new object[] { p }); + /// /// The committed canonical Postgres schema artifact. TypeScript is the single /// producer of this DDL (base tables + projection views, literal column naming); @@ -24,6 +34,21 @@ internal static class CorpusPaths public static readonly string CanonicalSchemaSql = Path.Combine(CanonicalDir, "schema.postgres.sql"); + /// + /// The committed canonical SQLite schema artifact — the same tables and report views, + /// produced by TypeScript (meta migrate --dialect sqlite's diff + emit). + /// + public static readonly string CanonicalSchemaSqliteSql = + Path.Combine(CanonicalDir, "schema.sqlite.sql"); + + /// + /// The committed canonical MySQL schema artifact: the adopter's tables (MySQL is not owned + /// by meta migrate, ADR-0015) plus the report views TypeScript lowers with + /// buildReportViews. + /// + public static readonly string CanonicalSchemaMysqlSql = + Path.Combine(CanonicalDir, "schema.mysql.sql"); + /// /// The canonical entity metadata for the corpus. The M:N runtime resolver /// () loads it to derive junction FK fields from the diff --git a/server/csharp/MetaObjects.IntegrationTests/Runner/MySqlDatabase.cs b/server/csharp/MetaObjects.IntegrationTests/Runner/MySqlDatabase.cs new file mode 100644 index 000000000..3a271f1a8 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Runner/MySqlDatabase.cs @@ -0,0 +1,113 @@ +// MySqlServer — one MySQL server shared by the MySQL test collection, and MySqlDatabase — a +// fresh, isolated database on it for one scenario. +// +// The server is, in order: +// 1. Existing server: METAOBJECTS_TEST_MYSQL_URL names a running MySQL as +// mysql://user:pass@host:port/db (an admin login), the form the TypeScript suite reads. +// 2. Otherwise one mysql:8.4 Testcontainers container, started on first use and removed when +// the collection finishes. +// Either way each scenario gets a uniquely-named database, dropped on dispose. + +using MySqlConnector; +using Testcontainers.MySql; +using Xunit; + +namespace MetaObjects.IntegrationTests.Runner; + +public sealed class MySqlServer : IAsyncLifetime +{ + private const string ExistingServerEnv = "METAOBJECTS_TEST_MYSQL_URL"; + + private readonly Lazy> _adminConnectionString; + private MySqlContainer? _container; // null until started, and when an existing server is used + + public MySqlServer() + { + _adminConnectionString = new Lazy>(StartAsync); + } + + private async Task StartAsync() + { + var existing = Environment.GetEnvironmentVariable(ExistingServerEnv); + if (!string.IsNullOrWhiteSpace(existing)) return FromUrl(existing); + + _container = new MySqlBuilder().WithImage("mysql:8.4").WithUsername("root").Build(); + await _container.StartAsync(); + return _container.GetConnectionString(); + } + + private static string FromUrl(string url) + { + var uri = new Uri(url); + var userInfo = uri.UserInfo.Split(':', 2); + var builder = new MySqlConnectionStringBuilder + { + Server = uri.Host, + Port = uri.Port == -1 ? 3306u : (uint)uri.Port, + UserID = Uri.UnescapeDataString(userInfo[0]), + Password = userInfo.Length > 1 ? Uri.UnescapeDataString(userInfo[1]) : "", + }; + var database = uri.AbsolutePath.TrimStart('/'); + if (database.Length > 0) builder.Database = Uri.UnescapeDataString(database); + return builder.ConnectionString; + } + + /// A fresh, empty database on this server, dropped when the result is disposed. + public async Task CreateDatabaseAsync() + { + var admin = await _adminConnectionString.Value; + var createdDb = "mo_test_cs_" + Guid.NewGuid().ToString("N"); + await using (var connection = new MySqlConnection(admin)) + { + await connection.OpenAsync(); + // Generated database name — no user input; safe to inline. + await using var cmd = new MySqlCommand($"CREATE DATABASE `{createdDb}`", connection); + await cmd.ExecuteNonQueryAsync(); + } + var scenario = new MySqlConnectionStringBuilder(admin) { Database = createdDb }; + return new MySqlDatabase(scenario.ConnectionString, admin, createdDb); + } + + public Task InitializeAsync() => Task.CompletedTask; + + public async Task DisposeAsync() + { + if (_container is not null) await _container.DisposeAsync(); + } +} + +[CollectionDefinition(Name)] +public sealed class MySqlCollection : ICollectionFixture +{ + public const string Name = "MySQL"; +} + +public sealed class MySqlDatabase : IAsyncDisposable +{ + public string ConnectionString { get; } + + private readonly string _adminConnectionString; + private readonly string _createdDb; + + internal MySqlDatabase(string connectionString, string adminConnectionString, string createdDb) + { + ConnectionString = connectionString; + _adminConnectionString = adminConnectionString; + _createdDb = createdDb; + } + + public async ValueTask DisposeAsync() + { + try + { + await using var admin = new MySqlConnection(_adminConnectionString); + await admin.OpenAsync(); + await using var cmd = new MySqlCommand($"DROP DATABASE IF EXISTS `{_createdDb}`", admin); + await cmd.ExecuteNonQueryAsync(); + } + catch + { + // Best-effort cleanup. + } + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Runner/QueryScenarioRunner.cs b/server/csharp/MetaObjects.IntegrationTests/Runner/QueryScenarioRunner.cs index 4daedc2ae..0208f80d7 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Runner/QueryScenarioRunner.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Runner/QueryScenarioRunner.cs @@ -30,21 +30,50 @@ namespace MetaObjects.IntegrationTests.Runner; public static class QueryScenarioRunner { - public static async Task RunAsync(QueryScenario scenario, PostgresContainer pg) + public static Task RunAsync(QueryScenario scenario, PostgresContainer pg) => + RunAsync(scenario, ScenarioEngine.Postgres, pg.ConnectionString); + + /// + /// Run a scenario against a SQLite database file the caller owns (an empty file path is + /// created on first use). The expectations are the Postgres wire ones: the engine + /// spelling is mapped only by on the actual side. + /// + public static Task RunSqliteAsync(QueryScenario scenario, string databasePath) => + RunAsync(scenario, ScenarioEngine.Sqlite, $"Data Source={databasePath}"); + + /// + /// Run a scenario against a MySQL database (a fresh, empty one). The expectations are the + /// Postgres wire ones, unchanged; maps the engine spelling. + /// + public static Task RunMySqlAsync(QueryScenario scenario, string connectionString) => + RunAsync(scenario, ScenarioEngine.MySql, connectionString); + + private static async Task RunAsync(QueryScenario scenario, ScenarioEngine engine, string connectionString) { // Provision the schema from the committed canonical DDL — the single // TS-produced artifact every port executes. (No per-scenario synthesis.) - await ExecuteAsync(pg.ConnectionString, ReadCanonicalSchemaSql()); + await engine.ExecuteScriptAsync(connectionString, ReadCanonicalSchemaSql(engine)); - // Optional seed data. - if (!string.IsNullOrWhiteSpace(scenario.SeedData)) - await ExecuteAsync(pg.ConnectionString, scenario.SeedData); + // Optional seed data (a per-engine `seed-data-engine` entry wins over `seed-data`). + var seed = scenario.SeedFor(engine.Name()); + if (!string.IsNullOrWhiteSpace(seed)) + { + // The corpus seed is Postgres SQL unless a per-engine entry spells it; MySQL's + // identifier quote differs, so a Postgres-spelled seed is translated. + if (engine == ScenarioEngine.MySql && !scenario.HasSeedFor(engine.Name())) + seed = MySqlSeed.FromPostgres(seed); + await engine.ExecuteScriptAsync(connectionString, seed); + } - // Open a DbContext + run each query. - var options = new DbContextOptionsBuilder() - .UseNpgsql(pg.ConnectionString) - .Options; - await using var db = new AppDbContext(options); + // Open a DbContext + run each query. (Provider choice: ScenarioEngineExtensions.UseEngine.) + var builder = new DbContextOptionsBuilder(); + builder.UseEngine(engine, connectionString); + var options = builder.Options; + // Postgres runs the whole generated context; the other engines read the report views + // through the view-only subset (see ReportViewsDbContext). + await using AppDbContext db = engine == ScenarioEngine.Postgres + ? new AppDbContext(options) + : new ReportViewsDbContext(options); // Metadata is loaded lazily — only `op: relate` (M:N traversal) needs it, // and it drives the junction-FK derivation via the shared M2MDerivation @@ -56,7 +85,9 @@ public static async Task RunAsync(QueryScenario scenario, PostgresContainer pg) if (spec.Op == "relate") { metaRoot ??= LoadCorpusMetadata(); - var actual = await ResolveRelateAsync(pg.ConnectionString, metaRoot, spec); + if (engine != ScenarioEngine.Postgres) + throw new InvalidOperationException($"{scenario.SourcePath}: op:relate runs on Postgres only"); + var actual = await ResolveRelateAsync(connectionString, metaRoot, spec); AssertResult(scenario.SourcePath, spec, actual); continue; } @@ -126,10 +157,15 @@ private static MetaRoot LoadCorpusMetadata() conn, root, spec.Entity, sourceId, spec.Relation!); } - /// Read the committed canonical Postgres schema artifact (TS-produced). - private static string ReadCanonicalSchemaSql() + /// Read the committed canonical schema artifact of (TS-produced). + private static string ReadCanonicalSchemaSql(ScenarioEngine engine) { - var path = CorpusPaths.CanonicalSchemaSql; + var path = engine switch + { + ScenarioEngine.Sqlite => CorpusPaths.CanonicalSchemaSqliteSql, + ScenarioEngine.MySql => CorpusPaths.CanonicalSchemaMysqlSql, + _ => CorpusPaths.CanonicalSchemaSql, + }; if (!File.Exists(path)) throw new InvalidOperationException( $"canonical schema artifact not found at {path}; it is produced by the " + @@ -137,14 +173,6 @@ private static string ReadCanonicalSchemaSql() return File.ReadAllText(path); } - private static async Task ExecuteAsync(string connString, string sql) - { - await using var conn = new NpgsqlConnection(connString); - await conn.OpenAsync(); - await using var cmd = new NpgsqlCommand(sql, conn); - await cmd.ExecuteNonQueryAsync(); - } - // ----------------------------------------------------------------------- // Result assertion (every comparison is JSON-canonical byte equality) // ----------------------------------------------------------------------- diff --git a/server/csharp/MetaObjects.IntegrationTests/Runner/ReportViewsDbContext.cs b/server/csharp/MetaObjects.IntegrationTests/Runner/ReportViewsDbContext.cs new file mode 100644 index 000000000..229cda065 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Runner/ReportViewsDbContext.cs @@ -0,0 +1,48 @@ +// ReportViewsDbContext — the generated AppDbContext, narrowed to its view-mapped entities +// so it builds on SQLite and MySQL. +// +// The generated context is Postgres-flavoured for the TABLE entities (jsonb JsonDocument, +// inet IPAddress, uuid conversions, `timestamp with time zone`), and EF's SQLite and MySQL +// providers refuse to build a model that contains a property they cannot map. The FR-044 report +// scenarios read only view-mapped entities (`ToView(...)`: reports and projections), whose +// columns are plain scalars, so this subclass ignores every entity that is not a view and +// leaves the generated mapping of the views untouched. Nothing here maps or renames a +// column: the views are the TypeScript-produced ones. +// +// One mapping is added: EF's SQLite provider refuses `ORDER BY` on a DateTimeOffset +// (it stores one as text with an offset, which does not sort as an instant), and a MySQL +// DATETIME holds no zone at all. A report whose time dimension is an instant +// (`RecordedAtHour`) is ordered by the generated query, so a DateTimeOffset reads and +// writes through its UTC DateTime. Both engines hold the instant as the UTC wall clock, +// so the value is unchanged. + +using Microsoft.EntityFrameworkCore; +using Microsoft.EntityFrameworkCore.Storage.ValueConversion; +using MetaObjects.IntegrationTests.Generated; + +namespace MetaObjects.IntegrationTests.Runner; + +public sealed class ReportViewsDbContext : AppDbContext +{ + public ReportViewsDbContext(DbContextOptions options) : base(options) { } + + protected override void OnModelCreating(ModelBuilder modelBuilder) + { + base.OnModelCreating(modelBuilder); + var tables = modelBuilder.Model.GetEntityTypes() + .Where(e => e.GetViewName() is null) + .Select(e => e.ClrType) + .ToList(); + foreach (var table in tables) modelBuilder.Ignore(table); + + var instant = new ValueConverter( + v => v.UtcDateTime, + // Microsoft.Data.Sqlite parses the `...Z` text to a LOCAL DateTime; a MySQL + // DATETIME arrives Unspecified. An Unspecified value carries no zone and is the + // UTC wall clock the views hold. + v => new DateTimeOffset(v.Kind == DateTimeKind.Unspecified ? DateTime.SpecifyKind(v, DateTimeKind.Utc) : v.ToUniversalTime())); + foreach (var property in modelBuilder.Model.GetEntityTypes().SelectMany(e => e.GetProperties()) + .Where(p => p.ClrType == typeof(DateTimeOffset))) + property.SetValueConverter(instant); + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Runner/Scenario.cs b/server/csharp/MetaObjects.IntegrationTests/Runner/Scenario.cs index 2c6df836e..7456c0aec 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Runner/Scenario.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Runner/Scenario.cs @@ -17,8 +17,17 @@ public sealed record QueryScenario( string Description, string SourcePath, string? SeedData, - IReadOnlyList Queries) - : Scenario(Name, Description, SourcePath); + IReadOnlyList Queries, + IReadOnlyDictionary? SeedDataEngine = null) // per-engine seed SQL (`seed-data-engine:`), keyed by engine name + : Scenario(Name, Description, SourcePath) +{ + /// The seed SQL for : its seed-data-engine entry, else the Postgres seed-data. + public string? SeedFor(string engine) => + SeedDataEngine is not null && SeedDataEngine.TryGetValue(engine, out var own) ? own : SeedData; + + /// True when seed-data-engine spells the seed for itself. + public bool HasSeedFor(string engine) => SeedDataEngine?.ContainsKey(engine) == true; +} /// A single query intent: op + entity + filter/sort/by + expected result. public sealed record QuerySpec( diff --git a/server/csharp/MetaObjects.IntegrationTests/Runner/ScenarioEngine.cs b/server/csharp/MetaObjects.IntegrationTests/Runner/ScenarioEngine.cs new file mode 100644 index 000000000..dddc121e8 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Runner/ScenarioEngine.cs @@ -0,0 +1,114 @@ +// ScenarioEngine — the database engine a query scenario runs against. Postgres is the +// reference; SQLite and MySQL are the other engines the FR-044 report scenarios run on in C#. + +using System.Data.Common; +using System.Text; +using System.Text.RegularExpressions; +using Microsoft.Data.Sqlite; +using Microsoft.EntityFrameworkCore; +using MySqlConnector; +using Npgsql; + +namespace MetaObjects.IntegrationTests.Runner; + +public enum ScenarioEngine +{ + Postgres, + Sqlite, + MySql, +} + +/// +/// The engine-keyed facts of the corpus lanes live HERE, not in per-caller switches: one +/// provider choice, one connection factory, one script executor, one identifier quote. A +/// fourth engine means one more arm in each of these switches (and the Postgres-default arm +/// is a loud TODO there), not a hunt through the fixture, runner and server factory. +/// +public static class ScenarioEngineExtensions +{ + /// The engine's name as the corpus spells it (the key of a seed-data-engine entry). + public static string Name(this ScenarioEngine engine) => engine switch + { + ScenarioEngine.Postgres => "postgres", + ScenarioEngine.Sqlite => "sqlite", + ScenarioEngine.MySql => "mysql", + _ => throw new ArgumentOutOfRangeException(nameof(engine), engine, null), + }; + + /// The identifier quote seeds and inserts use: MySQL backticks, everything else the corpus' double quotes. + public static string Quote(this ScenarioEngine engine) => + engine == ScenarioEngine.MySql ? "`" : "\""; + + /// The server version Pomelo generates against. Keep in step with the mysql:8.4 image MySqlServer starts. + public static readonly Version MySqlProviderVersion = new(8, 4, 0); + + /// Point this engine's EF Core provider at (mutates and returns the builder). + public static DbContextOptionsBuilder UseEngine(this DbContextOptionsBuilder builder, ScenarioEngine engine, string connString) => + engine switch + { + ScenarioEngine.Sqlite => builder.UseSqlite(connString), + ScenarioEngine.MySql => builder.UseMySql(connString, new MySqlServerVersion(MySqlProviderVersion)), + _ => builder.UseNpgsql(connString), + }; + + /// Open this engine's ADO.NET connection. Callers treat it as the provider-neutral . + public static async Task OpenConnectionAsync(this ScenarioEngine engine, string connString) + { + DbConnection conn = engine switch + { + ScenarioEngine.Sqlite => new SqliteConnection(connString), + ScenarioEngine.MySql => new MySqlConnection(connString), + _ => new NpgsqlConnection(connString), + }; + await conn.OpenAsync(); + return conn; + } + + /// Execute one whole SQL script (schema artifact or seed) on a fresh connection of this engine. + public static async Task ExecuteScriptAsync(this ScenarioEngine engine, string connString, string sql) + { + await using var conn = await engine.OpenConnectionAsync(connString); + await using var cmd = conn.CreateCommand(); + cmd.CommandText = sql; + await cmd.ExecuteNonQueryAsync(); + } +} + +public static partial class MySqlSeed +{ + [GeneratedRegex(@"'(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?)Z'")] + private static partial Regex UtcInstant(); + + /// + /// Postgres-quoted seed SQL to MySQL. Walks the text once so a double quote INSIDE a + /// single-quoted string (a JSON payload) is left alone and only a double-quoted identifier + /// becomes a backtick one; a UTC instant literal loses its Z (a DATETIME holds the UTC + /// wall clock). ANSI_QUOTES stays off: it would change how the view bodies parse. + /// + public static string FromPostgres(string seed) + { + var sb = new StringBuilder(seed.Length); + bool inString = false, inIdent = false; + for (var i = 0; i < seed.Length; i++) + { + var c = seed[i]; + if (inString) + { + sb.Append(c); + if (c == '\'') + { + if (i + 1 < seed.Length && seed[i + 1] == '\'') { sb.Append('\''); i++; } + else inString = false; + } + } + else if (inIdent) + { + if (c == '"') { sb.Append('`'); inIdent = false; } else sb.Append(c); + } + else if (c == '\'') { inString = true; sb.Append(c); } + else if (c == '"') { inIdent = true; sb.Append('`'); } + else sb.Append(c); + } + return UtcInstant().Replace(sb.ToString(), "'$1'"); + } +} diff --git a/server/csharp/MetaObjects.IntegrationTests/Runner/ScenarioLoader.cs b/server/csharp/MetaObjects.IntegrationTests/Runner/ScenarioLoader.cs index fe7892c18..bcf555689 100644 --- a/server/csharp/MetaObjects.IntegrationTests/Runner/ScenarioLoader.cs +++ b/server/csharp/MetaObjects.IntegrationTests/Runner/ScenarioLoader.cs @@ -84,6 +84,7 @@ public static QueryScenario LoadQuery(string yamlPath) Description: raw.Description ?? "", SourcePath: yamlPath, SeedData: raw.SeedData, + SeedDataEngine: raw.SeedDataEngine, Queries: (raw.Queries ?? []).Select(q => new QuerySpec( Name: q.Name ?? throw new InvalidOperationException($"{yamlPath}: query missing 'name'"), Op: q.Op ?? throw new InvalidOperationException($"{yamlPath}: query missing 'op'"), @@ -107,6 +108,9 @@ private sealed class QueryYaml public string? Name { get; set; } public string? Description { get; set; } public string? SeedData { get; set; } + // Per-engine seed SQL, keyed by engine name. Only a seed that cannot be one SQL + // text across engines (a clock-relative expression) carries one. + public Dictionary? SeedDataEngine { get; set; } public List? Queries { get; set; } } diff --git a/server/csharp/MetaObjects.IntegrationTests/Runner/SqliteTempDatabase.cs b/server/csharp/MetaObjects.IntegrationTests/Runner/SqliteTempDatabase.cs new file mode 100644 index 000000000..e4fbf6aa0 --- /dev/null +++ b/server/csharp/MetaObjects.IntegrationTests/Runner/SqliteTempDatabase.cs @@ -0,0 +1,25 @@ +// SqliteTempDatabase — a throwaway SQLite file for one scenario, owned by the scenario. +// +// The teardown order IS the contract: Microsoft.Data.Sqlite pools connections, so the pools +// must be cleared BEFORE the file is deleted — otherwise a pooled connection holds the file +// and the next "fresh" scenario reads the previous scenario's rows. That step is easy to +// forget in a copy-paste, so it lives once. + +using Microsoft.Data.Sqlite; + +namespace MetaObjects.IntegrationTests.Runner; + +internal sealed class SqliteTempDatabase : IDisposable +{ + /// Full path of the (not yet created) database file; the driver creates it on first use. + public string FilePath { get; } + + public SqliteTempDatabase(string prefix) => + FilePath = System.IO.Path.Combine(System.IO.Path.GetTempPath(), $"{prefix}{Guid.NewGuid():N}.db"); + + public void Dispose() + { + SqliteConnection.ClearAllPools(); + if (System.IO.File.Exists(FilePath)) System.IO.File.Delete(FilePath); + } +} diff --git a/server/java/integration-tests-kotlin/pom.xml b/server/java/integration-tests-kotlin/pom.xml index cdba6182f..f24e794e9 100644 --- a/server/java/integration-tests-kotlin/pom.xml +++ b/server/java/integration-tests-kotlin/pom.xml @@ -188,6 +188,24 @@ test + + + com.mysql + mysql-connector-j + 9.1.0 + test + + + + + org.xerial + sqlite-jdbc + 3.46.1.3 + test + + diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/MySqlContainer.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/MySqlContainer.kt new file mode 100644 index 000000000..c517e124f --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/MySqlContainer.kt @@ -0,0 +1,143 @@ +package com.metaobjects.integration.kotlin + +import java.io.BufferedReader +import java.io.InputStreamReader +import java.net.ServerSocket +import java.nio.charset.StandardCharsets +import java.sql.DriverManager +import java.util.UUID + +/** + * A throwaway MySQL database for the report scenarios, started through the `docker` CLI. + * + * Same approach as [PostgresContainer] and the Java port's `MySqlContainer.java`: testcontainers-java's + * bundled docker-java cannot negotiate with current Docker daemons. Only the per-container mode + * exists; there is no shared CI sidecar for MySQL. Set `METAOBJECTS_TEST_MYSQL_URL` + * (`jdbc:mysql://host:port/db`, with `METAOBJECTS_TEST_MYSQL_USER` / `_PASSWORD`) to run against an + * existing server instead: the test then owns that database. + */ +class MySqlContainer : AutoCloseable { + private val name: String? // null when an existing server is used + val jdbcUrl: String + val username: String + val password: String + + init { + val existing = System.getenv(URL_ENV) + if (!existing.isNullOrBlank()) { + name = null + jdbcUrl = existing + username = System.getenv("METAOBJECTS_TEST_MYSQL_USER") ?: "root" + password = System.getenv("METAOBJECTS_TEST_MYSQL_PASSWORD") ?: "" + } else { + name = "metaobjects-mysql-kt-" + UUID.randomUUID().toString().substring(0, 8) + username = "root" + password = PASSWORD + var started: String? = null + var last: RuntimeException? = null + for (attempt in 1..START_ATTEMPTS) { + if (started != null) break + val port = ServerSocket(0).use { it.localPort } + try { + runDocker( + "run", "-d", "--name", name, + "-e", "MYSQL_ROOT_PASSWORD=$PASSWORD", + "-e", "MYSQL_DATABASE=$DB", + "-p", "$port:3306", + IMAGE, + ) + } catch (e: RuntimeException) { + last = e + forceRemove() + continue + } + val candidate = "jdbc:mysql://localhost:$port/$DB" + try { + waitForReady(candidate) + started = candidate + } catch (e: RuntimeException) { + last = e + forceRemove() + } + } + jdbcUrl = started ?: throw RuntimeException( + "mysql container '$name' failed to start in $START_ATTEMPTS attempts", last, + ) + } + } + + override fun close() { + if (name != null) forceRemove() + } + + // MySQL's entrypoint runs a temporary server to initialise the data directory, stops it, then + // starts the real one. Readiness needs both: the final server has logged its start, and a JDBC + // connection to the created database succeeds. + private fun waitForReady(url: String) { + val deadline = System.currentTimeMillis() + READY_TIMEOUT_S * 1000L + while (System.currentTimeMillis() < deadline) { + val state = inspectState() + if (state != "running") { + throw RuntimeException("mysql container '$name' is '$state', not running. docker logs:\n" + tailLogs()) + } + if (canConnect(url) && initComplete()) return + try { + Thread.sleep(500) + } catch (_: InterruptedException) { + Thread.currentThread().interrupt() + return + } + } + throw RuntimeException("mysql container '$name' did not become ready within ${READY_TIMEOUT_S}s. docker logs:\n" + tailLogs()) + } + + // The temporary server listens on "port: 0"; only the final server logs port 3306. + private fun initComplete(): Boolean { + val logs = runDockerQuiet("logs", name!!) + return logs.contains("ready for connections") && logs.contains("port: 3306") + } + + private fun canConnect(url: String): Boolean = try { + DriverManager.getConnection(url, username, password).use { it.isValid(2) } + } catch (_: Exception) { + false + } + + private fun forceRemove() { + try { runDocker("rm", "-f", name!!) } catch (_: RuntimeException) { /* already gone */ } + } + + private fun inspectState(): String = try { + runDocker("inspect", "-f", "{{.State.Status}}", name!!) + } catch (_: RuntimeException) { + "gone" + } + + private fun tailLogs(): String = runDockerQuiet("logs", "--tail", "40", name!!) + + companion object { + private const val URL_ENV = "METAOBJECTS_TEST_MYSQL_URL" + private const val IMAGE = "mysql:8.4" + private const val DB = "mo_test" + private const val PASSWORD = "test" + private const val START_ATTEMPTS = 3 + private val READY_TIMEOUT_S: Int = (System.getenv("MO_MYSQL_READY_TIMEOUT_S") ?: "180").toInt() + + private fun runDockerQuiet(vararg args: String): String = try { + runDocker(*args) + } catch (e: RuntimeException) { + "(docker ${args[0]} unavailable: ${e.message})" + } + + private fun runDocker(vararg args: String): String { + val p = ProcessBuilder("docker", *args).redirectErrorStream(true).start() + val out = StringBuilder() + BufferedReader(InputStreamReader(p.inputStream, StandardCharsets.UTF_8)).use { r -> + r.forEachLine { out.append(it).append('\n') } + } + val exit = p.waitFor() + if (exit != 0) throw RuntimeException("docker ${args.joinToString(" ")} failed (exit=$exit): $out") + return out.toString().trim() + } + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioMySqlTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioMySqlTest.kt new file mode 100644 index 000000000..332dae68d --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioMySqlTest.kt @@ -0,0 +1,55 @@ +package com.metaobjects.integration.kotlin + +import org.junit.jupiter.api.AfterAll +import org.junit.jupiter.api.Assumptions +import org.junit.jupiter.api.BeforeAll +import org.junit.jupiter.api.TestInstance +import org.junit.jupiter.api.assertDoesNotThrow +import org.junit.jupiter.params.ParameterizedTest +import org.junit.jupiter.params.provider.Arguments +import org.junit.jupiter.params.provider.MethodSource +import java.util.stream.Stream + +/** + * The persistence corpus' report scenarios (`report-*.yaml`) on MySQL, through Exposed. The schema + * is the TS-produced `canonical/schema.mysql.sql` (hand tables plus the `buildReportViews` mysql + * views), executed verbatim; the same Postgres `expect` blocks are asserted, with the seed re-spelled + * by [ScenarioEngine.toMysqlSeed]. + * + * One MySQL server serves every scenario; each starts from an empty schema. Not registered in the + * parent reactor (it needs docker) -- run via `mvn -f server/java/integration-tests-kotlin/pom.xml test`. + */ +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +internal class QueryScenarioMySqlTest { + + private var mysql: MySqlContainer? = null + + @BeforeAll + fun startServer() { + mysql = MySqlContainer() + } + + @AfterAll + fun stopServer() { + mysql?.close() + } + + @ParameterizedTest(name = "{0}") + @MethodSource("scenarios") + fun reportScenario(name: String, scenario: Scenarios.QueryScenario) { + val server = mysql!! + assertDoesNotThrow { + QueryScenarioRunner.run(scenario, server.jdbcUrl, server.username, server.password, ScenarioEngine.MYSQL) + } + } + + companion object { + @JvmStatic + fun scenarios(): Stream { + val corpus = ScenarioLoader.findCorpusRoot() + val reports = ScenarioLoader.loadQueries(corpus.resolve("queries")).filter { it.name.startsWith("report-") } + Assumptions.assumeTrue(reports.isNotEmpty(), "No report-* scenarios under fixtures/persistence-conformance/queries") + return reports.stream().map { Arguments.of(it.name, it) } + } + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt index 3731906a4..a262ead8b 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioRunner.kt @@ -83,19 +83,32 @@ object QueryScenarioRunner { * Run a single query scenario end-to-end against a fresh container (one * container per scenario at the JUnit level). */ - fun run(scenario: QueryScenario, pg: PostgresContainer) { - val db = Database.connect(pg.jdbcUrl, user = pg.username, password = pg.password) + fun run(scenario: QueryScenario, pg: PostgresContainer) = + run(scenario, pg.jdbcUrl, pg.username, pg.password, ScenarioEngine.POSTGRES) + + /** + * Run a scenario against an already-provisioned database of [engine]. The schema is that + * engine's TS-produced artifact, executed verbatim; the seed is the scenario's seed SQL + * spelled for the engine. The Postgres `expect` blocks are asserted unchanged. + */ + fun run(scenario: QueryScenario, baseUrl: String, username: String, password: String, engine: ScenarioEngine) { + // The engine appends its own session parameters (timezone / date-format policy) — see + // ScenarioEngine.jdbcUrl. + val jdbcUrl = engine.jdbcUrl(baseUrl) + val db = Database.connect(jdbcUrl, user = username, password = password) val corpus = ScenarioLoader.findCorpusRoot() // 1. Provision the schema from the committed canonical DDL (base tables + - // projection views). Executed verbatim on a direct JDBC connection — - // schema authority is the TS-produced artifact, not Exposed. - val schemaDdl = ScenarioLoader.readCanonicalSchema(corpus) - execSql(pg, schemaDdl) + // projection views). Executed verbatim on a direct JDBC connection -- + // schema authority is the TS-produced artifact, not Exposed. A shared server + // (MySQL) starts each scenario from an empty schema. + val schemaDdl = ScenarioLoader.readCanonicalSchema(corpus, engine.schemaArtifact) + if (engine == ScenarioEngine.MYSQL) dropMysqlSchema(jdbcUrl, username, password, schemaDdl) + execSql(jdbcUrl, username, password, schemaDdl, engine) // 2. Seed via the YAML's raw SQL. - scenario.seedData?.takeIf { it.isNotBlank() }?.let { sql -> execSql(pg, sql) } + engine.seed(scenario)?.takeIf { it.isNotBlank() }?.let { sql -> execSql(jdbcUrl, username, password, sql, engine) } // 3. Load the canonical metadata root once per scenario — op:relate derives // the M:N junction FK fields + physical table/column names from it (the @@ -126,9 +139,32 @@ object QueryScenarioRunner { * through Exposed (they may use double-quoted identifiers Exposed wouldn't * synthesize). */ - private fun execSql(pg: PostgresContainer, sql: String) { - DriverManager.getConnection(pg.jdbcUrl, pg.username, pg.password).use { c -> - c.createStatement().use { it.execute(sql) } + private fun execSql(jdbcUrl: String, username: String, password: String, sql: String, engine: ScenarioEngine) { + DriverManager.getConnection(jdbcUrl, username, password).use { c -> + if (engine != ScenarioEngine.POSTGRES) { + // The MySQL and SQLite drivers run one statement per call. + c.createStatement().use { st -> ScenarioEngine.splitStatements(sql).forEach { st.execute(it) } } + } else { + c.createStatement().use { it.execute(sql) } + } + } + } + + private val MYSQL_CREATE = Regex("^CREATE (VIEW|TABLE) `([^`]+)`") + + /** + * Drop every view and table [schemaDdl] creates (views first, tables in reverse order), by + * name: `METAOBJECTS_TEST_MYSQL_URL` may point at a shared database. + */ + private fun dropMysqlSchema(jdbcUrl: String, username: String, password: String, schemaDdl: String) { + val created = ScenarioEngine.splitStatements(schemaDdl).mapNotNull { MYSQL_CREATE.find(it)?.groupValues } + val views = created.filter { it[1] == "VIEW" }.map { it[2] } + val tables = created.filter { it[1] == "TABLE" }.map { it[2] }.reversed() + DriverManager.getConnection(jdbcUrl, username, password).use { c -> + c.createStatement().use { st -> + views.forEach { st.execute("DROP VIEW IF EXISTS `$it`") } + tables.forEach { st.execute("DROP TABLE IF EXISTS `$it`") } + } } } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioSqliteTest.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioSqliteTest.kt new file mode 100644 index 000000000..a673c020f --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/QueryScenarioSqliteTest.kt @@ -0,0 +1,37 @@ +package com.metaobjects.integration.kotlin + +import org.junit.jupiter.api.Assumptions +import org.junit.jupiter.api.assertDoesNotThrow +import org.junit.jupiter.api.io.TempDir +import org.junit.jupiter.params.ParameterizedTest +import org.junit.jupiter.params.provider.Arguments +import org.junit.jupiter.params.provider.MethodSource +import java.nio.file.Path +import java.util.stream.Stream + +/** + * The persistence corpus' report scenarios (`report-*.yaml`) on SQLite, through Exposed. The schema + * is the TS-produced `canonical/schema.sqlite.sql` (the `meta migrate` tables plus report views), + * executed verbatim; the same Postgres `expect` blocks are asserted. SQLite is embedded, so each + * scenario gets a fresh database file and no container. + */ +internal class QueryScenarioSqliteTest { + + @ParameterizedTest(name = "{0}") + @MethodSource("scenarios") + fun reportScenario(name: String, scenario: Scenarios.QueryScenario, @TempDir dir: Path) { + assertDoesNotThrow { + QueryScenarioRunner.run(scenario, "jdbc:sqlite:${dir.resolve("scenario.db")}", "", "", ScenarioEngine.SQLITE) + } + } + + companion object { + @JvmStatic + fun scenarios(): Stream { + val corpus = ScenarioLoader.findCorpusRoot() + val reports = ScenarioLoader.loadQueries(corpus.resolve("queries")).filter { it.name.startsWith("report-") } + Assumptions.assumeTrue(reports.isNotEmpty(), "No report-* scenarios under fixtures/persistence-conformance/queries") + return reports.stream().map { Arguments.of(it.name, it) } + } + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/ScenarioEngine.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/ScenarioEngine.kt new file mode 100644 index 000000000..7538c6735 --- /dev/null +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/ScenarioEngine.kt @@ -0,0 +1,97 @@ +package com.metaobjects.integration.kotlin + +import com.metaobjects.integration.kotlin.Scenarios.QueryScenario + +/** + * The SQL engine a persistence scenario runs on. Postgres is the corpus' own engine: the scenario + * YAML holds its seed SQL and its wire-form `expect`. MySQL and SQLite run the same scenarios against + * their own TS-produced schema artifact (`canonical/schema..sql`), with the seed re-spelled for + * the engine where its spelling differs and the engine's spelling of a value mapped onto the corpus' + * wire form. Spelling only: a wrong VALUE still + * fails the comparison, and no `expect` is loosened. + */ +enum class ScenarioEngine(private val id: String) { + POSTGRES("postgres") { + override fun seed(scenario: QueryScenario): String? = scenario.seedData + }, + MYSQL("mysql") { + override fun seed(scenario: QueryScenario): String? = + scenario.seedDataEngine?.get("mysql") ?: scenario.seedData?.let { toMysqlSeed(it) } + + // A MySQL DATETIME has no zone and the corpus stores UTC wall clocks: read and write them + // as UTC whatever the JVM zone is, so a report's instant column reads back as the same instant. + override fun sessionParams() = "connectionTimeZone=UTC&forceConnectionTimeZoneToSession=true" + }, + SQLITE("sqlite") { + // SQLite takes the Postgres seed as written (double-quoted identifiers, ISO instants as text). + override fun seed(scenario: QueryScenario): String? = + scenario.seedDataEngine?.get("sqlite") ?: scenario.seedData + + // SQLite keeps an instant as the TEXT the view built (`...T03:00:00.000Z`); the driver + // parses a timestamp with this format, which reads that spelling back as the UTC instant. + override fun sessionParams() = "date_string_format=yyyy-MM-dd'T'HH:mm:ss.SSSX" + }; + + /** The schema artifact, relative to the corpus root. */ + val schemaArtifact: String get() = "canonical/schema.$id.sql" + + /** The scenario's seed SQL spelled for this engine, or null. */ + abstract fun seed(scenario: QueryScenario): String? + + /** Session parameters this engine MUST get on its JDBC URL (empty = none). */ + protected open fun sessionParams(): String = "" + + /** + * [base] plus this engine's session params, joined with `?`/`&` as base requires. On the + * enum, not an if-chain in the runner: a new engine that needs a session policy states it + * beside its seed, and one that needs none says so — it cannot silently get neither. + */ + fun jdbcUrl(base: String): String = + sessionParams().let { if (it.isEmpty()) base else base + (if ('?' in base) "&" else "?") + it } + + companion object { + private val UTC_INSTANT = Regex("'(\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(?:\\.\\d+)?)Z'") + + /** + * Postgres-quoted seed SQL to MySQL. Walks the text once so a double quote INSIDE a + * single-quoted string (a JSON payload) is left alone and only a double-quoted identifier is + * rewritten; a UTC instant literal loses its `Z` (a DATETIME holds the UTC wall clock). + */ + internal fun toMysqlSeed(seed: String): String { + val out = StringBuilder(seed.length) + var inString = false + var inIdent = false + var i = 0 + while (i < seed.length) { + val c = seed[i] + if (inString) { + out.append(c) + if (c == '\'') { + if (i + 1 < seed.length && seed[i + 1] == '\'') { out.append('\''); i++ } else inString = false + } + } else if (inIdent) { + if (c == '"') { out.append('`'); inIdent = false } else out.append(c) + } else if (c == '\'') { + inString = true + out.append(c) + } else if (c == '"') { + inIdent = true + out.append('`') + } else { + out.append(c) + } + i++ + } + return UTC_INSTANT.replace(out.toString(), "'$1'") + } + + /** The statements of a script: split on a `;` at the end of a line (no body carries one mid-line). */ + internal fun splitStatements(script: String): List { + val sb = StringBuilder() + for (line in script.split("\n")) { + if (!line.trim().startsWith("--")) sb.append(line).append('\n') + } + return sb.toString().split(Regex(";[ \\t]*(?:\\n|$)")).filter { it.isNotBlank() }.map { it.trim() } + } + } +} diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/ScenarioLoader.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/ScenarioLoader.kt index bbac99201..d85968f31 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/ScenarioLoader.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/ScenarioLoader.kt @@ -46,8 +46,8 @@ object ScenarioLoader { } /** Read the committed canonical Postgres schema DDL (executed verbatim by the query runner). */ - fun readCanonicalSchema(corpusRoot: Path): String { - val schema = corpusRoot.resolve(CANONICAL_SCHEMA_RELATIVE) + fun readCanonicalSchema(corpusRoot: Path, relative: String = CANONICAL_SCHEMA_RELATIVE): String { + val schema = corpusRoot.resolve(relative) require(Files.isRegularFile(schema)) { "Canonical schema not found: $schema" } return Files.readString(schema, StandardCharsets.UTF_8) } @@ -77,6 +77,8 @@ object ScenarioLoader { description = (root["description"] as? String) ?: "", sourcePath = file.toString(), seedData = root["seed-data"] as? String, + seedDataEngine = (root["seed-data-engine"] as? Map<*, *>) + ?.entries?.associate { it.key.toString() to it.value.toString() }, queries = queries, ) } diff --git a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/Scenarios.kt b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/Scenarios.kt index 33d79bd4b..3cdc2c0d2 100644 --- a/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/Scenarios.kt +++ b/server/java/integration-tests-kotlin/src/test/kotlin/com/metaobjects/integration/kotlin/Scenarios.kt @@ -30,6 +30,7 @@ object Scenarios { val description: String, val sourcePath: String, val seedData: String? = null, + val seedDataEngine: Map? = null, // `seed-data-engine`: seed SQL for an engine whose spelling differs val queries: List, ) } diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/JdbcTarget.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/JdbcTarget.java new file mode 100644 index 000000000..b4555c569 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/JdbcTarget.java @@ -0,0 +1,8 @@ +package com.metaobjects.integration; + +/** A database the query scenarios run against: what {@link PostgresContainer} and {@link MySqlContainer} both expose. */ +public interface JdbcTarget { + String jdbcUrl(); + String username(); + String password(); +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/MySqlContainer.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/MySqlContainer.java index 2c365ccf9..0c030496e 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/MySqlContainer.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/MySqlContainer.java @@ -21,7 +21,7 @@ * {@code METAOBJECTS_TEST_MYSQL_USER} / {@code _PASSWORD}) to run against an existing server * instead. The test then owns that database: it creates and drops its own table. */ -public final class MySqlContainer implements AutoCloseable { +public final class MySqlContainer implements AutoCloseable, JdbcTarget { private static final String URL_ENV = "METAOBJECTS_TEST_MYSQL_URL"; private static final String IMAGE = "mysql:8.4"; private static final String DB = "mo_test"; @@ -78,9 +78,9 @@ public MySqlContainer() { this.jdbcUrl = started; } - public String jdbcUrl() { return jdbcUrl; } - public String username() { return username; } - public String password() { return password; } + @Override public String jdbcUrl() { return jdbcUrl; } + @Override public String username() { return username; } + @Override public String password() { return password; } public Connection open() throws SQLException { return DriverManager.getConnection(jdbcUrl, username, password); diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/PostgresContainer.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/PostgresContainer.java index 6cc14665f..113dd3906 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/PostgresContainer.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/PostgresContainer.java @@ -38,7 +38,7 @@ * Mirrors the TS port's {@code postgres-container.ts}. The shared sidecar mode * sidesteps this entirely; the fallback keeps the docker path. */ -public final class PostgresContainer implements AutoCloseable { +public final class PostgresContainer implements AutoCloseable, JdbcTarget { /** Env var naming the shared CI Postgres sidecar (admin URL). Unset = per-container fallback. */ private static final String SHARED_PG_URL_ENV = "METAOBJECTS_TEST_PG_URL"; @@ -122,9 +122,9 @@ public PostgresContainer() { this.jdbcUrl = startedUrl; } - public String jdbcUrl() { return jdbcUrl; } - public String username() { return username; } - public String password() { return password; } + @Override public String jdbcUrl() { return jdbcUrl; } + @Override public String username() { return username; } + @Override public String password() { return password; } @Override public void close() { if (shared) { diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioMySqlTests.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioMySqlTests.java new file mode 100644 index 000000000..550d1e657 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioMySqlTests.java @@ -0,0 +1,67 @@ +package com.metaobjects.integration; + +import com.metaobjects.integration.Scenarios.QueryScenario; +import com.metaobjects.object.value.ValueObject; +import com.metaobjects.registry.ObjectClassRegistry; +import org.junit.jupiter.api.AfterAll; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.params.ParameterizedTest; +import org.junit.jupiter.params.provider.Arguments; +import org.junit.jupiter.params.provider.MethodSource; + +import java.nio.file.Path; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Stream; + +import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; + +/** + * The persistence corpus' report scenarios ({@code report-*.yaml}) on MySQL, through the OMDB + * {@code MySQLDriver}. The schema is the TS-produced {@code canonical/schema.mysql.sql} (hand + * tables plus the {@code buildReportViews} mysql views), executed verbatim; the same Postgres + * {@code expect} blocks are asserted, with MySQL's value spelling mapped onto the wire form by + * {@link QueryScenarioRunner} (a DATETIME is a UTC wall clock) and the seed re-spelled by + * {@link ScenarioEngine#toMysqlSeed}. + * + *

One MySQL server serves every scenario; each starts from an empty schema. Not part of the + * default Maven reactor build (it needs docker).

+ */ +final class QueryScenarioMySqlTests { + + private static final Path CORPUS = ScenarioLoader.findCorpusRoot(); + private static final List SCENARIOS = + ScenarioLoader.loadQueries(CORPUS.resolve("queries")).stream() + .filter(s -> s.name().startsWith("report-")).toList(); + + private static MySqlContainer mysql; + + @BeforeAll + static void beforeAll() { + ObjectClassRegistry reg = new ObjectClassRegistry(); + Map> bindings = new HashMap<>(); + // The entities the report views read; a report itself is served by its read model. + for (String entity : List.of("Program", "Week", "Asset")) { + bindings.put("fitness::" + entity, ValueObject.class); + } + reg.register(() -> bindings); + ObjectClassRegistry.setGlobal(reg); + mysql = new MySqlContainer(); + } + + @AfterAll + static void afterAll() { + if (mysql != null) mysql.close(); + } + + static Stream scenarios() { + return SCENARIOS.stream().map(s -> Arguments.of(s.name(), s)); + } + + @ParameterizedTest(name = "{0}") + @MethodSource("scenarios") + void reportScenario(String name, QueryScenario scenario) { + assertDoesNotThrow(() -> QueryScenarioRunner.run(scenario, mysql, CORPUS.resolve("canonical"), ScenarioEngine.MYSQL)); + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioRunner.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioRunner.java index 2078e59d5..486b03935 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioRunner.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/QueryScenarioRunner.java @@ -5,7 +5,6 @@ import com.metaobjects.loader.MetaDataLoader; import com.metaobjects.manager.ObjectConnection; import com.metaobjects.manager.db.ObjectManagerDB; -import com.metaobjects.manager.db.driver.PostgresDriver; import com.metaobjects.object.MetaObject; import com.metaobjects.reporting.ReportReadModel; @@ -45,22 +44,40 @@ public final class QueryScenarioRunner { private QueryScenarioRunner() {} public static void run(QueryScenario scenario, PostgresContainer pg, Path canonicalDir) throws Exception { + run(scenario, pg, canonicalDir, ScenarioEngine.POSTGRES); + } + + public static void run(QueryScenario scenario, JdbcTarget pg, Path canonicalDir, ScenarioEngine engine) throws Exception { // Pick a canonical-loader name unique per scenario so registry state // from a previous scenario doesn't leak across containers. String tag = "canonical-" + UUID.randomUUID().toString().substring(0, 8); MetaDataLoader loader = MetaDataLoader.fromDirectory(tag, canonicalDir); - ObjectManagerDB omdb = newOmdb(pg); + ObjectManagerDB omdb = newOmdb(pg, engine); // 1. Provision the schema by executing the committed canonical DDL // verbatim — the TS-produced artifact is the sole schema source // (ADR-0015). OMDB's runtime auto-create is intentionally NOT engaged. - String schemaDdl = ScenarioLoader.readCanonicalSchema(canonicalDir.getParent()); - try (Connection c = openConnection(pg)) { executeSql(c, schemaDdl); } + String schemaDdl = ScenarioLoader.readCanonicalSchema(canonicalDir.getParent(), engine.schemaArtifact()); + try (Connection c = openConnection(pg)) { + // A shared MySQL server serves several scenarios: start each from an empty schema. + // Split once — the same statement list drives the drop and the create (Postgres + // takes the whole script in one call and is never split at all). + if (engine == ScenarioEngine.MYSQL) { + List statements = ScenarioEngine.splitStatements(schemaDdl); + dropSchema(c, statements); + try (Statement s = c.createStatement()) { + for (String stmt : statements) s.execute(stmt); + } + } else { + executeSql(c, schemaDdl, engine); + } + } // 2. Seed data. - if (scenario.seedData() != null && !scenario.seedData().isBlank()) { - try (Connection c = openConnection(pg)) { executeSql(c, scenario.seedData()); } + String seed = engine.seed(scenario); + if (seed != null && !seed.isBlank()) { + try (Connection c = openConnection(pg)) { executeSql(c, seed, engine); } } // 3. Run queries. @@ -77,12 +94,12 @@ public static void run(QueryScenario scenario, PostgresContainer pg, Path canoni scenario.sourcePath() + " / " + spec.name() + ": no MetaObject named '" + spec.entity() + "' in canonical loader"); Map columnSqlTypes = columnTypeCache.computeIfAbsent( - mc, m -> probeColumnSqlTypes(pg, m)); + mc, m -> probeColumnSqlTypes(pg, m, engine)); // op:relate normalizes the TARGET entity's rows, so the adapter needs // to probe column types for an entity other than `mc`; hand it a probe // that hits the same per-scenario cache. ObjectManagerDbAdapter.ColumnTypeProbe probe = - m -> columnTypeCache.computeIfAbsent(m, x -> probeColumnSqlTypes(pg, x)); + m -> columnTypeCache.computeIfAbsent(m, x -> probeColumnSqlTypes(pg, x, engine)); // FR-017 TPH: an `expect-error: true` op (a cross-subtype write rejection) MUST // throw — any exception/assertion from the dispatch counts as the expected error. @@ -96,7 +113,7 @@ public static void run(QueryScenario scenario, PostgresContainer pg, Path canoni } Object actual = dispatch(omdb, oc, mc, spec, columnSqlTypes, loader, probe); - assertResult(scenario.sourcePath(), spec, actual); + assertResult(scenario.sourcePath(), spec, toWire(engine, mc, actual)); } } finally { omdb.releaseConnection(oc); @@ -129,7 +146,7 @@ private static Object dispatch(ObjectManagerDB omdb, ObjectConnection oc, MetaOb * name. A {@code SELECT * ... WHERE 1=0} returns no rows but a fully-typed * {@link ResultSetMetaData}. Returns an empty map for a non-persistent object. */ - private static Map probeColumnSqlTypes(PostgresContainer pg, MetaObject mc) { + private static Map probeColumnSqlTypes(JdbcTarget pg, MetaObject mc, ScenarioEngine engine) { // FR-044: a report's relation is its read model's view (named by the source's // kind-matching @view alias); the declared node carries no fields to key the probe by. String relation = ReportReadModel.isReport(mc) @@ -138,7 +155,8 @@ private static Map probeColumnSqlTypes(PostgresContainer pg, Me if (relation == null) relation = mc.getPrimaryRdbTableName(); if (relation == null) return Map.of(); Map types = new LinkedHashMap<>(); - String sql = "SELECT * FROM \"" + relation + "\" WHERE 1=0"; + String q = engine.quote(); + String sql = "SELECT * FROM " + q + relation + q + " WHERE 1=0"; try (Connection c = openConnection(pg); Statement s = c.createStatement(); ResultSet rs = s.executeQuery(sql)) { @@ -237,15 +255,15 @@ private static boolean asBoolean(Object v) { // Postgres + ObjectManagerDB plumbing // ----------------------------------------------------------------------- - private static ObjectManagerDB newOmdb(PostgresContainer pg) throws Exception { + private static ObjectManagerDB newOmdb(JdbcTarget pg, ScenarioEngine engine) throws Exception { ObjectManagerDB omdb = new ObjectManagerDB(); - omdb.setDatabaseDriver(new PostgresDriver()); + omdb.setDatabaseDriver(engine.newDriver()); omdb.setDataSource(simpleDataSource(pg)); omdb.init(); return omdb; } - private static DataSource simpleDataSource(PostgresContainer pg) { + private static DataSource simpleDataSource(JdbcTarget pg) { return new DataSource() { @Override public Connection getConnection() throws SQLException { return openConnection(pg); } @Override public Connection getConnection(String u, String p) throws SQLException { return openConnection(pg); } @@ -259,11 +277,59 @@ private static DataSource simpleDataSource(PostgresContainer pg) { }; } - private static Connection openConnection(PostgresContainer pg) throws SQLException { + private static Connection openConnection(JdbcTarget pg) throws SQLException { return DriverManager.getConnection(pg.jdbcUrl(), pg.username(), pg.password()); } - private static void executeSql(Connection c, String sql) throws SQLException { - try (Statement s = c.createStatement()) { s.execute(sql); } + /** Postgres runs a whole script in one statement; MySQL's driver takes one statement at a time. */ + private static void executeSql(Connection c, String sql, ScenarioEngine engine) throws SQLException { + try (Statement s = c.createStatement()) { + if (engine == ScenarioEngine.POSTGRES) { s.execute(sql); return; } + for (String stmt : ScenarioEngine.splitStatements(sql)) s.execute(stmt); + } + } + + /** Drop every view and table the split schema artifact creates (views first, tables in reverse order). */ + private static void dropSchema(Connection c, List statements) throws SQLException { + List views = new java.util.ArrayList<>(); + List tables = new java.util.ArrayList<>(); + java.util.regex.Pattern create = java.util.regex.Pattern.compile("^CREATE (VIEW|TABLE) `([^`]+)`"); + for (String stmt : statements) { + java.util.regex.Matcher m = create.matcher(stmt); + if (m.find()) ("VIEW".equals(m.group(1)) ? views : tables).add(m.group(2)); + } + try (Statement s = c.createStatement()) { + for (String v : views) s.execute("DROP VIEW IF EXISTS `" + v + "`"); + java.util.Collections.reverse(tables); + for (String t : tables) s.execute("DROP TABLE IF EXISTS `" + t + "`"); + } + } + + private static final java.util.regex.Pattern WALL_CLOCK = + java.util.regex.Pattern.compile("\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(?:\\.\\d+)?"); + + /** + * Map a non-Postgres engine's value spelling onto the corpus' wire form. A MySQL DATETIME + * holds the UTC wall clock with no zone, where the report's TIMESTAMPTZ column is a {@code ...Z} + * instant; every other normalized spelling (BIGINT as text, DECIMAL canonical) already agrees. + * Applies to report rows only, and changes spelling, never a value. + */ + @SuppressWarnings("unchecked") + private static Object toWire(ScenarioEngine engine, MetaObject mc, Object actual) { + if (engine != ScenarioEngine.MYSQL || actual == null || !ReportReadModel.isReport(mc)) return actual; + if (actual instanceof List rows) { + List out = new java.util.ArrayList<>(rows.size()); + for (Object r : rows) out.add(toWire(engine, mc, r)); + return out; + } + if (actual instanceof Map row) { + Map out = new LinkedHashMap<>(); + for (Map.Entry e : ((Map) row).entrySet()) { + Object v = e.getValue(); + out.put(e.getKey(), v instanceof String str && WALL_CLOCK.matcher(str).matches() ? str + "Z" : v); + } + return out; + } + return actual; } } diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/ScenarioEngine.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/ScenarioEngine.java new file mode 100644 index 000000000..5291e7b98 --- /dev/null +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/ScenarioEngine.java @@ -0,0 +1,92 @@ +package com.metaobjects.integration; + +import com.metaobjects.integration.Scenarios.QueryScenario; +import com.metaobjects.manager.db.DatabaseDriver; +import com.metaobjects.manager.db.driver.MySQLDriver; +import com.metaobjects.manager.db.driver.PostgresDriver; + +import java.util.Map; +import java.util.regex.Pattern; + +/** + * The SQL engine a persistence scenario runs on. Postgres is the corpus' own engine: the scenario + * YAML holds its seed SQL and its wire-form {@code expect}. Another engine runs the same scenarios + * against its own TS-produced schema artifact ({@code canonical/schema..sql}), with the + * seed re-spelled for it and the engine driver's spelling of a value mapped onto the corpus' wire + * form. Spelling only: a wrong VALUE still fails the comparison, and no {@code expect} is loosened. + */ +public enum ScenarioEngine { + POSTGRES("postgres", "\"") { + @Override public DatabaseDriver newDriver() { return new PostgresDriver(); } + @Override public String seed(QueryScenario s) { return s.seedData(); } + }, + MYSQL("mysql", "`") { + @Override public DatabaseDriver newDriver() { return new MySQLDriver(); } + @Override public String seed(QueryScenario s) { + Map own = s.seedDataEngine(); + if (own != null && own.get("mysql") != null) return own.get("mysql"); + return s.seedData() == null ? null : toMysqlSeed(s.seedData()); + } + }; + + private final String id; + private final String quote; + + ScenarioEngine(String id, String quote) { this.id = id; this.quote = quote; } + + /** The schema artifact name under the corpus' {@code canonical/}. */ + public String schemaArtifact() { return "canonical/schema." + id + ".sql"; } + /** The identifier quote the engine uses. */ + public String quote() { return quote; } + + public abstract DatabaseDriver newDriver(); + /** The scenario's seed SQL spelled for this engine, or null. */ + public abstract String seed(QueryScenario scenario); + + private static final Pattern UTC_INSTANT = Pattern.compile("'(\\d{4}-\\d{2}-\\d{2}T\\d{2}:\\d{2}:\\d{2}(?:\\.\\d+)?)Z'"); + + /** + * Postgres-quoted seed SQL to MySQL. Walks the text once so a double quote INSIDE a + * single-quoted string (a JSON payload) is left alone and only a double-quoted identifier is + * rewritten; a UTC instant literal loses its {@code Z} (a DATETIME holds the UTC wall clock). + */ + static String toMysqlSeed(String seed) { + StringBuilder out = new StringBuilder(seed.length()); + boolean inString = false; + boolean inIdent = false; + for (int i = 0; i < seed.length(); i++) { + char c = seed.charAt(i); + if (inString) { + out.append(c); + if (c == '\'') { + if (i + 1 < seed.length() && seed.charAt(i + 1) == '\'') { out.append('\''); i++; } + else inString = false; + } + } else if (inIdent) { + if (c == '"') { out.append('`'); inIdent = false; } else out.append(c); + } else if (c == '\'') { + inString = true; + out.append(c); + } else if (c == '"') { + inIdent = true; + out.append('`'); + } else { + out.append(c); + } + } + return UTC_INSTANT.matcher(out).replaceAll("'$1'"); + } + + /** The statements of a script: split on a {@code ;} at the end of a line (no body carries one mid-line). */ + static java.util.List splitStatements(String script) { + StringBuilder sb = new StringBuilder(); + for (String line : script.split("\n", -1)) { + if (!line.trim().startsWith("--")) sb.append(line).append('\n'); + } + java.util.List out = new java.util.ArrayList<>(); + for (String stmt : sb.toString().split(";[ \\t]*(?:\\n|$)")) { + if (!stmt.isBlank()) out.add(stmt.trim()); + } + return out; + } +} diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/ScenarioLoader.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/ScenarioLoader.java index 31bba6eb3..ac22055c4 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/ScenarioLoader.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/ScenarioLoader.java @@ -76,6 +76,7 @@ private static QueryScenario parseQuery(Path file, Map root) { stringOrDefault(root, "description", ""), file.toString(), (String) root.get("seed-data"), + (Map) root.get("seed-data-engine"), Collections.unmodifiableList(queries)); } @@ -143,7 +144,12 @@ public static Path findCorpusRoot() { * longer derives the conformance schema from metadata at test time. */ public static String readCanonicalSchema(Path corpusRoot) { - Path schema = corpusRoot.resolve(CANONICAL_SCHEMA_RELATIVE); + return readCanonicalSchema(corpusRoot, CANONICAL_SCHEMA_RELATIVE); + } + + /** A canonical schema artifact by its corpus-relative path ({@code canonical/schema.mysql.sql}). */ + public static String readCanonicalSchema(Path corpusRoot, String relative) { + Path schema = corpusRoot.resolve(relative); if (!Files.isRegularFile(schema)) { throw new IllegalStateException("Canonical schema not found: " + schema); } diff --git a/server/java/integration-tests/src/test/java/com/metaobjects/integration/Scenarios.java b/server/java/integration-tests/src/test/java/com/metaobjects/integration/Scenarios.java index 68c0e13bd..a19e88f93 100644 --- a/server/java/integration-tests/src/test/java/com/metaobjects/integration/Scenarios.java +++ b/server/java/integration-tests/src/test/java/com/metaobjects/integration/Scenarios.java @@ -33,6 +33,7 @@ public record QueryScenario( String description, String sourcePath, String seedData, + Map seedDataEngine, // engine -> seed SQL, when an engine needs its own spelling List queries ) {} } diff --git a/server/typescript/packages/codegen-ts/test/tmp-owned-skew-sqlite-uTNCWG/Article.queries.ts b/server/typescript/packages/codegen-ts/test/tmp-owned-skew-sqlite-uTNCWG/Article.queries.ts deleted file mode 100644 index 424fbd98c..000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-owned-skew-sqlite-uTNCWG/Article.queries.ts +++ /dev/null @@ -1,88 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — DO NOT EDIT. -// Source metadata: Article (Article) -// Extend in your own module (e.g. Article.extra.ts) — nothing imports it for you. -import { eq } from "drizzle-orm"; -import type { z } from "zod"; - -import type { BaseSQLiteDatabase } from "drizzle-orm/sqlite-core"; -type Db = BaseSQLiteDatabase< - "sync" | "async", - unknown, - Record ->; - -import { - type Article, - ArticleInsertPreservingSchema, - ArticleInsertSchema, - type ArticlePatch, - articles, - ArticleUpdateSchema, -} from "./Article"; -export async function findArticleById( - db: Db, - id: number, -): Promise
{ - const [article] = await db - .select() - .from(articles) - .where(eq(articles.id, id)) - .limit(1); - return article ?? null; -} -export async function listArticles( - db: Db, - opts?: { limit?: number; offset?: number }, -): Promise { - let q = db.select().from(articles).$dynamic(); - if (opts?.limit !== undefined) { - q = q.limit(opts.limit); - } - if (opts?.offset !== undefined) { - q = q.offset(opts.offset); - } - return q; -} -export async function createArticle( - db: Db, - data: z.input, -): Promise
{ - const validated = ArticleInsertSchema.parse(data); - const [article] = await db.insert(articles).values(validated).returning(); - return article!; -} -export async function insertPreservingArticle( - db: Db, - data: z.input, -): Promise
{ - const validated = ArticleInsertPreservingSchema.parse(data); - const [article] = await db.insert(articles).values(validated).returning(); - return article!; -} -export async function updateArticle( - db: Db, - id: number, - patch: ArticlePatch, -): Promise
{ - const validated = ArticleUpdateSchema.parse(patch); - // PATCH-5: an empty patch is a no-op — return the current row rather than let - // Drizzle throw on an empty SET clause. - if (Object.keys(validated).length === 0) { - return findArticleById(db, id); - } - const [article] = await db - .update(articles) - .set(validated) - .where(eq(articles.id, id)) - .returning(); - return article ?? null; -} -export async function deleteArticleById(db: Db, id: number): Promise { - // Use .returning() unconditionally — supported on SQLite ≥3.35 (covers D1, libsql/Turso) - // and Postgres. Result is an array of deleted rows; presence implies success. - const deleted = await db - .delete(articles) - .where(eq(articles.id, id)) - .returning(); - return deleted.length > 0; -} diff --git a/server/typescript/packages/codegen-ts/test/tmp-owned-skew-sqlite-uTNCWG/Article.ts b/server/typescript/packages/codegen-ts/test/tmp-owned-skew-sqlite-uTNCWG/Article.ts deleted file mode 100644 index 977c3f263..000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-owned-skew-sqlite-uTNCWG/Article.ts +++ /dev/null @@ -1,159 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: Article (Article) -// Extend in your own module (e.g. Article.extra.ts) — nothing imports it for you. -import type { InferInsertModel, InferSelectModel } from "drizzle-orm"; -import { integer, sqliteTable, text } from "drizzle-orm/sqlite-core"; -import { z } from "zod"; - -export const articles = sqliteTable("articles", { - id: integer("id").primaryKey({ autoIncrement: true }), - title: text("title").notNull(), - createdAt: text("created_at").$defaultFn(() => new Date().toISOString()), - updatedAt: text("updated_at").$defaultFn(() => new Date().toISOString()), -}); -export type Article = InferSelectModel; -export type ArticleInsert = InferInsertModel; -export type ArticleUpdate = Partial; -/** A field.timestamp instant in UTC (`YYYY-MM-DDTHH:MM:SS[.fff]Z`): a value already spelled so is kept - * as sent, an offset is applied, a zoneless value is UTC and a date alone is midnight UTC. - * SQLite/D1 compare timestamps as text, so instants are stored in UTC. Generated. */ -function utcIsoTimestamp(v: string): string { - if (/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?Z$/.test(v)) { - return v; - } - const m = - /^(\d{4}-\d{2}-\d{2})(?:[Tt ](\d{2}:\d{2})(?::(\d{2})(?:\.(\d+))?)?(?:([Zz])|([+-]\d{2}):?(\d{2})?)?)?$/.exec( - v, - ); - if (m === null) { - return v; - } - const t = Date.parse( - `${m[1]}T${m[2] ?? "00:00"}:${m[3] ?? "00"}.${(m[4] ?? "").slice(0, 3).padEnd(3, "0")}${ - m[6] === undefined ? "Z" : `${m[6]}:${m[7] ?? "00"}` - }`, - ); - return Number.isNaN(t) - ? v - : new Date(t).toISOString().replace(/\.?0+Z$/, "Z"); -} - -export const ArticleInsertSchema = z.object({ - title: z.string().min(1).max(200), - createdAt: z - .string() - .optional() - .transform(() => new Date().toISOString()), - updatedAt: z - .string() - .optional() - .transform(() => new Date().toISOString()), -}); - -export const ArticleUpdateSchema = z.object({ - title: z.string().min(1).max(200).optional(), - updatedAt: z - .string() - .optional() - .transform(() => new Date().toISOString()), -}); - -/** Typed create shape for Article: the insert schema's INPUT (pre-transform) type. A - * renamed/dropped/misspelt field is a compile error at every `createArticle` call site; - * the schema still validates at runtime. */ -export type ArticleCreate = z.input; - -/** Typed patch shape for Article: every settable field, optional (FR-035 PATCH). A - * renamed/dropped field is a compile error at every `updateArticle` call site. */ -export type ArticlePatch = z.input; - -/** Insert-shape for import / restore / replication of Article: identical to - * ArticleInsertSchema, but the @autoSet timestamp columns are written VERBATIM - * (no create-time now() stamp) so the caller's original values are preserved. */ -export const ArticleInsertPreservingSchema = z.object({ - title: z.string().min(1).max(200), - createdAt: z - .string() - .regex( - /^(?:\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|02-(?:0[1-9]|1\d|2[0-8]))|(?:\d{2}(?:0[48]|[2468][048]|[13579][26])|(?:[02468][048]|[13579][26])00)-02-29)(?:[Tt ](?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d+)?)?(?:[Zz]|[+-](?:[01]\d|2[0-3])(?::?[0-5]\d)?)?)?$/, - "must be an ISO 8601 timestamp (YYYY-MM-DD[THH:MM[:SS[.fff]]][Z|±HH:MM])", - ) - .transform(utcIsoTimestamp) - .optional(), - updatedAt: z - .string() - .regex( - /^(?:\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|02-(?:0[1-9]|1\d|2[0-8]))|(?:\d{2}(?:0[48]|[2468][048]|[13579][26])|(?:[02468][048]|[13579][26])00)-02-29)(?:[Tt ](?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d+)?)?(?:[Zz]|[+-](?:[01]\d|2[0-3])(?::?[0-5]\d)?)?)?$/, - "must be an ISO 8601 timestamp (YYYY-MM-DD[THH:MM[:SS[.fff]]][Z|±HH:MM])", - ) - .transform(utcIsoTimestamp) - .optional(), -}); - -/** Typed input of `insertPreservingArticle` — ArticleInsertPreservingSchema's pre-transform shape. */ -export type ArticleCreatePreserving = z.input< - typeof ArticleInsertPreservingSchema ->; -/** - * Metadata constants for Article. - * - * Use these instead of magic strings so TS catches typos and refactors stay - * coherent. Each non-dollar-prefixed key is a per-field object carrying - * name, label, view, an htmlType where the field's view maps to one, and the - * RHF-shaped validation rules derived from the field's validator children. - * - * placeholder and helpText are NOT emitted — no provider registers an attribute - * for them, so nothing here could derive one. (htmlType is emitted, but derived - * from the view subtype; the @htmlType override was removed for the same reason.) - * useEntityForm still honours placeholder if you add it: this file is generated, - * hand edits inside it survive regeneration through the three-way merge, and the - * consumers read this const rather than the metadata. That is the intended way - * to set it. - * - * Typical usage with the metaobjects React form helper: - * - * import { useEntityForm } from '@metaobjectsdev/react'; - * const form = useEntityForm(Article, ArticleInsertSchema); - * - */ -export const Article = { - $entity: "Article", - $table: "articles", - $path: "/articles", - id: { name: "id", label: "Id", view: "number", htmlType: "number" }, - title: { - name: "title", - label: "Title", - view: "text", - htmlType: "text", - rules: { - required: "Title is required", - maxLength: { value: 200, message: "Must be 200 characters or fewer" }, - }, - }, - createdAt: { - name: "createdAt", - label: "Created At", - view: "date", - htmlType: "datetime-local", - }, - updatedAt: { - name: "updatedAt", - label: "Updated At", - view: "date", - htmlType: "datetime-local", - }, -} as const; -import type { FilterAllowlist } from "@metaobjectsdev/runtime-ts/drizzle-fastify"; - -export const ArticleFilterAllowlist = {} as const satisfies FilterAllowlist; -import type { SortAllowlist } from "@metaobjectsdev/runtime-ts/drizzle-fastify"; - -export const ArticleSortAllowlist = {} as const satisfies SortAllowlist; -export type ArticleFilter = { - limit?: number; - offset?: number; - sort?: string; - or?: ArticleFilter[]; - and?: ArticleFilter[]; -}; diff --git a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Brewer.queries.ts b/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Brewer.queries.ts deleted file mode 100644 index 9d45e61b9..000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Brewer.queries.ts +++ /dev/null @@ -1,88 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: Brewer (Brewer) -// Extend in your own module (e.g. Brewer.extra.ts) — nothing imports it for you. -import { eq } from "drizzle-orm"; -import type { z } from "zod"; - -import type { BaseSQLiteDatabase } from "drizzle-orm/sqlite-core"; -type Db = BaseSQLiteDatabase< - "sync" | "async", - unknown, - Record ->; - -import { - type Brewer, - BrewerInsertPreservingSchema, - BrewerInsertSchema, - type BrewerPatch, - brewers, - BrewerUpdateSchema, -} from "./Brewer.js"; -export async function findBrewerById( - db: Db, - id: number, -): Promise { - const [brewer] = await db - .select() - .from(brewers) - .where(eq(brewers.id, id)) - .limit(1); - return brewer ?? null; -} -export async function listBrewers( - db: Db, - opts?: { limit?: number; offset?: number }, -): Promise { - let q = db.select().from(brewers).$dynamic(); - if (opts?.limit !== undefined) { - q = q.limit(opts.limit); - } - if (opts?.offset !== undefined) { - q = q.offset(opts.offset); - } - return q; -} -export async function createBrewer( - db: Db, - data: z.input, -): Promise { - const validated = BrewerInsertSchema.parse(data); - const [brewer] = await db.insert(brewers).values(validated).returning(); - return brewer!; -} -export async function insertPreservingBrewer( - db: Db, - data: z.input, -): Promise { - const validated = BrewerInsertPreservingSchema.parse(data); - const [brewer] = await db.insert(brewers).values(validated).returning(); - return brewer!; -} -export async function updateBrewer( - db: Db, - id: number, - patch: BrewerPatch, -): Promise { - const validated = BrewerUpdateSchema.parse(patch); - // PATCH-5: an empty patch is a no-op — return the current row rather than let - // Drizzle throw on an empty SET clause. - if (Object.keys(validated).length === 0) { - return findBrewerById(db, id); - } - const [brewer] = await db - .update(brewers) - .set(validated) - .where(eq(brewers.id, id)) - .returning(); - return brewer ?? null; -} -export async function deleteBrewerById(db: Db, id: number): Promise { - // Use .returning() unconditionally — supported on SQLite ≥3.35 (covers D1, libsql/Turso) - // and Postgres. Result is an array of deleted rows; presence implies success. - const deleted = await db - .delete(brewers) - .where(eq(brewers.id, id)) - .returning(); - return deleted.length > 0; -} diff --git a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Brewer.ts b/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Brewer.ts deleted file mode 100644 index 71819b99d..000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Brewer.ts +++ /dev/null @@ -1,143 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: Brewer (Brewer) -// Extend in your own module (e.g. Brewer.extra.ts) — nothing imports it for you. -import type { InferInsertModel, InferSelectModel } from "drizzle-orm"; -import { integer, sqliteTable, text } from "drizzle-orm/sqlite-core"; -import { z } from "zod"; - -export const brewers = sqliteTable("brewers", { - id: integer("id").primaryKey({ autoIncrement: true }), - displayName: text("display_name").notNull(), - email: text("email").notNull(), - createdAt: text("created_at") - .notNull() - .$defaultFn(() => new Date().toISOString()), -}); -export type Brewer = InferSelectModel; -export type BrewerInsert = InferInsertModel; -export type BrewerUpdate = Partial; -/** A field.timestamp instant in UTC (`YYYY-MM-DDTHH:MM:SS[.fff]Z`): a value already spelled so is kept - * as sent, an offset is applied, a zoneless value is UTC and a date alone is midnight UTC. - * SQLite/D1 compare timestamps as text, so instants are stored in UTC. Generated. */ -function utcIsoTimestamp(v: string): string { - if (/^\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?Z$/.test(v)) { - return v; - } - const m = - /^(\d{4}-\d{2}-\d{2})(?:[Tt ](\d{2}:\d{2})(?::(\d{2})(?:\.(\d+))?)?(?:([Zz])|([+-]\d{2}):?(\d{2})?)?)?$/.exec( - v, - ); - if (m === null) { - return v; - } - const t = Date.parse( - `${m[1]}T${m[2] ?? "00:00"}:${m[3] ?? "00"}.${(m[4] ?? "").slice(0, 3).padEnd(3, "0")}${ - m[6] === undefined ? "Z" : `${m[6]}:${m[7] ?? "00"}` - }`, - ); - return Number.isNaN(t) - ? v - : new Date(t).toISOString().replace(/\.?0+Z$/, "Z"); -} - -export const BrewerInsertSchema = z.object({ - displayName: z.string().min(1).max(80), - email: z.string().min(1), - createdAt: z - .string() - .optional() - .transform(() => new Date().toISOString()), -}); - -export const BrewerUpdateSchema = z.object({ - displayName: z.string().min(1).max(80).optional(), - email: z.string().min(1).optional(), -}); - -/** Typed create shape for Brewer: the insert schema's INPUT (pre-transform) type. A - * renamed/dropped/misspelt field is a compile error at every `createBrewer` call site; - * the schema still validates at runtime. */ -export type BrewerCreate = z.input; - -/** Typed patch shape for Brewer: every settable field, optional (FR-035 PATCH). A - * renamed/dropped field is a compile error at every `updateBrewer` call site. */ -export type BrewerPatch = z.input; - -/** Insert-shape for import / restore / replication of Brewer: identical to - * BrewerInsertSchema, but the @autoSet timestamp columns are written VERBATIM - * (no create-time now() stamp) so the caller's original values are preserved. */ -export const BrewerInsertPreservingSchema = z.object({ - displayName: z.string().min(1).max(80), - email: z.string().min(1), - createdAt: z - .string() - .regex( - /^(?:\d{4}-(?:(?:0[13578]|1[02])-(?:0[1-9]|[12]\d|3[01])|(?:0[469]|11)-(?:0[1-9]|[12]\d|30)|02-(?:0[1-9]|1\d|2[0-8]))|(?:\d{2}(?:0[48]|[2468][048]|[13579][26])|(?:[02468][048]|[13579][26])00)-02-29)(?:[Tt ](?:[01]\d|2[0-3]):[0-5]\d(?::[0-5]\d(?:\.\d+)?)?(?:[Zz]|[+-](?:[01]\d|2[0-3])(?::?[0-5]\d)?)?)?$/, - "must be an ISO 8601 timestamp (YYYY-MM-DD[THH:MM[:SS[.fff]]][Z|±HH:MM])", - ) - .transform(utcIsoTimestamp), -}); - -/** Typed input of `insertPreservingBrewer` — BrewerInsertPreservingSchema's pre-transform shape. */ -export type BrewerCreatePreserving = z.input< - typeof BrewerInsertPreservingSchema ->; -/** - * Metadata constants for Brewer. - * - * Use these instead of magic strings so TS catches typos and refactors stay - * coherent. Each non-dollar-prefixed key is a per-field object carrying - * name, label, view, an htmlType where the field's view maps to one, and the - * RHF-shaped validation rules derived from the field's validator children. - * - * placeholder and helpText are NOT emitted — no provider registers an attribute - * for them, so nothing here could derive one. (htmlType is emitted, but derived - * from the view subtype; the @htmlType override was removed for the same reason.) - * useEntityForm still honours placeholder if you add it: this file is generated, - * hand edits inside it survive regeneration through the three-way merge, and the - * consumers read this const rather than the metadata. That is the intended way - * to set it. - * - * Typical usage with the metaobjects React form helper: - * - * import { useEntityForm } from '@metaobjectsdev/react'; - * const form = useEntityForm(Brewer, BrewerInsertSchema); - * - */ -export const Brewer = { - $entity: "Brewer", - $table: "brewers", - $path: "/brewers", - id: { name: "id", label: "Id", view: "number", htmlType: "number" }, - displayName: { - name: "displayName", - label: "Display Name", - view: "text", - htmlType: "text", - rules: { - required: "Display Name is required", - maxLength: { value: 80, message: "Must be 80 characters or fewer" }, - }, - }, - email: { - name: "email", - label: "Email", - view: "text", - htmlType: "text", - rules: { required: "Email is required" }, - }, - createdAt: { - name: "createdAt", - label: "Created At", - view: "date", - htmlType: "datetime-local", - rules: { required: "Created At is required" }, - }, -} as const; -export type BrewerFilter = { - limit?: number; - offset?: number; - sort?: string; - or?: BrewerFilter[]; - and?: BrewerFilter[]; -}; diff --git a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Carrier.ts b/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Carrier.ts deleted file mode 100644 index 50441447e..000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Carrier.ts +++ /dev/null @@ -1,98 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: Carrier (Carrier) -// Extend in your own module (e.g. Carrier.extra.ts) — nothing imports it for you. -import { z } from "zod"; - -export interface Carrier { - id?: number | undefined; - partyType?: CarrierPartyType | null | undefined; - label: string; - fleetSize?: number | null | undefined; -} -export type CarrierPartyType = "Carrier"; -export const CarrierPartyTypeEnum = z.enum(["Carrier"]); -export const CarrierSchema = z.object({ - id: z.number().int(), - partyType: z.literal("Carrier"), - label: z.string().min(1), - fleetSize: z.number().int().nullable(), -}); -export const CarrierInsertSchema = z.object({ - partyType: z.literal("Carrier"), - label: z.string().min(1), - fleetSize: z.number().int().optional(), -}); - -export const CarrierUpdateSchema = z.object({ - label: z.string().min(1).optional(), - fleetSize: z.number().int().optional().nullable(), -}); - -/** Typed create shape for Carrier: the insert schema's INPUT (pre-transform) type. A - * renamed/dropped/misspelt field is a compile error at every `createCarrier` call site; - * the schema still validates at runtime. */ -export type CarrierCreate = z.input; - -/** Typed patch shape for Carrier: every settable field, optional (FR-035 PATCH). A - * renamed/dropped field is a compile error at every `updateCarrier` call site. */ -export type CarrierPatch = z.input; -/** - * Metadata constants for Carrier. - * - * Use these instead of magic strings so TS catches typos and refactors stay - * coherent. Each non-dollar-prefixed key is a per-field object carrying - * name, label, view, an htmlType where the field's view maps to one, and the - * RHF-shaped validation rules derived from the field's validator children. - * - * placeholder and helpText are NOT emitted — no provider registers an attribute - * for them, so nothing here could derive one. (htmlType is emitted, but derived - * from the view subtype; the @htmlType override was removed for the same reason.) - * useEntityForm still honours placeholder if you add it: this file is generated, - * hand edits inside it survive regeneration through the three-way merge, and the - * consumers read this const rather than the metadata. That is the intended way - * to set it. - * - * Typical usage with the metaobjects React form helper: - * - * import { useEntityForm } from '@metaobjectsdev/react'; - * const form = useEntityForm(Carrier, CarrierInsertSchema); - * - */ -export const Carrier = { - $entity: "Carrier", - $table: "parties", - $path: "/parties/carrier", - id: { name: "id", label: "Id", view: "number", htmlType: "number" }, - partyType: { - name: "partyType", - label: "Party Type", - view: "dropdown", - options: ["Carrier"] as const, - }, - label: { - name: "label", - label: "Label", - view: "text", - htmlType: "text", - rules: { required: "Label is required" }, - }, - fleetSize: { - name: "fleetSize", - label: "Fleet Size", - view: "number", - htmlType: "number", - }, -} as const; -import type { FilterAllowlist } from "@metaobjectsdev/runtime-ts/drizzle-fastify"; - -export const CarrierFilterAllowlist = {} as const satisfies FilterAllowlist; -import type { SortAllowlist } from "@metaobjectsdev/runtime-ts/drizzle-fastify"; - -export const CarrierSortAllowlist = {} as const satisfies SortAllowlist; -export type CarrierFilter = { - limit?: number; - offset?: number; - sort?: string; - or?: CarrierFilter[]; - and?: CarrierFilter[]; -}; diff --git a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Party.queries.ts b/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Party.queries.ts deleted file mode 100644 index 62a1de1fc..000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Party.queries.ts +++ /dev/null @@ -1,106 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: Party (Party) — TPH discriminator base -// Extend in your own module (e.g. Party.extra.ts) — nothing imports it for you. -import { and, eq } from "drizzle-orm"; -import type { z } from "zod"; -import { type Carrier, CarrierInsertSchema, CarrierSchema } from "./Carrier.js"; -import { parseParty, parties, type Party } from "./Party.js"; - -import type { BaseSQLiteDatabase } from "drizzle-orm/sqlite-core"; -type Db = BaseSQLiteDatabase< - "sync" | "async", - unknown, - Record ->; -export async function findPartyById(db: Db, id: number): Promise { - const [row] = await db - .select() - .from(parties) - .where(eq(parties.id, id)) - .limit(1); - return row ? parseParty(row) : null; -} - -export async function listParties( - db: Db, - opts?: { limit?: number; offset?: number }, -): Promise { - let q = db.select().from(parties).$dynamic(); - if (opts?.limit !== undefined) { - q = q.limit(opts.limit); - } - if (opts?.offset !== undefined) { - q = q.offset(opts.offset); - } - const rows = await q; - return rows.map((r) => parseParty(r)); -} -export async function listCarriers( - db: Db, - opts?: { limit?: number; offset?: number }, -): Promise { - let q = db - .select() - .from(parties) - .where(eq(parties.partyType, "Carrier")) - .$dynamic(); - if (opts?.limit !== undefined) { - q = q.limit(opts.limit); - } - if (opts?.offset !== undefined) { - q = q.offset(opts.offset); - } - const rows = await q; - return rows.map((r) => CarrierSchema.parse(r)); -} - -export async function findCarrierById( - db: Db, - id: number, -): Promise { - const [row] = await db - .select() - .from(parties) - .where(and(eq(parties.id, id), eq(parties.partyType, "Carrier"))) - .limit(1); - return row ? CarrierSchema.parse(row) : null; -} - -export async function createCarrier( - db: Db, - data: z.input, -): Promise { - const validated = CarrierInsertSchema.parse(data); - const [row] = await db - .insert(parties) - .values({ ...validated, partyType: "Carrier" }) - .returning(); - return CarrierSchema.parse(row!); -} - -export async function updateCarrierById( - db: Db, - id: number, - data: Partial>, -): Promise { - const validated = CarrierInsertSchema.partial().parse(data) as Record< - string, - unknown - >; - // The discriminator is immutable — a Carrier can never become another subtype. - const { ["partyType"]: _disc, ...safe } = validated; - const [row] = await db - .update(parties) - .set(safe) - .where(and(eq(parties.id, id), eq(parties.partyType, "Carrier"))) - .returning(); - return row ? CarrierSchema.parse(row) : null; -} - -export async function deleteCarrierById(db: Db, id: number): Promise { - const deleted = await db - .delete(parties) - .where(and(eq(parties.id, id), eq(parties.partyType, "Carrier"))) - .returning(); - return deleted.length > 0; -} diff --git a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Party.ts b/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Party.ts deleted file mode 100644 index 8d4e05c5a..000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/Party.ts +++ /dev/null @@ -1,108 +0,0 @@ -// @generated by @metaobjectsdev/codegen-ts — hand edits are kept by a three-way merge on regeneration; meta verify --codegen checks only the generated parts. -// Source metadata: Party (Party) -// Extend in your own module (e.g. Party.extra.ts) — nothing imports it for you. -import { type InferInsertModel, type InferSelectModel, sql } from "drizzle-orm"; -import { check, integer, sqliteTable, text } from "drizzle-orm/sqlite-core"; -import { z } from "zod"; -import { type Carrier, CarrierSchema } from "./Carrier.js"; - -export const parties = sqliteTable( - "parties", - { - id: integer("id").primaryKey({ autoIncrement: true }), - partyType: text("party_type", { enum: ["Carrier"] as const }), - label: text("label").notNull(), - fleetSize: integer("fleet_size"), - }, - (table) => [check("parties_party_type_chk", sql`party_type IN ('Carrier')`)], -); -export type PartyRow = InferSelectModel; -export type PartyInsert = InferInsertModel; -export type PartyUpdate = Partial; -export type PartyPartyType = "Carrier"; -export const PartyPartyTypeEnum = z.enum(["Carrier"]); -export const PartyInsertSchema = z.object({ - partyType: z.enum(["Carrier"]).optional(), - label: z.string().min(1), -}); - -export const PartyUpdateSchema = z.object({ - partyType: z.enum(["Carrier"]).optional().nullable(), - label: z.string().min(1).optional(), -}); - -/** Typed create shape for Party: the insert schema's INPUT (pre-transform) type. A - * renamed/dropped/misspelt field is a compile error at every `createParty` call site; - * the schema still validates at runtime. */ -export type PartyCreate = z.input; - -/** Typed patch shape for Party: every settable field, optional (FR-035 PATCH). A - * renamed/dropped field is a compile error at every `updateParty` call site. */ -export type PartyPatch = z.input; -/** - * Metadata constants for Party. - * - * Use these instead of magic strings so TS catches typos and refactors stay - * coherent. Each non-dollar-prefixed key is a per-field object carrying - * name, label, view, an htmlType where the field's view maps to one, and the - * RHF-shaped validation rules derived from the field's validator children. - * - * placeholder and helpText are NOT emitted — no provider registers an attribute - * for them, so nothing here could derive one. (htmlType is emitted, but derived - * from the view subtype; the @htmlType override was removed for the same reason.) - * useEntityForm still honours placeholder if you add it: this file is generated, - * hand edits inside it survive regeneration through the three-way merge, and the - * consumers read this const rather than the metadata. That is the intended way - * to set it. - * - * Typical usage with the metaobjects React form helper: - * - * import { useEntityForm } from '@metaobjectsdev/react'; - * const form = useEntityForm(Party, PartyInsertSchema); - * - */ -export const Party = { - $entity: "Party", - $table: "parties", - $path: "/parties", - id: { name: "id", label: "Id", view: "number", htmlType: "number" }, - partyType: { - name: "partyType", - label: "Party Type", - view: "dropdown", - options: ["Carrier"] as const, - }, - label: { - name: "label", - label: "Label", - view: "text", - htmlType: "text", - rules: { required: "Label is required" }, - }, -} as const; -export type PartyFilter = { - limit?: number; - offset?: number; - sort?: string; - or?: PartyFilter[]; - and?: PartyFilter[]; -}; -export type Party = Carrier; - -/** True when value is a Carrier (discriminated by partyType === "Carrier"). */ -export function isCarrier(value: Party): value is Carrier { - return value.partyType === "Carrier"; -} - -/** - * Parse a row from the Party table, dispatching by the - * `partyType` discriminator value to the matching subtype's - * Zod schema. Throws on unknown discriminator values. - */ -export function parseParty(row: unknown): Party { - const head = z.object({ partyType: z.enum(["Carrier"]) }).parse(row); - switch (head.partyType) { - case "Carrier": - return CarrierSchema.parse(row); - } -} diff --git a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/caller.ts b/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/caller.ts deleted file mode 100644 index 14770a217..000000000 --- a/server/typescript/packages/codegen-ts/test/tmp-typed-create-sqlite-NdCqpM/caller.ts +++ /dev/null @@ -1,7 +0,0 @@ -import { createBrewer, insertPreservingBrewer } from "./Brewer.queries"; -import { createCarrier, updateCarrierById } from "./Party.queries"; -declare const db: Parameters[0]; -declare const tphDb: Parameters[0]; -export async function run(): Promise { - await createBrewer(db, { email: "cy@example.com" }); -} diff --git a/server/typescript/packages/integration-tests/cube-live/cube-live-support.ts b/server/typescript/packages/integration-tests/cube-live/cube-live-support.ts new file mode 100644 index 000000000..337bd07af --- /dev/null +++ b/server/typescript/packages/integration-tests/cube-live/cube-live-support.ts @@ -0,0 +1,292 @@ +// cube-live-support.ts — what the Cube live checks share, whatever data source Cube reads: the +// Table F query that reproduces a report, Table I's normalization, and a client for Cube's REST +// API. cube-model.live.ts (Postgres) and cube-model-mysql.live.ts (MySQL) import it. + +import { expect } from "bun:test"; +import { + GRAIN_HOUR, + OBJECT_REPORT_ATTR_FILTER, + OBJECT_REPORT_ATTR_SEGMENT, + reportShape, + reportSpine, + type MetaObject, + type MetaRoot, +} from "@metaobjectsdev/metadata"; +import type { CubeStack } from "../src/cube-container.ts"; + +/** One HTTP request to Cube; a stalled socket is aborted and counts as a transport error. */ +export const REQUEST_TIMEOUT_MS = 60_000; +/** + * How long one `/v1/load` may keep retrying (`Continue wait` while a rollup builds, or a + * transport error). The worst case stays inside TEST_TIMEOUT_MS: this deadline, one request + * timeout and the last backoff, then a `/v1/sql` request and the view read. + */ +export const LOAD_DEADLINE_MS = 300_000; +/** Consecutive transport errors (refused, reset, aborted, a body that is not JSON) a load survives. */ +export const LOAD_TRANSPORT_RETRIES = 6; +/** Backoff after a transport error: 1 s, doubling, capped here. */ +export const LOAD_BACKOFF_CAP_MS = 10_000; + +// --------------------------------------------------------------------------------------------- +// Table F: the Cube query that reproduces a report. +// --------------------------------------------------------------------------------------------- + +export interface CubeTimeDimension { + readonly dimension: string; + readonly granularity: string; +} + +export interface CubeQuery { + readonly measures: readonly string[]; + readonly dimensions: readonly string[]; + readonly timeDimensions: readonly CubeTimeDimension[]; + readonly segments: readonly string[]; + /** Table I fixes the query time zone at UTC (Table J: another zone re-buckets). */ + readonly timezone: "UTC"; +} + +export type ColumnKind = "attribute" | "time" | "hour" | "measure"; + +/** One report field: its view column and the Cube row key that carries it. */ +export interface PlannedColumn { + readonly field: string; + readonly cubeKey: string; + readonly kind: ColumnKind; +} + +export interface QueryPlan { + readonly query: CubeQuery; + readonly columns: readonly PlannedColumn[]; +} + + + +/** Lower camel case of a report name: `RecentPrograms` → `recentPrograms` (Table F's scope segment). */ +export function lowerFirst(s: string): string { + return s.charAt(0).toLowerCase() + s.slice(1); +} + +export function cubeQueryFor(report: MetaObject, root: MetaRoot): QueryPlan { + const shape = reportShape(report, root); + // A @spine report is a Cube view of its own name, whose scope is inside its facts cube. + const spine = reportSpine(report) !== undefined; + const cube = spine ? report.name : shape.from.name; + const measures: string[] = []; + const dimensions: string[] = []; + const timeDimensions: CubeTimeDimension[] = []; + const columns: PlannedColumn[] = []; + for (const f of shape.fields) { + if (f.role === "measure") { + if (f.measure === undefined) throw new Error(`${report.name}.${f.name}: a measure field without its measure`); + const member = `${cube}.${f.measure.name}`; + measures.push(member); + columns.push({ field: f.name, cubeKey: member, kind: "measure" }); + } else { + if (f.dimension === undefined) throw new Error(`${report.name}.${f.name}: a dimension field without its dimension`); + const member = `${cube}.${f.dimension.name}`; + if (f.grain === undefined) { + dimensions.push(member); + columns.push({ field: f.name, cubeKey: member, kind: "attribute" }); + } else { + timeDimensions.push({ dimension: member, granularity: f.grain }); + columns.push({ field: f.name, cubeKey: `${member}.${f.grain}`, kind: f.grain === GRAIN_HOUR ? "hour" : "time" }); + } + } + } + const segments: string[] = []; + const segment = report.attr(OBJECT_REPORT_ATTR_SEGMENT); + if (!spine && typeof segment === "string") segments.push(`${cube}.${segment}`); + if (!spine && report.attr(OBJECT_REPORT_ATTR_FILTER) !== undefined) segments.push(`${cube}.${lowerFirst(report.name)}Scope`); + return { query: { measures, dimensions, timeDimensions, segments, timezone: "UTC" }, columns }; +} + +// --------------------------------------------------------------------------------------------- +// Table I: normalization and comparison. +// --------------------------------------------------------------------------------------------- + +export type NormalRow = Record; + +export const DECIMAL_RE = /^-?\d+(\.\d+)?$/; + +/** `60.0000000000000000` → `60`, `0.75000000000000000000` → `0.75`; an integer string is kept. */ +export function canonicalDecimal(s: string): string { + return s.includes(".") ? s.replace(/\.?0+$/, "") : s; +} + +/** True when the string already names its offset (`Z`, `+00:00`, `-0500`). */ +export const HAS_ZONE_RE = /(Z|[+-]\d{2}(:?\d{2})?)$/; + +export function normalize(kind: ColumnKind, raw: unknown, where: string): string | null { + if (raw === null || raw === undefined) return null; + if (typeof raw !== "string" && typeof raw !== "number") throw new Error(`${where}: unexpected value ${JSON.stringify(raw)}`); + const s = String(raw); + switch (kind) { + case "time": + // A day, week, month, quarter or year bucket: Cube sends `2026-05-01T00:00:00.000`, the view a DATE. + return s.slice(0, 10); + case "hour": { + // An instant. Cube sends the wall clock in the query time zone (UTC, fixed by the query); + // the view sends a timestamptz with its offset. + const d = new Date(HAS_ZONE_RE.test(s) ? s : `${s}Z`); + if (Number.isNaN(d.getTime())) throw new Error(`${where}: '${s}' is not a timestamp`); + return d.toISOString(); + } + case "attribute": + case "measure": + return DECIMAL_RE.test(s) ? canonicalDecimal(s) : s; + } +} + +export function fromCubeRow(row: Record, plan: QueryPlan): NormalRow { + const out: NormalRow = {}; + for (const c of plan.columns) { + if (!(c.cubeKey in row)) throw new Error(`Cube's row has no '${c.cubeKey}': ${JSON.stringify(row)}`); + out[c.field] = normalize(c.kind, row[c.cubeKey], `Cube ${c.cubeKey}`); + } + return out; +} + +export function fromViewRow(row: Record, plan: QueryPlan, view: string): NormalRow { + const want = plan.columns.map((c) => c.field).sort(); + expect(Object.keys(row).sort()).toEqual(want); + const out: NormalRow = {}; + for (const c of plan.columns) out[c.field] = normalize(c.kind, row[c.field], `${view}.${c.field}`); + return out; +} + +/** Rows as a set, ordered by their dimension values (then by the whole row, for determinism). */ +export function sortRows(rows: NormalRow[], plan: QueryPlan): NormalRow[] { + const dims = plan.columns.filter((c) => c.kind !== "measure").map((c) => c.field); + const key = (r: NormalRow): string => JSON.stringify([dims.map((d) => r[d] ?? null), plan.columns.map((c) => r[c.field] ?? null)]); + return rows.slice().sort((a, b) => (key(a) < key(b) ? -1 : key(a) > key(b) ? 1 : 0)); +} + +// --------------------------------------------------------------------------------------------- +// The Cube REST API. +// --------------------------------------------------------------------------------------------- + +export interface MetaMember { + readonly name: string; + readonly type?: string; + readonly aggType?: string; + readonly public?: boolean; + /** A member's own declared title (`title` is the cube's title and this one, joined). */ + readonly shortTitle?: string; + readonly description?: string; +} + +export interface MetaCube { + readonly name: string; + readonly public?: boolean; + readonly title?: string; + readonly description?: string; + readonly measures?: readonly MetaMember[]; + readonly dimensions?: readonly MetaMember[]; + readonly segments?: readonly MetaMember[]; +} + +export interface MetaResponse { + readonly error?: string; + readonly cubes: readonly MetaCube[]; +} + +export interface LoadResponse { + readonly data: Array>; + readonly usedPreAggregations?: Record; +} + +/** A request that never produced a JSON answer: refused, reset, aborted, or a body that is not JSON. */ +export class TransportError extends Error {} + +export async function getJson(url: string, timeoutMs = REQUEST_TIMEOUT_MS): Promise<{ status: number; body: unknown }> { + let res: Response; + let text: string; + try { + res = await fetch(url, { signal: AbortSignal.timeout(Math.max(1000, timeoutMs)) }); + text = await res.text(); + } catch (e) { + throw new TransportError(`${url.split("?")[0]}: ${e instanceof Error ? `${e.name}: ${e.message}` : String(e)}`, { cause: e }); + } + try { + return { status: res.status, body: JSON.parse(text) }; + } catch { + throw new TransportError(`${url.split("?")[0]}: HTTP ${res.status}, not JSON: ${text.slice(0, 500)}`); + } +} + +export function errorOf(body: unknown): string | undefined { + if (typeof body !== "object" || body === null) return `not an object: ${JSON.stringify(body)}`; + const e = (body as { error?: unknown }).error; + return e === undefined ? undefined : String(e); +} + +export async function getMeta(s: CubeStack, timeoutMs = REQUEST_TIMEOUT_MS): Promise { + const { body } = await getJson(`${s.apiBase}/meta`, timeoutMs); + const error = errorOf(body); + if (error !== undefined) return { error, cubes: [] }; + const cubes = (body as { cubes?: unknown }).cubes; + if (!Array.isArray(cubes)) throw new Error(`/meta answered without a cubes list: ${JSON.stringify(body).slice(0, 300)}`); + return { cubes: cubes as MetaCube[] }; +} + +export function queryUrl(s: CubeStack, path: "load" | "sql", query: CubeQuery): string { + return `${s.apiBase}/${path}?query=${encodeURIComponent(JSON.stringify(query))}`; +} + +/** + * `/v1/load`, retried every second while Cube answers `Continue wait` (a rollup is building), and + * with backoff after a transport error (at most LOAD_TRANSPORT_RETRIES in a row), all within + * LOAD_DEADLINE_MS. + */ +export async function load(s: CubeStack, query: CubeQuery): Promise { + const deadline = Date.now() + LOAD_DEADLINE_MS; + let transportErrors = 0; + for (;;) { + let answer: { status: number; body: unknown }; + try { + answer = await getJson(queryUrl(s, "load", query)); + transportErrors = 0; + } catch (e) { + if (!(e instanceof TransportError)) throw e; + transportErrors++; + if (transportErrors > LOAD_TRANSPORT_RETRIES || Date.now() > deadline) { + throw new Error(`/v1/load failed ${transportErrors} time(s) in a row, last: ${e.message}\n${s.cubeLogs(40)}`, { cause: e }); + } + await Bun.sleep(Math.min(1000 * 2 ** (transportErrors - 1), LOAD_BACKOFF_CAP_MS)); + continue; + } + const { status, body } = answer; + const error = errorOf(body); + if (error === "Continue wait") { + if (Date.now() > deadline) throw new Error(`Cube kept answering 'Continue wait' for ${LOAD_DEADLINE_MS / 1000}s:\n${s.cubeLogs(40)}`); + await Bun.sleep(1000); + continue; + } + if (error !== undefined) { + throw new Error(`/v1/load answered HTTP ${status}: ${error}\nquery: ${JSON.stringify(query)}\nSQL: ${await cubeSql(s, query)}`); + } + const data = (body as { data?: unknown }).data; + if (!Array.isArray(data)) throw new Error(`/v1/load answered without data: ${JSON.stringify(body).slice(0, 300)}`); + return body as LoadResponse; + } +} + +/** The SQL Cube generates for a query (`/v1/sql`); throws when Cube answers without it. */ +export async function generatedSql(s: CubeStack, query: CubeQuery): Promise { + const { status, body } = await getJson(queryUrl(s, "sql", query)); + const sql = (body as { sql?: { sql?: unknown } }).sql?.sql; + if (!Array.isArray(sql) || typeof sql[0] !== "string") { + throw new Error(`/v1/sql answered HTTP ${status} without SQL: ${JSON.stringify(body).slice(0, 2000)}`); + } + return sql[0]; +} + +/** The SQL Cube generates for a query, for a failure message. */ +export async function cubeSql(s: CubeStack, query: CubeQuery): Promise { + try { + return await generatedSql(s, query); + } catch (e) { + return `(no SQL: ${e instanceof Error ? e.message : String(e)})`; + } +} + diff --git a/server/typescript/packages/integration-tests/cube-live/cube-model-mysql.live.ts b/server/typescript/packages/integration-tests/cube-live/cube-model-mysql.live.ts new file mode 100644 index 000000000..269f78c26 --- /dev/null +++ b/server/typescript/packages/integration-tests/cube-live/cube-model-mysql.live.ts @@ -0,0 +1,186 @@ +/** + * The cube-model live check on MySQL — the Cube MySQL path executed, not just golden-compared. + * + * cube-model.live.ts runs the canonical model against Postgres. The exporter's MySQL output + * (backtick-quoted identifiers, `UTC_TIMESTAMP(3)`, `DATE_SUB(... WEEKDAY ...)` week buckets, + * `COUNT(DISTINCT a, b)`) was otherwise only compared with goldens and compiled through `/v1/meta`, + * which runs no SQL. This file closes that: it starts the same pinned Cube over a private MySQL 8.4 + * holding `fixtures/persistence-conformance/canonical/schema.mysql.sql` (the hand-DDL tables and + * the `buildReportViews` mysql views) plus `fixtures/cube-model/canonical/seed.mysql.sql`, and: + * + * 1. generates the canonical model with `cubeModel()` for the mysql dialect and requires Cube to + * compile it and to list every served report's cube; + * 2. for each served report, builds the Cube query Table F says reproduces it, runs it on MySQL + * through Cube, and compares the rows with `SELECT * FROM ` under the same + * normalization the Postgres check uses. + * + * Cube answers each report from the same rollup it does on Postgres (Table I's column), so the + * rollup build and read SQL Cube writes for MySQL runs too, not only the source-table SQL. + * + * `*.live.ts`: runs only by its path, from `bun run test:cube`. Without docker every test is + * SKIPPED behind a banner. + */ + +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { mkdtempSync, readFileSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { dirname, join, resolve } from "node:path"; +import { fileURLToPath } from "node:url"; +import { cubeModel, defineConfig, runGen, servedReport } from "@metaobjectsdev/codegen-ts"; +import { reportReadSource, type MetaObject, type MetaRoot } from "@metaobjectsdev/metadata"; +import { + CUBE_IMAGE, + CUBE_MYSQL_IMAGE, + cubeStackStartBudgetMs, + dockerUnavailableReason, + startCubeStack, + type CubeStack, +} from "../src/cube-container.ts"; +import { loadMetadataFile } from "../src/load-metadata.ts"; +import { CANONICAL_DIR } from "../src/paths.ts"; +import { readCanonicalMysqlSchemaSql } from "../src/canonical-schema-mysql.ts"; +import { cubeQueryFor, fromCubeRow, fromViewRow, getMeta, load, sortRows } from "./cube-live-support.ts"; + +const REPO_ROOT = resolve(dirname(fileURLToPath(import.meta.url)), "..", "..", "..", "..", ".."); +const CANONICAL_MODEL = join(CANONICAL_DIR, "meta.fitness.json"); +const CANONICAL_SEED = join(REPO_ROOT, "fixtures", "cube-model", "canonical", "seed.mysql.sql"); +const GEN_CONFIG = { dialect: "mysql", columnNamingStrategy: "literal" } as const; +const TEST_TIMEOUT_MS = 600_000; + +/** + * Each served canonical report, its view, the rollup Cube answers from (the same table Table I + * names on Postgres; undefined = the source tables) and its row count on seed.mysql.sql. + */ +const REPORTS: readonly { readonly report: string; readonly view: string; readonly preAggregation: string | undefined; readonly rows: number }[] = [ + { report: "ProgramMinutes", view: "v_program_minutes", preAggregation: "dev_pre_aggregations.week__program_minutes", rows: 3 }, + { report: "FitnessTotals", view: "v_fitness_totals", preAggregation: "dev_pre_aggregations.week__fitness_totals", rows: 1 }, + { report: "ProgramsByMonth", view: "v_programs_by_month", preAggregation: "dev_pre_aggregations.program__programs_by_month", rows: 5 }, + { report: "ProgramsByWeek", view: "v_programs_by_week", preAggregation: "dev_pre_aggregations.program__programs_by_week", rows: 5 }, + { report: "RecentPrograms", view: "v_recent_programs", preAggregation: undefined, rows: 1 }, + { report: "AssetActivity", view: "v_asset_activity", preAggregation: "dev_pre_aggregations.asset__asset_activity", rows: 2 }, + { report: "ProgramRoster", view: "v_program_roster", preAggregation: undefined, rows: 7 }, + { report: "ProgramLongWeeks", view: "v_program_long_weeks", preAggregation: undefined, rows: 7 }, + { report: "FitnessTotalsFilled", view: "v_fitness_totals_filled", preAggregation: "dev_pre_aggregations.week__fitness_totals_filled", rows: 1 }, +]; + +const unavailable = dockerUnavailableReason(); +if (unavailable !== undefined) { + const bar = "!".repeat(78); + console.warn( + `\n${bar}\n!! CUBE MYSQL LIVE CHECK SKIPPED: docker unavailable (${unavailable})\n` + + `!! Nothing was checked against ${CUBE_IMAGE} on ${CUBE_MYSQL_IMAGE}. Every test in this file reports SKIP.\n${bar}\n`, + ); +} +const skip = unavailable !== undefined; + +let work = ""; +let stack: CubeStack | undefined; +let canonical: MetaRoot | undefined; +const timings: string[] = []; + +beforeAll(async () => { + if (skip) return; + work = mkdtempSync(join(tmpdir(), "mo-cube-mysql-live-")); + process.on("exit", cleanUp); + try { + canonical = await loadMetadataFile(CANONICAL_MODEL); + const outDir = join(work, "out"); + await runGen({ + config: defineConfig({ outDir, ...GEN_CONFIG, generators: [cubeModel()] }), + metadata: canonical, + genStateDir: join(work, ".gen-state"), + }); + const t0 = Date.now(); + const started = await startCubeStack(join(outDir, "model"), { + dataSource: "mysql", + initSql: [readCanonicalMysqlSchemaSql(), readFileSync(CANONICAL_SEED, "utf8")], + }); + if (started.kind === "skipped") throw new Error(`docker answered a moment ago and no longer does: ${started.reason}`); + stack = started.stack; + timings.push(`stack up (MySQL + schema + seed, Cube /meta): ${Date.now() - t0} ms`); + } catch (e) { + cleanUp(); + throw e; + } +}, cubeStackStartBudgetMs() + 120_000); + +afterAll(() => { + cleanUp(); + process.off("exit", cleanUp); + if (timings.length > 0) console.log(`cube mysql live check timings:\n ${timings.join("\n ")}`); +}, 180_000); + +function cleanUp(): void { + stack?.stop(); + stack = undefined; + if (work !== "") rmSync(work, { recursive: true, force: true }); +} + +function requireStack(): CubeStack { + if (stack === undefined) throw new Error("the cube stack did not start (see beforeAll)"); + return stack; +} + +function requireCanonical(): MetaRoot { + if (canonical === undefined) throw new Error("the canonical model did not load (see beforeAll)"); + return canonical; +} + +function findReport(name: string): MetaObject { + const report = requireCanonical().objects().find((o) => servedReport(o) && o.name === name); + if (report === undefined) throw new Error(`the canonical model serves no report '${name}'`); + return report; +} + +/** + * `SELECT * FROM ` through the mysql client, every value as the text MySQL prints for it. A + * DATETIME prints `2026-05-04 03:30:00.000`; the space becomes `T` so the shared normalization + * reads it as a UTC instant, as it does Cube's `2026-05-04T03:30:00.000`. + */ +function selectView(s: CubeStack, view: string): Array> { + const columns = s.mysql(`SHOW COLUMNS FROM \`${view}\`;`).trim().split("\n").map((l) => l.split("\t")[0] ?? ""); + const out = s.mysql(`SET time_zone = '+00:00'; SELECT ${columns.map((c) => `\`${c}\``).join(", ")} FROM \`${view}\`;`); + return out + .split("\n") + .filter((l) => l !== "") + .map((line) => { + const cells = line.split("\t"); + const row: Record = {}; + columns.forEach((c, i) => { + const v = cells[i] ?? "NULL"; + row[c] = v === "NULL" ? null : /^\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2}/.test(v) ? v.replace(" ", "T") : v; + }); + return row; + }); +} + +describe(`cube-model MySQL live check (${CUBE_IMAGE} over ${CUBE_MYSQL_IMAGE}, development mode)`, () => { + test.skipIf(skip)("Table coverage: the reports below are exactly the canonical model's served reports", () => { + const served = requireCanonical().objects().filter(servedReport).map((r) => r.name).sort(); + expect(served).toEqual(REPORTS.map((e) => e.report).sort()); + for (const want of REPORTS) expect(reportReadSource(findReport(want.report))?.physicalName).toBe(want.view); + }); + + test.skipIf(skip)("Cube compiles the MySQL model and lists the cubes over the three tables the reports read", async () => { + const meta = await getMeta(requireStack()); + expect(meta.error).toBeUndefined(); + const names = new Set(meta.cubes.map((c) => c.name)); + for (const cube of ["Program", "Week", "Asset"]) expect(names.has(cube)).toBe(true); + }, TEST_TIMEOUT_MS); + + for (const want of REPORTS) { + test.skipIf(skip)(`${want.report}: the Table F query runs on MySQL and equals ${want.view}`, async () => { + const s = requireStack(); + const plan = cubeQueryFor(findReport(want.report), requireCanonical()); + const t0 = Date.now(); + const result = await load(s, plan.query); + const used = Object.keys(result.usedPreAggregations ?? {}).sort(); + const cubeRows = sortRows(result.data.map((row) => fromCubeRow(row, plan)), plan); + const viewRows = sortRows(selectView(s, want.view).map((row) => fromViewRow(row, plan, want.view)), plan); + timings.push(`${want.report}: ${cubeRows.length} row(s) compared, served from ${used[0] ?? "the source table"}, load ${Date.now() - t0} ms`); + expect(viewRows.length).toBe(want.rows); + expect(cubeRows).toEqual(viewRows); + expect(used).toEqual(want.preAggregation === undefined ? [] : [want.preAggregation]); + }, TEST_TIMEOUT_MS); + } +}); diff --git a/server/typescript/packages/integration-tests/cube-live/cube-model.live.ts b/server/typescript/packages/integration-tests/cube-live/cube-model.live.ts index a67fa6dbc..791acbd81 100644 --- a/server/typescript/packages/integration-tests/cube-live/cube-model.live.ts +++ b/server/typescript/packages/integration-tests/cube-live/cube-model.live.ts @@ -60,6 +60,26 @@ import { type CubeStack, } from "../src/cube-container.ts"; import { loadMetadataFile } from "../src/load-metadata.ts"; +import { + type CubeQuery, + type LoadResponse, + type MetaCube, + type MetaMember, + type MetaResponse, + type QueryPlan, + REQUEST_TIMEOUT_MS, + TransportError, + cubeQueryFor, + cubeSql, + generatedSql, + fromCubeRow, + fromViewRow, + getJson, + getMeta, + load, + normalize, + sortRows, +} from "./cube-live-support.ts"; import { CANONICAL_DIR } from "../src/paths.ts"; // cube-live → integration-tests → packages → typescript → server → repo root @@ -75,18 +95,6 @@ const CANONICAL_SEED = join(CUBE_CANONICAL_DIR, "seed.sql"); const GEN_CONFIG = { dialect: "postgres", columnNamingStrategy: "literal" } as const; const TEST_TIMEOUT_MS = 600_000; -/** One HTTP request to Cube; a stalled socket is aborted and counts as a transport error. */ -const REQUEST_TIMEOUT_MS = 60_000; -/** - * How long one `/v1/load` may keep retrying (`Continue wait` while a rollup builds, or a - * transport error). The worst case stays inside TEST_TIMEOUT_MS: this deadline, one request - * timeout and the last backoff, then a `/v1/sql` request and the view read. - */ -const LOAD_DEADLINE_MS = 300_000; -/** Consecutive transport errors (refused, reset, aborted, a body that is not JSON) a load survives. */ -const LOAD_TRANSPORT_RETRIES = 6; -/** Backoff after a transport error: 1 s, doubling, capped here. */ -const LOAD_BACKOFF_CAP_MS = 10_000; /** Each corpus case's share of the corpus pass's budget: its empty-model wait plus its load. */ const CASE_BUDGET_MS = 60_000; @@ -397,144 +405,12 @@ const ESCAPING_LITERALS: readonly { readonly segment: string; readonly literal: { segment: "lineBreak", literal: "'two\nlines'" }, ]; -// --------------------------------------------------------------------------------------------- -// Table F: the Cube query that reproduces a report. -// --------------------------------------------------------------------------------------------- - -interface CubeTimeDimension { - readonly dimension: string; - readonly granularity: string; -} - -interface CubeQuery { - readonly measures: readonly string[]; - readonly dimensions: readonly string[]; - readonly timeDimensions: readonly CubeTimeDimension[]; - readonly segments: readonly string[]; - /** Table I fixes the query time zone at UTC (Table J: another zone re-buckets). */ - readonly timezone: "UTC"; -} - -type ColumnKind = "attribute" | "time" | "hour" | "measure"; - -/** One report field: its view column and the Cube row key that carries it. */ -interface PlannedColumn { - readonly field: string; - readonly cubeKey: string; - readonly kind: ColumnKind; -} - -interface QueryPlan { - readonly query: CubeQuery; - readonly columns: readonly PlannedColumn[]; -} - function findReport(name: string): MetaObject { const report = requireCanonical().objects().find((o) => servedReport(o) && o.name === name); if (report === undefined) throw new Error(`the canonical model serves no report '${name}'`); return report; } -/** Lower camel case of a report name: `RecentPrograms` → `recentPrograms` (Table F's scope segment). */ -function lowerFirst(s: string): string { - return s.charAt(0).toLowerCase() + s.slice(1); -} - -function cubeQueryFor(report: MetaObject, root: MetaRoot): QueryPlan { - const shape = reportShape(report, root); - // A @spine report is a Cube view of its own name, whose scope is inside its facts cube. - const spine = reportSpine(report) !== undefined; - const cube = spine ? report.name : shape.from.name; - const measures: string[] = []; - const dimensions: string[] = []; - const timeDimensions: CubeTimeDimension[] = []; - const columns: PlannedColumn[] = []; - for (const f of shape.fields) { - if (f.role === "measure") { - if (f.measure === undefined) throw new Error(`${report.name}.${f.name}: a measure field without its measure`); - const member = `${cube}.${f.measure.name}`; - measures.push(member); - columns.push({ field: f.name, cubeKey: member, kind: "measure" }); - } else { - if (f.dimension === undefined) throw new Error(`${report.name}.${f.name}: a dimension field without its dimension`); - const member = `${cube}.${f.dimension.name}`; - if (f.grain === undefined) { - dimensions.push(member); - columns.push({ field: f.name, cubeKey: member, kind: "attribute" }); - } else { - timeDimensions.push({ dimension: member, granularity: f.grain }); - columns.push({ field: f.name, cubeKey: `${member}.${f.grain}`, kind: f.grain === GRAIN_HOUR ? "hour" : "time" }); - } - } - } - const segments: string[] = []; - const segment = report.attr(OBJECT_REPORT_ATTR_SEGMENT); - if (!spine && typeof segment === "string") segments.push(`${cube}.${segment}`); - if (!spine && report.attr(OBJECT_REPORT_ATTR_FILTER) !== undefined) segments.push(`${cube}.${lowerFirst(report.name)}Scope`); - return { query: { measures, dimensions, timeDimensions, segments, timezone: "UTC" }, columns }; -} - -// --------------------------------------------------------------------------------------------- -// Table I: normalization and comparison. -// --------------------------------------------------------------------------------------------- - -type NormalRow = Record; - -const DECIMAL_RE = /^-?\d+(\.\d+)?$/; - -/** `60.0000000000000000` → `60`, `0.75000000000000000000` → `0.75`; an integer string is kept. */ -function canonicalDecimal(s: string): string { - return s.includes(".") ? s.replace(/\.?0+$/, "") : s; -} - -/** True when the string already names its offset (`Z`, `+00:00`, `-0500`). */ -const HAS_ZONE_RE = /(Z|[+-]\d{2}(:?\d{2})?)$/; - -function normalize(kind: ColumnKind, raw: unknown, where: string): string | null { - if (raw === null || raw === undefined) return null; - if (typeof raw !== "string" && typeof raw !== "number") throw new Error(`${where}: unexpected value ${JSON.stringify(raw)}`); - const s = String(raw); - switch (kind) { - case "time": - // A day, week, month, quarter or year bucket: Cube sends `2026-05-01T00:00:00.000`, the view a DATE. - return s.slice(0, 10); - case "hour": { - // An instant. Cube sends the wall clock in the query time zone (UTC, fixed by the query); - // the view sends a timestamptz with its offset. - const d = new Date(HAS_ZONE_RE.test(s) ? s : `${s}Z`); - if (Number.isNaN(d.getTime())) throw new Error(`${where}: '${s}' is not a timestamp`); - return d.toISOString(); - } - case "attribute": - case "measure": - return DECIMAL_RE.test(s) ? canonicalDecimal(s) : s; - } -} - -function fromCubeRow(row: Record, plan: QueryPlan): NormalRow { - const out: NormalRow = {}; - for (const c of plan.columns) { - if (!(c.cubeKey in row)) throw new Error(`Cube's row has no '${c.cubeKey}': ${JSON.stringify(row)}`); - out[c.field] = normalize(c.kind, row[c.cubeKey], `Cube ${c.cubeKey}`); - } - return out; -} - -function fromViewRow(row: Record, plan: QueryPlan, view: string): NormalRow { - const want = plan.columns.map((c) => c.field).sort(); - expect(Object.keys(row).sort()).toEqual(want); - const out: NormalRow = {}; - for (const c of plan.columns) out[c.field] = normalize(c.kind, row[c.field], `${view}.${c.field}`); - return out; -} - -/** Rows as a set, ordered by their dimension values (then by the whole row, for determinism). */ -function sortRows(rows: NormalRow[], plan: QueryPlan): NormalRow[] { - const dims = plan.columns.filter((c) => c.kind !== "measure").map((c) => c.field); - const key = (r: NormalRow): string => JSON.stringify([dims.map((d) => r[d] ?? null), plan.columns.map((c) => r[c.field] ?? null)]); - return rows.slice().sort((a, b) => (key(a) < key(b) ? -1 : key(a) > key(b) ? 1 : 0)); -} - /** * `SELECT * FROM ` through psql, every value as the TEXT Postgres prints for it (a numeric * keeps its scale, a timestamptz its offset), in a UTC session. @@ -553,135 +429,6 @@ function selectView(s: CubeStack, view: string): Array>; } -// --------------------------------------------------------------------------------------------- -// The Cube REST API. -// --------------------------------------------------------------------------------------------- - -interface MetaMember { - readonly name: string; - readonly type?: string; - readonly aggType?: string; - readonly public?: boolean; - /** A member's own declared title (`title` is the cube's title and this one, joined). */ - readonly shortTitle?: string; - readonly description?: string; -} - -interface MetaCube { - readonly name: string; - readonly public?: boolean; - readonly title?: string; - readonly description?: string; - readonly measures?: readonly MetaMember[]; - readonly dimensions?: readonly MetaMember[]; - readonly segments?: readonly MetaMember[]; -} - -interface MetaResponse { - readonly error?: string; - readonly cubes: readonly MetaCube[]; -} - -interface LoadResponse { - readonly data: Array>; - readonly usedPreAggregations?: Record; -} - -/** A request that never produced a JSON answer: refused, reset, aborted, or a body that is not JSON. */ -class TransportError extends Error {} - -async function getJson(url: string, timeoutMs = REQUEST_TIMEOUT_MS): Promise<{ status: number; body: unknown }> { - let res: Response; - let text: string; - try { - res = await fetch(url, { signal: AbortSignal.timeout(Math.max(1000, timeoutMs)) }); - text = await res.text(); - } catch (e) { - throw new TransportError(`${url.split("?")[0]}: ${e instanceof Error ? `${e.name}: ${e.message}` : String(e)}`, { cause: e }); - } - try { - return { status: res.status, body: JSON.parse(text) }; - } catch { - throw new TransportError(`${url.split("?")[0]}: HTTP ${res.status}, not JSON: ${text.slice(0, 500)}`); - } -} - -function errorOf(body: unknown): string | undefined { - if (typeof body !== "object" || body === null) return `not an object: ${JSON.stringify(body)}`; - const e = (body as { error?: unknown }).error; - return e === undefined ? undefined : String(e); -} - -async function getMeta(s: CubeStack, timeoutMs = REQUEST_TIMEOUT_MS): Promise { - const { body } = await getJson(`${s.apiBase}/meta`, timeoutMs); - const error = errorOf(body); - if (error !== undefined) return { error, cubes: [] }; - const cubes = (body as { cubes?: unknown }).cubes; - if (!Array.isArray(cubes)) throw new Error(`/meta answered without a cubes list: ${JSON.stringify(body).slice(0, 300)}`); - return { cubes: cubes as MetaCube[] }; -} - -function queryUrl(s: CubeStack, path: "load" | "sql", query: CubeQuery): string { - return `${s.apiBase}/${path}?query=${encodeURIComponent(JSON.stringify(query))}`; -} - -/** - * `/v1/load`, retried every second while Cube answers `Continue wait` (a rollup is building), and - * with backoff after a transport error (at most LOAD_TRANSPORT_RETRIES in a row), all within - * LOAD_DEADLINE_MS. - */ -async function load(s: CubeStack, query: CubeQuery): Promise { - const deadline = Date.now() + LOAD_DEADLINE_MS; - let transportErrors = 0; - for (;;) { - let answer: { status: number; body: unknown }; - try { - answer = await getJson(queryUrl(s, "load", query)); - transportErrors = 0; - } catch (e) { - if (!(e instanceof TransportError)) throw e; - transportErrors++; - if (transportErrors > LOAD_TRANSPORT_RETRIES || Date.now() > deadline) { - throw new Error(`/v1/load failed ${transportErrors} time(s) in a row, last: ${e.message}\n${s.cubeLogs(40)}`, { cause: e }); - } - await Bun.sleep(Math.min(1000 * 2 ** (transportErrors - 1), LOAD_BACKOFF_CAP_MS)); - continue; - } - const { status, body } = answer; - const error = errorOf(body); - if (error === "Continue wait") { - if (Date.now() > deadline) throw new Error(`Cube kept answering 'Continue wait' for ${LOAD_DEADLINE_MS / 1000}s:\n${s.cubeLogs(40)}`); - await Bun.sleep(1000); - continue; - } - if (error !== undefined) { - throw new Error(`/v1/load answered HTTP ${status}: ${error}\nquery: ${JSON.stringify(query)}\nSQL: ${await cubeSql(s, query)}`); - } - const data = (body as { data?: unknown }).data; - if (!Array.isArray(data)) throw new Error(`/v1/load answered without data: ${JSON.stringify(body).slice(0, 300)}`); - return body as LoadResponse; - } -} - -/** The SQL Cube generates for a query (`/v1/sql`); throws when Cube answers without it. */ -async function generatedSql(s: CubeStack, query: CubeQuery): Promise { - const { status, body } = await getJson(queryUrl(s, "sql", query)); - const sql = (body as { sql?: { sql?: unknown } }).sql?.sql; - if (!Array.isArray(sql) || typeof sql[0] !== "string") { - throw new Error(`/v1/sql answered HTTP ${status} without SQL: ${JSON.stringify(body).slice(0, 2000)}`); - } - return sql[0]; -} - -/** The SQL Cube generates for a query, for a failure message. */ -async function cubeSql(s: CubeStack, query: CubeQuery): Promise { - try { - return await generatedSql(s, query); - } catch (e) { - return `(no SQL: ${e instanceof Error ? e.message : String(e)})`; - } -} - // --------------------------------------------------------------------------------------------- // What the files declare, and what Cube lists. // --------------------------------------------------------------------------------------------- diff --git a/server/typescript/packages/integration-tests/package.json b/server/typescript/packages/integration-tests/package.json index bc1e205c1..c70762423 100644 --- a/server/typescript/packages/integration-tests/package.json +++ b/server/typescript/packages/integration-tests/package.json @@ -6,7 +6,7 @@ "type": "module", "scripts": { "test": "bun test", - "test:cube": "bun test ./cube-live/cube-model.live.ts", + "test:cube": "bun test ./cube-live/cube-model.live.ts ./cube-live/cube-model-mysql.live.ts", "gen:schema": "bun run src/gen-canonical-schema.ts", "gen:report-api-schema": "bun run src/gen-api-contract-report-schema.ts", "gen:report-shapes": "bun run src/gen-report-shapes.ts", @@ -35,6 +35,7 @@ "@types/node": "^22.5.4", "@types/pg": "^8.11.6", "bun-types": "^1.1.27", + "miniflare": "4.20251210.0", "mysql2": "^3.11.0", "typescript": "^5.6.2" } diff --git a/server/typescript/packages/integration-tests/src/api-contract-report-generated-server.ts b/server/typescript/packages/integration-tests/src/api-contract-report-generated-server.ts index 112cb6a7b..bf2e66e02 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-report-generated-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-report-generated-server.ts @@ -85,6 +85,13 @@ export function seedInserts(root: MetaRoot, seed: ReportSeed): SeedInsert[] { return out; } +/** paidShare compares numerically (a decimal's spelling is the engine's own); every other key strictly. */ +export function rowsEqual(a: Record, b: Record): boolean { + const keys = Object.keys(a).sort(); + if (keys.join(",") !== Object.keys(b).sort().join(",")) return false; + return keys.every((k) => (k === "paidShare" ? Number(a[k]) === Number(b[k]) : a[k] === b[k])); +} + /** * The base tables, parents first, in the EMITTED snake_case spelling. The routes read the * views and the views read these, so they are provisioned by hand rather than generated. @@ -126,8 +133,9 @@ export interface GeneratedReportServerHandle { close(): Promise; } -/** The served reports: emitted registrar, and the route the corpus calls. */ -const SERVED_REPORTS = [ +/** The served reports: emitted registrar, and the route the corpus calls. One list for every + * engine's generated-report server — adding a report must reach all lanes at once. */ +export const SERVED_REPORTS = [ { name: "InvoiceStatusTotals", registrar: "invoiceStatusTotalsRoutes" }, { name: "InvoicesByMonth", registrar: "invoicesByMonthRoutes" }, { name: "InvoiceTotals", registrar: "invoiceTotalsRoutes" }, diff --git a/server/typescript/packages/integration-tests/src/api-contract-report-mysql-schema.ts b/server/typescript/packages/integration-tests/src/api-contract-report-mysql-schema.ts new file mode 100644 index 000000000..4d47839d0 --- /dev/null +++ b/server/typescript/packages/integration-tests/src/api-contract-report-mysql-schema.ts @@ -0,0 +1,47 @@ +// api-contract-report-mysql-schema.ts — the MySQL counterpart of the report sub-corpus schema +// artifact (report/schema.mysql.sql). +// +// `meta migrate` does not own a MySQL schema (ADR-0015), so the TABLES are the adopter's +// hand-written DDL, in the LITERAL column spelling the other committed artifacts use +// (amountCents, issuedOn) — the spelling the C# generated lane maps. The VIEWS are not +// hand-written: each is `buildReportViews(root, { dialect: "mysql" })`, the one producer of +// report SQL, wrapped in the recipe's `CREATE VIEW`. Drift-checked by +// test/api-contract-report-corpus.test.ts. + +import type { MetaRoot } from "@metaobjectsdev/metadata"; + +import { renderMysqlSchemaArtifact } from "./canonical-schema-mysql.ts"; + +const HEADER = [ + "-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT.", + "-- api-contract report sub-corpus schema (MySQL 8.4, literal column naming).", + "-- Tables: the adopter's hand-written DDL (meta migrate does not own MySQL, ADR-0015).", + "-- Views: buildReportViews(root, { dialect: \"mysql\" }), produced by TypeScript.", + "-- Regenerate: `bun run gen:report-api-schema` in server/typescript/packages/integration-tests.", +].join("\n"); + +/** The adopter's tables, parents first. */ +const TABLES = [ + `CREATE TABLE \`invoices\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`reference\` VARCHAR(40) NOT NULL, + \`status\` VARCHAR(20) NOT NULL, + \`amountCents\` BIGINT NOT NULL, + \`issuedOn\` DATE NOT NULL +)`, + `CREATE TABLE \`products\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`name\` VARCHAR(200) NOT NULL +)`, + `CREATE TABLE \`sales\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`productId\` BIGINT NOT NULL, + \`amountCents\` BIGINT NOT NULL, + FOREIGN KEY (\`productId\`) REFERENCES \`products\` (\`id\`) +)`, +]; + +/** The bytes of report/schema.mysql.sql for a loaded report/meta.json. */ +export function generateReportApiMysqlSchemaSql(root: MetaRoot): string { + return renderMysqlSchemaArtifact(HEADER, TABLES, root); +} diff --git a/server/typescript/packages/integration-tests/src/api-contract-report-mysql-server.ts b/server/typescript/packages/integration-tests/src/api-contract-report-mysql-server.ts new file mode 100644 index 000000000..e288fedef --- /dev/null +++ b/server/typescript/packages/integration-tests/src/api-contract-report-mysql-server.ts @@ -0,0 +1,140 @@ +// api-contract-report-mysql-server.ts — boots the GENERATED report routes on a real MySQL 8.4 +// and drives them over HTTP. The Postgres twin is api-contract-report-generated-server.ts, the +// SQLite one api-contract-report-sqlite-server.ts. +// +// Same shape: run the real codegen over report/meta.json (dialect mysql), provision the base +// tables by hand in the EMITTED snake_case spelling (MetaObjects does not own a MySQL schema, +// ADR-0015), create each view from buildReportViews (the real lowering, `dialect: "mysql"`), +// import the EMITTED .routes.ts files unmodified and mount them. + +import Fastify, { type FastifyInstance } from "fastify"; +import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; +import { dirname, join } from "node:path"; +import { fileURLToPath, pathToFileURL } from "node:url"; +import { runGen, defineConfig, buildReportViews } from "@metaobjectsdev/codegen-ts"; +import { DEFAULT_COLUMN_NAMING_STRATEGY } from "@metaobjectsdev/metadata"; +import { entityFile, routesFile } from "@metaobjectsdev/test-generators"; +import mysql from "mysql2/promise"; +import { seedInserts, SERVED_REPORTS, type ReportSeed } from "./api-contract-report-generated-server.ts"; +import { loadMetadataFile } from "./load-metadata.ts"; + +export interface MysqlReportServerHandle { + baseUrl: string; + applySeed(seed: ReportSeed): Promise; + close(): Promise; +} + +/** The base tables, parents first, in the EMITTED snake_case spelling. */ +const BASE_TABLES: ReadonlyArray<{ name: string; ddl: string }> = [ + { + name: "invoices", + ddl: `CREATE TABLE \`invoices\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`reference\` VARCHAR(40) NOT NULL, + \`status\` VARCHAR(20) NOT NULL, + \`amount_cents\` BIGINT NOT NULL, + \`issued_on\` DATE NOT NULL + )`, + }, + { + name: "products", + ddl: `CREATE TABLE \`products\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`name\` VARCHAR(200) NOT NULL + )`, + }, + { + name: "sales", + ddl: `CREATE TABLE \`sales\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`product_id\` BIGINT NOT NULL, + \`amount_cents\` BIGINT NOT NULL, + FOREIGN KEY (\`product_id\`) REFERENCES \`products\` (\`id\`) + )`, + }, +]; + +export async function startMysqlReportServer(mysqlUrl: string, metaPath: string): Promise { + const here = dirname(fileURLToPath(import.meta.url)); + const genTmpRoot = join(here, "..", ".gen-tmp"); + mkdirSync(genTmpRoot, { recursive: true }); + const tmp = mkdtempSync(join(genTmpRoot, "api-contract-report-mysql-")); + + const root = await loadMetadataFile(metaPath); + const lr = await runGen({ + config: defineConfig({ + outDir: tmp, + extStyle: "none", + dbImport: "./db", + dialect: "mysql", + apiPrefix: "/api", + generators: [entityFile(), routesFile()], + }), + metadata: root, + }); + if (existsSync(join(tmp, "InvoiceDays.routes.ts"))) { + rmSync(tmp, { recursive: true, force: true }); + throw new Error("codegen emitted a routes file for the sourceless report InvoiceDays"); + } + if (lr.warnings.length > 0) { + rmSync(tmp, { recursive: true, force: true }); + throw new Error(`codegen produced warnings: ${lr.warnings.join("; ")}`); + } + + // A BIGINT is a JS number while it fits (supportBigNumbers without bigNumberStrings), the + // spelling the Postgres lane gets from its bigint-as-number type parser; a DECIMAL stays text. + const dbModule = ` +import { drizzle } from "drizzle-orm/mysql2"; +import { createPool } from "mysql2"; +export const pool = createPool({ uri: ${JSON.stringify(mysqlUrl)}, timezone: "Z", supportBigNumbers: true }); +export const db = drizzle(pool); +`; + writeFileSync(join(tmp, "db.ts"), dbModule, "utf8"); + const dbMod = (await import(pathToFileURL(join(tmp, "db.ts")).href)) as { pool: { end(cb: () => void): void } }; + + const admin = await mysql.createConnection({ uri: mysqlUrl, timezone: "Z" }); + const views = buildReportViews(root, { dialect: "mysql", columnNamingStrategy: DEFAULT_COLUMN_NAMING_STRATEGY }); + for (const v of views) await admin.query(`DROP VIEW IF EXISTS \`${v.name}\``); + for (const t of [...BASE_TABLES].reverse()) await admin.query(`DROP TABLE IF EXISTS \`${t.name}\``); + for (const t of BASE_TABLES) await admin.query(t.ddl); + for (const v of views) { + if (v.sql === undefined) throw new Error(`view ${v.name} has no SQL`); + await admin.query(`CREATE VIEW \`${v.name}\` AS ${v.sql}`); + } + + const fastify = Fastify(); + for (const r of SERVED_REPORTS) { + const mod = (await import(pathToFileURL(join(tmp, `${r.name}.routes.ts`)).href)) as Record< + string, + (f: FastifyInstance) => Promise + >; + const registrar = mod[r.registrar]; + if (registrar === undefined) throw new Error(`${r.name}.routes.ts does not export ${r.registrar}`); + await fastify.register(registrar); + } + await fastify.ready(); + const baseUrl = await fastify.listen({ host: "127.0.0.1", port: 0 }); + + return { + baseUrl, + applySeed: async (seed: ReportSeed) => { + // Children before parents on the way out; the seed's file order (parents first) on the way in. + for (const t of [...BASE_TABLES].reverse()) await admin.query(`DELETE FROM \`${t.name}\``); + for (const r of seedInserts(root, seed)) { + await admin.query( + `INSERT INTO \`${r.table}\` (${r.columns.map((c) => `\`${c}\``).join(",")}) ` + + `VALUES (${r.columns.map(() => "?").join(",")})`, + r.values, + ); + } + }, + close: async () => { + await fastify.close(); + await new Promise((done) => dbMod.pool.end(done)); + for (const v of views) await admin.query(`DROP VIEW IF EXISTS \`${v.name}\``); + for (const t of [...BASE_TABLES].reverse()) await admin.query(`DROP TABLE IF EXISTS \`${t.name}\``); + await admin.end(); + rmSync(tmp, { recursive: true, force: true }); + }, + }; +} diff --git a/server/typescript/packages/integration-tests/src/api-contract-report-schema.ts b/server/typescript/packages/integration-tests/src/api-contract-report-schema.ts index e7de495e1..2bc0b7af2 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-report-schema.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-report-schema.ts @@ -6,14 +6,20 @@ import type { MetaRoot } from "@metaobjectsdev/metadata"; import { generateCanonicalSchemaSql } from "./canonical-schema.ts"; -export const REPORT_API_SCHEMA_HEADER = [ - "-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT.", - "-- api-contract report sub-corpus schema (Postgres, literal column naming).", - "-- Produced by TypeScript from report/meta.json; the C# generated lane executes it.", - "-- Regenerate: `bun run gen:report-api-schema` in this package.", -].join("\n"); +/** The dialects the report sub-corpus commits a schema artifact for (MySQL is the adopter's own DDL, ADR-0015). */ +export type ReportApiSchemaDialect = "postgres" | "sqlite"; -/** The bytes of report/schema.postgres.sql for a loaded report/meta.json. */ -export function generateReportApiSchemaSql(root: MetaRoot): Promise { - return generateCanonicalSchemaSql(root, { header: REPORT_API_SCHEMA_HEADER }); +/** The header of the committed artifact for `dialect`. */ +export function reportApiSchemaHeader(dialect: ReportApiSchemaDialect = "postgres"): string { + return [ + "-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT.", + `-- api-contract report sub-corpus schema (${dialect === "postgres" ? "Postgres" : "SQLite"}, literal column naming).`, + "-- Produced by TypeScript from report/meta.json; the C# generated lane executes it.", + "-- Regenerate: `bun run gen:report-api-schema` in this package.", + ].join("\n"); +} + +/** The bytes of report/schema..sql for a loaded report/meta.json. */ +export function generateReportApiSchemaSql(root: MetaRoot, dialect: ReportApiSchemaDialect = "postgres"): Promise { + return generateCanonicalSchemaSql(root, { header: reportApiSchemaHeader(dialect), dialect }); } diff --git a/server/typescript/packages/integration-tests/src/api-contract-report-sqlite-server.ts b/server/typescript/packages/integration-tests/src/api-contract-report-sqlite-server.ts index 90faa5a24..986380151 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-report-sqlite-server.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-report-sqlite-server.ts @@ -1,5 +1,6 @@ // api-contract-report-sqlite-server.ts — boots the GENERATED report routes on SQLite -// (bun:sqlite) and drives them over HTTP. The Postgres twin is +// (bun:sqlite), or on Cloudflare D1's local runtime (Miniflare, `engine: "d1"`), and drives +// them over HTTP. The Postgres twin is // api-contract-report-generated-server.ts; this one exists because the TypeScript view // read schema types a ratio (a decimal) as a string, SQLite has no decimal and returns a // REAL, and the corpus runs TypeScript on Postgres only. What the route answers for a @@ -11,7 +12,7 @@ // EMITTED .routes.ts files unmodified and mount them. import Fastify, { type FastifyInstance } from "fastify"; -import type { Database } from "bun:sqlite"; +import { randomUUID } from "node:crypto"; import { existsSync, mkdirSync, mkdtempSync, rmSync, writeFileSync } from "node:fs"; import { dirname, join } from "node:path"; import { fileURLToPath, pathToFileURL } from "node:url"; @@ -19,7 +20,17 @@ import { runGen, defineConfig, buildReportViews } from "@metaobjectsdev/codegen- import { DEFAULT_COLUMN_NAMING_STRATEGY } from "@metaobjectsdev/metadata"; import { entityFile, routesFile } from "@metaobjectsdev/test-generators"; import { loadMetadataFile } from "./load-metadata.ts"; -import { seedInserts, type ReportSeed } from "./api-contract-report-generated-server.ts"; +import { seedInserts, SERVED_REPORTS, type ReportSeed } from "./api-contract-report-generated-server.ts"; +import { startLocalD1 } from "./query-scenario-d1.ts"; + +/** The SQLite-family engines this server runs the emitted routes on. */ +export type SqliteReportEngine = "sqlite" | "d1"; + +/** What the server needs of an engine: run one statement, and close. */ +interface EngineClient { + run(sql: string, params?: unknown[]): Promise; + close(): Promise; +} export interface SqliteReportServerHandle { baseUrl: string; @@ -27,13 +38,6 @@ export interface SqliteReportServerHandle { close(): Promise; } -const SERVED_REPORTS = [ - { name: "InvoiceStatusTotals", registrar: "invoiceStatusTotalsRoutes" }, - { name: "InvoicesByMonth", registrar: "invoicesByMonthRoutes" }, - { name: "InvoiceTotals", registrar: "invoiceTotalsRoutes" }, - { name: "ProductRevenue", registrar: "productRevenueRoutes" }, -] as const; - /** The base tables, parents first, in the EMITTED snake_case spelling. */ const BASE_TABLES: ReadonlyArray<{ name: string; ddl: string }> = [ { @@ -66,7 +70,10 @@ const BASE_TABLES: ReadonlyArray<{ name: string; ddl: string }> = [ }, ]; -export async function startSqliteReportServer(metaPath: string): Promise { +export async function startSqliteReportServer( + metaPath: string, + engine: SqliteReportEngine = "sqlite", +): Promise { const here = dirname(fileURLToPath(import.meta.url)); const genTmpRoot = join(here, "..", ".gen-tmp"); mkdirSync(genTmpRoot, { recursive: true }); @@ -78,7 +85,7 @@ export async function startSqliteReportServer(metaPath: string): Promise { // Children before parents on the way out; the seed's file order (parents first) on the way in. - for (const t of [...BASE_TABLES].reverse()) dbMod.client.run(`DELETE FROM "${t.name}"`); + for (const t of [...BASE_TABLES].reverse()) await client.run(`DELETE FROM "${t.name}"`); for (const r of seedInserts(root, seed)) { - dbMod.client.run( + await client.run( `INSERT INTO "${r.table}" (${r.columns.map((c) => `"${c}"`).join(",")}) ` + `VALUES (${r.columns.map(() => "?").join(",")})`, r.values, @@ -142,8 +140,47 @@ export const db = drizzle(client); }, close: async () => { await fastify.close(); - dbMod.client.close(); + await client.close(); rmSync(tmp, { recursive: true, force: true }); }, }; } + +/** + * Write the `db` module the emitted routes import (`import { db } from "./db"`) and open the + * engine behind it. One database serves the routes and the seed. A REAL read back from SQLite + * is the same JS number on every SQLite driver. D1 runs in Miniflare; its binding is handed to + * the module through `globalThis`, which is how a Worker's env binding reaches drizzle here. + */ +async function openEngine(engine: SqliteReportEngine, tmp: string): Promise { + if (engine === "d1") { + const local = await startLocalD1(); + // One key per server, so two servers in flight never read each other's binding. + const key = `__reportD1_${randomUUID().replaceAll("-", "")}`; + (globalThis as Record)[key] = local.d1; + writeFileSync(join(tmp, "db.ts"), ` +import { drizzle } from "drizzle-orm/d1"; +export const db = drizzle((globalThis as Record)[${JSON.stringify(key)}]); +`, "utf8"); + return { + run: async (sql, params = []) => { + await local.d1.prepare(sql).bind(...params).all(); + }, + close: async () => { + delete (globalThis as Record)[key]; + await local.dispose(); + }, + }; + } + writeFileSync(join(tmp, "db.ts"), ` +import { Database } from "bun:sqlite"; +import { drizzle } from "drizzle-orm/bun-sqlite"; +export const client = new Database(":memory:"); +export const db = drizzle(client); +`, "utf8"); + const mod = (await import(pathToFileURL(join(tmp, "db.ts")).href)) as { client: import("bun:sqlite").Database }; + return { + run: async (sql, params = []) => { mod.client.run(sql, params as never); }, + close: async () => { mod.client.close(); }, + }; +} diff --git a/server/typescript/packages/integration-tests/src/api-contract-scenario.ts b/server/typescript/packages/integration-tests/src/api-contract-scenario.ts index d81009967..cbe87d59d 100644 --- a/server/typescript/packages/integration-tests/src/api-contract-scenario.ts +++ b/server/typescript/packages/integration-tests/src/api-contract-scenario.ts @@ -86,6 +86,28 @@ function parseScenario(file: string): ApiScenario { }; } +/** + * Send every request of a scenario against `baseUrl` and assert each response. Engine-neutral: + * shared by the report lane's variants (SQLite, D1, MySQL) so the request envelope lives once — + * the Postgres lane's own copy predates this and is the cross-port runner's reference. + */ +export async function runScenario(scenario: ApiScenario, baseUrl: string): Promise { + for (const req of scenario.requests) { + const init: RequestInit = { method: req.method }; + if (req.body !== undefined) { + init.body = JSON.stringify(req.body); + init.headers = { "content-type": "application/json" }; + } + const res = await fetch(baseUrl + req.path, init); + const bodyText = await res.text(); + let body: unknown = null; + if (bodyText.length > 0) { + try { body = JSON.parse(bodyText); } catch { body = bodyText; } + } + assertResponse(scenario.name, req, res.status, body); + } +} + // --------------------------------------------------------------------------- // Assertion engine // --------------------------------------------------------------------------- diff --git a/server/typescript/packages/integration-tests/src/canonical-schema-mysql.ts b/server/typescript/packages/integration-tests/src/canonical-schema-mysql.ts new file mode 100644 index 000000000..ff8c96d32 --- /dev/null +++ b/server/typescript/packages/integration-tests/src/canonical-schema-mysql.ts @@ -0,0 +1,84 @@ +// canonical-schema-mysql.ts — the MySQL counterpart of the canonical schema artifact. +// +// `meta migrate` does not own a MySQL schema (ADR-0015; `--dialect mysql` is refused), so the +// TABLES here are the adopter's hand-written DDL, exactly as docs/recipes/mysql.md tells an +// adopter to write them. The VIEWS are not hand-written: each is `buildReportViews(root, +// { dialect: "mysql" })`, the one producer of report SQL, wrapped in the recipe's +// `CREATE VIEW`. Only the tables the report scenarios read are declared (programs, weeks, +// assets) — the rest of the canonical model gates the entity runtime on Postgres. +// +// Committed as fixtures/persistence-conformance/canonical/schema.mysql.sql and drift-checked +// by test/schema-artifact.test.ts, so a port that runs the report scenarios on MySQL executes +// TypeScript's bytes instead of lowering a report itself. + +import { buildReportViews } from "@metaobjectsdev/codegen-ts"; +import type { MetaRoot } from "@metaobjectsdev/metadata"; +import { readFileSync } from "node:fs"; +import { resolve } from "node:path"; + +import { CANONICAL_COLUMN_NAMING } from "./canonical-schema.ts"; +import { CANONICAL_DIR } from "./paths.ts"; + +export const CANONICAL_MYSQL_SCHEMA_SQL_PATH = resolve(CANONICAL_DIR, "schema.mysql.sql"); + +const HEADER = [ + "-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT.", + "-- Canonical persistence-conformance schema (MySQL 8.4, literal column naming).", + "-- Tables: the adopter's hand-written DDL (meta migrate does not own MySQL, ADR-0015).", + "-- Views: buildReportViews(root, { dialect: \"mysql\" }), produced by TypeScript.", + "-- Only the tables the report-* scenarios read are declared.", + "-- Regenerate: `bun run gen:schema` in server/typescript/packages/integration-tests.", +].join("\n"); + +/** The adopter's tables, parents first. */ +const TABLES = [ + `CREATE TABLE \`programs\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`title\` VARCHAR(200) NOT NULL, + \`priceCents\` BIGINT NOT NULL, + \`status\` VARCHAR(9) NOT NULL, + \`created_ts\` DATETIME(3) NOT NULL +)`, + `CREATE TABLE \`weeks\` ( + \`id\` BIGINT NOT NULL AUTO_INCREMENT PRIMARY KEY, + \`programId\` BIGINT NOT NULL, + \`label\` VARCHAR(80), + \`durationMinutes\` INT NOT NULL, + FOREIGN KEY (\`programId\`) REFERENCES \`programs\` (\`id\`) +)`, + `CREATE TABLE \`assets\` ( + \`id\` VARCHAR(36) NOT NULL DEFAULT (UUID()) PRIMARY KEY, + \`ownerId\` VARCHAR(36) NOT NULL, + \`externalId\` VARCHAR(36), + \`payload\` JSON, + \`recordedAt\` DATETIME(3) NOT NULL, + \`observedAt\` DATETIME(3) NOT NULL, + \`asOfDate\` DATE NOT NULL, + \`atTime\` TIME(3) NOT NULL +)`, +]; + +/** + * The one MySQL artifact layout: header, the adopter's tables, then every report view TypeScript + * lowers wrapped in the recipe's CREATE VIEW, statement-per-block. Shared with the api-contract + * report sub-corpus (api-contract-report-mysql-schema.ts) so the two MySQL artifacts cannot + * drift apart in layout while each keeps its own HEADER/TABLES. + */ +export function renderMysqlSchemaArtifact(header: string, tables: readonly string[], root: MetaRoot): string { + const views = buildReportViews(root, { dialect: "mysql", columnNamingStrategy: CANONICAL_COLUMN_NAMING }); + const statements = [ + ...tables, + ...views.map((v) => `CREATE VIEW \`${v.name}\` AS\n${v.sql}`), + ]; + return `${header}\n\n${statements.map((s) => `${s};`).join("\n\n")}\n`; +} + +/** The bytes of schema.mysql.sql for a loaded canonical model. */ +export function generateCanonicalMysqlSchemaSql(root: MetaRoot): string { + return renderMysqlSchemaArtifact(HEADER, TABLES, root); +} + +/** Read the committed MySQL schema artifact. */ +export function readCanonicalMysqlSchemaSql(): string { + return readFileSync(CANONICAL_MYSQL_SCHEMA_SQL_PATH, "utf8"); +} diff --git a/server/typescript/packages/integration-tests/src/canonical-schema.ts b/server/typescript/packages/integration-tests/src/canonical-schema.ts index 147cd1f77..b16baa62b 100644 --- a/server/typescript/packages/integration-tests/src/canonical-schema.ts +++ b/server/typescript/packages/integration-tests/src/canonical-schema.ts @@ -13,7 +13,7 @@ // Postgres only (Decision 1 in the design): one schema.postgres.sql serves all // five ports' query scenarios against Testcontainers Postgres. -import { readFileSync } from "node:fs"; +import { existsSync, readFileSync, writeFileSync } from "node:fs"; import { resolve } from "node:path"; import { @@ -31,7 +31,7 @@ import { CANONICAL_DIR } from "./paths.ts"; * Postgres, and TS view synthesis emits the correlated-subquery view bodies * that match what introspection sees there. */ -export const CANONICAL_SCHEMA_DIALECT: Dialect = "postgres"; +export const CANONICAL_SCHEMA_DIALECT: CanonicalSchemaDialect = "postgres"; /** * Column-naming strategy for the canonical schema. Pinned to "literal" so the @@ -45,14 +45,39 @@ export const CANONICAL_COLUMN_NAMING: ColumnNamingStrategy = "literal"; /** Absolute path to the committed canonical Postgres schema artifact. */ export const CANONICAL_SCHEMA_SQL_PATH = resolve(CANONICAL_DIR, "schema.postgres.sql"); -const HEADER_LINES: readonly string[] = [ - "-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT.", - "-- Canonical persistence-conformance schema (Postgres).", - "-- Produced by TypeScript from canonical/meta.fitness.json; every port's", - "-- query-scenario runner executes this file verbatim to provision its test DB.", - "-- Regenerate: `bun run gen:schema` in this package. Drift-checked by", - "-- schema-artifact.test.ts (CI fails if this file diverges from metadata).", -]; +/** + * The dialects `meta migrate` owns a schema for, and so the ones TypeScript can produce a + * canonical schema artifact for. MySQL is absent on purpose (ADR-0015): its tables are the + * adopter's own DDL, see `canonical-schema-mysql.ts`. D1 is SQLite's dialect with its own + * migration spelling and gets no committed artifact: the D1 lane generates it on every run + * (`generateCanonicalSchemaSql(root, { dialect: "d1" })`, the diff + emit `meta migrate + * --dialect d1` runs) and executes it in Miniflare's local D1. + */ +export type CanonicalSchemaDialect = "postgres" | "sqlite"; + +/** Absolute path to the committed canonical schema artifact of `dialect`. */ +export function canonicalSchemaSqlPath(dialect: CanonicalSchemaDialect): string { + return resolve(CANONICAL_DIR, `schema.${dialect}.sql`); +} + +/** How each dialect names itself in the header of the artifact it generates. A dialect not + * listed here is a type error, not a header that silently keeps claiming to be Postgres. */ +const DIALECT_LABEL: Record = { + postgres: "Postgres", + sqlite: "SQLite", + d1: "D1", +}; + +function canonicalHeaderFor(dialect: CanonicalSchemaDialect | "d1"): string { + return [ + "-- @generated by @metaobjectsdev/integration-tests — DO NOT EDIT.", + `-- Canonical persistence-conformance schema (${DIALECT_LABEL[dialect]}).`, + "-- Produced by TypeScript from canonical/meta.fitness.json; every port's", + "-- query-scenario runner executes this file verbatim to provision its test DB.", + "-- Regenerate: `bun run gen:schema` in this package. Drift-checked by", + "-- schema-artifact.test.ts (CI fails if this file diverges from metadata).", + ].join("\n"); +} /** * Generate the canonical Postgres schema DDL from a loaded metadata root. @@ -68,25 +93,42 @@ const HEADER_LINES: readonly string[] = [ */ export async function generateCanonicalSchemaSql( root: MetaRoot, - opts?: { header?: string }, + opts?: { header?: string; dialect?: CanonicalSchemaDialect | "d1" }, ): Promise { + const dialectId = opts?.dialect ?? CANONICAL_SCHEMA_DIALECT; + const dialect: Dialect = dialectId; const expected = buildExpectedSchema(root, { - dialect: CANONICAL_SCHEMA_DIALECT, + dialect, columnNamingStrategy: CANONICAL_COLUMN_NAMING, views: buildProjectionViews(root, { - dialect: CANONICAL_SCHEMA_DIALECT, + dialect, columnNamingStrategy: CANONICAL_COLUMN_NAMING, }), }); - const r = await diff({ expected, actual: { tables: [], views: [] } }); - const { up } = emit(r.changes, { dialect: CANONICAL_SCHEMA_DIALECT }); - const header = opts?.header ?? HEADER_LINES.join("\n"); + const r = await diff({ expected, actual: { tables: [], views: [] }, dialect }); + const { up } = emit(r.changes, { dialect, expectedSchema: expected }); + const header = opts?.header ?? canonicalHeaderFor(dialectId); return `${header}\n\n${up}\n`; } -/** Read the committed canonical Postgres schema artifact. */ -export function readCanonicalSchemaSql(): string { - return readFileSync(CANONICAL_SCHEMA_SQL_PATH, "utf8"); +/** Read the committed canonical schema artifact (Postgres unless `dialect` says otherwise). */ +export function readCanonicalSchemaSql(dialect: CanonicalSchemaDialect = "postgres"): string { + return readFileSync(canonicalSchemaSqlPath(dialect), "utf8"); +} + +/** + * Write a committed artifact. Overwrite unconditionally — generated-wins, and refusing would + * block the command that repairs a red drift gate — but never SILENTLY (see + * describeRegenReplacement). Shared by every gen script so the policy lives once. + */ +export function writeArtifact(path: string, sql: string): void { + const existing = existsSync(path) ? readFileSync(path, "utf8") : undefined; + const replaced = describeRegenReplacement(existing, sql); + writeFileSync(path, sql, "utf8"); + /* eslint-disable no-console */ + if (replaced !== undefined) console.warn(replaced); + console.log(`wrote ${path} (${sql.length} bytes)`); + /* eslint-enable no-console */ } /** diff --git a/server/typescript/packages/integration-tests/src/cube-container.ts b/server/typescript/packages/integration-tests/src/cube-container.ts index 0c05a26e5..8d89204de 100644 --- a/server/typescript/packages/integration-tests/src/cube-container.ts +++ b/server/typescript/packages/integration-tests/src/cube-container.ts @@ -1,6 +1,9 @@ // CubeStack — a throwaway Cube instance over a throwaway Postgres, for the cube-model live check // (FR-044 Plan 4, Task 7; cube-live/cube-model.live.ts). // +// (A MySQL data source swaps `postgres:16-alpine` for `mysql:8.4` and Cube's driver for MySQL's; +// the rest is the same, and `CubeStackOptions.dataSource` picks.) +// // Everything runs through the docker CLI, as postgres-container.ts and mysql-container.ts do (and // for their reason: testcontainers-node hangs under Bun). One run owns: // @@ -26,6 +29,10 @@ import { randomUUID } from "node:crypto"; export const CUBE_IMAGE = "cubejs/cube:v1.7.43"; export const CUBE_POSTGRES_IMAGE = "postgres:16-alpine"; +export const CUBE_MYSQL_IMAGE = "mysql:8.4"; + +/** The database Cube reads. */ +export type CubeDataSource = "postgres" | "mysql"; /** Prefix of every container and network a run creates; the leftover check filters on it. */ export const CUBE_RESOURCE_PREFIX = "mo-cube-"; @@ -51,6 +58,7 @@ const PULL_TIMEOUT_MS = 15 * 60_000; const PROBE_TIMEOUT_MS = 10_000; const PG_PASSWORD = "test"; +const MYSQL_DATABASE = "mo_test"; /** The longest startCubeStack can take: two pulls, then both readiness windows. A hook's timeout. */ export function cubeStackStartBudgetMs(): number { @@ -63,8 +71,10 @@ export interface CubeStack { readonly network: string; readonly pgContainer: string; readonly cubeContainer: string; - /** Run SQL through `psql` inside the Postgres container (ON_ERROR_STOP, quiet, unaligned, tuples only). */ + /** Run SQL through `psql` inside the Postgres container (ON_ERROR_STOP, quiet, unaligned, tuples only). Postgres data source only. */ psql(sqlText: string): string; + /** Run SQL through `mysql` inside the MySQL container (batch mode, tab-separated, no header). MySQL data source only. */ + mysql(sqlText: string): string; /** The last `lines` lines of the Cube container's log, for a failure message. */ cubeLogs(lines?: number): string; /** Force-remove both containers and the network. Idempotent; never throws, but warns on stderr when a removal fails. */ @@ -72,8 +82,10 @@ export interface CubeStack { } export interface CubeStackOptions { + /** The database Cube reads; Postgres when absent. `pgContainer` names this database's container either way. */ + readonly dataSource?: CubeDataSource; /** - * SQL scripts applied, in order, once Postgres is ready and BEFORE Cube starts: the schema and + * SQL scripts applied, in order, once the database is ready and BEFORE Cube starts: the schema and * the seed. Cube builds a rollup on the first query that needs it and keeps it for its refresh * window, so data loaded after Cube is up could meet a rollup already built over empty tables. */ @@ -101,8 +113,9 @@ export async function startCubeStack(modelDir: string, options: CubeStackOptions const unavailable = dockerUnavailableReason(); if (unavailable !== undefined) return { kind: "skipped", reason: unavailable }; + const dataSource = options.dataSource ?? "postgres"; // Outside the clock: a pinned tag already present is used as is. - await ensureImage(CUBE_POSTGRES_IMAGE); + await ensureImage(dataSource === "mysql" ? CUBE_MYSQL_IMAGE : CUBE_POSTGRES_IMAGE); await ensureImage(CUBE_IMAGE); const id = randomUUID().slice(0, 8); @@ -133,20 +146,27 @@ export async function startCubeStack(modelDir: string, options: CubeStackOptions try { docker(["network", "create", network]); - docker(["run", "-d", "--name", pgContainer, "--network", network, - "-e", `POSTGRES_PASSWORD=${PG_PASSWORD}`, CUBE_POSTGRES_IMAGE]); - await waitForPostgres(pgContainer); - for (const script of options.initSql ?? []) psql(pgContainer, script); + if (dataSource === "mysql") { + docker(["run", "-d", "--name", pgContainer, "--network", network, + "-e", `MYSQL_ROOT_PASSWORD=${PG_PASSWORD}`, "-e", `MYSQL_DATABASE=${MYSQL_DATABASE}`, CUBE_MYSQL_IMAGE]); + await waitForMysql(pgContainer); + for (const script of options.initSql ?? []) mysqlExec(pgContainer, script); + } else { + docker(["run", "-d", "--name", pgContainer, "--network", network, + "-e", `POSTGRES_PASSWORD=${PG_PASSWORD}`, CUBE_POSTGRES_IMAGE]); + await waitForPostgres(pgContainer); + for (const script of options.initSql ?? []) psql(pgContainer, script); + } + const dbEnv = dataSource === "mysql" + ? ["CUBEJS_DB_TYPE=mysql", `CUBEJS_DB_HOST=${pgContainer}`, "CUBEJS_DB_PORT=3306", + `CUBEJS_DB_NAME=${MYSQL_DATABASE}`, "CUBEJS_DB_USER=root", `CUBEJS_DB_PASS=${PG_PASSWORD}`] + : ["CUBEJS_DB_TYPE=postgres", `CUBEJS_DB_HOST=${pgContainer}`, "CUBEJS_DB_PORT=5432", + "CUBEJS_DB_NAME=postgres", "CUBEJS_DB_USER=postgres", `CUBEJS_DB_PASS=${PG_PASSWORD}`]; docker([ "run", "-d", "--name", cubeContainer, "--network", network, "-p", "127.0.0.1::4000", - "-e", "CUBEJS_DB_TYPE=postgres", - "-e", `CUBEJS_DB_HOST=${pgContainer}`, - "-e", "CUBEJS_DB_PORT=5432", - "-e", "CUBEJS_DB_NAME=postgres", - "-e", "CUBEJS_DB_USER=postgres", - "-e", `CUBEJS_DB_PASS=${PG_PASSWORD}`, + ...dbEnv.flatMap((e) => ["-e", e]), "-e", "CUBEJS_DEV_MODE=true", "-e", "CUBEJS_TELEMETRY=false", // Throwaway: development mode does not check tokens, and the API is bound to 127.0.0.1. @@ -166,6 +186,7 @@ export async function startCubeStack(modelDir: string, options: CubeStackOptions pgContainer, cubeContainer, psql: (sqlText) => psql(pgContainer, sqlText), + mysql: (sqlText) => mysqlExec(pgContainer, sqlText), cubeLogs: (lines = 60) => tailLogs(cubeContainer, lines), stop, }, @@ -219,6 +240,21 @@ async function waitForPostgres(name: string): Promise { throw new Error(`postgres container '${name}' did not accept a TCP connection within ${PG_READY_TIMEOUT_S}s`); } +/** Like waitForPostgres: the entrypoint's temporary server has no TCP listener, so connect over TCP. */ +async function waitForMysql(name: string): Promise { + const deadline = Date.now() + PG_READY_TIMEOUT_S * 1000; + while (Date.now() < deadline) { + assertRunning(name, "mysql"); + const r = spawnSync("docker", ["exec", name, "mysql", "-h127.0.0.1", "-uroot", `-p${PG_PASSWORD}`, "-N", "-B", "-e", "select 1"], { + encoding: "utf8", + timeout: 30_000, + }); + if (r.status === 0 && r.stdout.trim() === "1") return; + await sleep(500); + } + throw new Error(`mysql container '${name}' did not accept a TCP connection within ${PG_READY_TIMEOUT_S}s`); +} + /** Until `/meta` answers with JSON: a model, or the model's compile error (the test reads which). */ async function waitForCube(name: string, apiBase: string): Promise { const deadline = Date.now() + READY_TIMEOUT_S * 1000; @@ -268,6 +304,17 @@ function psql(name: string, sqlText: string): string { return r.stdout; } +function mysqlExec(name: string, sqlText: string): string { + const r = spawnSync("docker", ["exec", "-i", name, "mysql", "-h127.0.0.1", "-uroot", `-p${PG_PASSWORD}`, MYSQL_DATABASE, "-N", "-B"], { + input: sqlText, + encoding: "utf8", + timeout: 120_000, + maxBuffer: 64 * 1024 * 1024, + }); + if (r.status !== 0) throw new Error(`mysql in '${name}' failed (exit ${String(r.status)}): ${r.stderr || r.stdout}`); + return r.stdout; +} + function tailLogs(name: string, lines: number): string { const r = spawnSync("docker", ["logs", "--tail", String(lines), name], { encoding: "utf8", timeout: 30_000 }); return r.status === 0 ? `${r.stdout}${r.stderr}`.trim() : "(docker logs unavailable)"; diff --git a/server/typescript/packages/integration-tests/src/d1-kysely.ts b/server/typescript/packages/integration-tests/src/d1-kysely.ts new file mode 100644 index 000000000..eb9a445ef --- /dev/null +++ b/server/typescript/packages/integration-tests/src/d1-kysely.ts @@ -0,0 +1,54 @@ +// d1-kysely.ts — a minimal Kysely dialect over a Cloudflare D1 binding, so the TypeScript +// ObjectManager can read through D1's own API (`prepare().bind().all()`), not libsql. D1 is +// SQLite at the SQL level, so the adapter, compiler and introspector are SQLite's. + +import { + CompiledQuery, SqliteAdapter, SqliteIntrospector, SqliteQueryCompiler, + type DatabaseConnection, type Dialect, type Driver, type Kysely, type QueryResult, +} from "kysely"; + +/** The slice of the D1 binding this adapter uses. */ +export interface D1Like { + prepare(sql: string): { + bind(...values: unknown[]): { + all(): Promise<{ results?: Array>; meta?: { changes?: number; last_row_id?: number } }>; + }; + }; +} + +class D1Connection implements DatabaseConnection { + constructor(private readonly d1: D1Like) {} + + async executeQuery(q: CompiledQuery): Promise> { + const r = await this.d1.prepare(q.sql).bind(...q.parameters).all(); + const rows = (r.results ?? []) as R[]; + return r.meta?.changes === undefined ? { rows } : { rows, numAffectedRows: BigInt(r.meta.changes) }; + } + + // eslint-disable-next-line require-yield + async *streamQuery(): AsyncIterableIterator> { + throw new Error("D1 does not stream queries"); + } +} + +class D1Driver implements Driver { + constructor(private readonly d1: D1Like) {} + async init(): Promise {} + async acquireConnection(): Promise { return new D1Connection(this.d1); } + // D1 has no interactive transactions; the report scenarios are read-only. + async beginTransaction(): Promise { throw new Error("D1 does not support interactive transactions"); } + async commitTransaction(): Promise {} + async rollbackTransaction(): Promise {} + async releaseConnection(): Promise {} + async destroy(): Promise {} +} + +export function d1Dialect(d1: D1Like): Dialect { + return { + createAdapter: () => new SqliteAdapter(), + createDriver: () => new D1Driver(d1), + createIntrospector: (db: Kysely) => new SqliteIntrospector(db), + createQueryCompiler: () => new SqliteQueryCompiler(), + }; +} + diff --git a/server/typescript/packages/integration-tests/src/engine-wire.ts b/server/typescript/packages/integration-tests/src/engine-wire.ts new file mode 100644 index 000000000..fceaf4218 --- /dev/null +++ b/server/typescript/packages/integration-tests/src/engine-wire.ts @@ -0,0 +1,96 @@ +// engine-wire.ts — what a non-Postgres engine's driver hands the runtime, mapped onto the +// wire form the corpus asserts (`expect`, which is the Postgres value and is never loosened). +// +// The corpus pins a report column's wire form by its declared type (report-shapes.json, +// contract Table B): a `long` measure is a string, a `decimal` is a canonical decimal string, +// a time bucket is `YYYY-MM-DD` or a UTC `...Z` instant. node-postgres happens to deliver those +// spellings. The other engines deliver the same VALUES in their drivers' spellings: +// +// SQLite one INTEGER storage class, so a count is a JS number; a ratio is a REAL, so a whole +// one (`1.0`) is a JS integer and a fractional one a double; an instant is the TEXT the +// writer stored (`...T03:00:00.000Z`). +// MySQL BIGINT and DECIMAL arrive as strings (the connection asks for `bigNumberStrings`), +// but a DECIMAL keeps its scale (`0.7500`, `60.0000`), and a DATETIME is a naive +// `YYYY-MM-DD HH:MM:SS.fff` holding the UTC wall clock. +// +// This module applies the declared type to the driver value. It maps spelling only: a wrong +// VALUE still fails the comparison, which is the point of the lane. It runs on the actual side +// of a report row; an unknown entity, a non-report row and a null pass through untouched. + +import { readFileSync } from "node:fs"; + +import { canonicalDecimal, canonicalFloat, DECIMAL_RE } from "./normalization.ts"; +import { REPORT_SHAPES_PATH } from "./paths.ts"; + +export type WireEngine = "sqlite" | "mysql"; + +interface ShapeField { readonly name: string; readonly subType: string } + +let shapes: Map> | undefined; + +/** report entity simple name -> (field name -> subType), from the committed report-shapes.json. */ +function reportShapes(): Map> { + if (shapes) return shapes; + const parsed = JSON.parse(readFileSync(REPORT_SHAPES_PATH, "utf8")) as { + reports: ReadonlyArray<{ report: string; fields: ReadonlyArray }>; + }; + shapes = new Map( + parsed.reports.map((r) => [ + r.report.slice(r.report.lastIndexOf(":") + 1), + new Map(r.fields.map((f) => [f.name, f.subType])), + ]), + ); + return shapes; +} + +/** Canonical decimal text: no trailing fractional zeros, no bare `.` — the normalization.ts rule. */ +function decimalText(v: unknown): unknown { + if (typeof v === "number") return Number.isInteger(v) ? String(v) : canonicalFloat(v); + if (typeof v === "bigint") return v.toString(); + if (typeof v === "string" && DECIMAL_RE.test(v)) { + return v.includes(".") ? canonicalDecimal(v) : v; + } + return v; +} + +const pad = (n: number): string => n.toString().padStart(2, "0"); + +/** An instant -> `YYYY-MM-DDTHH:MM:SS[.fff]Z` in UTC, the corpus' TIMESTAMPTZ spelling. */ +function instantText(engine: WireEngine, v: unknown): unknown { + if (typeof v !== "string") return v; + // MySQL DATETIME has no zone: the lane holds the UTC wall clock, so read it as UTC. + const iso = engine === "mysql" && !/[zZ]|[+-]\d{2}:?\d{2}$/.test(v) ? `${v.replace(" ", "T")}Z` : v; + const d = new Date(iso); + if (Number.isNaN(d.getTime())) return v; + const ms = d.getUTCMilliseconds(); + const frac = ms === 0 ? "" : `.${String(ms).padStart(3, "0").replace(/0+$/, "")}`; + return ( + `${d.getUTCFullYear()}-${pad(d.getUTCMonth() + 1)}-${pad(d.getUTCDate())}` + + `T${pad(d.getUTCHours())}:${pad(d.getUTCMinutes())}:${pad(d.getUTCSeconds())}${frac}Z` + ); +} + +/** Map one row of report `entity` from `engine`'s driver spelling onto the corpus' wire form. */ +export function toWireRow(engine: WireEngine, entity: string, row: Record): Record { + const fields = reportShapes().get(entity); + if (!fields) return row; + const out: Record = {}; + for (const [name, value] of Object.entries(row)) { + if (value === null || value === undefined) { out[name] = value; continue; } + switch (fields.get(name)) { + case "long": + case "currency": // integral minor units: a BIGINT on the wire, like a count + out[name] = typeof value === "number" || typeof value === "bigint" ? String(value) : value; + break; + case "decimal": + out[name] = decimalText(value); + break; + case "timestamp": + out[name] = instantText(engine, value); + break; + default: + out[name] = value; + } + } + return out; +} diff --git a/server/typescript/packages/integration-tests/src/gen-api-contract-report-schema.ts b/server/typescript/packages/integration-tests/src/gen-api-contract-report-schema.ts index a45d77235..4804ec6b5 100644 --- a/server/typescript/packages/integration-tests/src/gen-api-contract-report-schema.ts +++ b/server/typescript/packages/integration-tests/src/gen-api-contract-report-schema.ts @@ -1,31 +1,26 @@ // gen-api-contract-report-schema.ts — (re)generate the committed report sub-corpus schema. // // Run: `bun run gen:report-api-schema` (from this package). Pure metadata→SQL — no DB. -// Writes fixtures/api-contract-conformance/report/schema.postgres.sql. +// Writes fixtures/api-contract-conformance/report/schema.postgres.sql, schema.sqlite.sql and schema.mysql.sql. -import { existsSync, readFileSync, writeFileSync } from "node:fs"; import { join } from "node:path"; +import { generateReportApiMysqlSchemaSql } from "./api-contract-report-mysql-schema.ts"; import { generateReportApiSchemaSql } from "./api-contract-report-schema.ts"; -import { describeRegenReplacement } from "./canonical-schema.ts"; +import { writeArtifact } from "./canonical-schema.ts"; import { loadMetadataFile } from "./load-metadata.ts"; -import { API_CONTRACT_REPORT_DIR, API_CONTRACT_REPORT_SCHEMA_SQL_PATH } from "./paths.ts"; +import { + API_CONTRACT_REPORT_DIR, + API_CONTRACT_REPORT_MYSQL_SCHEMA_SQL_PATH, + API_CONTRACT_REPORT_SCHEMA_SQL_PATH, + API_CONTRACT_REPORT_SQLITE_SCHEMA_SQL_PATH, +} from "./paths.ts"; async function main(): Promise { const root = await loadMetadataFile(join(API_CONTRACT_REPORT_DIR, "meta.json")); - const sql = await generateReportApiSchemaSql(root); - - // Generated-wins, never silent: same policy as gen-canonical-schema.ts. - const existing = existsSync(API_CONTRACT_REPORT_SCHEMA_SQL_PATH) - ? readFileSync(API_CONTRACT_REPORT_SCHEMA_SQL_PATH, "utf8") - : undefined; - const replaced = describeRegenReplacement(existing, sql); - - writeFileSync(API_CONTRACT_REPORT_SCHEMA_SQL_PATH, sql, "utf8"); - /* eslint-disable no-console */ - if (replaced !== undefined) console.warn(replaced); - console.log(`wrote ${API_CONTRACT_REPORT_SCHEMA_SQL_PATH} (${sql.length} bytes)`); - /* eslint-enable no-console */ + writeArtifact(API_CONTRACT_REPORT_SCHEMA_SQL_PATH, await generateReportApiSchemaSql(root, "postgres")); + writeArtifact(API_CONTRACT_REPORT_SQLITE_SCHEMA_SQL_PATH, await generateReportApiSchemaSql(root, "sqlite")); + writeArtifact(API_CONTRACT_REPORT_MYSQL_SCHEMA_SQL_PATH, generateReportApiMysqlSchemaSql(root)); } main().catch((err: unknown) => { diff --git a/server/typescript/packages/integration-tests/src/gen-canonical-schema.ts b/server/typescript/packages/integration-tests/src/gen-canonical-schema.ts index ce39b5538..b6fc84cc5 100644 --- a/server/typescript/packages/integration-tests/src/gen-canonical-schema.ts +++ b/server/typescript/packages/integration-tests/src/gen-canonical-schema.ts @@ -1,35 +1,25 @@ -// gen-canonical-schema.ts — (re)generate the committed canonical schema artifact. +// gen-canonical-schema.ts — (re)generate the committed canonical schema artifacts. // // Run: `bun run gen:schema` (from this package). Pure metadata→SQL — no DB. -// Writes fixtures/persistence-conformance/canonical/schema.postgres.sql. - -import { existsSync, readFileSync, writeFileSync } from "node:fs"; +// Writes fixtures/persistence-conformance/canonical/schema.postgres.sql and schema.sqlite.sql, +// then schema.mysql.sql (the adopter's tables plus the views TypeScript lowers: ADR-0015). import { - CANONICAL_SCHEMA_SQL_PATH, - describeRegenReplacement, + type CanonicalSchemaDialect, + canonicalSchemaSqlPath, generateCanonicalSchemaSql, + writeArtifact, } from "./canonical-schema.ts"; +import { CANONICAL_MYSQL_SCHEMA_SQL_PATH, generateCanonicalMysqlSchemaSql } from "./canonical-schema-mysql.ts"; import { loadMetadataDir } from "./load-metadata.ts"; import { CANONICAL_DIR } from "./paths.ts"; async function main(): Promise { const root = await loadMetadataDir(CANONICAL_DIR); - const sql = await generateCanonicalSchemaSql(root); - - // Overwrite unconditionally — this artifact is generated-wins, and refusing here would - // block the command that repairs a red drift gate — but never SILENTLY. See - // describeRegenReplacement for why saying so is the whole fix. - const existing = existsSync(CANONICAL_SCHEMA_SQL_PATH) - ? readFileSync(CANONICAL_SCHEMA_SQL_PATH, "utf8") - : undefined; - const replaced = describeRegenReplacement(existing, sql); - - writeFileSync(CANONICAL_SCHEMA_SQL_PATH, sql, "utf8"); - /* eslint-disable no-console */ - if (replaced !== undefined) console.warn(replaced); - console.log(`wrote ${CANONICAL_SCHEMA_SQL_PATH} (${sql.length} bytes)`); - /* eslint-enable no-console */ + for (const dialect of ["postgres", "sqlite"] as const satisfies readonly CanonicalSchemaDialect[]) { + writeArtifact(canonicalSchemaSqlPath(dialect), await generateCanonicalSchemaSql(root, { dialect })); + } + writeArtifact(CANONICAL_MYSQL_SCHEMA_SQL_PATH, generateCanonicalMysqlSchemaSql(root)); } main().catch((err: unknown) => { diff --git a/server/typescript/packages/integration-tests/src/gen-report-shapes.ts b/server/typescript/packages/integration-tests/src/gen-report-shapes.ts index 53bd9fb10..429e44188 100644 --- a/server/typescript/packages/integration-tests/src/gen-report-shapes.ts +++ b/server/typescript/packages/integration-tests/src/gen-report-shapes.ts @@ -16,7 +16,6 @@ // canonical/ as metadata, and a non-metadata .json there fails the load. import { readFileSync, writeFileSync } from "node:fs"; -import { resolve } from "node:path"; import { OBJECT_SUBTYPE_REPORT, @@ -27,10 +26,7 @@ import { } from "@metaobjectsdev/metadata"; import { loadMetadataDir } from "./load-metadata.ts"; -import { CANONICAL_DIR, CORPUS_DIR } from "./paths.ts"; - -/** Absolute path to the committed report-shapes artifact. */ -export const REPORT_SHAPES_PATH = resolve(CORPUS_DIR, "report-shapes.json"); +import { CANONICAL_DIR, REPORT_SHAPES_PATH } from "./paths.ts"; interface ShapeFieldJson { name: string; diff --git a/server/typescript/packages/integration-tests/src/normalization.ts b/server/typescript/packages/integration-tests/src/normalization.ts index b59e3c6f9..a1b0480b7 100644 --- a/server/typescript/packages/integration-tests/src/normalization.ts +++ b/server/typescript/packages/integration-tests/src/normalization.ts @@ -14,7 +14,9 @@ export function normalizeRow(row: Record): Record; +} + +const D1_BINDING = "DB"; + +/** One local D1 database: a Miniflare instance and its D1 binding. */ +export interface LocalD1 { + d1: D1Database; + dispose(): Promise; +} + +export async function startLocalD1(): Promise { + const mf = new Miniflare({ + modules: true, + script: "export default { fetch() { return new Response('ok'); } }", + d1Databases: { [D1_BINDING]: "report-dialects-local-d1" }, + d1Persist: false, + }); + const d1 = (await mf.getD1Database(D1_BINDING)) as unknown as D1Database; + return { d1, dispose: () => mf.dispose() }; +} + +/** D1's `exec` runs one statement per line; fold each statement onto one line first. */ +export async function execScript(d1: D1Database, script: string): Promise { + for (const stmt of splitStatements(script)) { + await d1.exec(stmt.replace(/\s*\n\s*/g, " ")); + } +} + +export async function runQueryScenarioD1( + scenario: QueryScenario, + canonicalDir: string, + local: LocalD1, + /** `generateCanonicalSchemaSql(root, { dialect: "d1" })` — same bytes for every scenario, so the suite generates it once. */ + schemaSql: string, +): Promise { + const root = await loadMetadataDir(canonicalDir); + await execScript(local.d1, schemaSql); + const seed = scenario.seedDataEngine?.sqlite ?? scenario.seedData; + if (seed && seed.trim().length > 0) await execScript(local.d1, seed); + + const kysely = new Kysely({ dialect: d1Dialect(local.d1) }); + const driver = kyselyDriver({ db: kysely as never, dialect: "sqlite" }); + const om = new ObjectManager({ metadata: root, driver, columnNamingStrategy: CANONICAL_COLUMN_NAMING }); + await runScenarioQueries(scenario, om, root, "sqlite", "d1"); +} diff --git a/server/typescript/packages/integration-tests/src/query-scenario-mysql.ts b/server/typescript/packages/integration-tests/src/query-scenario-mysql.ts new file mode 100644 index 000000000..8922cc024 --- /dev/null +++ b/server/typescript/packages/integration-tests/src/query-scenario-mysql.ts @@ -0,0 +1,95 @@ +// query-scenario-mysql.ts — the persistence query scenarios on MySQL 8.4, through the +// ObjectManager's Kysely driver (MysqlDialect + mysql2). Same shape as the SQLite runner: +// execute the committed schema artifact, seed, run the shared DSL, hold the result to `expect` +// (or the query's `expect-engine` entry). +// +// The artifact is canonical/schema.mysql.sql: the adopter's tables plus the report views +// TypeScript lowers (buildReportViews, docs/recipes/mysql.md). The server keeps its default +// sql_mode, ONLY_FULL_GROUP_BY included — the view bodies must be valid under it. +// +// The corpus' seed is Postgres SQL with `"identifier"` quoting. MySQL quotes identifiers with +// backticks (ANSI_QUOTES would change how the VIEW bodies parse, so it stays off), so the seed +// is translated mechanically: identifier quotes become backticks, and an ISO instant's `Z` +// is dropped because DATETIME holds the UTC wall clock (the pool pins `timezone: "Z"`). + +import { ObjectManager } from "@metaobjectsdev/runtime-ts"; +import { kyselyDriver } from "@metaobjectsdev/runtime-ts/drivers"; +import { Kysely, MysqlDialect } from "kysely"; +import { createPool } from "mysql2"; +import mysql from "mysql2/promise"; + +import { splitStatements } from "./sql-script.ts"; + +import { CANONICAL_COLUMN_NAMING } from "./canonical-schema.ts"; +import { readCanonicalMysqlSchemaSql } from "./canonical-schema-mysql.ts"; +import { loadMetadataDir } from "./load-metadata.ts"; +import { runScenarioQueries } from "./query-scenario.ts"; +import type { QueryScenario } from "./scenario.ts"; + +/** + * Postgres-quoted seed SQL -> MySQL. Walks the text once so a `"` INSIDE a single-quoted + * string (a JSON payload) is left alone, and only a double-quoted identifier is rewritten. + */ +export function toMysqlSeed(seed: string): string { + let out = ""; + let inString = false; + let inIdent = false; + for (let i = 0; i < seed.length; i++) { + const c = seed[i]!; + if (inString) { + out += c; + if (c === "'") { + if (seed[i + 1] === "'") { out += "'"; i++; } else inString = false; + } + } else if (inIdent) { + if (c === '"') { out += "`"; inIdent = false; } else out += c; + } else if (c === "'") { + inString = true; + out += c; + } else if (c === '"') { + inIdent = true; + out += "`"; + } else { + out += c; + } + } + // '2026-05-04T03:30:00Z' -> '2026-05-04T03:30:00' (DATETIME is the UTC wall clock). + return out.replace(/'(\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:\.\d+)?)Z'/g, "'$1'"); +} + +/** Every table and view the schema artifact creates, dropped first so a rerun starts clean. */ +export async function resetSchema(conn: mysql.Connection, schemaSql: string): Promise { + const statements = splitStatements(schemaSql); + const views = statements.flatMap((s) => /^CREATE VIEW `([^`]+)`/.exec(s)?.[1] ?? []); + const tables = statements.flatMap((s) => /^CREATE TABLE `([^`]+)`/.exec(s)?.[1] ?? []); + for (const v of views) await conn.query(`DROP VIEW IF EXISTS \`${v}\``); + for (const t of [...tables].reverse()) await conn.query(`DROP TABLE IF EXISTS \`${t}\``); + for (const stmt of statements) await conn.query(stmt); +} + +export async function runQueryScenarioMysql( + scenario: QueryScenario, + mysqlUrl: string, + canonicalDir: string, +): Promise { + const admin = await mysql.createConnection({ uri: mysqlUrl, timezone: "Z" }); + // The pool is the runtime's: BIGINT and DECIMAL as text (a JS number cannot hold a BIGINT), + // DATE / DATETIME as the text MySQL sent rather than a Date shifted by the host zone. + const pool = createPool({ + uri: mysqlUrl, timezone: "Z", dateStrings: true, supportBigNumbers: true, bigNumberStrings: true, + }); + const kysely = new Kysely>({ dialect: new MysqlDialect({ pool }) }); + try { + await resetSchema(admin, readCanonicalMysqlSchemaSql()); + const seed = scenario.seedDataEngine?.mysql ?? (scenario.seedData ? toMysqlSeed(scenario.seedData) : ""); + for (const stmt of splitStatements(seed)) await admin.query(stmt); + + const root = await loadMetadataDir(canonicalDir); + const driver = kyselyDriver({ db: kysely as never, dialect: "mysql" }); + const om = new ObjectManager({ metadata: root, driver, columnNamingStrategy: CANONICAL_COLUMN_NAMING }); + await runScenarioQueries(scenario, om, root, "mysql"); + } finally { + await kysely.destroy(); + await admin.end(); + } +} diff --git a/server/typescript/packages/integration-tests/src/query-scenario-sqlite.ts b/server/typescript/packages/integration-tests/src/query-scenario-sqlite.ts new file mode 100644 index 000000000..6d42f4972 --- /dev/null +++ b/server/typescript/packages/integration-tests/src/query-scenario-sqlite.ts @@ -0,0 +1,51 @@ +// query-scenario-sqlite.ts — the persistence query scenarios on SQLite (libsql), and the +// shape every non-Postgres engine's runner follows: +// 1. execute the engine's committed schema artifact (TS-produced, drift-checked); +// 2. execute the scenario's seed data; +// 3. run the shared DSL through the ObjectManager and hold each result to `expect`, the +// Postgres wire value, after mapping the engine's own spelling (engine-wire.ts). +// +// The Postgres runner is query-scenario.ts; both call runScenarioQueries, so the DSL +// translation and the comparison are one implementation. + +import { ObjectManager } from "@metaobjectsdev/runtime-ts"; +import { kyselyDriver } from "@metaobjectsdev/runtime-ts/drivers"; +import { LibsqlDialect } from "@libsql/kysely-libsql"; +import { Kysely, sql } from "kysely"; +import { mkdtempSync, rmSync } from "node:fs"; +import { tmpdir } from "node:os"; +import { join } from "node:path"; + +import { CANONICAL_COLUMN_NAMING, readCanonicalSchemaSql } from "./canonical-schema.ts"; +import { loadMetadataDir } from "./load-metadata.ts"; +import { runScenarioQueries } from "./query-scenario.ts"; +import { splitStatements } from "./sql-script.ts"; +import type { QueryScenario } from "./scenario.ts"; + +/** libsql executes one statement per call. */ +export async function applySqlScript(db: Kysely, script: string): Promise { + for (const stmt of splitStatements(script)) { + await sql.raw(stmt).execute(db); + } +} + +export async function runQueryScenarioSqlite( + scenario: QueryScenario, + canonicalDir: string, +): Promise { + const dir = mkdtempSync(join(tmpdir(), "query-scenario-sqlite-")); + const kysely = new Kysely({ dialect: new LibsqlDialect({ url: `file:${join(dir, "test.db")}` }) }); + try { + await applySqlScript(kysely, readCanonicalSchemaSql("sqlite")); + const seed = scenario.seedDataEngine?.sqlite ?? scenario.seedData; + if (seed && seed.trim().length > 0) await applySqlScript(kysely, seed); + + const root = await loadMetadataDir(canonicalDir); + const driver = kyselyDriver({ db: kysely as never, dialect: "sqlite" }); + const om = new ObjectManager({ metadata: root, driver, columnNamingStrategy: CANONICAL_COLUMN_NAMING }); + await runScenarioQueries(scenario, om, root, "sqlite"); + } finally { + await kysely.destroy(); + rmSync(dir, { recursive: true, force: true }); + } +} diff --git a/server/typescript/packages/integration-tests/src/query-scenario.ts b/server/typescript/packages/integration-tests/src/query-scenario.ts index 31577118c..92b992be0 100644 --- a/server/typescript/packages/integration-tests/src/query-scenario.ts +++ b/server/typescript/packages/integration-tests/src/query-scenario.ts @@ -26,6 +26,7 @@ import { CANONICAL_COLUMN_NAMING, readCanonicalSchemaSql, } from "./canonical-schema.ts"; +import { toWireRow, type WireEngine } from "./engine-wire.ts"; import { loadMetadataDir } from "./load-metadata.ts"; import { canonicalJson, normalizeRow } from "./normalization.ts"; import { executeSql } from "./postgres-sql.ts"; @@ -71,30 +72,53 @@ export async function runQueryScenario( // columns the schema doesn't have. See migration-scenario.ts for why both // runners pin "literal". const om = new ObjectManager({ metadata: root, driver, columnNamingStrategy: CANONICAL_COLUMN_NAMING }); + await runScenarioQueries(scenario, om, root, "postgres"); + } finally { + await kysely.destroy(); + } +} - for (const spec of scenario.queries) { - if (spec.expectError) { - let threw = false; - try { - await execute(om, root, spec); - } catch { - threw = true; - } - if (!threw) { - throw new Error( - `${scenario.sourcePath} / ${spec.name}: expected the ${spec.op} to FAIL, but it succeeded`, - ); - } - continue; +/** + * Run a scenario's queries through `om` and hold each result to `expect`, the Postgres wire + * value, unchanged for every engine: a non-Postgres engine's driver spelling is mapped onto it + * first (engine-wire.ts), the expectation is never loosened. Shared by every engine's runner so + * the DSL translation and the comparison cannot drift between them. + */ +export async function runScenarioQueries( + scenario: QueryScenario, + om: ObjectManager, + root: MetaRoot, + /** The driver's wire spelling. The D1 lane passes "sqlite" — D1 IS SQLite at the SQL level. */ + engine: "postgres" | WireEngine, + /** Failure-message label, when the wire spelling is not the engine's name (the D1 lane). */ + label?: string, +): Promise { + for (const spec of scenario.queries) { + if (spec.expectError) { + let threw = false; + try { + await execute(om, root, spec); + } catch { + threw = true; + } + if (!threw) { + throw new Error( + `${scenario.sourcePath} / ${spec.name}: expected the ${spec.op} to FAIL, but it succeeded`, + ); } - const actual = await execute(om, root, spec); - assertResult(scenario.sourcePath, spec, actual); + continue; } - } finally { - await kysely.destroy(); + const actual = toEngineWire(engine, spec, await execute(om, root, spec)); + assertResult(scenario.sourcePath, spec, actual, label ?? engine); } } +/** A list result from a non-Postgres engine, mapped from its driver's spelling onto the corpus' wire form. */ +function toEngineWire(engine: "postgres" | WireEngine, spec: QuerySpec, actual: unknown): unknown { + if (engine === "postgres" || spec.op !== "list" || !Array.isArray(actual)) return actual; + return (actual as Record[]).map((row) => toWireRow(engine, spec.entity, row)); +} + // --------------------------------------------------------------------------- // DSL → ObjectManager // --------------------------------------------------------------------------- @@ -208,12 +232,12 @@ function toRuntimeFilter(filter: Record): Filter { // Assertions // --------------------------------------------------------------------------- -function assertResult(scenarioPath: string, spec: QuerySpec, actual: unknown): void { +function assertResult(scenarioPath: string, spec: QuerySpec, actual: unknown, engine: string): void { const expectedJson = canonicalizeExpected(spec.expect, spec.op); const actualJson = canonicalizeActual(actual, spec.op); if (expectedJson !== actualJson) throw new Error( - `${scenarioPath} / ${spec.name}: result mismatch\n expected: ${expectedJson}\n actual: ${actualJson}`, + `${scenarioPath} / ${spec.name}: result mismatch on ${engine}\n expected: ${expectedJson}\n actual: ${actualJson}`, ); } diff --git a/server/typescript/packages/integration-tests/src/scenario.ts b/server/typescript/packages/integration-tests/src/scenario.ts index b8286bdd5..7df13df4a 100644 --- a/server/typescript/packages/integration-tests/src/scenario.ts +++ b/server/typescript/packages/integration-tests/src/scenario.ts @@ -30,6 +30,13 @@ export interface QueryScenario { readonly description: string; readonly sourcePath: string; readonly seedData: string | null; + /** + * Per-engine replacement for `seed-data`, keyed by engine name (`sqlite`, `mysql`, `d1`). + * `seed-data` is Postgres SQL; an engine listed here runs ITS seed instead of a translation + * of it. Used only where an engine has no spelling to translate to (the relative-date seed + * calls the database clock, and every engine spells that differently). + */ + readonly seedDataEngine: Readonly> | null; readonly queries: ReadonlyArray; } @@ -89,6 +96,15 @@ export function loadQueries(dir: string): QueryScenario[] { return listYaml(dir).map(loadQuery); } +/** + * The `report-*` query scenarios — the only corpus subset the non-Postgres schema artifacts + * provision (their tables exist for what the reports read). One home for the predicate so a + * renamed or added dialect-gated scenario cannot be silently skipped by some lanes. + */ +export function loadReportQueries(dir: string): QueryScenario[] { + return loadQueries(dir).filter((s) => s.name.startsWith("report-")); +} + export function loadMigration(yamlPath: string): MigrationScenario { const raw = yamlLoad(readFileSync(yamlPath, "utf8")) as MigrationYaml; const base = dirname(yamlPath); @@ -126,6 +142,7 @@ export function loadQuery(yamlPath: string): QueryScenario { description: raw.description ?? "", sourcePath: yamlPath, seedData: raw["seed-data"] ?? null, + seedDataEngine: (raw["seed-data-engine"] as Record | undefined) ?? null, queries: (raw.queries ?? []).map((q) => ({ name: required(q.name, yamlPath, "query.name"), op: required(q.op, yamlPath, "query.op") as QuerySpec["op"], @@ -185,6 +202,7 @@ interface QueryYaml { name?: string; description?: string; "seed-data"?: string; + "seed-data-engine"?: unknown; queries?: { name?: string; op?: string; diff --git a/server/typescript/packages/integration-tests/src/sql-script.ts b/server/typescript/packages/integration-tests/src/sql-script.ts new file mode 100644 index 000000000..67f7e6b27 --- /dev/null +++ b/server/typescript/packages/integration-tests/src/sql-script.ts @@ -0,0 +1,11 @@ +// sql-script.ts — split a committed SQL script (the schema artifacts, a scenario's seed) into +// statements for a driver that runs one statement per call. +// +// A `;` ends a statement only at the end of a line, so one inside a string literal's data (a +// title, a JSON payload) does not break it. `--` comment lines are dropped first. + +export function splitStatements(script: string): string[] { + return script + .split("\n").filter((l) => !l.trim().startsWith("--")).join("\n") + .split(/;[ \t]*(?:\n|$)/).map((s) => s.trim()).filter(Boolean); +} diff --git a/server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts b/server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts index 09ba78a6b..ce402cccd 100644 --- a/server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts +++ b/server/typescript/packages/integration-tests/test/api-contract-report-corpus.test.ts @@ -7,10 +7,11 @@ import { } from "@metaobjectsdev/metadata"; import { loadMetadataFile } from "../src/load-metadata.ts"; import { loadScenarios } from "../src/api-contract-scenario.ts"; +import { generateReportApiMysqlSchemaSql } from "../src/api-contract-report-mysql-schema.ts"; import { generateReportApiSchemaSql } from "../src/api-contract-report-schema.ts"; import { SEED_REPORTS_KEY, seedInserts, type ReportSeed } from "../src/api-contract-report-generated-server.ts"; import { - API_CONTRACT_REPORT_DIR, API_CONTRACT_REPORT_SCENARIOS_DIR, API_CONTRACT_REPORT_SCHEMA_SQL_PATH, + API_CONTRACT_REPORT_DIR, API_CONTRACT_REPORT_SCENARIOS_DIR, API_CONTRACT_REPORT_SCHEMA_SQL_PATH, API_CONTRACT_REPORT_SQLITE_SCHEMA_SQL_PATH, API_CONTRACT_REPORT_MYSQL_SCHEMA_SQL_PATH, } from "../src/paths.ts"; const seed = JSON.parse(readFileSync(join(API_CONTRACT_REPORT_DIR, "seed.json"), "utf8")) as ReportSeed; @@ -84,4 +85,22 @@ describe("api-contract report corpus", () => { throw new Error("report/schema.postgres.sql is stale. Run `bun run gen:report-api-schema` in integration-tests."); } }); + + test("schema.sqlite.sql is what TypeScript produces from meta.json", async () => { + const root = await loadMetadataFile(join(API_CONTRACT_REPORT_DIR, "meta.json")); + const expected = await generateReportApiSchemaSql(root, "sqlite"); + const committed = readFileSync(API_CONTRACT_REPORT_SQLITE_SCHEMA_SQL_PATH, "utf8"); + if (committed !== expected) { + throw new Error("report/schema.sqlite.sql is stale. Run `bun run gen:report-api-schema` in integration-tests."); + } + }); + + test("schema.mysql.sql is what TypeScript produces from meta.json", async () => { + const root = await loadMetadataFile(join(API_CONTRACT_REPORT_DIR, "meta.json")); + const expected = generateReportApiMysqlSchemaSql(root); + const committed = readFileSync(API_CONTRACT_REPORT_MYSQL_SCHEMA_SQL_PATH, "utf8"); + if (committed !== expected) { + throw new Error("report/schema.mysql.sql is stale. Run `bun run gen:report-api-schema` in integration-tests."); + } + }); }); diff --git a/server/typescript/packages/integration-tests/test/api-contract-report-mysql.test.ts b/server/typescript/packages/integration-tests/test/api-contract-report-mysql.test.ts new file mode 100644 index 000000000..e4093590a --- /dev/null +++ b/server/typescript/packages/integration-tests/test/api-contract-report-mysql.test.ts @@ -0,0 +1,88 @@ +// FR-044 view-backed-report api-contract conformance, GENERATED lane, on MySQL 8.4. +// +// The cross-port corpus (api-contract-report.test.ts) runs the emitted routes on Postgres and +// api-contract-report-sqlite.test.ts on SQLite. This boots the same EMITTED report routes, +// unmodified, on a real MySQL with the views `buildReportViews({ dialect: "mysql" })` produces, +// and runs every shared scenario plus the seeded view rows. MySQL differs on the wire in the +// places the engine does: a ratio keeps its DECIMAL scale (`"0.4000"` where Postgres says +// `"0.4"`), so a decimal compares numerically, as the Postgres lane does. Requires Docker (or +// METAOBJECTS_TEST_MYSQL_URL). + +import { afterAll, beforeAll, describe, expect, test } from "bun:test"; +import { readFileSync } from "node:fs"; +import { join } from "node:path"; +import { API_CONTRACT_REPORT_DIR, API_CONTRACT_REPORT_SCENARIOS_DIR } from "../src/paths.ts"; +import { loadScenarios, runScenario } from "../src/api-contract-scenario.ts"; +import { rowsEqual, type ReportSeed } from "../src/api-contract-report-generated-server.ts"; +import { startMysqlReportServer, type MysqlReportServerHandle } from "../src/api-contract-report-mysql-server.ts"; +import { startMysql, type MysqlContainerHandle } from "../src/mysql-container.ts"; + +const SEED = JSON.parse(readFileSync(join(API_CONTRACT_REPORT_DIR, "seed.json"), "utf8")) as ReportSeed; +const META_PATH = join(API_CONTRACT_REPORT_DIR, "meta.json"); + +const SERVED_PATHS: Array<[string, string]> = [ + ["InvoiceStatusTotals", "/api/invoice_status_totals"], + ["InvoicesByMonth", "/api/invoices_by_months"], + ["InvoiceTotals", "/api/invoice_totals"], + ["ProductRevenue", "/api/product_revenues"], +]; + +let container: MysqlContainerHandle; +beforeAll(async () => { container = await startMysql(); }, 240_000); +afterAll(() => { container?.stop(); }, 60_000); + +async function withServer(fn: (server: MysqlReportServerHandle) => Promise, seed: ReportSeed = SEED): Promise { + const server = await startMysqlReportServer(container.url, META_PATH); + try { + await server.applySeed(seed); + await fn(server); + } finally { + await server.close(); + } +} + +describe("api contract report (FR-044) — GENERATED routes on MySQL", () => { + test("a ratio is a decimal string (MySQL keeps its scale), and the integers around it stay numbers", async () => { + await withServer(async (server) => { + const res = await fetch(server.baseUrl + "/api/invoice_totals"); + expect(res.status).toBe(200); + const rows = (await res.json()) as Array>; + expect(rows.length).toBe(1); + expect(typeof rows[0]!["paidShare"]).toBe("string"); + expect(Number(rows[0]!["paidShare"])).toBe(0.4); + expect(rows[0]!["invoices"]).toBe(5); + expect(rows[0]!["totalCents"]).toBe(315500); + }); + }, { timeout: 120_000 }); + + test("a ratio over zero rows is null", async () => { + await withServer(async (server) => { + const res = await fetch(server.baseUrl + "/api/invoice_totals"); + expect(await res.json()).toEqual([{ invoices: 0, totalCents: null, paidShare: null }]); + }, { invoices: [] }); + }, { timeout: 120_000 }); + + test("the seeded report rows are what the views return", async () => { + await withServer(async (server) => { + for (const [name, path] of SERVED_PATHS) { + const res = await fetch(server.baseUrl + path); + expect(res.status).toBe(200); + const actual = (await res.json()) as Array>; + const expected = SEED.reports![name]!; + expect(actual.length).toBe(expected.length); + const unmatched = [...actual]; + for (const want of expected) { + const idx = unmatched.findIndex((got) => rowsEqual(got, want)); + if (idx < 0) throw new Error(`${name}: no returned row equals ${JSON.stringify(want)}; got ${JSON.stringify(actual)}`); + unmatched.splice(idx, 1); + } + } + }); + }, { timeout: 120_000 }); + + for (const scenario of loadScenarios(API_CONTRACT_REPORT_SCENARIOS_DIR)) { + test(`shared scenario: ${scenario.name}`, async () => { + await withServer((server) => runScenario(scenario, server.baseUrl)); + }, { timeout: 120_000 }); + } +}); diff --git a/server/typescript/packages/integration-tests/test/api-contract-report-sqlite.test.ts b/server/typescript/packages/integration-tests/test/api-contract-report-sqlite.test.ts index 5f7cbc963..91aa717a4 100644 --- a/server/typescript/packages/integration-tests/test/api-contract-report-sqlite.test.ts +++ b/server/typescript/packages/integration-tests/test/api-contract-report-sqlite.test.ts @@ -1,4 +1,5 @@ -// FR-044 view-backed-report api-contract conformance, GENERATED lane, on SQLite. +// FR-044 view-backed-report api-contract conformance, GENERATED lane, on SQLite and on +// Cloudflare D1's local runtime (Miniflare, via `drizzle-orm/d1` and the `d1` codegen dialect). // // The cross-port corpus (api-contract-report.test.ts) runs TypeScript against Postgres // only, and its README does not assert a decimal's spelling. That left a hole on SQLite: @@ -17,15 +18,21 @@ import { describe, expect, test } from "bun:test"; import { readFileSync } from "node:fs"; import { join } from "node:path"; import { API_CONTRACT_REPORT_DIR, API_CONTRACT_REPORT_SCENARIOS_DIR } from "../src/paths.ts"; -import { loadScenarios, assertResponse, type ApiScenario } from "../src/api-contract-scenario.ts"; +import { loadScenarios, runScenario } from "../src/api-contract-scenario.ts"; import type { ReportSeed } from "../src/api-contract-report-generated-server.ts"; -import { startSqliteReportServer, type SqliteReportServerHandle } from "../src/api-contract-report-sqlite-server.ts"; +import { startSqliteReportServer, type SqliteReportEngine, type SqliteReportServerHandle } from "../src/api-contract-report-sqlite-server.ts"; const SEED = JSON.parse(readFileSync(join(API_CONTRACT_REPORT_DIR, "seed.json"), "utf8")) as ReportSeed; const META_PATH = join(API_CONTRACT_REPORT_DIR, "meta.json"); -async function withServer(fn: (server: SqliteReportServerHandle) => Promise, seed: ReportSeed = SEED): Promise { - const server = await startSqliteReportServer(META_PATH); +const ENGINES: readonly SqliteReportEngine[] = ["sqlite", "d1"]; + +async function withServer( + engine: SqliteReportEngine, + fn: (server: SqliteReportServerHandle) => Promise, + seed: ReportSeed = SEED, +): Promise { + const server = await startSqliteReportServer(META_PATH, engine); try { await server.applySeed(seed); await fn(server); @@ -40,9 +47,9 @@ async function getJson(server: SqliteReportServerHandle, path: string): Promise< return res.json(); } -describe("api contract report (FR-044) — GENERATED routes on SQLite", () => { +for (const engine of ENGINES) describe(`api contract report (FR-044) — GENERATED routes on ${engine === "d1" ? "Cloudflare D1 (local)" : "SQLite"}`, () => { test("a ratio is a JSON string, as it is on Postgres, and the integers around it stay numbers", async () => { - await withServer(async (server) => { + await withServer(engine, async (server) => { const rows = (await getJson(server, "/api/invoice_totals")) as Array>; expect(rows.length).toBe(1); const row = rows[0]!; @@ -54,7 +61,7 @@ describe("api contract report (FR-044) — GENERATED routes on SQLite", () => { }); test("a ratio is a string in the withCount envelope too", async () => { - await withServer(async (server) => { + await withServer(engine, async (server) => { const body = (await getJson(server, "/api/invoice_totals?withCount=1")) as { rows: Array>; total: number; }; @@ -64,14 +71,14 @@ describe("api contract report (FR-044) — GENERATED routes on SQLite", () => { }); test("a ratio over zero rows is null, not the string 'null' or 0", async () => { - await withServer(async (server) => { + await withServer(engine, async (server) => { const rows = (await getJson(server, "/api/invoice_totals")) as Array>; expect(rows).toEqual([{ invoices: 0, totalCents: null, paidShare: null }]); }, { invoices: [] }); }); test("a report with no decimal field is untouched: every value keeps its type", async () => { - await withServer(async (server) => { + await withServer(engine, async (server) => { expect(await getJson(server, "/api/invoice_status_totals?sort=status:asc")).toEqual( SEED.reports!["InvoiceStatusTotals"], ); @@ -85,7 +92,7 @@ describe("api contract report (FR-044) — GENERATED routes on SQLite", () => { }); test("the filter and the sort on a ratio still compare numerically after the wire change", async () => { - await withServer(async (server) => { + await withServer(engine, async (server) => { const hit = async (path: string): Promise => ((await getJson(server, path)) as unknown[]).length; expect(await hit("/api/invoice_totals?filter[paidShare][gt]=0.3")).toBe(1); expect(await hit("/api/invoice_totals?filter[paidShare][gt]=0.5")).toBe(0); @@ -98,24 +105,7 @@ describe("api contract report (FR-044) — GENERATED routes on SQLite", () => { for (const scenario of loadScenarios(API_CONTRACT_REPORT_SCENARIOS_DIR)) { test(`shared scenario: ${scenario.name}`, async () => { - await withServer((server) => runScenario(scenario, server)); + await withServer(engine, (server) => runScenario(scenario, server.baseUrl)); }); } }); - -async function runScenario(scenario: ApiScenario, server: SqliteReportServerHandle): Promise { - for (const req of scenario.requests) { - const init: RequestInit = { method: req.method }; - if (req.body !== undefined) { - init.body = JSON.stringify(req.body); - init.headers = { "content-type": "application/json" }; - } - const res = await fetch(server.baseUrl + req.path, init); - const bodyText = await res.text(); - let body: unknown = null; - if (bodyText.length > 0) { - try { body = JSON.parse(bodyText); } catch { body = bodyText; } - } - assertResponse(scenario.name, req, res.status, body); - } -} diff --git a/server/typescript/packages/integration-tests/test/engine-wire.test.ts b/server/typescript/packages/integration-tests/test/engine-wire.test.ts new file mode 100644 index 000000000..5d56fc65e --- /dev/null +++ b/server/typescript/packages/integration-tests/test/engine-wire.test.ts @@ -0,0 +1,66 @@ +// engine-wire.test.ts — the pure helpers behind the SQLite and MySQL persistence lanes. +// No DB: a spelling map that silently changed a value would let those lanes pass for the wrong +// reason, so each rule is pinned here, including the ones that must NOT fire. + +import { describe, expect, test } from "bun:test"; + +import { toWireRow } from "../src/engine-wire.ts"; +import { toMysqlSeed } from "../src/query-scenario-mysql.ts"; +import { splitStatements } from "../src/sql-script.ts"; + +describe("toWireRow", () => { + test("SQLite: a count and a currency are strings, a ratio is a canonical decimal string, min/max stay numbers", () => { + expect(toWireRow("sqlite", "ProgramMinutes", { + program: 1, weeks: 4, avgMinutes: 60, longShare: 0.75, minMinutes: 30, programTitle: "7", + })).toEqual({ + program: "1", weeks: "4", avgMinutes: "60", longShare: "0.75", minMinutes: 30, programTitle: "7", + }); + }); + + test("MySQL: a DECIMAL keeps its value and loses its scale; a BIGINT string is untouched", () => { + expect(toWireRow("mysql", "ProgramMinutes", { weeks: "4", avgMinutes: "60.0000", longShare: "0.6667" })) + .toEqual({ weeks: "4", avgMinutes: "60", longShare: "0.6667" }); + expect(toWireRow("mysql", "ProgramMinutes", { longShare: "0.0000" })).toEqual({ longShare: "0" }); + }); + + test("an instant becomes a UTC ...Z string: SQLite's stored text and MySQL's naive wall clock", () => { + expect(toWireRow("sqlite", "AssetActivity", { recordedAtHour: "2026-05-04T03:00:00.000Z" })) + .toEqual({ recordedAtHour: "2026-05-04T03:00:00Z" }); + expect(toWireRow("mysql", "AssetActivity", { recordedAtHour: "2026-05-04 03:00:00.000" })) + .toEqual({ recordedAtHour: "2026-05-04T03:00:00Z" }); + }); + + test("null, a date bucket and an unknown entity pass through", () => { + expect(toWireRow("sqlite", "ProgramMinutes", { totalMinutes: null })).toEqual({ totalMinutes: null }); + expect(toWireRow("sqlite", "AssetActivity", { asOfDateWeek: "2026-04-27" })).toEqual({ asOfDateWeek: "2026-04-27" }); + expect(toWireRow("mysql", "Program", { id: 1 })).toEqual({ id: 1 }); + }); + + test("a wrong value is still wrong: only the spelling is mapped", () => { + expect(toWireRow("sqlite", "ProgramMinutes", { weeks: 5 })).not.toEqual({ weeks: "4" }); + }); +}); + +describe("toMysqlSeed", () => { + test("a double-quoted identifier becomes a backtick one", () => { + expect(toMysqlSeed(`INSERT INTO "programs" ("id","title") VALUES (1, 'a');`)) + .toBe("INSERT INTO `programs` (`id`,`title`) VALUES (1, 'a');"); + }); + + test("a double quote inside a string literal is data, and '' does not end the string", () => { + expect(toMysqlSeed(`INSERT INTO "assets" ("payload") VALUES ('{"k": 1}'), ('it''s "x"');`)) + .toBe("INSERT INTO `assets` (`payload`) VALUES ('{\"k\": 1}'), ('it''s \"x\"');"); + }); + + test("an ISO instant loses its Z (DATETIME is the UTC wall clock); other strings keep theirs", () => { + expect(toMysqlSeed(`VALUES ('2026-05-04T03:30:00Z', '2026-05-04T03:30:00.5Z', 'Z')`)) + .toBe(`VALUES ('2026-05-04T03:30:00', '2026-05-04T03:30:00.5', 'Z')`); + }); +}); + +describe("splitStatements", () => { + test("drops -- comment lines and splits on a trailing semicolon only", () => { + expect(splitStatements("-- header\nCREATE TABLE a (x INT);\n\nINSERT INTO a VALUES (1), (2);\n")) + .toEqual(["CREATE TABLE a (x INT)", "INSERT INTO a VALUES (1), (2)"]); + }); +}); diff --git a/server/typescript/packages/integration-tests/test/query-d1.test.ts b/server/typescript/packages/integration-tests/test/query-d1.test.ts new file mode 100644 index 000000000..57675daff --- /dev/null +++ b/server/typescript/packages/integration-tests/test/query-d1.test.ts @@ -0,0 +1,32 @@ +// bun:test entry — the report-* persistence query scenarios on Cloudflare D1's local runtime. +// +// Miniflare runs the real workerd D1 binding on a SQLite file, so this reads the report views +// `meta migrate --dialect d1` creates through D1's own API. No Docker, no cloud account. One +// fresh D1 per scenario. See docs/CONFORMANCE.md "Split coverage". + +import { beforeAll, describe, test } from "bun:test"; +import { CANONICAL_DIR, QUERIES_DIR } from "../src/paths.ts"; +import { generateCanonicalSchemaSql } from "../src/canonical-schema.ts"; +import { loadMetadataDir } from "../src/load-metadata.ts"; +import { loadReportQueries } from "../src/scenario.ts"; +import { runQueryScenarioD1, startLocalD1 } from "../src/query-scenario-d1.ts"; + +// The D1 schema has no committed artifact (see query-scenario-d1.ts), but it depends only on the +// metadata — the whole migrate pipeline runs once for the suite instead of once per scenario. +let schemaSql: string; +beforeAll(async () => { + schemaSql = await generateCanonicalSchemaSql(await loadMetadataDir(CANONICAL_DIR), { dialect: "d1" }); +}, 120_000); + +describe("persistence conformance — report query scenarios on Cloudflare D1 (local runtime)", () => { + for (const scenario of loadReportQueries(QUERIES_DIR)) { + test(scenario.name, async () => { + const local = await startLocalD1(); + try { + await runQueryScenarioD1(scenario, CANONICAL_DIR, local, schemaSql); + } finally { + await local.dispose(); + } + }, { timeout: 60_000 }); + } +}); diff --git a/server/typescript/packages/integration-tests/test/query-mysql.test.ts b/server/typescript/packages/integration-tests/test/query-mysql.test.ts new file mode 100644 index 000000000..b5cdc0ea8 --- /dev/null +++ b/server/typescript/packages/integration-tests/test/query-mysql.test.ts @@ -0,0 +1,22 @@ +// bun:test entry — the report-* persistence query scenarios on MySQL 8.4. +// +// One MySQL server for the file, the schema artifact re-created per scenario. Requires Docker +// (or METAOBJECTS_TEST_MYSQL_URL). See query-sqlite.test.ts for why only the report scenarios. + +import { afterAll, beforeAll, describe, test } from "bun:test"; +import { CANONICAL_DIR, QUERIES_DIR } from "../src/paths.ts"; +import { loadReportQueries } from "../src/scenario.ts"; +import { runQueryScenarioMysql } from "../src/query-scenario-mysql.ts"; +import { startMysql, type MysqlContainerHandle } from "../src/mysql-container.ts"; + +let container: MysqlContainerHandle; +beforeAll(async () => { container = await startMysql(); }, 240_000); +afterAll(() => { container?.stop(); }, 60_000); + +describe("persistence conformance — report query scenarios on MySQL", () => { + for (const scenario of loadReportQueries(QUERIES_DIR)) { + test(scenario.name, async () => { + await runQueryScenarioMysql(scenario, container.url, CANONICAL_DIR); + }, { timeout: 120_000 }); + } +}); diff --git a/server/typescript/packages/integration-tests/test/query-sqlite.test.ts b/server/typescript/packages/integration-tests/test/query-sqlite.test.ts new file mode 100644 index 000000000..912a5513f --- /dev/null +++ b/server/typescript/packages/integration-tests/test/query-sqlite.test.ts @@ -0,0 +1,19 @@ +// bun:test entry — the report-* persistence query scenarios on SQLite. +// +// Only the report scenarios: the rest of the corpus gates the entity runtime on Postgres, and +// SQLite's gate there is its own (migrate-ts, the codegen lanes). A report is the part whose SQL +// is dialect-specific — conditional aggregates, grains, relative dates, the ratio's division — +// and that SQL is what an engine's lane has to run. See docs/CONFORMANCE.md "Split coverage". + +import { describe, test } from "bun:test"; +import { CANONICAL_DIR, QUERIES_DIR } from "../src/paths.ts"; +import { loadReportQueries } from "../src/scenario.ts"; +import { runQueryScenarioSqlite } from "../src/query-scenario-sqlite.ts"; + +describe("persistence conformance — report query scenarios on SQLite", () => { + for (const scenario of loadReportQueries(QUERIES_DIR)) { + test(scenario.name, async () => { + await runQueryScenarioSqlite(scenario, CANONICAL_DIR); + }, { timeout: 60_000 }); + } +}); diff --git a/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts b/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts index 126a590f7..a655dc872 100644 --- a/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts +++ b/server/typescript/packages/integration-tests/test/report-shapes-artifact.test.ts @@ -11,13 +11,9 @@ import { describe, expect, test } from "bun:test"; import { InMemoryStringSource, MetaDataLoader } from "@metaobjectsdev/metadata"; -import { - generateReportShapesJson, - readReportShapesJson, - REPORT_SHAPES_PATH, -} from "../src/gen-report-shapes.ts"; +import { generateReportShapesJson, readReportShapesJson } from "../src/gen-report-shapes.ts"; import { loadMetadataDir } from "../src/load-metadata.ts"; -import { CANONICAL_DIR } from "../src/paths.ts"; +import { CANONICAL_DIR, REPORT_SHAPES_PATH } from "../src/paths.ts"; describe("canonical report-shapes artifact (report-shapes.json)", () => { test("committed shapes match what TS derives from metadata (no drift)", async () => { diff --git a/server/typescript/packages/integration-tests/test/schema-artifact-engines.test.ts b/server/typescript/packages/integration-tests/test/schema-artifact-engines.test.ts new file mode 100644 index 000000000..fad9fd976 --- /dev/null +++ b/server/typescript/packages/integration-tests/test/schema-artifact-engines.test.ts @@ -0,0 +1,61 @@ +// schema-artifact-engines.test.ts — drift-check for the committed SQLite and MySQL canonical +// schema artifacts. No DB required: metadata -> SQL, byte-compared with the committed file. +// The Postgres twin is schema-artifact.test.ts. Regenerate all three with `bun run gen:schema`. +// +// SQLite's tables and views come from `meta migrate`'s own diff/emit (dialect sqlite). MySQL's +// tables are the adopter's hand-written DDL (ADR-0015) and its views `buildReportViews`. + +import { describe, expect, test } from "bun:test"; +import { buildReportViews } from "@metaobjectsdev/codegen-ts"; + +import { + CANONICAL_COLUMN_NAMING, + canonicalSchemaSqlPath, + generateCanonicalSchemaSql, + readCanonicalSchemaSql, +} from "../src/canonical-schema.ts"; +import { + CANONICAL_MYSQL_SCHEMA_SQL_PATH, + generateCanonicalMysqlSchemaSql, + readCanonicalMysqlSchemaSql, +} from "../src/canonical-schema-mysql.ts"; +import { loadMetadataDir } from "../src/load-metadata.ts"; +import { CANONICAL_DIR } from "../src/paths.ts"; + +describe("canonical schema artifacts for the other engines", () => { + test("schema.sqlite.sql matches what TS generates from metadata (no drift)", async () => { + const root = await loadMetadataDir(CANONICAL_DIR); + const generated = await generateCanonicalSchemaSql(root, { dialect: "sqlite" }); + if (generated !== readCanonicalSchemaSql("sqlite")) { + throw new Error( + `SQLite schema artifact is stale: ${canonicalSchemaSqlPath("sqlite")} differs from what TS ` + + `generates. Run \`bun run gen:schema\` and commit the result.`, + ); + } + }); + + test("schema.mysql.sql matches what TS generates from metadata (no drift)", async () => { + const root = await loadMetadataDir(CANONICAL_DIR); + if (generateCanonicalMysqlSchemaSql(root) !== readCanonicalMysqlSchemaSql()) { + throw new Error( + `MySQL schema artifact is stale: ${CANONICAL_MYSQL_SCHEMA_SQL_PATH} differs from what TS ` + + `generates. Run \`bun run gen:schema\` and commit the result.`, + ); + } + }); + + test("every report view is created in each engine's artifact", async () => { + // Derived from the model, not a hand list: a report added to canonical/meta.fitness.json + // must appear in every engine's artifact, and a lowering that emitted zero views is a + // failure, not an empty loop. + const root = await loadMetadataDir(CANONICAL_DIR); + const views = buildReportViews(root, { dialect: "mysql", columnNamingStrategy: CANONICAL_COLUMN_NAMING }).map((v) => v.name); + expect(views.length).toBeGreaterThan(0); + const sqlite = readCanonicalSchemaSql("sqlite"); + const mysql = readCanonicalMysqlSchemaSql(); + for (const v of views) { + expect(sqlite).toContain(`CREATE VIEW "${v}" AS`); + expect(mysql).toContain(`CREATE VIEW \`${v}\` AS`); + } + }); +});