diff --git a/.gitattributes b/.gitattributes index 6bc52b3d..8dc6e49d 100644 --- a/.gitattributes +++ b/.gitattributes @@ -1,2 +1,6 @@ +# Digests under corpus/ and vendor/ pin the exact bytes of text files, so a +# checkout must not rewrite line endings, whatever core.autocrlf says. +* text=auto eol=lf + # Upstream performance fixtures are immutable byte artifacts. perf/fixtures/upstream/* -text diff --git a/.gitignore b/.gitignore index 3610c84c..d47329e3 100644 --- a/.gitignore +++ b/.gitignore @@ -24,3 +24,4 @@ perf/fixtures/boundary/ !/.outline/gates/ !/.outline/GATES.md !/.outline/type-script-7.0.2-completion.md +/tasks/ diff --git a/.outline/GATES.md b/.outline/GATES.md index 9f8c8a0d..c9c0f4cc 100644 --- a/.outline/GATES.md +++ b/.outline/GATES.md @@ -3,7 +3,7 @@ Scope: Complete the clean-room Rust compiler, TypeScript-compatible Node 24 API package, ECMAScript runtime, native tiers, formal proof bindings, target cells, and release evidence for the stable TypeScript 7.0.2 product contract. - [ ] G1: Every leaf and integration ledger is met with recorded evidence - CHECK: python3 /home/alpha/.claude/plugins/cache/odin-marketplace/odin/1.17.108/skills/unlazy/scripts/gate_check.py --status .outline/gates/*.md + CHECK: just gate-status EXPECT: ALL MET EVIDENCE: 2026-08-28 not run to ALL MET; leaf ledgers still have unchecked G1-G6 boxes (F2.1, A3.1, C4.2, A2.7 G2-G6) @@ -16,35 +16,7 @@ Scope: Complete the clean-room Rust compiler, TypeScript-compatible Node 24 API CHECK: cargo run -p bamts-verification -- completion verify --root compiler EXPECT: COMPILER COMPLETE release=typescript-7.0.2 blocking=0 external_blocked=0 EVIDENCE: 2026-08-28 `completion verify --root compiler` exit 1: unproven B0.1, B1.1 through B1.4, B2.1, B2.2, B3.1 through B3.3, and 19515 more. Ledger n=78498 PASS=0 BLOCKING_FAIL=71750 - BASELINE: 2026-09-02 `bamts-verification suite run --catalog typescript-7.0.2 --shard i/64` with `BAMTS_SUITE_COMPILER_ADAPTER` set, receipts under `target/evidence-sweep/typescript-7.0.2/`. This is a shard sweep, not the CHECK above. 66786 rows: PASS 8951 (13.4%), BLOCKING_FAIL 51141 (76.6%), INAPPLICABLE_LANGUAGE_SERVICE 6693, TIMEOUT 1. Per-facet PASS: diagnostics 6525/15484, symbols 1846/14621, source-map 8/237, types 440/15918, declaration 61/2215, javascript 71/14096, parse 0/3718, trace 0/487, build-info 0/10. The 2026-08-28 line above reads PASS=0 because no adapter was registered, so every cell recorded `no registered adapter` and nothing was measured. These receipts pre-date 979b78c and 6d1ecc3, so they are a baseline for ranking work, never publishable completion evidence. - FACETS: 2026-09-02 the per-facet figures above are `PASS/total`, which reads every non-PASS row as outstanding work and is wrong for any facet carrying inapplicable rows. Cross-tabulating the same receipts by facet and state gives the applicable denominator, `PASS + BLOCKING_FAIL`. javascript 71/14060 pass, 13989 blocking, 36 inapplicable. types 440/13950, 13510 blocking, 1967 inapplicable. symbols 1846/13951, 12105 blocking, 670 inapplicable. diagnostics 6525/15182, 8657 blocking, 302 inapplicable. declaration 61/2215, 2154 blocking. trace 0/487, 487 blocking. source-map 8/237, 229 blocking. build-info 0/10, 10 blocking. parse has zero applicable rows: all 3718 are fourslash language-service cases and all are INAPPLICABLE_LANGUAGE_SERVICE, so `parse 0/3718` names an exclusion, never a gap. Pass share of applicable rows is 8951/60092 = 14.9%; the 13.4% above divides by all 66786 rows including exclusions. - RANKING: 2026-09-02 blocking mass by facet, which is the work order: javascript 13989, types 13510, symbols 12105, diagnostics 8657, declaration 2154, trace 487, source-map 229, build-info 10. Two facets have zero passing applicable rows: trace, whose largest single family is 78 rows of `classification/execution drift: no owned .trace.json baseline` and 31 of `invalid project trace fixture: missing field inputFiles`, and build-info, which has no observer function at all. Classification drift is not systemic: 218 of 51141 blocking rows (0.4%) carry it, over missing `.js` (127), `.trace.json` (78), `.js.map` (9), `.types` (2), and `.symbols` (2) baselines, so the blocking count stands essentially as measured. - JAVASCRIPT: 2026-09-02 12753 of 13989 javascript blocking rows (91.2%) carry one opaque detail, `javascript structural mismatch after formatting/order normalization`, which names no delta. Compiling 112 of them at HEAD and diffing against the reference `.js` classifies them as 72 reshape, 24 dropped-lines, 3 changed-lines, 12 emitting no `.js`, 1 identical, and every sampled family's first delta is the same missing `"use strict";` prologue. Extraction caveat for whoever re-measures: a reference baseline's last `//// [name]` section is often the `.d.ts`, so the section whose name ends in `.js` must be selected by name; taking the last section silently compares against declaration text and contaminated an earlier 86% figure. - PROLOGUE: 2026-09-02 measured over 10781 single-file baselines holding a real `.js` section. A file that is not an external module carries the prologue in 8644 cases and omits it in 82 (99.1%). An external module carries it in 1241 and omits it in 814, and the carriers concentrate in the script-format module kinds: `commonjs` 408 and a defaulted `module` 730. `crates/bamts-compiler/src/emitter/transforms.rs` gates emission on `always_strict || module_kind_implies_strict`, so it agrees on external modules and diverges on non-modules whenever `alwaysStrict` resolves false: 1437 of the 1597 baselines declaring `@strict: false` carry the prologue that gate omits. That is the single largest grounded javascript conversion available and it is not yet implemented. - JAVASCRIPT: 2026-09-03 v2 sweep (2048/2048 shards, lane binary on f8c28dc with the prologue implementation): javascript PASS 2015/14096 (14.3%), BLOCKING_FAIL 12045, up from 71/14060 on 2026-09-02. The +1944 spans the whole emitter campaign in range (prologue 93cc1eb/4756929 plus parens print 67790b5, declare/export align f36bbbc, the sourcemap series, jsdoc c51f49e, modern-js lowering ef67a7d), so it is not prologue-isolated. - PROLOGUE: 2026-09-03 residual sample at release binary 6cd8e0e (seed 20260903, N=300 of 12045 BLOCKING_FAIL javascript rows, row options from each receipt configuration plus harness strict/alwaysStrict defaults): 207 compared with 0 prologue deltas (95% upper bound about 1.4%); skips are multi-unit 69, no-reference 14, outfile-config 2, ambiguous-reference 1, missing .js section 2; 5 no-emit CLI gaps on exotic configs (system/isolatedModules/jsx, silent rc=1). Committed prologue unit tests pass 6/6. The prologue rule is done at HEAD within the comparable population; the residual is other families. - RESIDUAL: 2026-09-03 layered whole-document census at release binary (seed 600613, N=400 of 12045 BLOCKING_FAIL javascript rows; replicator calibrated EQUAL on 57 comparable PASS rows of 60 sampled (2 declaration-config + 1 multi-unit skipped), validating pairing, echo-strip, and framing assembly on those rows): WHITESPACE_ONLY 54, FIELD_INIT 29, COMMENT 27, DECORATOR 14, EXPORT_ELISION 2, OTHER_CODE 133, WOULD_PASS 5, COMPILE_FAIL 3; skips multi-unit 50, declaration-config 53, non-ts-source 25, outfile-config 5. The 5 WOULD_PASS rows (replication passes, sweep fails; 4 with sourcemap=true) are unexplained divergences with a worker-vs-CLI audit queued, so CLI-vs-worker equivalence is NOT established. No framing-only rows exist because the comparator's blank-sensitive document compare surfaces framing blanks as content diffs. Work order by mass: brace/paren whitespace shape, spurious field-init synthesis (`this.x = void 0`), comment drops, legacy decorator strategy. - FIELD_INIT: 2026-09-03 unconditional void-0 synthesis dropped under Assign semantics (b146af3; transforms.rs leaf-owned, no window needed; the CLI emit mapping takes 9 fixed keys with no useDefineForClassFields entry, so all knob runs defaulted and the construction default was unconditionally mode-blind): census bucket 29/400 to 8/400, remainder temp-naming and module-transform artifacts; headline baseline computedPropertyNames36_ES5 verified at CLI; suite 1801/0, clippy clean. Worker-side mapping under scout investigation; if both paths default the pin, the 75+110 pinned rows are an option-plumbing slice, not codegen. - TRIVIA: 2026-09-03 six whitespace exhibits confirm upstream preserves authored line structure in both directions (single-line stays single-line, multi-line stays multi-line) while bamts normalizes empties to `{}` and expands the rest; empty declaration bodies excepted (`export class Object {}` expands to two lines). Mechanism decision (reprint-with-trivia vs per-construct rules) pending red-team given source-map column blast radius over 2015 passing rows. - CLASS-EXPAND: 2026-09-03 class half of the trivia split resolved by strict doc-header-aware census (4501 single-unit `.ts` docs; full 17047-section enumeration with every tight line classified by case-source reading): zero single-line classes in any true `.js` output. No verbatim passthrough exists — even untransformed JS outputs reprint (`{}` to `{ }`, `{a}` to `{ a }`); every `{}` keeper is an echo, declaration-only, error, or comment artifact (definitive instrument leaves REVIEW bucket empty). Fix in `emit_class_body_js` (the `else newline` the declaration printer already had) plus byte-exact unit test; exhibit-1 CLI-verified converting. Suites 1804/0 and 598/0, clippy/fmt clean. Row conversions measured at next sweep, not claimed (315 `.ts` cases hold `class X {}`). Untouched with masses: empty objects keep `{}` (438 tight vs 0 spaced, `emit_object` correct); empty enums unobserved TS-side; es5/es3 class+sync-arrow downlevel absent by current error+preserve design (1334 js rows, needs its own lowering slice); amd/umd/system transforms unimplemented (583 js rows); conflict-marker scanner gap (6 js rows); empty `switch` unobserved either way (site untouched). - BLOCK-SPACE: 2026-09-03 same instrument proves empty blocks normalize to `{ }`: 2011 spaced paren-blocks plus 61 arrows/try/static and 243 constructors vs zero true-tight outputs; normalization proven tight-source to spaced-output (`expandoFunctionSymbolProperty` `function inner() {}` to `{ }`). Fix in `emit_block_with_braces` (synthesized unmapped space) and `emit_constructor_js` (second `raw("{}")` site found by its own failing decorator test), plus shape test, mapping-geometry update (close 16 to 17, EOL 17 to 18), and decorator-expectation updates. Suites 1804/0 and 598/0, clippy/fmt clean; CLI verified (expando inner, empty ctor/method). Non-empty single-line bodies are a different rule — PRESERVED, not expanded (292 fn + 579 obj + 3 arrow true outputs): follow-up architecture slice (trivia preservation), not a mirror fix. - ADVISORIES-2026-09-03: devin-blocker on unconditional class expansion REJECTED with the airtight evidence above (cited keepers are echoes: Test.js is declaration-only, node16 esm keeper sits in the echo section; transformed and untransformed JS outputs both expand). Default-advisory expectation of zero single-line functions FALSIFIED (292+579+3 preserved); its census-receipt proposal ADOPTED and it supersedes red-team for the class+block slices. qwen binding check ADOPTED and PASSED (2048 shards, four digests each count 2048, rows 66786 equal the catalog total). qwen digest paradox CONFIRMED by code reading (canonical homes `verification/evidence/` and `proof/completeness-ledger.json` are tracked; merge at completion.rs:972 and rebuild at rebuild.rs:401 demand binding equal HEAD tree; dirty-check rejects untracked): publishing a receipt set or ledger invalidates its own binding by construction, so the standing `ledger rebuild --check` E_SET_MISMATCH is partly structural (remainder is coverage: most obligations have no receipt). The GENERATED_BINDINGS exclusion fix is a verification-architecture change, QUEUED not implemented. qwen plumbing pair STALE: every cited item already landed (project.rs tri-state field/parse/test, program.rs carry, pipeline forward, apply_emit_fields Option, both path forwards); the 1558 decorator/helper rows were already recorded as separate modeling slices. - CTOR-CLOSE: 2026-09-03 non-empty constructor bodies closed with unmapped `raw("}")` while the empty fast path mapped its close; every ES2015 class with a non-empty constructor emitted the short map against the documented every-close-gets-a-segment invariant. One-line fix (`raw_mapped_char_end`), own commit 3c899ec, exact-segment suites referee: 1804/0 and 598/0, clippy/fmt clean. - INTERFACE: 2026-09-03 empty interfaces split by source shape in true declaration outputs (24 tight vs 13 expanded): tight-authored stays tight (`exportAssignedNamespaceIsVisibleInDeclarationEmit` `Bar {}`), multi-line-authored stays expanded (`commentsInterface` `i1 {`). The advisor's zero-either-way pass was wrong on both counts; the existing tight test pin STANDS with an authority citation added. General rule folds into the preservation slice, not a printer fix. - PARADOX-REVISION: 2026-09-03 the fixed-point wording is retracted to stale-by-design. Ignored shard paths never enter the porcelain check, so commit, sweep, and merge at one clean HEAD is paradox-free; staleness on the next commit is the currency design, and f8c28dc shards must not merge at HEAD (completion.rs:973 rejects the binding). What stands: publishing a tracked merged receipt or ledger means the next `--check` needs a same-HEAD re-sweep, and the standing E_SET_MISMATCH remains chiefly coverage (most obligations receiptless). Per-slice targeted recounts keep attribution that batched sweeps lose. - PRESERVATION-CELLS: 2026-09-03 open cells closed by measurement. Catch blocks space (`catch: 15` in the keyword buckets). Multi-line-authored objects show no collapse (177 sources screened, 8 single-line outputs all pair-matched to single-line sources, 2 verified byte-paired). Source-side singles undercount outputs (fn 115 vs 292, obj 220 vs 579) because outputs add synthesized forms (helpers, IIFEs, accessors) — consistent with 1:1 preservation plus synthesis, not reflow. The ES5 synthesized-constructor memory citation is retracted as unverified (no such form in the corpus; the lowering slice pins from data). Design for next slice: source-range newline check per non-empty body/object construct; classes stay always-expand, empty blocks always-space, empty objects always-tight. - PLUMBING: 2026-09-03 scout-mapped end to end: useDefineForClassFields is dropped on BOTH paths (CLI: no CompilerOptions field, project.rs:918; worker: build_tsconfig emits lowercase string-quoted key the case-sensitive lookup misses, plus no field), so both default to target and agree by coincidence — census valid, no contamination. Fix: Option field + parse arm + accessor (project.rs), new apply_emit_fields param set-when-Some (emitter.rs), pass-through (effective.rs, pipeline.rs), ResolvedProgram carry (program.rs, shared root, window required), build_tsconfig bool list + remap (check_cells.rs). 75 pinned-true rows below es2022 plus 110 pinned-false at es2022+ ride this one fix. alwaysStrict worker-pins-true vs CLI-derives-from-strict divergence noted, out of slice. - PLUMBING-RESULTS: 2026-09-03 fix landed c17ab5e (window acquired with delivered receipt, inbox empty throughout, released with pre/post hashes; suites compiler 1802/0 verification 598/0, clippy and fmt clean): camelCase pin produces defineProperty emit at CLI while lowercase keys sit in raw and are silently ignored (tsc errors TS5023; unknown-key tolerance is by design per project.rs:915, case gap noted). 88-row divergent set pre/post diff: 1 full conversion (override18 FIELD_INIT to satisfied), 4 mode-advanced with the next real delta exposed (2 comment, 1 decorator, 1 temp-naming), 2 relabeled by ref-side token noise in the bucket rule (zero void-0 in either emit), 62 unchanged (pin-irrelevant code or agreed defaults). Sweep effect pending worker rebuild. Follow-ups, same mechanism: import_helpers field exists but unplumbed on both paths (877 helper-policy rows). Separate modeling slices: experimentalDecorators/emitDecoratorMetadata (516/119 rows, no fields), downlevelIteration (46), isolatedModules (82). - SWEEP-FRESH: 2026-09-04 full 2048-shard resweep at 89b0455 with the pre-flighted HEAD worker (repo target rebuilt without the CARGO_TARGET_DIR override; header binary digest asserted equal before launch; Node 24.18.0; run sweep-89b0455-fresh): acceptance 2048 headers uniform on all four digests, 66786 rows, 0 logged SHARD_FAIL lines; 130 stranded tmp orphans accumulated across the three driver launches (cleaned post-acceptance). PASS 12354 to 13127 (+773), BLOCKING_FAIL down equally, 0 regressions, all flips in javascript (2015 to 2788, +38%). Exhibit-1 es2015 row flips FAIL to PASS at worker level (class rule causally pinned); its es5 twin stays failing by the recorded downlevel scope. Mechanism cuts are co-occurrence, batch-measured together, not isolated: tight-empty sources 23 class-pattern + 61 block-pattern (disjoint), 8 explicitly pinned rows, spaced/multi-line-authored empties dominate the remainder with field-init Assign rows, 47 residual without an isolated pattern (possibly export-synthesis side effects of the field-init guard). The first sweep-89b0455 run measured a stale 16:28 worker (zero flips, correctly discarded, never claimed). - INTERFACE-REVISION: 2026-09-04 the 89b0455 INTERFACE line is superseded: true declaration outputs ALWAYS expand empty interfaces (0 tight / 21 expand; importTag16 a.d.ts expands tight-authored `export interface I {}`). All 24 tights traced to d.ts input copies (Bar thing.d.ts echo sole occurrence; ISpinButton cross-dir stem collision). Fix is a caller-side pre-check in `emit_interface_decl` (expand + return); shared `emit_type_members_block` stays tight because empty OBJECT TYPES stay tight (mappedTypes1 `x1: {}`, es6Import `_default: {}` transformation-proven, 0 expands observed in the true-output census). Test flipped to expanded with the F415-byte citation (the 3c899ec preservation citation was wrong). Suites 1804/0 and 598/0, clippy/fmt clean. - PRESERVATION-BLOCK: 2026-09-04 block-body gate landed (8a3d98c): non-empty bodies authored without a line break keep `{ stmt; stmt; }` (292-function + 3-arrow families); synthesized, past-authored-text, or multi-line-authored bodies expand. Machinery: `spans_multiple_lines` (source.rs), rewriter taint set with child-mark climb (transforms.rs), `preserves_single_line` with taint/length/mint gates (emitter.rs). Window variant dropped (no consumer after the red-pen cut). Suites 1814/0 and 598/0, clippy/fmt clean. Map test restored to its multi-line authority fixture plus a new single-line method map test (hand-decoded golden: greet(4) return(14) 1(21) ;(22) }(24) EOL(25), class-close EOL(1) unchanged). CLI spots: single-line fixture emits `function f() { return 1; }`; expandoFunctionSymbolProperty emits `function inner() { }` (spaced guard holds end to end); pre/post-gate outputs byte-identical on both authority files. Limitation: mappedTypes1's x1 resolves to `any` in our checker, so no CLI run exercised the `{}`-tight object path (pinned by unit suite only); tsc bytes are `declare let x1: {};` at mappedTypes1.js:139, declaration embedded in the .js baseline. JIT `provision_lock_rejects_overlapping_attempts` failed once then passed solo and full rerun: flake, not the tree. Sweep restated: run sweep-89b0455-fresh, binary digest 0fa9235d, +773 with 84 mechanism-indicative (23 class-only + 61 block-only, disjoint), remainder co-occurrence. Follow-ups: ctor and object-literal gates (approved, sequenced next). - PRESERVATION-CTOR-OBJECT: 2026-09-04 ctor and object gates landed (e6e1f70): injection-free ctor bodies authored on one line keep `{ stmt; }` (class still expands around them); non-empty object literals authored on one line join as `{ a: 1, b: 2 }` with uniform spacing (tight-authored `{a:1}` normalizes). Parameter-property injections always expand. Suites 1819/0 and 598/0, clippy/fmt clean. CLI spots: `const o = { a: 1, b: 2 };` and `constructor() { console.log(1); }` single-line end to end. Next: sweep re-measurement of the 292/579/3 families. - SWEEP-PRESERVATION: 2026-09-04 full 2048-shard resweep at 2d76b09 (run sweep-2d76b09-preservation, worker binary 9f8c8ce9 asserted pre-flight, clean-tree digest uniform, Node 24.18.0 pinned via PATH; one bash-timeout driver restart, resume skipped footered shards): acceptance 2048/2048 footers, 0 SHARD_FAIL, 0 obligation drift (66786 rows both sides). PASS 13127 -> 13512 (+385), BLOCKING_FAIL 46653 -> 46268 (-385), INAPPLICABLE_LANGUAGE_SERVICE 6693 / INAPPLICABLE_CATALOG_ERROR 312 / TIMEOUT 1 all unchanged (controls). Transitions: 387 BLOCKING_FAIL->PASS, 2 PASS->BLOCKING_FAIL. Churn: compiler/tests/cases/compiler 231, conformance node 16, computedProperties 11, destructuring 10, Accessors 10, templates 8. Range e03e5de..2d76b09 is campaign-level (pin, empty class/block spacing, ctor-close, interface revision, block + ctor/object single-line preservation), not commit-isolated. Regressions, both javascript facet aot mode: override18.ts [noimplicitoverride=true,target=es2015,usedefineforclassfields=true#javascript] and parser768531.ts [allowunusedlabels=true,target=es2015#javascript], each PASS (empty detail) -> BLOCKING_FAIL `javascript structural mismatch after formatting/order normalization`; override18 was the c17ab5e FIELD_INIT conversion (PLUMBING-RESULTS), so the preservation range re-broke it; byte-diff attribution queued (worker request schema: binding, request_id u64, key, observables, argv, working_directory, timeout_ms). Tool local/diff-sweeps.py self-tested (baseline-vs-itself 0 transitions); full list local/sweep-2d76b09-transitions.txt; receipts .outline/evidence-sweep-2d76b09-preservation/. - SWEEP-PRESERVATION-AUDIT: 2026-09-04 post-push four-digest audit of all 2048 headers under .outline/evidence-sweep-2d76b09-preservation/: exactly one distinct (authority 47d60ce5, candidate_binary 9f8c8ce9, candidate_tree bd95c396, harness 9f323e56) tuple, covering the resume phase too. The entry's uniformity claim is now command-backed, not launch-time inferred. - REGRESSION-CLOSURE: 2026-09-04 both SWEEP-PRESERVATION regressions root-caused via CLI byte-diff (tsconfig-driven scratch projects mirroring build_tsconfig, strict:true) and fixed. parser768531: the block gate preserved statement-level blocks, but its authority is function-family bodies only — emit_block_with_braces now expands whenever map_open_brace is set (the same context bit that splits brace mapping), commit 2e0e39d. override18: object_literal minted through the untainted node() helper so wholly synthesized defineProperty descriptors collapsed single-line — both mints now record taint, commit 8806887. Post-fix CLI emit matches each baseline's .js section byte-for-byte (wrapper sections aside, which the harness strips). Suites 1821/0 (2 new regression tests) and 598/0, fmt/clippy clean. Next sweep at this HEAD should show 389 net conversions with zero regressions; re-measure before claiming. - SWEEP-ZERO-REGRESSION: 2026-09-04 confirmation sweep at e8b93bc (run sweep-e8b93bc-preservation, binary e3060e58 pre-flighted, tree 0b2742b6, four-digest audit 1 distinct tuple over 2048 headers, 0 SHARD_FAIL, 0 obligation drift): PASS 13514, BLOCKING_FAIL 46266, controls unchanged. Diff vs 2d76b09: exactly +2 (override18, parser768531 restored), 0 regressions. Diff vs 89b0455-fresh: 387 BLOCKING_FAIL->PASS, 0 PASS->BLOCKING_FAIL — the preservation campaign (e03e5de..e8b93bc) is strictly net-positive. Interim 217b37d sweep caught classThisReference breaking (static init blocks are brace-mapped yet layout-preserving); BlockLayout enum (e8b93bc) names the three contexts, verified byte-for-byte against all three baselines. Receipts .outline/evidence-sweep-e8b93bc-preservation/; transition lists local/sweep-e8b93bc-vs-{2d76b09,89b0455}.txt. - ID-FLOOR-STRUCTURAL: 2026-09-04 rewriter id seeding rederived from the file's real high-water mark, commit 139656d: one NodeIdSource flows desugar-into-rewriter (alloc_id = ids.fresh), Rewriter::new snapshots the seed as the per-file synthesized floor, PrintSource carries it, and the mint-gate range check reads per-file truth; SYNTHESIZED_ID_FLOOR (the 2d76b09 hazard) is deleted, untransformed prints pass u32::MAX so the range check applies universally. Disjointness of minted vs parser ids is now structural (no constant to cross); the dense-file test pins observables at 1M+ nodes (emit completes, no statement drops, authored single-line preservation holds, descriptor lowering mints). Mutation note: a black-box catch of the old floor is not cheaply constructible — the collision point depends on the unmeasured mint-per-statement consumption ratio (authored id ≈ 3k vs mint position ≈ 1M+k collide near k ≈ 500k only for the right ratio) — so the structural invariant is the guarantee; measuring that ratio and placing a probe is doable future work if a behavioral pin is ever wanted. Suites 1824/0 and 598/0, fmt/clippy/diagnostics clean. - BUILD-INFO-FACET: 2026-09-04 the 10-cell build-info facet re-derived against authority: zero baselines in the entire 7.0.2 reference tree carry a `//// [name.tsbuildinfo]` section (the harness drops the artifact as environment dependent), so content parity is unprovable and the prior "producer missing" HARNESS_ERROR was a dead end by construction. New contract is emission-shape: the producer emits only when incremental or composite is set (bare tsBuildInfoFile writes nothing, matching tsc's optionsTsBuildInfoFileWithoutIncrementalAndComposite), and the observer's four arms demand artifact-exactly-when-implied (696f9af); the earlier non-empty check was a tautology every cell passed — caught before commit, never landed. Output paths now accept absolute and root-escaping values (resolve_output_from, 413796f): tsc runs incrementalTsBuildInfoFile clean on /a.tsbuildinfo while source paths keep confinement. Verified by direct shard runs at clean HEAD: incrementalOut, incrementalConfig, incrementalTsBuildInfoFile (implied+produced) and optionsTsBuildInfoFileWithoutIncrementalAndComposite (not-implied+none, self-describing artifact) all PASS; 4/10 cells sampled across both arms and both path families, remaining 6 ride the next sweep. Suites 1824/0 and 598/0, fmt/clippy clean. + STATUS: progress is not recorded in this file. Run `just status-report`; it writes target/evidence-local/status.md from the receipts. Dated notes that used to be here are in .outline/notes/gates-progress-2026-09.md. - [ ] G4: The bamti Node 24 package implements tsc and all thirteen stable native-preview export subpaths, including API-reachable language-service behavior CHECK: cargo run -p bamts-verification -- completion verify --root package @@ -58,7 +30,7 @@ Scope: Complete the clean-room Rust compiler, TypeScript-compatible Node 24 API - [ ] G6: Interpreter, JIT, AOT, formal correspondence, target cells, and performance evidence satisfy every compiler-product release obligation CHECK: cargo run -p bamts-verification -- completion verify --root native - EXPECT: NATIVE COMPLETE formal=PASS targets=28/28 benchmarks=9/9 + EXPECT: NATIVE COMPLETE formal=PASS targets=49/49 benchmarks=9/9 EVIDENCE: 2026-08-28 `completion verify --root native` exit 1: unproven E1.1, E1.2, E2.1 through E2.3, E3.1 through E3.3, E4.1, E4.2, and 389 more - [ ] G7: The strict TypeScript-product completion gate accepts only receipt-backed PASS states or exact machine-evidenced policy exclusions diff --git a/.outline/gates/A3.1.md b/.outline/gates/A3.1.md index 107d95c3..d3068a6e 100644 --- a/.outline/gates/A3.1.md +++ b/.outline/gates/A3.1.md @@ -30,4 +30,4 @@ EVIDENCE: 2026-09-03 sweep complete: 2048/2048 shards merged to verification/evi - [ ] G6: Baseline generation and ledger rebuild pass their narrow repository checks CHECK: just leaf-gate A3.1 EXPECT: LEAF A3.1 PASS -EVIDENCE: 2026-09-03 structurally blocked, measured two ways: ledger rebuild --write exit 1 E_SET_MISMATCH with 248828 rejections (merged receipt StaleBinding on candidate_tree_digest plus unreceipted benchmark/formal/test262 obligations); re-running suite merge on 0b5bdde exit 1 E_DIGEST, shards stale for the current binding. Landing a receipt changes HEAD^{tree} which the receipt binds, so no committed receipt can be admitted; guards at crates/bamts-verification/src/rebuild.rs:613-624 plus the dirty-tree gate (28dc143, postdates ledger 5e83845). A2.6 G3 tests rejects_stale_receipt as a met gate, so the guard is intentional and must not be relaxed to force a green; the landing-sequence change belongs to its owners. +EVIDENCE: 2026-11-01 no longer structurally blocked: at f640f63 a target-cells receipt was measured, committed under verification/evidence/typescript-7.0.2/ in the next commit, and `ledger rebuild --check` there rejected only the older x86_64 receipt as stale, not it; one source change then made it stale too (355473 to 355474 rejections). The gate stays open until `just leaf-gate A3.1` passes. Earlier record: 2026-09-03 structurally blocked, measured two ways: ledger rebuild --write exit 1 E_SET_MISMATCH with 248828 rejections (merged receipt StaleBinding on candidate_tree_digest plus unreceipted benchmark/formal/test262 obligations); re-running suite merge on 0b5bdde exit 1 E_DIGEST, shards stale for the current binding. Landing a receipt changes HEAD^{tree} which the receipt binds, so no committed receipt can be admitted; guards at crates/bamts-verification/src/rebuild.rs:613-624 plus the dirty-tree gate (28dc143, postdates ledger 5e83845). A2.6 G3 tests rejects_stale_receipt as a met gate, so the guard is intentional and must not be relaxed to force a green; the landing-sequence change belongs to its owners. diff --git a/.outline/gates/B7.2.md b/.outline/gates/B7.2.md new file mode 100644 index 00000000..31f7930d --- /dev/null +++ b/.outline/gates/B7.2.md @@ -0,0 +1,33 @@ +# Gates: B7.2 historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter + +Scope: B7.2 owns crates/bamts-verification/src/historical.rs; it delivers historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter. + +- [ ] G1: historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter: the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf B7.2 --aspect contract + EXPECT: LEAF B7.2 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter: every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf B7.2 --aspect evidence + EXPECT: LEAF B7.2 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter: the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf B7.2 --aspect coverage + EXPECT: LEAF B7.2 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter: no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf B7.2 --aspect regression + EXPECT: LEAF B7.2 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter: the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf B7.2 --aspect mutation + EXPECT: LEAF B7.2 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter: the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate B7.2 + EXPECT: LEAF B7.2 PASS + EVIDENCE: pending diff --git a/.outline/gates/E2.4.md b/.outline/gates/E2.4.md new file mode 100644 index 00000000..a3670f04 --- /dev/null +++ b/.outline/gates/E2.4.md @@ -0,0 +1,33 @@ +# Gates: E2.4 JIT execution and W^X proof on Apple Silicon + +Scope: E2.4 owns crates/bamts-codegen/src/jit_memory.rs; it delivers JIT execution and W^X proof on Apple Silicon. + +- [ ] G1: JIT execution and W^X proof on Apple Silicon: the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf E2.4 --aspect contract + EXPECT: LEAF E2.4 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: JIT execution and W^X proof on Apple Silicon: every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf E2.4 --aspect evidence + EXPECT: LEAF E2.4 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: JIT execution and W^X proof on Apple Silicon: the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf E2.4 --aspect coverage + EXPECT: LEAF E2.4 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: JIT execution and W^X proof on Apple Silicon: no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf E2.4 --aspect regression + EXPECT: LEAF E2.4 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: JIT execution and W^X proof on Apple Silicon: the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf E2.4 --aspect mutation + EXPECT: LEAF E2.4 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: JIT execution and W^X proof on Apple Silicon: the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate E2.4 + EXPECT: LEAF E2.4 PASS + EVIDENCE: pending diff --git a/.outline/gates/E3.4.md b/.outline/gates/E3.4.md new file mode 100644 index 00000000..dffc750c --- /dev/null +++ b/.outline/gates/E3.4.md @@ -0,0 +1,33 @@ +# Gates: E3.4 native executable link and run on macOS hosts + +Scope: E3.4 owns crates/bamts-cli/src/driver.rs; it delivers native executable link and run on macOS hosts. + +- [ ] G1: native executable link and run on macOS hosts: the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.4 --aspect contract + EXPECT: LEAF E3.4 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: native executable link and run on macOS hosts: every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.4 --aspect evidence + EXPECT: LEAF E3.4 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: native executable link and run on macOS hosts: the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.4 --aspect coverage + EXPECT: LEAF E3.4 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: native executable link and run on macOS hosts: no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.4 --aspect regression + EXPECT: LEAF E3.4 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: native executable link and run on macOS hosts: the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.4 --aspect mutation + EXPECT: LEAF E3.4 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: native executable link and run on macOS hosts: the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate E3.4 + EXPECT: LEAF E3.4 PASS + EVIDENCE: pending diff --git a/.outline/gates/E3.5.md b/.outline/gates/E3.5.md new file mode 100644 index 00000000..18206d03 --- /dev/null +++ b/.outline/gates/E3.5.md @@ -0,0 +1,35 @@ +# Gates: E3.5 native executable link and run on Windows MSVC + +Scope: E3.5 owns crates/bamts-cli/src/windows_link.rs; it delivers native executable link and run on Windows MSVC. + +Status: the owned file does not exist at this commit. `crates/bamts-cli/src/driver.rs` still links with `cc ... -o`, which is not an MSVC command line, and the platform crate has no Windows implementation, so native output is unwritten. It is the deliverable of the `windows-native-output` card, which needs the Windows host. Until then this leaf stays open and ownership checks report the file as missing. + +- [ ] G1: native executable link and run on Windows MSVC: the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.5 --aspect contract + EXPECT: LEAF E3.5 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: native executable link and run on Windows MSVC: every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.5 --aspect evidence + EXPECT: LEAF E3.5 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: native executable link and run on Windows MSVC: the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.5 --aspect coverage + EXPECT: LEAF E3.5 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: native executable link and run on Windows MSVC: no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.5 --aspect regression + EXPECT: LEAF E3.5 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: native executable link and run on Windows MSVC: the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.5 --aspect mutation + EXPECT: LEAF E3.5 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: native executable link and run on Windows MSVC: the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate E3.5 + EXPECT: LEAF E3.5 PASS + EVIDENCE: pending diff --git a/.outline/gates/E3.6.md b/.outline/gates/E3.6.md new file mode 100644 index 00000000..b9c41446 --- /dev/null +++ b/.outline/gates/E3.6.md @@ -0,0 +1,33 @@ +# Gates: E3.6 shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host + +Scope: E3.6 owns crates/bamts-platform/; it delivers shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host. + +- [ ] G1: shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host: the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.6 --aspect contract + EXPECT: LEAF E3.6 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host: every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.6 --aspect evidence + EXPECT: LEAF E3.6 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host: the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.6 --aspect coverage + EXPECT: LEAF E3.6 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host: no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.6 --aspect regression + EXPECT: LEAF E3.6 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host: the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.6 --aspect mutation + EXPECT: LEAF E3.6 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host: the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate E3.6 + EXPECT: LEAF E3.6 PASS + EVIDENCE: pending diff --git a/.outline/gates/E3.7.md b/.outline/gates/E3.7.md new file mode 100644 index 00000000..5492432d --- /dev/null +++ b/.outline/gates/E3.7.md @@ -0,0 +1,33 @@ +# Gates: E3.7 in-process AOT linking on every execution host + +Scope: E3.7 owns crates/bamts-codegen/src/aot/in_process_link.rs; it delivers in-process AOT linking on every execution host. + +- [ ] G1: in-process AOT linking on every execution host: the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.7 --aspect contract + EXPECT: LEAF E3.7 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: in-process AOT linking on every execution host: every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.7 --aspect evidence + EXPECT: LEAF E3.7 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: in-process AOT linking on every execution host: the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.7 --aspect coverage + EXPECT: LEAF E3.7 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: in-process AOT linking on every execution host: no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.7 --aspect regression + EXPECT: LEAF E3.7 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: in-process AOT linking on every execution host: the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf E3.7 --aspect mutation + EXPECT: LEAF E3.7 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: in-process AOT linking on every execution host: the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate E3.7 + EXPECT: LEAF E3.7 PASS + EVIDENCE: pending diff --git a/.outline/gates/E4.2.md b/.outline/gates/E4.2.md index aad42efc..d802b030 100644 --- a/.outline/gates/E4.2.md +++ b/.outline/gates/E4.2.md @@ -2,6 +2,8 @@ Scope: E4.2 owns verification/toolchains/x86_64-unknown-linux-gnu.json; it delivers x86_64-unknown-linux-gnu cell closure (native host). +Status: the owned file does not exist at this commit. Commit 1352e7b removed it because its obligations were renamed, and evidence must be re-measured, not rewritten. A native `x86_64` `rustc` has to measure it again, on the `bamti-x64` VM profile. Until then this leaf stays open and ownership checks report the file as missing. + - [ ] G1: x86_64-unknown-linux-gnu cell closure (native host): the declared public contract vectors pass CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.2 --aspect contract EXPECT: LEAF E4.2 CONTRACT PASS diff --git a/.outline/gates/E4.6.md b/.outline/gates/E4.6.md new file mode 100644 index 00000000..cfd33d68 --- /dev/null +++ b/.outline/gates/E4.6.md @@ -0,0 +1,33 @@ +# Gates: E4.6 aarch64-apple-darwin cell (native host) + +Scope: E4.6 owns verification/toolchains/aarch64-apple-darwin.json; it delivers aarch64-apple-darwin cell (native host). + +- [ ] G1: aarch64-apple-darwin cell (native host): the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.6 --aspect contract + EXPECT: LEAF E4.6 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: aarch64-apple-darwin cell (native host): every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.6 --aspect evidence + EXPECT: LEAF E4.6 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: aarch64-apple-darwin cell (native host): the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.6 --aspect coverage + EXPECT: LEAF E4.6 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: aarch64-apple-darwin cell (native host): no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.6 --aspect regression + EXPECT: LEAF E4.6 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: aarch64-apple-darwin cell (native host): the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.6 --aspect mutation + EXPECT: LEAF E4.6 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: aarch64-apple-darwin cell (native host): the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate E4.6 + EXPECT: LEAF E4.6 PASS + EVIDENCE: pending diff --git a/.outline/gates/E4.7.md b/.outline/gates/E4.7.md new file mode 100644 index 00000000..8cf85fda --- /dev/null +++ b/.outline/gates/E4.7.md @@ -0,0 +1,35 @@ +# Gates: E4.7 x86_64-apple-darwin cell (Rosetta) + +Scope: E4.7 owns verification/toolchains/x86_64-apple-darwin.json; it delivers x86_64-apple-darwin cell (Rosetta). + +Status: the owned file does not exist at this commit. The record is measured by `toolchain measure --target x86_64-apple-darwin`, whose two run obligations need Rosetta, and Rosetta is not installed on the measuring Mac. That is the human task `install-rosetta`, then the `macos-target-cells` card. Until then this leaf stays open and ownership checks report the file as missing. + +- [ ] G1: x86_64-apple-darwin cell (Rosetta): the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.7 --aspect contract + EXPECT: LEAF E4.7 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: x86_64-apple-darwin cell (Rosetta): every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.7 --aspect evidence + EXPECT: LEAF E4.7 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: x86_64-apple-darwin cell (Rosetta): the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.7 --aspect coverage + EXPECT: LEAF E4.7 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: x86_64-apple-darwin cell (Rosetta): no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.7 --aspect regression + EXPECT: LEAF E4.7 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: x86_64-apple-darwin cell (Rosetta): the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.7 --aspect mutation + EXPECT: LEAF E4.7 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: x86_64-apple-darwin cell (Rosetta): the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate E4.7 + EXPECT: LEAF E4.7 PASS + EVIDENCE: pending diff --git a/.outline/gates/E4.8.md b/.outline/gates/E4.8.md new file mode 100644 index 00000000..811778d8 --- /dev/null +++ b/.outline/gates/E4.8.md @@ -0,0 +1,35 @@ +# Gates: E4.8 x86_64-pc-windows-msvc cell (native host) + +Scope: E4.8 owns verification/toolchains/x86_64-pc-windows-msvc.json; it delivers x86_64-pc-windows-msvc cell (native host). + +Status: the owned file does not exist at this commit. Measuring it needs a Windows host and a Windows linker path, which are the `windows-host` and `windows-native-output` cards. Until then this leaf stays open and ownership checks report the file as missing. + +- [ ] G1: x86_64-pc-windows-msvc cell (native host): the declared public contract vectors pass + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.8 --aspect contract + EXPECT: LEAF E4.8 CONTRACT PASS + EVIDENCE: pending + +- [ ] G2: x86_64-pc-windows-msvc cell (native host): every selected logical obligation has a current content-addressed receipt + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.8 --aspect evidence + EXPECT: LEAF E4.8 EVIDENCE PASS + EVIDENCE: pending + +- [ ] G3: x86_64-pc-windows-msvc cell (native host): the scoped obligation set has zero blocking, external-blocked, catalog-error, timeout, crash, skip, or missing-receipt states + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.8 --aspect coverage + EXPECT: LEAF E4.8 COVERAGE PASS + EVIDENCE: pending + +- [ ] G4: x86_64-pc-windows-msvc cell (native host): no previously passing obligation regressed + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.8 --aspect regression + EXPECT: LEAF E4.8 REGRESSION PASS + EVIDENCE: pending + +- [ ] G5: x86_64-pc-windows-msvc cell (native host): the registered mutation makes its contract gate fail + CHECK: cargo run -p bamts-verification -- completion verify --leaf E4.8 --aspect mutation + EXPECT: LEAF E4.8 MUTATION PASS + EVIDENCE: pending + +- [ ] G6: x86_64-pc-windows-msvc cell (native host): the owned code passes its narrow repository build, lint, format, and behavioral checks + CHECK: just leaf-gate E4.8 + EXPECT: LEAF E4.8 PASS + EVIDENCE: pending diff --git a/.outline/gates/root-native.md b/.outline/gates/root-native.md index ec6bb56e..11c73735 100644 --- a/.outline/gates/root-native.md +++ b/.outline/gates/root-native.md @@ -14,7 +14,7 @@ Scope: Track E native tiers, target cells, and performance with Track F formal e - [ ] G3: The native root has no incomplete state CHECK: cargo run -p bamts-verification -- completion verify --root native - EXPECT: NATIVE COMPLETE formal=PASS targets=28/28 benchmarks=9/9 + EXPECT: NATIVE COMPLETE formal=PASS targets=49/49 benchmarks=9/9 EVIDENCE: pending - [ ] G4: The native root has no PASS-to-non-PASS transition diff --git a/.outline/notes/gates-progress-2026-09.md b/.outline/notes/gates-progress-2026-09.md new file mode 100644 index 00000000..110579e2 --- /dev/null +++ b/.outline/notes/gates-progress-2026-09.md @@ -0,0 +1,33 @@ +# Progress notes moved out of GATES.md + +These dated notes sat in gate G3's block of `.outline/GATES.md` until 2026-11-01. A gate file states what must hold and how to check it; progress belongs in a report generated from the receipts (`just status-report`, which writes `target/evidence-local/status.md`). They are kept here unchanged because they record how the 2026-09 baseline was measured and what each finding meant. `git log -p -- .outline/GATES.md` has them in place. + + BASELINE: 2026-09-02 `bamts-verification suite run --catalog typescript-7.0.2 --shard i/64` with `BAMTS_SUITE_COMPILER_ADAPTER` set, receipts under `target/evidence-sweep/typescript-7.0.2/`. This is a shard sweep, not the CHECK above. 66786 rows: PASS 8951 (13.4%), BLOCKING_FAIL 51141 (76.6%), INAPPLICABLE_LANGUAGE_SERVICE 6693, TIMEOUT 1. Per-facet PASS: diagnostics 6525/15484, symbols 1846/14621, source-map 8/237, types 440/15918, declaration 61/2215, javascript 71/14096, parse 0/3718, trace 0/487, build-info 0/10. The 2026-08-28 line above reads PASS=0 because no adapter was registered, so every cell recorded `no registered adapter` and nothing was measured. These receipts pre-date 979b78c and 6d1ecc3, so they are a baseline for ranking work, never publishable completion evidence. + FACETS: 2026-09-02 the per-facet figures above are `PASS/total`, which reads every non-PASS row as outstanding work and is wrong for any facet carrying inapplicable rows. Cross-tabulating the same receipts by facet and state gives the applicable denominator, `PASS + BLOCKING_FAIL`. javascript 71/14060 pass, 13989 blocking, 36 inapplicable. types 440/13950, 13510 blocking, 1967 inapplicable. symbols 1846/13951, 12105 blocking, 670 inapplicable. diagnostics 6525/15182, 8657 blocking, 302 inapplicable. declaration 61/2215, 2154 blocking. trace 0/487, 487 blocking. source-map 8/237, 229 blocking. build-info 0/10, 10 blocking. parse has zero applicable rows: all 3718 are fourslash language-service cases and all are INAPPLICABLE_LANGUAGE_SERVICE, so `parse 0/3718` names an exclusion, never a gap. Pass share of applicable rows is 8951/60092 = 14.9%; the 13.4% above divides by all 66786 rows including exclusions. + RANKING: 2026-09-02 blocking mass by facet, which is the work order: javascript 13989, types 13510, symbols 12105, diagnostics 8657, declaration 2154, trace 487, source-map 229, build-info 10. Two facets have zero passing applicable rows: trace, whose largest single family is 78 rows of `classification/execution drift: no owned .trace.json baseline` and 31 of `invalid project trace fixture: missing field inputFiles`, and build-info, which has no observer function at all. Classification drift is not systemic: 218 of 51141 blocking rows (0.4%) carry it, over missing `.js` (127), `.trace.json` (78), `.js.map` (9), `.types` (2), and `.symbols` (2) baselines, so the blocking count stands essentially as measured. + JAVASCRIPT: 2026-09-02 12753 of 13989 javascript blocking rows (91.2%) carry one opaque detail, `javascript structural mismatch after formatting/order normalization`, which names no delta. Compiling 112 of them at HEAD and diffing against the reference `.js` classifies them as 72 reshape, 24 dropped-lines, 3 changed-lines, 12 emitting no `.js`, 1 identical, and every sampled family's first delta is the same missing `"use strict";` prologue. Extraction caveat for whoever re-measures: a reference baseline's last `//// [name]` section is often the `.d.ts`, so the section whose name ends in `.js` must be selected by name; taking the last section silently compares against declaration text and contaminated an earlier 86% figure. + PROLOGUE: 2026-09-02 measured over 10781 single-file baselines holding a real `.js` section. A file that is not an external module carries the prologue in 8644 cases and omits it in 82 (99.1%). An external module carries it in 1241 and omits it in 814, and the carriers concentrate in the script-format module kinds: `commonjs` 408 and a defaulted `module` 730. `crates/bamts-compiler/src/emitter/transforms.rs` gates emission on `always_strict || module_kind_implies_strict`, so it agrees on external modules and diverges on non-modules whenever `alwaysStrict` resolves false: 1437 of the 1597 baselines declaring `@strict: false` carry the prologue that gate omits. That is the single largest grounded javascript conversion available and it is not yet implemented. + JAVASCRIPT: 2026-09-03 v2 sweep (2048/2048 shards, lane binary on f8c28dc with the prologue implementation): javascript PASS 2015/14096 (14.3%), BLOCKING_FAIL 12045, up from 71/14060 on 2026-09-02. The +1944 spans the whole emitter campaign in range (prologue 93cc1eb/4756929 plus parens print 67790b5, declare/export align f36bbbc, the sourcemap series, jsdoc c51f49e, modern-js lowering ef67a7d), so it is not prologue-isolated. + PROLOGUE: 2026-09-03 residual sample at release binary 6cd8e0e (seed 20260903, N=300 of 12045 BLOCKING_FAIL javascript rows, row options from each receipt configuration plus harness strict/alwaysStrict defaults): 207 compared with 0 prologue deltas (95% upper bound about 1.4%); skips are multi-unit 69, no-reference 14, outfile-config 2, ambiguous-reference 1, missing .js section 2; 5 no-emit CLI gaps on exotic configs (system/isolatedModules/jsx, silent rc=1). Committed prologue unit tests pass 6/6. The prologue rule is done at HEAD within the comparable population; the residual is other families. + RESIDUAL: 2026-09-03 layered whole-document census at release binary (seed 600613, N=400 of 12045 BLOCKING_FAIL javascript rows; replicator calibrated EQUAL on 57 comparable PASS rows of 60 sampled (2 declaration-config + 1 multi-unit skipped), validating pairing, echo-strip, and framing assembly on those rows): WHITESPACE_ONLY 54, FIELD_INIT 29, COMMENT 27, DECORATOR 14, EXPORT_ELISION 2, OTHER_CODE 133, WOULD_PASS 5, COMPILE_FAIL 3; skips multi-unit 50, declaration-config 53, non-ts-source 25, outfile-config 5. The 5 WOULD_PASS rows (replication passes, sweep fails; 4 with sourcemap=true) are unexplained divergences with a worker-vs-CLI audit queued, so CLI-vs-worker equivalence is NOT established. No framing-only rows exist because the comparator's blank-sensitive document compare surfaces framing blanks as content diffs. Work order by mass: brace/paren whitespace shape, spurious field-init synthesis (`this.x = void 0`), comment drops, legacy decorator strategy. + FIELD_INIT: 2026-09-03 unconditional void-0 synthesis dropped under Assign semantics (b146af3; transforms.rs leaf-owned, no window needed; the CLI emit mapping takes 9 fixed keys with no useDefineForClassFields entry, so all knob runs defaulted and the construction default was unconditionally mode-blind): census bucket 29/400 to 8/400, remainder temp-naming and module-transform artifacts; headline baseline computedPropertyNames36_ES5 verified at CLI; suite 1801/0, clippy clean. Worker-side mapping under scout investigation; if both paths default the pin, the 75+110 pinned rows are an option-plumbing slice, not codegen. + TRIVIA: 2026-09-03 six whitespace exhibits confirm upstream preserves authored line structure in both directions (single-line stays single-line, multi-line stays multi-line) while bamts normalizes empties to `{}` and expands the rest; empty declaration bodies excepted (`export class Object {}` expands to two lines). Mechanism decision (reprint-with-trivia vs per-construct rules) pending red-team given source-map column blast radius over 2015 passing rows. + CLASS-EXPAND: 2026-09-03 class half of the trivia split resolved by strict doc-header-aware census (4501 single-unit `.ts` docs; full 17047-section enumeration with every tight line classified by case-source reading): zero single-line classes in any true `.js` output. No verbatim passthrough exists — even untransformed JS outputs reprint (`{}` to `{ }`, `{a}` to `{ a }`); every `{}` keeper is an echo, declaration-only, error, or comment artifact (definitive instrument leaves REVIEW bucket empty). Fix in `emit_class_body_js` (the `else newline` the declaration printer already had) plus byte-exact unit test; exhibit-1 CLI-verified converting. Suites 1804/0 and 598/0, clippy/fmt clean. Row conversions measured at next sweep, not claimed (315 `.ts` cases hold `class X {}`). Untouched with masses: empty objects keep `{}` (438 tight vs 0 spaced, `emit_object` correct); empty enums unobserved TS-side; es5/es3 class+sync-arrow downlevel absent by current error+preserve design (1334 js rows, needs its own lowering slice); amd/umd/system transforms unimplemented (583 js rows); conflict-marker scanner gap (6 js rows); empty `switch` unobserved either way (site untouched). + BLOCK-SPACE: 2026-09-03 same instrument proves empty blocks normalize to `{ }`: 2011 spaced paren-blocks plus 61 arrows/try/static and 243 constructors vs zero true-tight outputs; normalization proven tight-source to spaced-output (`expandoFunctionSymbolProperty` `function inner() {}` to `{ }`). Fix in `emit_block_with_braces` (synthesized unmapped space) and `emit_constructor_js` (second `raw("{}")` site found by its own failing decorator test), plus shape test, mapping-geometry update (close 16 to 17, EOL 17 to 18), and decorator-expectation updates. Suites 1804/0 and 598/0, clippy/fmt clean; CLI verified (expando inner, empty ctor/method). Non-empty single-line bodies are a different rule — PRESERVED, not expanded (292 fn + 579 obj + 3 arrow true outputs): follow-up architecture slice (trivia preservation), not a mirror fix. + ADVISORIES-2026-09-03: devin-blocker on unconditional class expansion REJECTED with the airtight evidence above (cited keepers are echoes: Test.js is declaration-only, node16 esm keeper sits in the echo section; transformed and untransformed JS outputs both expand). Default-advisory expectation of zero single-line functions FALSIFIED (292+579+3 preserved); its census-receipt proposal ADOPTED and it supersedes red-team for the class+block slices. qwen binding check ADOPTED and PASSED (2048 shards, four digests each count 2048, rows 66786 equal the catalog total). qwen digest paradox CONFIRMED by code reading (canonical homes `verification/evidence/` and `proof/completeness-ledger.json` are tracked; merge at completion.rs:972 and rebuild at rebuild.rs:401 demand binding equal HEAD tree; dirty-check rejects untracked): publishing a receipt set or ledger invalidates its own binding by construction, so the standing `ledger rebuild --check` E_SET_MISMATCH is partly structural (remainder is coverage: most obligations have no receipt). The GENERATED_BINDINGS exclusion fix is a verification-architecture change, QUEUED not implemented. qwen plumbing pair STALE: every cited item already landed (project.rs tri-state field/parse/test, program.rs carry, pipeline forward, apply_emit_fields Option, both path forwards); the 1558 decorator/helper rows were already recorded as separate modeling slices. + CTOR-CLOSE: 2026-09-03 non-empty constructor bodies closed with unmapped `raw("}")` while the empty fast path mapped its close; every ES2015 class with a non-empty constructor emitted the short map against the documented every-close-gets-a-segment invariant. One-line fix (`raw_mapped_char_end`), own commit 3c899ec, exact-segment suites referee: 1804/0 and 598/0, clippy/fmt clean. + INTERFACE: 2026-09-03 empty interfaces split by source shape in true declaration outputs (24 tight vs 13 expanded): tight-authored stays tight (`exportAssignedNamespaceIsVisibleInDeclarationEmit` `Bar {}`), multi-line-authored stays expanded (`commentsInterface` `i1 {`). The advisor's zero-either-way pass was wrong on both counts; the existing tight test pin STANDS with an authority citation added. General rule folds into the preservation slice, not a printer fix. + PARADOX-REVISION: 2026-09-03 the fixed-point wording is retracted to stale-by-design. Ignored shard paths never enter the porcelain check, so commit, sweep, and merge at one clean HEAD is paradox-free; staleness on the next commit is the currency design, and f8c28dc shards must not merge at HEAD (completion.rs:973 rejects the binding). What stands: publishing a tracked merged receipt or ledger means the next `--check` needs a same-HEAD re-sweep, and the standing E_SET_MISMATCH remains chiefly coverage (most obligations receiptless). Per-slice targeted recounts keep attribution that batched sweeps lose. + PRESERVATION-CELLS: 2026-09-03 open cells closed by measurement. Catch blocks space (`catch: 15` in the keyword buckets). Multi-line-authored objects show no collapse (177 sources screened, 8 single-line outputs all pair-matched to single-line sources, 2 verified byte-paired). Source-side singles undercount outputs (fn 115 vs 292, obj 220 vs 579) because outputs add synthesized forms (helpers, IIFEs, accessors) — consistent with 1:1 preservation plus synthesis, not reflow. The ES5 synthesized-constructor memory citation is retracted as unverified (no such form in the corpus; the lowering slice pins from data). Design for next slice: source-range newline check per non-empty body/object construct; classes stay always-expand, empty blocks always-space, empty objects always-tight. + PLUMBING: 2026-09-03 scout-mapped end to end: useDefineForClassFields is dropped on BOTH paths (CLI: no CompilerOptions field, project.rs:918; worker: build_tsconfig emits lowercase string-quoted key the case-sensitive lookup misses, plus no field), so both default to target and agree by coincidence — census valid, no contamination. Fix: Option field + parse arm + accessor (project.rs), new apply_emit_fields param set-when-Some (emitter.rs), pass-through (effective.rs, pipeline.rs), ResolvedProgram carry (program.rs, shared root, window required), build_tsconfig bool list + remap (check_cells.rs). 75 pinned-true rows below es2022 plus 110 pinned-false at es2022+ ride this one fix. alwaysStrict worker-pins-true vs CLI-derives-from-strict divergence noted, out of slice. + PLUMBING-RESULTS: 2026-09-03 fix landed c17ab5e (window acquired with delivered receipt, inbox empty throughout, released with pre/post hashes; suites compiler 1802/0 verification 598/0, clippy and fmt clean): camelCase pin produces defineProperty emit at CLI while lowercase keys sit in raw and are silently ignored (tsc errors TS5023; unknown-key tolerance is by design per project.rs:915, case gap noted). 88-row divergent set pre/post diff: 1 full conversion (override18 FIELD_INIT to satisfied), 4 mode-advanced with the next real delta exposed (2 comment, 1 decorator, 1 temp-naming), 2 relabeled by ref-side token noise in the bucket rule (zero void-0 in either emit), 62 unchanged (pin-irrelevant code or agreed defaults). Sweep effect pending worker rebuild. Follow-ups, same mechanism: import_helpers field exists but unplumbed on both paths (877 helper-policy rows). Separate modeling slices: experimentalDecorators/emitDecoratorMetadata (516/119 rows, no fields), downlevelIteration (46), isolatedModules (82). + SWEEP-FRESH: 2026-09-04 full 2048-shard resweep at 89b0455 with the pre-flighted HEAD worker (repo target rebuilt without the CARGO_TARGET_DIR override; header binary digest asserted equal before launch; Node 24.18.0; run sweep-89b0455-fresh): acceptance 2048 headers uniform on all four digests, 66786 rows, 0 logged SHARD_FAIL lines; 130 stranded tmp orphans accumulated across the three driver launches (cleaned post-acceptance). PASS 12354 to 13127 (+773), BLOCKING_FAIL down equally, 0 regressions, all flips in javascript (2015 to 2788, +38%). Exhibit-1 es2015 row flips FAIL to PASS at worker level (class rule causally pinned); its es5 twin stays failing by the recorded downlevel scope. Mechanism cuts are co-occurrence, batch-measured together, not isolated: tight-empty sources 23 class-pattern + 61 block-pattern (disjoint), 8 explicitly pinned rows, spaced/multi-line-authored empties dominate the remainder with field-init Assign rows, 47 residual without an isolated pattern (possibly export-synthesis side effects of the field-init guard). The first sweep-89b0455 run measured a stale 16:28 worker (zero flips, correctly discarded, never claimed). + INTERFACE-REVISION: 2026-09-04 the 89b0455 INTERFACE line is superseded: true declaration outputs ALWAYS expand empty interfaces (0 tight / 21 expand; importTag16 a.d.ts expands tight-authored `export interface I {}`). All 24 tights traced to d.ts input copies (Bar thing.d.ts echo sole occurrence; ISpinButton cross-dir stem collision). Fix is a caller-side pre-check in `emit_interface_decl` (expand + return); shared `emit_type_members_block` stays tight because empty OBJECT TYPES stay tight (mappedTypes1 `x1: {}`, es6Import `_default: {}` transformation-proven, 0 expands observed in the true-output census). Test flipped to expanded with the F415-byte citation (the 3c899ec preservation citation was wrong). Suites 1804/0 and 598/0, clippy/fmt clean. + PRESERVATION-BLOCK: 2026-09-04 block-body gate landed (8a3d98c): non-empty bodies authored without a line break keep `{ stmt; stmt; }` (292-function + 3-arrow families); synthesized, past-authored-text, or multi-line-authored bodies expand. Machinery: `spans_multiple_lines` (source.rs), rewriter taint set with child-mark climb (transforms.rs), `preserves_single_line` with taint/length/mint gates (emitter.rs). Window variant dropped (no consumer after the red-pen cut). Suites 1814/0 and 598/0, clippy/fmt clean. Map test restored to its multi-line authority fixture plus a new single-line method map test (hand-decoded golden: greet(4) return(14) 1(21) ;(22) }(24) EOL(25), class-close EOL(1) unchanged). CLI spots: single-line fixture emits `function f() { return 1; }`; expandoFunctionSymbolProperty emits `function inner() { }` (spaced guard holds end to end); pre/post-gate outputs byte-identical on both authority files. Limitation: mappedTypes1's x1 resolves to `any` in our checker, so no CLI run exercised the `{}`-tight object path (pinned by unit suite only); tsc bytes are `declare let x1: {};` at mappedTypes1.js:139, declaration embedded in the .js baseline. JIT `provision_lock_rejects_overlapping_attempts` failed once then passed solo and full rerun: flake, not the tree. Sweep restated: run sweep-89b0455-fresh, binary digest 0fa9235d, +773 with 84 mechanism-indicative (23 class-only + 61 block-only, disjoint), remainder co-occurrence. Follow-ups: ctor and object-literal gates (approved, sequenced next). + PRESERVATION-CTOR-OBJECT: 2026-09-04 ctor and object gates landed (e6e1f70): injection-free ctor bodies authored on one line keep `{ stmt; }` (class still expands around them); non-empty object literals authored on one line join as `{ a: 1, b: 2 }` with uniform spacing (tight-authored `{a:1}` normalizes). Parameter-property injections always expand. Suites 1819/0 and 598/0, clippy/fmt clean. CLI spots: `const o = { a: 1, b: 2 };` and `constructor() { console.log(1); }` single-line end to end. Next: sweep re-measurement of the 292/579/3 families. + SWEEP-PRESERVATION: 2026-09-04 full 2048-shard resweep at 2d76b09 (run sweep-2d76b09-preservation, worker binary 9f8c8ce9 asserted pre-flight, clean-tree digest uniform, Node 24.18.0 pinned via PATH; one bash-timeout driver restart, resume skipped footered shards): acceptance 2048/2048 footers, 0 SHARD_FAIL, 0 obligation drift (66786 rows both sides). PASS 13127 -> 13512 (+385), BLOCKING_FAIL 46653 -> 46268 (-385), INAPPLICABLE_LANGUAGE_SERVICE 6693 / INAPPLICABLE_CATALOG_ERROR 312 / TIMEOUT 1 all unchanged (controls). Transitions: 387 BLOCKING_FAIL->PASS, 2 PASS->BLOCKING_FAIL. Churn: compiler/tests/cases/compiler 231, conformance node 16, computedProperties 11, destructuring 10, Accessors 10, templates 8. Range e03e5de..2d76b09 is campaign-level (pin, empty class/block spacing, ctor-close, interface revision, block + ctor/object single-line preservation), not commit-isolated. Regressions, both javascript facet aot mode: override18.ts [noimplicitoverride=true,target=es2015,usedefineforclassfields=true#javascript] and parser768531.ts [allowunusedlabels=true,target=es2015#javascript], each PASS (empty detail) -> BLOCKING_FAIL `javascript structural mismatch after formatting/order normalization`; override18 was the c17ab5e FIELD_INIT conversion (PLUMBING-RESULTS), so the preservation range re-broke it; byte-diff attribution queued (worker request schema: binding, request_id u64, key, observables, argv, working_directory, timeout_ms). Tool local/diff-sweeps.py self-tested (baseline-vs-itself 0 transitions); full list local/sweep-2d76b09-transitions.txt; receipts .outline/evidence-sweep-2d76b09-preservation/. + SWEEP-PRESERVATION-AUDIT: 2026-09-04 post-push four-digest audit of all 2048 headers under .outline/evidence-sweep-2d76b09-preservation/: exactly one distinct (authority 47d60ce5, candidate_binary 9f8c8ce9, candidate_tree bd95c396, harness 9f323e56) tuple, covering the resume phase too. The entry's uniformity claim is now command-backed, not launch-time inferred. + REGRESSION-CLOSURE: 2026-09-04 both SWEEP-PRESERVATION regressions root-caused via CLI byte-diff (tsconfig-driven scratch projects mirroring build_tsconfig, strict:true) and fixed. parser768531: the block gate preserved statement-level blocks, but its authority is function-family bodies only — emit_block_with_braces now expands whenever map_open_brace is set (the same context bit that splits brace mapping), commit 2e0e39d. override18: object_literal minted through the untainted node() helper so wholly synthesized defineProperty descriptors collapsed single-line — both mints now record taint, commit 8806887. Post-fix CLI emit matches each baseline's .js section byte-for-byte (wrapper sections aside, which the harness strips). Suites 1821/0 (2 new regression tests) and 598/0, fmt/clippy clean. Next sweep at this HEAD should show 389 net conversions with zero regressions; re-measure before claiming. + SWEEP-ZERO-REGRESSION: 2026-09-04 confirmation sweep at e8b93bc (run sweep-e8b93bc-preservation, binary e3060e58 pre-flighted, tree 0b2742b6, four-digest audit 1 distinct tuple over 2048 headers, 0 SHARD_FAIL, 0 obligation drift): PASS 13514, BLOCKING_FAIL 46266, controls unchanged. Diff vs 2d76b09: exactly +2 (override18, parser768531 restored), 0 regressions. Diff vs 89b0455-fresh: 387 BLOCKING_FAIL->PASS, 0 PASS->BLOCKING_FAIL — the preservation campaign (e03e5de..e8b93bc) is strictly net-positive. Interim 217b37d sweep caught classThisReference breaking (static init blocks are brace-mapped yet layout-preserving); BlockLayout enum (e8b93bc) names the three contexts, verified byte-for-byte against all three baselines. Receipts .outline/evidence-sweep-e8b93bc-preservation/; transition lists local/sweep-e8b93bc-vs-{2d76b09,89b0455}.txt. + ID-FLOOR-STRUCTURAL: 2026-09-04 rewriter id seeding rederived from the file's real high-water mark, commit 139656d: one NodeIdSource flows desugar-into-rewriter (alloc_id = ids.fresh), Rewriter::new snapshots the seed as the per-file synthesized floor, PrintSource carries it, and the mint-gate range check reads per-file truth; SYNTHESIZED_ID_FLOOR (the 2d76b09 hazard) is deleted, untransformed prints pass u32::MAX so the range check applies universally. Disjointness of minted vs parser ids is now structural (no constant to cross); the dense-file test pins observables at 1M+ nodes (emit completes, no statement drops, authored single-line preservation holds, descriptor lowering mints). Mutation note: a black-box catch of the old floor is not cheaply constructible — the collision point depends on the unmeasured mint-per-statement consumption ratio (authored id ≈ 3k vs mint position ≈ 1M+k collide near k ≈ 500k only for the right ratio) — so the structural invariant is the guarantee; measuring that ratio and placing a probe is doable future work if a behavioral pin is ever wanted. Suites 1824/0 and 598/0, fmt/clippy/diagnostics clean. + BUILD-INFO-FACET: 2026-09-04 the 10-cell build-info facet re-derived against authority: zero baselines in the entire 7.0.2 reference tree carry a `//// [name.tsbuildinfo]` section (the harness drops the artifact as environment dependent), so content parity is unprovable and the prior "producer missing" HARNESS_ERROR was a dead end by construction. New contract is emission-shape: the producer emits only when incremental or composite is set (bare tsBuildInfoFile writes nothing, matching tsc's optionsTsBuildInfoFileWithoutIncrementalAndComposite), and the observer's four arms demand artifact-exactly-when-implied (696f9af); the earlier non-empty check was a tautology every cell passed — caught before commit, never landed. Output paths now accept absolute and root-escaping values (resolve_output_from, 413796f): tsc runs incrementalTsBuildInfoFile clean on /a.tsbuildinfo while source paths keep confinement. Verified by direct shard runs at clean HEAD: incrementalOut, incrementalConfig, incrementalTsBuildInfoFile (implied+produced) and optionsTsBuildInfoFileWithoutIncrementalAndComposite (not-implied+none, self-describing artifact) all PASS; 4/10 cells sampled across both arms and both path families, remaining 6 ride the next sweep. Suites 1824/0 and 598/0, fmt/clippy clean. diff --git a/.outline/type-script-7.0.2-completion.md b/.outline/type-script-7.0.2-completion.md index 45c9d2db..357d7f89 100644 --- a/.outline/type-script-7.0.2-completion.md +++ b/.outline/type-script-7.0.2-completion.md @@ -22,6 +22,10 @@ Complete the clean-room Rust TypeScript compiler and native execution product ag - Runtime: every applicable stable test262 obligation required by bamTiScript native execution must pass in Interpreter, JIT, and AOT modes. Stable `Intl` and `Temporal` stay in scope. - Final scoped roots permit zero `BLOCKING_FAIL`, zero `EXTERNAL_BLOCKED`, and zero `INAPPLICABLE_CATALOG_ERROR`. - `.references/` is study-only. Workers may inspect release metadata, public declarations, test inputs, baselines, and observable behavior. They must not copy implementation code. +- Execution hosts: Interpreter, JIT, AOT, and direct native executable output run on `aarch64-apple-darwin`, `x86_64-apple-darwin`, `x86_64-unknown-linux-gnu`, `aarch64-unknown-linux-gnu`, and `x86_64-pc-windows-msvc`. `s390x-unknown-linux-gnu` and `riscv64gc-unknown-linux-gnu` remain target cells. +- Evidence host: primary receipts for every catalog are produced on `aarch64-apple-darwin` through the repository `Justfile`. Other hosts are proven by their target cell, the corpus differential, and the clean-install contract. +- Continuous integration: hosted workflows remain. Local `Justfile` recipes are an additional runner, not a replacement. +- Historical catalogs are owned, measured locally, and filtered by the 7.0.2 applicability rule above. ## Product roots @@ -131,6 +135,7 @@ Depth is seven: product -> root -> track -> cluster -> leaf -> logical shard -> | B6.1 | W4 | B6 | tsc argv surface + exit-code parity | `crates/bamts-cli/src/cli/tsc_args.rs` | `.outline/gates/B6.1.md` | | B6.2 | W4 | B6 | diagnostic rendering parity (pretty false canonical text; json) | `crates/bamts-cli/src/cli/diagnostic_format.rs` | `.outline/gates/B6.2.md` | | B7.1 | W4 | B7 | corpus expansion wave (add projects per failure clusters; corpus/manifest.toml + specs + projects) | `corpus/manifest.toml; corpus/cases/*.ts` | `.outline/gates/B7.1.md` | +| B7.2 | W4 | B7 | historical 6.0.2 and 5.9.3 catalog ownership and 7.0.2 applicability filter | `crates/bamts-verification/src/historical.rs` | `.outline/gates/B7.2.md` | | C0.1 | W2 | C0 | $262 host + harness loader + async $DONE + agent/Atomics hooks (blocks every C leaf) | `crates/bamts-runtime/src/host_objects.rs; crates/bamts-verification/src/oracles/test262_harness.rs` | `.outline/gates/C0.1.md` | | C1.1 | W2 | C1 | BigInt full family (value model, operations, literals, asIntN/asUintN, JSON interplay) | `crates/bamts-runtime/src/builtins/bigint.rs` | `.outline/gates/C1.1.md` | | C1.2 | W2 | C1 | Number formatting/parsing exactness (toFixed/toPrecision/toExponential/radix) | `crates/bamts-runtime/src/builtins/number_format.rs` | `.outline/gates/C1.2.md` | @@ -177,14 +182,22 @@ Depth is seven: product -> root -> track -> cluster -> leaf -> logical shard -> | E2.1 | W3 | E2 | JIT helper coverage completeness (46 helpers vs ISA) | `crates/bamts-codegen/src/jit/helpers.rs` | `.outline/gates/E2.1.md` | | E2.2 | W3 | E2 | JIT tiering: OSR + deopt + warmup policy | `crates/bamts-codegen/src/jit/tiering.rs` | `.outline/gates/E2.2.md` | | E2.3 | W3 | E2 | JIT-vs-interpreter differential stress (random program corpus) | `crates/bamts-codegen/tests/jit_differential.rs` | `.outline/gates/E2.3.md` | +| E2.4 | W3 | E2 | JIT execution and W^X proof on Apple Silicon | `crates/bamts-codegen/src/jit_memory.rs` | `.outline/gates/E2.4.md` | | E3.1 | W4 | E3 | AOT object emission per target triple | `crates/bamts-codegen/src/aot/emission.rs` | `.outline/gates/E3.1.md` | | E3.2 | W4 | E3 | AOT linking + cache correctness (existing fallback-cache lessons) | `crates/bamts-codegen/src/aot/linking.rs` | `.outline/gates/E3.2.md` | | E3.3 | W4 | E3 | reproducible-build pipeline (deterministic objects) | `crates/bamts-codegen/src/aot/reproducible.rs` | `.outline/gates/E3.3.md` | +| E3.4 | W4 | E3 | native executable link and run on macOS hosts | `crates/bamts-cli/src/driver.rs` | `.outline/gates/E3.4.md` | +| E3.5 | W4 | E3 | native executable link and run on Windows MSVC | `crates/bamts-cli/src/windows_link.rs` | `.outline/gates/E3.5.md` | +| E3.6 | W4 | E3 | shared platform facts: host triple, machine facts, process regions, and peak memory on every execution host | `crates/bamts-platform/` | `.outline/gates/E3.6.md` | +| E3.7 | W4 | E3 | in-process AOT linking on every execution host | `crates/bamts-codegen/src/aot/in_process_link.rs` | `.outline/gates/E3.7.md` | | E4.1 | W5 | E4 | target-cell evidence schema + verification/toolchains/.json generator | `crates/bamts-verification/src/toolchain_schema.rs` | `.outline/gates/E4.1.md` | | E4.2 | W5 | E4 | x86_64-unknown-linux-gnu cell closure (native host) | `verification/toolchains/x86_64-unknown-linux-gnu.json` | `.outline/gates/E4.2.md` | | E4.3 | W5 | E4 | aarch64 cell (cross + QEMU) | `verification/toolchains/aarch64-unknown-linux-gnu.json` | `.outline/gates/E4.3.md` | | E4.4 | W5 | E4 | s390x cell (cross + QEMU) | `verification/toolchains/s390x-unknown-linux-gnu.json` | `.outline/gates/E4.4.md` | | E4.5 | W5 | E4 | riscv64gc cell (cross + QEMU) | `verification/toolchains/riscv64gc-unknown-linux-gnu.json` | `.outline/gates/E4.5.md` | +| E4.6 | W5 | E4 | aarch64-apple-darwin cell (native host) | `verification/toolchains/aarch64-apple-darwin.json` | `.outline/gates/E4.6.md` | +| E4.7 | W5 | E4 | x86_64-apple-darwin cell (Rosetta) | `verification/toolchains/x86_64-apple-darwin.json` | `.outline/gates/E4.7.md` | +| E4.8 | W5 | E4 | x86_64-pc-windows-msvc cell (native host) | `verification/toolchains/x86_64-pc-windows-msvc.json` | `.outline/gates/E4.8.md` | | E5.1 | W6 | E5 | bench/compiler-rules.toml schema + measure harness (conditions record: governor/affinity/swap; INVALID_CONDITIONS on mismatch; median-of-N) | `bench/compiler-rules.toml` | `.outline/gates/E5.1.md` | | E5.2 | W6 | E5 | jit.compile-cost/payback/queue-tail-latency accepted measurements | `bench/jit_benchmarks.rs` | `.outline/gates/E5.2.md` | | E5.3 | W6 | E5 | stage0.{correctness,integrity,no-rwx,event-completeness,aot-no-jit-allocator} evidence | `bench/stage0_evidence.rs` | `.outline/gates/E5.3.md` | @@ -212,7 +225,7 @@ Depth is seven: product -> root -> track -> cluster -> leaf -> logical shard -> | B4 | B4.1, B4.2, B4.3, B4.4, B4.5 | `.outline/gates/cluster-B4.md` | | B5 | B5.1, B5.2, B5.3, B5.4 | `.outline/gates/cluster-B5.md` | | B6 | B6.1, B6.2 | `.outline/gates/cluster-B6.md` | -| B7 | B7.1 | `.outline/gates/cluster-B7.md` | +| B7 | B7.1, B7.2 | `.outline/gates/cluster-B7.md` | | C0 | C0.1 | `.outline/gates/cluster-C0.md` | | C1 | C1.1, C1.2, C1.3 | `.outline/gates/cluster-C1.md` | | C2 | C2.1, C2.2, C2.3, C2.4 | `.outline/gates/cluster-C2.md` | @@ -226,9 +239,9 @@ Depth is seven: product -> root -> track -> cluster -> leaf -> logical shard -> | C10 | C10.1, C10.2, C10.3, C10.4, C10.5, C10.6 | `.outline/gates/cluster-C10.md` | | C11 | C11.1, C11.2, C11.3, C11.4 | `.outline/gates/cluster-C11.md` | | E1 | E1.1, E1.2 | `.outline/gates/cluster-E1.md` | -| E2 | E2.1, E2.2, E2.3 | `.outline/gates/cluster-E2.md` | -| E3 | E3.1, E3.2, E3.3 | `.outline/gates/cluster-E3.md` | -| E4 | E4.1, E4.2, E4.3, E4.4, E4.5 | `.outline/gates/cluster-E4.md` | +| E2 | E2.1, E2.2, E2.3, E2.4 | `.outline/gates/cluster-E2.md` | +| E3 | E3.1, E3.2, E3.3, E3.4, E3.5, E3.6, E3.7 | `.outline/gates/cluster-E3.md` | +| E4 | E4.1, E4.2, E4.3, E4.4, E4.5, E4.6, E4.7, E4.8 | `.outline/gates/cluster-E4.md` | | E5 | E5.1, E5.2, E5.3, E5.4 | `.outline/gates/cluster-E5.md` | | F1 | F1.1, F1.2 | `.outline/gates/cluster-F1.md` | | F2 | F2.1, F2.2, F2.3, F2.4 | `.outline/gates/cluster-F2.md` | @@ -292,7 +305,7 @@ Leaf ledgers contain the exact checks. Integration requires: - unpiped workspace format, Clippy `-D warnings`, and tests; - Interpreter/JIT/AOT corpus differential; - formal G1, G2, G5, G3, G4, and G6 in order; -- 28 target obligations and 9 pre-registered benchmark rules; +- 49 target obligations and 9 pre-registered benchmark rules; - clean Node 24 package install and import of all thirteen export subpaths across five artifacts; - byte-identical catalog, evidence merge, ledger, proof, package, and release regeneration. @@ -307,3 +320,4 @@ Append-only. - 2026-08-25 CONTRACT_WRITTEN: stable 7.0.2 TypeScript-product scope, interfaces, evidence, ownership, depth tree, waves, and unlazy gates fixed before production edits. - 2026-08-25 LEAF_STARTED A1.1-A1.5: authority, source materialization, logical catalog, manifest cutover, and exact-classification vertical slice dispatched under one integration owner. - 2026-08-25 LEAF_STARTED A2.1: closed shard, lane, receipt, bounded-memory merge, and atomic-publication core dispatched with disjoint ownership. +- 2026-11-01 CONTRACT_AMENDED: execution hosts widened to the five artifact targets, with `aarch64-apple-darwin` as the primary evidence host through the repository `Justfile` and hosted workflows kept beside it. Nine leaves added (B7.2, E2.4, E3.4 to E3.7, E4.6 to E4.8), taking the frozen authority from 105 to 114 leaves and the target obligations from 28 to 49 (seven cells of seven obligations). diff --git a/Cargo.lock b/Cargo.lock index cbad4e63..857b65f5 100644 --- a/Cargo.lock +++ b/Cargo.lock @@ -108,6 +108,7 @@ dependencies = [ "bamts-bytecode", "bamts-cancel", "bamts-native", + "bamts-platform", "bamts-runtime", "cranelift-codegen", "cranelift-frontend", @@ -163,6 +164,13 @@ dependencies = [ "tokio-util", ] +[[package]] +name = "bamts-platform" +version = "0.2.0" +dependencies = [ + "libc", +] + [[package]] name = "bamts-runtime" version = "0.2.0" @@ -170,6 +178,7 @@ dependencies = [ "bamts-bytecode", "bamts-cancel", "bamts-native", + "bamts-platform", "icu_properties", "unicode-normalization", ] @@ -186,6 +195,7 @@ dependencies = [ "bamts-compiler", "bamts-native", "bamts-node", + "bamts-platform", "bamts-runtime", "getrandom 0.3.4", "libc", diff --git a/Cargo.toml b/Cargo.toml index 1ffa9416..c9e2eab7 100644 --- a/Cargo.toml +++ b/Cargo.toml @@ -31,6 +31,7 @@ workspace = true members = [ "crates/bamts-compiler", "crates/bamts-cancel", + "crates/bamts-platform", "crates/bamts-bytecode", "crates/bamts-runtime", "crates/bamts-native", @@ -58,6 +59,7 @@ bamts-cli = { path = "crates/bamts-cli", version = "0.2.0" } bamts-codegen = { path = "crates/bamts-codegen", version = "0.2.0" } bamts-compiler = { path = "crates/bamts-compiler", version = "0.2.0" } bamts-native = { path = "crates/bamts-native", version = "0.2.0" } +bamts-platform = { path = "crates/bamts-platform", version = "0.2.0" } bamts-node = { path = "crates/bamts-node", version = "0.2.0" } bamts-runtime = { path = "crates/bamts-runtime", version = "0.2.0" } base64 = "=0.23.1" diff --git a/Justfile b/Justfile index 30ec55b5..0bf21f8c 100644 --- a/Justfile +++ b/Justfile @@ -1,5 +1,147 @@ set shell := ["bash", "-euo", "pipefail", "-c"] +host_triple := `rustc -vV | awk '/^host:/{print $2}'` +export PATH := env_var('HOME') + "/.nvm/versions/node/v24.18.0/bin:/opt/homebrew/opt/lld/bin:/opt/homebrew/opt/llvm/bin:" + env_var('PATH') + +doctor: + [[ "$(node -v)" == "v24.18.0" ]] + [[ "$(rustc -V)" == *"1.97.1"* ]] + [[ "$(racket --version)" == *"9.2"* ]] + command -v ld.lld + command -v llvm-readelf + command -v lake + test -x formal/quint/node_modules/.bin/quint + printf '%s\n' 'DOCTOR PASS' + +# Fails while any gate ledger has an unchecked box or pending evidence, and lists +# the open ledgers; prints ALL MET otherwise. This is the check for GATES.md G1. +gate-status: + python3 -c 'import glob,re,sys; rows=[(f,len(re.findall(r"(?m)^- \[ \]",t)),len(re.findall(r"(?m)^ EVIDENCE: pending\s*$",t))) for f in sorted(glob.glob(".outline/gates/*.md")) for t in [open(f).read()]]; open_=[r for r in rows if r[1] or r[2]]; [print("OPEN %s unchecked=%d pending=%d"%r) for r in open_]; print("NO LEDGERS FOUND" if not rows else "ALL MET" if not open_ else "OPEN LEDGERS: %d of %d"%(len(open_),len(rows))); sys.exit(1 if open_ or not rows else 0)' + +ci: ci-quality ci-npm ci-corpus xcheck portability-check + printf '%s\n' 'LOCAL CI PASS' + +ci-quality: + cargo fmt --all --check + cargo check --workspace --locked + cargo clippy --workspace --all-targets --locked -- -D warnings + cargo test --workspace --locked + +# Every npm test except the two groups that need the assembled five-host release +# set: the staged leaf archives and the clean-consumer install. One machine cannot +# build all five leaves, so those run in ci-npm-release instead of being dropped. +ci-npm: + npm ci + node --test --test-skip-pattern='staged cli-|clean consumer' npm/test/*.test.mjs + +# The whole npm suite. It needs every host's leaf archive in npm/dist, so it is the +# gate for the five-host install, not for a single machine. +ci-npm-release: + npm ci + node --test npm/test/*.test.mjs + +ci-corpus: + cargo test -p bamts-verification --test corpus_differential --locked -- --nocapture + +ci-accounting: + cargo run --locked -p bamts-verification --bin ts_conformance -- sync --verify-pin --write-snapshot + cargo run --locked -p bamts-verification --bin ts_conformance -- audit-ledger --require-complete + +authority: + cargo run --locked -p bamts-verification -- source fetch typescript-7-compiler --dest target/authority/typescript-7.0.2 + cargo run --locked -p bamts-verification -- source fetch typescript-primary-tests --dest target/authority/typescript-7.0.2-tests + cargo run --locked -p bamts-verification -- source fetch typescript-6.0-tests --dest target/authority/typescript-6.0.2-tests + cargo run --locked -p bamts-verification -- source fetch typescript-compat-tests --dest target/authority/typescript-5.9.3-tests + cargo run --locked -p bamts-verification -- source fetch test262 --dest target/authority/test262 + +ci-catalog: authority + cargo run --locked -p bamts-verification --bin bamts-verification -- catalog regenerate --release typescript-7.0.2 --check + +ci-formal: ci-catalog + cargo run --locked -p bamts-verification --bin bamts-verification -- formal audit --gates G1,G2,G5,G3,G4 + +ci-string-oracle: + node npm/scripts/native-string-oracle.mjs + +sweep catalog runner shards="32" jobs="10": + #!/usr/bin/env bash + set -euo pipefail + test -z "$(git status --porcelain)" || { echo "sweep needs a clean tree"; exit 1; } + cargo build --locked --release -p bamts-verification --bin bamts-verification --bin ts_lane_worker --bin runtime_lane_worker + sha="$(git rev-parse HEAD)" + run="local-${sha:0:12}" + out="target/evidence-local/{{catalog}}/{{runner}}/{{host_triple}}/$run" + mkdir -p "$out" target/evidence-local/merged + export BAMTS_SUITE_COMPILER_ADAPTER=target/release/ts_lane_worker + export BAMTS_SUITE_INTERPRETER_ADAPTER=target/release/runtime_lane_worker + export BAMTS_SUITE_JIT_ADAPTER=target/release/runtime_lane_worker + export BAMTS_SUITE_AOT_ADAPTER=target/release/runtime_lane_worker + test -f verification/ts-suite/index.json || cargo run --locked -p bamts-verification --bin ts_conformance -- sync --verify-pin --write-snapshot + seq 0 $(( {{shards}} - 1 )) | xargs -P {{jobs}} -I{} target/release/bamts-verification suite run \ + --catalog {{catalog}} --shard "{}/{{shards}}" --receipt "$out/{}.jsonl" \ + --runner {{runner}} --platform {{host_triple}} \ + --workflow Justfile --run-id "$run" --run-attempt 1 --source-sha "$sha" \ + --job sweep --host "$(uname -sm | tr ' ' '-')" \ + --runtime "node-v24.18.0;rust-$(rustc --version)" + merged="target/evidence-local/merged/{{catalog}}-{{runner}}-{{host_triple}}.jsonl" + target/release/bamts-verification suite merge --catalog {{catalog}} --receipts "$out" --out "$merged" + target/release/bamts-verification suite merge --catalog {{catalog}} --receipts "$out" --out "$merged" --check + target/release/bamts-verification suite status --receipts target/evidence-local/merged --out target/evidence-local/status.md + printf 'SWEEP PASS %s\n' "$merged" + +# Counts by state, facet and failure family for one sweep. `receipt` defaults to +# the merged file `sweep` writes; pass another, such as a smoke shard, to read it. +status catalog runner receipt="" top="20": + #!/usr/bin/env bash + set -euo pipefail + receipt="{{receipt}}" + receipt="${receipt:-target/evidence-local/merged/{{catalog}}-{{runner}}-{{host_triple}}.jsonl}" + cargo run --locked -q -p bamts-verification --bin bamts-verification -- suite families "$receipt" --top {{top}} + +# The rows of one failure family, smallest input first. Family names come from `status`. +family-show family catalog="typescript-7.0.2" runner="compiler" receipt="": + #!/usr/bin/env bash + set -euo pipefail + case "{{catalog}}" in + typescript-7.0.2) root=target/authority/typescript-7.0.2-tests ;; + typescript-6.0.2) root=target/authority/typescript-6.0.2-tests ;; + typescript-5.9.3) root=target/authority/typescript-5.9.3-tests ;; + test262) root=target/authority/test262 ;; + *) echo "family-show: no authority tree is known for catalog {{catalog}}"; exit 1 ;; + esac + receipt="{{receipt}}" + receipt="${receipt:-target/evidence-local/merged/{{catalog}}-{{runner}}-{{host_triple}}.jsonl}" + cargo run --locked -q -p bamts-verification --bin bamts-verification -- suite families "$receipt" --family "{{family}}" --root "$root" + +# Rebuilds the lane workers, reruns one family plus a sample of the passing rows, and +# fails if a sampled row no longer passes. A fix must convert a row and regress none. +# The sample catches a broad regression between sweeps; only a sweep proves none. +recount family catalog="typescript-7.0.2" runner="compiler" receipt="": + #!/usr/bin/env bash + set -euo pipefail + receipt="{{receipt}}" + receipt="${receipt:-target/evidence-local/merged/{{catalog}}-{{runner}}-{{host_triple}}.jsonl}" + cargo build --locked --release -p bamts-verification --bin ts_lane_worker --bin runtime_lane_worker + export BAMTS_SUITE_COMPILER_ADAPTER=target/release/ts_lane_worker + export BAMTS_SUITE_INTERPRETER_ADAPTER=target/release/runtime_lane_worker + export BAMTS_SUITE_JIT_ADAPTER=target/release/runtime_lane_worker + export BAMTS_SUITE_AOT_ADAPTER=target/release/runtime_lane_worker + cargo run --locked -q -p bamts-verification --bin bamts-verification -- suite recount --catalog {{catalog}} --runner {{runner}} --baseline "$receipt" --family "{{family}}" --fail-on-regression + +# Rewrites target/evidence-local/status.md from every receipt in a directory (the merged +# ones `sweep` writes) and shows what changed since the previous report. +status-report receipts="target/evidence-local/merged" out="target/evidence-local/status.md": + cargo run --locked -q -p bamts-verification --bin bamts-verification -- suite status --receipts {{receipts}} --out {{out}} + +nightly: ci ci-accounting ci-formal + just sweep typescript-7.0.2 compiler + just sweep typescript-6.0.2 compiler + just sweep typescript-5.9.3 compiler + just sweep test262 interpreter + just sweep test262 jit + just sweep test262 aot + printf '%s\n' 'NIGHTLY PASS' + # Deterministic receipt-aware TypeScript product release root. # # A receipt content-addresses the binary that measured it, so the process @@ -34,4 +176,44 @@ leaf-gate leaf: # "no registered adapter" and the receipt measures nothing at all. conformance-shard index count="4" catalog="typescript-7.0.2": cargo build --locked --release -p bamts-verification --bin ts_lane_worker - receipt="target/evidence-local/{{catalog}}/{{index}}.jsonl" && mkdir -p "$(dirname "$receipt")" && BAMTS_SUITE_COMPILER_ADAPTER=target/release/ts_lane_worker cargo run --locked --release -p bamts-verification --bin bamts-verification -- suite run --catalog {{catalog}} --shard "{{index}}/{{count}}" --receipt "$receipt" --runner compiler --platform x86_64-unknown-linux-gnu --workflow local --run-id local --run-attempt 1 --source-sha "$(git rev-parse HEAD)" --job conformance-shard --host "$(uname -sm | tr ' ' '-')" --runtime "node-v24.18.0;rust-$(rustc --version)" && printf 'SHARD %s/%s RECEIPT %s\n' "{{index}}" "{{count}}" "$receipt" + receipt="target/evidence-local/{{catalog}}/{{index}}.jsonl" && mkdir -p "$(dirname "$receipt")" && BAMTS_SUITE_COMPILER_ADAPTER=target/release/ts_lane_worker cargo run --locked --release -p bamts-verification --bin bamts-verification -- suite run --catalog {{catalog}} --shard "{{index}}/{{count}}" --receipt "$receipt" --runner compiler --platform {{host_triple}} --workflow local --run-id local --run-attempt 1 --source-sha "$(git rev-parse HEAD)" --job conformance-shard --host "$(uname -sm | tr ' ' '-')" --runtime "node-v24.18.0;rust-$(rustc --version)" && printf 'SHARD %s/%s RECEIPT %s\n' "{{index}}" "{{count}}" "$receipt" + +# Check that the whole workspace, tests included, still builds for every host +# the product targets. This proves only that it compiles. +xcheck: + cargo check --workspace --all-targets --locked --target x86_64-apple-darwin + cargo check --workspace --all-targets --locked --target x86_64-unknown-linux-gnu + cargo check --workspace --all-targets --locked --target aarch64-unknown-linux-gnu + cargo check --workspace --all-targets --locked --target x86_64-pc-windows-msvc + printf '%s\n' 'XCHECK PASS' + +# Fail when code outside bamts-platform reads /proc or /sys, or runs numactl or +# sysctl, so OS facts stay behind the platform crate. It is a tripwire for +# literals, not a proof: it misses a path built from parts or a command named by +# a variable. It uses grep because it exists on every host; an rg missing from +# PATH would make a ripgrep search print nothing and pass. +# +# A search that cannot see must not pass. BSD grep on macOS exits 1 for a missing +# directory, the same as "no match", so exit codes cannot tell. bamts-platform +# reads /proc and /sys by design, so a search that finds nothing there is blind. +portability-check: + #!/usr/bin/env bash + set -euo pipefail + pattern='"/(proc|sys)/|Command::new\("(numactl|sysctl)"' + status=0 + found="$(grep -rnE --include='*.rs' "$pattern" crates)" || status=$? + if [[ $status -gt 1 ]]; then + printf '%s\n' 'PORTABILITY ERROR: the search itself failed' + exit 1 + fi + if ! grep -q '^crates/bamts-platform/src/' <<<"$found"; then + printf '%s\n' 'PORTABILITY ERROR: the search found nothing in bamts-platform, so it cannot be trusted' + exit 1 + fi + hits="$(grep -v '^crates/bamts-platform/src/' <<<"$found" || true)" + if [[ -n "$hits" ]]; then + printf '%s\n' "$hits" + printf '%s\n' 'PORTABILITY FAIL' + exit 1 + fi + printf '%s\n' 'PORTABILITY PASS' diff --git a/README.md b/README.md index 3858942c..c67a37cf 100644 --- a/README.md +++ b/README.md @@ -12,7 +12,7 @@ bamTiScript is a pre-release Rust toolchain that type-checks and compiles TypeSc - Repository: `bamTiScript` - CLI binary: `bamts` -- Workspace crates: `bamts`, `bamts-cli`, `bamts-compiler`, `bamts-cancel`, `bamts-bytecode`, `bamts-runtime`, `bamts-codegen`, `bamts-native`, `bamts-node`, `bamts-verification`, and private `bamts-napi` +- Workspace crates: `bamts`, `bamts-cli`, `bamts-compiler`, `bamts-cancel`, `bamts-platform`, `bamts-bytecode`, `bamts-runtime`, `bamts-codegen`, `bamts-native`, `bamts-node`, `bamts-verification`, and private `bamts-napi` - npm packages: `bamti` (in-process Node 24+ interface) and `bamti-cli` (standalone CLI transport). Currently published 0.1.0 packages on npm do not provide native binary artifacts; the source implementation of Node-API bindings is not yet published. ## Quickstart diff --git a/crates/bamts-bytecode/src/lib.rs b/crates/bamts-bytecode/src/lib.rs index 47eeb3d8..9e2c05f6 100644 --- a/crates/bamts-bytecode/src/lib.rs +++ b/crates/bamts-bytecode/src/lib.rs @@ -381,6 +381,12 @@ pub enum UnaryOp { Negate, BitwiseNot, LogicalNot, + /// ECMAScript `ToString`, which a template literal applies to a substitution. + ToString, + /// ECMAScript `ToPropertyKey`, applied once to a key that is both read and + /// written. A primitive stays as it is, since it converts the same way each + /// time; only an object runs user code. + ToPropertyKey, } impl UnaryOp { @@ -392,6 +398,8 @@ impl UnaryOp { Self::Negate => 3, Self::BitwiseNot => 4, Self::LogicalNot => 5, + Self::ToString => 6, + Self::ToPropertyKey => 7, } } @@ -403,6 +411,8 @@ impl UnaryOp { 3 => Some(Self::Negate), 4 => Some(Self::BitwiseNot), 5 => Some(Self::LogicalNot), + 6 => Some(Self::ToString), + 7 => Some(Self::ToPropertyKey), _ => None, } } @@ -581,6 +591,10 @@ impl DisposeHint { pub enum AccessorKind { Getter, Setter, + /// A [`Self::Getter`] on a class, whose accessors are not enumerable. + ClassGetter, + /// A [`Self::Setter`] on a class, whose accessors are not enumerable. + ClassSetter, } impl AccessorKind { @@ -588,6 +602,8 @@ impl AccessorKind { match self { Self::Getter => 0, Self::Setter => 1, + Self::ClassGetter => 2, + Self::ClassSetter => 3, } } @@ -595,22 +611,39 @@ impl AccessorKind { match tag { 0 => Some(Self::Getter), 1 => Some(Self::Setter), + 2 => Some(Self::ClassGetter), + 3 => Some(Self::ClassSetter), _ => None, } } + + /// Whether this installs the getter half of the descriptor. + #[must_use] + pub const fn is_getter(self) -> bool { + matches!(self, Self::Getter | Self::ClassGetter) + } + + /// Whether a property this creates is enumerable. + #[must_use] + pub const fn is_enumerable(self) -> bool { + matches!(self, Self::Getter | Self::Setter) + } } /// Which own-property descriptor slot /// [`Instruction::LoadOwnDescriptorSlot`] / /// [`Instruction::DefineOwnDescriptorSlot`] addresses. /// -/// Wire tags are `0` = value, `1` = getter, `2` = setter. These opcodes never -/// invoke accessors and never walk the prototype chain. +/// Wire tags are `0` = value, `1` = getter, `2` = setter, `3` = entire. These +/// opcodes never invoke accessors and never walk the prototype chain. #[derive(Clone, Copy, Debug, Eq, Hash, PartialEq)] pub enum DescriptorSlot { Value, Getter, Setter, + /// The whole property, replaced by a writable, enumerable, configurable + /// data property: ECMAScript `CreateDataProperty`. It reads as `undefined`. + Entire, } impl DescriptorSlot { @@ -619,6 +652,7 @@ impl DescriptorSlot { Self::Value => 0, Self::Getter => 1, Self::Setter => 2, + Self::Entire => 3, } } @@ -627,6 +661,7 @@ impl DescriptorSlot { 0 => Some(Self::Value), 1 => Some(Self::Getter), 2 => Some(Self::Setter), + 3 => Some(Self::Entire), _ => None, } } @@ -1410,13 +1445,21 @@ pub struct FunctionFlags { pub is_async: bool, pub is_generator: bool, pub is_constructable: bool, + /// Non-strict ECMAScript code, which only a JavaScript script can contain. + pub is_sloppy: bool, + /// A generator whose parameters can run code. A call runs it to its first + /// `Suspend`, which ends the prologue, so a throw reaches the caller. + pub binds_at_call: bool, } impl FunctionFlags { const ASYNC: u8 = 0b001; const GENERATOR: u8 = 0b010; const CONSTRUCTABLE: u8 = 0b100; - const KNOWN: u8 = Self::ASYNC | Self::GENERATOR | Self::CONSTRUCTABLE; + const SLOPPY: u8 = 0b1000; + const BINDS_AT_CALL: u8 = 0b1_0000; + const KNOWN: u8 = + Self::ASYNC | Self::GENERATOR | Self::CONSTRUCTABLE | Self::SLOPPY | Self::BINDS_AT_CALL; const fn to_bits(self) -> u8 { let mut bits = 0; @@ -1429,6 +1472,12 @@ impl FunctionFlags { if self.is_constructable { bits |= Self::CONSTRUCTABLE; } + if self.is_sloppy { + bits |= Self::SLOPPY; + } + if self.binds_at_call { + bits |= Self::BINDS_AT_CALL; + } bits } @@ -1440,6 +1489,8 @@ impl FunctionFlags { is_async: bits & Self::ASYNC != 0, is_generator: bits & Self::GENERATOR != 0, is_constructable: bits & Self::CONSTRUCTABLE != 0, + is_sloppy: bits & Self::SLOPPY != 0, + binds_at_call: bits & Self::BINDS_AT_CALL != 0, }) } } @@ -4131,6 +4182,8 @@ mod tests { is_async: true, is_generator: false, is_constructable: true, + is_sloppy: true, + binds_at_call: true, }, code, handlers, @@ -4874,11 +4927,11 @@ mod tests { write_u32(0, &mut bad_flags); // capture count write_u32(0, &mut bad_flags); // params write_u32(1, &mut bad_flags); // registers - bad_flags.push(0b1000); // unknown flag bit + bad_flags.push(0b10_0000); // unknown flag bit assert!(matches!( decode(&bad_flags, &DecodeLimits::default()), Err(DecodeError { - kind: DecodeErrorKind::InvalidFunctionFlags { bits: 0b1000 }, + kind: DecodeErrorKind::InvalidFunctionFlags { bits: 0b10_0000 }, .. }) )); @@ -6155,7 +6208,7 @@ mod tests { assert_eq!(std::mem::size_of::(), 4); assert_eq!(std::mem::size_of::(), 4); assert_eq!(std::mem::size_of::(), 8); - assert_eq!(std::mem::size_of::(), 3); + assert_eq!(std::mem::size_of::(), 5); } #[test] diff --git a/crates/bamts-cli/build.rs b/crates/bamts-cli/build.rs index dc427e0a..78227508 100644 --- a/crates/bamts-cli/build.rs +++ b/crates/bamts-cli/build.rs @@ -16,6 +16,7 @@ fn main() { let host = required("HOST"); let archive = out_dir.join(staticlib_archive_name(&target)); + let mut native_libraries = String::new(); match node_staticlib_action(&target, &host) { NodeStaticlibAction::Assemble => { let wrapper = out_dir.join("embedded_node.rs"); @@ -33,7 +34,7 @@ fn main() { &cargo.fingerprint, ) { Some(node_rlib) => { - assemble_staticlib( + native_libraries = assemble_staticlib( &rustc, &target, &dependencies, @@ -55,6 +56,9 @@ fn main() { "rustc-env=BAMTS_NODE_STATICLIB={}", archive.display() )); + cargo_line(&format!( + "rustc-env=BAMTS_NODE_NATIVE_LIBS={native_libraries}" + )); cargo_line(&format!("rustc-env=BAMTS_HOST_TARGET={host}")); cargo_line(&format!("rustc-env=BAMTS_BUILD_TARGET={target}")); } @@ -425,7 +429,7 @@ fn assemble_staticlib( node_rlib: &Path, archive: &Path, wrapper: &Path, -) { +) -> String { let mut command = Command::new(rustc); command .arg("--crate-name") @@ -448,6 +452,12 @@ fn assemble_staticlib( .arg("-o") .arg(archive) .arg(wrapper); + // The MSVC linker takes no libraries from the archive, so the link step has to + // name the system libraries that the archive needs. Only rustc knows them. + let needs_libraries = targets_msvc(target); + if needs_libraries { + command.arg("--print=native-static-libs"); + } let output = command.output().unwrap_or_else(|error| { panic!("could not start `{}`: {error}", Path::new(rustc).display()) }); @@ -459,6 +469,26 @@ fn assemble_staticlib( String::from_utf8_lossy(&output.stderr) ); } + if !needs_libraries { + return String::new(); + } + native_static_libraries(&String::from_utf8_lossy(&output.stderr)).unwrap_or_else(|| { + panic!( + "rustc did not report the native libraries of the bamts-node staticlib\nstderr:\n{}", + String::from_utf8_lossy(&output.stderr) + ) + }) +} + +/// The libraries rustc says to link with a staticlib, from its +/// `note: native-static-libs: kernel32.lib ntdll.lib /defaultlib:msvcrt` line. +fn native_static_libraries(stderr: &str) -> Option { + const MARKER: &str = "native-static-libs:"; + stderr.lines().find_map(|line| { + let libraries = line.split_once(MARKER)?.1; + let libraries = libraries.split_whitespace().collect::>().join(" "); + (!libraries.is_empty()).then_some(libraries) + }) } fn write_check_placeholder(dependencies: &Path, archive: &Path) { @@ -529,8 +559,12 @@ fn node_staticlib_action(target: &str, host: &str) -> NodeStaticlibAction { } } +fn targets_msvc(target: &str) -> bool { + target.contains("windows-msvc") +} + fn staticlib_archive_name(target: &str) -> &'static str { - if target.contains("windows-msvc") { + if targets_msvc(target) { "bamts_node.lib" } else { "libbamts_node.a" @@ -554,6 +588,28 @@ mod tests { directory } + #[test] + fn native_libraries_come_from_the_rustc_note() { + let stderr = "warning: something\nnote: link against the following native artifacts when linking against this static library\nnote: native-static-libs: kernel32.lib ntdll.lib userenv.lib /defaultlib:msvcrt\n\nwarning: 1 warning emitted\n"; + assert_eq!( + native_static_libraries(stderr).as_deref(), + Some("kernel32.lib ntdll.lib userenv.lib /defaultlib:msvcrt") + ); + } + + #[test] + fn a_missing_or_empty_native_libraries_note_gives_nothing() { + assert_eq!(native_static_libraries(""), None); + assert_eq!( + native_static_libraries("note: native-static-libs: \n"), + None + ); + assert_eq!( + native_static_libraries("note: link against the following native artifacts\n"), + None + ); + } + fn fingerprint_json(dependency_hash: u64) -> String { format!(r#"{{"deps":[[8623995201299226029,"bamts_node",false,{dependency_hash}]]}}"#) } diff --git a/crates/bamts-cli/src/api_server/control.rs b/crates/bamts-cli/src/api_server/control.rs index cb52dc58..440e4d57 100644 --- a/crates/bamts-cli/src/api_server/control.rs +++ b/crates/bamts-cli/src/api_server/control.rs @@ -102,15 +102,25 @@ impl Control { pub(crate) fn offer(&self, request: Request, bytes: usize) -> Admission { let mut inner = self.inner.lock().expect("transport control poisoned"); - if inner.stop || inner.fatal.is_some() { + if inner.fatal.is_some() { + return Admission::Halt; + } + // A frame already read still earns its terminal response: the drain runs + // after the reader exits, so the request only has to reach the queue. + let stopping = inner.stop; + if stopping && request.id.is_none() { return Admission::Halt; } + let admission = self.admit_locked(&mut inner, request, bytes); + if stopping { Admission::Halt } else { admission } + } + fn admit_locked(&self, inner: &mut Inner, request: Request, bytes: usize) -> Admission { if request.id.is_none() { match request.method.as_str() { "$/cancelRequest" => { if let Some(id) = cancellation_id(&request) { - cancel_locked(&mut inner, &id); + cancel_locked(inner, &id); } self.ready.notify_one(); return Admission::Accepted; @@ -150,7 +160,7 @@ impl Control { }; if let Some((response_id, error)) = rejection { - return self.queue_reject_locked(&mut inner, ticket, response_id, error); + return self.queue_reject_locked(inner, ticket, response_id, error); } } @@ -169,9 +179,7 @@ impl Control { } match id { - Some(id) => { - self.queue_reject_locked(&mut inner, ticket, Some(id), ApiError::IntakeFull) - } + Some(id) => self.queue_reject_locked(inner, ticket, Some(id), ApiError::IntakeFull), None => Admission::Accepted, } } diff --git a/crates/bamts-cli/src/api_server/reader.rs b/crates/bamts-cli/src/api_server/reader.rs index 4da6bd7b..e1112d45 100644 --- a/crates/bamts-cli/src/api_server/reader.rs +++ b/crates/bamts-cli/src/api_server/reader.rs @@ -97,6 +97,21 @@ pub(crate) fn reader_main(mut input: R, control: Arc) -> io mod tests { use super::*; + #[test] + fn a_frame_read_after_stop_still_earns_its_response() { + let control = Control::new(); + control.stop(); + let request: Request = + serde_json::from_str(r#"{"id":7,"method":"compiler/version"}"#).expect("request"); + assert!(matches!(control.offer(request, 32), Admission::Halt)); + assert_eq!(control.drain().len(), 1, "the drain answers it"); + + let notification: Request = + serde_json::from_str(r#"{"method":"compiler/version"}"#).expect("notification"); + assert!(matches!(control.offer(notification, 32), Admission::Halt)); + assert!(control.drain().is_empty(), "a notification owes no answer"); + } + #[test] fn reader_guard_reports_panics() { let control = Arc::new(Control::new()); diff --git a/crates/bamts-cli/src/api_server/tests.rs b/crates/bamts-cli/src/api_server/tests.rs index b2c10613..8b15b945 100644 --- a/crates/bamts-cli/src/api_server/tests.rs +++ b/crates/bamts-cli/src/api_server/tests.rs @@ -476,6 +476,7 @@ fn cancelled_execute_reports_request_cancelled() { assert_eq!(error_code(&response), CODE_REQUEST_CANCELLED); } +#[cfg(unix)] #[test] fn late_cancel_after_response_does_not_poison_reused_id() { let root = temp_root("late-cancel"); @@ -521,6 +522,7 @@ fn late_cancel_after_response_does_not_poison_reused_id() { fs::remove_dir_all(root).expect("remove root"); } +#[cfg(unix)] #[test] fn queue_saturation_pipe_cancel() { let root = temp_root("queue-saturation"); @@ -585,6 +587,7 @@ fn queue_saturation_pipe_cancel() { fs::remove_dir_all(root).expect("remove root"); } +#[cfg(unix)] fn exit_join_elapsed() -> Duration { let (reader, mut writer) = io::pipe().expect("pipe"); let producer = std::thread::spawn(move || { @@ -600,6 +603,7 @@ fn exit_join_elapsed() -> Duration { elapsed } +#[cfg(unix)] #[test] fn repeated_exit_sessions_reap_readers() { for _ in 0..4 { @@ -607,6 +611,7 @@ fn repeated_exit_sessions_reap_readers() { } } +#[cfg(unix)] fn shutdown_join_elapsed() -> Duration { let (reader, mut writer) = io::pipe().expect("pipe"); let producer = std::thread::spawn(move || { @@ -626,11 +631,13 @@ fn shutdown_join_elapsed() -> Duration { elapsed } +#[cfg(unix)] #[test] fn shutdown_joins_the_reader() { assert!(shutdown_join_elapsed() < Duration::from_millis(500)); } +#[cfg(unix)] #[test] fn repeated_shutdown_sessions_reap_readers() { for _ in 0..4 { @@ -638,6 +645,7 @@ fn repeated_shutdown_sessions_reap_readers() { } } +#[cfg(unix)] #[test] fn in_flight_cancel_over_pipe_preserves_session() { let root = temp_root("pipe-cancel"); @@ -719,7 +727,8 @@ fn initialize_confines_paths_to_the_session_root() { input.extend(frame( r#"{"id":2,"method":"service/open","params":{"path":"../escape.ts","text":"const a = 1;","version":1}}"#, )); - let canonical_root = fs::canonicalize(&root).expect("canonical root"); + let canonical_root = + bamts_compiler::project::canonical::canonicalize(&root).expect("canonical root"); let responses = run(input); fs::remove_dir_all(&root).expect("remove root"); diff --git a/crates/bamts-cli/src/context.rs b/crates/bamts-cli/src/context.rs index a82a4f7c..abdf9c25 100644 --- a/crates/bamts-cli/src/context.rs +++ b/crates/bamts-cli/src/context.rs @@ -1,9 +1,9 @@ use std::collections::BTreeMap; use std::ffi::{OsStr, OsString}; -use std::fs; use std::path::{Path, PathBuf}; use bamts_compiler::program::ProgramLoadError; +use bamts_compiler::project::canonical::canonicalize; use crate::driver::DriverError; @@ -68,7 +68,7 @@ impl ExecutionContext { } fn canonicalize_cwd(cwd: &Path) -> Result { - fs::canonicalize(cwd) + canonicalize(cwd) .map_err(|error| DriverError::ProgramLoad(ProgramLoadError::InvalidRoot(error))) } @@ -87,21 +87,35 @@ fn temp_dir_from_env(env: &BTreeMap) -> PathBuf { .unwrap_or_else(|| PathBuf::from("/tmp")) } +/// The value of the variable `key`, found without regard to case: Windows spells +/// the search path `Path`, and an environment map compares keys exactly. +#[cfg(any(windows, test))] +pub(crate) fn env_ignoring_case<'a>( + env: impl IntoIterator, + key: &str, +) -> Option<&'a OsStr> { + env.into_iter() + .find(|(candidate, _)| candidate.to_string_lossy().eq_ignore_ascii_case(key)) + .map(|(_, value)| value) +} + #[cfg(windows)] fn temp_dir_from_env(env: &BTreeMap) -> PathBuf { - fn value<'a>(env: &'a BTreeMap, key: &str) -> Option<&'a OsStr> { - env.iter() - .find(|(candidate, _)| candidate.to_string_lossy().eq_ignore_ascii_case(key)) - .map(|(_, value)| value.as_os_str()) - } + let value = |key: &str| { + env_ignoring_case( + env.iter() + .map(|(name, value)| (name.as_os_str(), value.as_os_str())), + key, + ) + }; for key in ["TMP", "TEMP", "USERPROFILE"] { - if let Some(path) = value(env, key) { + if let Some(path) = value(key) { return PathBuf::from(path); } } - value(env, "SystemRoot") - .or_else(|| value(env, "WINDIR")) + value("SystemRoot") + .or_else(|| value("WINDIR")) .map(|root| PathBuf::from(root).join("Temp")) .unwrap_or_else(|| PathBuf::from(r"C:\Windows\Temp")) } @@ -110,6 +124,9 @@ fn temp_dir_from_env(env: &BTreeMap) -> PathBuf { mod tests { use super::*; + /// The variable that names the temporary directory on this platform. + const TEMP_VARIABLE: &str = if cfg!(windows) { "TMP" } else { "TMPDIR" }; + #[test] fn explicit_context_resolves_relative_and_empty_temp_paths_from_cwd() { let directory = tempfile::tempdir().expect("temporary directory"); @@ -119,7 +136,7 @@ mod tests { ] { let context = ExecutionContext::new( directory.path(), - BTreeMap::from([(OsString::from("TMPDIR"), value)]), + BTreeMap::from([(OsString::from(TEMP_VARIABLE), value)]), ) .expect("execution context"); assert_eq!(context.temp_dir(), expected); diff --git a/crates/bamts-cli/src/driver.rs b/crates/bamts-cli/src/driver.rs index 28278b9f..82d7d634 100644 --- a/crates/bamts-cli/src/driver.rs +++ b/crates/bamts-cli/src/driver.rs @@ -15,6 +15,7 @@ use bamts_compiler::{ project::{ JsonObject, JsonValue, ProjectConfig, ProjectRoot, build_mode::BuildInfo, + canonical::{canonicalize, slash_text}, effective::{ EffectiveProject, MaterializeError, ProjectBuildOptions, ProjectCompileError, ProjectCompileOptions, ProjectCompileResult, ProjectLoadRequest, @@ -670,7 +671,7 @@ fn initialize_tsc_config(context: &ExecutionContext) -> Result>(); let sources = result .sources @@ -1041,7 +1042,7 @@ fn config_not_found_outcome( "TS5081", format!( "Cannot find a tsconfig.json file at the current directory: {}.", - config_path.display() + slash_text(config_path) ), ); }; @@ -1053,7 +1054,7 @@ fn config_not_found_outcome( { return tsc_error_outcome( "TS5083", - format!("Cannot read file '{}'.", config_path.display()), + format!("Cannot read file '{}'.", slash_text(config_path)), ); } let directory = if project_path.is_absolute() { @@ -1065,7 +1066,7 @@ fn config_not_found_outcome( "TS5057", format!( "Cannot find a tsconfig.json file at the specified directory: '{}'.", - directory.display() + slash_text(&directory) ), ) } @@ -1722,6 +1723,7 @@ fn forbidden_lint_override(error: bamts_compiler::lint::ForbidOverrideError) -> fn lower_options(args: &CliArgs) -> LowerOptions { LowerOptions { javascript_compatibility: args.js_compat.enabled, + ..LowerOptions::default() } } @@ -1954,7 +1956,7 @@ fn populate_node_environment( Ok(()) } -fn capture_process_with_cancel( +pub(crate) fn capture_process_with_cancel( command: &mut Command, cancel: &CancellationToken, ) -> io::Result> { @@ -2154,41 +2156,77 @@ fn link_executable( source, })?; let archive = cached_node_archive(context)?; - let compiler = discover_toolchain(context, cancel)?; + let linker = discover_linker(context, cancel)?; let temporary = TemporaryLinkFiles::create(parent, &destination)?; fs::write(&temporary.object, object).map_err(|source| DriverError::WriteObject { path: temporary.object.clone(), source, })?; - let mut command = Command::new(&compiler); - command - .current_dir(context.cwd()) - .env_clear() - .envs(context.envs()) - .arg(&temporary.object) - .arg(&archive) - .arg("-o") - .arg(&temporary.executable); - if cfg!(target_os = "linux") { - command.args(["-ldl", "-lpthread", "-lm"]); - } + let mut command = linker.command(context, &temporary.object, &archive, &temporary.executable); let output = capture_process_with_cancel(&mut command, cancel) .map_err(|source| DriverError::LinkStart { - program: compiler.clone(), + program: command.get_program().to_owned(), source, })? .ok_or(DriverError::Cancelled)?; if !output.status.success() { return Err(DriverError::LinkFailed { - program: compiler, + program: command.get_program().to_owned(), status: output.status, - stderr: String::from_utf8_lossy(&output.stderr).into_owned(), + stderr: linker.diagnostics(&output), }); } publish_linked_executable(&temporary.executable, &destination) } +/// The system C compiler that links on every host except Windows MSVC. +#[cfg(not(target_env = "msvc"))] +struct Linker { + compiler: OsString, +} + +#[cfg(not(target_env = "msvc"))] +impl Linker { + fn command( + &self, + context: &ExecutionContext, + object: &Path, + archive: &Path, + executable: &Path, + ) -> Command { + let mut command = Command::new(&self.compiler); + command + .current_dir(context.cwd()) + .env_clear() + .envs(context.envs()) + .arg(object) + .arg(archive) + .arg("-o") + .arg(executable); + if cfg!(target_os = "linux") { + command.args(["-ldl", "-lpthread", "-lm"]); + } + command + } + + fn diagnostics(&self, output: &Output) -> String { + String::from_utf8_lossy(&output.stderr).into_owned() + } +} + +#[cfg(not(target_env = "msvc"))] +fn discover_linker( + context: &ExecutionContext, + cancel: &CancellationToken, +) -> Result { + discover_toolchain(context, cancel).map(|compiler| Linker { compiler }) +} + +#[cfg(target_env = "msvc")] +use crate::windows_link::discover as discover_linker; + +#[cfg(not(target_env = "msvc"))] fn discover_toolchain( context: &ExecutionContext, cancel: &CancellationToken, @@ -2199,6 +2237,7 @@ fn discover_toolchain( probe_toolchain_in_context(program, context, cancel) } +#[cfg(any(test, not(target_env = "msvc")))] fn probe_toolchain_in_context( program: OsString, context: &ExecutionContext, @@ -2491,7 +2530,7 @@ fn effective_uid_in_context(context: &ExecutionContext) -> Result Result { let context = ExecutionContext::ambient()?; effective_uid_in_context(&context) @@ -2615,7 +2654,7 @@ fn load_program_frontend_in_context_with_cancel( current_directory.join(entrypoint) }; let absolute_entrypoint = - fs::canonicalize(&absolute_entrypoint).map_err(|source| DriverError::ReadSource { + canonicalize(&absolute_entrypoint).map_err(|source| DriverError::ReadSource { path: absolute_entrypoint, source, })?; @@ -2628,7 +2667,7 @@ fn load_program_frontend_in_context_with_cancel( .to_path_buf() }; let project = discover_project(&absolute_entrypoint, fallback_root); - let canonical_root = fs::canonicalize(project.root()) + let canonical_root = canonicalize(project.root()) .map_err(|error| DriverError::ProgramLoad(ProgramLoadError::InvalidRoot(error)))?; let root = ProjectRoot::new(canonical_root).map_err(|error| { DriverError::ProgramLoad(ProgramLoadError::InvalidRoot(std::io::Error::new( @@ -2780,6 +2819,7 @@ mod tests { use bamts_compiler::{ CancellationToken, lint::{LintLevel, SourceDialect, rule_by_name}, + project::canonical::slash_text, }; use crate::args::{ArgsError, parse_args}; @@ -3046,7 +3086,7 @@ exit 2 String::from_utf8(existing.stdout).expect("UTF-8 TS5054"), format!( "error TS5054: A 'tsconfig.json' file is already defined at: '{}'.\n", - config.display() + slash_text(&config) ) ); assert_eq!(fs::read(config).expect("read unchanged config"), original); @@ -3073,7 +3113,7 @@ exit 2 current, format!( "error TS5081: Cannot find a tsconfig.json file at the current directory: {}.\n", - fixture.0.join("tsconfig.json").display() + slash_text(&fixture.0.join("tsconfig.json")) ), ); @@ -3088,7 +3128,7 @@ exit 2 explicit_directory, format!( "error TS5057: Cannot find a tsconfig.json file at the specified directory: '{}'.\n", - directory.display() + slash_text(&directory) ), ); @@ -3100,7 +3140,10 @@ exit 2 .expect("file project executes"); assert_error( explicit_file, - format!("error TS5083: Cannot read file '{}'.\n", config.display()), + format!( + "error TS5083: Cannot read file '{}'.\n", + slash_text(&config) + ), ); fixture.write("empty.json", r#"{"files":[]}"#); @@ -3113,7 +3156,7 @@ exit 2 empty, format!( "error TS18002: The 'files' list in config file '{}' is empty.\n", - fixture.0.join("empty.json").display() + slash_text(&fixture.0.join("empty.json")) ), ); @@ -3130,14 +3173,14 @@ exit 2 none, format!( "error TS18003: No inputs were found in config file '{}'. Specified 'include' paths were '[\"src/**/*.ts\"]' and 'exclude' paths were '[\"build\"]'.\n", - fixture.0.join("none.json").display() + slash_text(&fixture.0.join("none.json")) ), ); let absolute_pattern = fixture.0.join("absolute/**/*.ts"); fixture.write( "absolute.json", - &format!(r#"{{"include":["{}"]}}"#, absolute_pattern.display()), + &format!(r#"{{"include":["{}"]}}"#, slash_text(&absolute_pattern)), ); let absolute = execute_tsc_in( &parse_tsc_args(["--project", "absolute.json"]).expect("absolute project parses"), @@ -3148,8 +3191,8 @@ exit 2 absolute, format!( "error TS18003: No inputs were found in config file '{}'. Specified 'include' paths were '[\"{}\"]' and 'exclude' paths were '[]'.\n", - fixture.0.join("absolute.json").display(), - absolute_pattern.display() + slash_text(&fixture.0.join("absolute.json")), + slash_text(&absolute_pattern) ), ); @@ -3167,7 +3210,7 @@ exit 2 inherited, format!( "error TS18003: No inputs were found in config file '{}'. Specified 'include' paths were '[\"../base/src/**/*.ts\"]' and 'exclude' paths were '[\"../base/build\"]'.\n", - fixture.0.join("app/tsconfig.json").display() + slash_text(&fixture.0.join("app/tsconfig.json")) ), ); } @@ -3194,8 +3237,8 @@ exit 2 assert!(fixture.0.join("list-dist/a.js").is_file()); assert!(fixture.0.join("list-dist/b.js").is_file()); let listed = String::from_utf8(listed.stdout).unwrap(); - let dependency = fixture.0.join("src/b.ts").display().to_string(); - let root = fixture.0.join("src/a.ts").display().to_string(); + let dependency = slash_text(&fixture.0.join("src/b.ts")); + let root = slash_text(&fixture.0.join("src/a.ts")); assert!(listed.contains(&dependency), "{listed}"); assert!(listed.contains(&root), "{listed}"); @@ -3228,7 +3271,7 @@ exit 2 assert_eq!(bad.exit_code, TscExitStatus::Success.code()); assert_eq!( String::from_utf8(bad.stdout).unwrap(), - format!("{}\n", fixture.0.join("bad.ts").display()) + format!("{}\n", slash_text(&fixture.0.join("bad.ts"))) ); } @@ -3294,7 +3337,9 @@ exit 2 let id = NEXT_TSC_TEST_ID.fetch_add(1, Ordering::Relaxed); let path = std::env::temp_dir().join(format!("bamts-tsc-emit-{}-{id}.ts", std::process::id())); - let emitted = path.with_extension(""); + // The emitted executable has the platform's executable extension: none on + // Unix, `exe` on Windows. + let emitted = path.with_extension(std::env::consts::EXE_EXTENSION); std::fs::write(&path, "const value: number = \"text\";\n").expect("write source"); let outcome = execute_tsc( @@ -3897,7 +3942,7 @@ printf '#!/bin/sh\npwd\n' > "$output" Ok(()) } - #[cfg(target_os = "linux")] + #[cfg(unix)] #[test] fn managed_capture_kills_descendant_without_external_kill_before_reader_join() -> Result<(), Box> { @@ -3942,10 +3987,12 @@ printf '#!/bin/sh\npwd\n' > "$output" let pid = std::fs::read_to_string(&pid_file)?.trim().parse::()?; let deadline = Instant::now() + Duration::from_secs(2); loop { - let alive = std::fs::read_to_string(format!("/proc/{pid}/stat")) + let alive = process::Command::new("/bin/ps") + .args(["-o", "stat=", "-p", &pid.to_string()]) + .output() .ok() - .and_then(|stat| stat.split_whitespace().nth(2).map(str::to_owned)) - .is_some_and(|state| state != "Z"); + .map(|output| String::from_utf8_lossy(&output.stdout).trim().to_owned()) + .is_some_and(|state| !state.is_empty() && !state.starts_with('Z')); if !alive { break; } diff --git a/crates/bamts-cli/src/lib.rs b/crates/bamts-cli/src/lib.rs index 71ab3b62..5429c047 100755 --- a/crates/bamts-cli/src/lib.rs +++ b/crates/bamts-cli/src/lib.rs @@ -8,3 +8,5 @@ pub mod diagnostics; pub mod driver; pub mod lsp; pub mod output; +#[cfg(any(target_env = "msvc", test))] +mod windows_link; diff --git a/crates/bamts-cli/src/lsp.rs b/crates/bamts-cli/src/lsp.rs index 03654f7c..ed547be0 100644 --- a/crates/bamts-cli/src/lsp.rs +++ b/crates/bamts-cli/src/lsp.rs @@ -12,6 +12,7 @@ use std::{ use bamts_compiler::checker::SymbolKind; use bamts_compiler::diagnostic::DiagnosticSeverity; +use bamts_compiler::project::canonical::canonicalize; use bamts_compiler::service::{ DiagnosticEntry, DocumentSnapshot, Location, ServiceError, ServiceState, filesystem::{FileSystemError, OsFileSystem}, @@ -96,7 +97,7 @@ struct Session { impl Session { fn new(root: &Path) -> io::Result { - let process_root = root.canonicalize()?; + let process_root = canonicalize(root)?; let filesystem = OsFileSystem::new(&process_root).map_err(fs_io)?; Ok(Self { state: ServiceState::new(filesystem), @@ -732,15 +733,70 @@ fn uri_to_path(uri: &str) -> Result { None => return Err("file URI has no path".to_owned()), }; let decoded = percent_decode(&format!("/{path}"))?; - Ok(PathBuf::from(decoded)) + Ok(path_from_uri_text(decoded)) +} + +/// The file-system path that a decoded URI path names. +/// +/// RFC 8089 spells a Windows drive path as `/C:/dir/file.ts`; clients that +/// percent-encode the colon send `/c:/dir/file.ts` after decoding. The leading +/// slash is not part of the path. Everywhere else the text is the path. +#[cfg(windows)] +fn path_from_uri_text(decoded: String) -> PathBuf { + match drive_colon_index(&decoded) { + Some(_) => PathBuf::from(&decoded[1..]), + None => PathBuf::from(decoded), + } +} + +#[cfg(not(windows))] +fn path_from_uri_text(decoded: String) -> PathBuf { + PathBuf::from(decoded) +} + +/// The URI path text of `path`: segments are separated by `/`, and a Windows +/// drive path gains the leading slash that RFC 8089 puts before the drive letter. +#[cfg(windows)] +fn uri_path_text(path: &Path) -> String { + let text = path.to_string_lossy().replace('\\', "/"); + let rooted = format!("/{text}"); + if drive_colon_index(&rooted).is_some() { + rooted + } else { + text + } +} + +#[cfg(not(windows))] +fn uri_path_text(path: &Path) -> String { + path.to_string_lossy().into_owned() +} + +/// The byte index of the colon in a leading `/X:` drive designator that is +/// followed by `/` or ends the text. +#[cfg(windows)] +fn drive_colon_index(text: &str) -> Option { + match text.as_bytes() { + [b'/', drive, b':'] | [b'/', drive, b':', b'/', ..] if drive.is_ascii_alphabetic() => { + Some(2) + } + _ => None, + } +} + +#[cfg(not(windows))] +fn drive_colon_index(_text: &str) -> Option { + None } fn path_to_uri(path: &Path) -> String { - let display = path.to_string_lossy(); + let text = uri_path_text(path); + let drive_colon = drive_colon_index(&text); let mut encoded = String::from("file://"); - for ch in display.chars() { + for (index, ch) in text.char_indices() { match ch { 'A'..='Z' | 'a'..='z' | '0'..='9' | '/' | '_' | '-' | '.' | '~' => encoded.push(ch), + ':' if Some(index) == drive_colon => encoded.push(ch), _ => { for byte in ch.encode_utf8(&mut [0; 4]).bytes() { encoded.push_str(&format!("%{byte:02X}")); @@ -963,8 +1019,7 @@ fn initialize_root(params: &Value, process_root: &Path) -> Result Result { - let canonical = path - .canonicalize() + let canonical = canonicalize(path) .map_err(|error| format!("cannot canonicalize {}: {error}", path.display()))?; if canonical.strip_prefix(root).is_err() { return Err(format!( @@ -1408,6 +1463,55 @@ mod tests { assert!(uri_to_path("file://evil.com").is_err()); } + #[test] + fn uris_round_trip_through_percent_encoding() { + let path = Path::new("/tmp/a b/é~1.ts"); + let uri = path_to_uri(path); + + assert_eq!(uri, "file:///tmp/a%20b/%C3%A9~1.ts"); + assert_eq!(uri_to_path(&uri).expect("round trip"), path); + } + + #[test] + #[cfg(windows)] + fn drive_paths_use_rfc_8089_uris_in_both_directions() { + let path = Path::new(r"C:\Users\me\a b\é.ts"); + let uri = path_to_uri(path); + + assert_eq!(uri, "file:///C:/Users/me/a%20b/%C3%A9.ts"); + assert_eq!(uri_to_path(&uri).expect("round trip"), path); + for client_uri in [ + "file:///C:/Users/me/a%20b/%C3%A9.ts", + "file:///c%3A/Users/me/a%20b/%C3%A9.ts", + "file:///c%3a/Users/me/a%20b/%C3%A9.ts", + ] { + let parsed = uri_to_path(client_uri).expect(client_uri); + assert!(parsed.is_absolute(), "{client_uri} -> {}", parsed.display()); + assert!( + parsed + .to_string_lossy() + .eq_ignore_ascii_case(&path.to_string_lossy().replace('\\', "/")), + "{client_uri} -> {}", + parsed.display() + ); + } + assert_eq!(path_to_uri(Path::new(r"D:\")), "file:///D:/"); + assert_eq!( + uri_to_path("file:///D:").expect("drive root"), + Path::new("D:") + ); + } + + #[test] + #[cfg(windows)] + fn rooted_paths_without_a_drive_keep_a_single_leading_slash() { + assert_eq!(path_to_uri(Path::new(r"\tmp\a.ts")), "file:///tmp/a.ts"); + assert_eq!( + uri_to_path("file:///tmp/a.ts").expect("rooted"), + Path::new("/tmp/a.ts") + ); + } + #[test] fn references_exclude_declaration_only_when_asked() { let root = tempfile::tempdir().expect("temp"); @@ -2306,6 +2410,7 @@ mod tests { assert!(results_with_id(&messages, &json!(1))[0]["result"].is_object()); } + #[cfg(unix)] #[test] fn document_uris_must_stay_inside_the_workspace_root() { let root = tempfile::tempdir().expect("temp"); diff --git a/crates/bamts-cli/src/output.rs b/crates/bamts-cli/src/output.rs index 4cca246e..f7cee26f 100644 --- a/crates/bamts-cli/src/output.rs +++ b/crates/bamts-cli/src/output.rs @@ -438,6 +438,13 @@ impl AtomicFs for StdAtomicFs { } fn sync_dir(&mut self, dir: &Path) -> Result<(), Self::Error> { + // Windows is skipped: `File::open` cannot open a directory there. A handle + // opened with `FILE_FLAG_BACKUP_SEMANTICS` for writing can be flushed on + // NTFS, but this code does not do that. NTFS journals the rename, which is + // not a promise that it is on disk after a power loss. + if cfg!(windows) { + return Ok(()); + } File::open(dir)?.sync_all() } @@ -680,6 +687,15 @@ mod tests { use super::*; use std::collections::BTreeMap; + /// A fixture path written the Unix way, on the drive Windows needs to call it absolute. + fn abs(unix: &str) -> PathBuf { + if cfg!(windows) { + PathBuf::from(format!("C:{}", unix.replace('/', "\\"))) + } else { + PathBuf::from(unix) + } + } + fn request<'a>(root: &'a Path, entrypoints: &'a [PathBuf]) -> PlanRequest<'a> { PlanRequest { root, @@ -693,9 +709,9 @@ mod tests { #[test] fn plans_confined_source_relative_outputs_in_stable_order() { - let root = Path::new("/workspace"); + let root = abs("/workspace"); let entries = vec![PathBuf::from("src/main.mts"), PathBuf::from("lib/util.cts")]; - let mut input = request(root, &entries); + let mut input = request(&root, &entries); input.output_dir = Some(Path::new("dist/./generated")); input.emit_declarations = true; input.source_maps = true; @@ -711,27 +727,27 @@ mod tests { vec![ ( ArtifactKind::JavaScript, - Path::new("/workspace/dist/generated/src/main.mjs") + abs("/workspace/dist/generated/src/main.mjs").as_path() ), ( ArtifactKind::Declaration, - Path::new("/workspace/dist/generated/src/main.d.mts") + abs("/workspace/dist/generated/src/main.d.mts").as_path() ), ( ArtifactKind::SourceMap, - Path::new("/workspace/dist/generated/src/main.mjs.map") + abs("/workspace/dist/generated/src/main.mjs.map").as_path() ), ( ArtifactKind::JavaScript, - Path::new("/workspace/dist/generated/lib/util.cjs") + abs("/workspace/dist/generated/lib/util.cjs").as_path() ), ( ArtifactKind::Declaration, - Path::new("/workspace/dist/generated/lib/util.d.cts") + abs("/workspace/dist/generated/lib/util.d.cts").as_path() ), ( ArtifactKind::SourceMap, - Path::new("/workspace/dist/generated/lib/util.cjs.map") + abs("/workspace/dist/generated/lib/util.cjs.map").as_path() ), ] ); @@ -740,7 +756,8 @@ mod tests { #[test] fn exact_output_derives_declaration_and_map_companions() { let entries = vec![PathBuf::from("src/main.ts")]; - let mut input = request(Path::new("/workspace"), &entries); + let root = abs("/workspace"); + let mut input = request(&root, &entries); input.output_file = Some(Path::new("build/bundle.js")); input.emit_declarations = true; input.source_maps = true; @@ -754,9 +771,9 @@ mod tests { assert_eq!( paths, vec![ - Path::new("/workspace/build/bundle.js"), - Path::new("/workspace/build/bundle.d.ts"), - Path::new("/workspace/build/bundle.js.map"), + abs("/workspace/build/bundle.js").as_path(), + abs("/workspace/build/bundle.d.ts").as_path(), + abs("/workspace/build/bundle.js.map").as_path(), ] ); } @@ -770,7 +787,8 @@ mod tests { PathBuf::from(OsStr::from_bytes(b"src/\xff.ts")), PathBuf::from(OsStr::from_bytes(b"src/\xfe.mts")), ]; - let mut input = request(Path::new("/workspace"), &entries); + let root = abs("/workspace"); + let mut input = request(&root, &entries); input.output_dir = Some(Path::new("dist")); let plan = plan_outputs(input).unwrap(); @@ -789,7 +807,8 @@ mod tests { #[test] fn rejects_lexical_path_traversal() { let entries = vec![PathBuf::from("src/main.ts")]; - let mut input = request(Path::new("/workspace/project"), &entries); + let root = abs("/workspace/project"); + let mut input = request(&root, &entries); input.output_dir = Some(Path::new("../../escape")); assert!(matches!( @@ -800,8 +819,9 @@ mod tests { #[test] fn rejects_absolute_paths_outside_root() { - let entries = vec![PathBuf::from("/other/main.ts")]; - let input = request(Path::new("/workspace"), &entries); + let entries = vec![abs("/other/main.ts")]; + let root = abs("/workspace"); + let input = request(&root, &entries); assert!(matches!( plan_outputs(input), @@ -815,7 +835,8 @@ mod tests { PathBuf::from("src/value.ts"), PathBuf::from("src/value.tsx"), ]; - let mut input = request(Path::new("/workspace"), &entries); + let root = abs("/workspace"); + let mut input = request(&root, &entries); input.output_dir = Some(Path::new("dist/a/../")); assert!(matches!( @@ -824,19 +845,20 @@ mod tests { path, second_kind: ArtifactKind::JavaScript, .. - }) if path == Path::new("/workspace/dist/src/value.js") + }) if path == abs("/workspace/dist/src/value.js").as_path() )); } #[test] fn rejects_outputs_that_clobber_javascript_inputs() { let entries = vec![PathBuf::from("src/value.js")]; - let input = request(Path::new("/workspace"), &entries); + let root = abs("/workspace"); + let input = request(&root, &entries); assert!(matches!( plan_outputs(input), Err(PlanError::ClobbersInput { path, .. }) - if path == Path::new("/workspace/src/value.js") + if path == abs("/workspace/src/value.js").as_path() )); } @@ -1091,10 +1113,10 @@ mod tests { #[test] fn rejects_entrypoint_normalizing_to_root() { - let root = PathBuf::from("/workspace"); + let root = abs("/workspace"); for entrypoint in [ PathBuf::from("."), - PathBuf::from("/workspace"), + abs("/workspace"), PathBuf::from("src/.."), ] { let entrypoints = [entrypoint.clone()]; diff --git a/crates/bamts-cli/src/windows_link.rs b/crates/bamts-cli/src/windows_link.rs new file mode 100644 index 00000000..6b639be9 --- /dev/null +++ b/crates/bamts-cli/src/windows_link.rs @@ -0,0 +1,962 @@ +//! Linking a native executable with the Microsoft linker. +//! +//! The Unix path hands the object and the runtime archive to `cc`. On MSVC the +//! linker is `link.exe`, which is not a compiler driver. It needs the object, the +//! archive, the system libraries that the Rust runtime depends on, and a `LIB` +//! search path that only a Visual Studio command prompt sets. This module finds the +//! linker, builds that search path and builds the command line. +//! +//! The linker is found in this order, and the first answer wins: +//! +//! 1. `BAMTS_LINKER`, which names the linker. A wrong value is an error; the +//! search does not go on, so a typo cannot pick another linker silently. +//! 2. A `link.exe` on `PATH` that prints the Microsoft banner. Git for Windows puts +//! a GNU `link.exe` on `PATH`, and it is skipped. +//! 3. The newest Visual Studio with the C++ tools, found with `vswhere.exe` at its +//! fixed place in the Visual Studio Installer directory. +//! +//! The pure parts build on every host, so their tests run on Linux and macOS too. + +use std::ffi::{OsStr, OsString}; +use std::path::{Path, PathBuf}; + +#[cfg(target_env = "msvc")] +use std::{ + fs, io, + path::{Component, Prefix}, + process::Command, +}; + +#[cfg(target_env = "msvc")] +use bamts_compiler::CancellationToken; + +#[cfg(target_env = "msvc")] +use crate::context::ExecutionContext; +#[cfg(target_env = "msvc")] +use crate::driver::{DriverError, capture_process_with_cancel}; + +/// The text the Microsoft linker prints first, whatever it is asked. +const BANNER: &str = "Microsoft (R) Incremental Linker"; + +/// The component that a Visual Studio installation needs for `link.exe`. +#[cfg(target_env = "msvc")] +const CPP_TOOLS_COMPONENT: &str = "Microsoft.VisualStudio.Component.VC.Tools.x86.x64"; + +/// Whether what a program printed is the banner of the Microsoft linker. +fn is_microsoft_linker(stdout: &str, stderr: &str) -> bool { + stdout.contains(BANNER) || stderr.contains(BANNER) +} + +/// The numeric parts of a version-like directory name, such as `14.44.35207` or +/// `10.0.26100.0`. A name with any other part is not a version. +fn version_key(name: &str) -> Option> { + name.split('.').map(|part| part.parse().ok()).collect() +} + +/// The newest of the names that are versions, compared part by part, so `14.9` is +/// older than `14.44`. +fn newest_version<'a>(names: impl IntoIterator) -> Option<&'a str> { + names + .into_iter() + .filter_map(|name| version_key(name).map(|key| (key, name))) + .max_by(|left, right| left.0.cmp(&right.0)) + .map(|(_, name)| name) +} + +/// The install path in what `vswhere -property installationPath` printed: its +/// first non-empty line. +fn installation_path(vswhere_output: &str) -> Option { + vswhere_output + .lines() + .map(str::trim) + .find(|line| !line.is_empty()) + .map(PathBuf::from) +} + +/// The toolset directory of a `link.exe` that sits where Visual Studio puts it, +/// `...\VC\Tools\MSVC\\bin\Host\\link.exe`. +fn toolset_of(linker: &Path) -> Option { + let host = linker.parent()?.parent()?; + let binaries = host.parent()?; + let is_host = host.file_name()?.to_str().is_some_and(|name| { + name.len() > 4 + && name + .get(..4) + .is_some_and(|prefix| prefix.eq_ignore_ascii_case("host")) + }); + (is_host && binaries.file_name()?.eq_ignore_ascii_case("bin")) + .then(|| binaries.parent().map(Path::to_path_buf))? +} + +/// The library directories of a toolset and a Windows SDK, in the order that the +/// Visual Studio command prompt puts them in `LIB`. +fn library_directories(toolset: &Path, sdk_lib: &Path, sdk_version: &str) -> Vec { + let versioned = sdk_lib.join(sdk_version); + vec![ + toolset.join("lib").join("x64"), + versioned.join("ucrt").join("x64"), + versioned.join("um").join("x64"), + ] +} + +/// The arguments that link the object and the runtime archive into `output`. +/// +/// `/BREPRO` leaves the build time out of the file, so the same inputs give the +/// same bytes. `native_libraries` is what rustc reported for the archive, such as +/// `kernel32.lib ntdll.lib /defaultlib:msvcrt`; the archive carries none of them. +fn link_arguments( + object: &Path, + archive: &Path, + output: &Path, + native_libraries: &str, +) -> Vec { + let mut arguments: Vec = ["/NOLOGO", "/MACHINE:X64", "/SUBSYSTEM:CONSOLE", "/BREPRO"] + .map(OsString::from) + .to_vec(); + let mut output_option = OsString::from("/OUT:"); + output_option.push(output); + arguments.push(output_option); + arguments.push(object.as_os_str().to_owned()); + arguments.push(archive.as_os_str().to_owned()); + arguments.extend(native_libraries.split_whitespace().map(OsString::from)); + arguments +} + +/// What a failed link says. `link.exe` prints its errors on standard output and +/// leaves standard error empty, so both streams are kept, output first. +fn link_diagnostics(stdout: &str, stderr: &str) -> String { + [stdout, stderr] + .map(str::trim) + .into_iter() + .filter(|text| !text.is_empty()) + .collect::>() + .join("\n") +} + +/// A variable of the context, found without regard to case: Windows spells the +/// search path `Path`, and the environment of the context is a case-sensitive map. +fn variable(context: &crate::context::ExecutionContext, name: &str) -> Option { + crate::context::env_ignoring_case(context.envs(), name).map(OsStr::to_os_string) +} + +/// The linker and everything it needs to run. +#[cfg(target_env = "msvc")] +pub(crate) struct Linker { + program: PathBuf, + /// The `LIB` value to give the linker, or `None` when the context has its own. + library_path: Option, + native_libraries: &'static str, +} + +#[cfg(target_env = "msvc")] +impl Linker { + /// The text to show when the link failed. + pub(crate) fn diagnostics(&self, output: &std::process::Output) -> String { + link_diagnostics( + &String::from_utf8_lossy(&output.stdout), + &String::from_utf8_lossy(&output.stderr), + ) + } + + /// The command that links `object` and `archive` into `executable`. + /// + /// The linker gets the environment of the context. The two variables that + /// `link.exe` reads as extra options, `LINK` and `_LINK_`, are dropped, so an + /// option set in a shell cannot change what is built. + pub(crate) fn command( + &self, + context: &ExecutionContext, + object: &Path, + archive: &Path, + executable: &Path, + ) -> Command { + let mut command = context_command(&self.program, context); + command.env_remove("LINK").env_remove("_LINK_"); + if let Some(library_path) = &self.library_path { + command.env("LIB", library_path); + } + command.args(link_arguments( + object, + archive, + executable, + self.native_libraries, + )); + command + } +} + +/// The system libraries that rustc says the embedded runtime archive needs, which +/// the MSVC linker does not take from the archive. +#[cfg(target_env = "msvc")] +const NODE_NATIVE_LIBRARIES: &str = env!("BAMTS_NODE_NATIVE_LIBS"); + +/// Finds the linker and the library path to give it. +/// +/// # Errors +/// [`DriverError::ToolchainMissing`] when no linker is found, and the probe errors +/// when a program cannot be run. +#[cfg(target_env = "msvc")] +pub(crate) fn discover( + context: &ExecutionContext, + cancel: &CancellationToken, +) -> Result { + discover_with(context, cancel, NODE_NATIVE_LIBRARIES) +} + +/// [`discover`] for a given list of the libraries that rustc reported for the +/// runtime archive. +#[cfg(target_env = "msvc")] +fn discover_with( + context: &ExecutionContext, + cancel: &CancellationToken, + native_libraries: &'static str, +) -> Result { + let program = if let Some(program) = explicit_linker(context, cancel)? { + program + } else if let Some(program) = linker_on_path(context, cancel)? { + program + } else if let Some(program) = linker_from_visual_studio(context, cancel)? { + program + } else { + return Err(DriverError::ToolchainMissing { + program: OsString::from("link.exe"), + }); + }; + let library_path = library_path(context, &program); + Ok(Linker { + program, + library_path, + native_libraries, + }) +} + +/// The root of the drive in `path`, such as `C:\` for `C:\Windows`. +#[cfg(target_env = "msvc")] +fn drive_root(path: &Path) -> Option { + let mut components = path.components(); + match (components.next()?, components.next()?) { + (Component::Prefix(prefix), Component::RootDir) + if matches!(prefix.kind(), Prefix::Disk(_) | Prefix::VerbatimDisk(_)) => + { + let mut root = PathBuf::from(prefix.as_os_str()); + root.push(Component::RootDir); + Some(root) + } + _ => None, + } +} + +/// The root of the drive that holds Windows. The system folders that the context +/// does not name are looked for there, so a process with a cleared environment +/// finds them without asking the host process, which keeps the answer the same. +#[cfg(target_env = "msvc")] +fn system_drive(context: &ExecutionContext) -> PathBuf { + variable(context, "SystemRoot") + .and_then(|root| drive_root(Path::new(&root))) + .unwrap_or_else(|| PathBuf::from(r"C:\")) +} + +#[cfg(target_env = "msvc")] +fn program_files_x86(context: &ExecutionContext) -> PathBuf { + variable(context, "ProgramFiles(x86)").map_or_else( + || system_drive(context).join("Program Files (x86)"), + PathBuf::from, + ) +} + +/// `vswhere` reads the Visual Studio setup store under `ProgramData`. Without it +/// `vswhere` finds no installation and still exits with success. +#[cfg(target_env = "msvc")] +fn program_data(context: &ExecutionContext) -> PathBuf { + variable(context, "ProgramData") + .map_or_else(|| system_drive(context).join("ProgramData"), PathBuf::from) +} + +/// A command that runs in the working directory of the context with exactly the +/// environment of the context. +#[cfg(target_env = "msvc")] +fn context_command(program: impl AsRef, context: &ExecutionContext) -> Command { + let mut command = Command::new(program); + command + .current_dir(context.cwd()) + .env_clear() + .envs(context.envs()); + command +} + +/// Runs `program /?` and returns what it printed, or `None` when there is no such +/// program. +#[cfg(target_env = "msvc")] +fn ask_for_help( + program: &Path, + context: &ExecutionContext, + cancel: &CancellationToken, +) -> Result, DriverError> { + let mut command = context_command(program, context); + command.arg("/?"); + match capture_process_with_cancel(&mut command, cancel) { + Ok(Some(output)) => Ok(Some(output)), + Ok(None) => Err(DriverError::Cancelled), + Err(source) if source.kind() == io::ErrorKind::NotFound => Ok(None), + Err(source) => Err(DriverError::ToolchainProbe { + program: program.as_os_str().to_owned(), + source, + }), + } +} + +/// The linker that `BAMTS_LINKER` names. It is taken as it is: any program that +/// starts is accepted, so a compatible linker such as `lld-link.exe` can be named. +/// The library path is worked out only for a linker in a Visual Studio layout, so +/// any other linker finds its libraries through the `LIB` of the context. +#[cfg(target_env = "msvc")] +fn explicit_linker( + context: &ExecutionContext, + cancel: &CancellationToken, +) -> Result, DriverError> { + let Some(value) = variable(context, "BAMTS_LINKER") else { + return Ok(None); + }; + let named = Path::new(&value); + let program = if named.components().count() > 1 { + context.resolve_path(named) + } else { + path_candidates(context, named) + .next() + .unwrap_or_else(|| named.to_path_buf()) + }; + match ask_for_help(&program, context, cancel)? { + Some(_) => Ok(Some(program)), + None => Err(DriverError::ToolchainMissing { program: value }), + } +} + +/// The files called `name` in the directories of `PATH`, in search order. +#[cfg(target_env = "msvc")] +fn path_candidates(context: &ExecutionContext, name: &Path) -> impl Iterator { + let directories: Vec = variable(context, "PATH") + .map(|search| std::env::split_paths(&search).collect()) + .unwrap_or_default(); + directories + .into_iter() + .map(move |directory| directory.join(name)) + .filter(|candidate| candidate.is_file()) +} + +/// The first `link.exe` on `PATH` that is the Microsoft linker. +#[cfg(target_env = "msvc")] +fn linker_on_path( + context: &ExecutionContext, + cancel: &CancellationToken, +) -> Result, DriverError> { + for candidate in path_candidates(context, Path::new("link.exe")) { + match ask_for_help(&candidate, context, cancel) { + Ok(Some(output)) + if is_microsoft_linker( + &String::from_utf8_lossy(&output.stdout), + &String::from_utf8_lossy(&output.stderr), + ) => + { + return Ok(Some(candidate)); + } + Err(DriverError::Cancelled) => return Err(DriverError::Cancelled), + // Another program called link.exe, or one that does not start: not ours. + _ => {} + } + } + Ok(None) +} + +/// The `link.exe` of the newest Visual Studio that has the C++ tools. +#[cfg(target_env = "msvc")] +fn linker_from_visual_studio( + context: &ExecutionContext, + cancel: &CancellationToken, +) -> Result, DriverError> { + let vswhere = program_files_x86(context) + .join("Microsoft Visual Studio") + .join("Installer") + .join("vswhere.exe"); + if !vswhere.is_file() { + return Ok(None); + } + let mut command = context_command(&vswhere, context); + command + .env("ProgramData", program_data(context)) + .args([ + "-latest", + "-products", + "*", + "-requires", + CPP_TOOLS_COMPONENT, + ]) + .args(["-property", "installationPath", "-utf8"]); + let output = match capture_process_with_cancel(&mut command, cancel) { + Ok(Some(output)) => output, + Ok(None) => return Err(DriverError::Cancelled), + Err(source) => { + return Err(DriverError::ToolchainProbe { + program: vswhere.into_os_string(), + source, + }); + } + }; + if !output.status.success() { + return Ok(None); + } + let Some(installation) = installation_path(&String::from_utf8_lossy(&output.stdout)) else { + return Ok(None); + }; + let Some(toolset) = toolset_directory(&installation) else { + return Ok(None); + }; + let linker = toolset + .join("bin") + .join("Hostx64") + .join("x64") + .join("link.exe"); + Ok(linker.is_file().then_some(linker)) +} + +/// The toolset that Visual Studio's own command prompt uses: the one named in +/// `Microsoft.VCToolsVersion.default.txt`, or the newest one when that file does +/// not name an installed one. +#[cfg(target_env = "msvc")] +fn toolset_directory(installation: &Path) -> Option { + let tools = installation.join("VC").join("Tools").join("MSVC"); + let default = fs::read_to_string( + installation + .join("VC") + .join("Auxiliary") + .join("Build") + .join("Microsoft.VCToolsVersion.default.txt"), + ) + .ok() + .map(|text| text.trim().to_owned()) + .filter(|version| !version.is_empty() && tools.join(version).is_dir()); + if let Some(version) = default { + return Some(tools.join(version)); + } + let names = entry_names(&tools)?; + newest_version(names.iter().map(String::as_str)).map(|version| tools.join(version)) +} + +/// The names of the entries of `directory`, leaving out any that is not text. +#[cfg(target_env = "msvc")] +fn entry_names(directory: &Path) -> Option> { + Some( + fs::read_dir(directory) + .ok()? + .filter_map(Result::ok) + .filter_map(|entry| entry.file_name().into_string().ok()) + .collect(), + ) +} + +/// The `LIB` value to give the linker, or `None` when the context sets its own, as +/// a Visual Studio command prompt does, or when the directories cannot be worked +/// out. In that case the linker reports the first library it cannot open. +#[cfg(target_env = "msvc")] +fn library_path(context: &ExecutionContext, linker: &Path) -> Option { + if variable(context, "LIB").is_some() { + return None; + } + let toolset = toolset_of(linker)?; + let kits = variable(context, "WindowsSdkDir").map_or_else( + || program_files_x86(context).join("Windows Kits").join("10"), + PathBuf::from, + ); + let sdk_lib = kits.join("Lib"); + // An SDK counts when it has both the import libraries and the C runtime ones. + let complete = |version: &Path| { + version + .join("um") + .join("x64") + .join("kernel32.lib") + .is_file() + && version.join("ucrt").join("x64").join("ucrt.lib").is_file() + }; + let versions: Vec = entry_names(&sdk_lib)? + .into_iter() + .filter(|name| complete(&sdk_lib.join(name))) + .collect(); + let version = newest_version(versions.iter().map(String::as_str))?; + std::env::join_paths(library_directories(&toolset, &sdk_lib, version)).ok() +} + +#[cfg(test)] +mod tests { + use std::collections::BTreeMap; + + use super::*; + use crate::context::ExecutionContext; + + fn context_with(variables: &[(&str, &str)]) -> ExecutionContext { + let env = variables + .iter() + .map(|(key, value)| (OsString::from(key), OsString::from(value))) + .collect::>(); + ExecutionContext::new(std::env::temp_dir(), env).expect("execution context") + } + + #[test] + fn only_the_microsoft_banner_marks_the_microsoft_linker() { + let banner = "Microsoft (R) Incremental Linker Version 14.44.35228.0\nCopyright (C) Microsoft Corporation. All rights reserved.\n"; + assert!(is_microsoft_linker(banner, "")); + assert!(is_microsoft_linker("", banner)); + // GNU `link` and the Windows command processor are not the linker. + assert!(!is_microsoft_linker( + "", + "/usr/bin/link: missing operand after '/?'\n" + )); + assert!(!is_microsoft_linker( + "Starts a new instance of the Windows command interpreter\n", + "" + )); + assert!(!is_microsoft_linker("", "")); + } + + #[test] + fn versions_are_compared_part_by_part() { + assert_eq!(version_key("14.44.35207"), Some(vec![14, 44, 35207])); + assert_eq!(version_key("10.0.26100.0"), Some(vec![10, 0, 26100, 0])); + assert_eq!(version_key("latest"), None); + assert_eq!(version_key("14.x"), None); + assert_eq!(version_key(""), None); + + // A text comparison would put 14.9 after 14.44, and 10.0.9 after 10.0.26100. + assert_eq!( + newest_version(["14.9.1", "14.44.35207", "14.43.34808"]), + Some("14.44.35207") + ); + assert_eq!( + newest_version(["10.0.9600.0", "10.0.26100.0", "wdf", "10.0.22621.0"]), + Some("10.0.26100.0") + ); + assert_eq!(newest_version(["wdf", "um"]), None); + assert_eq!(newest_version([]), None); + } + + #[test] + fn the_install_path_is_the_first_line_vswhere_prints() { + assert_eq!( + installation_path("C:\\Program Files\\Microsoft Visual Studio\\2022\\Community\r\n"), + Some(PathBuf::from( + "C:\\Program Files\\Microsoft Visual Studio\\2022\\Community" + )) + ); + assert_eq!( + installation_path("\n C:\\VS \nC:\\Older\n"), + Some(PathBuf::from("C:\\VS")) + ); + assert_eq!(installation_path(""), None); + assert_eq!(installation_path("\r\n \r\n"), None); + } + + #[test] + fn the_toolset_is_found_from_the_place_of_link_exe() { + let linker = Path::new("root/VC/Tools/MSVC/14.44.35207/bin/Hostx64/x64/link.exe"); + assert_eq!( + toolset_of(linker), + Some(PathBuf::from("root/VC/Tools/MSVC/14.44.35207")) + ); + let upper = Path::new("root/MSVC/14.1/BIN/Hostx86/x64/link.exe"); + assert_eq!(toolset_of(upper), Some(PathBuf::from("root/MSVC/14.1"))); + // A linker that is not inside a toolset has none. + assert_eq!(toolset_of(Path::new("tools/link.exe")), None); + assert_eq!(toolset_of(Path::new("llvm/bin/x64/lld/link.exe")), None); + assert_eq!(toolset_of(Path::new("link.exe")), None); + } + + #[test] + fn a_directory_name_that_is_not_ascii_is_not_a_host_directory() { + // Byte 4 of each name is inside a character, so slicing there would panic. + for name in ["Café", "개발"] { + let linker = PathBuf::from("root").join("bin").join(name).join("x64"); + assert_eq!(toolset_of(&linker.join("link.exe")), None, "{name}"); + } + } + + #[test] + fn library_directories_follow_the_visual_studio_prompt_order() { + let directories = library_directories( + Path::new("tools/14.44"), + Path::new("kits/Lib"), + "10.0.26100.0", + ); + let expected: Vec = [ + "tools/14.44/lib/x64", + "kits/Lib/10.0.26100.0/ucrt/x64", + "kits/Lib/10.0.26100.0/um/x64", + ] + .iter() + .map(PathBuf::from) + .collect(); + assert_eq!(directories, expected); + } + + #[test] + fn the_link_command_line_names_the_inputs_and_the_system_libraries() { + let arguments = link_arguments( + Path::new("out/.app.o"), + Path::new("cache/bamts-node.lib"), + Path::new("out/app with space.exe"), + "kernel32.lib ntdll.lib\t/defaultlib:msvcrt", + ); + let text: Vec = arguments + .iter() + .map(|argument| argument.to_string_lossy().into_owned()) + .collect(); + assert_eq!( + text, + [ + "/NOLOGO", + "/MACHINE:X64", + "/SUBSYSTEM:CONSOLE", + "/BREPRO", + "/OUT:out/app with space.exe", + "out/.app.o", + "cache/bamts-node.lib", + "kernel32.lib", + "ntdll.lib", + "/defaultlib:msvcrt", + ] + ); + // No system libraries gives no extra arguments, not an empty one. + let bare = link_arguments(Path::new("a.o"), Path::new("b.lib"), Path::new("c.exe"), ""); + assert_eq!(bare.len(), 7); + } + + #[test] + fn a_failed_link_keeps_the_output_and_the_error_stream() { + let error = "LINK : fatal error LNK1561: entry point must be defined\r\n"; + assert_eq!(link_diagnostics(error, ""), error.trim()); + assert_eq!(link_diagnostics("", " cannot start \n"), "cannot start"); + assert_eq!(link_diagnostics("first\n", "second\n"), "first\nsecond"); + assert_eq!(link_diagnostics("", ""), ""); + } + + #[test] + fn variables_are_found_without_regard_to_case() { + let context = context_with(&[("Path", "C:\\a;C:\\b"), ("ProgramFiles(x86)", "D:\\PF")]); + assert_eq!( + variable(&context, "PATH"), + Some(OsString::from("C:\\a;C:\\b")) + ); + assert_eq!( + variable(&context, "path"), + Some(OsString::from("C:\\a;C:\\b")) + ); + assert_eq!( + variable(&context, "programfiles(x86)"), + Some(OsString::from("D:\\PF")) + ); + assert_eq!(variable(&context, "LIB"), None); + } + + #[cfg(target_env = "msvc")] + mod live { + use std::ffi::OsString; + use std::path::{Path, PathBuf}; + + use bamts_compiler::CancellationToken; + + use super::*; + use crate::driver::DriverError; + + /// What a hermetic process still has: no `PATH`, no `LIB`, no Visual Studio + /// command prompt. `extra` replaces or adds variables. + fn context(extra: &[(&str, &str)]) -> ExecutionContext { + let mut variables: Vec<(String, String)> = + ["SystemRoot", "windir", "TEMP", "TMP", "ProgramFiles(x86)"] + .into_iter() + .filter_map(|name| Some((name.to_owned(), std::env::var(name).ok()?))) + .collect(); + for (name, value) in extra { + variables.retain(|(existing, _)| !existing.eq_ignore_ascii_case(name)); + variables.push(((*name).to_owned(), (*value).to_owned())); + } + let pairs: Vec<(&str, &str)> = variables + .iter() + .map(|(name, value)| (name.as_str(), value.as_str())) + .collect(); + context_with(&pairs) + } + + fn discover_in(context: &ExecutionContext) -> Result { + discover_with(context, &CancellationToken::new(), "kernel32.lib ntdll.lib") + } + + fn visual_studio_linker() -> Linker { + discover_in(&context(&[])).expect("this host has the Visual Studio C++ tools") + } + + #[test] + fn system_folders_are_looked_for_on_the_drive_of_windows() { + assert_eq!( + drive_root(Path::new(r"D:\Windows")), + Some(PathBuf::from(r"D:\")) + ); + assert_eq!(drive_root(Path::new(r"\\server\share\Windows")), None); + assert_eq!(drive_root(Path::new("relative")), None); + + let on_d = context_with(&[("SystemRoot", r"D:\Windows")]); + assert_eq!(system_drive(&on_d), PathBuf::from(r"D:\")); + assert_eq!(program_data(&on_d), PathBuf::from(r"D:\ProgramData")); + assert_eq!( + program_files_x86(&on_d), + PathBuf::from(r"D:\Program Files (x86)") + ); + // What the context names is used as it is. + let named = context_with(&[ + ("SystemRoot", r"D:\Windows"), + ("ProgramData", r"E:\Data"), + ("ProgramFiles(x86)", r"E:\PF86"), + ]); + assert_eq!(program_data(&named), PathBuf::from(r"E:\Data")); + assert_eq!(program_files_x86(&named), PathBuf::from(r"E:\PF86")); + // With nothing to go by, the default drive. + let bare = context_with(&[]); + assert_eq!(program_data(&bare), PathBuf::from(r"C:\ProgramData")); + } + + #[test] + fn a_hermetic_environment_finds_the_linker_through_vswhere() { + let linker = visual_studio_linker(); + + assert_eq!( + linker.program.file_name().and_then(|n| n.to_str()), + Some("link.exe") + ); + assert!(linker.program.is_file(), "{}", linker.program.display()); + assert!( + toolset_of(&linker.program).is_some(), + "{}", + linker.program.display() + ); + let library_path = linker.library_path.expect("a hermetic process has no LIB"); + let directories: Vec = std::env::split_paths(&library_path).collect(); + assert_eq!(directories.len(), 3, "{directories:?}"); + assert!( + directories.iter().all(|directory| directory.is_dir()), + "{directories:?}" + ); + assert!(directories[0].join("msvcrt.lib").is_file()); + assert!(directories[1].join("ucrt.lib").is_file()); + assert!(directories[2].join("kernel32.lib").is_file()); + } + + #[test] + fn the_microsoft_linker_on_path_is_used_as_found() { + let found = visual_studio_linker().program; + let directory = found.parent().expect("link.exe has a directory"); + // No Visual Studio Installer here, so only PATH can give the linker. + let context = context(&[ + ("Path", &directory.to_string_lossy()), + ("ProgramFiles(x86)", r"C:\no-such-program-files"), + ( + "WindowsSdkDir", + &PathBuf::from(std::env::var("ProgramFiles(x86)").expect("ProgramFiles(x86)")) + .join("Windows Kits") + .join("10") + .to_string_lossy(), + ), + ]); + + let linker = discover_in(&context).expect("the linker on PATH"); + + assert_eq!(linker.program, found.parent().unwrap().join("link.exe")); + assert!( + linker.library_path.is_some(), + "the toolset gives the library path" + ); + } + + #[test] + fn a_link_exe_that_is_not_the_microsoft_linker_is_skipped() { + let scratch = tempfile::tempdir().expect("temporary directory"); + let directory = scratch.path(); + let imposter = directory.join("link.exe"); + let system = std::env::var_os("SystemRoot").expect("SystemRoot"); + std::fs::copy( + Path::new(&system).join("System32").join("cmd.exe"), + &imposter, + ) + .expect("a program that is not a linker"); + + let linker = discover_in(&context(&[("Path", &directory.to_string_lossy())])) + .expect("Visual Studio still gives the linker"); + assert_ne!(linker.program, imposter); + assert!( + toolset_of(&linker.program).is_some(), + "{}", + linker.program.display() + ); + } + + #[test] + fn an_explicit_linker_wins_and_a_wrong_one_is_an_error() { + let real = visual_studio_linker().program; + + let chosen = discover_in(&context(&[("BAMTS_LINKER", &real.to_string_lossy())])) + .expect("the named linker"); + assert_eq!(chosen.program, real); + + let wrong = discover_in(&context(&[("BAMTS_LINKER", r"C:\no\such\link.exe")])); + // The search does not go on to Visual Studio, which is installed here. + assert!( + matches!( + &wrong, + Err(DriverError::ToolchainMissing { program }) + if program == &OsString::from(r"C:\no\such\link.exe") + ), + "{:?}", + wrong.err() + ); + } + + fn system_program(name: &str) -> PathBuf { + let root = std::env::var_os("SystemRoot").expect("SystemRoot"); + PathBuf::from(root).join("System32").join(name) + } + + /// A `link.exe` in `directory` that prints the banner of the Microsoft linker + /// and nothing else, so it is told apart from the one of Visual Studio. + fn banner_linker(directory: &Path) -> PathBuf { + let source = directory.join("banner_linker.rs"); + std::fs::write( + &source, + format!("fn main() {{ println!(\"{BANNER} Version 0.0\"); }}"), + ) + .expect("write the source of the stand-in linker"); + let linker = directory.join("link.exe"); + let compiled = std::process::Command::new("rustc") + .arg("--edition=2021") + .arg(&source) + .arg("-o") + .arg(&linker) + .status() + .expect("rustc starts"); + assert!(compiled.success()); + linker + } + + #[test] + fn a_named_program_wins_over_the_linker_of_visual_studio() { + // Visual Studio is installed here and would give `link.exe`. + let named = system_program("cmd.exe"); + + let linker = discover_in(&context(&[("BAMTS_LINKER", &named.to_string_lossy())])) + .expect("a program that starts is taken as named"); + + assert_eq!(linker.program, named); + } + + #[test] + fn a_microsoft_linker_on_path_wins_over_visual_studio() { + let scratch = tempfile::tempdir().expect("temporary directory"); + let stand_in = banner_linker(scratch.path()); + + let linker = discover_in(&context(&[("Path", &scratch.path().to_string_lossy())])) + .expect("the linker on PATH"); + + assert_eq!(linker.program, stand_in); + } + + #[test] + fn cancelling_stops_each_stage_of_the_search() { + let scratch = tempfile::tempdir().expect("temporary directory"); + std::fs::copy(system_program("cmd.exe"), scratch.path().join("link.exe")) + .expect("a program called link.exe"); + let cancel = CancellationToken::new(); + cancel.cancel(); + let cancelled = |extra: &[(&str, &str)]| { + let result = discover_with(&context(extra), &cancel, "kernel32.lib"); + matches!(result, Err(DriverError::Cancelled)) + }; + + // The named program, the program on PATH, and vswhere each run a child. + assert!(cancelled(&[( + "BAMTS_LINKER", + &system_program("cmd.exe").to_string_lossy() + )])); + assert!(cancelled(&[("Path", &scratch.path().to_string_lossy())])); + assert!(cancelled(&[])); + } + + #[test] + fn no_linker_anywhere_is_reported_as_missing() { + let context = context(&[("ProgramFiles(x86)", r"C:\no-such-program-files")]); + + let error = discover_in(&context).err().expect("there is no linker"); + + assert!( + matches!(&error, DriverError::ToolchainMissing { program } + if program == &OsString::from("link.exe")), + "{error}" + ); + } + + #[test] + fn the_library_path_of_the_context_is_kept_and_link_options_are_dropped() { + let context = context(&[ + ("LIB", r"C:\my\libs"), + ("LINK", "/DEBUG"), + ("_LINK_", "/OPT:NOREF"), + ]); + let linker = discover_in(&context).expect("the linker"); + assert!(linker.library_path.is_none(), "the context has its own LIB"); + + let command = linker.command( + &context, + Path::new("a.o"), + Path::new("b.lib"), + Path::new("c.out"), + ); + let set = |name: &str| { + command + .get_envs() + .find(|(key, _)| { + key.to_str() + .is_some_and(|key| key.eq_ignore_ascii_case(name)) + }) + .map(|(_, value)| value.map(OsString::from)) + }; + assert_eq!(set("LIB"), Some(Some(OsString::from(r"C:\my\libs")))); + // The environment was cleared first, so a removed variable is absent. + assert_eq!(set("LINK"), None); + assert_eq!(set("_LINK_"), None); + } + + #[test] + fn the_command_runs_the_linker_with_the_link_arguments() { + let context = context(&[]); + let linker = discover_in(&context).expect("the linker"); + + let command = linker.command( + &context, + Path::new("a.o"), + Path::new("b.lib"), + Path::new("c.out"), + ); + + assert_eq!(command.get_program(), linker.program.as_os_str()); + let arguments: Vec = command.get_args().map(OsString::from).collect(); + assert_eq!( + arguments, + link_arguments( + Path::new("a.o"), + Path::new("b.lib"), + Path::new("c.out"), + "kernel32.lib ntdll.lib" + ) + ); + assert_eq!(command.get_current_dir(), Some(context.cwd())); + let library_path = command + .get_envs() + .find(|(key, _)| key.eq_ignore_ascii_case("LIB")) + .and_then(|(_, value)| value); + assert_eq!(library_path, linker.library_path.as_deref()); + } + } +} diff --git a/crates/bamts-cli/tests/cli.rs b/crates/bamts-cli/tests/cli.rs index ce4f4582..d01de34b 100644 --- a/crates/bamts-cli/tests/cli.rs +++ b/crates/bamts-cli/tests/cli.rs @@ -204,7 +204,8 @@ fn api_transport_serves_socket_backed_stdin() { drop(peer); assert_success(&output, "bamts --api"); - let canonical_root = fs::canonicalize(&directory.path).expect("session root canonicalizes"); + let canonical_root = bamts_compiler::project::canonical::canonicalize(&directory.path) + .expect("session root canonicalizes"); let responses = decode_frames(&output.stdout); assert_eq!( responses, @@ -292,7 +293,7 @@ process.stdout.write(process.env.BAMTS_AOT_ENTRYPOINT === undefined ? "hidden\n" let expected = format!( "bamts\n{}\nfirst\nsecond\nhidden\n", - fs::canonicalize(project.path.join("main.ts")) + bamts_compiler::project::canonical::canonicalize(project.path.join("main.ts")) .expect("entrypoint canonicalizes") .display() ); @@ -650,6 +651,26 @@ fn aot_fixture_matches_jit_stdout_and_exit_code() { assert_eq!(output.stdout, EXPECTED_STDOUT); } +/// `/BREPRO` leaves the link time out of the file, so the Microsoft linker gives the +/// same bytes for the same inputs. The test pins the flag, which only it can set. +#[cfg(target_env = "msvc")] +#[test] +fn aot_executables_for_the_same_source_have_the_same_bytes() { + let directory = ScratchDirectory::new(); + let mut images = Vec::new(); + for name in ["first", "second"] { + let executable = directory + .path + .join(format!("{name}{}", std::env::consts::EXE_SUFFIX)); + let compile = directory.emit(&fixture().to_string_lossy(), &executable); + assert_success(&compile, "bamts tsc-compatible emit"); + images.push(fs::read(&executable).expect("the linked executable")); + // The link time has a resolution of one second. + std::thread::sleep(Duration::from_millis(1100)); + } + assert!(images[0] == images[1], "the two executables differ"); +} + #[test] fn jit_runs_two_module_program_with_live_imported_mutation() { let project = ScratchDirectory::new(); diff --git a/crates/bamts-codegen/Cargo.toml b/crates/bamts-codegen/Cargo.toml index 91d131c4..ac3cbda1 100644 --- a/crates/bamts-codegen/Cargo.toml +++ b/crates/bamts-codegen/Cargo.toml @@ -25,6 +25,7 @@ wasmtime-internal-jit-icache-coherence = { version = "=47.0.2", optional = true libc = { version = "=0.2.189", optional = true } [dev-dependencies] +bamts-platform.workspace = true bamts-runtime.workspace = true [features] diff --git a/crates/bamts-codegen/src/aot.rs b/crates/bamts-codegen/src/aot.rs index 028d31a9..0a537d33 100644 --- a/crates/bamts-codegen/src/aot.rs +++ b/crates/bamts-codegen/src/aot.rs @@ -571,13 +571,16 @@ mod tests { .expect("test program verifies") } + /// Mirrors `host_target`, which needs the `host-jit` feature these tests + /// do not enable: the triple picks the host ABI and the container stays ELF. fn target() -> &'static str { - if cfg!(target_arch = "x86_64") { - "x86_64-unknown-linux-gnu" - } else if cfg!(target_arch = "aarch64") { - "aarch64-unknown-linux-gnu" - } else { - panic!("AOT object test needs a supported 64-bit host architecture") + match (std::env::consts::OS, std::env::consts::ARCH) { + ("macos", "aarch64") => "aarch64-apple-darwin-elf", + ("macos", "x86_64") => "x86_64-apple-darwin-elf", + ("windows", "x86_64") => "x86_64-pc-windows-msvc-elf", + (_, "x86_64") => "x86_64-unknown-linux-gnu", + (_, "aarch64") => "aarch64-unknown-linux-gnu", + _ => panic!("AOT object test needs a supported 64-bit host architecture"), } } diff --git a/crates/bamts-codegen/src/aot/emission.rs b/crates/bamts-codegen/src/aot/emission.rs index ac0d9c11..a8524252 100644 --- a/crates/bamts-codegen/src/aot/emission.rs +++ b/crates/bamts-codegen/src/aot/emission.rs @@ -13,11 +13,14 @@ use super::{AotError, AotObject, compile_aot_with_cancel}; use crate::{LowerError, ProgramLowerError}; /// The target cells for which BamTS publishes AOT evidence. -pub const SUPPORTED_TARGET_TRIPLES: [&str; 4] = [ +pub const SUPPORTED_TARGET_TRIPLES: [&str; 7] = [ "x86_64-unknown-linux-gnu", "aarch64-unknown-linux-gnu", "s390x-unknown-linux-gnu", "riscv64gc-unknown-linux-gnu", + "aarch64-apple-darwin", + "x86_64-apple-darwin", + "x86_64-pc-windows-msvc", ]; /// Canonical code-generation properties for one registered target cell. @@ -34,7 +37,7 @@ impl TargetDescriptor { pub fn lookup(target: &str) -> Result { if !SUPPORTED_TARGET_TRIPLES.contains(&target) { return Err(EmissionError::Target(AotError::TargetLookup(format!( - "`{target}` is not one of the four registered BamTS targets" + "`{target}` is not a registered BamTS target" )))); } @@ -340,6 +343,20 @@ mod tests { ); } + #[test] + fn each_registered_target_emits_its_own_object_format() { + for (triple, magic) in [ + ("x86_64-unknown-linux-gnu", &[0x7f, b'E', b'L', b'F'][..]), + ("aarch64-apple-darwin", &[0xcf, 0xfa, 0xed, 0xfe][..]), + ("x86_64-apple-darwin", &[0xcf, 0xfa, 0xed, 0xfe][..]), + ("x86_64-pc-windows-msvc", &[0x64, 0x86][..]), + ] { + let descriptor = TargetDescriptor::lookup(triple).expect("registered target resolves"); + let emitted = emit_for_target(&program(42), &descriptor).expect("object emits"); + assert!(emitted.bytes().starts_with(magic), "{triple}"); + } + } + #[test] fn unsupported_target_and_pointer_width_are_typed() { assert!(matches!( diff --git a/crates/bamts-codegen/src/aot/in_process_link.rs b/crates/bamts-codegen/src/aot/in_process_link.rs index 88f1b53c..6added26 100644 --- a/crates/bamts-codegen/src/aot/in_process_link.rs +++ b/crates/bamts-codegen/src/aot/in_process_link.rs @@ -29,8 +29,8 @@ use bamts_native::{ }; use cranelift_object::object::{ - File, Object, ObjectSection, ObjectSymbol, Relocation, RelocationEncoding, RelocationKind, - RelocationTarget, SectionIndex, SectionKind, SymbolIndex, + File, Object, ObjectSection, ObjectSymbol, Relocation, RelocationEncoding, RelocationFlags, + RelocationKind, RelocationTarget, SectionIndex, SectionKind, SymbolIndex, elf, }; use crate::AotObject; @@ -125,13 +125,20 @@ impl NativeEntryTable for LinkedAotProgram { /// Returns the Cranelift target triple for the host, or an error if the host /// architecture is not supported for in-process AOT. +/// +/// The triple selects the calling convention generated code uses to call the +/// `bamts_*` helpers, which are compiled for the real host. The container stays +/// ELF on every host: the `-elf` suffix makes Cranelift write ELF while the +/// rest of the triple still picks the host ABI. Do not pass these triples to +/// `linking::TargetFormat::for_triple`, which would read `darwin` as Mach-O. pub fn host_target() -> Result<&'static str, HostTargetError> { - if cfg!(target_arch = "x86_64") { - Ok("x86_64-unknown-linux-gnu") - } else if cfg!(target_arch = "aarch64") { - Ok("aarch64-unknown-linux-gnu") - } else { - Err(HostTargetError::UnsupportedArch(std::env::consts::ARCH)) + match (std::env::consts::OS, std::env::consts::ARCH) { + ("macos", "aarch64") => Ok("aarch64-apple-darwin-elf"), + ("macos", "x86_64") => Ok("x86_64-apple-darwin-elf"), + ("windows", "x86_64") => Ok("x86_64-pc-windows-msvc-elf"), + (_, "x86_64") => Ok("x86_64-unknown-linux-gnu"), + (_, "aarch64") => Ok("aarch64-unknown-linux-gnu"), + _ => Err(HostTargetError::UnsupportedArch(std::env::consts::ARCH)), } } @@ -154,12 +161,24 @@ pub enum InProcessLinkError { size: u8, encoding: RelocationEncoding, }, + /// A relocation value does not fit the field of the instruction it patches. + RelocationOutOfRange { + relocation: &'static str, + displacement: i64, + }, + /// A relocation needs an address with alignment the address does not have. + RelocationMisaligned { + relocation: &'static str, + address: u64, + }, /// The mapped ProgramDescriptor failed validation. DescriptorValidation(AbiError), /// A memory mapping operation failed. Map(String), /// A memory protection operation failed. Protect(String), + /// Making the patched code visible to instruction fetch failed. + InstructionCache(String), } impl fmt::Display for InProcessLinkError { @@ -181,11 +200,29 @@ impl fmt::Display for InProcessLinkError { "unsupported relocation: kind={kind:?} size={size} encoding={encoding:?}" ) } + Self::RelocationOutOfRange { + relocation, + displacement, + } => { + write!( + f, + "{relocation} displacement {displacement} is out of range" + ) + } + Self::RelocationMisaligned { + relocation, + address, + } => { + write!(f, "{relocation} address {address:#x} is misaligned") + } Self::DescriptorValidation(error) => { write!(f, "program descriptor validation failed: {error:?}") } Self::Map(msg) => write!(f, "memory mapping failed: {msg}"), Self::Protect(msg) => write!(f, "memory protection failed: {msg}"), + Self::InstructionCache(msg) => { + write!(f, "instruction cache maintenance failed: {msg}") + } } } } @@ -233,8 +270,8 @@ pub fn link_aot_in_process( } // Pre-scan relocations to find symbols that need GOT entries // (GotRelative relocations reference a GOT entry, not the symbol directly). - let mut got_symbols: Vec = Vec::new(); - let mut got_seen: std::collections::HashSet = std::collections::HashSet::new(); + let mut got_entries: Vec = Vec::new(); + let mut got_seen: std::collections::HashSet = std::collections::HashSet::new(); for section in §ions { if section.data.is_empty() { continue; @@ -243,20 +280,18 @@ pub fn link_aot_in_process( .section_by_index(section.original_index) .map_err(|e| InProcessLinkError::Parse(e.to_string()))?; for (_offset, reloc) in original_section.relocations() { - if reloc.kind() == RelocationKind::GotRelative + if needs_got_entry(&reloc) && let RelocationTarget::Symbol(idx) = reloc.target() - && got_seen.insert(idx) + && got_seen.insert((idx, got_entry_addend(&reloc))) { - got_symbols.push(idx); + got_entries.push((idx, got_entry_addend(&reloc))); } } } - let got_size = got_symbols.len() * 8; - // Total mapping size: text region (page-aligned) + data region (page-aligned) // + GOT region (8 bytes per entry, 8-aligned). let text_region_size = align_up(text_end, page_size); - let data_region_size = align_up(data_end + got_size, page_size); + let (got_offset, data_region_size) = got_layout(data_end, got_entries.len(), page_size); let total_size = text_region_size + data_region_size; if total_size == 0 { return Err(InProcessLinkError::Map("object has no sections".into())); @@ -323,10 +358,11 @@ pub fn link_aot_in_process( // -- 5b. Populate GOT entries ------------------------------------------- // The GOT lives at the end of the data region, after all data sections. // Each entry is 8 bytes containing the resolved symbol address. - let mut got_entry_addresses: HashMap = HashMap::new(); - if !got_symbols.is_empty() { - let got_base = unsafe { data_base.add(data_end) }; - for (i, sym_idx) in got_symbols.iter().enumerate() { + let mut got_entry_addresses: HashMap = HashMap::new(); + if !got_entries.is_empty() { + let got_base = unsafe { data_base.add(got_offset) }; + for (i, entry) in got_entries.iter().enumerate() { + let (sym_idx, addend) = entry; let sym = file .symbol_by_index(*sym_idx) .map_err(|e| InProcessLinkError::Parse(e.to_string()))?; @@ -346,9 +382,9 @@ pub fn link_aot_in_process( }; let got_entry = unsafe { got_base.add(i * 8) }; unsafe { - (got_entry as *mut u64).write_unaligned(addr as u64); + (got_entry as *mut u64).write_unaligned((addr as u64).wrapping_add_signed(*addend)); } - got_entry_addresses.insert(*sym_idx, got_entry as usize); + got_entry_addresses.insert(*entry, got_entry as usize); } } // -- 6. Apply relocations ----------------------------------------------- @@ -371,14 +407,15 @@ pub fn link_aot_in_process( for (reloc_offset, relocation) in original_section.relocations() { // For GotRelative, the "symbol address" is the GOT entry address. - let symbol_addr = if relocation.kind() == RelocationKind::GotRelative { + let symbol_addr = if needs_got_entry(&relocation) { let RelocationTarget::Symbol(sym_idx) = relocation.target() else { return Err(InProcessLinkError::UnknownRelocationTarget(format!( "{:?}", relocation.target() ))); }; - *got_entry_addresses.get(&sym_idx).ok_or_else(|| { + let entry = (sym_idx, got_entry_addend(&relocation)); + *got_entry_addresses.get(&entry).ok_or_else(|| { InProcessLinkError::MissingSymbol(format!("GOT entry for symbol {sym_idx:?}")) })? } else { @@ -393,7 +430,16 @@ pub fn link_aot_in_process( // -- 7. Protect text as executable -------------------------------------- if text_region_size > 0 { + // AArch64 fetches instructions through a cache that stores into the + // mapping do not update, so the patched code must be made visible. + // The order matches `jit_memory`: clear the cache before the RW -> RX + // transition, flush the pipeline after it. + // + // SAFETY: `text_base..text_base+text_region_size` is the text part of + // `mapping`, which this function owns and keeps alive. unsafe { + wasmtime_internal_jit_icache_coherence::clear_cache(text_base.cast(), text_region_size) + .map_err(|e| InProcessLinkError::InstructionCache(e.to_string()))?; region::protect( text_base as *const (), text_region_size, @@ -401,6 +447,8 @@ pub fn link_aot_in_process( ) .map_err(|e| InProcessLinkError::Protect(e.to_string()))?; } + wasmtime_internal_jit_icache_coherence::pipeline_flush_mt() + .map_err(|e| InProcessLinkError::InstructionCache(e.to_string()))?; } // -- 8. Build ProgramDescriptor from mapped data ------------------------ @@ -505,6 +553,14 @@ fn collect_sections(file: &File<'_>) -> Vec { sections } +/// Where the GOT starts in the data region, and the region's mapped size. An +/// AArch64 `ldr` can only reach an 8-aligned entry, and the last data section +/// can end anywhere. +fn got_layout(data_end: usize, entries: usize, page_size: usize) -> (usize, usize) { + let offset = align_up(data_end, 8); + (offset, align_up(offset + entries * 8, page_size)) +} + fn align_up(value: usize, alignment: usize) -> usize { if alignment == 0 { return value; @@ -561,6 +617,20 @@ fn apply_relocation( let size = relocation.size(); let encoding = relocation.encoding(); + // `object` leaves the AArch64 GOT forms as `Unknown`, so they are told + // apart by the raw ELF relocation type. + match elf_relocation_type(relocation) { + Some(elf::R_AARCH64_ADR_GOT_PAGE) => { + return patch_instruction(place, |word| { + patch_adrp_page(word, place_addr as u64, symbol_addr as u64) + }); + } + Some(elf::R_AARCH64_LD64_GOT_LO12_NC) => { + return patch_instruction(place, |word| patch_ldr_got_lo12(word, symbol_addr as u64)); + } + _ => {} + } + match (kind, size) { (RelocationKind::Absolute, 64) => { // S + A @@ -587,20 +657,13 @@ fn apply_relocation( place.cast::().write_unaligned(value); } } - (RelocationKind::Relative, 26) => { - // AArch64 call: (S + A - P) >> 2, 26-bit, masked into place[31:0] - // The lower 2 bits of the place are 00 (ARM alignment), so we - // encode the 26-bit immediate directly. - let displacement = (symbol_addr as i64) - .wrapping_add(addend) - .wrapping_sub(place_addr as i64); - let imm26 = (displacement >> 2) as u32 & 0x03FF_FFFF; - unsafe { - let existing = place.cast::().read_unaligned(); - place - .cast::() - .write_unaligned((existing & 0xFC00_0000) | imm26); - } + // AArch64 `B`/`BL`. `object` reports an ELF `R_AARCH64_CALL26` as + // `PltRelative`; COFF and Mach-O readers report `Relative`. + (RelocationKind::Relative | RelocationKind::PltRelative, 26) => { + let target = (symbol_addr as i64).wrapping_add(addend) as u64; + return patch_instruction(place, |word| { + patch_branch26(word, place_addr as u64, target) + }); } _ => { return Err(InProcessLinkError::UnsupportedRelocation { @@ -613,6 +676,100 @@ fn apply_relocation( Ok(()) } +fn elf_relocation_type(relocation: &Relocation) -> Option { + match relocation.flags() { + RelocationFlags::Elf { r_type } => Some(r_type), + _ => None, + } +} + +/// A GOT slot: the symbol and the addend its stored address includes. +type GotEntry = (SymbolIndex, i64); + +/// The addend a relocation's GOT slot stores. The AArch64 forms load +/// `GDAT(S + A)`, so slots differ by addend. x86-64 `GOTPCREL` applies its addend +/// to the displacement instead, where it reads `-4`. +fn got_entry_addend(relocation: &Relocation) -> i64 { + match elf_relocation_type(relocation) { + Some(elf::R_AARCH64_ADR_GOT_PAGE | elf::R_AARCH64_LD64_GOT_LO12_NC) => relocation.addend(), + _ => 0, + } +} + +/// Whether the relocation reads its symbol through a GOT entry instead of +/// addressing the symbol directly. +fn needs_got_entry(relocation: &Relocation) -> bool { + relocation.kind() == RelocationKind::GotRelative + || matches!( + elf_relocation_type(relocation), + Some(elf::R_AARCH64_ADR_GOT_PAGE | elf::R_AARCH64_LD64_GOT_LO12_NC) + ) +} + +/// Rewrites the 32-bit instruction at `place` through `patch`. +fn patch_instruction( + place: *mut u8, + patch: impl FnOnce(u32) -> Result, +) -> Result<(), InProcessLinkError> { + unsafe { + let patched = patch(place.cast::().read_unaligned())?; + place.cast::().write_unaligned(patched); + } + Ok(()) +} + +/// `R_AARCH64_ADR_GOT_PAGE`: points an `ADRP` at the 4 KiB page holding the +/// GOT entry. `ADRP` counts 4 KiB pages whatever the OS page size is. +fn patch_adrp_page( + instruction: u32, + place: u64, + got_entry: u64, +) -> Result { + let pages = (got_entry >> 12) as i64 - (place >> 12) as i64; + if !(-(1 << 20)..(1 << 20)).contains(&pages) { + return Err(InProcessLinkError::RelocationOutOfRange { + relocation: "R_AARCH64_ADR_GOT_PAGE", + displacement: pages, + }); + } + let immediate = pages as u32 & 0x1F_FFFF; + let immlo = immediate & 0x3; + let immhi = immediate >> 2; + Ok((instruction & !((0x3 << 29) | (0x7_FFFF << 5))) | (immlo << 29) | (immhi << 5)) +} + +/// `R_AARCH64_LD64_GOT_LO12_NC`: gives an 8-byte `LDR` the offset of the GOT +/// entry inside its page. +fn patch_ldr_got_lo12(instruction: u32, got_entry: u64) -> Result { + if got_entry & 0x7 != 0 { + return Err(InProcessLinkError::RelocationMisaligned { + relocation: "R_AARCH64_LD64_GOT_LO12_NC", + address: got_entry, + }); + } + let scaled_offset = ((got_entry & 0xFFF) >> 3) as u32; + Ok((instruction & !(0xFFF << 10)) | (scaled_offset << 10)) +} + +/// `R_AARCH64_CALL26`: gives a `B` or `BL` the word distance to `target`. The +/// field is 26 bits of signed 4-byte words, so the reach is +-128 MiB. +fn patch_branch26(instruction: u32, place: u64, target: u64) -> Result { + let displacement = (target as i64).wrapping_sub(place as i64); + if displacement & 0x3 != 0 { + return Err(InProcessLinkError::RelocationMisaligned { + relocation: "R_AARCH64_CALL26", + address: target, + }); + } + if !(-(1 << 27)..(1 << 27)).contains(&displacement) { + return Err(InProcessLinkError::RelocationOutOfRange { + relocation: "R_AARCH64_CALL26", + displacement, + }); + } + Ok((instruction & 0xFC00_0000) | ((displacement >> 2) as u32 & 0x03FF_FFFF)) +} + /// Builds a map of helper symbol names to their live function addresses. fn helper_address_map() -> HashMap<&'static str, usize> { let mut map = HashMap::new(); @@ -769,6 +926,249 @@ mod tests { .expect("test program verifies") } + /// `adrp x16, #0` and `ldr x17, [x16]`. The expected words below come from + /// assembling `adrp x16, #N` and `ldr x17, [x16, #N]` with clang. + const ADRP_X16: u32 = 0x9000_0010; + const LDR_X17_FROM_X16: u32 = 0xF940_0211; + + #[test] + fn a_got_slot_keeps_the_addend_only_where_the_entry_holds_it() { + use cranelift_object::object::write; + use cranelift_object::object::{ + Architecture, BinaryFormat, Endianness, SymbolFlags, SymbolKind, SymbolScope, + }; + + let parsed_addend = |architecture, r_type, addend| { + let mut object = + write::Object::new(BinaryFormat::Elf, architecture, Endianness::Little); + let text = object.add_section(Vec::new(), b".text".to_vec(), SectionKind::Text); + object.append_section_data(text, &[0; 8], 4); + let symbol = object.add_symbol(write::Symbol { + name: b"target".to_vec(), + value: 0, + size: 0, + kind: SymbolKind::Text, + scope: SymbolScope::Linkage, + weak: false, + section: write::SymbolSection::Undefined, + flags: SymbolFlags::None, + }); + object + .add_relocation( + text, + write::Relocation { + offset: 0, + symbol, + addend, + flags: RelocationFlags::Elf { r_type }, + }, + ) + .expect("relocation records"); + let bytes = object.write().expect("object writes"); + let file = File::parse(&*bytes).expect("object parses"); + let section = file.section_by_name(".text").expect("text section"); + let (_, relocation) = section.relocations().next().expect("one relocation"); + assert!(needs_got_entry(&relocation)); + got_entry_addend(&relocation) + }; + + for r_type in [elf::R_AARCH64_ADR_GOT_PAGE, elf::R_AARCH64_LD64_GOT_LO12_NC] { + assert_eq!(parsed_addend(Architecture::Aarch64, r_type, 16), 16); + } + assert_eq!( + parsed_addend(Architecture::X86_64, elf::R_X86_64_GOTPCREL, -4), + 0 + ); + } + + #[test] + fn the_got_starts_aligned_and_fits_its_region() { + for data_end in 0..32 { + for entries in 0..4 { + let (offset, size) = got_layout(data_end, entries, 16); + assert_eq!(offset % 8, 0); + assert!(offset >= data_end); + assert!(offset + entries * 8 <= size); + assert_eq!(size % 16, 0); + } + } + } + + #[test] + fn adrp_patch_matches_assembler_output() { + assert_eq!(patch_adrp_page(ADRP_X16, 0, 0x1000).unwrap(), 0xB000_0010); + assert_eq!(patch_adrp_page(ADRP_X16, 0x2000, 0).unwrap(), 0xD0FF_FFF0); + } + + #[test] + fn adrp_patch_counts_4k_pages_and_ignores_the_offset_in_the_page() { + assert_eq!( + patch_adrp_page(ADRP_X16, 0xFFC, 0x1008).unwrap(), + patch_adrp_page(ADRP_X16, 0, 0x1000).unwrap() + ); + } + + #[test] + fn adrp_patch_replaces_an_earlier_immediate() { + let stale = ADRP_X16 | (0x3 << 29) | (0x7_FFFF << 5); + assert_eq!(patch_adrp_page(stale, 0, 0x1000).unwrap(), 0xB000_0010); + } + + #[test] + fn adrp_patch_accepts_exactly_the_21_bit_page_range() { + let span = 1u64 << 32; + assert!(patch_adrp_page(ADRP_X16, 0, span - 0x1000).is_ok()); + assert!(patch_adrp_page(ADRP_X16, span, 0).is_ok()); + assert!(matches!( + patch_adrp_page(ADRP_X16, 0, span), + Err(InProcessLinkError::RelocationOutOfRange { .. }) + )); + assert!(matches!( + patch_adrp_page(ADRP_X16, span + 0x1000, 0), + Err(InProcessLinkError::RelocationOutOfRange { .. }) + )); + } + + #[test] + fn ldr_patch_matches_assembler_output() { + assert_eq!( + patch_ldr_got_lo12(LDR_X17_FROM_X16, 8).unwrap(), + 0xF940_0611 + ); + assert_eq!( + patch_ldr_got_lo12(LDR_X17_FROM_X16, 0x7008).unwrap(), + 0xF940_0611 + ); + } + + #[test] + fn ldr_patch_replaces_an_earlier_offset_and_covers_the_whole_page() { + let stale = LDR_X17_FROM_X16 | (0xFFF << 10); + assert_eq!(patch_ldr_got_lo12(stale, 0).unwrap(), LDR_X17_FROM_X16); + assert_eq!( + patch_ldr_got_lo12(LDR_X17_FROM_X16, 0xFF8).unwrap(), + LDR_X17_FROM_X16 | (0x1FF << 10) + ); + } + + #[test] + fn ldr_patch_rejects_an_entry_that_is_not_8_aligned() { + assert!(matches!( + patch_ldr_got_lo12(LDR_X17_FROM_X16, 4), + Err(InProcessLinkError::RelocationMisaligned { .. }) + )); + } + + /// `bl #0`. The expected words below come from assembling `bl #N` with clang. + const BL: u32 = 0x9400_0000; + + #[test] + fn branch_patch_matches_assembler_output() { + assert_eq!(patch_branch26(BL, 0, 8).unwrap(), 0x9400_0002); + assert_eq!(patch_branch26(BL, 4, 0).unwrap(), 0x97FF_FFFF); + } + + #[test] + fn branch_patch_keeps_the_opcode_of_b_and_bl() { + assert_eq!(patch_branch26(0x1400_0000, 0, 8).unwrap(), 0x1400_0002); + } + + #[test] + fn branch_patch_accepts_exactly_plus_or_minus_128_mib() { + let reach = 1u64 << 27; + assert_eq!(patch_branch26(BL, 0, reach - 4).unwrap(), 0x95FF_FFFF); + assert_eq!(patch_branch26(BL, reach, 0).unwrap(), 0x9600_0000); + assert!(matches!( + patch_branch26(BL, 0, reach), + Err(InProcessLinkError::RelocationOutOfRange { .. }) + )); + assert!(matches!( + patch_branch26(BL, reach + 4, 0), + Err(InProcessLinkError::RelocationOutOfRange { .. }) + )); + } + + #[test] + fn branch_patch_rejects_a_target_that_is_not_word_aligned() { + assert!(matches!( + patch_branch26(BL, 0, 6), + Err(InProcessLinkError::RelocationMisaligned { .. }) + )); + } + + #[test] + fn apply_relocation_patches_the_call26_that_elf_readers_call_plt_relative() { + use cranelift_object::object::write; + use cranelift_object::object::{ + Architecture, BinaryFormat, Endianness, SymbolFlags, SymbolKind, SymbolScope, + }; + + let mut object = + write::Object::new(BinaryFormat::Elf, Architecture::Aarch64, Endianness::Little); + let text = object.add_section(Vec::new(), b".text".to_vec(), SectionKind::Text); + object.append_section_data(text, &BL.to_le_bytes(), 4); + let callee = object.add_symbol(write::Symbol { + name: b"callee".to_vec(), + value: 0, + size: 0, + kind: SymbolKind::Text, + scope: SymbolScope::Linkage, + weak: false, + section: write::SymbolSection::Undefined, + flags: SymbolFlags::None, + }); + object + .add_relocation( + text, + write::Relocation { + offset: 0, + symbol: callee, + addend: 0, + flags: RelocationFlags::Elf { + r_type: elf::R_AARCH64_CALL26, + }, + }, + ) + .expect("relocation records"); + let bytes = object.write().expect("object writes"); + + let file = File::parse(&*bytes).expect("object parses"); + let section = file.section_by_name(".text").expect("text section"); + let (_, relocation) = section.relocations().next().expect("one relocation"); + assert_eq!(relocation.kind(), RelocationKind::PltRelative); + + let mut word = [BL]; + let place = word.as_mut_ptr().cast::(); + apply_relocation(place, place as usize + 8, &relocation).expect("call26 applies"); + assert_eq!(word[0], 0x9400_0002); + } + + #[test] + fn host_target_selects_the_host_abi_and_keeps_elf() { + use cranelift_codegen::isa::{self, CallConv}; + use cranelift_codegen::settings::{self, Flags}; + use cranelift_object::object::BinaryFormat; + + let target = host_target().expect("host target supported"); + let isa = isa::lookup_by_name(target) + .expect("target exists") + .finish(Flags::new(settings::builder())) + .expect("target builds"); + let expected = match (std::env::consts::OS, std::env::consts::ARCH) { + ("macos", "aarch64") => CallConv::AppleAarch64, + ("windows", "x86_64") => CallConv::WindowsFastcall, + _ => CallConv::SystemV, + }; + assert_eq!(isa.frontend_config().default_call_conv, expected); + if cfg!(target_os = "macos") { + assert!(target.contains("apple-darwin"), "target was {target}"); + } + + let object = crate::compile_aot(&test_program(), target).expect("AOT object emits"); + let file = File::parse(&*object.bytes).expect("object parses"); + assert_eq!(file.format(), BinaryFormat::Elf); + } + #[test] fn in_process_link_runs_native_entries() { let program = test_program(); diff --git a/crates/bamts-codegen/src/jit.rs b/crates/bamts-codegen/src/jit.rs index d4094969..fcb1c003 100644 --- a/crates/bamts-codegen/src/jit.rs +++ b/crates/bamts-codegen/src/jit.rs @@ -1170,27 +1170,50 @@ mod tests { assert!(host.stdout.is_empty(), "fuel failure precedes console.log"); } + /// The permissions of the region that contains `address`, as the host's + /// process map reports them. A region's `identity` begins with its hex + /// address range, `start-end`, on every host. + fn permissions_at(regions: &[bamts_platform::MemoryRegion], address: usize) -> Option<&str> { + regions.iter().find_map(|region| { + let range = region.identity.split_whitespace().next()?; + let (start, end) = range.split_once('-')?; + let start = usize::from_str_radix(start, 16).ok()?; + let end = usize::from_str_radix(end, 16).ok()?; + (start <= address && address < end).then_some(region.permissions.as_str()) + }) + } + + fn own_regions() -> Vec { + bamts_platform::process_regions(std::process::id()).expect("process regions are readable") + } + + /// The lookup above is the oracle for the next test, so it has to tell code + /// from data on a real process before its verdict on JIT code means anything. + #[test] + fn the_region_lookup_tells_code_from_data() { + let code = the_region_lookup_tells_code_from_data as fn() as usize; + let data = Box::new(0_u8); + let data = std::ptr::from_ref::(&data) as usize; + let regions = own_regions(); + + let code_permissions = permissions_at(®ions, code).expect("code is mapped"); + assert!(code_permissions.contains('x'), "{code_permissions}"); + assert!(!code_permissions.contains('w'), "{code_permissions}"); + let data_permissions = permissions_at(®ions, data).expect("data is mapped"); + assert!(data_permissions.contains('w'), "{data_permissions}"); + assert!(!data_permissions.contains('x'), "{data_permissions}"); + } + #[test] - #[cfg(target_os = "linux")] fn finalized_jit_code_is_executable_and_not_writable() { let program = compile_jit(&two_module_program()).expect("host JIT compiles"); - let maps = std::fs::read_to_string("/proc/self/maps").expect("process maps are readable"); + let regions = own_regions(); // Every compiled function address is executable and not writable, as an - // independent smoke test against `/proc/self/maps`. + // independent smoke test against the host's process map. for unit in &program.functions { let address = program.module.get_finalized_function(unit.function) as usize; - let permissions = maps - .lines() - .find_map(|line| { - let mut fields = line.split_whitespace(); - let range = fields.next()?; - let permissions = fields.next()?; - let (start, end) = range.split_once('-')?; - let start = usize::from_str_radix(start, 16).ok()?; - let end = usize::from_str_radix(end, 16).ok()?; - (start <= address && address < end).then_some(permissions) - }) + let permissions = permissions_at(®ions, address) .unwrap_or_else(|| panic!("finalized function at {address:#x} has no mapped page")); // `finalize_definitions` must publish code with an RW -> RX transition. diff --git a/crates/bamts-codegen/src/lib.rs b/crates/bamts-codegen/src/lib.rs index 330aed2e..b5263e81 100644 --- a/crates/bamts-codegen/src/lib.rs +++ b/crates/bamts-codegen/src/lib.rs @@ -853,6 +853,8 @@ const fn unary_op_selector(op: UnaryOp) -> i64 { UnaryOp::Negate => 3, UnaryOp::BitwiseNot => 4, UnaryOp::LogicalNot => 5, + UnaryOp::ToString => 6, + UnaryOp::ToPropertyKey => 7, } } @@ -919,6 +921,8 @@ const fn accessor_kind_selector(kind: AccessorKind) -> i64 { match kind { AccessorKind::Getter => 0, AccessorKind::Setter => 1, + AccessorKind::ClassGetter => 2, + AccessorKind::ClassSetter => 3, } } @@ -929,6 +933,7 @@ const fn descriptor_slot_selector(slot: DescriptorSlot) -> i64 { DescriptorSlot::Value => 0, DescriptorSlot::Getter => 1, DescriptorSlot::Setter => 2, + DescriptorSlot::Entire => 4, } } diff --git a/crates/bamts-compiler/src/checker.rs b/crates/bamts-compiler/src/checker.rs index 4bfab9c6..78200b9c 100644 --- a/crates/bamts-compiler/src/checker.rs +++ b/crates/bamts-compiler/src/checker.rs @@ -604,6 +604,7 @@ pub struct ProgramCheckOptions { /// Whether `noEmitHelpers` is enabled: emit references downleveling /// helpers without emitting their definitions. no_emit_helpers: bool, + remove_comments: bool, /// ECMAScript target — drives lib derivation and downleveling decisions. target: crate::emitter::ScriptTarget, /// Active lib set — controls which intrinsic globals are installed. @@ -623,6 +624,7 @@ impl ProgramCheckOptions { strict_property_initialization: false, always_strict: false, no_emit_helpers: false, + remove_comments: false, target: crate::emitter::ScriptTarget::EsNext, libs: LibSet::ALL, check_js: false, @@ -638,6 +640,7 @@ impl ProgramCheckOptions { strict_property_initialization: false, always_strict: false, no_emit_helpers: false, + remove_comments: false, target: crate::emitter::ScriptTarget::EsNext, libs: LibSet::ALL, check_js: false, @@ -693,6 +696,17 @@ impl ProgramCheckOptions { self.no_emit_helpers } + #[must_use] + pub const fn with_remove_comments(mut self, value: bool) -> Self { + self.remove_comments = value; + self + } + + #[must_use] + pub const fn remove_comments(&self) -> bool { + self.remove_comments + } + /// Sets the target from a string (e.g. `"es5"`, `"es2020"`) and derives /// its default lib set. `None` selects the default target; an /// unrecognized name leaves the options unchanged. diff --git a/crates/bamts-compiler/src/checker/binder.rs b/crates/bamts-compiler/src/checker/binder.rs index 532de8a5..c03d8f3a 100644 --- a/crates/bamts-compiler/src/checker/binder.rs +++ b/crates/bamts-compiler/src/checker/binder.rs @@ -241,6 +241,14 @@ impl Scope { pub fn type_binding(&self, name: &str) -> Option { self.types.get(name).copied() } + + /// The names this scope declares only as types. + pub fn type_only_names(&self) -> impl Iterator + '_ { + self.types + .keys() + .map(AsRef::as_ref) + .filter(|name: &&str| !self.values.contains_key(*name)) + } } /// Resolves a value name from `scope` outward through its ancestors. Class @@ -8013,7 +8021,7 @@ impl<'src> Binder<'src> { index.type_annotation.data().type_node.as_ref(), ); } - crate::syntax::ClassMember::Missing(_) => {} + crate::syntax::ClassMember::Missing(_) | crate::syntax::ClassMember::Semicolon => {} } } Ok(()) @@ -12148,7 +12156,9 @@ impl<'src> Binder<'src> { inventory, ); } - ClassMember::IndexSignature(_) | ClassMember::Missing(_) => {} + ClassMember::IndexSignature(_) + | ClassMember::Missing(_) + | ClassMember::Semicolon => {} } } if local_name.is_some() { @@ -13376,6 +13386,7 @@ impl<'src> Binder<'src> { self.complex_catch_scopes.insert(catch_scope); } self.bind_pattern(binding, VariableKind::Let, catch_scope, handler.id()); + self.resolve_pattern_expressions(binding, catch_scope); } // Body declarations live in a child block like the try // body above, so `function e` shadows the parameter @@ -13709,6 +13720,7 @@ impl<'src> Binder<'src> { let key_type = self.types.intersection(vec![object_key, string]); let keep_literal = variable.kind == VariableKind::Const; for declarator in &variable.declarations { + self.resolve_pattern_expressions(&declarator.data().binding, scope); if declarator.data().type_annotation.is_none() { self.assign_binding_pattern_types( &declarator.data().binding, @@ -13731,6 +13743,7 @@ impl<'src> Binder<'src> { self.bind_variable(variable, scope, NodeId::default()); for declarator_node in &variable.declarations { let declarator = declarator_node.data(); + self.resolve_pattern_expressions(&declarator.binding, scope); if matches!( variable.kind, VariableKind::Using | VariableKind::AwaitUsing @@ -13801,6 +13814,7 @@ impl<'src> Binder<'src> { if let Some(initializer) = &declarator.initializer { self.resolve_expr(initializer, scope); } + self.resolve_pattern_expressions(&declarator.binding, scope); if let Some(symbols) = self.declarator_symbols.get(&declarator_node.id()) && declarator.initializer.is_some() { @@ -13971,6 +13985,40 @@ impl<'src> Binder<'src> { } } + /// Resolves the defaults and computed keys a binding pattern carries. + fn resolve_pattern_expressions( + &mut self, + pattern: &'src crate::syntax::Pattern, + scope: ScopeId, + ) { + match pattern.data() { + BindingPattern::Object(object) => { + for property in &object.properties { + self.resolve_property_name(&property.name, scope); + if let Some(initializer) = &property.initializer { + self.resolve_expr(initializer, scope); + } + self.resolve_pattern_expressions(&property.binding, scope); + } + } + BindingPattern::Array(array) => { + for element in &array.elements { + if let crate::syntax::ArrayBindingElement::Binding(inner) = element { + self.resolve_pattern_expressions(inner, scope); + } + } + } + BindingPattern::Rest(rest) => { + self.resolve_pattern_expressions(&rest.argument, scope); + } + BindingPattern::Assignment(assignment) => { + self.resolve_expr(&assignment.right, scope); + self.resolve_pattern_expressions(&assignment.left, scope); + } + BindingPattern::Identifier(_) | BindingPattern::Missing(_) => {} + } + } + fn assign_binding_pattern_types( &mut self, pattern: &'src crate::syntax::Pattern, @@ -14416,6 +14464,7 @@ impl<'src> Binder<'src> { if let Some(initializer) = &data.initializer { self.resolve_expr(initializer, scope); } + self.resolve_pattern_expressions(&data.binding, scope); let initializer_type = data .initializer .as_ref() @@ -16227,7 +16276,8 @@ impl<'src> Binder<'src> { ClassMember::Method(_) | ClassMember::Constructor(_) | ClassMember::StaticBlock(_) - | ClassMember::Missing(_) => {} + | ClassMember::Missing(_) + | ClassMember::Semicolon => {} } } diff --git a/crates/bamts-compiler/src/early_errors.rs b/crates/bamts-compiler/src/early_errors.rs new file mode 100644 index 00000000..6457ee40 --- /dev/null +++ b/crates/bamts-compiler/src/early_errors.rs @@ -0,0 +1,2099 @@ +//! ECMAScript early errors for JavaScript the engine is about to run. +//! +//! `tsc` reports these differently, or not at all, and its diagnostics are held +//! to the TypeScript baselines, so this pass is not part of the compiler's +//! diagnostics. It runs only when a JavaScript source is compiled for execution, +//! where a program the specification calls invalid must never start. +//! +//! Every rule here is a static-semantics rule of the specification. A rule that +//! cannot be decided from the tree is left out: rejecting a valid program is +//! worse than running an invalid one, because the second fails anyway. + +use std::collections::{BTreeMap, BTreeSet}; + +use crate::{ + diagnostic::{Diagnostic, DiagnosticCode}, + source::{TextRange, Utf16Pos}, + syntax::{ + ArrayBindingElement, ArrayElement, AssignmentArrayElement, AssignmentOperator, + AssignmentTarget, AssignmentTargetNode, BinaryOperator, BindingPattern, BlockNode, + CallArgument, ClassDeclaration, ClassMember, ExportDeclaration, ExportDefaultValue, Expr, + Expression, ForBinding, ForInitializer, FunctionBody, FunctionLike, IdentifierNode, + Literal, LogicalOperator, MemberProperty, MetaProperty, ObjectMember, ParameterNode, + Pattern, PrivateIdentifierNode, PropertyModifier, PropertyName, SourceFile, Statement, + Stmt, Token, TokenKind, UnaryOperator, VariableKind, + }, +}; + +/// A program the specification rejects before it runs. +pub const EARLY_ERROR: DiagnosticCode = DiagnosticCode::new("BAMTS-P019"); + +/// Words reserved in strict code that the scanner reads as identifiers. +const STRICT_RESERVED: [&str; 9] = [ + "implements", + "interface", + "let", + "package", + "private", + "protected", + "public", + "static", + "yield", +]; + +/// Words no identifier can spell, even with escapes. +const RESERVED: [&str; 37] = [ + "break", + "case", + "catch", + "class", + "const", + "continue", + "debugger", + "default", + "delete", + "do", + "else", + "enum", + "export", + "extends", + "false", + "finally", + "for", + "function", + "if", + "import", + "in", + "instanceof", + "new", + "null", + "return", + "super", + "switch", + "this", + "throw", + "true", + "try", + "typeof", + "var", + "void", + "while", + "with", + "yield", +]; + +/// Reports every early error in `file`. +#[must_use] +pub fn check(file: &SourceFile) -> Vec { + let module = is_module(file); + walk( + file, + Context { + strict: module || has_use_strict(file, file.statements()), + module, + // The host did not name a goal, and top-level `await` implies one. + top_level_await: true, + ..Context::default() + }, + ) +} + +/// Reports every early error in `file`, which the host runs as a Script, so +/// module syntax is one more of them. +#[must_use] +pub fn check_script(file: &SourceFile) -> Vec { + let mut errors = walk( + file, + Context { + strict: has_use_strict(file, file.statements()), + script: true, + ..Context::default() + }, + ); + errors.extend( + file.statements() + .iter() + .filter(|statement| { + matches!( + statement.data(), + Statement::Import(_) | Statement::Export(_) + ) + }) + .map(|statement| { + Diagnostic::error( + EARLY_ERROR, + file.source_id(), + statement.range(), + "module syntax is not allowed in a script", + ) + }), + ); + errors +} + +fn walk(file: &SourceFile, context: Context) -> Vec { + let mut walker = Walker { + file, + errors: Vec::new(), + labels: Vec::new(), + private_scopes: Vec::new(), + }; + walker.scope(file.statements(), ScopeKind::Program, context); + walker.errors +} + +/// What the surrounding code allows at the node being walked. +#[derive(Clone, Copy, Debug, Default)] +struct Context { + strict: bool, + module: bool, + /// The host named the goal, so `import.meta` is decidable. + script: bool, + /// Inside a function body, where `return` is available. + in_function: bool, + /// At the top of a module, where `await` needs no async function. + top_level_await: bool, + /// Inside an async arrow's parameters, through any arrow nested there. + async_parameters: bool, + generator: bool, + is_async: bool, + in_parameters: bool, + /// Inside a class field initializer or static block, where `arguments` is + /// not available. + class_initializer: bool, + static_block: bool, + super_call: bool, + super_property: bool, + new_target: bool, + in_iteration: bool, + in_switch: bool, + /// Arrows and methods never allow a repeated parameter, even in sloppy code. + unique_parameters: bool, +} + +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum ScopeKind { + Program, + Function, + Block, +} + +/// How a private name is declared, which decides what may share it. +#[derive(Clone, Copy, Debug, PartialEq, Eq)] +enum PrivateKind { + Getter { is_static: bool }, + Setter { is_static: bool }, + Other, +} + +/// One class body's private names, and references it could not yet resolve. +#[derive(Default)] +struct PrivateScope { + declared: BTreeSet, + unresolved: Vec<(String, TextRange)>, +} + +struct Lexical { + name: String, + range: TextRange, + plain_function: bool, +} + +struct Walker<'a> { + file: &'a SourceFile, + errors: Vec, + labels: Vec<(String, bool)>, + private_scopes: Vec, +} + +/// Whether `file` has the module goal, which only its own syntax can tell. +pub(crate) fn is_module(file: &SourceFile) -> bool { + file.statements().iter().any(|statement| { + matches!( + statement.data(), + Statement::Import(_) | Statement::Export(_) + ) + }) +} + +pub(crate) fn has_use_strict(file: &SourceFile, statements: &[Stmt]) -> bool { + directives(statements).any(|token| { + file.token_text(token) + .is_some_and(|text| text == "'use strict'" || text == "\"use strict\"") + }) +} + +/// The string tokens of a directive prologue, in order. +fn directives(statements: &[Stmt]) -> impl Iterator { + statements.iter().map_while(|statement| { + let Statement::Expression(expression) = statement.data() else { + return None; + }; + let Expression::Literal(Literal::String(literal)) = expression.expression.data() else { + return None; + }; + Some(literal.data().token()) + }) +} + +/// The body statements of `pattern`'s statement, when it is a labeled function. +fn unlabeled(statement: &Stmt) -> &Stmt { + match statement.data() { + Statement::Labeled(labeled) => unlabeled(&labeled.body), + _ => statement, + } +} + +const fn is_iteration(statement: &Statement) -> bool { + matches!( + statement, + Statement::For(_) + | Statement::ForIn(_) + | Statement::ForOf(_) + | Statement::While(_) + | Statement::DoWhile(_) + ) +} + +/// The significant token at or after `position`. +fn next_token(file: &SourceFile, position: Utf16Pos) -> Option<&Token> { + let tokens = file.tokens(); + let start = tokens.partition_point(|token| token.range().start() < position); + tokens[start..].iter().find(|token| !is_trivia(token)) +} + +/// The last significant token before `position`. +fn previous_token(file: &SourceFile, position: Utf16Pos) -> Option<&Token> { + let tokens = file.tokens(); + let end = tokens.partition_point(|token| token.range().start() < position); + tokens[..end].iter().rev().find(|token| !is_trivia(token)) +} + +/// Whether parentheses wrap `range`, which the tree drops from a target. +pub(crate) fn is_parenthesized(file: &SourceFile, range: TextRange) -> bool { + previous_token(file, range.start()).is_some_and(|token| token.kind() == TokenKind::LParen) + && next_token(file, range.end()).is_some_and(|token| token.kind() == TokenKind::RParen) +} + +fn is_trivia(token: &Token) -> bool { + matches!( + token.kind(), + TokenKind::Whitespace | TokenKind::LineComment | TokenKind::BlockComment + ) +} + +/// Whether `text` holds an octal or `\\8`-style escape, which strict code and +/// every untagged template refuse. `\\0` alone is the null character. +fn has_legacy_escape(text: &str) -> bool { + let mut characters = text.chars().peekable(); + while let Some(character) = characters.next() { + if character != '\\' { + continue; + } + match characters.next() { + Some('1'..='9') => return true, + Some('0') if characters.peek().is_some_and(char::is_ascii_digit) => return true, + _ => {} + } + } + false +} + +/// Whether a member chain passes through `?.`, which a tag cannot do. +fn in_optional_chain(expression: &Expr) -> bool { + match expression.data() { + Expression::Member(member) => member.optional || in_optional_chain(&member.object), + Expression::Call(call) => call.optional || in_optional_chain(&call.callee), + _ => false, + } +} + +const fn is_mixed_logical(outer: LogicalOperator, inner: LogicalOperator) -> bool { + matches!(outer, LogicalOperator::Nullish) != matches!(inner, LogicalOperator::Nullish) +} + +/// Whether `expression` reads through an `import()` call, which `new` cannot +/// apply to at any depth of a member chain. +fn rooted_in_import(expression: &Expr) -> bool { + match expression.data() { + Expression::Import(_) => true, + Expression::Member(member) => rooted_in_import(&member.object), + _ => false, + } +} + +fn strip_parentheses(expression: &Expr) -> &Expr { + match expression.data() { + Expression::Parenthesized(inner) => strip_parentheses(inner), + _ => expression, + } +} + +impl Walker<'_> { + fn error(&mut self, range: TextRange, message: &'static str) { + self.errors.push(Diagnostic::error( + EARLY_ERROR, + self.file.source_id(), + range, + message, + )); + } + + /// Whether a comma follows `range`, which a rest element cannot have. + fn comma_follows(&self, range: TextRange) -> bool { + next_token(self.file, range.end()).is_some_and(|token| token.kind() == TokenKind::Comma) + } + + fn token_of(&self, token: &Token) -> &str { + self.file.token_text(token).unwrap_or_default() + } + + /// The word after `import.` or `new.`, exactly as written. + fn meta_word(&self, expression: &Expr) -> &str { + previous_token(self.file, expression.range().end()) + .and_then(|token| self.file.token_text(token)) + .unwrap_or_default() + } + + /// Whether `callee` is `import.defer` or `import.source`, which exist only + /// as the callee of a call. The parser reads every `import.x` as a meta + /// property, as `tsc` does, so the word is all that tells them apart. + fn is_import_phase(&self, callee: &Expr) -> bool { + matches!(callee.data(), Expression::Meta(MetaProperty::ImportMeta)) + && matches!(self.meta_word(callee), "defer" | "source") + } + + fn import_phase_call( + &mut self, + arguments: &[CallArgument], + range: TextRange, + context: Context, + ) { + if !(1..=2).contains(&arguments.len()) + || arguments + .iter() + .any(|argument| matches!(argument, CallArgument::Spread(_))) + { + self.error(range, "an import call takes a specifier and options"); + } + self.arguments(arguments, context); + } + + fn name(&self, identifier: &IdentifierNode) -> String { + self.file + .identifier_text(identifier.data().token()) + .map_or_else(String::new, std::borrow::Cow::into_owned) + } + + /// A private name as it compares: `#\u{6F}` and `#o` are the same name. + fn private_name(&self, private: &PrivateIdentifierNode) -> String { + Self::cooked_private( + self.file + .token_text(private.data().token()) + .unwrap_or_default(), + ) + } + + fn cooked_private(text: &str) -> String { + let tail = text.strip_prefix('#').unwrap_or(text); + let cooked = crate::syntax::cook_identifier_text(tail); + format!("#{}", cooked.as_deref().unwrap_or(tail)) + } + + fn property_text(&self, name: &PropertyName) -> Option { + match name { + PropertyName::Identifier(identifier) => Some(self.name(identifier)), + PropertyName::String(literal) => self + .file + .token_text(literal.data().token()) + .map(|text| text.trim_matches(['\'', '"']).to_owned()), + _ => None, + } + } + + // --- Names ----------------------------------------------------------- + + fn check_identifier(&mut self, name: &str, range: TextRange, context: Context) { + if name != "yield" && RESERVED.contains(&name) { + self.error(range, "a reserved word cannot name anything"); + } + if context.strict && STRICT_RESERVED.contains(&name) { + self.error(range, "this name is reserved in strict mode code"); + } + if name == "yield" && context.generator { + self.error(range, "`yield` cannot name anything in a generator"); + } + if name == "await" + && (context.is_async + || context.module + || context.static_block + || context.async_parameters) + { + self.error(range, "`await` cannot name anything here"); + } + } + + fn check_binding(&mut self, name: &str, range: TextRange, context: Context) { + self.check_identifier(name, range, context); + if context.strict && (name == "eval" || name == "arguments") { + self.error( + range, + "`eval` and `arguments` cannot be bound in strict mode", + ); + } + } + + fn reference(&mut self, identifier: &IdentifierNode, context: Context) { + let token = self + .file + .token_text(identifier.data().token()) + .unwrap_or_default(); + if token.starts_with('#') { + self.error(identifier.range(), "a private name needs `in` after it"); + return; + } + let name = self.name(identifier); + self.check_identifier(&name, identifier.range(), context); + if name == "arguments" && context.class_initializer { + self.error( + identifier.range(), + "`arguments` is not available in a class initializer", + ); + } + } + + /// The name of a private identifier standing alone as an operand. + fn private_operand(&self, expression: &Expr) -> Option { + let Expression::Identifier(identifier) = expression.data() else { + return None; + }; + let token = self.file.token_text(identifier.data().token())?; + token.starts_with('#').then(|| Self::cooked_private(token)) + } + + fn private_reference(&mut self, name: String, range: TextRange) { + match self.private_scopes.last_mut() { + Some(scope) => scope.unresolved.push((name, range)), + None => self.error(range, "this private name is not declared in a class"), + } + } + + fn bound_names(&self, pattern: &Pattern, names: &mut Vec<(String, TextRange)>) { + match pattern.data() { + BindingPattern::Identifier(identifier) => { + names.push((self.name(identifier), identifier.range())); + } + BindingPattern::Object(object) => { + for property in &object.properties { + self.bound_names(&property.binding, names); + } + } + BindingPattern::Array(array) => { + for element in &array.elements { + if let ArrayBindingElement::Binding(binding) = element { + self.bound_names(binding, names); + } + } + } + BindingPattern::Rest(rest) => self.bound_names(&rest.argument, names), + BindingPattern::Assignment(assignment) => self.bound_names(&assignment.left, names), + BindingPattern::Missing(_) => {} + } + } + + /// Checks one binding pattern and walks the expressions inside it. + fn pattern(&mut self, pattern: &Pattern, context: Context) { + match pattern.data() { + BindingPattern::Identifier(identifier) => { + let name = self.name(identifier); + self.check_binding(&name, identifier.range(), context); + } + BindingPattern::Object(object) => { + let last = object.properties.len().saturating_sub(1); + for (position, property) in object.properties.iter().enumerate() { + if let PropertyName::Computed(key) = &property.name { + self.expression(key, context); + } + if let BindingPattern::Rest(rest) = property.binding.data() { + if position != last { + self.error(property.binding.range(), "a rest property must come last"); + } + if !matches!(rest.argument.data(), BindingPattern::Identifier(_)) { + self.error( + property.binding.range(), + "a rest property must bind an identifier", + ); + } + } + self.pattern(&property.binding, context); + if let Some(initializer) = &property.initializer { + self.expression(initializer, context); + } + } + } + BindingPattern::Array(array) => { + let last = array.elements.len().saturating_sub(1); + for (position, element) in array.elements.iter().enumerate() { + if let ArrayBindingElement::Binding(binding) = element { + if matches!(binding.data(), BindingPattern::Rest(_)) + && (position != last || self.comma_follows(binding.range())) + { + self.error(binding.range(), "a rest element must come last"); + } + self.pattern(binding, context); + } + } + } + BindingPattern::Rest(rest) => { + if matches!(rest.argument.data(), BindingPattern::Assignment(_)) { + self.error(pattern.range(), "a rest element cannot have an initializer"); + } + self.pattern(&rest.argument, context); + } + BindingPattern::Assignment(assignment) => { + self.pattern(&assignment.left, context); + self.expression(&assignment.right, context); + } + BindingPattern::Missing(_) => {} + } + } + + // --- Scopes ---------------------------------------------------------- + + fn var_names(&self, statements: &[Stmt], names: &mut Vec<(String, TextRange)>) { + for statement in statements { + self.statement_var_names(statement, names); + } + } + + fn declarator_names( + &self, + declaration: &crate::syntax::VariableDeclaration, + names: &mut Vec<(String, TextRange)>, + ) { + if matches!(declaration.kind, VariableKind::Var) { + for declarator in &declaration.declarations { + self.bound_names(&declarator.data().binding, names); + } + } + } + + fn statement_var_names(&self, statement: &Stmt, names: &mut Vec<(String, TextRange)>) { + match statement.data() { + Statement::Variable(declaration) => self.declarator_names(declaration, names), + Statement::Block(block) => self.var_names(&block.data().statements, names), + Statement::If(conditional) => { + self.statement_var_names(&conditional.consequent, names); + if let Some(alternate) = &conditional.alternate { + self.statement_var_names(alternate, names); + } + } + Statement::For(for_statement) => { + if let Some(ForInitializer::Variable(declaration)) = &for_statement.initializer { + self.declarator_names(declaration, names); + } + self.statement_var_names(&for_statement.body, names); + } + Statement::ForIn(for_in) => { + if let ForBinding::Variable(declaration) = &for_in.binding { + self.declarator_names(declaration, names); + } + self.statement_var_names(&for_in.body, names); + } + Statement::ForOf(for_of) => { + if let ForBinding::Variable(declaration) = &for_of.binding { + self.declarator_names(declaration, names); + } + self.statement_var_names(&for_of.body, names); + } + Statement::While(while_statement) => { + self.statement_var_names(&while_statement.body, names); + } + Statement::DoWhile(do_while) => self.statement_var_names(&do_while.body, names), + Statement::With(with) => self.statement_var_names(&with.body, names), + Statement::Labeled(labeled) => self.statement_var_names(&labeled.body, names), + Statement::Try(try_statement) => { + self.var_names(&try_statement.block.data().statements, names); + if let Some(handler) = &try_statement.handler { + self.var_names(&handler.data().body.data().statements, names); + } + if let Some(finalizer) = &try_statement.finalizer { + self.var_names(&finalizer.data().statements, names); + } + } + Statement::Switch(switch) => { + for case in &switch.cases { + self.var_names(&case.data().consequent, names); + } + } + _ => {} + } + } + + fn lexical_names(&self, statements: &[Stmt], functions_are_lexical: bool) -> Vec { + let mut lexical = Vec::new(); + for statement in statements { + let statement = unlabeled(statement); + match statement.data() { + Statement::Variable(declaration) + if !matches!(declaration.kind, VariableKind::Var) => + { + let mut names = Vec::new(); + for declarator in &declaration.declarations { + self.bound_names(&declarator.data().binding, &mut names); + } + lexical.extend(names.into_iter().map(|(name, range)| Lexical { + name, + range, + plain_function: false, + })); + } + Statement::Class(class) => { + if let Some(name) = &class.name { + lexical.push(Lexical { + name: self.name(name), + range: name.range(), + plain_function: false, + }); + } + } + Statement::Function(function) if functions_are_lexical => { + let function = &function.function; + if let Some(name) = &function.name { + lexical.push(Lexical { + name: self.name(name), + range: name.range(), + plain_function: !function.is_async && !function.is_generator, + }); + } + } + _ => {} + } + } + lexical + } + + /// Reports redeclarations in one scope. `extra_vars` are var-like names from + /// outside the statement list, such as a function's parameters. + fn check_scope( + &mut self, + statements: &[Stmt], + kind: ScopeKind, + extra_vars: &[(String, TextRange)], + context: Context, + ) { + let functions_are_lexical = match kind { + ScopeKind::Block => true, + ScopeKind::Program => context.module, + ScopeKind::Function => false, + }; + let lexical = self.lexical_names(statements, functions_are_lexical); + let mut seen: BTreeMap<&str, bool> = BTreeMap::new(); + let mut reports = Vec::new(); + for entry in &lexical { + match seen.get(entry.name.as_str()) { + Some(previous_plain) + if !(kind == ScopeKind::Block + && !context.strict + && *previous_plain + && entry.plain_function) => + { + reports.push(entry.range); + } + _ => { + seen.insert(&entry.name, entry.plain_function); + } + } + } + let mut vars = Vec::new(); + self.var_names(statements, &mut vars); + if !functions_are_lexical { + for statement in statements { + if let Statement::Function(function) = unlabeled(statement).data() + && let Some(name) = &function.function.name + { + vars.push((self.name(name), name.range())); + } + } + } + for (name, range) in vars.iter().chain(extra_vars) { + if seen.contains_key(name.as_str()) { + reports.push(*range); + } + } + for range in reports { + self.error(range, "this name is already declared in this scope"); + } + } + + fn scope(&mut self, statements: &[Stmt], kind: ScopeKind, context: Context) { + self.check_scope(statements, kind, &[], context); + for statement in statements { + self.statement(statement, context); + } + } + + fn block(&mut self, block: &BlockNode, context: Context) { + self.scope(&block.data().statements, ScopeKind::Block, context); + } + + /// Checks a declaration that a loop or `if` holds without a block. + fn substatement(&mut self, statement: &Stmt, allow_function: bool, context: Context) { + match unlabeled(statement).data() { + Statement::Class(_) => { + self.error(statement.range(), "a class declaration needs a block here"); + } + Statement::Variable(declaration) if self.is_lexical_declaration(declaration) => { + self.error( + statement.range(), + "a lexical declaration needs a block here", + ); + } + Statement::Function(function) + if !allow_function + || context.strict + || function.function.is_async + || function.function.is_generator => + { + self.error( + statement.range(), + "a function declaration needs a block here", + ); + } + _ => {} + } + self.statement(statement, context); + } + + /// Checks a branch of an `if`, where Annex B allows a function declaration + /// only if no label stands in front of it. + fn branch(&mut self, statement: &Stmt, context: Context) { + if matches!(statement.data(), Statement::Labeled(_)) + && matches!(unlabeled(statement).data(), Statement::Function(_)) + { + self.error(statement.range(), "a labeled function needs a block here"); + } + self.substatement(statement, true, context); + } + + /// Whether a declaration in statement position is really one. `let` before + /// a line break there is an identifier followed by a new statement, unless + /// `[` comes next, which no statement may start after `let`. + fn is_lexical_declaration(&self, declaration: &crate::syntax::VariableDeclaration) -> bool { + if matches!(declaration.kind, VariableKind::Var) { + return false; + } + if !matches!(declaration.kind, VariableKind::Let) { + return true; + } + let Some(first) = declaration.declarations.first() else { + return true; + }; + let source = self.file.source_text(); + let line_break = match ( + source.utf16_to_byte(declaration.range.start()), + source.utf16_to_byte(first.data().binding.range().start()), + ) { + (Ok(start), Ok(end)) => source.as_str()[start..end].contains(['\n', '\r']), + _ => false, + }; + !line_break || matches!(first.data().binding.data(), BindingPattern::Array(_)) + } + + fn variable(&mut self, declaration: &crate::syntax::VariableDeclaration, context: Context) { + let lexical = !matches!(declaration.kind, VariableKind::Var); + let mut names = Vec::new(); + for declarator in &declaration.declarations { + let declarator = declarator.data(); + self.pattern(&declarator.binding, context); + if lexical { + self.bound_names(&declarator.binding, &mut names); + } + if let Some(initializer) = &declarator.initializer { + self.expression(initializer, context); + } + } + let mut seen = BTreeSet::new(); + for (name, range) in names { + if name == "let" { + self.error(range, "`let` cannot name a lexical binding"); + } + if !seen.insert(name) { + self.error(range, "this name is already declared in this scope"); + } + } + } + + /// Reports a declarator a `const` or `using` statement cannot have. A + /// `for-in` or `for-of` head gets its value from the loop, so it is exempt. + fn check_declarators(&mut self, declaration: &crate::syntax::VariableDeclaration) { + let needs_value = !matches!(declaration.kind, VariableKind::Var | VariableKind::Let); + let using = matches!( + declaration.kind, + VariableKind::Using | VariableKind::AwaitUsing + ); + for declarator in &declaration.declarations { + let declarator = declarator.data(); + if needs_value && declarator.initializer.is_none() { + self.error(declarator.binding.range(), "this declaration needs a value"); + } + if using && !matches!(declarator.binding.data(), BindingPattern::Identifier(_)) { + self.error(declarator.binding.range(), "`using` binds a single name"); + } + } + } + + /// Reports a name a `for` head binds lexically that its body redeclares. + fn check_for_head(&mut self, declaration: &crate::syntax::VariableDeclaration, body: &Stmt) { + if matches!(declaration.kind, VariableKind::Var) { + return; + } + let mut bound = Vec::new(); + for declarator in &declaration.declarations { + self.bound_names(&declarator.data().binding, &mut bound); + } + let mut vars = Vec::new(); + self.statement_var_names(body, &mut vars); + for (name, range) in vars { + if bound.iter().any(|(head, _)| *head == name) { + self.error(range, "this name is already declared in this scope"); + } + } + } + + // --- Statements ------------------------------------------------------ + + fn statement(&mut self, statement: &Stmt, context: Context) { + let looping = Context { + in_iteration: true, + ..context + }; + match statement.data() { + Statement::Variable(declaration) => { + self.check_declarators(declaration); + self.variable(declaration, context); + } + Statement::Function(function) => { + self.function(&function.function, false, false, context); + } + Statement::Class(class) => self.class(class, context), + Statement::Block(block) => self.block(block, context), + Statement::Expression(expression) => self.expression(&expression.expression, context), + Statement::If(conditional) => { + self.expression(&conditional.test, context); + self.branch(&conditional.consequent, context); + if let Some(alternate) = &conditional.alternate { + self.branch(alternate, context); + } + } + Statement::Switch(switch) => { + self.expression(&switch.discriminant, context); + let statements: Vec = switch + .cases + .iter() + .flat_map(|case| case.data().consequent.iter().cloned()) + .collect(); + self.check_scope(&statements, ScopeKind::Block, &[], context); + let inner = Context { + in_switch: true, + ..context + }; + for case in &switch.cases { + if let Some(test) = &case.data().test { + self.expression(test, context); + } + for statement in &case.data().consequent { + if let Statement::Variable(declaration) = statement.data() + && matches!( + declaration.kind, + VariableKind::Using | VariableKind::AwaitUsing + ) + { + self.error(statement.range(), "`using` needs a block here"); + } + self.statement(statement, inner); + } + } + } + Statement::For(for_statement) => { + match &for_statement.initializer { + Some(ForInitializer::Variable(declaration)) => { + self.check_declarators(declaration); + self.check_for_head(declaration, &for_statement.body); + self.variable(declaration, context); + } + Some(ForInitializer::Expression(expression)) => { + self.expression(expression, context); + } + None => {} + } + if let Some(test) = &for_statement.test { + self.expression(test, context); + } + if let Some(update) = &for_statement.update { + self.expression(update, context); + } + self.substatement(&for_statement.body, false, looping); + } + Statement::ForIn(for_in) => { + if let ForBinding::Variable(declaration) = &for_in.binding + && matches!( + declaration.kind, + VariableKind::Using | VariableKind::AwaitUsing + ) + { + self.error(statement.range(), "`using` cannot head a `for-in` loop"); + } + self.for_binding(&for_in.binding, &for_in.body, context); + self.expression(&for_in.object, context); + self.substatement(&for_in.body, false, looping); + } + Statement::ForOf(for_of) => { + self.for_binding(&for_of.binding, &for_of.body, context); + self.expression(&for_of.iterable, context); + self.substatement(&for_of.body, false, looping); + } + Statement::While(while_statement) => { + self.expression(&while_statement.test, context); + self.substatement(&while_statement.body, false, looping); + } + Statement::DoWhile(do_while) => { + self.substatement(&do_while.body, false, looping); + self.expression(&do_while.test, context); + } + Statement::Try(try_statement) => { + self.block(&try_statement.block, context); + if let Some(handler) = &try_statement.handler { + let handler = handler.data(); + if let Some(binding) = &handler.binding { + self.pattern(binding, context); + let mut names = Vec::new(); + self.bound_names(binding, &mut names); + let mut seen = BTreeSet::new(); + for (name, range) in &names { + if !seen.insert(name.clone()) { + self.error(*range, "this name is already declared in this scope"); + } + } + let body = &handler.body.data().statements; + for entry in self.lexical_names(body, true) { + if seen.contains(&entry.name) { + self.error( + entry.range, + "this name is already declared in this scope", + ); + } + } + } + self.block(&handler.body, context); + } + if let Some(finalizer) = &try_statement.finalizer { + self.block(finalizer, context); + } + } + Statement::With(with) => { + if context.strict { + self.error(statement.range(), "`with` is not allowed in strict mode"); + } + self.expression(&with.object, context); + self.substatement(&with.body, false, context); + } + Statement::Labeled(labeled) => { + let name = self.name(&labeled.label); + self.check_identifier(&name, labeled.label.range(), context); + if self.labels.iter().any(|(existing, _)| *existing == name) { + self.error(labeled.label.range(), "this label is already in use"); + } + let target = is_iteration(unlabeled(&labeled.body).data()); + self.labels.push((name, target)); + self.substatement(&labeled.body, true, context); + self.labels.pop(); + } + Statement::Break(jump) => match &jump.label { + Some(label) => { + let name = self.name(label); + if !self.labels.iter().any(|(existing, _)| *existing == name) { + self.error(label.range(), "`break` names an undefined label"); + } + } + None if !context.in_iteration && !context.in_switch => { + self.error(statement.range(), "`break` needs a loop or switch"); + } + None => {} + }, + Statement::Continue(jump) => match &jump.label { + Some(label) => { + let name = self.name(label); + if !self + .labels + .iter() + .any(|(existing, target)| *existing == name && *target) + { + self.error(label.range(), "`continue` needs a label on a loop"); + } + } + None if !context.in_iteration => { + self.error(statement.range(), "`continue` needs a loop"); + } + None => {} + }, + Statement::Return(return_statement) => { + if !context.in_function { + self.error(statement.range(), "`return` needs a function"); + } + if let Some(argument) = &return_statement.argument { + self.expression(argument, context); + } + } + Statement::Throw(throw) => self.expression(&throw.argument, context), + Statement::Export(export) => self.export(export, context), + _ => {} + } + } + + fn for_binding(&mut self, binding: &ForBinding, body: &Stmt, context: Context) { + match binding { + ForBinding::Variable(declaration) => { + self.check_for_head(declaration, body); + self.variable(declaration, context); + } + ForBinding::Target(target) => self.target(target, true, context), + } + } + + fn export(&mut self, export: &ExportDeclaration, context: Context) { + if let ExportDeclaration::Default(default) = export { + match &default.value { + ExportDefaultValue::Function(function) => { + self.function(function, false, false, context); + } + ExportDefaultValue::Class(class) => self.class(class, context), + ExportDefaultValue::Expression(expression) => { + self.expression(expression, context); + } + _ => {} + } + } + } + + // --- Functions and classes ------------------------------------------- + + /// Walks a non-arrow function. `method` allows `super.x` in it. An `expression` + /// binds its own name, so its own kind decides what that name can be; a + /// declaration's name belongs to the code around it. + fn function( + &mut self, + function: &FunctionLike, + method: bool, + expression: bool, + context: Context, + ) { + let inner = Context { + generator: function.is_generator, + is_async: function.is_async, + super_property: method, + unique_parameters: method, + ..Self::function_context(context) + }; + self.callable(&function.parameters, function.body.as_ref(), inner); + if let Some(name) = &function.name { + let text = self.name(name); + let strict = inner.strict + || function + .body + .as_ref() + .is_some_and(|body| self.body_is_strict(body)); + let around = if expression { + Context { + generator: function.is_generator, + is_async: function.is_async, + ..context + } + } else { + context + }; + self.check_binding(&text, name.range(), Context { strict, ..around }); + } + } + + /// What a new function scope resets. Strictness and the module goal carry in. + fn function_context(context: Context) -> Context { + Context { + strict: context.strict, + module: context.module, + script: context.script, + new_target: true, + in_function: true, + ..Context::default() + } + } + + fn body_is_strict(&self, body: &FunctionBody) -> bool { + match body { + FunctionBody::Block(block) => has_use_strict(self.file, &block.data().statements), + _ => false, + } + } + + fn callable( + &mut self, + parameters: &[ParameterNode], + body: Option<&FunctionBody>, + context: Context, + ) { + let saved_labels = std::mem::take(&mut self.labels); + let simple = parameters.iter().all(|parameter| { + parameter.data().initializer.is_none() + && matches!( + parameter.data().binding.data(), + BindingPattern::Identifier(_) + ) + }); + let strict = context.strict || body.is_some_and(|body| self.body_is_strict(body)); + if !simple + && body.is_some_and(|body| self.body_is_strict(body)) + && let Some(FunctionBody::Block(block)) = body + { + self.error(block.range(), "`use strict` needs simple parameters"); + } + let inner = Context { strict, ..context }; + let parameter_context = Context { + in_parameters: true, + ..inner + }; + let mut names = Vec::new(); + let last = parameters.len().saturating_sub(1); + for (position, parameter) in parameters.iter().enumerate() { + let data = parameter.data(); + if let BindingPattern::Rest(rest) = data.binding.data() { + if position != last || self.comma_follows(parameter.range()) { + self.error(parameter.range(), "a rest parameter must come last"); + } + if data.initializer.is_some() + || matches!(rest.argument.data(), BindingPattern::Assignment(_)) + { + self.error( + parameter.range(), + "a rest parameter cannot have an initializer", + ); + } + } + self.pattern(&data.binding, parameter_context); + if let Some(initializer) = &data.initializer { + self.expression(initializer, parameter_context); + } + self.bound_names(&data.binding, &mut names); + } + let mut seen = BTreeSet::new(); + let duplicates_allowed = simple && !strict && !context.unique_parameters; + for (name, range) in &names { + if !seen.insert(name.clone()) && !duplicates_allowed { + self.error(*range, "this parameter name is already used"); + } + } + match body { + Some(FunctionBody::Block(block)) => { + self.check_scope(&block.data().statements, ScopeKind::Function, &[], inner); + let lexical = self.lexical_names(&block.data().statements, false); + for entry in lexical { + if seen.contains(&entry.name) { + self.error(entry.range, "this name is already declared in this scope"); + } + } + for statement in &block.data().statements { + self.statement(statement, inner); + } + } + Some(FunctionBody::Expression(expression)) => self.expression(expression, inner), + _ => {} + } + self.labels = saved_labels; + } + + fn accessor_parameters( + &mut self, + modifier: PropertyModifier, + parameters: &[ParameterNode], + range: TextRange, + ) { + match modifier { + PropertyModifier::Get if !parameters.is_empty() => { + self.error(range, "a getter takes no parameters"); + } + PropertyModifier::Set + if parameters.len() != 1 + || matches!(parameters[0].data().binding.data(), BindingPattern::Rest(_)) => + { + self.error(range, "a setter takes exactly one parameter"); + } + _ => {} + } + } + + fn class(&mut self, class: &ClassDeclaration, context: Context) { + let context = Context { + strict: true, + ..context + }; + if let Some(name) = &class.name { + let text = self.name(name); + self.check_binding(&text, name.range(), context); + } + if let Some(heritage) = &class.extends { + self.expression(&heritage.expression, context); + } + let derived = class.extends.is_some(); + self.private_scopes.push(PrivateScope::default()); + self.declare_private_names(class); + let mut constructors = 0; + for member in &class.members { + match member.data() { + ClassMember::Constructor(constructor) => { + constructors += 1; + if constructors > 1 { + self.error(member.range(), "a class has one constructor"); + } + let inner = Context { + super_call: derived, + super_property: true, + unique_parameters: true, + ..Self::function_context(context) + }; + let body = FunctionBody::Block(constructor.body.clone()); + self.callable(&constructor.parameters, Some(&body), inner); + } + ClassMember::Method(method) => { + self.check_method_name(method, member.range()); + self.accessor_parameters( + method.modifier, + &method.function.parameters, + member.range(), + ); + if let PropertyName::Computed(key) = &method.name { + self.expression(key, context); + } + self.function(&method.function, true, false, context); + } + ClassMember::Property(property) => { + self.check_field_name( + &property.name, + property.modifiers.is_static, + member.range(), + ); + if let PropertyName::Computed(key) = &property.name { + self.expression(key, context); + } + if let Some(initializer) = &property.initializer { + self.expression(initializer, Self::initializer_context(context)); + } + } + ClassMember::AutoAccessor(accessor) => { + self.check_field_name( + &accessor.name, + accessor.modifiers.is_static, + member.range(), + ); + if let PropertyName::Computed(key) = &accessor.name { + self.expression(key, context); + } + if let Some(initializer) = &accessor.initializer { + self.expression(initializer, Self::initializer_context(context)); + } + } + ClassMember::StaticBlock(block) => { + let inner = Context { + static_block: true, + in_function: false, + ..Self::initializer_context(context) + }; + let saved_labels = std::mem::take(&mut self.labels); + self.check_scope(&block.data().statements, ScopeKind::Function, &[], inner); + for statement in &block.data().statements { + self.statement(statement, inner); + } + self.labels = saved_labels; + } + _ => {} + } + } + let scope = self.private_scopes.pop().unwrap_or_default(); + for (name, range) in scope.unresolved { + if !scope.declared.contains(&name) { + self.private_reference(name, range); + } + } + } + + /// A field initializer behaves as a method body with no arguments object. + fn initializer_context(context: Context) -> Context { + Context { + super_property: true, + class_initializer: true, + ..Self::function_context(context) + } + } + + fn check_method_name(&mut self, method: &crate::syntax::MethodDeclaration, range: TextRange) { + let special = method.modifier != PropertyModifier::None + || method.function.is_async + || method.function.is_generator; + let name = self.property_text(&method.name); + if method.modifiers.is_static { + if name.as_deref() == Some("prototype") { + self.error(range, "a static method cannot be named `prototype`"); + } + } else if special && name.as_deref() == Some("constructor") { + self.error(range, "`constructor` must be a plain method"); + } + } + + fn check_field_name(&mut self, name: &PropertyName, is_static: bool, range: TextRange) { + let text = self.property_text(name); + if text.as_deref() == Some("constructor") + || (is_static && text.as_deref() == Some("prototype")) + { + self.error(range, "a field cannot have this name"); + } + } + + fn declare_private_names(&mut self, class: &ClassDeclaration) { + let mut declared: BTreeMap> = BTreeMap::new(); + for member in &class.members { + let (name, kind) = match member.data() { + ClassMember::Method(method) => { + let is_static = method.modifiers.is_static; + let kind = match method.modifier { + PropertyModifier::Get => PrivateKind::Getter { is_static }, + PropertyModifier::Set => PrivateKind::Setter { is_static }, + PropertyModifier::None => PrivateKind::Other, + }; + (&method.name, kind) + } + ClassMember::Property(property) => (&property.name, PrivateKind::Other), + ClassMember::AutoAccessor(accessor) => (&accessor.name, PrivateKind::Other), + _ => continue, + }; + let PropertyName::Private(private) = name else { + continue; + }; + let text = self.private_name(private); + if text == "#constructor" { + self.error( + private.range(), + "`#constructor` is not a valid private name", + ); + } + let existing = declared.entry(text).or_default(); + let compatible = existing.iter().all(|previous| match (previous, &kind) { + ( + PrivateKind::Getter { is_static: first }, + PrivateKind::Setter { is_static: second }, + ) + | ( + PrivateKind::Setter { is_static: first }, + PrivateKind::Getter { is_static: second }, + ) => first == second, + _ => false, + }); + if !compatible { + self.error(private.range(), "this private name is already declared"); + } + existing.push(kind); + } + if let Some(scope) = self.private_scopes.last_mut() { + scope.declared = declared.into_keys().collect(); + } + } + + // --- Expressions ----------------------------------------------------- + + fn target(&mut self, target: &AssignmentTargetNode, patterns_allowed: bool, context: Context) { + match target.data() { + AssignmentTarget::Identifier(identifier) => { + let name = self.name(identifier); + if context.strict && (name == "eval" || name == "arguments") { + self.error( + identifier.range(), + "`eval` and `arguments` cannot be assigned in strict mode", + ); + } + self.reference(identifier, context); + } + AssignmentTarget::Member(member) => { + self.member(&member.object, &member.property, context); + } + AssignmentTarget::Object(object) => { + if !patterns_allowed || is_parenthesized(self.file, target.range()) { + self.error(target.range(), "this operator cannot assign to a pattern"); + } + let last = object.properties.len().saturating_sub(1); + for (position, property) in object.properties.iter().enumerate() { + if property.rest && position != last { + self.error(property.target.range(), "a rest property must come last"); + } + if property.rest + && matches!( + property.target.data(), + AssignmentTarget::Object(_) | AssignmentTarget::Array(_) + ) + { + self.error( + property.target.range(), + "a rest property cannot be a pattern", + ); + } + if let PropertyName::Computed(key) = &property.name { + self.expression(key, context); + } + self.target(&property.target, true, context); + if let Some(initializer) = &property.initializer { + self.expression(initializer, context); + } + } + } + AssignmentTarget::Array(array) => { + if !patterns_allowed || is_parenthesized(self.file, target.range()) { + self.error(target.range(), "this operator cannot assign to a pattern"); + } + let last = array.elements.len().saturating_sub(1); + for (position, element) in array.elements.iter().enumerate() { + match element { + AssignmentArrayElement::Target(element) => { + self.array_element(element, context); + } + AssignmentArrayElement::Rest(element) => { + if position != last || self.comma_follows(element.range()) { + self.error(element.range(), "a rest element must come last"); + } + if matches!(element.data(), AssignmentTarget::Invalid(_)) { + self.error( + element.range(), + "a rest element cannot have an initializer", + ); + } + self.target(element, true, context); + } + AssignmentArrayElement::Elision | AssignmentArrayElement::Missing(_) => {} + } + } + } + AssignmentTarget::Invalid(expression) => { + // Annex B keeps `f() = 1` a runtime error in sloppy code. + if context.strict || !matches!(expression.data(), Expression::Call(_)) { + self.error(target.range(), "this cannot be assigned to"); + } + self.expression(expression, context); + } + // A method or accessor folds to this, and reaching the walker means + // the parser reported nothing else here. + AssignmentTarget::Missing(_) => { + self.error(target.range(), "this cannot be assigned to") + } + } + } + + /// An element of an array pattern, which may carry a default: the parser + /// keeps `[a = 1]`'s element as the assignment it was read as. + fn array_element(&mut self, element: &AssignmentTargetNode, context: Context) { + if let AssignmentTarget::Invalid(expression) = element.data() + && let Expression::Assignment(assignment) = expression.data() + && assignment.operator == AssignmentOperator::Assign + { + self.target(&assignment.left, true, context); + self.expression(&assignment.right, context); + } else { + self.target(element, true, context); + } + } + + fn member(&mut self, object: &Expr, property: &MemberProperty, context: Context) { + if matches!(object.data(), Expression::Super) { + if !context.super_property { + self.error(object.range(), "`super` is not available here"); + } + } else { + self.expression(object, context); + } + match property { + MemberProperty::Private(private) => { + let name = self.private_name(private); + self.private_reference(name, private.range()); + } + MemberProperty::Computed(key) => self.expression(key, context), + MemberProperty::Named(_) => {} + } + } + + fn arguments(&mut self, arguments: &[CallArgument], context: Context) { + for argument in arguments { + match argument { + CallArgument::Expression(expression) => self.expression(expression, context), + CallArgument::Spread(spread) => self.expression(&spread.argument, context), + CallArgument::Missing(_) => {} + } + } + } + + fn object(&mut self, members: &[crate::syntax::ObjectMemberNode], context: Context) { + let mut proto = 0; + for member in members { + match member.data() { + ObjectMember::Property(property) => { + if property.shorthand + && (!matches!(property.name, PropertyName::Identifier(_)) + || matches!(property.value.data(), Expression::Assignment(_))) + { + self.error(member.range(), "this shorthand needs a pattern around it"); + } + if let PropertyName::Computed(key) = &property.name { + self.expression(key, context); + } + if !property.shorthand + && self.property_text(&property.name).as_deref() == Some("__proto__") + { + proto += 1; + if proto > 1 { + self.error(member.range(), "`__proto__` is set twice"); + } + } + self.expression(&property.value, context); + } + ObjectMember::Method(method) => { + self.accessor_parameters( + method.modifier, + &method.function.parameters, + member.range(), + ); + if let PropertyName::Computed(key) = &method.name { + self.expression(key, context); + } + self.function(&method.function, true, false, context); + } + ObjectMember::Spread(spread) => self.expression(&spread.argument, context), + ObjectMember::Missing(_) => {} + } + } + } + + fn expression(&mut self, expression: &Expr, context: Context) { + match expression.data() { + Expression::Identifier(identifier) => self.reference(identifier, context), + Expression::Super => self.error(expression.range(), "`super` is not available here"), + Expression::Literal(Literal::Number(literal)) if context.strict => { + let text = self.token_of(literal.data().token()); + if text.len() > 1 && text.starts_with('0') && text.as_bytes()[1].is_ascii_digit() { + self.error( + expression.range(), + "legacy octal is not allowed in strict code", + ); + } + } + Expression::Literal(Literal::String(literal)) if context.strict => { + if has_legacy_escape(self.token_of(literal.data().token())) { + self.error( + expression.range(), + "legacy octal is not allowed in strict code", + ); + } + } + Expression::Template(template) => { + for element in &template.elements { + if has_legacy_escape(self.token_of(element.data().token())) { + self.error(element.range(), "a template cannot use a legacy escape"); + } + } + for inner in &template.expressions { + self.expression(inner, context); + } + } + Expression::TaggedTemplate(tagged) => { + if in_optional_chain(&tagged.tag) { + self.error(expression.range(), "a template cannot follow `?.`"); + } + self.expression(&tagged.tag, context); + for inner in &tagged.template.expressions { + self.expression(inner, context); + } + } + Expression::Array(array) => { + for element in &array.elements { + match element { + ArrayElement::Expression(inner) => self.expression(inner, context), + ArrayElement::Spread(spread) => self.expression(&spread.argument, context), + ArrayElement::Elision | ArrayElement::Missing(_) => {} + } + } + } + Expression::Object(object) => self.object(&object.members, context), + // A function expression's name is bound inside it, so a static block + // around it has no say. + Expression::Function(function) => self.function( + &function.function, + false, + true, + Context { + static_block: false, + ..context + }, + ), + Expression::Class(class) => self.class(&class.class, context), + Expression::Arrow(arrow) => { + let inner = Context { + generator: false, + is_async: arrow.is_async, + static_block: false, + unique_parameters: true, + in_iteration: false, + in_switch: false, + in_parameters: false, + in_function: true, + top_level_await: false, + async_parameters: context.async_parameters + || (context.in_parameters && context.is_async), + ..context + }; + self.callable(&arrow.parameters, Some(&arrow.body), inner); + } + Expression::Call(call) if self.is_import_phase(&call.callee) => { + self.import_phase_call(&call.arguments, expression.range(), context); + } + Expression::Call(call) => { + if matches!(call.callee.data(), Expression::Super) { + if !context.super_call { + self.error(call.callee.range(), "`super()` is not available here"); + } + } else { + self.expression(&call.callee, context); + } + self.arguments(&call.arguments, context); + } + Expression::Member(member) => self.member(&member.object, &member.property, context), + Expression::New(new) => { + if rooted_in_import(&new.callee) { + self.error(new.callee.range(), "`import()` cannot be constructed"); + } + self.expression(&new.callee, context); + self.arguments(&new.arguments, context); + } + Expression::Await(await_expression) => { + if context.in_parameters { + self.error(expression.range(), "`await` cannot appear in parameters"); + } + if !context.is_async && !context.top_level_await { + self.error(expression.range(), "`await` needs an async function"); + } + self.expression(&await_expression.argument, context); + } + Expression::Yield(yield_expression) => { + if context.in_parameters && context.generator { + self.error(expression.range(), "`yield` cannot appear in parameters"); + } + if let Some(argument) = &yield_expression.argument { + self.expression(argument, context); + } + } + Expression::Unary(unary) => { + if unary.operator == UnaryOperator::Delete { + match strip_parentheses(&unary.argument).data() { + Expression::Identifier(_) if context.strict => { + self.error( + expression.range(), + "`delete` of a name is not allowed in strict mode", + ); + } + Expression::Member(member) + if matches!(member.property, MemberProperty::Private(_)) => + { + self.error(expression.range(), "a private name cannot be deleted"); + } + _ => {} + } + } + self.expression(&unary.argument, context); + } + Expression::Update(update) => self.target(&update.argument, false, context), + Expression::Binary(binary) => { + match self.private_operand(&binary.left) { + Some(name) if binary.operator == BinaryOperator::In => { + self.private_reference(name, binary.left.range()); + } + _ => self.expression(&binary.left, context), + } + self.expression(&binary.right, context); + } + Expression::Logical(logical) => { + for side in [&logical.left, &logical.right] { + if let Expression::Logical(inner) = side.data() + && is_mixed_logical(logical.operator, inner.operator) + { + self.error(side.range(), "`??` cannot mix with `&&` or `||`"); + } + } + self.expression(&logical.left, context); + self.expression(&logical.right, context); + } + Expression::Conditional(conditional) => { + self.expression(&conditional.test, context); + self.expression(&conditional.consequent, context); + self.expression(&conditional.alternate, context); + } + Expression::Assignment(assignment) => { + if matches!( + assignment.operator, + AssignmentOperator::LogicalAndAssign + | AssignmentOperator::LogicalOrAssign + | AssignmentOperator::NullishAssign + ) && matches!(assignment.left.data(), AssignmentTarget::Invalid(_)) + { + self.error(assignment.left.range(), "this cannot be assigned to"); + } + let patterns_allowed = assignment.operator == AssignmentOperator::Assign; + self.target(&assignment.left, patterns_allowed, context); + self.expression(&assignment.right, context); + } + Expression::Sequence(sequence) => { + for inner in &sequence.expressions { + self.expression(inner, context); + } + } + Expression::Parenthesized(inner) => self.expression(inner, context), + Expression::Import(import) => { + self.expression(&import.source, context); + if let Some(options) = &import.options { + self.expression(options, context); + } + } + Expression::Meta(MetaProperty::NewTarget) => { + if self.meta_word(expression) != "target" { + self.error(expression.range(), "`new.target` cannot contain an escape"); + } + if !context.new_target { + self.error(expression.range(), "`new.target` needs a function"); + } + } + Expression::Meta(MetaProperty::ImportMeta) => { + if self.meta_word(expression) != "meta" { + self.error(expression.range(), "this `import` form is not valid here"); + } + if context.script { + self.error(expression.range(), "`import.meta` needs a module"); + } + } + _ => {} + } + } +} + +#[cfg(test)] +mod tests { + use std::sync::Arc; + + use super::*; + use crate::{ + parser::parse, + scanner::scan, + source::{ScriptKind, SourceId, SourceText}, + }; + + fn errors(source: &str) -> Vec { + let text = Arc::new(SourceText::new(source).expect("source fits")); + let parsed = parse(scan(SourceId::new(0), ScriptKind::JavaScript, text)); + check(parsed.product()) + .iter() + .map(|diagnostic| diagnostic.message().to_owned()) + .collect() + } + + #[test] + fn invalid_programs_are_rejected() { + for source in [ + "{ let a; let a; }", + "{ let a; var a; }", + "{ var a; class a {} }", + "let a; function a() {}", + "function f(a) { let a; }", + "for (let a;;) { var a; }", + "try {} catch (e) { let e; }", + "switch (0) { case 1: let f; default: function* f() {} }", + "'use strict'; { function f() {} function f() {} }", + "a: a: ;", + "break;", + "continue;", + "x: { continue x; }", + "while (1) { break missing; }", + "'use strict'; var public;", + "'use strict'; eval = 1;", + "'use strict'; delete x;", + "'use strict'; with (a) {}", + "function f() { 'use strict'; var static; }", + "function f(a = 1) { 'use strict'; }", + "function* g() { var yield; }", + "async function f() { var await; }", + "function* g(x = yield) {}", + "async function f(x = await 1) {}", + "class C { #a; #a; }", + "class C { get #a() {} get #a() {} }", + "class C { static get #a() {} set #a(v) {} }", + "class C { #constructor; }", + "class C { m() { this.#missing; } }", + "class C { #a; m() { delete this.#a; } }", + "class C { #a; m() { delete ((this.#a)); } }", + "this.#outside;", + "class C { constructor() {} constructor() {} }", + "class C { get constructor() {} }", + "class C { *constructor() {} }", + "class C { async constructor() {} }", + "class C { static prototype() {} }", + "class C { constructor; }", + "class C { static prototype; }", + "class C { x = arguments; }", + "class C { x = () => arguments; }", + "class C { m() { super(); } }", + "class C { constructor() { super(); } }", + "function f() { super.x; }", + "class let {}", + "let let = 1;", + "let [a, a] = [];", + "1 = 2;", + "'use strict'; f() = 1;", + "class C { static { function await() {} } }", + "a + b = c;", + "[...a, b] = c;", + "[...a = 1] = c;", + "[a] += 1;", + "var [...a, b] = c;", + "var [...a = 1] = c;", + "var {...a, b} = c;", + "var {...{a}} = c;", + "function f(...a, b) {}", + "function f(...a = []) {}", + "function f(a, a) { 'use strict'; }", + "(a, a) => 1;", + "({ get a(b) {} });", + "({ set a() {} });", + "({ set a(...b) {} });", + "({ __proto__: 1, __proto__: 2 });", + "if (1) class C {}", + "while (1) function f() {}", + "if (1) async function f() {}", + "if (1) let [a] = b;", + "if (1) let\n[a] = b;", + "new.target;", + "new import('a');", + ] { + let found = errors(source); + assert!(!found.is_empty(), "{source:?} must be rejected"); + } + } + + #[test] + fn valid_programs_are_accepted() { + for source in [ + "{ function f() {} function f() {} }", + "function f() {} function f() {}", + "var a; var a;", + "function a() {} var a;", + "function f(a) { var a; }", + "function f(a, a) {}", + "try {} catch (e) { var e; }", + "for (var a;;) { var a; }", + "a: b: while (1) { continue a; }", + "a: { break a; }", + "a: ; a: ;", + "while (1) { function f() { a: ; } break; }", + "var public, static, let, yield, await;", + "eval = 1; arguments = 2;", + "delete x;", + "with (a) {}", + "if (1) function f() {}", + "a: function f() {}", + "function* g() { yield 1; function h(x = 1) { var yield; } }", + "async function f() { await 1; function g() { var await; } }", + "class C { #a; m() { this.#a; #a in this; } }", + "class C { get #a() {} set #a(v) {} static #b; }", + "class C { #a; m() { class D { n(x) { x.#a; } } } }", + "class C { static async *g() {} static x = 1; constructor() {} }", + "class C { 'constructor'() {} }", + "class C { static constructor() {} static get constructor() {} }", + "class C { m() { super.m(); } x = super.y; static { super.z; } }", + "class B {} class C extends B { constructor() { super(); () => super(); } }", + "({ m() { super.x; } });", + "({ __proto__: 1, ['__proto__']: 2, __proto__() {}, get __proto__() { return 1; } });", + "({ __proto__ });", + "var [a, ...b] = c, {d, ...e} = f;", + "[a, ...b] = c;", + "({ a, ...b } = c);", + "function f(a = 1, ...b) {}", + "(function () { new.target; });", + "class C { x = new.target; }", + "function f() { 'use strict'; }", + "function f(a) { 'use strict'; }", + "'use strict'; class C { static x = 1; }", + "a ? b : c; `x${a}y`; a`b${c}`;", + "if (1) let\na;", + "while (0) let\n{}", + "yield = 1; function f(a = yield) {}", + "class C { static { () => { var await; }; (function await(await) {}); } }", + "f() = 1; f()++; for (f() of []) ;", + "class C { #\\u{6F}; m() { this.#o; } }", + "class C { constructor() {} static constructor() {} static *constructor() {} }", + "[a = 1, [b = 2]] = c;", + "[...a] = b; [...[c, d]] = e; [...f.g] = h;", + "for (var a of b) ({ x: [c = 1] } = a);", + "for (let a of b) { let a; }", + "for (var a in b) var c;", + "switch (0) { case 1: var a; default: var a; }", + "class C { x = function () { arguments; }; }", + "class C { static { var await_ = 1; } }", + "async () => { await 1; };", + "async function f() { () => 1; }", + ] { + assert_eq!(errors(source), Vec::::new(), "{source:?}"); + } + } + + #[test] + fn syntax_the_grammar_alone_cannot_refuse_is_refused() { + for source in [ + "var x = ({ bre\\u0061k }) => {};", + "var this = 1;", + "\"use strict\"; 010;", + "\"use strict\"; '\\1';", + "\"use strict\"; '\\8';", + "function f() { '\\1'; 'use strict'; }", + "`\\8`;", + "`\\00`;", + "({ a = 1 });", + "({ 0 });", + "({ [a] });", + "(...a,) => {};", + "function f(...a,) {}", + "[...a,] = b;", + "let [...a,] = b;", + "const a;", + "for (const a;;) {}", + "{ using a; }", + "{ using a = null, [] = null; }", + "for (using a in b) {}", + "switch (0) { case 0: using a = null; }", + "return;", + "class C { static { return; } }", + "0 && 0 ?? 1;", + "0 ?? 0 || 1;", + "a?.b`c`;", + "f() &&= 1;", + "({}) = 1;", + "[a, ({ b })] = c;", + "if (0) a: function f() {}", + "class C { #a; m() { #a in #a in this; } }", + "class C { #a; m() { for (#a in []); } }", + "async(a = await => {}) => {};", + "(async function* await() {});", + "(async function* yield() {});", + "(function* yield() {});", + "async function f() { function await() {} }", + ] { + assert!(!errors(source).is_empty(), "{source} must be rejected"); + } + for source in [ + "'\\0'; 010; '\\1'; 09;", + "\"use strict\"; '\\0';", + "`\\0`;", + "a`\\8`;", + "({ a = 1 } = b);", + "[{ a = 1 }] = b;", + "({ a = 1 }) => a;", + "(...a) => {};", + "[...a, ] ;", + "for (const a of b) {}", + "for (const a in b) {}", + "{ using a = null; }", + "for (using a of b) {}", + "function f() { return; }", + "() => { return; };", + "class C { static { (() => { return; })(); } }", + "0 && (0 ?? 1);", + "(0 && 0) ?? 1;", + "a?.b.c;", + "f() = 1;", + "({ a } = b);", + "if (0) function f() {}", + "a: function f() {}", + "class C { #a; m() { return #a in this; } }", + "async function f() { await 0; }", + "async(a = 1) => {};", + "function await() {}", + "(function await() {});", + "(async function () { function yield() {} });", + "async function f() { (function await() {}); }", + "export {}; await 0;", + "function f() { return await(null); }", + "function await() {} await(1);", + "var await; await instanceof f;", + "await 0;", + "await f();", + "function f() { return g(await); }", + ] { + assert_eq!(errors(source), Vec::::new(), "{source}"); + } + } + + #[test] + fn only_import_calls_use_a_phase() { + for source in [ + "import.UNKNOWN('a');", + "import.defer();", + "import.source(...a);", + "new import.defer('a');", + "new import.source('a').b;", + "new import('a').b;", + "typeof import.source;", + "import.m\\u0065ta;", + "function f() { new.t\\u0061rget; }", + ] { + assert!(!errors(source).is_empty(), "{source} must be rejected"); + } + for source in [ + "import.defer('a');", + "import.source('a', {});", + "import . /* a.b */ meta;", + "function f() { new . target; }", + "new (import('a'));", + ] { + assert_eq!(errors(source), Vec::::new(), "{source}"); + } + } + + #[test] + fn an_object_pattern_keeps_its_rest_rules() { + for source in [ + "({ ...a, b } = c);", + "({ ...{ a } } = b);", + "({ ...[a] } = b);", + "({ get a() {} } = b);", + "[{ m() {} }] = a;", + ] { + assert!(!errors(source).is_empty(), "{source} must be rejected"); + } + for source in [ + "({ a, ...b } = c);", + "({ ...a.b } = c);", + "({ ...a[0] } = c);", + ] { + assert_eq!(errors(source), Vec::::new(), "{source}"); + } + } + + #[test] + fn a_module_is_strict_and_reserves_await() { + for source in [ + "export {}; var public;", + "export {}; var await;", + "export {}; with (a) {}", + "export {}; function f() {} function f() {}", + ] { + assert!(!errors(source).is_empty(), "{source:?} must be rejected"); + } + assert_eq!(errors("export {}; import.meta;"), Vec::::new()); + } +} diff --git a/crates/bamts-compiler/src/emitter.rs b/crates/bamts-compiler/src/emitter.rs index 23a0640d..30c8a9bd 100644 --- a/crates/bamts-compiler/src/emitter.rs +++ b/crates/bamts-compiler/src/emitter.rs @@ -24,14 +24,14 @@ pub mod transforms; pub mod transpile; use std::{ - collections::{BTreeMap, BTreeSet}, + collections::{BTreeMap, BTreeSet, HashMap}, fmt::Write as _, sync::Arc, }; use crate::checker::{SemanticModel, Type, TypeId, render_type_declaration}; use crate::diagnostic::{Diagnostic, DiagnosticCode}; -use crate::enum_plan::{EnumFacts, EnumMemberPlan, EnumScalar}; +use crate::enum_plan::{EnumFacts, EnumScalar}; use crate::jsx_desugar::JsxSourceDesugarPlan; use crate::source::{JsxEmit, SourceId, SourceText, TextRange, Utf16Pos}; use crate::syntax::*; @@ -87,6 +87,9 @@ pub enum ModuleKind { System, Es2015, EsNext, + /// A Node mode, where each file is CommonJS or an ES module. The project + /// resolves it per file before emit. + Node, } /// The structural line terminator the emitter inserts between lines. @@ -118,6 +121,14 @@ pub struct EmitOptions { /// Assume downleveling helpers exist globally; emit no helper prelude /// (`noEmitHelpers`, tsc's noEmitHelpers). pub no_emit_helpers: bool, + /// Leave authored comments out of the output (`removeComments`). + pub remove_comments: bool, + /// Keeps every import and export as written instead of eliding by use. + pub verbatim_module_syntax: bool, + /// `(specifier, name)` pairs the program knows have no runtime form. + pub erased_imports: Arc>, + /// Something outside the source reads imports, so none can be proven unused. + pub keeps_imports: bool, pub use_define_for_class_fields: Option, pub declaration: bool, pub emit_declaration_only: bool, @@ -148,6 +159,10 @@ impl Default for EmitOptions { always_strict: false, import_helpers: false, no_emit_helpers: false, + remove_comments: false, + verbatim_module_syntax: false, + erased_imports: Arc::default(), + keeps_imports: false, use_define_for_class_fields: None, declaration: false, emit_declaration_only: false, @@ -254,6 +269,12 @@ impl EmitOptions { }; self.no_emit_helpers = no_emit_helpers; } + "removecomments" => { + let Some(remove_comments) = parse_bool(value) else { + return Some(invalid()); + }; + self.remove_comments = remove_comments; + } "module" => { let Some(module) = parse_module(value) else { return Some(invalid()); @@ -392,6 +413,10 @@ impl EmitOptions { source_map: self.source_map, inline_source_map: self.inline_source_map, inline_sources: self.inline_sources, + remove_comments: self.remove_comments, + verbatim_module_syntax: self.verbatim_module_syntax, + erased_imports: Arc::clone(&self.erased_imports), + keeps_imports: self.keeps_imports, } } @@ -487,6 +512,11 @@ pub(crate) struct PrintOptions { pub(crate) source_map: bool, pub(crate) inline_source_map: bool, pub(crate) inline_sources: bool, + pub(crate) remove_comments: bool, + /// An ES module whose output has no module syntax left ends in `export {};`. + pub(crate) module_marker: bool, + /// An import or re-export of only types still loads its module. + pub(crate) verbatim_module_syntax: bool, } pub(crate) struct PrintSource<'a> { @@ -497,6 +527,8 @@ pub(crate) struct PrintSource<'a> { /// parser tree and any JSX-desugar consumption). Ids below it are /// parser-assigned and must carry ranges within the authored text. pub(crate) synthesized_floor: u32, + /// Raw lines that print in place of a marker statement. + pub(crate) inline_lines: Option<&'a HashMap>, } /// Print context for a braced block. Carries two tsc behaviors that split on @@ -581,6 +613,7 @@ pub(crate) fn print( // No rewriter ran: every id is parser-assigned, so the mint // gate's range check applies universally. synthesized_floor: u32::MAX, + inline_lines: None, }, model, options, @@ -606,6 +639,7 @@ pub(crate) fn print_with_jsx_plan( original_content, synthesized, synthesized_floor, + inline_lines, } = source; let mut source_map = (options.source_map || options.inline_source_map).then(|| { let mut builder = SourceMapBuilder::new() @@ -640,6 +674,8 @@ pub(crate) fn print_with_jsx_plan( .map(|token| token.range()) .collect(), tokens: file.tokens(), + pending_comments: Vec::new(), + comment_cursor: Utf16Pos::ZERO, source: file.source_text(), source_name: mapped_source_name, source_id: file.source_id(), @@ -666,6 +702,8 @@ pub(crate) fn print_with_jsx_plan( authored_len: Utf16Pos::new(utf16_len(original_content)), synthesized, synthesized_floor, + emitted_module_syntax: false, + inline_lines, }; if let Some(prelude) = prelude.filter(|prelude| !prelude.is_empty()) { let has_trailing_newline = prelude.ends_with('\n'); @@ -737,7 +775,8 @@ pub(crate) fn parse_module(value: &str) -> Option { "umd" => Some(ModuleKind::Umd), "system" => Some(ModuleKind::System), "es6" | "es2015" => Some(ModuleKind::Es2015), - "esnext" => Some(ModuleKind::EsNext), + "es2020" | "es2022" | "esnext" | "preserve" => Some(ModuleKind::EsNext), + "node16" | "node18" | "node20" | "nodenext" => Some(ModuleKind::Node), _ => None, } } @@ -822,6 +861,12 @@ struct Emitter<'a> { /// The full authored token stream, sorted by start. Backs authored-form /// lookups (arrow parameter parens) that the AST does not retain. tokens: &'a [Token], + /// Comments waiting for the next write. A statement that erases never + /// writes, so its comments go with it, as they do in `tsc`. + pending_comments: Vec, + /// Comments before this position are already spoken for, so rewritten + /// statements cannot print one twice. + comment_cursor: Utf16Pos, source_id: SourceId, model: &'a SemanticModel, enum_facts: &'a EnumFacts, @@ -863,6 +908,41 @@ struct Emitter<'a> { /// First rewriter-minted id for this file; ids below it are /// parser-assigned and must carry ranges within the authored text. synthesized_floor: u32, + /// An import or export reached the output. + emitted_module_syntax: bool, + inline_lines: Option<&'a HashMap>, +} + +/// The source text of an enum's numeric value. +pub(crate) fn enum_number_lexeme(value: f64) -> String { + if value.is_nan() { + "NaN".to_owned() + } else if value == f64::INFINITY { + "Infinity".to_owned() + } else if value == f64::NEG_INFINITY { + "-Infinity".to_owned() + } else { + value.to_string() + } +} + +/// The contents of a double-quoted literal holding `value`. +pub(crate) fn enum_string_lexeme(value: &bamts_bytecode::EcmaString) -> String { + let mut text = String::with_capacity(value.as_units().len()); + for &unit in value.as_units() { + match unit { + 0x08 => text.push_str("\\b"), + 0x09 => text.push_str("\\t"), + 0x0A => text.push_str("\\n"), + 0x0C => text.push_str("\\f"), + 0x0D => text.push_str("\\r"), + 0x22 => text.push_str("\\\""), + 0x5C => text.push_str("\\\\"), + 0x20..=0x7E => text.push(char::from_u32(u32::from(unit)).expect("ASCII unit")), + _ => write!(text, "\\u{unit:04X}").expect("writing to a String cannot fail"), + } + } + text } impl Emitter<'_> { @@ -895,18 +975,168 @@ impl Emitter<'_> { self.source.as_str().get(start..end) } - /// Prints a statement list with tsc's default comment preservation: each - /// statement that emits is followed by its same-line trailing comment - /// before the newline. Single-line-preserved bodies do not route here — - /// tsc expands a body whose statement carries a trailing comment, a - /// rule queued separately. + fn is_comment(token: &Token) -> bool { + matches!( + token.kind(), + TokenKind::LineComment | TokenKind::BlockComment + ) + } + + fn is_trivia(token: &Token) -> bool { + Self::is_comment(token) || token.kind() == TokenKind::Whitespace + } + + /// The comments before `position` back to the previous code token, which + /// comes with them. That token still exists for code that erased, which is + /// what keeps a comment from reaching past the declaration it belongs to. + fn comments_after_previous_token( + &self, + position: Utf16Pos, + ) -> (Vec, Option<&Token>) { + let end = self + .tokens + .partition_point(|token| token.range().start() < position); + let previous = self.tokens[..end] + .iter() + .rposition(|token| !Self::is_trivia(token)); + let first = previous.map_or(0, |index| index + 1); + let comments = self.tokens[first..end] + .iter() + .filter(|token| Self::is_comment(token) && token.range().start() >= self.comment_cursor) + .map(|token| token.range()) + .collect(); + (comments, previous.map(|index| &self.tokens[index])) + } + + fn source_line(&self, position: Utf16Pos) -> Option { + self.source.line_column(position).ok().map(|(line, _)| line) + } + + /// The comments that lead the code at `position`. One on the line of the + /// previous token trails that token instead, unless that token opens a block. + fn leading_comments(&self, position: Utf16Pos) -> Vec { + let (comments, previous) = self.comments_after_previous_token(position); + let trailing_line = previous + .filter(|token| token.kind() != TokenKind::LBrace) + .and_then(|token| self.source_line(token.range().end())); + comments + .into_iter() + .filter(|comment| { + trailing_line.is_none() || self.source_line(comment.start()) != trailing_line + }) + .collect() + } + + /// Queues the comments before a statement. They print on its first write. + fn queue_leading_comments(&mut self, start: Utf16Pos) { + if self.options.remove_comments || start > self.authored_len { + return; + } + self.pending_comments = self.leading_comments(start); + self.comment_cursor = self.comment_cursor.max(start); + } + + /// Prints the comments that close a file. + fn write_closing_comments(&mut self) { + self.queue_leading_comments(self.authored_len); + self.flush_comments(); + } + + /// The start of the first code in the file, or its end if there is none. + fn first_code_position(&self) -> Utf16Pos { + self.tokens + .iter() + .find(|token| !Self::is_trivia(token)) + .map_or(self.authored_len, |token| token.range().start()) + } + + /// Prints the comment header of a file: a run with no blank line inside it + /// and a blank line after it survives whatever the first statement does. + fn write_detached_comments(&mut self) { + if self.options.remove_comments { + return; + } + let first_code = self.first_code_position(); + let (comments, _) = self.comments_after_previous_token(first_code); + let starts_after = |emitter: &Self, index: usize| { + let next = comments + .get(index + 1) + .map_or(first_code, |next| next.start()); + emitter + .source_line(next) + .zip(emitter.source_line(comments[index].end())) + .is_some_and(|(next, end)| next >= end + 2) + }; + let Some(last) = (0..comments.len()).find(|&index| starts_after(self, index)) else { + return; + }; + self.comment_cursor = comments[last].end(); + self.pending_comments = comments[..=last].to_vec(); + self.flush_comments(); + } + + fn flush_comments(&mut self) { + for comment in std::mem::take(&mut self.pending_comments) { + self.write_comment(comment); + if self.comment_ends_its_line(comment) { + self.newline(); + } else { + self.raw(" "); + } + } + } + + /// Whether code follows a comment on its own last line. + fn comment_ends_its_line(&self, comment: TextRange) -> bool { + let start = self + .tokens + .partition_point(|token| token.range().start() < comment.end()); + let next = self.tokens[start..] + .iter() + .find(|token| !Self::is_trivia(token)); + next.is_none_or(|token| { + self.source_line(token.range().start()) != self.source_line(comment.end()) + }) + } + + /// Writes a comment at the current indentation. The lines of a block comment + /// keep their shape relative to where the comment began. + fn write_comment(&mut self, comment: TextRange) { + let Some(text) = self.source_slice(comment).map(str::to_owned) else { + return; + }; + let column = self + .source + .line_column(comment.start()) + .map_or(0, |(_, column)| column); + for (index, line) in text.split('\n').enumerate() { + let line = line.strip_suffix('\r').unwrap_or(line); + if index > 0 { + self.newline(); + } + let indent = line.len() - line.trim_start().len(); + self.raw(&line[if index == 0 { 0 } else { indent.min(column) }..]); + } + } + + /// Prints a statement list with tsc's default comment preservation: a + /// comment above a statement prints above it, and its same-line trailing + /// comment follows it before the newline. Single-line-preserved bodies do + /// not route here: tsc expands a body whose + /// statement carries a trailing comment, a rule queued separately. fn emit_statement_list(&mut self, statements: &[Stmt]) { for (index, statement) in statements.iter().enumerate() { - if self.emit_statement(statement) { + self.queue_leading_comments(statement.range().start()); + let emitted = self.emit_statement(statement); + self.pending_comments.clear(); + if emitted { let limit = statements .get(index + 1) .map_or_else(|| Utf16Pos::new(usize::MAX), |next| next.range().start()); - if let Some(comment) = self.trailing_line_comment(statement.range().end(), limit) { + if !self.options.remove_comments + && let Some(comment) = + self.trailing_line_comment(statement.range().end(), limit) + { self.raw(" "); if let Some(text) = self.source_slice(comment) { let text = text.to_owned(); @@ -1195,6 +1425,9 @@ impl<'a> Emitter<'a> { /// Flushes pending indentation, updating `generated_column`. fn flush_indent(&mut self) { + if !self.pending_comments.is_empty() { + self.flush_comments(); + } if self.pending_indent { let spaces = self.indent * self.options.indent_width as usize; for _ in 0..spaces { @@ -1323,7 +1556,13 @@ impl<'a> Emitter<'a> { // ---- module drivers --------------------------------------------------- fn emit_module_js(&mut self, statements: &[Stmt]) { + self.write_detached_comments(); self.emit_statement_list(statements); + if self.options.module_marker && !self.emitted_module_syntax { + self.raw("export {};"); + self.newline(); + } + self.write_closing_comments(); } fn emit_module_decl(&mut self, statements: &[Stmt]) { @@ -1340,20 +1579,41 @@ impl<'a> Emitter<'a> { // ======================================================================= fn emit_statement(&mut self, statement: &Stmt) -> bool { + if let Some(lines) = self + .inline_lines + .and_then(|lines| lines.get(&statement.id())) + { + for (index, line) in lines.lines().enumerate() { + if index > 0 { + self.newline(); + } + self.raw(line); + } + return true; + } let previous = self.anchor; self.anchor = statement.range(); let stmt_range = self.anchor; let emitted = match statement.data() { - Statement::Import(import) => self.emit_import(import, false), + Statement::Import(import) => { + let emitted = self.emit_import(import, false); + self.emitted_module_syntax |= emitted; + emitted + } Statement::ImportEquals(import) => { if import.is_type_only { false } else { self.emit_import_equals_js(import); + self.emitted_module_syntax = true; true } } - Statement::Export(export) => self.emit_export_js(export), + Statement::Export(export) => { + let emitted = self.emit_export_js(export); + self.emitted_module_syntax |= emitted; + emitted + } Statement::Variable(declaration) => { self.emit_variable_head(declaration); let semi_start = Utf16Pos::new(stmt_range.end().get().saturating_sub(1)); @@ -1375,7 +1635,7 @@ impl<'a> Emitter<'a> { true } Statement::Interface(_) | Statement::TypeAlias(_) | Statement::Declare(_) => false, - Statement::Enum(declaration) => self.emit_enum_js(statement.id(), declaration), + Statement::Enum(declaration) => self.emit_enum_js(declaration), Statement::Namespace(_) => { self.diag_here( codes::NAMESPACE_UNLOWERED, @@ -1874,7 +2134,10 @@ impl<'a> Emitter<'a> { // A clause that resolves to no value bindings is fully type-only and is // erased in JavaScript (a bare side-effect import keeps its clause-less // form below). - if !keep_types && let Some(clause) = &import.clause { + if !keep_types + && !self.options.verbatim_module_syntax + && let Some(clause) = &import.clause + { let has_value = clause.default.is_some() || matches!(&clause.binding, Some(ImportBinding::Namespace(_))) || matches!( @@ -2022,17 +2285,22 @@ impl<'a> Emitter<'a> { .iter() .filter(|s| matches!(s.data().mode, ExportSpecifierMode::Value)) .collect(); - if visible.is_empty() { + let loads_module = self.options.verbatim_module_syntax && source.is_some(); + if visible.is_empty() && !loads_module { return false; } - self.raw("export { "); - for (index, specifier) in visible.iter().enumerate() { - if index > 0 { - self.raw(", "); + if visible.is_empty() { + self.raw("export {}"); + } else { + self.raw("export { "); + for (index, specifier) in visible.iter().enumerate() { + if index > 0 { + self.raw(", "); + } + self.emit_export_specifier(specifier.data(), false); } - self.emit_export_specifier(specifier.data(), false); + self.raw(" }"); } - self.raw(" }"); if let Some(source) = source { self.raw(" from "); self.emit_string(source); @@ -2133,129 +2401,28 @@ impl<'a> Emitter<'a> { // enum lowering (JavaScript) // ======================================================================= - fn emit_enum_js(&mut self, declaration_id: NodeId, declaration: &EnumDeclaration) -> bool { - let Some(name) = self - .text(declaration.name.data().token()) - .map(str::to_owned) - else { - self.diag( - codes::UNRESOLVED_TOKEN, - "token text is not a valid slice of the source", - declaration.name.range(), - ); - return false; - }; - if name.is_empty() || declaration.is_const { - return false; - } - let Some(plan) = self.enum_facts.declaration(declaration_id) else { + /// The rewriter lowers every enum it has facts for, so one that reaches + /// the printer is either erased or has none. + fn emit_enum_js(&mut self, declaration: &EnumDeclaration) -> bool { + if !declaration.is_const { self.diag_here( codes::ENUM_FACTS_UNAVAILABLE, "runtime enum lowering requires checked enum facts", ); - return false; - }; - if plan.members().len() != declaration.members.len() { - self.diag_here( - codes::ENUM_FACTS_UNAVAILABLE, - "runtime enum lowering requires a complete checked enum plan", - ); - return false; - } - - self.raw("var "); - self.raw(&name); - self.raw(";"); - self.newline(); - self.raw("(function ("); - self.raw(&name); - self.raw(") {"); - self.newline(); - self.indent += 1; - for (member, plan_member) in declaration.members.iter().zip(plan.members()) { - if self.emit_enum_member(&name, member.data(), plan_member) { - self.newline(); - } - } - self.indent -= 1; - self.raw("})("); - self.raw(&name); - self.raw(" || ("); - self.raw(&name); - self.raw(" = {}));"); - true - } - - fn emit_enum_member( - &mut self, - object: &str, - member: &EnumMember, - plan_member: &EnumMemberPlan, - ) -> bool { - let (Some(key), Some(value)) = (plan_member.name(), plan_member.value()) else { - return false; - }; - let initializer = member.initializer.as_deref(); - if value.constant().is_none() && initializer.is_none() { - return false; - } - - self.raw(object); - self.raw("["); - if plan_member.reverse() { - self.raw(object); - self.raw("["); } - self.emit_enum_string(key); - self.raw("] = "); - if let Some(constant) = value.constant() { - self.emit_enum_scalar(constant); - } else if let Some(initializer) = initializer { - self.emit_expression_prec(initializer, P_ASSIGN); - } - if plan_member.reverse() { - self.raw("] = "); - self.emit_enum_string(key); - } - self.raw(";"); - true + false } fn emit_enum_scalar(&mut self, scalar: &EnumScalar) { match scalar { - EnumScalar::Number(value) => { - let value = value.to_f64(); - if value.is_nan() { - self.raw("NaN"); - } else if value == f64::INFINITY { - self.raw("Infinity"); - } else if value == f64::NEG_INFINITY { - self.raw("-Infinity"); - } else { - write!(self.out, "{value}").expect("writing to a String cannot fail"); - } - } + EnumScalar::Number(value) => self.raw(&enum_number_lexeme(value.to_f64())), EnumScalar::String(value) => self.emit_enum_string(value), } } fn emit_enum_string(&mut self, value: &bamts_bytecode::EcmaString) { self.raw("\""); - for &unit in value.as_units() { - match unit { - 0x08 => self.raw("\\b"), - 0x09 => self.raw("\\t"), - 0x0A => self.raw("\\n"), - 0x0C => self.raw("\\f"), - 0x0D => self.raw("\\r"), - 0x22 => self.raw("\\\""), - 0x5C => self.raw("\\\\"), - 0x20..=0x7E => self - .out - .push(char::from_u32(u32::from(unit)).expect("ASCII unit")), - _ => write!(self.out, "\\u{unit:04X}").expect("writing to a String cannot fail"), - } - } + self.raw(&enum_string_lexeme(value)); self.raw("\""); } @@ -2507,7 +2674,9 @@ impl<'a> Emitter<'a> { fn class_member_emits_js(&self, member: &ClassMember) -> bool { match member { - ClassMember::Constructor(_) | ClassMember::StaticBlock(_) => true, + ClassMember::Constructor(_) | ClassMember::StaticBlock(_) | ClassMember::Semicolon => { + true + } ClassMember::Method(method) => { method.function.body.is_some() && !method.modifiers.is_abstract } @@ -2579,6 +2748,10 @@ impl<'a> Emitter<'a> { self.diag_here(codes::MISSING_MEMBER, "cannot emit a missing class member"); false } + ClassMember::Semicolon => { + self.raw(";"); + true + } } } @@ -3560,6 +3733,10 @@ impl<'a> Emitter<'a> { AssignmentArrayElement::Target(target) => { self.emit_assignment_target(target) } + AssignmentArrayElement::Rest(target) => { + self.raw("..."); + self.emit_assignment_target(target); + } AssignmentArrayElement::Elision => {} AssignmentArrayElement::Missing(_) => { self.diag_here(codes::MISSING_TARGET, "cannot emit a missing target"); @@ -3587,6 +3764,11 @@ impl<'a> Emitter<'a> { } fn emit_assignment_object_property(&mut self, property: &AssignmentObjectProperty) { + if property.rest { + self.raw("..."); + self.emit_assignment_target(&property.target); + return; + } let shorthand = property.initializer.is_none() && self.target_matches_name(&property.name, &property.target); if shorthand { @@ -4362,7 +4544,7 @@ impl<'a> Emitter<'a> { fn class_member_emits_decl(&self, member: &ClassMember) -> bool { !matches!( member, - ClassMember::StaticBlock(_) | ClassMember::Missing(_) + ClassMember::StaticBlock(_) | ClassMember::Missing(_) | ClassMember::Semicolon ) } @@ -4442,7 +4624,7 @@ impl<'a> Emitter<'a> { self.raw(";"); true } - ClassMember::StaticBlock(_) => false, + ClassMember::StaticBlock(_) | ClassMember::Semicolon => false, ClassMember::Missing(_) => { self.diag_here(codes::MISSING_MEMBER, "cannot emit a missing class member"); false @@ -5984,6 +6166,41 @@ export default answer; assert_eq!(javascript(&output).code, ""); } + #[test] + fn array_assignment_rest_and_defaults_emit_as_written() { + let input = "[a = 1, ...b] = c;\n"; + let parsed = crate::parser::parse(crate::scanner::scan( + SourceId::new(0), + ScriptKind::TypeScript, + Arc::new(SourceText::new(input).expect("test source fits the per-file budget")), + )); + assert!(parsed.diagnostics().is_empty()); + let output = emit_output(parsed.product(), &EmitOptions::default()); + assert!(!output.has_errors()); + assert_eq!(javascript(&output).code, input); + } + + #[test] + fn a_bare_class_semicolon_survives_javascript_and_not_declarations() { + // Authority: `classWithSemicolonClassElement1` and `2` keep each `;`. + let input = "export class C {\n ;\n ;\n}\n"; + let parsed = crate::parser::parse(crate::scanner::scan( + SourceId::new(0), + ScriptKind::TypeScript, + Arc::new(SourceText::new(input).expect("test source fits the per-file budget")), + )); + assert!(parsed.diagnostics().is_empty()); + let output = emit_output(parsed.product(), &EmitOptions::default()); + assert!(!output.has_errors()); + assert_eq!( + javascript(&output).code, + "export class C {\n ;\n ;\n}\n" + ); + let declaration = emit_declaration(parsed.product()); + assert!(!declaration.has_errors()); + assert!(!format!("{declaration:?}").contains(";\\n ;")); + } + #[test] fn empty_class_body_expands_in_javascript() { // Authority: `exnextmodulekindExportClassNameWithObject(target=es2015)` @@ -6257,6 +6474,66 @@ export default answer; ); } + #[test] + fn comments_above_statements_survive_in_javascript() { + let emit = |input: &str, remove_comments: bool| { + let parsed = crate::parser::parse(crate::scanner::scan( + SourceId::new(0), + ScriptKind::TypeScript, + Arc::new(SourceText::new(input).expect("test source fits the per-file budget")), + )); + assert!(parsed.diagnostics().is_empty()); + let options = EmitOptions { + remove_comments, + ..EmitOptions::default() + }; + javascript(&emit_output(parsed.product(), &options)) + .code + .clone() + }; + assert_eq!( + emit("// above\nvar x = 1;\n// next\nvar y = 2;\n", false), + "// above\nvar x = 1;\n// next\nvar y = 2;\n" + ); + assert_eq!( + emit("/* one */ var x = 1;\n", false), + "/* one */ var x = 1;\n" + ); + assert_eq!( + emit("function f() {\n // inner\n return 1;\n}\n", false), + "function f() {\n // inner\n return 1;\n}\n" + ); + assert_eq!( + emit("var x; // t\n// next\nvar y;\n", false), + "var x; // t\n// next\nvar y;\n" + ); + // A comment on an erased declaration goes with it. + assert_eq!( + emit("// gone\ninterface I {}\n// kept\nvar x;\n", false), + "// kept\nvar x;\n" + ); + // A header stays even when the first statement erases. + assert_eq!( + emit("// header\n\n// gone\ninterface I {}\nvar x;\n", false), + "// header\nvar x;\n" + ); + assert_eq!(emit("var x;\n// end\n", false), "var x;\n// end\n"); + // The rewriter removes a `declare` from the list, but its comment stays + // with it rather than moving to the next statement that prints. + assert_eq!( + emit( + "// one\ndeclare var a;\n\n// two\ndeclare var b;\nc();\n", + false + ), + "c();\n" + ); + assert_eq!( + emit("if (a) {\n /* a\n b */\n f();\n}\n", false), + "if (a) {\n /* a\n b */\n f();\n}\n" + ); + assert_eq!(emit("// above\nvar x; // t\n// end\n", true), "var x;\n"); + } + #[test] fn trailing_line_comment_follows_statement_in_javascript() { // Authority: arrayAugment prints `var y; // Expect no error here` diff --git a/crates/bamts-compiler/src/emitter/declarations.rs b/crates/bamts-compiler/src/emitter/declarations.rs index 1486c6cb..3cc87244 100644 --- a/crates/bamts-compiler/src/emitter/declarations.rs +++ b/crates/bamts-compiler/src/emitter/declarations.rs @@ -111,6 +111,9 @@ pub fn emit_declarations( source_map: options.declaration_map, inline_source_map: false, inline_sources: false, + remove_comments: false, + module_marker: false, + verbatim_module_syntax: false, }, names, Surface::Declaration, diff --git a/crates/bamts-compiler/src/emitter/transforms.rs b/crates/bamts-compiler/src/emitter/transforms.rs index c3eec17d..3b758296 100644 --- a/crates/bamts-compiler/src/emitter/transforms.rs +++ b/crates/bamts-compiler/src/emitter/transforms.rs @@ -15,7 +15,7 @@ //! ES2015, and generators below ES2015 produce typed diagnostics instead of //! silently emitting later syntax. -use std::collections::{BTreeMap, BTreeSet}; +use std::collections::{BTreeMap, BTreeSet, HashMap}; use std::sync::Arc; use crate::checker::SemanticModel; @@ -35,6 +35,10 @@ use super::{ print_with_jsx_plan, }; +mod namespaces; + +use namespaces::{ContainerBinding, NamespaceFrame, is_qualified_alias}; + /// Stable diagnostic identifiers produced by target transforms. pub mod codes { use crate::diagnostic::DiagnosticCode; @@ -133,6 +137,12 @@ impl ScriptTarget { pub struct TransformOptions { /// Language target that drives downlevel decisions. pub target: ScriptTarget, + /// Keeps every import as written instead of eliding by use. + pub verbatim_module_syntax: bool, + /// `(specifier, name)` pairs the program knows have no runtime form. + pub erased_imports: std::sync::Arc>, + /// Something outside the source reads imports, so none can be proven unused. + pub keeps_imports: bool, /// Whether the emitted file begins with a `"use strict"` prologue. pub always_strict: bool, /// When true, class fields that survive downlevel use `define` semantics. @@ -158,6 +168,8 @@ pub struct TransformOptions { pub inline_source_map: bool, /// Whether JavaScript source maps include the original source text. pub inline_sources: bool, + /// Leave authored comments out of the output. + pub remove_comments: bool, } /// Features found in a file, helpers they require, and diagnostics they produce. @@ -282,12 +294,11 @@ pub fn emit_transformed( let helper_emit = helpers::emit_helpers_for_source(&used_helpers, &options.helpers, &rewritten, is_module); let cjs_marker = rewriter.cjs_marker_prelude(!runtime_prelude.is_empty()); - let cjs_requires = rewriter.cjs_require_prelude(); let prelude = join_preludes( &strict_prelude(file, options, is_module), &join_preludes( &helper_emit.prelude, - &join_preludes(&cjs_marker, &join_preludes(&runtime_prelude, &cjs_requires)), + &join_preludes(&cjs_marker, &runtime_prelude), ), ); let mut output = print_with_jsx_plan( @@ -296,6 +307,7 @@ pub fn emit_transformed( original_content: file.source_text().as_str(), synthesized: Some(&rewriter.synthesized_ids), synthesized_floor: rewriter.synthesized_floor, + inline_lines: Some(&rewriter.inline_lines), }, model, PrintOptions { @@ -304,6 +316,13 @@ pub fn emit_transformed( source_map: options.source_map, inline_source_map: options.inline_source_map, inline_sources: options.inline_sources, + remove_comments: options.remove_comments, + verbatim_module_syntax: options.verbatim_module_syntax, + module_marker: is_module + && matches!( + options.module_kind, + Some(ModuleKind::Es2015 | ModuleKind::EsNext) + ), }, names, Surface::JavaScript, @@ -593,7 +612,33 @@ struct Rewriter<'a> { expression_temp_scopes: Vec>, parameter_initializer_depths: Vec, cjs: Option, + /// Names `import type` binds, which an `export { name }` cannot publish. + type_imports: BTreeSet, + /// Names the file reads as values, or `None` when imports stay as written. + import_reads: Option>, + /// Raw lines for marker statements, keyed by the marker's node. + inline_lines: HashMap, + /// The expression of the expression statement being rewritten, whose value + /// nothing reads. + statement_expression: Option, + /// `module.exports = value;`, which closes the module. + export_assignment: Option, synthesized_ids: BTreeSet, + file: &'a SourceFile, + /// Namespace bodies being lowered, innermost last. + namespace_frames: Vec, + /// Names each enclosing statement list has declared so far. + declared_names: Vec>, + /// Identifiers a generated name must avoid, once one is needed. + occupied_names: Option>, + /// Where each value name is declared, once a namespace needs to know. + value_symbol_ranges: Option>>, + /// Where each alias read as a value is declared, once one needs to know. + read_aliases: Option>, + /// Temporaries taken outside any namespace body. + module_temps: usize, + /// The parameter holding each alias a namespace body publishes. + published_aliases: BTreeMap, } /// Per-file CommonJS lowering plan built before the rewrite walk. @@ -603,10 +648,30 @@ struct CjsPlan { is_module: bool, /// Exported names needing `exports. = void 0;` in the prelude. void_0: Vec, - /// `require` prelude lines in import-statement order. + /// The CommonJS lines of the statement being planned. requires: Vec, + /// Each import or re-export's lines, printed where the statement stood: + /// upstream keeps them in source order among the other statements. + statement_lines: HashMap, /// Import binding replacement by resolved module-scope symbol. imports: BTreeMap, + /// Exported `var`/`let` bindings, which live only on `exports`. + exported_variables: BTreeSet, + /// Helpers the prelude's re-exports call. + helpers: BTreeSet, + /// `(exported, local)` for functions. They hoist, so their assignments run + /// before any require, which also keeps a star export off those names. + hoisted_exports: Vec<(String, String)>, + /// Exported names an import's own lines publish. + import_exports: BTreeSet, + /// `(exported, local)` to publish right after a statement declares them. + declared_exports: HashMap>, + /// Exported names a declaration publishes. + published_locals: BTreeSet, + /// Exported names of a local, which every write to it must update. + live_locals: BTreeMap>, + /// `export =` replaces `exports`, so the file is no ES module interop. + export_assignment: bool, /// Require-local names whose member calls become `(0, m.p)()`. require_roots: BTreeSet, } @@ -636,21 +701,28 @@ impl RequireNames { } fn allocate(&mut self, module: &str) -> String { - let base = module.rsplit('/').next().unwrap_or(module); - let counter = self.counters.entry(base.to_owned()).or_insert(0); - // tsc binds require locals as `_` when the path's last - // segment can start an identifier (`enum_1`) and sanitizes with a - // leading underscore only when it cannot (`_10_lib_1`). - let starts_identifier = base + // tsc binds require locals as `_`, where the path's last segment + // becomes an identifier: other characters turn into `_` (`m_js_1`) and a + // leading digit gets a `_` (`_10_lib_1`). + let segment = module.rsplit('/').next().unwrap_or(module); + let slug: String = segment .chars() - .next() - .is_some_and(|c| c.is_ascii_alphabetic() || c == '_' || c == '$'); + .map(|c| { + if c.is_ascii_alphanumeric() || c == '_' { + c + } else { + '_' + } + }) + .collect(); + let counter = self.counters.entry(slug.clone()).or_insert(0); + let leading_digit = slug.starts_with(|c: char| c.is_ascii_digit()); loop { *counter += 1; - let candidate = if starts_identifier { - format!("{base}_{}", *counter) + let candidate = if leading_digit { + format!("_{slug}_{}", *counter) } else { - format!("_{base}_{}", *counter) + format!("{slug}_{}", *counter) }; if self.occupied.insert(candidate.clone()) { return candidate; @@ -671,6 +743,9 @@ impl<'a> Rewriter<'a> { } else { None }; + let used_helpers = cjs + .as_ref() + .map_or_else(BTreeSet::new, |plan| plan.helpers.clone()); let synthesized_floor = ids.position(); Self { options, @@ -684,12 +759,25 @@ impl<'a> Rewriter<'a> { next_temp: 0, diagnostics: Vec::new(), replace_await: false, - used_helpers: BTreeSet::new(), + used_helpers, key_prelude: Vec::new(), expression_temp_scopes: Vec::new(), parameter_initializer_depths: Vec::new(), cjs, + type_imports: type_import_names(file, options), + import_reads: import_reads(file, options, model), + inline_lines: HashMap::new(), + statement_expression: None, + export_assignment: None, synthesized_ids: BTreeSet::new(), + file, + namespace_frames: Vec::new(), + declared_names: Vec::new(), + occupied_names: None, + value_symbol_ranges: None, + read_aliases: None, + module_temps: 0, + published_aliases: BTreeMap::new(), } } /// Scans top-level module syntax into a CommonJS lowering plan. @@ -698,26 +786,14 @@ impl<'a> Rewriter<'a> { options: &TransformOptions, model: &SemanticModel, ) -> Option { - let occupied: BTreeSet = file - .tokens() - .iter() - .filter(|token| token.kind() == TokenKind::Identifier) - .filter_map(|token| { - let start = file - .source_text() - .utf16_to_byte(token.range().start()) - .ok()?; - let end = file.source_text().utf16_to_byte(token.range().end()).ok()?; - file.source_text() - .as_str() - .get(start..end) - .map(str::to_owned) - }) - .collect(); - let mut names = RequireNames::new(occupied); + let mut names = RequireNames::new(identifier_names(file)); let mut plan = CjsPlan::default(); let module_scope = model.module_scope(); let symbol_of = |name: &str| model.scope(module_scope).value(name); + let hoisted_functions = hoisted_function_names(file); + let type_imports = type_import_names(file, options); + let reads = import_reads(file, options, Some(model)); + let exported_locals = exported_locals(file); for statement in file.statements() { match statement.data() { Statement::Import(import) => { @@ -725,15 +801,24 @@ impl<'a> Rewriter<'a> { continue; } plan.is_module = true; - let Some(clause) = &import.clause else { - continue; - }; let raw = raw_token_text(file, import.source.data().token()); let module = raw.trim_matches(['"', '\'']); - let local = names.allocate(module); // Upstream re-quotes module specifiers with double // quotes in every require form. let quoted = quote_module_specifier(module); + let Some(clause) = &import.clause else { + plan.statement_lines + .insert(statement.id(), format!("require({quoted});")); + continue; + }; + let clause = match &reads { + Some(reads) => kept_import_clause(file, clause, reads), + None => Some(clause.clone()), + }; + let Some(clause) = &clause else { + continue; + }; + let local = names.allocate(module); let line = format!( "{} {local} = require({quoted});", if options.target <= ScriptTarget::Es5 { @@ -746,6 +831,11 @@ impl<'a> Rewriter<'a> { plan.require_roots.insert(local.clone()); if let Some(default) = &clause.default { let text = identifier_text(file, default); + for exported in exported_locals.get(&text).into_iter().flatten() { + plan.import_exports.insert(exported.clone()); + plan.requires + .push(format!("{} = {local}.default;", exports_member(exported))); + } if let Some(symbol) = symbol_of(&text) { plan.imports.insert( symbol, @@ -759,6 +849,11 @@ impl<'a> Rewriter<'a> { match &clause.binding { Some(ImportBinding::Namespace(name)) => { let text = identifier_text(file, name); + for exported in exported_locals.get(&text).into_iter().flatten() { + plan.import_exports.insert(exported.clone()); + plan.requires + .push(format!("{} = {local};", exports_member(exported))); + } if let Some(symbol) = symbol_of(&text) { plan.imports.insert( symbol, @@ -775,16 +870,12 @@ impl<'a> Rewriter<'a> { if !matches!(specifier.mode, ImportSpecifierMode::Value) { continue; } - let imported = match &specifier.imported { - ModuleExportName::Identifier(ident) => { - Some(identifier_text(file, ident)) - } - _ => None, - }; - let Some(property) = imported else { - continue; - }; + let property = module_export_text(file, &specifier.imported); let text = identifier_text(file, &specifier.local); + for exported in exported_locals.get(&text).into_iter().flatten() { + plan.import_exports.insert(exported.clone()); + plan.requires.push(live_export(exported, &local, &property)); + } if let Some(symbol) = symbol_of(&text) { plan.imports.insert( symbol, @@ -805,6 +896,12 @@ impl<'a> Rewriter<'a> { } Statement::Export(export) => { plan.is_module = true; + plan.export_assignment |= matches!(export, ExportDeclaration::Assignment(_)); + plan_cjs_reexport(file, export, &mut names, &mut plan, &options.erased_imports); + let binds_type = |name: &str| { + type_imports.contains(name) || is_type_only_export(model, name) + }; + plan_cjs_local_exports(file, export, &hoisted_functions, binds_type, &mut plan); if let ExportDeclaration::Named(ExportNamedDeclaration::Declaration(inner)) = export { @@ -817,17 +914,56 @@ impl<'a> Rewriter<'a> { Statement::Function(_) | Statement::Interface(_) | Statement::TypeAlias(_) => {} + Statement::ImportEquals(import) if is_qualified_alias(import) => { + let name = identifier_text(file, &import.local); + plan.exported_variables.extend(symbol_of(&name)); + plan.void_0.push(name); + } + Statement::ImportEquals(ImportEqualsDeclaration { + is_type_only: false, + local, + reference: ExternalModuleReference::Require(source), + .. + }) => { + let name = identifier_text(file, local); + plan.exported_variables.extend(symbol_of(&name)); + plan.requires.push(format!( + "{} = require({});", + exports_member(&name), + quote_module_specifier(&module_specifier(file, source)) + )); + } + Statement::Namespace(namespace) => { + if let NamespaceName::Identifier { name, .. } = &namespace.name + && model + .namespace_facts() + .declaration(inner.id()) + .is_some_and(|plan| plan.is_value_bearing()) + { + plan.void_0.push(identifier_text(file, name)); + } + } _ => { - for name in declared_binding_names(file, inner) { - plan.void_0.push(name); + let names = declared_binding_names(file, inner); + if let Statement::Variable(declaration) = inner.data() + && is_reassignable(declaration) + { + plan.exported_variables + .extend(names.iter().filter_map(|name| symbol_of(name))); } + plan.void_0.extend(names); } } } } _ => {} } + if !plan.requires.is_empty() { + let lines = std::mem::take(&mut plan.requires).join("\n"); + plan.statement_lines.insert(statement.id(), lines); + } } + plan_cjs_declared_exports(file, model, &exported_locals, &mut plan); // Script files keep the plan too: the marker stays off until a // require prelude (for example the JSX runtime) forces module form. plan.void_0.sort(); @@ -877,7 +1013,7 @@ impl<'a> Rewriter<'a> { fn cjs_export_target(&mut self, exported: &str, range: TextRange) -> AssignmentTargetNode { let exports_ident = self.ident("exports"); let object = self.syn_node(range, Expression::Identifier(exports_ident)); - let property = MemberProperty::Named(self.ident(exported)); + let property = self.named_or_computed(exported, range); self.syn_node( range, AssignmentTarget::Member(AssignmentMemberTarget { @@ -887,51 +1023,12 @@ impl<'a> Rewriter<'a> { ) } - /// `require("./m");` for a side-effect import, with the specifier - /// re-quoted the way upstream prints module text. - fn cjs_require_statement(&mut self, source: &StringLiteralNode) -> Stmt { - let range = source.range(); - let raw = self.token_text(source.data().token()).to_owned(); - let module = raw.trim_matches(['"', '\'']); - let require = self.ident("require"); - let callee = self.syn_node(range, Expression::Identifier(require)); - let literal = self.string_literal(module); - let module = self.syn_node(range, Expression::Literal(Literal::String(literal))); - let call = self.syn_node( - range, - Expression::Call(CallExpression { - callee: Box::new(callee), - optional: false, - type_arguments: None, - arguments: vec![CallArgument::Expression(Box::new(module))], - }), - ); - self.syn_node( - range, - Statement::Expression(ExpressionStatement { - expression: Box::new(call), - }), - ) - } - /// Replaces `export ` with assignments around the - /// declaration. Functions hoist, so the export assignment precedes them; + /// declaration. A function's assignment is hoisted into the prelude; /// classes and variables receive the `void 0` preamble instead. fn cjs_export_declaration(&mut self, range: TextRange, inner: &Stmt) -> Vec { match inner.data() { - Statement::Function(function) => { - let Some(name) = function - .function - .name - .as_ref() - .map(|ident| self.token_text(ident.data().token()).to_owned()) - else { - return self.rewrite_statement(inner); - }; - let mut out = vec![self.cjs_export_assignment(&name, &name, range)]; - out.extend(self.rewrite_statement(inner)); - out - } + Statement::Function(_) => self.rewrite_statement(inner), Statement::Class(class) => { let Some(name) = class .name @@ -944,8 +1041,12 @@ impl<'a> Rewriter<'a> { out.push(self.cjs_export_assignment(&name, &name, range)); out } + Statement::Variable(declaration) if is_reassignable(declaration) => { + self.namespace_export_variables(declaration, "exports", range) + } _ => { - let names = self.cjs_declared_names(inner); + // A const never changes, so its local copy stays accurate. + let names = declared_binding_names(self.file, inner); let mut out = self.rewrite_statement(inner); for name in &names { out.push(self.cjs_export_assignment(name, name, range)); @@ -988,39 +1089,176 @@ impl<'a> Rewriter<'a> { ModuleExportName::Identifier(local) => local.range(), _ => continue, }; - let (local, exported) = match (&specifier.local, &specifier.exported) { - (ModuleExportName::Identifier(local), ModuleExportName::Identifier(exported)) => ( - self.token_text(local.data().token()).to_owned(), - self.token_text(exported.data().token()).to_owned(), - ), - _ => continue, + let ModuleExportName::Identifier(local) = &specifier.local else { + continue; }; - out.push(self.cjs_export_assignment(&exported, &local, local_range)); + let local = self.token_text(local.data().token()).to_owned(); + let exported = module_export_text(self.file, &specifier.exported); + if self.exports_only_a_type(&local) { + continue; + } + let published = self.cjs.as_ref().is_some_and(|plan| { + plan.import_exports.contains(&exported) + || plan.published_locals.contains(&exported) + || plan + .hoisted_exports + .iter() + .any(|(name, value)| *name == exported && *value == local) + }); + if published { + continue; + } + let value = self.cjs_local_value(&local, local_range); + let left = self.cjs_export_target(&exported, local_range); + let assignment = self.syn_node( + local_range, + Expression::Assignment(AssignmentExpression { + operator: AssignmentOperator::Assign, + left, + right: Box::new(value), + }), + ); + out.push(self.syn_node( + local_range, + Statement::Expression(ExpressionStatement { + expression: Box::new(assignment), + }), + )); } out } - /// Declared value names of a statement, via the rewrite text. - fn cjs_declared_names(&self, statement: &Stmt) -> Vec { - match statement.data() { - Statement::Variable(declaration) => declaration - .declarations - .iter() - .filter_map(|declarator| match declarator.data().binding.data() { - BindingPattern::Identifier(ident) => { - Some(self.token_text(ident.data().token()).to_owned()) - } - _ => None, - }) - .collect(), - Statement::Enum(declaration) => { - vec![self.token_text(declaration.name.data().token()).to_owned()] + /// An ES import keeps only the bindings something reads as a value, and + /// goes away with the last one: naming a type would fail at load. + fn rewrite_esm_import(&mut self, statement: &Stmt, import: &ImportDeclaration) -> Vec { + let (Some(reads), Some(clause), false) = + (&self.import_reads, &import.clause, import.type_only) + else { + return vec![statement.clone()]; + }; + let Some(kept) = kept_import_clause(self.file, clause, reads) else { + return Vec::new(); + }; + if kept == *clause { + return vec![statement.clone()]; + } + vec![self.node( + statement.range(), + Statement::Import(ImportDeclaration { + clause: Some(kept), + ..import.clone() + }), + )] + } + + fn exports_only_a_type(&self, local: &str) -> bool { + self.type_imports.contains(local) + || self + .model + .is_some_and(|model| is_type_only_export(model, local)) + } + + /// `export { I }` of an interface or alias has no runtime form, so an ES + /// module must not name it, and a re-export with nothing left goes away. + fn rewrite_esm_export_specifiers( + &mut self, + statement: &Stmt, + specifiers: &[ExportSpecifierNode], + source: &Option, + ) -> Vec { + let module = source + .as_ref() + .map(|source| module_specifier(self.file, source)); + let kept: Vec = specifiers + .iter() + .filter(|specifier| match (&module, &specifier.data().local) { + (Some(module), _) => reexport_survives( + self.file, + specifier.data(), + module, + &self.options.erased_imports, + ), + (None, ModuleExportName::Identifier(local)) => { + !self.exports_only_a_type(self.token_text(local.data().token())) + } + (None, _) => true, + }) + .cloned() + .collect(); + if kept.len() == specifiers.len() { + return vec![statement.clone()]; + } + if kept.is_empty() && module.is_some() { + return Vec::new(); + } + let Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Specifiers { + attributes, + .. + })) = statement.data() + else { + return vec![statement.clone()]; + }; + vec![self.node( + statement.range(), + Statement::Export(ExportDeclaration::Named( + ExportNamedDeclaration::Specifiers { + type_only: false, + specifiers: kept, + source: source.clone(), + attributes: attributes.clone(), + }, + )), + )] + } + + /// What a module-scope name reads under CommonJS, where an exported + /// variable lives on `exports` and an import on its required module. + fn cjs_local_value(&mut self, local: &str, range: TextRange) -> Expr { + let symbol = self + .model + .and_then(|model| model.scope(model.module_scope()).value(local)); + let reference = symbol.and_then(|symbol| { + let plan = self.cjs.as_ref()?; + if plan.exported_variables.contains(&symbol) { + return Some(("exports".to_owned(), Some(local.to_owned()))); } - _ => Vec::new(), + let import = plan.imports.get(&symbol)?; + Some((import.root.clone(), import.property.clone())) + }); + match reference { + Some((root, Some(property))) => self.namespace_member(&root, &property, range), + Some((root, None)) => self.ident_expr(&root), + None => self.ident_expr(local), } } + /// Declared value names of a statement, via the rewrite text. fn rewrite_export_named_declaration(&mut self, statement: &Stmt, inner: &Stmt) -> Vec { + if self + .cjs + .as_ref() + .is_some_and(|plan| plan.statement_lines.contains_key(&statement.id())) + { + return self.cjs_statement_lines(statement); + } + let binding = if self.cjs.is_some() { + ContainerBinding::CommonJs + } else { + ContainerBinding::Export + }; + match inner.data() { + Statement::Namespace(namespace) => { + return self.lower_namespace(inner, namespace, binding, statement.range()); + } + Statement::Enum(declaration) if declaration.is_const => return Vec::new(), + Statement::Enum(declaration) => { + return self.lower_enum(inner, declaration, binding, statement.range()); + } + Statement::ImportEquals(import) if is_qualified_alias(import) => { + return self.lower_exported_alias(inner, import, binding, statement.range()); + } + _ => {} + } if self.cjs.is_some() { return self.cjs_export_declaration(statement.range(), inner); } @@ -1042,24 +1280,26 @@ impl<'a> Rewriter<'a> { } /// `export default function f() {}` / anonymous `default_1` renaming. fn cjs_default_function(&mut self, range: TextRange, function: &FunctionLike) -> Vec { - let (function, name) = match &function.name { - Some(ident) => ( - function.clone(), - self.token_text(ident.data().token()).to_owned(), - ), - None => { - let ident = self.ident("default_1"); - let mut renamed = function.clone(); - renamed.name = Some(ident); - (renamed, String::from("default_1")) - } - }; + let mut function = function.clone(); + if function.name.is_none() { + function.name = Some(self.ident("default_1")); + } let function = self.rewrite_function_like(&function, range); - let statement = self.node(range, Statement::Function(FunctionDeclaration { function })); - let mut out = vec![statement]; - let default_value = self.ident_expr(&name); - out.push(self.cjs_export_default_assignment(range, default_value)); - out + vec![self.node(range, Statement::Function(FunctionDeclaration { function }))] + } + + fn is_const_enum_use(&self, reference: NodeId) -> bool { + self.model + .is_some_and(|model| model.enum_facts().const_use(reference).is_some()) + } + + /// Whether a use resolved to a binding that lives only on `exports`. + fn cjs_reads_exported_variable(&self, reference: NodeId) -> bool { + self.cjs.as_ref().is_some_and(|plan| { + self.model + .and_then(|model| model.reference(reference)) + .is_some_and(|symbol| plan.exported_variables.contains(&symbol)) + }) } /// Rewrites a resolved import reference to its CommonJS member form. @@ -1076,12 +1316,12 @@ impl<'a> Rewriter<'a> { let Some(property) = property else { return Some(object); }; - let property = self.ident(&property); + let property = self.named_or_computed(&property, range); Some(self.node( range, Expression::Member(MemberExpression { object: Box::new(object), - property: MemberProperty::Named(property), + property, optional: false, }), )) @@ -1099,6 +1339,9 @@ impl<'a> Rewriter<'a> { let Some(symbol) = self.model.and_then(|model| model.reference(ident.id())) else { return false; }; + if plan.exported_variables.contains(&symbol) { + return true; + } plan.imports .get(&symbol) .is_some_and(|reference| reference.property.is_some()) @@ -1123,34 +1366,190 @@ impl<'a> Rewriter<'a> { if !(plan.is_module || force_module) { return String::new(); } - let mut prelude = - String::from("Object.defineProperty(exports, \"__esModule\", { value: true });\n"); + let mut prelude = if plan.export_assignment { + String::new() + } else { + String::from("Object.defineProperty(exports, \"__esModule\", { value: true });\n") + }; let mut names = plan.void_0.clone(); names.sort(); names.dedup(); if !names.is_empty() { - prelude.push_str("exports."); - for (index, name) in names.iter().enumerate() { - if index > 0 { - prelude.push_str(" = exports."); - } - prelude.push_str(name); - } + let members: Vec = names.iter().map(|name| exports_member(name)).collect(); + prelude.push_str(&members.join(" = ")); prelude.push_str(" = void 0;\n"); } + for (exported, local) in &plan.hoisted_exports { + prelude.push_str(&format!("{} = {local};\n", exports_member(exported))); + } prelude } - fn cjs_require_prelude(&self) -> String { - let Some(plan) = &self.cjs else { - return String::new(); + /// `.name`, or `["a b"]` for a name that is no identifier. + fn named_or_computed(&mut self, name: &str, range: TextRange) -> MemberProperty { + if is_identifier_name(name) { + return MemberProperty::Named(self.ident(name)); + } + let quoted = quote_string(name); + let literal = self.string_literal("ed[1..quoted.len() - 1]); + let key = self.syn_node(range, Expression::Literal(Literal::String(literal))); + MemberProperty::Computed(Box::new(key)) + } + + /// `module.exports = value;`, which `export =` means under CommonJS. + fn cjs_export_assignment_statement(&mut self, range: TextRange, value: &Expr) -> Stmt { + let value = self.rewrite_expr(value); + let module = self.ident("module"); + let object = self.syn_node(range, Expression::Identifier(module)); + let property = MemberProperty::Named(self.ident("exports")); + let left = self.syn_node( + range, + AssignmentTarget::Member(AssignmentMemberTarget { + object: Box::new(object), + property, + }), + ); + let assignment = self.syn_node( + range, + Expression::Assignment(AssignmentExpression { + operator: AssignmentOperator::Assign, + left, + right: Box::new(value), + }), + ); + self.syn_node( + range, + Statement::Expression(ExpressionStatement { + expression: Box::new(assignment), + }), + ) + } + + /// `exports.name = local;` for what a statement just declared. + fn cjs_publish_declared(&mut self, statement: &Stmt) -> Vec { + let pairs = self + .cjs + .as_mut() + .and_then(|plan| plan.declared_exports.remove(&statement.id())) + .unwrap_or_default(); + pairs + .iter() + .map(|(exported, local)| self.cjs_export_assignment(exported, local, statement.range())) + .collect() + } + + /// Exported names of what an assignment target writes, as `(local, names)`. + fn live_exports_of(&self, target: &AssignmentTargetNode) -> Vec<(String, Vec)> { + let (Some(plan), Some(model)) = (&self.cjs, self.model) else { + return Vec::new(); }; - let mut prelude = String::new(); - for line in &plan.requires { - prelude.push_str(line); - prelude.push('\n'); + let mut identifiers = Vec::new(); + collect_target_identifiers(target, &mut identifiers); + identifiers + .into_iter() + .filter_map(|identifier| { + let names = plan.live_locals.get(&model.reference(identifier.id())?)?; + Some((identifier_text(self.file, identifier), names.clone())) + }) + .collect() + } + + /// `exports.a = exports.b = value`. + fn cjs_assign_exports(&mut self, names: &[String], value: Expr, range: TextRange) -> Expr { + names.iter().fold(value, |value, name| { + let left = self.cjs_export_target(name, range); + self.syn_node( + range, + Expression::Assignment(AssignmentExpression { + operator: AssignmentOperator::Assign, + left, + right: Box::new(value), + }), + ) + }) + } + + fn sequence(&mut self, expressions: Vec, range: TextRange) -> Expr { + self.syn_node( + range, + Expression::Sequence(SequenceExpression { expressions }), + ) + } + + /// `temp = value`. + fn store_temp(&mut self, temp: &IdentifierNode, value: Expr, range: TextRange) -> Expr { + let left = self.syn_node(range, AssignmentTarget::Identifier(temp.clone())); + self.syn_node( + range, + Expression::Assignment(AssignmentExpression { + operator: AssignmentOperator::Assign, + left, + right: Box::new(value), + }), + ) + } + + /// Under CommonJS a local exported by `export { x }` has a copy on `exports`, + /// so a write to it also writes there, in the forms upstream prints. + fn publish_local_writes(&mut self, source: &Expr, rewritten: Expr) -> Expr { + let range = source.range(); + match source.data() { + Expression::Assignment(assignment) => { + let written = self.live_exports_of(&assignment.left); + if let AssignmentTarget::Identifier(_) = assignment.left.data() { + return match written.into_iter().next() { + Some((_, names)) => self.cjs_assign_exports(&names, rewritten, range), + None => rewritten, + }; + } + if written.is_empty() { + return rewritten; + } + let temp = self.expression_temp(); + let mut steps = vec![self.store_temp(&temp, rewritten, range)]; + for (local, names) in written { + let value = self.ident_expr(&local); + steps.push(self.cjs_assign_exports(&names, value, range)); + } + let result = self.node(range, Expression::Identifier(temp)); + steps.push(result); + self.sequence(steps, range) + } + Expression::Update(update) => { + let Some((local, names)) = self.live_exports_of(&update.argument).pop() else { + return rewritten; + }; + if update.prefix { + return self.cjs_assign_exports(&names, rewritten, range); + } + let current = self.ident_expr(&local); + if self.statement_expression == Some(source.id()) { + let value = self.sequence(vec![rewritten, current], range); + return self.cjs_assign_exports(&names, value, range); + } + let temp = self.expression_temp(); + let store = self.store_temp(&temp, rewritten, range); + let value = self.sequence(vec![store, current], range); + let publish = self.cjs_assign_exports(&names, value, range); + let result = self.node(range, Expression::Identifier(temp)); + self.sequence(vec![publish, result], range) + } + _ => rewritten, } - prelude + } + + /// An import or re-export's lines, as a statement the printer fills in. + fn cjs_statement_lines(&mut self, statement: &Stmt) -> Vec { + let lines = self + .cjs + .as_mut() + .and_then(|plan| plan.statement_lines.remove(&statement.id())); + let Some(lines) = lines else { + return Vec::new(); + }; + let marker = self.node(statement.range(), Statement::Empty); + self.inline_lines.insert(marker.id(), lines); + vec![marker] } /// Raw source text of a token, with escapes kept verbatim. @@ -1272,6 +1671,7 @@ impl<'a> Rewriter<'a> { fn rewrite_root_statements(&mut self, statements: &[Stmt], range: TextRange) -> Vec { self.expression_temp_scopes.push(Vec::new()); let mut rewritten = self.rewrite_statements(statements); + rewritten.extend(self.export_assignment.take()); let temps = self.expression_temp_scopes.pop().unwrap_or_default(); if let Some(declaration) = self.temp_declaration(temps, range) { rewritten.insert(0, declaration); @@ -1325,10 +1725,14 @@ impl<'a> Rewriter<'a> { } fn rewrite_statements(&mut self, statements: &[Stmt]) -> Vec { + self.declared_names.push(BTreeSet::new()); let mut out = Vec::new(); for statement in statements { + self.record_declaration(statement); out.extend(self.rewrite_statement(statement)); + out.extend(self.cjs_publish_declared(statement)); } + self.declared_names.pop(); out } @@ -1362,6 +1766,12 @@ impl<'a> Rewriter<'a> { vec![statement.clone()] } ExportDefaultValue::Expression(value) => { + // `export default T` of a type has no runtime form. + if let Expression::Identifier(identifier) = value.data() + && self.exports_only_a_type(self.token_text(identifier.data().token())) + { + return Vec::new(); + } if self.cjs.is_some() { let value = self.rewrite_expr(value); return vec![self.cjs_export_default_assignment(statement.range(), value)]; @@ -1384,25 +1794,40 @@ impl<'a> Rewriter<'a> { source, .. })) => { - if self.cjs.is_some() && source.is_none() && !*type_only { + if self.cjs.is_none() { + if *type_only || self.options.verbatim_module_syntax { + return vec![statement.clone()]; + } + return self.rewrite_esm_export_specifiers(statement, specifiers, source); + } + if source.is_none() && !*type_only { return self.cjs_export_specifiers(specifiers); } - vec![statement.clone()] + // A type-only re-export has no lines, so it is gone. + self.cjs_statement_lines(statement) + } + Statement::Export(ExportDeclaration::All(_)) if self.cjs.is_some() => { + self.cjs_statement_lines(statement) + } + Statement::Export(ExportDeclaration::Assignment(value)) if self.cjs.is_some() => { + // Upstream runs it after the module body, whatever its position. + let erased = matches!(value.data(), Expression::Identifier(identifier) + if self.exports_only_a_type(self.token_text(identifier.data().token()))); + if !erased { + self.export_assignment = + Some(self.cjs_export_assignment_statement(statement.range(), value)); + } + Vec::new() } Statement::Import(import) => { if self.cjs.is_none() { - return vec![statement.clone()]; + return self.rewrite_esm_import(statement, import); } if import.type_only { // Type-only imports carry no runtime form. return Vec::new(); } - match &import.clause { - // A side-effect import stays at its position as a bare - // require; clause imports move into the prelude. - None => vec![self.cjs_require_statement(&import.source)], - Some(_) => Vec::new(), - } + self.cjs_statement_lines(statement) } Statement::Block(block) => { let statements = self.rewrite_statements(&block.data().statements); @@ -1420,6 +1845,7 @@ impl<'a> Rewriter<'a> { )] } Statement::Expression(expr) => { + self.statement_expression = Some(expr.expression.id()); let mut candidate = &*expr.expression; while let Expression::Parenthesized(value) = candidate.data() { candidate = value; @@ -1527,13 +1953,21 @@ impl<'a> Rewriter<'a> { { return self.lower_sync_for_of(statement, for_of); } + let binding = match &for_of.binding { + ForBinding::Target(target) => { + ForBinding::Target(self.rewrite_assignment_target(target)) + } + ForBinding::Variable(declaration) => { + ForBinding::Variable(self.rewrite_for_declaration(declaration)) + } + }; let iterable = self.rewrite_expr(&for_of.iterable); let body = self.rewrite_single_statement(&for_of.body); vec![self.node( statement.range(), Statement::ForOf(ForOfStatement { mode: for_of.mode, - binding: for_of.binding.clone(), + binding, iterable: Box::new(iterable), body: Box::new(body), }), @@ -1544,13 +1978,15 @@ impl<'a> Rewriter<'a> { // const/let binding needs var conversion there. let binding = match &for_in.binding { ForBinding::Variable(declaration) => { - let mut declaration = declaration.clone(); + let mut declaration = self.rewrite_for_declaration(declaration); if self.options.target <= ScriptTarget::Es5 { declaration.kind = VariableKind::Var; } ForBinding::Variable(declaration) } - target @ ForBinding::Target(_) => target.clone(), + ForBinding::Target(target) => { + ForBinding::Target(self.rewrite_assignment_target(target)) + } }; let object = self.rewrite_expr(&for_in.object); let body = self.rewrite_single_statement(&for_in.body); @@ -1572,7 +2008,7 @@ impl<'a> Rewriter<'a> { ForInitializer::Expression(Box::new(self.rewrite_expr(expression))) } ForInitializer::Variable(declaration) => { - ForInitializer::Variable(declaration.clone()) + ForInitializer::Variable(self.rewrite_for_declaration(declaration)) } }); let test = value @@ -1597,14 +2033,13 @@ impl<'a> Rewriter<'a> { Statement::Try(value) => { let block = self.rewrite_block(&value.block); let handler = value.handler.as_ref().map(|handler| { + let binding = handler + .data() + .binding + .as_ref() + .map(|binding| self.rewrite_pattern(binding)); let body = self.rewrite_block(&handler.data().body); - self.node( - handler.range(), - CatchClause { - binding: handler.data().binding.clone(), - body, - }, - ) + self.node(handler.range(), CatchClause { binding, body }) }); let finalizer = value .finalizer @@ -1654,12 +2089,22 @@ impl<'a> Rewriter<'a> { vec![statement.clone()] } } - Statement::ImportEquals(_) - | Statement::Export(_) + Statement::Namespace(namespace) => self.lower_namespace( + statement, + namespace, + ContainerBinding::Local, + statement.range(), + ), + Statement::Enum(declaration) => self.lower_enum( + statement, + declaration, + ContainerBinding::Local, + statement.range(), + ), + Statement::ImportEquals(import) => self.lower_alias(statement, import), + Statement::Export(_) | Statement::Interface(_) | Statement::TypeAlias(_) - | Statement::Enum(_) - | Statement::Namespace(_) | Statement::Empty | Statement::Break(_) | Statement::Continue(_) @@ -1971,14 +2416,74 @@ impl<'a> Rewriter<'a> { }; Box::new(rewritten) }); + let binding = self.rewrite_pattern(&declarator.data().binding); self.node( declarator.range(), VariableDeclarator { + binding, initializer, ..declarator.data().clone() }, ) } + + fn rewrite_for_declaration( + &mut self, + declaration: &VariableDeclaration, + ) -> VariableDeclaration { + let declarations = declaration + .declarations + .iter() + .map(|declarator| self.rewrite_declarator_initializer(declarator)) + .collect(); + VariableDeclaration { + declarations, + ..declaration.clone() + } + } + + /// A binding pattern with its defaults and computed keys rewritten. + fn rewrite_pattern(&mut self, pattern: &Pattern) -> Pattern { + let rewritten = match pattern.data() { + BindingPattern::Object(object) => BindingPattern::Object(ObjectBindingPattern { + properties: object + .properties + .iter() + .map(|property| ObjectBindingProperty { + name: self.rewrite_property_name(&property.name), + binding: self.rewrite_pattern(&property.binding), + initializer: property + .initializer + .as_deref() + .map(|value| Box::new(self.rewrite_expr(value))), + }) + .collect(), + }), + BindingPattern::Array(array) => BindingPattern::Array(ArrayBindingPattern { + elements: array + .elements + .iter() + .map(|element| match element { + ArrayBindingElement::Binding(binding) => { + ArrayBindingElement::Binding(self.rewrite_pattern(binding)) + } + other => other.clone(), + }) + .collect(), + }), + BindingPattern::Rest(rest) => BindingPattern::Rest(RestBindingPattern { + argument: Box::new(self.rewrite_pattern(&rest.argument)), + }), + BindingPattern::Assignment(assignment) => { + BindingPattern::Assignment(AssignmentBindingPattern { + left: Box::new(self.rewrite_pattern(&assignment.left)), + right: Box::new(self.rewrite_expr(&assignment.right)), + }) + } + BindingPattern::Identifier(_) | BindingPattern::Missing(_) => return pattern.clone(), + }; + self.node(pattern.range(), rewritten) + } /// Lowers a declaration whose binding carries suspensions (yield in /// a default or computed key). Every bound name hoists first; reads /// become temp var statements; a default becomes an if/else whose @@ -3044,7 +3549,8 @@ impl<'a> Rewriter<'a> { ClassMember::Constructor(_) | ClassMember::StaticBlock(_) | ClassMember::IndexSignature(_) - | ClassMember::Missing(_) => None, + | ClassMember::Missing(_) + | ClassMember::Semicolon => None, }; name.is_some_and(|name| matches!(name, PropertyName::Computed(_))) }); @@ -3067,6 +3573,7 @@ impl<'a> Rewriter<'a> { match member.data() { ClassMember::Constructor(ctor) => { let mut ctor = ctor.clone(); + ctor.parameters = self.rewrite_parameters(&ctor.parameters); let statements = self.rewrite_statements(&ctor.body.data().statements); ctor.body = self.node(ctor.body.range(), Block { statements }); constructor = Some(self.node(member.range(), ClassMember::Constructor(ctor))); @@ -3159,13 +3666,18 @@ impl<'a> Rewriter<'a> { self.node(block.range(), Block { statements }), )); } - ClassMember::IndexSignature(_) | ClassMember::Missing(_) => { + ClassMember::IndexSignature(_) + | ClassMember::Missing(_) + | ClassMember::Semicolon => { members.push(member.clone()); } } } debug_assert!( - !has_computed || class.extends.is_none() || !prelude.is_empty(), + matches!(mode, FieldMode::Native) + || !has_computed + || class.extends.is_none() + || !prelude.is_empty(), "heritage must be captured before computed keys" ); if matches!(mode, FieldMode::Native) { @@ -3291,10 +3803,12 @@ impl<'a> Rewriter<'a> { } else { self.syn_node(ctor.body.range(), Block { statements }) }; + let parameters = self.rewrite_parameters(&ctor.parameters); Some(self.syn_node( existing.range(), ClassMember::Constructor(ConstructorDeclaration { body, + parameters, ..ctor.clone() }), )) @@ -4670,9 +5184,15 @@ impl<'a> Rewriter<'a> { debug_assert_eq!(popped, Some(depth)); Box::new(rewritten) }); + let depth = self.expression_temp_scopes.len(); + self.parameter_initializer_depths.push(depth); + let binding = self.rewrite_pattern(¶meter.data().binding); + let popped = self.parameter_initializer_depths.pop(); + debug_assert_eq!(popped, Some(depth)); self.node( parameter.range(), Parameter { + binding, initializer, ..parameter.data().clone() }, @@ -6687,6 +7207,18 @@ impl<'a> Rewriter<'a> { .collect() } fn rewrite_expr(&mut self, expression: &Expr) -> Expr { + let rewritten = self.rewrite_expression(expression); + if self + .cjs + .as_ref() + .is_some_and(|plan| !plan.live_locals.is_empty()) + { + return self.publish_local_writes(expression, rewritten); + } + rewritten + } + + fn rewrite_expression(&mut self, expression: &Expr) -> Expr { if self.replace_await && let Expression::Await(awaited) = expression.data() { @@ -6700,6 +7232,8 @@ impl<'a> Rewriter<'a> { ); } match expression.data() { + // The printer inlines this value by the node's identity. + Expression::Member(_) if self.is_const_enum_use(expression.id()) => expression.clone(), Expression::Class(class) => { let previous = self.replace_await; self.replace_await = false; @@ -6956,17 +7490,7 @@ impl<'a> Rewriter<'a> { return self.lower_logical_assignment(expression, assignment); } let right = self.rewrite_expr(&assignment.right); - let left = if self.replace_await - && let AssignmentTarget::Member(member) = assignment.left.data() - && target_contains_await(&member.object, &member.property) - { - // Convert awaits inside the target too, so a - // suspending target becomes a visible machine refusal - // instead of a live `await` in ES5 output. - self.rewrite_member_target(member, assignment.left.range()) - } else { - assignment.left.clone() - }; + let left = self.rewrite_assignment_target(&assignment.left); self.node( expression.range(), Expression::Assignment(AssignmentExpression { @@ -6976,17 +7500,216 @@ impl<'a> Rewriter<'a> { }), ) } - Expression::Identifier(_) => { + Expression::Identifier(ident) => { + if let Some(parameter) = self.namespace_member_parameter(expression.id()) + && let Some(name) = self.identifier_name(ident) + { + return self.namespace_member(¶meter, &name, expression.range()); + } + if self.cjs_reads_exported_variable(expression.id()) + && let Some(name) = self.identifier_name(ident) + { + return self.namespace_member("exports", &name, expression.range()); + } if let Some(replacement) = self.cjs_identifier_replacement(expression) { replacement } else { expression.clone() } } + Expression::Template(template) => { + let template = self.rewrite_template(template); + self.node(expression.range(), Expression::Template(template)) + } + Expression::TaggedTemplate(tagged) => { + let tag = self.rewrite_expr(&tagged.tag); + let template = self.rewrite_template(&tagged.template); + self.node( + expression.range(), + Expression::TaggedTemplate(TaggedTemplateExpression { + tag: Box::new(tag), + template, + }), + ) + } + Expression::Array(array) => { + let elements = array + .elements + .iter() + .map(|element| match element { + ArrayElement::Expression(value) => { + ArrayElement::Expression(Box::new(self.rewrite_expr(value))) + } + ArrayElement::Spread(spread) => ArrayElement::Spread(SpreadElement { + argument: Box::new(self.rewrite_expr(&spread.argument)), + }), + other => other.clone(), + }) + .collect(); + self.node( + expression.range(), + Expression::Array(ArrayLiteral { elements }), + ) + } + Expression::Object(object) => { + let members = object + .members + .iter() + .map(|member| self.rewrite_object_member(member)) + .collect(); + self.node( + expression.range(), + Expression::Object(ObjectLiteral { members }), + ) + } + Expression::Yield(value) => { + let argument = value + .argument + .as_deref() + .map(|argument| Box::new(self.rewrite_expr(argument))); + self.node( + expression.range(), + Expression::Yield(YieldExpression { + delegate: value.delegate, + argument, + }), + ) + } + Expression::Update(value) => { + let argument = self.rewrite_assignment_target(&value.argument); + self.node( + expression.range(), + Expression::Update(UpdateExpression { + operator: value.operator, + argument: Box::new(argument), + prefix: value.prefix, + }), + ) + } _ => expression.clone(), } } + fn rewrite_template(&mut self, template: &TemplateLiteral) -> TemplateLiteral { + TemplateLiteral { + elements: template.elements.clone(), + expressions: template + .expressions + .iter() + .map(|value| self.rewrite_expr(value)) + .collect(), + } + } + + fn rewrite_object_member(&mut self, member: &ObjectMemberNode) -> ObjectMemberNode { + let rewritten = match member.data() { + ObjectMember::Property(property) => { + let name = self.rewrite_property_name(&property.name); + let value = self.rewrite_expr(&property.value); + // A shorthand that no longer reads its own name has to spell + // the value out. + let shorthand = + property.shorthand && matches!(value.data(), Expression::Identifier(_)); + ObjectMember::Property(ObjectProperty { + name, + value: Box::new(value), + modifier: property.modifier, + shorthand, + }) + } + ObjectMember::Method(method) + if (method.function.is_generator + && Self::needs(LanguageFeature::Generators, self.options)) + || (method.function.is_async + && Self::needs(LanguageFeature::AsyncFunctions, self.options)) => + { + // A method owns its suspensions, so a target without them + // keeps the method as written rather than lowering it here. + return member.clone(); + } + ObjectMember::Method(method) => { + let name = self.rewrite_property_name(&method.name); + let previous = self.replace_await; + self.replace_await = false; + let function = self.rewrite_function_like(&method.function, member.range()); + self.replace_await = previous; + ObjectMember::Method(ObjectMethod { + name, + modifier: method.modifier, + function, + }) + } + ObjectMember::Spread(spread) => ObjectMember::Spread(SpreadElement { + argument: Box::new(self.rewrite_expr(&spread.argument)), + }), + ObjectMember::Missing(_) => return member.clone(), + }; + self.node(member.range(), rewritten) + } + + /// An assignment target with its nested expressions rewritten. A member + /// target is how a suspending target reaches the machine as a visible + /// refusal instead of a live `await` in ES5 output. + fn rewrite_assignment_target(&mut self, target: &AssignmentTargetNode) -> AssignmentTargetNode { + let rewritten = match target.data() { + AssignmentTarget::Identifier(ident) => { + let parameter = self + .namespace_member_parameter(target.id()) + .or_else(|| self.namespace_member_parameter(ident.id())) + .or_else(|| { + self.cjs_reads_exported_variable(ident.id()) + .then(|| "exports".to_owned()) + }); + let (Some(parameter), Some(name)) = (parameter, self.identifier_name(ident)) else { + return target.clone(); + }; + let object = self.ident_expr(¶meter); + let property = MemberProperty::Named(self.ident(&name)); + AssignmentTarget::Member(AssignmentMemberTarget { + object: Box::new(object), + property, + }) + } + AssignmentTarget::Member(member) => { + return self.rewrite_member_target(member, target.range()); + } + AssignmentTarget::Object(pattern) => { + let properties = pattern + .properties + .iter() + .map(|property| AssignmentObjectProperty { + name: self.rewrite_property_name(&property.name), + target: self.rewrite_assignment_target(&property.target), + initializer: property + .initializer + .as_deref() + .map(|value| Box::new(self.rewrite_expr(value))), + rest: property.rest, + }) + .collect(); + AssignmentTarget::Object(AssignmentObjectPattern { properties }) + } + AssignmentTarget::Array(pattern) => { + let elements = pattern + .elements + .iter() + .map(|element| match element { + AssignmentArrayElement::Target(target) => { + AssignmentArrayElement::Target(self.rewrite_assignment_target(target)) + } + AssignmentArrayElement::Rest(target) => { + AssignmentArrayElement::Rest(self.rewrite_assignment_target(target)) + } + other => other.clone(), + }) + .collect(); + AssignmentTarget::Array(AssignmentArrayPattern { elements }) + } + _ => return target.clone(), + }; + self.node(target.range(), rewritten) + } + fn rewrite_member_target( &mut self, member: &AssignmentMemberTarget, @@ -8790,539 +9513,868 @@ fn contains_yield(expression: &Expr) -> bool { } } +/// Every identifier the file spells, which a generated name has to avoid. +fn identifier_names(file: &SourceFile) -> BTreeSet { + file.tokens() + .iter() + .filter(|token| token.kind() == TokenKind::Identifier) + .filter_map(|token| { + let start = file + .source_text() + .utf16_to_byte(token.range().start()) + .ok()?; + let end = file.source_text().utf16_to_byte(token.range().end()).ok()?; + file.source_text() + .as_str() + .get(start..end) + .map(str::to_owned) + }) + .collect() +} + fn identifier_text(file: &SourceFile, ident: &IdentifierNode) -> String { raw_token_text(file, ident.data().token()) } -/// Declared value names of a statement, for export assignment generation. -fn declared_binding_names(file: &SourceFile, statement: &Stmt) -> Vec { - match statement.data() { - Statement::Variable(declaration) => declaration - .declarations - .iter() - .filter_map(|declarator| match declarator.data().binding.data() { - BindingPattern::Identifier(ident) => Some(identifier_text(file, ident)), - _ => None, - }) - .collect(), - Statement::Enum(declaration) => vec![raw_token_text(file, declaration.name.data().token())], - _ => Vec::new(), +/// Plain ASCII only: `exports["é"]` means the same as `exports.é`, while +/// telling `x²` from a real identifier would take the Unicode tables. +fn is_identifier_name(name: &str) -> bool { + let mut characters = name.chars(); + characters + .next() + .is_some_and(|first| first.is_ascii_alphabetic() || first == '_' || first == '$') + && characters.all(|rest| rest.is_ascii_alphanumeric() || rest == '_' || rest == '$') +} + +/// A double-quoted JavaScript string for already decoded text. +fn quote_string(text: &str) -> String { + let mut quoted = String::with_capacity(text.len() + 2); + quoted.push('"'); + for character in text.chars() { + match character { + '"' => quoted.push_str("\\\""), + '\\' => quoted.push_str("\\\\"), + '\n' => quoted.push_str("\\n"), + '\r' => quoted.push_str("\\r"), + '\u{2028}' => quoted.push_str("\\u2028"), + '\u{2029}' => quoted.push_str("\\u2029"), + character => quoted.push(character), + } + } + quoted.push('"'); + quoted +} + +/// `exports.name`, or `exports["a b"]` for a name that is no identifier. +fn exports_member(name: &str) -> String { + if is_identifier_name(name) { + format!("exports.{name}") + } else { + format!("exports[{}]", quote_string(name)) } } -fn property_key_text(rewriter: &mut Rewriter, property: &ObjectBindingProperty) -> Option { - match &property.name { - PropertyName::Identifier(ident) => { - Some(rewriter.token_text(ident.data().token()).to_owned()) - } - PropertyName::String(literal) => { - let raw = rewriter.token_text(literal.data().token()); - Some(raw.trim_matches(['"', '\'']).to_owned()) +/// A getter on `exports` that reads through to a module, so it stays live. +fn live_export(exported: &str, module: &str, imported: &str) -> String { + let read = if is_identifier_name(imported) { + format!("{module}.{imported}") + } else { + format!("{module}[{}]", quote_string(imported)) + }; + format!( + "Object.defineProperty(exports, {}, {{ enumerable: true, \ + get: function () {{ return {read}; }} }});", + quote_string(exported) + ) +} + +fn module_export_text(file: &SourceFile, name: &ModuleExportName) -> String { + match name { + ModuleExportName::Identifier(ident) => identifier_text(file, ident), + ModuleExportName::String(literal) => { + let raw = raw_token_text(file, literal.data().token()); + crate::literal::string_value(&raw) + .map_or_else(|| raw.clone(), |text| text.to_utf8_lossy()) } - _ => None, + ModuleExportName::Missing(_) => String::new(), } } -/// Source range of a property name, taken from its inner node. -fn property_name_range(name: &PropertyName) -> TextRange { - match name { - PropertyName::Identifier(ident) => ident.range(), - PropertyName::String(literal) => literal.range(), - PropertyName::Number(number) => number.range(), - PropertyName::Computed(expression) => expression.range(), - PropertyName::Private(private) => private.range(), - PropertyName::Missing(_) => { - TextRange::new(Utf16Pos::ZERO, Utf16Pos::ZERO).expect("zero-width range is ordered") +/// What each local is exported as through `export { local as name }`. +fn exported_locals(file: &SourceFile) -> BTreeMap> { + let mut locals: BTreeMap> = BTreeMap::new(); + for statement in file.statements() { + let Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Specifiers { + type_only: false, + specifiers, + source: None, + .. + })) = statement.data() + else { + continue; + }; + for specifier in specifiers { + let specifier = specifier.data(); + if let (ExportSpecifierMode::Value, ModuleExportName::Identifier(local)) = + (&specifier.mode, &specifier.local) + { + locals + .entry(identifier_text(file, local)) + .or_default() + .push(module_export_text(file, &specifier.exported)); + } } } + locals } -fn scan_statements( +/// Plans `export { local }` of the file's own declarations: each publishes +/// right after its statement, and writes to it stay in step. +fn plan_cjs_declared_exports( file: &SourceFile, - statements: &[Stmt], - options: &TransformOptions, - features: &mut BTreeSet, - diagnostics: &mut Vec, - in_async: bool, + model: &SemanticModel, + exported_locals: &BTreeMap>, + plan: &mut CjsPlan, ) { - let _ = in_async; - for statement in statements { - match statement.data() { - Statement::Class(class) => { - features.insert(LanguageFeature::Classes); - scan_class(file, class, options, features, diagnostics); + let scope = model.scope(model.module_scope()); + let imports = imported_bindings(file); + for (local, names) in exported_locals { + if !imports.contains_key(local) + && let Some(symbol) = scope.value(local) + && !plan.exported_variables.contains(&symbol) + { + plan.live_locals.insert(symbol, names.clone()); + } + } + for statement in file.statements() { + // An ambient declaration has nothing to publish; its `void 0` stands. + if let Statement::Declare(inner) = statement.data() { + let mut locals = declared_binding_names(file, inner); + if let Statement::Class(class) = inner.data() { + locals.extend(class.name.iter().map(|name| identifier_text(file, name))); } - Statement::Function(function) => { - scan_function( - file, - &function.function, - statement.range(), - options, - features, - diagnostics, - ); + for local in locals { + let names = exported_locals.get(&local).into_iter().flatten(); + plan.published_locals.extend(names.cloned()); } - Statement::Variable(declaration) => { - if matches!( - declaration.kind, - VariableKind::Using | VariableKind::AwaitUsing - ) { - features.insert(LanguageFeature::Using); - } - for declarator in &declaration.declarations { - scan_pattern(declarator.data().binding.data(), features); - if let Some(initializer) = declarator.data().initializer.as_deref() { - scan_expression(initializer, features); - } - } + continue; + } + let declared = match statement.data() { + Statement::Class(class) => class + .name + .iter() + .map(|name| identifier_text(file, name)) + .collect(), + _ => declared_binding_names(file, statement), + }; + let pairs: Vec<(String, String)> = declared + .iter() + .flat_map(|local| { + let names = exported_locals.get(local).into_iter().flatten(); + names.map(|name| (name.clone(), local.clone())) + }) + .collect(); + if !pairs.is_empty() { + plan.published_locals + .extend(pairs.iter().map(|(name, _)| name.clone())); + plan.declared_exports.insert(statement.id(), pairs); + } + } +} + +/// The identifiers an assignment target writes, through any destructuring. +fn collect_target_identifiers<'t>( + target: &'t AssignmentTargetNode, + identifiers: &mut Vec<&'t IdentifierNode>, +) { + match target.data() { + AssignmentTarget::Identifier(identifier) => identifiers.push(identifier), + AssignmentTarget::Object(object) => { + for property in &object.properties { + collect_target_identifiers(&property.target, identifiers); } - Statement::Expression(expr) => scan_expression(&expr.expression, features), - Statement::If(value) => { - scan_expression(&value.test, features); - scan_nested_statements(std::slice::from_ref(value.consequent.as_ref()), features); - if let Some(alternate) = value.alternate.as_deref() { - scan_nested_statements(std::slice::from_ref(alternate), features); + } + AssignmentTarget::Array(array) => { + for element in &array.elements { + if let AssignmentArrayElement::Target(inner) | AssignmentArrayElement::Rest(inner) = + element + { + collect_target_identifiers(inner, identifiers); } } - Statement::While(value) => { - scan_expression(&value.test, features); - scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); - } - Statement::DoWhile(value) => { - scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); - scan_expression(&value.test, features); - } - Statement::With(value) => { - scan_expression(&value.object, features); - scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); - } - Statement::Labeled(value) => { - scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); - } - Statement::Return(ret) => { - if let Some(argument) = ret.argument.as_deref() { - scan_expression(argument, features); - } - } - Statement::Throw(value) => scan_expression(&value.argument, features), - Statement::For(value) => { - if let Some(ForInitializer::Expression(initializer)) = &value.initializer { - scan_expression(initializer, features); - } - if let Some(test) = value.test.as_deref() { - scan_expression(test, features); - } - if let Some(update) = value.update.as_deref() { - scan_expression(update, features); - } - scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); - } - Statement::ForIn(for_in) => { - scan_expression(&for_in.object, features); - scan_nested_statements(std::slice::from_ref(for_in.body.as_ref()), features); - } - Statement::ForOf(for_of) => { - if for_of.mode == ForOfMode::Async { - features.insert(LanguageFeature::AsyncIteration); - } - scan_expression(&for_of.iterable, features); - scan_nested_statements(std::slice::from_ref(for_of.body.as_ref()), features); - } - Statement::Switch(value) => { - scan_expression(&value.discriminant, features); - for case in &value.cases { - if let Some(test) = case.data().test.as_deref() { - scan_expression(test, features); - } - scan_nested_statements(&case.data().consequent, features); - } - } - Statement::Try(value) => { - scan_nested_statements(&value.block.data().statements, features); - if let Some(handler) = &value.handler { - scan_nested_statements(&handler.data().body.data().statements, features); - } - if let Some(finalizer) = &value.finalizer { - scan_nested_statements(&finalizer.data().statements, features); - } - } - Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration( - inner, - ))) => scan_statements( - file, - std::slice::from_ref(inner), - options, - features, - diagnostics, - in_async, - ), - Statement::Export(ExportDeclaration::Default(default)) => { - if let ExportDefaultValue::Class(class) = &default.value { - features.insert(LanguageFeature::Classes); - scan_class(file, class, options, features, diagnostics); - } else if let ExportDefaultValue::Expression(value) = &default.value { - scan_expression(value, features); - } - } - Statement::Block(block) => { - scan_statements( - file, - &block.data().statements, - options, - features, - diagnostics, - in_async, - ); + } + AssignmentTarget::Invalid(operand) => { + if let Expression::Assignment(assignment) = operand.data() { + collect_target_identifiers(&assignment.left, identifiers); } - _ => {} } + _ => {} } } -fn scan_class( +/// Whether `export { name } from` still has something to publish. +fn reexport_survives( file: &SourceFile, - class: &ClassDeclaration, - options: &TransformOptions, - features: &mut BTreeSet, - diagnostics: &mut Vec, + specifier: &ExportSpecifier, + module: &str, + erased: &BTreeSet<(String, String)>, +) -> bool { + matches!(specifier.mode, ExportSpecifierMode::Value) + && !erased.contains(&( + module.to_owned(), + module_export_text(file, &specifier.local), + )) +} + +/// Plans a re-export's lines: a star export copies the module's names, and a named one reads through to the +/// module, so it stays a live binding. +fn plan_cjs_reexport( + file: &SourceFile, + export: &ExportDeclaration, + names: &mut RequireNames, + plan: &mut CjsPlan, + erased: &BTreeSet<(String, String)>, ) { - if let Some(heritage) = &class.extends { - scan_expression(&heritage.expression, features); - } - for member in &class.members { - match member.data() { - ClassMember::Property(property) => { - if !property.modifiers.is_declare && !property.modifiers.is_abstract { - features.insert(LanguageFeature::ClassFields); - } - scan_computed_name(&property.name, features); - if let Some(initializer) = property.initializer.as_deref() { - scan_expression(initializer, features); - } - if matches!(property.name, PropertyName::Private(_)) - && !options.target.supports(LanguageFeature::ClassFields) - { - diagnostics.push(Diagnostic::error( - codes::PRIVATE_FIELD_REQUIRES_ES2022, - file.source_id(), - member.range(), - "private class fields require ScriptTarget::Es2022 or later", + let quoted = |source: &StringLiteralNode| { + let raw = raw_token_text(file, source.data().token()); + quote_module_specifier(raw.trim_matches(['"', '\''])) + }; + match export { + ExportDeclaration::All(all) if !all.type_only => { + let module = quoted(&all.source); + match &all.exported { + Some(name) => { + let name = module_export_text(file, name); + plan.helpers.insert(HelperKind::ImportStar); + plan.requires.push(format!( + "{} = __importStar(require({module}));", + exports_member(&name) )); + plan.void_0.push(name); } - } - ClassMember::Method(method) => { - scan_function( - file, - &method.function, - member.range(), - options, - features, - diagnostics, - ); - } - ClassMember::AutoAccessor(accessor) => { - features.insert(LanguageFeature::ClassFields); - scan_computed_name(&accessor.name, features); - if let Some(initializer) = accessor.initializer.as_deref() { - scan_expression(initializer, features); - } - if !options.target.supports(LanguageFeature::ClassFields) { - diagnostics.push(Diagnostic::error( - codes::AUTO_ACCESSOR_REQUIRES_ES2022, - file.source_id(), - member.range(), - "auto-accessors require ScriptTarget::Es2022 or later", - )); + None => { + plan.helpers.insert(HelperKind::ExportStar); + plan.requires + .push(format!("__exportStar(require({module}), exports);")); } } - ClassMember::StaticBlock(block) => { - features.insert(LanguageFeature::ClassFields); - scan_statements( - file, - &block.data().statements, - options, - features, - diagnostics, - false, - ); + } + ExportDeclaration::Named(ExportNamedDeclaration::Specifiers { + type_only: false, + specifiers, + source: Some(source), + .. + }) => { + let module = module_specifier(file, source); + let kept: Vec<_> = specifiers + .iter() + .map(|specifier| specifier.data()) + .filter(|specifier| reexport_survives(file, specifier, &module, erased)) + .collect(); + if kept.is_empty() { + return; + } + let local = names.allocate(&module); + plan.requires + .push(format!("var {local} = require({});", quoted(source))); + for specifier in kept { + let imported = module_export_text(file, &specifier.local); + let exported = module_export_text(file, &specifier.exported); + plan.requires + .push(live_export(&exported, &local, &imported)); + plan.void_0.push(exported); } - _ => {} } + _ => {} } } -fn scan_function( +fn is_type_only_export(model: &SemanticModel, name: &str) -> bool { + let scope = model.scope(model.module_scope()); + scope.value(name).is_none() && scope.type_binding(name).is_some() +} + +/// The names that keep an import in this file. JavaScript keeps every import, +/// as does `verbatimModuleSyntax`, so those files have no answer. +fn import_reads( file: &SourceFile, - function: &FunctionLike, - _range: TextRange, options: &TransformOptions, - features: &mut BTreeSet, - diagnostics: &mut Vec, -) { - if function.is_async { - features.insert(LanguageFeature::AsyncFunctions); - } - if function.is_generator { - features.insert(LanguageFeature::Generators); + model: Option<&SemanticModel>, +) -> Option> { + if options.verbatim_module_syntax + || options.keeps_imports + || file.source_text().as_str().contains("@jsx") + || matches!( + file.script_kind(), + crate::source::ScriptKind::JavaScript | crate::source::ScriptKind::JavaScriptReact + ) + { + return None; } - for parameter in &function.parameters { - scan_pattern(parameter.data().binding.data(), features); - if let Some(initializer) = parameter.data().initializer.as_deref() { - scan_expression(initializer, features); + // tsc rejects reading a type as a value, so a read proves the name is one, + // whatever another file's exports appear to say. Only an export of the name + // is legal for a type, so only exports ask. + let mut reads = crate::lower::value_read_names(file, model.map(SemanticModel::enum_facts)); + let erased = erased_import_locals(file, options); + let exported = exported_locals(file) + .into_keys() + .chain(default_export_name(file)); + reads.extend(exported.filter(|name| !erased.contains(name))); + // The classic JSX factories read a name the source never spells. + if matches!( + options.jsx, + Some(JsxEmit::React | JsxEmit::Preserve | JsxEmit::ReactNative) + ) { + for factory in [&options.jsx_factory, &options.jsx_fragment_factory] { + let factory = factory.as_deref().unwrap_or("React"); + reads.extend(factory.split('.').next().map(str::to_owned)); } } - match function.body.as_ref() { - Some(FunctionBody::Block(body)) => scan_statements( - file, - &body.data().statements, - options, - features, - diagnostics, - function.is_async, - ), - Some(FunctionBody::Expression(expression)) => scan_expression(expression, features), - Some(FunctionBody::Missing(_)) | None => {} - } + Some(reads) } -fn scan_pattern(pattern: &BindingPattern, features: &mut BTreeSet) { - match pattern { - BindingPattern::Object(_) | BindingPattern::Array(_) => { - features.insert(LanguageFeature::Destructuring); - } - BindingPattern::Rest(_) => { - features.insert(LanguageFeature::ObjectRestSpread); - features.insert(LanguageFeature::Destructuring); +/// The name in `export default x` or `export = x`. +fn default_export_name(file: &SourceFile) -> Vec { + file.statements() + .iter() + .filter_map(|statement| match statement.data() { + Statement::Export(ExportDeclaration::Default(default)) => match &default.value { + ExportDefaultValue::Expression(value) => Some(value), + _ => None, + }, + Statement::Export(ExportDeclaration::Assignment(value)) => Some(value), + _ => None, + }) + .filter_map(|value| match value.data() { + Expression::Identifier(identifier) => Some(identifier_text(file, identifier)), + _ => None, + }) + .collect() +} + +/// An import clause without the bindings nothing reads, or `None` if none remain. +fn kept_import_clause( + file: &SourceFile, + clause: &ImportClause, + reads: &BTreeSet, +) -> Option { + let read = |name: &IdentifierNode| reads.contains(&identifier_text(file, name)); + let default = clause.default.clone().filter(read); + let binding = match &clause.binding { + Some(ImportBinding::Namespace(name)) if read(name) => clause.binding.clone(), + Some(ImportBinding::Named(specifiers)) => { + let kept: Vec<_> = specifiers + .iter() + .filter(|specifier| { + let specifier = specifier.data(); + matches!(specifier.mode, ImportSpecifierMode::Value) && read(&specifier.local) + }) + .cloned() + .collect(); + (!kept.is_empty()).then_some(ImportBinding::Named(kept)) } - _ => {} - } + _ => None, + }; + (default.is_some() || binding.is_some()).then_some(ImportClause { default, binding }) } -/// Recursively scans one expression for exponentiation syntax. The deep walk -/// intentionally reports only [`LanguageFeature::Exponentiation`]: every other -/// feature keeps the statement-level analysis surface this module has always -/// exposed, and only exponentiation is lowered in every expression position -/// the rewriter visits. -fn scan_expression(expression: &Expr, features: &mut BTreeSet) { - match expression.data() { - Expression::Binary(value) => { - scan_expression(&value.left, features); - if value.operator == BinaryOperator::Exponentiate { - features.insert(LanguageFeature::Exponentiation); +/// What a module's own text exports. +#[derive(Default)] +pub(crate) struct ModuleExports { + /// Names with no runtime form: types and const enums. + pub(crate) types: BTreeSet, + /// Names with a runtime value. + pub(crate) values: BTreeSet, + /// Specifiers of `export * from`. + pub(crate) stars: Vec, + /// `(exported, module, imported)`: a name whose kind another module decides. + pub(crate) forwards: Vec<(String, String, String)>, +} + +/// Reads exports rather than declarations: `interface Foo {}` beside +/// `export { value as Foo }` exports a value. +pub(crate) fn module_exports(file: &SourceFile, model: &SemanticModel) -> ModuleExports { + let scope = model.scope(model.module_scope()); + let const_enums: BTreeSet = const_enum_names(file).into_iter().collect(); + let erased_local = |name: &str| { + const_enums.contains(name) + || (scope.value(name).is_none() && scope.type_binding(name).is_some()) + }; + let imports = imported_bindings(file); + let type_imports = declared_type_imports(file); + let local_values = local_value_names(file); + let mut exports = ModuleExports::default(); + for statement in file.statements() { + let Statement::Export(export) = statement.data() else { + continue; + }; + match export { + ExportDeclaration::Named(ExportNamedDeclaration::Declaration(inner)) => { + let (inner, ambient) = match inner.data() { + Statement::Declare(declared) => (declared, true), + _ => (inner, false), + }; + match inner.data() { + Statement::Interface(declaration) => { + exports + .types + .insert(identifier_text(file, &declaration.name)); + } + Statement::TypeAlias(declaration) => { + exports + .types + .insert(identifier_text(file, &declaration.name)); + } + Statement::Enum(declaration) => { + let name = raw_token_text(file, declaration.name.data().token()); + if declaration.is_const { + exports.types.insert(name); + } else { + exports.values.insert(name); + } + } + Statement::Namespace(declaration) => { + if let NamespaceName::Identifier { name, .. } = &declaration.name { + // An ambient namespace can declare values too. + let bearing = ambient + || model + .namespace_facts() + .declaration(inner.id()) + .is_some_and(|plan| plan.is_value_bearing()); + let name = identifier_text(file, name); + if bearing { + exports.values.insert(name); + } else { + exports.types.insert(name); + } + } + } + Statement::Class(class) => { + exports + .values + .extend(class.name.iter().map(|name| identifier_text(file, name))); + } + Statement::Function(declaration) => { + exports.values.extend( + declaration + .function + .name + .iter() + .map(|name| identifier_text(file, name)), + ); + } + _ => exports.values.extend(declared_binding_names(file, inner)), + } } - scan_expression(&value.right, features); - } - Expression::Assignment(value) => { - scan_target(value.left.data(), features); - if value.operator == AssignmentOperator::ExponentiateAssign { - features.insert(LanguageFeature::Exponentiation); - } - if matches!( - value.operator, - AssignmentOperator::LogicalAndAssign - | AssignmentOperator::LogicalOrAssign - | AssignmentOperator::NullishAssign - ) { - features.insert(LanguageFeature::LogicalAssignment); - } - scan_expression(&value.right, features); - } - Expression::Template(value) => { - for expression in &value.expressions { - scan_expression(expression, features); - } - } - Expression::TaggedTemplate(value) => { - scan_expression(&value.tag, features); - for expression in &value.template.expressions { - scan_expression(expression, features); - } - } - Expression::Array(value) => { - for element in &value.elements { - if let ArrayElement::Expression(expression) = element { - scan_expression(expression, features); - } - } - } - Expression::Object(value) => { - for member in &value.members { - match member.data() { - ObjectMember::Property(property) => { - scan_computed_name(&property.name, features); - scan_expression(&property.value, features); - } - ObjectMember::Method(method) => { - scan_computed_name(&method.name, features); - scan_parameter_defaults(&method.function.parameters, features); - scan_nested_body(method.function.body.as_ref(), features); + ExportDeclaration::Named(ExportNamedDeclaration::Specifiers { + type_only, + specifiers, + source, + .. + }) => { + for specifier in specifiers { + let specifier = specifier.data(); + let local = module_export_text(file, &specifier.local); + let exported = module_export_text(file, &specifier.exported); + if *type_only || !matches!(specifier.mode, ExportSpecifierMode::Value) { + exports.types.insert(exported); + } else if let Some(source) = source { + let module = module_specifier(file, source); + exports.forwards.push((exported, module, local)); + } else if local_values.contains(&local) { + exports.values.insert(exported); + } else if type_imports.contains(&local) { + exports.types.insert(exported); + } else if let Some((module, imported)) = imports.get(&local) { + exports + .forwards + .push((exported, module.clone(), imported.clone())); + } else if erased_local(&local) { + exports.types.insert(exported); + } else { + exports.values.insert(exported); } - ObjectMember::Spread(spread) => scan_expression(&spread.argument, features), - ObjectMember::Missing(_) => {} } } - } - Expression::Function(value) => { - scan_parameter_defaults(&value.function.parameters, features); - scan_nested_body(value.function.body.as_ref(), features); - } - Expression::Class(value) => scan_class_expressions(&value.class, features), - Expression::Arrow(value) => { - scan_parameter_defaults(&value.parameters, features); - scan_nested_body(Some(&value.body), features); - } - Expression::Call(value) => { - if value.optional { - features.insert(LanguageFeature::OptionalChaining); - } - scan_expression(&value.callee, features); - for argument in &value.arguments { - match argument { - CallArgument::Expression(expression) => scan_expression(expression, features), - CallArgument::Spread(spread) => scan_expression(&spread.argument, features), - CallArgument::Missing(_) => {} + ExportDeclaration::All(all) => match &all.exported { + _ if all.type_only => {} + Some(name) => { + exports.values.insert(module_export_text(file, name)); } + None => exports.stars.push(module_specifier(file, &all.source)), + }, + ExportDeclaration::Default(default) => { + let read = match &default.value { + ExportDefaultValue::Expression(value) => match value.data() { + Expression::Identifier(identifier) => { + Some(identifier_text(file, identifier)) + } + _ => None, + }, + _ => None, + }; + let read = read.filter(|name| !local_values.contains(name)); + if let Some((module, imported)) = read.as_ref().and_then(|name| imports.get(name)) { + let default = String::from("default"); + exports + .forwards + .push((default, module.clone(), imported.clone())); + continue; + } + let erased = matches!(default.value, ExportDefaultValue::Interface(_)) + || read.is_some_and(|name| erased_local(&name) || type_imports.contains(&name)); + let names = if erased { + &mut exports.types + } else { + &mut exports.values + }; + names.insert(String::from("default")); } + _ => {} } - Expression::Member(value) => { - if value.optional { - features.insert(LanguageFeature::OptionalChaining); - } - scan_expression(&value.object, features); - if let MemberProperty::Computed(key) = &value.property { - scan_expression(key, features); - } + } + exports +} + +/// Each imported local, by the module and name it comes from. +fn imported_bindings(file: &SourceFile) -> BTreeMap { + let mut bindings = BTreeMap::new(); + for statement in file.statements() { + let Statement::Import(import) = statement.data() else { + continue; + }; + let Some(clause) = &import.clause else { + continue; + }; + let module = module_specifier(file, &import.source); + if let Some(default) = &clause.default { + bindings.insert( + identifier_text(file, default), + (module.clone(), String::from("default")), + ); } - Expression::New(value) => { - scan_expression(&value.callee, features); - for argument in &value.arguments { - match argument { - CallArgument::Expression(expression) => scan_expression(expression, features), - CallArgument::Spread(spread) => scan_expression(&spread.argument, features), - CallArgument::Missing(_) => {} - } + if let Some(ImportBinding::Named(specifiers)) = &clause.binding { + for specifier in specifiers { + let specifier = specifier.data(); + bindings.insert( + identifier_text(file, &specifier.local), + ( + module.clone(), + module_export_text(file, &specifier.imported), + ), + ); } } - Expression::Await(value) => scan_expression(&value.argument, features), - Expression::Yield(value) => { - if let Some(argument) = value.argument.as_deref() { - scan_expression(argument, features); + } + bindings +} + +/// The file's const enums, which emit erases. +#[must_use] +pub(crate) fn const_enum_names(file: &SourceFile) -> Vec { + file.statements() + .iter() + .filter_map(|statement| match statement.data() { + Statement::Enum(declaration) => Some(declaration), + Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration( + inner, + ))) => match inner.data() { + Statement::Enum(declaration) => Some(declaration), + _ => None, + }, + _ => None, + }) + .filter(|declaration| declaration.is_const) + .map(|declaration| raw_token_text(file, declaration.name.data().token())) + .collect() +} + +fn module_specifier(file: &SourceFile, source: &StringLiteralNode) -> String { + raw_token_text(file, source.data().token()) + .trim_matches(['"', '\'']) + .to_owned() +} + +/// Local names with no runtime value: `import type` bindings, imports of a +/// name its module erases, and const enums. +fn type_import_names(file: &SourceFile, options: &TransformOptions) -> BTreeSet { + let mut names: BTreeSet = const_enum_names(file).into_iter().collect(); + names.extend(erased_import_locals(file, options)); + names.extend(declared_type_imports(file)); + // A local value of the same name is the one an export means. + let values = local_value_names(file); + names.retain(|name| !values.contains(name)); + names +} + +/// Locals bound to what the imported module erases. +fn erased_import_locals(file: &SourceFile, options: &TransformOptions) -> BTreeSet { + imported_bindings(file) + .into_iter() + .filter(|(_, binding)| options.erased_imports.contains(binding)) + .map(|(local, _)| local) + .collect() +} + +/// What the file's own top-level statements declare as values. +fn local_value_names(file: &SourceFile) -> BTreeSet { + let mut names = hoisted_function_names(file); + for statement in file.statements() { + let statement = match statement.data() { + Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration( + inner, + ))) => inner, + _ => statement, + }; + names.extend(declared_binding_names(file, statement)); + match statement.data() { + Statement::Class(class) => { + names.extend(class.name.iter().map(|name| identifier_text(file, name))); } - } - Expression::Unary(value) => scan_expression(&value.argument, features), - Expression::Update(value) => scan_target(value.argument.data(), features), - Expression::Logical(value) => { - if value.operator == LogicalOperator::Nullish { - features.insert(LanguageFeature::NullishCoalescing); + Statement::Namespace(NamespaceDeclaration { + name: NamespaceName::Identifier { name, .. }, + .. + }) => { + names.insert(identifier_text(file, name)); } - scan_expression(&value.left, features); - scan_expression(&value.right, features); + _ => {} } - Expression::Conditional(value) => { - scan_expression(&value.test, features); - scan_expression(&value.consequent, features); - scan_expression(&value.alternate, features); + } + names +} + +/// The names `import type` and `import { type x }` bind. +fn declared_type_imports(file: &SourceFile) -> BTreeSet { + let mut names = BTreeSet::new(); + for statement in file.statements() { + let Statement::Import(import) = statement.data() else { + continue; + }; + let Some(clause) = &import.clause else { + continue; + }; + if import.type_only { + names.extend( + clause + .default + .iter() + .map(|name| identifier_text(file, name)), + ); } - Expression::Sequence(value) => { - for expression in &value.expressions { - scan_expression(expression, features); + match &clause.binding { + Some(ImportBinding::Namespace(name)) if import.type_only => { + names.insert(identifier_text(file, name)); } - } - Expression::Parenthesized(value) => scan_expression(value, features), - Expression::As(value) => scan_expression(&value.expression, features), - Expression::Satisfies(value) => scan_expression(&value.expression, features), - Expression::TypeAssertion(value) => scan_expression(&value.expression, features), - Expression::NonNull(value) => scan_expression(&value.expression, features), - Expression::Import(value) => { - scan_expression(&value.source, features); - if let Some(options) = value.options.as_deref() { - scan_expression(options, features); + Some(ImportBinding::Named(specifiers)) => { + names.extend( + specifiers + .iter() + .map(|specifier| specifier.data()) + .filter(|specifier| { + import.type_only + || matches!(specifier.mode, ImportSpecifierMode::TypeOnly) + }) + .map(|specifier| identifier_text(file, &specifier.local)), + ); } + _ => {} } - // JSX trees print through the JSX desugar plan, which owns their - // expression containers, so the transform walk never visits them. - _ => {} } + names } -/// Recursively scans an assignment target for exponentiation syntax. -fn scan_target(target: &AssignmentTarget, features: &mut BTreeSet) { - match target { - AssignmentTarget::Member(member) => { - scan_expression(&member.object, features); - if let MemberProperty::Computed(key) = &member.property { - scan_expression(key, features); +/// A function declaration with a body hoists; a signature is not a binding. +fn hoisted_function_name(file: &SourceFile, function: &FunctionLike) -> Option { + function.body.as_ref()?; + function + .name + .as_ref() + .map(|name| identifier_text(file, name)) +} + +/// The module's top-level functions, which `export { f }` hoists like +/// `export function f`. +fn hoisted_function_names(file: &SourceFile) -> BTreeSet { + file.statements() + .iter() + .filter_map(|statement| match statement.data() { + Statement::Function(declaration) => hoisted_function_name(file, &declaration.function), + Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration( + inner, + ))) => match inner.data() { + Statement::Function(declaration) => { + hoisted_function_name(file, &declaration.function) + } + _ => None, + }, + _ => None, + }) + .collect() +} + +/// Plans a local export's prelude line: a function's assignment, or the +/// `void 0` that claims a specifier's name before any star export runs. +fn plan_cjs_local_exports( + file: &SourceFile, + export: &ExportDeclaration, + hoisted_functions: &BTreeSet, + binds_type: impl Fn(&str) -> bool, + plan: &mut CjsPlan, +) { + match export { + ExportDeclaration::Named(ExportNamedDeclaration::Declaration(inner)) => { + if let Statement::Function(declaration) = inner.data() + && let Some(name) = hoisted_function_name(file, &declaration.function) + { + plan.hoisted_exports.push((name.clone(), name)); + } + } + ExportDeclaration::Default(ExportDefaultDeclaration { + value: ExportDefaultValue::Function(function), + }) if function.body.is_some() => { + let local = + hoisted_function_name(file, function).unwrap_or_else(|| String::from("default_1")); + plan.hoisted_exports.push((String::from("default"), local)); + } + ExportDeclaration::Named(ExportNamedDeclaration::Specifiers { + type_only: false, + specifiers, + source: None, + .. + }) => { + for specifier in specifiers { + let specifier = specifier.data(); + let ModuleExportName::Identifier(local) = &specifier.local else { + continue; + }; + let local = identifier_text(file, local); + if matches!(specifier.mode, ExportSpecifierMode::TypeOnly) || binds_type(&local) { + continue; + } + let exported = module_export_text(file, &specifier.exported); + if hoisted_functions.contains(&local) { + plan.hoisted_exports.push((exported, local)); + } else { + plan.void_0.push(exported); + } } } - AssignmentTarget::Object(pattern) => { - for property in &pattern.properties { - scan_computed_name(&property.name, features); - scan_target(property.target.data(), features); - if let Some(initializer) = property.initializer.as_deref() { - scan_expression(initializer, features); - } + _ => {} + } +} + +/// Whether a declaration's bindings can change after it runs. +fn is_reassignable(declaration: &VariableDeclaration) -> bool { + matches!(declaration.kind, VariableKind::Var | VariableKind::Let) +} + +fn collect_pattern_names(file: &SourceFile, pattern: &Pattern, names: &mut Vec) { + match pattern.data() { + BindingPattern::Identifier(ident) => names.push(identifier_text(file, ident)), + BindingPattern::Object(object) => { + for property in &object.properties { + collect_pattern_names(file, &property.binding, names); } } - AssignmentTarget::Array(pattern) => { - for element in &pattern.elements { - if let AssignmentArrayElement::Target(inner) = element { - scan_target(inner.data(), features); + BindingPattern::Array(array) => { + for element in &array.elements { + if let ArrayBindingElement::Binding(binding) = element { + collect_pattern_names(file, binding, names); } } } - AssignmentTarget::Identifier(_) | AssignmentTarget::Missing(_) => {} - AssignmentTarget::Invalid(operand) => { - scan_expression(operand, features); + BindingPattern::Rest(rest) => collect_pattern_names(file, &rest.argument, names), + BindingPattern::Assignment(assignment) => { + collect_pattern_names(file, &assignment.left, names); } + BindingPattern::Missing(_) => {} } } -/// Scans a property name for exponentiation inside a computed key. -fn scan_computed_name(name: &PropertyName, features: &mut BTreeSet) { - if let PropertyName::Computed(key) = name { - scan_expression(key, features); +/// Declared value names of a statement, for export assignment generation. +fn declared_binding_names(file: &SourceFile, statement: &Stmt) -> Vec { + match statement.data() { + Statement::Variable(declaration) => { + let mut names = Vec::new(); + for declarator in &declaration.declarations { + collect_pattern_names(file, &declarator.data().binding, &mut names); + } + names + } + Statement::Enum(declaration) if !declaration.is_const => { + vec![raw_token_text(file, declaration.name.data().token())] + } + _ => Vec::new(), } } -/// Scans parameter default values for exponentiation syntax. -fn scan_parameter_defaults(parameters: &[ParameterNode], features: &mut BTreeSet) { - for parameter in parameters { - if let Some(initializer) = parameter.data().initializer.as_deref() { - scan_expression(initializer, features); +fn property_key_text(rewriter: &mut Rewriter, property: &ObjectBindingProperty) -> Option { + match &property.name { + PropertyName::Identifier(ident) => { + Some(rewriter.token_text(ident.data().token()).to_owned()) + } + PropertyName::String(literal) => { + let raw = rewriter.token_text(literal.data().token()); + Some(raw.trim_matches(['"', '\'']).to_owned()) } + _ => None, } } -/// Scans a nested function body for exponentiation syntax. -fn scan_nested_body(body: Option<&FunctionBody>, features: &mut BTreeSet) { - match body { - Some(FunctionBody::Block(block)) => { - scan_nested_statements(&block.data().statements, features); +/// Source range of a property name, taken from its inner node. +fn property_name_range(name: &PropertyName) -> TextRange { + match name { + PropertyName::Identifier(ident) => ident.range(), + PropertyName::String(literal) => literal.range(), + PropertyName::Number(number) => number.range(), + PropertyName::Computed(expression) => expression.range(), + PropertyName::Private(private) => private.range(), + PropertyName::Missing(_) => { + TextRange::new(Utf16Pos::ZERO, Utf16Pos::ZERO).expect("zero-width range is ordered") } - Some(FunctionBody::Expression(expression)) => scan_expression(expression, features), - Some(FunctionBody::Missing(_)) | None => {} } } -/// Exponentiation-only statement walk for bodies the rewriter descends into -/// from expression position (nested functions, arrows, object methods). -fn scan_nested_statements(statements: &[Stmt], features: &mut BTreeSet) { +fn scan_statements( + file: &SourceFile, + statements: &[Stmt], + options: &TransformOptions, + features: &mut BTreeSet, + diagnostics: &mut Vec, + in_async: bool, +) { + let _ = in_async; for statement in statements { match statement.data() { + Statement::Class(class) => { + features.insert(LanguageFeature::Classes); + scan_class(file, class, options, features, diagnostics); + } + Statement::Function(function) => { + scan_function( + file, + &function.function, + statement.range(), + options, + features, + diagnostics, + ); + } Statement::Variable(declaration) => { + if matches!( + declaration.kind, + VariableKind::Using | VariableKind::AwaitUsing + ) { + features.insert(LanguageFeature::Using); + } for declarator in &declaration.declarations { + scan_pattern(declarator.data().binding.data(), features); if let Some(initializer) = declarator.data().initializer.as_deref() { scan_expression(initializer, features); } } } Statement::Expression(expr) => scan_expression(&expr.expression, features), - Statement::Return(ret) => { - if let Some(argument) = ret.argument.as_deref() { - scan_expression(argument, features); - } - } - Statement::Throw(value) => scan_expression(&value.argument, features), Statement::If(value) => { scan_expression(&value.test, features); scan_nested_statements(std::slice::from_ref(value.consequent.as_ref()), features); @@ -9345,6 +10397,12 @@ fn scan_nested_statements(statements: &[Stmt], features: &mut BTreeSet { scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); } + Statement::Return(ret) => { + if let Some(argument) = ret.argument.as_deref() { + scan_expression(argument, features); + } + } + Statement::Throw(value) => scan_expression(&value.argument, features), Statement::For(value) => { if let Some(ForInitializer::Expression(initializer)) = &value.initializer { scan_expression(initializer, features); @@ -9362,6 +10420,9 @@ fn scan_nested_statements(statements: &[Stmt], features: &mut BTreeSet { + if for_of.mode == ForOfMode::Async { + features.insert(LanguageFeature::AsyncIteration); + } scan_expression(&for_of.iterable, features); scan_nested_statements(std::slice::from_ref(for_of.body.as_ref()), features); } @@ -9383,159 +10444,975 @@ fn scan_nested_statements(statements: &[Stmt], features: &mut BTreeSet { - scan_parameter_defaults(&function.function.parameters, features); - scan_nested_body(function.function.body.as_ref(), features); - } - Statement::Class(class) => scan_class_expressions(class, features), - Statement::Block(block) => { - scan_nested_statements(&block.data().statements, features); - } Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration( inner, - ))) => scan_nested_statements(std::slice::from_ref(inner), features), + ))) => scan_statements( + file, + std::slice::from_ref(inner), + options, + features, + diagnostics, + in_async, + ), Statement::Export(ExportDeclaration::Default(default)) => { - if let ExportDefaultValue::Expression(value) = &default.value { + if let ExportDefaultValue::Class(class) = &default.value { + features.insert(LanguageFeature::Classes); + scan_class(file, class, options, features, diagnostics); + } else if let ExportDefaultValue::Expression(value) = &default.value { scan_expression(value, features); } } + Statement::Block(block) => { + scan_statements( + file, + &block.data().statements, + options, + features, + diagnostics, + in_async, + ); + } _ => {} } } } -/// Exponentiation-only walk over the expression positions of one class. -fn scan_class_expressions(class: &ClassDeclaration, features: &mut BTreeSet) { +fn scan_class( + file: &SourceFile, + class: &ClassDeclaration, + options: &TransformOptions, + features: &mut BTreeSet, + diagnostics: &mut Vec, +) { if let Some(heritage) = &class.extends { scan_expression(&heritage.expression, features); } for member in &class.members { match member.data() { ClassMember::Property(property) => { + if !property.modifiers.is_declare && !property.modifiers.is_abstract { + features.insert(LanguageFeature::ClassFields); + } scan_computed_name(&property.name, features); if let Some(initializer) = property.initializer.as_deref() { scan_expression(initializer, features); } + if matches!(property.name, PropertyName::Private(_)) + && !options.target.supports(LanguageFeature::ClassFields) + { + diagnostics.push(Diagnostic::error( + codes::PRIVATE_FIELD_REQUIRES_ES2022, + file.source_id(), + member.range(), + "private class fields require ScriptTarget::Es2022 or later", + )); + } } ClassMember::Method(method) => { - scan_computed_name(&method.name, features); - scan_parameter_defaults(&method.function.parameters, features); - scan_nested_body(method.function.body.as_ref(), features); + scan_function( + file, + &method.function, + member.range(), + options, + features, + diagnostics, + ); } ClassMember::AutoAccessor(accessor) => { + features.insert(LanguageFeature::ClassFields); scan_computed_name(&accessor.name, features); if let Some(initializer) = accessor.initializer.as_deref() { scan_expression(initializer, features); } + if !options.target.supports(LanguageFeature::ClassFields) { + diagnostics.push(Diagnostic::error( + codes::AUTO_ACCESSOR_REQUIRES_ES2022, + file.source_id(), + member.range(), + "auto-accessors require ScriptTarget::Es2022 or later", + )); + } } ClassMember::StaticBlock(block) => { - scan_nested_statements(&block.data().statements, features); + features.insert(LanguageFeature::ClassFields); + scan_statements( + file, + &block.data().statements, + options, + features, + diagnostics, + false, + ); } _ => {} } } } -#[cfg(test)] -mod tests { - use super::{LanguageFeature, ScriptTarget, analyze, codes, emit_transformed}; - use crate::checker; - use crate::diagnostic::Recovered; - use crate::emitter::{EmitFileNames, EmitOptions, EmitOutput, ModuleKind}; - use crate::parser; - use crate::scanner; - use crate::source::{ScriptKind, SourceId, SourceText}; - use std::sync::Arc; +fn scan_function( + file: &SourceFile, + function: &FunctionLike, + _range: TextRange, + options: &TransformOptions, + features: &mut BTreeSet, + diagnostics: &mut Vec, +) { + if function.is_async { + features.insert(LanguageFeature::AsyncFunctions); + } + if function.is_generator { + features.insert(LanguageFeature::Generators); + } + for parameter in &function.parameters { + scan_pattern(parameter.data().binding.data(), features); + if let Some(initializer) = parameter.data().initializer.as_deref() { + scan_expression(initializer, features); + } + } + match function.body.as_ref() { + Some(FunctionBody::Block(body)) => scan_statements( + file, + &body.data().statements, + options, + features, + diagnostics, + function.is_async, + ), + Some(FunctionBody::Expression(expression)) => scan_expression(expression, features), + Some(FunctionBody::Missing(_)) | None => {} + } +} - fn parse(source: &str) -> crate::syntax::SourceFile { - parser::parse(scanner::scan( - SourceId::new(0), - ScriptKind::TypeScript, - Arc::new(SourceText::new(source).expect("test source fits the per-file source budget")), - )) - .into_product() +fn scan_pattern(pattern: &BindingPattern, features: &mut BTreeSet) { + match pattern { + BindingPattern::Object(_) | BindingPattern::Array(_) => { + features.insert(LanguageFeature::Destructuring); + } + BindingPattern::Rest(_) => { + features.insert(LanguageFeature::ObjectRestSpread); + features.insert(LanguageFeature::Destructuring); + } + _ => {} } +} - fn names() -> EmitFileNames { - EmitFileNames { - source_name: Arc::from("input.ts"), - js_file_name: Some(Arc::from("output.js")), - declaration_file_name: Some(Arc::from("output.d.ts")), - source_root: None, - ..EmitFileNames::default() +/// Recursively scans one expression for exponentiation syntax. The deep walk +/// intentionally reports only [`LanguageFeature::Exponentiation`]: every other +/// feature keeps the statement-level analysis surface this module has always +/// exposed, and only exponentiation is lowered in every expression position +/// the rewriter visits. +fn scan_expression(expression: &Expr, features: &mut BTreeSet) { + match expression.data() { + Expression::Binary(value) => { + scan_expression(&value.left, features); + if value.operator == BinaryOperator::Exponentiate { + features.insert(LanguageFeature::Exponentiation); + } + scan_expression(&value.right, features); + } + Expression::Assignment(value) => { + scan_target(value.left.data(), features); + if value.operator == AssignmentOperator::ExponentiateAssign { + features.insert(LanguageFeature::Exponentiation); + } + if matches!( + value.operator, + AssignmentOperator::LogicalAndAssign + | AssignmentOperator::LogicalOrAssign + | AssignmentOperator::NullishAssign + ) { + features.insert(LanguageFeature::LogicalAssignment); + } + scan_expression(&value.right, features); } + Expression::Template(value) => { + for expression in &value.expressions { + scan_expression(expression, features); + } + } + Expression::TaggedTemplate(value) => { + scan_expression(&value.tag, features); + for expression in &value.template.expressions { + scan_expression(expression, features); + } + } + Expression::Array(value) => { + for element in &value.elements { + if let ArrayElement::Expression(expression) = element { + scan_expression(expression, features); + } + } + } + Expression::Object(value) => { + for member in &value.members { + match member.data() { + ObjectMember::Property(property) => { + scan_computed_name(&property.name, features); + scan_expression(&property.value, features); + } + ObjectMember::Method(method) => { + scan_computed_name(&method.name, features); + scan_parameter_defaults(&method.function.parameters, features); + scan_nested_body(method.function.body.as_ref(), features); + } + ObjectMember::Spread(spread) => scan_expression(&spread.argument, features), + ObjectMember::Missing(_) => {} + } + } + } + Expression::Function(value) => { + scan_parameter_defaults(&value.function.parameters, features); + scan_nested_body(value.function.body.as_ref(), features); + } + Expression::Class(value) => scan_class_expressions(&value.class, features), + Expression::Arrow(value) => { + scan_parameter_defaults(&value.parameters, features); + scan_nested_body(Some(&value.body), features); + } + Expression::Call(value) => { + if value.optional { + features.insert(LanguageFeature::OptionalChaining); + } + scan_expression(&value.callee, features); + for argument in &value.arguments { + match argument { + CallArgument::Expression(expression) => scan_expression(expression, features), + CallArgument::Spread(spread) => scan_expression(&spread.argument, features), + CallArgument::Missing(_) => {} + } + } + } + Expression::Member(value) => { + if value.optional { + features.insert(LanguageFeature::OptionalChaining); + } + scan_expression(&value.object, features); + if let MemberProperty::Computed(key) = &value.property { + scan_expression(key, features); + } + } + Expression::New(value) => { + scan_expression(&value.callee, features); + for argument in &value.arguments { + match argument { + CallArgument::Expression(expression) => scan_expression(expression, features), + CallArgument::Spread(spread) => scan_expression(&spread.argument, features), + CallArgument::Missing(_) => {} + } + } + } + Expression::Await(value) => scan_expression(&value.argument, features), + Expression::Yield(value) => { + if let Some(argument) = value.argument.as_deref() { + scan_expression(argument, features); + } + } + Expression::Unary(value) => scan_expression(&value.argument, features), + Expression::Update(value) => scan_target(value.argument.data(), features), + Expression::Logical(value) => { + if value.operator == LogicalOperator::Nullish { + features.insert(LanguageFeature::NullishCoalescing); + } + scan_expression(&value.left, features); + scan_expression(&value.right, features); + } + Expression::Conditional(value) => { + scan_expression(&value.test, features); + scan_expression(&value.consequent, features); + scan_expression(&value.alternate, features); + } + Expression::Sequence(value) => { + for expression in &value.expressions { + scan_expression(expression, features); + } + } + Expression::Parenthesized(value) => scan_expression(value, features), + Expression::As(value) => scan_expression(&value.expression, features), + Expression::Satisfies(value) => scan_expression(&value.expression, features), + Expression::TypeAssertion(value) => scan_expression(&value.expression, features), + Expression::NonNull(value) => scan_expression(&value.expression, features), + Expression::Import(value) => { + scan_expression(&value.source, features); + if let Some(options) = value.options.as_deref() { + scan_expression(options, features); + } + } + // JSX trees print through the JSX desugar plan, which owns their + // expression containers, so the transform walk never visits them. + _ => {} + } +} + +/// Recursively scans an assignment target for exponentiation syntax. +fn scan_target(target: &AssignmentTarget, features: &mut BTreeSet) { + match target { + AssignmentTarget::Member(member) => { + scan_expression(&member.object, features); + if let MemberProperty::Computed(key) = &member.property { + scan_expression(key, features); + } + } + AssignmentTarget::Object(pattern) => { + for property in &pattern.properties { + scan_computed_name(&property.name, features); + scan_target(property.target.data(), features); + if let Some(initializer) = property.initializer.as_deref() { + scan_expression(initializer, features); + } + } + } + AssignmentTarget::Array(pattern) => { + for element in &pattern.elements { + if let AssignmentArrayElement::Target(inner) = element { + scan_target(inner.data(), features); + } + } + } + AssignmentTarget::Identifier(_) | AssignmentTarget::Missing(_) => {} + AssignmentTarget::Invalid(operand) => { + scan_expression(operand, features); + } + } +} + +/// Scans a property name for exponentiation inside a computed key. +fn scan_computed_name(name: &PropertyName, features: &mut BTreeSet) { + if let PropertyName::Computed(key) = name { + scan_expression(key, features); + } +} + +/// Scans parameter default values for exponentiation syntax. +fn scan_parameter_defaults(parameters: &[ParameterNode], features: &mut BTreeSet) { + for parameter in parameters { + if let Some(initializer) = parameter.data().initializer.as_deref() { + scan_expression(initializer, features); + } + } +} + +/// Scans a nested function body for exponentiation syntax. +fn scan_nested_body(body: Option<&FunctionBody>, features: &mut BTreeSet) { + match body { + Some(FunctionBody::Block(block)) => { + scan_nested_statements(&block.data().statements, features); + } + Some(FunctionBody::Expression(expression)) => scan_expression(expression, features), + Some(FunctionBody::Missing(_)) | None => {} + } +} + +/// Exponentiation-only statement walk for bodies the rewriter descends into +/// from expression position (nested functions, arrows, object methods). +fn scan_nested_statements(statements: &[Stmt], features: &mut BTreeSet) { + for statement in statements { + match statement.data() { + Statement::Variable(declaration) => { + for declarator in &declaration.declarations { + if let Some(initializer) = declarator.data().initializer.as_deref() { + scan_expression(initializer, features); + } + } + } + Statement::Expression(expr) => scan_expression(&expr.expression, features), + Statement::Return(ret) => { + if let Some(argument) = ret.argument.as_deref() { + scan_expression(argument, features); + } + } + Statement::Throw(value) => scan_expression(&value.argument, features), + Statement::If(value) => { + scan_expression(&value.test, features); + scan_nested_statements(std::slice::from_ref(value.consequent.as_ref()), features); + if let Some(alternate) = value.alternate.as_deref() { + scan_nested_statements(std::slice::from_ref(alternate), features); + } + } + Statement::While(value) => { + scan_expression(&value.test, features); + scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); + } + Statement::DoWhile(value) => { + scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); + scan_expression(&value.test, features); + } + Statement::With(value) => { + scan_expression(&value.object, features); + scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); + } + Statement::Labeled(value) => { + scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); + } + Statement::For(value) => { + if let Some(ForInitializer::Expression(initializer)) = &value.initializer { + scan_expression(initializer, features); + } + if let Some(test) = value.test.as_deref() { + scan_expression(test, features); + } + if let Some(update) = value.update.as_deref() { + scan_expression(update, features); + } + scan_nested_statements(std::slice::from_ref(value.body.as_ref()), features); + } + Statement::ForIn(for_in) => { + scan_expression(&for_in.object, features); + scan_nested_statements(std::slice::from_ref(for_in.body.as_ref()), features); + } + Statement::ForOf(for_of) => { + scan_expression(&for_of.iterable, features); + scan_nested_statements(std::slice::from_ref(for_of.body.as_ref()), features); + } + Statement::Switch(value) => { + scan_expression(&value.discriminant, features); + for case in &value.cases { + if let Some(test) = case.data().test.as_deref() { + scan_expression(test, features); + } + scan_nested_statements(&case.data().consequent, features); + } + } + Statement::Try(value) => { + scan_nested_statements(&value.block.data().statements, features); + if let Some(handler) = &value.handler { + scan_nested_statements(&handler.data().body.data().statements, features); + } + if let Some(finalizer) = &value.finalizer { + scan_nested_statements(&finalizer.data().statements, features); + } + } + Statement::Function(function) => { + scan_parameter_defaults(&function.function.parameters, features); + scan_nested_body(function.function.body.as_ref(), features); + } + Statement::Class(class) => scan_class_expressions(class, features), + Statement::Block(block) => { + scan_nested_statements(&block.data().statements, features); + } + Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration( + inner, + ))) => scan_nested_statements(std::slice::from_ref(inner), features), + Statement::Export(ExportDeclaration::Default(default)) => { + if let ExportDefaultValue::Expression(value) = &default.value { + scan_expression(value, features); + } + } + _ => {} + } + } +} + +/// Exponentiation-only walk over the expression positions of one class. +fn scan_class_expressions(class: &ClassDeclaration, features: &mut BTreeSet) { + if let Some(heritage) = &class.extends { + scan_expression(&heritage.expression, features); + } + for member in &class.members { + match member.data() { + ClassMember::Property(property) => { + scan_computed_name(&property.name, features); + if let Some(initializer) = property.initializer.as_deref() { + scan_expression(initializer, features); + } + } + ClassMember::Method(method) => { + scan_computed_name(&method.name, features); + scan_parameter_defaults(&method.function.parameters, features); + scan_nested_body(method.function.body.as_ref(), features); + } + ClassMember::AutoAccessor(accessor) => { + scan_computed_name(&accessor.name, features); + if let Some(initializer) = accessor.initializer.as_deref() { + scan_expression(initializer, features); + } + } + ClassMember::StaticBlock(block) => { + scan_nested_statements(&block.data().statements, features); + } + _ => {} + } + } +} + +#[cfg(test)] +mod tests { + use super::{LanguageFeature, ScriptTarget, analyze, codes, emit_transformed}; + use crate::checker; + use crate::diagnostic::Recovered; + use crate::emitter::{EmitFileNames, EmitOptions, EmitOutput, ModuleKind}; + use crate::parser; + use crate::scanner; + use crate::source::{ScriptKind, SourceId, SourceText}; + use std::sync::Arc; + + fn parse(source: &str) -> crate::syntax::SourceFile { + parser::parse(scanner::scan( + SourceId::new(0), + ScriptKind::TypeScript, + Arc::new(SourceText::new(source).expect("test source fits the per-file source budget")), + )) + .into_product() + } + + fn names() -> EmitFileNames { + EmitFileNames { + source_name: Arc::from("input.ts"), + js_file_name: Some(Arc::from("output.js")), + declaration_file_name: Some(Arc::from("output.d.ts")), + source_root: None, + ..EmitFileNames::default() + } + } + + fn options_for(target: ScriptTarget) -> EmitOptions { + EmitOptions { + target, + ..EmitOptions::default() + } + } + + fn emit_at(source: &str, target: ScriptTarget) -> EmitOutput { + let file = parse(source); + let checked = checker::check(&Recovered::clean(parse(source))); + let options = options_for(target); + emit_transformed( + &file, + checked.product(), + &options.transform_view(), + &names(), + ) + } + + fn emit_with_options(source: &str, options: EmitOptions) -> EmitOutput { + let file = parse(source); + let checked = checker::check(&Recovered::clean(parse(source))); + emit_transformed( + &file, + checked.product(), + &options.transform_view(), + &names(), + ) + } + + fn javascript(output: &EmitOutput) -> &str { + &output.javascript.as_ref().expect("JavaScript output").code + } + + #[test] + fn dense_million_node_file_keeps_authored_preservation_unaliased() { + // The rewriter's id source is seeded past the parser high-water + // (and any JSX-desugar consumption), so minted ids are disjoint + // from authored ids BY CONSTRUCTION — the structural fix for the + // old constant floor. This pins the observable consequences on a + // dense million-node file: emit completes, no statements drop, + // authored single-line preservation holds, and field lowering + // still mints its descriptors. + let mut statements = 330_000usize; + let tail = "function probe() { return 1; }\nclass K { }"; + // Grow the prefix until the parser high-water crosses 1_000_100: + // node yield per statement amortizes, so calibrate on the real + // construction rather than a small sample. + let high_water = loop { + let probe = format!("{}{tail}", "a;".repeat(statements)); + let water = parse(&probe).id().get(); + if water >= 1_000_400 || statements > 600_000 { + break water; + } + statements += 2_000; + }; + let mut source = String::with_capacity(2_000_000); + source.push_str(&"a;".repeat(statements)); + source.push_str("function probe() { return 1; }\n"); + source.push_str("class K { "); + for i in 0..60 { + source.push_str(&format!("f{i} = 1; ")); + } + source.push_str("}\n"); + assert!( + (1_000_350..=1_004_500).contains(&high_water), + "calibration drifted: high water {high_water}" + ); + let output = emit_with_options( + &source, + EmitOptions { + target: ScriptTarget::Es2015, + use_define_for_class_fields: Some(true), + ..EmitOptions::default() + }, + ); + let code = javascript(&output); + assert!( + code.contains("function probe() { return 1; }"), + "authored single-line body lost preservation" + ); + assert!( + code.contains("enumerable: true"), + "descriptor lowering missing" + ); + } + + #[test] + fn esnext_keeps_async_and_class_fields() { + let source = "class C { x = 1; }\nasync function f() { return await 1; }\n"; + let output = emit_at(source, ScriptTarget::EsNext); + let code = javascript(&output); + assert!(code.contains("x = 1"), "{code}"); + assert!(code.contains("async function"), "{code}"); + assert!(!code.contains("__awaiter"), "{code}"); + } + + #[test] + fn es2015_downlevels_class_fields_into_the_constructor() { + let output = emit_at("class C { x = 1; }\n", ScriptTarget::Es2015); + let code = javascript(&output); + assert!(code.contains("constructor"), "{code}"); + assert!(code.contains("this.x = 1"), "{code}"); + assert!( + !code.lines().any(|line| line.trim() == "x = 1;"), + "field initializer should move: {code}" + ); + } + #[test] + fn require_locals_follow_the_tsc_slug_shape() { + // tsc binds `enum_1` for an identifier-starting slug and + // `_10_lib_1` when a leading digit forces sanitization. + let mut names = super::RequireNames::new(std::collections::BTreeSet::new()); + assert_eq!(names.allocate("./enum"), "enum_1"); + assert_eq!(names.allocate("./10_lib"), "_10_lib_1"); + assert_eq!(names.allocate("./enum"), "enum_2"); + } + + #[test] + fn class_expression_without_lowering_stays_inline() { + // Authority: classExpressionTest2 — a generic class expression with + // no lowered members emits inline with type parameters stripped + // (`var m = class C {`), never a temp-IIFE wrap. + let output = emit_at( + "function M() {\n var m = class C { f() { return 1; } };\n return new m();\n}\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + assert!(code.contains("var m = class C {"), "{code}"); + assert!(!code.contains("(() =>"), "{code}"); + } + + #[test] + fn derived_class_fields_follow_super_and_literal_names_are_preserved() { + let output = emit_at( + r#"class C extends B { "x" = 1; 2 = 3; constructor() { super(); after(); } } +"#, + ScriptTarget::Es2015, + ); + let code = javascript(&output); + let super_at = code + .find("super()") + .expect("derived constructor calls super"); + let string_at = code.find(r#"this["x"] = 1"#).expect("string field name"); + let number_at = code.find("this[2] = 3").expect("numeric field name"); + let after_at = code.find("after()").expect("original constructor body"); + assert!( + super_at < string_at && string_at < number_at && number_at < after_at, + "{code}" + ); + } + + #[test] + fn synthesized_derived_constructor_forwards_arguments_before_fields() { + let output = emit_at("class C extends B { x = 1; }\n", ScriptTarget::Es2015); + let code = javascript(&output); + assert!(code.contains("super(...arguments)"), "{code}"); + assert!( + code.find("super(...arguments)").unwrap() < code.find("this.x = 1").unwrap(), + "{code}" + ); + } + #[test] + fn uninitialized_fields_emit_nothing_under_assignment_semantics() { + // Baseline computedPropertyNames36_ES5(target=es2015).js keeps + // `class Foo {\n}` for `class Foo { x }`: assignment semantics + // define nothing for a declaration without an initializer. + let output = emit_at("class C { x; static y; }\n", ScriptTarget::Es2015); + let code = javascript(&output); + assert!(!code.contains("constructor"), "{code}"); + assert!(!code.contains("void 0"), "{code}"); + assert!(code.contains("class C {"), "{code}"); + } + + #[test] + fn uninitialized_fields_define_void_0_under_define_semantics() { + let output = emit_with_options( + "class C { x; static y; }\n", + EmitOptions { + target: ScriptTarget::Es2015, + use_define_for_class_fields: Some(true), + ..EmitOptions::default() + }, + ); + let code = javascript(&output); + assert!(code.contains("Object.defineProperty(this, \"x\""), "{code}"); + assert!(code.contains("Object.defineProperty(C, \"y\""), "{code}"); + assert!(code.contains("void 0"), "{code}"); + assert!(!code.contains("this.x ="), "{code}"); + } + + #[test] + fn computed_field_names_are_evaluated_once_at_class_definition() { + let output = emit_at( + "class C { [key()] = 1; } new C(); new C();\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + assert_eq!(code.matches("key()").count(), 1, "{code}"); + assert!(code.contains("let _t0 = key()"), "{code}"); + assert!(code.contains("this[_t0] = 1"), "{code}"); + } + #[test] + fn computed_keys_preserve_heritage_and_member_evaluation_order() { + let output = emit_at( + "class C extends heritage() { [field()] = 1; [method()]() {} [later()] = 2; }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + let heritage = code.find("heritage()").expect("heritage evaluation"); + let field = code.find("field()").expect("first computed field"); + let method = code.find("method()").expect("computed method"); + let later = code.find("later()").expect("later computed field"); + let class = code.find("class C extends _t0").expect("rewritten class"); + assert!( + heritage < field && field < method && method < later && later < class, + "{code}" + ); + assert!(code.contains("this[_t1] = 1"), "{code}"); + assert!(code.contains("[_t2]()"), "{code}"); + assert!(code.contains("this[_t3] = 2"), "{code}"); + } + #[test] + fn static_computed_field_runs_key_and_value_and_assigns_the_property() { + let output = emit_at( + "class C { static [key()] = value(); }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + assert_eq!(code.matches("key()").count(), 1, "{code}"); + assert_eq!(code.matches("value()").count(), 1, "{code}"); + let key = code.find("let _t0 = key()").expect("key setup"); + let class = code.find("class C").expect("class declaration"); + let assignment = code + .find("C[_t0] = value()") + .expect("post-class static property assignment"); + assert!(key < class && class < assignment, "{code}"); + } + #[test] + fn named_export_wraps_only_the_lowered_class() { + let output = emit_at( + "export class C { static [key()] = value(); }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + let prelude = code.find("let _t0 = key()").expect("ordinary key prelude"); + let class = code + .find("export class C") + .expect("exported rewritten class"); + let postlude = code + .find("C[_t0] = value()") + .expect("ordinary static postlude"); + assert!(prelude < class && class < postlude, "{code}"); + assert!(!code.contains("export let _t0"), "{code}"); + assert!(!code.contains("export C[_t0]"), "{code}"); + } + + #[test] + fn named_default_export_keeps_class_and_generated_statements_separate() { + let output = emit_at( + "export default class C { field = 1; static [key()] = value(); }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + let prelude = code.find("let _t0 = key()").expect("ordinary key prelude"); + let class = code + .find("export default class C") + .expect("named default class"); + let postlude = code + .find("C[_t0] = value()") + .expect("ordinary static postlude"); + assert!(prelude < class && class < postlude, "{code}"); + assert!(code.contains("this.field = 1"), "{code}"); + assert_eq!(code.matches("key()").count(), 1, "{code}"); + assert_eq!(code.matches("value()").count(), 1, "{code}"); + assert!(!code.contains("export default let"), "{code}"); + } + + #[test] + fn anonymous_default_export_lowers_instance_fields_in_place() { + let output = emit_at( + "export default class { field = 1; }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + assert!(code.contains("export default class"), "{code}"); + assert!(code.contains("constructor"), "{code}"); + assert!(code.contains("this.field = 1"), "{code}"); + assert!( + !output + .diagnostics + .iter() + .any(|diagnostic| { diagnostic.code() == codes::STATIC_FIELD_REQUIRES_CLASS_NAME }), + "{:?}", + output.diagnostics + ); + } + + #[test] + fn anonymous_default_static_field_uses_one_synthetic_class_target() { + let output = emit_at( + "export default class { static [key()] = value(); }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + let prelude = code.find("let _t1 = key()").expect("ordinary key prelude"); + let class = code + .find("export default class _t0") + .expect("synthetically named default class"); + let postlude = code + .find("_t0[_t1] = value()") + .expect("static assignment uses synthetic class name"); + assert!(prelude < class && class < postlude, "{code}"); + assert_eq!(code.matches("key()").count(), 1, "{code}"); + assert_eq!(code.matches("value()").count(), 1, "{code}"); + } + #[test] + fn anonymous_default_static_class_restores_imported_name_to_default() { + let output = emit_at( + "export default class { static x = 1; }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + let class = code + .find("export default class _t0") + .expect("synthetically named default class"); + let restore = code + .find(r#"Object.defineProperty(_t0, "name""#) + .expect("default name restoration"); + let static_assignment = code + .find("_t0.x = 1") + .expect("post-class static assignment"); + assert!(class < restore && restore < static_assignment, "{code}"); + assert!(code.contains(r#"value: "default""#), "{code}"); + assert!(code.contains("configurable: true"), "{code}"); } - fn options_for(target: ScriptTarget) -> EmitOptions { - EmitOptions { - target, - ..EmitOptions::default() - } + #[test] + fn es2015_downlevels_async_to_awaiter() { + // Authority: es5-importHelpersAsyncFunctions(target=es2015).js emits + // the awaiter alone — native generators need no __generator. + let output = emit_at( + "async function f(x) { return await x; }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + assert!(code.contains("__awaiter"), "{code}"); + assert!(code.contains("function*"), "{code}"); + assert!(code.contains("yield"), "{code}"); + assert!(!code.contains("async "), "{code}"); + assert!( + code.contains("var __awaiter = (this && this.__awaiter) || function"), + "{code}" + ); + assert!(!code.contains("__generator"), "{code}"); } - fn emit_at(source: &str, target: ScriptTarget) -> EmitOutput { + #[test] + fn helper_emission_uses_rewriter_demand_not_analysis_prediction() { + // Authority: es5-importHelpersAsyncFunctions(target=es2015).js. + let source = "function outer() { return async () => await 1; }\n"; let file = parse(source); let checked = checker::check(&Recovered::clean(parse(source))); - let options = options_for(target); - emit_transformed( - &file, - checked.product(), - &options.transform_view(), - &names(), - ) + let options = options_for(ScriptTarget::Es2015); + let transform_options = options.transform_view(); + let plan = analyze(&file, &transform_options); + assert!( + plan.helpers.is_empty(), + "fixture must remain outside the shallow analysis walk" + ); + + let output = emit_transformed(&file, checked.product(), &transform_options, &names()); + let code = javascript(&output); + assert!( + code.contains("var __awaiter = (this && this.__awaiter) || function"), + "{code}" + ); + assert!( + !code.contains("__generator"), + "native-generator targets emit no __generator: {code}" + ); } - fn emit_with_options(source: &str, options: EmitOptions) -> EmitOutput { - let file = parse(source); - let checked = checker::check(&Recovered::clean(parse(source))); - emit_transformed( - &file, - checked.product(), - &options.transform_view(), - &names(), - ) + #[test] + fn es5_downlevels_object_destructuring() { + let output = emit_at("const { a, b } = o;\n", ScriptTarget::Es5); + let code = javascript(&output); + assert!(code.contains("a = "), "{code}"); + assert!(code.contains("b = "), "{code}"); + assert!(!code.contains("{ a, b }"), "{code}"); } - fn javascript(output: &EmitOutput) -> &str { - &output.javascript.as_ref().expect("JavaScript output").code + #[test] + fn private_fields_below_es2022_are_errors() { + let output = emit_at("class C { #x = 1; }\n", ScriptTarget::Es2015); + assert!( + output + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code() == codes::PRIVATE_FIELD_REQUIRES_ES2022), + "{:?}", + output.diagnostics + ); } #[test] - fn dense_million_node_file_keeps_authored_preservation_unaliased() { - // The rewriter's id source is seeded past the parser high-water - // (and any JSX-desugar consumption), so minted ids are disjoint - // from authored ids BY CONSTRUCTION — the structural fix for the - // old constant floor. This pins the observable consequences on a - // dense million-node file: emit completes, no statements drop, - // authored single-line preservation holds, and field lowering - // still mints its descriptors. - let mut statements = 330_000usize; - let tail = "function probe() { return 1; }\nclass K { }"; - // Grow the prefix until the parser high-water crosses 1_000_100: - // node yield per statement amortizes, so calibrate on the real - // construction rather than a small sample. - let high_water = loop { - let probe = format!("{}{tail}", "a;".repeat(statements)); - let water = parse(&probe).id().get(); - if water >= 1_000_400 || statements > 600_000 { - break water; - } - statements += 2_000; - }; - let mut source = String::with_capacity(2_000_000); - source.push_str(&"a;".repeat(statements)); - source.push_str("function probe() { return 1; }\n"); - source.push_str("class K { "); - for i in 0..60 { - source.push_str(&format!("f{i} = 1; ")); - } - source.push_str("}\n"); + fn classes_below_es2015_are_errors() { + let output = emit_at("class C {}\n", ScriptTarget::Es5); assert!( - (1_000_350..=1_004_500).contains(&high_water), - "calibration drifted: high water {high_water}" + output + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code() == codes::CLASS_REQUIRES_ES2015), + "{:?}", + output.diagnostics + ); + } + + #[test] + fn using_declarations_are_errors_before_esnext() { + let output = emit_at("using x = y;\n", ScriptTarget::Es2022); + assert!( + output + .diagnostics + .iter() + .any(|diagnostic| diagnostic.code() == codes::USING_UNSUPPORTED), + "{:?}", + output.diagnostics + ); + } + + #[test] + fn static_fields_and_blocks_remain_interleaved() { + let output = emit_at( + "class C { static a = one(); static { step(); } static b = two(); }\n", + ScriptTarget::Es2015, ); + let code = javascript(&output); + let first = code.find("C.a = one()").expect("first static field"); + let block = code.find("step()").expect("static block"); + let second = code.find("C.b = two()").expect("second static field"); + assert!(first < block && block < second, "{code}"); + assert!(!code.contains("static {"), "{code}"); + } + + #[test] + fn define_mode_uses_complete_data_descriptors() { let output = emit_with_options( - &source, + "class C { x = 1; static y = 2; }\n", EmitOptions { target: ScriptTarget::Es2015, use_define_for_class_fields: Some(true), @@ -9543,784 +11420,1318 @@ mod tests { }, ); let code = javascript(&output); + assert!(code.contains("Object.defineProperty(this, \"x\""), "{code}"); + assert!(code.contains("Object.defineProperty(C, \"y\""), "{code}"); + let enumerable = code + .find("enumerable: true") + .expect("enumerable descriptor"); + let configurable = code + .find("configurable: true") + .expect("configurable descriptor"); + let writable = code.find("writable: true").expect("writable descriptor"); + let value = code.find("value: 1").expect("descriptor value"); assert!( - code.contains("function probe() { return 1; }"), - "authored single-line body lost preservation" + enumerable < configurable && configurable < writable && writable < value, + "{code}" + ); + } + + #[test] + fn synthesized_descriptor_object_expands_in_javascript() { + // Authority: override18(target=es2015).js — the defineProperty + // descriptor is wholly synthesized and prints multi-line even when + // the authored source is compact. An untainted mint collapsed it + // onto one line (sweep-2d76b09 regression). + let output = emit_with_options( + "class A { foo?: string; }\n", + EmitOptions { + target: ScriptTarget::Es2015, + use_define_for_class_fields: Some(true), + ..EmitOptions::default() + }, ); + let code = javascript(&output); assert!( - code.contains("enumerable: true"), - "descriptor lowering missing" + code.contains("Object.defineProperty(this, \"foo\", {\n enumerable: true,"), + "{code}" ); + assert!(!code.contains("{ enumerable: true, configurable"), "{code}"); } #[test] - fn esnext_keeps_async_and_class_fields() { - let source = "class C { x = 1; }\nasync function f() { return await 1; }\n"; - let output = emit_at(source, ScriptTarget::EsNext); + fn declare_and_abstract_fields_have_no_runtime_initializers() { + let output = emit_at( + "abstract class C { declare x: number; abstract y: string; z = 1; }\n", + ScriptTarget::Es2015, + ); let code = javascript(&output); - assert!(code.contains("x = 1"), "{code}"); - assert!(code.contains("async function"), "{code}"); - assert!(!code.contains("__awaiter"), "{code}"); + assert!(!code.contains("this.x"), "{code}"); + assert!(!code.contains("this.y"), "{code}"); + assert!(code.contains("this.z = 1"), "{code}"); } #[test] - fn es2015_downlevels_class_fields_into_the_constructor() { - let output = emit_at("class C { x = 1; }\n", ScriptTarget::Es2015); + fn nested_super_sites_receive_fields_after_each_completion() { + let output = emit_at( + "class C extends B { x = 1; constructor(flag) { if (flag) super(1); else { super(2); } } }\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + assert_eq!(code.matches("this.x = 1").count(), 2, "{code}"); + let first_super = code.find("super(1)").expect("first super"); + let first_init = code.find("this.x = 1").expect("first init"); + assert!(first_super < first_init, "{code}"); + } + + #[test] + fn class_expressions_lower_inside_call_arguments() { + let output = emit_at( + "consume(class { static x = 1; y = 2; });\n", + ScriptTarget::Es2015, + ); + let code = javascript(&output); + assert!(code.contains("consume((() =>"), "{code}"); + assert!(code.contains("this.y = 2"), "{code}"); + assert!(code.contains(".x = 1"), "{code}"); + assert!(!code.contains("static x"), "{code}"); + } + + #[test] + fn native_declaration_emits_suspending_class_key_prelude_in_order() { + let output = emit_at( + "function* g() { const left = before(), C = class { [yield key()]() {} }, right = after(); }\n", + ScriptTarget::Es2015, + ); + assert!(output.diagnostics.is_empty(), "{:?}", output.diagnostics); + let code = javascript(&output); + let positions = ["before()", "key()", "C =", "after()"].map(|needle| { + assert_eq!(code.matches(needle).count(), 1, "{needle}: {code}"); + code.find(needle).expect("emitted expression") + }); + assert!(positions.windows(2).all(|pair| pair[0] < pair[1]), "{code}"); + } + + #[test] + fn static_this_and_super_are_rebound_to_the_constructor() { + let output = emit_at( + "class B { static p() {} } class C extends B { static x = this; static { super.p(); } }\n", + ScriptTarget::Es2015, + ); let code = javascript(&output); - assert!(code.contains("constructor"), "{code}"); - assert!(code.contains("this.x = 1"), "{code}"); + assert!(code.contains("C.x = C"), "{code}"); assert!( - !code.lines().any(|line| line.trim() == "x = 1;"), - "field initializer should move: {code}" + code.contains("Reflect.get(Object.getPrototypeOf(C), \"p\", C).call(C)"), + "{code}" ); } + #[test] - fn require_locals_follow_the_tsc_slug_shape() { - // tsc binds `enum_1` for an identifier-starting slug and - // `_10_lib_1` when a leading digit forces sanitization. - let mut names = super::RequireNames::new(std::collections::BTreeSet::new()); - assert_eq!(names.allocate("./enum"), "enum_1"); - assert_eq!(names.allocate("./10_lib"), "_10_lib_1"); - assert_eq!(names.allocate("./enum"), "enum_2"); + fn static_block_only_source_is_detected_as_class_fields() { + let file = parse("class C { static { run(); } }\n"); + let options = options_for(ScriptTarget::Es2015).transform_view(); + let plan = analyze(&file, &options); + assert!(plan.features.contains(&LanguageFeature::ClassFields)); + assert!(plan.required.contains(&LanguageFeature::ClassFields)); } - #[test] - fn class_expression_without_lowering_stays_inline() { - // Authority: classExpressionTest2 — a generic class expression with - // no lowered members emits inline with type parameters stripped - // (`var m = class C {`), never a temp-IIFE wrap. + fn nested_static_blocks_rewrite_this_through_control_flow() { let output = emit_at( - "function M() {\n var m = class C { f() { return 1; } };\n return new m();\n}\n", + "class C { static { if (flag) { for (; once(); ) { this.x = this; break; } } try { throw this; } catch (e) { this.y = e; } finally { this.z = this; } } }\n", ScriptTarget::Es2015, ); let code = javascript(&output); - assert!(code.contains("var m = class C {"), "{code}"); - assert!(!code.contains("(() =>"), "{code}"); + assert!(code.contains("C.x = C"), "{code}"); + assert!(code.contains("throw C"), "{code}"); + assert!(code.contains("C.y = e"), "{code}"); + assert!(code.contains("C.z = C"), "{code}"); } #[test] - fn derived_class_fields_follow_super_and_literal_names_are_preserved() { + fn static_super_writes_use_reflect_with_single_evaluation() { let output = emit_at( - r#"class C extends B { "x" = 1; 2 = 3; constructor() { super(); after(); } } -"#, + "class B { static get x() { return 1; } static set x(v) {} } class C extends B { static { super.x = rhs(); super[key()] += more(); } }\n", ScriptTarget::Es2015, ); let code = javascript(&output); - let super_at = code - .find("super()") - .expect("derived constructor calls super"); - let string_at = code.find(r#"this["x"] = 1"#).expect("string field name"); - let number_at = code.find("this[2] = 3").expect("numeric field name"); - let after_at = code.find("after()").expect("original constructor body"); - assert!( - super_at < string_at && string_at < number_at && number_at < after_at, - "{code}" - ); + assert!(!code.contains("super.x ="), "{code}"); + assert!(!code.contains("super[key()]"), "{code}"); + assert_eq!(code.matches("key()").count(), 1, "{code}"); + assert_eq!(code.matches("rhs()").count(), 1, "{code}"); + assert_eq!(code.matches("more()").count(), 1, "{code}"); + assert!(code.matches("Reflect.set(").count() >= 2, "{code}"); + assert!(code.contains("Reflect.get("), "{code}"); } #[test] - fn synthesized_derived_constructor_forwards_arguments_before_fields() { - let output = emit_at("class C extends B { x = 1; }\n", ScriptTarget::Es2015); + fn node_executes_nested_static_super_writes() { + let source = r#" +const log = []; +function key() { log.push("key"); return "x"; } +function rhs() { log.push("rhs"); return 3; } +class B { + static get x() { log.push("get"); return this._x ?? 1; } + static set x(value) { log.push("set"); this._x = value; } +} +class C extends B { + static { + if (true) { + super.x = 2; + try { super[key()] += rhs(); this.self = this; } + finally { this.done = true; } + } + } +} +console.log(JSON.stringify([C._x, log, C.self === C, C.done])); +"#; + let output = emit_at(source, ScriptTarget::Es2015); let code = javascript(&output); - assert!(code.contains("super(...arguments)"), "{code}"); + let nonce = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("system clock") + .as_nanos(); + let path = std::env::temp_dir().join(format!("bamts-static-super-{nonce}.mjs")); + std::fs::write(&path, code).expect("write lowered JavaScript"); + let result = std::process::Command::new("node") + .arg(&path) + .output() + .expect("execute Node"); + let _ = std::fs::remove_file(&path); assert!( - code.find("super(...arguments)").unwrap() < code.find("this.x = 1").unwrap(), - "{code}" + result.status.success(), + "{}\n{code}", + String::from_utf8_lossy(&result.stderr) + ); + assert_eq!( + String::from_utf8(result.stdout).expect("Node stdout"), + "[5,[\"set\",\"key\",\"get\",\"rhs\",\"set\"],true,true]\n" ); } + #[test] - fn uninitialized_fields_emit_nothing_under_assignment_semantics() { - // Baseline computedPropertyNames36_ES5(target=es2015).js keeps - // `class Foo {\n}` for `class Foo { x }`: assignment semantics - // define nothing for a declaration without an initializer. - let output = emit_at("class C { x; static y; }\n", ScriptTarget::Es2015); - let code = javascript(&output); - assert!(!code.contains("constructor"), "{code}"); - assert!(!code.contains("void 0"), "{code}"); - assert!(code.contains("class C {"), "{code}"); + fn analysis_is_deterministic_and_lists_required_features() { + let file = parse("async function f() { const { a } = x; }\n"); + let options = options_for(ScriptTarget::Es5).transform_view(); + let left = analyze(&file, &options); + let right = analyze(&file, &options); + assert_eq!(left, right); + assert!(left.required.contains(&LanguageFeature::AsyncFunctions)); + assert!(left.required.contains(&LanguageFeature::Destructuring)); } #[test] - fn uninitialized_fields_define_void_0_under_define_semantics() { - let output = emit_with_options( - "class C { x; static y; }\n", + fn strict_prologue_follows_always_strict_and_module_kind() { + // A top-level external module takes the prologue exactly when it is + // transpiled to CommonJS; under ES module output it takes none even + // with alwaysStrict on (673 baselines). Scripts follow alwaysStrict + // alone, as conformance pairs like arrowFunctionContexts(alwaysstrict=...) + // show. + let module = emit_with_options( + "export const value = 1;\n", EmitOptions { - target: ScriptTarget::Es2015, - use_define_for_class_fields: Some(true), + module: Some(crate::emitter::ModuleKind::CommonJs), ..EmitOptions::default() }, ); - let code = javascript(&output); - assert!(code.contains("Object.defineProperty(this, \"x\""), "{code}"); - assert!(code.contains("Object.defineProperty(C, \"y\""), "{code}"); - assert!(code.contains("void 0"), "{code}"); - assert!(!code.contains("this.x ="), "{code}"); + assert!( + javascript(&module).starts_with("\"use strict\";\n"), + "a CommonJS external module implies strict: {}", + javascript(&module) + ); + let es_module = emit_with_options( + "export const value = 1;\n", + EmitOptions { + module: Some(crate::emitter::ModuleKind::Es2015), + always_strict: true, + ..EmitOptions::default() + }, + ); + assert!( + !javascript(&es_module).starts_with("\"use strict\";\n"), + "an ES external module takes no prologue even under alwaysStrict: {}", + javascript(&es_module) + ); + let commonjs_script = emit_with_options( + "const value = 1;\n", + EmitOptions { + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, + ); + assert!( + !javascript(&commonjs_script).starts_with("\"use strict\";\n"), + "a script under CommonJS still follows alwaysStrict: {}", + javascript(&commonjs_script) + ); + let script = emit_with_options( + "const value = 1;\n", + EmitOptions { + ..EmitOptions::default() + }, + ); + assert!( + !javascript(&script).starts_with("\"use strict\";\n"), + "a plain script has no prologue: {}", + javascript(&script) + ); + let always = emit_with_options( + "const value = 1;\n", + EmitOptions { + always_strict: true, + ..EmitOptions::default() + }, + ); + assert!( + javascript(&always).starts_with("\"use strict\";\n"), + "alwaysStrict emits the prologue: {}", + javascript(&always) + ); + let deduped = emit_with_options( + "\"use strict\";\nconst value = 1;\n", + EmitOptions { + always_strict: true, + ..EmitOptions::default() + }, + ); + let code = javascript(&deduped); + assert_eq!( + code.matches("\"use strict\";").count(), + 1, + "a source directive suppresses the synthetic one: {code}" + ); } #[test] - fn computed_field_names_are_evaluated_once_at_class_definition() { - let output = emit_at( - "class C { [key()] = 1; } new C(); new C();\n", - ScriptTarget::Es2015, + fn ambient_enum_and_module_produce_no_runtime_statements() { + // Upstream ambientEnumElementInitializer5 (43527a9c) emits only the + // strict prologue; ambientModules (df2119af) keeps only the trailing + // member assignment. + let ambient_enum = emit_at("declare enum E { e = -0xA }\n", ScriptTarget::Es2015); + assert_eq!( + javascript(&ambient_enum), + "", + "{:?}", + javascript(&ambient_enum) ); - let code = javascript(&output); - assert_eq!(code.matches("key()").count(), 1, "{code}"); - assert!(code.contains("let _t0 = key()"), "{code}"); - assert!(code.contains("this[_t0] = 1"), "{code}"); - } - #[test] - fn computed_keys_preserve_heritage_and_member_evaluation_order() { - let output = emit_at( - "class C extends heritage() { [field()] = 1; [method()]() {} [later()] = 2; }\n", + + let ambient_module = emit_at( + "declare namespace Foo.Bar { export var foo; };\nFoo.Bar.foo = 5;\n", ScriptTarget::Es2015, ); - let code = javascript(&output); - let heritage = code.find("heritage()").expect("heritage evaluation"); - let field = code.find("field()").expect("first computed field"); - let method = code.find("method()").expect("computed method"); - let later = code.find("later()").expect("later computed field"); - let class = code.find("class C extends _t0").expect("rewritten class"); + // The ambient declaration vanishes; the runtime assignment survives, + // matching upstream ambientModules (df2119af) after the prologue. assert!( - heritage < field && field < method && method < later && later < class, - "{code}" + !javascript(&ambient_module).starts_with("declare") + && !javascript(&ambient_module).contains("export var foo"), + "the ambient block is erased: {}", + javascript(&ambient_module) ); - assert!(code.contains("this[_t1] = 1"), "{code}"); - assert!(code.contains("[_t2]()"), "{code}"); - assert!(code.contains("this[_t3] = 2"), "{code}"); - } - #[test] - fn static_computed_field_runs_key_and_value_and_assigns_the_property() { - let output = emit_at( - "class C { static [key()] = value(); }\n", - ScriptTarget::Es2015, + + let runtime = emit_at("const enum K { A }\nlet kept = 1;\n", ScriptTarget::Es2015); + assert!( + javascript(&runtime).contains("kept"), + "non-ambient code survives: {}", + javascript(&runtime) ); - let code = javascript(&output); - assert_eq!(code.matches("key()").count(), 1, "{code}"); - assert_eq!(code.matches("value()").count(), 1, "{code}"); - let key = code.find("let _t0 = key()").expect("key setup"); - let class = code.find("class C").expect("class declaration"); - let assignment = code - .find("C[_t0] = value()") - .expect("post-class static property assignment"); - assert!(key < class && class < assignment, "{code}"); } + #[test] - fn named_export_wraps_only_the_lowered_class() { + fn generic_call_type_arguments_are_erased() { + // Upstream genericCallWithNonGenericArgs1 (e71ae63c): f(null) + // emits f(null); let output = emit_at( - "export class C { static [key()] = value(); }\n", + "function f(x: any) { }\nf(null);\n", ScriptTarget::Es2015, ); let code = javascript(&output); - let prelude = code.find("let _t0 = key()").expect("ordinary key prelude"); - let class = code - .find("export class C") - .expect("exported rewritten class"); - let postlude = code - .find("C[_t0] = value()") - .expect("ordinary static postlude"); - assert!(prelude < class && class < postlude, "{code}"); - assert!(!code.contains("export let _t0"), "{code}"); - assert!(!code.contains("export C[_t0]"), "{code}"); + assert!(code.contains("f(null);"), "{code}"); + assert!(!code.contains(""), "{code}"); } #[test] - fn named_default_export_keeps_class_and_generated_statements_separate() { - let output = emit_at( - "export default class C { field = 1; static [key()] = value(); }\n", - ScriptTarget::Es2015, + fn commonjs_named_import_and_function_export_match_upstream_shape() { + // Upstream commonjsSafeImport(target=es2015).js + // (282283386bc4040f0eba585ac33e142c5c30cfbb67f969b1706c655dc7bee408). + let export = emit_with_options( + "export function Foo() {}\n", + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, ); - let code = javascript(&output); - let prelude = code.find("let _t0 = key()").expect("ordinary key prelude"); - let class = code - .find("export default class C") - .expect("named default class"); - let postlude = code - .find("C[_t0] = value()") - .expect("ordinary static postlude"); - assert!(prelude < class && class < postlude, "{code}"); - assert!(code.contains("this.field = 1"), "{code}"); - assert_eq!(code.matches("key()").count(), 1, "{code}"); - assert_eq!(code.matches("value()").count(), 1, "{code}"); - assert!(!code.contains("export default let"), "{code}"); + let export_code = javascript(&export); + assert!( + export_code.starts_with( + "\"use strict\";\nObject.defineProperty(exports, \"__esModule\", { value: true });\nexports.Foo = Foo;\nfunction Foo()" + ), + "{export_code}" + ); + + let import = emit_with_options( + "import { Foo } from './10_lib';\nFoo();\n", + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, + ); + let import_code = javascript(&import); + assert!( + import_code.starts_with( + "\"use strict\";\nObject.defineProperty(exports, \"__esModule\", { value: true });\nconst _10_lib_1 = require(\"./10_lib\");\n" + ), + "{import_code}" + ); + assert!( + import_code.contains("(0, _10_lib_1.Foo)();"), + "{import_code}" + ); + assert!(!import_code.contains("import "), "{import_code}"); } #[test] - fn anonymous_default_export_lowers_instance_fields_in_place() { + fn es5_lowers_nested_computed_destructuring_assignment() { + // Upstream computedPropertiesInDestructuring1.js + // (50845e031ebf8203c61d4bfb72d37eea703fea245df4e215c7b5772583a8a898). let output = emit_at( - "export default class { field = 1; }\n", - ScriptTarget::Es2015, + "let foo = 'bar', bar4; [{ [foo]: bar4 }] = [{ bar: 'bar' }];\n", + ScriptTarget::Es5, ); let code = javascript(&output); - assert!(code.contains("export default class"), "{code}"); - assert!(code.contains("constructor"), "{code}"); - assert!(code.contains("this.field = 1"), "{code}"); assert!( !output .diagnostics .iter() - .any(|diagnostic| { diagnostic.code() == codes::STATIC_FIELD_REQUIRES_CLASS_NAME }), + .any(|diagnostic| diagnostic.code() + == codes::DESTRUCTURING_ASSIGNMENT_REQUIRES_ES2015), "{:?}", output.diagnostics ); + assert!(code.contains("bar4 ="), "{code}"); + assert!(code.contains("[foo]"), "{code}"); + assert!(!code.contains("[{ [foo]: bar4 }] ="), "{code}"); } #[test] - fn anonymous_default_static_field_uses_one_synthetic_class_target() { - let output = emit_at( - "export default class { static [key()] = value(); }\n", - ScriptTarget::Es2015, + fn commonjs_class_and_variable_exports_receive_void_0_preamble() { + // Upstream importHelpersWithImportStarAs a.js (a2836e68) emits + // `exports.A = void 0;` + class + `exports.A = A;`; the variable form + // follows es6ImportDefaultBindingWithExport client.js (adfd4388). + let output = emit_with_options( + "export class A {}\nexport const x = 1;\n", + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, ); let code = javascript(&output); - let prelude = code.find("let _t1 = key()").expect("ordinary key prelude"); - let class = code - .find("export default class _t0") - .expect("synthetically named default class"); - let postlude = code - .find("_t0[_t1] = value()") - .expect("static assignment uses synthetic class name"); - assert!(prelude < class && class < postlude, "{code}"); - assert_eq!(code.matches("key()").count(), 1, "{code}"); - assert_eq!(code.matches("value()").count(), 1, "{code}"); + assert!( + code.starts_with( + "\"use strict\";\nObject.defineProperty(exports, \"__esModule\", { value: true });\nexports.A = exports.x = void 0;\n" + ), + "{code}" + ); + let marker = code + .find("exports.A = exports.x = void 0;") + .expect("preamble"); + let class_at = code.find("class A").expect("class kept"); + let class_export = code.find("exports.A = A;").expect("class export"); + let variable = code.find("const x = 1;").expect("variable kept"); + let variable_export = code.find("exports.x = x;").expect("variable export"); + assert!( + marker < class_at + && class_at < class_export + && class_export < variable + && variable < variable_export, + "{code}" + ); } + #[test] - fn anonymous_default_static_class_restores_imported_name_to_default() { - let output = emit_at( - "export default class { static x = 1; }\n", - ScriptTarget::Es2015, + fn commonjs_local_specifier_exports_assign_each_name() { + // Same `exports.NAME = LOCAL;` identifier-assignment form as the + // identifier default export in es6ImportDefaultBindingWithExport + // server.js (adfd4388): `exports.default = a;`. + let output = emit_with_options( + "const a = 1, b = 2;\nexport { a, b as c };\n", + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, ); let code = javascript(&output); - let class = code - .find("export default class _t0") - .expect("synthetically named default class"); - let restore = code - .find(r#"Object.defineProperty(_t0, "name""#) - .expect("default name restoration"); - let static_assignment = code - .find("_t0.x = 1") - .expect("post-class static assignment"); - assert!(class < restore && restore < static_assignment, "{code}"); - assert!(code.contains(r#"value: "default""#), "{code}"); - assert!(code.contains("configurable: true"), "{code}"); + assert!(code.contains("exports.a = a;"), "{code}"); + assert!(code.contains("exports.c = b;"), "{code}"); + assert!(!code.contains("export {"), "{code}"); } #[test] - fn es2015_downlevels_async_to_awaiter() { - // Authority: es5-importHelpersAsyncFunctions(target=es2015).js emits - // the awaiter alone — native generators need no __generator. + fn a_namespace_fills_its_container_through_a_call() { let output = emit_at( - "async function f(x) { return await x; }\n", + "namespace A {\n\ + \x20 // a comment\n\ + \x20 export var x = 1, y;\n\ + \x20 var hidden = x;\n\ + \x20 export function f() { return x; }\n\ + \x20 export class C {}\n\ + \x20 export namespace B {\n\ + \x20 export var z = x + 1;\n\ + \x20 }\n\ + }\n\ + namespace A {\n\ + \x20 x = 2;\n\ + \x20 x++;\n\ + \x20 const box = { x };\n\ + \x20 f();\n\ + }\n", ScriptTarget::Es2015, ); - let code = javascript(&output); - assert!(code.contains("__awaiter"), "{code}"); - assert!(code.contains("function*"), "{code}"); - assert!(code.contains("yield"), "{code}"); - assert!(!code.contains("async "), "{code}"); - assert!( - code.contains("var __awaiter = (this && this.__awaiter) || function"), - "{code}" + assert_eq!( + javascript(&output), + "var A;\n\ + (function (A) {\n\ + \x20 // a comment\n\ + \x20 A.x = 1;\n\ + \x20 var hidden = A.x;\n\ + \x20 function f() { return A.x; }\n\ + \x20 A.f = f;\n\ + \x20 class C {\n\ + \x20 }\n\ + \x20 A.C = C;\n\ + \x20 let B;\n\ + \x20 (function (B) {\n\ + \x20 B.z = A.x + 1;\n\ + \x20 })(B = A.B || (A.B = {}));\n\ + })(A || (A = {}));\n\ + (function (A) {\n\ + \x20 A.x = 2;\n\ + \x20 A.x++;\n\ + \x20 const box = { x: A.x };\n\ + \x20 A.f();\n\ + })(A || (A = {}));\n" ); - assert!(!code.contains("__generator"), "{code}"); } #[test] - fn helper_emission_uses_rewriter_demand_not_analysis_prediction() { - // Authority: es5-importHelpersAsyncFunctions(target=es2015).js. - let source = "function outer() { return async () => await 1; }\n"; - let file = parse(source); - let checked = checker::check(&Recovered::clean(parse(source))); - let options = options_for(ScriptTarget::Es2015); - let transform_options = options.transform_view(); - let plan = analyze(&file, &transform_options); - assert!( - plan.helpers.is_empty(), - "fixture must remain outside the shallow analysis walk" - ); - - let output = emit_transformed(&file, checked.product(), &transform_options, &names()); - let code = javascript(&output); - assert!( - code.contains("var __awaiter = (this && this.__awaiter) || function"), - "{code}" + fn an_enum_is_declared_once_and_published_like_a_namespace() { + let output = emit_at( + "enum E { A }\n\ + enum E { B = 1 }\n\ + namespace N {\n\ + \x20 export enum F { C }\n\ + \x20 enum G { D }\n\ + }\n\ + function f() {\n\ + \x20 enum H { I }\n\ + }\n", + ScriptTarget::Es2015, ); - assert!( - !code.contains("__generator"), - "native-generator targets emit no __generator: {code}" + assert_eq!( + javascript(&output), + "var E;\n\ + (function (E) {\n\ + \x20 E[E[\"A\"] = 0] = \"A\";\n\ + })(E || (E = {}));\n\ + (function (E) {\n\ + \x20 E[E[\"B\"] = 1] = \"B\";\n\ + })(E || (E = {}));\n\ + var N;\n\ + (function (N) {\n\ + \x20 let F;\n\ + \x20 (function (F) {\n\ + \x20 F[F[\"C\"] = 0] = \"C\";\n\ + \x20 })(F = N.F || (N.F = {}));\n\ + \x20 let G;\n\ + \x20 (function (G) {\n\ + \x20 G[G[\"D\"] = 0] = \"D\";\n\ + \x20 })(G || (G = {}));\n\ + })(N || (N = {}));\n\ + function f() {\n\ + \x20 let H;\n\ + \x20 (function (H) {\n\ + \x20 H[H[\"I\"] = 0] = \"I\";\n\ + \x20 })(H || (H = {}));\n\ + }\n" ); } #[test] - fn es5_downlevels_object_destructuring() { - let output = emit_at("const { a, b } = o;\n", ScriptTarget::Es5); - let code = javascript(&output); - assert!(code.contains("a = "), "{code}"); - assert!(code.contains("b = "), "{code}"); - assert!(!code.contains("{ a, b }"), "{code}"); - } - - #[test] - fn private_fields_below_es2022_are_errors() { - let output = emit_at("class C { #x = 1; }\n", ScriptTarget::Es2015); - assert!( - output - .diagnostics - .iter() - .any(|diagnostic| diagnostic.code() == codes::PRIVATE_FIELD_REQUIRES_ES2022), - "{:?}", - output.diagnostics + fn an_exported_enum_publishes_its_container() { + let source = "export enum E { A }\nexport const enum F { B }\n"; + let esm = emit_at(source, ScriptTarget::Es2015); + assert_eq!( + javascript(&esm), + "export var E;\n(function (E) {\n E[E[\"A\"] = 0] = \"A\";\n})(E || (E = {}));\n" + ); + let cjs = emit_with_options( + source, + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, + ); + assert_eq!( + javascript(&cjs), + "\"use strict\";\n\ + Object.defineProperty(exports, \"__esModule\", { value: true });\n\ + exports.E = void 0;\n\ + var E;\n\ + (function (E) {\n\ + \x20 E[E[\"A\"] = 0] = \"A\";\n\ + })(E || (exports.E = E = {}));\n" ); } #[test] - fn classes_below_es2015_are_errors() { - let output = emit_at("class C {}\n", ScriptTarget::Es5); - assert!( - output - .diagnostics - .iter() - .any(|diagnostic| diagnostic.code() == codes::CLASS_REQUIRES_ES2015), - "{:?}", - output.diagnostics + fn an_alias_is_declared_only_where_it_holds_a_value() { + // Upstream aliasBug, unusedImports10 and + // usingModuleWithExportImportInValuePosition. + let output = emit_at( + "namespace A { export var x = 1; export interface I {} }\n\ + import top = A;\n\ + namespace B { import unused = A; export var y = 1; }\n\ + namespace C { import read = A; export var z = read.x; }\n\ + namespace D { export import published = A; }\n\ + namespace E { export import shape = A.I; export var w = 1; }\n", + ScriptTarget::Es2015, + ); + assert_eq!( + javascript(&output), + "var A;\n\ + (function (A) {\n\ + \x20 A.x = 1;\n\ + })(A || (A = {}));\n\ + var top = A;\n\ + var B;\n\ + (function (B) {\n\ + \x20 B.y = 1;\n\ + })(B || (B = {}));\n\ + var C;\n\ + (function (C) {\n\ + \x20 var read = A;\n\ + \x20 C.z = read.x;\n\ + })(C || (C = {}));\n\ + var D;\n\ + (function (D) {\n\ + \x20 D.published = A;\n\ + })(D || (D = {}));\n\ + var E;\n\ + (function (E) {\n\ + \x20 E.w = 1;\n\ + })(E || (E = {}));\n" ); } #[test] - fn using_declarations_are_errors_before_esnext() { - let output = emit_at("using x = y;\n", ScriptTarget::Es2022); - assert!( - output - .diagnostics - .iter() - .any(|diagnostic| diagnostic.code() == codes::USING_UNSUPPORTED), - "{:?}", - output.diagnostics + fn a_namespace_member_is_rewritten_wherever_it_is_used() { + let output = emit_at( + "namespace N {\n\ + \x20 export var x = 1;\n\ + \x20 export class C {\n\ + \x20 constructor(public p = x) {}\n\ + \x20 m(q = x) { return () => x + q; }\n\ + \x20 }\n\ + \x20 for (x of [1]) {}\n\ + \x20 for (x in {}) {}\n\ + \x20 function shadowed(x: number) { return x; }\n\ + }\n", + ScriptTarget::Es2015, ); + let code = javascript(&output); + for expected in [ + "constructor(p = N.x)", + "m(q = N.x)", + "return () => N.x + q;", + "for (N.x of [1])", + "for (N.x in {})", + "function shadowed(x) { return x; }", + ] { + assert!(code.contains(expected), "missing {expected}: {code}"); + } } #[test] - fn static_fields_and_blocks_remain_interleaved() { + fn a_name_is_rewritten_in_loop_heads_and_binding_patterns() { let output = emit_at( - "class C { static a = one(); static { step(); } static b = two(); }\n", + "namespace N {\n\ + \x20 export var n = 3;\n\ + \x20 for (let i = n; i > 0; i--) {}\n\ + \x20 for (const { a = n } of []) {}\n\ + \x20 for (const k in { [n]: 1 }) {}\n\ + \x20 function f({ a = n }, [b = n]) {}\n\ + \x20 const { c = n, [n]: d } = {};\n\ + }\n", ScriptTarget::Es2015, ); let code = javascript(&output); - let first = code.find("C.a = one()").expect("first static field"); - let block = code.find("step()").expect("static block"); - let second = code.find("C.b = two()").expect("second static field"); - assert!(first < block && block < second, "{code}"); - assert!(!code.contains("static {"), "{code}"); + for expected in [ + "for (let i = N.n; i > 0; i--)", + "for (const { a = N.n } of [])", + "[N.n]: 1", + "function f({ a = N.n }, [b = N.n])", + "const { c = N.n, [N.n]: d } = {};", + ] { + assert!(code.contains(expected), "missing {expected}: {code}"); + } } #[test] - fn define_mode_uses_complete_data_descriptors() { - let output = emit_with_options( - "class C { x = 1; static y = 2; }\n", - EmitOptions { - target: ScriptTarget::Es2015, - use_define_for_class_fields: Some(true), - ..EmitOptions::default() - }, + fn an_unknown_name_in_a_binding_pattern_is_reported() { + let source = "const { a = missing } = {};\nconst [b = alsoMissing] = [];\n"; + let checked = checker::check(&Recovered::clean(parse(source))); + let unknown = checked + .diagnostics() + .iter() + .filter(|diagnostic| diagnostic.code() == crate::checker::CANNOT_FIND_NAME) + .count(); + assert_eq!(unknown, 2, "{:?}", checked.diagnostics()); + } + + #[test] + fn a_const_enum_member_is_inlined() { + let output = emit_at( + "const enum K { A = 1 }\nconsole.log(K.A);\nconsole.log(K['A']);\n", + ScriptTarget::Es2015, ); let code = javascript(&output); - assert!(code.contains("Object.defineProperty(this, \"x\""), "{code}"); - assert!(code.contains("Object.defineProperty(C, \"y\""), "{code}"); - let enumerable = code - .find("enumerable: true") - .expect("enumerable descriptor"); - let configurable = code - .find("configurable: true") - .expect("configurable descriptor"); - let writable = code.find("writable: true").expect("writable descriptor"); - let value = code.find("value: 1").expect("descriptor value"); - assert!( - enumerable < configurable && configurable < writable && writable < value, - "{code}" - ); + assert!(!code.contains("K.A)"), "{code}"); + assert!(!code.contains("K['A']"), "{code}"); + assert!(code.contains("console.log(1"), "{code}"); } #[test] - fn synthesized_descriptor_object_expands_in_javascript() { - // Authority: override18(target=es2015).js — the defineProperty - // descriptor is wholly synthesized and prints multi-line even when - // the authored source is compact. An untainted mint collapsed it - // onto one line (sweep-2d76b09 regression). - let output = emit_with_options( - "class A { foo?: string; }\n", - EmitOptions { - target: ScriptTarget::Es2015, - use_define_for_class_fields: Some(true), - ..EmitOptions::default() - }, + fn a_published_alias_is_read_from_its_container() { + let output = emit_at( + "namespace A { export var x = 1; }\n\ + namespace C { export var v = b; export import b = A.x; export var w = b; }\n", + ScriptTarget::Es2015, ); let code = javascript(&output); - assert!( - code.contains("Object.defineProperty(this, \"foo\", {\n enumerable: true,"), - "{code}" - ); - assert!(!code.contains("{ enumerable: true, configurable"), "{code}"); + for expected in ["C.v = C.b;", "C.b = A.x;", "C.w = C.b;"] { + assert!(code.contains(expected), "missing {expected}: {code}"); + } } #[test] - fn declare_and_abstract_fields_have_no_runtime_initializers() { + fn a_namespace_without_a_value_emits_nothing() { let output = emit_at( - "abstract class C { declare x: number; abstract y: string; z = 1; }\n", + "namespace T { export interface I {} }\n\ + namespace U { import a = T; }\n\ + namespace V { export const enum E { A } }\n", ScriptTarget::Es2015, ); - let code = javascript(&output); - assert!(!code.contains("this.x"), "{code}"); - assert!(!code.contains("this.y"), "{code}"); - assert!(code.contains("this.z = 1"), "{code}"); + assert_eq!(javascript(&output), ""); } #[test] - fn nested_super_sites_receive_fields_after_each_completion() { + fn a_namespace_merged_into_a_class_uses_its_binding() { let output = emit_at( - "class C extends B { x = 1; constructor(flag) { if (flag) super(1); else { super(2); } } }\n", + "class C {}\nnamespace C { export var x = 1; }\n", ScriptTarget::Es2015, ); - let code = javascript(&output); - assert_eq!(code.matches("this.x = 1").count(), 2, "{code}"); - let first_super = code.find("super(1)").expect("first super"); - let first_init = code.find("this.x = 1").expect("first init"); - assert!(first_super < first_init, "{code}"); + assert_eq!( + javascript(&output), + "class C {\n}\n(function (C) {\n C.x = 1;\n})(C || (C = {}));\n" + ); } #[test] - fn class_expressions_lower_inside_call_arguments() { + fn a_namespace_whose_body_shadows_its_name_renames_the_parameter() { let output = emit_at( - "consume(class { static x = 1; y = 2; });\n", + "namespace M { export var M = 1; export var y = M; }\n", ScriptTarget::Es2015, ); - let code = javascript(&output); - assert!(code.contains("consume((() =>"), "{code}"); - assert!(code.contains("this.y = 2"), "{code}"); - assert!(code.contains(".x = 1"), "{code}"); - assert!(!code.contains("static x"), "{code}"); + assert_eq!( + javascript(&output), + "var M;\n(function (M_1) {\n M_1.M = 1;\n M_1.y = M_1.M;\n})(M || (M = {}));\n" + ); } #[test] - fn native_declaration_emits_suspending_class_key_prelude_in_order() { + fn an_exported_pattern_assigns_each_name_onto_the_container() { + // Upstream declarationEmitDestructuringArrayPattern3 and + // downlevelLetConst13: one read needs no temporary. let output = emit_at( - "function* g() { const left = before(), C = class { [yield key()]() {} }, right = after(); }\n", + "namespace M {\n\ + \x20 export var [a, b] = [1, 2];\n\ + \x20 export let [c] = [1];\n\ + \x20 export const { d: e } = { d: 1 };\n\ + }\n", ScriptTarget::Es2015, ); - assert!(output.diagnostics.is_empty(), "{:?}", output.diagnostics); - let code = javascript(&output); - let positions = ["before()", "key()", "C =", "after()"].map(|needle| { - assert_eq!(code.matches(needle).count(), 1, "{needle}: {code}"); - code.find(needle).expect("emitted expression") - }); - assert!(positions.windows(2).all(|pair| pair[0] < pair[1]), "{code}"); + assert_eq!( + javascript(&output), + "var M;\n\ + (function (M) {\n\ + \x20 var _a;\n\ + \x20 _a = [1, 2], M.a = _a[0], M.b = _a[1];\n\ + \x20 M.c = [1][0];\n\ + \x20 M.e = { d: 1 }.d;\n\ + })(M || (M = {}));\n" + ); } #[test] - fn static_this_and_super_are_rebound_to_the_constructor() { + fn an_exported_pattern_collects_its_rest() { let output = emit_at( - "class B { static p() {} } class C extends B { static x = this; static { super.p(); } }\n", + "namespace M {\n\ + \x20 export var [a, ...b] = [1, 2, 3];\n\ + \x20 export var { c, ...d } = { c: 1, e: 2 };\n\ + }\n", ScriptTarget::Es2015, ); let code = javascript(&output); - assert!(code.contains("C.x = C"), "{code}"); - assert!( - code.contains("Reflect.get(Object.getPrototypeOf(C), \"p\", C).call(C)"), - "{code}" + for expected in [ + "_a = [1, 2, 3], M.a = _a[0], M.b = _a.slice(1);", + "_b = { c: 1, e: 2 }, M.c = _b.c, M.d = __rest(_b, [\"c\"]);", + ] { + assert!(code.contains(expected), "missing {expected}: {code}"); + } + } + + #[test] + fn an_exported_namespace_publishes_its_container() { + let source = "export namespace M { export var x = 1; }\n"; + let esm = emit_at(source, ScriptTarget::Es2015); + assert_eq!( + javascript(&esm), + "export var M;\n(function (M) {\n M.x = 1;\n})(M || (M = {}));\n" + ); + let cjs = emit_with_options( + source, + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, + ); + assert_eq!( + javascript(&cjs), + "\"use strict\";\n\ + Object.defineProperty(exports, \"__esModule\", { value: true });\n\ + exports.M = void 0;\n\ + var M;\n\ + (function (M) {\n\ + \x20 M.x = 1;\n\ + })(M || (exports.M = M = {}));\n" ); } #[test] - fn static_block_only_source_is_detected_as_class_fields() { - let file = parse("class C { static { run(); } }\n"); - let options = options_for(ScriptTarget::Es2015).transform_view(); - let plan = analyze(&file, &options); - assert!(plan.features.contains(&LanguageFeature::ClassFields)); - assert!(plan.required.contains(&LanguageFeature::ClassFields)); + fn commonjs_keeps_an_exported_variable_on_exports() { + // Upstream downlevelLetConst13 and internalAliasVarInsideTopLevelModuleWithExport. + let output = emit_with_options( + "export var a = 1, b;\n\ + export let [c] = [1];\n\ + a = 2;\n\ + a++;\n\ + const box = { a };\n\ + a();\n\ + function f(a: number) { return a; }\n", + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, + ); + let code = javascript(&output); + let body = code + .split_once("void 0;\n") + .map(|(_, body)| body) + .expect("the preamble declares the exports"); + assert_eq!( + body, + "exports.a = 1;\n\ + exports.c = [1][0];\n\ + exports.a = 2;\n\ + exports.a++;\n\ + const box = { a: exports.a };\n\ + (0, exports.a)();\n\ + function f(a) { return a; }\n" + ); + for name in ["exports.a", "exports.b", "exports.c"] { + let preamble = code.split("void 0;").next().expect("split yields a head"); + assert!(preamble.contains(name), "{code}"); + } } + #[test] - fn nested_static_blocks_rewrite_this_through_control_flow() { - let output = emit_at( - "class C { static { if (flag) { for (; once(); ) { this.x = this; break; } } try { throw this; } catch (e) { this.y = e; } finally { this.z = this; } } }\n", - ScriptTarget::Es2015, + fn commonjs_export_specifiers_read_what_the_name_became() { + let output = emit_with_options( + "import { x } from './m';\n\ + import y from './n';\n\ + export var a = 1;\n\ + const local = 2;\n\ + export { a as b, x, y, local };\n", + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, ); let code = javascript(&output); - assert!(code.contains("C.x = C"), "{code}"); - assert!(code.contains("throw C"), "{code}"); - assert!(code.contains("C.y = e"), "{code}"); - assert!(code.contains("C.z = C"), "{code}"); + for expected in [ + "exports.b = exports.a;", + "Object.defineProperty(exports, \"x\", { enumerable: true, get: function () { return m_1.x; } });", + "exports.y = n_1.default;", + "exports.local = local;", + ] { + assert!(code.contains(expected), "missing {expected}: {code}"); + } } #[test] - fn static_super_writes_use_reflect_with_single_evaluation() { - let output = emit_at( - "class B { static get x() { return 1; } static set x(v) {} } class C extends B { static { super.x = rhs(); super[key()] += more(); } }\n", - ScriptTarget::Es2015, + fn commonjs_reads_a_published_alias_from_exports() { + let output = emit_with_options( + "export namespace a { export var x = 10; }\n\ + export import b = a.x;\n\ + export var c = b;\n", + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, ); let code = javascript(&output); - assert!(!code.contains("super.x ="), "{code}"); - assert!(!code.contains("super[key()]"), "{code}"); - assert_eq!(code.matches("key()").count(), 1, "{code}"); - assert_eq!(code.matches("rhs()").count(), 1, "{code}"); - assert_eq!(code.matches("more()").count(), 1, "{code}"); - assert!(code.matches("Reflect.set(").count() >= 2, "{code}"); - assert!(code.contains("Reflect.get("), "{code}"); + assert!(code.contains("exports.b = a.x;"), "{code}"); + assert!(code.contains("exports.c = exports.b;"), "{code}"); } #[test] - fn node_executes_nested_static_super_writes() { - let source = r#" -const log = []; -function key() { log.push("key"); return "x"; } -function rhs() { log.push("rhs"); return 3; } -class B { - static get x() { log.push("get"); return this._x ?? 1; } - static set x(value) { log.push("set"); this._x = value; } -} -class C extends B { - static { - if (true) { - super.x = 2; - try { super[key()] += rhs(); this.self = this; } - finally { this.done = true; } + fn commonjs_imports_stay_in_source_order_among_statements() { + let output = emit_with_options( + "// leading\n\ + import './s';\n\ + globalThis.a = 1;\n\ + import { q } from './m';\n\ + console.log(q);\n\ + export * from './n';\n\ + import './t';\n", + EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, + ); + let code = javascript(&output); + let order = [ + "// leading", + "require(\"./s\");", + "globalThis.a = 1;", + "const m_1 = require(\"./m\");", + "console.log(m_1.q);", + "__exportStar(require(\"./n\"), exports);", + "require(\"./t\");", + ]; + let positions: Vec = order + .iter() + .map(|line| code.find(line).unwrap_or_else(|| panic!("{line}: {code}"))) + .collect(); + assert!(positions.is_sorted(), "{code}"); } - } -} -console.log(JSON.stringify([C._x, log, C.self === C, C.done])); -"#; - let output = emit_at(source, ScriptTarget::Es2015); + + #[test] + fn commonjs_keeps_an_exported_local_in_step_with_exports() { + let output = emit_with_options( + "export { x };\n\ + let x = 1, y = 2, z = 3;\n\ + export { y as w, z };\n\ + x = 5;\n\ + y += 1;\n\ + ++z;\n\ + z++;\n\ + const old = z--;\n\ + [x] = [7];\n", + EmitOptions { + target: ScriptTarget::Es2022, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, + ); let code = javascript(&output); - let nonce = std::time::SystemTime::now() - .duration_since(std::time::UNIX_EPOCH) - .expect("system clock") - .as_nanos(); - let path = std::env::temp_dir().join(format!("bamts-static-super-{nonce}.mjs")); - std::fs::write(&path, code).expect("write lowered JavaScript"); - let result = std::process::Command::new("node") - .arg(&path) - .output() - .expect("execute Node"); - let _ = std::fs::remove_file(&path); - assert!( - result.status.success(), - "{}\n{code}", - String::from_utf8_lossy(&result.stderr) + let declared = code.find("let x = 1").expect("declaration"); + for line in ["exports.x = x;", "exports.w = y;", "exports.z = z;"] { + let published = code.find(line).unwrap_or_else(|| panic!("{line}: {code}")); + assert!(published > declared, "{line}: {code}"); + } + for line in [ + "exports.x = x = 5;", + "exports.w = y += 1;", + "exports.z = ++z;", + "exports.z = (z++, z);", + ", exports.x = x, ", + ] { + assert!(code.contains(line), "{line}: {code}"); + } + assert!(code.contains("exports.z = (_"), "{code}"); + } + + #[test] + fn export_assignment_closes_the_module_and_an_ambient_publishes_nothing() { + let commonjs = EmitOptions { + target: ScriptTarget::Es2022, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }; + let output = emit_with_options("export = C;\nclass C {}\n", commonjs.clone()); + let code = javascript(&output); + assert!(code.ends_with("module.exports = C;\n"), "{code}"); + let ambient = emit_with_options( + "declare const q: number;\nexport { q };\n", + commonjs.clone(), ); - assert_eq!( - String::from_utf8(result.stdout).expect("Node stdout"), - "[5,[\"set\",\"key\",\"get\",\"rhs\",\"set\"],true,true]\n" + let code = javascript(&ambient); + assert!(code.contains("exports.q = void 0;"), "{code}"); + assert!(!code.contains("exports.q = q"), "{code}"); + let parameter = emit_with_options( + "import { x } from './m';\nclass C { constructor(public a = x) {} }\nnew C();\n", + commonjs, ); + let code = javascript(¶meter); + assert!(code.contains("a = m_1.x"), "{code}"); } #[test] - fn analysis_is_deterministic_and_lists_required_features() { - let file = parse("async function f() { const { a } = x; }\n"); - let options = options_for(ScriptTarget::Es5).transform_view(); - let left = analyze(&file, &options); - let right = analyze(&file, &options); - assert_eq!(left, right); - assert!(left.required.contains(&LanguageFeature::AsyncFunctions)); - assert!(left.required.contains(&LanguageFeature::Destructuring)); + fn export_assignment_replaces_commonjs_exports() { + let output = emit_with_options( + "const K = 1;\nexport = K;\n", + EmitOptions { + target: ScriptTarget::Es2022, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, + ); + assert_eq!( + javascript(&output), + "\"use strict\";\nconst K = 1;\nmodule.exports = K;\n" + ); } #[test] - fn strict_prologue_follows_always_strict_and_module_kind() { - // A top-level external module takes the prologue exactly when it is - // transpiled to CommonJS; under ES module output it takes none even - // with alwaysStrict on (673 baselines). Scripts follow alwaysStrict - // alone, as conformance pairs like arrowFunctionContexts(alwaysstrict=...) - // show. - let module = emit_with_options( - "export const value = 1;\n", + fn string_export_names_quote_their_decoded_text() { + let output = emit_with_options( + "import { \"a b\" as ab } from './m';\n\ + const v = ab;\n\ + export { v as 'quo\"te', v as \"back\\\\slash\", v as \"x²\" };\n\ + export { q as \"c d\" } from './n';\n", EmitOptions { + target: ScriptTarget::Es2022, module: Some(crate::emitter::ModuleKind::CommonJs), ..EmitOptions::default() }, ); - assert!( - javascript(&module).starts_with("\"use strict\";\n"), - "a CommonJS external module implies strict: {}", - javascript(&module) - ); - let es_module = emit_with_options( - "export const value = 1;\n", + let code = javascript(&output); + for line in [ + "const v = m_1[\"a b\"];", + "exports[\"quo\\\"te\"] = v;", + "exports[\"back\\\\slash\"] = v;", + "exports[\"x²\"] = v;", + "Object.defineProperty(exports, \"c d\", {", + ] { + assert!(code.contains(line), "{line}: {code}"); + } + } + + #[test] + fn verbatim_module_syntax_keeps_type_only_imports_loading() { + let output = emit_with_options( + "import { type I } from './a';\n\ + import {} from './b';\n\ + export { type T } from './c';\n", EmitOptions { - module: Some(crate::emitter::ModuleKind::Es2015), - always_strict: true, + target: ScriptTarget::Es2022, + module: Some(crate::emitter::ModuleKind::EsNext), + verbatim_module_syntax: true, ..EmitOptions::default() }, ); - assert!( - !javascript(&es_module).starts_with("\"use strict\";\n"), - "an ES external module takes no prologue even under alwaysStrict: {}", - javascript(&es_module) - ); - let commonjs_script = emit_with_options( - "const value = 1;\n", + let code = javascript(&output); + for line in [ + "import {} from './a';", + "import {} from './b';", + "export {} from './c';", + ] { + assert!(code.contains(line), "{line}: {code}"); + } + } + + #[test] + fn a_require_local_is_always_an_identifier() { + let mut names = super::RequireNames::new(std::collections::BTreeSet::new()); + for (module, expected) in [ + ("./m.js", "m_js_1"), + ("./m.js", "m_js_2"), + ("@scope/my-lib", "my_lib_1"), + ("./10-lib", "_10_lib_1"), + ("enum", "enum_1"), + ] { + assert_eq!(names.allocate(module), expected); + } + } + + #[test] + fn commonjs_function_exports_precede_every_require() { + let output = emit_with_options( + "import './setup';\n\ + export * from './lib';\n\ + export function f() {}\n\ + function g() {}\n\ + export { g as h };\n\ + const c = 1;\n\ + export { c };\n\ + export default function d() {}\n\ + export declare function ambient(): void;\n", EmitOptions { + target: ScriptTarget::Es2015, module: Some(crate::emitter::ModuleKind::CommonJs), ..EmitOptions::default() }, ); - assert!( - !javascript(&commonjs_script).starts_with("\"use strict\";\n"), - "a script under CommonJS still follows alwaysStrict: {}", - javascript(&commonjs_script) - ); - let script = emit_with_options( - "const value = 1;\n", + let code = javascript(&output); + let order = [ + "exports.c = void 0;", + "exports.f = f;", + "exports.h = g;", + "exports.default = d;", + "require(\"./setup\");", + "__exportStar(require(\"./lib\"), exports);", + "function f() { }", + ]; + let positions: Vec = order + .iter() + .map(|line| code.find(line).unwrap_or_else(|| panic!("{line}: {code}"))) + .collect(); + assert!(positions.is_sorted(), "{code}"); + assert!(!code.contains("ambient"), "{code}"); + } + + #[test] + fn an_es_module_with_nothing_left_stays_a_module() { + let esm = EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::EsNext), + ..EmitOptions::default() + }; + for source in [ + "export interface I {}\n", + "interface I {}\nexport { I };\n", + "export {};\n", + "import type { T } from './t';\nconst x = 1;\nexport type { T };\n", + ] { + let output = emit_with_options(source, esm.clone()); + let code = javascript(&output); + assert!(code.ends_with("export {};\n"), "{source}: {code}"); + } + for source in ["export const x = 1;\n", "import './a';\n", "const x = 1;\n"] { + let output = emit_with_options(source, esm.clone()); + assert!(!javascript(&output).contains("export {}"), "{source}"); + } + let commonjs = emit_with_options( + "export interface I {}\n", EmitOptions { + target: ScriptTarget::Es2015, + module: Some(crate::emitter::ModuleKind::CommonJs), ..EmitOptions::default() }, ); - assert!( - !javascript(&script).starts_with("\"use strict\";\n"), - "a plain script has no prologue: {}", - javascript(&script) - ); - let always = emit_with_options( - "const value = 1;\n", + assert!(!javascript(&commonjs).contains("export {}")); + } + + #[test] + fn an_import_survives_only_through_a_value_read() { + let esm = EmitOptions { + target: ScriptTarget::Es2022, + module: Some(crate::emitter::ModuleKind::EsNext), + ..EmitOptions::default() + }; + for (source, expected) in [ + ( + "import { I, v } from './m';\nlet a: I;\nv;\n", + "import { v } from './m';", + ), + ("import { unused } from './m';\n", "export {};"), + ("import d from './m';\nlet a: d;\n", "export {};"), + ("import * as n from './m';\nlet a: n.T;\n", "export {};"), + ("import {} from './m';\n", "export {};"), + ( + "import { x } from './m';\nexport { x };\n", + "import { x } from './m';", + ), + ( + "import { d } from './m';\n@d class C {}\n", + "import { d } from './m';", + ), + ( + "import { d } from './m';\nclass C { @d m() {} }\n", + "import { d } from './m';", + ), + ( + "import { c } from './m';\ntry {} catch ({ a = c }) {}\n", + "import { c } from './m';", + ), + ( + "import { x } from './m';\nfunction f(a = x) { var x = 2; return a; }\n", + "import { x } from './m';", + ), + ( + "import { x } from './m';\n{ function x() {} }\nconsole.log(x);\n", + "import { x } from './m';", + ), + ( + "import { x } from './m';\nnamespace N { import q = x.v; export const w = q; }\n", + "import { x } from './m';", + ), + ( + "import { x } from './m';\nexport import w = x.v;\n", + "import { x } from './m';", + ), + ("import './m';\n", "import './m';"), + ] { + let output = emit_with_options(source, esm.clone()); + let code = javascript(&output); + assert!(code.contains(expected), "{source}: {code}"); + if expected == "export {};" { + assert!(!code.contains("import"), "{source}: {code}"); + } + } + let verbatim = emit_with_options( + "import { unused } from './m';\n", EmitOptions { - always_strict: true, - ..EmitOptions::default() + verbatim_module_syntax: true, + ..esm.clone() }, ); - assert!( - javascript(&always).starts_with("\"use strict\";\n"), - "alwaysStrict emits the prologue: {}", - javascript(&always) + assert!(javascript(&verbatim).contains("import { unused }")); + let commonjs = emit_with_options( + "import { I } from './m';\nlet a: I;\nexport { a };\n", + EmitOptions { + module: Some(crate::emitter::ModuleKind::CommonJs), + ..esm + }, ); - let deduped = emit_with_options( - "\"use strict\";\nconst value = 1;\n", + assert!(!javascript(&commonjs).contains("require")); + } + + #[test] + fn a_catch_pattern_reads_through_its_namespace() { + let output = emit_with_options( + "namespace N {\n\ + export var n = 3;\n\ + try { throw {}; } catch ({ a = n }) { a; }\n\ + }\n", EmitOptions { - always_strict: true, + target: ScriptTarget::Es2022, ..EmitOptions::default() }, ); - let code = javascript(&deduped); - assert_eq!( - code.matches("\"use strict\";").count(), - 1, - "a source directive suppresses the synthetic one: {code}" - ); + let code = javascript(&output); + assert!(code.contains("catch ({ a = N.n })"), "{code}"); } #[test] - fn ambient_enum_and_module_produce_no_runtime_statements() { - // Upstream ambientEnumElementInitializer5 (43527a9c) emits only the - // strict prologue; ambientModules (df2119af) keeps only the trailing - // member assignment. - let ambient_enum = emit_at("declare enum E { e = -0xA }\n", ScriptTarget::Es2015); - assert_eq!( - javascript(&ambient_enum), - "", - "{:?}", - javascript(&ambient_enum) + fn a_value_read_keeps_an_import_whatever_another_file_seems_to_say() { + let output = emit_with_options( + "import { Foo, Bar } from './m';\nconsole.log(Foo);\nexport { Bar };\n", + EmitOptions { + target: ScriptTarget::Es2022, + module: Some(crate::emitter::ModuleKind::EsNext), + erased_imports: std::sync::Arc::new( + [ + ("./m".to_owned(), "Foo".to_owned()), + ("./m".to_owned(), "Bar".to_owned()), + ] + .into_iter() + .collect(), + ), + ..EmitOptions::default() + }, ); + let code = javascript(&output); + assert!(code.contains("import { Foo } from './m';"), "{code}"); + assert!(!code.contains("Bar"), "{code}"); + } - let ambient_module = emit_at( - "declare namespace Foo.Bar { export var foo; };\nFoo.Bar.foo = 5;\n", - ScriptTarget::Es2015, - ); - // The ambient declaration vanishes; the runtime assignment survives, - // matching upstream ambientModules (df2119af) after the prologue. - assert!( - !javascript(&ambient_module).starts_with("declare") - && !javascript(&ambient_module).contains("export var foo"), - "the ambient block is erased: {}", - javascript(&ambient_module) + #[test] + fn something_reading_imports_unseen_keeps_them_all() { + let options = EmitOptions { + target: ScriptTarget::Es2022, + module: Some(crate::emitter::ModuleKind::EsNext), + ..EmitOptions::default() + }; + let pragma = emit_with_options( + "/** @jsx h */\nimport { h } from './m';\nexport {};\n", + options.clone(), ); - - let runtime = emit_at("const enum K { A }\nlet kept = 1;\n", ScriptTarget::Es2015); - assert!( - javascript(&runtime).contains("kept"), - "non-ambient code survives: {}", - javascript(&runtime) + assert!(javascript(&pragma).contains("import { h }")); + let metadata = emit_with_options( + "import { T } from './m';\nlet a: T;\n", + EmitOptions { + keeps_imports: true, + ..options + }, ); + assert!(javascript(&metadata).contains("import { T }")); } #[test] - fn generic_call_type_arguments_are_erased() { - // Upstream genericCallWithNonGenericArgs1 (e71ae63c): f(null) - // emits f(null); - let output = emit_at( - "function f(x: any) { }\nf(null);\n", - ScriptTarget::Es2015, + fn commonjs_export_import_require_lives_on_exports() { + let output = emit_with_options( + "export import ns = require('./m');\nconsole.log(ns);\n", + EmitOptions { + target: ScriptTarget::Es2022, + module: Some(crate::emitter::ModuleKind::CommonJs), + ..EmitOptions::default() + }, ); let code = javascript(&output); - assert!(code.contains("f(null);"), "{code}"); - assert!(!code.contains(""), "{code}"); + assert!(code.contains("exports.ns = require(\"./m\");"), "{code}"); + assert!(code.contains("console.log(exports.ns);"), "{code}"); } #[test] - fn commonjs_named_import_and_function_export_match_upstream_shape() { - // Upstream commonjsSafeImport(target=es2015).js - // (282283386bc4040f0eba585ac33e142c5c30cfbb67f969b1706c655dc7bee408). - let export = emit_with_options( - "export function Foo() {}\n", + fn a_type_has_no_runtime_export() { + const SOURCE: &str = "import type { Q } from './types';\n\ + import { type R } from './types';\n\ + interface I {}\n\ + type T = 1;\n\ + const v = 1;\n\ + export { I, T, Q, R, v };\n"; + let commonjs_output = emit_with_options( + SOURCE, EmitOptions { target: ScriptTarget::Es2015, module: Some(crate::emitter::ModuleKind::CommonJs), ..EmitOptions::default() }, ); - let export_code = javascript(&export); - assert!( - export_code.starts_with( - "\"use strict\";\nObject.defineProperty(exports, \"__esModule\", { value: true });\nexports.Foo = Foo;\nfunction Foo()" - ), - "{export_code}" - ); - - let import = emit_with_options( - "import { Foo } from './10_lib';\nFoo();\n", + let commonjs = javascript(&commonjs_output); + assert!(commonjs.contains("exports.v = void 0;"), "{commonjs}"); + assert!(commonjs.contains("exports.v = v;"), "{commonjs}"); + for name in ["I", "T", "Q", "R"] { + assert!(!commonjs.contains(&format!("exports.{name}")), "{commonjs}"); + } + let module_output = emit_with_options( + SOURCE, EmitOptions { target: ScriptTarget::Es2015, - module: Some(crate::emitter::ModuleKind::CommonJs), + module: Some(crate::emitter::ModuleKind::EsNext), ..EmitOptions::default() }, ); - let import_code = javascript(&import); - assert!( - import_code.starts_with( - "\"use strict\";\nObject.defineProperty(exports, \"__esModule\", { value: true });\nconst _10_lib_1 = require(\"./10_lib\");\n" - ), - "{import_code}" - ); - assert!( - import_code.contains("(0, _10_lib_1.Foo)();"), - "{import_code}" - ); - assert!(!import_code.contains("import "), "{import_code}"); - } - - #[test] - fn es5_lowers_nested_computed_destructuring_assignment() { - // Upstream computedPropertiesInDestructuring1.js - // (50845e031ebf8203c61d4bfb72d37eea703fea245df4e215c7b5772583a8a898). - let output = emit_at( - "let foo = 'bar', bar4; [{ [foo]: bar4 }] = [{ bar: 'bar' }];\n", - ScriptTarget::Es5, - ); - let code = javascript(&output); - assert!( - !output - .diagnostics - .iter() - .any(|diagnostic| diagnostic.code() - == codes::DESTRUCTURING_ASSIGNMENT_REQUIRES_ES2015), - "{:?}", - output.diagnostics - ); - assert!(code.contains("bar4 ="), "{code}"); - assert!(code.contains("[foo]"), "{code}"); - assert!(!code.contains("[{ [foo]: bar4 }] ="), "{code}"); + let module = javascript(&module_output); + assert!(module.contains("export { v };"), "{module}"); } #[test] - fn commonjs_class_and_variable_exports_receive_void_0_preamble() { - // Upstream importHelpersWithImportStarAs a.js (a2836e68) emits - // `exports.A = void 0;` + class + `exports.A = A;`; the variable form - // follows es6ImportDefaultBindingWithExport client.js (adfd4388). + fn commonjs_reexports_are_not_left_as_module_syntax() { let output = emit_with_options( - "export class A {}\nexport const x = 1;\n", + "export * from './a';\n\ + export * as ns from './b';\n\ + export { x, y as z } from './c';\n\ + export type { T } from './d';\n", EmitOptions { target: ScriptTarget::Es2015, module: Some(crate::emitter::ModuleKind::CommonJs), @@ -10328,35 +12739,30 @@ console.log(JSON.stringify([C._x, log, C.self === C, C.done])); }, ); let code = javascript(&output); - assert!( - code.starts_with( - "\"use strict\";\nObject.defineProperty(exports, \"__esModule\", { value: true });\nexports.A = exports.x = void 0;\n" - ), - "{code}" - ); - let marker = code - .find("exports.A = exports.x = void 0;") - .expect("preamble"); - let class_at = code.find("class A").expect("class kept"); - let class_export = code.find("exports.A = A;").expect("class export"); - let variable = code.find("const x = 1;").expect("variable kept"); - let variable_export = code.find("exports.x = x;").expect("variable export"); - assert!( - marker < class_at - && class_at < class_export - && class_export < variable - && variable < variable_export, - "{code}" - ); + assert!(!code.contains("export "), "{code}"); + assert!(!code.contains("./d"), "{code}"); + for expected in [ + "__exportStar(require(\"./a\"), exports);", + "exports.ns = __importStar(require(\"./b\"));", + "var c_1 = require(\"./c\");", + "Object.defineProperty(exports, \"x\", { enumerable: true, get: function () { return c_1.x; } });", + "Object.defineProperty(exports, \"z\", { enumerable: true, get: function () { return c_1.y; } });", + "function __exportStar(", + "function __importStar(", + ] { + assert!(code.contains(expected), "missing {expected}: {code}"); + } } #[test] - fn commonjs_local_specifier_exports_assign_each_name() { - // Same `exports.NAME = LOCAL;` identifier-assignment form as the - // identifier default export in es6ImportDefaultBindingWithExport - // server.js (adfd4388): `exports.default = a;`. + fn commonjs_rewrites_an_import_wherever_it_is_read() { let output = emit_with_options( - "const a = 1, b = 2;\nexport { a, b as c };\n", + "import { a } from './m';\n\ + const list = [a, ...a];\n\ + const box = { a, b: a, [a]: 1, c() { return a; } };\n\ + const text = `${a}`;\n\ + a.b = 1;\n\ + a.b++;\n", EmitOptions { target: ScriptTarget::Es2015, module: Some(crate::emitter::ModuleKind::CommonJs), @@ -10364,9 +12770,18 @@ console.log(JSON.stringify([C._x, log, C.self === C, C.done])); }, ); let code = javascript(&output); - assert!(code.contains("exports.a = a;"), "{code}"); - assert!(code.contains("exports.c = b;"), "{code}"); - assert!(!code.contains("export {"), "{code}"); + for expected in [ + "const list = [m_1.a, ...m_1.a];", + "a: m_1.a", + "b: m_1.a", + "[m_1.a]: 1", + "return m_1.a;", + "`${m_1.a}`", + "m_1.a.b = 1;", + "m_1.a.b++;", + ] { + assert!(code.contains(expected), "missing {expected}: {code}"); + } } #[test] diff --git a/crates/bamts-compiler/src/emitter/transforms/namespaces.rs b/crates/bamts-compiler/src/emitter/transforms/namespaces.rs new file mode 100644 index 00000000..76a7c0dd --- /dev/null +++ b/crates/bamts-compiler/src/emitter/transforms/namespaces.rs @@ -0,0 +1,871 @@ +//! Namespaces and enums as the function calls that build them at runtime. +//! +//! `namespace N { export var x = 1; }` becomes a `var N;` and a call that fills +//! the container: `(function (N) { N.x = 1; })(N || (N = {}));`. An enum fills +//! its container with its members the same way. Only the first declaration of +//! a name in a scope declares it, so merged blocks share one binding and a +//! namespace merged into a class or function reuses theirs. + +use std::collections::BTreeSet; + +use crate::checker::{SymbolId, SymbolKind}; +use crate::emitter::{enum_number_lexeme, enum_string_lexeme}; +use crate::enum_plan::{EnumMemberPlan, EnumScalar}; +use crate::source::TextRange; +use crate::syntax::*; + +use super::{RestExcludeKey, Rewriter, property_key_text}; + +/// One namespace body being lowered. +pub(super) struct NamespaceFrame { + container: SymbolId, + /// The body function's parameter, which holds the container. + parameter: String, + /// Temporaries taken in this body so far. + temps: usize, +} + +/// How a declaration reaches the container it fills. +pub(super) enum ContainerBinding { + /// `N || (N = {})` + Local, + /// `N = parent.N || (parent.N = {})`, for a member of `parent`. + Member(String), + /// `N || (exports.N = N = {})` + CommonJs, + /// `export var N;` declares it, and the call is then the local one. + Export, +} + +/// Whether an `import a = ...` aliases a name rather than requiring a module. +pub(super) fn is_qualified_alias(import: &ImportEqualsDeclaration) -> bool { + matches!(import.reference, ExternalModuleReference::Qualified(_)) +} + +impl Rewriter<'_> { + pub(super) fn lower_namespace( + &mut self, + statement: &Stmt, + namespace: &NamespaceDeclaration, + binding: ContainerBinding, + range: TextRange, + ) -> Vec { + let plan = self + .model + .and_then(|model| model.namespace_facts().declaration(statement.id())); + let (Some(plan), NamespaceName::Identifier { name, .. }) = (plan, &namespace.name) else { + return vec![statement.clone()]; + }; + if !plan.is_value_bearing() { + // Nothing to build; the printer decides what this one reports. + return vec![statement.clone()]; + } + let Some(local) = self.identifier_name(name) else { + return vec![statement.clone()]; + }; + let parameter = self.namespace_parameter(&local, namespace.body.range()); + let body = self.lower_namespace_body(namespace, plan.container(), ¶meter); + let mut out = Vec::new(); + out.extend(self.container_declaration(&local, &binding, range)); + let argument = self.container_argument(&local, &binding, range); + out.push(self.container_call(¶meter, body, argument, range)); + out + } + + pub(super) fn lower_enum( + &mut self, + statement: &Stmt, + declaration: &EnumDeclaration, + binding: ContainerBinding, + range: TextRange, + ) -> Vec { + let plan = self + .model + .and_then(|model| model.enum_facts().declaration(statement.id())) + .filter(|plan| plan.members().len() == declaration.members.len()); + let local = self + .identifier_name(&declaration.name) + .filter(|local| !local.is_empty()); + let (Some(plan), Some(local), false) = (plan, local, declaration.is_const) else { + return vec![statement.clone()]; + }; + let body = declaration + .members + .iter() + .zip(plan.members()) + .filter_map(|(member, plan)| self.enum_member_statement(&local, member, plan)) + .collect(); + let mut out = Vec::new(); + out.extend(self.container_declaration(&local, &binding, range)); + let argument = self.container_argument(&local, &binding, range); + out.push(self.container_call(&local, body, argument, range)); + out + } + + /// `var a = A.B;` for an alias something reads, or one a script declares at + /// its top level for a value. Any other alias only names a type. + pub(super) fn lower_alias( + &mut self, + statement: &Stmt, + import: &ImportEqualsDeclaration, + ) -> Vec { + let ExternalModuleReference::Qualified(name) = &import.reference else { + return vec![statement.clone()]; + }; + if import.is_type_only { + return vec![statement.clone()]; + } + let script_top_level = + self.declared_names.len() == 1 && !crate::checker::source_is_module(self.file); + if !self.alias_is_read(statement) + && !(script_top_level && self.alias_target_is_value(statement)) + { + return Vec::new(); + } + let range = statement.range(); + let Some(value) = self.entity_expression(name, range) else { + return vec![statement.clone()]; + }; + vec![self.alias_declaration(import, value, range)] + } + + /// An exported alias publishes its target, unless that only names a type. + pub(super) fn lower_exported_alias( + &mut self, + statement: &Stmt, + import: &ImportEqualsDeclaration, + binding: ContainerBinding, + range: TextRange, + ) -> Vec { + let ExternalModuleReference::Qualified(name) = &import.reference else { + return vec![statement.clone()]; + }; + if import.is_type_only + || !(self.alias_is_read(statement) || self.alias_target_is_value(statement)) + { + return Vec::new(); + } + let Some(local) = self.identifier_name(&import.local) else { + return vec![statement.clone()]; + }; + let Some(value) = self.entity_expression(name, range) else { + return vec![statement.clone()]; + }; + let expression = match binding { + ContainerBinding::Member(parent) => { + self.member_assignment(&parent, &local, value, range) + } + ContainerBinding::CommonJs => { + let left = self.cjs_export_target(&local, range); + self.assignment(left, value, range) + } + ContainerBinding::Local | ContainerBinding::Export => { + let declaration = self.alias_declaration(import, value, range); + return vec![self.syn_node( + range, + Statement::Export(ExportDeclaration::Named( + ExportNamedDeclaration::Declaration(Box::new(declaration)), + )), + )]; + } + }; + vec![self.expression_statement(expression, range)] + } + + fn alias_declaration( + &mut self, + import: &ImportEqualsDeclaration, + value: Expr, + range: TextRange, + ) -> Stmt { + let declarator = self.make_declarator(import.local.clone(), Some(value), range); + self.syn_node( + range, + Statement::Variable(VariableDeclaration { + range, + kind: VariableKind::Var, + declarations: vec![declarator], + }), + ) + } + + fn entity_expression(&mut self, name: &EntityName, range: TextRange) -> Option { + match name { + EntityName::Identifier(ident) => { + Some(self.node(ident.range(), Expression::Identifier(ident.clone()))) + } + EntityName::Qualified { left, right } => { + let object = self.entity_expression(left, range)?; + let property = MemberProperty::Named(right.clone()); + Some(self.member_expr(&object, property, range)) + } + EntityName::Missing(_) => None, + } + } + + fn alias_is_read(&mut self, statement: &Stmt) -> bool { + let Some(model) = self.model else { + return true; + }; + let read = self.read_aliases.get_or_insert_with(|| { + let read: BTreeSet = model.references().map(|(_, symbol)| symbol).collect(); + model + .symbols() + .iter() + .enumerate() + .filter(|(index, symbol)| { + symbol.kind() == SymbolKind::Import + && read.contains(&SymbolId::new(*index as u32)) + }) + .map(|(_, symbol)| symbol.range().start()) + .collect() + }); + read.contains(&statement.range().start()) + } + + /// Whether an alias resolves to something with a runtime value. One that + /// resolves to nothing counts, so the error it reports is not also erased. + fn alias_target_is_value(&self, statement: &Stmt) -> bool { + let Some(model) = self.model else { + return true; + }; + let facts = model.namespace_facts(); + let target = facts + .qualified_import_path(statement.id()) + .and_then(<[SymbolId]>::last); + let Some(&target) = target else { + return true; + }; + match model.symbol(target).kind() { + SymbolKind::Interface | SymbolKind::TypeAlias => false, + SymbolKind::Namespace => facts.merged_declarations(target).iter().any(|declaration| { + facts + .declaration(*declaration) + .is_some_and(|plan| plan.is_value_bearing()) + }), + _ => true, + } + } + + /// `E[E["A"] = 0] = "A";`, or `E["A"] = "a";` for a member with no reverse + /// mapping. A member with neither a value nor an initializer has nothing + /// to store. + fn enum_member_statement( + &mut self, + object: &str, + member: &EnumMemberNode, + plan: &EnumMemberPlan, + ) -> Option { + let key = enum_string_lexeme(plan.name()?); + let constant = plan.value()?.constant(); + let initializer = member.data().initializer.as_deref(); + let range = member.range(); + let value = match (constant, initializer) { + (Some(EnumScalar::Number(number)), _) => { + self.number_expr(&enum_number_lexeme(number.to_f64())) + } + (Some(EnumScalar::String(text)), _) => { + let text = enum_string_lexeme(text); + self.string_expr(&text, range) + } + (None, Some(initializer)) => self.rewrite_expr(initializer), + (None, None) => return None, + }; + let key_expr = self.string_expr(&key, range); + let store = self.computed_assignment(object, key_expr, value, range); + let expression = if plan.reverse() { + let name = self.string_expr(&key, range); + self.computed_assignment(object, store, name, range) + } else { + store + }; + Some(self.expression_statement(expression, range)) + } + + fn string_expr(&mut self, quoted_contents: &str, range: TextRange) -> Expr { + let literal = self.string_literal(quoted_contents); + self.syn_node(range, Expression::Literal(Literal::String(literal))) + } + + fn computed_assignment( + &mut self, + object: &str, + key: Expr, + value: Expr, + range: TextRange, + ) -> Expr { + let object = self.ident_expr(object); + let left = self.syn_node( + range, + AssignmentTarget::Member(AssignmentMemberTarget { + object: Box::new(object), + property: MemberProperty::Computed(Box::new(key)), + }), + ); + self.assignment(left, value, range) + } + + fn lower_namespace_body( + &mut self, + namespace: &NamespaceDeclaration, + container: SymbolId, + parameter: &str, + ) -> Vec { + self.namespace_frames.push(NamespaceFrame { + container, + parameter: parameter.to_owned(), + temps: 0, + }); + self.declared_names.push(BTreeSet::new()); + self.expression_temp_scopes.push(Vec::new()); + self.publish_aliases(namespace, parameter); + let mut body = Vec::new(); + for member in &namespace.body.data().statements { + self.record_declaration(member); + body.extend(self.rewrite_namespace_member(member, parameter)); + } + let temps = self.expression_temp_scopes.pop().unwrap_or_default(); + if let Some(declaration) = self.temp_declaration(temps, namespace.body.range()) { + body.insert(0, declaration); + } + self.declared_names.pop(); + self.namespace_frames.pop(); + body + } + + /// An exported alias has no local, so every read goes through the + /// container, including one above the declaration. + fn publish_aliases(&mut self, namespace: &NamespaceDeclaration, parameter: &str) { + let Some(model) = self.model else { + return; + }; + for member in &namespace.body.data().statements { + let Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration( + inner, + ))) = member.data() + else { + continue; + }; + let aliased = matches!(inner.data(), Statement::ImportEquals(import) + if is_qualified_alias(import)); + let position = model.symbols().iter().position(|symbol| { + symbol.kind() == SymbolKind::Import && symbol.range() == inner.range() + }); + if let (true, Some(position)) = (aliased, position) { + self.published_aliases + .insert(SymbolId::new(position as u32), parameter.to_owned()); + } + } + } + + fn rewrite_namespace_member(&mut self, member: &Stmt, parameter: &str) -> Vec { + let Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration(inner))) = + member.data() + else { + return self.rewrite_statement(member); + }; + let range = member.range(); + let exported = match inner.data() { + Statement::Variable(declaration) => { + return self.namespace_export_variables(declaration, parameter, range); + } + Statement::Namespace(namespace) => { + let binding = ContainerBinding::Member(parameter.to_owned()); + return self.lower_namespace(inner, namespace, binding, range); + } + Statement::Enum(declaration) if !declaration.is_const => { + let binding = ContainerBinding::Member(parameter.to_owned()); + return self.lower_enum(inner, declaration, binding, range); + } + Statement::ImportEquals(import) if is_qualified_alias(import) => { + let binding = ContainerBinding::Member(parameter.to_owned()); + return self.lower_exported_alias(inner, import, binding, range); + } + Statement::Function(function) if function.function.body.is_some() => { + function.function.name.as_ref() + } + Statement::Class(class) => class.name.as_ref(), + _ => None, + }; + let mut out = self.rewrite_statement(inner); + if !out.is_empty() { + // Claiming the `export` keyword brings the comments above it along. + let first = out.remove(0); + out.insert(0, Node::new(first.id(), range, first.into_data())); + } + if let Some(name) = exported.and_then(|name| self.identifier_name(name)) { + let value = self.ident_expr(&name); + let assignment = self.member_assignment(parameter, &name, value, range); + out.push(self.expression_statement(assignment, range)); + } + out + } + + /// An exported variable lives only on the container, so its declaration + /// is the assignment of its initializer, and it has none without one. + pub(super) fn namespace_export_variables( + &mut self, + declaration: &VariableDeclaration, + parameter: &str, + range: TextRange, + ) -> Vec { + let mut assignments = Vec::new(); + for declarator in &declaration.declarations { + let data = declarator.data(); + if let Some(initializer) = data.initializer.as_deref() { + let value = self.rewrite_expr(initializer); + self.flatten_export(&data.binding, value, parameter, &mut assignments); + } + } + let expression = match assignments.len() { + 0 => return Vec::new(), + 1 => assignments.remove(0), + _ => self.syn_node( + range, + Expression::Sequence(SequenceExpression { + expressions: assignments, + }), + ), + }; + vec![self.expression_statement(expression, range)] + } + + /// Assignments taking `pattern` apart onto the container. A pattern that + /// reads its source more than once holds it in a temporary first. + fn flatten_export( + &mut self, + pattern: &Pattern, + value: Expr, + parameter: &str, + out: &mut Vec, + ) { + let range = pattern.range(); + match pattern.data() { + BindingPattern::Identifier(ident) => { + if let Some(name) = self.identifier_name(ident) { + out.push(self.member_assignment(parameter, &name, value, ident.range())); + } + } + BindingPattern::Object(object) => { + let collects_rest = object + .properties + .iter() + .any(|property| matches!(property.binding.data(), BindingPattern::Rest(_))); + let reads = object.properties.len() + usize::from(collects_rest); + let source = self.single_read_source(value, reads, range, out); + let mut keys = Vec::new(); + for property in &object.properties { + if let BindingPattern::Rest(rest) = property.binding.data() { + if let Expression::Identifier(held) = source.data() { + let excluded = self.rest_exclude_array(&keys, range); + let collected = self.rest_call(held, excluded, range); + self.flatten_export(&rest.argument, collected, parameter, out); + } + continue; + } + if let (true, PropertyName::Computed(key)) = (collects_rest, &property.name) { + let key = self.rewrite_expr(key); + let held = self.hold_in_temp(key, range, out); + if let Expression::Identifier(temp) = held.data() { + keys.push(RestExcludeKey::Computed(temp.clone())); + } + let member = MemberProperty::Computed(Box::new(held)); + let read = self.member_expr(&source, member, range); + let read = self.with_default(read, property.initializer.as_deref(), out); + self.flatten_export(&property.binding, read, parameter, out); + continue; + } + keys.extend(property_key_text(self, property).map(RestExcludeKey::Static)); + let name = self.rewrite_property_name(&property.name); + let read = self.property_read(&source, name, range); + let read = self.with_default(read, property.initializer.as_deref(), out); + self.flatten_export(&property.binding, read, parameter, out); + } + } + BindingPattern::Array(array) => { + let collects_rest = array.elements.iter().any(|element| { + matches!(element, ArrayBindingElement::Binding(binding) + if matches!(binding.data(), BindingPattern::Rest(_))) + }); + let reads = array.elements.len() + usize::from(collects_rest); + let source = self.single_read_source(value, reads, range, out); + for (index, element) in array.elements.iter().enumerate() { + let ArrayBindingElement::Binding(binding) = element else { + continue; + }; + if let BindingPattern::Rest(rest) = binding.data() { + if let Expression::Identifier(held) = source.data() { + let collected = self.slice_call(held, index, range); + self.flatten_export(&rest.argument, collected, parameter, out); + } + continue; + } + let index = self.number_expr(&index.to_string()); + let property = MemberProperty::Computed(Box::new(index)); + let read = self.member_expr(&source, property, range); + self.flatten_export(binding, read, parameter, out); + } + } + BindingPattern::Assignment(assignment) => { + let read = self.with_default(value, Some(&*assignment.right), out); + self.flatten_export(&assignment.left, read, parameter, out); + } + BindingPattern::Rest(_) | BindingPattern::Missing(_) => {} + } + } + + fn single_read_source( + &mut self, + value: Expr, + reads: usize, + range: TextRange, + out: &mut Vec, + ) -> Expr { + if reads == 1 || matches!(value.data(), Expression::Identifier(_)) { + return value; + } + self.hold_in_temp(value, range, out) + } + + fn hold_in_temp(&mut self, value: Expr, range: TextRange, out: &mut Vec) -> Expr { + let temp = self.namespace_temp(); + let left = self.syn_node(range, AssignmentTarget::Identifier(temp.clone())); + out.push(self.assignment(left, value, range)); + self.node(temp.range(), Expression::Identifier(temp)) + } + + /// The next free `_a`-style name in the innermost body, declared with it. + fn namespace_temp(&mut self) -> IdentifierNode { + const LETTERS: &[u8] = b"abcdefghjklmopqrstuvwxyz"; + let file = self.file; + let occupied = self + .occupied_names + .get_or_insert_with(|| super::identifier_names(file)); + let temps = match self.namespace_frames.last_mut() { + Some(frame) => &mut frame.temps, + None => &mut self.module_temps, + }; + let (index, name) = (*temps..) + .map(|index| match LETTERS.get(index) { + Some(&letter) => (index, format!("_{}", char::from(letter))), + None => (index, format!("_{}", index - LETTERS.len())), + }) + .find(|(_, candidate)| !occupied.contains(candidate)) + .expect("an unbounded index range reaches an unused name"); + *temps = index + 1; + let temp = self.ident(&name); + if let Some(scope) = self.expression_temp_scopes.last_mut() { + scope.push(temp.clone()); + } + temp + } + + fn property_read(&mut self, source: &Expr, name: PropertyName, range: TextRange) -> Expr { + let property = match name { + PropertyName::Identifier(ident) => MemberProperty::Named(ident), + PropertyName::Computed(key) => MemberProperty::Computed(key), + PropertyName::String(literal) => MemberProperty::Computed(Box::new(self.node( + literal.range(), + Expression::Literal(Literal::String(literal)), + ))), + PropertyName::Number(literal) => MemberProperty::Computed(Box::new(self.node( + literal.range(), + Expression::Literal(Literal::Number(literal)), + ))), + PropertyName::Private(_) | PropertyName::Missing(_) => return source.clone(), + }; + self.member_expr(source, property, range) + } + + /// `_a === void 0 ? default : _a` over a read held in a temporary. + fn with_default(&mut self, read: Expr, default: Option<&Expr>, out: &mut Vec) -> Expr { + let Some(default) = default else { + return read; + }; + let range = read.range(); + let held = self.hold_in_temp(read, range, out); + let Expression::Identifier(temp) = held.data() else { + return held; + }; + let default = self.rewrite_expr(default); + self.default_ternary(temp, &default, range) + } + + /// The parameter name for a namespace body. It is the namespace's own name + /// unless something inside declares that name, which would shadow it. + fn namespace_parameter(&mut self, local: &str, body: TextRange) -> String { + if !self.declares_value_inside(local, body) { + return local.to_owned(); + } + let file = self.file; + let occupied = self + .occupied_names + .get_or_insert_with(|| super::identifier_names(file)); + (1u32..) + .map(|suffix| format!("{local}_{suffix}")) + .find(|candidate| occupied.insert(candidate.clone())) + .expect("an unbounded suffix range reaches an unused name") + } + + fn declares_value_inside(&mut self, name: &str, body: TextRange) -> bool { + let Some(model) = self.model else { + return false; + }; + let ranges = self.value_symbol_ranges.get_or_insert_with(|| { + let mut ranges: std::collections::HashMap> = + std::collections::HashMap::new(); + for symbol in model.symbols() { + if !matches!( + symbol.kind(), + SymbolKind::Interface + | SymbolKind::TypeAlias + | SymbolKind::TypeParameter + | SymbolKind::IntrinsicType + ) { + ranges + .entry(symbol.name().to_owned()) + .or_default() + .push(symbol.range()); + } + } + ranges + }); + ranges.get(name).is_some_and(|ranges| { + ranges + .iter() + .any(|range| range.start() >= body.start() && range.end() <= body.end()) + }) + } + + /// The name of the parameter holding the container a use resolved to, + /// when `reference` reads a namespace member rather than a local. + pub(super) fn namespace_member_parameter(&self, reference: NodeId) -> Option { + let model = self.model?; + if let Some(parameter) = model + .reference(reference) + .and_then(|symbol| self.published_aliases.get(&symbol)) + { + return Some(parameter.clone()); + } + let member = model.namespace_facts().member_use(reference)?; + self.namespace_frames + .iter() + .rev() + .find(|frame| frame.container == member.container()) + .map(|frame| frame.parameter.clone()) + } + + pub(super) fn namespace_member( + &mut self, + parameter: &str, + name: &str, + range: TextRange, + ) -> Expr { + let object = self.ident_expr(parameter); + let property = MemberProperty::Named(self.ident(name)); + self.member_expr(&object, property, range) + } + + /// Records a declaration that owns its name before anything merges with it. + pub(super) fn record_declaration(&mut self, statement: &Stmt) { + let declaration = match statement.data() { + Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration( + inner, + ))) => inner.data(), + data => data, + }; + let name = match declaration { + Statement::Class(class) => class.name.as_ref(), + Statement::Function(function) => function.function.name.as_ref(), + _ => None, + }; + if let Some(name) = name.and_then(|name| self.identifier_name(name)) + && let Some(names) = self.declared_names.last_mut() + { + names.insert(name); + } + } + + /// `var N;` for the first declaration of `local` in this scope. Below the + /// top level the binding is block scoped. + fn container_declaration( + &mut self, + local: &str, + binding: &ContainerBinding, + range: TextRange, + ) -> Option { + if !self.declared_names.last_mut()?.insert(local.to_owned()) { + return None; + } + let kind = if self.declared_names.len() > 1 { + VariableKind::Let + } else { + VariableKind::Var + }; + let name = self.ident(local); + let declarator = self.make_declarator(name, None, range); + let declaration = self.syn_node( + range, + Statement::Variable(VariableDeclaration { + range, + kind, + declarations: vec![declarator], + }), + ); + Some(if matches!(binding, ContainerBinding::Export) { + self.syn_node( + range, + Statement::Export(ExportDeclaration::Named( + ExportNamedDeclaration::Declaration(Box::new(declaration)), + )), + ) + } else { + declaration + }) + } + + fn container_argument( + &mut self, + local: &str, + binding: &ContainerBinding, + range: TextRange, + ) -> Expr { + let empty = self.syn_node( + range, + Expression::Object(ObjectLiteral { + members: Vec::new(), + }), + ); + match binding { + ContainerBinding::Local | ContainerBinding::Export => { + let read = self.ident_expr(local); + let create = self.name_assignment(local, empty, range); + self.or_create(read, create, range) + } + ContainerBinding::CommonJs => { + let read = self.ident_expr(local); + let create = self.name_assignment(local, empty, range); + let left = self.cjs_export_target(local, range); + let publish = self.assignment(left, create, range); + self.or_create(read, publish, range) + } + ContainerBinding::Member(parent) => { + let read = self.namespace_member(parent, local, range); + let create = self.member_assignment(parent, local, empty, range); + let either = self.or_create(read, create, range); + self.name_assignment(local, either, range) + } + } + } + + /// `(function (parameter) { body })(argument);` + fn container_call( + &mut self, + parameter: &str, + body: Vec, + argument: Expr, + range: TextRange, + ) -> Stmt { + let name = self.ident(parameter); + let binding = self.syn_node(range, BindingPattern::Identifier(name)); + let parameter = self.syn_node( + range, + Parameter { + decorators: Vec::new(), + modifiers: ParameterModifiers { + accessibility: None, + is_readonly: false, + is_override: false, + }, + binding, + optional: false, + type_annotation: None, + initializer: None, + }, + ); + let body = self.syn_node(range, Block { statements: body }); + let function = self.syn_node( + range, + Expression::Function(FunctionExpression { + function: FunctionLike { + decorators: Vec::new(), + name: None, + is_async: false, + is_generator: false, + type_parameters: None, + parameters: vec![parameter], + return_type: None, + body: Some(FunctionBody::Block(body)), + }, + }), + ); + let callee = self.syn_node(range, Expression::Parenthesized(Box::new(function))); + let call = self.syn_node( + range, + Expression::Call(CallExpression { + callee: Box::new(callee), + optional: false, + type_arguments: None, + arguments: vec![CallArgument::Expression(Box::new(argument))], + }), + ); + self.expression_statement(call, range) + } + + fn or_create(&mut self, read: Expr, create: Expr, range: TextRange) -> Expr { + let create = self.syn_node(range, Expression::Parenthesized(Box::new(create))); + self.syn_node( + range, + Expression::Logical(LogicalExpression { + operator: LogicalOperator::Or, + left: Box::new(read), + right: Box::new(create), + }), + ) + } + + fn name_assignment(&mut self, name: &str, value: Expr, range: TextRange) -> Expr { + let name = self.ident(name); + let left = self.syn_node(range, AssignmentTarget::Identifier(name)); + self.assignment(left, value, range) + } + + fn member_assignment( + &mut self, + parameter: &str, + name: &str, + value: Expr, + range: TextRange, + ) -> Expr { + let object = self.ident_expr(parameter); + let property = MemberProperty::Named(self.ident(name)); + let left = self.syn_node( + range, + AssignmentTarget::Member(AssignmentMemberTarget { + object: Box::new(object), + property, + }), + ); + self.assignment(left, value, range) + } + + fn assignment(&mut self, left: AssignmentTargetNode, value: Expr, range: TextRange) -> Expr { + self.syn_node( + range, + Expression::Assignment(AssignmentExpression { + operator: AssignmentOperator::Assign, + left, + right: Box::new(value), + }), + ) + } + + fn expression_statement(&mut self, expression: Expr, range: TextRange) -> Stmt { + self.syn_node( + range, + Statement::Expression(ExpressionStatement { + expression: Box::new(expression), + }), + ) + } +} diff --git a/crates/bamts-compiler/src/lib.rs b/crates/bamts-compiler/src/lib.rs index 884bce41..68de607a 100644 --- a/crates/bamts-compiler/src/lib.rs +++ b/crates/bamts-compiler/src/lib.rs @@ -2,6 +2,7 @@ pub mod binder; pub mod checker; pub mod diagnostic; pub mod diagnostics_parser; +pub mod early_errors; pub mod emitter; pub mod enum_plan; pub(crate) mod jsx_desugar; diff --git a/crates/bamts-compiler/src/lower.rs b/crates/bamts-compiler/src/lower.rs index 975a8436..4ebc4c10 100644 --- a/crates/bamts-compiler/src/lower.rs +++ b/crates/bamts-compiler/src/lower.rs @@ -20,10 +20,11 @@ use crate::syntax::{ ExportDefaultValue, ExportNamedDeclaration, ExportSpecifierMode, Expr, Expression, ForBinding, ForInStatement, ForInitializer, ForOfMode, ForOfStatement, ForStatement, FunctionBody, FunctionLike, IdentifierNode, IfStatement, ImportBinding, ImportDeclaration, - ImportSpecifierMode, LabeledStatement, Literal, LogicalExpression, LogicalOperator, - MemberExpression, MemberProperty, MetaProperty, ModuleExportName, NamespaceDeclaration, - NamespaceName, NewExpression, NodeId, NodeKind, NumericLiteralNode, ObjectLiteral, - ObjectMember, ParameterNode, Pattern, PrivateIdentifierNode, PropertyModifier, PropertyName, + ImportSpecifierMode, JsxAttributeInitializer, JsxAttributeItem, JsxChild, JsxElementName, + LabeledStatement, Literal, LogicalExpression, LogicalOperator, MemberExpression, + MemberProperty, MetaProperty, ModuleExportName, NamespaceDeclaration, NamespaceName, + NewExpression, NodeId, NodeKind, NumericLiteralNode, ObjectLiteral, ObjectMember, + ParameterNode, Pattern, PrivateIdentifierNode, PropertyModifier, PropertyName, RegexLiteralNode, SourceFile, Statement, Stmt, StringLiteralNode, SwitchStatement, TemplateElementNode, TemplateLiteral, TokenKind, UnaryOperator, UpdateExpression, UpdateOperator, VariableDeclaration, VariableKind, WhileStatement, WithStatement, @@ -37,7 +38,7 @@ use bamts_bytecode::{ UnaryOp, Verified, VerifyError, }; use std::borrow::Cow; -use std::collections::{BTreeSet, HashMap, HashSet}; +use std::collections::{BTreeMap, BTreeSet, HashMap, HashSet}; use std::error::Error; use std::fmt; fn zero_range() -> TextRange { @@ -49,6 +50,10 @@ fn zero_range() -> TextRange { #[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] pub struct LowerOptions { pub javascript_compatibility: bool, + /// The host runs the entry file as an ECMAScript Script, so its top-level + /// `var` and function names live on the global object. Nothing in a file + /// can say this: Node runs the same bytes as a CommonJS module. + pub script_entry: bool, } #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub(crate) enum LoweringGoal { @@ -356,6 +361,9 @@ fn assemble_with_linkage_strings( } else { None }; + if options.script_entry { + context.instantiate_script(&mut builder, file.statements())?; + } context.lower_top_level(&mut builder, file.statements())?; match completion { Some(value) => context.emit(file.range(), Instruction::Return { value })?, @@ -370,6 +378,30 @@ fn assemble_with_linkage_strings( .collect(); Ok(Module::new(builder.constants, functions, entry)) } +/// Whether binding `parameters` can neither run code nor throw. +fn has_simple_parameters(parameters: &[ParameterNode]) -> bool { + parameters.iter().all(|parameter| { + parameter.data().initializer.is_none() + && matches!( + parameter.data().binding.data(), + BindingPattern::Identifier(_) + ) + }) +} +fn strip_parentheses(expression: &Expr) -> &Expr { + match expression.data() { + Expression::Parenthesized(inner) => strip_parentheses(inner), + _ => expression, + } +} +/// Whether code at the top of `file` is strict. +fn source_is_strict(file: &SourceFile) -> bool { + !matches!( + file.script_kind(), + ScriptKind::JavaScript | ScriptKind::JavaScriptReact + ) || crate::early_errors::is_module(file) + || crate::early_errors::has_use_strict(file, file.statements()) +} fn validate_script_kind(file: &SourceFile, options: LowerOptions) -> Result<(), LowerError> { let kind = match file.script_kind() { ScriptKind::TypeScript | ScriptKind::TypeScriptReact => return Ok(()), @@ -456,6 +488,67 @@ struct CapturePlan { captured: HashSet, runtime_cells: HashSet, } +/// The names a file reads as values at any depth, which is what keeps an import. +/// `export default x` and `export = x` are left to the caller: unlike a read, they +/// are legal for a type, so the name alone does not make it a value. +pub(crate) fn value_read_names( + file: &SourceFile, + enum_facts: Option<&'_ crate::enum_plan::EnumFacts>, +) -> BTreeSet { + let mut scanner = FreeVarScanner::new(file, None); + scanner.scans_jsx = true; + scanner.inlined_enums = enum_facts; + for statement in file.statements() { + let exports_a_name = match statement.data() { + Statement::Export(ExportDeclaration::Default(default)) => matches!( + &default.value, + ExportDefaultValue::Expression(value) + if matches!(value.data(), Expression::Identifier(_)) + ), + Statement::Export(ExportDeclaration::Assignment(value)) => { + matches!(value.data(), Expression::Identifier(_)) + } + _ => false, + }; + if !exports_a_name { + scanner.scan_statement(statement); + } + } + scanner.reads +} +/// A `this` parameter types the receiver and takes no argument. +fn runtime_parameters<'p>( + file: &SourceFile, + parameters: &'p [ParameterNode], +) -> &'p [ParameterNode] { + match parameters + .first() + .map(|parameter| parameter.data().binding.data()) + { + Some(BindingPattern::Identifier(identifier)) + if identifier_name(file, identifier).as_deref() == Some("this") => + { + ¶meters[1..] + } + _ => parameters, + } +} +fn scan_constructor( + scanner: &mut FreeVarScanner<'_>, + parameters: &[ParameterNode], + body: Option<&Block>, +) { + scanner.preseed_parameters(parameters); + if let Some(block) = body { + scanner.preseed_vars(&block.statements); + } + scanner.scan_parameter_initializers(parameters); + if let Some(block) = body { + for statement in &block.statements { + scanner.scan_statement(statement); + } + } +} fn scan_instance_init_free_vars(scanner: &mut FreeVarScanner<'_>, instance_steps: &[InstanceInit]) { for step in instance_steps { match step { @@ -551,6 +644,23 @@ struct WithBase { found: Register, key: Register, } +/// What a member reference reads and writes through. +#[derive(Clone, Copy)] +enum MemberBase { + Object(Register), + Super { home: Register, receiver: Register }, +} + +impl MemberBase { + /// The `this` of a call through this reference. + const fn this_value(self) -> Register { + match self { + Self::Object(object) => object, + Self::Super { receiver, .. } => receiver, + } + } +} + #[derive(Clone, Debug, Eq, Ord, PartialEq, PartialOrd)] enum CaptureKey { /// Free outer binding plus the live `with` sites that still precede it at @@ -565,10 +675,33 @@ enum CaptureKey { NewTarget, Parent(Register), ClassElements(Register), + /// The object a method was defined on, which `super` starts above. + Home(Register), + /// The cell recording that a derived constructor's `super(...)` has run. + SuperGuard(Register), + /// A name only field initializers read. They see the class's scope, so it + /// stays out of the constructor's own. + FieldName { + name: String, + sites: Vec, + cell: Register, + }, Cell(Register), Container(crate::checker::SymbolId, ContainerKind, Register), WithObject(BindingSite, Register), } +/// The names a function resolves through, set aside while initializers lower. +struct NameView { + scopes: Vec>, + captured_names: HashMap>, +} +/// Whether `super(...)` has run. An arrow can reach `super` from outside the +/// constructor's registers, so a constructor with such an arrow keeps a cell. +#[derive(Clone, Copy)] +enum SuperGuard { + Register(Register), + Cell(Register), +} #[derive(Clone, Copy)] enum ArgumentsSource { Own, @@ -723,6 +856,8 @@ enum StaticInit { key: Register, init_chain: Register, extra_inits: Register, + /// Fields of an undecorated class define; decorated ones still assign. + defines: bool, }, AutoAccessor { initializer: Option>, @@ -780,12 +915,26 @@ struct FunctionContext<'a> { completion_depth: usize, this_capture: Option, this_cell: Option, - derived_super_guard: Option, + derived_super_guard: Option, + /// A derived constructor's `return` checks its value. An arrow that captured + /// the same state returns normally. + derived_constructor: bool, + /// The outer names field initializers read, by captured cell and the + /// `with` sites that precede each one. + field_scope: BTreeMap)>, instance_steps: Vec, new_target_capture: Option, parent_constructor_capture: Option, class_elements: Option, + /// The home object in this function, for `super` member references. + home_object: Option, + /// The home object of the method about to be built from this context. + method_home: Option, arguments_source: ArgumentsSource, + /// Whether this code is strict. Everything but a JavaScript script is. + strict: bool, + /// The name an anonymous class about to be lowered takes from its context. + inferred_class_name: Option, } impl<'a> FunctionContext<'a> { fn new_top_level( @@ -827,11 +976,17 @@ impl<'a> FunctionContext<'a> { this_capture: None, this_cell: None, derived_super_guard: None, + derived_constructor: false, + field_scope: BTreeMap::new(), instance_steps: Vec::new(), new_target_capture: None, parent_constructor_capture: None, class_elements: None, + home_object: None, + method_home: None, arguments_source: ArgumentsSource::None, + strict: source_is_strict(file), + inferred_class_name: None, } } /// Whether `await`, `await using`, or async disposal awaits are legal here. @@ -840,12 +995,13 @@ impl<'a> FunctionContext<'a> { self.is_async || (self.top_level && self.goal != LoweringGoal::ClassicScript) } fn into_function(self, name: Option, flags: FunctionFlags) -> Function { + let is_sloppy = !self.strict; Function::new( name, self.capture_count, self.parameter_count, self.registers, - flags, + FunctionFlags { is_sloppy, ..flags }, self.code, self.handlers, ) @@ -1522,6 +1678,12 @@ impl<'a> FunctionContext<'a> { }; } let id = builder.intern(Constant::String(EcmaString::encode(name)), range)?; + if self.strict { + // Strict code cannot create a global by assigning to it. The value + // is already computed here, which is when the specification checks. + let dst = self.alloc_register(range)?; + self.emit(range, Instruction::LoadGlobal { dst, name: id })?; + } self.emit(range, Instruction::StoreGlobal { name: id, value })?; Ok(()) } @@ -1586,6 +1748,65 @@ impl<'a> FunctionContext<'a> { ArgumentsSource::None => Ok(None), } } + /// The part of `GlobalDeclarationInstantiation` lowering must spell out: + /// each `var` name exists before the Script runs, and declaring one never + /// disturbs a global that already has it. + fn instantiate_script( + &mut self, + builder: &mut ModuleBuilder, + statements: &[Stmt], + ) -> Result<(), LowerError> { + let range = self.file.range(); + if self.strict { + // `this` is the global object at the top of a Script, even a strict + // one. No code has run yet that could have replaced `globalThis`. + let name = builder.intern(Constant::String(EcmaString::encode("globalThis")), range)?; + let dst = self.alloc_register(range)?; + self.emit(range, Instruction::LoadGlobal { dst, name })?; + self.this_capture = Some(dst); + } + let mut names = Vec::new(); + collect_var_names(self.file, statements, &mut names); + names.sort_unstable(); + names.dedup(); + names.retain(|name| name != "undefined"); + if names.is_empty() { + return Ok(()); + } + let undefined_type = self.string_reg(builder, EcmaString::encode("undefined"), range)?; + let value = self.undefined(builder, range)?; + for name in names { + let id = builder.intern(Constant::String(EcmaString::encode(&name)), range)?; + let declared = self.alloc_register(range)?; + self.emit( + range, + Instruction::TypeOfGlobal { + dst: declared, + name: id, + }, + )?; + let absent = self.alloc_register(range)?; + self.emit( + range, + Instruction::Binary { + dst: absent, + op: BinaryOp::StrictEqual, + left: declared, + right: undefined_type, + }, + )?; + let skip = self.emit( + range, + Instruction::JumpIfFalse { + condition: absent, + target: Pc::new(0), + }, + )?; + self.emit(range, Instruction::StoreGlobal { name: id, value })?; + self.patch_jump(skip, self.next_pc()); + } + Ok(()) + } fn lower_top_level( &mut self, builder: &mut ModuleBuilder, @@ -2969,8 +3190,16 @@ impl<'a> FunctionContext<'a> { let range = declarator.range(); let data = declarator.data(); self.predeclare_captured_pattern(&data.binding, declaration_scope)?; + // `var x;` only declares, and hoisting has done that. The name keeps + // whatever it holds, which may be a global of the same name. + if declaration.kind == VariableKind::Var && data.initializer.is_none() { + continue; + } let value = match &data.initializer { - Some(initializer) => self.lower_expression(builder, initializer)?, + Some(initializer) => { + let name = self.binding_name(&data.binding)?; + self.lower_maybe_named(builder, initializer, name.as_deref())? + } None => { // A bare declaration (`let x;`) binds undefined. self.undefined(builder, range)? @@ -3610,6 +3839,77 @@ impl<'a> FunctionContext<'a> { // ------------------------------------------------------------------ // Expressions // ------------------------------------------------------------------ + /// `NamedEvaluation`: an anonymous function or class directly initializing + /// a name takes that name. Parentheses do not matter; a comma does, because + /// then the value is no longer a function definition. + fn lower_named_expression( + &mut self, + builder: &mut ModuleBuilder, + expression: &Expr, + name: &str, + ) -> Result { + let range = expression.range(); + match strip_parentheses(expression).data() { + Expression::Function(function) if function.function.name.is_none() => self + .build_constructible_function_value( + builder, + range, + Some(name.to_owned()), + &function.function, + ), + Expression::Arrow(arrow) => { + self.lower_arrow_named(builder, range, arrow, Some(name.to_owned())) + } + Expression::Class(class) if class.class.name.is_none() => { + self.inferred_class_name = Some(name.to_owned()); + self.lower_class_value(builder, range, &class.class, None, None) + } + _ => self.lower_expression(builder, expression), + } + } + /// Lowers `value`, naming it after `name` when one applies. + fn lower_maybe_named( + &mut self, + builder: &mut ModuleBuilder, + value: &Expr, + name: Option<&str>, + ) -> Result { + match name { + Some(name) => self.lower_named_expression(builder, value, name), + None => self.lower_expression(builder, value), + } + } + /// The name a binding gives its initializer, when it is a plain identifier. + fn binding_name(&self, pattern: &Pattern) -> Result, LowerError> { + match pattern.data() { + BindingPattern::Identifier(identifier) => self.identifier_text(identifier).map(Some), + _ => Ok(None), + } + } + /// The name an assignment target gives its value. `(x) = f` names nothing: + /// the target must be the bare identifier. + fn target_name(&self, target: &AssignmentTargetNode) -> Result, LowerError> { + match target.data() { + AssignmentTarget::Identifier(identifier) + if !crate::early_errors::is_parenthesized(self.file, target.range()) => + { + self.identifier_text(identifier).map(Some) + } + _ => Ok(None), + } + } + /// The name a property key gives a value, when it is known here. + fn property_name_text(&self, name: &PropertyName) -> Result, LowerError> { + match name { + PropertyName::Identifier(identifier) => self.identifier_text(identifier).map(Some), + PropertyName::String(literal) => Ok(self + .file + .token_text(literal.data().token()) + .map(|text| text.trim_matches(['\'', '"']).to_owned())), + PropertyName::Private(private) => self.private_text(private).map(Some), + _ => Ok(None), + } + } fn lower_expression( &mut self, builder: &mut ModuleBuilder, @@ -3680,7 +3980,13 @@ impl<'a> FunctionContext<'a> { Expression::Array(array) => self.lower_array(builder, range, array), Expression::Object(object) => self.lower_object(builder, range, object), Expression::Function(function) => { - self.build_constructible_function_value(builder, range, None, &function.function) + let name = function + .function + .name + .as_ref() + .map(|identifier| self.identifier_text(identifier)) + .transpose()?; + self.build_constructible_function_value(builder, range, name, &function.function) } Expression::Class(class) => { let name = class @@ -3887,6 +4193,11 @@ impl<'a> FunctionContext<'a> { if Self::has_optional_chain(argument) { return self.lower_optional_delete(builder, range, member); } + if matches!(member.object.data(), Expression::Super) { + self.this_value(builder, range)?; + self.member_key(builder, &member.property)?; + return self.throw_reference_error(builder, range); + } let object = self.lower_expression(builder, &member.object)?; let key = self.member_key(builder, &member.property)?; let dst = self.alloc_register(range)?; @@ -4130,7 +4441,9 @@ impl<'a> FunctionContext<'a> { // assign_name so each performs a fresh membership walk. match compound_operator(assignment.operator) { None => { - let value = self.lower_expression(builder, &assignment.right)?; + let inferred = self.target_name(&assignment.left)?; + let value = + self.lower_maybe_named(builder, &assignment.right, inferred.as_deref())?; self.assign_name(builder, name, value, range)?; Ok(value) } @@ -4155,7 +4468,9 @@ impl<'a> FunctionContext<'a> { let current = self.read_name(builder, name, range)?; self.move_to(range, result, current)?; let skip = self.branch_on_short_circuit(builder, range, op, current)?; - let value = self.lower_expression(builder, &assignment.right)?; + let inferred = self.target_name(&assignment.left)?; + let value = + self.lower_maybe_named(builder, &assignment.right, inferred.as_deref())?; self.assign_name(builder, name, value, range)?; self.move_to(range, result, value)?; let end = self.next_pc(); @@ -4171,24 +4486,21 @@ impl<'a> FunctionContext<'a> { member: &AssignmentMemberTarget, assignment: &AssignmentExpression, ) -> Result { - let object = self.lower_expression(builder, &member.object)?; - let key = self.member_key(builder, &member.property)?; - match compound_operator(assignment.operator) { + let base = self.member_base(builder, range, &member.object)?; + let compound = compound_operator(assignment.operator); + let key = if compound.is_some() { + self.read_write_key(builder, range, base, &member.property)? + } else { + self.member_key(builder, &member.property)? + }; + match compound { None => { let value = self.lower_expression(builder, &assignment.right)?; - self.emit(range, Instruction::SetProperty { object, key, value })?; + self.set_member(range, base, key, value)?; Ok(value) } Some(CompoundOp::Arithmetic(op)) => { - let current = self.alloc_register(range)?; - self.emit( - range, - Instruction::GetProperty { - dst: current, - object, - key, - }, - )?; + let current = self.get_member(range, base, key)?; let right = self.lower_expression(builder, &assignment.right)?; let result = self.alloc_register(range)?; self.emit( @@ -4200,31 +4512,16 @@ impl<'a> FunctionContext<'a> { right, }, )?; - self.emit( - range, - Instruction::SetProperty { - object, - key, - value: result, - }, - )?; + self.set_member(range, base, key, result)?; Ok(result) } Some(CompoundOp::Logical(op)) => { let result = self.alloc_register(range)?; - let current = self.alloc_register(range)?; - self.emit( - range, - Instruction::GetProperty { - dst: current, - object, - key, - }, - )?; + let current = self.get_member(range, base, key)?; self.move_to(range, result, current)?; let skip = self.branch_on_short_circuit(builder, range, op, current)?; let value = self.lower_expression(builder, &assignment.right)?; - self.emit(range, Instruction::SetProperty { object, key, value })?; + self.set_member(range, base, key, value)?; self.move_to(range, result, value)?; let end = self.next_pc(); self.patch_jump(skip, end); @@ -4268,17 +4565,9 @@ impl<'a> FunctionContext<'a> { if self.is_const_enum_member_target(&update.argument, &member.object)? { return Err(self.const_enum_operation(range, ConstEnumOperation::Write)); } - let object = self.lower_expression(builder, &member.object)?; - let key = self.member_key(builder, &member.property)?; - let old = self.alloc_register(range)?; - self.emit( - range, - Instruction::GetProperty { - dst: old, - object, - key, - }, - )?; + let base = self.member_base(builder, range, &member.object)?; + let key = self.read_write_key(builder, range, base, &member.property)?; + let old = self.get_member(range, base, key)?; let one = self.load_constant(builder, Constant::Int32(1), range)?; let updated = self.alloc_register(range)?; self.emit( @@ -4290,14 +4579,7 @@ impl<'a> FunctionContext<'a> { right: one, }, )?; - self.emit( - range, - Instruction::SetProperty { - object, - key, - value: updated, - }, - )?; + self.set_member(range, base, key, updated)?; Ok(if update.prefix { updated } else { old }) } AssignmentTarget::Object(_) | AssignmentTarget::Array(_) => { @@ -4788,11 +5070,150 @@ impl<'a> FunctionContext<'a> { }; return Err(self.const_enum_operation(range, operation)); } - let object = self.lower_expression(builder, &member.object)?; + let base = self.member_base(builder, range, &member.object)?; let key = self.member_key(builder, &member.property)?; + let dst = self.get_member(range, base, key)?; + Ok((base.this_value(), dst)) + } + /// The base of a member reference. `super` reads from above the home + /// object, with the current `this` as the receiver. + fn member_base( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + object: &Expr, + ) -> Result { + if let (Expression::Super, Some(home)) = (object.data(), self.home_object) { + let receiver = self.this_value(builder, range)?; + return Ok(MemberBase::Super { home, receiver }); + } + Ok(MemberBase::Object(self.lower_expression(builder, object)?)) + } + /// A class constructor runs only through `new`. + fn reject_plain_call( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + ) -> Result<(), LowerError> { + let new_target = self.new_target_value(range)?; + let undefined = self.undefined(builder, range)?; + let called = self.alloc_register(range)?; + self.emit( + range, + Instruction::Binary { + dst: called, + op: BinaryOp::StrictEqual, + left: new_target, + right: undefined, + }, + )?; + let constructed = self.emit( + range, + Instruction::JumpIfFalse { + condition: called, + target: Pc::new(0), + }, + )?; + self.throw_error(builder, range, "TypeError")?; + self.patch_jump(constructed, self.next_pc()); + Ok(()) + } + /// Throws a `ReferenceError`, which is what deleting through `super` does. + fn throw_reference_error( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + ) -> Result { + self.throw_error(builder, range, "ReferenceError")?; + self.undefined(builder, range) + } + fn throw_error( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + kind: &str, + ) -> Result<(), LowerError> { + let name = builder.intern(Constant::String(EcmaString::encode(kind)), range)?; + let callee = self.alloc_register(range)?; + self.emit(range, Instruction::LoadGlobal { dst: callee, name })?; + let arguments = self.alloc_register(range)?; + self.emit(range, Instruction::CreateArray { dst: arguments })?; + let value = self.alloc_register(range)?; + self.emit( + range, + Instruction::Construct { + dst: value, + callee, + arguments, + }, + )?; + self.emit(range, Instruction::Throw { value })?; + Ok(()) + } + /// The key of a reference that is read and then written, converted now so + /// user conversion runs once. A reference through nothing still fails first. + fn read_write_key( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + base: MemberBase, + property: &MemberProperty, + ) -> Result { + let operand = self.member_key(builder, property)?; + if !matches!(property, MemberProperty::Computed(_)) { + return Ok(operand); + } + if let MemberBase::Object(object) = base { + self.require_object_coercible(range, object)?; + } let dst = self.alloc_register(range)?; - self.emit(range, Instruction::GetProperty { dst, object, key })?; - Ok((object, dst)) + self.emit( + range, + Instruction::Unary { + dst, + op: UnaryOp::ToPropertyKey, + operand, + }, + )?; + Ok(dst) + } + fn get_member( + &mut self, + range: TextRange, + base: MemberBase, + key: Register, + ) -> Result { + let dst = self.alloc_register(range)?; + let instruction = match base { + MemberBase::Object(object) => Instruction::GetProperty { dst, object, key }, + MemberBase::Super { home, receiver } => Instruction::GetSuper { + dst, + home, + receiver, + key, + }, + }; + self.emit(range, instruction)?; + Ok(dst) + } + fn set_member( + &mut self, + range: TextRange, + base: MemberBase, + key: Register, + value: Register, + ) -> Result<(), LowerError> { + let instruction = match base { + MemberBase::Object(object) => Instruction::SetProperty { object, key, value }, + MemberBase::Super { home, receiver } => Instruction::SetSuper { + home, + receiver, + key, + value, + }, + }; + self.emit(range, instruction)?; + Ok(()) } fn has_optional_chain(mut expression: &Expr) -> bool { loop { @@ -4952,17 +5373,15 @@ impl<'a> FunctionContext<'a> { Expression::Member(member) if !member.optional => { if matches!(member.object.data(), Expression::Super) { let this_value = self.this_value(builder, range)?; - let object = this_value; - let key = self.member_key(builder, &member.property)?; - let value = self.alloc_register(range)?; - self.emit( - range, - Instruction::GetProperty { - dst: value, - object, - key, + let base = match self.home_object { + Some(home) => MemberBase::Super { + home, + receiver: this_value, }, - )?; + None => MemberBase::Object(this_value), + }; + let key = self.member_key(builder, &member.property)?; + let value = self.get_member(range, base, key)?; return Ok((value, this_value)); } let (object, value) = self.lower_member(builder, callee.range(), callee)?; @@ -5173,16 +5592,7 @@ impl<'a> FunctionContext<'a> { }, )?; } - ArrayElement::Elision => { - let hole = self.undefined(builder, range)?; - self.emit( - range, - Instruction::ArrayPush { - array: dst, - value: hole, - }, - )?; - } + ArrayElement::Elision => self.extend_array_length(builder, range, dst)?, ArrayElement::Missing(missing) => { return Err(self.missing(range, missing.expected())); } @@ -5190,6 +5600,45 @@ impl<'a> FunctionContext<'a> { } Ok(dst) } + /// An elision has no element, so it only makes the array one longer. That + /// leaves a hole, which an element holding `undefined` would not be. + fn extend_array_length( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + array: Register, + ) -> Result<(), LowerError> { + let key = self.string_reg(builder, EcmaString::from("length"), range)?; + let length = self.alloc_register(range)?; + self.emit( + range, + Instruction::GetProperty { + dst: length, + object: array, + key, + }, + )?; + let one = self.load_constant(builder, Constant::Int32(1), range)?; + let value = self.alloc_register(range)?; + self.emit( + range, + Instruction::Binary { + dst: value, + op: BinaryOp::Add, + left: length, + right: one, + }, + )?; + self.emit( + range, + Instruction::SetProperty { + object: array, + key, + value, + }, + )?; + Ok(()) + } fn lower_object( &mut self, builder: &mut ModuleBuilder, @@ -5201,8 +5650,21 @@ impl<'a> FunctionContext<'a> { for member in &object.members { match member.data() { ObjectMember::Property(property) => { - let key = self.property_key(builder, &property.name)?; - let value = self.lower_expression(builder, &property.value)?; + let key = self.converted_property_key(builder, range, &property.name)?; + let name = self.property_name_text(&property.name)?; + let value = + self.lower_maybe_named(builder, &property.value, name.as_deref())?; + if sets_prototype(property, name.as_deref()) { + self.emit( + member.range(), + Instruction::SetProperty { + object: dst, + key, + value, + }, + )?; + continue; + } self.install_property( builder, member.range(), @@ -5213,9 +5675,12 @@ impl<'a> FunctionContext<'a> { )?; } ObjectMember::Method(method) => { - let key = self.property_key(builder, &method.name)?; + let key = self.converted_property_key(builder, range, &method.name)?; + self.method_home = Some(dst); let value = - self.build_function_value(builder, member.range(), None, &method.function)?; + self.build_function_value(builder, member.range(), None, &method.function); + self.method_home = None; + let value = value?; self.install_property( builder, member.range(), @@ -5242,6 +5707,28 @@ impl<'a> FunctionContext<'a> { } Ok(dst) } + /// A literal's key, converted before its value runs. + fn converted_property_key( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + name: &PropertyName, + ) -> Result { + let operand = self.property_key(builder, name)?; + if !matches!(name, PropertyName::Computed(_)) { + return Ok(operand); + } + let dst = self.alloc_register(range)?; + self.emit( + range, + Instruction::Unary { + dst, + op: UnaryOp::ToPropertyKey, + operand, + }, + )?; + Ok(dst) + } fn install_property( &mut self, _builder: &mut ModuleBuilder, @@ -5253,7 +5740,15 @@ impl<'a> FunctionContext<'a> { ) -> Result<(), LowerError> { match modifier { PropertyModifier::None => { - self.emit(range, Instruction::SetProperty { object, key, value })?; + self.emit( + range, + Instruction::DefineOwnDescriptorSlot { + object, + key, + src: value, + slot: DescriptorSlot::Entire, + }, + )?; } PropertyModifier::Get => { self.emit( @@ -5280,6 +5775,37 @@ impl<'a> FunctionContext<'a> { } Ok(()) } + /// Installs a method or accessor on a class, where neither is enumerable. + fn install_class_method( + &mut self, + range: TextRange, + object: Register, + key: Register, + value: Register, + modifier: PropertyModifier, + ) -> Result<(), LowerError> { + let kind = match modifier { + PropertyModifier::None => { + self.emit( + range, + Instruction::DefineDataProperty { object, key, value }, + )?; + return Ok(()); + } + PropertyModifier::Get => AccessorKind::ClassGetter, + PropertyModifier::Set => AccessorKind::ClassSetter, + }; + self.emit( + range, + Instruction::DefineAccessor { + object, + key, + accessor: value, + kind, + }, + )?; + Ok(()) + } fn set_named_entry( &mut self, builder: &mut ModuleBuilder, @@ -5694,6 +6220,17 @@ impl<'a> FunctionContext<'a> { let mut acc = self.string_reg(builder, first, range)?; for (index, expression) in template.expressions.iter().enumerate() { let value = self.lower_expression(builder, expression)?; + // `ToString`, not the `+` operator's `ToPrimitive`: `valueOf` has no + // say here, and a Symbol must throw. + let text = self.alloc_register(range)?; + self.emit( + range, + Instruction::Unary { + dst: text, + op: UnaryOp::ToString, + operand: value, + }, + )?; let joined = self.alloc_register(range)?; self.emit( range, @@ -5701,7 +6238,7 @@ impl<'a> FunctionContext<'a> { dst: joined, op: BinaryOp::Add, left: acc, - right: value, + right: text, }, )?; acc = joined; @@ -6381,6 +6918,7 @@ impl<'a> FunctionContext<'a> { ) } BindingPattern::Object(object) => { + self.require_object_coercible(range, value)?; let mut taken: Vec = Vec::new(); for property in &object.properties { if let BindingPattern::Rest(rest) = property.binding.data() { @@ -6388,7 +6926,7 @@ impl<'a> FunctionContext<'a> { self.bind_pattern(builder, &rest.argument, rest_value, declaration_scope)?; continue; } - let key = self.property_key(builder, &property.name)?; + let key = self.converted_property_key(builder, range, &property.name)?; taken.push(key); let element = self.alloc_register(range)?; self.emit( @@ -6400,7 +6938,10 @@ impl<'a> FunctionContext<'a> { }, )?; let element = match &property.initializer { - Some(default) => self.apply_default(builder, range, element, default)?, + Some(default) => { + let name = self.binding_name(&property.binding)?; + self.apply_default(builder, range, element, default, name.as_deref())? + } None => element, }; self.bind_pattern(builder, &property.binding, element, declaration_scope)?; @@ -6436,12 +6977,23 @@ impl<'a> FunctionContext<'a> { let value = self.iterator_step_value(builder, range, iterator)?; let value = match default { Some(default) => { - self.apply_default(builder, range, value, default)? + let name = self.binding_name(element_value)?; + self.apply_default( + builder, + range, + value, + default, + name.as_deref(), + )? } None => value, }; - let _ = element_value; - self.bind_pattern(builder, inner, value, declaration_scope)?; + self.bind_pattern( + builder, + element_value, + value, + declaration_scope, + )?; } } ArrayBindingElement::Missing(missing) => { @@ -6452,7 +7004,9 @@ impl<'a> FunctionContext<'a> { Ok(()) } BindingPattern::Assignment(assignment) => { - let value = self.apply_default(builder, range, value, &assignment.right)?; + let name = self.binding_name(&assignment.left)?; + let value = + self.apply_default(builder, range, value, &assignment.right, name.as_deref())?; self.bind_pattern(builder, &assignment.left, value, declaration_scope) } BindingPattern::Rest(rest) => { @@ -6474,6 +7028,7 @@ impl<'a> FunctionContext<'a> { range: TextRange, value: Register, default: &Expr, + name: Option<&str>, ) -> Result { let result = self.alloc_register(range)?; self.move_to(range, result, value)?; @@ -6495,7 +7050,7 @@ impl<'a> FunctionContext<'a> { target: Pc::new(0), }, )?; - let default_value = self.lower_expression(builder, default)?; + let default_value = self.lower_maybe_named(builder, default, name)?; self.move_to(range, result, default_value)?; let end = self.next_pc(); self.patch_jump(skip, end); @@ -6610,6 +7165,17 @@ impl<'a> FunctionContext<'a> { } Ok(rest) } + /// An object pattern refuses null and undefined before anything else, which + /// reading its first property could not do for `{}` or a computed key. + fn require_object_coercible( + &mut self, + range: TextRange, + value: Register, + ) -> Result<(), LowerError> { + let dst = self.alloc_register(range)?; + self.emit(range, Instruction::ToObject { dst, src: value })?; + Ok(()) + } fn assign_target( &mut self, builder: &mut ModuleBuilder, @@ -6623,15 +7189,11 @@ impl<'a> FunctionContext<'a> { self.assign_name(builder, &name, value, range) } AssignmentTarget::Member(member) => { - if self.is_const_enum_member_target(target, &member.object)? { - return Err(self.const_enum_operation(range, ConstEnumOperation::Write)); - } - let object = self.lower_expression(builder, &member.object)?; - let key = self.member_key(builder, &member.property)?; - self.emit(range, Instruction::SetProperty { object, key, value })?; - Ok(()) + let (base, key) = self.member_reference(builder, target, member)?; + self.set_member(range, base, key, value) } AssignmentTarget::Object(object) => { + self.require_object_coercible(range, value)?; let mut taken: Vec = Vec::new(); for property in &object.properties { self.assign_object_property(builder, range, value, property, &mut taken)?; @@ -6654,8 +7216,12 @@ impl<'a> FunctionContext<'a> { self.iterator_step_discard(range, iterator)?; } AssignmentArrayElement::Target(inner) => { - let element = self.iterator_step_value(builder, range, iterator)?; - self.assign_target(builder, inner, element)?; + self.assign_array_element(builder, range, inner, iterator)?; + } + AssignmentArrayElement::Rest(inner) => { + let reference = self.target_reference(builder, inner)?; + let rest = self.rest_array(builder, range, iterator)?; + self.assign_reference(builder, inner, reference, rest)?; } AssignmentArrayElement::Missing(missing) => { return Err(self.missing(range, missing.expected())); @@ -6670,6 +7236,75 @@ impl<'a> FunctionContext<'a> { } } } + fn member_reference( + &mut self, + builder: &mut ModuleBuilder, + target: &AssignmentTargetNode, + member: &AssignmentMemberTarget, + ) -> Result<(MemberBase, Register), LowerError> { + let range = target.range(); + if self.is_const_enum_member_target(target, &member.object)? { + return Err(self.const_enum_operation(range, ConstEnumOperation::Write)); + } + let base = self.member_base(builder, range, &member.object)?; + let key = self.member_key(builder, &member.property)?; + Ok((base, key)) + } + /// A destructuring target's reference comes before the value it receives, + /// so reading that value can observe the target's side effects. + fn target_reference( + &mut self, + builder: &mut ModuleBuilder, + target: &AssignmentTargetNode, + ) -> Result, LowerError> { + match target.data() { + AssignmentTarget::Member(member) => { + self.member_reference(builder, target, member).map(Some) + } + _ => Ok(None), + } + } + fn assign_reference( + &mut self, + builder: &mut ModuleBuilder, + target: &AssignmentTargetNode, + reference: Option<(MemberBase, Register)>, + value: Register, + ) -> Result<(), LowerError> { + match reference { + Some((base, key)) => self.set_member(target.range(), base, key, value), + None => self.assign_target(builder, target, value), + } + } + /// Assigns one array element. The parser keeps `[a = 1]`'s element as the + /// assignment it read, so a default arrives as that expression. + fn assign_array_element( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + target: &AssignmentTargetNode, + iterator: Register, + ) -> Result<(), LowerError> { + if let AssignmentTarget::Invalid(operand) = target.data() + && let Expression::Assignment(assignment) = operand.data() + && assignment.operator == AssignmentOperator::Assign + { + let reference = self.target_reference(builder, &assignment.left)?; + let value = self.iterator_step_value(builder, range, iterator)?; + let name = self.target_name(&assignment.left)?; + let value = self.apply_default( + builder, + target.range(), + value, + &assignment.right, + name.as_deref(), + )?; + return self.assign_reference(builder, &assignment.left, reference, value); + } + let reference = self.target_reference(builder, target)?; + let value = self.iterator_step_value(builder, range, iterator)?; + self.assign_reference(builder, target, reference, value) + } fn assign_object_property( &mut self, builder: &mut ModuleBuilder, @@ -6678,8 +7313,15 @@ impl<'a> FunctionContext<'a> { property: &AssignmentObjectProperty, taken: &mut Vec, ) -> Result<(), LowerError> { - let key = self.property_key(builder, &property.name)?; + if property.rest { + let reference = self.target_reference(builder, &property.target)?; + let rest = self.rest_object(builder, range, source, taken)?; + return self.assign_reference(builder, &property.target, reference, rest); + } + // Converted now: the key must not convert again for the target or `rest`. + let key = self.converted_property_key(builder, range, &property.name)?; taken.push(key); + let reference = self.target_reference(builder, &property.target)?; let element = self.alloc_register(range)?; self.emit( range, @@ -6690,21 +7332,33 @@ impl<'a> FunctionContext<'a> { }, )?; let element = match &property.initializer { - Some(default) => self.apply_default(builder, range, element, default)?, + Some(default) => { + let name = self.target_name(&property.target)?; + self.apply_default(builder, range, element, default, name.as_deref())? + } None => element, }; - self.assign_target(builder, &property.target, element) + self.assign_reference(builder, &property.target, reference, element) } fn lower_arrow( &mut self, builder: &mut ModuleBuilder, range: TextRange, arrow: &ArrowFunction, + ) -> Result { + self.lower_arrow_named(builder, range, arrow, None) + } + fn lower_arrow_named( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + arrow: &ArrowFunction, + name: Option, ) -> Result { let flags = FunctionFlags { is_async: arrow.is_async, is_generator: false, - is_constructable: false, + ..FunctionFlags::default() }; let captures = self.compute_captures(&arrow.parameters, LoweredBody::Arrow(&arrow.body), true); @@ -6713,7 +7367,7 @@ impl<'a> FunctionContext<'a> { builder, id, range, - None, + name, &arrow.parameters, LoweredBody::Arrow(&arrow.body), flags, @@ -6733,6 +7387,7 @@ impl<'a> FunctionContext<'a> { is_async: function.is_async, is_generator: function.is_generator, is_constructable: !function.is_async && !function.is_generator, + ..FunctionFlags::default() }; let closure = self.build_function_value_with_flags(builder, range, name, function, flags)?; @@ -6771,7 +7426,7 @@ impl<'a> FunctionContext<'a> { let flags = FunctionFlags { is_async: function.is_async, is_generator: function.is_generator, - is_constructable: false, + ..FunctionFlags::default() }; self.build_function_value_with_flags(builder, range, name, function, flags) } @@ -6849,15 +7504,26 @@ impl<'a> FunctionContext<'a> { this_capture: None, this_cell: None, derived_super_guard: None, - instance_steps: Vec::new(), + derived_constructor: false, + field_scope: BTreeMap::new(), + // A `super(...)` in an arrow initializes the fields. + instance_steps: if is_arrow { + self.instance_steps.clone() + } else { + Vec::new() + }, new_target_capture: None, parent_constructor_capture: None, class_elements: None, + home_object: None, + method_home: None, arguments_source: if is_arrow { ArgumentsSource::None } else { ArgumentsSource::Own }, + strict: self.strict || body.has_use_strict(self.file), + inferred_class_name: None, }; // Leading capture registers. for capture in captures { @@ -6880,6 +7546,15 @@ impl<'a> FunctionContext<'a> { CaptureKey::NewTarget => inner.new_target_capture = Some(register), CaptureKey::Parent(_) => inner.parent_constructor_capture = Some(register), CaptureKey::ClassElements(_) => inner.class_elements = Some(register), + CaptureKey::Home(_) => inner.home_object = Some(register), + CaptureKey::SuperGuard(_) => { + inner.derived_super_guard = Some(SuperGuard::Cell(register)); + } + CaptureKey::FieldName { name, sites, .. } => { + inner + .field_scope + .insert(name.clone(), (register, sites.clone())); + } CaptureKey::Cell(_) => {} CaptureKey::Container(symbol, kind, _) => inner.containers.push(Container { symbol: *symbol, @@ -6905,6 +7580,13 @@ impl<'a> FunctionContext<'a> { } else if let LoweredBody::Block(block) = body { inner.hoist_vars(builder, &block.data().statements, range)?; } + // A generator binds its parameters when called, before the body's + // first `next`. Parameters that can run code stop here for that call. + let binds_at_call = flags.is_generator && !has_simple_parameters(parameters); + if binds_at_call { + let value = inner.undefined(builder, range)?; + inner.emit_suspend(builder, range, value)?; + } match body { LoweredBody::Function(FunctionBody::Block(block)) | LoweredBody::Arrow(FunctionBody::Block(block)) @@ -6928,6 +7610,10 @@ impl<'a> FunctionContext<'a> { Some(name) => Some(builder.intern(Constant::String(EcmaString::encode(&name)), range)?), None => None, }; + let flags = FunctionFlags { + binds_at_call, + ..flags + }; let assembled = inner.into_function(name_constant, flags); builder.fill_function(id, assembled); Ok(()) @@ -6984,11 +7670,17 @@ impl<'a> FunctionContext<'a> { this_capture: None, this_cell: None, derived_super_guard: None, + derived_constructor: false, + field_scope: BTreeMap::new(), instance_steps: Vec::new(), new_target_capture: None, parent_constructor_capture: None, class_elements: None, + home_object: None, + method_home: None, arguments_source: ArgumentsSource::Own, + strict: self.strict, + inferred_class_name: None, }; // Leading capture registers, in the same order the materialized // captures array pushes them. @@ -7012,6 +7704,15 @@ impl<'a> FunctionContext<'a> { CaptureKey::NewTarget => inner.new_target_capture = Some(register), CaptureKey::Parent(_) => inner.parent_constructor_capture = Some(register), CaptureKey::ClassElements(_) => inner.class_elements = Some(register), + CaptureKey::Home(_) => inner.home_object = Some(register), + CaptureKey::SuperGuard(_) => { + inner.derived_super_guard = Some(SuperGuard::Cell(register)); + } + CaptureKey::FieldName { name, sites, .. } => { + inner + .field_scope + .insert(name.clone(), (register, sites.clone())); + } CaptureKey::Cell(_) => {} CaptureKey::Container(symbol, kind, _) => inner.containers.push(Container { symbol: *symbol, @@ -7047,6 +7748,7 @@ impl<'a> FunctionContext<'a> { parameters: &[ParameterNode], range: TextRange, ) -> Result<(Vec, Option), LowerError> { + let parameters = runtime_parameters(self.file, parameters); let rest_index = parameters.iter().position(|parameter| { matches!(parameter.data().binding.data(), BindingPattern::Rest(_)) }); @@ -7066,6 +7768,7 @@ impl<'a> FunctionContext<'a> { slots: &[Register], rest_index: Option, ) -> Result<(), LowerError> { + let parameters = runtime_parameters(self.file, parameters); let fixed = slots.len(); let mut undefined_seed = None; for (index, parameter) in parameters.iter().enumerate() { @@ -7113,7 +7816,10 @@ impl<'a> FunctionContext<'a> { let data = parameter.data(); let slot = slots[index]; let value = match &data.initializer { - Some(default) => self.apply_default(builder, parameter.range(), slot, default)?, + Some(default) => { + let name = self.binding_name(&data.binding)?; + self.apply_default(builder, parameter.range(), slot, default, name.as_deref())? + } None => slot, }; self.bind_pattern(builder, &data.binding, value, DeclarationScope::Function)?; @@ -7166,7 +7872,7 @@ impl<'a> FunctionContext<'a> { range: TextRange, value: Register, ) -> Result<(), LowerError> { - if self.this_cell.is_none() || self.parent_constructor_capture.is_none() { + if !self.derived_constructor { self.emit(range, Instruction::Return { value })?; return Ok(()); } @@ -7354,6 +8060,9 @@ impl<'a> FunctionContext<'a> { CaptureKey::NewTarget => self.new_target_value(range), CaptureKey::Parent(parent) => Ok(*parent), CaptureKey::ClassElements(table) => Ok(*table), + CaptureKey::Home(home) => Ok(*home), + CaptureKey::SuperGuard(guard) => Ok(*guard), + CaptureKey::FieldName { cell, .. } => Ok(*cell), CaptureKey::Cell(cell) => Ok(*cell), CaptureKey::Container(_, _, object) | CaptureKey::WithObject(_, object) => Ok(*object), } @@ -7366,6 +8075,9 @@ impl<'a> FunctionContext<'a> { ) -> Vec { let mut scanner = FreeVarScanner::new(self.file, self.jsx_plan); scanner.scan_function(parameters, body, is_arrow); + // `super(...)` in an arrow finishes the enclosing constructor's work. + let calls_parent = + is_arrow && scanner.calls_super && self.parent_constructor_capture.is_some(); let mut captures = Vec::new(); for name in &scanner.free { if matches!(self.resolve(name), Some(Binding::ConstEnum(_))) { @@ -7389,9 +8101,33 @@ impl<'a> FunctionContext<'a> { if scanner.uses_arguments { captures.push(CaptureKey::Arguments); } - if scanner.uses_new_target { + if scanner.uses_new_target || calls_parent { captures.push(CaptureKey::NewTarget); } + if calls_parent { + captures.extend(self.parent_constructor_capture.map(CaptureKey::Parent)); + captures.extend(self.class_elements.map(CaptureKey::ClassElements)); + if let Some(SuperGuard::Cell(guard)) = self.derived_super_guard { + captures.push(CaptureKey::SuperGuard(guard)); + } + // The arrow runs the field initializers there. + captures.extend(self.field_scope.iter().map(|(name, (cell, sites))| { + CaptureKey::FieldName { + name: name.clone(), + sites: sites.clone(), + cell: *cell, + } + })); + } + } + // An arrow's `super` is the enclosing method's. + let home = if is_arrow { + self.home_object + } else { + self.method_home + }; + if let (true, Some(home)) = (scanner.uses_super, home) { + captures.push(CaptureKey::Home(home)); } for container in &self.containers { captures.push(CaptureKey::Container( @@ -7678,21 +8414,35 @@ impl<'a> FunctionContext<'a> { key, init_chain, extra_inits, + defines, } => { let raw = match &property.initializer { - Some(initializer) => self.lower_expression(builder, initializer)?, + Some(initializer) => { + let name = self.property_name_text(&property.name)?; + self.lower_maybe_named(builder, initializer, name.as_deref())? + } None => self.undefined(builder, range)?, }; let value = self.run_initializer_chain(builder, range, final_class, *init_chain, raw)?; - self.emit( - range, + // A field defines its property, so `static name` replaces the + // function's own read-only one. A private name has none to define. + let instruction = if *defines && !matches!(property.name, PropertyName::Private(_)) + { + Instruction::DefineOwnDescriptorSlot { + object: raw_ctor, + key: *key, + src: value, + slot: DescriptorSlot::Entire, + } + } else { Instruction::SetProperty { object: raw_ctor, key: *key, value, - }, - )?; + } + }; + self.emit(range, instruction)?; self.run_extra_initializers(builder, range, final_class, *extra_inits)?; Ok(()) } @@ -7730,6 +8480,7 @@ impl<'a> FunctionContext<'a> { } } } + /// Lowers a class, every part of which is strict code. fn lower_class_value( &mut self, builder: &mut ModuleBuilder, @@ -7738,6 +8489,21 @@ impl<'a> FunctionContext<'a> { declaration_name: Option<&str>, expression_name: Option<(&str, BindingSite)>, ) -> Result { + let strict = std::mem::replace(&mut self.strict, true); + let value = + self.lower_strict_class_value(builder, range, class, declaration_name, expression_name); + self.strict = strict; + value + } + fn lower_strict_class_value( + &mut self, + builder: &mut ModuleBuilder, + range: TextRange, + class: &ClassDeclaration, + declaration_name: Option<&str>, + expression_name: Option<(&str, BindingSite)>, + ) -> Result { + let inferred_name = self.inferred_class_name.take(); let class_decorators = class .decorators .iter() @@ -7818,7 +8584,13 @@ impl<'a> FunctionContext<'a> { Some(identifier) => Some(self.identifier_text(identifier)?), None => None, }; - let source_name = owned_class_name.as_deref().or(declaration_name); + let source_name = owned_class_name + .as_deref() + .or(declaration_name) + .or(inferred_name.as_deref()); + // The constructor's `super` starts from this object, so it comes first. + let prototype = self.alloc_register(range)?; + self.emit(range, Instruction::CreateObject { dst: prototype })?; let ctor = self.build_constructor( builder, range, @@ -7830,10 +8602,9 @@ impl<'a> FunctionContext<'a> { } else { Some(class_elements) }, + prototype, source_name, )?; - let prototype = self.alloc_register(range)?; - self.emit(range, Instruction::CreateObject { dst: prototype })?; if let Some(parent) = parent { let parent_prototype = self.get_named(builder, range, parent, "prototype")?; self.emit( @@ -7888,6 +8659,13 @@ impl<'a> FunctionContext<'a> { &mut static_inits, )?; } + if !class.decorators.is_empty() { + for init in &mut static_inits { + if let StaticInit::Field { defines, .. } = init { + *defines = false; + } + } + } self.apply_member_decoration_stages(builder, stages)?; let class_extra_inits = self.alloc_register(range)?; self.emit( @@ -7924,6 +8702,7 @@ impl<'a> FunctionContext<'a> { // on `final_class`, then the remaining static timeline. let previous_this = self.this_capture; self.this_capture = Some(final_class); + let previous_home = self.home_object.replace(ctor); let mut static_result = Ok(()); let static_order = static_inits .iter() @@ -7940,6 +8719,7 @@ impl<'a> FunctionContext<'a> { } } self.this_capture = previous_this; + self.home_object = previous_home; static_result?; self.run_extra_initializers(builder, range, final_class, class_extra_inits)?; if let Some(name) = declaration_name { @@ -8098,6 +8878,7 @@ impl<'a> FunctionContext<'a> { parent: Option, instance_steps: &[InstanceInit], class_elements: Option, + home: Register, source_name: Option<&str>, ) -> Result { let (parameters, body_block): (&[ParameterNode], Option<&Block>) = match constructor { @@ -8117,6 +8898,7 @@ impl<'a> FunctionContext<'a> { instance_steps, parent, class_elements, + home, ); let id = builder.reserve_function(range)?; self.build_constructor_into( @@ -8139,29 +8921,29 @@ impl<'a> FunctionContext<'a> { instance_steps: &[InstanceInit], parent: Option, class_elements: Option, + home: Register, ) -> Vec { let mut scanner = FreeVarScanner::new(self.file, self.jsx_plan); - scanner.preseed_parameters(parameters); - if let Some(block) = body { - scanner.preseed_vars(&block.statements); - } - scanner.scan_parameter_initializers(parameters); - if let Some(block) = body { - for statement in &block.statements { - scanner.scan_statement(statement); - } - } - scan_instance_init_free_vars(&mut scanner, instance_steps); + scan_constructor(&mut scanner, parameters, body); + // Initializers scan alone: a constructor binding does not shadow them. + let mut fields = FreeVarScanner::new(self.file, self.jsx_plan); + scan_instance_init_free_vars(&mut fields, instance_steps); let mut captures = Vec::new(); - for name in &scanner.free { - if matches!(self.resolve(name), Some(Binding::ConstEnum(_))) { - continue; - } - if self.resolve(name).is_some() { - captures.push(CaptureKey::Name( + for name in scanner.free.union(&fields.free) { + match self.resolve(name) { + Some(Binding::ConstEnum(_)) | None => {} + Some(_) if scanner.free.contains(name) => captures.push(CaptureKey::Name( name.clone(), self.preceding_with_sites(name), - )); + )), + Some(Binding::Cell(cell)) => captures.push(CaptureKey::FieldName { + name: name.clone(), + sites: self.preceding_with_sites(name), + cell, + }), + Some(Binding::Local(_)) => { + panic!("capture plan resolved field capture `{name}` to Local") + } } } if let Some(parent) = parent { @@ -8170,6 +8952,9 @@ impl<'a> FunctionContext<'a> { if let Some(table) = class_elements { captures.push(CaptureKey::ClassElements(table)); } + if scanner.uses_super || fields.uses_super { + captures.push(CaptureKey::Home(home)); + } for container in &self.containers { captures.push(CaptureKey::Container( container.symbol, @@ -8177,22 +8962,42 @@ impl<'a> FunctionContext<'a> { container.object, )); } - if !scanner.free.is_empty() { + if !scanner.free.is_empty() || !fields.free.is_empty() { for region in &self.with_regions { captures.push(CaptureKey::WithObject(region.site, region.object)); } } captures } + /// Swaps in what field initializers see, returning what to restore. + fn enter_field_scope(&mut self) -> NameView { + let bindings = self + .field_scope + .iter() + .map(|(name, (cell, _))| (name.clone(), Binding::Cell(*cell))) + .collect(); + let sites = self + .field_scope + .iter() + .map(|(name, (_, sites))| (name.clone(), sites.clone())) + .collect(); + NameView { + scopes: std::mem::replace(&mut self.scopes, vec![bindings]), + captured_names: std::mem::replace(&mut self.captured_names, sites), + } + } fn initialize_instance_fields( &mut self, builder: &mut ModuleBuilder, range: TextRange, ) -> Result<(), LowerError> { - let steps = self.instance_steps.clone(); + // Taken so an initializer's own arrows carry no steps: only a + // diagnosed source has `super(...)` there, and it must still lower. + let steps = std::mem::take(&mut self.instance_steps); if steps.is_empty() { return Ok(()); } + let outer = self.enter_field_scope(); let table = self.class_elements; // Method/getter/setter extras are Decorated { initializer: None }. // Drain them as a prefix; field/auto-accessor steps keep source order. @@ -8263,6 +9068,9 @@ impl<'a> FunctionContext<'a> { } } } + self.scopes = outer.scopes; + self.captured_names = outer.captured_names; + self.instance_steps = steps; Ok(()) } fn read_table_slot( @@ -8308,13 +9116,15 @@ impl<'a> FunctionContext<'a> { builder: &mut ModuleBuilder, range: TextRange, ) -> Result<(), LowerError> { - let Some(guard) = self.derived_super_guard else { - return Ok(()); + let initialized = match self.derived_super_guard { + Some(SuperGuard::Register(guard)) => guard, + Some(SuperGuard::Cell(guard)) => self.cell_value(builder, guard, range)?, + None => return Ok(()), }; let first_call = self.emit( range, Instruction::JumpIfFalse { - condition: guard, + condition: initialized, target: Pc::new(0), }, )?; @@ -8329,9 +9139,13 @@ impl<'a> FunctionContext<'a> { builder: &mut ModuleBuilder, range: TextRange, ) -> Result<(), LowerError> { - if let Some(guard) = self.derived_super_guard { - let initialized = self.load_constant(builder, Constant::Boolean(true), range)?; - self.move_to(range, guard, initialized)?; + let Some(guard) = self.derived_super_guard else { + return Ok(()); + }; + let initialized = self.load_constant(builder, Constant::Boolean(true), range)?; + match guard { + SuperGuard::Register(guard) => self.move_to(range, guard, initialized)?, + SuperGuard::Cell(guard) => self.store_cell(builder, guard, initialized, range)?, } Ok(()) } @@ -8467,11 +9281,17 @@ impl<'a> FunctionContext<'a> { this_capture: None, this_cell: None, derived_super_guard: None, + derived_constructor: false, + field_scope: BTreeMap::new(), instance_steps: instance_steps.to_vec(), new_target_capture: None, parent_constructor_capture: None, class_elements: None, + home_object: None, + method_home: None, arguments_source: ArgumentsSource::Own, + strict: true, + inferred_class_name: None, }; for capture in captures { let register = inner.alloc_register(range)?; @@ -8484,9 +9304,18 @@ impl<'a> FunctionContext<'a> { DeclarationScope::Function, ); inner.captured_names.insert(name.clone(), sites.clone()); + inner + .field_scope + .insert(name.clone(), (register, sites.clone())); + } + CaptureKey::FieldName { name, sites, .. } => { + inner + .field_scope + .insert(name.clone(), (register, sites.clone())); } CaptureKey::Parent(_) => inner.parent_constructor_capture = Some(register), CaptureKey::ClassElements(_) => inner.class_elements = Some(register), + CaptureKey::Home(_) => inner.home_object = Some(register), CaptureKey::Cell(_) => {} CaptureKey::Container(symbol, kind, _) => inner.containers.push(Container { symbol: *symbol, @@ -8501,7 +9330,8 @@ impl<'a> FunctionContext<'a> { CaptureKey::This | CaptureKey::ThisCell | CaptureKey::Arguments - | CaptureKey::NewTarget => { + | CaptureKey::NewTarget + | CaptureKey::SuperGuard(_) => { unreachable!("constructors do not capture arrow-only bindings") } } @@ -8511,13 +9341,23 @@ impl<'a> FunctionContext<'a> { let cell = inner.alloc_register(range)?; inner.emit(range, Instruction::CreateCell { dst: cell })?; inner.this_cell = Some(cell); + inner.derived_constructor = true; if body.is_some() { + let mut scanner = FreeVarScanner::new(inner.file, inner.jsx_plan); + scan_constructor(&mut scanner, parameters, body); let guard = inner.alloc_register(range)?; let pending = inner.load_constant(builder, Constant::Boolean(false), range)?; - inner.move_to(range, guard, pending)?; - inner.derived_super_guard = Some(guard); + inner.derived_super_guard = Some(if scanner.arrow_calls_super { + inner.emit(range, Instruction::CreateCell { dst: guard })?; + inner.store_cell(builder, guard, pending, range)?; + SuperGuard::Cell(guard) + } else { + inner.move_to(range, guard, pending)?; + SuperGuard::Register(guard) + }); } } + inner.reject_plain_call(builder, range)?; inner.bind_parameters(builder, parameters, range, ¶meter_slots, rest_index)?; if let Some(block) = body { inner.hoist_vars(builder, &block.statements, range)?; @@ -8982,8 +9822,11 @@ impl<'a> FunctionContext<'a> { Some(lowered) => lowered.key, None => self.property_key(builder, &method.name)?, }; - let value = self.build_function_value(builder, range, None, &method.function)?; - self.install_property(builder, range, target, key, value, method.modifier)?; + self.method_home = Some(target); + let value = self.build_function_value(builder, range, None, &method.function); + self.method_home = None; + let value = value?; + self.install_class_method(range, target, key, value, method.modifier)?; let Some(lowered) = decorators.filter(|d| !d.decorators.is_empty()) else { return Ok(()); }; @@ -9049,6 +9892,7 @@ impl<'a> FunctionContext<'a> { key: lowered.key, init_chain, extra_inits, + defines: true, }); } else { let slot = *next_instance_slot; @@ -9090,6 +9934,7 @@ impl<'a> FunctionContext<'a> { key: lowered.key, init_chain, extra_inits, + defines: false, }); } else { stages.instance_fields.push(decoration); @@ -9126,7 +9971,7 @@ impl<'a> FunctionContext<'a> { object: target, key: public_key, accessor: generated_get, - kind: AccessorKind::Getter, + kind: AccessorKind::ClassGetter, }, )?; self.emit( @@ -9135,7 +9980,7 @@ impl<'a> FunctionContext<'a> { object: target, key: public_key, accessor: generated_set, - kind: AccessorKind::Setter, + kind: AccessorKind::ClassSetter, }, )?; let Some(lowered) = decorators.filter(|d| !d.decorators.is_empty()) else { @@ -9212,7 +10057,7 @@ impl<'a> FunctionContext<'a> { static_inits.push(StaticInit::Block(block.clone())); Ok(()) } - ClassMember::IndexSignature(_) => Ok(()), + ClassMember::IndexSignature(_) | ClassMember::Semicolon => Ok(()), ClassMember::Missing(missing) => Err(self.missing(range, missing.expected())), } } @@ -9223,6 +10068,18 @@ enum LoweredBody<'a> { Arrow(&'a FunctionBody), Block(&'a BlockNode), } +impl LoweredBody<'_> { + /// Whether this body's own directive prologue asks for strict code. + fn has_use_strict(self, file: &SourceFile) -> bool { + let block = match self { + Self::Function(FunctionBody::Block(block)) + | Self::Arrow(FunctionBody::Block(block)) + | Self::Block(block) => block, + _ => return false, + }; + crate::early_errors::has_use_strict(file, &block.data().statements) + } +} #[derive(Clone)] struct ScannedBinding { identity: BindingIdentity, @@ -9240,6 +10097,17 @@ struct FreeVarScanner<'a> { uses_this: bool, uses_arguments: bool, uses_new_target: bool, + uses_super: bool, + /// Walks source the way it is written, including JSX and decorators, which + /// lowering reaches some other way. + scans_jsx: bool, + /// Every name read as a value, shadowed or not. Keeping an import a shadowed + /// read did not need costs nothing; dropping one that was read would throw. + reads: BTreeSet, + /// Const enum uses the source scan skips, because emit inlines them. + inlined_enums: Option<&'a crate::enum_plan::EnumFacts>, + calls_super: bool, + arrow_calls_super: bool, fn_boundary: u32, function_depth: u32, } @@ -9313,6 +10181,12 @@ impl<'a> FreeVarScanner<'a> { uses_this: false, uses_arguments: false, uses_new_target: false, + uses_super: false, + scans_jsx: false, + reads: BTreeSet::new(), + inlined_enums: None, + calls_super: false, + arrow_calls_super: false, fn_boundary: 0, function_depth: 0, } @@ -9397,6 +10271,9 @@ impl<'a> FreeVarScanner<'a> { } return; } + if self.scans_jsx { + self.reads.insert(name.to_owned()); + } if let Some(binding) = self.resolve_binding(name).cloned() { if binding.owner_depth == 0 && !self.initialized.contains(&binding.identity) { self.runtime_cells.insert(binding.identity.clone()); @@ -9656,6 +10533,7 @@ impl<'a> FreeVarScanner<'a> { self.push(); if let Some(binding) = &handler.data().binding { self.bind_pattern(binding, DeclarationScope::Lexical); + self.scan_pattern_effects(binding); self.initialize_pattern(binding, DeclarationScope::Lexical); } self.predeclare_immediate(&handler.data().body.data().statements, false); @@ -9716,6 +10594,19 @@ impl<'a> FreeVarScanner<'a> { } } } + Statement::ImportEquals(import) if self.scans_jsx => { + if let crate::syntax::ExternalModuleReference::Qualified(name) = &import.reference { + let mut root = name; + while let crate::syntax::EntityName::Qualified { left, .. } = root { + root = left; + } + if let crate::syntax::EntityName::Identifier(identifier) = root + && let Some(text) = identifier_name(self.file, identifier) + { + self.use_name(&text); + } + } + } Statement::Namespace(declaration) => { let name = declaration .name @@ -9809,12 +10700,85 @@ impl<'a> FreeVarScanner<'a> { self.pop(); self.function_depth -= 1; } + /// A lowercase tag names an intrinsic, not a value in scope. + fn scan_jsx_name(&mut self, name: &JsxElementName) { + match name { + JsxElementName::Identifier(identifier) => { + if let Some(text) = identifier_name(self.file, identifier) + && !text.starts_with(|c: char| c.is_ascii_lowercase()) + { + self.use_name(&text); + } + } + JsxElementName::Member(member) => { + let mut root = &*member.object; + while let JsxElementName::Member(inner) = root { + root = &inner.object; + } + if let JsxElementName::Identifier(identifier) = root + && let Some(text) = identifier_name(self.file, identifier) + { + self.use_name(&text); + } + } + JsxElementName::Namespace(_) => {} + } + } + fn scan_jsx_attributes(&mut self, attributes: &[JsxAttributeItem]) { + for item in attributes { + match item { + JsxAttributeItem::Attribute(attribute) => { + if let Some(JsxAttributeInitializer::Expression(container)) = + &attribute.data().initializer + && let Some(expression) = &container.data().expression + { + self.scan_expression(expression); + } + } + JsxAttributeItem::Spread(spread) => { + self.scan_expression(&spread.data().expression); + } + } + } + } + fn scan_jsx_children(&mut self, children: &[JsxChild]) { + for child in children { + match child { + JsxChild::Text(_) => {} + JsxChild::ExpressionContainer(container) => { + if let Some(expression) = &container.data().expression { + self.scan_expression(expression); + } + } + JsxChild::Spread(spread) => self.scan_expression(&spread.data().expression), + JsxChild::Element(element) => self.scan_expression(element), + } + } + } fn scan_class_heritage(&mut self, class: &ClassDeclaration) { if let Some(heritage) = &class.extends { self.scan_expression(&heritage.expression); } } + /// Decorators run where the class is defined. Lowering reaches them there, + /// so only the source scan walks them here. + fn scan_decorators(&mut self, decorators: &[DecoratorNode]) { + if self.scans_jsx { + for decorator in decorators { + self.scan_expression(&decorator.data().expression); + } + } + } fn scan_class(&mut self, class: &ClassDeclaration) { + self.scan_decorators(&class.decorators); + for member in &class.members { + match member.data() { + ClassMember::Method(method) => self.scan_decorators(&method.function.decorators), + ClassMember::Property(property) => self.scan_decorators(&property.decorators), + ClassMember::AutoAccessor(accessor) => self.scan_decorators(&accessor.decorators), + _ => {} + } + } self.push(); let mut seen_private = HashSet::new(); for member in &class.members { @@ -9853,6 +10817,11 @@ impl<'a> FreeVarScanner<'a> { self.scan_statement(statement); } } + // Field initializers run in the constructor but see the class's scope. + self.function_roots.pop(); + self.pop(); + self.push(); + self.function_roots.push(self.bound.len() - 1); for member in &class.members { match member.data() { ClassMember::Property(property) @@ -9925,6 +10894,12 @@ impl<'a> FreeVarScanner<'a> { self.pop(); } fn scan_expression(&mut self, expression: &Expr) { + if self + .inlined_enums + .is_some_and(|facts| facts.const_use(expression.id()).is_some()) + { + return; + } if let Some(replacement) = self .jsx_plan .and_then(|plan| plan.expression_desugars.get(&expression.id())) @@ -9934,6 +10909,18 @@ impl<'a> FreeVarScanner<'a> { return; } match expression.data() { + Expression::JsxElement(element) if self.scans_jsx => { + self.scan_jsx_name(&element.opening.data().name); + self.scan_jsx_attributes(&element.opening.data().attributes); + self.scan_jsx_children(&element.children); + } + Expression::JsxSelfClosingElement(element) if self.scans_jsx => { + self.scan_jsx_name(&element.name); + self.scan_jsx_attributes(&element.attributes); + } + Expression::JsxFragment(fragment) if self.scans_jsx => { + self.scan_jsx_children(&fragment.children); + } Expression::JsxElement(_) | Expression::JsxFragment(_) | Expression::JsxSelfClosingElement(_) => {} @@ -9952,7 +10939,12 @@ impl<'a> FreeVarScanner<'a> { self.uses_this = true; } } - Expression::Super => {} + Expression::Super => { + if self.fn_boundary == 0 { + self.uses_super = true; + self.uses_this = true; + } + } Expression::Meta(MetaProperty::NewTarget) => { if self.fn_boundary == 0 { self.uses_new_target = true; @@ -10007,6 +10999,10 @@ impl<'a> FreeVarScanner<'a> { } Expression::Arrow(arrow) => self.scan_arrow(arrow), Expression::Call(call) => { + if self.fn_boundary == 0 && matches!(call.callee.data(), Expression::Super) { + self.calls_super = true; + self.arrow_calls_super |= self.function_depth > 0; + } self.scan_expression(&call.callee); for argument in &call.arguments { match argument { @@ -10114,7 +11110,9 @@ impl<'a> FreeVarScanner<'a> { } AssignmentTarget::Array(array) => { for element in &array.elements { - if let AssignmentArrayElement::Target(inner) = element { + if let AssignmentArrayElement::Target(inner) + | AssignmentArrayElement::Rest(inner) = element + { self.scan_assignment_target(inner); } } @@ -10556,6 +11554,14 @@ fn split_regex(lexeme: &str) -> Option<(String, String)> { let flags = &lexeme[last_slash + 1..]; Some((pattern.to_owned(), flags.to_owned())) } +/// Whether a property initializes the literal's prototype instead of naming a +/// property of it. Only `__proto__: value` does, not a computed or shorthand key. +fn sets_prototype(property: &crate::syntax::ObjectProperty, name: Option<&str>) -> bool { + name == Some("__proto__") + && !property.shorthand + && !matches!(property.name, PropertyName::Computed(_)) +} + fn number_constant(value: f64) -> Constant { if value.fract() == 0.0 && value.is_finite() @@ -10705,6 +11711,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, ) { Ok(module) => { @@ -10738,6 +11745,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, ) } @@ -10797,6 +11805,16 @@ mod tests { ); } #[test] + fn a_this_parameter_takes_no_argument() { + let module = lower_js("function f(this: any, a: number) { return a; }"); + let function = module + .functions() + .iter() + .find(|function| function.parameter_count() > 0) + .expect("the function has a parameter"); + assert_eq!(function.parameter_count(), 1); + } + #[test] fn checker_diagnosed_super_misuse_lowers_to_inert_values() { // The checker rejects every one of these (BAMTS-C024..C027); lowering // must still produce a verified module for diagnosed sources instead @@ -10808,6 +11826,7 @@ mod tests { "class A { constructor() { super(); } }", "class A extends Object { constructor() { super(); const g = () => super(); } }", "class A extends Object { m() { super(); } }", + "class A extends Object { f = () => super(); constructor() { super(); } }", ] { lower_js(src); } @@ -11506,13 +12525,21 @@ mod tests { .any(|function| function_has_name(&module, function, "C")), "named class constructor metadata uses the source class name" ); - let anonymous = lower_js("const C = class {};"); + let anonymous = lower_js("(0, class {});"); assert!( anonymous .functions() .iter() .all(|function| function.name().is_none()), - "anonymous class constructors keep empty function metadata" + "a class with nothing to name it keeps empty function metadata" + ); + let inferred = lower_js("const C = class {};"); + assert!( + inferred + .functions() + .iter() + .any(|function| function_has_name(&inferred, function, "C")), + "a class initializing a binding takes the binding's name" ); let expression = lower_js("const C = class Named {};"); assert!( @@ -12615,6 +13642,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, ) .expect_err("a trailing BigInt separator is invalid"); @@ -12681,6 +13709,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), @@ -12713,6 +13742,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), @@ -12740,6 +13770,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), @@ -12772,6 +13803,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), @@ -12825,6 +13857,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), @@ -12859,6 +13892,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), @@ -12881,6 +13915,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), @@ -12940,6 +13975,7 @@ mod tests { files[1].product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, facts.enum_facts(), facts.namespace_facts(), @@ -13037,6 +14073,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), @@ -13122,6 +14159,7 @@ mod tests { files[1].product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, facts.enum_facts(), facts.namespace_facts(), @@ -14163,6 +15201,42 @@ mod tests { Instruction::CreateRegExp { .. } ))); } + #[test] + fn array_assignment_patterns_take_defaults_and_rest() { + let module = lower_js("var a, b, c, d = {}; [a = 1, [b = 2], ...d.rest] = c;"); + assert!(any_instruction(&module, |i| matches!( + i, + Instruction::GetIterator { .. } + ))); + } + + #[test] + fn class_methods_are_defined_as_hidden_and_object_methods_are_not() { + let class = lower_js("class C { m() {} get v() { return 1; } set v(x) {} }"); + assert!(any_instruction(&class, |i| matches!( + i, + Instruction::DefineDataProperty { .. } + ))); + for expected in [ + bamts_bytecode::AccessorKind::ClassGetter, + bamts_bytecode::AccessorKind::ClassSetter, + ] { + assert!(any_instruction(&class, |i| matches!( + i, + Instruction::DefineAccessor { kind, .. } if *kind == expected + ))); + } + let object = lower_js("var o = { m() {}, get v() { return 1; } };"); + assert!(!any_instruction(&object, |i| matches!( + i, + Instruction::DefineDataProperty { .. } + ))); + assert!(any_instruction(&object, |i| matches!( + i, + Instruction::DefineAccessor { kind, .. } if *kind == bamts_bytecode::AccessorKind::Getter + ))); + } + #[test] fn class_with_extends_builds_prototype_chain() { let module = lower_js("class B {} class C extends B { m() { return 1; } }"); @@ -15843,6 +16917,7 @@ mod tests { parsed.product(), LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, checked.product().enum_facts(), checked.product().namespace_facts(), diff --git a/crates/bamts-compiler/src/namespace_plan.rs b/crates/bamts-compiler/src/namespace_plan.rs index 75a77473..4a429e15 100644 --- a/crates/bamts-compiler/src/namespace_plan.rs +++ b/crates/bamts-compiler/src/namespace_plan.rs @@ -532,8 +532,11 @@ fn statement_is_value_bearing( .get(&statement.id()) .is_some_and(|binding| namespace_is_value_bearing(binding, by_declaration)), Statement::Declare(_) => false, + // An alias holds a value only once it is published on the container. + Statement::ImportEquals(_) => false, Statement::Export(ExportDeclaration::Named(ExportNamedDeclaration::Declaration(inner))) => { - statement_is_value_bearing(inner, by_declaration) + matches!(inner.data(), Statement::ImportEquals(_)) + || statement_is_value_bearing(inner, by_declaration) } Statement::Empty => false, _ => true, diff --git a/crates/bamts-compiler/src/parser.rs b/crates/bamts-compiler/src/parser.rs index 0bef9661..58b2851c 100644 --- a/crates/bamts-compiler/src/parser.rs +++ b/crates/bamts-compiler/src/parser.rs @@ -214,11 +214,10 @@ pub fn parse_with_cancel( let mut parser = Parser::new_with_cancel(source_id, script_kind, source, tokens, eof, cancel); let statements = parser.parse_statements_until(&[]); - // The default scanner pass emits only `Slash`/`SlashEq`, never a - // `RegularExpressionLiteral`; every such token in the final stream is a - // committed parser regex rescan. The lexical diagnostics the default pass - // recorded inside those spans reflect the wrong (division) interpretation, - // so they are dropped and superseded by the rescan's own diagnostics. + // Every `RegularExpressionLiteral` in the final stream is either one the + // default pass was certain of or a committed parser regex rescan. Any lexical + // diagnostics recorded inside those spans reflect the wrong (division) + // interpretation, so they are dropped and superseded by the rescan's own. let regex_spans: Vec = parser .tokens .iter() @@ -1995,7 +1994,13 @@ impl Parser { break; } let before = self.cursor; - let member = self.parse_class_member(); + let member = if self.at(TokenKind::Semicolon) { + let start = self.cur_start(); + self.bump(); + self.node(start, ClassMember::Semicolon) + } else { + self.parse_class_member() + }; members.push(member); if self.cursor == before { let skipped = self.bump(); @@ -2151,8 +2156,14 @@ impl Parser { }; // Constructor. + // In JavaScript a static, async, or generator `constructor` is an ordinary + // method, which the early-error pass judges. TypeScript keeps one shape + // here because its checker reports these itself. + let plain_in_javascript = + self.is_typescript() || !(modifiers.is_static || is_async || is_generator); if !is_accessor && property_modifier == PropertyModifier::None + && plain_in_javascript && self.is_constructor_name(&name) && self.at(TokenKind::LParen) { @@ -3239,7 +3250,10 @@ impl Parser { fn parse_assignment_inner(&mut self, no_in: bool) -> Expr { let start = self.cur_start(); - if self.at(TokenKind::KwYield) { + // Outside a generator, sloppy JavaScript reads `yield` as an identifier. + if self.at(TokenKind::KwYield) + && (self.is_typescript() || self.keyword_context.yield_reserved) + { return self.parse_yield_expression(start, no_in); } @@ -3495,7 +3509,7 @@ impl Parser { ); } - if self.at(TokenKind::KwAwait) && self.can_start_expression_after(1) { + if self.at(TokenKind::KwAwait) && self.await_is_an_operator() { let keyword_range = self.cur().range(); self.bump(); let argument = self.parse_unary_expression(); @@ -3534,6 +3548,72 @@ impl Parser { } /// Whether the `n`-th token can begin the operand of a prefix operator. + /// Whether the `await` here starts an await expression. TypeScript always + /// reads one so it can report misuse. In JavaScript, `await` outside an async + /// function is an ordinary name, and `await instanceof f` uses that name. + fn await_is_an_operator(&self) -> bool { + if !self.can_start_expression_after(1) { + return false; + } + if self.is_typescript() || self.keyword_context.await_reserved { + return true; + } + if self.keyword_context.in_function { + return false; + } + let next = self.nth_kind(1); + if self.has_module_syntax() { + return !matches!(next, TokenKind::KwInstanceof | TokenKind::KwIn); + } + // Without module syntax nothing settles the goal yet. Like Node, read + // the name unless what follows could not continue an expression. + is_identifier_like(next) + || matches!( + next, + TokenKind::NumericLiteral + | TokenKind::BigIntLiteral + | TokenKind::StringLiteral + | TokenKind::LBrace + | TokenKind::Bang + | TokenKind::Tilde + | TokenKind::KwClass + | TokenKind::KwDelete + | TokenKind::KwFalse + | TokenKind::KwFunction + | TokenKind::KwImport + | TokenKind::KwNew + | TokenKind::KwNull + | TokenKind::KwSuper + | TokenKind::KwThis + | TokenKind::KwTrue + | TokenKind::KwTypeof + | TokenKind::KwVoid + ) + } + + /// Whether any token in the file looks like module syntax. `import(` and + /// `import.` also work in a script. A keyword used as a property name + /// counts too, which only keeps `await` an operator. + fn has_module_syntax(&self) -> bool { + let mut index = self.next_significant(0); + while index < self.tokens.len() { + let next = self.next_significant(index + 1); + match self.tokens[index].kind() { + TokenKind::KwExport => return true, + TokenKind::KwImport + if !self.tokens.get(next).is_some_and(|token| { + matches!(token.kind(), TokenKind::LParen | TokenKind::Dot) + }) => + { + return true; + } + _ => {} + } + index = next; + } + false + } + fn can_start_expression_after(&self, n: usize) -> bool { !matches!( self.nth_kind(n), @@ -3928,7 +4008,7 @@ impl Parser { fn parse_primary_expression(&mut self) -> Expr { let start = self.cur_start(); match self.kind() { - TokenKind::Slash | TokenKind::SlashEq => { + TokenKind::Slash | TokenKind::SlashEq | TokenKind::RegularExpressionLiteral => { self.rescan_regex_here(); let token = self.bump(); let range = token.range(); @@ -5472,12 +5552,9 @@ impl Parser { let target = self.expression_to_target(*expr); elements.push(AssignmentArrayElement::Target(target)); } - ArrayElement::Spread(_) => { - // A rest element has no array-target slot. Record a missing - // element so the checker can report it; the parser stays silent. - elements.push(AssignmentArrayElement::Missing(MissingNode::new( - NodeKind::MissingAssignmentTarget, - ))); + ArrayElement::Spread(spread) => { + let target = self.expression_to_target(*spread.argument); + elements.push(AssignmentArrayElement::Rest(target)); } ArrayElement::Missing(node) => { elements.push(AssignmentArrayElement::Missing(node)); @@ -5502,9 +5579,19 @@ impl Parser { name: property.name, target, initializer, + rest: false, }); } - ObjectMember::Spread(_) | ObjectMember::Method(_) | ObjectMember::Missing(_) => { + ObjectMember::Spread(spread) => { + let target = self.expression_to_target(*spread.argument); + properties.push(AssignmentObjectProperty { + name: PropertyName::Missing(MissingNode::new(NodeKind::Identifier)), + target, + initializer: None, + rest: true, + }); + } + ObjectMember::Method(_) | ObjectMember::Missing(_) => { properties.push(AssignmentObjectProperty { name: PropertyName::Missing(MissingNode::new(NodeKind::Identifier)), target: self.node_at( @@ -5514,6 +5601,7 @@ impl Parser { )), ), initializer: None, + rest: false, }); } } @@ -6956,6 +7044,34 @@ mod tests { } } + #[test] + fn a_bare_semicolon_is_a_class_member() { + for source in [ + "class C { ; }", + "class C { ;;; }", + "class C { a() {}; b = 1;; static c() {} ; }", + "class C { #a() {}; get #b() { return 1; }; }", + ] { + assert_clean(source); + } + let recovered = assert_clean("class C { a() {}; }"); + let Statement::Class(class) = recovered.product().statements()[0].data() else { + panic!("expected a class"); + }; + let members: Vec<_> = class.members.iter().map(|member| member.data()).collect(); + assert!(matches!( + members[..], + [ClassMember::Method(_), ClassMember::Semicolon] + )); + } + + #[test] + fn jsx_text_that_looks_like_a_regex_stays_text() { + let source = "const a =

, /x

;\nconst b =

= /'

;\n"; + let parsed = parse_text(source, ScriptKind::TypeScriptReact); + assert!(errors(&parsed).is_empty(), "{:?}", errors(&parsed)); + } + #[test] fn regex_rescan_produces_literal() { let recovered = assert_clean("const r = /a[/]b/gi;"); diff --git a/crates/bamts-compiler/src/pipeline.rs b/crates/bamts-compiler/src/pipeline.rs index c7cee279..925e264a 100644 --- a/crates/bamts-compiler/src/pipeline.rs +++ b/crates/bamts-compiler/src/pipeline.rs @@ -10,19 +10,29 @@ //! composes the existing scanner, parser, checker, and emitter surfaces. use bamts_cancel::{CancellationToken, Cancelled}; +use std::collections::{BTreeSet, HashMap, HashSet}; use std::sync::Arc; +use crate::checker::diagnostic_messages::{Args, by_u32}; use crate::checker::{self, ProgramCheckInput, ResolvedModuleEdge, SemanticModel}; -use crate::diagnostic::{Diagnostic, Recovered}; +use crate::diagnostic::{Diagnostic, DiagnosticCode, Recovered}; use crate::emitter::{self, EmitFileNames, EmitOptions, EmitOutput, ModuleKind}; use crate::lint::{LintProfile, LintTable}; use crate::parser; -use crate::program::{JsxRoutingDecision, ModuleTarget, ProgramOutputKind, ResolvedProgram}; +use crate::program::{ + JsxRoutingDecision, ModuleEdge, ModuleTarget, ProgramOutputKind, ResolvedModule, + ResolvedProgram, +}; use crate::scanner; use crate::source::{ScriptKind, SourceId, SourceText}; -use crate::syntax::{ExportDeclaration, ExportNamedDeclaration, SourceFile}; +use crate::syntax::{ + ExportDeclaration, ExportNamedDeclaration, NamespaceName, SourceFile, Statement, Stmt, +}; use crate::telemetry::{Phase, Telemetry}; +/// An import nothing satisfies, in `tsc`'s own numbering. +const CANNOT_FIND_MODULE: DiagnosticCode = DiagnosticCode::new("TS2307"); + struct EdgeNode { id: crate::syntax::NodeId, range: crate::source::TextRange, @@ -199,6 +209,32 @@ impl FrontendMode { } } +fn dependencies_of(program: &ResolvedProgram, source_id: SourceId) -> &[ModuleEdge] { + program + .modules() + .iter() + .find(|module| module.source_id() == source_id) + .map_or(&[], |module| module.dependencies()) +} + +/// The local module an edge's names come from; types may resolve apart. +fn edge_target(edge: &ModuleEdge) -> Option { + edge.type_target() + .unwrap_or_else(|| edge.target()) + .local_source_id() +} + +fn local_dependency( + program: &ResolvedProgram, + source_id: SourceId, + specifier: &str, +) -> Option { + dependencies_of(program, source_id) + .iter() + .find(|edge| edge.specifier() == specifier) + .and_then(edge_target) +} + fn program_emit_options(program: &ResolvedProgram, mode: FrontendMode) -> Option { let mut options = mode.emit_options()?; @@ -219,6 +255,8 @@ fn program_emit_options(program: &ResolvedProgram, mode: FrontendMode) -> Option program.import_helpers(), ); options.no_emit_helpers = check.no_emit_helpers(); + options.remove_comments = check.remove_comments(); + options.verbatim_module_syntax = program.verbatim_module_syntax(); // JSX routing is JavaScript-specific and already shared with the CLI path, // which applies the same overrides from `program.jsx()` after its base @@ -346,6 +384,100 @@ impl ProgramFrontendOutput { &self.modules } + /// Each module's export names with no runtime form. A name forwarded from + /// another module, by a re-export or `export *`, takes that module's answer. + #[must_use] + pub fn erased_exports(&self, program: &ResolvedProgram) -> HashMap> { + let mut erased = HashMap::new(); + for module in &self.modules { + let source_id = module.source_file().source_id(); + self.resolve_erased_exports(program, source_id, &mut erased, &mut HashSet::new()); + } + erased + } + + fn resolve_erased_exports( + &self, + program: &ResolvedProgram, + source_id: SourceId, + erased: &mut HashMap>, + visiting: &mut HashSet, + ) -> BTreeSet { + if let Some(names) = erased.get(&source_id) { + return names.clone(); + } + let Some(module) = self.module(source_id) else { + return BTreeSet::new(); + }; + // A cycle decides nothing: a name stays until something proves it erased. + if !visiting.insert(source_id) { + return BTreeSet::new(); + } + let exports = + emitter::transforms::module_exports(module.source_file(), module.semantic_model()); + let mut names = exports.types; + let forwarded: BTreeSet = exports + .forwards + .iter() + .map(|(exported, ..)| exported.clone()) + .collect(); + for (exported, specifier, imported) in exports.forwards { + if let Some(target) = local_dependency(program, source_id, &specifier) + && self + .resolve_erased_exports(program, target, erased, visiting) + .contains(&imported) + { + names.insert(exported); + } else { + // An explicit export outranks a local type of the same name. + names.remove(&exported); + } + } + let mut star_values = BTreeSet::new(); + for specifier in exports.stars { + if let Some(target) = local_dependency(program, source_id, &specifier) { + let starred = self.resolve_erased_exports(program, target, erased, visiting); + if let Some(module) = self.module(target) { + let exports = emitter::transforms::module_exports( + module.source_file(), + module.semantic_model(), + ); + star_values.extend(exports.values); + } + // An explicit export of the same name shadows what a star brings. + names.extend( + starred + .into_iter() + .filter(|name| name != "default" && !forwarded.contains(name)), + ); + } + } + // A name stays if any star brings it as a value. + names.retain(|name| !exports.values.contains(name) && !star_values.contains(name)); + erased.insert(source_id, names.clone()); + names + } + + /// `(specifier, name)` for what the module's imports can name that has no + /// runtime form. Emit cannot see other files, and an ES module naming one + /// of these fails at load. + #[must_use] + pub fn erased_imports( + &self, + program: &ResolvedProgram, + erased: &HashMap>, + source_id: SourceId, + ) -> BTreeSet<(String, String)> { + let mut pairs = BTreeSet::new(); + for edge in dependencies_of(program, source_id) { + let names = edge_target(edge).and_then(|target| erased.get(&target)); + for name in names.into_iter().flatten() { + pairs.insert((edge.specifier().to_owned(), name.clone())); + } + } + pairs + } + #[must_use] pub fn module(&self, source_id: SourceId) -> Option<&FrontendOutput> { self.modules @@ -385,6 +517,72 @@ pub fn compile_program_frontend_with_lints( /// Cancellation is observed between frontend stages and at every source and /// dependency iteration boundary. A completed result is otherwise identical to /// compile_program_frontend_with_lints. +/// The names of every `declare module "name"` in the program, which satisfy +/// an import that no file does. +fn ambient_module_names(parsed: &[Recovered]) -> Vec { + fn collect(file: &SourceFile, statement: &Stmt, names: &mut Vec) { + match statement.data() { + Statement::Declare(inner) => collect(file, inner, names), + Statement::Namespace(namespace) => { + if let NamespaceName::StringLiteral(literal) = &namespace.name + && let Some(text) = file.token_text(literal.data().token()) + { + names.push(text.trim_matches(['\'', '"']).to_owned()); + } + } + _ => {} + } + } + let mut names = Vec::new(); + for file in parsed { + for statement in file.product().statements() { + collect(file.product(), statement, &mut names); + } + } + names +} + +/// Whether an ambient module `pattern` covers `specifier`. One `*` matches +/// any text, as in `declare module "*.css"`. +fn ambient_module_matches(pattern: &str, specifier: &str) -> bool { + match pattern.split_once('*') { + Some((prefix, suffix)) => { + specifier.len() >= prefix.len() + suffix.len() + && specifier.starts_with(prefix) + && specifier.ends_with(suffix) + } + None => pattern == specifier, + } +} + +/// TS2307 for each import of `module` that nothing satisfies. +fn unresolved_import_diagnostics( + module: &ResolvedModule, + ambient_modules: &[String], +) -> Vec { + module + .dependencies() + .iter() + .filter(|edge| { + edge.is_unresolved() + && !ambient_modules + .iter() + .any(|pattern| ambient_module_matches(pattern, edge.specifier())) + }) + .map(|edge| { + let message = by_u32(2307) + .and_then(|entry| entry.format(Args::new([edge.specifier()])).ok()) + .unwrap_or_else(|| "Cannot find module.".to_owned()); + Diagnostic::error( + CANNOT_FIND_MODULE, + module.source_id(), + edge.range(), + message, + ) + }) + .collect() +} + pub fn compile_program_frontend_with_cancel( program: &ResolvedProgram, mode: FrontendMode, @@ -417,6 +615,7 @@ pub fn compile_program_frontend_with_cancel( let edges = resolved_checker_edges_with_cancel(program, &parsed, cancel)?; cancel.check()?; + let ambient_modules = ambient_module_names(&parsed); let checked = Telemetry::measure(Phase::Check, || { checker::check_program_with_options_and_cancel( ProgramCheckInput { @@ -483,6 +682,7 @@ pub fn compile_program_frontend_with_cancel( .filter(|diagnostic| diagnostic.source_id() == source_id) .cloned(), ); + diagnostics.extend(unresolved_import_diagnostics(module, &ambient_modules)); if let Some(output) = &emit { diagnostics.extend(output.diagnostics.iter().cloned()); } @@ -686,6 +886,82 @@ mod tests { (root_path, program) } + /// Writes `files` to a fresh project and returns its root and config. + fn project(files: &[(&str, &str)]) -> (PathBuf, ProjectRoot, ProjectConfig) { + let root_path = std::env::temp_dir().join(format!( + "bamts-pipeline-unresolved-{}-{}", + std::process::id(), + files.len() + )); + let _ = std::fs::remove_dir_all(&root_path); + for (name, text) in files { + let path = root_path.join(name); + std::fs::create_dir_all(path.parent().expect("a file has a parent")) + .expect("create fixture directory"); + std::fs::write(path, text).expect("write fixture file"); + } + let root = ProjectRoot::new(std::fs::canonicalize(&root_path).expect("canonical fixture")) + .expect("valid project root"); + let config = ProjectConfig::parse( + &root, + root_path.join("tsconfig.json"), + r#"{"compilerOptions":{"moduleResolution":"bundler"}}"#, + ) + .expect("valid project config"); + (root_path, root, config) + } + + #[test] + fn a_check_run_reports_every_unresolved_import() { + let (root_path, root, config) = project(&[ + ( + "main.ts", + "import a from 'missing';\n\ + import b from './gone';\n\ + import c from 'declared';\n\ + import d from 'pattern.css';\n\ + import e from 'typed';\n\ + declare module 'declared' {}\n\ + declare module '*.css' {}\n", + ), + ( + "node_modules/typed/package.json", + r#"{"name":"typed","types":"./lib/index.d.ts"}"#, + ), + ("node_modules/typed/lib/index.d.ts", "export default 1;\n"), + ]); + let loader = ProgramLoader::new(&root, config.options()).expect("construct loader"); + assert!( + loader.clone().load("main.ts").is_err(), + "a run that executes must still refuse an unresolved import" + ); + let program = loader + .tolerating_unresolved_imports() + .load("main.ts") + .expect("a check run keeps going"); + assert_eq!( + program.modules().len(), + 2, + "`typed` resolves as `tsc` would" + ); + let output = compile_program_frontend(&program, FrontendMode::Check); + let entry = output + .modules() + .iter() + .find(|module| module.source_file().source_id() == output.entrypoint_id()) + .expect("entrypoint module present"); + let reported: Vec<_> = entry + .diagnostics() + .iter() + .filter(|diagnostic| diagnostic.code().as_str() == "TS2307") + .map(|diagnostic| diagnostic.message().to_owned()) + .collect(); + assert_eq!(reported.len(), 2, "{reported:?}"); + assert!(reported[0].contains("'missing'"), "{reported:?}"); + assert!(reported[1].contains("'./gone'"), "{reported:?}"); + let _ = std::fs::remove_dir_all(root_path); + } + fn entrypoint_javascript(output: &ProgramFrontendOutput) -> String { let module = output .modules() diff --git a/crates/bamts-compiler/src/program.rs b/crates/bamts-compiler/src/program.rs index cb04202b..b01ba1f0 100644 --- a/crates/bamts-compiler/src/program.rs +++ b/crates/bamts-compiler/src/program.rs @@ -29,7 +29,10 @@ use crate::{ project::{ CompilerOptions, ModuleResolutionError, PackageError, PackageJson, PackageMode, PackageTarget, ProjectRoot, ResolutionConditions, ResolutionFlavor, plan_relative_module, - resolution::ResolutionMode, + resolution::{ + ModuleResolutionKind, ResolutionCache, ResolutionHost, ResolutionMode, + resolve_module_name, resolve_type_reference, + }, }, scanner, service::filesystem::{FileSystem, FileSystemError, OsFileSystem}, @@ -99,12 +102,19 @@ pub struct ModuleEdge { target: ModuleTarget, type_target: Option, range: TextRange, + unresolved: bool, } impl ModuleEdge { #[must_use] pub const fn kind(&self) -> ModuleEdgeKind { self.kind } + + /// Whether a tolerant load found nothing for this specifier. + #[must_use] + pub const fn is_unresolved(&self) -> bool { + self.unresolved + } /// Returns the resolution flavor used for `target`. /// /// A separate `type_target` overlay, when present, was resolved as @@ -221,8 +231,10 @@ pub struct ResolvedProgram { strict_property_initialization: bool, always_strict: bool, no_emit_helpers: bool, + remove_comments: bool, use_define_for_class_fields: Option, import_helpers: Option, + verbatim_module_syntax: bool, target: crate::emitter::ScriptTarget, libs: crate::checker::intrinsic_environment::LibSet, check_js: bool, @@ -248,6 +260,11 @@ impl ResolvedProgram { self.import_helpers } + #[must_use] + pub const fn verbatim_module_syntax(&self) -> bool { + self.verbatim_module_syntax + } + #[must_use] pub fn roots(&self) -> &[SourceId] { &self.roots @@ -343,6 +360,7 @@ impl ResolvedProgram { .with_strict_property_initialization(self.strict_property_initialization) .with_always_strict(self.always_strict) .with_no_emit_helpers(self.no_emit_helpers) + .with_remove_comments(self.remove_comments) .with_script_target(self.target) .with_libs(self.libs) .with_check_js(self.check_js) @@ -583,6 +601,7 @@ pub struct ProgramLoader { root: ProjectRoot, options: CompilerOptions, fs: Arc, + tolerate_unresolved: bool, } impl fmt::Debug for ProgramLoader { @@ -621,9 +640,20 @@ impl ProgramLoader { root, options: options.clone(), fs, + tolerate_unresolved: false, }) } + /// Keeps loading past an import nothing can satisfy, for a run that only + /// checks, as `tsc` does. The frontend reports each one, so a project sees + /// all of them at once. A run that will execute must not use this: the + /// program has to fail before it starts. + #[must_use] + pub const fn tolerating_unresolved_imports(mut self) -> Self { + self.tolerate_unresolved = true; + self + } + /// Resolves and loads an entrypoint. The entrypoint may be root-relative or absolute. pub fn load(&self, entrypoint: impl AsRef) -> Result { self.load_with_cancel(entrypoint, &CancellationToken::new()) @@ -721,8 +751,10 @@ impl ProgramLoader { strict_property_initialization: self.options.strict_property_initialization(), always_strict: self.options.always_strict(), no_emit_helpers: self.options.no_emit_helpers(), + remove_comments: self.options.remove_comments(), use_define_for_class_fields: self.options.use_define_for_class_fields(), import_helpers: self.options.import_helpers(), + verbatim_module_syntax: self.options.verbatim_module_syntax(), target, libs, check_js: self.options.check_js(), @@ -872,6 +904,9 @@ impl ProgramLoader { flavor, }); } + if self.tolerate_unresolved { + return self.resolve_edge_for_check(importer, edge, flavor, cancel); + } if let Some(target) = self.resolve_local(importer, edge, flavor, cancel)? { let type_overlay = self.type_overlay(importer, edge, &target, cancel)?; return Ok(ResolvedEdge { @@ -895,6 +930,70 @@ impl ProgramLoader { ))) } + /// Resolves an edge for a run that only checks. A package the runtime + /// could not use can still type-check, so `tsc`'s own algorithm gets the + /// last word, and what that cannot find becomes a reported edge. + fn resolve_edge_for_check( + &self, + importer: &Path, + edge: &UnresolvedEdge, + flavor: ResolutionFlavor, + cancel: &CancellationToken, + ) -> Result { + let local = Self::missing_when_invalid(self.resolve_local(importer, edge, flavor, cancel))?; + if let Some(target) = local { + let type_overlay = + Self::missing_when_invalid(self.type_overlay(importer, edge, &target, cancel))?; + return Ok(ResolvedEdge { + target, + type_overlay, + flavor, + }); + } + let target = match self.resolve_as_tsc(importer, edge) { + Some(path) => ResolvedEdgeTarget::Local(path), + None => ResolvedEdgeTarget::Unresolved(Arc::clone(&edge.specifier)), + }; + Ok(ResolvedEdge { + target, + type_overlay: None, + flavor, + }) + } + + /// Reads a package or specifier the runtime rules refused as not found. + fn missing_when_invalid( + resolved: Result, ProgramLoadError>, + ) -> Result, ProgramLoadError> { + match resolved { + Err( + ProgramLoadError::InvalidPackage { .. } | ProgramLoadError::InvalidSpecifier { .. }, + ) => Ok(None), + resolved => resolved, + } + } + + fn resolve_as_tsc(&self, importer: &Path, edge: &UnresolvedEdge) -> Option { + let kind = ModuleResolutionKind::from_options(&self.options) + .unwrap_or(ModuleResolutionKind::Bundler); + let strategy = (kind, edge.mode); + let host = LoaderResolutionHost(self.fs.as_ref()); + let mut cache = ResolutionCache::new(); + let specifier = &*edge.specifier; + let root = &self.root; + let options = &self.options; + resolve_type_reference( + root, options, importer, specifier, strategy, &host, &mut cache, + ) + .or_else(|_| { + resolve_module_name( + root, options, importer, specifier, strategy, &host, &mut cache, + ) + }) + .ok() + .and_then(|resolved| self.fs.normalize(resolved.path()).ok()) + } + fn resolve_local( &self, importer: &Path, @@ -1100,6 +1199,25 @@ impl ProgramLoader { enum ResolvedEdgeTarget { Local(PathBuf), External(Arc), + /// Nothing satisfies the specifier, and the load was told to continue. + Unresolved(Arc), +} + +/// The loader's filesystem, as `tsc`'s resolution algorithm reads it. +struct LoaderResolutionHost<'a>(&'a dyn FileSystem); + +impl ResolutionHost for LoaderResolutionHost<'_> { + fn file_exists(&self, path: &Path) -> bool { + self.0.metadata(path).is_ok() && !self.directory_exists(path) + } + + fn directory_exists(&self, path: &Path) -> bool { + self.0.read_dir(path).is_ok() + } + + fn read_file(&self, path: &Path) -> Option> { + self.0.read(path).ok().map(Arc::from) + } } #[derive(Clone, Debug)] @@ -1221,6 +1339,7 @@ impl LoadState<'_> { target: ModuleTarget::Local(source_id), type_target: None, range: pending.edge.range, + unresolved: false, }); if let Some(path) = pending.type_overlay { self.type_overlays.push(( @@ -1354,6 +1473,19 @@ impl LoadState<'_> { target: ModuleTarget::External(specifier), type_target: None, range: edge.range, + unresolved: false, + }); + } + ResolvedEdgeTarget::Unresolved(specifier) => { + frame.dependencies.push(ModuleEdge { + kind: edge.kind, + flavor: resolved.flavor, + mode: edge.mode, + specifier: edge.specifier, + target: ModuleTarget::External(specifier), + type_target: None, + range: edge.range, + unresolved: true, }); } } @@ -1393,6 +1525,7 @@ impl LoadState<'_> { target: ModuleTarget::Local(source_id), type_target: None, range: pending.edge.range, + unresolved: false, }); if let Some(path) = pending.type_overlay { self.type_overlays @@ -1803,7 +1936,9 @@ impl DynamicEdgeCollector<'_> { } } ClassMember::StaticBlock(block) => self.scan_statements(&block.data().statements), - ClassMember::IndexSignature(_) | ClassMember::Missing(_) => {} + ClassMember::IndexSignature(_) + | ClassMember::Missing(_) + | ClassMember::Semicolon => {} } } } @@ -2009,7 +2144,9 @@ impl DynamicEdgeCollector<'_> { } AssignmentTarget::Array(value) => { for element in &value.elements { - if let AssignmentArrayElement::Target(value) = element { + if let AssignmentArrayElement::Target(value) + | AssignmentArrayElement::Rest(value) = element + { self.scan_target(value); } } @@ -2481,14 +2618,21 @@ pub fn lower_program_with_cancel( ProgramLowerErrorKind::InvalidModuleName, ) })?; - raw_modules.push(collect_raw_module( + let mut raw = collect_raw_module( module, output.source_file(), output.semantic_model().enum_facts(), output.semantic_model().namespace_facts(), name, &module_ids, - )?); + )?; + if runs_as_script(resolved, module, output.source_file(), options) { + // Its `var` and function names belong to the global object, so a + // module cell must not capture them. + raw.bindings + .retain(|binding| !matches!(binding.kind, RawBindingKind::Hoisted)); + } + raw_modules.push(raw); check_lower_cancel(cancel, module.path(), ProgramLowerPhase::Metadata)?; } check_lower_cancel( @@ -2525,7 +2669,10 @@ pub fn lower_program_with_cancel( let strings = linkage_strings(raw); let code = lower::assemble_program_module( file, - options, + LowerOptions { + script_entry: runs_as_script(resolved, resolved_module, file, options), + ..options + }, &strings, raw.jsx_plan.as_ref(), output.semantic_model().enum_facts(), @@ -3379,6 +3526,19 @@ fn materialize_program_module( } } +/// Whether `module` is the Script the host asked for. A file with module syntax +/// is a module either way, and the host decides whether that is an error. +fn runs_as_script( + resolved: &ResolvedProgram, + module: &ResolvedModule, + file: &SourceFile, + options: LowerOptions, +) -> bool { + options.script_entry + && module.source_id() == resolved.entrypoint().source_id() + && !crate::early_errors::is_module(file) +} + fn normalized_module_name(root: &ProjectRoot, path: &Path) -> Option { let relative = path.strip_prefix(root.path()).ok()?; let mut name = String::new(); @@ -3644,6 +3804,7 @@ mod tests { &frontend, LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, ) .unwrap() @@ -3742,6 +3903,7 @@ mod tests { &frontend, LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, ) .expect_err("ill-formed metadata must not reach host String conversion"); @@ -5527,6 +5689,7 @@ mod tests { &frontend, LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, ) .unwrap_or_else(|error| panic!("{entrypoint}: {error}")); @@ -5693,6 +5856,7 @@ mod tests { &frontend, LowerOptions { javascript_compatibility: true, + ..LowerOptions::default() }, ) .unwrap(); diff --git a/crates/bamts-compiler/src/project.rs b/crates/bamts-compiler/src/project.rs index f4552ab0..b694edd3 100644 --- a/crates/bamts-compiler/src/project.rs +++ b/crates/bamts-compiler/src/project.rs @@ -1,4 +1,5 @@ pub mod build_mode; +pub mod canonical; pub mod effective; pub mod output; pub mod references; @@ -6,6 +7,9 @@ pub mod resolution; pub mod resolution_trace; pub mod tsconfig; +#[cfg(test)] +pub(crate) mod test_paths; + use crate::{ lint::{LintConfig, LintLevel, LintSetting}, source::JsxEmit, @@ -162,7 +166,7 @@ fn normalize_absolute(path: &Path) -> PathBuf { Component::Normal(part) => normalized.push(part), } } - normalized + canonical::simplify(&normalized) } /// The reason a JSON-with-comments document was rejected. @@ -958,6 +962,10 @@ pub struct CompilerOptions { always_strict: bool, use_define_for_class_fields: Option, import_helpers: Option, + verbatim_module_syntax: bool, + /// `emitDecoratorMetadata` and `reactNamespace` read imports the source scan + /// cannot see, so emit cannot prove one unused. + keeps_imports: bool, allow_js: bool, check_js: bool, resolve_json_module: bool, @@ -968,6 +976,7 @@ pub struct CompilerOptions { inline_source_map: bool, inline_sources: bool, no_emit_helpers: bool, + remove_comments: bool, map_root: Option>, source_root: Option>, no_emit: bool, @@ -1050,6 +1059,11 @@ impl CompilerOptions { self.always_strict } + #[must_use] + pub const fn remove_comments(&self) -> bool { + self.remove_comments + } + #[must_use] pub const fn no_emit_helpers(&self) -> bool { self.no_emit_helpers @@ -1064,6 +1078,16 @@ impl CompilerOptions { self.import_helpers } + #[must_use] + pub const fn verbatim_module_syntax(&self) -> bool { + self.verbatim_module_syntax + } + + #[must_use] + pub const fn keeps_imports(&self) -> bool { + self.keeps_imports + } + #[must_use] pub const fn allow_js(&self) -> bool { self.allow_js @@ -1255,6 +1279,10 @@ impl ProjectConfig { .unwrap_or(strict), always_strict: optional_bool(compiler, "alwaysStrict")?.unwrap_or(strict), use_define_for_class_fields: optional_bool(compiler, "useDefineForClassFields")?, + verbatim_module_syntax: optional_bool(compiler, "verbatimModuleSyntax")? + .unwrap_or(false), + keeps_imports: optional_bool(compiler, "emitDecoratorMetadata")?.unwrap_or(false) + || compiler.is_some_and(|options| options.get("reactNamespace").is_some()), import_helpers: optional_bool(compiler, "importHelpers")?, allow_js: optional_bool(compiler, "allowJs")?.unwrap_or(false), check_js: optional_bool(compiler, "checkJs")?.unwrap_or(false), @@ -1266,6 +1294,7 @@ impl ProjectConfig { inline_source_map: optional_bool(compiler, "inlineSourceMap")?.unwrap_or(false), inline_sources: optional_bool(compiler, "inlineSources")?.unwrap_or(false), no_emit_helpers: optional_bool(compiler, "noEmitHelpers")?.unwrap_or(false), + remove_comments: optional_bool(compiler, "removeComments")?.unwrap_or(false), map_root: optional_nested_string(compiler, "mapRoot")?, source_root: optional_nested_string(compiler, "sourceRoot")?, no_emit: optional_bool(compiler, "noEmit")?.unwrap_or(false), @@ -2341,16 +2370,16 @@ fn resolve_package_target( #[cfg(test)] mod tests { + use super::test_paths::abs; use super::{ ConfigError, JsonValue, JsoncErrorKind, LintTsConfig, ModuleResolutionError, PackageError, PackageJson, PackageMode, PackageTarget, ProjectConfig, ProjectRoot, ResolutionConditions, ResolutionFlavor, parse_bamts_toml, parse_jsonc, plan_relative_module, }; use crate::source::JsxEmit; - use std::path::{Path, PathBuf}; fn root() -> ProjectRoot { - ProjectRoot::new("/workspace/corpus").expect("absolute test root") + ProjectRoot::new(abs("/workspace/corpus")).expect("absolute test root") } #[test] @@ -2378,7 +2407,7 @@ mod tests { fn lint_tsconfig_ignores_typescript_strictness_options() { let config = LintTsConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{ "compilerOptions": { "target": "ES2022", @@ -2441,10 +2470,10 @@ mod tests { #[test] fn tsconfig_duplicate_compileroptions_keys_yield_last_value() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"strict": true, "strict": false}}"#, ) .expect("duplicate keys are accepted"); @@ -2460,10 +2489,10 @@ mod tests { // contextuallyTypedBooleanLiterals.ts, which carry duplicate @strict // and @declaration pragmas. The harness emits one key per pragma // occurrence, so the JSONC parser must accept duplicate keys. - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"target": "es2015", "strict": true, "declaration": true, "strict": true, "declaration": true}}"#, ) .expect("duplicate keys from duplicate pragmas must be accepted"); @@ -2491,7 +2520,7 @@ mod tests { project .resolve("projects/ohash/src/../src/index.ts") .expect("confined path"), - PathBuf::from("/workspace/corpus/projects/ohash/src/index.ts") + abs("/workspace/corpus/projects/ohash/src/index.ts") ); assert!(project.resolve("../secrets.ts").is_err()); assert!(ProjectRoot::new("relative/root").is_err()); @@ -2501,7 +2530,7 @@ mod tests { fn project_config_parses_corpus_shaped_jsonc_into_immutable_options() { let config = ProjectConfig::parse( &root(), - "/workspace/corpus/projects/hookable/tsconfig.json", + abs("/workspace/corpus/projects/hookable/tsconfig.json"), r##"{ "compilerOptions": { "target": "ESNext", @@ -2536,11 +2565,11 @@ mod tests { assert_eq!(config.options().jsx_import_source(), Some("@scope/runtime")); assert_eq!( config.options().base_url(), - Path::new("/workspace/corpus/projects/hookable") + abs("/workspace/corpus/projects/hookable").as_path() ); assert_eq!( config.options().paths()[0].targets()[0], - PathBuf::from("/workspace/corpus/projects/hookable/src/*") + abs("/workspace/corpus/projects/hookable/src/*") ); assert_eq!(config.include()[1].as_ref(), "test"); } @@ -2554,7 +2583,7 @@ mod tests { // b3dadd75912e490756bd1c0165909cbdbb175e7ee223a57351c9061e2f242e9a). let config = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"strict":true,"noImplicitAny":false,"strictNullChecks":false,"strictPropertyInitialization":false,"alwaysStrict":false}}"#, ) .expect("individual strict options"); @@ -2570,7 +2599,7 @@ mod tests { fn project_config_parses_use_define_for_class_fields_as_tri_state() { let pinned_true = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"useDefineForClassFields":true}}"#, ) .expect("pinned true"); @@ -2580,7 +2609,7 @@ mod tests { ); let pinned_false = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"useDefineForClassFields":false}}"#, ) .expect("pinned false"); @@ -2590,7 +2619,7 @@ mod tests { ); let silent = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"strict":true}}"#, ) .expect("silent key"); @@ -2601,21 +2630,21 @@ mod tests { fn project_config_parses_import_helpers_as_tri_state() { let pinned_true = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"importHelpers":true}}"#, ) .expect("pinned true"); assert_eq!(pinned_true.options().import_helpers(), Some(true)); let pinned_false = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"importHelpers":false}}"#, ) .expect("pinned false"); assert_eq!(pinned_false.options().import_helpers(), Some(false)); let silent = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"strict":true}}"#, ) .expect("silent key"); @@ -2626,7 +2655,7 @@ mod tests { fn project_config_rejects_wrong_types_and_every_root_escape() { let wrong = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"strict":"yes"}}"#, ) .expect_err("wrong type"); @@ -2634,7 +2663,7 @@ mod tests { let jsx = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"jsx":"invalid-mode"}}"#, ) .expect_err("jsx values use the canonical closed set"); @@ -2645,7 +2674,7 @@ mod tests { let escape = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"compilerOptions":{"outDir":"../outside"}}"#, ) .expect_err("outDir escape"); @@ -2653,7 +2682,7 @@ mod tests { let include_escape = ProjectConfig::parse( &root(), - "/workspace/corpus/tsconfig.json", + abs("/workspace/corpus/tsconfig.json"), r#"{"include":["../outside/**/*.ts"]}"#, ) .expect_err("include escape"); @@ -2665,7 +2694,7 @@ mod tests { let project = root(); let explicit = plan_relative_module( &project, - "/workspace/corpus/cases/dot-prop.ts", + abs("/workspace/corpus/cases/dot-prop.ts"), "../projects/dot-prop/index.js", ResolutionFlavor::Runtime, false, @@ -2674,40 +2703,37 @@ mod tests { assert_eq!( &explicit.candidates()[..4], &[ - PathBuf::from("/workspace/corpus/projects/dot-prop/index.ts"), - PathBuf::from("/workspace/corpus/projects/dot-prop/index.tsx"), - PathBuf::from("/workspace/corpus/projects/dot-prop/index.d.ts"), - PathBuf::from("/workspace/corpus/projects/dot-prop/index.js"), + abs("/workspace/corpus/projects/dot-prop/index.ts"), + abs("/workspace/corpus/projects/dot-prop/index.tsx"), + abs("/workspace/corpus/projects/dot-prop/index.d.ts"), + abs("/workspace/corpus/projects/dot-prop/index.js"), ] ); let directory = plan_relative_module( &project, - "/workspace/corpus/cases/ohash.ts", + abs("/workspace/corpus/cases/ohash.ts"), "../projects/ohash/src/crypto/node", ResolutionFlavor::Runtime, false, ) .expect("extension and index plan"); - assert!(directory.candidates().contains(&PathBuf::from( - "/workspace/corpus/projects/ohash/src/crypto/node/index.ts" - ))); + let expected_index = abs("/workspace/corpus/projects/ohash/src/crypto/node/index.ts"); + assert!(directory.candidates().contains(&expected_index)); assert_eq!( directory.select(|path| path.ends_with("index.ts")), - Some(Path::new( - "/workspace/corpus/projects/ohash/src/crypto/node/index.ts" - )) + Some(expected_index.as_path()) ); } #[test] fn relative_module_plan_rejects_bare_url_unsupported_and_escaping_specifiers() { let project = root(); - let importer = "/workspace/corpus/cases/main.ts"; + let importer = abs("/workspace/corpus/cases/main.ts"); assert!(matches!( plan_relative_module( &project, - importer, + &importer, "node:fs", ResolutionFlavor::Runtime, false @@ -2717,7 +2743,7 @@ mod tests { assert!(matches!( plan_relative_module( &project, - importer, + &importer, "./x.ts?raw", ResolutionFlavor::Runtime, false @@ -2727,7 +2753,7 @@ mod tests { assert!( plan_relative_module( &project, - importer, + &importer, "../../outside", ResolutionFlavor::Runtime, false @@ -2741,7 +2767,7 @@ mod tests { let project = root(); let flat = PackageJson::parse( &project, - "/workspace/corpus/projects/escape-string-regexp/package.json", + abs("/workspace/corpus/projects/escape-string-regexp/package.json"), r#"{"name":"escape-string-regexp","type":"module","exports":"./index.js"}"#, ) .expect("flat corpus package"); @@ -2753,12 +2779,12 @@ mod tests { &ResolutionConditions::for_mode(PackageMode::Import) ) .expect("root export"), - PathBuf::from("/workspace/corpus/projects/escape-string-regexp/index.js") + abs("/workspace/corpus/projects/escape-string-regexp/index.js") ); let nested = PackageJson::parse( &project, - "/workspace/corpus/projects/ohash/package.json", + abs("/workspace/corpus/projects/ohash/package.json"), r#"{ "name":"ohash", "exports": { @@ -2778,7 +2804,7 @@ mod tests { &ResolutionConditions::for_mode(PackageMode::Import) ) .expect("node condition"), - PathBuf::from("/workspace/corpus/projects/ohash/dist/crypto/node/index.mjs") + abs("/workspace/corpus/projects/ohash/dist/crypto/node/index.mjs") ); assert_eq!( nested @@ -2789,7 +2815,7 @@ mod tests { &ResolutionConditions::for_mode(PackageMode::Import) ) .expect("wildcard"), - PathBuf::from("/workspace/corpus/projects/ohash/src/serialize.ts") + abs("/workspace/corpus/projects/ohash/src/serialize.ts") ); } @@ -2798,7 +2824,7 @@ mod tests { let project = root(); let package = PackageJson::parse( &project, - "/workspace/corpus/projects/defu/package.json", + abs("/workspace/corpus/projects/defu/package.json"), r#"{ "exports": { ".": { @@ -2819,7 +2845,7 @@ mod tests { &ResolutionConditions::for_mode(PackageMode::Types) ) .expect("types condition"), - PathBuf::from("/workspace/corpus/projects/defu/dist/defu.d.mts") + abs("/workspace/corpus/projects/defu/dist/defu.d.mts") ); } @@ -2828,7 +2854,7 @@ mod tests { let project = root(); let package = PackageJson::parse( &project, - "/workspace/corpus/projects/demo/package.json", + abs("/workspace/corpus/projects/demo/package.json"), r##"{ "imports": { "#internal/*": "./src/*.ts", @@ -2845,9 +2871,7 @@ mod tests { &ResolutionConditions::for_mode(PackageMode::Import) ) .expect("local import"), - PackageTarget::Path(PathBuf::from( - "/workspace/corpus/projects/demo/src/value.ts" - )) + PackageTarget::Path(abs("/workspace/corpus/projects/demo/src/value.ts")) ); assert_eq!( package @@ -2866,7 +2890,7 @@ mod tests { let project = root(); let escape = PackageJson::parse( &project, - "/workspace/corpus/projects/demo/package.json", + abs("/workspace/corpus/projects/demo/package.json"), r#"{"exports":"./../outside.js"}"#, ) .expect("metadata parses"); @@ -2882,7 +2906,7 @@ mod tests { let node_modules = PackageJson::parse( &project, - "/workspace/corpus/projects/demo/package.json", + abs("/workspace/corpus/projects/demo/package.json"), r#"{"exports":"./node_modules/dep/index.js"}"#, ) .expect("metadata parses"); @@ -2902,7 +2926,7 @@ mod tests { let project = root(); let package = PackageJson::parse( &project, - "/workspace/corpus/projects/tslib/package.json", + abs("/workspace/corpus/projects/tslib/package.json"), r#"{ "main":"./tslib.js", "module":"./tslib.es6.js", @@ -2915,19 +2939,19 @@ mod tests { package .resolve_export(&project, ".", PackageMode::Import, &conditions) .expect("module entry"), - PathBuf::from("/workspace/corpus/projects/tslib/tslib.es6.js") + abs("/workspace/corpus/projects/tslib/tslib.es6.js") ); assert_eq!( package .resolve_export(&project, ".", PackageMode::Types, &conditions) .expect("types entry"), - PathBuf::from("/workspace/corpus/projects/tslib/tslib.d.ts") + abs("/workspace/corpus/projects/tslib/tslib.d.ts") ); // mitt ships bare (no leading `./`) legacy fields across all three modes. let bare = PackageJson::parse( &project, - "/workspace/corpus/projects/mitt/package.json", + abs("/workspace/corpus/projects/mitt/package.json"), r#"{ "main":"dist/mitt.js", "module":"dist/mitt.mjs", @@ -2938,17 +2962,17 @@ mod tests { assert_eq!( bare.resolve_export(&project, ".", PackageMode::Require, &conditions) .expect("bare main entry"), - PathBuf::from("/workspace/corpus/projects/mitt/dist/mitt.js") + abs("/workspace/corpus/projects/mitt/dist/mitt.js") ); assert_eq!( bare.resolve_export(&project, ".", PackageMode::Import, &conditions) .expect("bare module entry"), - PathBuf::from("/workspace/corpus/projects/mitt/dist/mitt.mjs") + abs("/workspace/corpus/projects/mitt/dist/mitt.mjs") ); assert_eq!( bare.resolve_export(&project, ".", PackageMode::Types, &conditions) .expect("bare typings entry"), - PathBuf::from("/workspace/corpus/projects/mitt/index.d.ts") + abs("/workspace/corpus/projects/mitt/index.d.ts") ); } @@ -2956,10 +2980,10 @@ mod tests { #[test] fn f13_string_true_parses_as_boolean_true() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"incremental": "true"}}"#, ) .expect("string \"true\" must coerce to boolean"); @@ -2968,10 +2992,10 @@ mod tests { #[test] fn f13_string_false_parses_as_boolean_false() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"incremental": "false"}}"#, ) .expect("string \"false\" must coerce to boolean"); @@ -2980,10 +3004,10 @@ mod tests { #[test] fn f13_string_true_case_insensitive_parses_as_boolean_true() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"composite": "True"}}"#, ) .expect("string \"True\" must coerce to boolean"); @@ -2992,10 +3016,10 @@ mod tests { #[test] fn f13_string_true_with_whitespace_parses_as_boolean_true() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"incremental": " true "}}"#, ) .expect("string \" true \" must coerce to boolean"); @@ -3004,10 +3028,10 @@ mod tests { #[test] fn f13_string_composite_true_parses_as_boolean_true() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"composite": "true"}}"#, ) .expect("string \"true\" must coerce to boolean for composite"); @@ -3016,10 +3040,10 @@ mod tests { #[test] fn f13_invalid_string_still_rejected_for_boolean_field() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let error = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"incremental": "yes"}}"#, ) .expect_err("non-boolean string must be rejected"); @@ -3031,10 +3055,10 @@ mod tests { #[test] fn f13_number_still_rejected_for_boolean_field() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let error = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"incremental": 1}}"#, ) .expect_err("number must be rejected for boolean field"); @@ -3046,10 +3070,10 @@ mod tests { #[test] fn f13_jsx_preserve_capital_parses_case_insensitively() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"jsx": "Preserve"}}"#, ) .expect("capitalized jsx value must be accepted"); @@ -3058,10 +3082,10 @@ mod tests { #[test] fn f13_jsx_react_uppercase_parses_case_insensitively() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"jsx": "REACT"}}"#, ) .expect("uppercase jsx value must be accepted"); @@ -3070,10 +3094,10 @@ mod tests { #[test] fn f13_jsx_react_native_mixed_case_parses_case_insensitively() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"jsx": "React-Native"}}"#, ) .expect("mixed-case jsx value must be accepted"); @@ -3082,10 +3106,10 @@ mod tests { #[test] fn f13_jsx_react_jsx_mixed_case_parses_case_insensitively() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"jsx": "React-JSX"}}"#, ) .expect("mixed-case jsx value must be accepted"); @@ -3094,10 +3118,10 @@ mod tests { #[test] fn f13_jsx_react_jsxdev_mixed_case_parses_case_insensitively() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"jsx": "React-JSXDev"}}"#, ) .expect("mixed-case jsx value must be accepted"); @@ -3106,10 +3130,10 @@ mod tests { #[test] fn f13_jsx_with_whitespace_parses_case_insensitively() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let config = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"jsx": " Preserve "}}"#, ) .expect("whitespace-padded jsx value must be accepted"); @@ -3118,10 +3142,10 @@ mod tests { #[test] fn f13_jsx_invalid_string_still_rejected() { - let root = ProjectRoot::new("/workspace").expect("valid root"); + let root = ProjectRoot::new(abs("/workspace")).expect("valid root"); let error = ProjectConfig::parse( &root, - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions": {"jsx": "react-jsx-foo"}}"#, ) .expect_err("invalid jsx value must be rejected"); diff --git a/crates/bamts-compiler/src/project/build_mode.rs b/crates/bamts-compiler/src/project/build_mode.rs index 8e9b5432..4e106938 100644 --- a/crates/bamts-compiler/src/project/build_mode.rs +++ b/crates/bamts-compiler/src/project/build_mode.rs @@ -1114,12 +1114,13 @@ pub fn clean( #[cfg(test)] mod tests { use super::super::references::ReferenceGraph; + use super::super::test_paths::{abs, abs_json, abs_text}; use super::super::tsconfig::TsConfig; use super::super::{ProjectConfig, ProjectRoot}; use super::*; fn test_root() -> ProjectRoot { - ProjectRoot::new("/workspace/test").expect("absolute test root") + ProjectRoot::new(abs("/workspace/test")).expect("absolute test root") } fn parse_tsconfig(root: &ProjectRoot, path: impl AsRef, source: &str) -> TsConfig { @@ -1276,16 +1277,16 @@ mod tests { } fn lib_path() -> PathBuf { - PathBuf::from("/workspace/test/lib/tsconfig.json") + abs("/workspace/test/lib/tsconfig.json") } fn app_path() -> PathBuf { - PathBuf::from("/workspace/test/app/tsconfig.json") + abs("/workspace/test/app/tsconfig.json") } - fn build_output(path: &str) -> BuildOutputs { + fn build_output(path: impl AsRef) -> BuildOutputs { let mut files = BTreeMap::new(); - files.insert(PathBuf::from(path), vec![b'j', b's']); + files.insert(path.as_ref().to_path_buf(), vec![b'j', b's']); BuildOutputs { files } } @@ -1302,8 +1303,8 @@ mod tests { &app_path(), parse_config(&root, app_path(), r#"{"references": [{"path": "../lib"}]}"#), ) - .with_compile_output(&lib_path(), build_output("/workspace/test/lib/lib.js")) - .with_compile_output(&app_path(), build_output("/workspace/test/app/app.js")); + .with_compile_output(&lib_path(), build_output(abs("/workspace/test/lib/lib.js"))) + .with_compile_output(&app_path(), build_output(abs("/workspace/test/app/app.js"))); let options = BuildOptions { force: true, @@ -1328,8 +1329,8 @@ mod tests { &app_path(), parse_config(&root, app_path(), r#"{"references": [{"path": "../lib"}]}"#), ) - .with_compile_output(&lib_path(), build_output("/workspace/test/lib/lib.js")) - .with_compile_output(&app_path(), build_output("/workspace/test/app/app.js")); + .with_compile_output(&lib_path(), build_output(abs("/workspace/test/lib/lib.js"))) + .with_compile_output(&app_path(), build_output(abs("/workspace/test/app/app.js"))); let options = BuildOptions { dry_run: true, @@ -1375,14 +1376,10 @@ mod tests { .clone() .unwrap(); - let lib_sources = BTreeMap::from([( - PathBuf::from("/workspace/test/lib/a.ts"), - source_signature("lib v1"), - )]); - let app_sources = BTreeMap::from([( - PathBuf::from("/workspace/test/app/b.ts"), - source_signature("app v1"), - )]); + let lib_sources = + BTreeMap::from([(abs("/workspace/test/lib/a.ts"), source_signature("lib v1"))]); + let app_sources = + BTreeMap::from([(abs("/workspace/test/app/b.ts"), source_signature("app v1"))]); let lib_config = parse_config( &root, @@ -1400,15 +1397,15 @@ mod tests { .with_config(&app_path(), app_config.clone()) .with_source_signatures(&lib_path(), lib_sources.clone()) .with_source_signatures(&app_path(), app_sources.clone()) - .with_compile_output(&lib_path(), build_output("/workspace/test/lib/lib.js")) - .with_compile_output(&app_path(), build_output("/workspace/test/app/app.js")) + .with_compile_output(&lib_path(), build_output(abs("/workspace/test/lib/lib.js"))) + .with_compile_output(&app_path(), build_output(abs("/workspace/test/app/app.js"))) .with_declared_outputs( &lib_path(), - BTreeSet::from([PathBuf::from("/workspace/test/lib/lib.js")]), + BTreeSet::from([abs("/workspace/test/lib/lib.js")]), ) .with_declared_outputs( &app_path(), - BTreeSet::from([PathBuf::from("/workspace/test/app/app.js")]), + BTreeSet::from([abs("/workspace/test/app/app.js")]), ); // First build. @@ -1439,7 +1436,7 @@ mod tests { version: Arc::from(BUILD_INFO_SCHEMA), options: options_signature(&lib_config), sources: lib_sources, - outputs: BTreeSet::from([PathBuf::from("/workspace/test/lib/lib.js")]), + outputs: BTreeSet::from([abs("/workspace/test/lib/lib.js")]), signature: lib_sig, }, ) @@ -1449,7 +1446,7 @@ mod tests { version: Arc::from(BUILD_INFO_SCHEMA), options: options_signature(&app_config), sources: app_sources, - outputs: BTreeSet::from([PathBuf::from("/workspace/test/app/app.js")]), + outputs: BTreeSet::from([abs("/workspace/test/app/app.js")]), signature: app_sig, }, ); @@ -1482,10 +1479,8 @@ mod tests { .clone() .unwrap(); - let lib_sources = BTreeMap::from([( - PathBuf::from("/workspace/test/lib/a.ts"), - source_signature("lib v1"), - )]); + let lib_sources = + BTreeMap::from([(abs("/workspace/test/lib/a.ts"), source_signature("lib v1"))]); let lib_config = parse_config( &root, lib_path(), @@ -1495,10 +1490,10 @@ mod tests { let mut host = InMemoryHost::new() .with_config(&lib_path(), lib_config.clone()) .with_source_signatures(&lib_path(), lib_sources.clone()) - .with_compile_output(&lib_path(), build_output("/workspace/test/lib/lib.js")) + .with_compile_output(&lib_path(), build_output(abs("/workspace/test/lib/lib.js"))) .with_declared_outputs( &lib_path(), - BTreeSet::from([PathBuf::from("/workspace/test/lib/lib.js")]), + BTreeSet::from([abs("/workspace/test/lib/lib.js")]), ); execute(&graph, &mut host, &root, &BuildOptions::default()).unwrap(); @@ -1511,10 +1506,7 @@ mod tests { ); let mut changed_sources = lib_sources; - changed_sources.insert( - PathBuf::from("/workspace/test/lib/a.ts"), - source_signature("lib v2"), - ); + changed_sources.insert(abs("/workspace/test/lib/a.ts"), source_signature("lib v2")); host = host .with_source_signatures(&lib_path(), changed_sources.clone()) @@ -1524,7 +1516,7 @@ mod tests { version: Arc::from(BUILD_INFO_SCHEMA), options: options_signature(&lib_config), sources: changed_sources.clone(), - outputs: BTreeSet::from([PathBuf::from("/workspace/test/lib/lib.js")]), + outputs: BTreeSet::from([abs("/workspace/test/lib/lib.js")]), signature: lib_sig, }, ); @@ -1572,8 +1564,8 @@ mod tests { let app = parse_tsconfig(&root, app_path(), r#"{"references": [{"path": "../lib"}]}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&app, &lib]).unwrap(); - let lib_out = PathBuf::from("/workspace/test/lib/lib.js"); - let app_out = PathBuf::from("/workspace/test/app/app.js"); + let lib_out = abs("/workspace/test/lib/lib.js"); + let app_out = abs("/workspace/test/app/app.js"); let mut host = InMemoryHost::new() .with_declared_outputs(&lib_path(), BTreeSet::from([lib_out.clone()])) @@ -1595,8 +1587,8 @@ mod tests { let app = parse_tsconfig(&root, app_path(), r#"{"references": [{"path": "../lib"}]}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&app, &lib]).unwrap(); - let lib_out = PathBuf::from("/workspace/test/lib/lib.js"); - let app_out = PathBuf::from("/workspace/test/app/app.js"); + let lib_out = abs("/workspace/test/lib/lib.js"); + let app_out = abs("/workspace/test/app/app.js"); let mut host = InMemoryHost::new() .with_declared_outputs(&lib_path(), BTreeSet::from([lib_out.clone()])) @@ -1621,10 +1613,8 @@ mod tests { let lib = parse_tsconfig(&root, lib_path(), r#"{}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&lib]).unwrap(); - let mut host = InMemoryHost::new().with_declared_outputs( - &lib_path(), - BTreeSet::from([PathBuf::from("/outside/root.js")]), - ); + let mut host = InMemoryHost::new() + .with_declared_outputs(&lib_path(), BTreeSet::from([abs("/outside/root.js")])); let options = BuildOptions { clean: true, @@ -1642,7 +1632,7 @@ mod tests { let mut host = InMemoryHost::new() .with_config(&lib_path(), parse_config(&root, lib_path(), r#"{}"#)) - .with_compile_output(&lib_path(), build_output("/workspace/test/lib/lib.js")); + .with_compile_output(&lib_path(), build_output(abs("/workspace/test/lib/lib.js"))); let options = BuildOptions { verbose: true, @@ -1676,10 +1666,8 @@ mod tests { lib_path(), r#"{"compilerOptions": {"composite": true}}"#, ); - let lib_sources = BTreeMap::from([( - PathBuf::from("/workspace/test/lib/a.ts"), - source_signature("lib v1"), - )]); + let lib_sources = + BTreeMap::from([(abs("/workspace/test/lib/a.ts"), source_signature("lib v1"))]); let lib_sig = project_signature( graph.node(lib_path().as_path()).unwrap(), &options_signature(&lib_config), @@ -1691,7 +1679,7 @@ mod tests { version: Arc::from(BUILD_INFO_SCHEMA), options: options_signature(&lib_config), sources: lib_sources.clone(), - outputs: BTreeSet::from([PathBuf::from("/workspace/test/lib/lib.js")]), + outputs: BTreeSet::from([abs("/workspace/test/lib/lib.js")]), signature: lib_sig, }; @@ -1701,12 +1689,9 @@ mod tests { .with_build_info(&lib_info, build_info) .with_declared_outputs( &lib_path(), - BTreeSet::from([PathBuf::from("/workspace/test/lib/lib.js")]), + BTreeSet::from([abs("/workspace/test/lib/lib.js")]), ) - .with_output( - PathBuf::from("/workspace/test/lib/lib.js"), - vec![b'j', b's'], - ); + .with_output(abs("/workspace/test/lib/lib.js"), vec![b'j', b's']); let first = execute(&graph, &mut host, &root, &BuildOptions::default()).unwrap(); let second = execute(&graph, &mut host, &root, &BuildOptions::default()).unwrap(); @@ -1720,10 +1705,10 @@ mod tests { let lib = parse_tsconfig(&root, lib_path(), r#"{}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&lib]).unwrap(); - let out = PathBuf::from("/workspace/test/lib/lib.js"); + let out = abs("/workspace/test/lib/lib.js"); let mut host = InMemoryHost::new() .with_config(&lib_path(), parse_config(&root, lib_path(), r#"{}"#)) - .with_compile_output(&lib_path(), build_output(out.to_str().unwrap())); + .with_compile_output(&lib_path(), build_output(&out)); execute(&graph, &mut host, &root, &BuildOptions::default()).unwrap(); @@ -1741,17 +1726,17 @@ mod tests { options: Arc::from(r#"{"composite":true,"strict":false}"#), sources: BTreeMap::from([ ( - PathBuf::from("/workspace/test/app/a.ts"), + abs("/workspace/test/app/a.ts"), source_signature("export const a = 1;"), ), ( - PathBuf::from("/workspace/test/app/b.ts"), + abs("/workspace/test/app/b.ts"), source_signature("export const b = 2;"), ), ]), outputs: BTreeSet::from([ - PathBuf::from("/workspace/test/app/a.js"), - PathBuf::from("/workspace/test/app/b.js"), + abs("/workspace/test/app/a.js"), + abs("/workspace/test/app/b.js"), ]), signature: Arc::from("0123456789abcdef"), } @@ -1833,32 +1818,35 @@ mod tests { #[test] fn build_info_decoder_rejects_duplicate_and_unsorted_paths() { let digest = source_signature("source"); + let a = abs_json("/workspace/a.ts"); let duplicate = format!( - "{{\"version\":\"{BUILD_INFO_SCHEMA}\",\"options\":\"null\",\"signature\":\"0123456789abcdef\",\"sources\":[[\"/workspace/a.ts\",\"{digest}\"],[\"/workspace/a.ts\",\"{digest}\"]],\"outputs\":[]}}" + "{{\"version\":\"{BUILD_INFO_SCHEMA}\",\"options\":\"null\",\"signature\":\"0123456789abcdef\",\"sources\":[[\"{a}\",\"{digest}\"],[\"{a}\",\"{digest}\"]],\"outputs\":[]}}" ); assert_eq!( BuildInfo::decode(duplicate.as_bytes()), Err(BuildInfoDecodeError::DuplicatePath { - path: PathBuf::from("/workspace/a.ts") + path: abs("/workspace/a.ts") }) ); + let b = abs_json("/workspace/b.ts"); let unsorted = format!( - "{{\"version\":\"{BUILD_INFO_SCHEMA}\",\"options\":\"null\",\"signature\":\"0123456789abcdef\",\"sources\":[[\"/workspace/b.ts\",\"{digest}\"],[\"/workspace/a.ts\",\"{digest}\"]],\"outputs\":[]}}" + "{{\"version\":\"{BUILD_INFO_SCHEMA}\",\"options\":\"null\",\"signature\":\"0123456789abcdef\",\"sources\":[[\"{b}\",\"{digest}\"],[\"{a}\",\"{digest}\"]],\"outputs\":[]}}" ); assert_eq!( BuildInfo::decode(unsorted.as_bytes()), Err(BuildInfoDecodeError::UnsortedPath { - previous: PathBuf::from("/workspace/b.ts"), - found: PathBuf::from("/workspace/a.ts") + previous: abs("/workspace/b.ts"), + found: abs("/workspace/a.ts") }) ); } #[test] fn build_info_decoder_rejects_invalid_digest_and_path() { + let a = abs_json("/workspace/a.ts"); let invalid_digest = format!( - "{{\"version\":\"{BUILD_INFO_SCHEMA}\",\"options\":\"null\",\"signature\":\"0123456789abcdef\",\"sources\":[[\"/workspace/a.ts\",\"g{}\"]],\"outputs\":[]}}", + "{{\"version\":\"{BUILD_INFO_SCHEMA}\",\"options\":\"null\",\"signature\":\"0123456789abcdef\",\"sources\":[[\"{a}\",\"g{}\"]],\"outputs\":[]}}", "0".repeat(SOURCE_DIGEST_HEX_LEN - 1) ); assert!(matches!( @@ -1867,13 +1855,14 @@ mod tests { )); let digest = source_signature("source"); + let outside = abs_json("/workspace/../outside.ts"); let traversal = format!( - "{{\"version\":\"{BUILD_INFO_SCHEMA}\",\"options\":\"null\",\"signature\":\"0123456789abcdef\",\"sources\":[[\"/workspace/../outside.ts\",\"{digest}\"]],\"outputs\":[]}}" + "{{\"version\":\"{BUILD_INFO_SCHEMA}\",\"options\":\"null\",\"signature\":\"0123456789abcdef\",\"sources\":[[\"{outside}\",\"{digest}\"]],\"outputs\":[]}}" ); assert_eq!( BuildInfo::decode(traversal.as_bytes()), Err(BuildInfoDecodeError::InvalidPath { - path: "/workspace/../outside.ts".to_string() + path: abs_text("/workspace/../outside.ts") }) ); } @@ -1906,9 +1895,9 @@ mod tests { r#"{"compilerOptions":{"composite":true}}"#, ); let options = options_signature(&config); - let source_path = PathBuf::from("/workspace/test/lib/a.ts"); + let source_path = abs("/workspace/test/lib/a.ts"); let sources = BTreeMap::from([(source_path.clone(), source_signature("original"))]); - let outputs = BTreeSet::from([PathBuf::from("/workspace/test/lib/lib.js")]); + let outputs = BTreeSet::from([abs("/workspace/test/lib/lib.js")]); let signature = project_signature(node, &options, &sources, &BTreeMap::new()); let info = BuildInfo { version: Arc::from(BUILD_INFO_SCHEMA), @@ -1956,10 +1945,10 @@ mod tests { .build_info_path .clone() .unwrap(); - let source_path = PathBuf::from("/workspace/test/lib/a.ts"); + let source_path = abs("/workspace/test/lib/a.ts"); let source_signatures = BTreeMap::from([(source_path, source_signature("export const a = 1;"))]); - let output_path = PathBuf::from("/workspace/test/lib/lib.js"); + let output_path = abs("/workspace/test/lib/lib.js"); let mut host = InMemoryHost::new() .with_config( &lib_path(), @@ -1970,7 +1959,7 @@ mod tests { ), ) .with_source_signatures(&lib_path(), source_signatures) - .with_compile_output(&lib_path(), build_output(output_path.to_str().unwrap())) + .with_compile_output(&lib_path(), build_output(&output_path)) .with_declared_outputs(&lib_path(), BTreeSet::from([output_path])); execute(&graph, &mut host, &root, &BuildOptions::default()).unwrap(); diff --git a/crates/bamts-compiler/src/project/canonical.rs b/crates/bamts-compiler/src/project/canonical.rs new file mode 100644 index 00000000..934fbf53 --- /dev/null +++ b/crates/bamts-compiler/src/project/canonical.rs @@ -0,0 +1,249 @@ +//! One spelling for every canonical path. +//! +//! On Windows, `fs::canonicalize` returns the verbatim spelling (`\\?\C:\dir`), +//! while the working directory, the temp directory and user input use the plain +//! spelling (`C:\dir`). `Path::starts_with` treats the two spellings as different +//! roots, so a file that lies inside a project looks like an escape. Every canonical +//! path that the compiler, the CLI and the `bamts` facade compare with a project +//! root goes through [`canonicalize`], which returns the plain spelling whenever +//! the plain spelling names the same file. +//! +//! Code that only compares a path with other `fs::canonicalize` results, such as +//! the repository checks in `bamts-verification`, may keep calling +//! `fs::canonicalize`. A path that is compared with compiler output must use this +//! function: `check_cells` pairs case units with loaded modules that way. + +use std::{ + ffi::{OsStr, OsString}, + fs, io, + path::{Component, MAIN_SEPARATOR, Path, PathBuf, Prefix}, +}; + +/// `fs::canonicalize` with the verbatim prefix removed when that is safe. +/// +/// On platforms without path prefixes this is exactly `fs::canonicalize`. +pub fn canonicalize(path: impl AsRef) -> io::Result { + fs::canonicalize(path).map(|canonical| simplify(&canonical)) +} + +/// Rewrites a verbatim disk path (`\\?\C:\a`) or verbatim UNC path +/// (`\\?\UNC\server\share\a`) to its plain spelling (`C:\a`, `\\server\share\a`). +/// +/// A verbatim path stays as it is when the plain spelling could name a different +/// file: when it has a relative `.` or `..` segment, when a segment is a reserved +/// device name, ends in a dot or a space, contains a character that Win32 does not +/// accept, or is not valid Unicode, and when it has no root directory. +#[must_use] +pub fn simplify(path: &Path) -> PathBuf { + let mut components = path.components(); + let Some(Component::Prefix(prefix)) = components.next() else { + return path.to_path_buf(); + }; + let mut plain = match prefix.kind() { + Prefix::VerbatimDisk(drive) => PathBuf::from(format!("{}:", char::from(drive))), + Prefix::VerbatimUNC(server, share) => { + if !(is_plain_name(server) && is_plain_name(share)) { + return path.to_path_buf(); + } + let mut unc = OsString::from(r"\\"); + unc.push(server); + unc.push(r"\"); + unc.push(share); + PathBuf::from(unc) + } + _ => return path.to_path_buf(), + }; + if components.next() != Some(Component::RootDir) { + return path.to_path_buf(); + } + plain.push(Component::RootDir.as_os_str()); + for component in components { + match component { + Component::Normal(name) if is_plain_name(name) => plain.push(name), + _ => return path.to_path_buf(), + } + } + plain +} + +/// The path as TypeScript prints it: `/` separates segments on every platform. +/// +/// Source-map URLs, `sources` entries and tsc messages use this text. Where the +/// platform separator is already `/`, the text is the path unchanged. +#[must_use] +pub fn slash_text(path: &Path) -> String { + path.to_string_lossy().replace(MAIN_SEPARATOR, "/") +} + +fn is_plain_name(name: &OsStr) -> bool { + let Some(name) = name.to_str() else { + return false; + }; + let invalid = |character: char| { + character.is_control() + || matches!( + character, + '<' | '>' | ':' | '"' | '/' | '\\' | '|' | '?' | '*' + ) + }; + !name.is_empty() + && !name.ends_with(['.', ' ']) + && !name.contains(invalid) + && !is_reserved_device_name(name) +} + +/// Win32 reads `NUL`, `nul.txt` and `NUL .txt` as the device, not as a file. +fn is_reserved_device_name(name: &str) -> bool { + let stem = name + .split('.') + .next() + .unwrap_or_default() + .trim_end_matches(' '); + const DEVICES: [&str; 6] = ["CON", "PRN", "AUX", "NUL", "CONIN$", "CONOUT$"]; + if DEVICES + .iter() + .any(|device| stem.eq_ignore_ascii_case(device)) + { + return true; + } + let mut characters = stem.chars(); + let family: String = characters.by_ref().take(3).collect(); + let numbered = matches!( + (characters.next(), characters.next()), + (Some('1'..='9' | '¹' | '²' | '³'), None) + ); + numbered && (family.eq_ignore_ascii_case("COM") || family.eq_ignore_ascii_case("LPT")) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn fresh_dir(label: &str) -> PathBuf { + let nonce = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("clock after epoch") + .as_nanos(); + let directory = std::env::temp_dir().join(format!( + "bamts-canonical-{label}-{}-{nonce}", + std::process::id() + )); + fs::create_dir(&directory).expect("create test directory"); + directory + } + + #[test] + fn canonicalize_of_a_missing_path_is_the_platform_error() { + let missing = std::env::temp_dir().join("bamts-canonical-path-that-does-not-exist"); + assert_eq!( + canonicalize(&missing).expect_err("missing").kind(), + io::ErrorKind::NotFound + ); + } + + #[test] + fn canonicalize_names_the_same_directory_as_the_platform() { + let directory = fresh_dir("path"); + let platform = fs::canonicalize(&directory).expect("platform canonical path"); + let canonical = canonicalize(&directory).expect("canonical path"); + + assert_eq!(simplify(&platform), canonical); + assert!(canonical.is_absolute()); + + fs::remove_dir_all(directory).expect("remove temporary directory"); + } + + #[test] + fn slash_text_uses_forward_slashes_and_keeps_relative_segments() { + let mut path = PathBuf::from(".."); + path.push("sub"); + path.push("name.part.js.map"); + + assert_eq!(slash_text(&path), "../sub/name.part.js.map"); + assert_eq!(slash_text(Path::new("a/b")), "a/b"); + assert_eq!(slash_text(Path::new("")), ""); + } + + #[test] + #[cfg(windows)] + fn slash_text_rewrites_windows_separators() { + assert_eq!(slash_text(Path::new(r"C:\a\b.ts")), "C:/a/b.ts"); + assert_eq!(slash_text(Path::new(r"..\a\*\*.ts")), "../a/*/*.ts"); + } + + #[test] + fn paths_without_a_verbatim_prefix_are_returned_unchanged() { + for input in ["", "a/b", "/", "/a/b/../c"] { + assert_eq!(simplify(Path::new(input)), Path::new(input), "{input}"); + } + } + + #[test] + #[cfg(windows)] + fn verbatim_disk_and_unc_paths_become_plain_paths() { + for (input, expected) in [ + (r"\\?\C:\", r"C:\"), + (r"\\?\C:\Users\me\project", r"C:\Users\me\project"), + (r"\\?\d:\a b\c.d", r"d:\a b\c.d"), + (r"\\?\UNC\server\share\dir", r"\\server\share\dir"), + (r"\\?\UNC\server\share\", r"\\server\share\"), + (r"C:\already\plain", r"C:\already\plain"), + (r"\\server\share\plain", r"\\server\share\plain"), + ] { + assert_eq!(simplify(Path::new(input)), Path::new(expected), "{input}"); + } + } + + #[test] + #[cfg(windows)] + fn verbatim_paths_stay_verbatim_when_the_plain_spelling_is_not_equivalent() { + for input in [ + r"\\?\C:", + r"\\?\C:\a\.\b", + r"\\?\C:\a\..\b", + r"\\?\C:\a.\b", + r"\\?\C:\a \b", + r"\\?\C:\dir\nul", + r"\\?\C:\dir\NUL.txt", + r"\\?\C:\dir\com1", + r"\\?\C:\dir\COM².txt", + r"\\?\C:\dir\LPT9.log", + r"\\?\C:\dir\CONOUT$", + r"\\?\C:\dir\file:stream", + r"\\?\C:\dir\what?", + r"\\?\UNC\server\sh*re\dir", + r"\\?\pipe\name", + r"\\?\Volume{01234567-89ab-cdef-0123-456789abcdef}\dir", + r"\\.\C:\dir", + ] { + assert_eq!(simplify(Path::new(input)), Path::new(input), "{input}"); + } + for input in [ + r"\\?\C:\dir\comma", + r"\\?\C:\dir\com10", + r"\\?\C:\dir\nullable", + r"\\?\C:\dir\console", + ] { + assert_ne!(simplify(Path::new(input)), Path::new(input), "{input}"); + } + } + + #[test] + #[cfg(windows)] + fn canonicalize_returns_the_plain_spelling_of_a_real_directory() { + let directory = fresh_dir("plain"); + let canonical = canonicalize(&directory).expect("canonical path"); + + assert!( + matches!( + canonical.components().next(), + Some(Component::Prefix(prefix)) if matches!(prefix.kind(), Prefix::Disk(_)) + ), + "{} is not a plain drive path", + canonical.display() + ); + assert!(canonical.starts_with(simplify(&directory))); + + fs::remove_dir_all(directory).expect("remove temporary directory"); + } +} diff --git a/crates/bamts-compiler/src/project/effective.rs b/crates/bamts-compiler/src/project/effective.rs index ba36b16b..8403e30b 100644 --- a/crates/bamts-compiler/src/project/effective.rs +++ b/crates/bamts-compiler/src/project/effective.rs @@ -1,16 +1,17 @@ use super::{ - CompilerOptions, JsonObject, JsonValue, PathError, ProjectRoot, + CompilerOptions, JsonObject, JsonValue, PackageJson, PathError, ProjectRoot, build_mode::{ BUILD_INFO_SCHEMA, BuildInfo, BuildOutputs, is_project_up_to_date, options_signature, project_signature, source_signature, }, + canonical::slash_text, output::{ArtifactKind, ArtifactSelection, OutputMapError, PlanRequest, map_output_paths}, references::{ReferenceError, ReferenceGraph, resolve_config_file_name}, tsconfig::{ProjectReference, ResolvedExtends, TsConfig, TsConfigError, resolve_extends}, }; use crate::{ diagnostic::Diagnostic, - emitter::{self, EmitFileNames, EmitOptions}, + emitter::{self, EmitFileNames, EmitOptions, ModuleKind}, lint::{LintProfile, LintTable}, pipeline::{FrontendMode, compile_program_frontend_with_lints}, program::{JsxRoutingDecision, ProgramLoader, ProgramOutputKind}, @@ -391,6 +392,7 @@ pub fn compile_project( .insert(artifact.kind, artifact.path.clone()); } let jsx_route = program.jsx_routing_decision(ProgramOutputKind::JavaScript); + let erased_exports = checked.erased_exports(&program); let mut outputs = BuildOutputs::default(); for module in checked.modules() { let source_path = sources @@ -405,6 +407,16 @@ pub fn compile_project( emit_options(project.options(), module.source_file().source_id()); emit_options.declaration = selection.declaration; emit_options.emit_declaration_only = selection.declaration && !selection.javascript; + emit_options.verbatim_module_syntax = project.options().verbatim_module_syntax(); + emit_options.keeps_imports = project.options().keeps_imports(); + emit_options.erased_imports = Arc::new(checked.erased_imports( + &program, + &erased_exports, + module.source_file().source_id(), + )); + if emit_options.module == Some(ModuleKind::Node) { + emit_options.module = Some(node_module_format(project, source_path, fs)); + } diagnostics.extend(option_diagnostics); match jsx_route { JsxRoutingDecision::Emit | JsxRoutingDecision::TransformAndEmit => { @@ -545,17 +557,17 @@ fn surface_map_names( // map directory back to the original source. let source_name = match source_root { Some(root) if has_explicit_source_root && !root.is_empty() => { - Some(Arc::::from(source_relative.to_string_lossy().as_ref())) + Some(Arc::::from(slash_text(&source_relative))) } _ => { let map_directory = map_path.parent().unwrap_or_else(|| Path::new("")); let relative_to_map = relative_path_between(map_directory, source_path); - Some(Arc::::from(relative_to_map.to_string_lossy().as_ref())) + Some(Arc::::from(slash_text(&relative_to_map))) } }; let source_map_url = map_root.map(|root| { - let source_relative_identity = format!("{}.map", output_relative.to_string_lossy()); + let source_relative_identity = format!("{}.map", slash_text(&output_relative)); if is_url_like(root) || root.starts_with('/') { Arc::::from(join_url_segments(root, &source_relative_identity)) } else { @@ -563,7 +575,7 @@ fn surface_map_names( let output_directory = output_path.parent().unwrap_or_else(|| Path::new("")); let url = relative_path_between(output_directory, &base.join(&source_relative_identity)); - Arc::::from(url.to_string_lossy().as_ref()) + Arc::::from(slash_text(&url)) } }); @@ -704,6 +716,7 @@ fn emit_options(options: &CompilerOptions, source_id: SourceId) -> (EmitOptions, ("inlineSources", options.inline_sources()), ("alwaysStrict", options.always_strict()), ("noEmitHelpers", options.no_emit_helpers()), + ("removeComments", options.remove_comments()), ] { if enabled { directives.insert(name.to_owned(), String::from("true")); @@ -729,6 +742,31 @@ fn emit_options(options: &CompilerOptions, source_id: SourceId) -> (EmitOptions, (emit_options, diagnostics) } +/// What a Node module mode emits for one file: the extension decides, then the +/// `type` of the nearest `package.json`, and a file with neither is CommonJS. +fn node_module_format( + project: &EffectiveProject, + source: &Path, + fs: &dyn FileSystem, +) -> ModuleKind { + match source.extension().and_then(|extension| extension.to_str()) { + Some("mts" | "mjs") => return ModuleKind::EsNext, + Some("cts" | "cjs") => return ModuleKind::CommonJs, + _ => {} + } + let root = project.root(); + let package = source.ancestors().skip(1).find_map(|directory| { + let path = directory.join("package.json"); + let text = fs.read(&path).ok()?; + PackageJson::parse(root, &path, &text).ok() + }); + if package.as_ref().and_then(PackageJson::package_type) == Some("module") { + ModuleKind::EsNext + } else { + ModuleKind::CommonJs + } +} + fn single_project_graph(project: &EffectiveProject) -> Result { ReferenceGraph::from_tsconfigs(project.root(), &[project.config()]) .map_err(|error| ProjectCompileError::Reference(Arc::from([error]))) @@ -902,7 +940,7 @@ impl fmt::Display for MaterializeError { Self::FilesListEmpty { config_path } => write!( formatter, "TS18002: The 'files' list in config file '{}' is empty.", - config_path.display() + slash_text(config_path) ), Self::NoInputs { config_path, @@ -911,7 +949,7 @@ impl fmt::Display for MaterializeError { } => write!( formatter, "TS18003: No inputs were found in config file '{}'. Specified 'include' paths were '{:?}' and 'exclude' paths were '{:?}'.", - config_path.display(), + slash_text(config_path), include, exclude ), @@ -1479,11 +1517,10 @@ fn materialize_sources( if value_path.is_absolute() { return Arc::clone(value); } - Arc::::from( - relative_path_between(directory, &source.directory.join(value_path)) - .to_string_lossy() - .as_ref(), - ) + Arc::::from(slash_text(&relative_path_between( + directory, + &source.directory.join(value_path), + ))) }) .collect::>(), ) @@ -2321,6 +2358,277 @@ mod tests { assert!(!javascript.contains(" Option { #[cfg(test)] mod tests { use super::*; + use crate::project::test_paths::abs; + use std::sync::OnceLock; + + fn project_root() -> &'static Path { + static ROOT: OnceLock = OnceLock::new(); + ROOT.get_or_init(|| abs("/project")).as_path() + } fn request<'a>(sources: &'a [PathBuf]) -> PlanRequest<'a> { PlanRequest { - project_root: Path::new("/project"), + project_root: project_root(), sources, root_dir: None, out_dir: Some(Path::new("out")), @@ -524,8 +531,8 @@ mod tests { assert_eq!( map_output_paths(request), Err(OutputMapError::RootDirViolation { - source: PathBuf::from("/project/shared/b.ts"), - root_dir: PathBuf::from("/project/src"), + source: abs("/project/shared/b.ts"), + root_dir: abs("/project/src"), }) ); } @@ -538,15 +545,12 @@ mod tests { PathBuf::from("types/outside.d.ts"), ]; let plan = map_output_paths(request(&sources)).unwrap(); - assert_eq!(plan.common_source_dir, Path::new("/project/src")); + assert_eq!(plan.common_source_dir, abs("/project/src").as_path()); assert_eq!( paths(&plan), vec![ - (PathBuf::from("/project/out/a.js"), ArtifactKind::JavaScript), - ( - PathBuf::from("/project/out/deep/b.js"), - ArtifactKind::JavaScript, - ), + (abs("/project/out/a.js"), ArtifactKind::JavaScript), + (abs("/project/out/deep/b.js"), ArtifactKind::JavaScript,), ] ); } @@ -578,7 +582,7 @@ mod tests { assert!( preserve .artifacts - .contains_key(Path::new("/project/out/b.jsx")) + .contains_key(abs("/project/out/b.jsx").as_path()) ); } @@ -637,17 +641,17 @@ mod tests { assert!( routed .artifacts - .contains_key(Path::new("/project/out/a.mjs")) + .contains_key(abs("/project/out/a.mjs").as_path()) ); assert!( routed .artifacts - .contains_key(Path::new("/project/types/a.d.mts")) + .contains_key(abs("/project/types/a.d.mts").as_path()) ); assert!( routed .artifacts - .contains_key(Path::new("/project/types/a.d.mts.map")) + .contains_key(abs("/project/types/a.d.mts.map").as_path()) ); } @@ -672,8 +676,8 @@ mod tests { assert_eq!( map_output_paths(request(&sources)), Err(OutputMapError::ClobbersInput { - path: PathBuf::from("/project/out/src/a.js"), - source: PathBuf::from("/project/out/src/a.js"), + path: abs("/project/out/src/a.js"), + source: abs("/project/out/src/a.js"), }) ); } diff --git a/crates/bamts-compiler/src/project/references.rs b/crates/bamts-compiler/src/project/references.rs index fc6c7435..e1b53933 100644 --- a/crates/bamts-compiler/src/project/references.rs +++ b/crates/bamts-compiler/src/project/references.rs @@ -683,19 +683,20 @@ fn feed_bytes(canonical: &mut Vec, bytes: &[u8]) { #[cfg(test)] mod tests { use super::*; + use crate::project::test_paths::abs; fn test_root() -> ProjectRoot { - ProjectRoot::new("/workspace/test").expect("absolute test root") + ProjectRoot::new(abs("/workspace/test")).expect("absolute test root") } - fn parse(root: &ProjectRoot, path: &str, source: &str) -> TsConfig { + fn parse(root: &ProjectRoot, path: impl AsRef, source: &str) -> TsConfig { TsConfig::parse(root, path, source).expect("valid test tsconfig") } #[test] fn empty_references_parses_to_empty_node() { let root = test_root(); - let config = parse(&root, "/workspace/test/app/tsconfig.json", r#"{}"#); + let config = parse(&root, abs("/workspace/test/app/tsconfig.json"), r#"{}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&config]).unwrap(); let node = graph.node(config.config().path()).expect("node exists"); @@ -709,10 +710,10 @@ mod tests { let root = test_root(); let app = parse( &root, - "/workspace/test/app/tsconfig.json", + abs("/workspace/test/app/tsconfig.json"), r#"{"references": [{"path": "../lib"}]}"#, ); - let lib = parse(&root, "/workspace/test/lib/tsconfig.json", r#"{}"#); + let lib = parse(&root, abs("/workspace/test/lib/tsconfig.json"), r#"{}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&app, &lib]).unwrap(); let node = graph.node(app.config().path()).unwrap(); @@ -720,7 +721,7 @@ mod tests { assert_eq!(node.references.len(), 1); assert_eq!( node.references[0].path(), - Path::new("/workspace/test/lib/tsconfig.json") + abs("/workspace/test/lib/tsconfig.json").as_path() ); assert_eq!(node.references[0].original_path().unwrap(), "../lib"); assert_eq!(node.references[0].circular(), None); @@ -732,7 +733,7 @@ mod tests { let root = test_root(); let app = parse( &root, - "/workspace/test/app/tsconfig.json", + abs("/workspace/test/app/tsconfig.json"), r#"{"references": [{"path": "../lib/custom.json"}]}"#, ); @@ -741,7 +742,7 @@ mod tests { assert_eq!( node.references[0].path(), - Path::new("/workspace/test/lib/custom.json") + abs("/workspace/test/lib/custom.json").as_path() ); assert!(is_json_path(node.references[0].path())); } @@ -751,10 +752,10 @@ mod tests { let root = test_root(); let app = parse( &root, - "/workspace/test/app/tsconfig.json", + abs("/workspace/test/app/tsconfig.json"), r#"{"references": [{"path": "../lib"}]}"#, ); - let lib = parse(&root, "/workspace/test/lib/tsconfig.json", r#"{}"#); + let lib = parse(&root, abs("/workspace/test/lib/tsconfig.json"), r#"{}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&app, &lib]).unwrap(); let order = graph.topological_order().unwrap(); @@ -762,8 +763,8 @@ mod tests { assert_eq!( order, vec![ - PathBuf::from("/workspace/test/lib/tsconfig.json"), - PathBuf::from("/workspace/test/app/tsconfig.json"), + abs("/workspace/test/lib/tsconfig.json"), + abs("/workspace/test/app/tsconfig.json"), ] ); } @@ -773,28 +774,25 @@ mod tests { let root = test_root(); let app = parse( &root, - "/workspace/test/app/tsconfig.json", + abs("/workspace/test/app/tsconfig.json"), r#"{"references": [{"path": "../left"}, {"path": "../right"}]}"#, ); let left = parse( &root, - "/workspace/test/left/tsconfig.json", + abs("/workspace/test/left/tsconfig.json"), r#"{"references": [{"path": "../shared"}]}"#, ); let right = parse( &root, - "/workspace/test/right/tsconfig.json", + abs("/workspace/test/right/tsconfig.json"), r#"{"references": [{"path": "../shared"}]}"#, ); - let shared = parse(&root, "/workspace/test/shared/tsconfig.json", r#"{}"#); + let shared = parse(&root, abs("/workspace/test/shared/tsconfig.json"), r#"{}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&app, &left, &right, &shared]).unwrap(); let order = graph.topological_order().unwrap(); - assert_eq!( - order[0], - PathBuf::from("/workspace/test/shared/tsconfig.json") - ); + assert_eq!(order[0], abs("/workspace/test/shared/tsconfig.json")); assert!( order .iter() @@ -836,12 +834,12 @@ mod tests { let root = test_root(); let a = parse( &root, - "/workspace/test/a/tsconfig.json", + abs("/workspace/test/a/tsconfig.json"), r#"{"references": [{"path": "../b"}]}"#, ); let b = parse( &root, - "/workspace/test/b/tsconfig.json", + abs("/workspace/test/b/tsconfig.json"), r#"{"references": [{"path": "../a"}]}"#, ); @@ -860,7 +858,7 @@ mod tests { let root = test_root(); let config = parse( &root, - "/workspace/test/p/tsconfig.json", + abs("/workspace/test/p/tsconfig.json"), r#"{"compilerOptions": {"composite": true, "declaration": false, "incremental": false}}"#, ); @@ -884,13 +882,13 @@ mod tests { let root = test_root(); let parent = parse( &root, - "/workspace/test/parent/tsconfig.json", + abs("/workspace/test/parent/tsconfig.json"), r#"{ "files": ["a.ts"], "references": [{"path": "../child"}] }"#, ); - let child = parse(&root, "/workspace/test/child/tsconfig.json", r#"{}"#); + let child = parse(&root, abs("/workspace/test/child/tsconfig.json"), r#"{}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&parent, &child]).unwrap(); let errors = graph.validate(); @@ -907,7 +905,7 @@ mod tests { let root = test_root(); let parent = parse( &root, - "/workspace/test/parent/tsconfig.json", + abs("/workspace/test/parent/tsconfig.json"), r#"{ "files": ["a.ts"], "compilerOptions": {"composite": true, "declaration": true}, @@ -916,7 +914,7 @@ mod tests { ); let child = parse( &root, - "/workspace/test/child/tsconfig.json", + abs("/workspace/test/child/tsconfig.json"), r#"{"compilerOptions": {"composite": true, "noEmit": true}}"#, ); @@ -935,7 +933,7 @@ mod tests { let root = test_root(); let config = parse( &root, - "/workspace/test/p/tsconfig.json", + abs("/workspace/test/p/tsconfig.json"), r#"{"compilerOptions": {"incremental": true}}"#, ); @@ -944,7 +942,7 @@ mod tests { assert_eq!( node.build_info_path.as_deref(), - Some(Path::new("/workspace/test/p/tsconfig.tsbuildinfo")) + Some(abs("/workspace/test/p/tsconfig.tsbuildinfo").as_path()) ); } @@ -953,7 +951,7 @@ mod tests { let root = test_root(); let config = parse( &root, - "/workspace/test/src/p/tsconfig.json", + abs("/workspace/test/src/p/tsconfig.json"), r#"{ "compilerOptions": { "composite": true, @@ -969,7 +967,7 @@ mod tests { assert_eq!( node.build_info_path.as_deref(), - Some(Path::new("/workspace/test/out/p/tsconfig.tsbuildinfo")) + Some(abs("/workspace/test/out/p/tsconfig.tsbuildinfo").as_path()) ); } @@ -978,7 +976,7 @@ mod tests { let root = test_root(); let config = parse( &root, - "/workspace/test/p/tsconfig.json", + abs("/workspace/test/p/tsconfig.json"), r#"{ "compilerOptions": { "incremental": true, @@ -992,7 +990,7 @@ mod tests { assert_eq!( node.build_info_path.as_deref(), - Some(Path::new("/workspace/test/cache/info.json")) + Some(abs("/workspace/test/cache/info.json").as_path()) ); } @@ -1001,7 +999,7 @@ mod tests { let root = test_root(); let parent = parse( &root, - "/workspace/test/parent/tsconfig.json", + abs("/workspace/test/parent/tsconfig.json"), r#"{ "files": ["a.ts"], "compilerOptions": {"composite": true, "declaration": true, "outDir": "../out"}, @@ -1010,7 +1008,7 @@ mod tests { ); let child = parse( &root, - "/workspace/test/child/tsconfig.json", + abs("/workspace/test/child/tsconfig.json"), r#"{ "compilerOptions": {"composite": true, "declaration": true, "outDir": "../out"} }"#, @@ -1031,7 +1029,7 @@ mod tests { let root = test_root(); let config = parse( &root, - "/workspace/test/p/tsconfig.json", + abs("/workspace/test/p/tsconfig.json"), r#"{ "files": ["a.ts"], "compilerOptions": {"incremental": true} @@ -1048,7 +1046,7 @@ mod tests { let different = parse( &root, - "/workspace/test/p/tsconfig.json", + abs("/workspace/test/p/tsconfig.json"), r#"{ "files": ["a.ts", "b.ts"], "compilerOptions": {"incremental": true} @@ -1066,7 +1064,7 @@ mod tests { let root = test_root(); let config = parse( &root, - "/workspace/test/p/tsconfig.json", + abs("/workspace/test/p/tsconfig.json"), r#"{"compilerOptions": {"incremental": true}}"#, ); @@ -1081,7 +1079,7 @@ mod tests { #[test] fn non_incremental_project_has_no_build_info_identity() { let root = test_root(); - let config = parse(&root, "/workspace/test/p/tsconfig.json", r#"{}"#); + let config = parse(&root, abs("/workspace/test/p/tsconfig.json"), r#"{}"#); let graph = ReferenceGraph::from_tsconfigs(&root, &[&config]).unwrap(); assert!(graph.tsbuildinfo_identity(config.config().path()).is_none()); @@ -1092,7 +1090,7 @@ mod tests { let root = test_root(); let config = parse( &root, - "/workspace/test/p/tsconfig.json", + abs("/workspace/test/p/tsconfig.json"), r#"{"references": [{"path": "../lib", "circular": true}]}"#, ); diff --git a/crates/bamts-compiler/src/project/resolution.rs b/crates/bamts-compiler/src/project/resolution.rs index 0a63d1ec..ee349c41 100644 --- a/crates/bamts-compiler/src/project/resolution.rs +++ b/crates/bamts-compiler/src/project/resolution.rs @@ -1264,6 +1264,8 @@ fn conditions_for( #[cfg(test)] mod tests { use super::*; + use crate::project::canonical::slash_text; + use crate::project::test_paths::abs; use crate::project::{JsonObject, JsonValue, PathMapping, ProjectConfig}; use std::collections::BTreeMap; @@ -1274,8 +1276,8 @@ mod tests { } impl MemoryHost { - fn file(&mut self, path: &str, source: &str) { - let path = PathBuf::from(path); + fn file(&mut self, path: impl AsRef, source: &str) { + let path = path.as_ref().to_path_buf(); if let Some(parent) = path.parent() { self.mark_dirs(parent); } @@ -1306,13 +1308,21 @@ mod tests { } fn root() -> ProjectRoot { - ProjectRoot::new("/workspace").expect("absolute root") + ProjectRoot::new(abs("/workspace")).expect("absolute root") + } + + /// How a trace line spells the root of the fixture file system: c: on Windows, + /// where upstream traces print a lowercase drive, and nothing elsewhere. + fn trace_drive() -> String { + slash_text(&abs("/")) + .trim_end_matches('/') + .to_ascii_lowercase() } fn options_with_resolution(module_resolution: &str) -> CompilerOptions { let config = ProjectConfig::parse( &root(), - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), &format!( r#"{{"compilerOptions":{{"moduleResolution":"{module_resolution}","resolveJsonModule":true}}}}"# ), @@ -1324,7 +1334,7 @@ mod tests { fn options_with_paths(paths_json: &str) -> CompilerOptions { let config = ProjectConfig::parse( &root(), - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), &format!( r#"{{"compilerOptions":{{"moduleResolution":"bundler","baseUrl":".","paths":{paths_json}}}}}"# ), @@ -1364,22 +1374,22 @@ mod tests { #[test] fn relative_extension_substitution_and_index_fallback() { let mut host = MemoryHost::default(); - host.file("/workspace/src/util.ts", "export const n = 1;\n"); - host.file("/workspace/src/nested/index.ts", "export {};\n"); + host.file(abs("/workspace/src/util.ts"), "export const n = 1;\n"); + host.file(abs("/workspace/src/nested/index.ts"), "export {};\n"); let options = options_with_resolution("node16"); let mut cache = ResolutionCache::new(); let resolved = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./util.js", (ModuleResolutionKind::Node16, ResolutionMode::Import), &host, &mut cache, ) .expect("util.js → util.ts"); - assert_eq!(resolved.path(), Path::new("/workspace/src/util.ts")); + assert_eq!(resolved.path(), abs("/workspace/src/util.ts").as_path()); assert!( resolved .trace() @@ -1391,25 +1401,28 @@ mod tests { let nested = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./nested", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, &mut cache, ) .expect("directory index"); - assert_eq!(nested.path(), Path::new("/workspace/src/nested/index.ts")); + assert_eq!( + nested.path(), + abs("/workspace/src/nested/index.ts").as_path() + ); } #[test] fn nodenext_import_requires_relative_extension_but_require_and_bundler_do_not() { let mut host = MemoryHost::default(); - host.file("/workspace/src/util.ts", "export const n = 1;\n"); + host.file(abs("/workspace/src/util.ts"), "export const n = 1;\n"); let options = options_with_resolution("nodenext"); let mut cache = ResolutionCache::new(); let missing = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./util", (ModuleResolutionKind::NodeNext, ResolutionMode::Import), &host, @@ -1423,34 +1436,34 @@ mod tests { let resolved = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./util", strategy, &host, &mut cache, ) .expect("extensionless resolution is permitted"); - assert_eq!(resolved.path(), Path::new("/workspace/src/util.ts")); + assert_eq!(resolved.path(), abs("/workspace/src/util.ts").as_path()); } } #[test] fn paths_longest_prefix_wins_with_star_substitution() { let mut host = MemoryHost::default(); - host.file("/workspace/lib/foo/bar.ts", "export {};\n"); + host.file(abs("/workspace/lib/foo/bar.ts"), "export {};\n"); let options = options_with_paths(r#"{ "@app/*": ["lib/*"], "@app/foo/*": ["lib/foo/*"] }"#); let mut cache = ResolutionCache::new(); let resolved = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "@app/foo/bar", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, &mut cache, ) .expect("paths"); - assert_eq!(resolved.path(), Path::new("/workspace/lib/foo/bar.ts")); + assert_eq!(resolved.path(), abs("/workspace/lib/foo/bar.ts").as_path()); assert!(resolved.trace().steps().iter().any(|step| matches!( step, ResolutionTraceStep::PathsMatch { pattern, .. } if pattern.as_ref() == "@app/foo/*" @@ -1461,7 +1474,7 @@ mod tests { fn package_exports_honor_resolution_mode_conditions() { let mut host = MemoryHost::default(); host.file( - "/workspace/node_modules/pkg/package.json", + abs("/workspace/node_modules/pkg/package.json"), r#"{ "name":"pkg", "exports":{ @@ -1472,14 +1485,14 @@ mod tests { } }"#, ); - host.file("/workspace/node_modules/pkg/esm.js", "export {};\n"); + host.file(abs("/workspace/node_modules/pkg/esm.js"), "export {};\n"); host.file( - "/workspace/node_modules/pkg/cjs.js", + abs("/workspace/node_modules/pkg/cjs.js"), "module.exports = {};\n", ); // Provide TS stand-ins so extension substitution can succeed for both modes. - host.file("/workspace/node_modules/pkg/esm.ts", "export {};\n"); - host.file("/workspace/node_modules/pkg/cjs.ts", "export {};\n"); + host.file(abs("/workspace/node_modules/pkg/esm.ts"), "export {};\n"); + host.file(abs("/workspace/node_modules/pkg/cjs.ts"), "export {};\n"); let options = options_with_resolution("nodenext"); let mut cache = ResolutionCache::new(); @@ -1487,7 +1500,7 @@ mod tests { let import_hit = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "pkg", (ModuleResolutionKind::NodeNext, ResolutionMode::Import), &host, @@ -1496,13 +1509,13 @@ mod tests { .expect("import condition"); assert_eq!( import_hit.path(), - Path::new("/workspace/node_modules/pkg/esm.ts") + abs("/workspace/node_modules/pkg/esm.ts").as_path() ); let require_hit = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "pkg", (ModuleResolutionKind::NodeNext, ResolutionMode::Require), &host, @@ -1511,25 +1524,31 @@ mod tests { .expect("require condition"); assert_eq!( require_hit.path(), - Path::new("/workspace/node_modules/pkg/cjs.ts") + abs("/workspace/node_modules/pkg/cjs.ts").as_path() ); } #[test] fn package_exports_block_private_and_null_subpaths() { let mut host = MemoryHost::default(); host.file( - "/workspace/node_modules/pkg/package.json", + abs("/workspace/node_modules/pkg/package.json"), r#"{"exports":{".":"./index.js","./blocked":null}}"#, ); - host.file("/workspace/node_modules/pkg/index.ts", "export {};\n"); - host.file("/workspace/node_modules/pkg/private.ts", "export {};\n"); - host.file("/workspace/node_modules/pkg/blocked.ts", "export {};\n"); + host.file(abs("/workspace/node_modules/pkg/index.ts"), "export {};\n"); + host.file( + abs("/workspace/node_modules/pkg/private.ts"), + "export {};\n", + ); + host.file( + abs("/workspace/node_modules/pkg/blocked.ts"), + "export {};\n", + ); let options = options_with_resolution("nodenext"); for specifier in ["pkg/private", "pkg/blocked"] { let error = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), specifier, (ModuleResolutionKind::NodeNext, ResolutionMode::Import), &host, @@ -1548,7 +1567,7 @@ mod tests { fn package_imports_resolve_and_detect_cycles() { let mut host = MemoryHost::default(); host.file( - "/workspace/package.json", + abs("/workspace/package.json"), r##"{ "name":"app", "imports":{ @@ -1557,27 +1576,27 @@ mod tests { } }"##, ); - host.file("/workspace/src/value.ts", "export const v = 1;\n"); - host.file("/workspace/src/main.ts", "import '#internal/value';\n"); + host.file(abs("/workspace/src/value.ts"), "export const v = 1;\n"); + host.file(abs("/workspace/src/main.ts"), "import '#internal/value';\n"); let options = options_with_resolution("bundler"); let mut cache = ResolutionCache::new(); let resolved = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "#internal/value", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, &mut cache, ) .expect("hash import"); - assert_eq!(resolved.path(), Path::new("/workspace/src/value.ts")); + assert_eq!(resolved.path(), abs("/workspace/src/value.ts").as_path()); let cycle = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "#loop", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, @@ -1591,12 +1610,15 @@ mod tests { fn node_modules_walk_stops_at_project_root_and_confinement_rejects_escape() { let mut host = MemoryHost::default(); host.file( - "/workspace/node_modules/left-pad/package.json", + abs("/workspace/node_modules/left-pad/package.json"), r#"{"name":"left-pad","main":"./index.js"}"#, ); - host.file("/workspace/node_modules/left-pad/index.ts", "export {};\n"); host.file( - "/workspace/node_modules/left-pad/index.js", + abs("/workspace/node_modules/left-pad/index.ts"), + "export {};\n", + ); + host.file( + abs("/workspace/node_modules/left-pad/index.js"), "module.exports = {};\n", ); @@ -1605,7 +1627,7 @@ mod tests { let resolved = resolve_module_name( &root(), &options, - "/workspace/packages/app/src/main.ts", + abs("/workspace/packages/app/src/main.ts"), "left-pad", (ModuleResolutionKind::Node16, ResolutionMode::Require), &host, @@ -1614,7 +1636,7 @@ mod tests { .expect("upward walk"); assert_eq!( resolved.path(), - Path::new("/workspace/node_modules/left-pad/index.ts") + abs("/workspace/node_modules/left-pad/index.ts").as_path() ); assert!( resolved @@ -1627,7 +1649,7 @@ mod tests { let escape = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "../../outside", (ModuleResolutionKind::Node16, ResolutionMode::Import), &host, @@ -1640,7 +1662,7 @@ mod tests { fn types_fallback_mangles_scoped_package_names() { let mut host = MemoryHost::default(); host.file( - "/workspace/node_modules/@types/scope__pkg/index.d.ts", + abs("/workspace/node_modules/@types/scope__pkg/index.d.ts"), "export {};\n", ); let options = options_with_resolution("nodenext"); @@ -1648,7 +1670,7 @@ mod tests { let resolved = resolve_type_reference( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "@scope/pkg", (ModuleResolutionKind::NodeNext, ResolutionMode::Import), &host, @@ -1657,7 +1679,7 @@ mod tests { .expect("@types scope mangling"); assert_eq!( resolved.path(), - Path::new("/workspace/node_modules/@types/scope__pkg/index.d.ts") + abs("/workspace/node_modules/@types/scope__pkg/index.d.ts").as_path() ); assert!(resolved.trace().steps().iter().any(|step| matches!( step, @@ -1669,13 +1691,13 @@ mod tests { #[test] fn cache_returns_identical_results_for_repeated_lookups() { let mut host = MemoryHost::default(); - host.file("/workspace/src/a.ts", "export {};\n"); + host.file(abs("/workspace/src/a.ts"), "export {};\n"); let options = options_with_resolution("bundler"); let mut cache = ResolutionCache::new(); let first = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./a", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, @@ -1685,7 +1707,7 @@ mod tests { let second = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./a", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, @@ -1698,7 +1720,7 @@ mod tests { let missing = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./missing", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, @@ -1708,7 +1730,7 @@ mod tests { let missing_again = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./missing", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, @@ -1725,18 +1747,18 @@ mod tests { fn trace_is_deterministic_across_two_fresh_caches() { let mut host = MemoryHost::default(); host.file( - "/workspace/node_modules/demo/package.json", + abs("/workspace/node_modules/demo/package.json"), r#"{"name":"demo","exports":{".":{"import":"./lib.js"}}}"#, ); - host.file("/workspace/node_modules/demo/lib.ts", "export {};\n"); - host.file("/workspace/node_modules/demo/lib.js", "export {};\n"); + host.file(abs("/workspace/node_modules/demo/lib.ts"), "export {};\n"); + host.file(abs("/workspace/node_modules/demo/lib.js"), "export {};\n"); let options = options_with_resolution("node16"); let mut cache_a = ResolutionCache::new(); let a = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "demo", (ModuleResolutionKind::Node16, ResolutionMode::Import), &host, @@ -1747,7 +1769,7 @@ mod tests { let b = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "demo", (ModuleResolutionKind::Node16, ResolutionMode::Import), &host, @@ -1802,17 +1824,17 @@ mod tests { "======== Module name './dir/index' was successfully resolved to '/dir/index.ts'. ========", ]; - let fs_root = ProjectRoot::new("/").expect("absolute root"); + let fs_root = ProjectRoot::new(abs("/")).expect("absolute root"); let config = ProjectConfig::parse( &fs_root, - "/tsconfig.json", + abs("/tsconfig.json"), r#"{"compilerOptions":{"moduleResolution":"bundler"}}"#, ) .expect("config"); let options = config.options().clone(); let mut host = MemoryHost::default(); - host.file("/main.ts", "import {} from './dir/index.js';\n"); - host.file("/dir/index.ts", "export {};\n"); + host.file(abs("/main.ts"), "import {} from './dir/index.js';\n"); + host.file(abs("/dir/index.ts"), "export {};\n"); let mut cache = ResolutionCache::new(); let mut log = ResolutionTraceLog::default(); @@ -1820,7 +1842,7 @@ mod tests { resolve_module_name_with_trace( &fs_root, &options, - "/main.ts", + abs("/main.ts"), specifier, ( ModuleResolutionKind::Bundler, @@ -1838,7 +1860,7 @@ mod tests { resolve_module_name_with_trace( &fs_root, &options, - "/main.ts", + abs("/main.ts"), "./dir/index", ( ModuleResolutionKind::Bundler, @@ -1853,22 +1875,27 @@ mod tests { ) .expect("cache hit"); + let drive = trace_drive(); + let expected: Vec = expected + .iter() + .map(|line| line.replace("'/", &format!("'{drive}/"))) + .collect(); let produced: Vec<&str> = log.lines().iter().map(String::as_str).collect(); assert_eq!(produced, expected); assert!(log.unsupported().is_empty()); } #[test] fn trace_mode_word_follows_resolution_mode() { - let fs_root = ProjectRoot::new("/").expect("absolute root"); + let fs_root = ProjectRoot::new(abs("/")).expect("absolute root"); let config = ProjectConfig::parse( &fs_root, - "/tsconfig.json", + abs("/tsconfig.json"), r#"{"compilerOptions":{"moduleResolution":"bundler"}}"#, ) .expect("config"); let options = config.options().clone(); let mut host = MemoryHost::default(); - host.file("/dir/index.ts", "export {};\n"); + host.file(abs("/dir/index.ts"), "export {};\n"); for (mode, word) in [ (ResolutionMode::Import, "ESM"), @@ -1879,7 +1906,7 @@ mod tests { resolve_module_name_with_trace( &fs_root, &options, - "/main.ts", + abs("/main.ts"), "./dir/index.ts", ( ModuleResolutionKind::Bundler, @@ -1907,17 +1934,17 @@ mod tests { #[test] fn trace_types_flavor_narrates_declaration_candidates_first() { - let fs_root = ProjectRoot::new("/").expect("absolute root"); + let fs_root = ProjectRoot::new(abs("/")).expect("absolute root"); let config = ProjectConfig::parse( &fs_root, - "/tsconfig.json", + abs("/tsconfig.json"), r#"{"compilerOptions":{"moduleResolution":"bundler"}}"#, ) .expect("config"); let options = config.options().clone(); let mut host = MemoryHost::default(); - host.file("/dir/index.d.ts", "export {};\n"); - host.file("/dir/index.ts", "export {};\n"); + host.file(abs("/dir/index.d.ts"), "export {};\n"); + host.file(abs("/dir/index.ts"), "export {};\n"); for (flavor, chosen, narrated) in [ (ResolutionFlavor::Runtime, "index.ts", "dir/index.ts"), @@ -1928,7 +1955,7 @@ mod tests { let resolved = resolve_module_name_with_trace( &fs_root, &options, - "/main.ts", + abs("/main.ts"), "./dir/index", ( ModuleResolutionKind::Bundler, @@ -1942,7 +1969,7 @@ mod tests { }, ) .expect("resolution"); - assert_eq!(resolved.path(), Path::new(&format!("/dir/{chosen}"))); + assert_eq!(resolved.path(), abs(&format!("/dir/{chosen}")).as_path()); // Narration stops at the first probed candidate, so the single // narrated file must be the one the resolver actually chose. let narrated_files: Vec = log @@ -1951,24 +1978,26 @@ mod tests { .filter(|line| line.starts_with("File '")) .cloned() .collect(); - let expected = - format!("File '/{narrated}' exists - use it as a name resolution result."); + let expected = format!( + "File '{}/{narrated}' exists - use it as a name resolution result.", + trace_drive() + ); assert_eq!(narrated_files, vec![expected]); } } #[test] fn cache_keys_distinguish_runtime_and_types_flavors() { - let fs_root = ProjectRoot::new("/").expect("absolute root"); + let fs_root = ProjectRoot::new(abs("/")).expect("absolute root"); let mut host = MemoryHost::default(); - host.file("/dir/index.d.ts", "export {};\n"); - host.file("/dir/index.ts", "export {};\n"); + host.file(abs("/dir/index.d.ts"), "export {};\n"); + host.file(abs("/dir/index.ts"), "export {};\n"); let options = options_with_resolution("bundler"); let mut cache = ResolutionCache::new(); let runtime = resolve_module_name( &fs_root, &options, - "/main.ts", + abs("/main.ts"), "./dir/index", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, @@ -1978,29 +2007,29 @@ mod tests { let types = resolve_type_reference( &fs_root, &options, - "/main.ts", + abs("/main.ts"), "./dir/index", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, &mut cache, ) .expect("types flavor"); - assert_eq!(runtime.path(), Path::new("/dir/index.ts")); - assert_eq!(types.path(), Path::new("/dir/index.d.ts")); + assert_eq!(runtime.path(), abs("/dir/index.ts").as_path()); + assert_eq!(types.path(), abs("/dir/index.d.ts").as_path()); assert_eq!(cache.len(), 2); } #[test] fn json_specifier_resolves_the_exact_file_when_resolve_json_module_is_on() { let mut host = MemoryHost::default(); - host.file("/workspace/src/data.json", "{\n \"a\": 1\n}\n"); + host.file(abs("/workspace/src/data.json"), "{\n \"a\": 1\n}\n"); // A script module beside the JSON file must not win: a `.json` // specifier is planned as an exact file with no extension // substitution. - host.file("/workspace/src/data.ts", "export const a = 1;\n"); + host.file(abs("/workspace/src/data.ts"), "export const a = 1;\n"); let config = ProjectConfig::parse( &root(), - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions":{"moduleResolution":"bundler","resolveJsonModule":true}}"#, ) .expect("config"); @@ -2009,14 +2038,14 @@ mod tests { let resolved = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./data.json", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, &mut cache, ) .expect("exact JSON module"); - assert_eq!(resolved.path(), Path::new("/workspace/src/data.json")); + assert_eq!(resolved.path(), abs("/workspace/src/data.json").as_path()); assert!( resolved.trace().steps().iter().all(|step| !matches!( step, @@ -2030,10 +2059,10 @@ mod tests { #[test] fn json_specifier_probes_the_exact_file_even_when_resolve_json_module_is_off() { let mut host = MemoryHost::default(); - host.file("/workspace/src/data.json", "{\n \"a\": 1\n}\n"); + host.file(abs("/workspace/src/data.json"), "{\n \"a\": 1\n}\n"); let config = ProjectConfig::parse( &root(), - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions":{"moduleResolution":"bundler"}}"#, ) .expect("config"); @@ -2046,14 +2075,14 @@ mod tests { let resolved = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./data.json", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, &mut cache, ) .expect("exact JSON module resolves without the flag"); - assert_eq!(resolved.path(), Path::new("/workspace/src/data.json")); + assert_eq!(resolved.path(), abs("/workspace/src/data.json").as_path()); // A `.json` specifier that names no file fails as a plain cannot-find // miss (upstream TS2732) — never as an unsupported-extension @@ -2061,7 +2090,7 @@ mod tests { let error = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "./missing.json", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, @@ -2074,10 +2103,13 @@ mod tests { #[test] fn json_specifier_with_paths_mapping_resolves_through_the_mapping() { let mut host = MemoryHost::default(); - host.file("/workspace/config/pkg.json", "{\n \"name\": \"pkg\"\n}\n"); + host.file( + abs("/workspace/config/pkg.json"), + "{\n \"name\": \"pkg\"\n}\n", + ); let config = ProjectConfig::parse( &root(), - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions":{"moduleResolution":"bundler","baseUrl":".","paths":{"@config/*":["config/*"]},"resolveJsonModule":true}}"#, ) .expect("config"); @@ -2086,14 +2118,14 @@ mod tests { let resolved = resolve_module_name( &root(), &options, - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "@config/pkg.json", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, &mut cache, ) .expect("paths-mapped JSON module"); - assert_eq!(resolved.path(), Path::new("/workspace/config/pkg.json")); + assert_eq!(resolved.path(), abs("/workspace/config/pkg.json").as_path()); // The searched routes stay flag-gated: with `resolveJsonModule` // disabled a paths-mapped `.json` cannot be found with the current @@ -2102,7 +2134,7 @@ mod tests { // though the mapped target file exists. let without_flag = ProjectConfig::parse( &root(), - "/workspace/tsconfig.json", + abs("/workspace/tsconfig.json"), r#"{"compilerOptions":{"moduleResolution":"bundler","baseUrl":".","paths":{"@config/*":["config/*"]}}}"#, ) .expect("config"); @@ -2112,7 +2144,7 @@ mod tests { let error = resolve_module_name( &root(), without_flag.options(), - "/workspace/src/main.ts", + abs("/workspace/src/main.ts"), "@config/pkg.json", (ModuleResolutionKind::Bundler, ResolutionMode::Import), &host, diff --git a/crates/bamts-compiler/src/project/resolution_trace.rs b/crates/bamts-compiler/src/project/resolution_trace.rs index 56d9d0c8..24ab594a 100644 --- a/crates/bamts-compiler/src/project/resolution_trace.rs +++ b/crates/bamts-compiler/src/project/resolution_trace.rs @@ -210,7 +210,11 @@ fn upstream_candidates( } /// Renders a path the way upstream traces do: forward slashes, lowercased drive. -fn display_path(path: &Path) -> String { +/// +/// Anything that looks for a path inside a rendered trace must build the text +/// with this function. +#[must_use] +pub fn display_path(path: &Path) -> String { let mut value = path.to_string_lossy().replace('\\', "/"); if value.as_bytes().get(1) == Some(&b':') { value.make_ascii_lowercase(); diff --git a/crates/bamts-compiler/src/project/test_paths.rs b/crates/bamts-compiler/src/project/test_paths.rs new file mode 100644 index 00000000..4dba9933 --- /dev/null +++ b/crates/bamts-compiler/src/project/test_paths.rs @@ -0,0 +1,53 @@ +//! Absolute paths for tests. +//! +//! Fixtures name absolute paths the Unix way (`/workspace/a.ts`). A path without +//! a drive is not absolute on Windows, so [`abs`] puts such a path on the `C:` +//! drive there. Expected values are built with the same helpers, never written +//! out, so a test says the same thing on every platform. + +use std::path::PathBuf; + +/// The platform's absolute spelling of a fixture path written with `/` separators. +#[cfg(not(windows))] +pub(crate) fn abs(unix: &str) -> PathBuf { + PathBuf::from(unix) +} + +/// The platform's absolute spelling of a fixture path written with `/` separators. +#[cfg(windows)] +pub(crate) fn abs(unix: &str) -> PathBuf { + assert!(unix.starts_with('/'), "fixture path {unix} is not rooted"); + PathBuf::from(format!("C:{}", unix.replace('/', "\\"))) +} + +/// [`abs`] as text, for expected messages. +pub(crate) fn abs_text(unix: &str) -> String { + abs(unix).to_string_lossy().into_owned() +} + +/// [`abs`] as the content of a JSON string literal: backslashes are escaped. +pub(crate) fn abs_json(unix: &str) -> String { + abs_text(unix).replace('\\', "\\\\") +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + #[cfg(not(windows))] + fn fixture_paths_are_unchanged_where_they_are_already_absolute() { + assert_eq!(abs("/workspace/a.ts"), PathBuf::from("/workspace/a.ts")); + assert_eq!(abs_json("/workspace/a.ts"), "/workspace/a.ts"); + } + + #[test] + #[cfg(windows)] + fn fixture_paths_move_onto_the_system_drive() { + assert_eq!(abs("/"), PathBuf::from(r"C:\")); + assert_eq!(abs("/workspace/a.ts"), PathBuf::from(r"C:\workspace\a.ts")); + assert!(abs("/workspace").is_absolute()); + assert_eq!(abs_text("/workspace/a.ts"), r"C:\workspace\a.ts"); + assert_eq!(abs_json("/workspace/a.ts"), r"C:\\workspace\\a.ts"); + } +} diff --git a/crates/bamts-compiler/src/project/tsconfig.rs b/crates/bamts-compiler/src/project/tsconfig.rs index 2f51a978..8e970379 100644 --- a/crates/bamts-compiler/src/project/tsconfig.rs +++ b/crates/bamts-compiler/src/project/tsconfig.rs @@ -609,9 +609,10 @@ fn utf16_offset(source: &str, byte: usize) -> usize { #[cfg(test)] mod tests { use super::*; + use crate::project::test_paths::abs; fn root() -> ProjectRoot { - ProjectRoot::new("/workspace").unwrap() + ProjectRoot::new(abs("/workspace")).unwrap() } #[test] @@ -663,7 +664,7 @@ mod tests { assert_eq!(cfg.references()[0].original_path(), Some("packages/foo")); assert_eq!( cfg.references()[0].path(), - Path::new("/workspace/packages/foo") + abs("/workspace/packages/foo").as_path() ); assert_eq!(cfg.references()[1].original_path(), Some("packages/bar")); assert_eq!(cfg.references()[1].prepend(), Some(true)); @@ -695,7 +696,7 @@ mod tests { fn relative_extends() { let source = r#"{"extends": "./base.json"}"#; let loader = |p: &Path| { - if p == Path::new("/workspace/base.json") { + if p == abs("/workspace/base.json").as_path() { Some(Arc::from(r#"{"compilerOptions": {"strict": true}}"#)) } else { None @@ -703,7 +704,7 @@ mod tests { }; let chain = resolve_extends(&root(), "tsconfig.json", source, &loader, 8).unwrap(); assert_eq!(chain.len(), 1); - assert_eq!(chain[0].path(), Path::new("/workspace/base.json")); + assert_eq!(chain[0].path(), abs("/workspace/base.json").as_path()); assert_eq!(chain[0].specifier(), "./base.json"); } @@ -711,7 +712,7 @@ mod tests { fn package_extends() { let source = r#"{"extends": "@tsconfig/strict"}"#; let loader = |p: &Path| { - if p == Path::new("/workspace/node_modules/@tsconfig/strict/tsconfig.json") { + if p == abs("/workspace/node_modules/@tsconfig/strict/tsconfig.json").as_path() { Some(Arc::from(r#"{}"#)) } else { None @@ -721,7 +722,7 @@ mod tests { assert_eq!(chain.len(), 1); assert_eq!( chain[0].path(), - Path::new("/workspace/node_modules/@tsconfig/strict/tsconfig.json") + abs("/workspace/node_modules/@tsconfig/strict/tsconfig.json").as_path() ); } @@ -729,7 +730,7 @@ mod tests { fn package_extends_with_subpath() { let source = r#"{"extends": "@tsconfig/strict/tsconfig.json"}"#; let loader = |p: &Path| { - if p == Path::new("/workspace/node_modules/@tsconfig/strict/tsconfig.json") { + if p == abs("/workspace/node_modules/@tsconfig/strict/tsconfig.json").as_path() { Some(Arc::from(r#"{}"#)) } else { None @@ -739,23 +740,26 @@ mod tests { assert_eq!(chain.len(), 1); assert_eq!( chain[0].path(), - Path::new("/workspace/node_modules/@tsconfig/strict/tsconfig.json") + abs("/workspace/node_modules/@tsconfig/strict/tsconfig.json").as_path() ); } #[test] fn extends_arrays_preserve_order_and_walk_ancestor_node_modules() { let source = r#"{"extends":["./base.json","@tsconfig/strict"]}"#; - let loader = |path: &Path| match path.to_str() { - Some("/workspace/packages/app/base.json") => Some(Arc::from(r#"{}"#)), - Some("/workspace/node_modules/@tsconfig/strict/tsconfig.json") => { - Some(Arc::from(r#"{}"#)) - } - _ => None, + let loader = |path: &Path| { + let known = [ + abs("/workspace/packages/app/base.json"), + abs("/workspace/node_modules/@tsconfig/strict/tsconfig.json"), + ]; + known + .iter() + .any(|candidate| candidate == path) + .then(|| Arc::from(r#"{}"#)) }; let chain = resolve_extends( &root(), - "/workspace/packages/app/tsconfig.json", + abs("/workspace/packages/app/tsconfig.json"), source, &loader, 8, @@ -766,7 +770,7 @@ mod tests { assert_eq!(chain[1].specifier(), "@tsconfig/strict"); assert_eq!( chain[1].path(), - Path::new("/workspace/node_modules/@tsconfig/strict/tsconfig.json") + abs("/workspace/node_modules/@tsconfig/strict/tsconfig.json").as_path() ); } @@ -790,9 +794,9 @@ mod tests { fn extends_cycle() { let source = r#"{"extends": "./a.json"}"#; let loader = |p: &Path| { - if p == Path::new("/workspace/a.json") { + if p == abs("/workspace/a.json").as_path() { Some(Arc::from(r#"{"extends": "./b.json"}"#)) - } else if p == Path::new("/workspace/b.json") { + } else if p == abs("/workspace/b.json").as_path() { Some(Arc::from(r#"{"extends": "./a.json"}"#)) } else { None diff --git a/crates/bamts-compiler/src/public_ast/visitor.rs b/crates/bamts-compiler/src/public_ast/visitor.rs index b775f506..6c4e487f 100644 --- a/crates/bamts-compiler/src/public_ast/visitor.rs +++ b/crates/bamts-compiler/src/public_ast/visitor.rs @@ -748,7 +748,9 @@ where } AssignmentTarget::Array(array) => { for element in &array.elements { - if let AssignmentArrayElement::Target(target) = element { + if let AssignmentArrayElement::Target(target) + | AssignmentArrayElement::Rest(target) = element + { visit_node(NodeRef::AssignmentTarget(target), visitor)?; } } @@ -837,7 +839,7 @@ where } visit_node(NodeRef::TypeAnnotation(&signature.type_annotation), visitor)?; } - ClassMember::Missing(_) => {} + ClassMember::Missing(_) | ClassMember::Semicolon => {} } ControlFlow::Continue(()) } diff --git a/crates/bamts-compiler/src/rules/mod.rs b/crates/bamts-compiler/src/rules/mod.rs index 52926b24..c822906e 100644 --- a/crates/bamts-compiler/src/rules/mod.rs +++ b/crates/bamts-compiler/src/rules/mod.rs @@ -1258,7 +1258,7 @@ fn visit_class( ClassMember::StaticBlock(block) => { visit_statement_list(&block.data().statements, script_kind, findings) } - ClassMember::IndexSignature(_) | ClassMember::Missing(_) => {} + ClassMember::IndexSignature(_) | ClassMember::Missing(_) | ClassMember::Semicolon => {} } } } diff --git a/crates/bamts-compiler/src/scanner.rs b/crates/bamts-compiler/src/scanner.rs index 3771e251..084031c7 100644 --- a/crates/bamts-compiler/src/scanner.rs +++ b/crates/bamts-compiler/src/scanner.rs @@ -14,9 +14,10 @@ //! resolved through *explicit* scanner operations rather than guesses: //! //! * A `/` is ambiguous between division and a regular-expression literal. The -//! default pass always emits [`TokenKind::Slash`]/[`TokenKind::SlashEq`]; a -//! caller with grammar context calls [`Scanner::rescan_regex`] to reinterpret -//! it as a [`TokenKind::RegularExpressionLiteral`]. +//! default pass emits [`TokenKind::Slash`]/[`TokenKind::SlashEq`] unless the +//! token before it cannot end an expression; a caller with grammar context +//! calls [`Scanner::rescan_regex`] to reinterpret it as a +//! [`TokenKind::RegularExpressionLiteral`]. //! * A `>` is ambiguous between a single relational operator and the start of a //! shift/compound operator. The default pass greedily forms the longest //! operator; a caller closing type arguments or a JSX tag calls @@ -386,10 +387,10 @@ impl ScannedSource { /// Scans a whole source into an ordered token stream with recovery diagnostics. /// -/// This is the default single-pass driver: `/` stays division and `>` forms the -/// longest operator, both of which a grammar-aware caller can reinterpret with -/// [`Scanner::rescan_regex`] and [`Scanner::rescan_greater_than`]. Template -/// literals are fully segmented here. +/// This is the default single-pass driver: `>` forms the longest operator and `/` +/// is division unless nothing before it could be divided. A grammar-aware caller +/// can reinterpret either with [`Scanner::rescan_regex`] and +/// [`Scanner::rescan_greater_than`]. Template literals are fully segmented here. #[must_use] pub fn scan( source_id: SourceId, @@ -420,14 +421,27 @@ pub fn scan_with_cancel( let (tokens, eof, diagnostics) = { let mut scanner = Scanner::new_with_cancel(source_id, script_kind, &source, cancel.clone()); let mut tokens = Vec::new(); + let mut previous = None; let eof = loop { if scanner.is_cancelled() { break scanner.make(TokenKind::EndOfFile, scanner.position().get()); } - let token = scanner.next_token(); + let mut token = scanner.next_token(); + if matches!(token.kind(), TokenKind::Slash | TokenKind::SlashEq) + && previous.is_some_and(cannot_end_expression) + && scanner.regex_closes_at_last_token() + { + token = scanner.rescan_regex(); + } if token.kind() == TokenKind::EndOfFile || scanner.is_cancelled() { break token; } + if !matches!( + token.kind(), + TokenKind::Whitespace | TokenKind::LineComment | TokenKind::BlockComment + ) { + previous = Some(token.kind()); + } tokens.push(token); }; (tokens, eof, scanner.into_diagnostics()) @@ -447,6 +461,71 @@ pub fn scan_with_cancel( Ok(Recovered::new(product, diagnostics)) } +/// Whether no expression can end with `kind`, so a `/` after it has nothing to +/// divide and can only start a regular expression. This needs no grammar: reading +/// that `/` as division here would let a quote in the regex open a false string. +/// Tokens an expression can end with, such as `)`, `}`, `]`, `!`, and `>`, stay +/// with the parser, which knows what they closed. So does `<`, which opens a +/// closing JSX tag. +const fn cannot_end_expression(kind: TokenKind) -> bool { + matches!( + kind, + TokenKind::LParen + | TokenKind::LBracket + | TokenKind::LBrace + | TokenKind::Comma + | TokenKind::Semicolon + | TokenKind::Colon + | TokenKind::Question + | TokenKind::QuestionQuestion + | TokenKind::Arrow + | TokenKind::Plus + | TokenKind::Minus + | TokenKind::Star + | TokenKind::StarStar + | TokenKind::Percent + | TokenKind::Eq + | TokenKind::EqEq + | TokenKind::EqEqEq + | TokenKind::BangEq + | TokenKind::BangEqEq + | TokenKind::Amp + | TokenKind::AmpAmp + | TokenKind::Pipe + | TokenKind::PipePipe + | TokenKind::Caret + | TokenKind::Tilde + | TokenKind::PlusEq + | TokenKind::MinusEq + | TokenKind::StarEq + | TokenKind::StarStarEq + | TokenKind::SlashEq + | TokenKind::PercentEq + | TokenKind::LessLessEq + | TokenKind::GreaterGreaterEq + | TokenKind::GreaterGreaterGreaterEq + | TokenKind::AmpEq + | TokenKind::AmpAmpEq + | TokenKind::PipeEq + | TokenKind::PipePipeEq + | TokenKind::CaretEq + | TokenKind::QuestionQuestionEq + | TokenKind::TemplateHead + | TokenKind::TemplateMiddle + | TokenKind::KwReturn + | TokenKind::KwTypeof + | TokenKind::KwCase + | TokenKind::KwThrow + | TokenKind::KwVoid + | TokenKind::KwDelete + | TokenKind::KwIn + | TokenKind::KwInstanceof + | TokenKind::KwNew + | TokenKind::KwElse + | TokenKind::KwDo + ) +} + /// The kind of an open brace the scanner is currently inside, used to segment /// template literals in one pass without grammar feedback. #[derive(Clone, Copy, Debug, Eq, PartialEq)] @@ -629,6 +708,12 @@ impl<'a> Scanner<'a> { self.make(kind, start_u) } + /// Whether a regular expression starting at the most recent token closes on + /// its own line. One that does not is left for the parser to judge. + fn regex_closes_at_last_token(&self) -> bool { + scan_regex_slice(&self.text[self.last_start_byte..]).1 + } + /// Reinterprets the most recent `>`-family token as a single `>`, advancing /// exactly one code unit past the last token's start. /// @@ -2197,6 +2282,74 @@ mod tests { ); } + #[test] + fn a_regex_holding_a_quote_does_not_break_the_template_around_it() { + // Read as a division, the `'` opens a string that swallows the `}` and + // leaves the template unterminated. + let text = "`${k.replace(/'/g, \"\\\\'\")}`;\n"; + let recovered = scan_text(text); + assert_eq!(recovered.diagnostics(), &[]); + let tokens = significant(text); + assert!(tokens.contains(&(TokenKind::RegularExpressionLiteral, "/'/g".to_owned()))); + assert_eq!( + tokens.last().map(|token| token.0), + Some(TokenKind::Semicolon) + ); + } + + #[test] + fn a_slash_that_cannot_divide_starts_a_regex() { + for text in [ + "x = /'/;", + "f(/'/);", + "[/'/];", + "a, /'/;", + "a ? /'/ : b;", + "return /'/;", + "typeof /'/;", + "x = /=/;", + ] { + assert!( + kinds(text).contains(&TokenKind::RegularExpressionLiteral), + "{text}" + ); + assert_eq!(scan_text(text).diagnostics(), &[], "{text}"); + } + } + + #[test] + fn a_slash_with_an_operand_before_it_stays_division() { + for text in [ + "a / b / c", + "1 / 2 / 3", + "(a) / b / c", + "a[0] / b / c", + "x! / 2 / 3", + "a++ / b / c", + "a-- / b / c", + "f / b / c", + "{} / b / c", + "this / b / c", + ] { + assert!( + !kinds(text).contains(&TokenKind::RegularExpressionLiteral), + "{text}" + ); + } + } + + #[test] + fn a_closing_tag_slash_stays_for_the_parser() { + assert!(!kinds("").contains(&TokenKind::RegularExpressionLiteral)); + } + + #[test] + fn a_slash_that_never_closes_is_left_for_the_parser_to_judge() { + let text = "x = /abc\n"; + assert!(kinds(text).contains(&TokenKind::Slash)); + assert_eq!(scan_text(text).diagnostics(), &[]); + } + #[test] fn templates_segment_with_nested_braces() { // `${ {a:1} }` nests an object literal whose braces must not close the @@ -2386,15 +2539,8 @@ mod tests { #[test] fn corpus_cases_lex_totally_and_tile() { - // The default pass does not guess `/` as a regular expression, so a - // file that uses a regex literal will not lex cleanly without the - // explicit `rescan_regex` a parser would drive. These files still must - // tile and round-trip; only the clean-diagnostics claim is waived. - const REGEX_LITERAL_CASES: &[&str] = &["escape-string-regexp.ts"]; - let root = PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../corpus/cases"); let mut scanned_any = false; - let mut regex_case_seen = false; for entry in std::fs::read_dir(&root).expect("corpus/cases must be readable") { let path = entry.expect("directory entry").path(); if path.extension().and_then(|ext| ext.to_str()) != Some("ts") { @@ -2437,29 +2583,14 @@ mod tests { "{name}: lexemes did not reproduce the source" ); - if REGEX_LITERAL_CASES.contains(&name.as_str()) { - regex_case_seen = true; - // The no-guess contract means the default pass reports at least - // one diagnostic on a raw regex literal it cannot recognize. - assert!( - !recovered.diagnostics().is_empty(), - "{name}: expected the default pass to reject a regex literal" - ); - } else { - // Every regex-free corpus driver is valid TypeScript and must - // lex cleanly under the default pass. - assert!( - recovered.diagnostics().is_empty(), - "{name}: unexpected diagnostics {:?}", - recovered.diagnostics() - ); - } + // Every corpus driver is valid TypeScript and must lex cleanly. + assert!( + recovered.diagnostics().is_empty(), + "{name}: unexpected diagnostics {:?}", + recovered.diagnostics() + ); } assert!(scanned_any, "expected at least one corpus case"); - assert!( - regex_case_seen, - "expected the regex-literal case to be present" - ); } #[test] diff --git a/crates/bamts-compiler/src/script.rs b/crates/bamts-compiler/src/script.rs index 242a4489..6fa933be 100644 --- a/crates/bamts-compiler/src/script.rs +++ b/crates/bamts-compiler/src/script.rs @@ -78,6 +78,7 @@ pub fn compile_classic_script( let module_name = normalized_module_name(resource_name).unwrap_or(DEFAULT_MODULE_NAME); let options = LowerOptions { javascript_compatibility: true, + script_entry: true, }; let enum_facts = EnumFacts::unchecked(); let namespace_facts = NamespaceFacts::unchecked(); @@ -89,6 +90,15 @@ pub fn compile_classic_script( &namespace_facts, ) .map_err(|error| map_lower_error(&source, error))?; + // Lowering reports the structural errors first; these are the rest. + if let Some(diagnostic) = crate::early_errors::check_script(parsed.product()).first() { + let (line, column) = line_column(&source, diagnostic.range().start()); + return Err(ScriptCompileError::Syntax { + message: diagnostic.message().to_owned(), + line, + column, + }); + } let encoded_name = EcmaString::encode(module_name); let name = assembled .constants() diff --git a/crates/bamts-compiler/src/service/async.rs b/crates/bamts-compiler/src/service/async.rs index d037b016..972ac7f5 100644 --- a/crates/bamts-compiler/src/service/async.rs +++ b/crates/bamts-compiler/src/service/async.rs @@ -228,7 +228,6 @@ mod tests { num::NonZeroUsize, sync::Arc, task::{Context, Poll, Waker}, - time::{SystemTime, UNIX_EPOCH}, }; use super::{AsyncService, CancellationToken}; @@ -247,15 +246,7 @@ mod tests { } fn service() -> (std::path::PathBuf, AsyncService) { - let nonce = SystemTime::now() - .duration_since(UNIX_EPOCH) - .expect("clock") - .as_nanos(); - let root = std::env::temp_dir().join(format!( - "bamts-async-service-{}-{nonce}", - std::process::id() - )); - fs::create_dir(&root).expect("create root"); + let root = crate::service::test_support::fresh_dir("bamts-async-service"); let service = AsyncService::new( OsFileSystem::new(&root).expect("filesystem"), NonZeroUsize::new(1).expect("nonzero"), diff --git a/crates/bamts-compiler/src/service/filesystem.rs b/crates/bamts-compiler/src/service/filesystem.rs index dcc32c0a..15c3c7b4 100644 --- a/crates/bamts-compiler/src/service/filesystem.rs +++ b/crates/bamts-compiler/src/service/filesystem.rs @@ -4,6 +4,8 @@ use std::{ time::SystemTime, }; +use crate::project::canonical; + /// Metadata used to decide whether a closed file needs a fresh compiler snapshot. #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct FileMetadata { @@ -43,7 +45,17 @@ impl std::error::Error for FileSystemError {} impl From for FileSystemError { fn from(error: io::Error) -> Self { - Self::new(error.kind(), error.to_string()) + Self::new(absent_aware_kind(&error), error.to_string()) + } +} + +/// Windows answers `InvalidFilename` for a name it cannot hold, such as +/// `engine:clock`, where Unix answers `NotFound`. No file can carry such a name, +/// so both platforms report the file as absent. +fn absent_aware_kind(error: &io::Error) -> io::ErrorKind { + match error.kind() { + io::ErrorKind::InvalidFilename => io::ErrorKind::NotFound, + kind => kind, } } @@ -64,7 +76,7 @@ pub struct OsFileSystem { impl OsFileSystem { pub fn new(root: impl AsRef) -> Result { - let root = fs::canonicalize(root).map_err(FileSystemError::from)?; + let root = canonical::canonicalize(root).map_err(FileSystemError::from)?; if !root.is_dir() { return Err(FileSystemError::new( io::ErrorKind::NotADirectory, @@ -103,7 +115,7 @@ impl FileSystem for OsFileSystem { loop { match fs::symlink_metadata(ancestor) { Ok(_) => break, - Err(error) if error.kind() == io::ErrorKind::NotFound => { + Err(error) if absent_aware_kind(&error) == io::ErrorKind::NotFound => { ancestor = ancestor.parent().ok_or_else(|| { FileSystemError::new( io::ErrorKind::NotFound, @@ -114,14 +126,14 @@ impl FileSystem for OsFileSystem { Err(error) => return Err(FileSystemError::from(error)), } } - let mut canonical = fs::canonicalize(ancestor).map_err(FileSystemError::from)?; + let mut resolved = canonical::canonicalize(ancestor).map_err(FileSystemError::from)?; let suffix = lexical .strip_prefix(ancestor) .expect("selected ancestor is a lexical path prefix"); for component in suffix.components() { - canonical.push(component); + resolved.push(component); } - self.confined(canonical) + self.confined(resolved) } fn read(&self, path: &Path) -> Result { @@ -150,8 +162,8 @@ impl FileSystem for OsFileSystem { for entry in fs::read_dir(&path).map_err(FileSystemError::from)? { let entry = entry.map_err(FileSystemError::from)?; let child = path.join(entry.file_name()); - let canonical = fs::canonicalize(&child).map_err(FileSystemError::from)?; - entries.push(self.confined(canonical)?); + let resolved = canonical::canonicalize(&child).map_err(FileSystemError::from)?; + entries.push(self.confined(resolved)?); } entries.sort(); entries.dedup(); @@ -163,12 +175,7 @@ fn normalize_lexically(path: &Path) -> Result { let mut normalized = PathBuf::new(); for component in path.components() { match component { - Component::Prefix(_) => { - return Err(FileSystemError::new( - io::ErrorKind::InvalidInput, - "platform path prefix is unsupported", - )); - } + Component::Prefix(prefix) => normalized.push(prefix.as_os_str()), Component::RootDir => normalized.push(component.as_os_str()), Component::CurDir => {} Component::ParentDir => { @@ -187,23 +194,12 @@ fn normalize_lexically(path: &Path) -> Result { #[cfg(test)] mod tests { - use std::{ - fs, io, - path::Path, - time::{SystemTime, UNIX_EPOCH}, - }; + use std::{fs, io, path::Path}; use super::{FileSystem, OsFileSystem}; fn temporary_root() -> std::path::PathBuf { - let nonce = SystemTime::now() - .duration_since(UNIX_EPOCH) - .expect("clock after epoch") - .as_nanos(); - let root = - std::env::temp_dir().join(format!("bamts-service-fs-{}-{nonce}", std::process::id())); - fs::create_dir(&root).expect("create temporary root"); - root + crate::service::test_support::fresh_dir("bamts-service-fs") } fn collect_names(entries: Vec) -> Vec { @@ -241,6 +237,72 @@ mod tests { fs::remove_dir_all(root).expect("remove temporary root"); } + #[test] + fn absolute_paths_are_accepted_inside_the_root_and_confined_outside_it() { + let root = temporary_root(); + fs::write(root.join("a.ts"), "const a = 1;").expect("write source"); + let filesystem = OsFileSystem::new(&root).expect("filesystem"); + + let inside = filesystem.root().join("a.ts"); + assert_eq!(filesystem.normalize(&inside).expect("inside"), inside); + assert_eq!(filesystem.read(&inside).expect("read"), "const a = 1;"); + + let outside = filesystem.root().parent().expect("root has a parent"); + assert_eq!( + filesystem + .normalize(outside) + .expect_err("outside the root") + .kind(), + io::ErrorKind::PermissionDenied + ); + + fs::remove_dir_all(root).expect("remove temporary root"); + } + + #[test] + #[cfg(windows)] + fn lexical_normalization_keeps_drive_and_verbatim_prefixes() { + use super::normalize_lexically; + + for (input, expected) in [ + (r"C:\a\.\b\..\c", r"C:\a\c"), + (r"\\?\C:\a\..\b", r"\\?\C:\b"), + (r"\\server\share\a\..\b", r"\\server\share\b"), + ] { + assert_eq!( + normalize_lexically(Path::new(input)).expect(input), + Path::new(expected), + "{input}" + ); + } + assert_eq!( + normalize_lexically(Path::new(r"C:\..")) + .expect_err("above the drive root") + .kind(), + io::ErrorKind::PermissionDenied + ); + } + + #[test] + fn names_the_platform_cannot_hold_are_absent_not_invalid() { + let root = temporary_root(); + let filesystem = OsFileSystem::new(&root).expect("filesystem"); + let scheme = Path::new("node_modules/engine:clock/package.json"); + + let normalized = filesystem.normalize(scheme).expect("normalize is lexical"); + assert_eq!(normalized, filesystem.root().join(scheme)); + assert_eq!( + filesystem.metadata(scheme).expect_err("absent").kind(), + io::ErrorKind::NotFound + ); + assert_eq!( + filesystem.read(scheme).expect_err("absent").kind(), + io::ErrorKind::NotFound + ); + + fs::remove_dir_all(root).expect("remove temporary root"); + } + #[test] fn read_dir_sorts_entries_and_includes_directories() { let root = temporary_root(); @@ -324,7 +386,10 @@ mod tests { #[test] #[cfg(unix)] fn read_dir_rejects_escaping_symlink() { - use std::os::unix::fs::symlink; + use std::{ + os::unix::fs::symlink, + time::{SystemTime, UNIX_EPOCH}, + }; let root = temporary_root(); let outside = root.parent().unwrap().join(format!("bamts-outside-{}", { diff --git a/crates/bamts-compiler/src/service/language_service.rs b/crates/bamts-compiler/src/service/language_service.rs index 12555caf..0deffe1c 100644 --- a/crates/bamts-compiler/src/service/language_service.rs +++ b/crates/bamts-compiler/src/service/language_service.rs @@ -557,24 +557,13 @@ const fn symbol_index(symbol: SymbolId) -> usize { #[cfg(test)] mod tests { - use std::{ - fs, - time::{SystemTime, UNIX_EPOCH}, - }; + use std::fs; use super::*; use crate::service::{ServiceError, ServiceState, filesystem::OsFileSystem}; fn state(source: &str) -> (PathBuf, ServiceState) { - let nonce = SystemTime::now() - .duration_since(UNIX_EPOCH) - .expect("clock") - .as_nanos(); - let root = std::env::temp_dir().join(format!( - "bamts-language-service-{}-{nonce}", - std::process::id() - )); - fs::create_dir(&root).expect("create root"); + let root = crate::service::test_support::fresh_dir("bamts-language-service"); let mut state = ServiceState::new(OsFileSystem::new(&root).expect("filesystem")); state.open("a.ts", source, 1).expect("open"); (root, state) diff --git a/crates/bamts-compiler/src/service/mod.rs b/crates/bamts-compiler/src/service/mod.rs index 8c6aa916..27af612f 100644 --- a/crates/bamts-compiler/src/service/mod.rs +++ b/crates/bamts-compiler/src/service/mod.rs @@ -8,6 +8,8 @@ pub mod r#async; pub mod filesystem; pub mod language_service; pub mod sync; +#[cfg(test)] +mod test_support; use bamts_cancel::CancellationToken; use std::{ @@ -811,12 +813,7 @@ fn import_clause_is_type_only(clause: Option<&crate::syntax::ImportClause>) -> b #[cfg(test)] mod tests { - use std::{ - fs, - path::Path, - sync::Arc, - time::{SystemTime, UNIX_EPOCH}, - }; + use std::{fs, path::Path, sync::Arc}; use super::{ CancellationToken, ServiceError, ServiceState, SnapshotContent, build_snapshot_with_cancel, @@ -824,15 +821,7 @@ mod tests { use crate::{service::filesystem::OsFileSystem, source::SourceId}; fn state() -> (std::path::PathBuf, ServiceState) { - let nonce = SystemTime::now() - .duration_since(UNIX_EPOCH) - .expect("clock") - .as_nanos(); - let root = std::env::temp_dir().join(format!( - "bamts-service-state-{}-{nonce}", - std::process::id() - )); - fs::create_dir(&root).expect("create root"); + let root = crate::service::test_support::fresh_dir("bamts-service-state"); let state = ServiceState::new(OsFileSystem::new(&root).expect("filesystem")); (root, state) } diff --git a/crates/bamts-compiler/src/service/sync.rs b/crates/bamts-compiler/src/service/sync.rs index adea4885..e113c6ed 100644 --- a/crates/bamts-compiler/src/service/sync.rs +++ b/crates/bamts-compiler/src/service/sync.rs @@ -222,24 +222,14 @@ impl SyncService { #[cfg(test)] mod tests { - use std::{ - fs, - sync::Arc, - time::{SystemTime, UNIX_EPOCH}, - }; + use std::{fs, sync::Arc}; use super::SyncService; use crate::service::{ServiceError, filesystem::OsFileSystem}; #[test] fn sync_api_preserves_version_failures_across_clones() { - let nonce = SystemTime::now() - .duration_since(UNIX_EPOCH) - .expect("clock") - .as_nanos(); - let root = - std::env::temp_dir().join(format!("bamts-sync-service-{}-{nonce}", std::process::id())); - fs::create_dir(&root).expect("create root"); + let root = crate::service::test_support::fresh_dir("bamts-sync-service"); let service = SyncService::new(OsFileSystem::new(&root).expect("filesystem")); service .open("a.ts", Arc::::from("const a = 1;"), 1) diff --git a/crates/bamts-compiler/src/service/test_support.rs b/crates/bamts-compiler/src/service/test_support.rs new file mode 100644 index 00000000..1269b8c6 --- /dev/null +++ b/crates/bamts-compiler/src/service/test_support.rs @@ -0,0 +1,70 @@ +//! Scratch directories for the service tests. + +use std::{ + fs, + path::PathBuf, + sync::atomic::{AtomicU64, Ordering}, + time::{SystemTime, UNIX_EPOCH}, +}; + +static NEXT_DIRECTORY: AtomicU64 = AtomicU64::new(0); + +/// Creates a fresh, empty directory under the system temp directory. +/// +/// The clock alone cannot make the name unique. macOS reads it in microseconds, +/// so tests that start together get the same value and collide on +/// `AlreadyExists`. The process-wide sequence number is what separates them, +/// and the process id and clock keep a directory left behind by an earlier run +/// from colliding with a new one. +pub(crate) fn fresh_dir(prefix: &str) -> PathBuf { + let sequence = NEXT_DIRECTORY.fetch_add(1, Ordering::Relaxed); + let nonce = SystemTime::now() + .duration_since(UNIX_EPOCH) + .expect("clock after epoch") + .as_nanos(); + let root = std::env::temp_dir().join(format!( + "{prefix}-{}-{nonce}-{sequence}", + std::process::id() + )); + fs::create_dir(&root).expect("create test directory"); + root +} + +#[cfg(test)] +mod tests { + use std::{ + collections::BTreeSet, + sync::{Arc, Barrier}, + thread, + }; + + use super::fresh_dir; + + /// Threads that start together read the same clock value, so only the + /// sequence number keeps their directories apart. + #[test] + fn directories_made_at_the_same_moment_are_distinct() { + const THREADS: usize = 64; + let start = Arc::new(Barrier::new(THREADS)); + let workers: Vec<_> = (0..THREADS) + .map(|_| { + let start = Arc::clone(&start); + thread::spawn(move || { + start.wait(); + fresh_dir("bamts-test-support") + }) + }) + .collect(); + let roots: Vec<_> = workers + .into_iter() + .map(|worker| worker.join().expect("a directory is created without error")) + .collect(); + + let distinct: BTreeSet<_> = roots.iter().collect(); + assert_eq!(distinct.len(), THREADS); + for root in &roots { + assert!(root.is_dir(), "{} is not a directory", root.display()); + std::fs::remove_dir(root).expect("remove"); + } + } +} diff --git a/crates/bamts-compiler/src/syntax.rs b/crates/bamts-compiler/src/syntax.rs index 97f993fc..25415a69 100644 --- a/crates/bamts-compiler/src/syntax.rs +++ b/crates/bamts-compiler/src/syntax.rs @@ -904,6 +904,9 @@ pub struct AssignmentObjectProperty { pub name: PropertyName, pub target: AssignmentTargetNode, pub initializer: Option>, + /// `...target`, which collects the properties the others left. It has no + /// name, and kept here it is visited with the targets around it. + pub rest: bool, } #[derive(Clone, Debug, Eq, PartialEq)] @@ -916,6 +919,8 @@ pub enum AssignmentArrayElement { Target(AssignmentTargetNode), Elision, Missing(MissingNode), + /// `...target`, which collects what the iterator has left. + Rest(AssignmentTargetNode), } #[derive(Clone, Debug, Eq, PartialEq)] @@ -1071,6 +1076,8 @@ pub enum ClassMember { StaticBlock(BlockNode), IndexSignature(IndexSignature), Missing(MissingNode), + /// A bare `;` between members. It declares nothing, and emit keeps it. + Semicolon, } impl NodeData for ClassMember { diff --git a/crates/bamts-platform/Cargo.toml b/crates/bamts-platform/Cargo.toml new file mode 100644 index 00000000..364247d9 --- /dev/null +++ b/crates/bamts-platform/Cargo.toml @@ -0,0 +1,18 @@ +[package] +name = "bamts-platform" +version.workspace = true +edition.workspace = true +rust-version.workspace = true +publish = true +license.workspace = true +repository.workspace = true +description = "Host facts and process observation for BamTS" + +[dependencies] +libc = "=0.2.189" + +[lints.rust] +# `unsafe_code` is downgraded from the workspace `forbid` to `deny` so that the +# OS backend modules alone can carry `#[allow(unsafe_code)]`. +unsafe_code = "deny" +unsafe_op_in_unsafe_fn = "deny" diff --git a/crates/bamts-platform/src/lib.rs b/crates/bamts-platform/src/lib.rs new file mode 100644 index 00000000..f0720d9d --- /dev/null +++ b/crates/bamts-platform/src/lib.rs @@ -0,0 +1,524 @@ +//! Host facts and process observation. +//! +//! OS-specific reads live here so callers stay portable. A fact the host +//! cannot provide is [`UNAVAILABLE`] or a typed error, never a fabricated value. + +use std::fmt; + +#[cfg(any(target_os = "linux", test))] +mod linux; +#[cfg(any(target_os = "macos", test))] +mod macos; +#[cfg(any(target_os = "windows", test))] +mod windows; + +/// Recorded for a fact this host does not have, such as a CPU governor on macOS. +pub const UNAVAILABLE: &str = "unavailable"; + +/// The file the Linux backend reads for the CPU governor. A host manifest +/// records it as the evidence source of its `governor` condition, so a test can +/// hold the manifest to the code that does the reading. +pub const LINUX_GOVERNOR_SOURCE: &str = "/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor"; + +/// Why a host fact could not be read. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum PlatformError { + /// This host has no implementation of the named capability. + Unsupported(&'static str), + /// The host cannot find the process, which has exited or is exiting. + ProcessGone(u32), + /// The process shows no address space at this moment. It may be exiting, + /// replacing its image with `exec` on macOS, or running on after its main + /// thread exited on Linux. Only a later read can tell which. + NoAddressSpace(u32), + /// The host failed the read. + Read { source: String, detail: String }, + /// The host answered in a shape the parser does not accept. + Malformed { source: String, detail: String }, +} + +impl fmt::Display for PlatformError { + fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { + match self { + Self::Unsupported(what) => write!(formatter, "{what} is not available on this host"), + Self::ProcessGone(pid) => write!(formatter, "process {pid} no longer exists"), + Self::NoAddressSpace(pid) => { + write!(formatter, "process {pid} has no address space right now") + } + Self::Read { source, detail } | Self::Malformed { source, detail } => { + write!(formatter, "{source}: {detail}") + } + } + } +} + +impl std::error::Error for PlatformError {} + +/// Facts that identify the machine, for matching a measurement host. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct MachineFacts { + /// Kernel release, such as `7.0.0-30-generic`. + pub kernel_release: String, + /// Kernel build version tail. + pub kernel_version: String, + /// CPU model name. + pub cpu_model: String, + /// Distinct physical sockets. + pub sockets: u32, + /// Physical cores per socket. + pub cores_per_socket: u32, + /// CPU microcode revision, or [`UNAVAILABLE`]. + pub microcode: String, + /// CPU frequency governor, or the host's marker for none. + pub governor: String, + /// Total swap in KiB. + pub swap_total_kib: u64, +} + +/// How the calling thread is placed on the machine. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ThreadPlacement { + /// CPUs the thread may run on, such as `0-19`, or [`UNAVAILABLE`]. + pub cpu_affinity: String, + /// NUMA memory policy, or [`UNAVAILABLE`]. + pub memory_policy: String, + /// NUMA memory nodes, or [`UNAVAILABLE`]. + pub memory_nodes: String, +} + +/// Reads the facts that identify this machine. +/// +/// # Errors +/// Returns [`PlatformError`] when this host has no implementation or the host +/// answers in a shape the reader does not accept. +pub fn machine_facts() -> Result { + #[cfg(target_os = "linux")] + { + linux::machine_facts() + } + #[cfg(target_os = "macos")] + { + macos::machine_facts() + } + #[cfg(target_os = "windows")] + { + windows::machine_facts() + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + Err(PlatformError::Unsupported("machine facts")) + } +} + +/// Reads how the calling thread is placed on this machine. +/// +/// # Errors +/// Returns [`PlatformError`] when this host has no implementation or the host +/// answers in a shape the reader does not accept. +pub fn thread_placement() -> Result { + #[cfg(target_os = "linux")] + { + linux::thread_placement() + } + #[cfg(target_os = "macos")] + { + macos::thread_placement() + } + #[cfg(target_os = "windows")] + { + windows::thread_placement() + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + Err(PlatformError::Unsupported("thread placement")) + } +} + +/// The governor of CPU 0, or [`UNAVAILABLE`] on a host without one. A Linux host +/// that cannot say fails here, unlike [`MachineFacts::governor`], which identifies +/// a machine and tolerates the gap. +/// +/// # Errors +/// Returns [`PlatformError`] when this host has no implementation or the host +/// fails the read. +pub fn cpu_governor() -> Result { + #[cfg(target_os = "linux")] + { + linux::cpu_governor() + } + #[cfg(any(target_os = "macos", target_os = "windows"))] + { + Ok(UNAVAILABLE.to_owned()) + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + Err(PlatformError::Unsupported("cpu governor")) + } +} + +/// Total swap in KiB. +/// +/// # Errors +/// Returns [`PlatformError`] when this host has no implementation or the host +/// answers in a shape the reader does not accept. +pub fn swap_total_kib() -> Result { + #[cfg(target_os = "linux")] + { + linux::swap_total_kib() + } + #[cfg(target_os = "macos")] + { + macos::swap_total_kib() + } + #[cfg(target_os = "windows")] + { + windows::swap_total_kib() + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + Err(PlatformError::Unsupported("swap total")) + } +} + +/// The CPUs the calling thread may run on, such as `0-19`, or [`UNAVAILABLE`]. +/// +/// # Errors +/// Returns [`PlatformError`] when this host has no implementation or the host +/// answers in a shape the reader does not accept. +pub fn cpu_affinity() -> Result { + #[cfg(target_os = "linux")] + { + linux::cpu_affinity() + } + #[cfg(target_os = "macos")] + { + Ok(UNAVAILABLE.to_owned()) + } + #[cfg(target_os = "windows")] + { + windows::cpu_affinity() + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + Err(PlatformError::Unsupported("cpu affinity")) + } +} + +/// The CPUs that are online, as a kernel CPU list such as `0-19`. +/// +/// # Errors +/// Returns [`PlatformError`] when this host has no implementation or the host +/// fails the read. +pub fn online_cpu_list() -> Result { + #[cfg(target_os = "linux")] + { + linux::online_cpu_list() + } + #[cfg(target_os = "macos")] + { + macos::online_cpu_list() + } + #[cfg(target_os = "windows")] + { + windows::online_cpu_list() + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + Err(PlatformError::Unsupported("online CPU list")) + } +} + +/// The frequency governor of each online CPU, sorted. A host that has none, +/// such as macOS, reports a single [`UNAVAILABLE`]. +/// +/// # Errors +/// Returns [`PlatformError`] when this host has no implementation or the host +/// fails the read. +pub fn cpu_governors() -> Result, PlatformError> { + #[cfg(target_os = "linux")] + { + linux::cpu_governors() + } + #[cfg(any(target_os = "macos", target_os = "windows"))] + { + Ok(vec![UNAVAILABLE.to_owned()]) + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + Err(PlatformError::Unsupported("CPU governors")) + } +} + +/// The memory policy of this process as `numactl --show` reports it, or +/// [`UNAVAILABLE`] when the host or the tool does not say. +#[must_use] +pub fn numa_policy_mode() -> String { + #[cfg(target_os = "linux")] + { + linux::numa_policy_mode() + } + #[cfg(not(target_os = "linux"))] + { + UNAVAILABLE.to_owned() + } +} + +/// The memory nodes this process is bound to. A host without NUMA nodes, such as +/// macOS, has none and returns an empty list. Windows binds a process to no subset +/// of its nodes, so it lists every node the host reports. +/// +/// # Errors +/// Returns [`PlatformError`] when this host has no implementation or the host +/// fails the read. +pub fn numa_memory_nodes() -> Result, PlatformError> { + #[cfg(target_os = "linux")] + { + linux::numa_memory_nodes() + } + #[cfg(target_os = "macos")] + { + Ok(Vec::new()) + } + #[cfg(target_os = "windows")] + { + windows::numa_memory_nodes() + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + Err(PlatformError::Unsupported("memory nodes")) + } +} + +/// One contiguous memory region of a process. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct MemoryRegion { + /// Names this region. Stable across samples of one address space. + pub identity: String, + /// Four characters: `r`, `w`, and `x`, each `-` when absent, then `p`. + /// For example `r-xp`. + pub permissions: String, + /// The backing file, or `None` for anonymous memory. + pub pathname: Option, +} + +/// Lists the memory regions of a process. +/// +/// The read is not atomic. While a process execs or exits, a list can be partial +/// or mix its old and new images, so sample a changing process repeatedly. A +/// mapping that comes and goes between samples is not seen at all. +/// +/// # Errors +/// Returns [`PlatformError::ProcessGone`] when the host cannot find the process +/// and [`PlatformError::NoAddressSpace`] when it has no regions at this moment. +/// Neither means a child can be reaped yet, so poll it rather than concluding +/// from one read, and do not block on it: after `NoAddressSpace` it may still be +/// running. Returns another [`PlatformError`] when this host has no +/// implementation or the host fails the read. +/// +/// On Windows a region is a run of committed pages with one protection. Reserved +/// and free address space is left out, and a copy-on-write page counts as +/// writable. A file-backed region names its file as a DOS path, or as an NT path +/// when its device has no drive letter. +pub fn process_regions(pid: u32) -> Result, PlatformError> { + #[cfg(target_os = "linux")] + { + linux::process_regions(pid) + } + #[cfg(target_os = "macos")] + { + macos::process_regions(pid) + } + #[cfg(target_os = "windows")] + { + windows::process_regions(pid) + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + let _ = pid; + Err(PlatformError::Unsupported("process regions")) + } +} + +#[cfg(target_os = "linux")] +use linux::Window as WindowImpl; +#[cfg(target_os = "macos")] +use macos::Window as WindowImpl; +#[cfg(target_os = "windows")] +use windows::Window as WindowImpl; + +/// A host with no peak-memory window cannot construct one. +#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] +enum WindowImpl {} + +#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] +impl WindowImpl { + fn start() -> Result { + Err(PlatformError::Unsupported("peak memory window")) + } + + fn finish(self) -> Result { + match self {} + } +} + +/// Peak memory of a process since it started or last execed, in bytes. +/// +/// This is the resident-set peak on Linux, the physical-footprint peak on macOS +/// and the peak working set on Windows. +/// +/// # Errors +/// Returns [`PlatformError::ProcessGone`] when the host cannot find the process +/// and [`PlatformError::NoAddressSpace`] when it has no address space at this +/// moment. Returns another [`PlatformError`] when this host has no +/// implementation or the host fails the read. +/// +/// To follow a child to its end, poll beside a `wait` until `ProcessGone`, which +/// comes only after the reap, and keep polling through `NoAddressSpace`. Linux +/// reads the main thread, so it reports that during an exit and for a process +/// running on after its main thread exited. macOS keeps answering with the +/// final peak until the reap. Windows answers [PlatformError::ProcessGone] as soon +/// as the process has exited, even while its parent still holds a handle to it, +/// because the host would otherwise keep answering until that handle closed. +pub fn peak_memory_bytes(pid: u32) -> Result { + #[cfg(target_os = "linux")] + { + linux::peak_memory_bytes(pid) + } + #[cfg(target_os = "macos")] + { + macos::peak_memory_bytes(pid) + } + #[cfg(target_os = "windows")] + { + windows::peak_memory_bytes(pid) + } + #[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))] + { + let _ = pid; + Err(PlatformError::Unsupported("peak memory")) + } +} + +/// Measures the peak memory of the current process across a span of work. +/// +/// On Linux the host resets its high-water mark when the window starts, so +/// the result is exact. macOS has no public way to reset the peak, so the +/// window samples the process footprint every millisecond and a spike shorter +/// than that can be missed. Treat the macOS value as functional evidence, not +/// as a benchmark measurement. +/// +/// Windows has no way to reset its peak working set either, so the window samples +/// it the same way and the same caveat applies. +/// +/// Windows cannot overlap on Linux. Starting one resets a mark shared by the +/// whole process, which would lower the result of a window already open. +pub struct PeakWindow(WindowImpl); + +impl PeakWindow { + /// Starts measuring. + /// + /// # Errors + /// Returns [`PlatformError`] when this host has no implementation or the + /// host refuses to start the window. + pub fn start() -> Result { + WindowImpl::start().map(Self) + } + + /// Stops measuring and returns the peak in bytes since [`PeakWindow::start`]. + /// + /// # Errors + /// Returns [`PlatformError`] when the host fails the final read. + pub fn finish(self) -> Result { + self.0.finish() + } +} + +/// The Rust target triple of this host, in the form receipts record. +#[must_use] +pub fn host_triple() -> String { + let environment = if cfg!(target_env = "musl") { + "musl" + } else if cfg!(target_env = "msvc") { + "msvc" + } else { + "gnu" + }; + match std::env::consts::OS { + "linux" => format!("{}-unknown-linux-{environment}", std::env::consts::ARCH), + "macos" => format!("{}-apple-darwin", std::env::consts::ARCH), + "windows" => format!("{}-pc-windows-{environment}", std::env::consts::ARCH), + os => format!("{}-unknown-{os}", std::env::consts::ARCH), + } +} + +#[cfg(test)] +mod tests { + use super::*; + + /// On Linux a window resets a mark shared by the whole process, so tests that + /// open one run one at a time. + #[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] + static WINDOW_TESTS: std::sync::Mutex<()> = std::sync::Mutex::new(()); + + #[cfg(target_os = "linux")] + #[test] + fn a_window_does_not_count_an_earlier_peak() { + const BYTES: usize = 64 * 1024 * 1024; + let _serial = WINDOW_TESTS + .lock() + .unwrap_or_else(|poison| poison.into_inner()); + let block = vec![1_u8; BYTES]; + assert_eq!(block.iter().filter(|byte| **byte == 1).count(), BYTES); + drop(block); + let peak = PeakWindow::start().unwrap().finish().unwrap(); + assert!( + peak < BYTES as u64, + "peak {peak} still includes the allocation made before the window" + ); + } + + #[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] + #[test] + fn a_touched_allocation_raises_the_peak_window() { + const BYTES: usize = 64 * 1024 * 1024; + let _serial = WINDOW_TESTS + .lock() + .unwrap_or_else(|poison| poison.into_inner()); + let idle = PeakWindow::start().unwrap(); + let baseline = idle.finish().unwrap(); + + let window = PeakWindow::start().unwrap(); + let block = vec![1_u8; BYTES]; + // Hold the allocation live long enough for the 1 ms macOS sampler to + // be scheduled and see it, even on a busy machine. + std::thread::sleep(std::time::Duration::from_millis(150)); + assert_eq!(block.iter().filter(|byte| **byte == 1).count(), BYTES); + drop(block); + let peak = window.finish().unwrap(); + + let margin = 16 * 1024 * 1024; + assert!( + peak >= baseline + (BYTES - margin) as u64, + "peak {peak} did not rise from the idle {baseline} by the allocation" + ); + assert!(peak >= BYTES as u64, "peak {peak} is below the allocation"); + } + + #[cfg(any(target_os = "linux", target_os = "macos", target_os = "windows"))] + #[test] + fn the_process_peak_covers_its_current_footprint() { + assert!(peak_memory_bytes(std::process::id()).unwrap() > 0); + } + + #[test] + fn host_triple_matches_the_running_host() { + let triple = host_triple(); + assert!(triple.starts_with(std::env::consts::ARCH), "{triple}"); + if cfg!(target_os = "macos") { + assert_eq!(triple, format!("{}-apple-darwin", std::env::consts::ARCH)); + } + if cfg!(target_os = "linux") { + assert!(triple.contains("-unknown-linux-"), "{triple}"); + } + } +} diff --git a/crates/bamts-platform/src/linux.rs b/crates/bamts-platform/src/linux.rs new file mode 100644 index 00000000..1d0938ba --- /dev/null +++ b/crates/bamts-platform/src/linux.rs @@ -0,0 +1,889 @@ +//! Linux backend: `/proc`, `/sys`, and `get_mempolicy(2)`. + +use std::collections::{BTreeMap, BTreeSet}; +#[cfg(target_os = "linux")] +use std::fs; +#[cfg(target_os = "linux")] +use std::process::Command; + +#[cfg(target_os = "linux")] +use crate::{MachineFacts, ThreadPlacement}; +use crate::{MemoryRegion, PlatformError, UNAVAILABLE}; + +type Result = std::result::Result; + +fn malformed(source: &str, detail: impl Into) -> PlatformError { + PlatformError::Malformed { + source: source.to_owned(), + detail: detail.into(), + } +} + +#[cfg(target_os = "linux")] +fn read_evidence(path: &str) -> Result { + fs::read_to_string(path).map_err(|error| PlatformError::Read { + source: path.to_owned(), + detail: error.to_string(), + }) +} + +/// Parses the kernel release and build-version tail from `/proc/version`. +fn parse_proc_version(content: &str) -> Result<(String, String)> { + let mut tokens = content.split_whitespace(); + // Expected: "Linux version ..." + let release = tokens + .nth(2) + .ok_or_else(|| malformed("/proc/version", "missing kernel release"))? + .to_owned(); + let version = content + .find('#') + .map(|start| content[start..].trim_end().to_owned()) + .unwrap_or_default(); + Ok((release, version)) +} + +/// Parses `(cpu_model, sockets, cores_per_socket, microcode)` from cpuinfo. +fn parse_proc_cpuinfo(content: &str) -> Result<(String, u32, u32, String)> { + let mut model = None; + let mut microcode = None; + let mut cores_per_socket = None; + let mut sockets = BTreeSet::new(); + + for line in content.lines() { + let Some((key, value)) = line.split_once(':') else { + continue; + }; + let (key, value) = (key.trim(), value.trim()); + match key { + "model name" if model.is_none() => model = Some(value.to_owned()), + "microcode" if microcode.is_none() => microcode = Some(value.to_owned()), + "cpu cores" if cores_per_socket.is_none() => cores_per_socket = value.parse().ok(), + "physical id" => { + sockets.insert(value.to_owned()); + } + _ => {} + } + } + + let model = model.ok_or_else(|| malformed("/proc/cpuinfo", "missing model name"))?; + let microcode = microcode.ok_or_else(|| malformed("/proc/cpuinfo", "missing microcode"))?; + let cores_per_socket = + cores_per_socket.ok_or_else(|| malformed("/proc/cpuinfo", "missing cpu cores"))?; + let socket_count = u32::try_from(sockets.len()) + .map_err(|_| malformed("/proc/cpuinfo", "socket count overflow"))? + .max(1); + Ok((model, socket_count, cores_per_socket, microcode)) +} + +/// Parses `/proc//maps` into regions. +fn parse_process_maps(text: &str) -> Result> { + const SOURCE: &str = "/proc//maps"; + text.lines() + .map(|line| { + let mut fields = line.split_whitespace(); + let range = fields + .next() + .ok_or_else(|| malformed(SOURCE, "mapping lacks range"))?; + let permissions = fields + .next() + .ok_or_else(|| malformed(SOURCE, "mapping lacks permissions"))? + .to_owned(); + let offset = fields + .next() + .ok_or_else(|| malformed(SOURCE, "mapping lacks offset"))?; + let device = fields + .next() + .ok_or_else(|| malformed(SOURCE, "mapping lacks device"))?; + let inode = fields + .next() + .ok_or_else(|| malformed(SOURCE, "mapping lacks inode"))?; + let pathname = fields.next().map(str::to_owned); + Ok(MemoryRegion { + identity: format!( + "{range} {offset} {device} {inode} {}", + pathname.as_deref().unwrap_or("") + ), + permissions, + pathname, + }) + }) + .collect() +} + +/// Classifies a failed read of a `/proc/` file. A process reaped before +/// `open` is `ENOENT`, and one reaped between `open` and `read` is `ESRCH`. +fn process_read_error(pid: u32, source: String, error: &std::io::Error) -> PlatformError { + if error.kind() == std::io::ErrorKind::NotFound || error.raw_os_error() == Some(libc::ESRCH) { + PlatformError::ProcessGone(pid) + } else { + PlatformError::Read { + source, + detail: error.to_string(), + } + } +} + +/// A process with an empty map has released its address space. +fn regions_from_maps(pid: u32, text: &str) -> Result> { + let regions = parse_process_maps(text)?; + if regions.is_empty() { + return Err(PlatformError::NoAddressSpace(pid)); + } + Ok(regions) +} + +#[cfg(target_os = "linux")] +pub(crate) fn process_regions(pid: u32) -> Result> { + let path = format!("/proc/{pid}/maps"); + match fs::read_to_string(&path) { + Ok(text) => regions_from_maps(pid, &text), + Err(error) => Err(process_read_error(pid, path, &error)), + } +} + +/// Parses the peak resident set (`VmHWM`) in bytes from a +/// `/proc//status` document. +fn parse_peak_memory_bytes(status: &str) -> Result { + const SOURCE: &str = "/proc//status"; + let rest = status + .lines() + .find_map(|line| line.strip_prefix("VmHWM:")) + .ok_or_else(|| malformed(SOURCE, "VmHWM field not found"))?; + let kib: u64 = rest + .split_whitespace() + .next() + .ok_or_else(|| malformed(SOURCE, "VmHWM has no value"))? + .parse() + .map_err(|error| malformed(SOURCE, format!("VmHWM is not a number: {error}")))?; + kib.checked_mul(1024) + .ok_or_else(|| malformed(SOURCE, "VmHWM overflows")) +} + +/// The kernel prints `VmHWM` only for a task that holds an address space. An +/// exiting process drops it long before it becomes a zombie, and its state can +/// still read as running. A process whose main thread has exited reads the same +/// way while its other threads run, so this does not prove an exit. +fn has_no_address_space(status: &str) -> bool { + let has_field = |name| status.lines().any(|line| line.starts_with(name)); + has_field("State:") && !has_field("VmHWM:") +} + +fn peak_from_status(pid: u32, status: &str) -> Result { + if has_no_address_space(status) { + return Err(PlatformError::NoAddressSpace(pid)); + } + parse_peak_memory_bytes(status) +} + +#[cfg(target_os = "linux")] +pub(crate) fn peak_memory_bytes(pid: u32) -> Result { + let path = format!("/proc/{pid}/status"); + let status = + fs::read_to_string(&path).map_err(|error| process_read_error(pid, path, &error))?; + peak_from_status(pid, &status) +} + +/// Resets the kernel's high-water mark when it starts, so the final read is +/// the peak since then. +#[cfg(target_os = "linux")] +pub(crate) struct Window; + +#[cfg(target_os = "linux")] +impl Window { + pub(crate) fn start() -> Result { + const SOURCE: &str = "/proc/self/clear_refs"; + fs::write(SOURCE, "5").map_err(|error| PlatformError::Read { + source: SOURCE.to_owned(), + detail: error.to_string(), + })?; + Ok(Self) + } + + pub(crate) fn finish(self) -> Result { + peak_memory_bytes(std::process::id()) + } +} + +/// Builds the CPU identity from cpuinfo. A `model name` line selects the +/// original parser, so x86 hosts read exactly what they always have. A host +/// without it but with ARM identification fields takes its model from those +/// fields and its socket and core counts from the sysfs topology. +fn cpu_identity( + content: &str, + topology: impl FnOnce() -> Result>, +) -> Result<(String, u32, u32, String)> { + let has_model_name = content + .lines() + .filter_map(|line| line.split_once(':')) + .any(|(key, _)| key.trim() == "model name"); + if !has_model_name && let Some(model) = parse_arm_cpu_model(content) { + let (sockets, cores_per_socket) = sockets_and_cores(&topology()?)?; + return Ok((model, sockets, cores_per_socket, UNAVAILABLE.to_owned())); + } + parse_proc_cpuinfo(content) +} + +/// Names an ARM CPU from the identification fields cpuinfo does print. +fn parse_arm_cpu_model(content: &str) -> Option { + let mut implementer = None; + let mut part = None; + for (key, value) in content.lines().filter_map(|line| line.split_once(':')) { + let (key, value) = (key.trim(), value.trim()); + match key { + "CPU implementer" if implementer.is_none() => implementer = Some(value), + "CPU part" if part.is_none() => part = Some(value), + _ => {} + } + } + Some(format!("arm64 implementer={} part={}", implementer?, part?)) +} + +/// Counts sockets and cores per socket from one `(package id, core id)` pair +/// per online CPU. +fn sockets_and_cores(cpus: &[(i64, i64)]) -> Result<(u32, u32)> { + const SOURCE: &str = "/sys/devices/system/cpu topology"; + let mut packages: BTreeMap> = BTreeMap::new(); + for &(package, core) in cpus { + if package < 0 { + return Err(malformed(SOURCE, format!("negative package id {package}"))); + } + packages.entry(package).or_default().insert(core); + } + let mut counts = packages.values().map(BTreeSet::len); + let first = counts.next().ok_or_else(|| malformed(SOURCE, "no CPUs"))?; + if counts.any(|count| count != first) { + return Err(malformed(SOURCE, "packages have different core counts")); + } + let sockets = + u32::try_from(packages.len()).map_err(|_| malformed(SOURCE, "socket count overflow"))?; + let cores = u32::try_from(first).map_err(|_| malformed(SOURCE, "core count overflow"))?; + Ok((sockets, cores)) +} + +#[cfg(target_os = "linux")] +fn read_topology() -> Result> { + const ROOT: &str = "/sys/devices/system/cpu"; + let read_error = |error: std::io::Error| PlatformError::Read { + source: ROOT.to_owned(), + detail: error.to_string(), + }; + let mut cpus = Vec::new(); + for entry in fs::read_dir(ROOT).map_err(read_error)? { + let entry = entry.map_err(read_error)?; + let name = entry.file_name(); + let name = name.to_string_lossy(); + let Some(number) = name.strip_prefix("cpu") else { + continue; + }; + if number.is_empty() || !number.bytes().all(|byte| byte.is_ascii_digit()) { + continue; + } + let topology = entry.path().join("topology"); + // An offline CPU has no topology directory. + let (Ok(package), Ok(core)) = ( + fs::read_to_string(topology.join("physical_package_id")), + fs::read_to_string(topology.join("core_id")), + ) else { + continue; + }; + let parse = |text: &str, field: &str| { + text.trim() + .parse::() + .map_err(|_| malformed(ROOT, format!("{field} is not a number"))) + }; + cpus.push(( + parse(&package, "physical_package_id")?, + parse(&core, "core_id")?, + )); + } + Ok(cpus) +} + +/// Parses `SwapTotal` (KiB) from `/proc/meminfo`. +fn parse_proc_meminfo_swap(content: &str) -> Result { + for line in content.lines() { + if let Some(rest) = line.strip_prefix("SwapTotal:") { + let value = rest + .split_whitespace() + .next() + .ok_or_else(|| malformed("/proc/meminfo", "SwapTotal malformed"))?; + return value + .parse() + .map_err(|_| malformed("/proc/meminfo", "SwapTotal not a number")); + } + } + Err(malformed("/proc/meminfo", "missing SwapTotal")) +} + +fn parse_cpu_affinity(status: &str) -> Result { + status + .lines() + .find_map(|line| line.strip_prefix("Cpus_allowed_list:")) + .map(str::trim) + .filter(|value| !value.is_empty()) + .map(str::to_owned) + .ok_or_else(|| malformed("/proc/thread-self/status", "missing Cpus_allowed_list")) +} + +#[cfg(target_os = "linux")] +const NUMA_MASK_BITS: usize = 1024; +#[cfg(target_os = "linux")] +const NUMA_MASK_WORDS: usize = NUMA_MASK_BITS.div_ceil(libc::c_ulong::BITS as usize); + +#[cfg(target_os = "linux")] +fn format_memory_nodes(mask: &[libc::c_ulong]) -> String { + let mut ranges = Vec::new(); + let mut start = None; + for node in 0..NUMA_MASK_BITS { + let word = node / libc::c_ulong::BITS as usize; + let bit = node % libc::c_ulong::BITS as usize; + let is_set = mask[word] & (1 as libc::c_ulong) << bit != 0; + if is_set && start.is_none() { + start = Some(node); + } else if !is_set && start.is_some() { + let first = start.take().expect("checked above"); + let last = node - 1; + ranges.push(if first == last { + first.to_string() + } else { + format!("{first}-{last}") + }); + } + } + if let Some(first) = start { + let last = NUMA_MASK_BITS - 1; + ranges.push(if first == last { + first.to_string() + } else { + format!("{first}-{last}") + }); + } + ranges.join(",") +} + +#[cfg(target_os = "linux")] +fn decode_memory_policy(mode: libc::c_int, mask: &[libc::c_ulong]) -> Result<(String, String)> { + let mode_flags = libc::MPOL_F_STATIC_NODES | libc::MPOL_F_RELATIVE_NODES; + let base_mode = mode & !mode_flags; + let mut policy = match base_mode { + libc::MPOL_DEFAULT => "default".to_owned(), + libc::MPOL_PREFERRED => "preferred".to_owned(), + libc::MPOL_BIND => "bind".to_owned(), + libc::MPOL_INTERLEAVE => "interleave".to_owned(), + libc::MPOL_LOCAL => "local".to_owned(), + _ => { + return Err(malformed( + "get_mempolicy(2)", + format!("returned unsupported mode {mode}"), + )); + } + }; + if mode & libc::MPOL_F_STATIC_NODES != 0 { + policy.push_str("|static"); + } + if mode & libc::MPOL_F_RELATIVE_NODES != 0 { + policy.push_str("|relative"); + } + Ok((policy, format_memory_nodes(mask))) +} + +#[cfg(target_os = "linux")] +#[allow(unsafe_code)] +fn read_memory_policy() -> Result<(String, String)> { + let mut mode = 0 as libc::c_int; + let mut mask = [0 as libc::c_ulong; NUMA_MASK_WORDS]; + // SAFETY: get_mempolicy writes one c_int and at most NUMA_MASK_BITS into + // the two live buffers. flags=0 and addr=NULL request this thread's + // default policy, as required by get_mempolicy(2). + let status = unsafe { + libc::syscall( + libc::SYS_get_mempolicy, + std::ptr::addr_of_mut!(mode), + mask.as_mut_ptr(), + NUMA_MASK_BITS as libc::c_ulong, + std::ptr::null::(), + 0 as libc::c_ulong, + ) + }; + if status != 0 { + return Err(PlatformError::Read { + source: "get_mempolicy(2)".to_owned(), + detail: std::io::Error::last_os_error().to_string(), + }); + } + decode_memory_policy(mode, &mask) +} + +#[cfg(target_os = "linux")] +pub(crate) fn machine_facts() -> Result { + let version = read_evidence("/proc/version")?; + let cpuinfo = read_evidence("/proc/cpuinfo")?; + let governor = cpu_governor().unwrap_or_else(|_| "unknown".to_owned()); + + let (kernel_release, kernel_version) = parse_proc_version(&version)?; + let (cpu_model, sockets, cores_per_socket, microcode) = cpu_identity(&cpuinfo, read_topology)?; + let swap_total_kib = swap_total_kib()?; + + Ok(MachineFacts { + kernel_release, + kernel_version, + cpu_model, + sockets, + cores_per_socket, + microcode, + governor, + swap_total_kib, + }) +} + +#[cfg(target_os = "linux")] +pub(crate) fn cpu_governor() -> Result { + read_evidence(crate::LINUX_GOVERNOR_SOURCE).map(|text| text.trim().to_owned()) +} + +#[cfg(target_os = "linux")] +pub(crate) fn swap_total_kib() -> Result { + parse_proc_meminfo_swap(&read_evidence("/proc/meminfo")?) +} + +#[cfg(target_os = "linux")] +pub(crate) fn cpu_affinity() -> Result { + parse_cpu_affinity(&read_evidence("/proc/thread-self/status")?) +} + +#[cfg(target_os = "linux")] +pub(crate) fn thread_placement() -> Result { + let cpu_affinity = cpu_affinity()?; + let (memory_policy, memory_nodes) = read_memory_policy()?; + Ok(ThreadPlacement { + cpu_affinity, + memory_policy, + memory_nodes, + }) +} + +/// Parses a kernel CPU list such as `0-3,5,7-9`. +fn parse_cpu_list(text: &str) -> Result> { + const SOURCE: &str = "CPU list"; + let number = |token: &str| -> Result { + token + .trim() + .parse() + .map_err(|_| malformed(SOURCE, format!("`{token}` is not a CPU number"))) + }; + let mut cpus = BTreeSet::new(); + for part in text + .split(',') + .map(str::trim) + .filter(|part| !part.is_empty()) + { + if let Some((first, last)) = part.split_once('-') { + cpus.extend(number(first)?..=number(last)?); + } else { + cpus.insert(number(part)?); + } + } + Ok(cpus) +} + +#[cfg(target_os = "linux")] +pub(crate) fn online_cpu_list() -> Result { + read_evidence("/sys/devices/system/cpu/online").map(|text| text.trim().to_owned()) +} + +/// The scaling governor of each online CPU, sorted. A CPU without cpufreq +/// reports an empty string. +#[cfg(target_os = "linux")] +pub(crate) fn cpu_governors() -> Result> { + const ROOT: &str = "/sys/devices/system/cpu"; + let read_error = |error: std::io::Error| PlatformError::Read { + source: ROOT.to_owned(), + detail: error.to_string(), + }; + let online = parse_cpu_list(&online_cpu_list()?)?; + let mut governors = Vec::new(); + for entry in fs::read_dir(ROOT).map_err(read_error)? { + let entry = entry.map_err(read_error)?; + let name = entry.file_name(); + let name = name.to_string_lossy(); + let Some(digits) = name.strip_prefix("cpu") else { + continue; + }; + if digits.is_empty() || !digits.bytes().all(|byte| byte.is_ascii_digit()) { + continue; + } + let number: u32 = digits + .parse() + .map_err(|_| malformed(ROOT, format!("`{name}` is not a CPU directory")))?; + if online.contains(&number) { + let path = entry.path().join("cpufreq/scaling_governor"); + governors.push( + fs::read_to_string(path) + .unwrap_or_default() + .trim() + .to_owned(), + ); + } + } + governors.sort(); + Ok(governors) +} + +/// Runs `numactl --show` and returns its output, or `None` when the tool is +/// missing, fails, or prints something that is not UTF-8. +#[cfg(target_os = "linux")] +fn numactl_show() -> Option { + Command::new("numactl") + .arg("--show") + .output() + .ok() + .filter(|output| output.status.success()) + .and_then(|output| String::from_utf8(output.stdout).ok()) +} + +/// The memory policy `numactl --show` reports, or [`UNAVAILABLE`] when it does +/// not say. +#[cfg(target_os = "linux")] +pub(crate) fn numa_policy_mode() -> String { + numactl_show() + .and_then(|text| { + text.lines().find_map(|line| { + line.strip_prefix("policy:") + .map(str::trim) + .map(str::to_owned) + }) + }) + .unwrap_or_else(|| UNAVAILABLE.to_owned()) +} + +/// The memory nodes this process is bound to: the `nodebind` line of +/// `numactl --show`, else `Mems_allowed_list` from `/proc/self/status`. +#[cfg(target_os = "linux")] +pub(crate) fn numa_memory_nodes() -> Result> { + if let Some(text) = numactl_show() + && let Some(nodes) = text.lines().find_map(|line| line.strip_prefix("nodebind:")) + { + return parse_whitespace_list(nodes.trim()); + } + let status = read_evidence("/proc/self/status")?; + status + .lines() + .find_map(|line| line.strip_prefix("Mems_allowed_list:")) + .map(str::trim) + .ok_or_else(|| malformed("/proc/self/status", "missing Mems_allowed_list")) + .and_then(parse_node_range_list) +} + +fn node_number(token: &str) -> Result { + token.parse().map_err(|_| { + malformed( + "memory-node list", + format!("`{token}` is not a node number"), + ) + }) +} + +/// Parses a kernel node list such as `0`, `0-3`, or `0,2-3`. +fn parse_node_range_list(text: &str) -> Result> { + const SOURCE: &str = "memory-node list"; + if text.trim().is_empty() { + return Err(malformed(SOURCE, "memory-node list is empty")); + } + let mut nodes = Vec::new(); + for token in text.split(',').map(str::trim) { + if token.is_empty() { + return Err(malformed( + SOURCE, + "memory-node list contains an empty token", + )); + } + if let Some((start, end)) = token.split_once('-') { + let (start, end) = (node_number(start)?, node_number(end)?); + if start > end { + return Err(malformed( + SOURCE, + format!("reversed memory-node range: {token}"), + )); + } + nodes.extend(start..=end); + } else { + nodes.push(node_number(token)?); + } + } + Ok(nodes) +} + +/// Parses the node numbers on a `numactl --show` `nodebind:` line. +fn parse_whitespace_list(text: &str) -> Result> { + text.split_whitespace().map(node_number).collect() +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parse_proc_version_extracts_release_and_tail() { + let content = "Linux version 7.0.0-28-generic (buildd@lcy02) (gcc 15) #28-Ubuntu SMP PREEMPT_DYNAMIC Sun Jun 21 01:01:36 UTC 2026\n"; + let (release, version) = parse_proc_version(content).unwrap(); + assert_eq!(release, "7.0.0-28-generic"); + assert!(version.starts_with("#28-Ubuntu SMP PREEMPT_DYNAMIC")); + } + + #[test] + fn parse_proc_cpuinfo_counts_sockets_and_cores() { + let content = "model name\t: Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz\nmicrocode\t: 0x2007006\ncpu cores\t: 20\nphysical id\t: 0\n\nphysical id\t: 1\ncpu cores\t: 20\n"; + let (model, sockets, cores, micro) = parse_proc_cpuinfo(content).unwrap(); + assert_eq!(model, "Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz"); + assert_eq!(sockets, 2); + assert_eq!(cores, 20); + assert_eq!(micro, "0x2007006"); + } + + const ARM_CPUINFO: &str = "processor\t: 0\nBogoMIPS\t: 48.00\nFeatures\t: fp asimd\nCPU implementer\t: 0x61\nCPU architecture: 8\nCPU variant\t: 0x0\nCPU part\t: 0x000\nCPU revision\t: 0\n"; + const X86_CPUINFO: &str = "model name\t: Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz\nmicrocode\t: 0x2007006\ncpu cores\t: 20\nphysical id\t: 0\n\nphysical id\t: 1\ncpu cores\t: 20\n"; + + #[test] + fn arm_cpu_model_comes_from_identification_fields() { + assert_eq!( + parse_arm_cpu_model(ARM_CPUINFO).as_deref(), + Some("arm64 implementer=0x61 part=0x000") + ); + assert_eq!(parse_arm_cpu_model("CPU implementer\t: 0x61\n"), None); + assert_eq!(parse_arm_cpu_model(X86_CPUINFO), None); + } + + #[test] + fn cpu_identity_keeps_the_x86_path_and_its_errors() { + let unreached = || unreachable!("an x86 host never reads the topology"); + assert_eq!( + cpu_identity(X86_CPUINFO, unreached).unwrap(), + parse_proc_cpuinfo(X86_CPUINFO).unwrap() + ); + let no_microcode = "model name\t: Intel(R) Xeon(R)\ncpu cores\t: 4\n"; + assert!( + cpu_identity(no_microcode, unreached) + .unwrap_err() + .to_string() + .contains("missing microcode") + ); + assert!( + cpu_identity("", unreached) + .unwrap_err() + .to_string() + .contains("missing model name") + ); + } + + #[test] + fn cpu_identity_reads_the_topology_for_arm() { + let cpus = (0..8).map(|core| (0, core)).collect::>(); + assert_eq!( + cpu_identity(ARM_CPUINFO, || Ok(cpus)).unwrap(), + ( + "arm64 implementer=0x61 part=0x000".to_owned(), + 1, + 8, + UNAVAILABLE.to_owned() + ) + ); + assert!(cpu_identity(ARM_CPUINFO, || Ok(Vec::new())).is_err()); + } + + #[test] + fn counts_sockets_and_cores_per_socket() { + assert_eq!( + sockets_and_cores(&[(0, 0), (0, 1), (0, 2), (0, 3)]).unwrap(), + (1, 4) + ); + assert_eq!( + sockets_and_cores(&[(0, 0), (0, 1), (1, 0), (1, 1)]).unwrap(), + (2, 2) + ); + // SMT siblings share a core id and must not be counted twice. + assert_eq!( + sockets_and_cores(&[(0, 0), (0, 0), (0, 1), (0, 1)]).unwrap(), + (1, 2) + ); + for bad in [ + &[][..], + &[(0, 0), (0, 1), (1, 0)][..], + &[(-1, 0), (-1, 1)][..], + ] { + assert!(sockets_and_cores(bad).is_err(), "{bad:?}"); + } + } + + #[test] + fn parses_process_maps_into_regions() { + let text = "55d0c8a00000-55d0c8a21000 r-xp 00000000 fd:01 1048 /usr/bin/cat\n7f1c2a000000-7f1c2a021000 rwxp 00000000 00:00 0\n"; + let regions = parse_process_maps(text).unwrap(); + assert_eq!(regions.len(), 2); + assert_eq!( + regions[0], + MemoryRegion { + identity: "55d0c8a00000-55d0c8a21000 00000000 fd:01 1048 /usr/bin/cat".to_owned(), + permissions: "r-xp".to_owned(), + pathname: Some("/usr/bin/cat".to_owned()), + } + ); + assert_eq!(regions[1].permissions, "rwxp"); + assert_eq!(regions[1].pathname, None); + assert!(parse_process_maps("55d0c8a00000-55d0c8a21000\n").is_err()); + } + + #[test] + fn an_empty_map_is_no_address_space() { + assert_eq!( + regions_from_maps(7, ""), + Err(PlatformError::NoAddressSpace(7)) + ); + } + + #[test] + fn a_process_reaped_around_a_read_is_gone() { + for errno in [libc::ENOENT, libc::ESRCH] { + let error = std::io::Error::from_raw_os_error(errno); + assert_eq!( + process_read_error(7, "/proc/7/maps".to_owned(), &error), + PlatformError::ProcessGone(7) + ); + } + let denied = std::io::Error::from_raw_os_error(libc::EACCES); + assert!(matches!( + process_read_error(7, "/proc/7/maps".to_owned(), &denied), + PlatformError::Read { .. } + )); + } + + #[cfg(target_os = "linux")] + #[test] + fn reads_this_process_regions() { + let regions = process_regions(std::process::id()).unwrap(); + assert!( + regions + .iter() + .any(|region| region.permissions.contains('x') && region.pathname.is_some()) + ); + assert!( + regions + .iter() + .all(|region| !region.permissions.contains("rwx")) + ); + } + + #[test] + fn peak_memory_is_the_high_water_mark_not_the_current_set() { + let status = "VmPeak:\t 900000 kB\nVmHWM:\t 32000 kB\nVmRSS:\t 20000 kB\n"; + assert_eq!(parse_peak_memory_bytes(status).unwrap(), 32_768_000); + for bad in [ + "VmRSS:\t 20000 kB\n", + "VmHWM:\n", + "VmHWM:\t abc kB\n", + "VmHWM:\t 18446744073709551615 kB\n", + ] { + assert!(parse_peak_memory_bytes(bad).is_err(), "{bad:?}"); + } + } + + #[test] + fn a_status_without_vmhwm_has_no_address_space() { + // Measured on Linux 6.8: an exiting process reads `R (running)` with no + // `VmHWM` line for most of its exit, before it becomes a zombie. + for state in ["R (running)", "D (disk sleep)", "Z (zombie)"] { + let status = format!("Name:\tcat\nState:\t{state}\nPid:\t7\n"); + assert_eq!( + peak_from_status(7, &status), + Err(PlatformError::NoAddressSpace(7)), + "{state}" + ); + } + } + + #[test] + fn a_live_process_reports_its_peak() { + let status = "State:\tS (sleeping)\nVmHWM:\t 32000 kB\n"; + assert_eq!(peak_from_status(7, status), Ok(32_768_000)); + } + + #[test] + fn a_status_without_a_state_is_malformed_not_gone() { + assert!(matches!( + peak_from_status(7, "Name:\tcat\n"), + Err(PlatformError::Malformed { .. }) + )); + } + + #[test] + fn parse_proc_meminfo_reads_swap_total() { + let content = "MemTotal: 791180784 kB\nSwapTotal: 0 kB\n"; + assert_eq!(parse_proc_meminfo_swap(content).unwrap(), 0); + } + + #[test] + fn parses_thread_cpu_affinity() { + let status = + "Name:\tperf_budget\nCpus_allowed:\t00000000,000fffff\nCpus_allowed_list:\t0-19\n"; + assert_eq!(parse_cpu_affinity(status).unwrap(), "0-19"); + assert!(parse_cpu_affinity("Name:\tperf_budget\n").is_err()); + } + + #[cfg(target_os = "linux")] + #[test] + fn decodes_memory_policy_and_node_ranges() { + let mut mask = [0 as libc::c_ulong; NUMA_MASK_WORDS]; + mask[0] = 0b1_1111; + assert_eq!( + decode_memory_policy(libc::MPOL_BIND, &mask).unwrap(), + ("bind".to_owned(), "0-4".to_owned()) + ); + assert!(decode_memory_policy(-1, &mask).is_err()); + } + + #[test] + fn node_lists_parse_singletons_ranges_and_mixes() { + assert_eq!(parse_node_range_list("0").unwrap(), [0]); + assert_eq!(parse_node_range_list("0-3").unwrap(), [0, 1, 2, 3]); + assert_eq!(parse_node_range_list("0,2-3").unwrap(), [0, 2, 3]); + assert_eq!(parse_node_range_list(" 1 , 3 ").unwrap(), [1, 3]); + } + + #[test] + fn malformed_node_lists_are_rejected() { + for text in ["", " ", "0,", ",0", "3-1", "a", "0-b", "-1", "0--1"] { + assert!(parse_node_range_list(text).is_err(), "{text:?}"); + } + } + + #[test] + fn nodebind_lines_parse_as_whitespace_lists() { + assert_eq!(parse_whitespace_list("0 1").unwrap(), [0, 1]); + assert_eq!(parse_whitespace_list("0").unwrap(), [0]); + assert_eq!(parse_whitespace_list("").unwrap(), Vec::::new()); + assert!(parse_whitespace_list("0 x").is_err()); + } + + #[test] + fn cpu_lists_parse_ranges_and_singletons() { + let expected: BTreeSet = [0, 1, 2, 3, 5, 7, 8, 9].into(); + assert_eq!(parse_cpu_list("0-3,5,7-9").unwrap(), expected); + assert_eq!(parse_cpu_list("0-79").unwrap().len(), 80); + assert_eq!(parse_cpu_list(" 4 ,").unwrap(), BTreeSet::from([4])); + assert!(parse_cpu_list("").unwrap().is_empty()); + } + + #[test] + fn malformed_cpu_lists_are_rejected() { + for text in ["a", "0-b", "-1", "1,x"] { + assert!(parse_cpu_list(text).is_err(), "{text:?}"); + } + } + + #[cfg(target_os = "linux")] + #[test] + fn there_is_one_governor_per_online_cpu() { + let online = parse_cpu_list(&online_cpu_list().unwrap()).unwrap(); + assert!(!online.is_empty()); + assert_eq!(cpu_governors().unwrap().len(), online.len()); + } +} diff --git a/crates/bamts-platform/src/macos.rs b/crates/bamts-platform/src/macos.rs new file mode 100644 index 00000000..f38ec86d --- /dev/null +++ b/crates/bamts-platform/src/macos.rs @@ -0,0 +1,800 @@ +//! macOS backend: `sysctl` and `libproc`. + +#[cfg(target_os = "macos")] +use std::mem::{MaybeUninit, size_of}; +#[cfg(target_os = "macos")] +use std::process::Command; +#[cfg(target_os = "macos")] +use std::sync::Arc; +#[cfg(target_os = "macos")] +use std::sync::atomic::{AtomicBool, Ordering}; +#[cfg(target_os = "macos")] +use std::thread; +#[cfg(target_os = "macos")] +use std::time::Duration; + +use crate::PlatformError; +#[cfg(target_os = "macos")] +use crate::{MachineFacts, MemoryRegion, ThreadPlacement, UNAVAILABLE}; + +type Result = std::result::Result; + +fn malformed(source: &str, detail: impl Into) -> PlatformError { + PlatformError::Malformed { + source: source.to_owned(), + detail: detail.into(), + } +} + +#[cfg(target_os = "macos")] +const SYSCTL: &str = "/usr/sbin/sysctl"; + +#[cfg(target_os = "macos")] +fn sysctl(key: &str) -> Result { + let source = format!("sysctl {key}"); + let output = Command::new(SYSCTL) + .args(["-n", key]) + .output() + .map_err(|error| PlatformError::Read { + source: source.clone(), + detail: error.to_string(), + })?; + if !output.status.success() { + return Err(PlatformError::Read { + source, + detail: format!("exited with {}", output.status), + }); + } + Ok(String::from_utf8_lossy(&output.stdout).trim().to_owned()) +} + +#[cfg(target_os = "macos")] +fn sysctl_u32(key: &str) -> Result { + sysctl(key)? + .parse() + .map_err(|_| malformed(&format!("sysctl {key}"), "not a number")) +} + +/// Parses the swap total in KiB from `sysctl vm.swapusage`, such as +/// `total = 4096.00M used = 3601.38M free = 494.62M (encrypted)`. +fn parse_swap_total_kib(text: &str) -> Result { + const SOURCE: &str = "sysctl vm.swapusage"; + let token = text + .split("total =") + .nth(1) + .and_then(|rest| rest.split_whitespace().next()) + .ok_or_else(|| malformed(SOURCE, "missing total"))?; + let split = token.len().saturating_sub(1); + if !token.is_char_boundary(split) { + return Err(malformed(SOURCE, "total has no unit")); + } + let (number, unit) = token.split_at(split); + let value: f64 = number + .parse() + .map_err(|_| malformed(SOURCE, "total is not a number"))?; + if !value.is_finite() || value < 0.0 { + return Err(malformed(SOURCE, "total is out of range")); + } + let kib_per_unit = match unit { + "K" => 1.0, + "M" => 1024.0, + "G" => 1024.0 * 1024.0, + _ => return Err(malformed(SOURCE, format!("unknown unit `{unit}`"))), + }; + Ok((value * kib_per_unit).round() as u64) +} + +#[cfg(target_os = "macos")] +pub(crate) fn machine_facts() -> Result { + let packages = sysctl_u32("hw.packages")?; + let physical = sysctl_u32("hw.physicalcpu")?; + if packages == 0 || physical % packages != 0 { + return Err(malformed( + "sysctl hw.packages", + format!("{physical} physical cores do not divide across {packages} packages"), + )); + } + Ok(MachineFacts { + kernel_release: sysctl("kern.osrelease")?, + kernel_version: sysctl("kern.version")?, + cpu_model: sysctl("machdep.cpu.brand_string")?, + sockets: packages, + cores_per_socket: physical / packages, + microcode: UNAVAILABLE.to_owned(), + governor: UNAVAILABLE.to_owned(), + swap_total_kib: swap_total_kib()?, + }) +} + +#[cfg(target_os = "macos")] +pub(crate) fn swap_total_kib() -> Result { + parse_swap_total_kib(&sysctl("vm.swapusage")?) +} + +#[cfg(target_os = "macos")] +pub(crate) fn thread_placement() -> Result { + Ok(ThreadPlacement { + cpu_affinity: UNAVAILABLE.to_owned(), + memory_policy: UNAVAILABLE.to_owned(), + memory_nodes: UNAVAILABLE.to_owned(), + }) +} + +/// Formats the CPUs `0` to `count - 1` as a kernel CPU list, such as `0-13`. +fn cpu_list_for(count: u32) -> Result { + match count { + 0 => Err(malformed("sysctl hw.logicalcpu", "no logical CPUs")), + 1 => Ok("0".to_owned()), + _ => Ok(format!("0-{}", count - 1)), + } +} + +#[cfg(target_os = "macos")] +pub(crate) fn online_cpu_list() -> Result { + cpu_list_for(sysctl_u32("hw.logicalcpu")?) +} + +/// Formats Mach VM protection bits in the `rwxp` style of `/proc//maps`. +fn permissions_string(protection: u32) -> String { + const READ: u32 = 0x01; + const WRITE: u32 = 0x02; + const EXECUTE: u32 = 0x04; + let flag = |bit: u32, letter: char| if protection & bit != 0 { letter } else { '-' }; + format!( + "{}{}{}p", + flag(READ, 'r'), + flag(WRITE, 'w'), + flag(EXECUTE, 'x') + ) +} + +/// `PROC_PIDREGIONPATHINFO` in ``. +#[cfg(target_os = "macos")] +const PROC_PIDREGIONPATHINFO: libc::c_int = 8; + +/// Mirrors `struct proc_regioninfo` in ``. The `libc` crate +/// declares `proc_pidinfo` and `vnode_info_path` but not the region structs. +#[cfg(target_os = "macos")] +#[repr(C)] +struct ProcRegionInfo { + pri_protection: u32, + pri_max_protection: u32, + pri_inheritance: u32, + pri_flags: u32, + pri_offset: u64, + pri_behavior: u32, + pri_user_wired_count: u32, + pri_user_tag: u32, + pri_pages_resident: u32, + pri_pages_shared_now_private: u32, + pri_pages_swapped_out: u32, + pri_pages_dirtied: u32, + pri_ref_count: u32, + pri_shadow_depth: u32, + pri_share_mode: u32, + pri_private_pages_resident: u32, + pri_shared_pages_resident: u32, + pri_obj_id: u32, + pri_depth: u32, + pri_address: u64, + pri_size: u64, +} + +/// Mirrors `struct proc_regionwithpathinfo` in ``. +#[cfg(target_os = "macos")] +#[repr(C)] +struct ProcRegionWithPathInfo { + prp_prinfo: ProcRegionInfo, + prp_vip: libc::vnode_info_path, +} + +// Sizes measured from the macOS SDK headers with `sizeof`. A layout drift +// fails the build here, not at the kernel boundary. +#[cfg(target_os = "macos")] +const _: () = assert!(size_of::() == 96); +#[cfg(target_os = "macos")] +const _: () = assert!(size_of::() == 1272); + +#[cfg(target_os = "macos")] +const REGION_SOURCE: &str = "proc_pidinfo(PROC_PIDREGIONPATHINFO)"; + +/// Decodes `vip_path`, which `libc` declares as an array of arrays. +#[cfg(target_os = "macos")] +fn region_path(field: &[[libc::c_char; 32]; 32]) -> Result { + let field = field.as_flattened(); + let length = field + .iter() + .position(|byte| *byte == 0) + .ok_or_else(|| malformed(REGION_SOURCE, "region path is not NUL-terminated"))?; + let bytes: Vec = field[..length].iter().map(|byte| *byte as u8).collect(); + Ok(String::from_utf8_lossy(&bytes).into_owned()) +} + +#[cfg(target_os = "macos")] +fn region_from(info: &ProcRegionWithPathInfo) -> Result { + let path = region_path(&info.prp_vip.vip_path)?; + let region = &info.prp_prinfo; + let end = region + .pri_address + .checked_add(region.pri_size) + .ok_or_else(|| malformed(REGION_SOURCE, "region end overflows the address space"))?; + Ok(MemoryRegion { + identity: format!( + "{:x}-{end:x} {:x} {:x} {path}", + region.pri_address, region.pri_offset, region.pri_obj_id + ), + permissions: permissions_string(region.pri_protection), + pathname: (!path.is_empty()).then_some(path), + }) +} + +/// One answer from the kernel, as a walk needs it. +#[cfg(target_os = "macos")] +struct WalkedRegion { + start: u64, + size: u64, + region: MemoryRegion, +} + +/// Reads the first region at or after `address`, or `None` when the kernel has +/// no region there. +#[cfg(target_os = "macos")] +fn region_at(raw_pid: libc::c_int, pid: u32, address: u64) -> Result> { + const BUFFER_SIZE: libc::c_int = 1272; + let mut info = MaybeUninit::::zeroed(); + // SAFETY: `info` is a live, writable buffer of exactly `BUFFER_SIZE` bytes, + // the size the kernel is told. `proc_pidinfo` writes at most that many + // bytes and keeps no pointer to it after returning. + #[allow(unsafe_code)] + let written = unsafe { + libc::proc_pidinfo( + raw_pid, + PROC_PIDREGIONPATHINFO, + address, + info.as_mut_ptr().cast::(), + BUFFER_SIZE, + ) + }; + if written != BUFFER_SIZE { + let error = std::io::Error::last_os_error(); + return match (written, error.raw_os_error()) { + (0, Some(libc::EINVAL)) => Ok(None), + // Measured on macOS 27: never seen on a live process, in 363 + // verdicts taken while one execed. + (0, Some(libc::ESRCH)) => Err(PlatformError::ProcessGone(pid)), + (0, _) => Err(PlatformError::Read { + source: REGION_SOURCE.to_owned(), + detail: error.to_string(), + }), + _ => Err(malformed( + REGION_SOURCE, + format!("short read of {written} bytes, expected {BUFFER_SIZE}"), + )), + }; + } + // SAFETY: the buffer was zeroed and the kernel then wrote it. Every field + // is an integer or an array of integers, so any bytes are a valid value. + #[allow(unsafe_code)] + let info = unsafe { info.assume_init() }; + Ok(Some(WalkedRegion { + start: info.prp_prinfo.pri_address, + size: info.prp_prinfo.pri_size, + region: region_from(&info)?, + })) +} + +/// Walks regions through `read_at` until it runs out. +/// +/// Running out is ambiguous. It ends every walk, but a process also reads this +/// way for a moment when it exits or execs: the kernel has no region to report +/// until the process is gone or has its new image. A process with an address +/// space always has a region at or after address 0, so reading there again +/// catches an address space that is already gone. +/// +/// It cannot catch one still being torn down, or an exec that lands between two +/// reads, so a list taken during either can be partial or span both images. +#[cfg(target_os = "macos")] +fn walk_regions( + pid: u32, + mut read_at: impl FnMut(u64) -> Result>, +) -> Result> { + let mut regions = Vec::new(); + let mut address = 0_u64; + while let Some(found) = read_at(address)? { + let next = found + .start + .checked_add(found.size) + .filter(|next| *next > address) + .ok_or_else(|| malformed(REGION_SOURCE, "region walk does not advance"))?; + regions.push(found.region); + address = next; + } + if regions.is_empty() || read_at(0)?.is_none() { + return Err(PlatformError::NoAddressSpace(pid)); + } + Ok(regions) +} + +#[cfg(target_os = "macos")] +pub(crate) fn process_regions(pid: u32) -> Result> { + let raw_pid = libc::c_int::try_from(pid).map_err(|_| { + malformed( + REGION_SOURCE, + format!("pid {pid} does not fit the host pid type"), + ) + })?; + walk_regions(pid, |address| region_at(raw_pid, pid, address)) +} + +#[cfg(target_os = "macos")] +const RUSAGE_SOURCE: &str = "proc_pid_rusage(RUSAGE_INFO_V4)"; + +/// Reads the resource usage record of a process. +#[cfg(target_os = "macos")] +fn rusage_v4(pid: u32) -> Result { + let raw_pid = libc::c_int::try_from(pid).map_err(|_| { + malformed( + RUSAGE_SOURCE, + format!("pid {pid} does not fit the host pid type"), + ) + })?; + let mut info = MaybeUninit::::zeroed(); + // SAFETY: `info` is a live, writable `rusage_info_v4`, the record that + // `RUSAGE_INFO_V4` asks the kernel to fill. The pointer is used only for + // the duration of the call. + #[allow(unsafe_code)] + let status = unsafe { + libc::proc_pid_rusage( + raw_pid, + libc::RUSAGE_INFO_V4, + info.as_mut_ptr().cast::(), + ) + }; + if status != 0 { + let error = std::io::Error::last_os_error(); + return Err(if error.raw_os_error() == Some(libc::ESRCH) { + PlatformError::ProcessGone(pid) + } else { + PlatformError::Read { + source: RUSAGE_SOURCE.to_owned(), + detail: error.to_string(), + } + }); + } + // SAFETY: the buffer was zeroed and the kernel then wrote it. Every field + // is an integer or an array of integers, so any bytes are a valid value. + #[allow(unsafe_code)] + Ok(unsafe { info.assume_init() }) +} + +#[cfg(target_os = "macos")] +pub(crate) fn peak_memory_bytes(pid: u32) -> Result { + Ok(rusage_v4(pid)?.ri_lifetime_max_phys_footprint) +} + +/// Samples the physical footprint of this process while the window is open. +/// The public SDK has no way to reset the footprint peak, so a spike shorter +/// than the sampling interval can be missed. +#[cfg(target_os = "macos")] +pub(crate) struct Window { + stop: Arc, + sampler: Option>, +} + +#[cfg(target_os = "macos")] +impl Window { + const INTERVAL: Duration = Duration::from_millis(1); + + pub(crate) fn start() -> Result { + let pid = std::process::id(); + let first = rusage_v4(pid)?.ri_phys_footprint; + let stop = Arc::new(AtomicBool::new(false)); + let flag = Arc::clone(&stop); + let sampler = thread::Builder::new() + .name("bamts-peak-window".to_owned()) + .spawn(move || { + let mut peak = first; + while !flag.load(Ordering::Acquire) { + if let Ok(info) = rusage_v4(pid) { + peak = peak.max(info.ri_phys_footprint); + } + thread::sleep(Self::INTERVAL); + } + peak + }) + .map_err(|error| PlatformError::Read { + source: "peak memory sampler thread".to_owned(), + detail: error.to_string(), + })?; + Ok(Self { + stop, + sampler: Some(sampler), + }) + } + + pub(crate) fn finish(mut self) -> Result { + self.stop.store(true, Ordering::Release); + let sampled = self + .sampler + .take() + .expect("the sampler is present until the window finishes") + .join() + .map_err(|_| PlatformError::Read { + source: "peak memory sampler thread".to_owned(), + detail: "the sampler panicked".to_owned(), + })?; + let last = rusage_v4(std::process::id())?.ri_phys_footprint; + Ok(sampled.max(last)) + } +} + +#[cfg(target_os = "macos")] +impl Drop for Window { + fn drop(&mut self) { + self.stop.store(true, Ordering::Release); + if let Some(sampler) = self.sampler.take() { + let _ = sampler.join(); + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn parses_swap_total_in_every_unit() { + let text = "total = 4096.00M used = 3601.38M free = 494.62M (encrypted)"; + assert_eq!(parse_swap_total_kib(text).unwrap(), 4_194_304); + assert_eq!( + parse_swap_total_kib("total = 0.00M used = 0.00M").unwrap(), + 0 + ); + assert_eq!( + parse_swap_total_kib("total = 1.50G used = 0.00M").unwrap(), + 1_572_864 + ); + assert_eq!( + parse_swap_total_kib("total = 512.00K used = 0.00K").unwrap(), + 512 + ); + } + + #[test] + fn rejects_malformed_swap_totals() { + for text in [ + "", + "nonsense", + "total =", + "total = 12", + "total = 12X", + "total = -3.00M", + "total = abcM", + ] { + assert!(parse_swap_total_kib(text).is_err(), "{text:?}"); + } + } + + #[cfg(target_os = "macos")] + #[test] + fn reads_live_machine_facts() { + let facts = machine_facts().unwrap(); + let release = Command::new("/usr/bin/uname").arg("-r").output().unwrap(); + assert_eq!( + facts.kernel_release, + String::from_utf8_lossy(&release.stdout).trim() + ); + assert!(facts.kernel_version.starts_with("Darwin Kernel Version")); + assert!(!facts.cpu_model.is_empty()); + assert!(facts.sockets >= 1); + assert!(facts.cores_per_socket >= 1); + assert_eq!(facts.microcode, UNAVAILABLE); + assert_eq!(facts.governor, UNAVAILABLE); + } + + #[test] + fn formats_protection_bits_like_proc_maps() { + assert_eq!(permissions_string(0), "---p"); + assert_eq!(permissions_string(1), "r--p"); + assert_eq!(permissions_string(3), "rw-p"); + assert_eq!(permissions_string(5), "r-xp"); + assert_eq!(permissions_string(7), "rwxp"); + } + + #[cfg(target_os = "macos")] + fn region(start: u64, size: u64) -> Option { + Some(WalkedRegion { + start, + size, + region: MemoryRegion { + identity: String::new(), + permissions: permissions_string(0), + pathname: None, + }, + }) + } + + #[cfg(target_os = "macos")] + #[test] + fn reads_a_region_path_across_the_inner_arrays() { + let path = format!("/{}", "a".repeat(63)); + let mut field = [[0; 32]; 32]; + for (slot, byte) in field.as_flattened_mut().iter_mut().zip(path.bytes()) { + *slot = byte as libc::c_char; + } + assert_eq!(region_path(&field).unwrap(), path); + } + + #[cfg(target_os = "macos")] + #[test] + fn a_region_path_without_a_terminator_is_malformed() { + let field = [[b'a' as libc::c_char; 32]; 32]; + assert!(matches!( + region_path(&field), + Err(PlatformError::Malformed { .. }) + )); + } + + /// The anonymous-executable check tests only `is_none`, so a path that is + /// not UTF-8 must stay a path. + #[cfg(target_os = "macos")] + #[test] + fn a_region_path_that_is_not_utf8_is_still_a_path() { + let mut field = [[0; 32]; 32]; + field[0][0] = 0xFF_u8 as libc::c_char; + assert_eq!(region_path(&field).unwrap(), "\u{FFFD}"); + } + + /// Runs a walk against scripted kernel answers, and returns its result with + /// the addresses it asked for. + #[cfg(target_os = "macos")] + fn scripted_walk( + script: Vec>>, + ) -> (Result>, Vec) { + let mut asked = Vec::new(); + let mut answers = script.into_iter(); + let result = walk_regions(7, |address| { + asked.push(address); + answers.next().expect("the walk read past its script") + }); + (result, asked) + } + + #[cfg(target_os = "macos")] + #[test] + fn a_finished_walk_confirms_the_process_still_has_regions() { + let (result, asked) = scripted_walk(vec![ + Ok(region(0, 16)), + Ok(region(16, 16)), + Ok(None), + Ok(region(0, 16)), + ]); + assert_eq!(result.unwrap().len(), 2); + assert_eq!(asked, [0, 16, 32, 0]); + } + + #[cfg(target_os = "macos")] + #[test] + fn a_walk_cut_short_by_an_exit_is_not_a_complete_list() { + // The address space went away after the first region. Measured on + // macOS 27: a walk ended this way after 6 regions of a larger map. + let (result, _) = scripted_walk(vec![Ok(region(0, 16)), Ok(None), Ok(None)]); + assert_eq!(result, Err(PlatformError::NoAddressSpace(7))); + } + + #[cfg(target_os = "macos")] + #[test] + fn a_process_with_no_regions_has_no_address_space() { + let (result, _) = scripted_walk(vec![Ok(None)]); + assert_eq!(result, Err(PlatformError::NoAddressSpace(7))); + } + + #[cfg(target_os = "macos")] + #[test] + fn an_exit_during_the_confirming_read_is_process_gone() { + let (result, _) = scripted_walk(vec![ + Ok(region(0, 16)), + Ok(None), + Err(PlatformError::ProcessGone(7)), + ]); + assert_eq!(result, Err(PlatformError::ProcessGone(7))); + } + + #[cfg(target_os = "macos")] + #[test] + fn a_walk_that_stops_advancing_is_malformed() { + let (result, _) = scripted_walk(vec![Ok(region(16, 0)), Ok(region(16, 0))]); + assert!(matches!(result, Err(PlatformError::Malformed { .. }))); + } + + /// The page tests map and unmap at the kernel's choice of address. Running + /// one at a time keeps another test from being handed a freed address. + #[cfg(target_os = "macos")] + static PAGE_TESTS: std::sync::Mutex<()> = std::sync::Mutex::new(()); + + #[cfg(target_os = "macos")] + fn region_containing(regions: &[MemoryRegion], address: usize) -> Option<&MemoryRegion> { + regions.iter().find(|region| { + let range = region.identity.split_whitespace().next().unwrap(); + let (start, end) = range.split_once('-').unwrap(); + let start = usize::from_str_radix(start, 16).unwrap(); + let end = usize::from_str_radix(end, 16).unwrap(); + (start..end).contains(&address) + }) + } + + #[cfg(target_os = "macos")] + #[test] + fn reads_a_live_child_then_reports_it_gone() { + use std::io::Write; + use std::process::Stdio; + let mut child = Command::new("/bin/cat") + .stdin(Stdio::piped()) + .stdout(Stdio::null()) + .spawn() + .unwrap(); + let pid = child.id(); + let regions = process_regions(pid).unwrap(); + assert!( + regions + .iter() + .any(|r| r.permissions == "r-xp" && r.pathname.as_deref() == Some("/bin/cat")), + "{regions:#?}" + ); + assert!(regions.iter().all(|r| !r.permissions.contains("rwx"))); + assert!( + regions + .iter() + .all(|r| !(r.permissions.contains('x') && r.pathname.is_none())) + ); + child.stdin.take().unwrap().flush().unwrap(); + child.wait().unwrap(); + assert_eq!(process_regions(pid), Err(PlatformError::ProcessGone(pid))); + } + + #[cfg(target_os = "macos")] + #[test] + fn reads_the_lifetime_peak_of_a_child_then_reports_it_gone() { + use std::io::{BufRead, BufReader}; + use std::process::Stdio; + // The child allocates and touches about 120 MiB, announces it, then + // waits for its standard input to close. + let mut child = Command::new("/usr/bin/perl") + .args([ + "-e", + r#"$| = 1; my $x = "a" x 125000000; print "ready\n"; ;"#, + ]) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .spawn() + .unwrap(); + let pid = child.id(); + let mut line = String::new(); + BufReader::new(child.stdout.take().unwrap()) + .read_line(&mut line) + .unwrap(); + assert_eq!(line.trim(), "ready"); + let peak = peak_memory_bytes(pid).unwrap(); + assert!(peak >= 100 * 1024 * 1024, "peak was {peak}"); + assert!(peak < 4 * 1024 * 1024 * 1024, "peak was {peak}"); + drop(child.stdin.take()); + child.wait().unwrap(); + assert_eq!(peak_memory_bytes(pid), Err(PlatformError::ProcessGone(pid))); + } + + #[cfg(target_os = "macos")] + #[test] + fn rejects_a_peak_memory_pid_that_does_not_fit() { + assert!(matches!( + peak_memory_bytes(u32::MAX), + Err(PlatformError::Malformed { .. }) + )); + } + + #[cfg(target_os = "macos")] + #[test] + fn a_dropped_window_stops_its_sampler() { + let window = Window::start().unwrap(); + let stop = Arc::clone(&window.stop); + drop(window); + // The sampler holds the only other clone until it returns. + assert_eq!(Arc::strong_count(&stop), 1); + } + + #[cfg(target_os = "macos")] + #[test] + fn rejects_a_pid_that_does_not_fit() { + assert!(matches!( + process_regions(u32::MAX), + Err(PlatformError::Malformed { .. }) + )); + } + + #[cfg(target_os = "macos")] + #[allow(unsafe_code)] + #[test] + fn sees_an_anonymous_executable_page_come_and_go() { + let _serial = PAGE_TESTS + .lock() + .unwrap_or_else(|poison| poison.into_inner()); + // SAFETY: one anonymous page is mapped, protected, inspected by address + // only, and unmapped. Nothing in it is executed. + unsafe { + let size = libc::sysconf(libc::_SC_PAGESIZE) as usize; + let page = libc::mmap( + std::ptr::null_mut(), + size, + libc::PROT_READ | libc::PROT_WRITE, + libc::MAP_ANON | libc::MAP_PRIVATE, + -1, + 0, + ); + assert_ne!(page, libc::MAP_FAILED); + assert_eq!( + libc::mprotect(page, size, libc::PROT_READ | libc::PROT_EXEC), + 0 + ); + let regions = process_regions(std::process::id()).unwrap(); + let region = region_containing(®ions, page as usize).unwrap(); + assert_eq!(region.permissions, "r-xp"); + assert_eq!(region.pathname, None); + assert_eq!(libc::munmap(page, size), 0); + let regions = process_regions(std::process::id()).unwrap(); + assert!(region_containing(®ions, page as usize).is_none()); + } + } + + #[cfg(target_os = "macos")] + #[allow(unsafe_code)] + #[test] + fn sees_a_map_jit_page_as_read_write_execute() { + let _serial = PAGE_TESTS + .lock() + .unwrap_or_else(|poison| poison.into_inner()); + // SAFETY: as above. `MAP_JIT` is what makes the OS allow this mapping. + unsafe { + let size = libc::sysconf(libc::_SC_PAGESIZE) as usize; + let page = libc::mmap( + std::ptr::null_mut(), + size, + libc::PROT_READ | libc::PROT_WRITE | libc::PROT_EXEC, + libc::MAP_ANON | libc::MAP_PRIVATE | libc::MAP_JIT, + -1, + 0, + ); + assert_ne!(page, libc::MAP_FAILED); + let regions = process_regions(std::process::id()).unwrap(); + assert_eq!( + region_containing(®ions, page as usize) + .unwrap() + .permissions, + "rwxp" + ); + assert_eq!(libc::munmap(page, size), 0); + } + } + + #[cfg(target_os = "macos")] + #[test] + fn thread_placement_is_unavailable_on_macos() { + let placement = thread_placement().unwrap(); + assert_eq!(placement.cpu_affinity, UNAVAILABLE); + assert_eq!(placement.memory_policy, UNAVAILABLE); + assert_eq!(placement.memory_nodes, UNAVAILABLE); + } + + #[test] + fn formats_the_logical_cpus_as_a_kernel_list() { + assert_eq!(cpu_list_for(14).unwrap(), "0-13"); + assert_eq!(cpu_list_for(2).unwrap(), "0-1"); + assert_eq!(cpu_list_for(1).unwrap(), "0"); + assert!(cpu_list_for(0).is_err()); + } + + #[cfg(target_os = "macos")] + #[test] + fn the_online_list_covers_the_logical_cpus() { + let count = sysctl_u32("hw.logicalcpu").unwrap(); + assert_eq!(online_cpu_list().unwrap(), cpu_list_for(count).unwrap()); + } +} diff --git a/crates/bamts-platform/src/windows.rs b/crates/bamts-platform/src/windows.rs new file mode 100644 index 00000000..35a77f24 --- /dev/null +++ b/crates/bamts-platform/src/windows.rs @@ -0,0 +1,1734 @@ +//! Windows backend: process regions through `VirtualQueryEx` and peak memory +//! through `GetProcessMemoryInfo`. +//! +//! The crate takes no dependency that declares the Win32 calls, so the few it +//! needs are declared here by hand, and every unsafe block says why it is sound. +//! The pure parts, the permission letters and the NT-to-DOS path mapping, build +//! on every host so their tests run everywhere. + +#[cfg(target_os = "windows")] +use std::collections::HashMap; +#[cfg(target_os = "windows")] +use std::mem::size_of; +#[cfg(target_os = "windows")] +use std::sync::Arc; +#[cfg(target_os = "windows")] +use std::sync::atomic::{AtomicBool, Ordering}; +#[cfg(target_os = "windows")] +use std::thread; +#[cfg(target_os = "windows")] +use std::time::Duration; + +#[cfg(target_os = "windows")] +use crate::{MachineFacts, MemoryRegion, PlatformError, ThreadPlacement, UNAVAILABLE}; + +#[cfg(target_os = "windows")] +type Result = std::result::Result; + +/// `MEMORY_BASIC_INFORMATION::State` of pages that are backed by memory or the +/// paging file. Reserved and free address space is not mapped. +#[cfg(target_os = "windows")] +const MEM_COMMIT: u32 = 0x1000; +/// `MEMORY_BASIC_INFORMATION::Type` of an executable image or one of its sections. +const MEM_IMAGE: u32 = 0x0100_0000; +/// `MEMORY_BASIC_INFORMATION::Type` of a mapped file or shared section. +const MEM_MAPPED: u32 = 0x0004_0000; + +const PAGE_READONLY: u32 = 0x02; +const PAGE_READWRITE: u32 = 0x04; +const PAGE_WRITECOPY: u32 = 0x08; +const PAGE_EXECUTE: u32 = 0x10; +const PAGE_EXECUTE_READ: u32 = 0x20; +const PAGE_EXECUTE_READWRITE: u32 = 0x40; +const PAGE_EXECUTE_WRITECOPY: u32 = 0x80; +/// A guard page faults on first touch, so no access is possible until then. +const PAGE_GUARD: u32 = 0x100; +/// The protection modes sit in the low byte; higher bits are modifiers. +const PAGE_MODE_MASK: u32 = 0xFF; + +/// Formats Windows page protection in the `rwxp` style of `/proc//maps`. +/// +/// Copy-on-write pages count as writable: a write through them succeeds. A guard +/// page and `PAGE_NOACCESS` are `---p`, as a guard page is on Linux. +fn permissions_string(protect: u32) -> String { + if protect & PAGE_GUARD != 0 { + return "---p".to_owned(); + } + let (read, write, execute) = match protect & PAGE_MODE_MASK { + PAGE_READONLY => (true, false, false), + PAGE_READWRITE | PAGE_WRITECOPY => (true, true, false), + PAGE_EXECUTE => (false, false, true), + PAGE_EXECUTE_READ => (true, false, true), + PAGE_EXECUTE_READWRITE | PAGE_EXECUTE_WRITECOPY => (true, true, true), + _ => (false, false, false), + }; + let flag = |present: bool, letter: char| if present { letter } else { '-' }; + format!( + "{}{}{}p", + flag(read, 'r'), + flag(write, 'w'), + flag(execute, 'x') + ) +} + +/// What backs a committed region, for its identity. +fn kind_name(kind: u32) -> &'static str { + match kind { + MEM_IMAGE => "image", + MEM_MAPPED => "mapped", + _ => "private", + } +} + +/// One drive letter and the NT device it names, such as `C:` and +/// `\Device\HarddiskVolume7`. +#[derive(Debug, Clone, PartialEq, Eq)] +struct DosDevice { + drive: String, + target: String, +} + +/// Turns the NT path that `GetMappedFileName` returns into the path a program +/// opens, such as `C:\Windows\System32\sort.exe`. +/// +/// A path on a device with no drive letter keeps its NT spelling: it still names +/// the backing file, and a made-up drive would not. +fn dos_path(nt_path: &str, devices: &[DosDevice]) -> String { + let under_device = |device: &DosDevice| { + let prefix = nt_path.get(..device.target.len())?; + let rest = nt_path.get(device.target.len()..)?; + (prefix.eq_ignore_ascii_case(&device.target) && rest.starts_with('\\')) + .then(|| format!("{}{rest}", device.drive)) + }; + if let Some(path) = devices.iter().find_map(under_device) { + return path; + } + const MUP: &str = "\\Device\\Mup\\"; + match nt_path.get(..MUP.len()) { + Some(prefix) if prefix.eq_ignore_ascii_case(MUP) => { + format!("\\\\{}", &nt_path[MUP.len()..]) + } + _ => nt_path.to_owned(), + } +} + +/// Names a region by its address range first, as every host does, then by the +/// allocation it belongs to and what backs it. +fn region_identity(start: u64, end: u64, allocation_base: u64, kind: u32, path: &str) -> String { + format!( + "{start:x}-{end:x} {allocation_base:x} {} {path}", + kind_name(kind) + ) +} + +/// Formats CPU numbers as a kernel CPU list, such as `0-3,5,7-9`. +fn cpu_list(cpus: impl IntoIterator) -> String { + let cpus: std::collections::BTreeSet = cpus.into_iter().collect(); + let mut parts = Vec::new(); + let mut remaining = cpus.into_iter().peekable(); + while let Some(first) = remaining.next() { + let mut last = first; + while last + .checked_add(1) + .is_some_and(|next| remaining.peek() == Some(&next)) + { + last = remaining.next().unwrap_or(last); + } + parts.push(if first == last { + first.to_string() + } else { + format!("{first}-{last}") + }); + } + parts.join(",") +} + +/// The CPUs that `mask` sets in a processor group whose first CPU is numbered +/// `first_cpu` across the machine. +fn mask_cpus(first_cpu: u32, mask: u64) -> Vec { + (0..u64::BITS) + .filter(|bit| mask >> bit & 1 == 1) + .map(|bit| first_cpu + bit) + .collect() +} + +/// `LOGICAL_PROCESSOR_RELATIONSHIP` of a physical core. +const RELATION_PROCESSOR_CORE: u32 = 0; +/// `LOGICAL_PROCESSOR_RELATIONSHIP` of a processor package, which is a socket. +const RELATION_PROCESSOR_PACKAGE: u32 = 3; + +/// `LOGICAL_PROCESSOR_RELATIONSHIP` of a NUMA node. +const RELATION_NUMA_NODE: u32 = 1; + +/// Splits the records that `GetLogicalProcessorInformationEx` returns into +/// `(relationship, whole record)`. +/// +/// Each record starts with its relationship and its own size, both `u32`. +fn split_records(records: &[u8]) -> std::result::Result, String> { + let mut found = Vec::new(); + let mut offset = 0_usize; + while offset < records.len() { + let header = records + .get(offset..offset + 8) + .ok_or_else(|| format!("a record header is cut short at byte {offset}"))?; + let relationship = u32::from_le_bytes([header[0], header[1], header[2], header[3]]); + let size = u32::from_le_bytes([header[4], header[5], header[6], header[7]]); + let size = usize::try_from(size).unwrap_or(usize::MAX); + let end = offset + .checked_add(size) + .filter(|end| size >= 8 && *end <= records.len()); + let Some(end) = end else { + return Err(format!("the record at byte {offset} has size {size}")); + }; + found.push((relationship, &records[offset..end])); + offset = end; + } + Ok(found) +} + +/// Reads `(packages, cores per package)` from the records. +/// +/// The cores must divide evenly across the packages, or the host describes a +/// machine this fact cannot state. +fn packages_and_cores(records: &[u8]) -> std::result::Result<(u32, u32), String> { + let (mut packages, mut cores) = (0_u32, 0_u32); + for (relationship, _) in split_records(records)? { + match relationship { + RELATION_PROCESSOR_PACKAGE => packages += 1, + RELATION_PROCESSOR_CORE => cores += 1, + _ => {} + } + } + if packages == 0 || cores == 0 || cores % packages != 0 { + return Err(format!( + "{cores} physical cores do not divide across {packages} packages" + )); + } + Ok((packages, cores / packages)) +} + +/// Reads the NUMA node numbers from the records, sorted. A node record holds its +/// number right after the header. +fn numa_nodes(records: &[u8]) -> std::result::Result, String> { + let mut nodes = std::collections::BTreeSet::new(); + for (relationship, record) in split_records(records)? { + if relationship != RELATION_NUMA_NODE { + continue; + } + let number = record + .get(8..12) + .ok_or_else(|| "a NUMA node record is cut short".to_owned())?; + nodes.insert(u32::from_le_bytes([ + number[0], number[1], number[2], number[3], + ])); + } + if nodes.is_empty() { + return Err("the host reports no NUMA node".to_owned()); + } + Ok(nodes.into_iter().collect()) +} + +/// The Windows version as `major.minor.build.revision`, such as `10.0.26200.9711`. +fn version_text(major: u32, minor: u32, build: u32, revision: u32) -> String { + format!("{major}.{minor}.{build}.{revision}") +} + +/// The microcode revision as Linux prints it, such as `0x8701030`. +/// +/// The `Update Revision` registry value is the revision register of the CPU. A +/// four-byte value holds the revision itself. Any other length is a form this +/// reader has not checked against a host, so it gives `None` rather than a guess. +fn microcode_text(value: &[u8]) -> Option { + let bytes: [u8; 4] = value.try_into().ok()?; + Some(format!("0x{:x}", u32::from_le_bytes(bytes))) +} + +/// Decodes a registry string: UTF-16 up to the first NUL, without the padding that +/// CPU vendors put around the name. +fn registry_text(bytes: &[u8]) -> String { + let units: Vec = bytes + .chunks_exact(2) + .map(|pair| u16::from_le_bytes([pair[0], pair[1]])) + .take_while(|unit| *unit != 0) + .collect(); + String::from_utf16_lossy(&units).trim().to_owned() +} + +#[cfg(target_os = "windows")] +mod ffi { + use std::ffi::c_void; + + pub(super) type Handle = *mut c_void; + + /// `MEMORY_BASIC_INFORMATION`. `PartitionId` exists in the 64-bit layout only. + #[repr(C)] + #[derive(Default)] + pub(super) struct MemoryBasicInformation { + pub(super) base_address: usize, + pub(super) allocation_base: usize, + pub(super) _allocation_protect: u32, + #[cfg(target_pointer_width = "64")] + pub(super) _partition_id: u16, + pub(super) region_size: usize, + pub(super) state: u32, + pub(super) protect: u32, + pub(super) kind: u32, + } + + /// `PROCESS_MEMORY_COUNTERS`. + #[repr(C)] + #[derive(Default)] + pub(super) struct ProcessMemoryCounters { + pub(super) cb: u32, + pub(super) _page_fault_count: u32, + pub(super) peak_working_set_size: usize, + pub(super) working_set_size: usize, + pub(super) _quota_peak_paged_pool_usage: usize, + pub(super) _quota_paged_pool_usage: usize, + pub(super) _quota_peak_non_paged_pool_usage: usize, + pub(super) _quota_non_paged_pool_usage: usize, + pub(super) _pagefile_usage: usize, + pub(super) _peak_pagefile_usage: usize, + } + + /// `RTL_OSVERSIONINFOW`. + #[repr(C)] + pub(super) struct OsVersionInfo { + pub(super) size: u32, + pub(super) major: u32, + pub(super) minor: u32, + pub(super) build: u32, + pub(super) _platform: u32, + pub(super) _service_pack: [u16; 128], + } + + /// `GROUP_AFFINITY`. + #[repr(C)] + #[derive(Default)] + pub(super) struct GroupAffinity { + pub(super) mask: usize, + pub(super) group: u16, + pub(super) _reserved: [u16; 3], + } + + /// `PERFORMANCE_INFORMATION`. + #[repr(C)] + #[derive(Default)] + pub(super) struct PerformanceInformation { + pub(super) cb: u32, + pub(super) _commit_total: usize, + pub(super) _commit_limit: usize, + pub(super) _commit_peak: usize, + pub(super) _physical_total: usize, + pub(super) _physical_available: usize, + pub(super) _system_cache: usize, + pub(super) _kernel_total: usize, + pub(super) _kernel_paged: usize, + pub(super) _kernel_non_paged: usize, + pub(super) page_size: usize, + pub(super) _handle_count: u32, + pub(super) _process_count: u32, + pub(super) _thread_count: u32, + } + + /// `ENUM_PAGE_FILE_INFORMATION`. + #[repr(C)] + pub(super) struct EnumPageFileInformation { + pub(super) _size: u32, + pub(super) _reserved: u32, + pub(super) total_size: usize, + pub(super) _total_in_use: usize, + pub(super) _peak_usage: usize, + } + + /// `PENUM_PAGE_FILE_CALLBACKW`. + pub(super) type PageFileCallback = + unsafe extern "system" fn(*mut c_void, *const EnumPageFileInformation, *const u16) -> i32; + + // Sizes measured from the SDK headers. A layout drift fails the build here, + // not at the kernel boundary. + #[cfg(target_pointer_width = "64")] + const _: () = assert!(size_of::() == 48); + #[cfg(target_pointer_width = "64")] + const _: () = assert!(size_of::() == 72); + #[cfg(target_pointer_width = "64")] + const _: () = assert!(size_of::() == 16); + #[cfg(target_pointer_width = "64")] + const _: () = assert!(size_of::() == 104); + #[cfg(target_pointer_width = "64")] + const _: () = assert!(size_of::() == 32); + #[cfg(target_pointer_width = "32")] + const _: () = assert!(size_of::() == 28); + #[cfg(target_pointer_width = "32")] + const _: () = assert!(size_of::() == 40); + const _: () = assert!(size_of::() == 276); + + #[link(name = "kernel32")] + #[allow(unsafe_code)] + unsafe extern "system" { + pub(super) fn OpenProcess( + desired_access: u32, + inherit_handle: i32, + process_id: u32, + ) -> Handle; + pub(super) fn CloseHandle(object: Handle) -> i32; + pub(super) fn WaitForSingleObject(object: Handle, milliseconds: u32) -> u32; + pub(super) fn GetCurrentProcess() -> Handle; + pub(super) fn GetLogicalDrives() -> u32; + pub(super) fn VirtualQueryEx( + process: Handle, + address: *const c_void, + buffer: *mut MemoryBasicInformation, + length: usize, + ) -> usize; + pub(super) fn K32GetMappedFileNameW( + process: Handle, + address: *mut c_void, + file_name: *mut u16, + size: u32, + ) -> u32; + pub(super) fn K32GetProcessMemoryInfo( + process: Handle, + counters: *mut ProcessMemoryCounters, + size: u32, + ) -> i32; + pub(super) fn QueryDosDeviceW( + device_name: *const u16, + target_path: *mut u16, + max: u32, + ) -> u32; + pub(super) fn GetCurrentThread() -> Handle; + pub(super) fn GetThreadGroupAffinity(thread: Handle, affinity: *mut GroupAffinity) -> i32; + pub(super) fn GetActiveProcessorCount(group: u16) -> u32; + pub(super) fn GetLogicalProcessorInformationEx( + relationship: u32, + buffer: *mut c_void, + length: *mut u32, + ) -> i32; + pub(super) fn K32GetPerformanceInfo(info: *mut PerformanceInformation, size: u32) -> i32; + pub(super) fn K32EnumPageFilesW(callback: PageFileCallback, context: *mut c_void) -> i32; + } + + #[link(name = "ntdll")] + #[allow(unsafe_code)] + unsafe extern "system" { + pub(super) fn RtlGetVersion(info: *mut OsVersionInfo) -> i32; + } + + #[link(name = "advapi32")] + #[allow(unsafe_code)] + unsafe extern "system" { + pub(super) fn RegGetValueW( + key: Handle, + sub_key: *const u16, + value: *const u16, + flags: u32, + kind: *mut u32, + data: *mut c_void, + size: *mut u32, + ) -> i32; + } +} + +/// `ERROR_ACCESS_DENIED`: what a process that has exited, or is exiting, answers +/// to a read through a handle that is still open. +#[cfg(target_os = "windows")] +const ERROR_ACCESS_DENIED: i32 = 5; +/// `ERROR_INVALID_PARAMETER`: no process has this id, and no region starts at or +/// after an address past the last one. +#[cfg(target_os = "windows")] +const ERROR_INVALID_PARAMETER: i32 = 87; +/// `ERROR_PARTIAL_COPY`: the process went away in the middle of a read. +#[cfg(target_os = "windows")] +const ERROR_PARTIAL_COPY: i32 = 299; +/// `ERROR_UNEXP_NET_ERR`: what `GetMappedFileNameW` answers for an address at which +/// no file is mapped, as when the file was unmapped after the region was listed. +#[cfg(target_os = "windows")] +const ERROR_UNEXP_NET_ERR: i32 = 59; + +/// How often a walk looks again at a region whose file was unmapped while the walk +/// asked for its name. A region that still has no file after this is listed without +/// a path, so memory that keeps changing is never left out of the walk. +#[cfg(target_os = "windows")] +const UNMAPPED_LOOKUPS: u32 = 8; + +/// `PROCESS_QUERY_INFORMATION | PROCESS_VM_READ | SYNCHRONIZE`. +#[cfg(target_os = "windows")] +const PROCESS_ACCESS: u32 = 0x0400 | 0x0010 | 0x0010_0000; +#[cfg(target_os = "windows")] +const WAIT_OBJECT_0: u32 = 0; +#[cfg(target_os = "windows")] +const WAIT_TIMEOUT: u32 = 0x102; + +#[cfg(target_os = "windows")] +const OPEN_SOURCE: &str = "OpenProcess"; +#[cfg(target_os = "windows")] +const WAIT_SOURCE: &str = "WaitForSingleObject"; +#[cfg(target_os = "windows")] +const REGION_SOURCE: &str = "VirtualQueryEx"; +#[cfg(target_os = "windows")] +const MEMORY_SOURCE: &str = "GetProcessMemoryInfo"; +#[cfg(target_os = "windows")] +const SAMPLER_SOURCE: &str = "peak memory sampler thread"; + +/// The host failed to answer a read. +#[cfg(target_os = "windows")] +fn read_failed(source: impl Into, detail: impl Into) -> PlatformError { + PlatformError::Read { + source: source.into(), + detail: detail.into(), + } +} + +#[cfg(target_os = "windows")] +fn read_error(source: &str, error: &std::io::Error) -> PlatformError { + read_failed(source, error.to_string()) +} + +/// The host answered, but not with data of the shape this reader expects. +#[cfg(target_os = "windows")] +fn malformed(source: impl Into, detail: impl Into) -> PlatformError { + PlatformError::Malformed { + source: source.into(), + detail: detail.into(), + } +} + +/// What the host says about the file behind an allocation. +#[cfg(target_os = "windows")] +#[derive(Debug, PartialEq, Eq)] +enum Backing { + /// The NT path of the file. + File(String), + /// The allocation has no file, as with memory that processes share through the + /// page file. + Anonymous, + /// No file is mapped at the address any more. + Unmapped, +} + +/// An open process handle, closed on drop. +#[cfg(target_os = "windows")] +struct Process { + handle: ffi::Handle, + pid: u32, +} + +#[cfg(target_os = "windows")] +impl Process { + fn open(pid: u32) -> Result { + // SAFETY: `OpenProcess` takes three integers and returns a new handle + // that this value owns, or null. + #[allow(unsafe_code)] + let handle = unsafe { ffi::OpenProcess(PROCESS_ACCESS, 0, pid) }; + if handle.is_null() { + let error = std::io::Error::last_os_error(); + return Err(if error.raw_os_error() == Some(ERROR_INVALID_PARAMETER) { + PlatformError::ProcessGone(pid) + } else { + read_error(OPEN_SOURCE, &error) + }); + } + Ok(Self { handle, pid }) + } + + /// Whether the process has ended. The host keeps answering reads for a process + /// that has exited for as long as anyone holds a handle to it, so a caller that + /// waits for `ProcessGone` would wait for the parent to close its own handle. + fn has_exited(&self) -> Result { + // SAFETY: `self.handle` is a live handle with the `SYNCHRONIZE` right, and + // a zero timeout makes the call return at once. + #[allow(unsafe_code)] + let wait = unsafe { ffi::WaitForSingleObject(self.handle, 0) }; + match wait { + WAIT_OBJECT_0 => Ok(true), + WAIT_TIMEOUT => Ok(false), + _ => Err(read_error(WAIT_SOURCE, &std::io::Error::last_os_error())), + } + } + + /// Classifies a failed read: the process has gone, it has no address space + /// right now, or the host failed the read. + fn failure(&self, source: &str, error: &std::io::Error) -> PlatformError { + match self.has_exited() { + Ok(true) => PlatformError::ProcessGone(self.pid), + Ok(false) + if matches!( + error.raw_os_error(), + Some(ERROR_ACCESS_DENIED | ERROR_PARTIAL_COPY) + ) => + { + PlatformError::NoAddressSpace(self.pid) + } + Ok(false) => read_error(source, error), + Err(wait_error) => wait_error, + } + } + + /// Reads the region that starts at or after `address`, or `None` past the last. + fn query( + &self, + address: usize, + ) -> std::result::Result, std::io::Error> { + let mut info = ffi::MemoryBasicInformation::default(); + // SAFETY: `info` is a live, writable `MEMORY_BASIC_INFORMATION` and its + // size is the length the host is told. The host keeps no pointer to it. + #[allow(unsafe_code)] + let written = unsafe { + ffi::VirtualQueryEx( + self.handle, + address as *const std::ffi::c_void, + &raw mut info, + size_of::(), + ) + }; + if written != 0 { + return Ok(Some(info)); + } + let error = std::io::Error::last_os_error(); + if error.raw_os_error() == Some(ERROR_INVALID_PARAMETER) { + Ok(None) + } else { + Err(error) + } + } + + /// Asks for the file that backs the allocation at `address`. + fn mapped_file(&self, address: usize, buffer: &mut [u16]) -> Backing { + let Ok(capacity) = u32::try_from(buffer.len()) else { + return Backing::Anonymous; + }; + // SAFETY: `buffer` is live and writable for `capacity` UTF-16 units, the + // size the host is told. `address` is only looked up, never read. + #[allow(unsafe_code)] + let written = unsafe { + ffi::K32GetMappedFileNameW( + self.handle, + address as *mut std::ffi::c_void, + buffer.as_mut_ptr(), + capacity, + ) + }; + let error = std::io::Error::last_os_error(); + match written as usize { + 0 if error.raw_os_error() == Some(ERROR_UNEXP_NET_ERR) => Backing::Unmapped, + // A full buffer was cut short, and a cut name names nothing. + 0 => Backing::Anonymous, + length if length < buffer.len() => { + Backing::File(String::from_utf16_lossy(&buffer[..length])) + } + _ => Backing::Anonymous, + } + } + + /// Why a walk found no region at all: the process has ended, or it has no + /// address space right now. + fn without_regions(&self) -> Result { + Ok(if self.has_exited()? { + PlatformError::ProcessGone(self.pid) + } else { + PlatformError::NoAddressSpace(self.pid) + }) + } + + fn memory_counters(&self) -> std::result::Result { + counters_of(self.handle) + } +} + +#[cfg(target_os = "windows")] +impl Drop for Process { + fn drop(&mut self) { + // SAFETY: `self.handle` came from `OpenProcess` and is closed only here. + #[allow(unsafe_code)] + unsafe { + ffi::CloseHandle(self.handle); + } + } +} + +#[cfg(target_os = "windows")] +fn counters_of( + handle: ffi::Handle, +) -> std::result::Result { + let mut counters = ffi::ProcessMemoryCounters::default(); + let size = u32::try_from(size_of::()) + .expect("the counters record fits a DWORD"); + counters.cb = size; + // SAFETY: `counters` is a live, writable `PROCESS_MEMORY_COUNTERS` whose size + // is both in `cb` and the length the host is told. `handle` is a process + // handle, or the pseudo-handle of the calling process, which needs no closing. + #[allow(unsafe_code)] + let status = unsafe { ffi::K32GetProcessMemoryInfo(handle, &raw mut counters, size) }; + if status == 0 { + Err(std::io::Error::last_os_error()) + } else { + Ok(counters) + } +} + +/// UTF-16 text with the NUL terminator that Win32 calls expect. +#[cfg(target_os = "windows")] +fn wide(text: &str) -> Vec { + text.encode_utf16().chain(std::iter::once(0)).collect() +} + +/// Maps each drive letter to its NT device, so a mapped file name can be shown as +/// the path a program opens. A letter with no device is left out. +/// +/// Only the letters that `GetLogicalDrives` reports are asked. If it reports none, +/// every letter is asked, as a host that cannot list its drives may still map them. +#[cfg(target_os = "windows")] +fn dos_devices() -> Vec { + // SAFETY: `GetLogicalDrives` takes no arguments and returns a bit mask, with + // bit 0 for `A:`. + #[allow(unsafe_code)] + let logical_drives = unsafe { ffi::GetLogicalDrives() }; + let mut buffer = [0_u16; 1024]; + let mut devices = Vec::new(); + for (bit, letter) in ('A'..='Z').enumerate() { + if logical_drives != 0 && logical_drives & (1 << bit) == 0 { + continue; + } + let drive = format!("{letter}:"); + let name = wide(&drive); + let capacity = u32::try_from(buffer.len()).expect("the buffer fits a DWORD"); + // SAFETY: `name` is NUL-terminated and `buffer` is live and writable for + // `capacity` UTF-16 units, the size the host is told. + #[allow(unsafe_code)] + let written = unsafe { ffi::QueryDosDeviceW(name.as_ptr(), buffer.as_mut_ptr(), capacity) }; + if written == 0 { + continue; + } + // The answer is a list of NUL-terminated names; the first is the current one. + let first = buffer + .iter() + .position(|unit| *unit == 0) + .unwrap_or(buffer.len()); + devices.push(DosDevice { + drive, + target: String::from_utf16_lossy(&buffer[..first]), + }); + } + devices +} + +#[cfg(target_os = "windows")] +pub(crate) fn process_regions(pid: u32) -> Result> { + let process = Process::open(pid)?; + let devices = dos_devices(); + let mut name_buffer = vec![0_u16; 32 * 1024]; + // One lookup per allocation: the sections of an image share their allocation. + let mut names: HashMap> = HashMap::new(); + let mut regions = Vec::new(); + let mut address = 0_usize; + let mut unmapped_lookups = 0_u32; + loop { + let info = match process.query(address) { + Ok(Some(info)) => info, + Ok(None) => break, + Err(error) => return Err(process.failure(REGION_SOURCE, &error)), + }; + let next = info + .base_address + .checked_add(info.region_size) + .filter(|next| *next > address) + .ok_or_else(|| malformed(REGION_SOURCE, "region walk does not advance"))?; + if info.state == MEM_COMMIT { + let backed = info.kind == MEM_IMAGE || info.kind == MEM_MAPPED; + let path = if !backed { + None + } else if let Some(known) = names.get(&info.allocation_base) { + known.clone() + } else { + let name = match process.mapped_file(info.allocation_base, &mut name_buffer) { + Backing::File(nt_path) => Some(dos_path(&nt_path, &devices)), + Backing::Anonymous => None, + // The file was unmapped after the region was listed. Ask for the + // region again: it is gone, or it is a new one. Naming it as + // memory without a file would make a library that unloads look + // like anonymous code. + Backing::Unmapped if unmapped_lookups < UNMAPPED_LOOKUPS => { + unmapped_lookups += 1; + continue; + } + Backing::Unmapped => None, + }; + names.insert(info.allocation_base, name.clone()); + name + }; + regions.push(MemoryRegion { + identity: region_identity( + info.base_address as u64, + next as u64, + info.allocation_base as u64, + info.kind, + path.as_deref().unwrap_or(""), + ), + permissions: permissions_string(info.protect), + pathname: path, + }); + } + address = next; + unmapped_lookups = 0; + } + if regions.is_empty() { + return Err(process.without_regions()?); + } + Ok(regions) +} + +#[cfg(target_os = "windows")] +pub(crate) fn peak_memory_bytes(pid: u32) -> Result { + let process = Process::open(pid)?; + // The counters of a process that has exited stay readable while a handle to it + // is open. Stop here so that `ProcessGone` does not wait for that handle. + if process.has_exited()? { + return Err(PlatformError::ProcessGone(pid)); + } + match process.memory_counters() { + Ok(counters) => Ok(counters.peak_working_set_size as u64), + Err(error) => Err(process.failure(MEMORY_SOURCE, &error)), + } +} + +/// The working set of this process now, in bytes. +#[cfg(target_os = "windows")] +fn working_set_bytes() -> Result { + // SAFETY: `GetCurrentProcess` takes no arguments and returns a pseudo-handle + // for the calling process. + #[allow(unsafe_code)] + let current = unsafe { ffi::GetCurrentProcess() }; + counters_of(current) + .map(|counters| counters.working_set_size as u64) + .map_err(|error| read_error(MEMORY_SOURCE, &error)) +} + +/// Samples the working set of this process while the window is open. Windows has +/// no way to reset the peak working set, so a spike shorter than the sampling +/// interval can be missed. +#[cfg(target_os = "windows")] +pub(crate) struct Window { + stop: Arc, + sampler: Option>, +} + +#[cfg(target_os = "windows")] +impl Window { + const INTERVAL: Duration = Duration::from_millis(1); + + pub(crate) fn start() -> Result { + let first = working_set_bytes()?; + let stop = Arc::new(AtomicBool::new(false)); + let flag = Arc::clone(&stop); + let sampler = thread::Builder::new() + .name("bamts-peak-window".to_owned()) + .spawn(move || { + let mut peak = first; + while !flag.load(Ordering::Acquire) { + if let Ok(bytes) = working_set_bytes() { + peak = peak.max(bytes); + } + thread::sleep(Self::INTERVAL); + } + peak + }) + .map_err(|error| read_error(SAMPLER_SOURCE, &error))?; + Ok(Self { + stop, + sampler: Some(sampler), + }) + } + + pub(crate) fn finish(mut self) -> Result { + self.stop.store(true, Ordering::Release); + let sampled = self + .sampler + .take() + .expect("the sampler is present until the window finishes") + .join() + .map_err(|_| read_failed(SAMPLER_SOURCE, "the sampler panicked"))?; + Ok(sampled.max(working_set_bytes()?)) + } +} + +#[cfg(target_os = "windows")] +impl Drop for Window { + fn drop(&mut self) { + self.stop.store(true, Ordering::Release); + if let Some(sampler) = self.sampler.take() { + let _ = sampler.join(); + } + } +} + +/// `RRF_RT_REG_SZ`, `RRF_RT_REG_BINARY` and `RRF_RT_REG_DWORD`: the registry value +/// types `RegGetValueW` may return. +#[cfg(target_os = "windows")] +const RRF_RT_REG_SZ: u32 = 0x0000_0002; +#[cfg(target_os = "windows")] +const RRF_RT_REG_BINARY: u32 = 0x0000_0008; +#[cfg(target_os = "windows")] +const RRF_RT_REG_DWORD: u32 = 0x0000_0010; +/// `ERROR_FILE_NOT_FOUND`: the registry has no such value. +#[cfg(target_os = "windows")] +const ERROR_FILE_NOT_FOUND: i32 = 2; +/// `ERROR_INSUFFICIENT_BUFFER`: the answer to a size query. +#[cfg(target_os = "windows")] +const ERROR_INSUFFICIENT_BUFFER: i32 = 122; +/// `RelationAll` and `ALL_PROCESSOR_GROUPS`. +#[cfg(target_os = "windows")] +const RELATION_ALL: u32 = 0xFFFF; +#[cfg(target_os = "windows")] +const ALL_PROCESSOR_GROUPS: u16 = 0xFFFF; + +#[cfg(target_os = "windows")] +const PROCESSOR_KEY: &str = "HARDWARE\\DESCRIPTION\\System\\CentralProcessor\\0"; +#[cfg(target_os = "windows")] +const VERSION_KEY: &str = "SOFTWARE\\Microsoft\\Windows NT\\CurrentVersion"; + +/// Reads one value of `HKEY_LOCAL_MACHINE`. `Ok(None)` means the value does not exist. +#[cfg(target_os = "windows")] +fn registry_value( + sub_key: &str, + value: &str, + flags: u32, + capacity: usize, +) -> Result>> { + let (sub_key_wide, value_wide) = (wide(sub_key), wide(value)); + let mut data = vec![0_u8; capacity]; + let mut size = u32::try_from(capacity).expect("the capacity fits a DWORD"); + // `HKEY_LOCAL_MACHINE` is `0x80000002` sign-extended to a pointer. + let local_machine = 0x8000_0002_u32 as i32 as isize as ffi::Handle; + // SAFETY: both names are NUL-terminated and live for the call. `data` is live + // and writable for `size` bytes, and `size` is the length the host is told. + #[allow(unsafe_code)] + let status = unsafe { + ffi::RegGetValueW( + local_machine, + sub_key_wide.as_ptr(), + value_wide.as_ptr(), + flags, + std::ptr::null_mut(), + data.as_mut_ptr().cast::(), + &raw mut size, + ) + }; + match status { + 0 => { + data.truncate(size as usize); + Ok(Some(data)) + } + ERROR_FILE_NOT_FOUND => Ok(None), + _ => Err(read_error( + ®istry_source(sub_key, value), + &std::io::Error::from_raw_os_error(status), + )), + } +} + +/// Names a registry value in an error, the way `source` names the call elsewhere. +#[cfg(target_os = "windows")] +fn registry_source(sub_key: &str, value: &str) -> String { + format!("RegGetValueW {sub_key}\\{value}") +} + +/// A registry value that this host has to have. +#[cfg(target_os = "windows")] +fn required_registry_value( + sub_key: &str, + value: &str, + flags: u32, + capacity: usize, +) -> Result> { + registry_value(sub_key, value, flags, capacity)? + .ok_or_else(|| malformed(registry_source(sub_key, value), "the value does not exist")) +} + +/// The Windows version `(major, minor, build)`. `RtlGetVersion` tells the real +/// version; the documented `GetVersionEx` reports what the manifest of the caller +/// asks for. +#[cfg(target_os = "windows")] +fn windows_version() -> Result<(u32, u32, u32)> { + let mut info = ffi::OsVersionInfo { + size: u32::try_from(size_of::()).expect("the record fits a DWORD"), + major: 0, + minor: 0, + build: 0, + _platform: 0, + _service_pack: [0; 128], + }; + // SAFETY: `info` is a live, writable `RTL_OSVERSIONINFOW` whose size field + // holds its size, as the call requires. + #[allow(unsafe_code)] + let status = unsafe { ffi::RtlGetVersion(&raw mut info) }; + if status != 0 { + return Err(read_failed( + "RtlGetVersion", + format!("NTSTATUS {status:#x}"), + )); + } + Ok((info.major, info.minor, info.build)) +} + +#[cfg(target_os = "windows")] +const TOPOLOGY_SOURCE: &str = "GetLogicalProcessorInformationEx"; + +/// Reads the topology records of the host, for every relationship. +#[cfg(target_os = "windows")] +fn processor_records() -> Result> { + const SOURCE: &str = TOPOLOGY_SOURCE; + let mut length = 0_u32; + // SAFETY: a null buffer with a zero length asks the host for the size it + // needs; it writes only `length`. + #[allow(unsafe_code)] + let sized = unsafe { + ffi::GetLogicalProcessorInformationEx(RELATION_ALL, std::ptr::null_mut(), &raw mut length) + }; + let error = std::io::Error::last_os_error(); + if sized != 0 || error.raw_os_error() != Some(ERROR_INSUFFICIENT_BUFFER) { + return Err(read_error(SOURCE, &error)); + } + // The records hold pointer-sized fields, so the buffer is aligned for them. + let mut buffer = vec![0_u64; (length as usize).div_ceil(size_of::())]; + // SAFETY: `buffer` is live and writable for at least `length` bytes, the size + // the host asked for and is told again. + #[allow(unsafe_code)] + let read = unsafe { + ffi::GetLogicalProcessorInformationEx( + RELATION_ALL, + buffer.as_mut_ptr().cast::(), + &raw mut length, + ) + }; + if read == 0 { + return Err(read_error(SOURCE, &std::io::Error::last_os_error())); + } + let mut records: Vec = buffer.iter().flat_map(|word| word.to_ne_bytes()).collect(); + records.truncate(length as usize); + Ok(records) +} + +/// Reads `(packages, cores per package)` from the host. +#[cfg(target_os = "windows")] +fn processor_topology() -> Result<(u32, u32)> { + packages_and_cores(&processor_records()?).map_err(|detail| malformed(TOPOLOGY_SOURCE, detail)) +} + +/// The NUMA nodes of the host. A Windows process is not bound to a subset of +/// them, so these are the nodes its memory may come from. +#[cfg(target_os = "windows")] +pub(crate) fn numa_memory_nodes() -> Result> { + numa_nodes(&processor_records()?).map_err(|detail| malformed(TOPOLOGY_SOURCE, detail)) +} + +#[cfg(target_os = "windows")] +pub(crate) fn machine_facts() -> Result { + let (major, minor, build) = windows_version()?; + let revision = u32::from_le_bytes( + required_registry_value(VERSION_KEY, "UBR", RRF_RT_REG_DWORD, 4)? + .try_into() + .map_err(|_| { + malformed( + registry_source(VERSION_KEY, "UBR"), + "the revision is not a DWORD", + ) + })?, + ); + let cpu_model = registry_text(&required_registry_value( + PROCESSOR_KEY, + "ProcessorNameString", + RRF_RT_REG_SZ, + 1024, + )?); + // The buffer is far larger than any known form, so that a value of another length + // reaches `microcode_text` and reads as unavailable instead of failing the read. + let microcode = registry_value(PROCESSOR_KEY, "Update Revision", RRF_RT_REG_BINARY, 1024)? + .and_then(|value| microcode_text(&value)) + .unwrap_or_else(|| UNAVAILABLE.to_owned()); + let (sockets, cores_per_socket) = processor_topology()?; + Ok(MachineFacts { + kernel_release: format!("{major}.{minor}.{build}"), + kernel_version: version_text(major, minor, build, revision), + cpu_model, + sockets, + cores_per_socket, + microcode, + governor: UNAVAILABLE.to_owned(), + swap_total_kib: swap_total_kib()?, + }) +} + +/// The size of one page, from `GetPerformanceInfo`. +#[cfg(target_os = "windows")] +fn page_size_bytes() -> Result { + let size = + u32::try_from(size_of::()).expect("the record fits a DWORD"); + let mut info = ffi::PerformanceInformation { + cb: size, + ..ffi::PerformanceInformation::default() + }; + // SAFETY: `info` is a live, writable `PERFORMANCE_INFORMATION` whose size is + // both in `cb` and the length the host is told. + #[allow(unsafe_code)] + let status = unsafe { ffi::K32GetPerformanceInfo(&raw mut info, size) }; + if status == 0 { + return Err(read_error( + "GetPerformanceInfo", + &std::io::Error::last_os_error(), + )); + } + Ok(info.page_size as u64) +} + +/// Adds the size of one page file, in pages, to the total behind `context`. +/// +/// # Safety +/// `context` must point to the `u64` that `swap_total_kib` passed, and `info` to a +/// record the host owns for the length of the call. `K32EnumPageFilesW` upholds this. +#[cfg(target_os = "windows")] +#[allow(unsafe_code)] +unsafe extern "system" fn add_page_file( + context: *mut std::ffi::c_void, + info: *const ffi::EnumPageFileInformation, + _name: *const u16, +) -> i32 { + // SAFETY: the contract of this function; the total outlives the enumeration. + #[allow(unsafe_code)] + unsafe { + *context.cast::() += (*info).total_size as u64; + } + 1 +} + +#[cfg(target_os = "windows")] +pub(crate) fn swap_total_kib() -> Result { + const SOURCE: &str = "EnumPageFiles"; + let page = page_size_bytes()?; + let mut pages = 0_u64; + // SAFETY: the callback and the total outlive the call, and the callback only + // adds the sizes the host hands it to the total. + #[allow(unsafe_code)] + let status = unsafe { ffi::K32EnumPageFilesW(add_page_file, (&raw mut pages).cast()) }; + if status == 0 { + return Err(read_error(SOURCE, &std::io::Error::last_os_error())); + } + pages + .checked_mul(page) + .map(|bytes| bytes / 1024) + .ok_or_else(|| malformed(SOURCE, "the page file size overflows")) +} + +/// How many logical CPUs are active in `group`, or in all groups. +#[cfg(target_os = "windows")] +fn active_processors(group: u16) -> Result { + // SAFETY: `GetActiveProcessorCount` takes a group number and returns a count. + #[allow(unsafe_code)] + let count = unsafe { ffi::GetActiveProcessorCount(group) }; + if count == 0 { + return Err(read_error( + "GetActiveProcessorCount", + &std::io::Error::last_os_error(), + )); + } + Ok(count) +} + +#[cfg(target_os = "windows")] +pub(crate) fn online_cpu_list() -> Result { + Ok(cpu_list(0..active_processors(ALL_PROCESSOR_GROUPS)?)) +} + +/// The CPUs the calling thread may run on, numbered across processor groups. +#[cfg(target_os = "windows")] +pub(crate) fn cpu_affinity() -> Result { + const SOURCE: &str = "GetThreadGroupAffinity"; + let mut affinity = ffi::GroupAffinity::default(); + // SAFETY: `GetCurrentThread` returns a pseudo-handle that needs no closing, + // and `affinity` is a live, writable `GROUP_AFFINITY`. + #[allow(unsafe_code)] + let status = unsafe { ffi::GetThreadGroupAffinity(ffi::GetCurrentThread(), &raw mut affinity) }; + if status == 0 { + return Err(read_error(SOURCE, &std::io::Error::last_os_error())); + } + let mut first_cpu = 0_u32; + for group in 0..affinity.group { + first_cpu += active_processors(group)?; + } + let cpus = mask_cpus(first_cpu, affinity.mask as u64); + if cpus.is_empty() { + return Err(malformed(SOURCE, "the thread may run on no CPU")); + } + Ok(cpu_list(cpus)) +} + +/// A Windows thread has a CPU affinity to read, and the host lists its NUMA nodes. +/// The memory policy is `numactl`'s notion and has no Windows reader, so it is +/// [`UNAVAILABLE`]. +#[cfg(target_os = "windows")] +pub(crate) fn thread_placement() -> Result { + Ok(ThreadPlacement { + cpu_affinity: cpu_affinity()?, + memory_policy: UNAVAILABLE.to_owned(), + memory_nodes: cpu_list(numa_memory_nodes()?), + }) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn volume(drive: &str, target: &str) -> DosDevice { + DosDevice { + drive: drive.to_owned(), + target: target.to_owned(), + } + } + + #[test] + fn protection_modes_become_rwx_letters() { + for (protect, expected) in [ + (0x01, "---p"), + (0x02, "r--p"), + (0x04, "rw-p"), + (0x08, "rw-p"), + (0x10, "--xp"), + (0x20, "r-xp"), + (0x40, "rwxp"), + (0x80, "rwxp"), + // Modifier bits do not change the mode. + (0x204, "rw-p"), + (0x0200_0020, "r-xp"), + (0, "---p"), + ] { + assert_eq!(permissions_string(protect), expected, "{protect:#x}"); + } + } + + #[test] + fn a_guard_page_allows_no_access() { + assert_eq!(permissions_string(0x104), "---p"); + assert_eq!(permissions_string(0x140), "---p"); + } + + #[test] + fn a_mapped_file_name_loses_its_nt_device() { + let devices = [ + volume("C:", "\\Device\\HarddiskVolume7"), + volume("D:", "\\Device\\HarddiskVolume70"), + ]; + assert_eq!( + dos_path( + "\\Device\\HarddiskVolume7\\Windows\\System32\\sort.exe", + &devices + ), + "C:\\Windows\\System32\\sort.exe" + ); + // A device whose name only starts the same is another device. + assert_eq!( + dos_path("\\Device\\HarddiskVolume70\\data\\a.bin", &devices), + "D:\\data\\a.bin" + ); + assert_eq!( + dos_path("\\device\\harddiskvolume7\\Users\\me\\a.exe", &devices), + "C:\\Users\\me\\a.exe" + ); + } + + #[test] + fn a_network_share_becomes_a_unc_path() { + assert_eq!( + dos_path("\\Device\\Mup\\server\\share\\dir\\a.dll", &[]), + "\\\\server\\share\\dir\\a.dll" + ); + } + + #[test] + fn a_device_without_a_drive_letter_keeps_its_nt_path() { + let devices = [volume("C:", "\\Device\\HarddiskVolume7")]; + let path = "\\Device\\HarddiskVolume9\\x.dll"; + assert_eq!(dos_path(path, &devices), path); + // A path that ends at the device name is not under it. + assert_eq!( + dos_path("\\Device\\HarddiskVolume7", &devices), + "\\Device\\HarddiskVolume7" + ); + assert_eq!(dos_path("", &devices), ""); + } + + #[test] + fn cpu_lists_join_runs_and_keep_gaps() { + assert_eq!(cpu_list([0, 1, 2, 3]), "0-3"); + assert_eq!(cpu_list([5, 0, 1, 7, 8, 9, 3, 3]), "0-1,3,5,7-9"); + assert_eq!(cpu_list([4]), "4"); + assert_eq!(cpu_list(std::iter::empty()), ""); + // The last number of the range type does not overflow the run test. + assert_eq!( + cpu_list([u32::MAX - 1, u32::MAX]), + format!("{}-{}", u32::MAX - 1, u32::MAX) + ); + } + + #[test] + fn a_group_mask_lists_its_cpus_across_groups() { + assert_eq!(mask_cpus(0, 0b1011), vec![0, 1, 3]); + assert_eq!(mask_cpus(64, 0b101), vec![64, 66]); + assert_eq!(mask_cpus(0, u64::MAX).len(), 64); + assert!(mask_cpus(0, 0).is_empty()); + } + + /// A record as `GetLogicalProcessorInformationEx` returns it: the relationship + /// and the size, then the rest of the record. + fn record(relationship: u32, size: u32) -> Vec { + let mut bytes = Vec::new(); + bytes.extend(relationship.to_le_bytes()); + bytes.extend(size.to_le_bytes()); + bytes.resize(size as usize, 0); + bytes + } + + fn records(parts: &[(u32, u32)]) -> Vec { + parts + .iter() + .flat_map(|(relationship, size)| record(*relationship, *size)) + .collect() + } + + #[test] + fn the_topology_counts_packages_and_cores_among_other_records() { + let one = records(&[ + (RELATION_PROCESSOR_PACKAGE, 32), + (RELATION_PROCESSOR_CORE, 48), + (RELATION_PROCESSOR_CORE, 48), + (2, 64), + (4, 24), + (RELATION_PROCESSOR_CORE, 48), + (RELATION_PROCESSOR_CORE, 48), + ]); + assert_eq!(packages_and_cores(&one), Ok((1, 4))); + + let two = records(&[ + (RELATION_PROCESSOR_PACKAGE, 32), + (RELATION_PROCESSOR_PACKAGE, 32), + (RELATION_PROCESSOR_CORE, 48), + (RELATION_PROCESSOR_CORE, 48), + (RELATION_PROCESSOR_CORE, 48), + (RELATION_PROCESSOR_CORE, 48), + ]); + assert_eq!(packages_and_cores(&two), Ok((2, 2))); + } + + #[test] + fn a_topology_that_cannot_be_stated_is_an_error() { + let uneven = records(&[ + (RELATION_PROCESSOR_PACKAGE, 32), + (RELATION_PROCESSOR_PACKAGE, 32), + (RELATION_PROCESSOR_CORE, 48), + (RELATION_PROCESSOR_CORE, 48), + (RELATION_PROCESSOR_CORE, 48), + ]); + assert!(packages_and_cores(&uneven).is_err()); + assert!(packages_and_cores(&records(&[(RELATION_PROCESSOR_CORE, 48)])).is_err()); + assert!(packages_and_cores(&[]).is_err()); + // A header cut short, and a size past the end. A size below the header is + // the subject of a test of its own. + assert!(packages_and_cores(&[3, 0, 0, 0, 8]).is_err()); + assert!(packages_and_cores(&records(&[(RELATION_PROCESSOR_PACKAGE, 4)])).is_err()); + let mut overrun = record(RELATION_PROCESSOR_PACKAGE, 32); + overrun.truncate(20); + assert!(packages_and_cores(&overrun).is_err()); + } + + #[test] + fn a_record_smaller_than_its_header_ends_the_walk_with_an_error() { + // The header is whole and its size field is below the size of a header. A + // size of zero would leave the offset where it is and the walk would not end. + for size in [0_u32, 4, 7] { + let mut header = Vec::new(); + header.extend(RELATION_PROCESSOR_PACKAGE.to_le_bytes()); + header.extend(size.to_le_bytes()); + let error = split_records(&header).unwrap_err(); + assert!(error.contains(&format!("has size {size}")), "{error}"); + } + } + + #[test] + fn the_numa_nodes_are_read_from_their_records_sorted_and_once() { + let mut records = Vec::new(); + for node in [1_u32, 0, 1] { + let mut record = record(RELATION_NUMA_NODE, 64); + record[8..12].copy_from_slice(&node.to_le_bytes()); + records.extend(record); + } + records.extend(record(RELATION_PROCESSOR_CORE, 48)); + assert_eq!(numa_nodes(&records), Ok(vec![0, 1])); + + // No node record, and a node record without room for its number. + assert!(numa_nodes(&record(RELATION_PROCESSOR_CORE, 48)).is_err()); + assert!(numa_nodes(&record(RELATION_NUMA_NODE, 8)).is_err()); + assert!(numa_nodes(&[]).is_err()); + } + #[test] + fn the_version_and_the_microcode_read_like_the_linux_facts() { + assert_eq!(version_text(10, 0, 26200, 9711), "10.0.26200.9711"); + assert_eq!( + microcode_text(&[0x30, 0x10, 0x70, 0x08]), + Some("0x8701030".to_owned()) + ); + // A form this reader has not seen on a host is not decoded. + assert_eq!(microcode_text(&[1, 2, 3, 4, 5, 6, 7, 8]), None); + assert_eq!(microcode_text(&[]), None); + } + + #[test] + fn a_registry_string_loses_its_padding_and_its_terminator() { + let bytes: Vec = " AMD Ryzen 7 \0left over" + .encode_utf16() + .flat_map(u16::to_le_bytes) + .collect(); + assert_eq!(registry_text(&bytes), "AMD Ryzen 7"); + assert_eq!(registry_text(&[]), ""); + } + + #[test] + fn a_region_identity_starts_with_its_address_range() { + let identity = region_identity( + 0x7ff6_606d_0000, + 0x7ff6_606d_1000, + 0x7ff6_606d_0000, + MEM_IMAGE, + "C:\\a b\\x.exe", + ); + let range = identity.split_whitespace().next().expect("a first field"); + assert_eq!(range, "7ff6606d0000-7ff6606d1000"); + assert!(identity.ends_with("image C:\\a b\\x.exe"), "{identity}"); + assert!(region_identity(1, 2, 1, 0, "").contains(" private")); + } + + #[cfg(target_os = "windows")] + mod live { + use std::io::Write; + use std::process::{Command, Stdio}; + + use super::super::*; + + /// A child that stays alive until its standard input closes. + fn waiting_child() -> std::process::Child { + let system = std::env::var_os("SystemRoot").expect("SystemRoot is set on Windows"); + Command::new( + std::path::PathBuf::from(system) + .join("System32") + .join("sort.exe"), + ) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .spawn() + .expect("start sort.exe") + } + + fn region_at(regions: &[MemoryRegion], address: usize) -> Option<&MemoryRegion> { + regions.iter().find(|region| { + let range = region.identity.split_whitespace().next(); + let Some((start, end)) = range.and_then(|range| range.split_once('-')) else { + return false; + }; + matches!( + (usize::from_str_radix(start, 16), usize::from_str_radix(end, 16)), + (Ok(start), Ok(end)) if start <= address && address < end + ) + }) + } + + fn permissions_at(regions: &[MemoryRegion], address: usize) -> Option<&str> { + region_at(regions, address).map(|region| region.permissions.as_str()) + } + + /// A view of page-file memory, which no file backs: `(section, view address)`. + fn shared_view() -> (ffi::Handle, usize) { + #[link(name = "kernel32")] + #[allow(unsafe_code)] + unsafe extern "system" { + fn CreateFileMappingW( + file: ffi::Handle, + attributes: *mut std::ffi::c_void, + protect: u32, + size_high: u32, + size_low: u32, + name: *const u16, + ) -> ffi::Handle; + fn MapViewOfFile( + section: ffi::Handle, + access: u32, + offset_high: u32, + offset_low: u32, + bytes: usize, + ) -> *mut std::ffi::c_void; + } + const PAGE_READWRITE: u32 = 0x04; + const FILE_MAP_WRITE: u32 = 0x02; + // SAFETY: `INVALID_HANDLE_VALUE` asks for page-file memory, and the null + // attributes and name are allowed. The call keeps no pointer. + #[allow(unsafe_code)] + let section = unsafe { + CreateFileMappingW( + -1_isize as ffi::Handle, + std::ptr::null_mut(), + PAGE_READWRITE, + 0, + 4096, + std::ptr::null(), + ) + }; + assert!(!section.is_null(), "the host made no section"); + // SAFETY: `section` is the live section made above. + #[allow(unsafe_code)] + let view = unsafe { MapViewOfFile(section, FILE_MAP_WRITE, 0, 0, 4096) }; + assert!(!view.is_null(), "the host mapped no view"); + (section, view as usize) + } + + /// Unmaps the view of [`shared_view`] and closes its section. + fn unmap(section: ffi::Handle, view: usize) { + #[link(name = "kernel32")] + #[allow(unsafe_code)] + unsafe extern "system" { + fn UnmapViewOfFile(view: *const std::ffi::c_void) -> i32; + } + // SAFETY: `view` is the live view of `section`, and neither is used + // again. The section handle is owned by the caller. + #[allow(unsafe_code)] + unsafe { + assert_ne!(UnmapViewOfFile(view as *const std::ffi::c_void), 0); + ffi::CloseHandle(section); + } + } + + #[test] + fn the_host_tells_a_file_from_memory_without_a_file_from_nothing() { + let own = Process::open(std::process::id()).expect("own process"); + let mut buffer = vec![0_u16; 1024]; + let code = + the_host_tells_a_file_from_memory_without_a_file_from_nothing as fn() as usize; + match own.mapped_file(code, &mut buffer) { + Backing::File(path) => assert!(path.starts_with("\\Device\\"), "{path}"), + other => panic!("the test image has no file: {other:?}"), + } + + let (section, view) = shared_view(); + assert_eq!(own.mapped_file(view, &mut buffer), Backing::Anonymous); + unmap(section, view); + assert_eq!(own.mapped_file(view, &mut buffer), Backing::Unmapped); + + let mut short = vec![0_u16; 4]; + assert_eq!(own.mapped_file(code, &mut short), Backing::Anonymous); + } + + #[test] + fn memory_without_a_file_is_listed_without_a_path() { + let (section, view) = shared_view(); + let regions = process_regions(std::process::id()).expect("own regions"); + unmap(section, view); + + let region = region_at(®ions, view).expect("the view is listed"); + assert_eq!(region.pathname, None); + assert_eq!(region.permissions, "rw-p"); + assert!(region.identity.contains(" mapped"), "{}", region.identity); + } + + #[test] + fn this_process_shows_code_as_executable_and_data_as_writable() { + let regions = process_regions(std::process::id()).expect("own regions"); + let code_address = + this_process_shows_code_as_executable_and_data_as_writable as fn() as usize; + let code = permissions_at(®ions, code_address).expect("code is mapped"); + assert!(code.contains('x') && !code.contains('w'), "{code}"); + let data = Box::new([0_u8; 64]); + let regions = process_regions(std::process::id()).expect("own regions"); + let heap_address = std::ptr::from_ref::<[u8; 64]>(&data) as usize; + let heap = permissions_at(®ions, heap_address).expect("heap is mapped"); + assert!(heap.contains('w') && !heap.contains('x'), "{heap}"); + } + + #[test] + fn a_live_child_shows_its_image_under_a_dos_path() { + let mut child = waiting_child(); + let regions = process_regions(child.id()).expect("child regions"); + let image = regions + .iter() + .filter_map(|region| region.pathname.as_deref()) + .find(|path| path.to_ascii_lowercase().ends_with("\\sort.exe")) + .expect("the image is mapped"); + assert!(image.as_bytes()[1] == b':', "{image}"); + assert!(peak_memory_bytes(child.id()).expect("child peak") > 0); + + drop(child.stdin.take()); + child.wait().expect("child exits"); + } + + #[test] + fn an_exited_child_is_gone_while_the_parent_still_holds_its_handle() { + let mut child = waiting_child(); + let pid = child.id(); + child.stdin.take().expect("stdin").flush().expect("flush"); + drop(child.stdin.take()); + child.wait().expect("child exits"); + + assert_eq!(peak_memory_bytes(pid), Err(PlatformError::ProcessGone(pid))); + assert_eq!(process_regions(pid), Err(PlatformError::ProcessGone(pid))); + } + + #[test] + fn a_failed_read_is_classified_by_whether_the_process_lives() { + let own = Process::open(std::process::id()).expect("own process"); + for code in [ERROR_ACCESS_DENIED, ERROR_PARTIAL_COPY] { + let error = std::io::Error::from_raw_os_error(code); + assert_eq!( + own.failure("read", &error), + PlatformError::NoAddressSpace(own.pid) + ); + } + let other = std::io::Error::from_raw_os_error(ERROR_INVALID_PARAMETER); + assert_eq!( + own.failure("read", &other), + PlatformError::Read { + source: "read".to_owned(), + detail: other.to_string(), + } + ); + + let mut child = waiting_child(); + let gone = Process::open(child.id()).expect("child process"); + drop(child.stdin.take()); + child.wait().expect("child exits"); + let denied = std::io::Error::from_raw_os_error(ERROR_ACCESS_DENIED); + assert_eq!( + gone.failure("read", &denied), + PlatformError::ProcessGone(gone.pid) + ); + } + + #[test] + fn a_walk_without_regions_is_classified_by_whether_the_process_lives() { + let own = Process::open(std::process::id()).expect("own process"); + assert_eq!( + own.without_regions(), + Ok(PlatformError::NoAddressSpace(own.pid)) + ); + + let mut child = waiting_child(); + let gone = Process::open(child.id()).expect("child process"); + drop(child.stdin.take()); + child.wait().expect("child exits"); + assert_eq!( + gone.without_regions(), + Ok(PlatformError::ProcessGone(gone.pid)) + ); + } + + fn parse_cpu_list(text: &str) -> std::collections::BTreeSet { + text.split(',') + .flat_map(|part| match part.split_once('-') { + Some((first, last)) => { + first.parse::().expect("a number")..=last.parse().expect("a number") + } + None => { + let cpu = part.parse::().expect("a number"); + cpu..=cpu + } + }) + .collect() + } + + #[test] + fn this_host_describes_itself() { + let facts = machine_facts().expect("machine facts"); + + let release: Vec<&str> = facts.kernel_release.split('.').collect(); + assert_eq!(release.len(), 3, "{}", facts.kernel_release); + assert!(release.iter().all(|part| part.parse::().is_ok())); + assert!( + facts + .kernel_version + .starts_with(&format!("{}.", facts.kernel_release)), + "{} / {}", + facts.kernel_release, + facts.kernel_version + ); + assert!(!facts.cpu_model.is_empty()); + assert_eq!(facts.cpu_model, facts.cpu_model.trim()); + assert!(facts.sockets >= 1 && facts.cores_per_socket >= 1); + assert!(facts.microcode == UNAVAILABLE || facts.microcode.starts_with("0x")); + assert_eq!(facts.governor, UNAVAILABLE); + assert_eq!(facts.swap_total_kib, swap_total_kib().expect("swap")); + } + + #[test] + fn the_online_list_covers_every_logical_cpu_and_the_thread_runs_inside_it() { + let online = parse_cpu_list(&online_cpu_list().expect("online CPUs")); + let logical = std::thread::available_parallelism().expect("parallelism"); + // Parallelism follows the affinity of the process, the list does not. + assert!(online.len() >= logical.get(), "{online:?}"); + assert_eq!(online.first(), Some(&0)); + assert_eq!( + online.last().copied(), + Some(u32::try_from(online.len()).expect("a small count") - 1), + "the online CPUs are numbered without gaps" + ); + + let affinity = parse_cpu_list(&cpu_affinity().expect("affinity")); + assert!( + !affinity.is_empty() && affinity.is_subset(&online), + "{affinity:?}" + ); + let nodes = numa_memory_nodes().expect("NUMA nodes"); + assert!( + !nodes.is_empty() && nodes.windows(2).all(|pair| pair[0] < pair[1]), + "{nodes:?}" + ); + let placement = thread_placement().expect("placement"); + assert_eq!(parse_cpu_list(&placement.cpu_affinity), affinity); + assert_eq!(placement.memory_policy, UNAVAILABLE); + assert_eq!(placement.memory_nodes, cpu_list(nodes)); + } + + #[test] + fn a_thread_pinned_to_one_cpu_reports_that_cpu() { + // Rust's standard library has no affinity call, so pin through the host. + #[link(name = "kernel32")] + #[allow(unsafe_code)] + unsafe extern "system" { + fn SetThreadAffinityMask(thread: ffi::Handle, mask: usize) -> usize; + } + std::thread::spawn(|| { + // The process may be limited to other CPUs than the first, so pin to + // the lowest CPU that the thread may use now. + let first = *parse_cpu_list(&cpu_affinity().expect("affinity")) + .first() + .expect("an allowed CPU"); + let mut group = ffi::GroupAffinity::default(); + // SAFETY: the pseudo-handle of the calling thread needs no closing, + // and `group` is a live, writable `GROUP_AFFINITY`. + #[allow(unsafe_code)] + let status = + unsafe { ffi::GetThreadGroupAffinity(ffi::GetCurrentThread(), &raw mut group) }; + assert_ne!(status, 0, "the host did not name the thread's CPUs"); + let lowest = 1_usize << group.mask.trailing_zeros(); + // SAFETY: the pseudo-handle of the calling thread needs no closing, + // and the mask names one CPU of the group the thread is in. + #[allow(unsafe_code)] + let before = unsafe { SetThreadAffinityMask(ffi::GetCurrentThread(), lowest) }; + assert_ne!(before, 0, "the host refused to pin the thread"); + assert_eq!(cpu_affinity().expect("affinity"), first.to_string()); + }) + .join() + .expect("the pinned thread"); + } + + #[test] + fn a_process_that_does_not_exist_is_gone() { + let pid = 0x7FFF_FFF0; + assert_eq!(peak_memory_bytes(pid), Err(PlatformError::ProcessGone(pid))); + assert_eq!(process_regions(pid), Err(PlatformError::ProcessGone(pid))); + } + } +} diff --git a/crates/bamts-runtime/Cargo.toml b/crates/bamts-runtime/Cargo.toml index b6a1fe01..03f67c1d 100644 --- a/crates/bamts-runtime/Cargo.toml +++ b/crates/bamts-runtime/Cargo.toml @@ -23,6 +23,7 @@ aot-main = [] bamts-cancel.workspace = true bamts-bytecode.workspace = true bamts-native = { workspace = true, features = ["gc"] } +bamts-platform.workspace = true icu_properties = { version = "=2.3.0", features = ["compiled_data"] } unicode-normalization = "=0.1.24" diff --git a/crates/bamts-runtime/src/builtins/annex_b.rs b/crates/bamts-runtime/src/builtins/annex_b.rs index 11e59b4a..4c284fcb 100644 --- a/crates/bamts-runtime/src/builtins/annex_b.rs +++ b/crates/bamts-runtime/src/builtins/annex_b.rs @@ -229,15 +229,12 @@ fn lookup_legacy_accessor( let key = machine.observable_property_key(args.first().copied().unwrap_or(Value::UNDEFINED))?; let mut current = Some(object); while let Some(object) = current { - if let Some(property) = machine.own_descriptor(object, &key)? { - return Ok(BuiltinOutcome::Value(match property { - Property::Accessor { getter, setter, .. } => match slot { - LegacyAccessorSlot::Getter => getter, - LegacyAccessorSlot::Setter => setter, - } - .unwrap_or(Value::UNDEFINED), - Property::Data { .. } => Value::UNDEFINED, - })); + if let Some(descriptor) = machine.internal_get_own_property(object, &key)? { + let accessor = match slot { + LegacyAccessorSlot::Getter => descriptor.getter, + LegacyAccessorSlot::Setter => descriptor.setter, + }; + return Ok(BuiltinOutcome::Value(accessor.unwrap_or(Value::UNDEFINED))); } current = machine.internal_get_prototype_of(object)?; } @@ -693,7 +690,9 @@ fn properties_of(heap: &mut [HeapEntry], object: Value) -> &mut PropertyMap { | HeapEntry::Script { properties, .. } | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } - | HeapEntry::Date { properties, .. } => properties, + | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } => properties, _ => panic!("Annex B property target must be an ordinary object"), } } @@ -711,7 +710,9 @@ fn clone_property(heap: &[HeapEntry], object: Value, name: &str) -> Property { | HeapEntry::Script { properties, .. } | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } - | HeapEntry::Date { properties, .. } => properties, + | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } => properties, _ => panic!("Annex B property source must be an ordinary object"), }; properties @@ -878,84 +879,6 @@ fn proto_getter( )) } -/// Ordinary `[[SetPrototypeOf]]` over the machine heap representation: -/// same-prototype fast path, non-extensible rejection, and the cycle walk -/// that stops at exotic `[[GetPrototypeOf]]` (Proxy) carriers without running -/// their traps. Returns the spec Boolean status; failures map to a `TypeError` -/// at the accessor call site. -fn ordinary_set_prototype_of( - machine: &mut Machine<'_, H>, - object: Value, - prototype: Option, -) -> Result { - if machine.internal_get_prototype_of(object)? == prototype { - return Ok(true); - } - let Some(index) = machine.runtime_slot(object).map_err(EvalFailure::Runtime)? else { - return Ok(false); - }; - let extensible = match &machine.heap[index] { - HeapEntry::Object { extensible, .. } - | HeapEntry::Array { extensible, .. } - | HeapEntry::Function { extensible, .. } - | HeapEntry::Script { extensible, .. } - | HeapEntry::RegExp { extensible, .. } - | HeapEntry::Date { extensible, .. } - | HeapEntry::BuiltinIterator { extensible, .. } - | HeapEntry::Collection { extensible, .. } - | HeapEntry::TypedArray { extensible, .. } - | HeapEntry::DataView { extensible, .. } - | HeapEntry::ArrayBuffer { extensible, .. } - | HeapEntry::SharedArrayBuffer { extensible, .. } - | HeapEntry::Generator { extensible, .. } - | HeapEntry::AsyncGenerator { extensible, .. } - | HeapEntry::AsyncFromSync { extensible, .. } - | HeapEntry::Promise { extensible, .. } - | HeapEntry::DisposableStack { extensible, .. } - | HeapEntry::ProcessEnv { extensible, .. } - | HeapEntry::Timeout { extensible, .. } - | HeapEntry::WeakRef { extensible, .. } - | HeapEntry::FinalizationRegistry { extensible, .. } - | HeapEntry::ProxyRevoker { extensible, .. } - | HeapEntry::NativeFunction { extensible, .. } => *extensible, - HeapEntry::Proxy { .. } => { - return machine.internal_is_extensible(object); - } - HeapEntry::Vacant - | HeapEntry::String(_) - | HeapEntry::BigInt(_) - | HeapEntry::ModuleNamespace { .. } - | HeapEntry::ExternalModuleNamespace { .. } - | HeapEntry::HashState { .. } - | HeapEntry::Symbol { .. } - | HeapEntry::PrivateName { .. } - | HeapEntry::Iterator { .. } - | HeapEntry::PromiseResolver { .. } - | HeapEntry::PromiseAll { .. } - | HeapEntry::PromiseAllElement { .. } - | HeapEntry::AsyncActivation { .. } => return Ok(false), - }; - if !extensible { - return Ok(false); - } - let mut candidate = prototype; - let mut traversed = 0; - while let Some(value) = candidate { - if value == object { - return Ok(false); - } - candidate = machine.internal_get_prototype_of(value)?; - traversed += 1; - if traversed > machine.heap.len() { - return Ok(false); - } - } - if !machine.internal_set_prototype_of(object, prototype)? { - return Err(type_error("Cannot set prototype: object is not extensible")); - } - Ok(true) -} - fn is_annex_object(machine: &Machine<'_, H>, value: Value) -> Result { let Some(index) = machine.runtime_slot(value).map_err(EvalFailure::Runtime)? else { return Ok(false); @@ -990,7 +913,7 @@ fn proto_setter( } let object = machine.value_to_object(this)?; let prototype = (value != Value::NULL).then_some(value); - if ordinary_set_prototype_of(machine, object, prototype)? { + if machine.internal_set_prototype_of(object, prototype)? { Ok(BuiltinOutcome::Value(Value::UNDEFINED)) } else { Err(type_error("Object.prototype.__proto__ setter failed")) @@ -2383,7 +2306,7 @@ mod tests { let matched = Match { range: 3..6, captures: vec![None, Some(3..4), None, Some(5..6)], - named: std::collections::BTreeMap::new(), + named: Vec::new(), }; record_legacy_match(&mut machine, &input, &matched).unwrap(); for (property, expected) in [ diff --git a/crates/bamts-runtime/src/builtins/array.rs b/crates/bamts-runtime/src/builtins/array.rs index 77f58a6c..a4894499 100644 --- a/crates/bamts-runtime/src/builtins/array.rs +++ b/crates/bamts-runtime/src/builtins/array.rs @@ -1,13 +1,11 @@ -use std::cmp::Ordering; use std::collections::BTreeMap; use bamts_bytecode::EcmaString; use bamts_native::{Decoded, Value}; -use super::{ - allocate_array, allocate_string, define_data, install_constructor_function, install_function, - range_error, to_integer_or_infinity, type_error, value_number, -}; +use super::array_generic as generic; +use super::{allocate_string, array_es2023}; +use super::{define_data, install_constructor_function, install_function, type_error}; use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; use crate::{ EvalFailure, HeapEntry, Host, IterationKind, Machine, Property, PropertyKey, PropertyMap, @@ -19,50 +17,73 @@ pub(super) fn install( builtins: &mut BuiltinTable, ) { let prototype = builtins.array_prototype(); - let constructor = install_constructor_function(heap, builtins, "Array", 1, constructor::); + let constructor = + install_constructor_function(heap, builtins, "Array", 1, generic::constructor::); builtins.set_constructor_prototype(heap, constructor, prototype); globals.insert(EcmaString::encode("Array"), constructor); for (name, length, handler) in [ - ("isArray", 1, is_array:: as BuiltinHandler), - ("from", 1, from::), - ("of", 0, of::), + ( + "isArray", + 1, + generic::is_array_method:: as BuiltinHandler, + ), + ("from", 1, generic::from::), + ("of", 0, generic::of::), ] { let function = install_function(heap, builtins, name, length, handler); define_static(heap, constructor, name, function); } for (name, length, handler) in [ ("toString", 0, to_string:: as BuiltinHandler), - ("push", 1, push:: as BuiltinHandler), - ("pop", 0, pop::), - ("shift", 0, shift::), - ("unshift", 1, unshift::), - ("slice", 2, slice::), - ("splice", 2, splice::), - ("concat", 1, concat::), - ("join", 1, join::), - ("indexOf", 1, index_of::), - ("lastIndexOf", 1, last_index_of::), - ("includes", 1, includes::), - ("find", 1, find::), - ("findIndex", 1, find_index::), - ("findLast", 1, find_last::), - ("filter", 1, filter::), - ("map", 1, map::), - ("forEach", 1, for_each::), - ("reduce", 1, reduce::), - ("reduceRight", 1, reduce_right::), - ("some", 1, some::), - ("every", 1, every::), - ("sort", 1, sort::), - ("reverse", 0, reverse::), - ("flat", 0, flat::), - ("flatMap", 1, flat_map::), - ("fill", 1, fill::), - ("at", 1, at::), + ("toLocaleString", 0, to_locale_string::), + ("copyWithin", 2, copy_within::), + ("push", 1, generic::push::), + ("pop", 0, generic::pop::), + ("shift", 0, generic::shift::), + ("unshift", 1, generic::unshift::), + ("slice", 2, generic::slice::), + ("splice", 2, generic::splice::), + ("concat", 1, generic::concat::), + ("join", 1, generic::join::), + ("indexOf", 1, generic::index_of::), + ("lastIndexOf", 1, generic::last_index_of::), + ("includes", 1, generic::includes::), + ("find", 1, generic::find::), + ("findIndex", 1, generic::find_index::), + ("filter", 1, generic::filter::), + ("map", 1, generic::map::), + ("forEach", 1, generic::for_each::), + ("reduce", 1, generic::reduce::), + ("reduceRight", 1, generic::reduce_right::), + ("some", 1, generic::some::), + ("every", 1, generic::every::), + ("sort", 1, generic::sort::), + ("reverse", 0, generic::reverse::), + ("flat", 0, generic::flat::), + ("flatMap", 1, generic::flat_map::), + ("fill", 1, generic::fill::), ] { let function = install_function(heap, builtins, name, length, handler); define_data(heap, prototype, name, function); } + let species = install_function( + heap, + builtins, + "get [Symbol.species]", + 0, + super::regexp_protocol::species_getter::, + ); + super::define_symbol_property( + heap, + constructor, + builtins.symbol_species(), + Property::Accessor { + getter: Some(species), + setter: None, + enumerable: false, + configurable: true, + }, + ); let keys = install_function(heap, builtins, "keys", 0, keys_iterator::); let values = install_function(heap, builtins, "values", 0, values_iterator::); let entries = install_function(heap, builtins, "entries", 0, entries_iterator::); @@ -141,154 +162,37 @@ fn define_static(heap: &mut [HeapEntry], constructor: Value, name: &str, value: ); } -fn constructor( - machine: &mut Machine<'_, H>, - _this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - let elements = if args.len() == 1 { - let n = value_number(args[0]); - if n.is_finite() && n >= 0.0 && n.fract() == 0.0 && n <= u32::MAX as f64 { - vec![Value::HOLE; n as usize] - } else { - args.to_vec() - } - } else { - args.to_vec() - }; - Ok(BuiltinOutcome::Value(allocate_array(machine, elements)?)) -} -fn is_array( - machine: &mut Machine<'_, H>, - _this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - Ok(BuiltinOutcome::Value(Value::boolean( - machine - .array_elements(args.first().copied().unwrap_or(Value::UNDEFINED))? - .is_some(), - ))) -} -fn from( +/// The most elements a non-array receiver is read into a list for. A receiver may +/// claim a length of 2^53 - 1. +const MAX_ARRAY_LIKE_ELEMENTS: usize = 1 << 20; + +/// Reads `this` as a list. Array methods are generic, so any object with a length +/// works; one that is not an array is read once, up front, with a missing index +/// kept as a hole. +pub(super) fn elements( machine: &mut Machine<'_, H>, - _this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - let source = args.first().copied().unwrap_or(Value::UNDEFINED); - let callback = args - .get(1) - .copied() - .filter(|value| *value != Value::UNDEFINED); - if let Some(callback) = callback - && !machine.is_callable(callback)? + this: Value, +) -> Result, EvalFailure> { + if let Some(values) = machine.array_elements(this)? + && !values.contains(&Value::HOLE) { - return Err(type_error("Array.from mapper is not callable")); - } - let this_arg = args.get(2).copied().unwrap_or(Value::UNDEFINED); - let iterator_key = machine.to_property_key(machine.intrinsics.builtins.symbol_iterator())?; - let iterator_method = machine.get_property_key(source, &iterator_key)?; - if matches!( - iterator_method.decode(), - Some(Decoded::Undefined | Decoded::Null) - ) { - return from_array_like(machine, source, callback, this_arg); - } - if !machine.is_callable(iterator_method)? { - return Err(type_error("value is not iterable")); - } - let iterator_object = machine.call_value(iterator_method, source, &[])?; - if !machine.is_object(iterator_object) { - return Err(type_error("iterator method returned a non-object")); + return Ok(values); } - let next = machine.get_named_property(iterator_object, "next")?; - let iterator = machine.create_protocol_iterator(iterator_object, next)?; - let mut elements = Vec::new(); - loop { - let (done, value) = machine.iterator_next(iterator)?; - if done { - break; - } - let value = if let Some(callback) = callback { - match machine.call_value( - callback, - this_arg, - &[value, crate::number_value(elements.len() as f64)], - ) { - Ok(value) => value, - Err(failure) => { - let (close, _) = machine.close_iterator_raw(iterator); - if let Err(EvalFailure::Runtime(kind)) = close { - return Err(EvalFailure::Runtime(kind)); - } - return Err(failure); - } - } - } else { - value - }; - elements.push(value); - } - Ok(BuiltinOutcome::Value(allocate_array(machine, elements)?)) -} - -fn from_array_like( - machine: &mut Machine<'_, H>, - source: Value, - callback: Option, - this_arg: Value, -) -> Result { - const MAX_SAFE_INTEGER: f64 = 9_007_199_254_740_991.0; - let length_value = machine.get_named_property(source, "length")?; - let number = value_number(machine.coerce_number_observable(length_value)?); - let integer = if number.is_nan() || number == 0.0 { - 0.0 - } else { - number.trunc() - }; - let length = if integer <= 0.0 { - 0.0 - } else if integer.is_infinite() { - MAX_SAFE_INTEGER - } else { - integer.min(MAX_SAFE_INTEGER) - }; - if length > machine.limits.max_heap_slots as f64 { - return Err(range_error("Invalid Array.from length")); + let object = array_es2023::to_object(machine, this)?; + let length = array_es2023::length_of_array_like(machine, object)?; + if length > MAX_ARRAY_LIKE_ELEMENTS { + return Err(type_error("Array method received too long an array-like")); } - let length = length as usize; - let mut elements = Vec::with_capacity(length); + let mut values = Vec::with_capacity(length); for index in 0..length { - let value = machine.get_named_property(source, &index.to_string())?; - let value = match callback { - Some(callback) => machine.call_value( - callback, - this_arg, - &[value, crate::number_value(index as f64)], - )?, - None => value, - }; - elements.push(value); + let key = array_es2023::index_key(index); + values.push(if machine.internal_has_property(object, &key)? { + machine.get_property_key(object, &key)? + } else { + Value::HOLE + }); } - Ok(BuiltinOutcome::Value(allocate_array(machine, elements)?)) -} -fn of( - machine: &mut Machine<'_, H>, - _this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - Ok(BuiltinOutcome::Value(allocate_array( - machine, - args.to_vec(), - )?)) -} -fn elements(machine: &Machine<'_, H>, this: Value) -> Result, EvalFailure> { - machine - .array_elements(this)? - .ok_or_else(|| type_error("Array method called on incompatible receiver")) + Ok(values) } /// Validates that `this` is an Array receiver without materializing its /// elements. The iterator factories only need receiver validation — the @@ -299,189 +203,14 @@ fn array_slot(machine: &Machine<'_, H>, object: Value) -> Result<(), Ev let Some(index) = machine.runtime_slot(object).map_err(EvalFailure::Runtime)? else { return Err(type_error("Array method called on incompatible receiver")); }; - if !matches!(machine.heap[index], HeapEntry::Array { .. }) { + if !matches!( + machine.heap[index], + HeapEntry::Array { .. } | HeapEntry::TypedArray { .. } + ) { return Err(type_error("Array method called on incompatible receiver")); } Ok(()) } -fn write_elements( - machine: &mut Machine<'_, H>, - this: Value, - values: Vec, -) -> Result<(), EvalFailure> { - machine.replace_array_elements(this, values) -} -fn push( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - let mut values = elements(machine, this)?; - values.extend_from_slice(args); - let len = values.len(); - write_elements(machine, this, values)?; - Ok(BuiltinOutcome::Value(crate::number_value(len as f64))) -} -fn pop( - machine: &mut Machine<'_, H>, - this: Value, - _args: &[Value], - _constructing: bool, -) -> Result { - let mut values = elements(machine, this)?; - let value = values - .pop() - .filter(|v| *v != Value::HOLE) - .unwrap_or(Value::UNDEFINED); - write_elements(machine, this, values)?; - Ok(BuiltinOutcome::Value(value)) -} -fn shift( - machine: &mut Machine<'_, H>, - this: Value, - _args: &[Value], - _constructing: bool, -) -> Result { - let mut values = elements(machine, this)?; - let value = if values.is_empty() { - Value::UNDEFINED - } else { - values.remove(0) - }; - write_elements(machine, this, values)?; - Ok(BuiltinOutcome::Value(if value == Value::HOLE { - Value::UNDEFINED - } else { - value - })) -} -fn unshift( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - let old = elements(machine, this)?; - let mut values = Vec::with_capacity(args.len() + old.len()); - values.extend_from_slice(args); - values.extend(old); - let len = values.len(); - write_elements(machine, this, values)?; - Ok(BuiltinOutcome::Value(crate::number_value(len as f64))) -} -fn relative_index( - machine: &Machine<'_, H>, - value: Value, - len: usize, -) -> Result { - let n = to_integer_or_infinity(machine, value)?; - Ok(if n < 0.0 { - (len as f64 + n).max(0.0) as usize - } else { - n.min(len as f64) as usize - }) -} -fn slice( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let start = relative_index( - machine, - args.first().copied().unwrap_or(Value::int32(0)), - values.len(), - )?; - let end = relative_index( - machine, - args.get(1) - .copied() - .unwrap_or(crate::number_value(values.len() as f64)), - values.len(), - )?; - Ok(BuiltinOutcome::Value(allocate_array( - machine, - if end > start { - values[start..end].to_vec() - } else { - Vec::new() - }, - )?)) -} -fn splice( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let mut values = elements(machine, this)?; - let start = relative_index( - machine, - args.first().copied().unwrap_or(Value::int32(0)), - values.len(), - )?; - let delete = if args.len() < 2 { - values.len() - start - } else { - to_integer_or_infinity(machine, args[1])? - .max(0.0) - .min((values.len() - start) as f64) as usize - }; - let removed: Vec<_> = values - .splice(start..start + delete, args.iter().copied().skip(2)) - .collect(); - write_elements(machine, this, values)?; - Ok(BuiltinOutcome::Value(allocate_array(machine, removed)?)) -} -fn concat( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let mut out = elements(machine, this)?; - for arg in args { - if let Some(values) = machine.array_elements(*arg)? { - out.extend(values) - } else { - out.push(*arg) - } - } - Ok(BuiltinOutcome::Value(allocate_array(machine, out)?)) -} -fn join( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let length = machine.array_length(this)?; - let separator = if args.is_empty() || args[0] == Value::UNDEFINED { - EcmaString::encode(",") - } else { - machine.to_string(args[0])? - }; - let mut output = bamts_bytecode::EcmaStringBuilder::new(); - for index in 0..length { - if index != 0 { - for &unit in separator.as_units() { - output.push_unit(unit); - } - } - let value = machine.get_named_property(this, &index.to_string())?; - if !matches!(value.decode(), Some(Decoded::Undefined | Decoded::Null)) { - for &unit in machine.to_string(value)?.as_units() { - output.push_unit(unit); - } - } - } - Ok(BuiltinOutcome::Value(allocate_string( - machine, - output.finish(), - )?)) -} fn to_string( machine: &mut Machine<'_, H>, this: Value, @@ -501,491 +230,107 @@ fn to_string( &[], )?)) } -fn from_index( - machine: &Machine<'_, H>, - arg: Option, - len: usize, - reverse: bool, -) -> Result, EvalFailure> { - let default = if reverse { len.saturating_sub(1) } else { 0 }; - let Some(value) = arg else { - return Ok((len > 0).then_some(default)); - }; - let n = to_integer_or_infinity(machine, value)?; - if n >= len as f64 { - return Ok(None); - }; - if n < -(len as f64) { - return Ok((!reverse && len > 0).then_some(0)); - } - Ok(Some(if n < 0.0 { - (len as f64 + n) as usize - } else { - n as usize - })) -} -fn index_of( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let needle = args.first().copied().unwrap_or(Value::UNDEFINED); - let found = match from_index(machine, args.get(1).copied(), values.len(), false)? { - Some(start) => values - .iter() - .enumerate() - .skip(start) - .find(|(_, v)| **v != Value::HOLE && machine.strict_equal(**v, needle)) - .map(|(i, _)| i as f64) - .unwrap_or(-1.0), - None => -1.0, - }; - Ok(BuiltinOutcome::Value(crate::number_value(found))) -} -fn last_index_of( +/// `Array.prototype.toLocaleString`: each element's own `toLocaleString`, +/// joined with a comma. ECMA-402 passes `locales` and `options` on to every +/// element. `object` stays rooted because every element can run user code. +fn to_locale_string( machine: &mut Machine<'_, H>, this: Value, args: &[Value], - _: bool, + _constructing: bool, ) -> Result { - let values = elements(machine, this)?; - let needle = args.first().copied().unwrap_or(Value::UNDEFINED); - let start = from_index(machine, args.get(1).copied(), values.len(), true)?; - let found = start - .and_then(|s| { - (0..=s) - .rev() - .find(|i| values[*i] != Value::HOLE && machine.strict_equal(values[*i], needle)) + let object = array_es2023::to_object(machine, this)?; + let argument = |index: usize| args.get(index).copied().unwrap_or(Value::UNDEFINED); + let forwarded = [argument(0), argument(1)]; + let joined = super::with_roots(machine, &[object], |machine, _| { + generic::guarded_join(machine, object, |machine| { + locale_join(machine, object, &forwarded) }) - .map(|i| i as f64) - .unwrap_or(-1.0); - Ok(BuiltinOutcome::Value(crate::number_value(found))) + })?; + Ok(BuiltinOutcome::Value(allocate_string(machine, joined)?)) } -fn includes( +fn locale_join( machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let needle = args.first().copied().unwrap_or(Value::UNDEFINED); - let found = - from_index(machine, args.get(1).copied(), values.len(), false)?.is_some_and(|start| { - values.iter().skip(start).any(|v| { - machine.same_value_zero( - if *v == Value::HOLE { - Value::UNDEFINED - } else { - *v - }, - needle, - ) - }) - }); - Ok(BuiltinOutcome::Value(Value::boolean(found))) -} -fn callback(args: &[Value]) -> Result { - args.first() - .copied() - .filter(|v| *v != Value::UNDEFINED) - .ok_or_else(|| type_error("callback is not a function")) -} -fn find( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - for (i, v) in values.iter().enumerate() { - let value = if *v == Value::HOLE { - Value::UNDEFINED - } else { - *v - }; - if machine.call_truthy( - cb, - Value::UNDEFINED, - &[value, crate::number_value(i as f64), this], - )? { - return Ok(BuiltinOutcome::Value(value)); + object: Value, + forwarded: &[Value], +) -> Result { + let length = array_es2023::length_of_array_like(machine, object)?; + let mut output = bamts_bytecode::EcmaStringBuilder::new(); + for index in 0..length { + if index != 0 { + output.push_unit(u16::from(b',')); } - } - Ok(BuiltinOutcome::Value(Value::UNDEFINED)) -} -fn find_index( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - for (i, v) in values.iter().enumerate() { - let value = if *v == Value::HOLE { - Value::UNDEFINED - } else { - *v - }; - if machine.call_truthy( - cb, - Value::UNDEFINED, - &[value, crate::number_value(i as f64), this], - )? { - return Ok(BuiltinOutcome::Value(crate::number_value(i as f64))); + let element = machine.get_property_key(object, &array_es2023::index_key(index))?; + if matches!(element.decode(), Some(Decoded::Undefined | Decoded::Null)) { + continue; } - } - Ok(BuiltinOutcome::Value(crate::number_value(-1.0))) -} -fn find_last( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - for i in (0..values.len()).rev() { - let value = if values[i] == Value::HOLE { - Value::UNDEFINED - } else { - values[i] - }; - if machine.call_truthy( - cb, - Value::UNDEFINED, - &[value, crate::number_value(i as f64), this], - )? { - return Ok(BuiltinOutcome::Value(value)); + let method = machine.get_named_property(element, "toLocaleString")?; + let text = machine.call_value(method, element, forwarded)?; + for &unit in machine.coerce_string_observable(text)?.as_units() { + output.push_unit(unit); } } - Ok(BuiltinOutcome::Value(Value::UNDEFINED)) + Ok(output.finish()) } -fn filter( +/// `Array.prototype.copyWithin`. Each step reads and writes through the +/// object's own internal methods, so a hole deletes its destination and a +/// proxy or accessor observes every step in specification order. +fn copy_within( machine: &mut Machine<'_, H>, this: Value, args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - let mut out = Vec::new(); - for (i, v) in values - .iter() - .enumerate() - .filter(|(_, v)| **v != Value::HOLE) - { - if machine.call_truthy( - cb, - Value::UNDEFINED, - &[*v, crate::number_value(i as f64), this], - )? { - out.push(*v) - } - } - Ok(BuiltinOutcome::Value(allocate_array(machine, out)?)) -} -fn map( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - let mut out = vec![Value::HOLE; values.len()]; - for (i, v) in values - .iter() - .enumerate() - .filter(|(_, v)| **v != Value::HOLE) - { - out[i] = machine.call_value( - cb, - Value::UNDEFINED, - &[*v, crate::number_value(i as f64), this], - )? - } - Ok(BuiltinOutcome::Value(allocate_array(machine, out)?)) -} -fn for_each( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, + _constructing: bool, ) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - for (i, v) in values - .iter() - .enumerate() - .filter(|(_, v)| **v != Value::HOLE) - { - machine.call_value( - cb, - Value::UNDEFINED, - &[*v, crate::number_value(i as f64), this], - )?; - } - Ok(BuiltinOutcome::Value(Value::UNDEFINED)) + let object = array_es2023::to_object(machine, this)?; + let depth = machine.native_roots.len(); + machine.push_native_roots(depth, &[object]); + let copied = copy_within_steps(machine, object, args); + machine.pop_native_roots(depth); + copied?; + Ok(BuiltinOutcome::Value(object)) } -fn reduce_impl( +fn copy_within_steps( machine: &mut Machine<'_, H>, - this: Value, + object: Value, args: &[Value], - reverse: bool, -) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - let indices: Vec<_> = if reverse { - (0..values.len()).rev().collect() - } else { - (0..values.len()).collect() - }; - let mut iter = indices.into_iter().filter(|i| values[*i] != Value::HOLE); - let mut acc = if let Some(initial) = args.get(1) { - *initial - } else { - let i = iter - .next() - .ok_or_else(|| type_error("Reduce of empty array with no initial value"))?; - values[i] +) -> Result<(), EvalFailure> { + let length = array_es2023::length_of_array_like(machine, object)?; + let target = args.first().copied().unwrap_or(Value::UNDEFINED); + let mut to = super::array_generic::relative_position(machine, target, length)?; + let start = args.get(1).copied().unwrap_or(Value::UNDEFINED); + let mut from = super::array_generic::relative_position(machine, start, length)?; + let end = match args.get(2).copied() { + None | Some(Value::UNDEFINED) => length, + Some(end) => super::array_generic::relative_position(machine, end, length)?, }; - for i in iter { - acc = machine.call_value( - cb, - Value::UNDEFINED, - &[acc, values[i], crate::number_value(i as f64), this], - )? - } - Ok(acc) -} -fn reduce( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - Ok(BuiltinOutcome::Value(reduce_impl( - machine, this, args, false, - )?)) -} -fn reduce_right( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - Ok(BuiltinOutcome::Value(reduce_impl( - machine, this, args, true, - )?)) -} -fn some( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - for (i, v) in values - .iter() - .enumerate() - .filter(|(_, v)| **v != Value::HOLE) - { - if machine.call_truthy( - cb, - Value::UNDEFINED, - &[*v, crate::number_value(i as f64), this], - )? { - return Ok(BuiltinOutcome::Value(Value::TRUE)); - } + let mut count = end.saturating_sub(from).min(length - to); + let backward = from < to && to < from + count; + if backward && count > 0 { + from += count - 1; + to += count - 1; } - Ok(BuiltinOutcome::Value(Value::FALSE)) -} -fn every( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let cb = callback(args)?; - for (i, v) in values - .iter() - .enumerate() - .filter(|(_, v)| **v != Value::HOLE) - { - if !machine.call_truthy( - cb, - Value::UNDEFINED, - &[*v, crate::number_value(i as f64), this], - )? { - return Ok(BuiltinOutcome::Value(Value::FALSE)); + while count > 0 { + let from_key = array_es2023::index_key(from); + let to_key = array_es2023::index_key(to); + if machine.internal_has_property(object, &from_key)? { + let value = machine.get_property_key(object, &from_key)?; + machine.set_data_property_key(object, to_key, value)?; + } else if !machine.internal_delete(object, &to_key)? { + return Err(type_error("Cannot delete property during copyWithin")); } - } - Ok(BuiltinOutcome::Value(Value::TRUE)) -} -fn sort( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let comparator = args.first().copied().filter(|v| *v != Value::UNDEFINED); - let mut present: Vec<(usize, Value)> = values - .into_iter() - .enumerate() - .filter(|(_, v)| *v != Value::HOLE) - .collect(); - let mut error = None; - present.sort_by(|(ia, a), (ib, b)| { - if error.is_some() { - return Ordering::Equal; - } - let ordering = if *a == Value::UNDEFINED { - if *b == Value::UNDEFINED { - Ordering::Equal + count -= 1; + if count > 0 { + if backward { + from -= 1; + to -= 1; } else { - Ordering::Greater - } - } else if *b == Value::UNDEFINED { - Ordering::Less - } else if let Some(cb) = comparator { - match machine - .call_value(cb, Value::UNDEFINED, &[*a, *b]) - .and_then(|v| machine.to_number(v)) - { - Ok(v) => value_number(v).partial_cmp(&0.0).unwrap_or(Ordering::Equal), - Err(e) => { - error = Some(e); - Ordering::Equal - } - } - } else { - match (machine.to_string(*a), machine.to_string(*b)) { - (Ok(a), Ok(b)) => a.cmp(&b), - (Err(e), _) | (_, Err(e)) => { - error = Some(e); - Ordering::Equal - } + from += 1; + to += 1; } - }; - ordering.then_with(|| ia.cmp(ib)) - }); - if let Some(e) = error { - return Err(e); - } - let len = present.len(); - let mut out: Vec<_> = present.into_iter().map(|(_, v)| v).collect(); - out.resize(machine.array_length(this)?, Value::HOLE); - debug_assert!(len <= out.len()); - write_elements(machine, this, out)?; - Ok(BuiltinOutcome::Value(this)) -} -fn reverse( - machine: &mut Machine<'_, H>, - this: Value, - _: &[Value], - _: bool, -) -> Result { - let mut values = elements(machine, this)?; - values.reverse(); - write_elements(machine, this, values)?; - Ok(BuiltinOutcome::Value(this)) -} -fn flatten( - machine: &Machine<'_, H>, - values: Vec, - depth: usize, - out: &mut Vec, -) -> Result<(), EvalFailure> { - for value in values { - if depth > 0 - && let Some(inner) = machine.array_elements(value)? - { - flatten(machine, inner, depth - 1, out)?; - continue; - } - if value != Value::HOLE { - out.push(value) } } Ok(()) } -fn flat( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let depth = to_integer_or_infinity(machine, args.first().copied().unwrap_or(Value::int32(1)))? - .max(0.0) as usize; - let mut out = Vec::new(); - flatten(machine, values, depth, &mut out)?; - Ok(BuiltinOutcome::Value(allocate_array(machine, out)?)) -} -fn flat_map( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let BuiltinOutcome::Value(mapped) = map(machine, this, args, false)? else { - unreachable!() - }; - flat(machine, mapped, &[Value::int32(1)], false) -} -fn fill( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let mut values = elements(machine, this)?; - let start = relative_index( - machine, - args.get(1).copied().unwrap_or(Value::int32(0)), - values.len(), - )?; - let end = relative_index( - machine, - args.get(2) - .copied() - .unwrap_or(crate::number_value(values.len() as f64)), - values.len(), - )?; - for slot in values.iter_mut().take(end).skip(start) { - *slot = args.first().copied().unwrap_or(Value::UNDEFINED) - } - write_elements(machine, this, values)?; - Ok(BuiltinOutcome::Value(this)) -} -fn at( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - let values = elements(machine, this)?; - let n = to_integer_or_infinity(machine, args.first().copied().unwrap_or(Value::UNDEFINED))?; - let index = if n < 0.0 { values.len() as f64 + n } else { n }; - let value = if index < 0.0 || index >= values.len() as f64 { - Value::UNDEFINED - } else { - let v = values[index as usize]; - if v == Value::HOLE { - Value::UNDEFINED - } else { - v - } - }; - Ok(BuiltinOutcome::Value(value)) -} fn keys_iterator( machine: &mut Machine<'_, H>, @@ -1029,12 +374,72 @@ fn entries_iterator( )?)) } +#[cfg(test)] +mod copy_within_tests { + use super::super::allocate_array; + use super::super::test_support::{TestHost, blank_program}; + use super::*; + use crate::Limits; + + fn array_of(machine: &mut Machine<'_, TestHost>, values: &[i32]) -> Value { + let elements = values.iter().map(|value| Value::int32(*value as u32)); + allocate_array(machine, elements.collect()).unwrap() + } + + fn copy(machine: &mut Machine<'_, TestHost>, array: Value, args: &[Value]) -> Vec { + let method = machine + .get_named_property(array, "copyWithin") + .expect("copyWithin is installed"); + let result = machine.call_value(method, array, args).unwrap(); + assert_eq!(result, array, "copyWithin returns its receiver"); + machine.array_elements(array).unwrap().unwrap() + } + + #[test] + fn overlapping_ranges_copy_as_if_through_a_temporary() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let forward = array_of(&mut machine, &[1, 2, 3, 4, 5]); + assert_eq!( + copy(&mut machine, forward, &[Value::int32(0), Value::int32(1)]), + [2, 3, 4, 5, 5].map(Value::int32) + ); + let backward = array_of(&mut machine, &[1, 2, 3, 4, 5]); + assert_eq!( + copy(&mut machine, backward, &[Value::int32(1), Value::int32(0)]), + [1, 1, 2, 3, 4].map(Value::int32) + ); + } + + #[test] + fn a_hole_at_the_source_deletes_the_destination() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = allocate_array( + &mut machine, + vec![Value::HOLE, Value::int32(2), Value::int32(3)], + ) + .unwrap(); + assert_eq!( + copy( + &mut machine, + array, + &[Value::int32(2), Value::int32(0), Value::int32(1)] + ), + [Value::HOLE, Value::int32(2), Value::HOLE] + ); + } +} + #[cfg(test)] mod tests { use std::cell::RefCell; use bamts_native::Decoded; + use super::super::allocate_array; use super::super::test_support::{TestHost, blank_program, custom_iterable, ordinary_object}; use super::*; use crate::Limits; @@ -1884,6 +1289,95 @@ mod tests { assert!(done); } + fn call_prototype( + machine: &mut Machine<'_, TestHost>, + name: &str, + this: Value, + args: &[Value], + ) -> Result { + let proto = machine.intrinsics.array_prototype; + let method = machine.get_named_property(proto, name)?; + machine.call_value(method, this, args) + } + + #[test] + fn array_methods_read_any_object_with_a_length() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let like = ordinary_object(&mut machine); + machine + .set_data_property(like, "0", Value::int32(7)) + .unwrap(); + machine + .set_data_property(like, "2", Value::int32(9)) + .unwrap(); + machine + .set_data_property(like, "length", Value::int32(3)) + .unwrap(); + + assert_eq!( + elements(&mut machine, like).unwrap(), + [Value::int32(7), Value::HOLE, Value::int32(9)], + "a missing index is a hole, not undefined" + ); + assert_eq!( + call_prototype(&mut machine, "indexOf", like, &[Value::int32(9)]).unwrap(), + Value::int32(2) + ); + } + + #[test] + fn array_mutators_write_back_to_any_object_with_a_length() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let like = ordinary_object(&mut machine); + machine + .set_data_property(like, "0", Value::int32(1)) + .unwrap(); + machine + .set_data_property(like, "1", Value::int32(2)) + .unwrap(); + machine + .set_data_property(like, "length", Value::int32(2)) + .unwrap(); + + call_prototype(&mut machine, "push", like, &[Value::int32(3)]).unwrap(); + assert_eq!( + machine.get_named_property(like, "2").unwrap(), + Value::int32(3) + ); + assert_eq!( + machine.get_named_property(like, "length").unwrap(), + Value::int32(3) + ); + + let popped = call_prototype(&mut machine, "pop", like, &[]).unwrap(); + assert_eq!(popped, Value::int32(3)); + let key = array_es2023::index_key(2); + assert!( + !machine.internal_has_property(like, &key).unwrap(), + "a removed element is deleted" + ); + assert_eq!( + machine.get_named_property(like, "length").unwrap(), + Value::int32(2) + ); + } + + #[test] + fn an_absurd_array_like_length_is_refused_before_allocating() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let like = ordinary_object(&mut machine); + machine + .set_data_property(like, "length", crate::number_value(9_007_199_254_740_991.0)) + .unwrap(); + assert!(elements(&mut machine, like).is_err()); + } + #[test] fn iterator_rejects_non_array_receiver() { let module = blank_program(""); diff --git a/crates/bamts-runtime/src/builtins/array_es2023.rs b/crates/bamts-runtime/src/builtins/array_es2023.rs index 95e4109d..9260ae8e 100644 --- a/crates/bamts-runtime/src/builtins/array_es2023.rs +++ b/crates/bamts-runtime/src/builtins/array_es2023.rs @@ -72,7 +72,10 @@ fn argument(args: &[Value], index: usize) -> Value { args.get(index).copied().unwrap_or(Value::UNDEFINED) } -fn to_object(machine: &mut Machine<'_, H>, value: Value) -> Result { +pub(super) fn to_object( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { machine.value_to_object(value) } @@ -80,7 +83,7 @@ fn number(machine: &mut Machine<'_, H>, value: Value) -> Result( +pub(super) fn integer_or_infinity( machine: &mut Machine<'_, H>, value: Value, ) -> Result { @@ -92,7 +95,7 @@ fn integer_or_infinity( }) } -fn length_of_array_like( +pub(super) fn length_of_array_like( machine: &mut Machine<'_, H>, object: Value, ) -> Result { @@ -107,14 +110,14 @@ fn length_of_array_like( Ok(length.trunc().min(MAX_SAFE_INTEGER) as usize) } -fn index_key(index: usize) -> PropertyKey { +pub(super) fn index_key(index: usize) -> PropertyKey { PropertyKey::Named(EcmaString::encode(&index.to_string())) } /// `ArrayCreate(length)` with a preflight before Rust allocates the backing /// vector. Copy-by-change methods always create the intrinsic Array, never a /// species-derived result. -fn array_create( +pub(super) fn array_create( machine: &mut Machine<'_, H>, length: usize, ) -> Result { @@ -145,7 +148,7 @@ fn array_create( allocate_array(machine, elements) } -fn create_data_property( +pub(super) fn create_data_property( machine: &mut Machine<'_, H>, array: Value, index: usize, @@ -340,6 +343,10 @@ fn stable_sort( values: &mut [Value], comparator: Option, ) -> Result<(), EvalFailure> { + // The comparator can remove these from wherever they came from. + for value in values.iter() { + machine.keep_temporary(*value); + } for end in 1..values.len() { let mut index = end; while index > 0 diff --git a/crates/bamts-runtime/src/builtins/array_generic.rs b/crates/bamts-runtime/src/builtins/array_generic.rs new file mode 100644 index 00000000..4521a268 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/array_generic.rs @@ -0,0 +1,1513 @@ +//! The generic `Array.prototype` algorithms. +//! +//! These methods work on any object with a length, and each step is observable: +//! `length` is read once, then every element goes through `HasProperty`, `Get` +//! and `Set`, so callbacks and proxies see them in specification order. +//! +//! Stringifying an index for every step would dominate on a plain array, so the +//! element accessors below take a shortcut when it is unobservable. A non-hole +//! entry in `elements` is always a plain writable data property, because +//! `define_descriptor` moves any other kind of index into `properties` and leaves +//! a hole behind. Holes take the full path, which reaches the prototype chain. + +use std::cmp::Ordering; + +use bamts_bytecode::{EcmaString, EcmaStringBuilder}; +use bamts_native::{Decoded, Value}; + +use super::array_es2023::{ + array_create, create_data_property, index_key, integer_or_infinity, length_of_array_like, + to_object, +}; +use super::property_descriptor::descriptor_from_property; +use super::{allocate_string, keep_root, range_error, type_error, value_number, with_roots}; +use crate::intrinsics::BuiltinOutcome; +use crate::{EvalFailure, HeapEntry, Host, Machine, Property}; + +const MAX_SAFE_INTEGER: usize = 9_007_199_254_740_991; +/// How deeply `flat` follows nested arrays before treating the input as cyclic. +const MAX_FLATTEN_DEPTH: usize = 4096; + +fn argument(args: &[Value], index: usize) -> Value { + args.get(index).copied().unwrap_or(Value::UNDEFINED) +} + +/// Charges one step, so a claimed length of 2^53 - 1 ends in fuel exhaustion +/// rather than a loop nothing can interrupt. +fn tick(machine: &mut Machine<'_, H>) -> Result<(), EvalFailure> { + machine.consume_fuel(1).map_err(EvalFailure::Runtime) +} + +fn dense_element(machine: &Machine<'_, H>, object: Value, index: usize) -> Option { + let slot = machine.runtime_slot(object).ok().flatten()?; + let HeapEntry::Array { elements, .. } = &machine.heap[slot] else { + return None; + }; + elements + .get(index) + .copied() + .filter(|element| *element != Value::HOLE) +} + +fn has_index( + machine: &mut Machine<'_, H>, + object: Value, + index: usize, +) -> Result { + if dense_element(machine, object, index).is_some() { + return Ok(true); + } + machine.internal_has_property(object, &index_key(index)) +} + +fn get_index( + machine: &mut Machine<'_, H>, + object: Value, + index: usize, +) -> Result { + if let Some(value) = dense_element(machine, object, index) { + return Ok(value); + } + machine.get_property_key(object, &index_key(index)) +} + +fn set_index( + machine: &mut Machine<'_, H>, + object: Value, + index: usize, + value: Value, +) -> Result<(), EvalFailure> { + if dense_element(machine, object, index).is_some() { + let slot = machine + .runtime_slot(object) + .map_err(EvalFailure::Runtime)? + .expect("a dense element belongs to a runtime array"); + if let HeapEntry::Array { elements, .. } = &mut machine.heap[slot] { + elements[index] = value; + } + return Ok(()); + } + machine.set_data_property_key(object, index_key(index), value) +} + +fn delete_index( + machine: &mut Machine<'_, H>, + object: Value, + index: usize, +) -> Result<(), EvalFailure> { + if machine.internal_delete(object, &index_key(index))? { + Ok(()) + } else { + Err(type_error("Cannot delete array element")) + } +} + +/// `CreateDataPropertyOrThrow`. A species can construct anything, proxies +/// included, so only a real Array takes the direct path. +fn create_index( + machine: &mut Machine<'_, H>, + target: Value, + index: usize, + value: Value, +) -> Result<(), EvalFailure> { + if machine.array_len(target)?.is_some() { + return create_data_property(machine, target, index, value); + } + let descriptor = descriptor_from_property(Property::Data { + value, + writable: true, + enumerable: true, + configurable: true, + }); + if machine.internal_define_own_property(target, index_key(index), descriptor)? { + Ok(()) + } else { + Err(type_error("Cannot define array element")) + } +} + +fn set_length( + machine: &mut Machine<'_, H>, + object: Value, + length: usize, +) -> Result<(), EvalFailure> { + machine.set_data_property(object, "length", crate::number_value(length as f64)) +} + +/// `IsArray`, which sees through proxies and throws on a revoked one. +pub(crate) fn is_array( + machine: &Machine<'_, H>, + value: Value, +) -> Result { + let mut current = value; + loop { + let index = machine + .runtime_slot(current) + .map_err(EvalFailure::Runtime)?; + match index.map(|index| &machine.heap[index]) { + Some(HeapEntry::Array { .. }) => return Ok(true), + Some(HeapEntry::Proxy { record }) => match record.live_target() { + Some(target) => current = target, + None => return Err(type_error("IsArray on a revoked proxy")), + }, + _ => return Ok(false), + } + } +} + +/// `ArraySpeciesCreate`. +fn array_species_create( + machine: &mut Machine<'_, H>, + original: Value, + length: usize, +) -> Result { + if !is_array(machine, original)? { + return array_create(machine, length); + } + let mut constructor = machine.get_named_property(original, "constructor")?; + if machine.is_object(constructor) { + let key = machine.to_property_key(machine.intrinsics.builtins.symbol_species())?; + constructor = machine.get_property_key(constructor, &key)?; + if constructor == Value::NULL { + constructor = Value::UNDEFINED; + } + } + if constructor == Value::UNDEFINED { + return array_create(machine, length); + } + if !machine.is_constructor(constructor)? { + return Err(type_error("Array species is not a constructor")); + } + machine.construct_value(constructor, &[crate::number_value(length as f64)]) +} + +fn require_callable(machine: &Machine<'_, H>, callback: Value) -> Result<(), EvalFailure> { + if machine.is_callable(callback)? { + Ok(()) + } else { + Err(type_error("Array callback is not callable")) + } +} + +/// The clamped position a relative argument names, with user conversion running +/// in specification order. +pub(super) fn relative_position( + machine: &mut Machine<'_, H>, + value: Value, + length: usize, +) -> Result { + let relative = integer_or_infinity(machine, value)?; + let length = length as f64; + Ok(if relative < 0.0 { + (length + relative).max(0.0) as usize + } else { + relative.min(length) as usize + }) +} + +pub(super) fn push( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, _| { + let mut length = length_of_array_like(machine, object)?; + if length.saturating_add(args.len()) > MAX_SAFE_INTEGER { + return Err(type_error("Array length exceeds the safe integer limit")); + } + for &value in args { + machine.set_data_property_key(object, index_key(length), value)?; + length += 1; + } + set_length(machine, object, length)?; + Ok(crate::number_value(length as f64)) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn pop( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, depth| { + let length = length_of_array_like(machine, object)?; + if length == 0 { + set_length(machine, object, 0)?; + return Ok(Value::UNDEFINED); + } + let index = length - 1; + let element = get_index(machine, object, index)?; + keep_root(machine, depth, element); + delete_index(machine, object, index)?; + set_length(machine, object, index)?; + Ok(element) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn shift( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, depth| { + let length = length_of_array_like(machine, object)?; + if length == 0 { + set_length(machine, object, 0)?; + return Ok(Value::UNDEFINED); + } + let first = get_index(machine, object, 0)?; + keep_root(machine, depth, first); + for from in 1..length { + tick(machine)?; + move_element(machine, object, from, from - 1)?; + } + delete_index(machine, object, length - 1)?; + set_length(machine, object, length - 1)?; + Ok(first) + }) + .map(BuiltinOutcome::Value) +} + +/// One step of a shifting loop: the destination takes the source's value, or is +/// deleted when the source is absent. +fn move_element( + machine: &mut Machine<'_, H>, + object: Value, + from: usize, + to: usize, +) -> Result<(), EvalFailure> { + if has_index(machine, object, from)? { + let value = get_index(machine, object, from)?; + set_index(machine, object, to, value) + } else { + delete_index(machine, object, to) + } +} + +pub(super) fn unshift( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, _| { + let length = length_of_array_like(machine, object)?; + let count = args.len(); + if count > 0 { + if length.saturating_add(count) > MAX_SAFE_INTEGER { + return Err(type_error("Array length exceeds the safe integer limit")); + } + for index in (0..length).rev() { + tick(machine)?; + move_element(machine, object, index, index + count)?; + } + for (index, &value) in args.iter().enumerate() { + set_index(machine, object, index, value)?; + } + } + set_length(machine, object, length + count)?; + Ok(crate::number_value((length + count) as f64)) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn slice( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, depth| { + let length = length_of_array_like(machine, object)?; + let start = relative_position(machine, argument(args, 0), length)?; + let end = match argument(args, 1) { + Value::UNDEFINED => length, + end => relative_position(machine, end, length)?, + }; + let count = end.saturating_sub(start); + let target = array_species_create(machine, object, count)?; + keep_root(machine, depth, target); + let mut to = 0; + for from in start..end { + tick(machine)?; + if has_index(machine, object, from)? { + let value = get_index(machine, object, from)?; + create_index(machine, target, to, value)?; + } + to += 1; + } + set_length(machine, target, to)?; + Ok(target) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn splice( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, depth| { + let length = length_of_array_like(machine, object)?; + let start = relative_position(machine, argument(args, 0), length)?; + let items = args.get(2..).unwrap_or_default(); + let delete_count = match args.len() { + 0 => 0, + 1 => length - start, + _ => { + let requested = integer_or_infinity(machine, args[1])?; + requested.clamp(0.0, (length - start) as f64) as usize + } + }; + if (length + items.len()).saturating_sub(delete_count) > MAX_SAFE_INTEGER { + return Err(type_error("Array length exceeds the safe integer limit")); + } + let removed = array_species_create(machine, object, delete_count)?; + keep_root(machine, depth, removed); + for offset in 0..delete_count { + tick(machine)?; + let from = start + offset; + if has_index(machine, object, from)? { + let value = get_index(machine, object, from)?; + create_index(machine, removed, offset, value)?; + } + } + set_length(machine, removed, delete_count)?; + let new_length = length - delete_count + items.len(); + if items.len() < delete_count { + for index in start..length - delete_count { + tick(machine)?; + move_element(machine, object, index + delete_count, index + items.len())?; + } + for index in (new_length..length).rev() { + tick(machine)?; + delete_index(machine, object, index)?; + } + } else if items.len() > delete_count { + for index in (start..length - delete_count).rev() { + tick(machine)?; + move_element(machine, object, index + delete_count, index + items.len())?; + } + } + for (offset, &value) in items.iter().enumerate() { + set_index(machine, object, start + offset, value)?; + } + set_length(machine, object, new_length)?; + Ok(removed) + }) + .map(BuiltinOutcome::Value) +} + +/// `IsConcatSpreadable`: the symbol wins, and an array spreads by default. +fn is_concat_spreadable( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + if !machine.is_object(value) { + return Ok(false); + } + let symbol = machine.intrinsics.builtins.symbol_is_concat_spreadable(); + let key = machine.to_property_key(symbol)?; + let spreadable = machine.get_property_key(value, &key)?; + if spreadable != Value::UNDEFINED { + return Ok(machine.to_boolean(spreadable)); + } + is_array(machine, value) +} + +pub(super) fn concat( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, depth| { + let target = array_species_create(machine, object, 0)?; + keep_root(machine, depth, target); + let mut to: usize = 0; + for item in std::iter::once(object).chain(args.iter().copied()) { + if is_concat_spreadable(machine, item)? { + let length = length_of_array_like(machine, item)?; + if to.saturating_add(length) > MAX_SAFE_INTEGER { + return Err(type_error("Array length exceeds the safe integer limit")); + } + for from in 0..length { + tick(machine)?; + if has_index(machine, item, from)? { + let value = get_index(machine, item, from)?; + create_index(machine, target, to, value)?; + } + to += 1; + } + } else { + if to >= MAX_SAFE_INTEGER { + return Err(type_error("Array length exceeds the safe integer limit")); + } + create_index(machine, target, to, item)?; + to += 1; + } + } + set_length(machine, target, to)?; + Ok(target) + }) + .map(BuiltinOutcome::Value) +} + +fn append(output: &mut EcmaStringBuilder, text: &EcmaString) { + for &unit in text.as_units() { + output.push_unit(unit); + } +} + +/// Runs a join over `object`, which an array reached again through its own +/// elements contributes nothing to. Nothing else stops that recursion: it passes +/// through native code, so no activation limit applies. +pub(super) fn guarded_join( + machine: &mut Machine<'_, H>, + object: Value, + body: impl FnOnce(&mut Machine<'_, H>) -> Result, +) -> Result { + if machine.array_join_stack.contains(&object) { + return Ok(EcmaString::default()); + } + machine.array_join_stack.push(object); + let result = body(machine); + machine.array_join_stack.pop(); + result +} + +pub(super) fn join( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + let joined = with_roots(machine, &[object], |machine, _| { + let length = length_of_array_like(machine, object)?; + let separator = match argument(args, 0) { + Value::UNDEFINED => EcmaString::encode(","), + separator => machine.coerce_string_observable(separator)?, + }; + guarded_join(machine, object, |machine| { + let mut output = EcmaStringBuilder::new(); + for index in 0..length { + tick(machine)?; + if index != 0 { + append(&mut output, &separator); + } + let element = get_index(machine, object, index)?; + if !matches!(element.decode(), Some(Decoded::Undefined | Decoded::Null)) { + append(&mut output, &machine.coerce_string_observable(element)?); + } + } + Ok(output.finish()) + }) + })?; + Ok(BuiltinOutcome::Value(allocate_string(machine, joined)?)) +} + +/// The first position `fromIndex` selects, or `None` when it is past the end. +fn start_index( + machine: &mut Machine<'_, H>, + from_index: Value, + length: usize, +) -> Result, EvalFailure> { + let relative = integer_or_infinity(machine, from_index)?; + if relative == f64::INFINITY { + return Ok(None); + } + Ok(Some(if relative >= 0.0 { + relative as usize + } else { + (length as f64 + relative).max(0.0) as usize + })) +} + +pub(super) fn index_of( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + let needle = argument(args, 0); + with_roots(machine, &[object, needle], |machine, _| { + let length = length_of_array_like(machine, object)?; + if length == 0 { + return Ok(-1.0); + } + let Some(start) = start_index(machine, argument(args, 1), length)? else { + return Ok(-1.0); + }; + for index in start..length { + tick(machine)?; + if has_index(machine, object, index)? { + let element = get_index(machine, object, index)?; + if machine.strict_equal(element, needle) { + return Ok(index as f64); + } + } + } + Ok(-1.0) + }) + .map(|found| BuiltinOutcome::Value(crate::number_value(found))) +} + +pub(super) fn last_index_of( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + let needle = argument(args, 0); + with_roots(machine, &[object, needle], |machine, _| { + let length = length_of_array_like(machine, object)?; + if length == 0 { + return Ok(-1.0); + } + let relative = if args.len() > 1 { + integer_or_infinity(machine, args[1])? + } else { + (length - 1) as f64 + }; + let position = if relative >= 0.0 { + relative.min((length - 1) as f64) + } else { + length as f64 + relative + }; + if position < 0.0 { + return Ok(-1.0); + } + let mut index = position as usize; + loop { + tick(machine)?; + if has_index(machine, object, index)? { + let element = get_index(machine, object, index)?; + if machine.strict_equal(element, needle) { + return Ok(index as f64); + } + } + if index == 0 { + return Ok(-1.0); + } + index -= 1; + } + }) + .map(|found| BuiltinOutcome::Value(crate::number_value(found))) +} + +pub(super) fn includes( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + let needle = argument(args, 0); + with_roots(machine, &[object, needle], |machine, _| { + let length = length_of_array_like(machine, object)?; + if length == 0 { + return Ok(false); + } + let Some(start) = start_index(machine, argument(args, 1), length)? else { + return Ok(false); + }; + for index in start..length { + tick(machine)?; + let element = get_index(machine, object, index)?; + if machine.same_value_zero(element, needle) { + return Ok(true); + } + } + Ok(false) + }) + .map(|found| BuiltinOutcome::Value(Value::boolean(found))) +} + +/// Which element-wise method is running. The loop is one algorithm with a +/// different reaction to each callback result. +#[derive(Clone, Copy, Eq, PartialEq)] +enum Iteration { + ForEach, + Map, + Filter, + Some, + Every, +} + +fn iterate( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + iteration: Iteration, +) -> Result { + let object = to_object(machine, this)?; + let (callback, this_arg) = (argument(args, 0), argument(args, 1)); + with_roots(machine, &[object, callback, this_arg], |machine, depth| { + let length = length_of_array_like(machine, object)?; + require_callable(machine, callback)?; + let target = match iteration { + Iteration::Map => array_species_create(machine, object, length)?, + Iteration::Filter => array_species_create(machine, object, 0)?, + _ => Value::UNDEFINED, + }; + keep_root(machine, depth, target); + let mut selected = 0; + for index in 0..length { + // A present element is real storage or runs user code, and its callback + // is charged for itself. Only a gap can spin on a claimed length. + if !has_index(machine, object, index)? { + tick(machine)?; + continue; + } + let value = get_index(machine, object, index)?; + machine.refresh_native_roots(depth, &[object, callback, this_arg, target, value]); + let position = crate::number_value(index as f64); + let result = machine.call_callback(callback, this_arg, &[value, position, object])?; + match iteration { + Iteration::ForEach => {} + Iteration::Map => create_index(machine, target, index, result)?, + Iteration::Filter => { + if machine.to_boolean(result) { + create_index(machine, target, selected, value)?; + selected += 1; + } + } + Iteration::Some => { + if machine.to_boolean(result) { + return Ok(Value::TRUE); + } + } + Iteration::Every => { + if !machine.to_boolean(result) { + return Ok(Value::FALSE); + } + } + } + } + Ok(match iteration { + Iteration::ForEach => Value::UNDEFINED, + Iteration::Map | Iteration::Filter => target, + Iteration::Some => Value::FALSE, + Iteration::Every => Value::TRUE, + }) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn for_each( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + iterate(machine, this, args, Iteration::ForEach) +} + +pub(super) fn map( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + iterate(machine, this, args, Iteration::Map) +} + +pub(super) fn filter( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + iterate(machine, this, args, Iteration::Filter) +} + +pub(super) fn some( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + iterate(machine, this, args, Iteration::Some) +} + +pub(super) fn every( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + iterate(machine, this, args, Iteration::Every) +} + +/// `find` and `findIndex` visit holes, unlike the methods above. +fn find_from( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + want_index: bool, +) -> Result { + let object = to_object(machine, this)?; + let (callback, this_arg) = (argument(args, 0), argument(args, 1)); + with_roots(machine, &[object, callback, this_arg], |machine, depth| { + let length = length_of_array_like(machine, object)?; + require_callable(machine, callback)?; + for index in 0..length { + tick(machine)?; + let value = get_index(machine, object, index)?; + machine.refresh_native_roots(depth, &[object, callback, this_arg, value]); + let position = crate::number_value(index as f64); + let result = machine.call_callback(callback, this_arg, &[value, position, object])?; + if machine.to_boolean(result) { + return Ok(if want_index { position } else { value }); + } + } + Ok(if want_index { + crate::number_value(-1.0) + } else { + Value::UNDEFINED + }) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn find( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + find_from(machine, this, args, false) +} + +pub(super) fn find_index( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + find_from(machine, this, args, true) +} + +fn reduce_from( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + reverse: bool, +) -> Result { + let object = to_object(machine, this)?; + let callback = argument(args, 0); + with_roots(machine, &[object, callback], |machine, depth| { + let length = length_of_array_like(machine, object)?; + require_callable(machine, callback)?; + let mut indices: Box> = if reverse { + Box::new((0..length).rev()) + } else { + Box::new(0..length) + }; + let mut accumulator = args.get(1).copied(); + while accumulator.is_none() { + let Some(index) = indices.next() else { + return Err(type_error("Reduce of an empty array with no initial value")); + }; + tick(machine)?; + if has_index(machine, object, index)? { + accumulator = Some(get_index(machine, object, index)?); + } + } + let mut accumulator = accumulator.expect("the loop above found an initial value"); + for index in indices { + tick(machine)?; + machine.refresh_native_roots(depth, &[object, callback, accumulator]); + if has_index(machine, object, index)? { + let value = get_index(machine, object, index)?; + let position = crate::number_value(index as f64); + accumulator = machine.call_callback( + callback, + Value::UNDEFINED, + &[accumulator, value, position, object], + )?; + } + } + Ok(accumulator) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn reduce( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + reduce_from(machine, this, args, false) +} + +pub(super) fn reduce_right( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + reduce_from(machine, this, args, true) +} + +/// `SortCompare`. `undefined` sorts last without reaching the comparator. +fn sort_compare( + machine: &mut Machine<'_, H>, + comparator: Value, + left: Value, + right: Value, +) -> Result { + match (left == Value::UNDEFINED, right == Value::UNDEFINED) { + (true, true) => return Ok(Ordering::Equal), + (true, false) => return Ok(Ordering::Greater), + (false, true) => return Ok(Ordering::Less), + (false, false) => {} + } + if comparator != Value::UNDEFINED { + let result = machine.call_value(comparator, Value::UNDEFINED, &[left, right])?; + let order = value_number(machine.coerce_number_observable(result)?); + return Ok(order.partial_cmp(&0.0).unwrap_or(Ordering::Equal)); + } + let left = machine.coerce_string_observable(left)?; + let right = machine.coerce_string_observable(right)?; + Ok(left.as_units().cmp(right.as_units())) +} + +/// A stable bottom-up merge sort. A user comparator can be inconsistent or throw, +/// which the standard library's sort does not tolerate. +fn merge_sort( + machine: &mut Machine<'_, H>, + mut items: Vec, + comparator: Value, +) -> Result, EvalFailure> { + let length = items.len(); + let mut width = 1; + while width < length { + let mut merged = Vec::with_capacity(length); + for start in (0..length).step_by(width * 2) { + let middle = (start + width).min(length); + let end = (start + width * 2).min(length); + let (mut left, mut right) = (start, middle); + while left < middle && right < end { + let order = sort_compare(machine, comparator, items[left], items[right])?; + if order == Ordering::Greater { + merged.push(items[right]); + right += 1; + } else { + merged.push(items[left]); + left += 1; + } + } + merged.extend_from_slice(&items[left..middle]); + merged.extend_from_slice(&items[right..end]); + } + items = merged; + width *= 2; + } + Ok(items) +} + +pub(super) fn sort( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let comparator = argument(args, 0); + if comparator != Value::UNDEFINED && !machine.is_callable(comparator)? { + return Err(type_error("Array sort comparator is not callable")); + } + let object = to_object(machine, this)?; + with_roots(machine, &[object, comparator], |machine, depth| { + let values = super::array::elements(machine, object)?; + let length = values.len(); + let present: Vec = values + .into_iter() + .filter(|value| *value != Value::HOLE) + .collect(); + for &value in &present { + keep_root(machine, depth, value); + } + let count = present.len(); + let sorted = merge_sort(machine, present, comparator)?; + for (index, value) in sorted.into_iter().enumerate() { + set_index(machine, object, index, value)?; + } + for index in count..length { + tick(machine)?; + delete_index(machine, object, index)?; + } + Ok(object) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn reverse( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, depth| { + let length = length_of_array_like(machine, object)?; + for lower in 0..length / 2 { + tick(machine)?; + let upper = length - lower - 1; + let lower_exists = has_index(machine, object, lower)?; + let lower_value = if lower_exists { + get_index(machine, object, lower)? + } else { + Value::UNDEFINED + }; + keep_root(machine, depth, lower_value); + let upper_exists = has_index(machine, object, upper)?; + let upper_value = if upper_exists { + get_index(machine, object, upper)? + } else { + Value::UNDEFINED + }; + keep_root(machine, depth, upper_value); + match (lower_exists, upper_exists) { + (true, true) => { + set_index(machine, object, lower, upper_value)?; + set_index(machine, object, upper, lower_value)?; + } + (false, true) => { + set_index(machine, object, lower, upper_value)?; + delete_index(machine, object, upper)?; + } + (true, false) => { + delete_index(machine, object, lower)?; + set_index(machine, object, upper, lower_value)?; + } + (false, false) => {} + } + machine.native_roots[depth].truncate(1); + } + Ok(object) + }) + .map(BuiltinOutcome::Value) +} + +/// One array that `FlattenIntoArray` is part way through. +struct FlattenLevel { + source: Value, + length: usize, + index: usize, + remaining: Option, +} + +/// `FlattenIntoArray` with an explicit stack, because the script controls how +/// deeply arrays nest. `remaining` is `None` for an infinite depth, and `mapper` +/// applies to the top level only. +fn flatten_into( + machine: &mut Machine<'_, H>, + target: Value, + source: Value, + length: usize, + remaining: Option, + mapper: Option<(Value, Value)>, +) -> Result<(), EvalFailure> { + with_roots(machine, &[source], |machine, depth| { + let mut levels = vec![FlattenLevel { + source, + length, + index: 0, + remaining, + }]; + let mut to = 0; + while let Some(level) = levels.last_mut() { + tick(machine)?; + if level.index >= level.length { + levels.pop(); + machine.native_roots[depth].pop(); + continue; + } + let (source, index, remaining) = (level.source, level.index, level.remaining); + level.index += 1; + if !has_index(machine, source, index)? { + continue; + } + let mut element = get_index(machine, source, index)?; + if let Some((mapper, this_arg)) = mapper.filter(|_| levels.len() == 1) { + let position = crate::number_value(index as f64); + element = machine.call_value(mapper, this_arg, &[element, position, source])?; + } + if remaining != Some(0) && is_array(machine, element)? { + if levels.len() > MAX_FLATTEN_DEPTH { + return Err(range_error("Array is nested too deeply to flatten")); + } + keep_root(machine, depth, element); + let length = length_of_array_like(machine, element)?; + levels.push(FlattenLevel { + source: element, + length, + index: 0, + remaining: remaining.map(|remaining| remaining - 1), + }); + } else { + if to >= MAX_SAFE_INTEGER { + return Err(type_error("Array length exceeds the safe integer limit")); + } + create_index(machine, target, to, element)?; + to += 1; + } + } + Ok(()) + }) +} + +pub(super) fn flat( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + with_roots(machine, &[object], |machine, depth| { + let length = length_of_array_like(machine, object)?; + let requested = match argument(args, 0) { + Value::UNDEFINED => 1.0, + requested => integer_or_infinity(machine, requested)?.max(0.0), + }; + let remaining = requested.is_finite().then_some(requested as usize); + let target = array_species_create(machine, object, 0)?; + keep_root(machine, depth, target); + flatten_into(machine, target, object, length, remaining, None)?; + Ok(target) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn flat_map( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + let (mapper, this_arg) = (argument(args, 0), argument(args, 1)); + with_roots(machine, &[object, mapper, this_arg], |machine, depth| { + let length = length_of_array_like(machine, object)?; + require_callable(machine, mapper)?; + let target = array_species_create(machine, object, 0)?; + keep_root(machine, depth, target); + flatten_into( + machine, + target, + object, + length, + Some(1), + Some((mapper, this_arg)), + )?; + Ok(target) + }) + .map(BuiltinOutcome::Value) +} + +pub(super) fn fill( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let object = to_object(machine, this)?; + let value = argument(args, 0); + with_roots(machine, &[object, value], |machine, _| { + let length = length_of_array_like(machine, object)?; + let start = relative_position(machine, argument(args, 1), length)?; + let end = match argument(args, 2) { + Value::UNDEFINED => length, + end => relative_position(machine, end, length)?, + }; + for index in start..end { + tick(machine)?; + set_index(machine, object, index, value)?; + } + Ok(object) + }) + .map(BuiltinOutcome::Value) +} + +/// `Array(...)`. A single Number argument is a length, which must be a valid one; +/// any other single argument is an element. +pub(super) fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let new_target = machine.current_new_target(); + let default = machine.intrinsics.array_prototype; + let prototype = if new_target == Value::UNDEFINED { + default + } else { + let candidate = machine.get_named_property(new_target, "prototype")?; + if machine.is_object(candidate) { + candidate + } else { + default + } + }; + let array = match args { + [length] + if matches!( + length.decode(), + Some(Decoded::Int32(_) | Decoded::Number(_)) + ) => + { + let length = value_number(*length); + let integer = crate::to_uint32(length); + if f64::from(integer) != length { + return Err(range_error("Invalid array length")); + } + array_create(machine, integer as usize)? + } + _ => super::allocate_array(machine, args.to_vec())?, + }; + if prototype != default { + machine.set_prototype(array, prototype)?; + } + Ok(BuiltinOutcome::Value(array)) +} + +pub(super) fn is_array_method( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + Ok(BuiltinOutcome::Value(Value::boolean(is_array( + machine, + argument(args, 0), + )?))) +} + +/// The receiver a static `Array` method builds into: `C` when it is a +/// constructor, and an intrinsic array otherwise. +fn construct_from_receiver( + machine: &mut Machine<'_, H>, + receiver: Value, + length: Option, +) -> Result { + if machine.is_constructor(receiver)? { + return match length { + Some(length) => { + machine.construct_value(receiver, &[crate::number_value(length as f64)]) + } + None => machine.construct_value(receiver, &[]), + }; + } + array_create(machine, length.unwrap_or(0)) +} + +pub(super) fn of( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + with_roots(machine, args, |machine, depth| { + let target = construct_from_receiver(machine, this, Some(args.len()))?; + keep_root(machine, depth, target); + for (index, &value) in args.iter().enumerate() { + create_index(machine, target, index, value)?; + } + set_length(machine, target, args.len())?; + Ok(target) + }) + .map(BuiltinOutcome::Value) +} + +fn iterator_method( + machine: &mut Machine<'_, H>, + items: Value, +) -> Result, EvalFailure> { + let key = machine.to_property_key(machine.intrinsics.builtins.symbol_iterator())?; + let method = machine.get_property_key(items, &key)?; + if matches!(method.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Ok(None); + } + if !machine.is_callable(method)? { + return Err(type_error("value is not iterable")); + } + Ok(Some(method)) +} + +fn close_and_fail( + machine: &mut Machine<'_, H>, + iterator: Value, + failure: EvalFailure, +) -> EvalFailure { + let (closed, _) = machine.close_iterator_raw(iterator); + match closed { + Err(EvalFailure::Runtime(kind)) => EvalFailure::Runtime(kind), + _ => failure, + } +} + +pub(super) fn from( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (items, mapper, this_arg) = (argument(args, 0), argument(args, 1), argument(args, 2)); + if mapper != Value::UNDEFINED && !machine.is_callable(mapper)? { + return Err(type_error("Array.from mapper is not callable")); + } + with_roots(machine, &[items, mapper, this_arg], |machine, depth| { + if let Some(method) = iterator_method(machine, items)? { + let target = construct_from_receiver(machine, this, None)?; + keep_root(machine, depth, target); + let opened = machine.call_value(method, items, &[])?; + if !machine.is_object(opened) { + return Err(type_error("iterator method returned a non-object")); + } + let next = machine.get_named_property(opened, "next")?; + let iterator = machine.create_protocol_iterator(opened, next)?; + keep_root(machine, depth, iterator); + let mut index = 0; + loop { + let (done, value) = machine.iterator_next(iterator)?; + if done { + set_length(machine, target, index)?; + return Ok(target); + } + let mapped = if mapper == Value::UNDEFINED { + value + } else { + let position = crate::number_value(index as f64); + match machine.call_value(mapper, this_arg, &[value, position]) { + Ok(mapped) => mapped, + Err(failure) => return Err(close_and_fail(machine, iterator, failure)), + } + }; + if let Err(failure) = create_index(machine, target, index, mapped) { + return Err(close_and_fail(machine, iterator, failure)); + } + index += 1; + } + } + let source = to_object(machine, items)?; + keep_root(machine, depth, source); + let length = length_of_array_like(machine, source)?; + let target = construct_from_receiver(machine, this, Some(length))?; + keep_root(machine, depth, target); + for index in 0..length { + tick(machine)?; + let value = get_index(machine, source, index)?; + let mapped = if mapper == Value::UNDEFINED { + value + } else { + let position = crate::number_value(index as f64); + machine.call_value(mapper, this_arg, &[value, position])? + }; + create_index(machine, target, index, mapped)?; + } + set_length(machine, target, length)?; + Ok(target) + }) + .map(BuiltinOutcome::Value) +} + +#[cfg(test)] +mod tests { + use super::super::allocate_array; + use super::super::test_support::{TestHost, blank_program, ordinary_object}; + use super::*; + use crate::intrinsics::{BuiltinDef, BuiltinHandler, native_function}; + use crate::{Limits, ThrowOrigin}; + + fn numbers(machine: &mut Machine<'_, TestHost>, values: &[i32]) -> Value { + let elements = values.iter().map(|value| Value::int32(*value as u32)); + allocate_array(machine, elements.collect()).unwrap() + } + + fn invoke( + machine: &mut Machine<'_, TestHost>, + receiver: Value, + name: &str, + args: &[Value], + ) -> Result { + let prototype = machine.intrinsics.array_prototype; + let method = machine.get_named_property(prototype, name)?; + machine.call_value(method, receiver, args) + } + + fn native( + machine: &mut Machine<'_, TestHost>, + name: &'static str, + handler: BuiltinHandler, + ) -> Value { + let id = machine.intrinsics.builtins.register(BuiltinDef { + name, + length: 1, + handler, + }); + native_function(&mut machine.heap, id, name, 1) + } + + fn return_this( + _machine: &mut Machine<'_, TestHost>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + Ok(BuiltinOutcome::Value(this)) + } + + /// Answers "less" whatever it is given, which is no consistent order at all. + fn always_less( + _machine: &mut Machine<'_, TestHost>, + _this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + Ok(BuiltinOutcome::Value(Value::int32(-1i32 as u32))) + } + + fn wrap_in_array( + machine: &mut Machine<'_, TestHost>, + _this: Value, + args: &[Value], + _constructing: bool, + ) -> Result { + let inner = allocate_array(machine, vec![args[0]])?; + let outer = allocate_array(machine, vec![inner])?; + Ok(BuiltinOutcome::Value(outer)) + } + + #[test] + fn callbacks_receive_this_arg() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = numbers(&mut machine, &[1]); + let callback = native(&mut machine, "return this", return_this); + let this_arg = ordinary_object(&mut machine); + let mapped = invoke(&mut machine, array, "map", &[callback, this_arg]).unwrap(); + assert_eq!(machine.array_elements(mapped).unwrap().unwrap(), [this_arg]); + } + + #[test] + fn concat_honours_is_concat_spreadable_in_both_directions() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let key = machine + .to_property_key(machine.intrinsics.builtins.symbol_is_concat_spreadable()) + .unwrap(); + let spreadable_object = ordinary_object(&mut machine); + machine + .set_data_property(spreadable_object, "length", Value::int32(1)) + .unwrap(); + machine + .set_data_property(spreadable_object, "0", Value::int32(5)) + .unwrap(); + machine + .set_data_property_key(spreadable_object, key.clone(), Value::TRUE) + .unwrap(); + let unspread_array = numbers(&mut machine, &[6]); + machine + .set_data_property_key(unspread_array, key, Value::FALSE) + .unwrap(); + let receiver = numbers(&mut machine, &[4]); + let result = invoke( + &mut machine, + receiver, + "concat", + &[spreadable_object, unspread_array], + ) + .unwrap(); + assert_eq!( + machine.array_elements(result).unwrap().unwrap(), + [Value::int32(4), Value::int32(5), unspread_array] + ); + } + + #[test] + fn a_cyclic_array_joins_without_overflowing_the_stack() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = numbers(&mut machine, &[1]); + invoke(&mut machine, array, "push", &[array]).unwrap(); + let joined = invoke(&mut machine, array, "join", &[]).unwrap(); + assert!(machine.string_value(joined).unwrap().eq_ascii("1,")); + assert!(machine.array_join_stack.is_empty()); + } + + #[test] + fn flattening_a_cyclic_array_is_a_range_error_rather_than_a_crash() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = numbers(&mut machine, &[]); + invoke(&mut machine, array, "push", &[array]).unwrap(); + let depth = Value::number(f64::INFINITY); + assert!(matches!( + invoke(&mut machine, array, "flat", &[depth]), + Err(EvalFailure::Throw(ThrowOrigin::RangeError { .. })) + )); + } + + #[test] + fn flat_map_applies_its_mapper_once_and_flattens_one_level() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = numbers(&mut machine, &[7]); + let mapper = native(&mut machine, "wrap in array", wrap_in_array); + let result = invoke(&mut machine, array, "flatMap", &[mapper]).unwrap(); + let items = machine.array_elements(result).unwrap().unwrap(); + assert_eq!(items.len(), 1, "one level is flattened"); + assert_eq!( + machine.array_elements(items[0]).unwrap().unwrap(), + [Value::int32(7)], + "the nested array keeps its element, unmapped" + ); + } + + #[test] + fn an_inconsistent_comparator_cannot_panic_the_sort() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = numbers(&mut machine, &(0..64).rev().collect::>()); + let comparator = native(&mut machine, "always less", always_less); + invoke(&mut machine, array, "sort", &[comparator]).unwrap(); + assert_eq!(machine.array_elements(array).unwrap().unwrap().len(), 64); + } + + #[test] + fn a_sort_comparator_must_be_callable_before_anything_else_happens() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + assert!(matches!( + invoke(&mut machine, Value::UNDEFINED, "sort", &[Value::int32(1)]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "Array sort comparator is not callable" + })) + )); + } +} diff --git a/crates/bamts-runtime/src/builtins/arraybuffer.rs b/crates/bamts-runtime/src/builtins/arraybuffer.rs index bd526460..43811909 100644 --- a/crates/bamts-runtime/src/builtins/arraybuffer.rs +++ b/crates/bamts-runtime/src/builtins/arraybuffer.rs @@ -1,9 +1,9 @@ -use std::collections::BTreeMap; -#[cfg(test)] -use std::sync::Arc; -use std::sync::Mutex; +use std::collections::{BTreeMap, VecDeque}; +use std::sync::{Arc, Condvar, Mutex, MutexGuard}; +use std::time::{Duration, Instant}; use bamts_bytecode::EcmaString; +use bamts_cancel::CancellationToken; use bamts_native::{Decoded, Value}; use super::{ @@ -15,9 +15,110 @@ use crate::{EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, Proper const MAX_SAFE_INTEGER: f64 = 9_007_199_254_740_991.0; +/// How often a suspended agent looks for cancellation, which has no wake signal. +const CANCELLATION_POLL: Duration = Duration::from_millis(20); + +/// One agent suspended in `Atomics.wait`. +#[derive(Debug, Default)] +pub(crate) struct Suspension { + notified: Mutex, + ready: Condvar, +} + +impl Suspension { + pub(crate) fn wake(&self) { + *self + .notified + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) = true; + self.ready.notify_one(); + } + + /// Suspends until woken, past `deadline`, or until `cancel` fires, and + /// reports whether this agent was woken. + pub(crate) fn wait(&self, deadline: Option, cancel: &CancellationToken) -> bool { + let mut notified = self + .notified + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()); + while !*notified && !cancel.is_cancelled() { + let remaining = deadline.map_or(CANCELLATION_POLL, |deadline| { + deadline.saturating_duration_since(Instant::now()) + }); + if remaining.is_zero() { + break; + } + (notified, _) = self + .ready + .wait_timeout(notified, remaining.min(CANCELLATION_POLL)) + .unwrap_or_else(|poisoned| poisoned.into_inner()); + } + *notified + } +} + +/// A Waiter Record. +#[derive(Clone, Debug)] +pub(crate) enum Waiter { + Blocking(Arc), + /// `Atomics.waitAsync`, as the id of the machine's record of it. + Async(u64), +} + +impl Waiter { + fn is(&self, other: &Self) -> bool { + match (self, other) { + (Self::Blocking(left), Self::Blocking(right)) => Arc::ptr_eq(left, right), + (Self::Async(left), Self::Async(right)) => left == right, + _ => false, + } + } +} + +/// The WaiterList of one shared block: a queue per byte position, oldest first. +#[derive(Debug, Default)] +pub(crate) struct WaiterList { + queues: BTreeMap>, +} + +impl WaiterList { + pub(crate) fn add(&mut self, position: usize, waiter: Waiter) { + self.queues.entry(position).or_default().push_back(waiter); + } + + /// Reports whether `waiter` was still queued. It is not once a notifier has + /// taken it, which is how a timeout that lost that race learns it did. + pub(crate) fn remove(&mut self, position: usize, waiter: &Waiter) -> bool { + let Some(queue) = self.queues.get_mut(&position) else { + return false; + }; + let Some(index) = queue.iter().position(|queued| queued.is(waiter)) else { + return false; + }; + queue.remove(index); + if queue.is_empty() { + self.queues.remove(&position); + } + true + } + + /// RemoveWaiters: up to `count` waiters at `position`, oldest first. + pub(crate) fn take(&mut self, position: usize, count: usize) -> Vec { + let Some(queue) = self.queues.get_mut(&position) else { + return Vec::new(); + }; + let taken = queue.drain(..count.min(queue.len())).collect(); + if queue.is_empty() { + self.queues.remove(&position); + } + taken + } +} + #[derive(Debug)] pub(crate) struct SharedBlock { state: Mutex, + waiters: Mutex, } #[derive(Debug)] @@ -27,16 +128,23 @@ struct SharedBlockState { } impl SharedBlock { - #[cfg(test)] pub(crate) fn new(bytes: Vec, max_byte_length: Option) -> Self { Self { state: Mutex::new(SharedBlockState { bytes, max_byte_length, }), + waiters: Mutex::default(), } } + /// Enters this block's waiter critical section. + pub(crate) fn waiters(&self) -> MutexGuard<'_, WaiterList> { + self.waiters + .lock() + .unwrap_or_else(|poisoned| poisoned.into_inner()) + } + pub(crate) fn byte_length(&self) -> usize { self.state .lock() @@ -45,7 +153,6 @@ impl SharedBlock { .len() } - #[cfg(test)] pub(crate) fn max_byte_length(&self) -> usize { let state = self .state @@ -84,7 +191,6 @@ impl SharedBlock { /// Mutates the shared backing store after the caller has charged the owning /// heap slot. Length validation belongs to [`grow_shared_array_buffer`]. - #[cfg(test)] fn resize_backing(&self, new_length: usize) -> Result<(), ()> { let mut state = self .state @@ -112,7 +218,6 @@ impl SharedBlock { /// Grows a SharedArrayBuffer through its owning runtime slot so the byte delta /// is charged exactly once, even when other heap entries alias the same /// `Arc`. -#[cfg(test)] pub(crate) fn grow_shared_array_buffer( machine: &mut Machine<'_, H>, buffer: Value, @@ -748,7 +853,7 @@ fn clamp_index(relative: f64, length: usize) -> usize { relative.min(length as f64) as usize } } -fn to_clamped_index( +pub(super) fn to_clamped_index( machine: &mut Machine<'_, H>, value: Value, length: usize, @@ -766,11 +871,11 @@ fn to_clamped_index( Ok(clamp_index(integer, length)) } -fn species_constructor( +pub(super) fn species_constructor( machine: &mut Machine<'_, H>, object: Value, + default: Value, ) -> Result { - let default = machine.intrinsics.builtins.arraybuffer_constructor(); let constructor = machine.get_named_property(object, "constructor")?; if constructor == Value::UNDEFINED { return Ok(default); @@ -810,7 +915,8 @@ fn slice( Some(value) => to_clamped_index(machine, value, source_length)?, }; let new_length = end.saturating_sub(start); - let constructor = species_constructor(machine, this)?; + let default = machine.intrinsics.builtins.arraybuffer_constructor(); + let constructor = species_constructor(machine, this, default)?; let target_value = machine.construct_value(constructor, &[crate::number_value(new_length as f64)])?; let target = handle(machine, target_value)?; diff --git a/crates/bamts-runtime/src/builtins/atomics.rs b/crates/bamts-runtime/src/builtins/atomics.rs index d1c88a66..b1939fe9 100644 --- a/crates/bamts-runtime/src/builtins/atomics.rs +++ b/crates/bamts-runtime/src/builtins/atomics.rs @@ -1,20 +1,25 @@ -//! SharedArrayBuffer and the synchronous Atomics operations. +//! SharedArrayBuffer over the shared data block in `arraybuffer.rs`, and the +//! Atomics operations over TypedArray views of it. use std::collections::BTreeMap; use std::sync::Arc; -use std::time::Duration; +use std::time::{Duration, Instant}; use bamts_bytecode::EcmaString; use bamts_native::{Decoded, Value}; -use super::arraybuffer::{SharedBlock, Waiter, WaiterListHandle, WaiterState}; +use super::arraybuffer::{ + SharedBlock, Suspension, Waiter, grow_shared_array_buffer, species_constructor, + to_clamped_index, to_index, +}; +use super::bigint; use super::typedarray_all::{ - ElementKind, TypedArraySnapshot, ViewBuffer, storage_from_value, typed_array_snapshot, - value_from_storage, + ElementKind, TypedArraySnapshot, ViewBuffer, integer_or_infinity, storage_from_value, + typed_array_snapshot, value_from_storage, }; use super::{ - allocate_string, builtin_property, define_data, heap_index, install_function, range_error, - type_error, + allocate_string, define_data, define_to_string_tag, heap_index, install_constructor_function, + install_function, range_error, type_error, value_number, }; use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; use crate::{EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, PropertyMap}; @@ -34,37 +39,34 @@ fn install_shared_array_buffer( builtins: &mut BuiltinTable, ) { let prototype = super::super::ordinary_prototype(heap, builtins.object_prototype()); - let constructor = install_function( - heap, - builtins, - "SharedArrayBuffer", - 1, - shared_array_buffer_constructor::, - ); + let constructor = + install_constructor_function(heap, builtins, "SharedArrayBuffer", 1, constructor::); builtins.set_constructor_prototype(heap, constructor, prototype); + builtins.set_sharedarraybuffer_constructor(constructor); builtins.set_sharedarraybuffer_prototype(prototype); define_data(heap, prototype, "constructor", constructor); - let grow = install_function(heap, builtins, "grow", 1, grow::); - define_data(heap, prototype, "grow", grow); + for (name, length, handler) in [ + ("grow", 1, grow:: as BuiltinHandler), + ("slice", 2, slice::), + ] { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, prototype, name, function); + } for (property_name, function_name, handler) in [ ( "byteLength", "get byteLength", - sab_byte_length:: as BuiltinHandler, - ), - ( - "maxByteLength", - "get maxByteLength", - sab_max_byte_length::, + byte_length:: as BuiltinHandler, ), - ("growable", "get growable", sab_growable::), + ("maxByteLength", "get maxByteLength", max_byte_length::), + ("growable", "get growable", growable::), ] { let getter = install_function(heap, builtins, function_name, 0, handler); let HeapEntry::Object { properties, .. } = &mut heap[heap_index(prototype)] else { unreachable!("SharedArrayBuffer.prototype is ordinary") }; properties.insert( - PropertyKey::Named(EcmaString::from_utf8(property_name)), + PropertyKey::Named(EcmaString::encode(property_name)), Property::Accessor { getter: Some(getter), setter: None, @@ -86,18 +88,13 @@ fn install_shared_array_buffer( configurable: true, }, ); - let tag = super::super::push( + define_to_string_tag( heap, - HeapEntry::String(EcmaString::from_utf8("SharedArrayBuffer")), - ); - let HeapEntry::Object { properties, .. } = &mut heap[heap_index(prototype)] else { - unreachable!("SharedArrayBuffer.prototype is ordinary") - }; - properties.insert( - PropertyKey::Symbol(heap_index(builtins.symbol_to_string_tag()) as u32), - builtin_property(tag), + prototype, + builtins.symbol_to_string_tag(), + "SharedArrayBuffer", ); - globals.insert(EcmaString::from_utf8("SharedArrayBuffer"), constructor); + globals.insert(EcmaString::encode("SharedArrayBuffer"), constructor); } fn install_atomics( @@ -107,36 +104,50 @@ fn install_atomics( ) { let atomics = super::super::ordinary_prototype(heap, builtins.object_prototype()); for (name, length, handler) in [ - ("load", 2, load:: as BuiltinHandler), - ("store", 3, store::), - ("add", 3, add::), - ("sub", 3, sub::), - ("and", 3, bit_and::), - ("or", 3, bit_or::), - ("xor", 3, bit_xor::), - ("exchange", 3, exchange::), + ("add", 3, add:: as BuiltinHandler), + ("and", 3, and::), ("compareExchange", 4, compare_exchange::), + ("exchange", 3, exchange::), ("isLockFree", 1, is_lock_free::), - ("wait", 4, wait::), - ("waitAsync", 4, wait_async::), + ("load", 2, load::), ("notify", 3, notify::), + ("or", 3, or::), ("pause", 0, pause::), + ("store", 3, store::), + ("sub", 3, sub::), + ("wait", 4, wait::), + ("waitAsync", 4, wait_async::), + ("xor", 3, xor::), ] { let function = install_function(heap, builtins, name, length, handler); define_data(heap, atomics, name, function); } - let tag = super::super::push(heap, HeapEntry::String(EcmaString::from_utf8("Atomics"))); - let HeapEntry::Object { properties, .. } = &mut heap[heap_index(atomics)] else { - unreachable!("Atomics is ordinary") - }; - properties.insert( - PropertyKey::Symbol(heap_index(builtins.symbol_to_string_tag()) as u32), - builtin_property(tag), - ); - globals.insert(EcmaString::from_utf8("Atomics"), atomics); + let timeout = install_function(heap, builtins, "", 1, wait_async_timeout::); + builtins.set_wait_async_timeout(timeout); + define_to_string_tag(heap, atomics, builtins.symbol_to_string_tag(), "Atomics"); + globals.insert(EcmaString::encode("Atomics"), atomics); } -fn shared_array_buffer_constructor( +fn argument(args: &[Value], index: usize) -> Value { + args.get(index).copied().unwrap_or(Value::UNDEFINED) +} + +/// GetArrayBufferMaxByteLengthOption: anything but an object means "fixed". +fn max_byte_length_option( + machine: &mut Machine<'_, H>, + options: Value, +) -> Result, EvalFailure> { + if !machine.is_object(options) { + return Ok(None); + } + let maximum = machine.get_named_property(options, "maxByteLength")?; + if maximum == Value::UNDEFINED { + return Ok(None); + } + to_index(machine, maximum).map(Some) +} + +fn constructor( machine: &mut Machine<'_, H>, _callee: Value, args: &[Value], @@ -145,33 +156,13 @@ fn shared_array_buffer_constructor( if !constructing { return Err(type_error("SharedArrayBuffer constructor requires 'new'")); } - let byte_length = - super::arraybuffer::to_index(machine, args.first().copied().unwrap_or(Value::UNDEFINED))?; - let max_byte_length = match args.get(1).copied() { - None | Some(Value::UNDEFINED) => None, - Some(options) if !machine.is_object(options) => None, - Some(options) => { - let maximum = machine.get_named_property(options, "maxByteLength")?; - if maximum == Value::UNDEFINED { - None - } else { - Some(super::arraybuffer::to_index(machine, maximum)?) - } - } - }; + let byte_length = to_index(machine, argument(args, 0))?; + let max_byte_length = max_byte_length_option(machine, argument(args, 1))?; if max_byte_length.is_some_and(|maximum| byte_length > maximum) { return Err(range_error( "SharedArrayBuffer byteLength exceeds maxByteLength", )); } - machine - .ensure_allocation_capacity(1, byte_length.saturating_add(1)) - .map_err(EvalFailure::Runtime)?; - let mut bytes = Vec::new(); - bytes - .try_reserve_exact(byte_length) - .map_err(|_| range_error("SharedArrayBuffer allocation failed"))?; - bytes.resize(byte_length, 0); let intrinsic = machine.intrinsics.builtins.sharedarraybuffer_prototype(); let new_target = machine.current_new_target(); let prototype = if new_target == Value::UNDEFINED { @@ -184,6 +175,20 @@ fn shared_array_buffer_constructor( intrinsic } }; + // A growable block must be able to reach its maximum, so that much is + // required of the heap even though only the current length is charged. + let allocation_length = max_byte_length.unwrap_or(byte_length); + if machine + .ensure_allocation_capacity(1, allocation_length.saturating_add(1)) + .is_err() + { + return Err(range_error("SharedArrayBuffer allocation failed")); + } + let mut bytes = Vec::new(); + bytes + .try_reserve_exact(byte_length) + .map_err(|_| range_error("SharedArrayBuffer allocation failed"))?; + bytes.resize(byte_length, 0); let value = machine .allocate(HeapEntry::SharedArrayBuffer { data: Arc::new(SharedBlock::new(bytes, max_byte_length)), @@ -191,7 +196,7 @@ fn shared_array_buffer_constructor( prototype: Some(prototype), extensible: true, }) - .map_err(EvalFailure::Runtime)?; + .map_err(|_| range_error("SharedArrayBuffer allocation failed"))?; Ok(BuiltinOutcome::Value(value)) } @@ -199,20 +204,18 @@ fn shared_block( machine: &Machine<'_, H>, value: Value, ) -> Result, EvalFailure> { - let Some(index) = machine.runtime_slot(value).map_err(EvalFailure::Runtime)? else { - return Err(type_error( - "SharedArrayBuffer method called on incompatible receiver", - )); - }; + let incompatible = || type_error("SharedArrayBuffer method called on incompatible receiver"); + let index = machine + .runtime_slot(value) + .map_err(EvalFailure::Runtime)? + .ok_or_else(incompatible)?; let HeapEntry::SharedArrayBuffer { data, .. } = &machine.heap[index] else { - return Err(type_error( - "SharedArrayBuffer method called on incompatible receiver", - )); + return Err(incompatible()); }; Ok(Arc::clone(data)) } -fn sab_byte_length( +fn byte_length( machine: &mut Machine<'_, H>, this: Value, _args: &[Value], @@ -223,7 +226,7 @@ fn sab_byte_length( ))) } -fn sab_max_byte_length( +fn max_byte_length( machine: &mut Machine<'_, H>, this: Value, _args: &[Value], @@ -234,7 +237,7 @@ fn sab_max_byte_length( ))) } -fn sab_growable( +fn growable( machine: &mut Machine<'_, H>, this: Value, _args: &[Value], @@ -251,15 +254,50 @@ fn grow( args: &[Value], _constructing: bool, ) -> Result { - let block = shared_block(machine, this)?; - let length = - super::arraybuffer::to_index(machine, args.first().copied().unwrap_or(Value::UNDEFINED))?; - block - .grow(length) - .map_err(|()| range_error("Invalid SharedArrayBuffer grow length"))?; + // The [[ArrayBufferMaxByteLength]] slot check precedes argument coercion. + if !shared_block(machine, this)?.is_growable() { + return Err(type_error("SharedArrayBuffer is not growable")); + } + let new_length = to_index(machine, argument(args, 0))?; + grow_shared_array_buffer(machine, this, new_length)?; Ok(BuiltinOutcome::Value(Value::UNDEFINED)) } +fn slice( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let source = shared_block(machine, this)?; + let length = source.byte_length(); + let first = to_clamped_index(machine, argument(args, 0), length)?; + let end = match argument(args, 1) { + Value::UNDEFINED => length, + value => to_clamped_index(machine, value, length)?, + }; + let new_length = end.saturating_sub(first); + let default = machine.intrinsics.builtins.sharedarraybuffer_constructor(); + let constructor = species_constructor(machine, this, default)?; + let target_value = + machine.construct_value(constructor, &[crate::number_value(new_length as f64)])?; + let target = shared_block(machine, target_value)?; + if Arc::ptr_eq(&source, &target) { + return Err(type_error( + "SharedArrayBuffer species returned the source buffer", + )); + } + if target.byte_length() < new_length { + return Err(type_error( + "SharedArrayBuffer species returned a buffer that is too small", + )); + } + // A block only grows, so the range clamped above is still inside it. + let copied = source.with_bytes(|bytes| bytes[first..first + new_length].to_vec()); + target.with_bytes_mut(|bytes| bytes[..new_length].copy_from_slice(&copied)); + Ok(BuiltinOutcome::Value(target_value)) +} + fn species( _machine: &mut Machine<'_, H>, this: Value, @@ -269,376 +307,323 @@ fn species( Ok(BuiltinOutcome::Value(this)) } -#[derive(Clone, Copy)] -enum AtomicOperation { - Load, - Store, - Add, - Sub, - And, - Or, - Xor, - Exchange, - CompareExchange, -} - -fn integer_snapshot( - machine: &mut Machine<'_, H>, +/// ValidateIntegerTypedArray. The returned snapshot's bounds are the ones every +/// later length check of this operation must use, so they are taken here, +/// before any argument coercion can run guest code. +fn validate_integer_typed_array( + machine: &Machine<'_, H>, view: Value, - index_value: Value, -) -> Result<(TypedArraySnapshot, usize), EvalFailure> { + waitable: bool, +) -> Result { let snapshot = typed_array_snapshot(machine, view)?; - if snapshot.bounds.detached || snapshot.bounds.out_of_bounds { + if snapshot.bounds.out_of_bounds { return Err(type_error( "Atomics operation requires an in-bounds TypedArray", )); } - if matches!( - snapshot.kind, - ElementKind::Uint8Clamped - | ElementKind::Float16 - | ElementKind::Float32 - | ElementKind::Float64 - ) { + let supported = if waitable { + matches!(snapshot.kind, ElementKind::Int32 | ElementKind::BigInt64) + } else { + !matches!( + snapshot.kind, + ElementKind::Uint8Clamped + | ElementKind::Float16 + | ElementKind::Float32 + | ElementKind::Float64 + ) + }; + if !supported { + return Err(type_error(if waitable { + "Atomics wait operations require an Int32Array or BigInt64Array" + } else { + "Atomics operation requires an integer TypedArray" + })); + } + Ok(snapshot) +} + +/// ValidateAtomicAccess: the byte position of `request` in the buffer. +fn validate_atomic_access( + machine: &mut Machine<'_, H>, + snapshot: &TypedArraySnapshot, + request: Value, +) -> Result { + let index = to_index(machine, request)?; + if index >= snapshot.bounds.element_length { + return Err(range_error("Atomics index is outside the TypedArray")); + } + Ok(snapshot.byte_offset + index * snapshot.kind.element_size()) +} + +/// RevalidateAtomicAccess, for after a coercion that may have run guest code. +fn revalidate_atomic_access( + machine: &Machine<'_, H>, + view: Value, + position: usize, +) -> Result<(), EvalFailure> { + let snapshot = typed_array_snapshot(machine, view)?; + if snapshot.bounds.out_of_bounds { return Err(type_error( - "Atomics operation requires an integer TypedArray", + "Atomics operation requires an in-bounds TypedArray", )); } - let index = super::arraybuffer::to_index(machine, index_value)?; - if index >= snapshot.bounds.element_length { + if position >= snapshot.buffer.byte_length(machine)? { return Err(range_error("Atomics index is outside the TypedArray")); } - Ok((snapshot, index)) + Ok(()) } -fn atomic( +/// The coerced operand `v` of a write, which `Atomics.store` returns as is, and +/// the element bytes it stores, which wrap to the element type. +fn coerce_operand( machine: &mut Machine<'_, H>, - args: &[Value], - operation: AtomicOperation, -) -> Result { - let view = args.first().copied().unwrap_or(Value::UNDEFINED); - let (snapshot, index) = integer_snapshot( - machine, - view, - args.get(1).copied().unwrap_or(Value::UNDEFINED), - )?; - let size = snapshot.kind.element_size(); - let replacement = match operation { - AtomicOperation::Load => None, - AtomicOperation::CompareExchange => Some(storage_from_value( - machine, - snapshot.kind, - args.get(3).copied().unwrap_or(Value::UNDEFINED), - )?), - _ => Some(storage_from_value( - machine, - snapshot.kind, - args.get(2).copied().unwrap_or(Value::UNDEFINED), - )?), - }; - let expected = if matches!(operation, AtomicOperation::CompareExchange) { - Some(storage_from_value( - machine, - snapshot.kind, - args.get(2).copied().unwrap_or(Value::UNDEFINED), - )?) + kind: ElementKind, + value: Value, +) -> Result<(Value, u64), EvalFailure> { + let operand = if kind.is_bigint() { + bigint::to_bigint(machine, value)? } else { - None + crate::number_value(integer_or_infinity(machine, value)?) }; - let start = snapshot.byte_offset + index * size; - let mut old = [0_u8; 8]; + let storage = storage_from_value(machine, kind, operand)?; + Ok((operand, u64::from_le_bytes(storage))) +} + +/// Applies `update` to one element as a single read-modify-write and returns +/// the previous element bytes. For a shared block the whole closure runs under +/// the block's lock, which every other access to those bytes also takes. +fn update_element( + machine: &mut Machine<'_, H>, + snapshot: &TypedArraySnapshot, + position: usize, + update: impl FnOnce(u64) -> u64, +) -> Result<[u8; 8], EvalFailure> { + let size = snapshot.kind.element_size(); snapshot.buffer.with_bytes_mut(machine, |bytes| { - old[..size].copy_from_slice(&bytes[start..start + size]); - if let Some(replacement) = replacement { - let old_raw = raw(&old, size); - let argument = raw(&replacement, size); - let mask = if size == 8 { - u64::MAX - } else { - (1_u64 << (size * 8)) - 1 - }; - let next = match operation { - AtomicOperation::Store | AtomicOperation::Exchange => argument, - AtomicOperation::Add => old_raw.wrapping_add(argument) & mask, - AtomicOperation::Sub => old_raw.wrapping_sub(argument) & mask, - AtomicOperation::And => old_raw & argument, - AtomicOperation::Or => old_raw | argument, - AtomicOperation::Xor => old_raw ^ argument, - AtomicOperation::CompareExchange => { - if old[..size] == expected.unwrap()[..size] { - argument - } else { - old_raw - } - } - AtomicOperation::Load => old_raw, - }; - bytes[start..start + size].copy_from_slice(&next.to_le_bytes()[..size]); + let element = &mut bytes[position..position + size]; + let mut old = [0_u8; 8]; + old[..size].copy_from_slice(element); + element.copy_from_slice(&update(u64::from_le_bytes(old)).to_le_bytes()[..size]); + old + }) +} + +#[derive(Clone, Copy)] +enum Modification { + Add, + And, + Exchange, + Or, + Sub, + Xor, +} + +impl Modification { + /// Both inputs are element bytes in the low bits, so an overflow past the + /// element's width is discarded when the result is written back. + const fn apply(self, old: u64, operand: u64) -> u64 { + match self { + Self::Add => old.wrapping_add(operand), + Self::And => old & operand, + Self::Exchange => operand, + Self::Or => old | operand, + Self::Sub => old.wrapping_sub(operand), + Self::Xor => old ^ operand, } - })?; - let result = if matches!(operation, AtomicOperation::Store) { - value_from_storage(machine, snapshot.kind, replacement.unwrap())? - } else { - value_from_storage(machine, snapshot.kind, old)? - }; - Ok(BuiltinOutcome::Value(result)) + } } -fn raw(storage: &[u8; 8], size: usize) -> u64 { - let mut bytes = [0_u8; 8]; - bytes[..size].copy_from_slice(&storage[..size]); - u64::from_le_bytes(bytes) +/// AtomicReadModifyWrite. +fn read_modify_write( + machine: &mut Machine<'_, H>, + args: &[Value], + modification: Modification, +) -> Result { + let view = argument(args, 0); + let snapshot = validate_integer_typed_array(machine, view, false)?; + let position = validate_atomic_access(machine, &snapshot, argument(args, 1))?; + let (_, operand) = coerce_operand(machine, snapshot.kind, argument(args, 2))?; + revalidate_atomic_access(machine, view, position)?; + let old = update_element(machine, &snapshot, position, |old| { + modification.apply(old, operand) + })?; + value_from_storage(machine, snapshot.kind, old).map(BuiltinOutcome::Value) } -macro_rules! atomics_handlers { - ($(($name:ident, $operation:expr)),* $(,)?) => {$ ( +macro_rules! read_modify_write_handlers { + ($(($name:ident, $modification:expr)),* $(,)?) => {$( fn $name( - machine: &mut Machine<'_, H>, _this: Value, args: &[Value], _constructing: bool, - ) -> Result { atomic(machine, args, $operation) } - )* }; -} - -atomics_handlers!( - (load, AtomicOperation::Load), - (store, AtomicOperation::Store), - (add, AtomicOperation::Add), - (sub, AtomicOperation::Sub), - (bit_and, AtomicOperation::And), - (bit_or, AtomicOperation::Or), - (bit_xor, AtomicOperation::Xor), - (exchange, AtomicOperation::Exchange), - (compare_exchange, AtomicOperation::CompareExchange), + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, + ) -> Result { + read_modify_write(machine, args, $modification) + } + )*}; +} + +read_modify_write_handlers!( + (add, Modification::Add), + (and, Modification::And), + (exchange, Modification::Exchange), + (or, Modification::Or), + (sub, Modification::Sub), + (xor, Modification::Xor), ); -fn waitable_snapshot( +fn compare_exchange( machine: &mut Machine<'_, H>, + _this: Value, args: &[Value], -) -> Result<(TypedArraySnapshot, usize), EvalFailure> { - let result = integer_snapshot( - machine, - args.first().copied().unwrap_or(Value::UNDEFINED), - args.get(1).copied().unwrap_or(Value::UNDEFINED), - )?; - if !matches!(result.0.kind, ElementKind::Int32 | ElementKind::BigInt64) { - return Err(type_error( - "Atomics.wait requires an Int32Array or BigInt64Array", - )); - } - Ok(result) + _constructing: bool, +) -> Result { + let view = argument(args, 0); + let snapshot = validate_integer_typed_array(machine, view, false)?; + let position = validate_atomic_access(machine, &snapshot, argument(args, 1))?; + let (_, expected) = coerce_operand(machine, snapshot.kind, argument(args, 2))?; + let (_, replacement) = coerce_operand(machine, snapshot.kind, argument(args, 3))?; + revalidate_atomic_access(machine, view, position)?; + let old = update_element(machine, &snapshot, position, |old| { + if old == expected { replacement } else { old } + })?; + value_from_storage(machine, snapshot.kind, old).map(BuiltinOutcome::Value) } -fn waiter_list( - machine: &Machine<'_, H>, - buffer: &ViewBuffer, -) -> Result { - match buffer { - ViewBuffer::Array(buffer) => buffer.waiter_list(machine), - ViewBuffer::Shared(buffer) => Ok(buffer.waiter_list()), - } +fn load( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let view = argument(args, 0); + let snapshot = validate_integer_typed_array(machine, view, false)?; + let position = validate_atomic_access(machine, &snapshot, argument(args, 1))?; + revalidate_atomic_access(machine, view, position)?; + let size = snapshot.kind.element_size(); + let mut storage = [0_u8; 8]; + snapshot.buffer.with_bytes(machine, |bytes| { + storage[..size].copy_from_slice(&bytes[position..position + size]); + })?; + value_from_storage(machine, snapshot.kind, storage).map(BuiltinOutcome::Value) } -fn wait_result( +fn store( machine: &mut Machine<'_, H>, - text: &str, + _this: Value, + args: &[Value], + _constructing: bool, ) -> Result { - Ok(BuiltinOutcome::Value(allocate_string( - machine, - EcmaString::from_utf8(text), - )?)) + let view = argument(args, 0); + let snapshot = validate_integer_typed_array(machine, view, false)?; + let position = validate_atomic_access(machine, &snapshot, argument(args, 1))?; + let (operand, storage) = coerce_operand(machine, snapshot.kind, argument(args, 2))?; + revalidate_atomic_access(machine, view, position)?; + update_element(machine, &snapshot, position, |_| storage)?; + Ok(BuiltinOutcome::Value(operand)) } -fn wait( +/// Every element size is a plain machine word on the supported targets. The +/// runtime serializes access with the block's lock, which only one agent ever +/// takes, so `true` overstates nothing a program can observe. +fn is_lock_free( machine: &mut Machine<'_, H>, _this: Value, args: &[Value], _constructing: bool, ) -> Result { - if !machine.limits.agent_can_suspend { - return Err(type_error("Atomics.wait cannot suspend this agent")); - } - let (snapshot, index) = waitable_snapshot(machine, args)?; - let expected = storage_from_value( - machine, - snapshot.kind, - args.get(2).copied().unwrap_or(Value::UNDEFINED), - )?; - let timeout = match args.get(3).copied() { - None | Some(Value::UNDEFINED) => None, - Some(value) => { - let number = match machine.coerce_number_observable(value)?.decode() { - Some(Decoded::Int32(value)) => f64::from(value as i32), - Some(Decoded::Number(value)) => value, - _ => unreachable!("ToNumber returns a number"), - }; - if number.is_nan() || number == f64::INFINITY { - None - } else { - Some(Duration::from_secs_f64(number.max(0.0) / 1000.0)) - } - } - }; - let byte_position = snapshot.byte_offset + index * snapshot.kind.element_size(); - let waiters = waiter_list(machine, &snapshot.buffer)?; - let mut lists = waiters - .lists - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - let equal = snapshot.buffer.with_bytes(machine, |bytes| { - bytes[byte_position..byte_position + snapshot.kind.element_size()] - == expected[..snapshot.kind.element_size()] - })?; - if !equal { - drop(lists); - return wait_result(machine, "not-equal"); - } - let waiter = Waiter::handle(); - lists - .entry(byte_position) - .or_default() - .push_back(Arc::clone(&waiter)); - drop(lists); - - let state = waiter - .state - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - let notified = if let Some(timeout) = timeout { - let (state, _) = waiter - .ready - .wait_timeout_while(state, timeout, |state| *state == WaiterState::Waiting) - .unwrap_or_else(|poisoned| poisoned.into_inner()); - *state == WaiterState::Notified - } else { - let state = waiter - .ready - .wait_while(state, |state| *state == WaiterState::Waiting) - .unwrap_or_else(|poisoned| poisoned.into_inner()); - *state == WaiterState::Notified + let size = integer_or_infinity(machine, argument(args, 0))?; + Ok(BuiltinOutcome::Value(Value::boolean( + [1.0, 2.0, 4.0, 8.0].contains(&size), + ))) +} + +fn pause( + _machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let integral = match argument(args, 0).decode() { + Some(Decoded::Undefined | Decoded::Int32(_)) => true, + Some(Decoded::Number(number)) => number.is_finite() && number.fract() == 0.0, + _ => false, }; - if !notified { - let mut lists = waiters - .lists - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - if let Some(queue) = lists.get_mut(&byte_position) { - queue.retain(|candidate| !Arc::ptr_eq(candidate, &waiter)); - if queue.is_empty() { - lists.remove(&byte_position); - } - } + if !integral { + return Err(type_error("Atomics.pause requires an integral Number")); } - wait_result(machine, if notified { "ok" } else { "timed-out" }) + std::hint::spin_loop(); + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) } -fn wait_async_result( - machine: &mut Machine<'_, H>, - asynchronous: bool, - value: Value, -) -> Result { - let object = machine - .allocate(HeapEntry::Object { - properties: PropertyMap::default(), - prototype: Some(machine.intrinsics.builtins.object_prototype()), - boxed_primitive: None, - extensible: true, +/// The validated, coerced arguments of DoWait, up to the critical section. +struct WaitRequest { + block: Arc, + position: usize, + size: usize, + expected: [u8; 8], + /// `None` is a wait with no timeout. + timeout_ms: Option, +} + +impl WaitRequest { + fn parse(machine: &mut Machine<'_, H>, args: &[Value]) -> Result { + let snapshot = validate_integer_typed_array(machine, argument(args, 0), true)?; + let ViewBuffer::Shared(block) = &snapshot.buffer else { + return Err(type_error("Atomics wait operations require shared memory")); + }; + let position = validate_atomic_access(machine, &snapshot, argument(args, 1))?; + let expected = storage_from_value(machine, snapshot.kind, argument(args, 2))?; + let timeout = value_number(machine.coerce_number_observable(argument(args, 3))?); + Ok(Self { + block: Arc::clone(block), + position, + size: snapshot.kind.element_size(), + expected, + timeout_ms: (!timeout.is_nan() && timeout != f64::INFINITY).then(|| timeout.max(0.0)), }) - .map_err(EvalFailure::Runtime)?; - machine.set_data_property(object, "async", Value::boolean(asynchronous))?; - machine.set_data_property(object, "value", value)?; - Ok(BuiltinOutcome::Value(object)) + } + + /// Must be asked inside the waiter critical section: a writer changes the + /// value before it notifies, so it either is seen here or finds the waiter. + fn value_is_expected(&self) -> bool { + self.block.with_bytes(|bytes| { + bytes[self.position..self.position + self.size] == self.expected[..self.size] + }) + } } -fn wait_async( +fn wait( machine: &mut Machine<'_, H>, _this: Value, args: &[Value], _constructing: bool, ) -> Result { - let (snapshot, index) = waitable_snapshot(machine, args)?; - let expected = storage_from_value( - machine, - snapshot.kind, - args.get(2).copied().unwrap_or(Value::UNDEFINED), - )?; - let timeout = match args.get(3).copied() { - None | Some(Value::UNDEFINED) => None, - Some(value) => { - let number = match machine.coerce_number_observable(value)?.decode() { - Some(Decoded::Int32(value)) => f64::from(value as i32), - Some(Decoded::Number(value)) => value, - _ => unreachable!("ToNumber returns a number"), - }; - if number.is_nan() || number == f64::INFINITY { - None - } else { - Some(Duration::from_secs_f64(number.max(0.0) / 1000.0)) - } - } - }; - let byte_position = snapshot.byte_offset + index * snapshot.kind.element_size(); - let waiters = waiter_list(machine, &snapshot.buffer)?; - let mut lists = waiters - .lists - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - let equal = snapshot.buffer.with_bytes(machine, |bytes| { - bytes[byte_position..byte_position + snapshot.kind.element_size()] - == expected[..snapshot.kind.element_size()] - })?; - if !equal { - drop(lists); - let value = allocate_string(machine, EcmaString::from_utf8("not-equal"))?; - return wait_async_result(machine, false, value); - } - let waiter = Waiter::handle(); - lists - .entry(byte_position) - .or_default() - .push_back(Arc::clone(&waiter)); - drop(lists); - let promise = machine.create_promise()?; - let promise_bits = promise.to_bits(); - let inbox = machine.external_inbox(); - inbox.reserve(); - std::thread::spawn({ - let waiters = Arc::clone(&waiters); - move || { - let state = waiter - .state - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - let notified = if let Some(timeout) = timeout { - let (state, _) = waiter - .ready - .wait_timeout_while(state, timeout, |state| *state == WaiterState::Waiting) - .unwrap_or_else(|poisoned| poisoned.into_inner()); - *state == WaiterState::Notified - } else { - let state = waiter - .ready - .wait_while(state, |state| *state == WaiterState::Waiting) - .unwrap_or_else(|poisoned| poisoned.into_inner()); - *state == WaiterState::Notified - }; - if !notified { - let mut lists = waiters - .lists - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - if let Some(queue) = lists.get_mut(&byte_position) { - queue.retain(|candidate| !Arc::ptr_eq(candidate, &waiter)); - if queue.is_empty() { - lists.remove(&byte_position); - } - } - } - inbox.push(crate::ExternalJob::ResolveWaitAsync { - promise: promise_bits, - result: if notified { "ok" } else { "timed-out" }, - }); + let request = WaitRequest::parse(machine, args)?; + if !machine.limits.agent_can_suspend { + return Err(type_error("Atomics.wait cannot suspend this agent")); + } + let suspension = Arc::new(Suspension::default()); + let waiter = Waiter::Blocking(Arc::clone(&suspension)); + { + let mut waiters = request.block.waiters(); + if !request.value_is_expected() { + drop(waiters); + return string_outcome(machine, "not-equal"); } - }); - wait_async_result(machine, true, promise) + waiters.add(request.position, waiter.clone()); + } + let deadline = request + .timeout_ms + .and_then(|milliseconds| Duration::try_from_secs_f64(milliseconds / 1000.0).ok()) + .and_then(|timeout| Instant::now().checked_add(timeout)); + let woken = suspension.wait(deadline, &machine.cancel) + || !request.block.waiters().remove(request.position, &waiter); + if woken { + return string_outcome(machine, "ok"); + } + machine.check_cancel().map_err(EvalFailure::Runtime)?; + string_outcome(machine, "timed-out") } fn notify( @@ -647,170 +632,851 @@ fn notify( args: &[Value], _constructing: bool, ) -> Result { - let (snapshot, index) = waitable_snapshot(machine, args)?; - let count = match args.get(2).copied() { - None | Some(Value::UNDEFINED) => usize::MAX, - Some(value) => super::arraybuffer::to_index(machine, value)?, + let snapshot = validate_integer_typed_array(machine, argument(args, 0), true)?; + let position = validate_atomic_access(machine, &snapshot, argument(args, 1))?; + let count = match argument(args, 2) { + Value::UNDEFINED => usize::MAX, + // A float-to-integer cast saturates, which is what infinity asks for. + value => integer_or_infinity(machine, value)?.max(0.0) as usize, }; - let byte_position = snapshot.byte_offset + index * snapshot.kind.element_size(); - let waiters = waiter_list(machine, &snapshot.buffer)?; - let mut lists = waiters - .lists - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - let mut woken = Vec::new(); - if let Some(queue) = lists.get_mut(&byte_position) { - for _ in 0..count { - let Some(waiter) = queue.pop_front() else { - break; - }; - woken.push(waiter); - } - if queue.is_empty() { - lists.remove(&byte_position); - } - } - drop(lists); + let ViewBuffer::Shared(block) = &snapshot.buffer else { + return Ok(BuiltinOutcome::Value(Value::int32(0))); + }; + let woken = block.waiters().take(position, count); for waiter in &woken { - *waiter - .state - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) = WaiterState::Notified; - waiter.ready.notify_one(); + match waiter { + Waiter::Blocking(suspension) => suspension.wake(), + Waiter::Async(id) => settle_async_waiter(machine, *id, "ok")?, + } } Ok(BuiltinOutcome::Value(crate::number_value( woken.len() as f64 ))) } -fn is_lock_free( +/// A pending `Atomics.waitAsync`. The block's waiter list holds only its id, +/// because the collector cannot see a `Value` inside a block; the machine owns +/// this record and traces `promise`. +#[derive(Debug)] +pub(crate) struct AsyncWaiter { + pub(crate) promise: Value, + block: Arc, + position: usize, + /// The armed timeout's handle, when the wait has one. + timer: Option, + /// What is left of a timeout too long for one timer. + remaining_ms: u64, +} + +/// Arms a timer for as much of `timeout_ms` as one [`crate::TimerProvider`] delay +/// can hold, and returns its handle with the part that is left over. +fn arm_async_timeout( + machine: &mut Machine<'_, H>, + id: u64, + timeout_ms: u64, +) -> Result<(Value, u64), EvalFailure> { + let delay = u32::try_from(timeout_ms).unwrap_or(u32::MAX); + let callback = machine.intrinsics.builtins.wait_async_timeout(); + let handle = machine.schedule_timeout(callback, delay, vec![crate::number_value(id as f64)])?; + Ok((handle, timeout_ms - u64::from(delay))) +} + +/// NotifyWaiter for a waiter of this agent: resolves the promise at once, so +/// its reactions run at the next microtask checkpoint. +fn settle_async_waiter( + machine: &mut Machine<'_, H>, + id: u64, + result: &str, +) -> Result<(), EvalFailure> { + let Some(waiter) = machine.async_waiters.remove(&id) else { + return Ok(()); + }; + if let Some(timer) = waiter.timer { + machine.clear_timeout(timer)?; + } + let value = allocate_string(machine, EcmaString::encode(result))?; + machine + .fulfill_promise(waiter.promise, value) + .map_err(EvalFailure::Runtime) +} + +/// The timer callback behind every timed `Atomics.waitAsync`. No script can +/// reach it: the only reference is the timer record that calls it. +fn wait_async_timeout( machine: &mut Machine<'_, H>, _this: Value, args: &[Value], _constructing: bool, ) -> Result { - let size = - super::arraybuffer::to_index(machine, args.first().copied().unwrap_or(Value::UNDEFINED))?; - Ok(BuiltinOutcome::Value(Value::boolean(matches!( - size, - 1 | 2 | 4 | 8 - )))) + let id = value_number(argument(args, 0)) as u64; + if let Some(waiter) = machine.async_waiters.get(&id) { + if waiter.remaining_ms == 0 { + waiter + .block + .waiters() + .remove(waiter.position, &Waiter::Async(id)); + settle_async_waiter(machine, id, "timed-out")?; + } else { + let remaining_ms = waiter.remaining_ms; + let (timer, remaining_ms) = arm_async_timeout(machine, id, remaining_ms)?; + if let Some(waiter) = machine.async_waiters.get_mut(&id) { + waiter.timer = Some(timer); + waiter.remaining_ms = remaining_ms; + } + } + } + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) } -fn pause( - _machine: &mut Machine<'_, H>, +fn wait_async_result( + machine: &mut Machine<'_, H>, + asynchronous: bool, + value: Value, +) -> Result { + let prototype = machine.intrinsics.object_prototype; + let object = super::ordinary_runtime(machine, Some(prototype))?; + for (name, value) in [("async", Value::boolean(asynchronous)), ("value", value)] { + let key = PropertyKey::Named(EcmaString::encode(name)); + machine.create_data_property_key(object, key, value)?; + } + Ok(object) +} + +fn wait_async_settled( + machine: &mut Machine<'_, H>, + outcome: &str, +) -> Result { + let value = allocate_string(machine, EcmaString::encode(outcome))?; + wait_async_result(machine, false, value).map(BuiltinOutcome::Value) +} + +fn wait_async( + machine: &mut Machine<'_, H>, _this: Value, - _args: &[Value], + args: &[Value], _constructing: bool, ) -> Result { - Ok(BuiltinOutcome::Value(Value::UNDEFINED)) + let request = WaitRequest::parse(machine, args)?; + let block = Arc::clone(&request.block); + let mut waiters = block.waiters(); + if !request.value_is_expected() { + drop(waiters); + return wait_async_settled(machine, "not-equal"); + } + if request.timeout_ms == Some(0.0) { + drop(waiters); + return wait_async_settled(machine, "timed-out"); + } + let promise = machine.create_promise()?; + let result = wait_async_result(machine, true, promise)?; + let id = machine.next_async_waiter; + machine.next_async_waiter += 1; + let (timer, remaining_ms) = match request.timeout_ms { + None => (None, 0), + // A cast saturates, and a fraction of a millisecond still has to pass. + Some(milliseconds) => { + let (timer, remaining_ms) = arm_async_timeout(machine, id, milliseconds.ceil() as u64)?; + (Some(timer), remaining_ms) + } + }; + machine.async_waiters.insert( + id, + AsyncWaiter { + promise, + block: Arc::clone(&request.block), + position: request.position, + timer, + remaining_ms, + }, + ); + waiters.add(request.position, Waiter::Async(id)); + Ok(BuiltinOutcome::Value(result)) +} + +fn string_outcome( + machine: &mut Machine<'_, H>, + text: &str, +) -> Result { + allocate_string(machine, EcmaString::encode(text)).map(BuiltinOutcome::Value) } #[cfg(test)] mod tests { use super::super::test_support::{TestHost, blank_program}; use super::*; - use crate::{Limits, PromiseState}; + use crate::{ + CancellationToken, Limits, PromiseState, RuntimeErrorKind, ThrowOrigin, TimerError, + TimerProvider, TimerWakeup, + }; - fn shared_int32(machine: &mut Machine<'_, TestHost>) -> Value { - let buffer_constructor = machine + /// A clock that moves only when the event loop waits, straight to the next + /// deadline, so a timeout of any length takes no real time. + #[derive(Default)] + struct ManualTimers { + armed: Vec, + now_ms: u64, + } + + impl TimerProvider for ManualTimers { + fn schedule(&mut self, id: u64, delay_ms: u32) -> Result { + let deadline_ms = self.now_ms + u64::from(delay_ms); + self.armed.push(TimerWakeup { id, deadline_ms }); + Ok(deadline_ms) + } + + fn cancel(&mut self, id: u64) -> Result { + let armed = self.armed.len(); + self.armed.retain(|timer| timer.id != id); + Ok(self.armed.len() != armed) + } + + fn poll_expired(&mut self, _output: &mut Vec) -> Result<(), TimerError> { + Ok(()) + } + + fn wait_expired( + &mut self, + _cancel: &CancellationToken, + ) -> Result, TimerError> { + let next = (0..self.armed.len()).min_by_key(|&index| self.armed[index].deadline_ms); + Ok(next.map(|index| { + let wakeup = self.armed.remove(index); + self.now_ms = wakeup.deadline_ms; + wakeup + })) + } + + fn has_pending(&self) -> bool { + !self.armed.is_empty() + } + } + + #[derive(Default)] + struct TimerHost { + timers: ManualTimers, + } + + impl Host for TimerHost { + fn timers(&mut self) -> Option<&mut (dyn TimerProvider + 'static)> { + Some(&mut self.timers) + } + } + + fn with_timers(test: impl FnOnce(&mut Machine<'_, TimerHost>)) -> TimerHost { + let program = blank_program(""); + let mut host = TimerHost::default(); + let mut machine = Machine::new(&program, &mut host, Limits::default()); + test(&mut machine); + drop(machine); + host + } + + fn atomics(machine: &Machine<'_, H>) -> Value { + machine + .intrinsics + .global("Atomics") + .expect("Atomics is installed") + } + + fn with_machine(test: impl FnOnce(&mut Machine<'_, TestHost>)) { + let program = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + test(&mut machine); + } + + fn shared_buffer( + machine: &mut Machine<'_, H>, + length: i32, + maximum: Option, + ) -> Value { + let constructor = machine .intrinsics .global("SharedArrayBuffer") .expect("SharedArrayBuffer is installed"); - let buffer = machine - .construct_value(buffer_constructor, &[Value::int32(4)]) - .expect("SharedArrayBuffer construction succeeds"); - let view_constructor = machine - .intrinsics - .global("Int32Array") - .expect("Int32Array is installed"); + let options = match maximum { + None => Value::UNDEFINED, + Some(maximum) => { + let options = crate::builtins::ordinary_runtime(machine, None).unwrap(); + machine + .set_data_property(options, "maxByteLength", Value::int32(maximum)) + .unwrap(); + options + } + }; machine - .construct_value(view_constructor, &[buffer]) - .expect("Int32Array construction succeeds") + .construct_value(constructor, &[Value::int32(length as u32), options]) + .expect("SharedArrayBuffer construction succeeds") } - fn promise_state(machine: &Machine<'_, TestHost>, promise: Value) -> PromiseState { - let index = machine.runtime_slot(promise).unwrap().unwrap(); - let HeapEntry::Promise { state, .. } = &machine.heap[index] else { - panic!("waitAsync value is a Promise"); - }; - state.clone() + fn view_of(machine: &mut Machine<'_, H>, kind: &str, buffer: Value) -> Value { + let constructor = machine.intrinsics.global(kind).expect("view constructor"); + machine + .construct_value(constructor, &[buffer]) + .expect("view construction succeeds") + } + + fn call( + machine: &mut Machine<'_, H>, + receiver: Value, + method: &str, + args: &[Value], + ) -> Result { + let function = machine.get_named_property(receiver, method).unwrap(); + machine.call_value(function, receiver, args) } #[test] - fn wait_async_reports_mismatch_without_a_promise() { - let program = blank_program("Atomics.waitAsync mismatch"); - let mut host = TestHost; - let mut machine = Machine::new(&program, &mut host, Limits::default()); - let view = shared_int32(&mut machine); - let BuiltinOutcome::Value(result) = wait_async( - &mut machine, - Value::UNDEFINED, - &[view, Value::int32(0), Value::int32(1)], - false, - ) - .unwrap() else { - panic!("waitAsync returns a result object"); - }; - assert_eq!( - machine.get_named_property(result, "async").unwrap(), - Value::FALSE - ); - let value = machine.get_named_property(result, "value").unwrap(); - assert!( + fn views_of_one_block_observe_each_other_through_slice_and_grow() { + with_machine(|machine| { + let buffer = shared_buffer(machine, 4, Some(8)); + let bytes = view_of(machine, "Uint8Array", buffer); machine - .string_value(value) - .is_some_and(|text| text.eq_ascii("not-equal")) - ); + .set_data_property(bytes, "1", Value::int32(7)) + .unwrap(); + + let sliced = call(machine, buffer, "slice", &[Value::int32(1)]).unwrap(); + assert_eq!( + machine.get_named_property(sliced, "byteLength").unwrap(), + Value::int32(3) + ); + let sliced_bytes = view_of(machine, "Uint8Array", sliced); + assert_eq!( + machine.get_named_property(sliced_bytes, "0").unwrap(), + Value::int32(7) + ); + + call(machine, buffer, "grow", &[Value::int32(8)]).unwrap(); + assert_eq!( + machine.get_named_property(bytes, "length").unwrap(), + Value::int32(8), + "a length-tracking view follows the block" + ); + assert_eq!( + machine.get_named_property(bytes, "1").unwrap(), + Value::int32(7), + "growing keeps the existing bytes" + ); + }); + } + + #[test] + fn store_returns_the_coerced_operand_while_the_element_wraps() { + with_machine(|machine| { + let buffer = shared_buffer(machine, 8, None); + let bytes = view_of(machine, "Int8Array", buffer); + let namespace = atomics(machine); + let stored = call( + machine, + namespace, + "store", + &[bytes, Value::int32(0), crate::number_value(257.75)], + ) + .unwrap(); + assert_eq!(stored, crate::number_value(257.0)); + assert_eq!( + machine.get_named_property(bytes, "0").unwrap(), + Value::int32(1) + ); + let stored = call( + machine, + namespace, + "store", + &[bytes, Value::int32(0), crate::number_value(-0.5)], + ) + .unwrap(); + assert!( + super::super::value_number(stored).is_sign_positive(), + "ToIntegerOrInfinity(-0.5) is +0" + ); + }); + } + + #[test] + fn read_modify_write_wraps_at_the_element_width_and_returns_the_old_value() { + with_machine(|machine| { + let buffer = shared_buffer(machine, 8, None); + let bytes = view_of(machine, "Uint8Array", buffer); + machine + .set_data_property(bytes, "1", Value::int32(9)) + .unwrap(); + machine + .set_data_property(bytes, "0", Value::int32(255)) + .unwrap(); + let namespace = atomics(machine); + let old = call( + machine, + namespace, + "add", + &[bytes, Value::int32(0), Value::int32(2)], + ) + .unwrap(); + assert_eq!(old, Value::int32(255)); + assert_eq!( + machine.get_named_property(bytes, "0").unwrap(), + Value::int32(1) + ); + assert_eq!( + machine.get_named_property(bytes, "1").unwrap(), + Value::int32(9), + "the carry must not reach the next element" + ); + }); + } + + #[test] + fn compare_exchange_replaces_only_on_a_match_of_the_wrapped_bytes() { + with_machine(|machine| { + let buffer = shared_buffer(machine, 8, None); + let bytes = view_of(machine, "Uint8Array", buffer); + machine + .set_data_property(bytes, "0", Value::int32(1)) + .unwrap(); + let namespace = atomics(machine); + for (expected, replacement, element) in [(2, 7, 1), (257, 7, 7)] { + call( + machine, + namespace, + "compareExchange", + &[ + bytes, + Value::int32(0), + Value::int32(expected), + Value::int32(replacement), + ], + ) + .unwrap(); + assert_eq!( + machine.get_named_property(bytes, "0").unwrap(), + Value::int32(element) + ); + } + }); + } + + #[test] + fn floating_and_clamped_views_are_rejected_before_the_index_is_read() { + with_machine(|machine| { + let buffer = shared_buffer(machine, 8, None); + for kind in ["Float64Array", "Uint8ClampedArray"] { + let view = view_of(machine, kind, buffer); + let namespace = atomics(machine); + assert!(matches!( + call( + machine, + namespace, + "load", + &[view, crate::number_value(-1.0)] + ), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + }); + } + + fn result_is(machine: &Machine<'_, H>, value: Value, expected: &str) -> bool { + machine + .string_value(value) + .is_some_and(|text| text.eq_ascii(expected)) + } + + #[test] + fn wait_compares_the_value_before_it_suspends() { + with_machine(|machine| { + let buffer = shared_buffer(machine, 8, None); + let view = view_of(machine, "Int32Array", buffer); + let namespace = atomics(machine); + let result = call( + machine, + namespace, + "wait", + &[view, Value::int32(0), Value::int32(1)], + ) + .unwrap(); + assert!(result_is(machine, result, "not-equal")); + }); + } + + #[test] + fn wait_blocks_for_its_timeout_and_leaves_no_waiter_behind() { + with_machine(|machine| { + let buffer = shared_buffer(machine, 8, None); + let view = view_of(machine, "Int32Array", buffer); + let namespace = atomics(machine); + let started = Instant::now(); + let result = call( + machine, + namespace, + "wait", + &[view, Value::int32(0), Value::int32(0), Value::int32(30)], + ) + .unwrap(); + assert!(result_is(machine, result, "timed-out")); + assert!(started.elapsed() >= Duration::from_millis(30)); + let woken = call(machine, namespace, "notify", &[view, Value::int32(0)]).unwrap(); + assert_eq!(woken, Value::int32(0)); + }); + } + + #[test] + fn a_notifier_on_another_thread_wakes_a_suspended_wait() { + with_machine(|machine| { + let buffer = shared_buffer(machine, 8, None); + let view = view_of(machine, "Int32Array", buffer); + let block = shared_block(machine, buffer).unwrap(); + let namespace = atomics(machine); + let notifier = std::thread::spawn(move || { + loop { + let woken = block.waiters().take(0, 1); + if let [Waiter::Blocking(suspension)] = woken.as_slice() { + suspension.wake(); + return; + } + std::thread::yield_now(); + } + }); + let result = call( + machine, + namespace, + "wait", + &[view, Value::int32(0), Value::int32(0)], + ) + .unwrap(); + notifier.join().unwrap(); + assert!(result_is(machine, result, "ok")); + }); } #[test] - fn wait_async_resolves_after_notify_through_the_external_job_queue() { - let program = blank_program("Atomics.waitAsync notify"); + fn cancellation_ends_an_unbounded_wait_and_withdraws_its_waiter() { + let program = blank_program(""); let mut host = TestHost; - let mut machine = Machine::new(&program, &mut host, Limits::default()); - let view = shared_int32(&mut machine); - let BuiltinOutcome::Value(result) = wait_async( + let cancel = CancellationToken::new(); + let mut machine = + Machine::new_with_cancel(&program, &mut host, Limits::default(), cancel.clone()); + let buffer = shared_buffer(&mut machine, 8, None); + let view = view_of(&mut machine, "Int32Array", buffer); + let block = shared_block(&machine, buffer).unwrap(); + let namespace = atomics(&machine); + let canceller = std::thread::spawn(move || { + std::thread::sleep(Duration::from_millis(10)); + cancel.cancel(); + }); + let result = call( &mut machine, - Value::UNDEFINED, - &[ - view, - Value::int32(0), + namespace, + "wait", + &[view, Value::int32(0), Value::int32(0)], + ); + canceller.join().unwrap(); + assert!(matches!( + result, + Err(EvalFailure::Runtime(RuntimeErrorKind::Cancelled)) + )); + assert!(block.waiters().take(0, usize::MAX).is_empty()); + } + + #[test] + fn waiters_queue_per_position_and_wake_oldest_first() { + let block = SharedBlock::new(vec![0; 16], None); + let mut list = block.waiters(); + list.add(0, Waiter::Async(1)); + list.add(8, Waiter::Async(2)); + list.add(0, Waiter::Async(3)); + assert!(matches!(list.take(0, 1).as_slice(), [Waiter::Async(1)])); + assert!(!list.remove(0, &Waiter::Async(1)), "a taken waiter is gone"); + assert!(list.remove(0, &Waiter::Async(3))); + assert!(list.take(0, usize::MAX).is_empty()); + assert!(matches!( + list.take(8, usize::MAX).as_slice(), + [Waiter::Async(2)] + )); + } + + #[test] + fn wait_is_refused_on_unshared_memory_and_views_that_cannot_wait() { + with_machine(|machine| { + let namespace = atomics(machine); + let unshared = machine.intrinsics.global("ArrayBuffer").unwrap(); + let unshared = machine + .construct_value(unshared, &[Value::int32(8)]) + .unwrap(); + let unshared = view_of(machine, "Int32Array", unshared); + let shared = shared_buffer(machine, 8, None); + let bytes = view_of(machine, "Uint8Array", shared); + for view in [unshared, bytes] { + assert!(matches!( + call( + machine, + namespace, + "wait", + &[view, Value::int32(0), Value::int32(0), Value::int32(0)] + ), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + let woken = call(machine, namespace, "notify", &[unshared, Value::int32(0)]).unwrap(); + assert_eq!( + woken, Value::int32(0), - crate::number_value(1000.0), - ], - false, - ) - .unwrap() else { - panic!("waitAsync returns a result object"); + "notify on unshared memory is a no-op" + ); + }); + } + + #[test] + fn an_agent_that_cannot_suspend_rejects_wait_after_validating_its_arguments() { + let program = blank_program(""); + let mut host = TestHost; + let limits = Limits { + agent_can_suspend: false, + ..Limits::default() }; + let mut machine = Machine::new(&program, &mut host, limits); + let buffer = shared_buffer(&mut machine, 8, None); + let view = view_of(&mut machine, "Int32Array", buffer); + let namespace = atomics(&machine); + assert!(matches!( + call( + &mut machine, + namespace, + "wait", + &[view, Value::int32(0), Value::int32(0), Value::int32(0)] + ), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + assert!(matches!( + call( + &mut machine, + namespace, + "wait", + &[view, Value::int32(99), Value::int32(0), Value::int32(0)] + ), + Err(EvalFailure::Throw(ThrowOrigin::RangeError { .. })) + )); + } + + fn int32_view(machine: &mut Machine<'_, H>) -> Value { + let buffer = shared_buffer(machine, 8, None); + view_of(machine, "Int32Array", buffer) + } + + /// Returns the promise of an `Atomics.waitAsync(view, 0, 0, timeout)` that + /// went asynchronous. + fn pending_wait(machine: &mut Machine<'_, H>, view: Value, timeout: Value) -> Value { + let namespace = atomics(machine); + let result = call( + machine, + namespace, + "waitAsync", + &[view, Value::int32(0), Value::int32(0), timeout], + ) + .unwrap(); assert_eq!( machine.get_named_property(result, "async").unwrap(), Value::TRUE ); - let promise = machine.get_named_property(result, "value").unwrap(); - assert!(matches!( - promise_state(&machine, promise), - PromiseState::Pending { .. } - )); - let BuiltinOutcome::Value(count) = notify( - &mut machine, - Value::UNDEFINED, - &[view, Value::int32(0), Value::int32(1)], - false, - ) - .unwrap() else { - panic!("Atomics.notify returns a count"); - }; - assert_eq!(count, crate::number_value(1.0)); - machine.run_to_quiescence().unwrap(); - let PromiseState::Fulfilled { value } = promise_state(&machine, promise) else { - panic!("waitAsync promise was not fulfilled"); + machine.get_named_property(result, "value").unwrap() + } + + fn fulfilled_with(machine: &Machine<'_, H>, promise: Value, expected: &str) -> bool { + let index = machine.runtime_slot(promise).unwrap().unwrap(); + let HeapEntry::Promise { state, .. } = &machine.heap[index] else { + panic!("waitAsync value is a Promise"); }; - assert!( + matches!(state, PromiseState::Fulfilled { value } if result_is(machine, *value, expected)) + } + + fn notify_all(machine: &mut Machine<'_, H>, view: Value) -> Value { + let namespace = atomics(machine); + call(machine, namespace, "notify", &[view, Value::int32(0)]).unwrap() + } + + #[test] + fn wait_async_answers_without_a_promise_when_it_need_not_wait() { + with_machine(|machine| { + let view = int32_view(machine); + let namespace = atomics(machine); + for (expected, timeout, outcome) in [ + (Value::int32(1), Value::UNDEFINED, "not-equal"), + (Value::int32(0), Value::int32(0), "timed-out"), + (Value::int32(0), crate::number_value(-1.0), "timed-out"), + ] { + let result = call( + machine, + namespace, + "waitAsync", + &[view, Value::int32(0), expected, timeout], + ) + .unwrap(); + assert_eq!( + machine.get_named_property(result, "async").unwrap(), + Value::FALSE + ); + let value = machine.get_named_property(result, "value").unwrap(); + assert!(result_is(machine, value, outcome)); + } + assert_eq!(notify_all(machine, view), Value::int32(0)); + }); + } + + #[test] + fn notify_settles_a_timed_wait_at_once_and_disarms_its_timer() { + with_timers(|machine| { + let view = int32_view(machine); + let promise = pending_wait(machine, view, Value::int32(1000)); + assert!(machine.has_pending_timers()); + assert_eq!(notify_all(machine, view), Value::int32(1)); + assert!( + fulfilled_with(machine, promise, "ok"), + "a waiter of this agent resolves inside notify" + ); + assert!(!machine.has_pending_timers()); + assert_eq!(notify_all(machine, view), Value::int32(0)); + }); + } + + #[test] + fn a_timeout_settles_through_the_host_timer_and_empties_the_list() { + let host = with_timers(|machine| { + let view = int32_view(machine); + let promise = pending_wait(machine, view, crate::number_value(10.5)); + machine.run_to_quiescence().unwrap(); + assert!(fulfilled_with(machine, promise, "timed-out")); + assert_eq!(notify_all(machine, view), Value::int32(0)); + }); + assert_eq!(host.timers.now_ms, 11, "a timeout is never cut short"); + } + + #[test] + fn a_timeout_longer_than_one_timer_waits_out_all_of_it() { + let timeout = 3 * (u64::from(u32::MAX) + 1); + let host = with_timers(|machine| { + let view = int32_view(machine); + let promise = pending_wait(machine, view, crate::number_value(timeout as f64)); + machine.run_to_quiescence().unwrap(); + assert!(fulfilled_with(machine, promise, "timed-out")); + }); + assert_eq!(host.timers.now_ms, timeout); + } + + #[test] + fn notify_wakes_async_waiters_oldest_first_up_to_its_count() { + with_machine(|machine| { + let view = int32_view(machine); + let first = pending_wait(machine, view, Value::UNDEFINED); + let second = pending_wait(machine, view, Value::UNDEFINED); + let namespace = atomics(machine); + let woken = call( + machine, + namespace, + "notify", + &[view, Value::int32(0), Value::int32(1)], + ) + .unwrap(); + assert_eq!(woken, Value::int32(1)); + assert!(fulfilled_with(machine, first, "ok")); + assert!(!fulfilled_with(machine, second, "ok")); + assert_eq!(notify_all(machine, view), Value::int32(1)); + assert!(fulfilled_with(machine, second, "ok")); + }); + } + + #[test] + fn an_untimed_wait_does_not_keep_the_event_loop_alive() { + with_machine(|machine| { + let view = int32_view(machine); + let promise = pending_wait(machine, view, Value::UNDEFINED); + machine.run_to_quiescence().unwrap(); + assert!(!fulfilled_with(machine, promise, "ok")); + }); + } + + #[test] + fn a_pending_promise_survives_collection_with_nothing_else_holding_it() { + with_machine(|machine| { + let view = int32_view(machine); + machine.test_set_global("view", view); + let promise = pending_wait(machine, view, Value::UNDEFINED); + machine.collect_garbage(); + assert_eq!(notify_all(machine, view), Value::int32(1)); + assert!(fulfilled_with(machine, promise, "ok")); + }); + } + + #[test] + fn a_script_cannot_cancel_a_wait_timeout_by_guessing_its_timer_number() { + with_timers(|machine| { + let view = int32_view(machine); + let promise = pending_wait(machine, view, Value::int32(5)); + let timer = *machine.timers.keys().next().unwrap(); machine - .string_value(value) - .is_some_and(|text| text.eq_ascii("ok")) - ); + .clear_timeout(Value::int32(u32::try_from(timer).unwrap())) + .unwrap(); + machine.run_to_quiescence().unwrap(); + assert!(fulfilled_with(machine, promise, "timed-out")); + }); + } + + #[test] + fn a_timed_wait_fails_loudly_on_a_host_without_timers() { + with_machine(|machine| { + let view = int32_view(machine); + let namespace = atomics(machine); + assert!(matches!( + call( + machine, + namespace, + "waitAsync", + &[view, Value::int32(0), Value::int32(0), Value::int32(5)] + ), + Err(EvalFailure::Runtime( + RuntimeErrorKind::TimerProviderFailure { .. } + )) + )); + assert_eq!( + notify_all(machine, view), + Value::int32(0), + "a wait that failed to arm leaves no waiter" + ); + }); + } + + #[test] + fn buffers_and_data_views_report_their_own_properties() { + with_machine(|machine| { + let shared = shared_buffer(machine, 8, None); + let data_view = view_of(machine, "DataView", shared); + let unshared = machine.intrinsics.global("ArrayBuffer").unwrap(); + let unshared = machine + .construct_value(unshared, &[Value::int32(8)]) + .unwrap(); + for object in [shared, data_view, unshared] { + machine + .set_data_property(object, "marker", Value::int32(1)) + .unwrap(); + let key = PropertyKey::Named(EcmaString::encode("marker")); + assert!(machine.own_descriptor(object, &key).unwrap().is_some()); + assert_eq!(machine.enumerable_keys(object).unwrap().len(), 1); + } + }); + } + + #[test] + fn grow_rejects_shrinking_and_passing_the_maximum_and_fixed_blocks() { + with_machine(|machine| { + let growable = shared_buffer(machine, 4, Some(8)); + for length in [3, 9] { + assert!(matches!( + call(machine, growable, "grow", &[Value::int32(length)]), + Err(EvalFailure::Throw(ThrowOrigin::RangeError { .. })) + )); + } + let fixed = shared_buffer(machine, 4, None); + assert!(matches!( + call(machine, fixed, "grow", &[Value::int32(4)]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + }); } } diff --git a/crates/bamts-runtime/src/builtins/bigint.rs b/crates/bamts-runtime/src/builtins/bigint.rs index 00e1b2e4..871b7149 100644 --- a/crates/bamts-runtime/src/builtins/bigint.rs +++ b/crates/bamts-runtime/src/builtins/bigint.rs @@ -12,7 +12,7 @@ //! - mixed BigInt/Number arithmetic is a `TypeError`; //! - non-integral `Number -> BigInt` conversion is a `RangeError`; //! - `StringToBigInt` failures are real `SyntaxError` objects built through -//! the installed `SyntaxError` constructor (mirrors `json.rs`); a zero-length +//! the installed `SyntaxError` constructor (mirrors `json_edge.rs`); a zero-length //! or whitespace-only string is `0n`, never an error. use std::collections::BTreeMap; @@ -20,7 +20,10 @@ use std::collections::BTreeMap; use bamts_bytecode::EcmaString; use bamts_native::{Decoded, Value}; -use super::{allocate_string, define_data, heap_index, install_function, range_error, type_error}; +use super::{ + allocate_string, define_data, heap_index, install_constructor_function, install_function, + range_error, type_error, +}; use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; use crate::{ EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, ThrowOrigin, numeric_f64, @@ -1271,7 +1274,7 @@ pub(crate) enum BigIntBinaryOp { /// Maps a core `BigIntError` to an abrupt completion. Type and range stays in /// the typed `ThrowOrigin` channel; syntax failures build a real `SyntaxError` -/// instance through the installed error constructor, exactly as `json.rs` does. +/// instance through the installed error constructor, exactly as `json_edge.rs` does. fn bigint_failure(machine: &mut Machine<'_, H>, error: BigIntError) -> EvalFailure { match error { BigIntError::Range(operation) => range_error(operation), @@ -1663,7 +1666,7 @@ pub(super) fn install( builtins: &mut BuiltinTable, ) -> Value { let prototype = super::super::ordinary_prototype(heap, builtins.object_prototype()); - let constructor = install_function(heap, builtins, "BigInt", 1, constructor::); + let constructor = install_constructor_function(heap, builtins, "BigInt", 1, constructor::); builtins.set_bigint_constructor(constructor); builtins.set_constructor_prototype(heap, constructor, prototype); @@ -1714,7 +1717,8 @@ fn define_native_property(heap: &mut [HeapEntry], object: Value, name: &str, val ); } -/// §21.2.1.1: `BigInt(value)` is callable but not constructible. +/// §21.2.1.1: `BigInt(value)` is callable. It has `[[Construct]]` so it can name +/// an `extends` clause, but reaching it through `new` or `super()` throws here. fn constructor( machine: &mut Machine<'_, H>, _this: Value, @@ -1772,14 +1776,11 @@ fn as_uint_n( /// `ToIndex`: integer truncating, must be in `[0, 2^53 - 1]`. fn to_index(machine: &mut Machine<'_, H>, value: Value) -> Result { - if value == Value::UNDEFINED { - return Ok(0); - } - let number = numeric_f64(machine.to_number(value)?).expect("ToNumber returns numeric"); - if !number.is_finite() || !(0.0..=9_007_199_254_740_991.0).contains(&number) { + let integer = super::to_integer_or_infinity(machine, value)?; + if !(0.0..=9_007_199_254_740_991.0).contains(&integer) { return Err(range_error("Invalid index range")); } - usize::try_from(number.trunc() as u64).map_err(|_| range_error("Invalid index range")) + usize::try_from(integer as u64).map_err(|_| range_error("Invalid index range")) } /// §21.2.3.1: `BigInt.prototype.toString([radix])`. @@ -1817,20 +1818,18 @@ fn value_of( Ok(BuiltinOutcome::Value(allocate_bigint(machine, &bigint)?)) } -/// §21.2.3.3: `BigInt.prototype.toLocaleString()`. Without an ECMA-402 host -/// the fallback the spec permits is the plain decimal `toString` text; locale -/// formatting is an unsupported host capability, not silently faked beyond -/// this spec-sanctioned equivalence. +/// ECMA-402 `BigInt.prototype.toLocaleString(locales, options)`: formats +/// through a new `Intl.NumberFormat`, exactly. fn to_locale_string( machine: &mut Machine<'_, H>, this: Value, - _args: &[Value], + args: &[Value], _constructing: bool, ) -> Result { let bigint = this_bigint_value(machine, this)?; - let text = bigint.to_string_radix(10); - let string = allocate_string(machine, EcmaString::encode(&text))?; - Ok(BuiltinOutcome::Value(string)) + let locales = args.first().copied().unwrap_or(Value::UNDEFINED); + let options = args.get(1).copied().unwrap_or(Value::UNDEFINED); + super::intl::bigint_to_locale_string(machine, &bigint.to_string_radix(10), locales, options) } #[cfg(test)] @@ -2079,6 +2078,32 @@ mod tests { assert_type_error(failure); } + #[test] + fn bigint_has_construct_but_new_throws_from_the_function_body() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + install_bigint(&mut machine); + let bigint = machine.intrinsics.global("BigInt").unwrap(); + assert!( + machine.is_constructor(bigint).unwrap(), + "an extends clause needs BigInt to have [[Construct]]" + ); + assert!(matches!( + machine.construct_value(bigint, &[Value::int32(1)]), + Err(EvalFailure::Throw(crate::ThrowOrigin::TypeError { + operation: "BigInt is not a constructor" + })) + )); + let result = machine + .call_value(bigint, Value::UNDEFINED, &[Value::int32(1)]) + .unwrap(); + assert!( + !machine.is_object(result), + "a plain call still returns a primitive bigint" + ); + } + #[test] fn constructor_and_prototype_descriptors_are_exact() { let module = blank_program(""); @@ -2388,8 +2413,8 @@ mod tests { .expect("toLocaleString succeeds"); let text = machine.to_string(locale).expect("locale result is text"); assert!( - text.eq_ascii("-9099"), - "toLocaleString matches decimal toString" + text.eq_ascii("-9,099"), + "toLocaleString groups like Intl.NumberFormat" ); } diff --git a/crates/bamts-runtime/src/builtins/collections.rs b/crates/bamts-runtime/src/builtins/collections.rs index f69b2d61..31c4c49c 100644 --- a/crates/bamts-runtime/src/builtins/collections.rs +++ b/crates/bamts-runtime/src/builtins/collections.rs @@ -7,7 +7,7 @@ use super::{ allocate_array, builtin_property, define_data, heap_index, install_constructor_function, install_function, type_error, }; -use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; +use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable, IteratorFamily}; use crate::vm::async_iterators::{ASYNC_FROM_SYNC_METHODS, ASYNC_GENERATOR_METHODS}; use crate::vm::generator_async::{GENERATOR_METHODS, GeneratorCompletion, GeneratorOperation}; use crate::{ @@ -139,7 +139,6 @@ pub(super) fn install_iterator_prototype( builtins: &mut BuiltinTable, ) { let prototype = ordinary(heap, Some(builtins.object_prototype())); - let next = install_function(heap, builtins, "next", 0, iterator_next::); let identity = install_function( heap, builtins, @@ -147,9 +146,25 @@ pub(super) fn install_iterator_prototype( 0, iterator_identity::, ); - define_data(heap, prototype, "next", next); define_symbol(heap, prototype, builtins.symbol_iterator(), identity); builtins.set_iterator_prototype(prototype); + let family_prototypes = [ + ( + "Array Iterator", + array_iterator_next:: as BuiltinHandler, + ), + ("Map Iterator", map_iterator_next::), + ("Set Iterator", set_iterator_next::), + ("String Iterator", string_iterator_next::), + ] + .map(|(tag, handler)| { + let family = ordinary(heap, Some(prototype)); + let next = install_function(heap, builtins, "next", 0, handler); + define_data(heap, family, "next", next); + super::define_to_string_tag(heap, family, builtins.symbol_to_string_tag(), tag); + family + }); + builtins.set_iterator_family_prototypes(family_prototypes); } pub(super) fn install_async_iterator_prototype( @@ -327,18 +342,46 @@ fn generator_throw( }) } +fn iterator_family( + machine: &Machine<'_, H>, + source: Value, + kind: IterationKind, +) -> Result { + if kind == IterationKind::StringCodePoint { + return Ok(IteratorFamily::String); + } + let index = machine.runtime_slot(source).map_err(EvalFailure::Runtime)?; + Ok(match index.map(|index| &machine.heap[index]) { + Some(HeapEntry::Collection { + kind: CollectionKind::Map, + .. + }) => IteratorFamily::Map, + Some(HeapEntry::Collection { + kind: CollectionKind::Set, + .. + }) => IteratorFamily::Set, + _ => IteratorFamily::Array, + }) +} + pub(super) fn iterator( machine: &mut Machine<'_, H>, source: Value, kind: IterationKind, ) -> Result { + let family = iterator_family(machine, source, kind)?; machine .allocate(HeapEntry::BuiltinIterator { source, kind, position: Some(0), properties: PropertyMap::default(), - prototype: Some(machine.intrinsics.builtins.iterator_prototype()), + prototype: Some( + machine + .intrinsics + .builtins + .iterator_family_prototype(family), + ), extensible: true, }) .map_err(EvalFailure::Runtime) @@ -353,11 +396,46 @@ fn iterator_identity( Ok(BuiltinOutcome::Value(this)) } -fn iterator_next( +fn array_iterator_next( machine: &mut Machine<'_, H>, this: Value, _args: &[Value], _constructing: bool, +) -> Result { + iterator_next(machine, this, IteratorFamily::Array) +} + +fn map_iterator_next( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + iterator_next(machine, this, IteratorFamily::Map) +} + +fn set_iterator_next( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + iterator_next(machine, this, IteratorFamily::Set) +} + +fn string_iterator_next( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + iterator_next(machine, this, IteratorFamily::String) +} + +fn iterator_next( + machine: &mut Machine<'_, H>, + this: Value, + family: IteratorFamily, ) -> Result { let Some(iterator_index) = machine.runtime_slot(this).map_err(EvalFailure::Runtime)? else { return Err(type_error("iterator next called on incompatible receiver")); @@ -371,6 +449,16 @@ fn iterator_next( else { return Err(type_error("iterator next called on incompatible receiver")); }; + let builtin_family = matches!( + kind, + IterationKind::Key + | IterationKind::Value + | IterationKind::Entry + | IterationKind::StringCodePoint + ); + if !builtin_family || iterator_family(machine, source, kind)? != family { + return Err(type_error("iterator next called on incompatible receiver")); + } let (done, value, next_position) = match position { None => (true, Value::UNDEFINED, None), Some(cursor) => { @@ -382,6 +470,9 @@ fn iterator_next( let bounds = crate::intrinsics::builtins::typedarray_all::typed_array_bounds( machine, source, )?; + if bounds.out_of_bounds { + return Err(type_error("iterated typed array is out of bounds")); + } match usize::try_from(cursor) { Ok(index) if !bounds.out_of_bounds && index < bounds.element_length => { let value = crate::intrinsics::builtins::typedarray_all::read_element( @@ -439,8 +530,13 @@ fn iterator_next( Some((next, key, item_value)) => { let value = match kind { IterationKind::Key => key, - IterationKind::Value => item_value, + IterationKind::Value | IterationKind::StringCodePoint => item_value, IterationKind::Entry => allocate_array(machine, vec![key, item_value])?, + IterationKind::RegExpString + | IterationKind::IteratorHelper(_) + | IterationKind::WrapForValid => { + unreachable!("the family check admits only the four families") + } }; (false, value, Some(next)) } @@ -920,7 +1016,7 @@ fn map_for_each( collection_next(machine, this, CollectionKind::Map, cursor)? { cursor = next; - machine.call_value( + machine.call_callback( callback, args.get(1).copied().unwrap_or(Value::UNDEFINED), &[value, key, this], @@ -1048,7 +1144,7 @@ fn set_for_each( while let Some((next, _, value)) = collection_next(machine, this, CollectionKind::Set, cursor)? { cursor = next; - machine.call_value( + machine.call_callback( callback, args.get(1).copied().unwrap_or(Value::UNDEFINED), &[value, value, this], @@ -1057,7 +1153,7 @@ fn set_for_each( Ok(BuiltinOutcome::Value(Value::UNDEFINED)) } -fn map_put( +pub(super) fn map_put( machine: &mut Machine<'_, H>, object: Value, key: Value, @@ -1196,7 +1292,7 @@ fn collection( .map_err(EvalFailure::Runtime) } -fn collection_slot( +pub(super) fn collection_slot( machine: &Machine<'_, H>, object: Value, expected: CollectionKind, @@ -1254,7 +1350,10 @@ fn collection_iterator( Ok(BuiltinOutcome::Value(iterator(machine, object, kind)?)) } -fn require_weak_key(machine: &Machine<'_, H>, key: Value) -> Result<(), EvalFailure> { +pub(super) fn require_weak_key( + machine: &Machine<'_, H>, + key: Value, +) -> Result<(), EvalFailure> { let Some(index) = machine.runtime_slot(key).map_err(EvalFailure::Runtime)? else { return Err(type_error("Invalid value used as weak collection key")); }; @@ -1279,6 +1378,8 @@ fn require_weak_key(machine: &Machine<'_, H>, key: Value) -> Result<(), | HeapEntry::HashState { .. } | HeapEntry::RegExp { .. } | HeapEntry::Date { .. } + | HeapEntry::Temporal { .. } + | HeapEntry::Intl { .. } | HeapEntry::Collection { .. } | HeapEntry::TypedArray { .. } | HeapEntry::ArrayBuffer { .. } @@ -1327,6 +1428,9 @@ fn constructor_prototype( }; Ok(prototype) } +pub(super) fn ordinary_intrinsic(heap: &mut Vec, prototype: Value) -> Value { + ordinary(heap, Some(prototype)) +} fn ordinary(heap: &mut Vec, prototype: Option) -> Value { super::super::push( heap, @@ -1812,6 +1916,35 @@ mod tests { // ---- %GeneratorPrototype% tests ---------------------------------------- + #[test] + fn each_iterator_family_owns_next_and_rejects_the_others() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let iterator_prototype = machine.intrinsics.builtins.iterator_prototype(); + assert_eq!( + machine + .get_named_property(iterator_prototype, "next") + .unwrap(), + Value::UNDEFINED, + "a subclass of Iterator must not inherit a builtin next" + ); + let array = allocate_array(&mut machine, Vec::new()).unwrap(); + let array_iterator = iterator(&mut machine, array, IterationKind::Value).unwrap(); + let map_constructor = machine.intrinsics.global("Map").unwrap(); + let map = machine.construct_value(map_constructor, &[]).unwrap(); + let map_iterator = iterator(&mut machine, map, IterationKind::Entry).unwrap(); + + let array_next = machine.get_named_property(array_iterator, "next").unwrap(); + let map_next = machine.get_named_property(map_iterator, "next").unwrap(); + assert_ne!(array_next, map_next); + assert!(machine.call_value(array_next, array_iterator, &[]).is_ok()); + assert!(matches!( + machine.call_value(array_next, map_iterator, &[]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + #[test] fn generator_prototype_chains_to_iterator_prototype() { let module = blank_program(""); diff --git a/crates/bamts-runtime/src/builtins/date.rs b/crates/bamts-runtime/src/builtins/date.rs deleted file mode 100644 index 629b57d5..00000000 --- a/crates/bamts-runtime/src/builtins/date.rs +++ /dev/null @@ -1,1013 +0,0 @@ -use std::collections::BTreeMap; - -use bamts_bytecode::EcmaString; -use bamts_native::Value; - -use super::{ - allocate_string, define_data, install_function, range_error, type_error, value_number, -}; -use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; -use crate::{EvalFailure, HeapEntry, Host, Machine, PropertyKey, PropertyMap}; - -pub(super) fn install( - heap: &mut Vec, - globals: &mut BTreeMap, - builtins: &mut BuiltinTable, -) { - let prototype = super::super::ordinary_prototype(heap, builtins.object_prototype()); - builtins.set_date_prototype(prototype); - let constructor = install_function(heap, builtins, "Date", 7, constructor::); - builtins.set_constructor_prototype(heap, constructor, prototype); - let now = install_function(heap, builtins, "now", 0, now::); - let HeapEntry::NativeFunction { properties, .. } = &mut heap[super::heap_index(constructor)] - else { - unreachable!() - }; - properties.insert( - PropertyKey::Named(EcmaString::encode("now")), - super::builtin_property(now), - ); - for (name, handler) in [ - ("getTime", get_time:: as BuiltinHandler), - ("valueOf", get_time::), - ("toISOString", to_iso_string::), - ] { - let function = install_function(heap, builtins, name, 0, handler); - define_data(heap, prototype, name, function); - } - globals.insert(EcmaString::encode("Date"), constructor); -} - -fn constructor( - machine: &mut Machine<'_, H>, - _this: Value, - args: &[Value], - constructing: bool, -) -> Result { - if !constructing { - let text = to_date_string(time_clip(machine.host.now_ms() as f64)) - .unwrap_or_else(|| "Invalid Date".to_owned()); - return Ok(BuiltinOutcome::Value(allocate_string( - machine, - EcmaString::encode(&text), - )?)); - } - - let milliseconds = if args.is_empty() { - time_clip(machine.host.now_ms() as f64) - } else if args.len() == 1 { - let value = args[0]; - let copied_time = machine - .runtime_slot(value) - .map_err(EvalFailure::Runtime)? - .and_then(|index| match &machine.heap[index] { - HeapEntry::Date { time, .. } => Some(*time), - _ => None, - }); - if let Some(time) = copied_time { - time_clip(time) - } else { - let primitive = machine.coerce_primitive_default(value)?; - if let Some(text) = machine.string_value(primitive) { - parse_iso_date(&text).unwrap_or(f64::NAN) - } else { - time_clip(value_number(machine.coerce_number_observable(primitive)?)) - } - } - } else { - let mut components = [0.0; 7]; - components[2] = 1.0; - for (i, component) in components.iter_mut().enumerate() { - if let Some(&argument) = args.get(i) { - *component = value_number(machine.coerce_number_observable(argument)?); - } - } - date_from_components(components) - }; - let default_prototype = machine.intrinsics.builtins.date_prototype(); - let new_target = machine.current_new_target(); - let prototype = if new_target != Value::UNDEFINED { - let candidate = machine.get_named_property(new_target, "prototype")?; - if machine.is_object(candidate) { - candidate - } else { - default_prototype - } - } else { - default_prototype - }; - let object = machine - .allocate(HeapEntry::Date { - time: milliseconds, - properties: PropertyMap::default(), - prototype: Some(prototype), - extensible: true, - }) - .map_err(EvalFailure::Runtime)?; - Ok(BuiltinOutcome::Value(object)) -} - -fn now( - machine: &mut Machine<'_, H>, - _this: Value, - _args: &[Value], - _constructing: bool, -) -> Result { - Ok(BuiltinOutcome::Value(crate::number_value( - machine.host.now_ms() as f64, - ))) -} - -fn get_time( - machine: &mut Machine<'_, H>, - this: Value, - _args: &[Value], - _constructing: bool, -) -> Result { - Ok(BuiltinOutcome::Value(crate::number_value(date_time( - machine, this, - )?))) -} - -fn to_iso_string( - machine: &mut Machine<'_, H>, - this: Value, - _args: &[Value], - _constructing: bool, -) -> Result { - let milliseconds = date_time(machine, this)?; - let text = iso_string(milliseconds).ok_or_else(|| range_error("Invalid time value"))?; - Ok(BuiltinOutcome::Value(allocate_string( - machine, - EcmaString::encode(&text), - )?)) -} - -fn date_time(machine: &Machine<'_, H>, this: Value) -> Result { - let Some(index) = machine.runtime_slot(this).map_err(EvalFailure::Runtime)? else { - return Err(type_error("Date method called on incompatible receiver")); - }; - let HeapEntry::Date { time, .. } = &machine.heap[index] else { - return Err(type_error("Date method called on incompatible receiver")); - }; - Ok(*time) -} - -fn iso_string(milliseconds: f64) -> Option { - if !milliseconds.is_finite() || milliseconds.abs() > 8_640_000_000_000_000.0 { - return None; - } - let millis = milliseconds.trunc() as i64; - let seconds = millis.div_euclid(1000); - let millisecond = millis.rem_euclid(1000); - let days = seconds.div_euclid(86_400); - let day_seconds = seconds.rem_euclid(86_400); - let (year, month, day) = civil_from_days(days); - let hour = day_seconds / 3600; - let minute = day_seconds % 3600 / 60; - let second = day_seconds % 60; - let year_text = if (0..=9999).contains(&year) { - format!("{year:04}") - } else if year < 0 { - format!("-{abs:06}", abs = year.unsigned_abs()) - } else { - format!("+{year:06}") - }; - Some(format!( - "{year_text}-{month:02}-{day:02}T{hour:02}:{minute:02}:{second:02}.{millisecond:03}Z" - )) -} - -/// `ToDateString`: the implementation-defined human-readable form used by both -/// `Date()` (no `new`) and `Date.prototype.toString`. The runtime is UTC-only, -/// so the offset is fixed at `GMT+0000` with the `(Coordinated Universal Time)` -/// time-zone name, matching the host's `now_ms` UTC epoch. -fn to_date_string(milliseconds: f64) -> Option { - if !milliseconds.is_finite() || milliseconds.abs() > 8_640_000_000_000_000.0 { - return None; - } - let millis = milliseconds.trunc() as i64; - let seconds = millis.div_euclid(1000); - let days = seconds.div_euclid(86_400); - let day_seconds = seconds.rem_euclid(86_400); - let (year, month, day) = civil_from_days(days); - let hour = day_seconds / 3600; - let minute = day_seconds % 3600 / 60; - let second = day_seconds % 60; - const WEEKDAYS: [&str; 7] = ["Sun", "Mon", "Tue", "Wed", "Thu", "Fri", "Sat"]; - const MONTHS: [&str; 12] = [ - "Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dec", - ]; - // 1970-01-01 was a Thursday (index 4); rem_euclid keeps negative days correct. - let weekday = ((days.rem_euclid(7) + 4) % 7) as usize; - let year_text = if (0..=9999).contains(&year) { - format!("{year:04}") - } else if year < 0 { - format!("-{abs:06}", abs = year.unsigned_abs()) - } else { - format!("+{year:06}") - }; - Some(format!( - "{} {} {:02} {} {:02}:{:02}:{:02} GMT+0000 (Coordinated Universal Time)", - WEEKDAYS[weekday], - MONTHS[(month - 1) as usize], - day, - year_text, - hour, - minute, - second - )) -} - -fn parse_iso_date(text: &EcmaString) -> Option { - let units = text.as_units(); - let mut cursor = 0; - let year = parse_year(units, &mut cursor)?; - if cursor == units.len() { - return time_clip_option(milliseconds_from_civil(year, 1, 1, 0, 0, 0, 0)?); - } - - consume(units, &mut cursor, b'-')?; - let month = parse_component(units, &mut cursor, 2)?; - if !(1..=12).contains(&month) { - return None; - } - if cursor == units.len() { - return time_clip_option(milliseconds_from_civil(year, month, 1, 0, 0, 0, 0)?); - } - - consume(units, &mut cursor, b'-')?; - let day = parse_component(units, &mut cursor, 2)?; - if day == 0 || day > days_in_month(year, month) { - return None; - } - if cursor == units.len() { - return time_clip_option(milliseconds_from_civil(year, month, day, 0, 0, 0, 0)?); - } - - consume(units, &mut cursor, b'T')?; - let hour = parse_component(units, &mut cursor, 2)?; - consume(units, &mut cursor, b':')?; - let minute = parse_component(units, &mut cursor, 2)?; - let mut second = 0; - let mut millisecond = 0; - if matches!(units.get(cursor), Some(unit) if *unit == u16::from(b':')) { - cursor += 1; - second = parse_component(units, &mut cursor, 2)?; - millisecond = parse_fraction(units, &mut cursor)?; - } - if hour > 24 - || minute > 59 - || second > 59 - || (hour == 24 && (minute != 0 || second != 0 || millisecond != 0)) - { - return None; - } - - let offset_minutes = match units.get(cursor).copied() { - None => 0, - Some(unit) if unit == u16::from(b'Z') => { - cursor += 1; - 0 - } - Some(unit @ (0x002B | 0x002D)) => { - cursor += 1; - let offset_hour = parse_component(units, &mut cursor, 2)?; - consume(units, &mut cursor, b':')?; - let offset_minute = parse_component(units, &mut cursor, 2)?; - if offset_hour > 23 || offset_minute > 59 { - return None; - } - let minutes = offset_hour.checked_mul(60)?.checked_add(offset_minute)?; - if unit == u16::from(b'-') { - -minutes - } else { - minutes - } - } - Some(_) => return None, - }; - if cursor != units.len() { - return None; - } - - let milliseconds = - milliseconds_from_civil(year, month, day, hour, minute, second, millisecond)? - .checked_sub(offset_minutes.checked_mul(60_000)?)?; - time_clip_option(milliseconds) -} - -fn date_from_components(components: [f64; 7]) -> f64 { - let [year, month, day, hour, minute, second, millisecond] = components; - let Some(mut year) = integer_component(year) else { - return f64::NAN; - }; - let Some(month) = integer_component(month) else { - return f64::NAN; - }; - let Some(day) = integer_component(day) else { - return f64::NAN; - }; - let Some(hour) = integer_component(hour) else { - return f64::NAN; - }; - let Some(minute) = integer_component(minute) else { - return f64::NAN; - }; - let Some(second) = integer_component(second) else { - return f64::NAN; - }; - let Some(millisecond) = integer_component(millisecond) else { - return f64::NAN; - }; - - if (0..=99).contains(&year) { - year = year.saturating_add(1900); - } - let Some(year) = year.checked_add(month.div_euclid(12)) else { - return f64::NAN; - }; - let month = month.rem_euclid(12) + 1; - let Some(milliseconds) = - milliseconds_from_civil(year, month, day, hour, minute, second, millisecond) - else { - return f64::NAN; - }; - time_clip(milliseconds as f64) -} - -fn time_clip(milliseconds: f64) -> f64 { - if !milliseconds.is_finite() || milliseconds.abs() > 8_640_000_000_000_000.0 { - return f64::NAN; - } - let milliseconds = milliseconds.trunc(); - if milliseconds == 0.0 { - 0.0 - } else { - milliseconds - } -} - -fn time_clip_option(milliseconds: i64) -> Option { - let milliseconds = time_clip(milliseconds as f64); - milliseconds.is_finite().then_some(milliseconds) -} - -fn integer_component(value: f64) -> Option { - (value.is_finite() - && (-9_223_372_036_854_775_808.0..9_223_372_036_854_775_808.0).contains(&value)) - .then(|| value.trunc() as i64) -} - -fn parse_year(units: &[u16], cursor: &mut usize) -> Option { - let signed = matches!(units.get(*cursor), Some(unit) if *unit == u16::from(b'+') || *unit == u16::from(b'-')); - let sign = if signed { - let sign = units[*cursor]; - *cursor += 1; - sign - } else { - u16::from(b'+') - }; - let year = parse_component(units, cursor, if signed { 6 } else { 4 })?; - if sign == u16::from(b'-') { - (year != 0).then_some(-year) - } else { - Some(year) - } -} - -fn parse_component(units: &[u16], cursor: &mut usize, len: usize) -> Option { - let value = decimal_component(units, *cursor, len)?; - *cursor = cursor.checked_add(len)?; - Some(value) -} - -fn consume(units: &[u16], cursor: &mut usize, expected: u8) -> Option<()> { - (units.get(*cursor) == Some(&u16::from(expected))).then(|| { - *cursor += 1; - }) -} - -fn parse_fraction(units: &[u16], cursor: &mut usize) -> Option { - if units.get(*cursor) != Some(&u16::from(b'.')) { - return Some(0); - } - *cursor += 1; - let mut millisecond = 0; - let mut place = 100; - let mut digits = 0; - while let Some(&unit) = units.get(*cursor) { - if !(u16::from(b'0')..=u16::from(b'9')).contains(&unit) { - break; - } - if place > 0 { - millisecond += i64::from(unit - u16::from(b'0')) * place; - place /= 10; - } - digits += 1; - *cursor += 1; - } - (digits > 0).then_some(millisecond) -} - -fn milliseconds_from_civil( - year: i64, - month: i64, - day: i64, - hour: i64, - minute: i64, - second: i64, - millisecond: i64, -) -> Option { - let days = days_from_civil_checked(year, month, day)?; - days.checked_mul(86_400_000)? - .checked_add(hour.checked_mul(3_600_000)?)? - .checked_add(minute.checked_mul(60_000)?)? - .checked_add(second.checked_mul(1_000)?)? - .checked_add(millisecond) -} - -fn days_from_civil_checked(year: i64, month: i64, day: i64) -> Option { - let year = year.checked_sub(i64::from(month <= 2))?; - let era = year.div_euclid(400); - let year_of_era = year.checked_sub(era.checked_mul(400)?)?; - let month_prime = month + if month > 2 { -3 } else { 9 }; - let day_of_year = (153 * month_prime + 2) / 5 + day.checked_sub(1)?; - era.checked_mul(146_097)? - .checked_add(year_of_era * 365)? - .checked_add(year_of_era / 4)? - .checked_sub(year_of_era / 100)? - .checked_add(day_of_year)? - .checked_sub(719_468) -} - -fn decimal_component(units: &[u16], start: usize, len: usize) -> Option { - let end = start.checked_add(len)?; - let mut value = 0_i64; - for &unit in units.get(start..end)? { - if !(u16::from(b'0')..=u16::from(b'9')).contains(&unit) { - return None; - } - value = value * 10 + i64::from(unit - u16::from(b'0')); - } - Some(value) -} - -fn days_in_month(year: i64, month: i64) -> i64 { - match month { - 1 | 3 | 5 | 7 | 8 | 10 | 12 => 31, - 4 | 6 | 9 | 11 => 30, - 2 if year % 4 == 0 && (year % 100 != 0 || year % 400 == 0) => 29, - 2 => 28, - _ => 0, - } -} - -fn civil_from_days(days_since_epoch: i64) -> (i64, i64, i64) { - let z = days_since_epoch + 719_468; - let era = if z >= 0 { z } else { z - 146_096 } / 146_097; - let day_of_era = z - era * 146_097; - let year_of_era = - (day_of_era - day_of_era / 1460 + day_of_era / 36_524 - day_of_era / 146_096) / 365; - let mut year = year_of_era + era * 400; - let day_of_year = day_of_era - (365 * year_of_era + year_of_era / 4 - year_of_era / 100); - let month_prime = (5 * day_of_year + 2) / 153; - let day = day_of_year - (153 * month_prime + 2) / 5 + 1; - let month = month_prime + if month_prime < 10 { 3 } else { -9 }; - year += i64::from(month <= 2); - (year, month, day) -} - -#[cfg(test)] -mod tests { - use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering}; - - use bamts_native::Value; - - use super::super::test_support::{blank_program, ordinary_object}; - use super::*; - use crate::intrinsics::{BuiltinDef, BuiltinHandler, native_function}; - use crate::{Limits, Property}; - use bamts_bytecode::{FunctionId, ModuleId}; - - struct TestHost; - - impl Host for TestHost { - fn now_ms(&mut self) -> u64 { - 1_704_067_200_123 - } - } - - fn call_date(machine: &mut Machine<'_, TestHost>, args: &[Value], constructing: bool) -> Value { - let constructor = machine.intrinsics.global("Date").expect("Date exists"); - let index = machine.runtime_slot(constructor).unwrap().unwrap(); - let HeapEntry::NativeFunction { - callable: crate::NativeCallable::Builtin(id), - .. - } = machine.heap[index] - else { - panic!("Date constructor is native"); - }; - let BuiltinOutcome::Value(value) = machine - .call_builtin(id, Value::UNDEFINED, args, constructing) - .expect("Date call succeeds") - else { - panic!("Date returns a value"); - }; - value - } - - #[test] - fn formats_node_24_iso_dates() { - assert_eq!(iso_string(0.0).as_deref(), Some("1970-01-01T00:00:00.000Z")); - assert_eq!( - iso_string(-1.0).as_deref(), - Some("1969-12-31T23:59:59.999Z") - ); - assert_eq!( - iso_string(1_704_067_200_123.0).as_deref(), - Some("2024-01-01T00:00:00.123Z") - ); - } - - #[test] - fn call_form_ignores_every_argument_and_uses_host_time() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let garbage = machine - .allocate(HeapEntry::String(EcmaString::encode("not a date"))) - .expect("string allocation succeeds"); - - // Date() without `new` returns ToDateString — the human-readable - // toString form — NOT the ISO-8601 string. ECMA-262 §21.4.2.1. - let expected = to_date_string(1_704_067_200_123.0).unwrap(); - for args in [&[][..], &[Value::int32(0)][..], &[garbage][..]] { - let value = call_date(&mut machine, args, false); - assert_eq!( - machine.string_value(value).unwrap().as_units(), - EcmaString::encode(&expected).as_units(), - "Date() must return the toString form, not ISO-8601" - ); - // Guard against regression to the ISO form. - assert_ne!( - machine.string_value(value).unwrap().as_units(), - EcmaString::encode("2024-01-01T00:00:00.123Z").as_units() - ); - } - } - - #[test] - fn call_form_returns_to_date_string_not_iso() { - // Direct regression: the spec ToDateString form starts with a weekday - // and contains "GMT", never the ISO-8601 date/time separator. - let text = to_date_string(1_704_067_200_123.0).unwrap(); - assert!(text.starts_with("Mon Jan 01 2024")); - assert!(text.contains("GMT+0000 (Coordinated Universal Time)")); - // ISO-8601 is `YYYY-MM-DDTHH:MM:SS.sssZ`; the ToDateString form is - // `Www Mmm DD YYYY HH:MM:SS GMT+0000 (...)`. The two are told apart - // by the `T` date/time separator at the fixed offset 10 — a `T` that - // appears inside "Time" or weekday "Tue"/"Thu" lives elsewhere, so a - // bare `contains('T')` cannot distinguish the forms. - let bytes = text.as_bytes(); - let is_iso_prefix = - bytes.len() >= 11 && bytes[4] == b'-' && bytes[7] == b'-' && bytes[10] == b'T'; - assert!(!is_iso_prefix, "ToDateString must not be ISO-8601"); - - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let value = call_date(&mut machine, &[], false); - let result = machine.string_value(value).unwrap(); - assert_eq!(result.as_units(), EcmaString::encode(&text).as_units()); - } - - #[test] - fn date_subclass_instance_has_subclass_prototype() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - - // Build a stand-in for `class D extends Date {}`: a constructor - // function whose `prototype` is a fresh ordinary object that itself - // inherits from Date.prototype. - let date_constructor = machine.intrinsics.global("Date").expect("Date installed"); - let date_prototype = machine - .get_named_property(date_constructor, "prototype") - .unwrap(); - - let sub_prototype = ordinary_object(&mut machine); - machine - .set_prototype(sub_prototype, date_prototype) - .unwrap(); - - let mut properties = PropertyMap::default(); - properties.insert( - PropertyKey::Named(EcmaString::encode("prototype")), - Property::Data { - value: sub_prototype, - writable: true, - enumerable: false, - configurable: false, - }, - ); - let sub_constructor = machine - .allocate(HeapEntry::Function { - module: ModuleId::new(0), - function: FunctionId::new(0), - captures: Vec::new(), - context: None, - properties, - prototype: Some(machine.intrinsics.function_prototype), - extensible: true, - }) - .unwrap(); - - // Construct via Date with new.target = sub_constructor, simulating - // `super()` inside `class D extends Date`. - let date_index = machine.runtime_slot(date_constructor).unwrap().unwrap(); - let HeapEntry::NativeFunction { - callable: crate::NativeCallable::Builtin(date_id), - .. - } = machine.heap[date_index] - else { - panic!("Date is a builtin"); - }; - let BuiltinOutcome::Value(instance) = machine - .call_builtin_with_new_target(date_id, Value::UNDEFINED, &[], true, sub_constructor) - .expect("Date construct succeeds") - else { - panic!("Date construct returns a value"); - }; - - // The instance must inherit from the subclass prototype, not directly - // from Date.prototype. - assert_eq!( - machine.internal_get_prototype_of(instance).unwrap(), - Some(sub_prototype), - "subclass instance must carry the subclass prototype" - ); - assert!( - machine.instance_of(instance, sub_constructor).unwrap(), - "instanceof SubClass must be true" - ); - assert!( - machine.instance_of(instance, date_constructor).unwrap(), - "instanceof Date must still be true" - ); - } - - #[test] - fn multi_argument_construction_normalizes_components_and_time_clips() { - assert_eq!( - date_from_components([2024.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0]), - 1_704_067_200_000.0 - ); - assert_eq!( - date_from_components([99.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0]), - 915_148_800_000.0 - ); - assert_eq!( - date_from_components([2024.0, 1.0, 30.0, 24.0, 0.0, 0.0, 0.0]), - 1_709_337_600_000.0 - ); - assert!(date_from_components([1.0e9, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0]).is_nan()); - - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let value = call_date( - &mut machine, - &[Value::int32(2024), Value::int32(0), Value::int32(1)], - true, - ); - assert_eq!(date_time(&machine, value).unwrap(), 1_704_067_200_000.0); - } - - #[test] - fn parses_node_date_only_time_offset_and_extended_year_forms() { - for (text, milliseconds) in [ - ("2024", 1_704_067_200_000.0), - ("2024-01", 1_704_067_200_000.0), - ("2024-01-01", 1_704_067_200_000.0), - ("2024-01-01T00:00Z", 1_704_067_200_000.0), - ("2024-01-01T00:00:00Z", 1_704_067_200_000.0), - ("2024-01-01T00:00:00", 1_704_067_200_000.0), - ("2024-01-01T00:00:00.1Z", 1_704_067_200_100.0), - ("2024-01-01T01:30:00+01:30", 1_704_067_200_000.0), - ("2024-01-01T00:00:00-02:30", 1_704_076_200_000.0), - ("2024-01-01T24:00:00Z", 1_704_153_600_000.0), - ] { - assert_eq!( - parse_iso_date(&EcmaString::encode(text)), - Some(milliseconds), - "{text}" - ); - } - - for (text, expected) in [ - ("+006024-02-29T23:59:59.123Z", "6024-02-29T23:59:59.123Z"), - ("+010000-01-01T00:00:00.000Z", "+010000-01-01T00:00:00.000Z"), - ("-000001-01-01T00:00:00.000Z", "-000001-01-01T00:00:00.000Z"), - ] { - let milliseconds = - parse_iso_date(&EcmaString::encode(text)).expect("valid extended year"); - assert_eq!( - iso_string(milliseconds).as_deref(), - Some(expected), - "{text}" - ); - } - - for text in [ - "2024-02-30T00:00:00.000Z", - "2023-02-29T00:00:00.000Z", - "2024-01-01T24:00:00.001Z", - "2024-01-01T00:60:00.000Z", - "2024-01-01T00:00:60.000Z", - "2024-01-01T00:00:00.Z", - "2024-01-01 00:00:00.000Z", - "-000000-01-01T00:00:00.000Z", - ] { - assert_eq!(parse_iso_date(&EcmaString::encode(text)), None, "{text}"); - } - } - static VALUE_OF_CALLED: AtomicBool = AtomicBool::new(false); - static DATE_VALUE_OF_CALLED: AtomicBool = AtomicBool::new(false); - - fn native( - machine: &mut Machine<'_, TestHost>, - name: &'static str, - handler: BuiltinHandler, - ) -> Value { - let id = machine.intrinsics.builtins.register(BuiltinDef { - name, - length: 0, - handler, - }); - native_function(&mut machine.heap, id, name, 0) - } - - fn generic_value_of( - _machine: &mut Machine<'_, TestHost>, - _this: Value, - _args: &[Value], - _constructing: bool, - ) -> Result { - VALUE_OF_CALLED.store(true, Ordering::SeqCst); - Ok(BuiltinOutcome::Value(Value::int32(12_345))) - } - - fn date_value_of_override( - _machine: &mut Machine<'_, TestHost>, - _this: Value, - _args: &[Value], - _constructing: bool, - ) -> Result { - DATE_VALUE_OF_CALLED.store(true, Ordering::SeqCst); - Ok(BuiltinOutcome::Value(Value::int32(99_999))) - } - - static CALL_ORDER: AtomicUsize = AtomicUsize::new(0); - static YEAR_ORDER: AtomicUsize = AtomicUsize::new(0); - static MONTH_ORDER: AtomicUsize = AtomicUsize::new(0); - static DAY_ORDER: AtomicUsize = AtomicUsize::new(0); - static TO_STRING_CALLED: AtomicBool = AtomicBool::new(false); - - fn year_value_of( - _machine: &mut Machine<'_, TestHost>, - _this: Value, - _args: &[Value], - _constructing: bool, - ) -> Result { - YEAR_ORDER.store( - CALL_ORDER.fetch_add(1, Ordering::SeqCst) + 1, - Ordering::SeqCst, - ); - Ok(BuiltinOutcome::Value(Value::int32(2024))) - } - - fn month_value_of( - _machine: &mut Machine<'_, TestHost>, - _this: Value, - _args: &[Value], - _constructing: bool, - ) -> Result { - MONTH_ORDER.store( - CALL_ORDER.fetch_add(1, Ordering::SeqCst) + 1, - Ordering::SeqCst, - ); - Ok(BuiltinOutcome::Value(Value::int32(0))) - } - - fn day_value_of( - _machine: &mut Machine<'_, TestHost>, - _this: Value, - _args: &[Value], - _constructing: bool, - ) -> Result { - DAY_ORDER.store( - CALL_ORDER.fetch_add(1, Ordering::SeqCst) + 1, - Ordering::SeqCst, - ); - Ok(BuiltinOutcome::Value(Value::int32(1))) - } - - fn to_string_date_string( - machine: &mut Machine<'_, TestHost>, - _this: Value, - _args: &[Value], - _constructing: bool, - ) -> Result { - TO_STRING_CALLED.store(true, Ordering::SeqCst); - Ok(BuiltinOutcome::Value(allocate_string( - machine, - EcmaString::encode("2024-01-01T00:00:00.000Z"), - )?)) - } - - #[test] - fn one_argument_copies_valid_and_invalid_date_time_values() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - - let valid = call_date(&mut machine, &[Value::int32(0)], true); - let valid_copy = call_date(&mut machine, &[valid], true); - assert_eq!(date_time(&machine, valid_copy).unwrap(), 0.0); - - let invalid = call_date(&mut machine, &[Value::number(f64::NAN)], true); - assert!(date_time(&machine, invalid).unwrap().is_nan()); - let invalid_copy = call_date(&mut machine, &[invalid], true); - assert!(date_time(&machine, invalid_copy).unwrap().is_nan()); - } - - #[test] - fn one_argument_observes_generic_object_value_of() { - VALUE_OF_CALLED.store(false, Ordering::SeqCst); - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - - let source = ordinary_object(&mut machine); - let value_of = native(&mut machine, "valueOf", generic_value_of); - machine - .set_data_property(source, "valueOf", value_of) - .expect("valueOf install succeeds"); - - let copy = call_date(&mut machine, &[source], true); - assert!( - VALUE_OF_CALLED.load(Ordering::SeqCst), - "valueOf must be called" - ); - assert_eq!(date_time(&machine, copy).unwrap(), 12_345.0); - } - - #[test] - fn one_argument_copies_date_time_without_calling_overridden_value_of() { - DATE_VALUE_OF_CALLED.store(false, Ordering::SeqCst); - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - - let source = call_date(&mut machine, &[Value::int32(0)], true); - let value_of = native(&mut machine, "valueOf", date_value_of_override); - machine - .set_data_property(source, "valueOf", value_of) - .expect("valueOf install succeeds"); - - let copy = call_date(&mut machine, &[source], true); - assert!( - !DATE_VALUE_OF_CALLED.load(Ordering::SeqCst), - "Date valueOf must not be called when copying" - ); - assert_eq!(date_time(&machine, copy).unwrap(), 0.0); - } - - #[test] - fn one_argument_parses_object_to_string_primitive() { - TO_STRING_CALLED.store(false, Ordering::SeqCst); - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - - let source = ordinary_object(&mut machine); - let to_string = native(&mut machine, "toString", to_string_date_string); - machine - .set_data_property(source, "toString", to_string) - .expect("toString install succeeds"); - - let value = call_date(&mut machine, &[source], true); - assert!( - TO_STRING_CALLED.load(Ordering::SeqCst), - "toString must be called" - ); - assert_eq!( - date_time(&machine, value).unwrap(), - 1_704_067_200_000.0, - "object toString must be parsed as an ISO date string" - ); - } - - #[test] - fn multi_argument_components_call_value_of_in_order() { - CALL_ORDER.store(0, Ordering::SeqCst); - YEAR_ORDER.store(0, Ordering::SeqCst); - MONTH_ORDER.store(0, Ordering::SeqCst); - DAY_ORDER.store(0, Ordering::SeqCst); - - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - - let year = ordinary_object(&mut machine); - let month = ordinary_object(&mut machine); - let day = ordinary_object(&mut machine); - - let year_fn = native(&mut machine, "yearValueOf", year_value_of); - let month_fn = native(&mut machine, "monthValueOf", month_value_of); - let day_fn = native(&mut machine, "dayValueOf", day_value_of); - - machine - .set_data_property(year, "valueOf", year_fn) - .expect("year valueOf install succeeds"); - machine - .set_data_property(month, "valueOf", month_fn) - .expect("month valueOf install succeeds"); - machine - .set_data_property(day, "valueOf", day_fn) - .expect("day valueOf install succeeds"); - - let value = call_date(&mut machine, &[year, month, day], true); - assert_eq!( - date_time(&machine, value).unwrap(), - 1_704_067_200_000.0, - "multi-argument components must coerce to 2024-01-01" - ); - assert_eq!( - YEAR_ORDER.load(Ordering::SeqCst), - 1, - "year must coerce first" - ); - assert_eq!( - MONTH_ORDER.load(Ordering::SeqCst), - 2, - "month must coerce second" - ); - assert_eq!(DAY_ORDER.load(Ordering::SeqCst), 3, "day must coerce third"); - } - - #[test] - fn date_prototype_is_cached_not_global_lookup() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - - let ctor_before = machine - .intrinsics - .global("Date") - .expect("Date global exists"); - let date_id = machine - .intrinsics - .builtins - .id_named("Date") - .expect("Date builtin id"); - - // Delete and overwrite the global name. Construction through the saved - // builtin id must still use the cached intrinsic prototype. - machine - .intrinsics - .globals - .remove(&EcmaString::encode("Date")); - machine - .intrinsics - .globals - .insert(EcmaString::encode("Date"), Value::int32(99)); - - let BuiltinOutcome::Value(instance) = machine - .call_builtin(date_id, Value::UNDEFINED, &[], true) - .expect("Date construct succeeds") - else { - panic!("Date construct returns a value"); - }; - - let index = machine - .runtime_slot(instance) - .expect("valid instance") - .expect("slot"); - let HeapEntry::Date { prototype, .. } = &machine.heap[index] else { - panic!("Date instance"); - }; - assert_eq!( - *prototype, - Some(machine.intrinsics.builtins.date_prototype()) - ); - - // The cached prototype's "constructor" is still the original Date constructor. - let proto_val = machine.intrinsics.builtins.date_prototype(); - let ctor = machine - .get_named_property(proto_val, "constructor") - .expect("prototype has constructor"); - assert_eq!(ctor, ctor_before); - } -} diff --git a/crates/bamts-runtime/src/builtins/date_full.rs b/crates/bamts-runtime/src/builtins/date_full.rs index fdfb1a64..091cda15 100644 --- a/crates/bamts-runtime/src/builtins/date_full.rs +++ b/crates/bamts-runtime/src/builtins/date_full.rs @@ -8,6 +8,7 @@ use super::{ allocate_string, define_data, install_constructor_function, install_function, range_error, type_error, value_number, }; +use crate::intl::date_time_format::{DefaultComponents, RequiredComponents}; use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; use crate::{EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, PropertyMap}; @@ -545,9 +546,9 @@ pub(super) fn install( ("toDateString", 0, to_date_string::), ("toTimeString", 0, to_time_string::), ("toUTCString", 0, to_utc_string::), - ("toLocaleString", 0, to_string::), - ("toLocaleDateString", 0, to_date_string::), - ("toLocaleTimeString", 0, to_time_string::), + ("toLocaleString", 0, to_locale_string::), + ("toLocaleDateString", 0, to_locale_date_string::), + ("toLocaleTimeString", 0, to_locale_time_string::), ]; let mut utc_string_function = None; for &(name, length, handler) in methods { @@ -563,9 +564,58 @@ pub(super) fn install( "toGMTString", utc_string_function.expect("toUTCString is installed"), ); + let to_primitive = install_function( + heap, + builtins, + "[Symbol.toPrimitive]", + 1, + date_to_primitive::, + ); + super::define_symbol_property( + heap, + prototype, + builtins.symbol_to_primitive(), + Property::Data { + value: to_primitive, + writable: false, + enumerable: false, + configurable: true, + }, + ); globals.insert(EcmaString::encode("Date"), constructor); } +/// `Date.prototype[@@toPrimitive]`: unlike every other object, a Date reads +/// the `default` hint as `string`. +fn date_to_primitive( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if !machine.is_object(this) { + return Err(type_error( + "Date.prototype[Symbol.toPrimitive] requires an object receiver", + )); + } + let hint = args + .first() + .and_then(|value| machine.string_value(*value)) + .ok_or_else(|| type_error("Invalid Date.prototype[Symbol.toPrimitive] hint"))?; + let prefer_string = if hint.eq_ascii("string") || hint.eq_ascii("default") { + true + } else if hint.eq_ascii("number") { + false + } else { + return Err(type_error( + "Invalid Date.prototype[Symbol.toPrimitive] hint", + )); + }; + Ok(BuiltinOutcome::Value( + machine.ordinary_to_primitive(this, prefer_string)?, + )) +} + fn named(name: &str) -> PropertyKey { PropertyKey::Named(EcmaString::encode(name)) } @@ -657,7 +707,7 @@ fn constructor( if let Some(time) = copied { time } else { - let primitive = machine.coerce_primitive_observable(value, false)?; + let primitive = machine.coerce_primitive_default(value)?; if let Some(text) = machine.string_value(primitive) { let text = text.to_utf8_lossy(); let zone = if date_text_uses_local_zone(&text) { @@ -674,11 +724,13 @@ fn constructor( let values = numbers(machine, &args[..args.len().min(7)])?; components_to_time(&values, &timezone(machine)?, true) }; + let default_prototype = machine.intrinsics.builtins.date_prototype(); + let prototype = super::prototype_from_new_target(machine, default_prototype)?; let object = machine .allocate(HeapEntry::Date { time: milliseconds, properties: PropertyMap::default(), - prototype: Some(machine.intrinsics.builtins.date_prototype()), + prototype: Some(prototype), extensible: true, }) .map_err(EvalFailure::Runtime)?; @@ -837,7 +889,7 @@ fn set_time_fields( supplied.push(f64::NAN); } if raw.is_nan() { - return write_date_time(machine, this, f64::NAN); + return Ok(BuiltinOutcome::Value(Value::number(f64::NAN))); } let zone = if local { Some(timezone(machine)?) @@ -875,7 +927,7 @@ fn set_calendar_fields( supplied.push(f64::NAN); } if raw.is_nan() && !invalid_zero { - return write_date_time(machine, this, f64::NAN); + return Ok(BuiltinOutcome::Value(Value::number(f64::NAN))); } let zone = if local { Some(timezone(machine)?) @@ -1001,6 +1053,62 @@ enum DateStringKind { Date, Time, } +/// The shared body of the three `toLocale*` methods. ECMA-402 gives each a +/// component requirement and defaults of its own. +fn locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + required: RequiredComponents, + defaults: DefaultComponents, +) -> Result { + let time = date_time(machine, this)?; + let locales = args.first().copied().unwrap_or(Value::UNDEFINED); + let options = args.get(1).copied().unwrap_or(Value::UNDEFINED); + super::intl::date_to_locale_string(machine, time, locales, options, required, defaults) +} +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + locale_string( + machine, + this, + args, + RequiredComponents::Any, + DefaultComponents::All, + ) +} +fn to_locale_date_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + locale_string( + machine, + this, + args, + RequiredComponents::Date, + DefaultComponents::Date, + ) +} +fn to_locale_time_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + locale_string( + machine, + this, + args, + RequiredComponents::Time, + DefaultComponents::Time, + ) +} fn to_string( machine: &mut Machine<'_, H>, this: Value, @@ -1081,13 +1189,19 @@ fn iso_string(time: f64) -> Option { ms_from_time(time) )) } +/// At least four digits, after a `-` for a negative year. +fn year_string(year: impl Into) -> String { + let year = year.into(); + let sign = if year < 0 { "-" } else { "" }; + format!("{sign}{:04}", year.unsigned_abs()) +} fn utc_string(time: f64) -> String { format!( "{}, {:02} {} {} {:02}:{:02}:{:02} GMT", WEEKDAYS[week_day(time) as usize], date_from_time(time), MONTHS[month_from_time(time) as usize], - year_from_time(time), + year_string(year_from_time(time)), hour_from_time(time), min_from_time(time), sec_from_time(time) @@ -1103,7 +1217,7 @@ fn date_string(time: f64, zone: &TimeZoneRule, kind: DateStringKind) -> String { WEEKDAYS[week_day(local) as usize], MONTHS[month_from_time(local) as usize], date_from_time(local), - year_from_time(local) + year_string(year_from_time(local)) ); let offset = zone.offset_at_utc(time); let sign = if offset < 0 { '-' } else { '+' }; @@ -1308,6 +1422,68 @@ fn parse_local_output(text: &str, zone: &TimeZoneRule) -> Option { Some(time_clip(local - f64::from(offset) * MS_PER_SECOND)) } +#[cfg(test)] +mod to_primitive_tests { + use super::super::test_support::{TestHost, blank_program, ordinary_object}; + use super::*; + use crate::{Limits, ThrowOrigin}; + + fn hint(machine: &mut Machine<'_, TestHost>, text: &str) -> Value { + machine + .allocate(HeapEntry::String(EcmaString::encode(text))) + .unwrap() + } + + #[test] + fn to_primitive_reads_each_hint_and_rejects_the_rest() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let date_constructor = machine.intrinsics.global("Date").unwrap(); + let prototype = machine + .get_named_property(date_constructor, "prototype") + .unwrap(); + let key = machine + .to_property_key(machine.intrinsics.builtins.symbol_to_primitive()) + .unwrap(); + let method = machine.get_property_key(prototype, &key).unwrap(); + + let object = ordinary_object(&mut machine); + let to_string = hint(&mut machine, "from toString"); + let to_string_fn = machine + .get_named_property(machine.intrinsics.object_prototype, "valueOf") + .unwrap(); + machine + .set_data_property(object, "toString", to_string_fn) + .unwrap(); + machine.set_data_property(object, "tag", to_string).unwrap(); + + for valid in ["string", "default", "number"] { + let argument = hint(&mut machine, valid); + assert!(matches!( + machine.call_value(method, object, &[argument]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + for invalid in [hint(&mut machine, "bogus"), Value::UNDEFINED] { + let failure = machine.call_value(method, object, &[invalid]); + assert!(matches!( + failure, + Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "Invalid Date.prototype[Symbol.toPrimitive] hint" + })) + )); + } + let number_hint = hint(&mut machine, "number"); + assert!(matches!( + machine.call_value(method, Value::int32(1), &[number_hint]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "Date.prototype[Symbol.toPrimitive] requires an object receiver" + })) + )); + } +} + #[cfg(test)] mod tests { use super::*; @@ -1792,6 +1968,24 @@ mod tests { assert_eq!(COERCION_STEP.load(Ordering::SeqCst), 2); } + #[test] + fn a_setter_on_an_invalid_date_leaves_the_stored_time_alone() { + let module = blank_program(""); + let mut host = ControlledHost { zone: "UTC" }; + let mut machine = machine(&module, &mut host); + let date = construct_date(&mut machine, &[Value::number(f64::NAN)]); + let result = call(&mut machine, date, "setUTCHours", &[Value::int32(1)]).unwrap(); + assert!(value_number(result).is_nan()); + assert!(value_number(call(&mut machine, date, "getTime", &[]).unwrap()).is_nan()); + } + + #[test] + fn years_print_with_four_digits_and_a_sign() { + assert_eq!(year_string(20), "0020"); + assert_eq!(year_string(-1), "-0001"); + assert_eq!(year_string(12345), "12345"); + } + static RECEIVER_ARGUMENT_COERCED: AtomicBool = AtomicBool::new(false); fn receiver_order_value_of( diff --git a/crates/bamts-runtime/src/builtins/error_edge.rs b/crates/bamts-runtime/src/builtins/error_edge.rs index 3ba0df2b..a4890d0b 100644 --- a/crates/bamts-runtime/src/builtins/error_edge.rs +++ b/crates/bamts-runtime/src/builtins/error_edge.rs @@ -5,7 +5,7 @@ use bamts_native::Value; use super::{ allocate_array, allocate_string, heap_index, install_constructor_function, install_function, - type_error, + type_error, with_roots, }; use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; use crate::{EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, PropertyMap}; @@ -33,11 +33,32 @@ pub(super) fn install( globals: &mut BTreeMap, builtins: &mut BuiltinTable, ) { + let brand = crate::intrinsics::push( + heap, + HeapEntry::PrivateName { + description: EcmaString::encode("[[ErrorData]]"), + }, + ); + builtins.set_error_brand(brand); let error_prototype = prototype_of(heap, globals, "Error"); let stack_getter = install_function(heap, builtins, "get stack", 0, stack_get::); let stack_setter = install_function(heap, builtins, "set stack", 1, stack_set::); define_hidden(heap, error_prototype, STACK_GETTER, stack_getter); define_hidden(heap, error_prototype, STACK_SETTER, stack_setter); + let getter = install_function(heap, builtins, "get stack", 0, prototype_stack_get::); + let setter = install_function(heap, builtins, "set stack", 1, prototype_stack_set::); + let HeapEntry::Object { properties, .. } = &mut heap[heap_index(error_prototype)] else { + panic!("Error.prototype is an ordinary object") + }; + properties.insert( + PropertyKey::Named(EcmaString::encode("stack")), + Property::Accessor { + getter: Some(getter), + setter: Some(setter), + enumerable: false, + configurable: true, + }, + ); for (name, length) in ERROR_TYPES { let prototype = prototype_of(heap, globals, name); @@ -54,6 +75,14 @@ pub(super) fn install( capture_stack_trace::, ); define_static(heap, constructor, "captureStackTrace", capture); + let is_error = install_function(heap, builtins, "isError", 1, is_error::); + define_static(heap, constructor, "isError", is_error); + } else { + let HeapEntry::NativeFunction { prototype, .. } = &mut heap[heap_index(constructor)] + else { + panic!("error constructors are native"); + }; + *prototype = Some(globals[&EcmaString::encode("Error")]); } globals.insert(EcmaString::encode(name), constructor); } @@ -127,6 +156,33 @@ fn define_own( ) } +fn has_error_brand(machine: &Machine<'_, H>, value: Value) -> Result { + let brand = machine.intrinsics.builtins.error_brand_key(); + Ok(machine + .runtime_slot(value) + .map_err(EvalFailure::Runtime)? + .is_some_and(|index| { + matches!( + &machine.heap[index], + HeapEntry::Object { properties, .. } if properties.contains_key(&brand) + ) + })) +} + +/// `Error.isError`: true only for objects an Error constructor made. The brand +/// is a private key, so a proxy for an error or an object that merely inherits +/// from `Error.prototype` does not have it. +fn is_error( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = args.first().copied().unwrap_or(Value::UNDEFINED); + let branded = has_error_brand(machine, value)?; + Ok(BuiltinOutcome::Value(Value::boolean(branded))) +} + pub(super) fn error_constructor( machine: &mut Machine<'_, H>, this: Value, @@ -162,58 +218,63 @@ pub(super) fn error_constructor( .map_err(EvalFailure::Runtime)? }; + let brand = machine.intrinsics.builtins.error_brand_key(); + machine.define_descriptor(object, brand, crate::error_brand())?; + let (message_index, options_index) = match name { "AggregateError" => (1, 2), "SuppressedError" => (2, 3), _ => (0, 1), }; - let message = args - .get(message_index) - .copied() - .filter(|value| *value != Value::UNDEFINED) - .map(|value| machine.to_string(value)) - .transpose()?; - if let Some(text) = &message { - let value = allocate_string(machine, text.clone())?; - define_own(machine, object, "message", value)?; - } - - if let Some(options) = args.get(options_index).copied() - && machine.is_object(options) - { - let key = PropertyKey::Named(EcmaString::encode("cause")); - if machine.internal_has_property(options, &key)? { - let cause = machine.get_named_property(options, "cause")?; - define_own(machine, object, "cause", cause)?; + with_roots(machine, &[object], |machine, _| { + let message = args + .get(message_index) + .copied() + .filter(|value| *value != Value::UNDEFINED) + .map(|value| machine.coerce_string_observable(value)) + .transpose()?; + if let Some(text) = &message { + let value = allocate_string(machine, text.clone())?; + define_own(machine, object, "message", value)?; } - } - match name { - "AggregateError" => { - let source = args.first().copied().unwrap_or(Value::UNDEFINED); - let values = machine.iterable_values(source)?; - let array = allocate_array(machine, values)?; - define_own(machine, object, "errors", array)?; + if let Some(options) = args.get(options_index).copied() + && machine.is_object(options) + { + let key = PropertyKey::Named(EcmaString::encode("cause")); + if machine.internal_has_property(options, &key)? { + let cause = machine.get_named_property(options, "cause")?; + define_own(machine, object, "cause", cause)?; + } } - "SuppressedError" => { - define_own( - machine, - object, - "error", - args.first().copied().unwrap_or(Value::UNDEFINED), - )?; - define_own( - machine, - object, - "suppressed", - args.get(1).copied().unwrap_or(Value::UNDEFINED), - )?; + + match name { + "AggregateError" => { + let source = args.first().copied().unwrap_or(Value::UNDEFINED); + let values = machine.iterable_values(source)?; + let array = allocate_array(machine, values)?; + define_own(machine, object, "errors", array)?; + } + "SuppressedError" => { + define_own( + machine, + object, + "error", + args.first().copied().unwrap_or(Value::UNDEFINED), + )?; + define_own( + machine, + object, + "suppressed", + args.get(1).copied().unwrap_or(Value::UNDEFINED), + )?; + } + _ => {} } - _ => {} - } - install_stack(machine, object, name, message.as_ref(), None)?; - Ok(BuiltinOutcome::Value(object)) + install_stack(machine, object, name, message.as_ref(), None)?; + Ok(BuiltinOutcome::Value(object)) + }) } fn install_stack( @@ -404,6 +465,56 @@ fn stack_set( Ok(BuiltinOutcome::Value(Value::UNDEFINED)) } +/// `get Error.prototype.stack`, reached by an error that has lost its own `stack`. +fn prototype_stack_get( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + constructing: bool, +) -> Result { + if !machine.is_object(this) { + return Err(type_error("Error.prototype.stack requires an object")); + } + if has_error_brand(machine, this)? { + return super::error_to_string(machine, this, &[], constructing); + } + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) +} + +/// `set Error.prototype.stack`, which never writes to the prototype itself. +fn prototype_stack_set( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if !machine.is_object(this) { + return Err(type_error("Error.prototype.stack requires an object")); + } + let value = args.first().copied().unwrap_or(Value::UNDEFINED); + if machine.string_value(value).is_none() { + return Err(type_error("Error.prototype.stack requires a string")); + } + if this == prototype_of_error(machine)? { + return Err(type_error("Cannot set stack on Error.prototype")); + } + let key = PropertyKey::Named(EcmaString::encode("stack")); + if machine.internal_get_own_property(this, &key)?.is_some() { + machine.set_data_property_key(this, key, value)?; + } else { + machine.create_data_property_key(this, key, value)?; + } + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) +} + +fn prototype_of_error(machine: &mut Machine<'_, H>) -> Result { + let error = machine + .intrinsics + .global("Error") + .ok_or_else(|| type_error("Error constructor is not installed"))?; + machine.get_named_property(error, "prototype") +} + fn capture_stack_trace( machine: &mut Machine<'_, H>, _this: Value, @@ -431,6 +542,75 @@ fn capture_stack_trace( Ok(BuiltinOutcome::Value(Value::UNDEFINED)) } +#[cfg(test)] +mod is_error_tests { + use super::super::test_support::{TestHost, blank_program, ordinary_object}; + use super::*; + use crate::{Limits, ThrowOrigin}; + + fn is_error_of(machine: &mut Machine<'_, TestHost>, value: Value) -> Value { + let error = machine.intrinsics.global("Error").unwrap(); + let method = machine.get_named_property(error, "isError").unwrap(); + machine.call_value(method, error, &[value]).unwrap() + } + + #[test] + fn the_brand_decides_rather_than_the_prototype_chain() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let constructor = machine.intrinsics.global("RangeError").unwrap(); + let real = machine.construct_value(constructor, &[]).unwrap(); + assert_eq!(is_error_of(&mut machine, real), Value::TRUE); + + let engine = machine + .materialize_engine_origin(ThrowOrigin::TypeError { operation: "test" }) + .unwrap(); + assert_eq!( + is_error_of(&mut machine, engine), + Value::TRUE, + "errors the engine throws carry the brand too" + ); + + let prototype = machine + .get_named_property(constructor, "prototype") + .unwrap(); + assert_eq!(is_error_of(&mut machine, prototype), Value::FALSE); + let fake = machine + .allocate(HeapEntry::Object { + properties: PropertyMap::default(), + prototype: Some(prototype), + extensible: true, + boxed_primitive: None, + }) + .unwrap(); + assert_eq!( + is_error_of(&mut machine, fake), + Value::FALSE, + "inheriting from an error prototype does not make an error" + ); + let handler = ordinary_object(&mut machine); + let proxy = super::super::proxy::create(&mut machine, real, handler).unwrap(); + assert_eq!(is_error_of(&mut machine, proxy), Value::FALSE); + assert_eq!(is_error_of(&mut machine, Value::int32(1)), Value::FALSE); + } + + #[test] + fn the_brand_is_invisible_to_reflection() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let constructor = machine.intrinsics.global("Error").unwrap(); + let error = machine.construct_value(constructor, &[]).unwrap(); + let keys = machine.internal_own_property_keys(error).unwrap(); + assert!( + !keys + .iter() + .any(|key| matches!(key, PropertyKey::Private(_))) + ); + } +} + #[cfg(test)] mod tests { use super::super::test_support::{TestHost, blank_program, custom_iterable, ordinary_object}; @@ -544,6 +724,48 @@ mod tests { native_function(&mut machine.heap, id, name, 0) } + #[test] + fn error_prototype_stack_setter_ignores_the_prototype_itself() { + machine!(machine); + let error = machine.intrinsics.global("Error").unwrap(); + let prototype = machine.get_named_property(error, "prototype").unwrap(); + let key = PropertyKey::Named(EcmaString::encode("stack")); + let Some(Property::Accessor { + setter: Some(setter), + .. + }) = machine.own_descriptor(prototype, &key).unwrap() + else { + panic!("Error.prototype.stack is an accessor"); + }; + let text = allocate_string(&mut machine, EcmaString::encode("trace")).unwrap(); + assert!(machine.call_value(setter, prototype, &[text]).is_err()); + let plain = ordinary_object(&mut machine); + assert!( + machine + .call_value(setter, plain, &[Value::int32(1)]) + .is_err() + ); + machine.call_value(setter, plain, &[text]).unwrap(); + assert_eq!(machine.get_named_property(plain, "stack").unwrap(), text); + } + + #[test] + fn error_to_string_defaults_missing_fields_and_requires_an_object() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let error = machine.intrinsics.global("Error").unwrap(); + let prototype = machine.get_named_property(error, "prototype").unwrap(); + let to_string = machine.get_named_property(prototype, "toString").unwrap(); + assert!(matches!( + machine.call_value(to_string, Value::UNDEFINED, &[]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + let plain = ordinary_object(&mut machine); + let text = machine.call_value(to_string, plain, &[]).unwrap(); + assert!(machine.string_value(text).unwrap().eq_ascii("Error")); + } + #[test] fn cause_distinguishes_absence_from_present_undefined() { machine!(machine); diff --git a/crates/bamts-runtime/src/builtins/function_constructor.rs b/crates/bamts-runtime/src/builtins/function_constructor.rs index 625fde59..2b09472a 100644 --- a/crates/bamts-runtime/src/builtins/function_constructor.rs +++ b/crates/bamts-runtime/src/builtins/function_constructor.rs @@ -14,14 +14,22 @@ use std::collections::BTreeMap; use bamts_bytecode::EcmaString; use bamts_native::Value; -use super::{define_data, define_frozen_data, install_constructor_function}; -use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; -use crate::{EvalFailure, HeapEntry, Host, Machine, PropertyMap, ScriptSource, ThrowOrigin}; +use super::{ + define_data, define_frozen_data, define_to_string_tag, heap_index, install_constructor_function, +}; +use crate::intrinsics::{ + BuiltinDef, BuiltinHandler, BuiltinOutcome, BuiltinTable, native_function, +}; +use crate::{ + EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, PropertyMap, ScriptSource, + ThrowOrigin, +}; pub(crate) fn install( heap: &mut Vec, globals: &mut BTreeMap, builtins: &mut BuiltinTable, + script_compiler: bool, ) { let prototype = builtins.function_prototype(); let constructor = install_constructor_function( @@ -34,51 +42,143 @@ pub(crate) fn install( define_frozen_data(heap, constructor, "prototype", prototype); define_data(heap, prototype, "constructor", constructor); globals.insert(EcmaString::encode("Function"), constructor); + let generator_prototype = builtins.generator_prototype(); + let async_generator_prototype = builtins.async_generator_prototype(); + let tag = builtins.symbol_to_string_tag(); + define_to_string_tag(heap, generator_prototype, tag, "Generator"); + define_to_string_tag(heap, async_generator_prototype, tag, "AsyncGenerator"); + let generator = install_function_kind( + heap, + builtins, + "GeneratorFunction", + generator_function_constructor::, + constructor, + Some(generator_prototype), + ); + let asynchronous = install_function_kind( + heap, + builtins, + "AsyncFunction", + async_function_constructor::, + constructor, + None, + ); + let async_generator = install_function_kind( + heap, + builtins, + "AsyncGeneratorFunction", + async_generator_function_constructor::, + constructor, + Some(async_generator_prototype), + ); + builtins.set_function_kind_prototypes(generator, asynchronous, async_generator); + if script_compiler { + let id = builtins.register(BuiltinDef { + name: "eval", + length: 1, + handler: eval::, + }); + let function = native_function(heap, id, "eval", 1); + globals.insert(EcmaString::encode("eval"), function); + } } -/// `Function(p1, ..., pn, body)` / `new Function(p1, ..., pn, body)`. -/// -/// The arguments synthesize into a `function anonymous(...) { ... }` -/// expression compiled as a classic script; classic scripts return their -/// completion value, so the entry's result is the new function itself. -fn function_constructor( +/// A read-only link between function-kind intrinsics. +fn define_link(heap: &mut [HeapEntry], object: Value, name: &str, value: Value) { + let HeapEntry::Object { properties, .. } = &mut heap[heap_index(object)] else { + panic!("function kind intrinsics are ordinary objects"); + }; + properties.insert( + PropertyKey::Named(EcmaString::encode(name)), + Property::Data { + value, + writable: false, + enumerable: false, + configurable: true, + }, + ); +} + +/// Installs one of the hidden kinds of function constructor and returns the +/// prototype its closures inherit from. Neither is reachable from a global: +/// they are found through `Object.getPrototypeOf(function* () {})`. +fn install_function_kind( + heap: &mut Vec, + builtins: &mut BuiltinTable, + name: &'static str, + handler: BuiltinHandler, + function_constructor: Value, + instance_prototype: Option, +) -> Value { + let prototype = crate::intrinsics::push( + heap, + HeapEntry::Object { + properties: PropertyMap::default(), + prototype: Some(builtins.function_prototype()), + extensible: true, + boxed_primitive: None, + }, + ); + let constructor = install_constructor_function(heap, builtins, name, 1, handler); + let HeapEntry::NativeFunction { + prototype: parent, .. + } = &mut heap[heap_index(constructor)] + else { + panic!("function kind constructor is a native function"); + }; + *parent = Some(function_constructor); + define_frozen_data(heap, constructor, "prototype", prototype); + define_link(heap, prototype, "constructor", constructor); + define_to_string_tag(heap, prototype, builtins.symbol_to_string_tag(), name); + if let Some(instance_prototype) = instance_prototype { + define_link(heap, prototype, "prototype", instance_prototype); + define_link(heap, instance_prototype, "constructor", prototype); + } + prototype +} + +/// `eval(x)` as an indirect call: the source runs in the global scope, as a +/// classic script, and yields its completion value. Anything but a string comes +/// back unchanged. A host without a script compiler has no `eval` at all, so +/// the name reads as undeclared there rather than failing when called. +fn eval( machine: &mut Machine<'_, H>, _this: Value, args: &[Value], - _constructing: bool, + constructing: bool, ) -> Result { - let (parameter_source, body_source) = match args.split_last() { - None => (String::new(), String::new()), - Some((body, parameters)) => { - let mut names = Vec::with_capacity(parameters.len()); - for parameter in parameters { - let text = machine.to_string(*parameter)?; - names.push(String::from_utf16_lossy(text.as_units())); - } - let body_text = machine.to_string(*body)?; - ( - names.join(","), - String::from_utf16_lossy(body_text.as_units()), - ) - } - }; - // Line offsets inside the body must match the spec's synthesis, so the - // body starts on its own source line. - let mut source = String::from("(function anonymous("); - source.push_str(¶meter_source); - source.push_str("\n)\n{\n"); - source.push_str(&body_source); - source.push_str("\n})"); - let script = EcmaString::encode(&source); - let script_name = EcmaString::encode("Function constructor"); + if constructing { + return Err(type_error("eval is not a constructor")); + } + let source = args.first().copied().unwrap_or(Value::UNDEFINED); + match machine.string_value(source) { + Some(text) => run_script( + machine, + &text, + "eval", + "eval: this host provides no script compiler", + ), + None => Ok(BuiltinOutcome::Value(source)), + } +} + +/// Compiles `source` as a classic script, runs it against the global object, +/// and returns its completion value. +fn run_script( + machine: &mut Machine<'_, H>, + source: &EcmaString, + name: &str, + no_compiler: &'static str, +) -> Result { + let name = EcmaString::encode(name); let compiled = { let provider = machine .host .script_compiler() - .ok_or_else(|| type_error("Function: this host provides no script compiler"))?; + .ok_or_else(|| type_error(no_compiler))?; provider.compile_script(ScriptSource { - source: script.as_units(), - name: script_name.as_units(), + source: source.as_units(), + name: name.as_units(), }) } .map_err(|error| crate::vm::compile_error(machine, error))?; @@ -102,6 +202,77 @@ fn function_constructor( result.map(BuiltinOutcome::Value) } +/// `Function(p1, ..., pn, body)` / `new Function(p1, ..., pn, body)`. +fn function_constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + create_dynamic_function(machine, args, "function") +} + +fn generator_function_constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + create_dynamic_function(machine, args, "function*") +} + +fn async_function_constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + create_dynamic_function(machine, args, "async function") +} + +fn async_generator_function_constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + create_dynamic_function(machine, args, "async function*") +} + +/// The arguments synthesize into an `anonymous` function expression compiled +/// as a classic script; classic scripts return their completion value, so the +/// entry's result is the new function itself. +fn create_dynamic_function( + machine: &mut Machine<'_, H>, + args: &[Value], + keyword: &str, +) -> Result { + let (parameter_source, body_source) = match args.split_last() { + None => (String::new(), String::new()), + Some((body, parameters)) => { + let mut names = Vec::with_capacity(parameters.len()); + for parameter in parameters { + let text = machine.coerce_string_observable(*parameter)?; + names.push(String::from_utf16_lossy(text.as_units())); + } + let body_text = machine.coerce_string_observable(*body)?; + ( + names.join(","), + String::from_utf16_lossy(body_text.as_units()), + ) + } + }; + // Line offsets inside the body must match the spec's synthesis, so the + // body starts on its own source line. + let source = format!("({keyword} anonymous({parameter_source}\n)\n{{\n{body_source}\n}})"); + run_script( + machine, + &EcmaString::encode(&source), + "Function constructor", + "Function: this host provides no script compiler", + ) +} + fn type_error(operation: &'static str) -> EvalFailure { EvalFailure::Throw(ThrowOrigin::TypeError { operation }) } diff --git a/crates/bamts-runtime/src/builtins/intl/case_mapping.rs b/crates/bamts-runtime/src/builtins/intl/case_mapping.rs new file mode 100644 index 00000000..b4cfd0b7 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/case_mapping.rs @@ -0,0 +1,251 @@ +//! TransformCase: `String.prototype.toLocaleUpperCase` and `toLocaleLowerCase`. +//! +//! The Unicode Character Database itself carries the language-sensitive case +//! mappings, in SpecialCasing.txt, for exactly three languages, so these need +//! no locale data. Everything else is the default case conversion. + +use bamts_bytecode::{EcmaString, EcmaStringBuilder}; +use bamts_native::Value; +use icu_properties::CodePointSetData; +use icu_properties::props::{CaseIgnorable, Cased, SoftDotted}; +use unicode_normalization::char::canonical_combining_class; + +use super::super::allocate_string; +use super::options::canonicalize_locale_list; +use crate::intrinsics::BuiltinOutcome; +use crate::{EvalFailure, Host, Machine}; + +const CAPITAL_I: u32 = 0x49; +const CAPITAL_J: u32 = 0x4A; +const SMALL_I: u32 = 0x69; +const SMALL_J: u32 = 0x6A; +const DOTLESS_SMALL_I: u32 = 0x131; +const DOTTED_CAPITAL_I: u32 = 0x130; +const CAPITAL_I_OGONEK: u32 = 0x12E; +const SMALL_I_OGONEK: u32 = 0x12F; +const CAPITAL_I_GRAVE: u32 = 0xCC; +const CAPITAL_I_ACUTE: u32 = 0xCD; +const CAPITAL_I_TILDE: u32 = 0x128; +const COMBINING_DOT_ABOVE: u32 = 0x307; +const COMBINING_GRAVE: u32 = 0x300; +const COMBINING_ACUTE: u32 = 0x301; +const COMBINING_TILDE: u32 = 0x303; +const CAPITAL_SIGMA: u32 = 0x3A3; +const FINAL_SIGMA: u32 = 0x3C2; +/// The canonical combining class of marks written above the base. +const CLASS_ABOVE: u8 = 230; + +/// Which language-sensitive mappings of SpecialCasing.txt apply. +#[derive(Clone, Copy, Eq, PartialEq)] +enum Tailoring { + Default, + Turkic, + Lithuanian, +} + +/// The case to convert to. +#[derive(Clone, Copy, Eq, PartialEq)] +pub(in crate::intrinsics::builtins) enum TargetCase { + Upper, + Lower, +} + +fn tailoring_of(locale: Option<&String>) -> Tailoring { + let language = locale.map_or("", |tag| tag.split('-').next().unwrap_or("")); + match language { + "tr" | "az" => Tailoring::Turkic, + "lt" => Tailoring::Lithuanian, + _ => Tailoring::Default, + } +} + +fn combining_class(code_point: u32) -> u8 { + char::from_u32(code_point).map_or(0, canonical_combining_class) +} + +/// The nearest code point among `candidates` that a combining sequence's +/// conditions stop at: a base, or a mark above. +fn anchor(points: &[u32], mut candidates: impl Iterator) -> Option { + candidates.find_map(|index| { + let point = points[index]; + matches!(combining_class(point), 0 | CLASS_ABOVE).then_some(point) + }) +} + +fn preceding_anchor(points: &[u32], index: usize) -> Option { + anchor(points, (0..index).rev()) +} + +fn following_anchor(points: &[u32], index: usize) -> Option { + anchor(points, index + 1..points.len()) +} + +fn after_capital_i(points: &[u32], index: usize) -> bool { + preceding_anchor(points, index) == Some(CAPITAL_I) +} + +fn before_dot(points: &[u32], index: usize) -> bool { + following_anchor(points, index) == Some(COMBINING_DOT_ABOVE) +} + +fn more_above(points: &[u32], index: usize) -> bool { + following_anchor(points, index).is_some_and(|point| combining_class(point) == CLASS_ABOVE) +} + +fn after_soft_dotted(points: &[u32], index: usize) -> bool { + preceding_anchor(points, index) + .is_some_and(|point| CodePointSetData::new::().contains32(point)) +} + +/// Final_Sigma: a cased letter comes before, skipping case-ignorable code +/// points, and none comes after. +fn is_final_sigma(points: &[u32], index: usize) -> bool { + let cased = CodePointSetData::new::(); + let ignorable = CodePointSetData::new::(); + let significant = |point: &&u32| !ignorable.contains32(**point); + let before = points[..index].iter().rev().find(significant); + let after = points[index + 1..].iter().find(significant); + before.is_some_and(|point| cased.contains32(*point)) + && !after.is_some_and(|point| cased.contains32(*point)) +} + +/// The tailored lower case of the code point at `index`, if it has one. +fn tailored_lower(tailoring: Tailoring, points: &[u32], index: usize) -> Option<&'static [u32]> { + let point = points[index]; + match tailoring { + Tailoring::Default => None, + Tailoring::Turkic => match point { + DOTTED_CAPITAL_I => Some(&[SMALL_I]), + COMBINING_DOT_ABOVE if after_capital_i(points, index) => Some(&[]), + CAPITAL_I if !before_dot(points, index) => Some(&[DOTLESS_SMALL_I]), + _ => None, + }, + Tailoring::Lithuanian => match point { + CAPITAL_I if more_above(points, index) => Some(&[SMALL_I, COMBINING_DOT_ABOVE]), + CAPITAL_J if more_above(points, index) => Some(&[SMALL_J, COMBINING_DOT_ABOVE]), + CAPITAL_I_OGONEK if more_above(points, index) => { + Some(&[SMALL_I_OGONEK, COMBINING_DOT_ABOVE]) + } + CAPITAL_I_GRAVE => Some(&[SMALL_I, COMBINING_DOT_ABOVE, COMBINING_GRAVE]), + CAPITAL_I_ACUTE => Some(&[SMALL_I, COMBINING_DOT_ABOVE, COMBINING_ACUTE]), + CAPITAL_I_TILDE => Some(&[SMALL_I, COMBINING_DOT_ABOVE, COMBINING_TILDE]), + _ => None, + }, + } +} + +/// The tailored upper case of the code point at `index`, if it has one. +fn tailored_upper(tailoring: Tailoring, points: &[u32], index: usize) -> Option<&'static [u32]> { + match (tailoring, points[index]) { + (Tailoring::Turkic, SMALL_I) => Some(&[DOTTED_CAPITAL_I]), + (Tailoring::Lithuanian, COMBINING_DOT_ABOVE) if after_soft_dotted(points, index) => { + Some(&[]) + } + _ => None, + } +} + +fn push_point(output: &mut EcmaStringBuilder, point: u32) { + output + .push_code_point(point) + .expect("case mappings produce valid code points"); +} + +fn convert(source: &EcmaString, tailoring: Tailoring, target: TargetCase) -> EcmaString { + let points: Vec = source.code_points().map(|(_, point)| point).collect(); + let mut output = EcmaStringBuilder::new(); + for (index, &point) in points.iter().enumerate() { + let tailored = match target { + TargetCase::Upper => tailored_upper(tailoring, &points, index), + TargetCase::Lower => tailored_lower(tailoring, &points, index), + }; + if let Some(mapped) = tailored { + for &code_point in mapped { + push_point(&mut output, code_point); + } + continue; + } + let Some(character) = char::from_u32(point) else { + output.push_unit(point as u16); + continue; + }; + match target { + TargetCase::Upper => { + for mapped in character.to_uppercase() { + push_point(&mut output, mapped as u32); + } + } + TargetCase::Lower if point == CAPITAL_SIGMA && is_final_sigma(&points, index) => { + push_point(&mut output, FINAL_SIGMA); + } + TargetCase::Lower => { + for mapped in character.to_lowercase() { + push_point(&mut output, mapped as u32); + } + } + } + } + output.finish() +} + +/// TransformCase, after the receiver has become a string. +pub(in crate::intrinsics::builtins) fn transform_case( + machine: &mut Machine<'_, H>, + source: &EcmaString, + locales: Value, + target: TargetCase, +) -> Result { + let requested = canonicalize_locale_list(machine, locales)?; + let converted = convert(source, tailoring_of(requested.first()), target); + allocate_string(machine, converted).map(BuiltinOutcome::Value) +} + +#[cfg(test)] +mod tests { + use super::*; + + fn cased(text: &str, tailoring: Tailoring, target: TargetCase) -> String { + convert(&EcmaString::encode(text), tailoring, target).to_utf8_lossy() + } + + #[test] + fn languages_choose_their_tailoring_by_primary_subtag() { + let tailoring = |tag: &str| tailoring_of(Some(&tag.to_owned())); + assert!(tailoring("tr-TR") == Tailoring::Turkic); + assert!(tailoring("az-Cyrl") == Tailoring::Turkic); + assert!(tailoring("lt") == Tailoring::Lithuanian); + assert!(tailoring("trv") == Tailoring::Default); + assert!(tailoring_of(None) == Tailoring::Default); + } + + #[test] + fn turkic_dots_follow_the_letter() { + let lower = |text| cased(text, Tailoring::Turkic, TargetCase::Lower); + assert_eq!(lower("I"), "\u{131}"); + assert_eq!(lower("I\u{307}"), "i"); + assert_eq!(lower("\u{130}"), "i"); + assert_eq!(cased("i", Tailoring::Turkic, TargetCase::Upper), "\u{130}"); + assert_eq!(cased("i", Tailoring::Default, TargetCase::Upper), "I"); + } + + #[test] + fn lithuanian_keeps_the_dot_under_accents() { + let lower = |text| cased(text, Tailoring::Lithuanian, TargetCase::Lower); + assert_eq!(lower("I"), "i"); + assert_eq!(lower("I\u{300}"), "i\u{307}\u{300}"); + assert_eq!(lower("\u{CC}"), "i\u{307}\u{300}"); + assert_eq!( + cased("i\u{307}", Tailoring::Lithuanian, TargetCase::Upper), + "I" + ); + } + + #[test] + fn sigma_is_final_only_at_the_end_of_a_word() { + let lower = |text| cased(text, Tailoring::Default, TargetCase::Lower); + assert_eq!(lower("A\u{3A3}"), "a\u{3C2}"); + assert_eq!(lower("A\u{3A3}B"), "a\u{3C3}b"); + assert_eq!(lower("\u{3A3}"), "\u{3C3}"); + assert_eq!(lower("A.\u{3A3}"), "a.\u{3C2}"); + } +} diff --git a/crates/bamts-runtime/src/builtins/intl/collator.rs b/crates/bamts-runtime/src/builtins/intl/collator.rs new file mode 100644 index 00000000..64fdfaaf --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/collator.rs @@ -0,0 +1,213 @@ +//! `Intl.Collator`. + +use bamts_native::Value; + +use super::options::{Options, canonicalize_locale_list, coerce_options_to_object, read}; +use super::{ + Class, allocate, arg, bound_function, define_field, install_class, instance_prototype, + intl_record, new_object, register_hidden, string_value, supported_locales_outcome, type_error, +}; +use crate::intl::collator::{ + CoercedCollatorOption, Collator, CollatorError, CollatorErrorKind, CollatorOptionType, + CollatorOptionsSource, +}; +use crate::intl::locale_data::DATA; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +const COMPARE: &str = "Intl.Collator compare"; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + register_hidden(builtins, COMPARE, 2, compare_strings::); + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::Collator, + name: "Collator", + tag: "Intl.Collator", + length: 0, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[("compare", "get compare", compare_getter::)], + methods: &[("resolvedOptions", 0, resolved_options::)], + }, + ); +} + +/// The library reads options itself, in specification order, through this +/// adapter. A JavaScript failure cannot cross its error type, so it waits +/// here for the constructor to rethrow. +struct LiveOptions<'a, 'm, H: Host> { + machine: &'a mut Machine<'m, H>, + options: Options, + failure: Option, +} + +impl LiveOptions<'_, '_, H> { + fn fail(&mut self, failure: EvalFailure) -> CollatorError { + self.failure = Some(failure); + CollatorError::type_error("option read failed") + } +} + +impl CollatorOptionsSource for LiveOptions<'_, '_, H> { + fn coerce_options_to_object(&mut self) -> Result<(), CollatorError> { + Ok(()) + } + + fn get_option( + &mut self, + property: &'static str, + option_type: CollatorOptionType, + ) -> Result, CollatorError> { + let value = match read(self.machine, self.options, property) { + Ok(value) => value, + Err(failure) => return Err(self.fail(failure)), + }; + if value == Value::UNDEFINED { + return Ok(None); + } + match option_type { + CollatorOptionType::Boolean => Ok(Some(CoercedCollatorOption::Boolean( + self.machine.to_boolean(value), + ))), + CollatorOptionType::String => match self.machine.coerce_string_observable(value) { + Ok(text) => Ok(Some(CoercedCollatorOption::String(text.to_utf8_lossy()))), + Err(failure) => Err(self.fail(failure)), + }, + } + } +} + +fn collator_failure(machine: &mut Machine<'_, H>, error: CollatorError) -> EvalFailure { + match error.kind { + CollatorErrorKind::TypeError => type_error(machine, error.message), + CollatorErrorKind::RangeError => super::range_error(machine, error.message), + } +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + let prototype = instance_prototype(machine, IntlKind::Collator, constructing)?; + let collator = create_collator(machine, arg(args, 0), arg(args, 1))?; + allocate(machine, IntlObject::Collator(collator), prototype).map(BuiltinOutcome::Value) +} + +/// InitializeCollator without the object it initializes. +pub(super) fn create_collator( + machine: &mut Machine<'_, H>, + locales: Value, + options: Value, +) -> Result { + let requested = canonicalize_locale_list(machine, locales)?; + let options = coerce_options_to_object(machine, options)?; + let mut live = LiveOptions { + machine, + options, + failure: None, + }; + let source: Option<&mut dyn CollatorOptionsSource> = if options.is_some() { + Some(&mut live) + } else { + None + }; + let result = Collator::new(&requested, source, &*DATA); + let LiveOptions { + machine, failure, .. + } = live; + match (result, failure) { + (_, Some(failure)) => Err(failure), + (Ok(collator), None) => Ok(collator), + (Err(error), None) => Err(collator_failure(machine, error)), + } +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn compare_getter( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + if !matches!(intl_record(machine, this)?, Some(IntlObject::Collator(_))) { + return Err(type_error(machine, "receiver is not an Intl.Collator")); + } + bound_function(machine, this, COMPARE, 2) +} + +/// The function `compare` hands out, bound to its collator. +fn compare_strings( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let left = machine.coerce_string_observable(arg(args, 0))?; + let right = machine.coerce_string_observable(arg(args, 1))?; + let Some(IntlObject::Collator(collator)) = intl_record(machine, this)? else { + return Err(type_error(machine, "receiver is not an Intl.Collator")); + }; + let result = collator.compare(&*DATA, left.as_units(), right.as_units()); + Ok(BuiltinOutcome::Value(crate::number_value(f64::from( + result, + )))) +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let Some(IntlObject::Collator(collator)) = intl_record(machine, this)? else { + return Err(type_error(machine, "receiver is not an Intl.Collator")); + }; + let resolved = collator.resolved_options(); + let locale = resolved.locale.to_owned(); + let usage = resolved.usage.as_str(); + let sensitivity = resolved.sensitivity.as_str(); + let ignore_punctuation = resolved.ignore_punctuation; + let collation = resolved.collation.to_owned(); + let numeric = resolved.numeric; + let case_first = resolved.case_first.as_str(); + let result = new_object(machine)?; + for (name, text) in [ + ("locale", locale.as_str()), + ("usage", usage), + ("sensitivity", sensitivity), + ] { + let value = string_value(machine, text)?; + define_field(machine, result, name, value)?; + } + define_field( + machine, + result, + "ignorePunctuation", + Value::boolean(ignore_punctuation), + )?; + let value = string_value(machine, &collation)?; + define_field(machine, result, "collation", value)?; + define_field(machine, result, "numeric", Value::boolean(numeric))?; + let value = string_value(machine, case_first)?; + define_field(machine, result, "caseFirst", value)?; + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/date_time_format.rs b/crates/bamts-runtime/src/builtins/intl/date_time_format.rs new file mode 100644 index 00000000..fe53bc5e --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/date_time_format.rs @@ -0,0 +1,657 @@ +//! `Intl.DateTimeFormat`. + +use bamts_native::{Decoded, Value}; + +use super::super::temporal::temporal_record; +use super::options::{ + Options, boolean_option, canonicalize_locale_list, coerce_options_to_object, + locale_matcher_option, number_option, read, string_option, +}; +use super::{ + Class, allocate, arg, bound_function, chain_legacy_instance, define_field, install_class, + instance_prototype, intl_record, is_type_sequence, new_object, part_object, range_error, + register_hidden, string_outcome, string_value, supported_locales_outcome, type_error, + unwrap_legacy_instance, +}; +use crate::intl::date_time_format::{ + DateTimeComponents, DateTimeFormat, DateTimeFormatError, DateTimeFormatErrorKind, + DateTimeFormatOptions, DateTimePart, DateTimeRangePart, DateTimeStyle, DefaultComponents, + FormatMatcher, HourCycle, MonthStyle, NumericStyle, RangeSource, RequiredComponents, + TemporalFormatKind, TextWidth, TimeZoneNameStyle, is_valid_time_zone, +}; +use crate::intl::locale_data::DATA; +use crate::intl::number_data::NoHostLocale; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::TemporalObject; +use crate::temporal::instant_duration::epoch_days_from_ymd; +use crate::temporal::plain_types::{PlainDate, PlainTime}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +const FORMAT: &str = "Intl.DateTimeFormat format"; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + register_hidden(builtins, FORMAT, 1, format_value::); + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::DateTimeFormat, + name: "DateTimeFormat", + tag: "Intl.DateTimeFormat", + length: 0, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[("format", "get format", format_getter::)], + methods: &[ + ("formatRange", 2, format_range::), + ("formatRangeToParts", 2, format_range_to_parts::), + ("formatToParts", 1, format_to_parts::), + ("resolvedOptions", 0, resolved_options::), + ], + }, + ); +} + +pub(super) fn date_failure( + machine: &mut Machine<'_, H>, + error: &DateTimeFormatError, +) -> EvalFailure { + match error.kind() { + DateTimeFormatErrorKind::TypeError => type_error(machine, error.to_string()), + _ => range_error(machine, error.to_string()), + } +} + +const WIDTHS: &[&str] = &["narrow", "short", "long"]; +const NUMERIC: &[&str] = &["2-digit", "numeric"]; + +fn width(value: Option) -> Option { + value.map(|value| match value.as_str() { + "narrow" => TextWidth::Narrow, + "short" => TextWidth::Short, + _ => TextWidth::Long, + }) +} + +fn numeric(value: Option) -> Option { + value.map(|value| { + if value == "2-digit" { + NumericStyle::TwoDigit + } else { + NumericStyle::Numeric + } + }) +} + +fn date_style(value: Option) -> Option { + value.map(|value| match value.as_str() { + "full" => DateTimeStyle::Full, + "long" => DateTimeStyle::Long, + "medium" => DateTimeStyle::Medium, + _ => DateTimeStyle::Short, + }) +} + +/// The date and time component options, in table order. +fn component_options( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let weekday = width(string_option(machine, options, "weekday", WIDTHS)?); + let era = width(string_option(machine, options, "era", WIDTHS)?); + let year = numeric(string_option(machine, options, "year", NUMERIC)?); + let month = string_option( + machine, + options, + "month", + &["2-digit", "numeric", "narrow", "short", "long"], + )? + .map(|value| match value.as_str() { + "2-digit" => MonthStyle::TwoDigit, + "numeric" => MonthStyle::Numeric, + "narrow" => MonthStyle::Narrow, + "short" => MonthStyle::Short, + _ => MonthStyle::Long, + }); + let day = numeric(string_option(machine, options, "day", NUMERIC)?); + let day_period = width(string_option(machine, options, "dayPeriod", WIDTHS)?); + let hour = numeric(string_option(machine, options, "hour", NUMERIC)?); + let minute = numeric(string_option(machine, options, "minute", NUMERIC)?); + let second = numeric(string_option(machine, options, "second", NUMERIC)?); + let fractional_second_digits = + number_option(machine, options, "fractionalSecondDigits", 1.0, 3.0)? + .map(|digits| digits as u8); + let time_zone_name = string_option( + machine, + options, + "timeZoneName", + &[ + "short", + "long", + "shortOffset", + "longOffset", + "shortGeneric", + "longGeneric", + ], + )? + .map(|value| match value.as_str() { + "short" => TimeZoneNameStyle::Short, + "long" => TimeZoneNameStyle::Long, + "shortOffset" => TimeZoneNameStyle::ShortOffset, + "longOffset" => TimeZoneNameStyle::LongOffset, + "shortGeneric" => TimeZoneNameStyle::ShortGeneric, + _ => TimeZoneNameStyle::LongGeneric, + }); + Ok(DateTimeComponents { + weekday, + era, + year, + month, + day, + day_period, + hour, + minute, + second, + fractional_second_digits, + time_zone_name, + }) +} + +/// The options of CreateDateTimeFormat, read in its order. +pub(super) fn read_options( + machine: &mut Machine<'_, H>, + options: Options, + zone_of_value: Option<&str>, +) -> Result { + let locale_matcher = locale_matcher_option(machine, options)?; + let calendar = string_option(machine, options, "calendar", &[])?; + if let Some(name) = &calendar + && !is_type_sequence(name) + { + let message = format!("{name} is not a valid calendar"); + return Err(range_error(machine, message)); + } + let numbering_system = string_option(machine, options, "numberingSystem", &[])?; + if let Some(name) = &numbering_system + && !is_type_sequence(name) + { + let message = format!("{name} is not a valid numbering system"); + return Err(range_error(machine, message)); + } + let hour12 = boolean_option(machine, options, "hour12")?; + let hour_cycle = string_option(machine, options, "hourCycle", &["h11", "h12", "h23", "h24"])? + .map(|value| match value.as_str() { + "h11" => HourCycle::H11, + "h12" => HourCycle::H12, + "h23" => HourCycle::H23, + _ => HourCycle::H24, + }); + let time_zone = read(machine, options, "timeZone")?; + let time_zone = if time_zone == Value::UNDEFINED { + zone_of_value.map(str::to_owned) + } else if zone_of_value.is_some() { + return Err(type_error( + machine, + "a ZonedDateTime already has its time zone", + )); + } else { + let name = machine.coerce_string_observable(time_zone)?.to_utf8_lossy(); + if !is_valid_time_zone(&name, &*DATA) { + let message = format!("{name} is not a supported time zone"); + return Err(range_error(machine, message)); + } + Some(name) + }; + let components = component_options(machine, options)?; + let format_matcher = string_option(machine, options, "formatMatcher", &["basic", "best fit"])?; + let styles = ["full", "long", "medium", "short"]; + let date = date_style(string_option(machine, options, "dateStyle", &styles)?); + let time = date_style(string_option(machine, options, "timeStyle", &styles)?); + Ok(DateTimeFormatOptions { + locale_matcher, + calendar, + numbering_system, + time_zone, + hour12, + hour_cycle, + format_matcher: if format_matcher.as_deref() == Some("basic") { + FormatMatcher::Basic + } else { + FormatMatcher::BestFit + }, + date_style: date, + time_style: time, + components, + }) +} + +/// CreateDateTimeFormat over already canonicalized locales. +pub(super) fn create_date_time_format( + machine: &mut Machine<'_, H>, + requested: &[String], + options: &DateTimeFormatOptions, + required: RequiredComponents, + defaults: DefaultComponents, +) -> Result { + DateTimeFormat::try_new_with_mode( + requested, + options, + required, + defaults, + &*DATA, + &NoHostLocale, + ) + .map_err(|error| date_failure(machine, &error)) +} + +fn constructor( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + constructing: bool, +) -> Result { + let prototype = instance_prototype(machine, IntlKind::DateTimeFormat, constructing)?; + let requested = canonicalize_locale_list(machine, arg(args, 0))?; + let options = coerce_options_to_object(machine, arg(args, 1))?; + let options = read_options(machine, options, None)?; + let format = create_date_time_format( + machine, + &requested, + &options, + RequiredComponents::Any, + DefaultComponents::Date, + )?; + let instance = allocate(machine, IntlObject::DateTimeFormat(format), prototype)?; + chain_legacy_instance( + machine, + instance, + IntlKind::DateTimeFormat, + constructing, + this, + ) +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn this_format<'a, H: Host>( + machine: &'a Machine<'_, H>, + this: Value, +) -> Result, EvalFailure> { + Ok(match intl_record(machine, this)? { + Some(IntlObject::DateTimeFormat(format)) => Some(format), + _ => None, + }) +} + +fn not_a_date_time_format(machine: &mut Machine<'_, H>) -> EvalFailure { + type_error(machine, "receiver is not an Intl.DateTimeFormat") +} + +fn format_getter( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let format = unwrap_legacy_instance(machine, this, IntlKind::DateTimeFormat)?; + if this_format(machine, format)?.is_none() { + return Err(not_a_date_time_format(machine)); + } + bound_function(machine, format, FORMAT, 1) +} + +/// What ToDateTimeFormattable leaves: a time value or a Temporal object. +enum Formattable { + Time(f64), + Temporal(TemporalObject), +} + +/// ToDateTimeFormattable, with `undefined` meaning now. Temporal.Duration is +/// not a formattable Temporal object; it converts like any other object. +fn to_formattable( + machine: &mut Machine<'_, H>, + date: Value, +) -> Result { + if date == Value::UNDEFINED { + return Ok(Formattable::Time(machine.host.now_ms() as f64)); + } + match temporal_record(machine, date)? { + None | Some(TemporalObject::Duration(_)) => {} + Some(record) => return Ok(Formattable::Temporal(record)), + } + let number = machine.coerce_number_observable(date)?; + Ok(Formattable::Time(match number.decode() { + Some(Decoded::Int32(number)) => f64::from(number as i32), + Some(Decoded::Number(number)) => number, + _ => f64::NAN, + })) +} + +const NANOSECONDS_PER_DAY: i128 = 86_400_000_000_000; + +fn time_nanoseconds(time: PlainTime) -> i128 { + let seconds = + i128::from(time.hour) * 3600 + i128::from(time.minute) * 60 + i128::from(time.second); + seconds * 1_000_000_000 + + i128::from(time.millisecond) * 1_000_000 + + i128::from(time.microsecond) * 1000 + + i128::from(time.nanosecond) +} + +fn noon(date: PlainDate) -> i128 { + i128::from(date.epoch_days()) * NANOSECONDS_PER_DAY + NANOSECONDS_PER_DAY / 2 +} + +/// HandleDateTimeValue for a Temporal object: its formatter and instant, after +/// the checks each type's calendar must pass. +pub(super) fn temporal_value( + machine: &mut Machine<'_, H>, + format: &DateTimeFormat, + record: &TemporalObject, +) -> Result<(TemporalFormatKind, i128), EvalFailure> { + let calendar = format.resolved_options().calendar.clone(); + let check = |machine: &mut Machine<'_, H>, name: &str, iso_allowed: bool| { + if name == calendar || (iso_allowed && name == "iso8601") { + Ok(()) + } else { + let message = format!("calendar {name} does not match the formatter's {calendar}"); + Err(range_error(machine, message)) + } + }; + Ok(match record { + TemporalObject::PlainDate { value, calendar } => { + check(machine, calendar.as_str(), true)?; + (TemporalFormatKind::PlainDate, noon(*value)) + } + TemporalObject::PlainDateTime { value, calendar } => { + check(machine, calendar.as_str(), true)?; + let days = i128::from(value.date.epoch_days()); + let nanoseconds = days * NANOSECONDS_PER_DAY + time_nanoseconds(value.time); + (TemporalFormatKind::PlainDateTime, nanoseconds) + } + TemporalObject::PlainYearMonth { value, calendar } => { + check(machine, calendar.as_str(), false)?; + let days = epoch_days_from_ymd(value.year, value.month, value.reference_day()); + let nanoseconds = i128::from(days) * NANOSECONDS_PER_DAY + NANOSECONDS_PER_DAY / 2; + (TemporalFormatKind::PlainYearMonth, nanoseconds) + } + TemporalObject::PlainMonthDay { value, calendar } => { + check(machine, calendar.as_str(), false)?; + let days = epoch_days_from_ymd(value.reference_year(), value.month, value.day); + let nanoseconds = i128::from(days) * NANOSECONDS_PER_DAY + NANOSECONDS_PER_DAY / 2; + (TemporalFormatKind::PlainMonthDay, nanoseconds) + } + TemporalObject::PlainTime(value) => { + (TemporalFormatKind::PlainTime, time_nanoseconds(*value)) + } + TemporalObject::Instant(value) => (TemporalFormatKind::Instant, value.epoch_nanoseconds()), + TemporalObject::ZonedDateTime(_) | TemporalObject::Duration(_) => { + return Err(type_error( + machine, + "Temporal.ZonedDateTime is not formattable here", + )); + } + }) +} + +fn format_parts( + machine: &mut Machine<'_, H>, + this: Value, + value: &Formattable, +) -> Result, EvalFailure> { + let Some(format) = this_format(machine, this)? else { + return Err(not_a_date_time_format(machine)); + }; + let parts = match value { + Formattable::Time(date) => format.format_to_parts(*date, &*DATA).map(Some), + Formattable::Temporal(record) => { + let format = format.clone(); + let (kind, instant) = temporal_value(machine, &format, record)?; + format.format_temporal_to_parts(kind, instant, &*DATA) + } + }; + match parts.map_err(|error| date_failure(machine, &error))? { + Some(parts) => Ok(parts), + None => Err(type_error( + machine, + "this formatter's options do not apply to the value", + )), + } +} + +/// The function `format` hands out, bound to its DateTimeFormat. +fn format_value( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = to_formattable(machine, arg(args, 0))?; + let parts = format_parts(machine, this, &value)?; + let text: String = parts.iter().map(|part| part.value.as_str()).collect(); + string_outcome(machine, &text) +} + +fn format_to_parts( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if this_format(machine, this)?.is_none() { + return Err(not_a_date_time_format(machine)); + } + let value = to_formattable(machine, arg(args, 0))?; + let parts = format_parts(machine, this, &value)?; + let mut elements = Vec::with_capacity(parts.len()); + for part in &parts { + elements.push(part_object( + machine, + part.part_type.as_str(), + &part.value, + None, + )?); + } + super::super::allocate_array(machine, elements).map(BuiltinOutcome::Value) +} + +/// PartitionDateTimeRangePattern's dispatch on what the two ends are. +fn range_parts( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], +) -> Result, EvalFailure> { + if this_format(machine, this)?.is_none() { + return Err(not_a_date_time_format(machine)); + } + let (start, end) = (arg(args, 0), arg(args, 1)); + if start == Value::UNDEFINED || end == Value::UNDEFINED { + return Err(type_error(machine, "a range needs two ends")); + } + let start = to_formattable(machine, start)?; + let end = to_formattable(machine, end)?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_date_time_format(machine)); + }; + let format = format.clone(); + let parts = match (&start, &end) { + (Formattable::Time(start), Formattable::Time(end)) => { + format.format_range_to_parts(*start, *end, &*DATA).map(Some) + } + (Formattable::Temporal(first), Formattable::Temporal(second)) => { + let (kind, start) = temporal_value(machine, &format, first)?; + let (other, end) = temporal_value(machine, &format, second)?; + if kind != other { + return Err(type_error(machine, "a range needs two ends of one type")); + } + format.format_temporal_range_to_parts(kind, start, end, &*DATA) + } + _ => return Err(type_error(machine, "a range needs two ends of one type")), + }; + match parts.map_err(|error| date_failure(machine, &error))? { + Some(parts) => Ok(parts), + None => Err(type_error( + machine, + "this formatter's options do not apply to the value", + )), + } +} + +fn format_range( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let parts = range_parts(machine, this, args)?; + let text: String = parts.iter().map(|part| part.value.as_str()).collect(); + string_outcome(machine, &text) +} + +const fn source_name(source: RangeSource) -> &'static str { + match source { + RangeSource::Shared => "shared", + RangeSource::StartRange => "startRange", + RangeSource::EndRange => "endRange", + } +} + +fn format_range_to_parts( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let parts = range_parts(machine, this, args)?; + let mut elements = Vec::with_capacity(parts.len()); + for part in &parts { + elements.push(part_object( + machine, + part.part_type.as_str(), + &part.value, + Some(("source", source_name(part.source))), + )?); + } + super::super::allocate_array(machine, elements).map(BuiltinOutcome::Value) +} + +const fn width_name(width: TextWidth) -> &'static str { + match width { + TextWidth::Narrow => "narrow", + TextWidth::Short => "short", + TextWidth::Long => "long", + } +} + +const fn numeric_name(style: NumericStyle) -> &'static str { + match style { + NumericStyle::Numeric => "numeric", + NumericStyle::TwoDigit => "2-digit", + } +} + +const fn month_name(style: MonthStyle) -> &'static str { + match style { + MonthStyle::Numeric => "numeric", + MonthStyle::TwoDigit => "2-digit", + MonthStyle::Narrow => "narrow", + MonthStyle::Short => "short", + MonthStyle::Long => "long", + } +} + +const fn zone_name(style: TimeZoneNameStyle) -> &'static str { + match style { + TimeZoneNameStyle::Short => "short", + TimeZoneNameStyle::Long => "long", + TimeZoneNameStyle::ShortOffset => "shortOffset", + TimeZoneNameStyle::LongOffset => "longOffset", + TimeZoneNameStyle::ShortGeneric => "shortGeneric", + TimeZoneNameStyle::LongGeneric => "longGeneric", + } +} + +const fn style_name(style: DateTimeStyle) -> &'static str { + match style { + DateTimeStyle::Full => "full", + DateTimeStyle::Long => "long", + DateTimeStyle::Medium => "medium", + DateTimeStyle::Short => "short", + } +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let format = unwrap_legacy_instance(machine, this, IntlKind::DateTimeFormat)?; + let Some(format) = this_format(machine, format)? else { + return Err(not_a_date_time_format(machine)); + }; + let resolved = format.resolved_options().clone(); + let result = new_object(machine)?; + let text = |machine: &mut Machine<'_, H>, name: &str, value: &str| { + let value = string_value(machine, value)?; + define_field(machine, result, name, value) + }; + text(machine, "locale", &resolved.locale)?; + text(machine, "calendar", &resolved.calendar)?; + text(machine, "numberingSystem", &resolved.numbering_system)?; + text(machine, "timeZone", &resolved.time_zone)?; + if let Some(cycle) = resolved.hour_cycle { + text(machine, "hourCycle", cycle.as_str())?; + } + if let Some(hour12) = resolved.hour12 { + define_field(machine, result, "hour12", Value::boolean(hour12))?; + } + let components = &resolved.components; + let fields = [ + ("weekday", components.weekday.map(width_name)), + ("era", components.era.map(width_name)), + ("year", components.year.map(numeric_name)), + ("month", components.month.map(month_name)), + ("day", components.day.map(numeric_name)), + ("dayPeriod", components.day_period.map(width_name)), + ("hour", components.hour.map(numeric_name)), + ("minute", components.minute.map(numeric_name)), + ("second", components.second.map(numeric_name)), + ]; + for (name, value) in fields { + if let Some(value) = value { + text(machine, name, value)?; + } + } + if let Some(digits) = components.fractional_second_digits { + define_field( + machine, + result, + "fractionalSecondDigits", + crate::number_value(f64::from(digits)), + )?; + } + if let Some(name) = components.time_zone_name { + text(machine, "timeZoneName", zone_name(name))?; + } + if let Some(style) = resolved.date_style { + text(machine, "dateStyle", style_name(style))?; + } + if let Some(style) = resolved.time_style { + text(machine, "timeStyle", style_name(style))?; + } + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/display_names.rs b/crates/bamts-runtime/src/builtins/intl/display_names.rs new file mode 100644 index 00000000..9bf80daf --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/display_names.rs @@ -0,0 +1,165 @@ +//! `Intl.DisplayNames`. + +use bamts_native::Value; + +use super::options::{ + canonicalize_locale_list, get_options_object, locale_matcher_option, matcher_name, + string_option, +}; +use super::{ + Class, allocate, arg, define_field, install_class, instance_prototype, intl_record, new_object, + optional_string_outcome, require_new, service_failure, string_value, supported_locales_outcome, + type_error, +}; +use crate::intl::locale_data::{DATA, IntlData}; +use crate::intl::number_data::NoHostLocale; +use crate::intl::plural_rules::{DisplayNames, DisplayNamesOptions, DisplayNamesType}; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::DisplayNames, + name: "DisplayNames", + tag: "Intl.DisplayNames", + length: 2, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[], + methods: &[ + ("of", 1, of::), + ("resolvedOptions", 0, resolved_options::), + ], + }, + ); +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "DisplayNames")?; + let prototype = instance_prototype(machine, IntlKind::DisplayNames, constructing)?; + let requested = canonicalize_locale_list(machine, arg(args, 0))?; + if arg(args, 1) == Value::UNDEFINED { + return Err(type_error(machine, "options are required")); + } + let options = get_options_object(machine, arg(args, 1))?; + let matcher = locale_matcher_option(machine, options)?; + let style = string_option(machine, options, "style", &["narrow", "short", "long"])?; + let name_type = string_option( + machine, + options, + "type", + &[ + "language", + "region", + "script", + "currency", + "calendar", + "dateTimeField", + ], + )?; + if name_type.is_none() { + return Err(type_error(machine, "type is required")); + } + let fallback = string_option(machine, options, "fallback", &["code", "none"])?; + let language_display = string_option( + machine, + options, + "languageDisplay", + &["dialect", "standard"], + )?; + let resolved = DisplayNamesOptions { + locale_matcher: Some(matcher_name(matcher).to_owned()), + name_type, + style, + fallback, + language_display, + }; + let data: &'static IntlData = &DATA; + let names = DisplayNames::try_new(&requested, &resolved, data, &NoHostLocale) + .map_err(|error| service_failure(machine, &error))?; + allocate(machine, IntlObject::DisplayNames(names), prototype).map(BuiltinOutcome::Value) +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn this_names<'a, H: Host>( + machine: &'a Machine<'_, H>, + this: Value, +) -> Result>, EvalFailure> { + Ok(match intl_record(machine, this)? { + Some(IntlObject::DisplayNames(names)) => Some(names), + _ => None, + }) +} + +fn not_display_names(machine: &mut Machine<'_, H>) -> EvalFailure { + type_error(machine, "receiver is not an Intl.DisplayNames") +} + +fn of( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if this_names(machine, this)?.is_none() { + return Err(not_display_names(machine)); + } + let code = machine + .coerce_string_observable(arg(args, 0))? + .to_utf8_lossy(); + let Some(names) = this_names(machine, this)? else { + return Err(not_display_names(machine)); + }; + let name = names.of(&code); + let name = name.map_err(|error| service_failure(machine, &error))?; + optional_string_outcome(machine, name.as_deref()) +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let Some(names) = this_names(machine, this)? else { + return Err(not_display_names(machine)); + }; + let resolved = names.resolved_options(); + let result = new_object(machine)?; + let mut fields = vec![ + ("locale", resolved.locale.as_str()), + ("style", resolved.style.as_str()), + ("type", resolved.name_type.as_str()), + ("fallback", resolved.fallback.as_str()), + ]; + if resolved.name_type == DisplayNamesType::Language { + fields.push(("languageDisplay", resolved.language_display.as_str())); + } + for (name, text) in fields { + let value = string_value(machine, text)?; + define_field(machine, result, name, value)?; + } + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/duration_format.rs b/crates/bamts-runtime/src/builtins/intl/duration_format.rs new file mode 100644 index 00000000..b0bf1073 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/duration_format.rs @@ -0,0 +1,300 @@ +//! `Intl.DurationFormat`. + +use std::collections::BTreeMap; + +use bamts_native::Value; + +use super::super::allocate_array; +use super::super::temporal::to_temporal_duration; +use super::options::{ + Options, canonicalize_locale_list, get_options_object, locale_matcher_option, number_option, + string_option, +}; +use super::{ + Class, allocate, arg, define_field, install_class, instance_prototype, intl_record, + is_type_sequence, new_object, part_object, range_error, require_new, string_outcome, + string_value, supported_locales_outcome, type_error, +}; +use crate::intl::duration_format::{ + DurationFormat, DurationPart, UNITS, UnitOptions, UnitStyle, default_unit_options, + is_valid_unit_style, settled_unit_style, +}; +use crate::intl::locale_data::{DATA, IntlData}; +use crate::intl::locale_negotiation::resolve_locale; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::instant_duration::Duration; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::DurationFormat, + name: "DurationFormat", + tag: "Intl.DurationFormat", + length: 0, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[], + methods: &[ + ("format", 1, format::), + ("formatToParts", 1, format_to_parts::), + ("resolvedOptions", 0, resolved_options::), + ], + }, + ); +} + +/// GetDurationUnitOptions for the unit at `index`. +fn unit_options( + machine: &mut Machine<'_, H>, + options: Options, + index: usize, + base: &str, + previous: Option, +) -> Result { + const DATE_STYLES: &[&str] = &["long", "short", "narrow"]; + const TIME_STYLES: &[&str] = &["long", "short", "narrow", "numeric", "2-digit"]; + const SUBSECOND_STYLES: &[&str] = &["long", "short", "narrow", "numeric"]; + let allowed = match index { + 0..=3 => DATE_STYLES, + 4..=6 => TIME_STYLES, + _ => SUBSECOND_STYLES, + }; + let unit = UNITS[index]; + let requested = string_option(machine, options, unit, allowed)?; + let base_style = if base == "digital" { + UnitStyle::Short + } else { + UnitStyle::parse(base) + }; + let (mut style, mut always) = match requested { + Some(style) => (UnitStyle::parse(&style), true), + None => default_unit_options(index, base == "digital", base_style, previous), + }; + if style == UnitStyle::Numeric && index > 6 { + style = UnitStyle::Fractional; + always = false; + } + let display = string_option( + machine, + options, + &format!("{unit}Display"), + &["auto", "always"], + )?; + let always = display.map_or(always, |display| display == "always"); + if !is_valid_unit_style(style, always, previous) { + return Err(range_error( + machine, + format!("{unit} cannot use this style here"), + )); + } + Ok(UnitOptions { + style: settled_unit_style(index, style, previous), + always, + }) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "DurationFormat")?; + let prototype = instance_prototype(machine, IntlKind::DurationFormat, constructing)?; + let format = create_duration_format(machine, arg(args, 0), arg(args, 1))?; + allocate( + machine, + IntlObject::DurationFormat(Box::new(format)), + prototype, + ) + .map(BuiltinOutcome::Value) +} + +/// The body of the constructor, which `toLocaleString` shares. +pub(super) fn create_duration_format( + machine: &mut Machine<'_, H>, + locales: Value, + options: Value, +) -> Result { + let requested = canonicalize_locale_list(machine, locales)?; + let options = get_options_object(machine, options)?; + let matcher = locale_matcher_option(machine, options)?; + let numbering_system = string_option(machine, options, "numberingSystem", &[])?; + let mut locale_options = BTreeMap::new(); + if let Some(name) = numbering_system { + if !is_type_sequence(&name) { + let message = format!("{name} is not a valid numbering system"); + return Err(range_error(machine, message)); + } + locale_options.insert(String::from("nu"), name.to_ascii_lowercase()); + } + let resolved = resolve_locale( + &requested, + &locale_options, + &[String::from("nu")], + matcher, + &*DATA, + ) + .map_err(|error| range_error(machine, error.to_string()))?; + let style = string_option( + machine, + options, + "style", + &["long", "short", "narrow", "digital"], + )? + .unwrap_or_else(|| String::from("short")); + let mut units = [UnitOptions { + style: UnitStyle::Short, + always: false, + }; 10]; + let mut previous = None; + for (index, slot) in units.iter_mut().enumerate() { + let unit = unit_options(machine, options, index, &style, previous)?; + *slot = unit; + previous = Some(unit.style); + } + let fractional_digits = + number_option(machine, options, "fractionalDigits", 0.0, 9.0)?.map(|digits| digits as u8); + Ok(DurationFormat { + locale: resolved.locale, + numbering_system: resolved.values.get("nu").cloned().unwrap_or_default(), + style, + units, + fractional_digits, + }) +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn this_format<'a, H: Host>( + machine: &'a Machine<'_, H>, + this: Value, +) -> Result, EvalFailure> { + Ok(match intl_record(machine, this)? { + Some(IntlObject::DurationFormat(format)) => Some(format), + _ => None, + }) +} + +fn not_a_duration_format(machine: &mut Machine<'_, H>) -> EvalFailure { + type_error(machine, "receiver is not an Intl.DurationFormat") +} + +/// ToDurationRecord, then the partition. +fn parts( + machine: &mut Machine<'_, H>, + this: Value, + duration: Value, +) -> Result, EvalFailure> { + if this_format(machine, this)?.is_none() { + return Err(not_a_duration_format(machine)); + } + let duration = to_temporal_duration(machine, duration)?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_duration_format(machine)); + }; + let format = format.clone(); + duration_parts(machine, &format, &duration) +} + +/// PartitionDurationFormatPattern. +pub(super) fn duration_parts( + machine: &mut Machine<'_, H>, + format: &DurationFormat, + duration: &Duration, +) -> Result, EvalFailure> { + let fields = [ + duration.years, + duration.months, + duration.weeks, + duration.days, + duration.hours, + duration.minutes, + duration.seconds, + duration.milliseconds, + duration.microseconds, + duration.nanoseconds, + ]; + let data: &'static IntlData = &DATA; + let result = format.partition(&fields, data); + result.map_err(|error| range_error(machine, error.to_string())) +} + +fn format( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let parts = parts(machine, this, arg(args, 0))?; + let text: String = parts.iter().map(|part| part.value.as_str()).collect(); + string_outcome(machine, &text) +} + +fn format_to_parts( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let parts = parts(machine, this, arg(args, 0))?; + let mut elements = Vec::with_capacity(parts.len()); + for part in &parts { + let extra = part.unit.map(|unit| ("unit", unit)); + elements.push(part_object(machine, part.part_type, &part.value, extra)?); + } + allocate_array(machine, elements).map(BuiltinOutcome::Value) +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let Some(format) = this_format(machine, this)? else { + return Err(not_a_duration_format(machine)); + }; + let format = format.clone(); + let result = new_object(machine)?; + for (name, text) in [ + ("locale", format.locale.as_str()), + ("numberingSystem", format.numbering_system.as_str()), + ("style", format.style.as_str()), + ] { + let value = string_value(machine, text)?; + define_field(machine, result, name, value)?; + } + for (unit, options) in UNITS.iter().zip(format.units) { + let value = string_value(machine, options.style.as_str())?; + define_field(machine, result, unit, value)?; + let display = if options.always { "always" } else { "auto" }; + let value = string_value(machine, display)?; + define_field(machine, result, &format!("{unit}Display"), value)?; + } + if let Some(digits) = format.fractional_digits { + define_field( + machine, + result, + "fractionalDigits", + crate::number_value(f64::from(digits)), + )?; + } + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/integration.rs b/crates/bamts-runtime/src/builtins/intl/integration.rs new file mode 100644 index 00000000..7022ce40 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/integration.rs @@ -0,0 +1,209 @@ +//! The `toLocale*` methods of the rest of the runtime, which ECMA-402 defines +//! as creating a formatter and using it once. + +use bamts_bytecode::EcmaString; +use bamts_native::Value; + +use super::super::temporal::temporal_record; +use super::collator::create_collator; +use super::date_time_format::{ + create_date_time_format, date_failure, read_options, temporal_value, +}; +use super::duration_format::{create_duration_format, duration_parts}; +use super::number_format::{create_number_format, number_failure, read_number_format_options}; +use super::options::{canonicalize_locale_list, coerce_options_to_object}; +use super::{range_error, string_outcome, type_error}; +use crate::intl::date_time_format::{ + DateTimeFormat, DateTimeFormatOptions, DefaultComponents, RequiredComponents, + TemporalFormatKind, TimeZoneNameStyle, +}; +use crate::intl::locale_data::DATA; +use crate::intl::number_format::IntlMathematicalValue; +use crate::intrinsics::BuiltinOutcome; +use crate::temporal::TemporalObject; +use crate::temporal::instant_duration::Duration; +use crate::{EvalFailure, Host, Machine}; + +/// `Number.prototype.toLocaleString` after the receiver check. +pub(in crate::intrinsics::builtins) fn number_to_locale_string( + machine: &mut Machine<'_, H>, + number: f64, + locales: Value, + options: Value, +) -> Result { + number_to_string( + machine, + &IntlMathematicalValue::from_f64(number), + locales, + options, + ) +} + +/// `String.prototype.localeCompare` through a collator made for this call. +pub(in crate::intrinsics::builtins) fn compare_with_collator( + machine: &mut Machine<'_, H>, + left: &EcmaString, + right: &EcmaString, + locales: Value, + options: Value, +) -> Result { + let collator = create_collator(machine, locales, options)?; + let result = collator.compare(&*DATA, left.as_units(), right.as_units()); + Ok(BuiltinOutcome::Value(crate::number_value(f64::from( + result, + )))) +} + +/// `BigInt.prototype.toLocaleString` after the receiver check. +pub(in crate::intrinsics::builtins) fn bigint_to_locale_string( + machine: &mut Machine<'_, H>, + decimal: &str, + locales: Value, + options: Value, +) -> Result { + let value = IntlMathematicalValue::from_bigint(decimal) + .map_err(|error| number_failure(machine, error))?; + number_to_string(machine, &value, locales, options) +} + +fn number_to_string( + machine: &mut Machine<'_, H>, + value: &IntlMathematicalValue, + locales: Value, + options: Value, +) -> Result { + let requested = canonicalize_locale_list(machine, locales)?; + let options = coerce_options_to_object(machine, options)?; + let options = read_number_format_options(machine, options)?; + let format = create_number_format(machine, &requested, &options)?; + let text = format + .format(value) + .map_err(|error| number_failure(machine, error))?; + string_outcome(machine, &text) +} + +/// CreateDateTimeFormat for one `toLocale*` call. +fn date_format( + machine: &mut Machine<'_, H>, + locales: Value, + options: Value, + required: RequiredComponents, + defaults: DefaultComponents, + adjust: impl FnOnce(&mut DateTimeFormatOptions), + zone_of_value: Option<&str>, +) -> Result { + let requested = canonicalize_locale_list(machine, locales)?; + let options = coerce_options_to_object(machine, options)?; + let mut options = read_options(machine, options, zone_of_value)?; + adjust(&mut options); + create_date_time_format(machine, &requested, &options, required, defaults) +} + +/// `Date.prototype.toLocaleString` and its two peers, after the receiver check. +/// An invalid date never reaches the options. +pub(in crate::intrinsics::builtins) fn date_to_locale_string( + machine: &mut Machine<'_, H>, + time: f64, + locales: Value, + options: Value, + required: RequiredComponents, + defaults: DefaultComponents, +) -> Result { + if time.is_nan() { + return string_outcome(machine, "Invalid Date"); + } + let format = date_format(machine, locales, options, required, defaults, |_| {}, None)?; + let text = format + .format(time, &*DATA) + .map_err(|error| date_failure(machine, &error))?; + string_outcome(machine, &text) +} + +fn temporal_text( + machine: &mut Machine<'_, H>, + format: &DateTimeFormat, + kind: TemporalFormatKind, + instant: i128, +) -> Result { + let parts = format + .format_temporal_to_parts(kind, instant, &*DATA) + .map_err(|error| date_failure(machine, &error))?; + let Some(parts) = parts else { + return Err(type_error( + machine, + "these options do not apply to the value", + )); + }; + let text: String = parts.iter().map(|part| part.value.as_str()).collect(); + string_outcome(machine, &text) +} + +/// `toLocaleString` of a Temporal class, after its receiver check. +pub(in crate::intrinsics::builtins) fn temporal_to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + locales: Value, + options: Value, +) -> Result { + use DefaultComponents as Defaults; + use RequiredComponents as Required; + let Some(record) = temporal_record(machine, this)? else { + return Err(type_error(machine, "receiver is not a Temporal object")); + }; + if let TemporalObject::ZonedDateTime(zoned) = &record { + let zone = zoned.time_zone_id().as_str().to_owned(); + let calendar = zoned.calendar_id().as_str().to_owned(); + let instant = zoned.epoch_nanoseconds(); + let format = date_format( + machine, + locales, + options, + Required::Any, + Defaults::All, + |options| { + // The zone is part of the default rendering, not of a requested one. + let styled = options.date_style.is_some() || options.time_style.is_some(); + if !styled && !options.components.has_any() { + options.components.time_zone_name = options + .components + .time_zone_name + .or(Some(TimeZoneNameStyle::Short)); + } + }, + Some(&zone), + )?; + if calendar != "iso8601" && calendar != format.resolved_options().calendar { + return Err(range_error( + machine, + format!("calendar {calendar} does not match the formatter's"), + )); + } + return temporal_text(machine, &format, TemporalFormatKind::Instant, instant); + } + let (required, defaults) = match &record { + TemporalObject::PlainDate { .. } => (Required::Date, Defaults::Date), + TemporalObject::PlainTime(_) => (Required::Time, Defaults::Time), + TemporalObject::PlainYearMonth { .. } => (Required::YearMonth, Defaults::YearMonth), + TemporalObject::PlainMonthDay { .. } => (Required::MonthDay, Defaults::MonthDay), + _ => (Required::Any, Defaults::All), + }; + let format = date_format(machine, locales, options, required, defaults, |_| {}, None)?; + let (kind, instant) = temporal_value(machine, &format, &record)?; + temporal_text(machine, &format, kind, instant) +} + +/// `Temporal.Duration.prototype.toLocaleString`, after the receiver check. +pub(in crate::intrinsics::builtins) fn duration_to_locale_string( + machine: &mut Machine<'_, H>, + duration: &Duration, + locales: Value, + options: Value, +) -> Result { + let format = create_duration_format(machine, locales, options)?; + let parts = duration_parts(machine, &format, duration)?; + let mut text = String::new(); + for part in &parts { + text.push_str(&part.value); + } + super::super::allocate_string(machine, EcmaString::encode(&text)).map(BuiltinOutcome::Value) +} diff --git a/crates/bamts-runtime/src/builtins/intl/list_format.rs b/crates/bamts-runtime/src/builtins/intl/list_format.rs new file mode 100644 index 00000000..a240189c --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/list_format.rs @@ -0,0 +1,164 @@ +//! `Intl.ListFormat`. + +use bamts_native::Value; + +use super::super::allocate_array; +use super::options::{ + canonicalize_locale_list, get_options_object, locale_matcher_option, matcher_name, + string_option, +}; +use super::{ + Class, allocate, arg, define_field, install_class, instance_prototype, intl_record, new_object, + part_object, require_new, service_failure, string_iterable_to_list, string_outcome, + string_value, supported_locales_outcome, type_error, +}; +use crate::intl::locale_data::{DATA, IntlData}; +use crate::intl::number_data::NoHostLocale; +use crate::intl::plural_rules::{ListFormat, ListFormatOptions, ListPartKind}; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::ListFormat, + name: "ListFormat", + tag: "Intl.ListFormat", + length: 0, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[], + methods: &[ + ("format", 1, format::), + ("formatToParts", 1, format_to_parts::), + ("resolvedOptions", 0, resolved_options::), + ], + }, + ); +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "ListFormat")?; + let prototype = instance_prototype(machine, IntlKind::ListFormat, constructing)?; + let requested = canonicalize_locale_list(machine, arg(args, 0))?; + let options = get_options_object(machine, arg(args, 1))?; + let matcher = locale_matcher_option(machine, options)?; + let list_type = string_option( + machine, + options, + "type", + &["conjunction", "disjunction", "unit"], + )?; + let style = string_option(machine, options, "style", &["long", "short", "narrow"])?; + let resolved = ListFormatOptions { + locale_matcher: Some(matcher_name(matcher).to_owned()), + list_type, + style, + }; + let data: &'static IntlData = &DATA; + let format = ListFormat::try_new(&requested, &resolved, data, &NoHostLocale) + .map_err(|error| service_failure(machine, &error))?; + allocate(machine, IntlObject::ListFormat(format), prototype).map(BuiltinOutcome::Value) +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn this_format<'a, H: Host>( + machine: &'a Machine<'_, H>, + this: Value, +) -> Result>, EvalFailure> { + Ok(match intl_record(machine, this)? { + Some(IntlObject::ListFormat(format)) => Some(format), + _ => None, + }) +} + +fn not_a_list_format(machine: &mut Machine<'_, H>) -> EvalFailure { + type_error(machine, "receiver is not an Intl.ListFormat") +} + +fn format( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if this_format(machine, this)?.is_none() { + return Err(not_a_list_format(machine)); + } + let elements = string_iterable_to_list(machine, arg(args, 0))?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_list_format(machine)); + }; + let text = format.format(&elements); + let text = text.map_err(|error| service_failure(machine, &error))?; + string_outcome(machine, &text) +} + +fn format_to_parts( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if this_format(machine, this)?.is_none() { + return Err(not_a_list_format(machine)); + } + let elements = string_iterable_to_list(machine, arg(args, 0))?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_list_format(machine)); + }; + let parts = format.format_to_parts(&elements); + let parts = parts.map_err(|error| service_failure(machine, &error))?; + let mut elements = Vec::with_capacity(parts.len()); + for part in &parts { + let part_type = match part.kind { + ListPartKind::Element => "element", + ListPartKind::Literal => "literal", + }; + elements.push(part_object(machine, part_type, &part.value, None)?); + } + allocate_array(machine, elements).map(BuiltinOutcome::Value) +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let Some(format) = this_format(machine, this)? else { + return Err(not_a_list_format(machine)); + }; + let resolved = format.resolved_options(); + let result = new_object(machine)?; + for (name, text) in [ + ("locale", resolved.locale.as_str()), + ("type", resolved.list_type.as_str()), + ("style", resolved.style.as_str()), + ] { + let value = string_value(machine, text)?; + define_field(machine, result, name, value)?; + } + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/locale.rs b/crates/bamts-runtime/src/builtins/intl/locale.rs new file mode 100644 index 00000000..cb44c231 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/locale.rs @@ -0,0 +1,487 @@ +//! `Intl.Locale`. + +use bamts_native::Value; + +use super::super::allocate_array; +use super::options::{Options, boolean_option, coerce_options_to_object, string_option}; +use super::{ + Class, arg, boolean_outcome, construct, create, define_field, install_class, intl_record, + is_object, is_type_sequence, locale_record, new_object, optional_string_outcome, range_error, + require_new, string_array_outcome, string_outcome, string_value, type_error, +}; +use crate::intl::locale_data::{self, DATA}; +use crate::intl::locale_negotiation::{ + LanguageId, LanguageTag, UnicodeExtension, canonicalize_unicode_locale_id, +}; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::Locale, + name: "Locale", + tag: "Intl.Locale", + length: 1, + constructor: constructor::, + statics: &[], + getters: &[ + ("baseName", "get baseName", base_name::), + ("calendar", "get calendar", calendar::), + ("caseFirst", "get caseFirst", case_first::), + ("collation", "get collation", collation::), + ( + "firstDayOfWeek", + "get firstDayOfWeek", + first_day_of_week::, + ), + ("hourCycle", "get hourCycle", hour_cycle::), + ("language", "get language", language::), + ( + "numberingSystem", + "get numberingSystem", + numbering_system::, + ), + ("numeric", "get numeric", numeric::), + ("region", "get region", region::), + ("script", "get script", script::), + ("variants", "get variants", variants::), + ], + methods: &[ + ("getCalendars", 0, get_calendars::), + ("getCollations", 0, get_collations::), + ("getHourCycles", 0, get_hour_cycles::), + ("getNumberingSystems", 0, get_numbering_systems::), + ("getTextInfo", 0, get_text_info::), + ("getTimeZones", 0, get_time_zones::), + ("getWeekInfo", 0, get_week_info::), + ("maximize", 0, maximize::), + ("minimize", 0, minimize::), + ("toString", 0, to_string::), + ], + }, + ); +} + +fn this_locale( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result { + match intl_record(machine, this)? { + Some(IntlObject::Locale(tag)) => Ok(tag.clone()), + _ => Err(type_error(machine, "receiver is not an Intl.Locale")), + } +} + +/// GetOption for a keyword whose value must match the `type` production. +fn type_option( + machine: &mut Machine<'_, H>, + options: Options, + name: &str, +) -> Result, EvalFailure> { + let value = string_option(machine, options, name, &[])?; + checked_type(machine, name, value) +} + +fn checked_type( + machine: &mut Machine<'_, H>, + name: &str, + value: Option, +) -> Result, EvalFailure> { + match value { + Some(value) if !is_type_sequence(&value) => Err(range_error( + machine, + format!("{value} is not a valid value for {name}"), + )), + value => Ok(value), + } +} + +/// `WeekdayToString`: a number names a day, Sunday being both 0 and 7. +fn weekday_keyword(value: &str) -> &str { + match value { + "0" | "7" => "sun", + "1" => "mon", + "2" => "tue", + "3" => "wed", + "4" => "thu", + "5" => "fri", + "6" => "sat", + other => other, + } +} + +/// UpdateLanguageId: the language options, validated as they are read. +fn update_language_id( + machine: &mut Machine<'_, H>, + tag: &LanguageTag, + options: Options, +) -> Result { + let mut id = tag.id.clone(); + if let Some(language) = string_option(machine, options, "language", &[])? { + if !LanguageTag::is_language_subtag(&language) { + return Err(range_error(machine, format!("invalid language {language}"))); + } + id.language = language; + } + if let Some(script) = string_option(machine, options, "script", &[])? { + if !LanguageTag::is_script_subtag(&script) { + return Err(range_error(machine, format!("invalid script {script}"))); + } + id.script = Some(script); + } + if let Some(region) = string_option(machine, options, "region", &[])? { + if !LanguageTag::is_region_subtag(®ion) { + return Err(range_error(machine, format!("invalid region {region}"))); + } + id.region = Some(region); + } + if let Some(variants) = string_option(machine, options, "variants", &[])? { + let lower = variants.to_ascii_lowercase(); + let subtags: Vec = lower.split('-').map(str::to_owned).collect(); + let duplicate = subtags + .iter() + .enumerate() + .any(|(index, subtag)| subtags[..index].contains(subtag)); + if duplicate + || !subtags + .iter() + .all(|subtag| LanguageTag::is_variant_subtag(subtag)) + { + return Err(range_error(machine, format!("invalid variants {variants}"))); + } + id.variants = subtags; + } + Ok(id) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "Locale")?; + let tag = arg(args, 0); + if machine.string_value(tag).is_none() && !is_object(machine, tag)? { + return Err(type_error( + machine, + "a locale must be a string or an object", + )); + } + let text = match locale_record(machine, tag)? { + Some(tag) => tag.to_string(), + None => machine.coerce_string_observable(tag)?.to_utf8_lossy(), + }; + let options = coerce_options_to_object(machine, arg(args, 1))?; + let canonical = canonicalize_unicode_locale_id(&text, &*DATA) + .map_err(|error| range_error(machine, error.to_string()))?; + let mut tag = parse(machine, &canonical)?; + tag.id = update_language_id(machine, &tag, options)?; + + let calendar = type_option(machine, options, "calendar")?; + let collation = type_option(machine, options, "collation")?; + let first_day = string_option(machine, options, "firstDayOfWeek", &[])? + .map(|day| weekday_keyword(&day).to_owned()); + let first_day = checked_type(machine, "firstDayOfWeek", first_day)?; + let hour_cycle = string_option(machine, options, "hourCycle", &["h11", "h12", "h23", "h24"])?; + let case_first = string_option(machine, options, "caseFirst", &["upper", "lower", "false"])?; + let numeric = boolean_option(machine, options, "numeric")?; + let numbering_system = type_option(machine, options, "numberingSystem")?; + + let keywords = [ + ("ca", calendar), + ("co", collation), + ("fw", first_day), + ("hc", hour_cycle), + ("kf", case_first), + ("kn", numeric.map(|value| value.to_string())), + ("nu", numbering_system), + ]; + if keywords.iter().any(|(_, value)| value.is_some()) { + let extension = tag.unicode.get_or_insert_with(UnicodeExtension::default); + for (key, value) in keywords { + if let Some(value) = value { + extension.keywords.insert(key.to_owned(), value); + } + } + } + let canonical = canonicalize_unicode_locale_id(&tag.to_string(), &*DATA) + .map_err(|error| range_error(machine, error.to_string()))?; + let tag = parse(machine, &canonical)?; + construct(machine, IntlKind::Locale, IntlObject::Locale(tag)) +} + +fn parse(machine: &mut Machine<'_, H>, text: &str) -> Result { + LanguageTag::parse(text).map_err(|error| range_error(machine, error.to_string())) +} + +fn keyword(tag: &LanguageTag, key: &str) -> Option { + tag.unicode + .as_ref() + .and_then(|extension| extension.keywords.get(key)) + .cloned() +} + +macro_rules! keyword_getters { + ($($name:ident: $key:literal),+ $(,)?) => { + $(fn $name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let tag = this_locale(machine, this)?; + optional_string_outcome(machine, keyword(&tag, $key).as_deref()) + })+ + }; +} + +keyword_getters! { + calendar: "ca", + case_first: "kf", + collation: "co", + first_day_of_week: "fw", + hour_cycle: "hc", + numbering_system: "nu", +} + +fn numeric( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + Ok(boolean_outcome( + keyword(&tag, "kn").is_some_and(|value| value.is_empty() || value == "true"), + )) +} + +fn base_name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + string_outcome(machine, &tag.id.render()) +} + +fn language( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + string_outcome(machine, &tag.id.language) +} + +fn script( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + optional_string_outcome(machine, tag.id.script.as_deref()) +} + +fn region( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + optional_string_outcome(machine, tag.id.region.as_deref()) +} + +fn variants( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + if tag.id.variants.is_empty() { + return Ok(BuiltinOutcome::Value(Value::UNDEFINED)); + } + string_outcome(machine, &tag.id.variants.join("-")) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + string_outcome(machine, &tag.to_string()) +} + +/// A new `Intl.Locale` with `tag`'s extensions over a replaced identifier. +fn with_id( + machine: &mut Machine<'_, H>, + mut tag: LanguageTag, + id: Option, +) -> Result { + if let Some(id) = id { + tag.id = id; + } + create(machine, IntlObject::Locale(tag)).map(BuiltinOutcome::Value) +} + +fn maximize( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + let maximal = locale_data::add_likely_subtags(&tag.id); + with_id(machine, tag, maximal) +} + +fn minimize( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + let minimal = locale_data::remove_likely_subtags(&tag.id); + with_id(machine, tag, minimal) +} + +/// The answer of a `get...` method: the keyword alone when it is present. +fn keyword_or_list( + machine: &mut Machine<'_, H>, + tag: &LanguageTag, + key: &str, + list: &[&str], +) -> Result { + let values: Vec = match keyword(tag, key) { + Some(value) if !value.is_empty() => vec![value], + _ => list.iter().map(|value| (*value).to_owned()).collect(), + }; + string_array_outcome(machine, &values) +} + +fn get_calendars( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + keyword_or_list(machine, &tag, "ca", locale_data::calendars_of(&tag)) +} + +fn get_collations( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + keyword_or_list(machine, &tag, "co", locale_data::COLLATIONS) +} + +fn get_hour_cycles( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + keyword_or_list(machine, &tag, "hc", locale_data::hour_cycles(&tag)) +} + +fn get_numbering_systems( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + keyword_or_list(machine, &tag, "nu", locale_data::numbering_systems_of(&tag)) +} + +fn get_time_zones( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + match locale_data::time_zones_of(&tag) { + Some(zones) => { + let zones: Vec = zones.iter().map(|zone| (*zone).to_owned()).collect(); + string_array_outcome(machine, &zones) + } + None => Ok(BuiltinOutcome::Value(Value::UNDEFINED)), + } +} + +fn get_text_info( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + let direction = if locale_data::is_right_to_left(&tag) { + "rtl" + } else { + "ltr" + }; + let result = new_object(machine)?; + let direction = string_value(machine, direction)?; + define_field(machine, result, "direction", direction)?; + Ok(BuiltinOutcome::Value(result)) +} + +/// The ISO weekday a `-u-fw` value names. +fn weekday_number(value: &str) -> Option { + ["mon", "tue", "wed", "thu", "fri", "sat", "sun"] + .iter() + .position(|day| *day == value) + .and_then(|index| u8::try_from(index + 1).ok()) +} + +fn get_week_info( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let tag = this_locale(machine, this)?; + let (first_day, weekend) = locale_data::week_info(&tag); + let first_day = keyword(&tag, "fw") + .and_then(|value| weekday_number(&value)) + .unwrap_or(first_day); + let result = new_object(machine)?; + define_field( + machine, + result, + "firstDay", + Value::int32(u32::from(first_day)), + )?; + let days = weekend + .iter() + .map(|day| Value::int32(u32::from(*day))) + .collect(); + let weekend = allocate_array(machine, days)?; + define_field(machine, result, "weekend", weekend)?; + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/mod.rs b/crates/bamts-runtime/src/builtins/intl/mod.rs new file mode 100644 index 00000000..75d40b1d --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/mod.rs @@ -0,0 +1,645 @@ +//! The JavaScript binding of ECMA-402. +//! +//! Algorithms live in `crate::intl`; this module owns what only JavaScript can +//! observe: argument conversion, the order options and properties are read in, +//! receiver checks, and the shape of the installed objects. + +mod case_mapping; +mod collator; +mod date_time_format; +mod display_names; +mod duration_format; +mod integration; +mod list_format; +mod locale; +mod number_format; +mod options; +mod plural_rules; +mod relative_time_format; +mod segmenter; + +pub(super) use case_mapping::{TargetCase, transform_case}; +pub(super) use integration::{ + bigint_to_locale_string, compare_with_collator, date_to_locale_string, + duration_to_locale_string, number_to_locale_string, temporal_to_locale_string, +}; + +use std::collections::BTreeMap; + +use bamts_bytecode::EcmaString; +use bamts_native::Value; + +use super::super::push; +use super::{ + allocate_array, allocate_string, define_data, install_constructor_function, install_function, +}; +use crate::intl::canonical_locales; +use crate::intl::locale_data::DATA; +use crate::intl::locale_negotiation::{LanguageTag, supported_locales}; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinDef, BuiltinHandler, BuiltinOutcome, BuiltinTable}; +use crate::{ + BoundCallable, EvalFailure, HeapEntry, Host, Machine, NativeCallable, Property, PropertyKey, + PropertyMap, +}; + +/// Name, `length`, and implementation of one installed function. +pub(super) type Method = (&'static str, u32, BuiltinHandler); + +/// Property name, function name, and implementation of one accessor. +pub(super) type Getter = (&'static str, &'static str, BuiltinHandler); + +/// A function the runtime hands out but never installs, found again by `name`. +pub(super) fn register_hidden( + builtins: &mut BuiltinTable, + name: &'static str, + length: u32, + handler: BuiltinHandler, +) { + builtins.register(BuiltinDef { + name, + length, + handler, + }); +} + +/// The installed shape of one Intl class. +struct Class<'a, H: Host> { + kind: IntlKind, + name: &'static str, + tag: &'static str, + length: u32, + constructor: BuiltinHandler, + statics: &'a [Method], + getters: &'a [Getter], + methods: &'a [Method], +} + +fn install_class( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, + class: &Class<'_, H>, +) { + let prototype = builtins.intl_prototype(class.kind); + let constructor = + install_constructor_function(heap, builtins, class.name, class.length, class.constructor); + builtins.set_constructor_prototype(heap, constructor, prototype); + for &(name, length, handler) in class.statics { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, constructor, name, function); + } + for &(property, name, handler) in class.getters { + let getter = install_function(heap, builtins, name, 0, handler); + super::temporal::define_getter(heap, prototype, property, getter); + } + for &(name, length, handler) in class.methods { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, prototype, name, function); + } + let tag_symbol = builtins.symbol_to_string_tag(); + super::define_to_string_tag(heap, prototype, tag_symbol, class.tag); + define_data(heap, namespace, class.name, constructor); +} + +pub(super) fn install( + heap: &mut Vec, + globals: &mut BTreeMap, + builtins: &mut BuiltinTable, +) { + let namespace = super::super::ordinary_prototype(heap, builtins.object_prototype()); + let tag_symbol = builtins.symbol_to_string_tag(); + super::define_to_string_tag(heap, namespace, tag_symbol, "Intl"); + let fallback_symbol = push( + heap, + HeapEntry::Symbol { + description: EcmaString::encode("IntlLegacyConstructedSymbol"), + }, + ); + builtins.set_intl_fallback_symbol(fallback_symbol); + // Classes refer to each other's instances, so every prototype exists before + // any class is populated. + for kind in IntlKind::ALL { + let prototype = super::super::ordinary_prototype(heap, builtins.object_prototype()); + builtins.set_intl_prototype(kind, prototype); + } + let functions: [Method; 2] = [ + ("getCanonicalLocales", 1, get_canonical_locales::), + ("supportedValuesOf", 1, supported_values_of::), + ]; + for (name, length, handler) in functions { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, namespace, name, function); + } + locale::install(heap, builtins, namespace); + collator::install(heap, builtins, namespace); + number_format::install(heap, builtins, namespace); + date_time_format::install(heap, builtins, namespace); + plural_rules::install(heap, builtins, namespace); + list_format::install(heap, builtins, namespace); + relative_time_format::install(heap, builtins, namespace); + segmenter::install(heap, builtins, namespace); + display_names::install(heap, builtins, namespace); + duration_format::install(heap, builtins, namespace); + globals.insert(EcmaString::encode("Intl"), namespace); +} + +/// `Intl.getCanonicalLocales`. +fn get_canonical_locales( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let locales = options::canonicalize_locale_list(machine, arg(args, 0))?; + string_array_outcome(machine, &locales) +} + +/// `Intl.supportedValuesOf`. +fn supported_values_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let key = machine + .coerce_string_observable(arg(args, 0))? + .to_utf8_lossy(); + let values = canonical_locales::supported_values_of(&key, &*DATA) + .map_err(|error| range_error(machine, error.message))?; + string_array_outcome(machine, &values) +} + +/// Maps a service-core error to the exception it stands for. +pub(super) fn service_failure( + machine: &mut Machine<'_, H>, + error: &crate::intl::plural_rules::IntlError, +) -> EvalFailure { + match error.kind() { + crate::intl::plural_rules::IntlErrorKind::TypeError => { + type_error(machine, error.to_string()) + } + _ => range_error(machine, error.to_string()), + } +} + +fn throw(machine: &mut Machine<'_, H>, name: &str, message: String) -> EvalFailure { + let id = machine + .intrinsics + .builtins + .id_named(name) + .expect("error constructor is installed"); + machine.throw_error(id, message) +} + +pub(super) fn range_error( + machine: &mut Machine<'_, H>, + message: impl Into, +) -> EvalFailure { + throw(machine, "RangeError", message.into()) +} + +pub(super) fn type_error( + machine: &mut Machine<'_, H>, + message: impl Into, +) -> EvalFailure { + throw(machine, "TypeError", message.into()) +} + +/// `Type(value) is Object`. `Machine::is_object` also answers true for a +/// Symbol, which every "must be an object" check here has to reject. +pub(super) fn is_object( + machine: &Machine<'_, H>, + value: Value, +) -> Result { + Ok(!machine.coercion_is_primitive(value)?) +} + +/// The Intl record `value` carries, if it is an Intl instance. +pub(super) fn intl_record<'a, H: Host>( + machine: &'a Machine<'_, H>, + value: Value, +) -> Result, EvalFailure> { + let Some(index) = machine.runtime_slot(value).map_err(EvalFailure::Runtime)? else { + return Ok(None); + }; + Ok(match &machine.heap[index] { + HeapEntry::Intl { value, .. } => Some(value), + _ => None, + }) +} + +/// The language tag `value` carries, if it is an `Intl.Locale`. +pub(super) fn locale_record<'a, H: Host>( + machine: &'a Machine<'_, H>, + value: Value, +) -> Result, EvalFailure> { + Ok(match intl_record(machine, value)? { + Some(IntlObject::Locale(tag)) => Some(tag), + _ => None, + }) +} + +/// The prototype OrdinaryCreateFromConstructor picks. The service constructors +/// read it first, before their arguments; a plain call has no `new.target`. +pub(super) fn instance_prototype( + machine: &mut Machine<'_, H>, + kind: IntlKind, + constructing: bool, +) -> Result { + if constructing { + let new_target = machine.current_new_target(); + let prototype = machine.get_named_property(new_target, "prototype")?; + if is_object(machine, prototype)? { + return Ok(prototype); + } + } + Ok(machine.intrinsics.builtins.intl_prototype(kind)) +} + +/// OrdinaryHasInstance against an Intl constructor, whose `prototype` property +/// is fixed. +fn inherits_from( + machine: &mut Machine<'_, H>, + value: Value, + kind: IntlKind, +) -> Result { + if !is_object(machine, value)? { + return Ok(false); + } + let prototype = machine.intrinsics.builtins.intl_prototype(kind); + let mut object = value; + while let Some(next) = machine.internal_get_prototype_of(object)? { + if next == prototype { + return Ok(true); + } + object = next; + } + Ok(false) +} + +/// ChainNumberFormat and ChainDateTimeFormat: a plain call on an object that +/// inherits from the constructor initializes that object, for the sake of +/// `Intl.Constructor.call(instance)`, by hiding the real instance on it. +pub(super) fn chain_legacy_instance( + machine: &mut Machine<'_, H>, + instance: Value, + kind: IntlKind, + constructing: bool, + this: Value, +) -> Result { + if constructing || !inherits_from(machine, this, kind)? { + return Ok(BuiltinOutcome::Value(instance)); + } + let symbol = machine.intrinsics.builtins.intl_fallback_symbol(); + let key = machine.to_property_key(symbol)?; + machine.define_descriptor( + this, + key, + Property::Data { + value: instance, + writable: false, + enumerable: false, + configurable: false, + }, + )?; + Ok(BuiltinOutcome::Value(this)) +} + +/// UnwrapNumberFormat and UnwrapDateTimeFormat: the instance a legacy-chained +/// object hides, or the value itself. +pub(super) fn unwrap_legacy_instance( + machine: &mut Machine<'_, H>, + value: Value, + kind: IntlKind, +) -> Result { + let is_instance = intl_record(machine, value)?.is_some_and(|record| record.kind() == kind); + if is_instance || !inherits_from(machine, value, kind)? { + return Ok(value); + } + let symbol = machine.intrinsics.builtins.intl_fallback_symbol(); + let key = machine.to_property_key(symbol)?; + machine.get_property_key(value, &key) +} + +/// The cached function of a `format` or `compare` getter. The bound receiver +/// is what makes one function per instance, as those getters promise. +pub(super) fn bound_function( + machine: &mut Machine<'_, H>, + instance: Value, + builtin: &str, + length: u32, +) -> Result { + let index = machine + .runtime_slot(instance) + .map_err(EvalFailure::Runtime)? + .expect("the getter checked its receiver"); + if let HeapEntry::Intl { + bound: Some(function), + .. + } = &machine.heap[index] + { + return Ok(BuiltinOutcome::Value(*function)); + } + let id = machine + .intrinsics + .builtins + .id_named(builtin) + .expect("Intl registers its hidden functions"); + let target = machine + .allocate(HeapEntry::native_function( + NativeCallable::Builtin(id), + PropertyMap::default(), + None, + )) + .map_err(EvalFailure::Runtime)?; + let function = machine + .allocate(HeapEntry::native_function( + NativeCallable::Bound(Box::new(BoundCallable { + target, + this_value: instance, + arguments: Vec::new(), + })), + PropertyMap::default(), + None, + )) + .map_err(EvalFailure::Runtime)?; + let name = string_value(machine, "")?; + for (key, value) in [ + ("length", crate::number_value(f64::from(length))), + ("name", name), + ] { + machine.define_descriptor( + function, + PropertyKey::Named(EcmaString::encode(key)), + Property::Data { + value, + writable: false, + enumerable: false, + configurable: true, + }, + )?; + } + if let HeapEntry::Intl { bound, .. } = &mut machine.heap[index] { + *bound = Some(function); + } + Ok(BuiltinOutcome::Value(function)) +} + +/// `supportedLocalesOf` of every service. +pub(super) fn supported_locales_outcome( + machine: &mut Machine<'_, H>, + args: &[Value], +) -> Result { + let requested = options::canonicalize_locale_list(machine, arg(args, 0))?; + let options = options::coerce_options_to_object(machine, arg(args, 1))?; + let matcher = options::locale_matcher_option(machine, options)?; + let supported = supported_locales(&requested, matcher, &*DATA) + .map_err(|error| range_error(machine, error.to_string()))?; + string_array_outcome(machine, &supported) +} + +/// Allocates an instance with the class's own prototype. +pub(super) fn create( + machine: &mut Machine<'_, H>, + value: IntlObject, +) -> Result { + let prototype = machine.intrinsics.builtins.intl_prototype(value.kind()); + allocate(machine, value, prototype) +} + +/// Allocates the result of `new`, honouring `new.target`: an object whose +/// `prototype` is not an object gets the class's own prototype rather than +/// `Object.prototype`. +pub(super) fn construct( + machine: &mut Machine<'_, H>, + kind: IntlKind, + value: IntlObject, +) -> Result { + let new_target = machine.current_new_target(); + let prototype = machine.get_named_property(new_target, "prototype")?; + let prototype = if is_object(machine, prototype)? { + prototype + } else { + machine.intrinsics.builtins.intl_prototype(kind) + }; + allocate(machine, value, prototype).map(BuiltinOutcome::Value) +} + +pub(super) fn allocate( + machine: &mut Machine<'_, H>, + value: IntlObject, + prototype: Value, +) -> Result { + machine + .allocate(HeapEntry::Intl { + value: Box::new(value), + bound: None, + properties: PropertyMap::default(), + prototype: Some(prototype), + extensible: true, + }) + .map_err(EvalFailure::Runtime) +} + +/// Throws unless this call came through `new`. +pub(super) fn require_new( + machine: &mut Machine<'_, H>, + constructing: bool, + name: &str, +) -> Result<(), EvalFailure> { + if constructing { + Ok(()) + } else { + Err(type_error( + machine, + format!("Intl.{name} constructor requires new"), + )) + } +} + +pub(super) fn string_value( + machine: &mut Machine<'_, H>, + text: &str, +) -> Result { + allocate_string(machine, EcmaString::encode(text)) +} + +pub(super) fn string_outcome( + machine: &mut Machine<'_, H>, + text: &str, +) -> Result { + string_value(machine, text).map(BuiltinOutcome::Value) +} + +pub(super) fn optional_string_outcome( + machine: &mut Machine<'_, H>, + text: Option<&str>, +) -> Result { + match text { + Some(text) => string_outcome(machine, text), + None => Ok(BuiltinOutcome::Value(Value::UNDEFINED)), + } +} + +/// An ordinary object for a result. +pub(super) fn new_object(machine: &mut Machine<'_, H>) -> Result { + machine + .allocate(HeapEntry::Object { + properties: PropertyMap::default(), + prototype: Some(machine.intrinsics.object_prototype), + boxed_primitive: None, + extensible: true, + }) + .map_err(EvalFailure::Runtime) +} + +/// CreateDataProperty on a result object nothing else can see yet. +pub(super) fn define_field( + machine: &mut Machine<'_, H>, + object: Value, + name: &str, + value: Value, +) -> Result<(), EvalFailure> { + machine.create_data_property_key(object, PropertyKey::Named(EcmaString::encode(name)), value) +} + +/// One element of a `formatToParts` result. `source` and `unit` are the +/// properties some services add. +pub(super) fn part_object( + machine: &mut Machine<'_, H>, + part_type: &str, + value: &str, + extra: Option<(&str, &str)>, +) -> Result { + let part = new_object(machine)?; + let type_value = string_value(machine, part_type)?; + define_field(machine, part, "type", type_value)?; + let text = string_value(machine, value)?; + define_field(machine, part, "value", text)?; + if let Some((name, text)) = extra { + let text = string_value(machine, text)?; + define_field(machine, part, name, text)?; + } + Ok(part) +} + +/// An Array of strings, as `CreateArrayFromList` makes it. +pub(super) fn string_array( + machine: &mut Machine<'_, H>, + values: &[String], +) -> Result { + let mut elements = Vec::with_capacity(values.len()); + for value in values { + elements.push(string_value(machine, value)?); + } + allocate_array(machine, elements) +} + +pub(super) fn string_array_outcome( + machine: &mut Machine<'_, H>, + values: &[String], +) -> Result { + string_array(machine, values).map(BuiltinOutcome::Value) +} + +/// The `type` production of a Unicode locale extension. +pub(super) fn is_type_sequence(value: &str) -> bool { + !value.is_empty() + && value.split('-').all(|part| { + (3..=8).contains(&part.len()) && part.bytes().all(|byte| byte.is_ascii_alphanumeric()) + }) +} + +/// Sets a symbol-keyed method, as a data property like any other. +fn define_symbol_method(heap: &mut [HeapEntry], object: Value, symbol: Value, function: Value) { + let symbol = super::heap_index(symbol); + let HeapEntry::Object { properties, .. } = &mut heap[super::heap_index(object)] else { + panic!("method target is an ordinary object"); + }; + properties.insert( + PropertyKey::Symbol(symbol as u32), + Property::Data { + value: function, + writable: true, + enumerable: false, + configurable: true, + }, + ); +} + +fn set_prototype(heap: &mut [HeapEntry], object: Value, parent: Value) { + let HeapEntry::Object { prototype, .. } = &mut heap[super::heap_index(object)] else { + panic!("prototype target is an ordinary object"); + }; + *prototype = Some(parent); +} + +fn define_tag(heap: &mut Vec, object: Value, symbol: Value, tag: &str) { + super::define_to_string_tag(heap, object, symbol, tag); +} + +/// StringListFromIterable. +pub(super) fn string_iterable_to_list( + machine: &mut Machine<'_, H>, + iterable: Value, +) -> Result, EvalFailure> { + let mut list = Vec::new(); + if iterable == Value::UNDEFINED { + return Ok(list); + } + let key = machine.to_property_key(machine.intrinsics.builtins.symbol_iterator())?; + let object = machine.value_to_object(iterable)?; + let method = machine.get_property_key(object, &key)?; + if !machine.is_callable(method)? { + return Err(type_error(machine, "value is not iterable")); + } + let target = machine.call_value(method, iterable, &[])?; + if !is_object(machine, target)? { + return Err(type_error(machine, "iterator method returned a non-object")); + } + let next = machine.get_named_property(target, "next")?; + let iterator = machine.create_protocol_iterator(target, next)?; + let depth = machine.native_roots.len(); + machine.push_native_roots(depth, &[iterator]); + let result = (|| loop { + let step = machine.iterator_step(iterator)?; + let (done, value) = machine.iterator_result_parts(step)?; + if done { + return Ok(()); + } + let Some(text) = machine.string_value(value) else { + let failure = type_error(machine, "a list element must be a string"); + let _ = machine.close_iterator_raw(iterator); + return Err(failure); + }; + list.push(text.to_utf8_lossy()); + })(); + machine.pop_native_roots(depth); + result?; + Ok(list) +} + +/// StrWhiteSpaceChar. +pub(super) fn is_string_whitespace(character: char) -> bool { + matches!( + character, + '\u{9}' | '\u{a}' | '\u{b}' | '\u{c}' | '\u{d}' | ' ' | '\u{a0}' | '\u{1680}' | '\u{2000}' + ..='\u{200a}' + | '\u{2028}' + | '\u{2029}' + | '\u{202f}' + | '\u{205f}' + | '\u{3000}' + | '\u{feff}' + ) +} + +pub(super) fn boolean_outcome(value: bool) -> BuiltinOutcome { + BuiltinOutcome::Value(Value::boolean(value)) +} + +/// The argument at `index`, `undefined` when absent. +pub(super) fn arg(args: &[Value], index: usize) -> Value { + args.get(index).copied().unwrap_or(Value::UNDEFINED) +} diff --git a/crates/bamts-runtime/src/builtins/intl/number_format.rs b/crates/bamts-runtime/src/builtins/intl/number_format.rs new file mode 100644 index 00000000..d1233316 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/number_format.rs @@ -0,0 +1,565 @@ +//! `Intl.NumberFormat`. + +use bamts_native::{Decoded, Value}; + +use super::super::bigint::bigint_from_value; +use super::options::{ + Options, canonicalize_locale_list, coerce_options_to_object, digit_options, + locale_matcher_option, matcher_name, read, string_option, +}; +use super::{ + Class, allocate, arg, bound_function, chain_legacy_instance, define_field, install_class, + instance_prototype, intl_record, is_type_sequence, new_object, part_object, range_error, + register_hidden, string_outcome, string_value, supported_locales_outcome, type_error, + unwrap_legacy_instance, +}; +use crate::intl::locale_data::DATA; +use crate::intl::number_data::NoHostLocale; +use crate::intl::number_format::{ + IntlMathematicalValue, NumberFormat, NumberFormatError, NumberFormatErrorKind, + NumberFormatOptions, NumberPart, UseGrouping, UseGroupingOption, +}; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +const FORMAT: &str = "Intl.NumberFormat format"; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + register_hidden(builtins, FORMAT, 1, format_value::); + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::NumberFormat, + name: "NumberFormat", + tag: "Intl.NumberFormat", + length: 0, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[("format", "get format", format_getter::)], + methods: &[ + ("formatRange", 2, format_range::), + ("formatRangeToParts", 2, format_range_to_parts::), + ("formatToParts", 1, format_to_parts::), + ("resolvedOptions", 0, resolved_options::), + ], + }, + ); +} + +pub(super) fn number_failure( + machine: &mut Machine<'_, H>, + error: NumberFormatError, +) -> EvalFailure { + match error.kind { + NumberFormatErrorKind::TypeError => type_error(machine, error.message), + NumberFormatErrorKind::RangeError + | NumberFormatErrorKind::MissingData + | NumberFormatErrorKind::InvalidData => range_error(machine, error.message), + } +} + +/// SetNumberFormatUnitOptions, which validates each value as it is read. +fn unit_options( + machine: &mut Machine<'_, H>, + options: Options, + result: &mut NumberFormatOptions, +) -> Result<(), EvalFailure> { + result.style = string_option( + machine, + options, + "style", + &["decimal", "percent", "currency", "unit"], + )?; + let style = result.style.clone(); + result.currency = string_option(machine, options, "currency", &[])?; + match &result.currency { + None if style.as_deref() == Some("currency") => { + return Err(type_error(machine, "currency is required")); + } + Some(code) if code.len() != 3 || !code.bytes().all(|byte| byte.is_ascii_alphabetic()) => { + let message = format!("{code} is not a well-formed currency code"); + return Err(range_error(machine, message)); + } + _ => {} + } + result.currency_display = string_option( + machine, + options, + "currencyDisplay", + &["code", "symbol", "narrowSymbol", "name"], + )?; + result.currency_sign = string_option( + machine, + options, + "currencySign", + &["standard", "accounting"], + )?; + result.unit = string_option(machine, options, "unit", &[])?; + match &result.unit { + None if style.as_deref() == Some("unit") => { + return Err(type_error(machine, "unit is required")); + } + Some(unit) if !crate::intl::number_format::is_well_formed_unit(unit) => { + let message = format!("{unit} is not a well-formed unit identifier"); + return Err(range_error(machine, message)); + } + _ => {} + } + result.unit_display = string_option( + machine, + options, + "unitDisplay", + &["short", "narrow", "long"], + )?; + Ok(()) +} + +/// GetBooleanOrStringNumberFormatOption for `useGrouping`. +fn use_grouping_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result, EvalFailure> { + let value = read(machine, options, "useGrouping")?; + if value == Value::UNDEFINED { + return Ok(None); + } + if value == Value::boolean(true) { + return Ok(Some(UseGroupingOption::Boolean(true))); + } + if !machine.to_boolean(value) { + return Ok(Some(UseGroupingOption::Boolean(false))); + } + let text = machine.coerce_string_observable(value)?.to_utf8_lossy(); + if ["min2", "auto", "always", "true", "false"].contains(&text.as_str()) { + Ok(Some(UseGroupingOption::String(text))) + } else { + Err(range_error( + machine, + format!("{text} is not a valid value for useGrouping"), + )) + } +} + +/// The options of `new Intl.NumberFormat`, read in constructor order. +pub(super) fn read_number_format_options( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let mut result = NumberFormatOptions::default(); + let matcher = locale_matcher_option(machine, options)?; + result.locale_matcher = Some(matcher_name(matcher).to_owned()); + result.numbering_system = string_option(machine, options, "numberingSystem", &[])?; + if let Some(name) = &result.numbering_system + && !is_type_sequence(name) + { + let message = format!("{name} is not a valid numbering system"); + return Err(range_error(machine, message)); + } + unit_options(machine, options, &mut result)?; + result.notation = string_option( + machine, + options, + "notation", + &["standard", "scientific", "engineering", "compact"], + )?; + let compact = result.notation.as_deref() == Some("compact"); + let digits = digit_options(machine, options, compact)?; + result.minimum_integer_digits = digits.minimum_integer_digits; + result.minimum_fraction_digits = digits.minimum_fraction_digits; + result.maximum_fraction_digits = digits.maximum_fraction_digits; + result.minimum_significant_digits = digits.minimum_significant_digits; + result.maximum_significant_digits = digits.maximum_significant_digits; + result.rounding_increment = digits.rounding_increment; + result.rounding_mode = digits.rounding_mode; + result.rounding_priority = digits.rounding_priority; + result.trailing_zero_display = digits.trailing_zero_display; + result.compact_display = string_option(machine, options, "compactDisplay", &["short", "long"])?; + result.use_grouping = use_grouping_option(machine, options)?; + result.sign_display = string_option( + machine, + options, + "signDisplay", + &["auto", "never", "always", "exceptZero", "negative"], + )?; + Ok(result) +} + +/// CreateNumberFormat over already canonicalized locales. +pub(super) fn create_number_format( + machine: &mut Machine<'_, H>, + requested: &[String], + options: &NumberFormatOptions, +) -> Result, EvalFailure> { + let data: &'static crate::intl::locale_data::IntlData = &DATA; + NumberFormat::new(requested, options, data, &NoHostLocale) + .map_err(|error| number_failure(machine, error)) +} + +fn constructor( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + constructing: bool, +) -> Result { + let prototype = instance_prototype(machine, IntlKind::NumberFormat, constructing)?; + let requested = canonicalize_locale_list(machine, arg(args, 0))?; + let options = coerce_options_to_object(machine, arg(args, 1))?; + let options = read_number_format_options(machine, options)?; + let format = create_number_format(machine, &requested, &options)?; + let instance = allocate( + machine, + IntlObject::NumberFormat(Box::new(format)), + prototype, + )?; + chain_legacy_instance( + machine, + instance, + IntlKind::NumberFormat, + constructing, + this, + ) +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn this_format<'a, H: Host>( + machine: &'a Machine<'_, H>, + this: Value, +) -> Result>, EvalFailure> { + Ok(match intl_record(machine, this)? { + Some(IntlObject::NumberFormat(format)) => Some(format), + _ => None, + }) +} + +fn not_a_number_format(machine: &mut Machine<'_, H>) -> EvalFailure { + type_error(machine, "receiver is not an Intl.NumberFormat") +} + +fn format_getter( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let format = unwrap_legacy_instance(machine, this, IntlKind::NumberFormat)?; + if this_format(machine, format)?.is_none() { + return Err(not_a_number_format(machine)); + } + bound_function(machine, format, FORMAT, 1) +} + +/// The value of a StringNumericLiteral as an exact number, as +/// ToIntlMathematicalValue reads it: magnitudes a Number cannot hold saturate. +fn string_mathematical_value(text: &str) -> IntlMathematicalValue { + let trimmed = text.trim_matches(super::is_string_whitespace); + if trimmed.is_empty() { + return IntlMathematicalValue::from_f64(0.0); + } + let (negative, body) = match trimmed.as_bytes()[0] { + b'-' => (true, &trimmed[1..]), + b'+' => (false, &trimmed[1..]), + _ => (false, trimmed), + }; + if body == "Infinity" { + return IntlMathematicalValue::from_f64(if negative { + f64::NEG_INFINITY + } else { + f64::INFINITY + }); + } + let radix = match body.get(..2) { + Some("0x" | "0X") => 16, + Some("0o" | "0O") => 8, + Some("0b" | "0B") => 2, + _ => 10, + }; + if radix != 10 { + let unsigned = trimmed.len() == body.len(); + return match (unsigned, radix_digits(&body[2..], radix)) { + (true, Some(decimal)) => IntlMathematicalValue::from_bigint(&decimal) + .unwrap_or_else(|_| IntlMathematicalValue::from_f64(f64::NAN)), + _ => IntlMathematicalValue::from_f64(f64::NAN), + }; + } + if !is_decimal_literal(body) { + return IntlMathematicalValue::from_f64(f64::NAN); + } + let approximate: f64 = body.parse().unwrap_or(f64::NAN); + if approximate.is_infinite() || approximate == 0.0 { + let saturated = if negative { -approximate } else { approximate }; + return IntlMathematicalValue::from_f64(saturated); + } + IntlMathematicalValue::from_decimal_string(trimmed) + .unwrap_or_else(|_| IntlMathematicalValue::from_f64(f64::NAN)) +} + +/// Converts digits of `radix` to a decimal string, by long multiplication. +fn radix_digits(digits: &str, radix: u32) -> Option { + if digits.is_empty() { + return None; + } + let mut decimal: Vec = vec![0]; + for character in digits.chars() { + let mut carry = character.to_digit(radix)?; + for digit in decimal.iter_mut().rev() { + let value = *digit * radix + carry; + *digit = value % 10; + carry = value / 10; + } + while carry > 0 { + decimal.insert(0, carry % 10); + carry /= 10; + } + } + decimal + .iter() + .map(|digit| char::from_digit(*digit, 10)) + .collect() +} + +/// StrUnsignedDecimalLiteral without `Infinity`. +fn is_decimal_literal(text: &str) -> bool { + let (mantissa, exponent) = match text.find(['e', 'E']) { + Some(index) => (&text[..index], Some(&text[index + 1..])), + None => (text, None), + }; + let (whole, fraction) = mantissa.split_once('.').unwrap_or((mantissa, "")); + let digits = |part: &str| part.bytes().all(|byte| byte.is_ascii_digit()); + if !digits(whole) || !digits(fraction) || (whole.is_empty() && fraction.is_empty()) { + return false; + } + exponent.is_none_or(|exponent| { + let exponent = exponent.strip_prefix(['+', '-']).unwrap_or(exponent); + !exponent.is_empty() && digits(exponent) + }) +} + +/// ToIntlMathematicalValue. +pub(super) fn to_mathematical_value( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let primitive = machine.coerce_primitive_observable(value, false)?; + if let Some(bigint) = bigint_from_value(machine, primitive)? { + return IntlMathematicalValue::from_bigint(&bigint.to_string_radix(10)) + .map_err(|error| number_failure(machine, error)); + } + if let Some(text) = machine.string_value(primitive) { + return Ok(string_mathematical_value(&text.to_utf8_lossy())); + } + let number = machine.coerce_number_observable(primitive)?; + let number = match number.decode() { + Some(Decoded::Int32(number)) => f64::from(number as i32), + Some(Decoded::Number(number)) => number, + _ => f64::NAN, + }; + Ok(IntlMathematicalValue::from_f64(number)) +} + +/// The function `format` hands out, bound to its NumberFormat. +fn format_value( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = to_mathematical_value(machine, arg(args, 0))?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_number_format(machine)); + }; + let text = format.format(&value); + let text = text.map_err(|error| number_failure(machine, error))?; + string_outcome(machine, &text) +} + +fn format_to_parts( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if this_format(machine, this)?.is_none() { + return Err(not_a_number_format(machine)); + } + let value = to_mathematical_value(machine, arg(args, 0))?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_number_format(machine)); + }; + let parts = format + .format_to_parts(&value) + .map_err(|error| number_failure(machine, error))?; + parts_outcome(machine, &parts) +} + +pub(super) fn parts_outcome( + machine: &mut Machine<'_, H>, + parts: &[NumberPart], +) -> Result { + let mut elements = Vec::with_capacity(parts.len()); + for part in parts { + elements.push(part_object( + machine, + part.part_type.as_str(), + &part.value, + None, + )?); + } + super::super::allocate_array(machine, elements).map(BuiltinOutcome::Value) +} + +/// The argument handling `formatRange` and `formatRangeToParts` share. +fn range_arguments( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], +) -> Result<(IntlMathematicalValue, IntlMathematicalValue), EvalFailure> { + if this_format(machine, this)?.is_none() { + return Err(not_a_number_format(machine)); + } + let (start, end) = (arg(args, 0), arg(args, 1)); + if start == Value::UNDEFINED || end == Value::UNDEFINED { + return Err(type_error(machine, "a range needs two ends")); + } + let start = to_mathematical_value(machine, start)?; + let end = to_mathematical_value(machine, end)?; + Ok((start, end)) +} + +fn format_range( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (start, end) = range_arguments(machine, this, args)?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_number_format(machine)); + }; + let text = format.format_range(&start, &end); + let text = text.map_err(|error| number_failure(machine, error))?; + string_outcome(machine, &text) +} + +fn format_range_to_parts( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (start, end) = range_arguments(machine, this, args)?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_number_format(machine)); + }; + let parts = format + .format_range_to_parts(&start, &end) + .map_err(|error| number_failure(machine, error))?; + let mut elements = Vec::with_capacity(parts.len()); + for part in &parts { + elements.push(part_object( + machine, + part.part_type.as_str(), + &part.value, + Some(("source", part.source.as_str())), + )?); + } + super::super::allocate_array(machine, elements).map(BuiltinOutcome::Value) +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let format = unwrap_legacy_instance(machine, this, IntlKind::NumberFormat)?; + let Some(format) = this_format(machine, format)? else { + return Err(not_a_number_format(machine)); + }; + let resolved = format.resolved_options(); + let result = new_object(machine)?; + let text = |machine: &mut Machine<'_, H>, name: &str, value: &str| { + let value = string_value(machine, value)?; + define_field(machine, result, name, value) + }; + let number = |machine: &mut Machine<'_, H>, name: &str, value: u16| { + define_field(machine, result, name, crate::number_value(f64::from(value))) + }; + text(machine, "locale", &resolved.locale)?; + text(machine, "numberingSystem", &resolved.numbering_system)?; + text(machine, "style", resolved.style.as_str())?; + if let Some(currency) = &resolved.currency { + text(machine, "currency", currency)?; + } + if let Some(display) = resolved.currency_display { + text(machine, "currencyDisplay", display.as_str())?; + } + if let Some(sign) = resolved.currency_sign { + text(machine, "currencySign", sign.as_str())?; + } + if let Some(unit) = &resolved.unit { + text(machine, "unit", unit)?; + } + if let Some(display) = resolved.unit_display { + text(machine, "unitDisplay", display.as_str())?; + } + number( + machine, + "minimumIntegerDigits", + u16::from(resolved.minimum_integer_digits), + )?; + for (name, value) in [ + ("minimumFractionDigits", resolved.minimum_fraction_digits), + ("maximumFractionDigits", resolved.maximum_fraction_digits), + ( + "minimumSignificantDigits", + resolved.minimum_significant_digits, + ), + ( + "maximumSignificantDigits", + resolved.maximum_significant_digits, + ), + ] { + if let Some(value) = value { + number(machine, name, u16::from(value))?; + } + } + match resolved.use_grouping { + UseGrouping::False => define_field(machine, result, "useGrouping", Value::boolean(false))?, + UseGrouping::Min2 => text(machine, "useGrouping", "min2")?, + UseGrouping::Auto => text(machine, "useGrouping", "auto")?, + UseGrouping::Always => text(machine, "useGrouping", "always")?, + } + text(machine, "notation", resolved.notation.as_str())?; + if let Some(display) = resolved.compact_display { + text(machine, "compactDisplay", display.as_str())?; + } + text(machine, "signDisplay", resolved.sign_display.as_str())?; + number(machine, "roundingIncrement", resolved.rounding_increment)?; + text(machine, "roundingMode", resolved.rounding_mode.as_str())?; + text( + machine, + "roundingPriority", + resolved.rounding_priority.as_str(), + )?; + text( + machine, + "trailingZeroDisplay", + resolved.trailing_zero_display.as_str(), + )?; + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/options.rs b/crates/bamts-runtime/src/builtins/intl/options.rs new file mode 100644 index 00000000..42f19b94 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/options.rs @@ -0,0 +1,315 @@ +//! Option reading. Every function here performs exactly the property reads and +//! conversions its abstract operation does; callers sequence them in the order +//! ECMA-402 lists them, which test262 observes through getters and Proxies. + +use bamts_native::{Decoded, Value}; + +use super::{is_object, locale_record, range_error, type_error}; +use crate::intl::locale_data::DATA; +use crate::intl::locale_negotiation::{LocaleMatcher, canonicalize_unicode_locale_id}; +use crate::intrinsics::builtins::array_es2023::{index_key, length_of_array_like}; +use crate::{EvalFailure, Host, Machine}; + +/// A resolved options argument. `None` stands for the empty null-prototype +/// object of `undefined`, which no read can observe. +pub(super) type Options = Option; + +/// GetOptionsObject. +pub(super) fn get_options_object( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + if value == Value::UNDEFINED { + Ok(None) + } else if is_object(machine, value)? { + Ok(Some(value)) + } else { + Err(type_error(machine, "options must be an object")) + } +} + +/// CoerceOptionsToObject. +pub(super) fn coerce_options_to_object( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + if value == Value::UNDEFINED { + Ok(None) + } else { + machine.value_to_object(value).map(Some) + } +} + +/// The value of one option, `undefined` when absent. +pub(super) fn read( + machine: &mut Machine<'_, H>, + options: Options, + name: &str, +) -> Result { + match options { + Some(object) => machine.get_named_property(object, name), + None => Ok(Value::UNDEFINED), + } +} + +/// GetOption with type string and no default. An empty `allowed` accepts any +/// value. +pub(super) fn string_option( + machine: &mut Machine<'_, H>, + options: Options, + name: &str, + allowed: &[&str], +) -> Result, EvalFailure> { + let value = read(machine, options, name)?; + if value == Value::UNDEFINED { + return Ok(None); + } + let text = machine.coerce_string_observable(value)?.to_utf8_lossy(); + if allowed.is_empty() || allowed.contains(&text.as_str()) { + Ok(Some(text)) + } else { + Err(range_error( + machine, + format!("{text} is not a valid value for {name}"), + )) + } +} + +/// GetOption with type boolean and no default. +pub(super) fn boolean_option( + machine: &mut Machine<'_, H>, + options: Options, + name: &str, +) -> Result, EvalFailure> { + let value = read(machine, options, name)?; + if value == Value::UNDEFINED { + Ok(None) + } else { + Ok(Some(machine.to_boolean(value))) + } +} + +/// The `localeMatcher` option every constructor reads. +pub(super) fn locale_matcher_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = string_option(machine, options, "localeMatcher", &["lookup", "best fit"])?; + Ok(if value.as_deref() == Some("lookup") { + LocaleMatcher::Lookup + } else { + LocaleMatcher::BestFit + }) +} + +/// The library's spelling of a matcher. +pub(super) const fn matcher_name(matcher: LocaleMatcher) -> &'static str { + match matcher { + LocaleMatcher::Lookup => "lookup", + LocaleMatcher::BestFit => "best fit", + } +} + +/// DefaultNumberOption. +pub(super) fn default_number_option( + machine: &mut Machine<'_, H>, + value: Value, + minimum: f64, + maximum: f64, + name: &str, +) -> Result, EvalFailure> { + if value == Value::UNDEFINED { + return Ok(None); + } + let number = machine.coerce_number_observable(value)?; + let number = match number.decode() { + Some(Decoded::Int32(number)) => f64::from(number as i32), + Some(Decoded::Number(number)) => number, + _ => f64::NAN, + }; + if !number.is_finite() || number < minimum || number > maximum { + return Err(range_error(machine, format!("{name} is out of range"))); + } + Ok(Some(number.floor())) +} + +/// GetNumberOption. +pub(super) fn number_option( + machine: &mut Machine<'_, H>, + options: Options, + name: &str, + minimum: f64, + maximum: f64, +) -> Result, EvalFailure> { + let value = read(machine, options, name)?; + default_number_option(machine, value, minimum, maximum, name) +} + +/// CanonicalizeLocaleList. Elements are read and validated one at a time, so +/// an invalid tag stops the walk before any later element is touched. +pub(super) fn canonicalize_locale_list( + machine: &mut Machine<'_, H>, + locales: Value, +) -> Result, EvalFailure> { + let mut seen = Vec::new(); + if locales == Value::UNDEFINED { + return Ok(seen); + } + if machine.string_value(locales).is_some() || locale_record(machine, locales)?.is_some() { + add_locale(machine, &mut seen, locales)?; + return Ok(seen); + } + let object = machine.value_to_object(locales)?; + let length = length_of_array_like(machine, object)?; + for index in 0..length { + let key = index_key(index); + if !machine.internal_has_property(object, &key)? { + continue; + } + let element = machine.get_property_key(object, &key)?; + if machine.string_value(element).is_none() && !is_object(machine, element)? { + return Err(type_error( + machine, + "a locale must be a string or an object", + )); + } + add_locale(machine, &mut seen, element)?; + } + Ok(seen) +} + +fn add_locale( + machine: &mut Machine<'_, H>, + seen: &mut Vec, + element: Value, +) -> Result<(), EvalFailure> { + let tag = match locale_record(machine, element)? { + Some(tag) => tag.to_string(), + None => machine.coerce_string_observable(element)?.to_utf8_lossy(), + }; + let canonical = canonicalize_unicode_locale_id(&tag, &*DATA) + .map_err(|error| range_error(machine, error.to_string()))?; + if !seen.contains(&canonical) { + seen.push(canonical); + } + Ok(()) +} + +/// What SetNumberFormatDigitOptions reads, shared by the services that round. +/// Only the values the resolution needs have gone through ToNumber. +pub(super) struct DigitOptions { + pub(super) minimum_integer_digits: Option, + pub(super) minimum_fraction_digits: Option, + pub(super) maximum_fraction_digits: Option, + pub(super) minimum_significant_digits: Option, + pub(super) maximum_significant_digits: Option, + pub(super) rounding_increment: Option, + pub(super) rounding_mode: Option, + pub(super) rounding_priority: Option, + pub(super) trailing_zero_display: Option, +} + +const ROUNDING_INCREMENTS: [f64; 15] = [ + 1.0, 2.0, 5.0, 10.0, 20.0, 25.0, 50.0, 100.0, 200.0, 250.0, 500.0, 1000.0, 2000.0, 2500.0, + 5000.0, +]; + +/// The reads of SetNumberFormatDigitOptions, in order. The four digit limits +/// are fetched early but converted late, and only when rounding uses them. +pub(super) fn digit_options( + machine: &mut Machine<'_, H>, + options: Options, + compact: bool, +) -> Result { + let minimum_integer_digits = + number_option(machine, options, "minimumIntegerDigits", 1.0, 21.0)?; + let minimum_fraction = read(machine, options, "minimumFractionDigits")?; + let maximum_fraction = read(machine, options, "maximumFractionDigits")?; + let minimum_significant = read(machine, options, "minimumSignificantDigits")?; + let maximum_significant = read(machine, options, "maximumSignificantDigits")?; + let rounding_increment = number_option(machine, options, "roundingIncrement", 1.0, 5000.0)?; + if rounding_increment.is_some_and(|increment| !ROUNDING_INCREMENTS.contains(&increment)) { + return Err(range_error(machine, "roundingIncrement is not allowed")); + } + let rounding_mode = string_option( + machine, + options, + "roundingMode", + &[ + "ceil", + "floor", + "expand", + "trunc", + "halfCeil", + "halfFloor", + "halfExpand", + "halfTrunc", + "halfEven", + ], + )?; + let rounding_priority = string_option( + machine, + options, + "roundingPriority", + &["auto", "morePrecision", "lessPrecision"], + )?; + let trailing_zero_display = string_option( + machine, + options, + "trailingZeroDisplay", + &["auto", "stripIfInteger"], + )?; + let has_significant = + minimum_significant != Value::UNDEFINED || maximum_significant != Value::UNDEFINED; + let has_fraction = minimum_fraction != Value::UNDEFINED || maximum_fraction != Value::UNDEFINED; + let automatic = matches!(rounding_priority.as_deref(), None | Some("auto")); + let need_significant = !automatic || has_significant; + let need_fraction = !automatic || !(has_significant || (!has_fraction && compact)); + let (mut minimum_significant_digits, mut maximum_significant_digits) = (None, None); + if need_significant && has_significant { + let minimum = default_number_option( + machine, + minimum_significant, + 1.0, + 21.0, + "minimumSignificantDigits", + )?; + let maximum = default_number_option( + machine, + maximum_significant, + minimum.unwrap_or(1.0), + 21.0, + "maximumSignificantDigits", + )?; + minimum_significant_digits = minimum; + maximum_significant_digits = maximum; + } + let (mut minimum_fraction_digits, mut maximum_fraction_digits) = (None, None); + if need_fraction && has_fraction { + minimum_fraction_digits = default_number_option( + machine, + minimum_fraction, + 0.0, + 100.0, + "minimumFractionDigits", + )?; + maximum_fraction_digits = default_number_option( + machine, + maximum_fraction, + 0.0, + 100.0, + "maximumFractionDigits", + )?; + } + Ok(DigitOptions { + minimum_integer_digits, + minimum_fraction_digits, + maximum_fraction_digits, + minimum_significant_digits, + maximum_significant_digits, + rounding_increment, + rounding_mode, + rounding_priority, + trailing_zero_display, + }) +} diff --git a/crates/bamts-runtime/src/builtins/intl/plural_rules.rs b/crates/bamts-runtime/src/builtins/intl/plural_rules.rs new file mode 100644 index 00000000..ea708bc6 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/plural_rules.rs @@ -0,0 +1,234 @@ +//! `Intl.PluralRules`. + +use bamts_native::{Decoded, Value}; + +use super::options::{ + canonicalize_locale_list, coerce_options_to_object, digit_options, locale_matcher_option, + matcher_name, string_option, +}; +use super::{ + Class, allocate, arg, define_field, install_class, instance_prototype, intl_record, new_object, + range_error, require_new, service_failure, string_array, string_outcome, string_value, + supported_locales_outcome, type_error, +}; +use crate::intl::locale_data::{DATA, IntlData}; +use crate::intl::number_data::NoHostLocale; +use crate::intl::plural_rules::{PluralRules, PluralRulesOptions}; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::PluralRules, + name: "PluralRules", + tag: "Intl.PluralRules", + length: 0, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[], + methods: &[ + ("resolvedOptions", 0, resolved_options::), + ("select", 1, select::), + ("selectRange", 2, select_range::), + ], + }, + ); +} + +fn to_u8(value: Option) -> Option { + value.map(|value| value as u8) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "PluralRules")?; + let prototype = instance_prototype(machine, IntlKind::PluralRules, constructing)?; + let requested = canonicalize_locale_list(machine, arg(args, 0))?; + let options = coerce_options_to_object(machine, arg(args, 1))?; + let matcher = locale_matcher_option(machine, options)?; + let plural_type = string_option(machine, options, "type", &["cardinal", "ordinal"])?; + let notation = string_option( + machine, + options, + "notation", + &["standard", "scientific", "engineering", "compact"], + )?; + let compact_display = string_option(machine, options, "compactDisplay", &["short", "long"])?; + let compact = notation.as_deref() == Some("compact"); + let digits = digit_options(machine, options, compact)?; + let resolved = PluralRulesOptions { + locale_matcher: Some(matcher_name(matcher).to_owned()), + plural_type, + minimum_integer_digits: to_u8(digits.minimum_integer_digits), + minimum_fraction_digits: to_u8(digits.minimum_fraction_digits), + maximum_fraction_digits: to_u8(digits.maximum_fraction_digits), + minimum_significant_digits: to_u8(digits.minimum_significant_digits), + maximum_significant_digits: to_u8(digits.maximum_significant_digits), + rounding_priority: digits.rounding_priority, + rounding_increment: digits.rounding_increment.map(|value| value as u16), + rounding_mode: digits.rounding_mode, + trailing_zero_display: digits.trailing_zero_display, + notation, + compact_display, + }; + let data: &'static IntlData = &DATA; + let rules = PluralRules::try_new(&requested, &resolved, data, &NoHostLocale) + .map_err(|error| service_failure(machine, &error))?; + allocate(machine, IntlObject::PluralRules(rules), prototype).map(BuiltinOutcome::Value) +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn this_rules<'a, H: Host>( + machine: &'a Machine<'_, H>, + this: Value, +) -> Result>, EvalFailure> { + Ok(match intl_record(machine, this)? { + Some(IntlObject::PluralRules(rules)) => Some(rules), + _ => None, + }) +} + +fn not_plural_rules(machine: &mut Machine<'_, H>) -> EvalFailure { + type_error(machine, "receiver is not an Intl.PluralRules") +} + +fn to_number(machine: &mut Machine<'_, H>, value: Value) -> Result { + let number = machine.coerce_number_observable(value)?; + Ok(match number.decode() { + Some(Decoded::Int32(number)) => f64::from(number as i32), + Some(Decoded::Number(number)) => number, + _ => f64::NAN, + }) +} + +fn select( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if this_rules(machine, this)?.is_none() { + return Err(not_plural_rules(machine)); + } + let number = to_number(machine, arg(args, 0))?; + let Some(rules) = this_rules(machine, this)? else { + return Err(not_plural_rules(machine)); + }; + let category = rules.select(number); + let category = category.map_err(|error| service_failure(machine, &error))?; + string_outcome(machine, category.as_str()) +} + +fn select_range( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if this_rules(machine, this)?.is_none() { + return Err(not_plural_rules(machine)); + } + let (start, end) = (arg(args, 0), arg(args, 1)); + if start == Value::UNDEFINED || end == Value::UNDEFINED { + return Err(type_error(machine, "a range needs two ends")); + } + let start = to_number(machine, start)?; + let end = to_number(machine, end)?; + if start.is_nan() || end.is_nan() { + return Err(range_error(machine, "a range end is not a number")); + } + let Some(rules) = this_rules(machine, this)? else { + return Err(not_plural_rules(machine)); + }; + let category = rules.select_range(start, end); + let category = category.map_err(|error| service_failure(machine, &error))?; + string_outcome(machine, category.as_str()) +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let Some(rules) = this_rules(machine, this)? else { + return Err(not_plural_rules(machine)); + }; + let resolved = rules.resolved_options(); + let result = new_object(machine)?; + let text = |machine: &mut Machine<'_, H>, name: &str, value: &str| { + let value = string_value(machine, value)?; + define_field(machine, result, name, value) + }; + let number = |machine: &mut Machine<'_, H>, name: &str, value: u16| { + define_field(machine, result, name, crate::number_value(f64::from(value))) + }; + text(machine, "locale", &resolved.locale)?; + text(machine, "type", resolved.plural_type.as_str())?; + text(machine, "notation", &resolved.notation)?; + if let Some(display) = &resolved.compact_display { + text(machine, "compactDisplay", display)?; + } + number( + machine, + "minimumIntegerDigits", + u16::from(resolved.minimum_integer_digits), + )?; + for (name, value) in [ + ("minimumFractionDigits", resolved.minimum_fraction_digits), + ("maximumFractionDigits", resolved.maximum_fraction_digits), + ( + "minimumSignificantDigits", + resolved.minimum_significant_digits, + ), + ( + "maximumSignificantDigits", + resolved.maximum_significant_digits, + ), + ] { + if let Some(value) = value { + number(machine, name, u16::from(value))?; + } + } + let categories: Vec = resolved + .plural_categories + .iter() + .map(|category| category.as_str().to_owned()) + .collect(); + let categories = string_array(machine, &categories)?; + define_field(machine, result, "pluralCategories", categories)?; + number(machine, "roundingIncrement", resolved.rounding_increment)?; + text(machine, "roundingMode", resolved.rounding_mode.as_str())?; + text( + machine, + "roundingPriority", + resolved.rounding_priority.as_str(), + )?; + text( + machine, + "trailingZeroDisplay", + resolved.trailing_zero_display.as_str(), + )?; + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/relative_time_format.rs b/crates/bamts-runtime/src/builtins/intl/relative_time_format.rs new file mode 100644 index 00000000..16d134f8 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/relative_time_format.rs @@ -0,0 +1,207 @@ +//! `Intl.RelativeTimeFormat`. + +use bamts_native::{Decoded, Value}; + +use super::super::allocate_array; +use super::options::{ + canonicalize_locale_list, coerce_options_to_object, locale_matcher_option, matcher_name, + string_option, +}; +use super::{ + Class, allocate, arg, define_field, install_class, instance_prototype, intl_record, + is_type_sequence, new_object, part_object, range_error, require_new, service_failure, + string_outcome, string_value, supported_locales_outcome, type_error, +}; +use crate::intl::locale_data::{DATA, IntlData}; +use crate::intl::number_data::NoHostLocale; +use crate::intl::plural_rules::{ + NumberPartKind, RelativeTimeFormat, RelativeTimeFormatOptions, RelativeTimePartKind, +}; +use crate::intl::{IntlKind, IntlObject}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::RelativeTimeFormat, + name: "RelativeTimeFormat", + tag: "Intl.RelativeTimeFormat", + length: 0, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[], + methods: &[ + ("format", 2, format::), + ("formatToParts", 2, format_to_parts::), + ("resolvedOptions", 0, resolved_options::), + ], + }, + ); +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "RelativeTimeFormat")?; + let prototype = instance_prototype(machine, IntlKind::RelativeTimeFormat, constructing)?; + let requested = canonicalize_locale_list(machine, arg(args, 0))?; + let options = coerce_options_to_object(machine, arg(args, 1))?; + let matcher = locale_matcher_option(machine, options)?; + let numbering_system = string_option(machine, options, "numberingSystem", &[])?; + if let Some(name) = &numbering_system + && !is_type_sequence(name) + { + let message = format!("{name} is not a valid numbering system"); + return Err(range_error(machine, message)); + } + let style = string_option(machine, options, "style", &["long", "short", "narrow"])?; + let numeric = string_option(machine, options, "numeric", &["always", "auto"])?; + let resolved = RelativeTimeFormatOptions { + locale_matcher: Some(matcher_name(matcher).to_owned()), + numeric, + style, + numbering_system, + }; + let data: &'static IntlData = &DATA; + let format = RelativeTimeFormat::try_new(&requested, &resolved, data, &NoHostLocale) + .map_err(|error| service_failure(machine, &error))?; + allocate(machine, IntlObject::RelativeTimeFormat(format), prototype).map(BuiltinOutcome::Value) +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn this_format<'a, H: Host>( + machine: &'a Machine<'_, H>, + this: Value, +) -> Result>, EvalFailure> { + Ok(match intl_record(machine, this)? { + Some(IntlObject::RelativeTimeFormat(format)) => Some(format), + _ => None, + }) +} + +fn not_a_relative_time_format(machine: &mut Machine<'_, H>) -> EvalFailure { + type_error(machine, "receiver is not an Intl.RelativeTimeFormat") +} + +/// The argument conversions `format` and `formatToParts` share. +fn value_and_unit( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], +) -> Result<(f64, String), EvalFailure> { + if this_format(machine, this)?.is_none() { + return Err(not_a_relative_time_format(machine)); + } + let number = machine.coerce_number_observable(arg(args, 0))?; + let number = match number.decode() { + Some(Decoded::Int32(number)) => f64::from(number as i32), + Some(Decoded::Number(number)) => number, + _ => f64::NAN, + }; + let unit = machine + .coerce_string_observable(arg(args, 1))? + .to_utf8_lossy(); + Ok((number, unit)) +} + +fn format( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, unit) = value_and_unit(machine, this, args)?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_relative_time_format(machine)); + }; + let text = format.format(value, &unit); + let text = text.map_err(|error| service_failure(machine, &error))?; + string_outcome(machine, &text) +} + +const fn number_part_type(kind: NumberPartKind) -> &'static str { + match kind { + NumberPartKind::Integer => "integer", + NumberPartKind::Group => "group", + NumberPartKind::Decimal => "decimal", + NumberPartKind::Fraction => "fraction", + NumberPartKind::PlusSign => "plusSign", + NumberPartKind::MinusSign => "minusSign", + NumberPartKind::Nan => "nan", + NumberPartKind::Infinity => "infinity", + NumberPartKind::ExponentSeparator => "exponentSeparator", + NumberPartKind::ExponentMinusSign => "exponentMinusSign", + NumberPartKind::ExponentInteger => "exponentInteger", + NumberPartKind::Compact => "compact", + NumberPartKind::PercentSign => "percentSign", + NumberPartKind::Currency => "currency", + NumberPartKind::Unit => "unit", + NumberPartKind::Literal => "literal", + } +} + +fn format_to_parts( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, unit) = value_and_unit(machine, this, args)?; + let Some(format) = this_format(machine, this)? else { + return Err(not_a_relative_time_format(machine)); + }; + let parts = format.format_to_parts(value, &unit); + let parts = parts.map_err(|error| service_failure(machine, &error))?; + let mut elements = Vec::with_capacity(parts.len()); + for part in &parts { + let part_type = match part.kind { + RelativeTimePartKind::Literal => "literal", + RelativeTimePartKind::Number(kind) => number_part_type(kind), + }; + let extra = part.unit.map(|unit| ("unit", unit.as_str())); + elements.push(part_object(machine, part_type, &part.value, extra)?); + } + allocate_array(machine, elements).map(BuiltinOutcome::Value) +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let Some(format) = this_format(machine, this)? else { + return Err(not_a_relative_time_format(machine)); + }; + let resolved = format.resolved_options(); + let result = new_object(machine)?; + for (name, text) in [ + ("locale", resolved.locale.as_str()), + ("style", resolved.style.as_str()), + ("numeric", resolved.numeric.as_str()), + ("numberingSystem", resolved.numbering_system.as_str()), + ] { + let value = string_value(machine, text)?; + define_field(machine, result, name, value)?; + } + Ok(BuiltinOutcome::Value(result)) +} diff --git a/crates/bamts-runtime/src/builtins/intl/segmenter.rs b/crates/bamts-runtime/src/builtins/intl/segmenter.rs new file mode 100644 index 00000000..cdbc9d88 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/intl/segmenter.rs @@ -0,0 +1,257 @@ +//! `Intl.Segmenter`, its `Segments` objects, and their iterator. + +use bamts_bytecode::EcmaString; +use bamts_native::{Decoded, Value}; + +use super::options::{ + canonicalize_locale_list, get_options_object, locale_matcher_option, matcher_name, + string_option, +}; +use super::{ + Class, allocate, arg, create, define_field, install_class, instance_prototype, intl_record, + new_object, require_new, service_failure, string_value, supported_locales_outcome, type_error, +}; +use crate::intl::locale_data::{DATA, IntlData}; +use crate::intl::number_data::NoHostLocale; +use crate::intl::plural_rules::{ + SegmentRecord, SegmentedText, Segmenter, SegmenterGranularity, SegmenterOptions, +}; +use crate::intl::{IntlKind, IntlObject, SegmentIteratorRecord, SegmentsRecord}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: IntlKind::Segmenter, + name: "Segmenter", + tag: "Intl.Segmenter", + length: 0, + constructor: constructor::, + statics: &[("supportedLocalesOf", 1, supported_locales_of::)], + getters: &[], + methods: &[ + ("resolvedOptions", 0, resolved_options::), + ("segment", 1, segment::), + ], + }, + ); + let segments = builtins.intl_prototype(IntlKind::Segments); + let containing = super::install_function(heap, builtins, "containing", 1, containing::); + super::define_data(heap, segments, "containing", containing); + let iterator = builtins.symbol_iterator(); + let function = + super::install_function(heap, builtins, "[Symbol.iterator]", 0, create_iterator::); + super::define_symbol_method(heap, segments, iterator, function); + + let prototype = builtins.intl_prototype(IntlKind::SegmentIterator); + let iterator_prototype = builtins.iterator_prototype(); + super::set_prototype(heap, prototype, iterator_prototype); + let next = super::install_function(heap, builtins, "next", 0, next::); + super::define_data(heap, prototype, "next", next); + let tag_symbol = builtins.symbol_to_string_tag(); + super::define_tag(heap, prototype, tag_symbol, "Segmenter String Iterator"); +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "Segmenter")?; + let prototype = instance_prototype(machine, IntlKind::Segmenter, constructing)?; + let requested = canonicalize_locale_list(machine, arg(args, 0))?; + let options = get_options_object(machine, arg(args, 1))?; + let matcher = locale_matcher_option(machine, options)?; + let granularity = string_option( + machine, + options, + "granularity", + &["grapheme", "word", "sentence"], + )?; + let resolved = SegmenterOptions { + locale_matcher: Some(matcher_name(matcher).to_owned()), + granularity, + }; + let data: &'static IntlData = &DATA; + let segmenter = Segmenter::try_new(&requested, &resolved, data, &NoHostLocale) + .map_err(|error| service_failure(machine, &error))?; + allocate(machine, IntlObject::Segmenter(segmenter), prototype).map(BuiltinOutcome::Value) +} + +fn supported_locales_of( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + supported_locales_outcome(machine, args) +} + +fn this_segmenter<'a, H: Host>( + machine: &'a Machine<'_, H>, + this: Value, +) -> Result>, EvalFailure> { + Ok(match intl_record(machine, this)? { + Some(IntlObject::Segmenter(segmenter)) => Some(segmenter), + _ => None, + }) +} + +fn not_a_segmenter(machine: &mut Machine<'_, H>) -> EvalFailure { + type_error(machine, "receiver is not an Intl.Segmenter") +} + +fn segment( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if this_segmenter(machine, this)?.is_none() { + return Err(not_a_segmenter(machine)); + } + let text = machine.coerce_string_observable(arg(args, 0))?; + let Some(segmenter) = this_segmenter(machine, this)? else { + return Err(not_a_segmenter(machine)); + }; + let granularity = segmenter.resolved_options().granularity; + let segmented = segmenter.segment_utf16(text.as_units()); + let text = segmented.map_err(|error| service_failure(machine, &error))?; + create( + machine, + IntlObject::Segments(Box::new(SegmentsRecord { text, granularity })), + ) + .map(BuiltinOutcome::Value) +} + +fn resolved_options( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let Some(segmenter) = this_segmenter(machine, this)? else { + return Err(not_a_segmenter(machine)); + }; + let resolved = segmenter.resolved_options(); + let result = new_object(machine)?; + for (name, text) in [ + ("locale", resolved.locale.as_str()), + ("granularity", resolved.granularity.as_str()), + ] { + let value = string_value(machine, text)?; + define_field(machine, result, name, value)?; + } + Ok(BuiltinOutcome::Value(result)) +} + +/// CreateSegmentDataObject. +fn segment_data( + machine: &mut Machine<'_, H>, + text: &SegmentedText, + record: &SegmentRecord, + granularity: SegmenterGranularity, +) -> Result { + let result = new_object(machine)?; + let segment = + super::super::allocate_string(machine, EcmaString::from_units(text.segment(record)))?; + define_field(machine, result, "segment", segment)?; + let index = crate::number_value(record.start as f64); + define_field(machine, result, "index", index)?; + let input = super::super::allocate_string(machine, EcmaString::from_units(text.input()))?; + define_field(machine, result, "input", input)?; + if granularity == SegmenterGranularity::Word { + let word_like = Value::boolean(record.is_word_like.unwrap_or(false)); + define_field(machine, result, "isWordLike", word_like)?; + } + Ok(result) +} + +fn this_segments( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result { + match intl_record(machine, this)? { + Some(IntlObject::Segments(record)) => Ok((**record).clone()), + _ => Err(type_error(machine, "receiver is not a Segments object")), + } +} + +fn containing( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let record = this_segments(machine, this)?; + let number = machine.coerce_number_observable(arg(args, 0))?; + let number = match number.decode() { + Some(Decoded::Int32(number)) => f64::from(number as i32), + Some(Decoded::Number(number)) if number.is_nan() => 0.0, + Some(Decoded::Number(number)) => number.trunc(), + _ => 0.0, + }; + if number < 0.0 || number >= record.text.input().len() as f64 { + return Ok(BuiltinOutcome::Value(Value::UNDEFINED)); + } + let Some(segment) = record.text.containing(number as usize) else { + return Ok(BuiltinOutcome::Value(Value::UNDEFINED)); + }; + segment_data(machine, &record.text, segment, record.granularity).map(BuiltinOutcome::Value) +} + +fn create_iterator( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let record = this_segments(machine, this)?; + create( + machine, + IntlObject::SegmentIterator(Box::new(SegmentIteratorRecord { + text: record.text, + granularity: record.granularity, + next_record: 0, + })), + ) + .map(BuiltinOutcome::Value) +} + +fn next( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let Some(index) = machine.runtime_slot(this).map_err(EvalFailure::Runtime)? else { + return Err(type_error(machine, "receiver is not a Segment Iterator")); + }; + let HeapEntry::Intl { value, .. } = &mut machine.heap[index] else { + return Err(type_error(machine, "receiver is not a Segment Iterator")); + }; + let IntlObject::SegmentIterator(iterator) = &mut **value else { + return Err(type_error(machine, "receiver is not a Segment Iterator")); + }; + let Some(record) = iterator.text.records().get(iterator.next_record).cloned() else { + return machine + .iterator_result(Value::UNDEFINED, true) + .map(BuiltinOutcome::Value); + }; + iterator.next_record += 1; + let text = iterator.text.clone(); + let granularity = iterator.granularity; + let data = segment_data(machine, &text, &record, granularity)?; + machine + .iterator_result(data, false) + .map(BuiltinOutcome::Value) +} diff --git a/crates/bamts-runtime/src/builtins/iterator_helpers.rs b/crates/bamts-runtime/src/builtins/iterator_helpers.rs new file mode 100644 index 00000000..319952d2 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/iterator_helpers.rs @@ -0,0 +1,1727 @@ +//! The `Iterator` constructor, its helpers, and `Iterator.from/concat/zip`. +//! +//! The specification describes a helper as a generator closure. There is no +//! native generator here, so each helper is a `BuiltinIterator` whose `source` is +//! a hidden state array reachable only from it, and `advance` runs one step of +//! the closure per `next()`. The `status` slot carries the generator states, +//! which is what makes a helper reject re-entry while it is running. + +use std::collections::BTreeMap; + +use bamts_bytecode::EcmaString; +use bamts_native::{Decoded, Value}; + +use super::property_descriptor::descriptor_from_property; +use super::{ + allocate_array, allocate_string, define_data, define_frozen_data, define_symbol_property, + heap_index, install_constructor_function, install_function, keep_root, range_error, type_error, + value_number, with_roots, +}; +use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; +use crate::{ + EvalFailure, HeapEntry, Host, IterationKind, Machine, Property, PropertyKey, PropertyMap, +}; + +/// What a helper does on each step. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub(crate) enum Operation { + Map, + Filter, + Take, + Drop, + FlatMap, + Concat, + Zip, + ZipKeyed, +} + +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +enum ZipMode { + Shortest, + Longest, + Strict, +} + +const STATUS: usize = 0; +const ITERATOR: usize = 1; +const NEXT: usize = 2; +const CALLBACK: usize = 3; +const COUNTER: usize = 4; +const REMAINING: usize = 5; +const INNER: usize = 6; +const INNER_NEXT: usize = 7; +/// `Iterator.concat`'s `[method, iterable, ...]`, or `zip`'s `[iterator, next, ...]` +/// with a finished iterator's entries set to null. +const LIST: usize = 8; +const PADDING: usize = 9; +const KEYS: usize = 10; +const MODE: usize = 11; +const SLOTS: usize = 12; + +const SUSPENDED_START: u32 = 0; +const SUSPENDED_YIELD: u32 = 1; +const RUNNING: u32 = 2; +const COMPLETED: u32 = 3; + +fn argument(args: &[Value], index: usize) -> Value { + args.get(index).copied().unwrap_or(Value::UNDEFINED) +} + +fn is_nullish(value: Value) -> bool { + matches!(value.decode(), Some(Decoded::Undefined | Decoded::Null)) +} + +fn hidden_array( + machine: &mut Machine<'_, H>, + elements: Vec, +) -> Result { + machine + .allocate(HeapEntry::Array { + elements, + properties: PropertyMap::default(), + prototype: None, + extensible: false, + length_writable: false, + }) + .map_err(EvalFailure::Runtime) +} + +fn slot(machine: &Machine<'_, H>, state: Value, index: usize) -> Value { + let HeapEntry::Array { elements, .. } = &machine.heap[heap_index(state)] else { + unreachable!("helper state is a hidden array") + }; + elements[index] +} + +fn set_slot(machine: &mut Machine<'_, H>, state: Value, index: usize, value: Value) { + let HeapEntry::Array { elements, .. } = &mut machine.heap[heap_index(state)] else { + unreachable!("helper state is a hidden array") + }; + elements[index] = value; +} + +fn status(machine: &Machine<'_, H>, state: Value) -> u32 { + value_number(slot(machine, state, STATUS)) as u32 +} + +fn set_status(machine: &mut Machine<'_, H>, state: Value, status: u32) { + set_slot(machine, state, STATUS, Value::int32(status)); +} + +fn bump_counter(machine: &mut Machine<'_, H>, state: Value) { + let next = value_number(slot(machine, state, COUNTER)) + 1.0; + set_slot(machine, state, COUNTER, crate::number_value(next)); +} + +fn new_helper( + machine: &mut Machine<'_, H>, + operation: Operation, + fields: &[(usize, Value)], +) -> Result { + let mut elements = vec![Value::UNDEFINED; SLOTS]; + elements[STATUS] = Value::int32(SUSPENDED_START); + elements[COUNTER] = Value::int32(0); + for &(index, value) in fields { + elements[index] = value; + } + let source = hidden_array(machine, elements)?; + machine + .allocate(HeapEntry::BuiltinIterator { + source, + kind: IterationKind::IteratorHelper(operation), + position: Some(0), + properties: PropertyMap::default(), + prototype: Some(machine.intrinsics.builtins.iterator_helper_prototype()), + extensible: true, + }) + .map_err(EvalFailure::Runtime) +} + +/// One `IteratorStepValue`. A failure here does not close the iterator: it is the +/// iterator that failed. +fn step_value( + machine: &mut Machine<'_, H>, + iterator: Value, + next: Value, +) -> Result, EvalFailure> { + let result = machine.call_value(next, iterator, &[])?; + let (done, value) = machine.iterator_result_parts(result)?; + Ok((!done).then_some(value)) +} + +/// One `IteratorStep`: `value` is never read, so skipping stays unobservable. +fn step( + machine: &mut Machine<'_, H>, + iterator: Value, + next: Value, +) -> Result { + let result = machine.call_value(next, iterator, &[])?; + if !machine.is_object(result) { + return Err(type_error("iterator result is not an object")); + } + let done = machine.get_named_property(result, "done")?; + Ok(machine.to_boolean(done)) +} + +fn call_return( + machine: &mut Machine<'_, H>, + iterator: Value, +) -> Result, EvalFailure> { + let method = machine.get_named_property(iterator, "return")?; + if is_nullish(method) { + return Ok(None); + } + if !machine.is_callable(method)? { + return Err(type_error("iterator return method is not callable")); + } + machine.call_value(method, iterator, &[]).map(Some) +} + +fn failure_value(failure: &EvalFailure) -> Option { + match failure { + EvalFailure::ThrowValue(value) | EvalFailure::ThrowValueOrigin { value, .. } => { + Some(*value) + } + _ => None, + } +} + +/// `IteratorClose`. An existing failure wins over anything `return` does, except +/// an engine failure, which cannot be replaced by a user throw. +fn close_iterator( + machine: &mut Machine<'_, H>, + iterator: Value, + completion: Result<(), EvalFailure>, +) -> Result<(), EvalFailure> { + let held = completion.as_ref().err().and_then(failure_value); + let roots: Vec = [iterator].into_iter().chain(held).collect(); + let closed = with_roots(machine, &roots, |machine, _| call_return(machine, iterator)); + match completion { + Err(failure) => match closed { + Err(EvalFailure::Runtime(kind)) => Err(EvalFailure::Runtime(kind)), + _ => Err(failure), + }, + Ok(()) => match closed? { + Some(result) if !machine.is_object(result) => { + Err(type_error("iterator return result is not an object")) + } + _ => Ok(()), + }, + } +} + +fn close_after( + machine: &mut Machine<'_, H>, + iterator: Value, + failure: EvalFailure, +) -> EvalFailure { + match close_iterator(machine, iterator, Err(failure)) { + Err(failure) => failure, + Ok(()) => unreachable!("closing with a failure returns that failure"), + } +} + +fn call_closing( + machine: &mut Machine<'_, H>, + iterator: Value, + callback: Value, + arguments: &[Value], +) -> Result { + match machine.call_callback(callback, Value::UNDEFINED, arguments) { + Ok(value) => Ok(value), + Err(failure) => Err(close_after(machine, iterator, failure)), + } +} + +/// `GetMethod(value, @@iterator)`. +fn iterator_method( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result, EvalFailure> { + let key = machine.to_property_key(machine.intrinsics.builtins.symbol_iterator())?; + let method = machine.get_property_key(value, &key)?; + if is_nullish(method) { + return Ok(None); + } + if !machine.is_callable(method)? { + return Err(type_error("Symbol.iterator is not callable")); + } + Ok(Some(method)) +} + +/// `GetIteratorDirect`, once `iterator` is known to be an object. +fn direct_record( + machine: &mut Machine<'_, H>, + iterator: Value, +) -> Result<(Value, Value), EvalFailure> { + if !machine.is_object(iterator) { + return Err(type_error("iterator is not an object")); + } + let next = machine.get_named_property(iterator, "next")?; + Ok((iterator, next)) +} + +/// `GetIteratorFlattenable`. A value with no `@@iterator` is taken to be an +/// iterator already, and only `Iterator.from` lets a string through. +fn flattenable( + machine: &mut Machine<'_, H>, + value: Value, + allow_string: bool, +) -> Result<(Value, Value), EvalFailure> { + let is_string = machine.string_value(value).is_some(); + if !machine.is_object(value) && !(allow_string && is_string) { + return Err(type_error("value is not iterable")); + } + let iterator = match iterator_method(machine, value)? { + Some(method) => machine.call_value(method, value, &[])?, + None => value, + }; + direct_record(machine, iterator) +} + +/// `GetIterator(value, sync)`. +fn get_iterator( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result<(Value, Value), EvalFailure> { + let Some(method) = iterator_method(machine, value)? else { + return Err(type_error("value is not iterable")); + }; + let iterator = machine.call_value(method, value, &[])?; + direct_record(machine, iterator) +} + +fn advance( + machine: &mut Machine<'_, H>, + state: Value, + operation: Operation, + depth: usize, +) -> Result, EvalFailure> { + let iterator = slot(machine, state, ITERATOR); + let next = slot(machine, state, NEXT); + let callback = slot(machine, state, CALLBACK); + match operation { + Operation::Map => { + let Some(value) = step_value(machine, iterator, next)? else { + return Ok(None); + }; + let counter = slot(machine, state, COUNTER); + let mapped = call_closing(machine, iterator, callback, &[value, counter])?; + bump_counter(machine, state); + Ok(Some(mapped)) + } + Operation::Filter => loop { + let Some(value) = step_value(machine, iterator, next)? else { + return Ok(None); + }; + keep_root(machine, depth, value); + let counter = slot(machine, state, COUNTER); + let selected = call_closing(machine, iterator, callback, &[value, counter])?; + bump_counter(machine, state); + if machine.to_boolean(selected) { + return Ok(Some(value)); + } + }, + Operation::Take => { + let remaining = value_number(slot(machine, state, REMAINING)); + if remaining == 0.0 { + close_iterator(machine, iterator, Ok(()))?; + return Ok(None); + } + set_slot( + machine, + state, + REMAINING, + crate::number_value(remaining - 1.0), + ); + step_value(machine, iterator, next) + } + Operation::Drop => { + let mut remaining = value_number(slot(machine, state, REMAINING)); + while remaining > 0.0 { + remaining -= 1.0; + set_slot(machine, state, REMAINING, crate::number_value(remaining)); + if step(machine, iterator, next)? { + return Ok(None); + } + } + step_value(machine, iterator, next) + } + Operation::FlatMap => flat_map_step(machine, state, iterator, next, callback, depth), + Operation::Concat => concat_step(machine, state), + Operation::Zip | Operation::ZipKeyed => zip_step(machine, state, operation, depth), + } +} + +fn flat_map_step( + machine: &mut Machine<'_, H>, + state: Value, + iterator: Value, + next: Value, + callback: Value, + depth: usize, +) -> Result, EvalFailure> { + loop { + let inner = slot(machine, state, INNER); + if inner != Value::UNDEFINED { + let inner_next = slot(machine, state, INNER_NEXT); + match step_value(machine, inner, inner_next) { + Ok(Some(value)) => return Ok(Some(value)), + Ok(None) => { + set_slot(machine, state, INNER, Value::UNDEFINED); + bump_counter(machine, state); + continue; + } + Err(failure) => return Err(close_after(machine, iterator, failure)), + } + } + let Some(value) = step_value(machine, iterator, next)? else { + return Ok(None); + }; + let counter = slot(machine, state, COUNTER); + let mapped = call_closing(machine, iterator, callback, &[value, counter])?; + keep_root(machine, depth, mapped); + match flattenable(machine, mapped, false) { + Ok((inner, inner_next)) => { + set_slot(machine, state, INNER, inner); + set_slot(machine, state, INNER_NEXT, inner_next); + } + Err(failure) => return Err(close_after(machine, iterator, failure)), + } + } +} + +fn concat_step( + machine: &mut Machine<'_, H>, + state: Value, +) -> Result, EvalFailure> { + loop { + let iterator = slot(machine, state, ITERATOR); + if iterator != Value::UNDEFINED { + let next = slot(machine, state, NEXT); + if let Some(value) = step_value(machine, iterator, next)? { + return Ok(Some(value)); + } + set_slot(machine, state, ITERATOR, Value::UNDEFINED); + } + let list = slot(machine, state, LIST); + let index = value_number(slot(machine, state, COUNTER)) as usize; + let items = machine.array_elements(list)?.unwrap_or_default(); + let (Some(method), Some(iterable)) = (items.get(index * 2), items.get(index * 2 + 1)) + else { + return Ok(None); + }; + bump_counter(machine, state); + let opened = machine.call_value(*method, *iterable, &[])?; + let (iterator, next) = direct_record(machine, opened)?; + set_slot(machine, state, ITERATOR, iterator); + set_slot(machine, state, NEXT, next); + } +} + +/// `IteratorCloseAll` over the open entries of a flat `[iterator, next, ...]` +/// list, last first. +fn close_list( + machine: &mut Machine<'_, H>, + list: &[Value], + mut completion: Result<(), EvalFailure>, +) -> Result<(), EvalFailure> { + for pair in list.chunks_exact(2).rev() { + if pair[0] != Value::NULL { + completion = close_iterator(machine, pair[0], completion); + } + } + completion +} + +fn close_open_iterators( + machine: &mut Machine<'_, H>, + state: Value, + completion: Result<(), EvalFailure>, +) -> Result<(), EvalFailure> { + let list = slot(machine, state, LIST); + let open = machine.array_elements(list)?.unwrap_or_default(); + close_list(machine, &open, completion) +} + +fn mark_finished(machine: &mut Machine<'_, H>, state: Value, index: usize) { + let list = slot(machine, state, LIST); + let HeapEntry::Array { elements, .. } = &mut machine.heap[heap_index(list)] else { + unreachable!("zip list is a hidden array") + }; + elements[index * 2] = Value::NULL; +} + +fn zip_mode(machine: &Machine<'_, H>, state: Value) -> ZipMode { + match value_number(slot(machine, state, MODE)) as u32 { + 0 => ZipMode::Shortest, + 1 => ZipMode::Longest, + _ => ZipMode::Strict, + } +} + +/// The strict-mode tail of a zip whose first iterator ended: every other one must +/// end on this same step. +fn finish_strict_zip( + machine: &mut Machine<'_, H>, + state: Value, + count: usize, +) -> Result, EvalFailure> { + let list = slot(machine, state, LIST); + for index in 1..count { + let pairs = machine.array_elements(list)?.unwrap_or_default(); + let (iterator, next) = (pairs[index * 2], pairs[index * 2 + 1]); + match step(machine, iterator, next) { + Err(failure) => { + mark_finished(machine, state, index); + close_open_iterators(machine, state, Err(failure))?; + } + Ok(true) => mark_finished(machine, state, index), + Ok(false) => { + let mismatch = type_error("Iterator.zip iterators have different lengths"); + close_open_iterators(machine, state, Err(mismatch))?; + } + } + } + Ok(None) +} + +fn zip_step( + machine: &mut Machine<'_, H>, + state: Value, + operation: Operation, + depth: usize, +) -> Result, EvalFailure> { + let mode = zip_mode(machine, state); + let list = slot(machine, state, LIST); + let count = machine.array_elements(list)?.unwrap_or_default().len() / 2; + if count == 0 { + return Ok(None); + } + let mut results = Vec::with_capacity(count); + for index in 0..count { + let pairs = machine.array_elements(list)?.unwrap_or_default(); + let (iterator, next) = (pairs[index * 2], pairs[index * 2 + 1]); + let padding = slot(machine, state, PADDING); + let value = if iterator == Value::NULL { + machine.array_elements(padding)?.unwrap_or_default()[index] + } else { + match step_value(machine, iterator, next) { + Err(failure) => { + mark_finished(machine, state, index); + close_open_iterators(machine, state, Err(failure))?; + unreachable!("closing with a failure returns that failure") + } + Ok(Some(value)) => value, + Ok(None) => { + mark_finished(machine, state, index); + match mode { + ZipMode::Shortest => { + close_open_iterators(machine, state, Ok(()))?; + return Ok(None); + } + ZipMode::Strict if index != 0 => { + let mismatch = + type_error("Iterator.zip iterators have different lengths"); + close_open_iterators(machine, state, Err(mismatch))?; + unreachable!("closing with a failure returns that failure") + } + ZipMode::Strict => return finish_strict_zip(machine, state, count), + ZipMode::Longest => { + let open = machine.array_elements(list)?.unwrap_or_default(); + if open.chunks_exact(2).all(|pair| pair[0] == Value::NULL) { + return Ok(None); + } + machine.array_elements(padding)?.unwrap_or_default()[index] + } + } + } + } + }; + keep_root(machine, depth, value); + results.push(value); + } + if operation == Operation::Zip { + return allocate_array(machine, results).map(Some); + } + let keys = slot(machine, state, KEYS); + let keys = machine.array_elements(keys)?.unwrap_or_default(); + let object = super::ordinary_runtime(machine, None)?; + for (key, value) in keys.into_iter().zip(results) { + let key = machine.to_property_key(key)?; + machine.create_data_property_key(object, key, value)?; + } + Ok(Some(object)) +} + +fn helper_state( + machine: &Machine<'_, H>, + this: Value, +) -> Result<(Value, Operation), EvalFailure> { + let index = machine.runtime_slot(this).map_err(EvalFailure::Runtime)?; + match index.map(|index| &machine.heap[index]) { + Some(HeapEntry::BuiltinIterator { + source, + kind: IterationKind::IteratorHelper(operation), + .. + }) => Ok((*source, *operation)), + _ => Err(type_error( + "Iterator helper method on incompatible receiver", + )), + } +} + +fn iterator_result( + machine: &mut Machine<'_, H>, + value: Value, + done: bool, +) -> Result { + machine + .iterator_result(value, done) + .map(BuiltinOutcome::Value) +} + +/// `%IteratorHelperPrototype%.next`. +fn helper_next( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (state, operation) = helper_state(machine, this)?; + match status(machine, state) { + RUNNING => return Err(type_error("Iterator helper is already running")), + COMPLETED => return iterator_result(machine, Value::UNDEFINED, true), + _ => {} + } + set_status(machine, state, RUNNING); + let advanced = with_roots(machine, &[this], |machine, depth| { + advance(machine, state, operation, depth) + }); + match advanced { + Ok(Some(value)) => { + set_status(machine, state, SUSPENDED_YIELD); + iterator_result(machine, value, false) + } + Ok(None) => { + set_status(machine, state, COMPLETED); + iterator_result(machine, Value::UNDEFINED, true) + } + Err(failure) => { + set_status(machine, state, COMPLETED); + Err(failure) + } + } +} + +/// Closes whatever a helper holds open, as a return completion would. +fn close_helper( + machine: &mut Machine<'_, H>, + state: Value, + operation: Operation, + started: bool, +) -> Result<(), EvalFailure> { + let iterator = slot(machine, state, ITERATOR); + match operation { + Operation::Map | Operation::Filter | Operation::Take | Operation::Drop => { + close_iterator(machine, iterator, Ok(())) + } + Operation::FlatMap => { + let inner = slot(machine, state, INNER); + if started + && inner != Value::UNDEFINED + && let Err(failure) = close_iterator(machine, inner, Ok(())) + { + return Err(close_after(machine, iterator, failure)); + } + close_iterator(machine, iterator, Ok(())) + } + Operation::Concat if started && iterator != Value::UNDEFINED => { + close_iterator(machine, iterator, Ok(())) + } + Operation::Concat => Ok(()), + Operation::Zip | Operation::ZipKeyed => close_open_iterators(machine, state, Ok(())), + } +} + +/// `%IteratorHelperPrototype%.return`. +fn helper_return( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (state, operation) = helper_state(machine, this)?; + let current = status(machine, state); + match current { + RUNNING => return Err(type_error("Iterator helper is already running")), + COMPLETED => return iterator_result(machine, Value::UNDEFINED, true), + _ => {} + } + let started = current == SUSPENDED_YIELD; + set_status(machine, state, if started { RUNNING } else { COMPLETED }); + let closed = with_roots(machine, &[this], |machine, _| { + close_helper(machine, state, operation, started) + }); + set_status(machine, state, COMPLETED); + closed?; + iterator_result(machine, Value::UNDEFINED, true) +} + +fn require_object( + machine: &Machine<'_, H>, + value: Value, + operation: &'static str, +) -> Result<(), EvalFailure> { + if machine.is_object(value) { + Ok(()) + } else { + Err(type_error(operation)) + } +} + +/// The shared opening of the callback helpers: a bad argument closes the receiver +/// before `next` is read. +fn checked_callback( + machine: &mut Machine<'_, H>, + this: Value, + callback: Value, +) -> Result<(), EvalFailure> { + require_object(machine, this, "Iterator helper requires an object receiver")?; + if machine.is_callable(callback)? { + return Ok(()); + } + let failure = type_error("Iterator helper callback is not callable"); + Err(close_after(machine, this, failure)) +} + +fn callback_helper( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + operation: Operation, +) -> Result { + let callback = argument(args, 0); + checked_callback(machine, this, callback)?; + let (iterator, next) = direct_record(machine, this)?; + new_helper( + machine, + operation, + &[(ITERATOR, iterator), (NEXT, next), (CALLBACK, callback)], + ) + .map(BuiltinOutcome::Value) +} + +fn map( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + callback_helper(machine, this, args, Operation::Map) +} + +fn filter( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + callback_helper(machine, this, args, Operation::Filter) +} + +fn flat_map( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + callback_helper(machine, this, args, Operation::FlatMap) +} + +/// The shared opening of `take` and `drop`. Each failure closes the receiver, and +/// `next` is read only once the limit is known to be valid. +fn limit_helper( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + operation: Operation, +) -> Result { + require_object(machine, this, "Iterator helper requires an object receiver")?; + let limit = match machine.coerce_number_observable(argument(args, 0)) { + Ok(limit) => value_number(limit), + Err(failure) => return Err(close_after(machine, this, failure)), + }; + if limit.is_nan() { + let failure = range_error("Iterator helper limit is NaN"); + return Err(close_after(machine, this, failure)); + } + let limit = limit.trunc(); + if limit < 0.0 { + let failure = range_error("Iterator helper limit is negative"); + return Err(close_after(machine, this, failure)); + } + let (iterator, next) = direct_record(machine, this)?; + let remaining = crate::number_value(limit); + new_helper( + machine, + operation, + &[(ITERATOR, iterator), (NEXT, next), (REMAINING, remaining)], + ) + .map(BuiltinOutcome::Value) +} + +fn take( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + limit_helper(machine, this, args, Operation::Take) +} + +fn drop( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + limit_helper(machine, this, args, Operation::Drop) +} + +fn to_array( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + require_object(machine, this, "Iterator helper requires an object receiver")?; + let (iterator, next) = direct_record(machine, this)?; + with_roots(machine, &[iterator, next], |machine, depth| { + let mut values = Vec::new(); + while let Some(value) = step_value(machine, iterator, next)? { + keep_root(machine, depth, value); + values.push(value); + } + allocate_array(machine, values) + }) + .map(BuiltinOutcome::Value) +} + +/// How a consumer treats each callback result. +#[derive(Clone, Copy, Eq, PartialEq)] +enum Consumer { + ForEach, + Some, + Every, + Find, +} + +fn consume( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + consumer: Consumer, +) -> Result { + let callback = argument(args, 0); + checked_callback(machine, this, callback)?; + let (iterator, next) = direct_record(machine, this)?; + with_roots(machine, &[iterator, next, callback], |machine, _| { + let mut counter = 0.0; + loop { + let Some(value) = step_value(machine, iterator, next)? else { + return Ok(match consumer { + Consumer::ForEach | Consumer::Find => Value::UNDEFINED, + Consumer::Some => Value::FALSE, + Consumer::Every => Value::TRUE, + }); + }; + let arguments = [value, crate::number_value(counter)]; + let result = call_closing(machine, iterator, callback, &arguments)?; + let truthy = machine.to_boolean(result); + let finished = match consumer { + Consumer::ForEach => None, + Consumer::Some => truthy.then_some(Value::TRUE), + Consumer::Every => (!truthy).then_some(Value::FALSE), + Consumer::Find => truthy.then_some(value), + }; + if let Some(answer) = finished { + close_iterator(machine, iterator, Ok(()))?; + return Ok(answer); + } + counter += 1.0; + } + }) + .map(BuiltinOutcome::Value) +} + +fn for_each( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + consume(machine, this, args, Consumer::ForEach) +} + +fn some( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + consume(machine, this, args, Consumer::Some) +} + +fn every( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + consume(machine, this, args, Consumer::Every) +} + +fn find( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + consume(machine, this, args, Consumer::Find) +} + +fn reduce( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let reducer = argument(args, 0); + checked_callback(machine, this, reducer)?; + let (iterator, next) = direct_record(machine, this)?; + with_roots(machine, &[iterator, next, reducer], |machine, depth| { + let (mut accumulator, mut counter) = match args.get(1) { + Some(initial) => (*initial, 0.0), + None => match step_value(machine, iterator, next)? { + Some(first) => (first, 1.0), + None => { + return Err(type_error( + "Iterator.prototype.reduce of an empty iterator with no initial value", + )); + } + }, + }; + machine.refresh_native_roots(depth, &[iterator, next, reducer, accumulator]); + while let Some(value) = step_value(machine, iterator, next)? { + let arguments = [accumulator, value, crate::number_value(counter)]; + accumulator = call_closing(machine, iterator, reducer, &arguments)?; + machine.refresh_native_roots(depth, &[iterator, next, reducer, accumulator]); + counter += 1.0; + } + Ok(accumulator) + }) + .map(BuiltinOutcome::Value) +} + +/// `Iterator.prototype[@@dispose]`. +fn dispose( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + call_return(machine, this)?; + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) +} + +/// `new Iterator()` is abstract: only a subclass may construct it. +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let new_target = machine.current_new_target(); + let active = machine + .intrinsics + .global("Iterator") + .expect("Iterator is installed"); + if new_target == Value::UNDEFINED || new_target == active { + return Err(type_error("Iterator is an abstract class")); + } + let candidate = machine.get_named_property(new_target, "prototype")?; + let prototype = if machine.is_object(candidate) { + candidate + } else { + machine.intrinsics.builtins.iterator_prototype() + }; + machine + .allocate_constructed_receiver_with(prototype) + .map(BuiltinOutcome::Value) + .map_err(EvalFailure::Runtime) +} + +/// `Iterator.from`: an iterator that already inherits from `Iterator.prototype` +/// comes back as it is, and anything else gets a wrapper with the helpers. +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (iterator, next) = flattenable(machine, argument(args, 0), true)?; + let active = machine + .intrinsics + .global("Iterator") + .expect("Iterator is installed"); + let inherits = with_roots(machine, &[iterator, next], |machine, _| { + machine.ordinary_has_instance(iterator, active) + })?; + if inherits { + return Ok(BuiltinOutcome::Value(iterator)); + } + let source = hidden_array(machine, vec![iterator, next])?; + machine + .allocate(HeapEntry::BuiltinIterator { + source, + kind: IterationKind::WrapForValid, + position: Some(0), + properties: PropertyMap::default(), + prototype: Some( + machine + .intrinsics + .builtins + .wrap_for_valid_iterator_prototype(), + ), + extensible: true, + }) + .map(BuiltinOutcome::Value) + .map_err(EvalFailure::Runtime) +} + +fn wrapped_record( + machine: &Machine<'_, H>, + this: Value, +) -> Result<(Value, Value), EvalFailure> { + let index = machine.runtime_slot(this).map_err(EvalFailure::Runtime)?; + let Some(HeapEntry::BuiltinIterator { + source, + kind: IterationKind::WrapForValid, + .. + }) = index.map(|index| &machine.heap[index]) + else { + return Err(type_error( + "%WrapForValidIteratorPrototype% method on incompatible receiver", + )); + }; + Ok((slot(machine, *source, 0), slot(machine, *source, 1))) +} + +fn wrap_next( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (iterator, next) = wrapped_record(machine, this)?; + machine + .call_value(next, iterator, &[]) + .map(BuiltinOutcome::Value) +} + +fn wrap_return( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (iterator, _) = wrapped_record(machine, this)?; + match call_return(machine, iterator)? { + Some(result) => Ok(BuiltinOutcome::Value(result)), + None => iterator_result(machine, Value::UNDEFINED, true), + } +} + +/// `Iterator.concat`: every argument is checked before anything is opened. +fn concat( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + with_roots(machine, args, |machine, depth| { + let mut list = Vec::with_capacity(args.len() * 2); + for &item in args { + require_object(machine, item, "Iterator.concat requires objects")?; + let Some(method) = iterator_method(machine, item)? else { + return Err(type_error("Iterator.concat requires iterables")); + }; + keep_root(machine, depth, method); + list.extend([method, item]); + } + let list = hidden_array(machine, list)?; + new_helper(machine, Operation::Concat, &[(LIST, list)]) + }) + .map(BuiltinOutcome::Value) +} + +/// `GetOptionsObject`, then the `mode` and `padding` options shared by the zips. +fn zip_options( + machine: &mut Machine<'_, H>, + options: Value, +) -> Result<(ZipMode, Value), EvalFailure> { + if options == Value::UNDEFINED { + return Ok((ZipMode::Shortest, Value::UNDEFINED)); + } + require_object(machine, options, "Iterator.zip options must be an object")?; + let mode = machine.get_named_property(options, "mode")?; + let mode = if mode == Value::UNDEFINED { + ZipMode::Shortest + } else { + match machine.string_value(mode) { + Some(text) if text.eq_ascii("shortest") => ZipMode::Shortest, + Some(text) if text.eq_ascii("longest") => ZipMode::Longest, + Some(text) if text.eq_ascii("strict") => ZipMode::Strict, + _ => return Err(type_error("Iterator.zip mode is not recognized")), + } + }; + if mode != ZipMode::Longest { + return Ok((mode, Value::UNDEFINED)); + } + let padding = machine.get_named_property(options, "padding")?; + if padding != Value::UNDEFINED && !machine.is_object(padding) { + return Err(type_error("Iterator.zip padding must be an object")); + } + Ok((mode, padding)) +} + +fn new_zip( + machine: &mut Machine<'_, H>, + operation: Operation, + mode: ZipMode, + iterators: Vec, + padding: Vec, + keys: Vec, +) -> Result { + let list = hidden_array(machine, iterators)?; + let padding = hidden_array(machine, padding)?; + let keys = hidden_array(machine, keys)?; + let mode = Value::int32(match mode { + ZipMode::Shortest => 0, + ZipMode::Longest => 1, + ZipMode::Strict => 2, + }); + new_helper( + machine, + operation, + &[(LIST, list), (PADDING, padding), (KEYS, keys), (MODE, mode)], + ) +} + +/// Reads the padding iterable for `Iterator.zip`'s longest mode. A short one pads +/// with `undefined`, and one that still has values left is closed. +fn zip_padding( + machine: &mut Machine<'_, H>, + depth: usize, + padding_option: Value, + count: usize, +) -> Result, EvalFailure> { + if padding_option == Value::UNDEFINED { + return Ok(vec![Value::UNDEFINED; count]); + } + let (iterator, next) = get_iterator(machine, padding_option)?; + keep_root(machine, depth, iterator); + keep_root(machine, depth, next); + let mut padding = Vec::with_capacity(count); + let mut exhausted = false; + for _ in 0..count { + let value = if exhausted { + None + } else { + step_value(machine, iterator, next)? + }; + exhausted = value.is_none(); + let value = value.unwrap_or(Value::UNDEFINED); + keep_root(machine, depth, value); + padding.push(value); + } + if !exhausted { + close_iterator(machine, iterator, Ok(()))?; + } + Ok(padding) +} + +fn close_on_failure( + machine: &mut Machine<'_, H>, + iterators: &[Value], + result: Result, +) -> Result { + match result { + Ok(value) => Ok(value), + Err(failure) => { + close_list(machine, iterators, Err(failure))?; + unreachable!("closing with a failure returns that failure") + } + } +} + +/// `Iterator.zip(iterables, options)`. +fn zip( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (iterables, options) = (argument(args, 0), argument(args, 1)); + require_object(machine, iterables, "Iterator.zip requires an object")?; + with_roots(machine, &[iterables, options], |machine, depth| { + let (mode, padding_option) = zip_options(machine, options)?; + keep_root(machine, depth, padding_option); + let (input, input_next) = get_iterator(machine, iterables)?; + keep_root(machine, depth, input); + keep_root(machine, depth, input_next); + let mut iterators = Vec::new(); + loop { + let stepped = step_value(machine, input, input_next); + let Some(value) = close_on_failure(machine, &iterators, stepped)? else { + break; + }; + keep_root(machine, depth, value); + match flattenable(machine, value, false) { + Ok((iterator, next)) => { + keep_root(machine, depth, iterator); + keep_root(machine, depth, next); + iterators.extend([iterator, next]); + } + Err(failure) => { + let failure = match close_list(machine, &iterators, Err(failure)) { + Err(failure) => failure, + Ok(()) => unreachable!("closing with a failure returns that failure"), + }; + return Err(close_after(machine, input, failure)); + } + } + } + let count = iterators.len() / 2; + let padding = if mode == ZipMode::Longest { + let read = zip_padding(machine, depth, padding_option, count); + close_on_failure(machine, &iterators, read)? + } else { + vec![Value::UNDEFINED; count] + }; + new_zip( + machine, + Operation::Zip, + mode, + iterators, + padding, + Vec::new(), + ) + }) + .map(BuiltinOutcome::Value) +} + +fn key_value( + machine: &mut Machine<'_, H>, + key: &PropertyKey, +) -> Result { + match key { + PropertyKey::Named(name) => allocate_string(machine, name.clone()), + PropertyKey::Symbol(index) | PropertyKey::Private(index) => Ok(Value::heap_ref( + bamts_native::SlotId::from_parts(crate::RUNTIME_HEAP_SEGMENT, index + 1) + .expect("property-key heap slots are nonzero"), + )), + } +} + +/// `Iterator.zipKeyed(iterables, options)`: `iterables` is read by own enumerable +/// key, and each result object is keyed the same way. +fn zip_keyed( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (iterables, options) = (argument(args, 0), argument(args, 1)); + require_object(machine, iterables, "Iterator.zipKeyed requires an object")?; + with_roots(machine, &[iterables, options], |machine, depth| { + let (mode, padding_option) = zip_options(machine, options)?; + keep_root(machine, depth, padding_option); + let all_keys = machine.internal_own_property_keys(iterables)?; + let mut iterators = Vec::new(); + let mut keys = Vec::new(); + let mut key_values = Vec::new(); + for key in all_keys { + let descriptor = machine.internal_get_own_property(iterables, &key); + let descriptor = close_on_failure(machine, &iterators, descriptor)?; + if !descriptor.is_some_and(|descriptor| descriptor.enumerable == Some(true)) { + continue; + } + let value = machine.get_property_key(iterables, &key); + let value = close_on_failure(machine, &iterators, value)?; + if value == Value::UNDEFINED { + continue; + } + keep_root(machine, depth, value); + let key_handle = key_value(machine, &key)?; + keep_root(machine, depth, key_handle); + let record = flattenable(machine, value, false); + let (iterator, next) = close_on_failure(machine, &iterators, record)?; + keep_root(machine, depth, iterator); + keep_root(machine, depth, next); + iterators.extend([iterator, next]); + key_values.push(key_handle); + keys.push(key); + } + let mut padding = Vec::with_capacity(keys.len()); + for key in &keys { + let value = if mode == ZipMode::Longest && padding_option != Value::UNDEFINED { + let read = machine.get_property_key(padding_option, key); + close_on_failure(machine, &iterators, read)? + } else { + Value::UNDEFINED + }; + keep_root(machine, depth, value); + padding.push(value); + } + new_zip( + machine, + Operation::ZipKeyed, + mode, + iterators, + padding, + key_values, + ) + }) + .map(BuiltinOutcome::Value) +} + +/// `SetterThatIgnoresPrototypeProperties`: a write through an inheriting object +/// defines an own property instead of failing on the accessor it found. +fn set_ignoring_prototype( + machine: &mut Machine<'_, H>, + this: Value, + key: PropertyKey, + value: Value, +) -> Result<(), EvalFailure> { + require_object( + machine, + this, + "Iterator.prototype setter requires an object", + )?; + if this == machine.intrinsics.builtins.iterator_prototype() { + return Err(type_error("Iterator.prototype property cannot be assigned")); + } + if machine.internal_get_own_property(this, &key)?.is_some() { + return machine.set_data_property_key(this, key, value); + } + let descriptor = descriptor_from_property(Property::Data { + value, + writable: true, + enumerable: true, + configurable: true, + }); + if machine.internal_define_own_property(this, key, descriptor)? { + Ok(()) + } else { + Err(type_error("Cannot define property")) + } +} + +fn constructor_getter( + machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + Ok(BuiltinOutcome::Value( + machine + .intrinsics + .global("Iterator") + .expect("Iterator is installed"), + )) +} + +fn constructor_setter( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let key = PropertyKey::Named(EcmaString::encode("constructor")); + set_ignoring_prototype(machine, this, key, argument(args, 0))?; + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) +} + +fn tag_getter( + machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + allocate_string(machine, EcmaString::encode("Iterator")).map(BuiltinOutcome::Value) +} + +fn tag_setter( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let key = machine.to_property_key(machine.intrinsics.builtins.symbol_to_string_tag())?; + set_ignoring_prototype(machine, this, key, argument(args, 0))?; + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) +} + +fn accessor(getter: Value, setter: Value) -> Property { + Property::Accessor { + getter: Some(getter), + setter: Some(setter), + enumerable: false, + configurable: true, + } +} + +pub(super) fn install( + heap: &mut Vec, + globals: &mut BTreeMap, + builtins: &mut BuiltinTable, +) { + let prototype = builtins.iterator_prototype(); + let constructor_function = + install_constructor_function(heap, builtins, "Iterator", 0, constructor::); + define_frozen_data(heap, constructor_function, "prototype", prototype); + globals.insert(EcmaString::encode("Iterator"), constructor_function); + for (name, length, handler) in [ + ("from", 1, from:: as BuiltinHandler), + ("concat", 0, concat::), + ("zip", 1, zip::), + ("zipKeyed", 1, zip_keyed::), + ] { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, constructor_function, name, function); + } + for (name, length, handler) in [ + ("map", 1, map:: as BuiltinHandler), + ("filter", 1, filter::), + ("take", 1, take::), + ("drop", 1, drop::), + ("flatMap", 1, flat_map::), + ("reduce", 1, reduce::), + ("toArray", 0, to_array::), + ("forEach", 1, for_each::), + ("some", 1, some::), + ("every", 1, every::), + ("find", 1, find::), + ] { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, prototype, name, function); + } + let dispose_function = install_function(heap, builtins, "[Symbol.dispose]", 0, dispose::); + define_symbol_property( + heap, + prototype, + builtins.symbol_dispose(), + super::builtin_property(dispose_function), + ); + let get_constructor = install_function( + heap, + builtins, + "get constructor", + 0, + constructor_getter::, + ); + let set_constructor = install_function( + heap, + builtins, + "set constructor", + 1, + constructor_setter::, + ); + let HeapEntry::Object { properties, .. } = &mut heap[heap_index(prototype)] else { + panic!("%Iterator.prototype% is an ordinary object"); + }; + properties.insert( + PropertyKey::Named(EcmaString::encode("constructor")), + accessor(get_constructor, set_constructor), + ); + let get_tag = install_function( + heap, + builtins, + "get [Symbol.toStringTag]", + 0, + tag_getter::, + ); + let set_tag = install_function( + heap, + builtins, + "set [Symbol.toStringTag]", + 1, + tag_setter::, + ); + define_symbol_property( + heap, + prototype, + builtins.symbol_to_string_tag(), + accessor(get_tag, set_tag), + ); + + let helper_prototype = super::collections::ordinary_intrinsic(heap, prototype); + let next = install_function(heap, builtins, "next", 0, helper_next::); + define_data(heap, helper_prototype, "next", next); + let return_method = install_function(heap, builtins, "return", 0, helper_return::); + define_data(heap, helper_prototype, "return", return_method); + super::define_to_string_tag( + heap, + helper_prototype, + builtins.symbol_to_string_tag(), + "Iterator Helper", + ); + let wrap_prototype = super::collections::ordinary_intrinsic(heap, prototype); + let next = install_function(heap, builtins, "next", 0, wrap_next::); + define_data(heap, wrap_prototype, "next", next); + let return_method = install_function(heap, builtins, "return", 0, wrap_return::); + define_data(heap, wrap_prototype, "return", return_method); + builtins.set_iterator_helper_prototypes(helper_prototype, wrap_prototype); +} + +#[cfg(test)] +mod tests { + use super::super::test_support::{TestHost, blank_program, ordinary_object}; + use super::*; + use crate::intrinsics::{BuiltinDef, native_function}; + use crate::{Limits, ThrowOrigin}; + + fn is_type_error(result: Result) -> bool { + matches!( + result, + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + ) + } + + fn invoke( + machine: &mut Machine<'_, TestHost>, + receiver: Value, + name: &str, + args: &[Value], + ) -> Result { + let method = machine.get_named_property(receiver, name)?; + machine.call_value(method, receiver, args) + } + + fn numbers(machine: &mut Machine<'_, TestHost>, values: &[i32]) -> Value { + let elements = values.iter().map(|value| Value::int32(*value as u32)); + let array = allocate_array(machine, elements.collect()).unwrap(); + invoke(machine, array, "values", &[]).unwrap() + } + + fn native( + machine: &mut Machine<'_, TestHost>, + name: &'static str, + handler: BuiltinHandler, + ) -> Value { + let id = machine.intrinsics.builtins.register(BuiltinDef { + name, + length: 1, + handler, + }); + native_function(&mut machine.heap, id, name, 1) + } + + fn collected(machine: &mut Machine<'_, TestHost>, iterator: Value) -> Vec { + let array = invoke(machine, iterator, "toArray", &[]).unwrap(); + machine.array_elements(array).unwrap().unwrap() + } + + fn double( + machine: &mut Machine<'_, TestHost>, + _this: Value, + args: &[Value], + _constructing: bool, + ) -> Result { + let value = value_number(machine.to_number(args[0])?); + Ok(BuiltinOutcome::Value(crate::number_value(value * 2.0))) + } + + fn is_even( + machine: &mut Machine<'_, TestHost>, + _this: Value, + args: &[Value], + _constructing: bool, + ) -> Result { + let value = value_number(machine.to_number(args[0])?); + Ok(BuiltinOutcome::Value(Value::boolean(value % 2.0 == 0.0))) + } + + /// `next` of an object whose `helper` property is the helper consuming it. + fn reenter( + machine: &mut Machine<'_, TestHost>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let helper = machine.get_named_property(this, "helper")?; + let result = invoke(machine, helper, "next", &[]); + assert!(is_type_error(result), "a running helper rejects next"); + Ok(BuiltinOutcome::Value( + machine.iterator_result(Value::UNDEFINED, true)?, + )) + } + + fn record_return( + machine: &mut Machine<'_, TestHost>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + machine.set_data_property(this, "closed", Value::TRUE)?; + Ok(BuiltinOutcome::Value(this)) + } + + #[test] + fn iterator_is_abstract_and_only_a_subclass_constructs_it() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let iterator = machine.intrinsics.global("Iterator").unwrap(); + assert!(is_type_error(machine.construct_value(iterator, &[]))); + assert!(is_type_error(machine.call_value( + iterator, + Value::UNDEFINED, + &[] + ))); + let subclass_prototype = ordinary_object(&mut machine); + let subclass = ordinary_object(&mut machine); + machine + .set_data_property(subclass, "prototype", subclass_prototype) + .unwrap(); + let instance = machine.internal_construct(iterator, &[], subclass).unwrap(); + assert_eq!( + machine.internal_get_prototype_of(instance).unwrap(), + Some(subclass_prototype) + ); + } + + #[test] + fn helpers_compose_and_produce_nothing_until_asked() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let source = numbers(&mut machine, &[1, 2, 3, 4, 5, 6]); + let predicate = native(&mut machine, "is even", is_even); + let filtered = invoke(&mut machine, source, "filter", &[predicate]).unwrap(); + let mapper = native(&mut machine, "double", double); + let mapped = invoke(&mut machine, filtered, "map", &[mapper]).unwrap(); + let dropped = invoke(&mut machine, mapped, "drop", &[Value::int32(1)]).unwrap(); + let taken = invoke(&mut machine, dropped, "take", &[Value::int32(1)]).unwrap(); + assert_eq!(collected(&mut machine, taken), [Value::int32(8)]); + } + + #[test] + fn a_running_helper_rejects_reentry() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.builtins.iterator_prototype(); + let underlying = ordinary_object(&mut machine); + machine + .internal_set_prototype_of(underlying, Some(prototype)) + .unwrap(); + let next = native(&mut machine, "reenter", reenter); + machine.set_data_property(underlying, "next", next).unwrap(); + let mapper = native(&mut machine, "double", double); + let helper = invoke(&mut machine, underlying, "map", &[mapper]).unwrap(); + machine + .set_data_property(underlying, "helper", helper) + .unwrap(); + let result = invoke(&mut machine, helper, "next", &[]).unwrap(); + assert_eq!( + machine.get_named_property(result, "done").unwrap(), + Value::TRUE + ); + } + + #[test] + fn return_closes_the_underlying_iterator_once() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.builtins.iterator_prototype(); + let underlying = ordinary_object(&mut machine); + machine + .internal_set_prototype_of(underlying, Some(prototype)) + .unwrap(); + let close = native(&mut machine, "record return", record_return); + machine + .set_data_property(underlying, "return", close) + .unwrap(); + let mapper = native(&mut machine, "double", double); + let helper = invoke(&mut machine, underlying, "map", &[mapper]).unwrap(); + invoke(&mut machine, helper, "return", &[]).unwrap(); + assert_eq!( + machine.get_named_property(underlying, "closed").unwrap(), + Value::TRUE + ); + machine + .set_data_property(underlying, "closed", Value::FALSE) + .unwrap(); + invoke(&mut machine, helper, "return", &[]).unwrap(); + assert_eq!( + machine.get_named_property(underlying, "closed").unwrap(), + Value::FALSE, + "a completed helper does not close again" + ); + } + + #[test] + fn a_bad_argument_closes_the_receiver_before_reading_next() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.builtins.iterator_prototype(); + let underlying = ordinary_object(&mut machine); + machine + .internal_set_prototype_of(underlying, Some(prototype)) + .unwrap(); + let close = native(&mut machine, "record return", record_return); + machine + .set_data_property(underlying, "return", close) + .unwrap(); + assert!(is_type_error(invoke( + &mut machine, + underlying, + "map", + &[Value::int32(1)] + ))); + assert_eq!( + machine.get_named_property(underlying, "closed").unwrap(), + Value::TRUE + ); + } + + #[test] + fn zip_stops_at_the_shortest_and_pads_to_the_longest() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let iterator = machine.intrinsics.global("Iterator").unwrap(); + let (one, two) = (numbers(&mut machine, &[1]), numbers(&mut machine, &[2, 3])); + let inputs = allocate_array(&mut machine, vec![one, two]).unwrap(); + let zipped = invoke(&mut machine, iterator, "zip", &[inputs]).unwrap(); + assert_eq!(collected(&mut machine, zipped).len(), 1); + + let (one, two) = (numbers(&mut machine, &[1]), numbers(&mut machine, &[2, 3])); + let inputs = allocate_array(&mut machine, vec![one, two]).unwrap(); + let options = ordinary_object(&mut machine); + let longest = allocate_string(&mut machine, EcmaString::encode("longest")).unwrap(); + machine.set_data_property(options, "mode", longest).unwrap(); + let zipped = invoke(&mut machine, iterator, "zip", &[inputs, options]).unwrap(); + let rows = collected(&mut machine, zipped); + assert_eq!(rows.len(), 2); + assert_eq!( + machine.array_elements(rows[1]).unwrap().unwrap(), + [Value::UNDEFINED, Value::int32(3)] + ); + } +} diff --git a/crates/bamts-runtime/src/builtins/json.rs b/crates/bamts-runtime/src/builtins/json.rs deleted file mode 100644 index af4ded89..00000000 --- a/crates/bamts-runtime/src/builtins/json.rs +++ /dev/null @@ -1,1023 +0,0 @@ -use std::collections::{BTreeMap, BTreeSet}; - -use bamts_bytecode::{EcmaString, EcmaStringBuilder}; -use bamts_native::{Decoded, Value}; - -use super::{allocate_array, allocate_string, define_data, install_function, type_error}; -use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; -use crate::{EvalFailure, HeapEntry, Host, Machine, PropertyKey, PropertyMap}; - -// JSON input and callbacks are untrusted; stop recursive Rust traversal well -// before stack exhaustion. -const MAX_JSON_DEPTH: usize = 256; - -fn json_depth_error() -> EvalFailure { - EvalFailure::Runtime(crate::RuntimeErrorKind::CallDepthExceeded { - limit: MAX_JSON_DEPTH, - }) -} - -pub(super) fn install( - heap: &mut Vec, - globals: &mut BTreeMap, - builtins: &mut BuiltinTable, -) { - let json = super::push( - heap, - HeapEntry::Object { - properties: PropertyMap::default(), - prototype: Some(builtins.object_prototype()), - extensible: true, - boxed_primitive: None, - }, - ); - let parse = install_function(heap, builtins, "parse", 2, parse::); - let stringify = install_function(heap, builtins, "stringify", 3, stringify::); - define_data(heap, json, "parse", parse); - define_data(heap, json, "stringify", stringify); - globals.insert(EcmaString::encode("JSON"), json); -} - -fn parse( - machine: &mut Machine<'_, H>, - _: Value, - args: &[Value], - _: bool, -) -> Result { - let source = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; - let value = match Parser::new(source.as_units()).parse(machine) { - Ok(value) => value, - Err(ParseFailure::Runtime(error)) => return Err(error), - Err(ParseFailure::Syntax(message)) => { - let id = machine - .intrinsics - .builtins - .id_named("SyntaxError") - .expect("SyntaxError installed"); - return Err(machine.throw_error(id, message)); - } - }; - let reviver = if let Some(value) = args.get(1).copied() { - machine.is_callable(value)?.then_some(value) - } else { - None - }; - if let Some(reviver) = reviver { - let root = machine - .allocate(HeapEntry::Object { - properties: PropertyMap::default(), - prototype: Some(machine.intrinsics.object_prototype), - extensible: true, - boxed_primitive: None, - }) - .map_err(EvalFailure::Runtime)?; - let key = EcmaString::default(); - machine.create_data_property_key(root, PropertyKey::Named(key.clone()), value)?; - return Ok(BuiltinOutcome::Value(walk_reviver( - machine, root, key, reviver, 0, - )?)); - } - Ok(BuiltinOutcome::Value(value)) -} - -fn walk_reviver( - machine: &mut Machine<'_, H>, - holder: Value, - key: EcmaString, - reviver: Value, - depth: usize, -) -> Result { - let property_key = PropertyKey::Named(key.clone()); - let value = machine.get_property_key(holder, &property_key)?; - if let Some(length) = machine.array_len(value)? { - if depth >= MAX_JSON_DEPTH { - return Err(json_depth_error()); - } - for index in 0..length { - let name = EcmaString::encode(&index.to_string()); - let child = walk_reviver(machine, value, name.clone(), reviver, depth + 1)?; - let key = PropertyKey::Named(name); - if child == Value::UNDEFINED { - machine.delete_property(value, &key)?; - } else { - machine.create_data_property_key(value, key, child)?; - } - } - } else if machine.is_object(value) { - if depth >= MAX_JSON_DEPTH { - return Err(json_depth_error()); - } - for name in machine.enumerable_keys(value)? { - let child = walk_reviver(machine, value, name.clone(), reviver, depth + 1)?; - let key = PropertyKey::Named(name); - if child == Value::UNDEFINED { - machine.delete_property(value, &key)?; - } else { - machine.create_data_property_key(value, key, child)?; - } - } - } - let name = allocate_string(machine, key)?; - machine.call_value(reviver, holder, &[name, value]) -} - -fn stringify( - machine: &mut Machine<'_, H>, - _: Value, - args: &[Value], - _: bool, -) -> Result { - let value = args.first().copied().unwrap_or(Value::UNDEFINED); - let replacer = args - .get(1) - .copied() - .filter(|value| *value != Value::UNDEFINED); - let space = args.get(2).copied().unwrap_or(Value::UNDEFINED); - let gap = match space.decode() { - Some(Decoded::Int32(number)) => { - EcmaString::encode(&" ".repeat(((number as i32).max(0) as usize).min(10))) - } - Some(Decoded::Number(number)) => { - EcmaString::encode(&" ".repeat((number.max(0.0) as usize).min(10))) - } - Some(Decoded::HeapRef(_)) => { - let text = machine.to_string(machine.unbox_primitive_or_self(space)?)?; - text.slice_units(0..text.len_units().min(10)) - .expect("range is bounded by the string length") - } - _ => EcmaString::default(), - }; - let property_list = if let Some(replacer) = replacer { - if let Some(values) = machine.array_elements(replacer)? { - let mut seen = BTreeSet::new(); - let mut keys = Vec::new(); - for value in values { - let key = match value.decode() { - Some(Decoded::Int32(_) | Decoded::Number(_)) => Some(machine.to_string(value)?), - Some(Decoded::HeapRef(_)) => machine.string_value(value), - _ => None, - }; - if let Some(key) = key.filter(|key| seen.insert(key.clone())) { - keys.push(key); - } - } - Some(keys) - } else { - None - } - } else { - None - }; - let callable_replacer = if let Some(value) = replacer { - machine.is_callable(value)?.then_some(value) - } else { - None - }; - let wrapper = machine - .allocate(HeapEntry::Object { - properties: PropertyMap::default(), - prototype: Some(machine.intrinsics.object_prototype), - extensible: true, - boxed_primitive: None, - }) - .map_err(EvalFailure::Runtime)?; - let root_key = EcmaString::default(); - machine.create_data_property_key(wrapper, PropertyKey::Named(root_key.clone()), value)?; - let options = SerializeOptions { - replacer: callable_replacer, - property_list: property_list.as_deref(), - gap: &gap, - }; - let mut stack = Vec::new(); - let result = serialize_property(machine, wrapper, root_key, &options, 0, &mut stack)?; - match result { - Some(text) => Ok(BuiltinOutcome::Value(allocate_string(machine, text)?)), - None => Ok(BuiltinOutcome::Value(Value::UNDEFINED)), - } -} - -struct SerializeOptions<'a> { - replacer: Option, - property_list: Option<&'a [EcmaString]>, - gap: &'a EcmaString, -} - -fn serialize_property( - machine: &mut Machine<'_, H>, - holder: Value, - key: EcmaString, - options: &SerializeOptions<'_>, - depth: usize, - stack: &mut Vec, -) -> Result, EvalFailure> { - if depth >= MAX_JSON_DEPTH { - return Err(json_depth_error()); - } - let mut value = machine.get_property_key(holder, &PropertyKey::Named(key.clone()))?; - if machine.is_object(value) { - let to_json = machine.get_named_property(value, "toJSON")?; - if machine.is_callable(to_json)? { - let key_value = allocate_string(machine, key.clone())?; - value = machine.call_value(to_json, value, &[key_value])?; - } - } - if let Some(replacer) = options.replacer { - let key_value = allocate_string(machine, key)?; - value = machine.call_value(replacer, holder, &[key_value, value])?; - } - value = machine.unbox_primitive_or_self(value)?; - match value.decode() { - Some(Decoded::Null) => Ok(Some(EcmaString::encode("null"))), - Some(Decoded::Boolean(value)) => Ok(Some(EcmaString::encode(if value { - "true" - } else { - "false" - }))), - Some(Decoded::Int32(value)) => Ok(Some(EcmaString::encode(&(value as i32).to_string()))), - Some(Decoded::Number(value)) => { - let text = if value.is_finite() { - crate::format_number(value) - } else { - "null".to_owned() - }; - Ok(Some(EcmaString::encode(&text))) - } - Some(Decoded::HeapRef(_)) => { - if let Some(text) = machine.string_value(value) { - return Ok(Some(quote(&text))); - } - if stack.contains(&value) { - return Err(type_error("Converting circular structure to JSON")); - } - if !machine.is_object(value) { - return Ok(None); - } - stack.push(value); - let result = if let Some(length) = machine.array_len(value)? { - serialize_array(machine, value, length, options, depth, stack) - } else { - serialize_object(machine, value, options, depth, stack) - }; - stack.pop(); - result.map(Some) - } - Some(Decoded::Undefined | Decoded::Hole | Decoded::Uninitialized) | None => Ok(None), - } -} - -fn serialize_array( - machine: &mut Machine<'_, H>, - array: Value, - length: usize, - options: &SerializeOptions<'_>, - depth: usize, - stack: &mut Vec, -) -> Result { - let mut partial = Vec::with_capacity(length); - for index in 0..length { - partial.push( - serialize_property( - machine, - array, - EcmaString::encode(&index.to_string()), - options, - depth + 1, - stack, - )? - .unwrap_or_else(|| EcmaString::encode("null")), - ); - } - Ok(compose(b'[', b']', partial, options.gap, depth)) -} - -fn serialize_object( - machine: &mut Machine<'_, H>, - object: Value, - options: &SerializeOptions<'_>, - depth: usize, - stack: &mut Vec, -) -> Result { - let keys = options - .property_list - .map_or_else(|| machine.enumerable_keys(object), |keys| Ok(keys.to_vec()))?; - let mut partial = Vec::new(); - for key in keys { - if let Some(value) = - serialize_property(machine, object, key.clone(), options, depth + 1, stack)? - { - let mut member = EcmaStringBuilder::new(); - append(&mut member, "e(&key)); - member.push_unit(u16::from(b':')); - if !options.gap.is_empty() { - member.push_unit(u16::from(b' ')); - } - append(&mut member, &value); - partial.push(member.finish()); - } - } - Ok(compose(b'{', b'}', partial, options.gap, depth)) -} - -fn append(output: &mut EcmaStringBuilder, text: &EcmaString) { - for &unit in text.as_units() { - output.push_unit(unit); - } -} - -fn compose( - open: u8, - close: u8, - parts: Vec, - gap: &EcmaString, - depth: usize, -) -> EcmaString { - let mut output = EcmaStringBuilder::new(); - output.push_unit(u16::from(open)); - if parts.is_empty() { - output.push_unit(u16::from(close)); - return output.finish(); - } - if gap.is_empty() { - for (index, part) in parts.iter().enumerate() { - if index != 0 { - output.push_unit(u16::from(b',')); - } - append(&mut output, part); - } - } else { - output.push_unit(u16::from(b'\n')); - for (index, part) in parts.iter().enumerate() { - if index != 0 { - output.push_unit(u16::from(b',')); - output.push_unit(u16::from(b'\n')); - } - for _ in 0..=depth { - append(&mut output, gap); - } - append(&mut output, part); - } - output.push_unit(u16::from(b'\n')); - for _ in 0..depth { - append(&mut output, gap); - } - } - output.push_unit(u16::from(close)); - output.finish() -} - -fn push_escape(output: &mut EcmaStringBuilder, escape: &str) { - output.push_utf8(escape); -} - -fn push_hex_escape(output: &mut EcmaStringBuilder, unit: u16) { - output.push_utf8(&format!("\\u{unit:04x}")); -} - -fn quote(text: &EcmaString) -> EcmaString { - let mut output = EcmaStringBuilder::new(); - output.push_unit(u16::from(b'"')); - let units = text.as_units(); - let mut offset = 0; - while offset < units.len() { - let unit = units[offset]; - match unit { - 0x0022 => push_escape(&mut output, "\\\""), - 0x005C => push_escape(&mut output, "\\\\"), - 0x0008 => push_escape(&mut output, "\\b"), - 0x000C => push_escape(&mut output, "\\f"), - 0x000A => push_escape(&mut output, "\\n"), - 0x000D => push_escape(&mut output, "\\r"), - 0x0009 => push_escape(&mut output, "\\t"), - 0x0000..=0x001F => push_hex_escape(&mut output, unit), - 0xD800..=0xDBFF - if units - .get(offset + 1) - .is_some_and(|next| (0xDC00..=0xDFFF).contains(next)) => - { - output.push_unit(unit); - offset += 1; - output.push_unit(units[offset]); - } - 0xD800..=0xDFFF => push_hex_escape(&mut output, unit), - _ => output.push_unit(unit), - } - offset += 1; - } - output.push_unit(u16::from(b'"')); - output.finish() -} - -struct Parser<'a> { - source: &'a [u16], - pos: usize, -} - -enum ParseFailure { - Syntax(String), - Runtime(EvalFailure), -} - -type ParseResult = Result; - -impl<'a> Parser<'a> { - fn new(source: &'a [u16]) -> Self { - Self { source, pos: 0 } - } - - fn parse(mut self, machine: &mut Machine<'_, H>) -> ParseResult { - self.ws(); - let value = self.value(machine, 0)?; - self.ws(); - if self.pos != self.source.len() { - return Err(ParseFailure::Syntax(self.error_unexpected())); - } - Ok(value) - } - - fn ws(&mut self) { - while self - .peek() - .is_some_and(|unit| matches!(unit, 0x20 | 0x0A | 0x0D | 0x09)) - { - self.pos += 1; - } - } - - fn peek(&self) -> Option { - self.source.get(self.pos).copied() - } - - fn value(&mut self, machine: &mut Machine<'_, H>, depth: usize) -> ParseResult { - self.ws(); - match self.peek() { - Some(unit) if unit == u16::from(b'n') => { - self.literal("null")?; - Ok(Value::NULL) - } - Some(unit) if unit == u16::from(b't') => { - self.literal("true")?; - Ok(Value::TRUE) - } - Some(unit) if unit == u16::from(b'f') => { - self.literal("false")?; - Ok(Value::FALSE) - } - Some(unit) if unit == u16::from(b'"') => { - let string = self.string()?; - allocate_string(machine, string).map_err(ParseFailure::Runtime) - } - Some(unit) if unit == u16::from(b'[') => self.array(machine, depth), - Some(unit) if unit == u16::from(b'{') => self.object(machine, depth), - Some(unit) - if unit == u16::from(b'-') - || (u16::from(b'0')..=u16::from(b'9')).contains(&unit) => - { - self.number() - } - _ => Err(ParseFailure::Syntax(self.error_unexpected())), - } - } - - fn literal(&mut self, literal: &str) -> ParseResult<()> { - let matches = self.source[self.pos..] - .iter() - .zip(literal.bytes()) - .take(literal.len()) - .all(|(unit, byte)| *unit == u16::from(byte)); - if matches && self.source.len() >= self.pos + literal.len() { - self.pos += literal.len(); - Ok(()) - } else { - Err(ParseFailure::Syntax(self.error_unexpected())) - } - } - - fn string(&mut self) -> ParseResult { - self.pos += 1; - let mut output = EcmaStringBuilder::new(); - loop { - let Some(unit) = self.peek() else { - return Err(ParseFailure::Syntax(self.at("Unterminated string in JSON"))); - }; - self.pos += 1; - if unit == u16::from(b'"') { - return Ok(output.finish()); - } - if unit == u16::from(b'\\') { - let Some(escape) = self.peek() else { - return Err(ParseFailure::Syntax(self.at("Unterminated string in JSON"))); - }; - self.pos += 1; - match escape { - unit if unit == u16::from(b'"') => output.push_unit(u16::from(b'"')), - unit if unit == u16::from(b'\\') => output.push_unit(u16::from(b'\\')), - unit if unit == u16::from(b'/') => output.push_unit(u16::from(b'/')), - unit if unit == u16::from(b'b') => output.push_unit(0x0008), - unit if unit == u16::from(b'f') => output.push_unit(0x000C), - unit if unit == u16::from(b'n') => output.push_unit(0x000A), - unit if unit == u16::from(b'r') => output.push_unit(0x000D), - unit if unit == u16::from(b't') => output.push_unit(0x0009), - unit if unit == u16::from(b'u') => output.push_unit(self.hex_unit()?), - _ => return Err(ParseFailure::Syntax(self.error_unexpected())), - } - } else if unit <= 0x001F { - return Err(ParseFailure::Syntax(self.error_unexpected())); - } else { - output.push_unit(unit); - } - } - } - - fn hex_unit(&mut self) -> ParseResult { - if self.pos + 4 > self.source.len() { - return Err(ParseFailure::Syntax(self.error_unexpected())); - } - let mut value = 0_u16; - for &unit in &self.source[self.pos..self.pos + 4] { - let digit = match unit { - unit if (u16::from(b'0')..=u16::from(b'9')).contains(&unit) => { - unit - u16::from(b'0') - } - unit if (u16::from(b'a')..=u16::from(b'f')).contains(&unit) => { - unit - u16::from(b'a') + 10 - } - unit if (u16::from(b'A')..=u16::from(b'F')).contains(&unit) => { - unit - u16::from(b'A') + 10 - } - _ => return Err(ParseFailure::Syntax(self.error_unexpected())), - }; - value = value * 16 + digit; - } - self.pos += 4; - Ok(value) - } - - fn number(&mut self) -> ParseResult { - let start = self.pos; - if self.peek() == Some(u16::from(b'-')) { - self.pos += 1; - } - if self.peek() == Some(u16::from(b'0')) { - self.pos += 1; - if self - .peek() - .is_some_and(|unit| (u16::from(b'0')..=u16::from(b'9')).contains(&unit)) - { - return Err(ParseFailure::Syntax(self.at("Unexpected number in JSON"))); - } - } else { - self.digits()?; - } - if self.peek() == Some(u16::from(b'.')) { - self.pos += 1; - self.digits()?; - } - if self - .peek() - .is_some_and(|unit| unit == u16::from(b'e') || unit == u16::from(b'E')) - { - self.pos += 1; - if self - .peek() - .is_some_and(|unit| unit == u16::from(b'+') || unit == u16::from(b'-')) - { - self.pos += 1; - } - self.digits()?; - } - let bytes: Vec = self.source[start..self.pos] - .iter() - .map(|unit| *unit as u8) - .collect(); - let number = std::str::from_utf8(&bytes) - .ok() - .and_then(|text| text.parse::().ok()) - .ok_or_else(|| ParseFailure::Syntax(self.error_unexpected()))?; - Ok(crate::number_value(number)) - } - - fn digits(&mut self) -> ParseResult<()> { - let start = self.pos; - while self - .peek() - .is_some_and(|unit| (u16::from(b'0')..=u16::from(b'9')).contains(&unit)) - { - self.pos += 1; - } - (self.pos != start) - .then_some(()) - .ok_or_else(|| ParseFailure::Syntax(self.error_unexpected())) - } - - fn array(&mut self, machine: &mut Machine<'_, H>, depth: usize) -> ParseResult { - if depth >= MAX_JSON_DEPTH { - return Err(ParseFailure::Runtime(json_depth_error())); - } - self.pos += 1; - self.ws(); - let mut output = Vec::new(); - if self.peek() == Some(u16::from(b']')) { - self.pos += 1; - return allocate_array(machine, output).map_err(ParseFailure::Runtime); - } - loop { - output.push(self.value(machine, depth + 1)?); - self.ws(); - match self.peek() { - Some(unit) if unit == u16::from(b',') => { - self.pos += 1; - self.ws(); - } - Some(unit) if unit == u16::from(b']') => { - self.pos += 1; - break; - } - _ => return Err(ParseFailure::Syntax(self.error_unexpected())), - } - } - allocate_array(machine, output).map_err(ParseFailure::Runtime) - } - - fn object( - &mut self, - machine: &mut Machine<'_, H>, - depth: usize, - ) -> ParseResult { - if depth >= MAX_JSON_DEPTH { - return Err(ParseFailure::Runtime(json_depth_error())); - } - self.pos += 1; - self.ws(); - let object = machine - .allocate(HeapEntry::Object { - properties: PropertyMap::default(), - prototype: Some(machine.intrinsics.object_prototype), - extensible: true, - boxed_primitive: None, - }) - .map_err(EvalFailure::Runtime) - .map_err(ParseFailure::Runtime)?; - if self.peek() == Some(u16::from(b'}')) { - self.pos += 1; - return Ok(object); - } - loop { - if self.peek() != Some(u16::from(b'"')) { - return Err(ParseFailure::Syntax( - self.at("Expected property name or '}' in JSON"), - )); - } - let key = self.string()?; - self.ws(); - if self.peek() != Some(u16::from(b':')) { - return Err(ParseFailure::Syntax( - self.at("Expected ':' after property name in JSON"), - )); - } - self.pos += 1; - let value = self.value(machine, depth + 1)?; - machine - .create_data_property_key(object, PropertyKey::Named(key), value) - .map_err(ParseFailure::Runtime)?; - self.ws(); - match self.peek() { - Some(unit) if unit == u16::from(b',') => { - self.pos += 1; - self.ws(); - } - Some(unit) if unit == u16::from(b'}') => { - self.pos += 1; - break; - } - _ => return Err(ParseFailure::Syntax(self.error_unexpected())), - } - } - Ok(object) - } - - fn error_unexpected(&self) -> String { - match self.peek() { - None => self.at("Unexpected end of JSON input"), - Some(unit) if unit <= 0x7F => self.at(&format!( - "Unexpected token '{}' in JSON", - char::from_u32(u32::from(unit)).expect("ASCII unit is scalar") - )), - Some(unit) => self.at(&format!("Unexpected code unit U+{unit:04X} in JSON")), - } - } - - fn at(&self, message: &str) -> String { - let prefix = &self.source[..self.pos.min(self.source.len())]; - let line = prefix - .iter() - .filter(|unit| **unit == u16::from(b'\n')) - .count() - + 1; - let column = prefix - .iter() - .rposition(|unit| *unit == u16::from(b'\n')) - .map_or(prefix.len() + 1, |offset| prefix.len() - offset); - format!( - "{message} at position {} (line {line} column {column})", - self.pos - ) - } -} - -#[cfg(test)] -mod tests { - use super::super::test_support::{TestHost, blank_program}; - use super::*; - use crate::intrinsics::{BuiltinDef, BuiltinHandler, native_function}; - use crate::{Limits, RuntimeErrorKind, ThrowOrigin}; - - fn call_json(machine: &mut Machine<'_, TestHost>, method: &str, source: EcmaString) -> Value { - let json = machine.intrinsics.global("JSON").expect("JSON exists"); - let function = machine - .get_named_property(json, method) - .expect("method exists"); - let source = allocate_string(machine, source).expect("string allocation succeeds"); - machine - .call_value(function, json, &[source]) - .expect("JSON call succeeds") - } - - fn native( - machine: &mut Machine<'_, TestHost>, - name: &'static str, - handler: BuiltinHandler, - ) -> Value { - let id = machine.intrinsics.builtins.register(BuiltinDef { - name, - length: 2, - handler, - }); - native_function(&mut machine.heap, id, name, 2) - } - - fn identity_reviver( - _: &mut Machine<'_, TestHost>, - _: Value, - args: &[Value], - _: bool, - ) -> Result { - Ok(BuiltinOutcome::Value( - args.get(1).copied().unwrap_or(Value::UNDEFINED), - )) - } - - fn nested_array(machine: &mut Machine<'_, TestHost>, depth: usize) -> Value { - let mut value = Value::int32(0); - for _ in 0..depth { - value = allocate_array(machine, vec![value]).expect("array allocation succeeds"); - } - value - } - - #[test] - fn parse_ignores_non_callable_revivers() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let json = machine.intrinsics.global("JSON").expect("JSON exists"); - let parse = machine - .get_named_property(json, "parse") - .expect("JSON.parse exists"); - let object = machine - .allocate(HeapEntry::Object { - properties: PropertyMap::default(), - prototype: Some(machine.intrinsics.object_prototype), - extensible: true, - boxed_primitive: None, - }) - .expect("object allocation succeeds"); - - for reviver in [Value::NULL, Value::FALSE, object] { - let source = allocate_string(&mut machine, EcmaString::encode("1")) - .expect("string allocation succeeds"); - let value = machine - .call_value(parse, json, &[source, reviver]) - .expect("non-callable reviver is ignored"); - assert_eq!(value, Value::int32(1)); - } - } - - #[test] - fn parse_preserves_utf16_surrogate_units() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let lone = call_json(&mut machine, "parse", EcmaString::encode("\"\\uD83D\"")); - assert_eq!(machine.string_value(lone).unwrap().as_units(), &[0xD83D]); - let pair = call_json( - &mut machine, - "parse", - EcmaString::encode("\"\\uD83D\\uDE03\""), - ); - assert_eq!( - machine.string_value(pair).unwrap().as_units(), - &[0xD83D, 0xDE03] - ); - } - - #[test] - fn parse_uses_syntax_error_only_for_grammar_failures() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let json = machine.intrinsics.global("JSON").expect("JSON exists"); - let parse = machine - .get_named_property(json, "parse") - .expect("JSON.parse exists"); - let source = allocate_string(&mut machine, EcmaString::encode("{]")) - .expect("string allocation succeeds"); - let error = machine - .call_value(parse, json, &[source]) - .expect_err("malformed JSON throws"); - let EvalFailure::ThrowValue(value) = error else { - panic!("malformed JSON must throw a SyntaxError"); - }; - let syntax_error = machine - .intrinsics - .global("SyntaxError") - .expect("SyntaxError exists"); - let prototype = machine - .get_named_property(syntax_error, "prototype") - .expect("SyntaxError.prototype exists"); - assert!( - machine - .inherits_from_prototype(value, prototype) - .expect("error has a valid prototype chain") - ); - } - - #[test] - fn parse_preserves_allocation_failures_as_runtime_errors() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new( - &module, - &mut host, - Limits { - max_heap_slots: 1, - ..Limits::default() - }, - ); - let json = machine.intrinsics.global("JSON").expect("JSON exists"); - let parse = machine - .get_named_property(json, "parse") - .expect("JSON.parse exists"); - let source = allocate_string(&mut machine, EcmaString::encode("\"x\"")) - .expect("input consumes the final slot"); - assert!(matches!( - machine.call_value(parse, json, &[source]), - Err(EvalFailure::Runtime( - RuntimeErrorKind::HeapSlotLimitExceeded { .. } - )) - )); - } - - #[test] - fn nested_json_round_trips_within_the_depth_budget() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let source = format!("{}0{}", "[".repeat(16), "]".repeat(16)); - let value = call_json(&mut machine, "parse", EcmaString::encode(&source)); - let json = machine.intrinsics.global("JSON").expect("JSON exists"); - let stringify = machine - .get_named_property(json, "stringify") - .expect("JSON.stringify exists"); - let output = machine - .call_value(stringify, json, &[value]) - .expect("nested JSON stringifies"); - assert!( - machine - .string_value(output) - .expect("string result") - .eq_ascii(&source) - ); - } - - #[test] - fn parser_reviver_and_stringify_bound_hostile_nesting() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let source = format!( - "{}0{}", - "[".repeat(MAX_JSON_DEPTH + 1), - "]".repeat(MAX_JSON_DEPTH + 1) - ); - let json = machine.intrinsics.global("JSON").expect("JSON exists"); - let parse = machine - .get_named_property(json, "parse") - .expect("JSON.parse exists"); - let source = allocate_string(&mut machine, EcmaString::encode(&source)) - .expect("string allocation succeeds"); - assert!(matches!( - machine.call_value(parse, json, &[source]), - Err(EvalFailure::Runtime(RuntimeErrorKind::CallDepthExceeded { - limit: MAX_JSON_DEPTH - })) - )); - - let value = nested_array(&mut machine, MAX_JSON_DEPTH + 1); - let wrapper = machine - .allocate(HeapEntry::Object { - properties: PropertyMap::default(), - prototype: Some(machine.intrinsics.object_prototype), - extensible: true, - boxed_primitive: None, - }) - .expect("wrapper allocation succeeds"); - let key = EcmaString::default(); - machine - .create_data_property_key(wrapper, PropertyKey::Named(key.clone()), value) - .expect("wrapper property set succeeds"); - let reviver = native(&mut machine, "identityReviver", identity_reviver); - assert!(matches!( - walk_reviver(&mut machine, wrapper, key, reviver, 0), - Err(EvalFailure::Runtime(RuntimeErrorKind::CallDepthExceeded { - limit: MAX_JSON_DEPTH - })) - )); - - let stringify = machine - .get_named_property(json, "stringify") - .expect("JSON.stringify exists"); - assert!(matches!( - machine.call_value(stringify, json, &[value]), - Err(EvalFailure::Runtime(RuntimeErrorKind::CallDepthExceeded { - limit: MAX_JSON_DEPTH - })) - )); - } - - #[test] - fn stringify_escapes_unpaired_surrogates() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let json = machine.intrinsics.global("JSON").expect("JSON exists"); - let stringify = machine.get_named_property(json, "stringify").unwrap(); - let value = allocate_string(&mut machine, EcmaString::from_units(&[0xD800])).unwrap(); - let result = machine.call_value(stringify, json, &[value]).unwrap(); - assert!( - machine - .string_value(result) - .unwrap() - .eq_ascii("\"\\ud800\"") - ); - } - - #[test] - fn stringify_preserves_sparse_array_order() { - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - let json = machine.intrinsics.global("JSON").expect("JSON exists"); - let stringify = machine.get_named_property(json, "stringify").unwrap(); - let value = allocate_array( - &mut machine, - vec![Value::int32(1), Value::HOLE, Value::int32(3)], - ) - .expect("array allocation succeeds"); - let result = machine - .call_value(stringify, json, &[value]) - .expect("stringify sparse array succeeds"); - let text = machine.string_value(result).expect("result is a string"); - assert!(text.eq_ascii("[1,null,3]")); - } - - #[test] - fn parse_creates_own_property_ignoring_prototype_setter() { - use bamts_bytecode::AccessorKind; - - let module = blank_program(""); - let mut host = TestHost; - let mut machine = Machine::new(&module, &mut host, Limits::default()); - // Install a throwing setter on Object.prototype for "x". - // If JSON.parse used [[Set]], parsing {"x":1} would invoke this setter and throw. - // With CreateDataProperty ([[DefineOwnProperty]]) it must create an own data property. - let setter = native(&mut machine, "evilSetter", |_, _, _, _| { - Err(EvalFailure::Throw(ThrowOrigin::TypeError { - operation: "prototype setter called", - })) - }); - machine - .define_accessor( - machine.intrinsics.object_prototype, - PropertyKey::Named(EcmaString::encode("x")), - setter, - AccessorKind::Setter, - ) - .expect("define setter succeeds"); - let value = call_json(&mut machine, "parse", EcmaString::encode("{\"x\":1}")); - assert!(machine.is_object(value)); - let x = machine - .get_property_key(value, &PropertyKey::Named(EcmaString::encode("x"))) - .expect("get succeeds"); - assert_eq!(x, Value::int32(1)); - } -} diff --git a/crates/bamts-runtime/src/builtins/map_set_edge.rs b/crates/bamts-runtime/src/builtins/map_set_edge.rs index 08b0746b..bfd21904 100644 --- a/crates/bamts-runtime/src/builtins/map_set_edge.rs +++ b/crates/bamts-runtime/src/builtins/map_set_edge.rs @@ -36,10 +36,56 @@ pub(super) fn install( weak_set_constructor::, ); + for constructor in [map.constructor, set.constructor] { + let species = install_function( + heap, + builtins, + "get [Symbol.species]", + 0, + super::regexp_protocol::species_getter::, + ); + super::define_symbol_property( + heap, + constructor, + builtins.symbol_species(), + crate::Property::Accessor { + getter: Some(species), + setter: None, + enumerable: false, + configurable: true, + }, + ); + } + let group_by = install_function(heap, builtins, "groupBy", 2, map_group_by::); define_constructor_data(heap, map.constructor, "groupBy", group_by); let map_set = install_function(heap, builtins, "set", 2, strong_map_set::); define_data(heap, map.prototype, "set", map_set); + for (target, name, handler) in [ + ( + map.prototype, + "getOrInsert", + map_get_or_insert:: as BuiltinHandler, + ), + ( + map.prototype, + "getOrInsertComputed", + map_get_or_insert_computed::, + ), + ( + weak_map.prototype, + "getOrInsert", + weak_map_get_or_insert::, + ), + ( + weak_map.prototype, + "getOrInsertComputed", + weak_map_get_or_insert_computed::, + ), + ] { + let function = install_function(heap, builtins, name, 2, handler); + define_data(heap, target, name, function); + } let set_add = install_function(heap, builtins, "add", 1, strong_set_add::); define_data(heap, set.prototype, "add", set_add); @@ -201,6 +247,112 @@ fn strong_set_add( Ok(BuiltinOutcome::Value(this)) } +/// The key `getOrInsert*` stores: `Map` canonicalizes `-0`, and `WeakMap` +/// refuses anything that cannot be held weakly. +fn upsert_key( + machine: &Machine<'_, H>, + key: Value, + kind: CollectionKind, +) -> Result { + if kind == CollectionKind::WeakMap { + super::collections::require_weak_key(machine, key)?; + return Ok(key); + } + Ok(canonical_collection_key(key)) +} + +fn existing_value(machine: &Machine<'_, H>, slot: usize, key: Value) -> Option { + let HeapEntry::Collection { entries, index, .. } = &machine.heap[slot] else { + unreachable!("collection brand was checked") + }; + index + .get(machine, entries, key) + .map(|entry_index| entries[entry_index].value) +} + +fn get_or_insert( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + kind: CollectionKind, +) -> Result { + let slot = super::collections::collection_slot(machine, this, kind)?; + let key = upsert_key( + machine, + args.first().copied().unwrap_or(Value::UNDEFINED), + kind, + )?; + if let Some(existing) = existing_value(machine, slot, key) { + return Ok(BuiltinOutcome::Value(existing)); + } + let value = args.get(1).copied().unwrap_or(Value::UNDEFINED); + super::collections::append_collection_entry(machine, slot, key, value)?; + Ok(BuiltinOutcome::Value(value)) +} + +/// The callback may change the collection, so its result is stored by a fresh +/// lookup rather than at a position found before the call. +fn get_or_insert_computed( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + kind: CollectionKind, +) -> Result { + let slot = super::collections::collection_slot(machine, this, kind)?; + let key = args.first().copied().unwrap_or(Value::UNDEFINED); + let key = upsert_key(machine, key, kind)?; + let callback = args.get(1).copied().unwrap_or(Value::UNDEFINED); + if !machine.is_callable(callback)? { + return Err(type_error("getOrInsertComputed callback is not callable")); + } + if let Some(existing) = existing_value(machine, slot, key) { + return Ok(BuiltinOutcome::Value(existing)); + } + let depth = machine.native_roots.len(); + machine.push_native_roots(depth, &[this, key, callback]); + let value = machine.call_value(callback, Value::UNDEFINED, &[key]); + machine.pop_native_roots(depth); + let value = value?; + super::collections::map_put(machine, this, key, value, kind)?; + Ok(BuiltinOutcome::Value(value)) +} + +fn map_get_or_insert( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + get_or_insert(machine, this, args, CollectionKind::Map) +} + +fn map_get_or_insert_computed( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + get_or_insert_computed(machine, this, args, CollectionKind::Map) +} + +fn weak_map_get_or_insert( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + get_or_insert(machine, this, args, CollectionKind::WeakMap) +} + +fn weak_map_get_or_insert_computed( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + get_or_insert_computed(machine, this, args, CollectionKind::WeakMap) +} + fn canonical_collection_key(value: Value) -> Value { match value.decode() { Some(Decoded::Number(number)) if number == 0.0 && number.is_sign_negative() => { @@ -525,6 +677,178 @@ mod tests { test(&mut machine); } + fn invoke( + machine: &mut Machine<'_, TestHost>, + receiver: Value, + name: &str, + args: &[Value], + ) -> Result { + let method = machine.get_named_property(receiver, name).unwrap(); + machine.call_value(method, receiver, args) + } + + fn compute_seven( + _machine: &mut Machine<'_, TestHost>, + _this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + Ok(BuiltinOutcome::Value(Value::int32(7))) + } + + /// Stores into the map before returning, as a user callback may. + fn insert_during_callback( + machine: &mut Machine<'_, TestHost>, + _this: Value, + args: &[Value], + _constructing: bool, + ) -> Result { + let global = machine.intrinsics.global("globalThis").unwrap(); + let map = machine.get_named_property(global, "upsertTarget").unwrap(); + invoke(machine, map, "set", &[args[0], Value::int32(1)])?; + Ok(BuiltinOutcome::Value(Value::int32(2))) + } + + fn native_callback( + machine: &mut Machine<'_, TestHost>, + name: &'static str, + handler: crate::intrinsics::BuiltinHandler, + ) -> Value { + let id = machine.intrinsics.builtins.register(BuiltinDef { + name, + length: 1, + handler, + }); + crate::intrinsics::native_function(&mut machine.heap, id, name, 1) + } + + fn new_collection(machine: &mut Machine<'_, TestHost>, name: &str) -> Value { + let constructor = machine.intrinsics.global(name).unwrap(); + machine.construct_value(constructor, &[]).unwrap() + } + + #[test] + fn get_or_insert_keeps_an_existing_value_and_canonicalizes_negative_zero() { + with_machine(|machine| { + let map = new_collection(machine, "Map"); + let negative_zero = Value::number(-0.0); + assert_eq!( + invoke( + machine, + map, + "getOrInsert", + &[negative_zero, Value::int32(1)] + ) + .unwrap(), + Value::int32(1) + ); + assert_eq!( + invoke( + machine, + map, + "getOrInsert", + &[Value::int32(0), Value::int32(2)] + ) + .unwrap(), + Value::int32(1), + "-0 and +0 are one key" + ); + let keys = invoke(machine, map, "keys", &[]).unwrap(); + let first = invoke(machine, keys, "next", &[]).unwrap(); + let key = machine.get_named_property(first, "value").unwrap(); + assert!( + matches!(key.decode(), Some(Decoded::Number(zero)) if zero.is_sign_positive()) + || key == Value::int32(0), + "the stored key is +0" + ); + }); + } + + #[test] + fn get_or_insert_computed_calls_back_only_for_a_missing_key() { + with_machine(|machine| { + let map = new_collection(machine, "Map"); + let compute = native_callback(machine, "compute seven", compute_seven); + assert_eq!( + invoke( + machine, + map, + "getOrInsertComputed", + &[Value::int32(1), compute] + ) + .unwrap(), + Value::int32(7) + ); + let non_callable = Value::int32(0); + assert!(matches!( + invoke( + machine, + map, + "getOrInsertComputed", + &[Value::int32(1), non_callable] + ), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + }); + } + + #[test] + fn a_key_stored_by_the_callback_is_overwritten_not_duplicated() { + with_machine(|machine| { + let map = new_collection(machine, "Map"); + let global = machine.intrinsics.global("globalThis").unwrap(); + machine + .set_data_property(global, "upsertTarget", map) + .unwrap(); + let callback = + native_callback(machine, "insert during callback", insert_during_callback); + assert_eq!( + invoke( + machine, + map, + "getOrInsertComputed", + &[Value::int32(5), callback] + ) + .unwrap(), + Value::int32(2) + ); + assert_eq!( + invoke(machine, map, "get", &[Value::int32(5)]).unwrap(), + Value::int32(2) + ); + assert_eq!( + machine.get_named_property(map, "size").unwrap(), + Value::int32(1) + ); + }); + } + + #[test] + fn weak_map_get_or_insert_rejects_keys_that_cannot_be_held_weakly() { + with_machine(|machine| { + let weak_map = new_collection(machine, "WeakMap"); + let key = ordinary_object(machine); + assert_eq!( + invoke(machine, weak_map, "getOrInsert", &[key, Value::int32(1)]).unwrap(), + Value::int32(1) + ); + assert!(matches!( + invoke(machine, weak_map, "getOrInsert", &[Value::int32(1), key]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + let compute = native_callback(machine, "compute seven", compute_seven); + assert!(matches!( + invoke( + machine, + weak_map, + "getOrInsertComputed", + &[Value::int32(1), compute] + ), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + }); + } + fn builtin_id(machine: &Machine<'_, TestHost>, value: Value) -> crate::intrinsics::BuiltinId { let slot = machine.runtime_slot(value).unwrap().unwrap(); match machine.heap[slot] { diff --git a/crates/bamts-runtime/src/builtins/math_edge.rs b/crates/bamts-runtime/src/builtins/math_edge.rs index 62a501c3..279e01f6 100644 --- a/crates/bamts-runtime/src/builtins/math_edge.rs +++ b/crates/bamts-runtime/src/builtins/math_edge.rs @@ -1,8 +1,9 @@ #[cfg(test)] use bamts_bytecode::EcmaString; -use bamts_native::Value; +use bamts_bytecode::IteratorKind; +use bamts_native::{Decoded, Value}; -use super::{define_data, define_frozen_data, install_function, value_number}; +use super::{define_data, define_frozen_data, install_function, type_error, value_number}; use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; use crate::{EvalFailure, HeapEntry, Host, Machine}; @@ -41,7 +42,7 @@ const CONSTANTS: [(&str, f64); 8] = [ ("SQRT2", std::f64::consts::SQRT_2), ]; -fn functions() -> [(&'static str, u32, BuiltinHandler); 29] { +fn functions() -> [(&'static str, u32, BuiltinHandler); 30] { [ ("acos", 1, acos:: as BuiltinHandler), ("acosh", 1, acosh::), @@ -69,6 +70,7 @@ fn functions() -> [(&'static str, u32, BuiltinHandler); 29] { ("sign", 1, sign::), ("sin", 1, sin::), ("sinh", 1, sinh::), + ("sumPrecise", 1, sum_precise::), ("tan", 1, tan::), ("tanh", 1, tanh::), ("trunc", 1, trunc::), @@ -79,6 +81,182 @@ fn argument(args: &[Value], index: usize) -> Value { args.get(index).copied().unwrap_or(Value::UNDEFINED) } +/// Enough limbs for any sum the iterator protocol can produce: a finite Number +/// is below 2^2098 of these units, and a list holds fewer than 2^53 of them. +const EXACT_SUM_LIMBS: usize = 36; + +/// A two's-complement fixed-point total in units of 2^-1074, the spacing of +/// every Number. Each finite Number is a whole count of those units, so the +/// total is exact and `Math.sumPrecise` rounds exactly once, at the end. +struct ExactSum { + limbs: [u64; EXACT_SUM_LIMBS], +} + +impl ExactSum { + fn new() -> Self { + Self { + limbs: [0; EXACT_SUM_LIMBS], + } + } + + fn add(&mut self, value: f64) { + let bits = value.to_bits(); + let biased_exponent = (bits >> 52) & 0x7ff; + let fraction = bits & ((1 << 52) - 1); + let (mantissa, shift) = if biased_exponent == 0 { + (fraction, 0) + } else { + (fraction | (1 << 52), biased_exponent - 1) + }; + let first = (shift / 64) as usize; + let wide = u128::from(mantissa) << (shift % 64); + let parts = [wide as u64, (wide >> 64) as u64]; + let mut carry = false; + for (offset, limb) in self.limbs[first..].iter_mut().enumerate() { + let part = parts.get(offset).copied().unwrap_or(0); + if offset >= parts.len() && !carry { + break; + } + let (partial, first_carry, second_carry) = if value.is_sign_negative() { + let (partial, borrowed) = limb.overflowing_sub(part); + let (partial, again) = partial.overflowing_sub(u64::from(carry)); + (partial, borrowed, again) + } else { + let (partial, carried) = limb.overflowing_add(part); + let (partial, again) = partial.overflowing_add(u64::from(carry)); + (partial, carried, again) + }; + *limb = partial; + carry = first_carry || second_carry; + } + } + + /// The total rounded to the nearest Number, ties to even. An exact zero is + /// `+0`; the all-`-0` case never reaches here. + fn round(&self) -> f64 { + let negative = self.limbs[EXACT_SUM_LIMBS - 1] >> 63 == 1; + let mut magnitude = self.limbs; + if negative { + let mut carry = true; + for limb in &mut magnitude { + let (negated, carried) = (!*limb).overflowing_add(u64::from(carry)); + *limb = negated; + carry = carried; + } + } + let Some(top) = magnitude.iter().rposition(|limb| *limb != 0) else { + return 0.0; + }; + let bit_length = top * 64 + (64 - magnitude[top].leading_zeros() as usize); + let result = if bit_length <= 53 { + // The encoding of Numbers below 2^-1021 is continuous, so a count + // of units that fits in 53 bits is its own bit pattern. + f64::from_bits(magnitude[0]) + } else { + round_magnitude(&magnitude, bit_length) + }; + if negative { -result } else { result } + } +} + +fn round_magnitude(magnitude: &[u64; EXACT_SUM_LIMBS], bit_length: usize) -> f64 { + let bit = |position: usize| magnitude[position / 64] >> (position % 64) & 1 == 1; + let mut shift = bit_length - 53; + let (limb, offset) = (shift / 64, shift % 64); + let low = magnitude[limb] >> offset; + let high = match offset { + 0 => 0, + _ => magnitude + .get(limb + 1) + .map_or(0, |next| next << (64 - offset)), + }; + let mut mantissa = (low | high) & ((1 << 53) - 1); + let round_position = shift - 1; + let sticky = magnitude[..round_position / 64] + .iter() + .any(|limb| *limb != 0) + || magnitude[round_position / 64] & ((1u64 << (round_position % 64)) - 1) != 0; + if bit(round_position) && (sticky || mantissa & 1 == 1) { + mantissa += 1; + if mantissa == 1 << 53 { + mantissa >>= 1; + shift += 1; + } + } + let biased_exponent = shift as u64 + 1; + if biased_exponent >= 0x7ff { + return f64::INFINITY; + } + f64::from_bits((biased_exponent << 52) | (mantissa & ((1 << 52) - 1))) +} + +/// `Math.sumPrecise(items)`: the exact sum of an iterable of Numbers. A +/// non-Number closes the iterator; infinities and NaN are only recorded, since +/// later items must still be checked. +fn sum_precise( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let items = argument(args, 0); + if matches!(items.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Err(type_error("Math.sumPrecise requires an iterable")); + } + let iterator = machine.create_iterator(items, IteratorKind::Sync)?; + let depth = machine.native_roots.len(); + machine.push_native_roots(depth, &[iterator]); + let total = sum_iterator(machine, iterator); + machine.pop_native_roots(depth); + Ok(BuiltinOutcome::Value( + number_value_preserving_negative_zero(total?), + )) +} + +fn sum_iterator( + machine: &mut Machine<'_, H>, + iterator: Value, +) -> Result { + let mut sum = ExactSum::new(); + let (mut saw_nan, mut saw_positive, mut saw_negative) = (false, false, false); + let mut only_negative_zero = true; + loop { + let (done, next) = machine.iterator_next(iterator)?; + if done { + break; + } + if !matches!(next.decode(), Some(Decoded::Int32(_) | Decoded::Number(_))) { + let (closed, _) = machine.close_iterator_raw(iterator); + if let Err(EvalFailure::Runtime(kind)) = closed { + return Err(EvalFailure::Runtime(kind)); + } + return Err(type_error("Math.sumPrecise requires Number items")); + } + let number = value_number(next); + if number.is_nan() { + saw_nan = true; + } else if number == f64::INFINITY { + saw_positive = true; + } else if number == f64::NEG_INFINITY { + saw_negative = true; + } else { + only_negative_zero &= number == 0.0 && number.is_sign_negative(); + sum.add(number); + } + } + Ok(if saw_nan || (saw_positive && saw_negative) { + f64::NAN + } else if saw_positive { + f64::INFINITY + } else if saw_negative { + f64::NEG_INFINITY + } else if only_negative_zero { + -0.0 + } else { + sum.round() + }) +} + /// `? ToNumber(arg)` on the observable path: an object argument runs its /// `valueOf`/`toString` hook and propagates an abrupt completion. fn to_number(machine: &mut Machine<'_, H>, value: Value) -> Result { @@ -385,6 +563,62 @@ fn round_to_binary16(value: f64) -> f64 { if negative { -magnitude } else { magnitude } } +#[cfg(test)] +mod exact_sum_tests { + use super::ExactSum; + + fn total(values: &[f64]) -> f64 { + let mut sum = ExactSum::new(); + for value in values { + sum.add(*value); + } + sum.round() + } + + #[test] + fn intermediate_overflow_and_cancellation_stay_exact() { + assert_eq!(total(&[f64::MAX, f64::MAX, -f64::MAX]), f64::MAX); + assert_eq!(total(&[1e308, 1e308, -1e308, -1e308]), 0.0); + assert_eq!(total(&[1e30, 0.1, -1e30]), 0.1); + assert_eq!(total(&[f64::MAX, f64::MAX]), f64::INFINITY); + assert_eq!(total(&[-f64::MAX, -f64::MAX]), f64::NEG_INFINITY); + } + + #[test] + fn rounding_is_to_nearest_with_ties_to_even() { + let tiny = 2f64.powi(-53); + assert_eq!(total(&[1.0, tiny]), 1.0, "a tie rounds down to even"); + let odd = 1.0 + 2f64.powi(-52); + assert_eq!( + total(&[odd, tiny]), + 1.0 + 2f64.powi(-51), + "a tie rounds up to even" + ); + assert_eq!( + total(&[1.0, tiny, 2f64.powi(-1000)]), + 1.0 + 2f64.powi(-52), + "anything past the tie breaks it upward" + ); + } + + #[test] + fn subnormal_totals_are_exact() { + let smallest = f64::from_bits(1); + assert_eq!(total(&[smallest, smallest]), f64::from_bits(2)); + assert_eq!( + total(&[f64::MIN_POSITIVE, -smallest]).to_bits(), + (1 << 52) - 1 + ); + assert_eq!(total(&[-smallest]), -smallest); + } + + #[test] + fn an_exact_zero_is_positive() { + assert!(total(&[1.5, -1.5]).is_sign_positive()); + assert!(total(&[]).is_sign_positive()); + } +} + #[cfg(test)] mod tests { use super::super::test_support::{TestHost, blank_program, ordinary_object}; diff --git a/crates/bamts-runtime/src/builtins/mod.rs b/crates/bamts-runtime/src/builtins/mod.rs index 9c41b039..57cd3ae1 100644 --- a/crates/bamts-runtime/src/builtins/mod.rs +++ b/crates/bamts-runtime/src/builtins/mod.rs @@ -7,10 +7,14 @@ use bamts_bytecode::{DescriptorSlot, EcmaString}; use bamts_native::{Decoded, Value}; use std::collections::BTreeMap; mod array; +mod array_generic; +pub(crate) use array_generic::is_array; pub(crate) mod bigint; mod collections; mod date_full; mod function_constructor; +mod intl; +pub(crate) mod iterator_helpers; mod json_edge; mod number; mod object; @@ -20,22 +24,26 @@ pub(crate) mod property_descriptor; pub(crate) mod proxy; pub(crate) mod reflect; mod regexp; -pub(crate) use regexp::{canonical_source, initial_regexp_properties}; +pub(crate) use regexp::initial_regexp_properties; mod annex_b; mod array_es2023; pub(crate) mod arraybuffer; +pub(crate) mod atomics; pub(crate) mod dataview; mod disposable_stack; mod error_edge; mod map_set_edge; mod math_edge; mod number_format; -mod regexp_v; +pub(crate) use number_format::string_to_number; +mod regexp_protocol; +pub(crate) mod regexp_v; mod set_methods; mod string; mod string_edge; mod structured_clone; mod symbol; +mod temporal; #[cfg(test)] pub(crate) mod test_support; mod timers; @@ -48,7 +56,9 @@ pub(crate) fn install( globals: &mut BTreeMap, builtins: &mut BuiltinTable, timers_available: bool, + script_compiler: bool, ) { + object::install_function_prototype(heap, builtins); symbol::install(heap, globals, builtins); let bigint_prototype = bigint::install(heap, globals, builtins); builtins.set_bigint_prototype(bigint_prototype); @@ -57,11 +67,12 @@ pub(crate) fn install( timers::install(heap, globals, builtins); } collections::install_iterator_prototype(heap, builtins); + iterator_helpers::install(heap, globals, builtins); collections::install_async_iterator_prototype(heap, builtins); collections::install_generator_prototype(heap, builtins); collections::install_async_generator_prototype(heap, builtins); collections::install(heap, globals, builtins); - function_constructor::install(heap, globals, builtins); + function_constructor::install(heap, globals, builtins, script_compiler); map_set_edge::install(heap, globals, builtins); date_full::install(heap, globals, builtins); object::install(heap, globals, builtins); @@ -75,6 +86,7 @@ pub(crate) fn install( annex_b::install(heap, globals, builtins); weakref_finalization::install(heap, globals, builtins); arraybuffer::install(heap, globals, builtins); + atomics::install(heap, globals, builtins); typedarray_all::install(heap, globals, builtins); dataview::install(heap, globals, builtins); install_globals(heap, globals, builtins); @@ -88,6 +100,24 @@ pub(crate) fn install( reflect::install(heap, globals, builtins); uri::install(heap, globals, builtins); disposable_stack::install(heap, globals, builtins); + temporal::install(heap, globals, builtins); + intl::install(heap, globals, builtins); +} +/// Runs `body` with `roots` kept alive across any user code it reaches. The +/// frame's depth is passed in, so `keep_root` can add values found along the way. +fn with_roots( + machine: &mut Machine<'_, H>, + roots: &[Value], + body: impl FnOnce(&mut Machine<'_, H>, usize) -> Result, +) -> Result { + let depth = machine.native_roots.len(); + machine.push_native_roots(depth, roots); + let result = body(machine, depth); + machine.pop_native_roots(depth); + result +} +fn keep_root(machine: &mut Machine<'_, H>, depth: usize, value: Value) { + machine.native_roots[depth].push(value); } fn install_function( heap: &mut Vec, @@ -132,7 +162,9 @@ pub(crate) fn define_data(heap: &mut [HeapEntry], object: Value, name: &str, val | HeapEntry::Script { properties, .. } | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } - | HeapEntry::Date { properties, .. } => { + | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } => { properties.insert( PropertyKey::Named(EcmaString::encode(name)), builtin_property(value), @@ -141,6 +173,27 @@ pub(crate) fn define_data(heap: &mut [HeapEntry], object: Value, name: &str, val _ => panic!("intrinsic property target must be an ordinary object"), } } +/// Installs a symbol-keyed property on an intrinsic object of any shape. +pub(crate) fn define_symbol_property( + heap: &mut [HeapEntry], + object: Value, + symbol: Value, + property: Property, +) { + let key = PropertyKey::Symbol(heap_index(symbol) as u32); + let index = heap_index(object); + match &mut heap[index] { + HeapEntry::Object { properties, .. } + | HeapEntry::Array { properties, .. } + | HeapEntry::Function { properties, .. } + | HeapEntry::NativeFunction { properties, .. } + | HeapEntry::RegExp { properties, .. } + | HeapEntry::Date { properties, .. } => { + properties.insert(key, property); + } + _ => panic!("intrinsic property target must be an ordinary object"), + } +} pub(super) fn define_to_string_tag( heap: &mut Vec, object: Value, @@ -257,11 +310,45 @@ fn to_uint8_from_f64(number: f64) -> u8 { } } +/// `GetPrototypeFromConstructor` for the running built-in constructor. +fn prototype_from_new_target( + machine: &mut Machine<'_, H>, + default_prototype: Value, +) -> Result { + let new_target = machine.current_new_target(); + if new_target == Value::UNDEFINED { + return Ok(default_prototype); + } + let prototype = machine.get_named_property(new_target, "prototype")?; + Ok(if machine.is_object(prototype) { + prototype + } else { + default_prototype + }) +} +/// The wrapper object a primitive-wrapping constructor returns. +fn wrap_for_new_target( + machine: &mut Machine<'_, H>, + primitive: Value, + default_prototype: Value, +) -> Result { + let prototype = with_roots(machine, &[primitive], |machine, _| { + prototype_from_new_target(machine, default_prototype) + })?; + let wrapper = machine.box_primitive(primitive)?; + if let HeapEntry::Object { + prototype: slot, .. + } = &mut machine.heap[heap_index(wrapper)] + { + *slot = Some(prototype); + } + Ok(BuiltinOutcome::Value(wrapper)) +} fn to_integer_or_infinity( - machine: &Machine<'_, H>, + machine: &mut Machine<'_, H>, value: Value, ) -> Result { - let number = value_number(machine.to_number(value)?); + let number = value_number(machine.coerce_number_observable(value)?); Ok(if number.is_nan() || number == 0.0 { 0.0 } else { @@ -286,7 +373,11 @@ fn define_array_length( if !length_writable && length != elements.len() { return Err(type_error("Cannot redefine non-writable array length")); } - crate::apply_array_length(elements, properties, length, "define array length") + if crate::apply_array_length(elements, properties, length) { + Ok(()) + } else { + Err(type_error("define array length")) + } } fn install_boolean( heap: &mut Vec, @@ -300,6 +391,8 @@ fn install_boolean( globals.insert(EcmaString::encode("Boolean"), constructor); let value_of = install_function(heap, builtins, "valueOf", 0, boolean_value_of::); define_data(heap, prototype, "valueOf", value_of); + let to_string = install_function(heap, builtins, "toString", 0, boolean_to_string::); + define_data(heap, prototype, "toString", to_string); } fn boolean_constructor( machine: &mut Machine<'_, H>, @@ -310,7 +403,8 @@ fn boolean_constructor( let value = Value::boolean(machine.to_boolean(args.first().copied().unwrap_or(Value::UNDEFINED))); if constructing { - return machine.box_primitive(value).map(BuiltinOutcome::Value); + let prototype = machine.intrinsics.boolean_prototype; + return wrap_for_new_target(machine, value, prototype); } Ok(BuiltinOutcome::Value(value)) } @@ -328,6 +422,21 @@ fn boolean_value_of( )), } } +fn boolean_to_string( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = machine.unbox_primitive(this, "Boolean.prototype.toString")?; + let Some(bamts_native::Decoded::Boolean(value)) = value.decode() else { + return Err(type_error( + "Boolean.prototype.toString called on incompatible receiver", + )); + }; + let text = EcmaString::encode(if value { "true" } else { "false" }); + allocate_string(machine, text).map(BuiltinOutcome::Value) +} fn install_math( heap: &mut Vec, globals: &mut BTreeMap, @@ -366,11 +475,11 @@ fn install_math( math_edge::install(heap, builtins, math); } fn numeric_args( - machine: &Machine<'_, H>, + machine: &mut Machine<'_, H>, args: &[Value], ) -> Result, EvalFailure> { args.iter() - .map(|value| machine.to_number(*value).map(value_number)) + .map(|value| machine.coerce_number_observable(*value).map(value_number)) .collect() } macro_rules! unary_math { @@ -381,9 +490,10 @@ macro_rules! unary_math { args: &[Value], _constructing: bool, ) -> Result { - let x = - value_number(machine.to_number(args.first().copied().unwrap_or(Value::UNDEFINED))?) - as f64; + let x = value_number( + machine + .coerce_number_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?, + ) as f64; Ok(BuiltinOutcome::Value(crate::number_value(($body)(x)))) } }; @@ -394,19 +504,31 @@ unary_math!(math_ceil, f64::ceil); unary_math!(math_trunc, f64::trunc); unary_math!(math_sqrt, f64::sqrt); unary_math!(math_log2, f64::log2); +/// The integer nearest `x`, ties toward positive infinity. Comparing the +/// distance from the floor avoids `x + 0.5`, which itself rounds: the largest +/// double below 0.5 would come back as 1. A negative `x` that rounds to zero +/// gives negative zero. +fn round_half_up(x: f64) -> f64 { + if x.fract() == 0.0 || !x.is_finite() { + return x; + } + let floor = x.floor(); + let rounded = if x - floor >= 0.5 { floor + 1.0 } else { floor }; + if rounded == 0.0 && x < 0.0 { + -0.0 + } else { + rounded + } +} fn math_round( machine: &mut Machine<'_, H>, _this: Value, args: &[Value], _constructing: bool, ) -> Result { - let x = value_number(machine.to_number(args.first().copied().unwrap_or(Value::UNDEFINED))?); - let rounded = if (-0.5..0.0).contains(&x) { - -0.0 - } else { - (x + 0.5).floor() - }; - Ok(BuiltinOutcome::Value(crate::number_value(rounded))) + let x = args.first().copied().unwrap_or(Value::UNDEFINED); + let x = value_number(machine.coerce_number_observable(x)?); + Ok(BuiltinOutcome::Value(crate::number_value(round_half_up(x)))) } fn math_sign( machine: &mut Machine<'_, H>, @@ -414,7 +536,9 @@ fn math_sign( args: &[Value], _constructing: bool, ) -> Result { - let x = value_number(machine.to_number(args.first().copied().unwrap_or(Value::UNDEFINED))?); + let x = value_number( + machine.coerce_number_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?, + ); let sign = if x == 0.0 || x.is_nan() { x } else { @@ -472,9 +596,15 @@ fn math_pow( args: &[Value], _constructing: bool, ) -> Result { - let x = value_number(machine.to_number(args.first().copied().unwrap_or(Value::UNDEFINED))?); - let y = value_number(machine.to_number(args.get(1).copied().unwrap_or(Value::UNDEFINED))?); - Ok(BuiltinOutcome::Value(crate::number_value(x.powf(y)))) + let x = value_number( + machine.coerce_number_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?, + ); + let y = value_number( + machine.coerce_number_observable(args.get(1).copied().unwrap_or(Value::UNDEFINED))?, + ); + Ok(BuiltinOutcome::Value(crate::number_value( + crate::number_exponentiate(x, y), + ))) } fn math_random( machine: &mut Machine<'_, H>, @@ -511,10 +641,12 @@ fn math_imul( args: &[Value], _constructing: bool, ) -> Result { - let a = - value_number(machine.to_number(args.first().copied().unwrap_or(Value::UNDEFINED))?) as u32; - let b = - value_number(machine.to_number(args.get(1).copied().unwrap_or(Value::UNDEFINED))?) as u32; + let a = value_number( + machine.coerce_number_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?, + ) as u32; + let b = value_number( + machine.coerce_number_observable(args.get(1).copied().unwrap_or(Value::UNDEFINED))?, + ) as u32; Ok(BuiltinOutcome::Value(Value::int32(a.wrapping_mul(b)))) } fn install_globals( @@ -548,22 +680,8 @@ fn install_globals( crate::number_value(f64::INFINITY), ); globals.insert(EcmaString::encode("NaN"), crate::number_value(f64::NAN)); - // Node exposes `Atomics` as a global, and the corpus differential compares - // our output against Node byte-for-byte. `corpus/cases/is-plain-obj.ts` - // reads the global, so omitting it throws a ReferenceError and fails Node - // parity. Populating the namespace requires SharedArrayBuffer / shared - // memory, which the runtime does not have yet, so it is deliberately empty. - let atomics = push( - heap, - HeapEntry::Object { - properties: PropertyMap::default(), - prototype: Some(builtins.object_prototype()), - extensible: true, - boxed_primitive: None, - }, - ); - define_to_string_tag(heap, atomics, builtins.symbol_to_string_tag(), "Atomics"); - globals.insert(EcmaString::encode("Atomics"), atomics); + globals.insert(EcmaString::encode("undefined"), Value::UNDEFINED); + number::share_global_parsers(heap, globals); } fn install_errors( heap: &mut Vec, @@ -774,10 +892,19 @@ fn error_to_string( _args: &[Value], _constructing: bool, ) -> Result { + if !machine.is_object(this) { + return Err(type_error("Error.prototype.toString requires an object")); + } let name_value = machine.get_named_property(this, "name")?; - let name = machine.to_string(name_value)?; + let name = match name_value { + Value::UNDEFINED => EcmaString::encode("Error"), + value => machine.coerce_string_observable(value)?, + }; let message_value = machine.get_named_property(this, "message")?; - let message = machine.to_string(message_value)?; + let message = match message_value { + Value::UNDEFINED => EcmaString::encode(""), + value => machine.coerce_string_observable(value)?, + }; let text = if name.is_empty() { message } else if message.is_empty() { @@ -884,6 +1011,11 @@ impl<'a, H: Host> Machine<'a, H> { proxy::define_own_property(machine, object, key, descriptor) }); } + if let Some(index) = self.runtime_slot(object).map_err(EvalFailure::Runtime)? + && self.is_array_length_key(index, &key) + { + return self.array_define_length(index, descriptor); + } let current = self.own_descriptor(object, &key)?; let extensible = property_descriptor::is_extensible(self, object)?; property_descriptor::validate_and_apply_property_descriptor( @@ -938,11 +1070,16 @@ impl<'a, H: Host> Machine<'a, H> { if self.same_value(proposed, current) { return Ok(true); } - if !property_descriptor::is_extensible(self, object)? { + // %Object.prototype% is an immutable prototype exotic object. + if object == self.intrinsics.object_prototype + || !property_descriptor::is_extensible(self, object)? + { return Ok(false); } + // The cycle walk stops at a proxy, whose [[GetPrototypeOf]] is not ordinary. let mut node = proposed; - while !matches!(node.decode(), Some(Decoded::Null)) { + while !matches!(node.decode(), Some(Decoded::Null)) && !self.heap_entry_is_proxy_value(node) + { if node == object { return Ok(false); } @@ -985,6 +1122,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { extensible, .. } | HeapEntry::RegExp { extensible, .. } | HeapEntry::Date { extensible, .. } + | HeapEntry::Temporal { extensible, .. } + | HeapEntry::Intl { extensible, .. } | HeapEntry::BuiltinIterator { extensible, .. } | HeapEntry::Collection { extensible, .. } | HeapEntry::TypedArray { extensible, .. } @@ -1026,7 +1165,7 @@ impl<'a, H: Host> Machine<'a, H> { this_value: Value, arguments: &[Value], ) -> Result { - let result = self.call_value(callee, this_value, arguments)?; + let result = self.call_callback(callee, this_value, arguments)?; Ok(self.to_boolean(result)) } #[cfg(test)] @@ -1070,6 +1209,16 @@ impl<'a, H: Host> Machine<'a, H> { extensible, .. } + | HeapEntry::Temporal { + properties, + extensible, + .. + } + | HeapEntry::Intl { + properties, + extensible, + .. + } | HeapEntry::BuiltinIterator { properties, extensible, @@ -1155,6 +1304,10 @@ impl<'a, H: Host> Machine<'a, H> { let Some(index) = self.runtime_slot(object).map_err(EvalFailure::Runtime)? else { return Ok(None); }; + if let Some(property) = self.string_exotic_property(index, key)? { + return Ok(Some(property)); + } + self.reify_function_properties(index)?; let properties = match &self.heap[index] { HeapEntry::ModuleNamespace { module } => { let PropertyKey::Named(name) = key else { @@ -1229,12 +1382,22 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::WeakRef { properties, .. } | HeapEntry::FinalizationRegistry { properties, .. } | HeapEntry::DisposableStack { properties, .. } - | HeapEntry::Timeout { properties, .. } => properties, + | HeapEntry::Timeout { properties, .. } + | HeapEntry::ArrayBuffer { properties, .. } + | HeapEntry::SharedArrayBuffer { properties, .. } + | HeapEntry::DataView { properties, .. } + | HeapEntry::Promise { properties, .. } + | HeapEntry::Generator { properties, .. } + | HeapEntry::AsyncGenerator { properties, .. } + | HeapEntry::AsyncFromSync { properties, .. } + | HeapEntry::ProxyRevoker { properties, .. } => properties, _ => return Ok(None), }; Ok(properties.get(key).cloned()) @@ -1358,6 +1521,23 @@ impl<'a, H: Host> Machine<'a, H> { let Some(index) = self.runtime_slot(object).map_err(EvalFailure::Runtime)? else { return Err(type_error("Object.defineProperty called on non-object")); }; + self.reify_function_properties(index)?; + if let Some(Property::Data { + value: stored, + enumerable, + .. + }) = self.string_exotic_property(index, &key)? + { + return match descriptor { + Property::Data { + value, + writable: false, + enumerable: requested, + configurable: false, + } if requested == enumerable && self.same_value(value, stored) => Ok(()), + _ => Err(type_error("Cannot redefine property")), + }; + } if let PropertyKey::Named(name) = &key && let Some(typed_index) = typedarray_all::typed_array_index(name) && matches!(self.heap[index], HeapEntry::TypedArray { .. }) @@ -1480,6 +1660,16 @@ impl<'a, H: Host> Machine<'a, H> { extensible, .. } + | HeapEntry::Temporal { + properties, + extensible, + .. + } + | HeapEntry::Intl { + properties, + extensible, + .. + } | HeapEntry::BuiltinIterator { properties, extensible, @@ -1529,6 +1719,21 @@ impl<'a, H: Host> Machine<'a, H> { properties, extensible, .. + } + | HeapEntry::Promise { + properties, + extensible, + .. + } + | HeapEntry::Generator { + properties, + extensible, + .. + } + | HeapEntry::AsyncGenerator { + properties, + extensible, + .. } => { let exists = properties.contains_key(&key); let (property_growth, property_refund) = @@ -1626,6 +1831,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::TypedArray { properties, .. } @@ -1635,7 +1842,10 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::WeakRef { properties, .. } | HeapEntry::FinalizationRegistry { properties, .. } | HeapEntry::DisposableStack { properties, .. } - | HeapEntry::Timeout { properties, .. } => { + | HeapEntry::Timeout { properties, .. } + | HeapEntry::Promise { properties, .. } + | HeapEntry::Generator { properties, .. } + | HeapEntry::AsyncGenerator { properties, .. } => { properties.insert(key, descriptor); } _ => unreachable!("validated object cannot change heap entry kind"), @@ -1658,6 +1868,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { extensible, .. } | HeapEntry::RegExp { extensible, .. } | HeapEntry::Date { extensible, .. } + | HeapEntry::Temporal { extensible, .. } + | HeapEntry::Intl { extensible, .. } | HeapEntry::BuiltinIterator { extensible, .. } | HeapEntry::Collection { extensible, .. } | HeapEntry::TypedArray { extensible, .. } @@ -1706,6 +1918,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::TypedArray { properties, .. } @@ -1731,6 +1945,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::TypedArray { properties, .. } @@ -1748,7 +1964,10 @@ impl<'a, H: Host> Machine<'a, H> { self.refund_slot(index, refund); Ok(()) } - fn observable_property_key(&mut self, key: Value) -> Result { + pub(crate) fn observable_property_key( + &mut self, + key: Value, + ) -> Result { let primitive = self.coerce_primitive_observable(key, true)?; Ok( match self.runtime_slot(primitive).map_err(EvalFailure::Runtime)? { @@ -1794,6 +2013,15 @@ impl<'a, H: Host> Machine<'a, H> { slot: DescriptorSlot, ) -> Result<(), EvalFailure> { let key = self.observable_property_key(key)?; + if slot == DescriptorSlot::Entire { + let property = Property::Data { + value: src, + writable: true, + enumerable: true, + configurable: true, + }; + return self.define_descriptor(object, key, property); + } let half = (src != Value::UNDEFINED).then_some(src); let updated = match (self.own_descriptor(object, &key)?, slot) { (None, DescriptorSlot::Value) => Property::Data { @@ -2055,6 +2283,43 @@ mod tests { } } #[test] + fn round_half_up_keeps_zero_signs_and_the_value_just_below_a_tie() { + assert!(round_half_up(-0.0).is_sign_negative()); + assert!(round_half_up(-0.5).is_sign_negative()); + assert_eq!(round_half_up(-0.5), 0.0); + assert!(round_half_up(0.2).is_sign_positive()); + assert_eq!(round_half_up(0.49999999999999994), 0.0); + assert_eq!(round_half_up(2.5), 3.0); + assert_eq!(round_half_up(-2.5), -2.0); + assert_eq!(round_half_up(9007199254740991.0), 9007199254740991.0); + assert!(round_half_up(f64::NAN).is_nan()); + assert_eq!(round_half_up(f64::NEG_INFINITY), f64::NEG_INFINITY); + } + #[test] + fn global_undefined_descriptor_is_frozen() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let global_this = machine + .intrinsics + .global("globalThis") + .expect("globalThis is installed"); + let key = PropertyKey::Named(EcmaString::encode("undefined")); + let descriptor = machine + .own_descriptor(global_this, &key) + .expect("descriptor lookup succeeds") + .expect("undefined is defined on globalThis"); + assert!(matches!( + descriptor, + Property::Data { + value: Value::UNDEFINED, + writable: false, + enumerable: false, + configurable: false, + } + )); + } + #[test] fn global_infinity_and_nan_are_stable_across_reads() { let module = blank_program(""); let mut host = TestHost; @@ -2166,7 +2431,7 @@ mod tests { } } #[test] - fn atomics_claims_no_methods() { + fn atomics_claims_exactly_the_methods_it_implements() { let module = blank_program(""); let mut host = TestHost; let machine = Machine::new(&module, &mut host, Limits::default()); @@ -2177,13 +2442,32 @@ mod tests { let keys = machine .own_property_keys(atomics) .expect("Atomics is an object"); - let method_keys: Vec<_> = keys + let mut method_names: Vec<_> = keys .into_iter() - .filter(|key| !matches!(key, PropertyKey::Symbol(_))) + .filter_map(|key| match key { + PropertyKey::Named(name) => Some(name.to_utf8_lossy()), + _ => None, + }) .collect(); - assert!( - method_keys.is_empty(), - "Atomics must not claim any named methods" + method_names.sort(); + assert_eq!( + method_names, + [ + "add", + "and", + "compareExchange", + "exchange", + "isLockFree", + "load", + "notify", + "or", + "pause", + "store", + "sub", + "wait", + "waitAsync", + "xor", + ] ); } #[test] diff --git a/crates/bamts-runtime/src/builtins/number.rs b/crates/bamts-runtime/src/builtins/number.rs index 91455b6f..6b5159da 100644 --- a/crates/bamts-runtime/src/builtins/number.rs +++ b/crates/bamts-runtime/src/builtins/number.rs @@ -3,6 +3,7 @@ use std::collections::BTreeMap; use bamts_bytecode::EcmaString; use bamts_native::{Decoded, Value}; +use super::intl::number_to_locale_string; use super::{ allocate_string, define_data, install_constructor_function, install_function, number_format, range_error, to_integer_or_infinity, type_error, value_number, @@ -24,8 +25,6 @@ pub(super) fn install( ("isSafeInteger", 1, is_safe_integer::), ("isFinite", 1, is_finite::), ("isNaN", 1, is_nan::), - ("parseFloat", 1, parse_float::), - ("parseInt", 2, parse_int::), ] { let f = install_function(heap, builtins, name, length, handler); define_static(heap, constructor, name, f) @@ -40,11 +39,25 @@ pub(super) fn install( ("NEGATIVE_INFINITY", f64::NEG_INFINITY), ("NaN", f64::NAN), ] { - define_static(heap, constructor, name, crate::number_value(value)) + let HeapEntry::NativeFunction { properties, .. } = + &mut heap[super::heap_index(constructor)] + else { + panic!("Number constructor must be native") + }; + properties.insert( + PropertyKey::Named(EcmaString::encode(name)), + crate::Property::Data { + value: crate::number_value(value), + writable: false, + enumerable: false, + configurable: false, + }, + ); } for (name, length, handler) in [ ("toString", 1, to_string:: as BuiltinHandler), ("toFixed", 1, to_fixed::), + ("toLocaleString", 0, to_locale_string::), ("toExponential", 1, to_exponential::), ("toPrecision", 1, to_precision::), ("valueOf", 0, value_of::), @@ -53,6 +66,13 @@ pub(super) fn install( define_data(heap, prototype, name, f) } } +/// `Number.parseFloat` and `Number.parseInt` are the global functions themselves. +pub(super) fn share_global_parsers(heap: &mut [HeapEntry], globals: &BTreeMap) { + let constructor = globals[&EcmaString::encode("Number")]; + for name in ["parseFloat", "parseInt"] { + define_static(heap, constructor, name, globals[&EcmaString::encode(name)]); + } +} fn define_static(heap: &mut [HeapEntry], constructor: Value, name: &str, value: Value) { let HeapEntry::NativeFunction { properties, .. } = &mut heap[super::heap_index(constructor)] else { @@ -69,15 +89,18 @@ fn constructor( args: &[Value], constructing: bool, ) -> Result { - let value = if args.is_empty() { - Value::int32(0) - } else if let Some(number) = super::bigint::bigint_to_number(machine, args[0])? { - number + let value = if let Some(value) = args.first() { + let primitive = machine.coerce_primitive_observable(*value, false)?; + match super::bigint::bigint_to_number(machine, primitive)? { + Some(number) => number, + None => machine.coerce_number_observable(primitive)?, + } } else { - machine.to_number(args[0])? + Value::int32(0) }; if constructing { - Ok(BuiltinOutcome::Value(machine.box_primitive(value)?)) + let prototype = machine.intrinsics.number_prototype; + super::wrap_for_new_target(machine, value, prototype) } else { Ok(BuiltinOutcome::Value(value)) } @@ -156,7 +179,7 @@ pub(super) fn parse_float( _: bool, ) -> Result { let text = machine - .to_string(args.first().copied().unwrap_or(Value::UNDEFINED))? + .coerce_string_observable(args.first().copied().unwrap_or(Value::UNDEFINED))? .to_utf8_lossy(); Ok(BuiltinOutcome::Value(crate::number_value( number_format::parse_float(&text), @@ -180,10 +203,10 @@ pub(super) fn parse_int( _: bool, ) -> Result { let text = machine - .to_string(args.first().copied().unwrap_or(Value::UNDEFINED))? + .coerce_string_observable(args.first().copied().unwrap_or(Value::UNDEFINED))? .to_utf8_lossy(); let requested = to_int32_radix(value_number( - machine.to_number(args.get(1).copied().unwrap_or(Value::UNDEFINED))?, + machine.coerce_number_observable(args.get(1).copied().unwrap_or(Value::UNDEFINED))?, )); Ok(BuiltinOutcome::Value(crate::number_value( number_format::parse_int(&text, requested), @@ -195,7 +218,9 @@ pub(super) fn global_is_nan( args: &[Value], _: bool, ) -> Result { - let n = value_number(machine.to_number(args.first().copied().unwrap_or(Value::UNDEFINED))?); + let n = value_number( + machine.coerce_number_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?, + ); Ok(BuiltinOutcome::Value(Value::boolean(n.is_nan()))) } pub(super) fn global_is_finite( @@ -204,7 +229,9 @@ pub(super) fn global_is_finite( args: &[Value], _: bool, ) -> Result { - let n = value_number(machine.to_number(args.first().copied().unwrap_or(Value::UNDEFINED))?); + let n = value_number( + machine.coerce_number_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?, + ); Ok(BuiltinOutcome::Value(Value::boolean(n.is_finite()))) } fn to_string( @@ -229,6 +256,21 @@ fn to_string( EcmaString::encode(&text), )?)) } +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + let n = primitive_number( + machine, + this, + "Number.prototype.toLocaleString requires that 'this' be a Number", + )?; + let argument = |index: usize| args.get(index).copied().unwrap_or(Value::UNDEFINED); + number_to_locale_string(machine, n, argument(0), argument(1)) +} + fn to_fixed( machine: &mut Machine<'_, H>, this: Value, @@ -314,6 +356,58 @@ mod tests { use super::*; use crate::Limits; + #[test] + fn exponentiation_differs_from_powf_for_a_unit_base() { + assert!(crate::number_exponentiate(1.0, f64::INFINITY).is_nan()); + assert!(crate::number_exponentiate(-1.0, f64::NEG_INFINITY).is_nan()); + assert!(crate::number_exponentiate(1.0, f64::NAN).is_nan()); + assert_eq!(crate::number_exponentiate(f64::NAN, 0.0), 1.0); + assert_eq!(crate::number_exponentiate(2.0, 10.0), 1024.0); + } + + #[test] + fn number_parsers_are_the_global_functions() { + let program = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + let constructor = machine.intrinsics.global("Number").unwrap(); + for name in ["parseFloat", "parseInt"] { + assert_eq!( + machine.get_named_property(constructor, name).unwrap(), + machine.intrinsics.global(name).unwrap() + ); + } + } + + #[test] + fn number_constants_are_read_only() { + let program = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + let constructor = machine.intrinsics.global("Number").unwrap(); + for name in [ + "MAX_SAFE_INTEGER", + "MIN_SAFE_INTEGER", + "EPSILON", + "MAX_VALUE", + "MIN_VALUE", + "POSITIVE_INFINITY", + "NEGATIVE_INFINITY", + "NaN", + ] { + let key = PropertyKey::Named(EcmaString::encode(name)); + assert!(matches!( + machine.own_descriptor(constructor, &key).unwrap(), + Some(crate::Property::Data { + writable: false, + enumerable: false, + configurable: false, + .. + }) + )); + } + } + #[test] fn numeric_prefixes_stop_at_lone_surrogates() { let program = blank_program(""); diff --git a/crates/bamts-runtime/src/builtins/number_format.rs b/crates/bamts-runtime/src/builtins/number_format.rs index bad2e463..d928e0f1 100644 --- a/crates/bamts-runtime/src/builtins/number_format.rs +++ b/crates/bamts-runtime/src/builtins/number_format.rs @@ -154,7 +154,9 @@ impl Big { fn set_bit(&mut self, index: u32) { let word = (index / 32) as usize; - self.0.resize(word + 1, 0); + if self.0.len() <= word { + self.0.resize(word + 1, 0); + } self.0[word] |= 1_u32 << (index % 32); } @@ -614,6 +616,50 @@ pub fn parse_int(text: &str, mut radix: i32) -> f64 { if negative { -magnitude } else { magnitude } } +/// `StringToNumber`. Only a decimal literal takes a sign, and this grammar has no +/// numeric separators. +#[must_use] +pub(crate) fn string_to_number(text: &str) -> f64 { + let trimmed = text.trim_matches(is_js_whitespace); + if trimmed.is_empty() { + return 0.0; + } + for (prefixes, radix) in [(["0x", "0X"], 16), (["0o", "0O"], 8), (["0b", "0B"], 2)] { + let Some(digits) = prefixes + .iter() + .find_map(|prefix| trimmed.strip_prefix(prefix)) + else { + continue; + }; + let valid = !digits.is_empty() + && digits + .chars() + .all(|digit| digit_value(digit).is_some_and(|value| value < radix as u32)); + return if valid { + parse_int(digits, radix) + } else { + f64::NAN + }; + } + let unsigned = trimmed.strip_prefix(['+', '-']).unwrap_or(trimmed); + if unsigned == "Infinity" { + return if trimmed.starts_with('-') { + f64::NEG_INFINITY + } else { + f64::INFINITY + }; + } + // Rust's own grammar also takes `inf`, `infinity` and `nan` in any case. + if unsigned + .bytes() + .all(|byte| byte.is_ascii_digit() || matches!(byte, b'.' | b'e' | b'E' | b'+' | b'-')) + { + trimmed.parse().unwrap_or(f64::NAN) + } else { + f64::NAN + } +} + fn decimal_prefix_len(text: &str) -> usize { if text.starts_with("Infinity") { return "Infinity".len(); @@ -737,6 +783,30 @@ mod tests { assert_eq!(to_exponential(77.0, None), Ok("7.7e+1".to_owned())); } + /// `div_rem` sets quotient bits from the top down, so a quotient of 2^32 or + /// more is only right if setting a low bit keeps the limbs above it. Losing + /// them made `rounded_significand` oscillate between exponents forever. + #[test] + fn precision_past_one_limb_of_quotient_terminates_with_exact_digits() { + let cases = [ + (1.5, 16.0, "1.500000000000000"), + (1.1, 16.0, "1.100000000000000"), + (123.456, 21.0, "123.456000000000003070"), + (0.5, 16.0, "0.5000000000000000"), + (100.0, 16.0, "100.0000000000000"), + ]; + for (value, precision, expected) in cases { + assert_eq!( + to_precision(value, Some(precision)), + Ok(expected.to_owned()) + ); + } + assert_eq!( + to_fixed(0.1, 25.0), + Ok("0.1000000000000000055511151".to_owned()) + ); + } + #[test] fn radix_formatting_covers_every_accepted_radix() { for radix in 2..=36 { @@ -811,6 +881,35 @@ mod tests { } } + #[test] + fn string_to_number_follows_the_string_numeric_literal_grammar() { + for (text, expected) in [ + ("", 0.0), + ("\u{feff} \u{2029}12\t", 12.0), + ("0x10", 16.0), + ("0XfF", 255.0), + ("0o17", 15.0), + ("0b101", 5.0), + ("1e3", 1000.0), + (".5", 0.5), + ("5.", 5.0), + ("Infinity", f64::INFINITY), + ("+Infinity", f64::INFINITY), + ("-Infinity", f64::NEG_INFINITY), + // Rounding after each digit loses the low bits and gives 2^57. + ("0x200000000000011", 144_115_188_075_855_904.0), + ] { + assert_eq!(string_to_number(text), expected, "{text:?}"); + } + assert!(string_to_number("-0").is_sign_negative()); + for text in [ + "-0x10", "+0b1", "0x", "0xg", "0b2", "0o8", "1_000", "infinity", "inf", "nan", "NaN", + "INFINITY", "e5", ".", "+", "\u{85}1", + ] { + assert!(string_to_number(text).is_nan(), "{text:?}"); + } + } + #[test] fn parse_int_implements_prefix_radix_whitespace_and_overflow_rules() { assert_eq!(parse_int("0x10", 0), 16.0); diff --git a/crates/bamts-runtime/src/builtins/object.rs b/crates/bamts-runtime/src/builtins/object.rs index ef041032..22f49d09 100644 --- a/crates/bamts-runtime/src/builtins/object.rs +++ b/crates/bamts-runtime/src/builtins/object.rs @@ -2,22 +2,52 @@ use std::collections::BTreeMap; use super::property_descriptor::{ PropertyDescriptor as CanonicalPropertyDescriptor, collect_property_descriptors, - is_extensible as descriptor_is_extensible, to_property_descriptor, - validate_and_apply_property_descriptor, + to_property_descriptor, }; use bamts_bytecode::{EcmaString, EcmaStringBuilder}; use bamts_native::{Decoded, Value}; use super::{ - allocate_string, define_data, install_constructor_function, install_function, range_error, - to_integer_or_infinity, type_error, value_number, + allocate_string, define_data, define_symbol_property, heap_index, install_constructor_function, + install_function, to_integer_or_infinity, type_error, value_number, +}; +use crate::intrinsics::{ + BuiltinDef, BuiltinHandler, BuiltinOutcome, BuiltinTable, native_function_properties, }; -use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; use crate::{ BoundCallable, EvalFailure, HeapEntry, Host, Machine, NativeCallable, Property, PropertyKey, PropertyMap, RuntimeErrorKind, }; +/// `%Function.prototype%` is itself a function, so this replaces the ordinary +/// object that had to exist before the builtin table could. Its prototype is +/// stated explicitly: a native function without one falls back to this object. +pub(super) fn install_function_prototype( + heap: &mut Vec, + builtins: &mut BuiltinTable, +) { + let id = builtins.register(BuiltinDef { + name: "", + length: 0, + handler: function_prototype_call::, + }); + let properties = native_function_properties(heap, "", 0); + heap[heap_index(builtins.function_prototype())] = HeapEntry::native_function( + NativeCallable::Builtin(id), + properties, + Some(builtins.object_prototype()), + ); +} + +fn function_prototype_call( + _machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) +} + pub(super) fn install( heap: &mut Vec, globals: &mut BTreeMap, @@ -32,6 +62,7 @@ pub(super) fn install( for (name, length, handler) in [ ("toString", 0, prototype_to_string:: as BuiltinHandler), + ("toLocaleString", 0, prototype_to_locale_string::), ("hasOwnProperty", 1, has_own_property::), ("isPrototypeOf", 1, is_prototype_of::), ("valueOf", 0, value_of::), @@ -47,10 +78,197 @@ pub(super) fn install( ("call", 1, function_call:: as BuiltinHandler), ("apply", 2, function_apply::), ("bind", 1, function_bind::), + ("toString", 0, function_to_string::), ] { let function = install_function(heap, builtins, name, length, handler); define_data(heap, builtins.function_prototype(), name, function); } + install_restricted_function_properties(heap, builtins); + let reified = crate::intrinsics::push( + heap, + HeapEntry::PrivateName { + description: EcmaString::encode("[[FunctionPropertiesStored]]"), + }, + ); + builtins.set_function_reified_brand(reified); + let has_instance = install_function( + heap, + builtins, + "[Symbol.hasInstance]", + 1, + function_has_instance::, + ); + define_symbol_property( + heap, + builtins.function_prototype(), + builtins.symbol_has_instance(), + Property::Data { + value: has_instance, + writable: false, + enumerable: false, + configurable: false, + }, + ); +} + +/// `AddRestrictedFunctionProperties` for %Function.prototype%, around a frozen +/// %ThrowTypeError%. +fn install_restricted_function_properties( + heap: &mut Vec, + builtins: &mut BuiltinTable, +) { + let thrower = install_function(heap, builtins, "", 0, throw_type_error::); + builtins.set_throw_type_error(thrower); + let HeapEntry::NativeFunction { + properties, + extensible, + .. + } = &mut heap[heap_index(thrower)] + else { + panic!("%ThrowTypeError% must be native"); + }; + *extensible = false; + for (_, property) in &mut properties.0 { + if let Property::Data { configurable, .. } = property { + *configurable = false; + } + } + let HeapEntry::NativeFunction { properties, .. } = + &mut heap[heap_index(builtins.function_prototype())] + else { + panic!("%Function.prototype% must be native"); + }; + for name in ["caller", "arguments"] { + properties.insert( + PropertyKey::Named(EcmaString::encode(name)), + Property::Accessor { + getter: Some(thrower), + setter: Some(thrower), + enumerable: false, + configurable: true, + }, + ); + } +} + +/// %ThrowTypeError%. A non-strict `function` has no call-stack introspection here, +/// so it reads `caller` and `arguments` as undefined instead of throwing. +fn throw_type_error( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + if machine.is_sloppy_ordinary_function(this)? { + return Ok(BuiltinOutcome::Value(Value::UNDEFINED)); + } + Err(type_error( + "caller and arguments are restricted function properties", + )) +} + +/// Whether `text` can follow `function` in the `NativeFunction` syntax: +/// an identifier or a computed symbol name, optionally after `get` or `set`. +/// Engine-internal labels such as `Promise resolve target` are neither. +fn is_native_function_name(text: &str) -> bool { + fn is_identifier(text: &str) -> bool { + let mut characters = text.chars(); + characters + .next() + .is_some_and(|first| first.is_alphabetic() || matches!(first, '$' | '_')) + && characters.all(|next| next.is_alphanumeric() || matches!(next, '$' | '_')) + } + let property = text + .strip_prefix("get ") + .or_else(|| text.strip_prefix("set ")) + .unwrap_or(text); + match property + .strip_prefix('[') + .and_then(|inner| inner.strip_suffix(']')) + { + Some(computed) => computed.split('.').all(is_identifier), + None => is_identifier(property), + } +} + +/// The name a function was created with. This reads metadata rather than the +/// `name` property, which user code can redefine as an accessor. +fn function_metadata_name( + machine: &Machine<'_, H>, + function: Value, +) -> Result, EvalFailure> { + let Some(index) = machine + .runtime_slot(function) + .map_err(EvalFailure::Runtime)? + else { + return Ok(None); + }; + Ok(match &machine.heap[index] { + HeapEntry::NativeFunction { + callable: NativeCallable::Builtin(id), + .. + } => Some(machine.intrinsics.builtins.get(*id).name.to_owned()), + HeapEntry::Function { + module, function, .. + } => machine.module_code(*module).functions()[function.get() as usize] + .name() + .map(|id| machine.constant_text(*module, id).to_utf8_lossy()), + _ => None, + }) +} + +/// `Function.prototype.toString`. Every callable answers in the +/// `NativeFunction` form: bytecode does not retain source text, so a user +/// function's exact source is not available here. +fn function_to_string( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + if !machine.is_callable(this)? { + return Err(type_error( + "Function.prototype.toString requires a callable receiver", + )); + } + let name = function_metadata_name(machine, this)? + .filter(|name| is_native_function_name(name)) + .unwrap_or_default(); + let source = EcmaString::encode(&format!("function {name}() {{ [native code] }}")); + Ok(BuiltinOutcome::Value(allocate_string(machine, source)?)) +} + +/// `Object.prototype.toLocaleString`: `Invoke(this, "toString")`, with the +/// receiver passed through unboxed so a primitive `this` stays primitive. +fn prototype_to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + if matches!(this.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Err(type_error( + "Object.prototype.toLocaleString called on null or undefined", + )); + } + let method = machine.get_named_property(this, "toString")?; + Ok(BuiltinOutcome::Value(machine.call_value( + method, + this, + &[], + )?)) +} + +fn function_has_instance( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = args.first().copied().unwrap_or(Value::UNDEFINED); + Ok(BuiltinOutcome::Value(Value::boolean( + machine.ordinary_has_instance(value, this)?, + ))) } fn constructor( @@ -134,76 +352,17 @@ pub(super) fn create( Ok(BuiltinOutcome::Value(object)) } -fn define_array_length_descriptor( - machine: &mut Machine<'_, H>, - object: Value, - key: &PropertyKey, - descriptor: CanonicalPropertyDescriptor, -) -> Result { - let Some(index) = machine.runtime_slot(object).map_err(EvalFailure::Runtime)? else { - return Ok(false); - }; - if !matches!(key, PropertyKey::Named(name) if name.eq_ascii("length")) - || !matches!(machine.heap[index], HeapEntry::Array { .. }) - { - return Ok(false); - } - if descriptor.is_accessor() { - return Err(type_error("Invalid property descriptor")); - } - if descriptor.enumerable == Some(true) || descriptor.configurable == Some(true) { - return Err(type_error("Cannot redefine array length")); - } - let length = descriptor - .value - .map(|value| { - crate::exact_array_length(value).ok_or_else(|| range_error("define array length")) - }) - .transpose()?; - let HeapEntry::Array { - elements, - properties, - length_writable, - .. - } = &mut machine.heap[index] - else { - unreachable!("array checked above"); - }; - if descriptor.writable == Some(true) && !*length_writable { - return Err(type_error("Cannot make array length writable")); - } - let result = match length { - Some(length) => super::define_array_length(elements, properties, *length_writable, length), - None => Ok(()), - }; - if descriptor.writable == Some(false) { - *length_writable = false; - } - result?; - Ok(true) -} - pub(super) fn apply_property_descriptor( machine: &mut Machine<'_, H>, target: Value, key: PropertyKey, descriptor: CanonicalPropertyDescriptor, ) -> Result<(), EvalFailure> { - if define_array_length_descriptor(machine, target, &key, descriptor)? { - return Ok(()); - } - let current = machine.own_descriptor(target, &key)?; - let extensible = descriptor_is_extensible(machine, target)?; - if !validate_and_apply_property_descriptor( - machine, - Some((target, key)), - extensible, - descriptor, - current, - )? { - return Err(type_error("Cannot redefine property")); + if machine.internal_define_own_property(target, key, descriptor)? { + Ok(()) + } else { + Err(type_error("Cannot redefine property")) } - Ok(()) } pub(super) fn define_property( @@ -265,7 +424,8 @@ fn prototype_to_string( Some(Decoded::Undefined) => EcmaString::encode("Undefined"), Some(Decoded::Null) => EcmaString::encode("Null"), _ => { - let fallback = machine.object_to_string_tag(this)?; + let object = machine.value_to_object(this)?; + let fallback = machine.object_to_string_tag(object)?; let symbol = machine.intrinsics.builtins.symbol_to_string_tag(); let key = PropertyKey::Symbol( machine @@ -273,7 +433,7 @@ fn prototype_to_string( .map_err(EvalFailure::Runtime)? .expect("well-known symbol belongs to the runtime heap") as u32, ); - let tag_value = machine.get_property_key(this, &key)?; + let tag_value = machine.get_property_key(object, &key)?; machine .string_value(tag_value) .unwrap_or_else(|| EcmaString::encode(fallback)) @@ -298,10 +458,10 @@ fn has_own_property( args: &[Value], _constructing: bool, ) -> Result { - let key = machine.to_property_key(args.first().copied().unwrap_or(Value::UNDEFINED))?; - Ok(BuiltinOutcome::Value(Value::boolean( - machine.has_own_property_key(this, &key)?, - ))) + let key = machine.observable_property_key(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let object = machine.value_to_object(this)?; + let own = machine.internal_get_own_property(object, &key)?.is_some(); + Ok(BuiltinOutcome::Value(Value::boolean(own))) } fn is_prototype_of( @@ -311,8 +471,12 @@ fn is_prototype_of( _constructing: bool, ) -> Result { let mut value = args.first().copied().unwrap_or(Value::UNDEFINED); + if !machine.is_object(value) { + return Ok(BuiltinOutcome::Value(Value::FALSE)); + } + let object = machine.value_to_object(this)?; while let Some(prototype) = machine.internal_get_prototype_of(value)? { - if prototype == this { + if prototype == object { return Ok(BuiltinOutcome::Value(Value::TRUE)); } value = prototype; @@ -321,12 +485,12 @@ fn is_prototype_of( } fn value_of( - _machine: &mut Machine<'_, H>, + machine: &mut Machine<'_, H>, this: Value, _args: &[Value], _constructing: bool, ) -> Result { - Ok(BuiltinOutcome::Value(this)) + machine.value_to_object(this).map(BuiltinOutcome::Value) } fn property_is_enumerable( @@ -335,10 +499,11 @@ fn property_is_enumerable( args: &[Value], _constructing: bool, ) -> Result { - let key = machine.to_property_key(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let key = machine.observable_property_key(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let object = machine.value_to_object(this)?; let enumerable = machine - .own_descriptor(this, &key)? - .is_some_and(|property| property.enumerable()); + .internal_get_own_property(object, &key)? + .is_some_and(|descriptor| descriptor.enumerable == Some(true)); Ok(BuiltinOutcome::Value(Value::boolean(enumerable))) } @@ -507,6 +672,508 @@ mod tests { use crate::{Limits, RuntimeErrorKind, ThrowOrigin}; use bamts_bytecode::{FunctionId, ModuleId}; + fn named(name: &str) -> PropertyKey { + PropertyKey::Named(EcmaString::encode(name)) + } + + fn pinned_element(value: Value) -> Property { + Property::Data { + value, + writable: true, + enumerable: true, + configurable: false, + } + } + + #[test] + fn a_blocked_array_truncation_stops_at_the_element_and_reports_failure() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let values = vec![Value::int32(1), Value::int32(2), Value::int32(3)]; + let array = allocate_array(&mut machine, values).unwrap(); + machine + .define_descriptor(array, named("1"), pinned_element(Value::int32(2))) + .unwrap(); + let zero = CanonicalPropertyDescriptor { + value: Some(Value::int32(0)), + ..Default::default() + }; + assert!( + !machine + .internal_define_own_property(array, named("length"), zero) + .unwrap() + ); + assert_eq!(machine.array_length(array).unwrap(), 2); + assert!( + !machine + .internal_set(array, named("length"), Value::int32(0), array) + .unwrap() + ); + assert_eq!(machine.array_length(array).unwrap(), 2); + } + + #[test] + fn an_invalid_array_length_is_a_range_error_before_the_descriptor_is_judged() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = allocate_array(&mut machine, Vec::new()).unwrap(); + let negative = CanonicalPropertyDescriptor { + value: Some(crate::number_value(-1.0)), + configurable: Some(true), + ..Default::default() + }; + assert!(matches!( + machine.internal_define_own_property(array, named("length"), negative), + Err(EvalFailure::Throw(ThrowOrigin::RangeError { .. })) + )); + } + + #[test] + fn defining_through_a_proxy_without_a_trap_reaches_the_target() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let target = ordinary_object(&mut machine); + let handler = ordinary_object(&mut machine); + let proxy_constructor = machine.intrinsics.global("Proxy").unwrap(); + let proxy = machine + .construct_value(proxy_constructor, &[target, handler]) + .unwrap(); + let descriptor = CanonicalPropertyDescriptor { + value: Some(Value::int32(7)), + ..Default::default() + }; + apply_property_descriptor(&mut machine, proxy, named("x"), descriptor).unwrap(); + assert!(matches!( + machine.own_descriptor(target, &named("x")).unwrap(), + Some(Property::Data { value, .. }) if value == Value::int32(7) + )); + } + + #[test] + fn writing_an_element_with_a_descriptor_keeps_its_attributes() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = allocate_array(&mut machine, vec![Value::int32(1)]).unwrap(); + machine + .define_descriptor(array, named("0"), pinned_element(Value::int32(1))) + .unwrap(); + machine + .set_data_property(array, "0", Value::int32(9)) + .unwrap(); + assert!(matches!( + machine.own_descriptor(array, &named("0")).unwrap(), + Some(Property::Data { + value, + configurable: false, + .. + }) if value == Value::int32(9) + )); + } + + #[test] + fn array_own_keys_put_length_after_indices_and_before_other_names() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array = allocate_array(&mut machine, vec![Value::int32(1)]).unwrap(); + machine + .set_data_property(array, "tag", Value::TRUE) + .unwrap(); + assert_eq!( + machine.internal_own_property_keys(array).unwrap(), + [named("0"), named("length"), named("tag")] + ); + assert!(machine.enumerable_keys(array).unwrap().len() == 2); + } + + #[test] + fn string_wrappers_own_read_only_length_and_code_units() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let text = allocate_string(&mut machine, EcmaString::encode("ab")).unwrap(); + let wrapper = machine.value_to_object(text).unwrap(); + for (name, expected_enumerable) in [("0", true), ("1", true), ("length", false)] { + assert!(matches!( + machine.own_descriptor(wrapper, &named(name)).unwrap(), + Some(Property::Data { + writable: false, + enumerable, + configurable: false, + .. + }) if enumerable == expected_enumerable + )); + assert!(!machine.internal_delete(wrapper, &named(name)).unwrap()); + assert!( + !machine + .internal_set(wrapper, named(name), Value::int32(0), wrapper) + .unwrap() + ); + } + assert!( + machine + .own_descriptor(wrapper, &named("2")) + .unwrap() + .is_none() + ); + assert!( + machine + .internal_set(wrapper, named("2"), Value::int32(0), wrapper) + .unwrap() + ); + assert_eq!(machine.enumerable_keys(wrapper).unwrap().len(), 3); + } + + #[test] + fn freezing_a_string_wrapper_keeps_its_exotic_properties() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let text = allocate_string(&mut machine, EcmaString::encode("ab")).unwrap(); + let wrapper = machine.value_to_object(text).unwrap(); + call_object(&mut machine, "freeze", &[wrapper]).unwrap(); + let frozen = call_object(&mut machine, "isFrozen", &[wrapper]).unwrap(); + assert_eq!(frozen, Value::TRUE); + } + + #[test] + fn object_prototype_methods_convert_their_receiver_after_the_key() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.object_prototype; + for name in ["hasOwnProperty", "propertyIsEnumerable"] { + let method = machine.get_named_property(prototype, name).unwrap(); + assert!(matches!( + machine.call_value(method, Value::UNDEFINED, &[Value::int32(0)]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + let value_of = machine.get_named_property(prototype, "valueOf").unwrap(); + assert!(machine.call_value(value_of, Value::NULL, &[]).is_err()); + let boxed = machine.call_value(value_of, Value::TRUE, &[]).unwrap(); + assert!(machine.is_object(boxed)); + let is_prototype_of = machine + .get_named_property(prototype, "isPrototypeOf") + .unwrap(); + assert_eq!( + machine + .call_value(is_prototype_of, Value::UNDEFINED, &[Value::int32(1)]) + .unwrap(), + Value::FALSE, + "a primitive argument answers before the receiver is converted" + ); + } + + #[test] + fn object_prototype_to_string_is_an_ordinary_property() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.object_prototype; + assert!(matches!( + machine + .own_descriptor(prototype, &named("toString")) + .unwrap(), + Some(Property::Data { + writable: true, + configurable: true, + .. + }) + )); + assert!( + machine + .internal_delete(prototype, &named("toString")) + .unwrap() + ); + assert_eq!( + machine.get_named_property(prototype, "toString").unwrap(), + Value::UNDEFINED + ); + } + + #[test] + fn object_prototype_has_an_immutable_prototype() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.object_prototype; + let parent = ordinary_object(&mut machine); + assert!( + !machine + .internal_set_prototype_of(prototype, Some(parent)) + .unwrap() + ); + assert!(machine.internal_set_prototype_of(prototype, None).unwrap()); + } + + #[test] + fn promises_and_generators_accept_defined_properties() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let promise = machine.intrinsics.global("Promise").unwrap(); + let resolve = machine.get_named_property(promise, "resolve").unwrap(); + let settled = machine + .call_value(resolve, promise, &[Value::int32(1)]) + .unwrap(); + let descriptor = CanonicalPropertyDescriptor { + value: Some(Value::int32(7)), + ..Default::default() + }; + apply_property_descriptor(&mut machine, settled, named("tag"), descriptor).unwrap(); + assert_eq!( + machine.get_named_property(settled, "tag").unwrap(), + Value::int32(7) + ); + } + + #[test] + fn deleting_a_builtin_function_name_refunds_what_it_was_charged() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let target = machine.intrinsics.global("parseInt").unwrap(); + let bound = call_method(&mut machine, target, "bind", &[]).unwrap(); + assert!(machine.internal_delete(bound, &named("name")).unwrap()); + assert!(machine.internal_delete(bound, &named("length")).unwrap()); + } + + #[test] + fn enumerable_own_listing_boxes_a_primitive_source() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let text = allocate_string(&mut machine, EcmaString::encode("ab")).unwrap(); + for name in ["keys", "values", "entries"] { + let listed = call_object(&mut machine, name, &[text]).unwrap(); + assert_eq!(machine.array_length(listed).unwrap(), 2, "{name}"); + } + let target = ordinary_object(&mut machine); + call_object(&mut machine, "assign", &[target, text]).unwrap(); + assert!(machine.has_own_property_key(target, &named("1")).unwrap()); + } + + #[test] + fn function_prototype_poisons_caller_and_arguments_for_strict_receivers() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.function_prototype; + let thrower = machine.intrinsics.builtins.throw_type_error(); + for name in ["caller", "arguments"] { + assert!(matches!( + machine.own_descriptor(prototype, &named(name)).unwrap(), + Some(Property::Accessor { + getter: Some(getter), + setter: Some(setter), + enumerable: false, + configurable: true, + }) if getter == thrower && setter == thrower + )); + assert!(machine.get_named_property(prototype, name).is_err()); + } + assert!(matches!( + machine.own_descriptor(thrower, &named("length")).unwrap(), + Some(Property::Data { + configurable: false, + .. + }) + )); + assert!(matches!( + machine.heap[heap_index(thrower)], + HeapEntry::NativeFunction { + extensible: false, + .. + } + )); + } + + #[test] + fn function_length_and_name_become_own_properties_that_can_be_deleted() { + let module = constructable_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let function = machine + .allocate(HeapEntry::Function { + module: ModuleId::new(0), + function: FunctionId::new(0), + captures: Vec::new(), + context: None, + properties: PropertyMap::default(), + prototype: Some(machine.intrinsics.function_prototype), + extensible: true, + }) + .unwrap(); + assert_eq!( + machine.get_named_property(function, "length").unwrap(), + Value::int32(0), + "reads work before anything stores them" + ); + let keys = machine.internal_own_property_keys(function).unwrap(); + assert_eq!(keys[..2], [named("length"), named("name")]); + assert!(matches!( + machine.own_descriptor(function, &named("name")).unwrap(), + Some(Property::Data { + writable: false, + enumerable: false, + configurable: true, + .. + }) + )); + assert!(machine.internal_delete(function, &named("length")).unwrap()); + assert!( + !machine + .has_own_property_key(function, &named("length")) + .unwrap() + ); + assert!( + machine + .own_descriptor(function, &named("length")) + .unwrap() + .is_none() + ); + } + + #[test] + fn a_property_reference_rejects_a_nullish_base_before_converting_its_key() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let key = ordinary_object(&mut machine); + for base in [Value::NULL, Value::UNDEFINED] { + assert!(matches!( + machine.computed_property_key(base, key), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + } + + #[test] + fn function_prototype_is_a_function_without_construct() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.function_prototype; + assert!(machine.is_callable(prototype).unwrap()); + assert!(!machine.is_constructor(prototype).unwrap()); + assert_eq!( + machine + .call_value(prototype, Value::UNDEFINED, &[Value::int32(1)]) + .unwrap(), + Value::UNDEFINED + ); + assert_eq!( + machine.internal_get_prototype_of(prototype).unwrap(), + Some(machine.intrinsics.object_prototype), + "the chain must not loop back to Function.prototype" + ); + assert_eq!( + machine.get_named_property(prototype, "length").unwrap(), + Value::int32(0) + ); + let name = machine.get_named_property(prototype, "name").unwrap(); + assert!( + machine + .string_value(name) + .is_some_and(|text| text.is_empty()) + ); + } + + #[test] + fn native_function_names_exclude_engine_labels() { + for name in ["push", "get source", "set x", "[Symbol.replace]", "$_a1"] { + assert!(is_native_function_name(name), "{name}"); + } + for name in ["", "Promise resolve target", "#private", "1a", "[a b]"] { + assert!(!is_native_function_name(name), "{name}"); + } + } + + fn function_source(machine: &mut Machine<'_, TestHost>, function: Value) -> EcmaString { + let prototype = machine.intrinsics.builtins.function_prototype(); + let to_string = machine.get_named_property(prototype, "toString").unwrap(); + let source = machine.call_value(to_string, function, &[]).unwrap(); + machine.string_value(source).unwrap() + } + + #[test] + fn function_to_string_uses_metadata_rather_than_the_name_property() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let array_prototype = machine.intrinsics.array_prototype; + let push = machine.get_named_property(array_prototype, "push").unwrap(); + assert!(function_source(&mut machine, push).eq_ascii("function push() { [native code] }")); + + let renamed = machine + .allocate(HeapEntry::String(EcmaString::encode("renamed"))) + .unwrap(); + machine + .define_descriptor( + push, + PropertyKey::Named(EcmaString::encode("name")), + Property::Data { + value: renamed, + writable: false, + enumerable: false, + configurable: true, + }, + ) + .unwrap(); + assert!(function_source(&mut machine, push).eq_ascii("function push() { [native code] }")); + } + + #[test] + fn function_to_string_rejects_non_callable_receivers() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.builtins.function_prototype(); + let to_string = machine.get_named_property(prototype, "toString").unwrap(); + let object = ordinary_object(&mut machine); + for receiver in [Value::UNDEFINED, Value::int32(1), object] { + assert!(matches!( + machine.call_value(to_string, receiver, &[]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + } + + #[test] + fn has_instance_is_false_for_non_callable_receivers_and_primitives() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let prototype = machine.intrinsics.builtins.function_prototype(); + let key = machine + .to_property_key(machine.intrinsics.builtins.symbol_has_instance()) + .unwrap(); + let has_instance = machine.get_property_key(prototype, &key).unwrap(); + let object = ordinary_object(&mut machine); + let array = machine.intrinsics.global("Array").unwrap(); + assert_eq!( + machine.call_value(has_instance, object, &[object]).unwrap(), + Value::FALSE, + "a non-callable receiver is not a constructor of anything" + ); + assert_eq!( + machine + .call_value(has_instance, array, &[Value::int32(1)]) + .unwrap(), + Value::FALSE, + "a primitive has no prototype chain to walk" + ); + assert!(matches!( + machine.instance_of(object, Value::int32(1)), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + fn data_descriptor(machine: &mut Machine<'_, TestHost>, value: Value) -> Value { let descriptor = ordinary_object(machine); machine diff --git a/crates/bamts-runtime/src/builtins/object_statics.rs b/crates/bamts-runtime/src/builtins/object_statics.rs index 6a34ef7e..e550a1b5 100644 --- a/crates/bamts-runtime/src/builtins/object_statics.rs +++ b/crates/bamts-runtime/src/builtins/object_statics.rs @@ -4,7 +4,7 @@ use bamts_bytecode::EcmaString; use bamts_native::{Decoded, Value}; use super::property_descriptor::{PropertyDescriptor, from_property_descriptor, same_value}; -use super::{allocate_array, allocate_string, install_function, type_error}; +use super::{allocate_array, allocate_string, install_function, keep_root, type_error, with_roots}; use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; use crate::{EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, PropertyMap}; @@ -137,6 +137,9 @@ pub(super) fn own_keys( machine: &mut Machine<'_, H>, object: Value, ) -> Result, EvalFailure> { + if let Some(index) = machine.runtime_slot(object).map_err(EvalFailure::Runtime)? { + machine.reify_function_properties(index)?; + } let stored = machine.own_property_keys(object)?; let primitive = machine.unbox_primitive_or_self(object)?; let Some(index) = machine @@ -191,22 +194,33 @@ fn own_descriptor( machine.internal_get_own_property(object, key) } -fn enumerable_string_keys( +/// `EnumerableOwnProperties`, reading each key's descriptor immediately before +/// its value. `item` builds the list entry from the name and that value. +fn enumerable_own_properties( machine: &mut Machine<'_, H>, - object: Value, -) -> Result, EvalFailure> { - let mut names = Vec::new(); - for key in machine.internal_own_property_keys(object)? { - let PropertyKey::Named(name) = key else { - continue; - }; - if own_descriptor(machine, object, &PropertyKey::Named(name.clone()))? - .is_some_and(|descriptor| descriptor.enumerable == Some(true)) - { - names.push(name); + source: Value, + item: impl Fn(&mut Machine<'_, H>, EcmaString, Value) -> Result, +) -> Result { + require_object_coercible(source, "Cannot convert undefined or null to object")?; + let object = machine.value_to_object(source)?; + with_roots(machine, &[object], |machine, depth| { + let mut items = Vec::new(); + for key in machine.internal_own_property_keys(object)? { + let PropertyKey::Named(name) = &key else { + continue; + }; + if !own_descriptor(machine, object, &key)? + .is_some_and(|descriptor| descriptor.enumerable == Some(true)) + { + continue; + } + let value = machine.get_property_key(object, &key)?; + let entry = item(machine, name.clone(), value)?; + keep_root(machine, depth, entry); + items.push(entry); } - } - Ok(names) + allocate_array(machine, items).map(BuiltinOutcome::Value) + }) } fn assign( @@ -224,6 +238,7 @@ fn assign( if matches!(source.decode(), Some(Decoded::Undefined | Decoded::Null)) { continue; } + let source = machine.value_to_object(source)?; for key in machine.internal_own_property_keys(source)? { if !machine .internal_get_own_property(source, &key)? @@ -244,13 +259,23 @@ fn keys( args: &[Value], _constructing: bool, ) -> Result { - let source = argument(args, 0); - require_object_coercible(source, "Cannot convert undefined or null to object")?; - let values = enumerable_string_keys(machine, source)? - .into_iter() - .map(|name| allocate_string(machine, name)) - .collect::, _>>()?; - Ok(BuiltinOutcome::Value(allocate_array(machine, values)?)) + let object = argument(args, 0); + require_object_coercible(object, "Cannot convert undefined or null to object")?; + let object = machine.value_to_object(object)?; + with_roots(machine, &[object], |machine, depth| { + let mut names = Vec::new(); + for key in machine.internal_own_property_keys(object)? { + if let PropertyKey::Named(name) = &key + && own_descriptor(machine, object, &key)? + .is_some_and(|descriptor| descriptor.enumerable == Some(true)) + { + let name = allocate_string(machine, name.clone())?; + keep_root(machine, depth, name); + names.push(name); + } + } + allocate_array(machine, names).map(BuiltinOutcome::Value) + }) } fn values( @@ -259,13 +284,7 @@ fn values( args: &[Value], _constructing: bool, ) -> Result { - let source = argument(args, 0); - require_object_coercible(source, "Cannot convert undefined or null to object")?; - let mut output = Vec::new(); - for name in enumerable_string_keys(machine, source)? { - output.push(machine.get_property_key(source, &PropertyKey::Named(name))?); - } - Ok(BuiltinOutcome::Value(allocate_array(machine, output)?)) + enumerable_own_properties(machine, argument(args, 0), |_, _, value| Ok(value)) } fn entries( @@ -274,15 +293,10 @@ fn entries( args: &[Value], _constructing: bool, ) -> Result { - let source = argument(args, 0); - require_object_coercible(source, "Cannot convert undefined or null to object")?; - let mut output = Vec::new(); - for name in enumerable_string_keys(machine, source)? { - let value = machine.get_property_key(source, &PropertyKey::Named(name.clone()))?; + enumerable_own_properties(machine, argument(args, 0), |machine, name, value| { let key = allocate_string(machine, name)?; - output.push(allocate_array(machine, vec![key, value])?); - } - Ok(BuiltinOutcome::Value(allocate_array(machine, output)?)) + allocate_array(machine, vec![key, value]) + }) } fn get_own_property_descriptor( @@ -293,6 +307,7 @@ fn get_own_property_descriptor( ) -> Result { let source = argument(args, 0); require_object_coercible(source, "Cannot convert undefined or null to object")?; + let source = machine.value_to_object(source)?; let key = machine.observable_property_key(argument(args, 1))?; let descriptor = own_descriptor(machine, source, &key)?; Ok(BuiltinOutcome::Value(from_property_descriptor( @@ -308,6 +323,7 @@ fn get_own_property_descriptors( ) -> Result { let source = argument(args, 0); require_object_coercible(source, "Cannot convert undefined or null to object")?; + let source = machine.value_to_object(source)?; let keys = machine.internal_own_property_keys(source)?; let result = ordinary_object(machine, Some(machine.intrinsics.object_prototype))?; for key in keys { diff --git a/crates/bamts-runtime/src/builtins/promise.rs b/crates/bamts-runtime/src/builtins/promise.rs index 8d8487b4..f05776ed 100644 --- a/crates/bamts-runtime/src/builtins/promise.rs +++ b/crates/bamts-runtime/src/builtins/promise.rs @@ -106,12 +106,16 @@ pub(super) fn install( let race = install_function(heap, builtins, "race", 1, race::); let all_settled = install_function(heap, builtins, "allSettled", 1, all_settled::); let any = install_function(heap, builtins, "any", 1, any::); + let try_method = install_function(heap, builtins, "try", 1, promise_try::); + let with_resolvers = install_function(heap, builtins, "withResolvers", 0, with_resolvers::); define_static(heap, constructor, "resolve", resolve); define_static(heap, constructor, "reject", reject); define_static(heap, constructor, "all", all); define_static(heap, constructor, "race", race); define_static(heap, constructor, "allSettled", all_settled); define_static(heap, constructor, "any", any); + define_static(heap, constructor, "try", try_method); + define_static(heap, constructor, "withResolvers", with_resolvers); builtins.set_promise_capability_executor(capability_executor); builtins.set_promise_finally_value(finally_value); builtins.set_promise_finally_throw(finally_throw); @@ -227,7 +231,9 @@ fn capability_executor( })); } let record = args.first().copied().unwrap_or(Value::UNDEFINED); - if machine.get_named_property(record, "resolve")? != Value::UNDEFINED { + if machine.get_named_property(record, "resolve")? != Value::UNDEFINED + || machine.get_named_property(record, "reject")? != Value::UNDEFINED + { return Err(EvalFailure::Throw(ThrowOrigin::TypeError { operation: "Promise capability executor", })); @@ -460,6 +466,64 @@ fn reject( Ok(BuiltinOutcome::Value(capability.promise)) } +/// `Promise.try(callback, ...args)`: a synchronous throw rejects the result +/// rather than reaching the caller. The capability stays rooted across the +/// callback, which can run arbitrary code. +fn promise_try( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + if !machine.is_constructor(this)? { + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "Promise.try", + })); + } + let capability = new_promise_capability(machine, this)?; + let depth = machine.native_roots.len(); + machine.push_native_roots( + depth, + &[capability.promise, capability.resolve, capability.reject], + ); + let callback = args.first().copied().unwrap_or(Value::UNDEFINED); + let callback_args = args.get(1..).unwrap_or_default(); + let outcome = match machine.call_value(callback, Value::UNDEFINED, callback_args) { + Ok(value) => machine + .call_value(capability.resolve, Value::UNDEFINED, &[value]) + .map(|_| BuiltinOutcome::Value(capability.promise)), + Err(failure) => reject_capability_failure(machine, &capability, failure), + }; + machine.pop_native_roots(depth); + outcome +} + +/// `Promise.withResolvers()`: a capability exposed as an ordinary object. +fn with_resolvers( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + if !machine.is_constructor(this)? { + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "Promise.withResolvers", + })); + } + let capability = new_promise_capability(machine, this)?; + let prototype = machine.intrinsics.object_prototype; + let result = super::ordinary_runtime(machine, Some(prototype))?; + for (name, value) in [ + ("promise", capability.promise), + ("resolve", capability.resolve), + ("reject", capability.reject), + ] { + let key = PropertyKey::Named(EcmaString::encode(name)); + machine.create_data_property_key(result, key, value)?; + } + Ok(BuiltinOutcome::Value(result)) +} + fn all( machine: &mut Machine<'_, H>, this: Value, @@ -521,9 +585,7 @@ fn race( let value = match machine.iterator_next(iterator) { Ok((true, _)) => break, Ok((false, value)) => value, - Err(failure) => { - return reject_iterator_failure(machine, &capability, iterator, failure); - } + Err(failure) => return reject_capability_failure(machine, &capability, failure), }; let resolved = match machine.call_value(resolve, this, &[value]) { Ok(resolved) => resolved, @@ -674,13 +736,8 @@ fn perform_ordered_aggregate( Ok((true, _)) => break, Ok((false, value)) => value, Err(failure) => { - return reject_aggregate_failure( - machine, - aggregate, - capability.reject, - iterator, - failure, - ); + machine.mark_promise_all_settled(aggregate)?; + return reject_failure(machine, capability.reject, failure); } }; let index = match machine.add_promise_all_element(aggregate) { @@ -1240,6 +1297,76 @@ mod tests { use crate::intrinsics::BuiltinHandler; use crate::{Limits, PromiseState}; + fn promise_state<'m>(machine: &'m Machine<'_, TestHost>, promise: Value) -> &'m PromiseState { + let index = machine.runtime_slot(promise).unwrap().unwrap(); + let HeapEntry::Promise { state, .. } = &machine.heap[index] else { + panic!("expected a promise"); + }; + state + } + + fn static_method(machine: &mut Machine<'_, TestHost>, name: &str) -> (Value, Value) { + let constructor = machine.intrinsics.global("Promise").unwrap(); + let method = machine.get_named_property(constructor, name).unwrap(); + (constructor, method) + } + + #[test] + fn promise_try_rejects_a_synchronous_throw_instead_of_propagating_it() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let (constructor, method) = static_method(&mut machine, "try"); + let thrower = callback(&mut machine, "thrower", throw_99); + let promise = machine + .call_value(method, constructor, &[thrower]) + .expect("the throw is captured by the promise"); + assert!(matches!( + promise_state(&machine, promise), + PromiseState::Rejected { .. } + )); + } + + #[test] + fn promise_try_and_with_resolvers_require_a_constructor() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let parse_int = machine.intrinsics.global("parseInt").unwrap(); + for name in ["try", "withResolvers"] { + let (_, method) = static_method(&mut machine, name); + for receiver in [Value::UNDEFINED, parse_int] { + assert!(matches!( + machine.call_value(method, receiver, &[]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + } + } + + #[test] + fn with_resolvers_exposes_a_working_capability() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let (constructor, method) = static_method(&mut machine, "withResolvers"); + let record = machine.call_value(method, constructor, &[]).unwrap(); + let promise = machine.get_named_property(record, "promise").unwrap(); + let resolve = machine.get_named_property(record, "resolve").unwrap(); + machine.get_named_property(record, "reject").unwrap(); + assert!(matches!( + promise_state(&machine, promise), + PromiseState::Pending { .. } + )); + machine + .call_value(resolve, Value::UNDEFINED, &[Value::int32(1)]) + .unwrap(); + assert!(matches!( + promise_state(&machine, promise), + PromiseState::Fulfilled { .. } + )); + } + fn callback( machine: &mut Machine<'_, TestHost>, name: &'static str, diff --git a/crates/bamts-runtime/src/builtins/property_descriptor.rs b/crates/bamts-runtime/src/builtins/property_descriptor.rs index f26ea4ab..2dcb2ea5 100644 --- a/crates/bamts-runtime/src/builtins/property_descriptor.rs +++ b/crates/bamts-runtime/src/builtins/property_descriptor.rs @@ -412,6 +412,8 @@ pub(crate) fn is_extensible( | HeapEntry::Script { extensible, .. } | HeapEntry::RegExp { extensible, .. } | HeapEntry::Date { extensible, .. } + | HeapEntry::Temporal { extensible, .. } + | HeapEntry::Intl { extensible, .. } | HeapEntry::Collection { extensible, .. } | HeapEntry::BuiltinIterator { extensible, .. } | HeapEntry::Generator { extensible, .. } diff --git a/crates/bamts-runtime/src/builtins/proxy.rs b/crates/bamts-runtime/src/builtins/proxy.rs index f6073c74..7d5c5534 100644 --- a/crates/bamts-runtime/src/builtins/proxy.rs +++ b/crates/bamts-runtime/src/builtins/proxy.rs @@ -46,6 +46,14 @@ impl ProxyRecord { self.constructable } + /// The target of a live proxy; a revoked proxy has none. + pub(crate) fn live_target(&self) -> Option { + match self.state { + ProxyState::Live { target, .. } => Some(target), + ProxyState::Revoked => None, + } + } + pub(crate) fn for_each_value(&self, mut visit: impl FnMut(Value)) { if let ProxyState::Live { target, handler } = &self.state { visit(*target); diff --git a/crates/bamts-runtime/src/builtins/reflect.rs b/crates/bamts-runtime/src/builtins/reflect.rs index e5eae896..99a46d9c 100644 --- a/crates/bamts-runtime/src/builtins/reflect.rs +++ b/crates/bamts-runtime/src/builtins/reflect.rs @@ -243,7 +243,7 @@ fn is_extensible( ))) } -fn property_key_value( +pub(crate) fn property_key_value( machine: &mut Machine<'_, H>, key: PropertyKey, ) -> Result { diff --git a/crates/bamts-runtime/src/builtins/regexp.rs b/crates/bamts-runtime/src/builtins/regexp.rs index b6804cd7..6036d083 100644 --- a/crates/bamts-runtime/src/builtins/regexp.rs +++ b/crates/bamts-runtime/src/builtins/regexp.rs @@ -1,18 +1,25 @@ use std::collections::BTreeMap; use std::ops::Range; +use std::rc::Rc; use bamts_bytecode::EcmaString; use bamts_native::Value; use super::annex_b::record_legacy_match; +use super::regexp_protocol::{flags_getter, test, to_length}; +use super::regexp_v::{CompiledV, create_result_property, execution_failure}; +use super::string_edge::is_regexp; use super::{ - allocate_array, allocate_string, builtin_property, define_data, heap_index, - install_constructor_function, install_function, type_error, + allocate_array, allocate_string, define_data, install_constructor_function, install_function, + type_error, with_roots, }; -use crate::intrinsics::regexp::{Match, Regex, RegexErrorKind, STEP_BUDGET, canonical_flags}; +use crate::intrinsics::regexp::{Match, canonical_flags}; use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; use crate::{EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, PropertyMap}; +/// How many compiled patterns the machine keeps. A program's hot loop uses few. +const COMPILE_CACHE_ENTRIES: usize = 8; + pub(super) fn install( heap: &mut Vec, globals: &mut BTreeMap, @@ -33,20 +40,7 @@ pub(super) fn install( define_getter(heap, prototype, "source", source_get); let flags_get = install_function(heap, builtins, "get flags", 0, flags_getter::); define_getter(heap, prototype, "flags", flags_get); - let symbol_replace_fn = install_function( - heap, - builtins, - "[Symbol.replace]", - 2, - symbol_replace:: as BuiltinHandler, - ); - let HeapEntry::Object { properties, .. } = &mut heap[heap_index(prototype)] else { - panic!("RegExp prototype must be an ordinary object"); - }; - properties.insert( - PropertyKey::Symbol(heap_index(builtins.symbol_replace()) as u32), - builtin_property(symbol_replace_fn), - ); + super::regexp_protocol::install(heap, builtins, prototype, constructor); builtins.set_regexp_prototype(prototype); globals.insert(EcmaString::encode("RegExp"), constructor); } @@ -57,49 +51,96 @@ fn constructor( args: &[Value], _constructing: bool, ) -> Result { - let (pattern, inherited_flags) = args.first().copied().map_or_else( - || Ok((EcmaString::default(), EcmaString::default())), - |value| { - if let Some(parts) = regexp_parts(machine, value) { - Ok(parts) + let pattern = args.first().copied().unwrap_or(Value::UNDEFINED); + let flags = args.get(1).copied().unwrap_or(Value::UNDEFINED); + with_roots(machine, &[pattern, flags], |machine, depth| { + let pattern_is_regexp = is_regexp(machine, pattern)?; + let new_target = machine.current_new_target(); + let active = machine + .intrinsics + .global("RegExp") + .expect("RegExp is installed"); + if new_target == Value::UNDEFINED && pattern_is_regexp && flags == Value::UNDEFINED { + let pattern_constructor = machine.get_named_property(pattern, "constructor")?; + if pattern_constructor == active { + return Ok(pattern); + } + } + let (source, flags) = if let Some((source, original)) = regexp_parts(machine, pattern) { + let flags = if flags == Value::UNDEFINED { + allocate_string(machine, original)? + } else { + flags + }; + (allocate_string(machine, source)?, flags) + } else if pattern_is_regexp { + let source = machine.get_named_property(pattern, "source")?; + let flags = if flags == Value::UNDEFINED { + machine.get_named_property(pattern, "flags")? + } else { + flags + }; + (source, flags) + } else { + (pattern, flags) + }; + super::keep_root(machine, depth, source); + super::keep_root(machine, depth, flags); + let default_prototype = machine.intrinsics.regexp_prototype(); + let prototype = if new_target == Value::UNDEFINED { + default_prototype + } else { + let candidate = machine.get_named_property(new_target, "prototype")?; + if machine.is_object(candidate) { + candidate } else { - Ok((machine.to_string(value)?, EcmaString::default())) + default_prototype } - }, - )?; - let flags = if let Some(value) = args - .get(1) - .copied() - .filter(|value| *value != Value::UNDEFINED) - { - machine.to_string(value)? + }; + super::keep_root(machine, depth, prototype); + regexp_initialize(machine, prototype, source, flags) + }) + .map(BuiltinOutcome::Value) +} + +/// `RegExpAlloc` and `RegExpInitialize` together. The two conversions run after +/// the prototype is read, and a bad pattern or flag is a `SyntaxError`. +fn regexp_initialize( + machine: &mut Machine<'_, H>, + prototype: Value, + source: Value, + flags: Value, +) -> Result { + let pattern = if source == Value::UNDEFINED { + EcmaString::default() } else { - inherited_flags + machine.coerce_string_observable(source)? }; - compile(machine, &pattern, &flags)?; - let properties = initial_regexp_properties(); - let default_prototype = machine.intrinsics.regexp_prototype(); - let new_target = machine.current_new_target(); - let prototype = if new_target != Value::UNDEFINED { - let candidate = machine.get_named_property(new_target, "prototype")?; - if machine.is_object(candidate) { - candidate - } else { - default_prototype - } + let flags = if flags == Value::UNDEFINED { + EcmaString::default() } else { - default_prototype + machine.coerce_string_observable(flags)? }; - let value = machine + compile(machine, &pattern, &flags)?; + machine .allocate(HeapEntry::RegExp { pattern, flags, - properties, + properties: initial_regexp_properties(), prototype: Some(prototype), extensible: true, }) - .map_err(EvalFailure::Runtime)?; - Ok(BuiltinOutcome::Value(value)) + .map_err(EvalFailure::Runtime) +} + +/// `RegExpCreate`, for the String methods that take a pattern. +pub(super) fn regexp_create( + machine: &mut Machine<'_, H>, + source: Value, + flags: Value, +) -> Result { + let prototype = machine.intrinsics.regexp_prototype(); + regexp_initialize(machine, prototype, source, flags) } fn uses_extended_flags(flags: &EcmaString) -> bool { @@ -123,20 +164,23 @@ pub(super) fn compile( machine: &mut Machine<'_, H>, pattern: &EcmaString, flags: &EcmaString, -) -> Result { - if uses_extended_flags(flags) { - return Ok(super::regexp_v::compile(machine, pattern, flags)? - .engine_regex() - .clone()); +) -> Result, EvalFailure> { + let cached = machine + .regexp_cache + .iter() + .position(|(source, options, _)| source == pattern && options == flags); + if let Some(position) = cached { + let entry = machine.regexp_cache.remove(position); + let compiled = Rc::clone(&entry.2); + machine.regexp_cache.insert(0, entry); + return Ok(compiled); } - Regex::compile(pattern, flags).map_err(|error| { - let id = machine - .intrinsics - .builtins - .id_named("SyntaxError") - .expect("SyntaxError installed"); - machine.throw_error(id, error.message().to_owned()) - }) + let compiled = Rc::new(super::regexp_v::compile(machine, pattern, flags)?); + machine.regexp_cache.truncate(COMPILE_CACHE_ENTRIES - 1); + machine + .regexp_cache + .insert(0, (pattern.clone(), flags.clone(), Rc::clone(&compiled))); + Ok(compiled) } /// Build the initial own-property map for a newly constructed RegExp. @@ -168,43 +212,24 @@ pub(super) fn regexp_parts( } } +/// `RegExpBuiltinExec` up to the point of building a result. `lastIndex` is read +/// first even when it will not be used, and a coercion it triggers can recompile +/// the receiver, so the pattern and flags are read after it. pub(super) fn execute( machine: &mut Machine<'_, H>, regexp: Value, input: &EcmaString, ) -> Result, EvalFailure> { - let (pattern, flags) = regexp_parts(machine, regexp) - .ok_or_else(|| type_error("RegExp method called on incompatible receiver"))?; - if uses_extended_flags(&flags) { - let (_compiled, matched) = - super::regexp_v::execute(machine, regexp, &pattern, &flags, input)?; - if let Some(ref matched) = matched { - record_legacy_match(machine, input, matched)?; - } - return Ok(matched); - } - let regex = compile(machine, &pattern, &flags)?; - let uses_last_index = regex.flags().global || regex.flags().sticky; - let start = if uses_last_index { - index_value(machine.get_named_property(regexp, "lastIndex")?) - } else { - 0 - }; - let matched = regex.exec(input, start).map_err(|error| { - // Budget exhaustion must surface as a runtime error, not a silent - // non-match — a caller that validates input with .test() must see a - // failure it can handle instead of a false `false`. Compile errors - // are impossible here (the regex was already compiled successfully - // above), so only BudgetExhausted can reach this point. - match error.kind() { - RegexErrorKind::BudgetExhausted => { - EvalFailure::Runtime(crate::RuntimeErrorKind::RegexpStepBudgetExceeded { - limit: STEP_BUDGET, - }) - } - RegexErrorKind::Compile => unreachable!("regex already compiled successfully"), - } - })?; + let incompatible = || type_error("RegExp method called on incompatible receiver"); + regexp_parts(machine, regexp).ok_or_else(incompatible)?; + let last_index = machine.get_named_property(regexp, "lastIndex")?; + let last_index = to_length(machine, last_index)?; + let (pattern, flags) = regexp_parts(machine, regexp).ok_or_else(incompatible)?; + let compiled = compile(machine, &pattern, &flags)?; + let engine_flags = compiled.engine_regex().flags(); + let uses_last_index = engine_flags.global || engine_flags.sticky; + let start = if uses_last_index { last_index } else { 0 }; + let matched = compiled.exec(input, start).map_err(execution_failure)?; if let Some(ref matched) = matched { record_legacy_match(machine, input, matched)?; } @@ -215,41 +240,16 @@ pub(super) fn execute( Ok(matched) } -/// RegExpExec for consumers (`test`, String methods). The builtin `exec` method -/// calls `execute` directly and must not dispatch through this path. -pub(super) fn regexp_exec( +/// `RegExpBuiltinExec`. +pub(super) fn builtin_exec( machine: &mut Machine<'_, H>, regexp: Value, input: &EcmaString, ) -> Result { - let input_value = allocate_string(machine, input.clone())?; - if let Some(result) = super::regexp_v::call_exec_override(machine, regexp, input_value)? { - return Ok(result); - } let Some(matched) = execute(machine, regexp, input)? else { return Ok(Value::NULL); }; - match_array_for(machine, Some(regexp), input, matched) -} - -fn symbol_replace( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - regexp_parts(machine, this).ok_or_else(|| { - type_error("RegExp.prototype[Symbol.replace] called on incompatible receiver") - })?; - let string = args.first().copied().unwrap_or(Value::UNDEFINED); - let replacer = args.get(1).copied().unwrap_or(Value::UNDEFINED); - let replace = - machine.get_named_property(machine.intrinsics.builtins.string_prototype(), "replace")?; - Ok(BuiltinOutcome::Value(machine.call_value( - replace, - string, - &[this, replacer], - )?)) + match_array_for(machine, regexp, input, matched) } fn exec( @@ -260,30 +260,9 @@ fn exec( ) -> Result { regexp_parts(machine, this) .ok_or_else(|| type_error("RegExp method called on incompatible receiver"))?; - let input = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; - let Some(matched) = execute(machine, this, &input)? else { - return Ok(BuiltinOutcome::Value(Value::NULL)); - }; - Ok(BuiltinOutcome::Value(match_array_for( - machine, - Some(this), - &input, - matched, - )?)) -} - -fn test( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - regexp_parts(machine, this) - .ok_or_else(|| type_error("RegExp method called on incompatible receiver"))?; - let input = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; - Ok(BuiltinOutcome::Value(Value::boolean( - regexp_exec(machine, this, &input)? != Value::NULL, - ))) + let input = + machine.coerce_string_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?; + Ok(BuiltinOutcome::Value(builtin_exec(machine, this, &input)?)) } fn to_string( @@ -292,13 +271,19 @@ fn to_string( _args: &[Value], _constructing: bool, ) -> Result { - let (pattern, flags) = regexp_parts(machine, this) - .ok_or_else(|| type_error("RegExp method called on incompatible receiver"))?; + if !machine.is_object(this) { + return Err(type_error("RegExp.prototype.toString requires an object")); + } + let source = machine.get_named_property(this, "source")?; + let source = machine.coerce_string_observable(source)?; + let flags = machine.get_named_property(this, "flags")?; + let flags = machine.coerce_string_observable(flags)?; let mut output = bamts_bytecode::EcmaStringBuilder::new(); output.push_unit(u16::from(b'/')); - append_canonical_source(&pattern, &mut output); + for &unit in source.as_units() { + output.push_unit(unit); + } output.push_unit(u16::from(b'/')); - let flags = canonical_regexp_flags(&flags); for &unit in flags.as_units() { output.push_unit(unit); } @@ -314,31 +299,29 @@ fn source_getter( _args: &[Value], _constructing: bool, ) -> Result { - let (pattern, _) = regexp_parts(machine, this).ok_or_else(|| { - type_error("RegExp.prototype.source getter called on incompatible receiver") - })?; - Ok(BuiltinOutcome::Value(allocate_string( - machine, - canonical_source(&pattern), - )?)) -} - -fn flags_getter( - machine: &mut Machine<'_, H>, - this: Value, - _args: &[Value], - _constructing: bool, -) -> Result { - let (_pattern, flags) = regexp_parts(machine, this).ok_or_else(|| { - type_error("RegExp.prototype.flags getter called on incompatible receiver") - })?; - Ok(BuiltinOutcome::Value(allocate_string( - machine, - canonical_regexp_flags(&flags), - )?)) + if !machine.is_object(this) { + return Err(type_error( + "RegExp.prototype.source getter requires an object", + )); + } + if let Some((pattern, _)) = regexp_parts(machine, this) { + return Ok(BuiltinOutcome::Value(allocate_string( + machine, + canonical_source(&pattern), + )?)); + } + if this == machine.intrinsics.regexp_prototype() { + return Ok(BuiltinOutcome::Value(allocate_string( + machine, + EcmaString::encode("(?:)"), + )?)); + } + Err(type_error( + "RegExp.prototype.source getter called on incompatible receiver", + )) } -fn define_getter(heap: &mut [HeapEntry], object: Value, name: &str, getter: Value) { +pub(super) fn define_getter(heap: &mut [HeapEntry], object: Value, name: &str, getter: Value) { let HeapEntry::Object { properties, .. } = &mut heap[super::heap_index(object)] else { panic!("accessor target must be an ordinary object"); }; @@ -355,15 +338,14 @@ fn define_getter(heap: &mut [HeapEntry], object: Value, name: &str, getter: Valu pub(super) fn match_array_for( machine: &mut Machine<'_, H>, - regexp: Option, + regexp: Value, input: &EcmaString, matched: Match, ) -> Result { - if let Some(regexp) = regexp - && let Some((pattern, flags)) = regexp_parts(machine, regexp) + if let Some((pattern, flags)) = regexp_parts(machine, regexp) && uses_extended_flags(&flags) { - let compiled = super::regexp_v::compile(machine, &pattern, &flags)?; + let compiled = compile(machine, &pattern, &flags)?; return super::regexp_v::match_array(machine, input, &compiled, &matched); } match_array(machine, input, matched) @@ -382,15 +364,16 @@ pub(super) fn match_array( }); } let array = allocate_array(machine, values)?; - machine.set_data_property( + create_result_property( + machine, array, "index", crate::number_value(matched.range.start as f64), )?; let input_value = allocate_string(machine, input.clone())?; - machine.set_data_property(array, "input", input_value)?; + create_result_property(machine, array, "input", input_value)?; if matched.named.is_empty() { - machine.set_data_property(array, "groups", Value::UNDEFINED)?; + create_result_property(machine, array, "groups", Value::UNDEFINED)?; } else { let groups = machine .allocate(HeapEntry::Object { @@ -405,9 +388,9 @@ pub(super) fn match_array( Some(range) => allocate_string(machine, slice_units(input, range))?, None => Value::UNDEFINED, }; - machine.set_data_property(groups, &name, value)?; + create_result_property(machine, groups, &name, value)?; } - machine.set_data_property(array, "groups", groups)?; + create_result_property(machine, array, "groups", groups)?; } Ok(array) } @@ -415,16 +398,6 @@ pub(super) fn match_array( pub(super) fn slice_units(input: &EcmaString, range: Range) -> EcmaString { input.slice_units(range).unwrap_or_default() } -fn index_value(value: Value) -> usize { - match value.decode() { - Some(bamts_native::Decoded::Int32(value)) => value as usize, - Some(bamts_native::Decoded::Number(value)) if value.is_finite() && value > 0.0 => { - value as usize - } - _ => 0, - } -} - pub(crate) fn canonical_source(pattern: &EcmaString) -> EcmaString { let mut escaped = bamts_bytecode::EcmaStringBuilder::new(); append_canonical_source(pattern, &mut escaped); diff --git a/crates/bamts-runtime/src/builtins/regexp_protocol.rs b/crates/bamts-runtime/src/builtins/regexp_protocol.rs new file mode 100644 index 00000000..7519a007 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/regexp_protocol.rs @@ -0,0 +1,1286 @@ +//! The generic RegExp protocol: `RegExpExec`, the `Symbol`-keyed methods of +//! `%RegExp.prototype%`, and `GetSubstitution`. +//! +//! Every algorithm here is written against the observable object protocol, not +//! the matcher. A receiver need not be a RegExp, `exec` may be user code, and +//! the order of `Get`, `Set` and coercion is part of the specification, so the +//! only path to the engine is the builtin `exec`. + +use bamts_bytecode::{EcmaString, EcmaStringBuilder}; +use bamts_native::{Decoded, Value}; + +use super::array_es2023::{index_key, integer_or_infinity, length_of_array_like}; +use super::regexp::{builtin_exec, regexp_parts}; +use super::{allocate_array, allocate_string, keep_root, type_error, with_roots}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::{EvalFailure, HeapEntry, Host, Machine, Property}; + +const MAX_SAFE_INTEGER: f64 = 9_007_199_254_740_991.0; + +fn argument(args: &[Value], index: usize) -> Value { + args.get(index).copied().unwrap_or(Value::UNDEFINED) +} + +fn require_object( + machine: &Machine<'_, H>, + value: Value, + operation: &'static str, +) -> Result<(), EvalFailure> { + if machine.is_object(value) { + Ok(()) + } else { + Err(type_error(operation)) + } +} + +fn append(output: &mut EcmaStringBuilder, units: &[u16]) { + for &unit in units { + output.push_unit(unit); + } +} + +fn contains_flag(flags: &EcmaString, flag: u8) -> bool { + flags.as_units().contains(&u16::from(flag)) +} + +fn is_full_unicode(flags: &EcmaString) -> bool { + contains_flag(flags, b'u') || contains_flag(flags, b'v') +} + +/// `ToString(? Get(rx, "flags"))`. +fn flags_text(machine: &mut Machine<'_, H>, rx: Value) -> Result { + let flags = machine.get_named_property(rx, "flags")?; + machine.coerce_string_observable(flags) +} + +/// `ToLength`, with user conversion. Lengths are bounded by 2^53 - 1, which fits +/// a `usize` on every supported target. +pub(super) fn to_length( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let length = integer_or_infinity(machine, value)?; + Ok(length.clamp(0.0, MAX_SAFE_INTEGER) as usize) +} + +/// `AdvanceStringIndex`. Positions past the end stay simple increments, because +/// `lastIndex` can hold any length. +fn advance_string_index(string: &EcmaString, index: usize, unicode: bool) -> usize { + if !unicode || index + 1 >= string.len_units() { + return index + 1; + } + index + crate::intrinsics::regexp::next_code_point(string.as_units(), index, true).1 +} + +fn set_last_index( + machine: &mut Machine<'_, H>, + rx: Value, + index: usize, +) -> Result<(), EvalFailure> { + machine.set_data_property(rx, "lastIndex", crate::number_value(index as f64)) +} + +/// `RegExpExec`: a callable `exec` wins, whatever the receiver is. The builtin +/// runs only when `exec` is not callable, and then the receiver must be a RegExp. +pub(super) fn regexp_exec( + machine: &mut Machine<'_, H>, + regexp: Value, + input: Value, +) -> Result { + let exec = machine.get_named_property(regexp, "exec")?; + if machine.is_callable(exec)? { + let result = machine.call_value(exec, regexp, &[input])?; + if result == Value::NULL || machine.is_object(result) { + return Ok(result); + } + return Err(type_error("RegExp exec method returned a non-object")); + } + let text = machine.string_value(input).unwrap_or_default(); + builtin_exec(machine, regexp, &text) +} + +/// The step every global loop shares: an empty match would never advance, so it +/// moves `lastIndex` past itself explicitly. +fn advance_past_empty_match( + machine: &mut Machine<'_, H>, + rx: Value, + string: &EcmaString, + unicode: bool, +) -> Result<(), EvalFailure> { + let last_index = machine.get_named_property(rx, "lastIndex")?; + let this_index = to_length(machine, last_index)?; + let next_index = advance_string_index(string, this_index, unicode); + set_last_index(machine, rx, next_index) +} + +/// `RegExp.prototype.test`. +pub(super) fn test( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + require_object(machine, this, "RegExp.prototype.test requires an object")?; + let string = machine.coerce_string_observable(argument(args, 0))?; + let input = allocate_string(machine, string)?; + let result = with_roots(machine, &[this, input], |machine, _| { + regexp_exec(machine, this, input) + })?; + Ok(BuiltinOutcome::Value(Value::boolean(result != Value::NULL))) +} + +/// `RegExp.prototype[@@match]`. +pub(super) fn symbol_match( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + require_object( + machine, + this, + "RegExp.prototype[Symbol.match] requires an object", + )?; + let string = machine.coerce_string_observable(argument(args, 0))?; + let input = allocate_string(machine, string.clone())?; + with_roots(machine, &[this, input], |machine, depth| { + let flags = flags_text(machine, this)?; + if !contains_flag(&flags, b'g') { + return regexp_exec(machine, this, input); + } + let unicode = is_full_unicode(&flags); + set_last_index(machine, this, 0)?; + let mut matches = Vec::new(); + loop { + let result = regexp_exec(machine, this, input)?; + if result == Value::NULL { + return if matches.is_empty() { + Ok(Value::NULL) + } else { + allocate_array(machine, matches) + }; + } + let matched = machine.get_named_property(result, "0")?; + let matched = machine.coerce_string_observable(matched)?; + let empty = matched.is_empty(); + let value = allocate_string(machine, matched)?; + keep_root(machine, depth, value); + matches.push(value); + if empty { + advance_past_empty_match(machine, this, &string, unicode)?; + } + } + }) + .map(BuiltinOutcome::Value) +} + +/// `RegExp.prototype[@@search]`. `lastIndex` is restored, so a search leaves the +/// receiver as it found it. +pub(super) fn symbol_search( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + require_object( + machine, + this, + "RegExp.prototype[Symbol.search] requires an object", + )?; + let string = machine.coerce_string_observable(argument(args, 0))?; + let input = allocate_string(machine, string)?; + with_roots(machine, &[this, input], |machine, depth| { + let previous = machine.get_named_property(this, "lastIndex")?; + keep_root(machine, depth, previous); + if !machine.same_value(previous, Value::int32(0)) { + set_last_index(machine, this, 0)?; + } + let result = regexp_exec(machine, this, input)?; + keep_root(machine, depth, result); + let current = machine.get_named_property(this, "lastIndex")?; + if !machine.same_value(current, previous) { + machine.set_data_property(this, "lastIndex", previous)?; + } + if result == Value::NULL { + return Ok(Value::int32(-1i32 as u32)); + } + machine.get_named_property(result, "index") + }) + .map(BuiltinOutcome::Value) +} + +/// `SpeciesConstructor`. +pub(super) fn species_constructor( + machine: &mut Machine<'_, H>, + object: Value, + default: Value, +) -> Result { + let constructor = machine.get_named_property(object, "constructor")?; + if constructor == Value::UNDEFINED { + return Ok(default); + } + if !machine.is_object(constructor) { + return Err(type_error("constructor property is not an object")); + } + let key = machine.to_property_key(machine.intrinsics.builtins.symbol_species())?; + let species = machine.get_property_key(constructor, &key)?; + if matches!(species.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Ok(default); + } + if machine.is_constructor(species)? { + Ok(species) + } else { + Err(type_error("Symbol.species is not a constructor")) + } +} + +fn regexp_constructor(machine: &Machine<'_, H>) -> Result { + machine + .intrinsics + .global("RegExp") + .ok_or_else(|| type_error("RegExp constructor is unavailable")) +} + +/// `RegExp.prototype[@@split]`. The search runs on a sticky copy, so each step +/// asks "does a separator start exactly here" and the loop advances itself. +pub(super) fn symbol_split( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + require_object( + machine, + this, + "RegExp.prototype[Symbol.split] requires an object", + )?; + let string = machine.coerce_string_observable(argument(args, 0))?; + let input = allocate_string(machine, string.clone())?; + let limit = argument(args, 1); + with_roots(machine, &[this, input, limit], |machine, depth| { + let default = regexp_constructor(machine)?; + let constructor = species_constructor(machine, this, default)?; + keep_root(machine, depth, constructor); + let flags = flags_text(machine, this)?; + let unicode = is_full_unicode(&flags); + let new_flags = if contains_flag(&flags, b'y') { + flags + } else { + let mut sticky = EcmaStringBuilder::with_capacity(flags.len_units() + 1); + for &unit in flags.as_units() { + sticky.push_unit(unit); + } + sticky.push_unit(u16::from(b'y')); + sticky.finish() + }; + let new_flags = allocate_string(machine, new_flags)?; + let splitter = machine.construct_value(constructor, &[this, new_flags])?; + keep_root(machine, depth, splitter); + let limit = if limit == Value::UNDEFINED { + u32::MAX + } else { + crate::to_uint32(super::value_number( + machine.coerce_number_observable(limit)?, + )) + }; + if limit == 0 { + return allocate_array(machine, Vec::new()); + } + split_steps(machine, depth, splitter, &string, input, limit, unicode) + }) + .map(BuiltinOutcome::Value) +} + +fn split_steps( + machine: &mut Machine<'_, H>, + depth: usize, + splitter: Value, + string: &EcmaString, + input: Value, + limit: u32, + unicode: bool, +) -> Result { + let limit = limit as usize; + let size = string.len_units(); + if size == 0 { + if regexp_exec(machine, splitter, input)? != Value::NULL { + return allocate_array(machine, Vec::new()); + } + return allocate_array(machine, vec![input]); + } + let mut pieces = Vec::new(); + let (mut piece_start, mut search_at) = (0, 0); + while search_at < size { + set_last_index(machine, splitter, search_at)?; + let matched = regexp_exec(machine, splitter, input)?; + if matched == Value::NULL { + search_at = advance_string_index(string, search_at, unicode); + continue; + } + keep_root(machine, depth, matched); + let end = machine.get_named_property(splitter, "lastIndex")?; + let end = to_length(machine, end)?.min(size); + if end == piece_start { + search_at = advance_string_index(string, search_at, unicode); + continue; + } + let piece = allocate_string( + machine, + super::regexp::slice_units(string, piece_start..search_at), + )?; + keep_root(machine, depth, piece); + pieces.push(piece); + if pieces.len() == limit { + return allocate_array(machine, pieces); + } + piece_start = end; + let captures = length_of_array_like(machine, matched)?.saturating_sub(1); + for index in 1..=captures { + let capture = machine.get_property_key(matched, &index_key(index))?; + keep_root(machine, depth, capture); + pieces.push(capture); + if pieces.len() == limit { + return allocate_array(machine, pieces); + } + } + search_at = piece_start; + } + let tail = allocate_string( + machine, + super::regexp::slice_units(string, piece_start..size), + )?; + pieces.push(tail); + allocate_array(machine, pieces) +} + +/// One match result read in full, as `@@replace` requires before it calls user +/// code for any replacement. +struct ReplaceStep { + matched: EcmaString, + position: usize, + captures: Vec>, + named: Value, +} + +fn read_replace_step( + machine: &mut Machine<'_, H>, + result: Value, + length: usize, +) -> Result { + let capture_count = length_of_array_like(machine, result)?.saturating_sub(1); + let matched = machine.get_named_property(result, "0")?; + let matched = machine.coerce_string_observable(matched)?; + let position = machine.get_named_property(result, "index")?; + let position = integer_or_infinity(machine, position)?.clamp(0.0, length as f64) as usize; + let mut captures = Vec::with_capacity(capture_count); + for index in 1..=capture_count { + let capture = machine.get_property_key(result, &index_key(index))?; + captures.push(if capture == Value::UNDEFINED { + None + } else { + Some(machine.coerce_string_observable(capture)?) + }); + } + let named = machine.get_named_property(result, "groups")?; + Ok(ReplaceStep { + matched, + position, + captures, + named, + }) +} + +fn functional_replacement( + machine: &mut Machine<'_, H>, + replacer: Value, + step: &ReplaceStep, + input: Value, +) -> Result { + let mut arguments = Vec::with_capacity(step.captures.len() + 4); + arguments.push(allocate_string(machine, step.matched.clone())?); + for capture in &step.captures { + arguments.push(match capture { + Some(text) => allocate_string(machine, text.clone())?, + None => Value::UNDEFINED, + }); + } + arguments.push(crate::number_value(step.position as f64)); + arguments.push(input); + if step.named != Value::UNDEFINED { + arguments.push(step.named); + } + let value = machine.call_value(replacer, Value::UNDEFINED, &arguments)?; + machine.coerce_string_observable(value) +} + +/// `RegExp.prototype[@@replace]`. All matching finishes before the first +/// replacement runs, so a replacer cannot change what was matched. +pub(super) fn symbol_replace( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + require_object( + machine, + this, + "RegExp.prototype[Symbol.replace] requires an object", + )?; + let string = machine.coerce_string_observable(argument(args, 0))?; + let replace_value = argument(args, 1); + let functional = machine.is_callable(replace_value)?; + let template = if functional { + None + } else { + Some(machine.coerce_string_observable(replace_value)?) + }; + let input = allocate_string(machine, string.clone())?; + with_roots(machine, &[this, input, replace_value], |machine, depth| { + let flags = flags_text(machine, this)?; + let global = contains_flag(&flags, b'g'); + let unicode = is_full_unicode(&flags); + if global { + set_last_index(machine, this, 0)?; + } + let mut results = Vec::new(); + loop { + let result = regexp_exec(machine, this, input)?; + if result == Value::NULL { + break; + } + keep_root(machine, depth, result); + results.push(result); + if !global { + break; + } + let matched = machine.get_named_property(result, "0")?; + if machine.coerce_string_observable(matched)?.is_empty() { + advance_past_empty_match(machine, this, &string, unicode)?; + } + } + let length = string.len_units(); + let mut output = EcmaStringBuilder::new(); + let mut next_source_position = 0; + for result in results { + let step = read_replace_step(machine, result, length)?; + keep_root(machine, depth, step.named); + let replacement = match &template { + None => functional_replacement(machine, replace_value, &step, input)?, + Some(template) => { + let named = if step.named == Value::UNDEFINED { + Value::UNDEFINED + } else { + let object = machine.value_to_object(step.named)?; + keep_root(machine, depth, object); + object + }; + get_substitution( + machine, + &step.matched, + &string, + step.position, + &step.captures, + named, + template, + )? + } + }; + if step.position >= next_source_position { + append( + &mut output, + &string.as_units()[next_source_position..step.position], + ); + append(&mut output, replacement.as_units()); + next_source_position = step.position + step.matched.len_units(); + } + } + if next_source_position < length { + append(&mut output, &string.as_units()[next_source_position..]); + } + allocate_string(machine, output.finish()) + }) + .map(BuiltinOutcome::Value) +} + +fn ascii_digit(unit: u16) -> Option { + (0x30..=0x39) + .contains(&unit) + .then(|| usize::from(unit - 0x30)) +} + +/// `GetSubstitution`. `captures` and `named` are empty and `undefined` for a +/// string search, which leaves `$1` and `$<` literal. `named` is an object or +/// `undefined`, and looking a group up in it can run user code. +pub(super) fn get_substitution( + machine: &mut Machine<'_, H>, + matched: &EcmaString, + string: &EcmaString, + position: usize, + captures: &[Option], + named: Value, + template: &EcmaString, +) -> Result { + let units = template.as_units(); + let source = string.as_units(); + let mut output = EcmaStringBuilder::new(); + let mut offset = 0; + while offset < units.len() { + let unit = units[offset]; + if unit != u16::from(b'$') || offset + 1 == units.len() { + output.push_unit(unit); + offset += 1; + continue; + } + let next = units[offset + 1]; + if next == u16::from(b'$') { + output.push_unit(unit); + offset += 2; + } else if next == u16::from(b'&') { + append(&mut output, matched.as_units()); + offset += 2; + } else if next == u16::from(b'`') { + append(&mut output, &source[..position.min(source.len())]); + offset += 2; + } else if next == u16::from(b'\'') { + let tail = position + .saturating_add(matched.len_units()) + .min(source.len()); + append(&mut output, &source[tail..]); + offset += 2; + } else if let Some(first) = ascii_digit(next) { + let second = units.get(offset + 2).copied().and_then(ascii_digit); + let (index, width) = match second { + Some(second) if first * 10 + second <= captures.len() => (first * 10 + second, 2), + _ => (first, 1), + }; + if index == 0 || index > captures.len() { + append(&mut output, &units[offset..offset + 1 + width]); + } else if let Some(capture) = &captures[index - 1] { + append(&mut output, capture.as_units()); + } + offset += 1 + width; + } else if next == u16::from(b'<') { + let close = units[offset + 2..] + .iter() + .position(|candidate| *candidate == u16::from(b'>')); + let (Some(close), false) = (close, named == Value::UNDEFINED) else { + append(&mut output, &units[offset..offset + 2]); + offset += 2; + continue; + }; + let name = EcmaString::from_units(&units[offset + 2..offset + 2 + close]); + let capture = machine.get_property_key(named, &crate::PropertyKey::Named(name))?; + if capture != Value::UNDEFINED { + append( + &mut output, + machine.coerce_string_observable(capture)?.as_units(), + ); + } + offset += close + 3; + } else { + output.push_unit(unit); + offset += 1; + } + } + Ok(output.finish()) +} + +/// `CreateRegExpStringIterator`. The hidden state array is reachable only from +/// the iterator, and `position` going to `None` is the `done` state. +fn regexp_string_iterator( + machine: &mut Machine<'_, H>, + matcher: Value, + input: Value, + global: bool, + unicode: bool, +) -> Result { + let state = machine + .allocate(HeapEntry::Array { + elements: vec![ + matcher, + input, + Value::boolean(global), + Value::boolean(unicode), + ], + properties: crate::PropertyMap::default(), + prototype: None, + extensible: false, + length_writable: false, + }) + .map_err(EvalFailure::Runtime)?; + machine + .allocate(HeapEntry::BuiltinIterator { + source: state, + kind: crate::IterationKind::RegExpString, + position: Some(0), + properties: crate::PropertyMap::default(), + prototype: Some( + machine + .intrinsics + .builtins + .regexp_string_iterator_prototype(), + ), + extensible: true, + }) + .map_err(EvalFailure::Runtime) +} + +/// `RegExp.prototype[@@matchAll]`. +pub(super) fn symbol_match_all( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + require_object( + machine, + this, + "RegExp.prototype[Symbol.matchAll] requires an object", + )?; + let string = machine.coerce_string_observable(argument(args, 0))?; + let input = allocate_string(machine, string)?; + with_roots(machine, &[this, input], |machine, depth| { + let default = regexp_constructor(machine)?; + let constructor = species_constructor(machine, this, default)?; + keep_root(machine, depth, constructor); + let flags = flags_text(machine, this)?; + let global = contains_flag(&flags, b'g'); + let unicode = is_full_unicode(&flags); + let flags_value = allocate_string(machine, flags)?; + let matcher = machine.construct_value(constructor, &[this, flags_value])?; + keep_root(machine, depth, matcher); + let last_index = machine.get_named_property(this, "lastIndex")?; + let last_index = to_length(machine, last_index)?; + set_last_index(machine, matcher, last_index)?; + regexp_string_iterator(machine, matcher, input, global, unicode) + }) + .map(BuiltinOutcome::Value) +} + +fn iterator_result( + machine: &mut Machine<'_, H>, + value: Value, + done: bool, +) -> Result { + machine + .iterator_result(value, done) + .map(BuiltinOutcome::Value) +} + +fn finish_iterator( + machine: &mut Machine<'_, H>, + iterator: Value, +) -> Result<(), EvalFailure> { + let index = machine + .runtime_slot(iterator) + .map_err(EvalFailure::Runtime)?; + if let Some(index) = index + && let HeapEntry::BuiltinIterator { position, .. } = &mut machine.heap[index] + { + *position = None; + } + Ok(()) +} + +/// `%RegExpStringIteratorPrototype%.next`. +pub(super) fn regexp_string_iterator_next( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let index = machine.runtime_slot(this).map_err(EvalFailure::Runtime)?; + let Some(HeapEntry::BuiltinIterator { + source, + kind: crate::IterationKind::RegExpString, + position, + .. + }) = index.map(|index| &machine.heap[index]) + else { + return Err(type_error( + "%RegExpStringIteratorPrototype%.next called on incompatible receiver", + )); + }; + let (source, done) = (*source, position.is_none()); + if done { + return iterator_result(machine, Value::UNDEFINED, true); + } + let state = machine.array_elements(source)?.unwrap_or_default(); + let &[matcher, input, global, unicode] = state.as_slice() else { + unreachable!("the hidden iterator state has four slots") + }; + let result = with_roots(machine, &[this], |machine, depth| { + let result = regexp_exec(machine, matcher, input)?; + keep_root(machine, depth, result); + if result == Value::NULL { + finish_iterator(machine, this)?; + return iterator_result(machine, Value::UNDEFINED, true); + } + if global != Value::TRUE { + finish_iterator(machine, this)?; + return iterator_result(machine, result, false); + } + let matched = machine.get_named_property(result, "0")?; + if machine.coerce_string_observable(matched)?.is_empty() { + let text = machine.string_value(input).unwrap_or_default(); + advance_past_empty_match(machine, matcher, &text, unicode == Value::TRUE)?; + } + iterator_result(machine, result, false) + })?; + Ok(result) +} + +/// Whether a RegExp receiver owns flag `flag`, for the flag accessors. The +/// prototype itself answers `undefined`, which is what makes it inspectable. +fn has_flag( + machine: &mut Machine<'_, H>, + this: Value, + flag: u8, +) -> Result { + require_object(machine, this, "RegExp flag getter requires an object")?; + if let Some((_, flags)) = regexp_parts(machine, this) { + return Ok(BuiltinOutcome::Value(Value::boolean(contains_flag( + &flags, flag, + )))); + } + if this == machine.intrinsics.regexp_prototype() { + return Ok(BuiltinOutcome::Value(Value::UNDEFINED)); + } + Err(type_error("RegExp flag getter requires a RegExp")) +} + +macro_rules! flag_getter { + ($name:ident, $flag:literal) => { + fn $name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + has_flag(machine, this, $flag) + } + }; +} +flag_getter!(dot_all, b's'); +flag_getter!(global, b'g'); +flag_getter!(has_indices, b'd'); +flag_getter!(ignore_case, b'i'); +flag_getter!(multiline, b'm'); +flag_getter!(sticky, b'y'); +flag_getter!(unicode, b'u'); +flag_getter!(unicode_sets, b'v'); + +/// `get RegExp.prototype.flags`: built from the individual accessors, in +/// `dgimsuvy` order, so each one is an observable `Get`. +pub(super) fn flags_getter( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + require_object( + machine, + this, + "RegExp.prototype.flags getter requires an object", + )?; + let mut flags = EcmaStringBuilder::new(); + for (name, flag) in [ + ("hasIndices", b'd'), + ("global", b'g'), + ("ignoreCase", b'i'), + ("multiline", b'm'), + ("dotAll", b's'), + ("unicode", b'u'), + ("unicodeSets", b'v'), + ("sticky", b'y'), + ] { + let value = machine.get_named_property(this, name)?; + if machine.to_boolean(value) { + flags.push_unit(u16::from(flag)); + } + } + Ok(BuiltinOutcome::Value(allocate_string( + machine, + flags.finish(), + )?)) +} + +pub(super) fn species_getter( + _machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + Ok(BuiltinOutcome::Value(this)) +} + +fn is_escaped_punctuator(unit: u32) -> bool { + char::from_u32(unit).is_some_and(|c| ",-=<>#&!%:;@~'`\"".contains(c)) +} + +fn is_whitespace_or_line_terminator(unit: u32) -> bool { + matches!( + unit, + 0x09..=0x0d + | 0x20 + | 0xa0 + | 0x1680 + | 0x2000..=0x200a + | 0x2028 + | 0x2029 + | 0x202f + | 0x205f + | 0x3000 + | 0xfeff + ) +} + +/// `EncodeForRegExpEscape`. +fn encode_for_escape(output: &mut EcmaStringBuilder, code_point: u32) { + let control = match code_point { + 0x09 => Some("\\t"), + 0x0a => Some("\\n"), + 0x0b => Some("\\v"), + 0x0c => Some("\\f"), + 0x0d => Some("\\r"), + _ => None, + }; + if let Some(control) = control { + output.push_utf8(control); + } else if char::from_u32(code_point).is_some_and(|c| "^$\\.*+?()[]{}|/".contains(c)) { + output.push_unit(u16::from(b'\\')); + output.push_unit(code_point as u16); + } else if is_escaped_punctuator(code_point) + || is_whitespace_or_line_terminator(code_point) + || (0xd800..=0xdfff).contains(&code_point) + { + if code_point <= 0xff { + output.push_utf8(&format!("\\x{code_point:02x}")); + } else { + output.push_utf8(&format!("\\u{code_point:04x}")); + } + } else { + output + .push_code_point(code_point) + .expect("a code point read from a string is valid"); + } +} + +/// `RegExp.escape(S)`. The first character is hex-escaped when it could join a +/// preceding `\0`-style escape or a `\c` control, so the result is safe anywhere +/// in a pattern. +fn escape( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let Some(text) = machine.string_value(argument(args, 0)) else { + return Err(type_error("RegExp.escape requires a string")); + }; + let mut output = EcmaStringBuilder::with_capacity(text.len_units()); + for (offset, code_point) in text.code_points() { + let initial_alphanumeric = + offset == 0 && u8::try_from(code_point).is_ok_and(|byte| byte.is_ascii_alphanumeric()); + if initial_alphanumeric { + output.push_utf8(&format!("\\x{code_point:02x}")); + } else { + encode_for_escape(&mut output, code_point); + } + } + Ok(BuiltinOutcome::Value(allocate_string( + machine, + output.finish(), + )?)) +} + +/// Installs the flag accessors, the `Symbol`-keyed methods and `RegExp.escape`. +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + prototype: Value, + constructor: Value, +) { + use super::{define_symbol_property, install_function}; + use crate::intrinsics::BuiltinHandler; + + for (property, getter_name, handler) in [ + ("dotAll", "get dotAll", dot_all:: as BuiltinHandler), + ("global", "get global", global::), + ("hasIndices", "get hasIndices", has_indices::), + ("ignoreCase", "get ignoreCase", ignore_case::), + ("multiline", "get multiline", multiline::), + ("sticky", "get sticky", sticky::), + ("unicode", "get unicode", unicode::), + ("unicodeSets", "get unicodeSets", unicode_sets::), + ] { + let getter = install_function(heap, builtins, getter_name, 0, handler); + super::regexp::define_getter(heap, prototype, property, getter); + } + let symbols = [ + ( + builtins.symbol_match(), + "[Symbol.match]", + 1, + symbol_match:: as BuiltinHandler, + ), + ( + builtins.symbol_match_all(), + "[Symbol.matchAll]", + 1, + symbol_match_all::, + ), + ( + builtins.symbol_replace(), + "[Symbol.replace]", + 2, + symbol_replace::, + ), + ( + builtins.symbol_search(), + "[Symbol.search]", + 1, + symbol_search::, + ), + ( + builtins.symbol_split(), + "[Symbol.split]", + 2, + symbol_split::, + ), + ]; + for (symbol, name, length, handler) in symbols { + let function = install_function(heap, builtins, name, length, handler); + define_symbol_property(heap, prototype, symbol, super::builtin_property(function)); + } + let escape_function = install_function(heap, builtins, "escape", 1, escape::); + super::define_data(heap, constructor, "escape", escape_function); + let species = install_function( + heap, + builtins, + "get [Symbol.species]", + 0, + species_getter::, + ); + define_symbol_property( + heap, + constructor, + builtins.symbol_species(), + Property::Accessor { + getter: Some(species), + setter: None, + enumerable: false, + configurable: true, + }, + ); + let iterator_prototype = crate::intrinsics::push( + heap, + HeapEntry::Object { + properties: crate::PropertyMap::default(), + prototype: Some(builtins.iterator_prototype()), + extensible: true, + boxed_primitive: None, + }, + ); + let next = install_function(heap, builtins, "next", 0, regexp_string_iterator_next::); + super::define_data(heap, iterator_prototype, "next", next); + super::define_to_string_tag( + heap, + iterator_prototype, + builtins.symbol_to_string_tag(), + "RegExp String Iterator", + ); + builtins.set_regexp_string_iterator_prototype(iterator_prototype); +} + +#[cfg(test)] +mod tests { + use super::super::test_support::{TestHost, blank_program, ordinary_object}; + use super::*; + use crate::intrinsics::{BuiltinDef, native_function}; + use crate::{Limits, ThrowOrigin}; + + fn regexp(machine: &mut Machine<'_, TestHost>, pattern: &str, flags: &str) -> Value { + let constructor = machine.intrinsics.global("RegExp").unwrap(); + let pattern = allocate_string(machine, EcmaString::encode(pattern)).unwrap(); + let flags = allocate_string(machine, EcmaString::encode(flags)).unwrap(); + machine + .construct_value(constructor, &[pattern, flags]) + .unwrap() + } + + #[test] + fn flag_accessors_answer_for_regexps_and_the_prototype_only() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let value = regexp(&mut machine, "x", "gi"); + assert_eq!( + machine.get_named_property(value, "global").unwrap(), + Value::TRUE + ); + assert_eq!( + machine.get_named_property(value, "sticky").unwrap(), + Value::FALSE + ); + let key = crate::PropertyKey::Named(EcmaString::encode("global")); + assert_eq!( + machine.own_descriptor(value, &key).unwrap(), + None, + "a flag is an accessor on the prototype, not an own property" + ); + let prototype = machine.intrinsics.regexp_prototype(); + assert_eq!( + machine.get_named_property(prototype, "global").unwrap(), + Value::UNDEFINED + ); + let object = ordinary_object(&mut machine); + machine + .internal_set_prototype_of(object, Some(prototype)) + .unwrap(); + assert!(matches!( + machine.get_named_property(object, "global"), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + + #[test] + fn flags_reads_each_accessor_so_an_override_is_seen() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let object = ordinary_object(&mut machine); + for (name, value) in [("global", Value::TRUE), ("sticky", Value::int32(1))] { + machine.set_data_property(object, name, value).unwrap(); + } + let BuiltinOutcome::Value(flags) = flags_getter(&mut machine, object, &[], false).unwrap() + else { + panic!("the flags getter returns a value"); + }; + assert!(machine.string_value(flags).unwrap().eq_ascii("gy")); + assert!(matches!( + flags_getter(&mut machine, Value::int32(1), &[], false), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + + fn text(machine: &mut Machine<'_, TestHost>, value: &str) -> Value { + allocate_string(machine, EcmaString::encode(value)).unwrap() + } + + fn outcome_value(outcome: BuiltinOutcome) -> Value { + match outcome { + BuiltinOutcome::Value(value) => value, + _ => panic!("the builtin returns a value"), + } + } + + /// A user `exec` that reports a match of `b` at index 1, whatever the input. + fn fixed_exec( + machine: &mut Machine<'_, TestHost>, + _this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let matched = allocate_string(machine, EcmaString::encode("b"))?; + let result = allocate_array(machine, vec![matched])?; + machine.set_data_property(result, "index", Value::int32(1))?; + Ok(BuiltinOutcome::Value(result)) + } + + #[test] + fn replace_is_driven_by_a_user_exec_rather_than_the_matcher() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let rx = regexp(&mut machine, "never", ""); + let id = machine.intrinsics.builtins.register(BuiltinDef { + name: "fixed exec", + length: 1, + handler: fixed_exec, + }); + let exec = native_function(&mut machine.heap, id, "fixed exec", 1); + machine.set_data_property(rx, "exec", exec).unwrap(); + let (string, replacement) = (text(&mut machine, "abc"), text(&mut machine, "[$&]")); + let result = + outcome_value(symbol_replace(&mut machine, rx, &[string, replacement], false).unwrap()); + assert!(machine.string_value(result).unwrap().eq_ascii("a[b]c")); + } + + #[test] + fn a_receiver_without_exec_must_be_a_regexp() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let object = ordinary_object(&mut machine); + let string = text(&mut machine, "abc"); + assert!(matches!( + symbol_search(&mut machine, object, &[string], false), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + + #[test] + fn a_global_match_steps_past_an_empty_match_by_code_point_under_unicode() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let string = text(&mut machine, "😀"); + for (flags, expected) in [("g", 3), ("gu", 2)] { + let rx = regexp(&mut machine, "(?:)", flags); + let result = outcome_value(symbol_match(&mut machine, rx, &[string], false).unwrap()); + assert_eq!( + machine.array_elements(result).unwrap().unwrap().len(), + expected, + "with flags {flags}" + ); + } + } + + #[test] + fn search_leaves_last_index_as_it_found_it() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let rx = regexp(&mut machine, "b", "g"); + machine + .set_data_property(rx, "lastIndex", Value::int32(9)) + .unwrap(); + let string = text(&mut machine, "abc"); + assert_eq!( + outcome_value(symbol_search(&mut machine, rx, &[string], false).unwrap()), + Value::int32(1) + ); + assert_eq!( + machine.get_named_property(rx, "lastIndex").unwrap(), + Value::int32(9) + ); + } + + #[test] + fn split_keeps_captures_and_stops_at_the_limit() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let rx = regexp(&mut machine, "(b)", ""); + let string = text(&mut machine, "abcbd"); + let pieces = |machine: &mut Machine<'_, TestHost>, result| -> Vec { + let values = machine.array_elements(result).unwrap().unwrap(); + values + .into_iter() + .map(|value| machine.string_value(value).unwrap().to_utf8_lossy()) + .collect() + }; + let all = outcome_value(symbol_split(&mut machine, rx, &[string], false).unwrap()); + assert_eq!(pieces(&mut machine, all), ["a", "b", "c", "b", "d"]); + let limited = outcome_value( + symbol_split(&mut machine, rx, &[string, Value::int32(2)], false).unwrap(), + ); + assert_eq!(pieces(&mut machine, limited), ["a", "b"]); + } + + #[test] + fn match_all_stays_done_once_exhausted() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let rx = regexp(&mut machine, "a", "g"); + let string = text(&mut machine, "aa"); + let iterator = outcome_value(symbol_match_all(&mut machine, rx, &[string], false).unwrap()); + let step = |machine: &mut Machine<'_, TestHost>| { + let result = + outcome_value(regexp_string_iterator_next(machine, iterator, &[], false).unwrap()); + machine.get_named_property(result, "done").unwrap() + }; + assert_eq!(step(&mut machine), Value::FALSE); + assert_eq!(step(&mut machine), Value::FALSE); + assert_eq!(step(&mut machine), Value::TRUE); + assert_eq!(step(&mut machine), Value::TRUE); + assert!(matches!( + regexp_string_iterator_next(&mut machine, string, &[], false), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + + fn escaped(machine: &mut Machine<'_, TestHost>, units: &[u16]) -> Vec { + let input = allocate_string(machine, EcmaString::from_units(units)).unwrap(); + let result = outcome_value(escape(machine, Value::UNDEFINED, &[input], false).unwrap()); + machine.string_value(result).unwrap().as_units().to_vec() + } + + #[test] + fn escape_is_safe_at_the_start_of_a_pattern_and_after_any_syntax() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + for (input, expected) in [ + ("abc", "\\x61bc"), + ("1", "\\x31"), + ("_a", "_a"), + (".*", "\\.\\*"), + ("a/b", "\\x61\\/b"), + ("\n", "\\n"), + ("-", "\\x2d"), + (" ", "\\x20"), + ("\u{2028}", "\\u2028"), + ] { + let units: Vec = input.encode_utf16().collect(); + let expected: Vec = expected.encode_utf16().collect(); + assert_eq!(escaped(&mut machine, &units), expected, "{input:?}"); + } + let lone: Vec = "\\ud800".encode_utf16().collect(); + assert_eq!(escaped(&mut machine, &[0xd800]), lone); + assert!(matches!( + escape(&mut machine, Value::UNDEFINED, &[Value::int32(1)], false), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + + #[test] + fn substitution_reads_each_reference_form() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let captures = [Some(EcmaString::encode("x")), None]; + for (template, expected) in [ + ("$$", "$"), + ("$&", "b"), + ("$`", "a"), + ("$'", "c"), + ("$1", "x"), + ("$01", "x"), + ("$2", ""), + ("$10", "x0"), + ("$0", "$0"), + ("$3", "$3"), + ("$", "$"), + ("$", "$"), + ] { + let result = get_substitution( + &mut machine, + &EcmaString::encode("b"), + &EcmaString::encode("abc"), + 1, + &captures, + Value::UNDEFINED, + &EcmaString::encode(template), + ) + .unwrap(); + assert!(result.eq_ascii(expected), "{template}"); + } + let groups = ordinary_object(&mut machine); + let value = text(&mut machine, "named"); + machine.set_data_property(groups, "n", value).unwrap(); + let result = get_substitution( + &mut machine, + &EcmaString::encode("b"), + &EcmaString::encode("abc"), + 1, + &[], + groups, + &EcmaString::encode("$$"), + ) + .unwrap(); + assert!(result.eq_ascii("named")); + } +} diff --git a/crates/bamts-runtime/src/builtins/regexp_v/mod.rs b/crates/bamts-runtime/src/builtins/regexp_v/mod.rs index 65dee046..e766e229 100644 --- a/crates/bamts-runtime/src/builtins/regexp_v/mod.rs +++ b/crates/bamts-runtime/src/builtins/regexp_v/mod.rs @@ -336,7 +336,7 @@ fn is_pattern_syntax_character(value: u16) -> bool { /// A compiled pattern under the full flag set: the engine regex, plus the /// public-to-internal capture-name mapping that duplicate named groups need. #[derive(Clone, Debug)] -pub(super) struct CompiledV { +pub(crate) struct CompiledV { regex: Regex, flags: VFlags, /// Public name in source order, paired with the internal names the lowered @@ -347,10 +347,6 @@ pub(super) struct CompiledV { } impl CompiledV { - pub(super) fn flags(&self) -> VFlags { - self.flags - } - pub(super) fn engine_regex(&self) -> &Regex { &self.regex } @@ -381,10 +377,10 @@ impl CompiledV { self.names .iter() .map(|(public, internal)| { - let resolved = internal - .iter() - .filter_map(|name| matched.named.get(name).cloned().flatten()) - .next(); + let resolved = internal.iter().find_map(|name| { + let found = matched.named.iter().find(|(known, _)| known == name)?; + found.1.clone() + }); (public.clone(), resolved) }) .collect() @@ -444,21 +440,7 @@ pub(super) fn compile( }) } -fn last_index_value(machine: &Machine<'_, H>, value: Value) -> Result { - let numeric = machine.to_number(value)?; - let number = match numeric.decode() { - Some(bamts_native::Decoded::Int32(value)) => f64::from(value as i32), - Some(bamts_native::Decoded::Number(value)) => value, - _ => f64::NAN, - }; - if number.is_nan() || number <= 0.0 { - return Ok(0); - } - let upper = (usize::MAX as f64).min(9_007_199_254_740_991.0); - Ok(number.floor().min(upper) as usize) -} - -fn execution_failure(error: RegexError) -> EvalFailure { +pub(super) fn execution_failure(error: RegexError) -> EvalFailure { match error.kind() { RegexErrorKind::BudgetExhausted => { EvalFailure::Runtime(crate::RuntimeErrorKind::RegexpStepBudgetExceeded { @@ -471,47 +453,16 @@ fn execution_failure(error: RegexError) -> EvalFailure { } } -/// Executes the compiled matcher and performs the observable lastIndex -/// transitions required by RegExpBuiltinExec. -pub(super) fn execute( - machine: &mut Machine<'_, H>, - regexp: Value, - pattern: &EcmaString, - flags: &EcmaString, - input: &EcmaString, -) -> Result<(CompiledV, Option), EvalFailure> { - let compiled = compile(machine, pattern, flags)?; - let uses_last_index = compiled.flags().global || compiled.flags().sticky; - let start = if uses_last_index { - let last_index = machine.get_named_property(regexp, "lastIndex")?; - last_index_value(machine, last_index)? - } else { - 0 - }; - let matched = compiled.exec(input, start).map_err(execution_failure)?; - if uses_last_index { - let next = matched.as_ref().map_or(0, |matched| matched.range.end); - machine.set_data_property(regexp, "lastIndex", crate::number_value(next as f64))?; - } - Ok((compiled, matched)) -} - -/// Implements the dynamic RegExpExec protocol. None asks the caller to use -/// RegExpBuiltinExec after checking the receiver's RegExp internal state. -pub(super) fn call_exec_override( +/// `CreateDataProperty`: the result of a match is built with own properties, so +/// a setter on `Array.prototype` cannot observe it. +pub(super) fn create_result_property( machine: &mut Machine<'_, H>, - regexp: Value, - input: Value, -) -> Result, EvalFailure> { - let exec = machine.get_named_property(regexp, "exec")?; - if !machine.is_callable(exec)? { - return Ok(None); - } - let result = machine.call_value(exec, regexp, &[input])?; - if result == Value::NULL || machine.is_object(result) { - return Ok(Some(result)); - } - Err(super::type_error("RegExp exec method returned non-object")) + array: Value, + name: &str, + value: Value, +) -> Result<(), EvalFailure> { + let key = crate::PropertyKey::Named(EcmaString::encode(name)); + machine.create_data_property_key(array, key, value) } pub(super) fn plain_object(machine: &mut Machine<'_, H>) -> Result { @@ -559,13 +510,14 @@ pub(super) fn match_array( }); } let array = allocate_array(machine, values)?; - machine.set_data_property( + create_result_property( + machine, array, "index", crate::number_value(matched.range.start as f64), )?; let input_value = allocate_string(machine, input.clone())?; - machine.set_data_property(array, "input", input_value)?; + create_result_property(machine, array, "input", input_value)?; let resolved = compiled.public_named(matched); let groups = if resolved.is_empty() { @@ -579,15 +531,15 @@ pub(super) fn match_array( } None => Value::UNDEFINED, }; - machine.set_data_property(groups, name, value)?; + create_result_property(machine, groups, name, value)?; } groups }; - machine.set_data_property(array, "groups", groups)?; + create_result_property(machine, array, "groups", groups)?; if compiled.flags.has_indices { let indices = build_indices(machine, matched, &resolved)?; - machine.set_data_property(array, "indices", indices)?; + create_result_property(machine, array, "indices", indices)?; } Ok(array) } @@ -608,11 +560,11 @@ fn build_indices( let groups = plain_object(machine)?; for (name, range) in resolved { let value = range_pair(machine, range.as_ref())?; - machine.set_data_property(groups, name, value)?; + create_result_property(machine, groups, name, value)?; } groups }; - machine.set_data_property(indices, "groups", groups)?; + create_result_property(machine, indices, "groups", groups)?; Ok(indices) } @@ -838,6 +790,17 @@ impl<'a> VParser<'a> { names.push(public); } }, + Some(unit @ (0x69 | 0x6d | 0x73 | 0x2d)) => { + self.out.push_str("(?"); + self.out.push(char::from(unit as u8)); + while self.out.as_bytes().last() != Some(&b':') { + let modifier = self + .next() + .and_then(|unit| u8::try_from(unit).ok()) + .ok_or_else(|| self.error("invalid group"))?; + self.out.push(char::from(modifier)); + } + } _ => return Err(self.error("invalid group")), } } else { diff --git a/crates/bamts-runtime/src/builtins/string.rs b/crates/bamts-runtime/src/builtins/string.rs index 792bc576..d4dd006e 100644 --- a/crates/bamts-runtime/src/builtins/string.rs +++ b/crates/bamts-runtime/src/builtins/string.rs @@ -3,12 +3,13 @@ use std::collections::BTreeMap; use bamts_bytecode::{EcmaString, EcmaStringBuilder}; use bamts_native::{Decoded, Value}; +use super::intl::{TargetCase, compare_with_collator, transform_case}; use super::{ allocate_array, allocate_string, define_data, install_constructor_function, install_function, - range_error, to_integer_or_infinity, type_error, uri_error, value_number, + range_error, type_error, uri_error, value_number, with_roots, }; use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; -use crate::{EvalFailure, HeapEntry, Host, IterationKind, Machine, PropertyKey, PropertyMap}; +use crate::{EvalFailure, HeapEntry, Host, IterationKind, Machine, PropertyKey}; pub(super) fn install( heap: &mut Vec, @@ -27,7 +28,9 @@ pub(super) fn install( define_static(heap, constructor, name, f) } for (name, length, handler) in [ - ("charAt", 1, char_at:: as BuiltinHandler), + ("toString", 0, this_string_value:: as BuiltinHandler), + ("valueOf", 0, this_string_value::), + ("charAt", 1, char_at::), ("charCodeAt", 1, char_code_at::), ("codePointAt", 1, code_point_at::), ("at", 1, at::), @@ -40,12 +43,13 @@ pub(super) fn install( ("endsWith", 1, ends_with::), ("split", 2, split::), ("replace", 2, replace::), - ("replaceAll", 2, replace_all::), ("trim", 0, trim::), ("trimStart", 0, trim_start::), ("trimEnd", 0, trim_end::), ("toUpperCase", 0, to_upper::), ("toLowerCase", 0, to_lower::), + ("toLocaleUpperCase", 0, to_locale_upper::), + ("toLocaleLowerCase", 0, to_locale_lower::), ("padStart", 1, pad_start::), ("padEnd", 1, pad_end::), ("repeat", 1, repeat::), @@ -86,21 +90,24 @@ fn constructor( ) -> Result { let text = if args.is_empty() { EcmaString::default() + } else if constructing { + machine.coerce_string_observable(args[0])? } else { machine.string_constructor_text(args[0])? }; let value = allocate_string(machine, text)?; if constructing { - Ok(BuiltinOutcome::Value(machine.box_primitive(value)?)) + let prototype = machine.intrinsics.string_prototype; + super::wrap_for_new_target(machine, value, prototype) } else { Ok(BuiltinOutcome::Value(value)) } } -fn text(machine: &Machine<'_, H>, this: Value) -> Result { +fn text(machine: &mut Machine<'_, H>, this: Value) -> Result { if matches!(this.decode(), Some(Decoded::Undefined | Decoded::Null)) { return Err(type_error("String method called on null or undefined")); } - machine.to_string(machine.unbox_primitive_or_self(this)?) + machine.coerce_string_observable(this) } fn locale_compare( machine: &mut Machine<'_, H>, @@ -114,6 +121,11 @@ fn locale_compare( let this = machine.coerce_string_observable(this)?; let other = machine.coerce_string_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let locales = args.get(1).copied().unwrap_or(Value::UNDEFINED); + let options = args.get(2).copied().unwrap_or(Value::UNDEFINED); + if locales != Value::UNDEFINED || options != Value::UNDEFINED { + return compare_with_collator(machine, &this, &other, locales, options); + } let result: i32 = match this.as_units().cmp(other.as_units()) { std::cmp::Ordering::Less => -1, std::cmp::Ordering::Equal => 0, @@ -126,9 +138,38 @@ fn append(out: &mut EcmaStringBuilder, text: &EcmaString) { out.push_unit(unit); } } -fn integer(machine: &Machine<'_, H>, value: Value) -> Result { - Ok(to_integer_or_infinity(machine, value)? as isize) +fn integer(machine: &mut Machine<'_, H>, value: Value) -> Result { + Ok(super::array_es2023::integer_or_infinity(machine, value)? as isize) } +/// Rejects a search string that is a RegExp, for the methods that take text only. +fn require_not_regexp( + machine: &mut Machine<'_, H>, + value: Value, + operation: &'static str, +) -> Result<(), EvalFailure> { + if super::string_edge::is_regexp(machine, value)? { + return Err(type_error(operation)); + } + Ok(()) +} +/// `String.prototype.toString` and `valueOf`: the receiver's own string. Neither +/// coerces, so a receiver that is not a String throws. +fn this_string_value( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = machine.unbox_primitive(this, "String.prototype.toString")?; + if machine.string_value(value).is_some() { + Ok(BuiltinOutcome::Value(value)) + } else { + Err(type_error( + "String.prototype.toString called on incompatible receiver", + )) + } +} + fn from_char_code( machine: &mut Machine<'_, H>, _: Value, @@ -137,7 +178,9 @@ fn from_char_code( ) -> Result { let mut out = EcmaStringBuilder::with_capacity(args.len()); for arg in args { - out.push_unit(value_number(machine.to_number(*arg)?) as u16); + out.push_unit( + crate::to_uint32(value_number(machine.coerce_number_observable(*arg)?)) as u16, + ); } Ok(BuiltinOutcome::Value(allocate_string( machine, @@ -157,11 +200,13 @@ fn raw( .ok_or_else(|| type_error("String.raw requires template.raw"))?; let mut out = EcmaStringBuilder::new(); for (i, value) in values.iter().enumerate() { - append(&mut out, &machine.to_string(*value)?); + append(&mut out, &machine.coerce_string_observable(*value)?); if i + 1 < values.len() { append( &mut out, - &machine.to_string(args.get(i + 1).copied().unwrap_or(Value::UNDEFINED))?, + &machine.coerce_string_observable( + args.get(i + 1).copied().unwrap_or(Value::UNDEFINED), + )?, ); } } @@ -269,15 +314,10 @@ fn slice( integer(machine, args.first().copied().unwrap_or(Value::int32(0)))?, string.len_units(), ); - let end = rel( - integer( - machine, - args.get(1) - .copied() - .unwrap_or(crate::number_value(string.len_units() as f64)), - )?, - string.len_units(), - ); + let end = match args.get(1).copied() { + None | Some(Value::UNDEFINED) => string.len_units(), + Some(end) => rel(integer(machine, end)?, string.len_units()), + }; let result = if end > start { string .slice_units(start..end) @@ -329,7 +369,8 @@ fn index_of( _: bool, ) -> Result { let haystack = text(machine, this)?; - let needle = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let needle = + machine.coerce_string_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?; let start = integer(machine, args.get(1).copied().unwrap_or(Value::int32(0)))?.max(0) as usize; Ok(BuiltinOutcome::Value(crate::number_value( search_units(haystack.as_units(), needle.as_units(), start).map_or(-1.0, |i| i as f64), @@ -342,13 +383,16 @@ fn last_index_of( _: bool, ) -> Result { let haystack = text(machine, this)?; - let needle = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; - let end = if args.len() < 2 || args[1] == Value::UNDEFINED { + let needle = + machine.coerce_string_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let position = value_number( + machine.coerce_number_observable(args.get(1).copied().unwrap_or(Value::UNDEFINED))?, + ); + let end = if position.is_nan() { haystack.len_units() } else { - integer(machine, args[1])?.max(0) as usize - } - .min(haystack.len_units()); + (position.max(0.0) as usize).min(haystack.len_units()) + }; let found = (0..=end).rev().find(|index| { haystack .as_units() @@ -365,11 +409,17 @@ fn includes( args: &[Value], _: bool, ) -> Result { - let BuiltinOutcome::Value(v) = index_of(machine, this, args, false)? else { - unreachable!() - }; + let haystack = text(machine, this)?; + let search = args.first().copied().unwrap_or(Value::UNDEFINED); + require_not_regexp( + machine, + search, + "String.prototype.includes cannot search for a RegExp", + )?; + let needle = machine.coerce_string_observable(search)?; + let start = integer(machine, args.get(1).copied().unwrap_or(Value::int32(0)))?.max(0) as usize; Ok(BuiltinOutcome::Value(Value::boolean( - value_number(v) >= 0.0, + search_units(haystack.as_units(), needle.as_units(), start).is_some(), ))) } fn starts_with( @@ -379,9 +429,16 @@ fn starts_with( _: bool, ) -> Result { let haystack = text(machine, this)?; - let needle = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; - let position = - integer(machine, args.get(1).copied().unwrap_or(Value::int32(0)))?.max(0) as usize; + let search = args.first().copied().unwrap_or(Value::UNDEFINED); + require_not_regexp( + machine, + search, + "String.prototype.startsWith cannot search for a RegExp", + )?; + let needle = machine.coerce_string_observable(search)?; + let position = (integer(machine, args.get(1).copied().unwrap_or(Value::int32(0)))?.max(0) + as usize) + .min(haystack.len_units()); Ok(BuiltinOutcome::Value(Value::boolean( haystack .as_units() @@ -396,7 +453,13 @@ fn ends_with( _: bool, ) -> Result { let haystack = text(machine, this)?; - let needle = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let search = args.first().copied().unwrap_or(Value::UNDEFINED); + require_not_regexp( + machine, + search, + "String.prototype.endsWith cannot search for a RegExp", + )?; + let needle = machine.coerce_string_observable(search)?; let end = if args.len() < 2 || args[1] == Value::UNDEFINED { haystack.len_units() } else { @@ -409,56 +472,104 @@ fn ends_with( ))) } +fn require_coercible(value: Value) -> Result<(), EvalFailure> { + if matches!(value.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Err(type_error("String method called on null or undefined")); + } + Ok(()) +} + +/// `GetMethod(argument, symbol)` for the methods that delegate to a pattern. A +/// primitive argument is never consulted, so a method added to +/// `Number.prototype` cannot take over a plain string search. +fn pattern_method( + machine: &mut Machine<'_, H>, + argument: Value, + symbol: Value, +) -> Result, EvalFailure> { + if !machine.is_object(argument) { + return Ok(None); + } + let key = machine.to_property_key(symbol)?; + let method = machine.get_property_key(argument, &key)?; + if matches!(method.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Ok(None); + } + if machine.is_callable(method)? { + Ok(Some(method)) + } else { + Err(type_error("pattern method is not callable")) + } +} + +/// `Invoke(rx, symbol, «input»)` on a RegExp this method just created. +fn invoke_on_new_regexp( + machine: &mut Machine<'_, H>, + regexp: Value, + symbol: Value, + input: Value, +) -> Result { + with_roots(machine, &[regexp, input], |machine, _| { + let key = machine.to_property_key(symbol)?; + let method = machine.get_property_key(regexp, &key)?; + machine.call_value(method, regexp, &[input]) + }) + .map(BuiltinOutcome::Value) +} + fn split( machine: &mut Machine<'_, H>, this: Value, args: &[Value], _: bool, ) -> Result { - if let Some(separator) = args.first().copied() - && super::regexp::regexp_parts(machine, separator).is_some() - { - return split_regexp(machine, this, args); + require_coercible(this)?; + let separator = args.first().copied().unwrap_or(Value::UNDEFINED); + let limit = args.get(1).copied().unwrap_or(Value::UNDEFINED); + let symbol = machine.intrinsics.builtins.symbol_split(); + if let Some(method) = pattern_method(machine, separator, symbol)? { + return Ok(BuiltinOutcome::Value(machine.call_value( + method, + separator, + &[this, limit], + )?)); } let string = text(machine, this)?; - let limit = value_number( - machine.to_number( - args.get(1) - .copied() - .unwrap_or(crate::number_value(u32::MAX as f64)), - )?, - ) as u32 as usize; + let limit = if limit == Value::UNDEFINED { + u32::MAX + } else { + crate::to_uint32(value_number(machine.coerce_number_observable(limit)?)) + } as usize; + let separator_text = machine.coerce_string_observable(separator)?; if limit == 0 { return Ok(BuiltinOutcome::Value(allocate_array(machine, Vec::new())?)); } - let parts = if args.is_empty() || args[0] == Value::UNDEFINED { + let parts = if separator == Value::UNDEFINED { vec![string] + } else if separator_text.is_empty() { + string + .as_units() + .iter() + .map(|unit| EcmaString::from_units(&[*unit])) + .collect() } else { - let separator = machine.to_string(args[0])?; - if separator.is_empty() { - string - .as_units() - .iter() - .map(|unit| EcmaString::from_units(&[*unit])) - .collect() - } else { - let mut parts = Vec::new(); - let mut cursor = 0; - while let Some(offset) = search_units(string.as_units(), separator.as_units(), cursor) { - parts.push( - string - .slice_units(cursor..offset) - .expect("separator search returns string bounds"), - ); - cursor = offset + separator.len_units(); - } + let mut parts = Vec::new(); + let mut cursor = 0; + while let Some(offset) = search_units(string.as_units(), separator_text.as_units(), cursor) + { parts.push( string - .slice_units(cursor..string.len_units()) - .expect("separator search preserves string bounds"), + .slice_units(cursor..offset) + .expect("separator search returns string bounds"), ); - parts + cursor = offset + separator_text.len_units(); } + parts.push( + string + .slice_units(cursor..string.len_units()) + .expect("separator search preserves string bounds"), + ); + parts }; let mut values = Vec::new(); for part in parts.into_iter().take(limit) { @@ -466,135 +577,71 @@ fn split( } Ok(BuiltinOutcome::Value(allocate_array(machine, values)?)) } -fn replacement( - machine: &mut Machine<'_, H>, - replacer: Value, - matched: &EcmaString, - index: usize, - whole: &EcmaString, -) -> Result { - if machine.is_callable(replacer)? { - let matched_value = allocate_string(machine, matched.clone())?; - let whole_value = allocate_string(machine, whole.clone())?; - return machine - .call_value( - replacer, - Value::UNDEFINED, - &[ - matched_value, - crate::number_value(index as f64), - whole_value, - ], - ) - .and_then(|value| machine.to_string(value)); - } - let template = machine.to_string(replacer)?; - let mut output = EcmaStringBuilder::new(); - let units = template.as_units(); - let mut offset = 0; - while offset < units.len() { - if units[offset] != u16::from(b'$') || offset + 1 == units.len() { - output.push_unit(units[offset]); - offset += 1; - continue; - } - match units[offset + 1] { - unit if unit == u16::from(b'$') => output.push_unit(u16::from(b'$')), - unit if unit == u16::from(b'&') => append(&mut output, matched), - unit if unit == u16::from(b'`') => { - append( - &mut output, - &whole - .slice_units(0..index) - .expect("match index is within the source string"), - ); - } - unit if unit == u16::from(b'\'') => { - append( - &mut output, - &whole - .slice_units(index + matched.len_units()..whole.len_units()) - .expect("match range is within the source string"), - ); - } - _ => { - output.push_unit(units[offset]); - offset += 1; - continue; - } - } - offset += 2; - } - Ok(output.finish()) -} -fn replace_impl( + +fn replace( machine: &mut Machine<'_, H>, this: Value, args: &[Value], - all: bool, + _: bool, ) -> Result { - if let Some(search) = args.first().copied() - && super::regexp::regexp_parts(machine, search).is_some() - { - return replace_regexp(machine, this, args, all); + require_coercible(this)?; + let search = args.first().copied().unwrap_or(Value::UNDEFINED); + let replace_value = args.get(1).copied().unwrap_or(Value::UNDEFINED); + let symbol = machine.intrinsics.builtins.symbol_replace(); + if let Some(method) = pattern_method(machine, search, symbol)? { + return Ok(BuiltinOutcome::Value(machine.call_value( + method, + search, + &[this, replace_value], + )?)); } let string = text(machine, this)?; - let needle = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; - let replacer = args.get(1).copied().unwrap_or(Value::UNDEFINED); - let mut output = EcmaStringBuilder::new(); - let mut cursor = 0; - while cursor <= string.len_units() { - let Some(index) = search_units(string.as_units(), needle.as_units(), cursor) else { - break; - }; - append( - &mut output, - &string - .slice_units(cursor..index) - .expect("search result is within the source string"), - ); - append( - &mut output, - &replacement(machine, replacer, &needle, index, &string)?, - ); - cursor = index + needle.len_units(); - if !all { - break; - } - if needle.is_empty() && cursor < string.len_units() { - output.push_unit(string.unit_at(cursor).expect("cursor is in bounds")); - cursor += 1; - } else if needle.is_empty() && cursor == string.len_units() { - break; + let needle = machine.coerce_string_observable(search)?; + let functional = machine.is_callable(replace_value)?; + let template = if functional { + None + } else { + Some(machine.coerce_string_observable(replace_value)?) + }; + let Some(position) = search_units(string.as_units(), needle.as_units(), 0) else { + return Ok(BuiltinOutcome::Value(allocate_string(machine, string)?)); + }; + let replacement = match template { + Some(template) => super::regexp_protocol::get_substitution( + machine, + &needle, + &string, + position, + &[], + Value::UNDEFINED, + &template, + )?, + None => { + let matched = allocate_string(machine, needle.clone())?; + let whole = allocate_string(machine, string.clone())?; + let value = machine.call_value( + replace_value, + Value::UNDEFINED, + &[matched, crate::number_value(position as f64), whole], + )?; + machine.coerce_string_observable(value)? } - } + }; + let mut output = EcmaStringBuilder::new(); append( &mut output, - &string - .slice_units(cursor.min(string.len_units())..string.len_units()) - .expect("cursor was clamped to the source string"), + &super::regexp::slice_units(&string, 0..position), + ); + append(&mut output, &replacement); + append( + &mut output, + &super::regexp::slice_units(&string, position + needle.len_units()..string.len_units()), ); Ok(BuiltinOutcome::Value(allocate_string( machine, output.finish(), )?)) } -fn replace( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - replace_impl(machine, this, args, false) -} -fn replace_all( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _: bool, -) -> Result { - replace_impl(machine, this, args, true) -} fn is_js_whitespace(unit: u16) -> bool { matches!( unit, @@ -677,6 +724,32 @@ fn to_upper( output.finish(), )?)) } +fn to_locale_upper( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + if matches!(this.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Err(type_error("String method called on null or undefined")); + } + let source = machine.coerce_string_observable(this)?; + let locales = args.first().copied().unwrap_or(Value::UNDEFINED); + transform_case(machine, &source, locales, TargetCase::Upper) +} +fn to_locale_lower( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _: bool, +) -> Result { + if matches!(this.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Err(type_error("String method called on null or undefined")); + } + let source = machine.coerce_string_observable(this)?; + let locales = args.first().copied().unwrap_or(Value::UNDEFINED); + transform_case(machine, &source, locales, TargetCase::Lower) +} fn to_lower( machine: &mut Machine<'_, H>, this: Value, @@ -685,17 +758,18 @@ fn to_lower( ) -> Result { let source = text(machine, this)?; let mut output = EcmaStringBuilder::new(); + // Lowering whole runs lets str::to_lowercase see the context Final_Sigma needs. + let mut run = String::new(); for (_, code_point) in source.code_points() { if let Some(character) = char::from_u32(code_point) { - for mapped in character.to_lowercase() { - output - .push_code_point(mapped as u32) - .expect("Rust char is valid"); - } + run.push(character); } else { + output.push_utf8(&run.to_lowercase()); + run.clear(); output.push_unit(code_point as u16); } } + output.push_utf8(&run.to_lowercase()); Ok(BuiltinOutcome::Value(allocate_string( machine, output.finish(), @@ -708,16 +782,18 @@ fn pad( start: bool, ) -> Result { let string = text(machine, this)?; - let target = - to_integer_or_infinity(machine, args.first().copied().unwrap_or(Value::UNDEFINED))? - .max(0.0); + let target = super::array_es2023::integer_or_infinity( + machine, + args.first().copied().unwrap_or(Value::UNDEFINED), + )? + .max(0.0); if target <= string.len_units() as f64 { return Ok(BuiltinOutcome::Value(allocate_string(machine, string)?)); } let filler = if args.len() < 2 || args[1] == Value::UNDEFINED { EcmaString::encode(" ") } else { - machine.to_string(args[1])? + machine.coerce_string_observable(args[1])? }; if filler.is_empty() { return Ok(BuiltinOutcome::Value(allocate_string(machine, string)?)); @@ -787,7 +863,10 @@ fn repeat( _: bool, ) -> Result { let string = text(machine, this)?; - let count = to_integer_or_infinity(machine, args.first().copied().unwrap_or(Value::UNDEFINED))?; + let count = super::array_es2023::integer_or_infinity( + machine, + args.first().copied().unwrap_or(Value::UNDEFINED), + )?; if count < 0.0 || count.is_infinite() { return Err(range_error("Invalid count value")); } @@ -829,7 +908,7 @@ fn concat( let mut output = EcmaStringBuilder::new(); append(&mut output, &text(machine, this)?); for argument in args { - append(&mut output, &machine.to_string(*argument)?); + append(&mut output, &machine.coerce_string_observable(*argument)?); } Ok(BuiltinOutcome::Value(allocate_string( machine, @@ -843,7 +922,7 @@ fn normalize( _: bool, ) -> Result { if let Some(form) = args.first().filter(|value| **value != Value::UNDEFINED) { - let form = machine.to_string(*form)?; + let form = machine.coerce_string_observable(*form)?; if !form.eq_ascii("NFC") && !form.eq_ascii("NFD") && !form.eq_ascii("NFKC") @@ -854,10 +933,8 @@ fn normalize( )); } } - Ok(BuiltinOutcome::Value(allocate_string( - machine, - text(machine, this)?, - )?)) + let string = text(machine, this)?; + Ok(BuiltinOutcome::Value(allocate_string(machine, string)?)) } fn uri_unescaped(byte: u8) -> bool { byte.is_ascii_alphanumeric() || b"-_.!~*'()".contains(&byte) @@ -960,7 +1037,8 @@ pub(super) fn decode_uri_component( args: &[Value], _: bool, ) -> Result { - let source = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let source = + machine.coerce_string_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?; let units = source.as_units(); let mut output = EcmaStringBuilder::with_capacity(units.len()); let mut offset = 0; @@ -1030,116 +1108,22 @@ fn string_iterator( Ok(BuiltinOutcome::Value(super::collections::iterator( machine, source, - IterationKind::Value, + IterationKind::StringCodePoint, )?)) } -fn regexp_for_argument( - machine: &mut Machine<'_, H>, - value: Value, -) -> Result<(crate::intrinsics::regexp::Regex, Option), EvalFailure> { - if let Some((pattern, flags)) = super::regexp::regexp_parts(machine, value) { - Ok(( - super::regexp::compile(machine, &pattern, &flags)?, - Some(value), - )) - } else { - let pattern = machine.to_string(value)?; - Ok(( - super::regexp::compile(machine, &pattern, &EcmaString::default())?, - None, - )) - } -} - -/// Run a compiled regex, mapping step-budget exhaustion to a runtime error -/// instead of a silent non-match. Compile errors are impossible here (the -/// regex was already compiled by `regexp_for_argument` or `compile`). -fn exec_regex( - regex: &crate::intrinsics::regexp::Regex, - input: &EcmaString, - start: usize, -) -> Result, EvalFailure> { - regex - .exec(input, start) - .map_err(|error| match error.kind() { - crate::intrinsics::regexp::RegexErrorKind::BudgetExhausted => { - EvalFailure::Runtime(crate::RuntimeErrorKind::RegexpStepBudgetExceeded { - limit: crate::intrinsics::regexp::STEP_BUDGET, - }) - } - crate::intrinsics::regexp::RegexErrorKind::Compile => { - unreachable!("regex already compiled successfully") - } - }) -} - fn string_match( machine: &mut Machine<'_, H>, this: Value, args: &[Value], _constructing: bool, ) -> Result { - let input = text(machine, this)?; - let argument = args.first().copied().unwrap_or(Value::UNDEFINED); - let (regex, object) = regexp_for_argument(machine, argument)?; - if !regex.flags().global { - if let Some(regexp) = object { - return Ok(BuiltinOutcome::Value(super::regexp::regexp_exec( - machine, regexp, &input, - )?)); - } - let matched = exec_regex(®ex, &input, 0)?; - return match matched { - Some(matched) => Ok(BuiltinOutcome::Value(super::regexp::match_array_for( - machine, None, &input, matched, - )?)), - None => Ok(BuiltinOutcome::Value(Value::NULL)), - }; - } - if let Some(regexp) = object { - machine.set_data_property(regexp, "lastIndex", Value::int32(0))?; - } - let matches = collect_matches(®ex, &input)?; - if matches.is_empty() { - return Ok(BuiltinOutcome::Value(Value::NULL)); - } - let mut values = Vec::with_capacity(matches.len()); - for matched in matches { - values.push(allocate_string( - machine, - super::regexp::slice_units(&input, matched.range), - )?); - } - Ok(BuiltinOutcome::Value(allocate_array(machine, values)?)) -} - -fn match_all( - machine: &mut Machine<'_, H>, - this: Value, - args: &[Value], - _constructing: bool, -) -> Result { - let input = text(machine, this)?; - let argument = args.first().copied().unwrap_or(Value::UNDEFINED); - let (regex, object) = regexp_for_argument(machine, argument)?; - if object.is_some() && !regex.flags().global { - return Err(type_error( - "String.prototype.matchAll requires a global RegExp", - )); - } - let mut values = Vec::new(); - for matched in collect_matches(®ex, &input)? { - values.push(super::regexp::match_array_for( - machine, object, &input, matched, - )?); - } - let source = allocate_array(machine, values)?; - Ok(BuiltinOutcome::Value(super::collections::iterator( + delegate_to_pattern( machine, - source, - IterationKind::Value, - )?)) + this, + args, + machine.intrinsics.builtins.symbol_match(), + ) } fn search( @@ -1148,320 +1132,68 @@ fn search( args: &[Value], _constructing: bool, ) -> Result { - let input = text(machine, this)?; - let argument = args.first().copied().unwrap_or(Value::UNDEFINED); - let (regex, _) = regexp_for_argument(machine, argument)?; - Ok(BuiltinOutcome::Value(crate::number_value( - exec_regex(®ex, &input, 0)?.map_or(-1.0, |matched| matched.range.start as f64), - ))) -} - -fn collect_matches( - regex: &crate::intrinsics::regexp::Regex, - input: &EcmaString, -) -> Result, EvalFailure> { - let mut matches = Vec::new(); - let mut start = 0; - let length = input.len_units(); - while start <= length { - let Some(matched) = exec_regex(regex, input, start)? else { - break; - }; - let next = if matched.range.end == matched.range.start && matched.range.end < length { - matched.range.end - + crate::intrinsics::regexp::next_code_point( - input.as_units(), - matched.range.end, - regex.flags().unicode, - ) - .1 - } else { - matched.range.end + usize::from(matched.range.end == matched.range.start) - }; - matches.push(matched); - if !regex.flags().global || next > length { - break; - } - start = next; - } - Ok(matches) + delegate_to_pattern( + machine, + this, + args, + machine.intrinsics.builtins.symbol_search(), + ) } -fn split_regexp( +/// The shared shape of `match` and `search`: use the argument's own method, or +/// build a RegExp from the argument and use that one. +fn delegate_to_pattern( machine: &mut Machine<'_, H>, this: Value, args: &[Value], + symbol: Value, ) -> Result { - let input = text(machine, this)?; - let separator = args[0]; - let (pattern, flags) = - super::regexp::regexp_parts(machine, separator).expect("caller checked RegExp argument"); - let stickyless_flags = EcmaString::from_units( - &flags - .as_units() - .iter() - .copied() - .filter(|unit| *unit != u16::from(b'y')) - .collect::>(), - ); - let regex = super::regexp::compile(machine, &pattern, &stickyless_flags)?; - let limit = value_number( - machine.to_number( - args.get(1) - .copied() - .unwrap_or(crate::number_value(u32::MAX as f64)), - )?, - ) as u32 as usize; - let mut pieces = Vec::new(); - // ECMA-262 §22.2.6.14 keeps two cursors: `p` (start of the pending - // piece) and `q` (search position). An empty match advances `q` only — - // it never pushes and never touches `p` — so the skipped text is not - // dropped. Collapsing both into one `cursor` made an empty match - // indistinguishable from a consumed one and silently deleted characters. - let mut piece_start = 0; - let mut cursor = 0; - let length = input.len_units(); - while cursor < length && pieces.len() < limit { - let Some(matched) = exec_regex(®ex, &input, cursor)? else { - break; - }; - if matched.range.start == matched.range.end && matched.range.start == piece_start { - // Empty match at the pending piece start: advance the search - // cursor only, exactly as the spec moves `q` past `p`. Push - // nothing and leave `piece_start` untouched. - cursor = matched.range.end - + crate::intrinsics::regexp::next_code_point( - input.as_units(), - matched.range.end, - regex.flags().unicode, - ) - .1; - continue; - } - pieces.push(super::regexp::slice_units( - &input, - piece_start..matched.range.start, - )); - for capture in matched.captures.iter().skip(1) { - if pieces.len() == limit { - break; - } - pieces.push(capture.clone().map_or_else(EcmaString::default, |range| { - super::regexp::slice_units(&input, range) - })); - } - piece_start = matched.range.end; - cursor = if matched.range.end == matched.range.start && matched.range.end < length { - matched.range.end - + crate::intrinsics::regexp::next_code_point( - input.as_units(), - matched.range.end, - regex.flags().unicode, - ) - .1 - } else { - matched.range.end + usize::from(matched.range.end == matched.range.start) - }; + require_coercible(this)?; + let pattern = args.first().copied().unwrap_or(Value::UNDEFINED); + if let Some(method) = pattern_method(machine, pattern, symbol)? { + return Ok(BuiltinOutcome::Value(machine.call_value( + method, + pattern, + &[this], + )?)); } - if pieces.len() < limit { - pieces.push(super::regexp::slice_units( - &input, - piece_start.min(length)..length, - )); - } - let mut values = Vec::new(); - for piece in pieces.into_iter().take(limit) { - values.push(allocate_string(machine, piece)?); - } - Ok(BuiltinOutcome::Value(allocate_array(machine, values)?)) + let string = text(machine, this)?; + let regexp = super::regexp::regexp_create(machine, pattern, Value::UNDEFINED)?; + let input = allocate_string(machine, string)?; + invoke_on_new_regexp(machine, regexp, symbol, input) } -fn replace_regexp( +fn match_all( machine: &mut Machine<'_, H>, this: Value, args: &[Value], - replace_all_call: bool, + _constructing: bool, ) -> Result { - let input = text(machine, this)?; - let regexp = args[0]; - let (pattern, flags) = - super::regexp::regexp_parts(machine, regexp).expect("caller checked RegExp argument"); - let regex = super::regexp::compile(machine, &pattern, &flags)?; - if replace_all_call && !regex.flags().global { - return Err(type_error( - "String.prototype.replaceAll requires a global RegExp", - )); - } - let replacer = args.get(1).copied().unwrap_or(Value::UNDEFINED); - let matches = collect_matches(®ex, &input)?; - let mut output = EcmaStringBuilder::new(); - let mut cursor = 0; - for matched in matches { - append( - &mut output, - &super::regexp::slice_units(&input, cursor..matched.range.start), - ); - append( - &mut output, - ®exp_replacement(machine, replacer, &input, &matched)?, - ); - cursor = matched.range.end; - if !regex.flags().global { - break; - } - } - append( - &mut output, - &super::regexp::slice_units(&input, cursor..input.len_units()), - ); - Ok(BuiltinOutcome::Value(allocate_string( - machine, - output.finish(), - )?)) -} - -fn regexp_replacement( - machine: &mut Machine<'_, H>, - replacer: Value, - input: &EcmaString, - matched: &crate::intrinsics::regexp::Match, -) -> Result { - let matched_text = super::regexp::slice_units(input, matched.range.clone()); - if machine.is_callable(replacer)? { - let mut arguments = Vec::with_capacity(matched.captures.len() + 3); - for capture in &matched.captures { - arguments.push(match capture { - Some(range) => { - allocate_string(machine, super::regexp::slice_units(input, range.clone()))? - } - None => Value::UNDEFINED, - }); - } - arguments.push(crate::number_value(matched.range.start as f64)); - arguments.push(allocate_string(machine, input.clone())?); - // ECMA-262 §22.1.3.2.1 appends `groups` only when the pattern - // defines named capture metadata. The map retains unmatched groups - // as `None`, so metadata presence does not depend on participation. - if !matched.named.is_empty() { - let groups = machine - .allocate(HeapEntry::Object { - properties: PropertyMap::default(), - prototype: None, - extensible: true, - boxed_primitive: None, - }) - .map_err(EvalFailure::Runtime)?; - for (name, range) in &matched.named { - let value = match range { - Some(range) => { - allocate_string(machine, super::regexp::slice_units(input, range.clone()))? - } - None => Value::UNDEFINED, - }; - machine.set_data_property(groups, name, value)?; - } - arguments.push(groups); + require_coercible(this)?; + let pattern = args.first().copied().unwrap_or(Value::UNDEFINED); + if machine.is_object(pattern) && super::string_edge::is_regexp(machine, pattern)? { + let flags = machine.get_named_property(pattern, "flags")?; + require_coercible(flags)?; + let flags = machine.coerce_string_observable(flags)?; + if !flags.as_units().contains(&u16::from(b'g')) { + return Err(type_error( + "String.prototype.matchAll requires a global RegExp", + )); } - return machine - .call_value(replacer, Value::UNDEFINED, &arguments) - .and_then(|value| machine.to_string(value)); } - let replacement = machine.to_string(replacer)?; - let before = super::regexp::slice_units(input, 0..matched.range.start); - let after = super::regexp::slice_units(input, matched.range.end..input.len_units()); - let mut output = EcmaStringBuilder::new(); - let units = replacement.as_units(); - let mut offset = 0; - while offset < units.len() { - if units[offset] != u16::from(b'$') || offset + 1 == units.len() { - output.push_unit(units[offset]); - offset += 1; - continue; - } - let next = units[offset + 1]; - if next == u16::from(b'$') { - output.push_unit(u16::from(b'$')); - offset += 2; - continue; - } else if next == u16::from(b'&') { - append(&mut output, &matched_text); - offset += 2; - continue; - } else if next == u16::from(b'`') { - append(&mut output, &before); - offset += 2; - continue; - } else if next == u16::from(b'\'') { - append(&mut output, &after); - offset += 2; - continue; - } else if next == u16::from(b'<') { - // ECMA-262 §22.1.3.19.1 GetSubstitution: $ resolves a - // named capture group. If the name is absent or the group did - // not participate, the substitution is the empty string. - if !matched.named.is_empty() - && let Some(close) = units[offset + 2..] - .iter() - .position(|&unit| unit == u16::from(b'>')) - { - let name_units = &units[offset + 2..offset + 2 + close]; - let name = String::from_utf16_lossy(name_units); - if let Some(Some(range)) = matched.named.get(&name) { - append( - &mut output, - &super::regexp::slice_units(input, range.clone()), - ); - } - offset += 2 + close + 1; - continue; - } - // Without named capture metadata, or without a closing `>`, - // `$<` starts a literal sequence. - output.push_unit(units[offset]); - offset += 1; - continue; - } else if (u16::from(b'1')..=u16::from(b'9')).contains(&next) { - // ECMA-262 §22.1.3.19.1: $nn (01–99). When two digits form a - // valid group number, prefer that reading. Otherwise use the - // single-digit group and leave the second digit as a literal. - let one_digit = usize::from(next - u16::from(b'0')); - if let Some(second) = units.get(offset + 2).copied() - && (u16::from(b'0')..=u16::from(b'9')).contains(&second) - { - let two_digit = one_digit * 10 + usize::from(second - u16::from(b'0')); - if two_digit < matched.captures.len() { - if let Some(Some(range)) = matched.captures.get(two_digit) { - append( - &mut output, - &super::regexp::slice_units(input, range.clone()), - ); - } - offset += 3; - continue; - } - } - if one_digit < matched.captures.len() { - if let Some(Some(range)) = matched.captures.get(one_digit) { - append( - &mut output, - &super::regexp::slice_units(input, range.clone()), - ); - } - offset += 2; - continue; - } - // No matching group — emit '$' literally and re-examine the - // digit on the next iteration. - output.push_unit(units[offset]); - offset += 1; - continue; - } else { - output.push_unit(units[offset]); - offset += 1; - continue; - } + let symbol = machine.intrinsics.builtins.symbol_match_all(); + if let Some(method) = pattern_method(machine, pattern, symbol)? { + return Ok(BuiltinOutcome::Value(machine.call_value( + method, + pattern, + &[this], + )?)); } - Ok(output.finish()) + let string = text(machine, this)?; + let global = allocate_string(machine, EcmaString::encode("g"))?; + let regexp = super::regexp::regexp_create(machine, pattern, global)?; + let input = allocate_string(machine, string)?; + invoke_on_new_regexp(machine, regexp, symbol, input) } #[cfg(test)] diff --git a/crates/bamts-runtime/src/builtins/string_edge.rs b/crates/bamts-runtime/src/builtins/string_edge.rs index a86ac90f..b5460243 100644 --- a/crates/bamts-runtime/src/builtins/string_edge.rs +++ b/crates/bamts-runtime/src/builtins/string_edge.rs @@ -349,7 +349,10 @@ fn symbol_key( Ok(PropertyKey::Symbol(index as u32)) } -fn is_regexp(machine: &mut Machine<'_, H>, value: Value) -> Result { +pub(super) fn is_regexp( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { if !machine.is_object(value) { return Ok(false); } @@ -375,7 +378,7 @@ fn replace_all( let search = args.first().copied().unwrap_or(Value::UNDEFINED); let replacement = args.get(1).copied().unwrap_or(Value::UNDEFINED); - if !matches!(search.decode(), Some(Decoded::Undefined | Decoded::Null)) { + if machine.is_object(search) { if is_regexp(machine, search)? { let flags = machine.get_named_property(search, "flags")?; require_coercible(flags)?; @@ -720,7 +723,8 @@ fn repeat( return Err(range_error("Invalid count value")); } if source.is_empty() || count == 0.0 { - return Ok(BuiltinOutcome::Value(allocate_string(machine, source)?)); + let empty = EcmaString::encode(""); + return Ok(BuiltinOutcome::Value(allocate_string(machine, empty)?)); } if count > usize::MAX as f64 { return Err(heap_limit(machine)); @@ -1148,6 +1152,8 @@ mod tests { let huge_empty = repeat(&mut machine, empty, &[crate::number_value(f64::MAX)], false).unwrap(); assert!(result_text(&machine, huge_empty).is_empty()); + let none = repeat(&mut machine, x, &[Value::int32(0)], false).unwrap(); + assert!(result_text(&machine, none).is_empty()); assert!(matches!( repeat( &mut machine, diff --git a/crates/bamts-runtime/src/builtins/structured_clone.rs b/crates/bamts-runtime/src/builtins/structured_clone.rs index 3b58a754..de4c7647 100644 --- a/crates/bamts-runtime/src/builtins/structured_clone.rs +++ b/crates/bamts-runtime/src/builtins/structured_clone.rs @@ -161,6 +161,7 @@ struct HeapCheckpoint { slot_bytes_len: usize, heap_bytes: usize, vacant_count: usize, + temporaries: usize, } impl HeapCheckpoint { fn capture(machine: &Machine<'_, H>) -> Self { @@ -169,9 +170,11 @@ impl HeapCheckpoint { slot_bytes_len: machine.slot_bytes.len(), heap_bytes: machine.heap_bytes, vacant_count: machine.vacant_count, + temporaries: machine.builtin_temporaries.len(), } } fn rollback(self, machine: &mut Machine<'_, H>) { + machine.builtin_temporaries.truncate(self.temporaries); machine.heap.truncate(self.heap_len); machine.slot_bytes.truncate(self.slot_bytes_len); machine.heap_bytes = self.heap_bytes; @@ -439,6 +442,8 @@ fn build_plan( | HeapEntry::AsyncFromSync { .. } | HeapEntry::DisposableStack { .. } | HeapEntry::ProcessEnv { .. } + | HeapEntry::Temporal { .. } + | HeapEntry::Intl { .. } | HeapEntry::Promise { .. } | HeapEntry::Timeout { .. } | HeapEntry::PromiseResolver { .. } diff --git a/crates/bamts-runtime/src/builtins/symbol.rs b/crates/bamts-runtime/src/builtins/symbol.rs index dcef7202..f1bef329 100644 --- a/crates/bamts-runtime/src/builtins/symbol.rs +++ b/crates/bamts-runtime/src/builtins/symbol.rs @@ -4,7 +4,8 @@ use bamts_bytecode::EcmaString; use bamts_native::Value; use super::{ - allocate_string, builtin_property, define_data, heap_index, install_function, type_error, + allocate_string, builtin_property, define_data, define_to_string_tag, heap_index, + install_constructor_function, install_function, type_error, }; use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; use crate::{EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey}; @@ -24,11 +25,22 @@ pub(super) fn install( let async_dispose = symbol(heap, "Symbol.asyncDispose"); let match_symbol = symbol(heap, "Symbol.match"); let replace_symbol = symbol(heap, "Symbol.replace"); + let match_all_symbol = symbol(heap, "Symbol.matchAll"); + let search_symbol = symbol(heap, "Symbol.search"); + let split_symbol = symbol(heap, "Symbol.split"); + let is_concat_spreadable = symbol(heap, "Symbol.isConcatSpreadable"); + let to_primitive = symbol(heap, "Symbol.toPrimitive"); let unscopables = symbol(heap, "Symbol.unscopables"); builtins.set_symbol_iterator(iterator); builtins.set_symbol_async_iterator(async_iterator); builtins.set_symbol_match(match_symbol); builtins.set_symbol_replace(replace_symbol); + builtins.set_symbol_match_all(match_all_symbol); + builtins.set_symbol_search(search_symbol); + builtins.set_symbol_split(split_symbol); + builtins.set_symbol_is_concat_spreadable(is_concat_spreadable); + builtins.set_symbol_has_instance(has_instance); + builtins.set_symbol_to_primitive(to_primitive); builtins.set_symbol_to_string_tag(to_string_tag); builtins.set_symbol_species(species); builtins.set_symbol_dispose(dispose); @@ -36,16 +48,23 @@ pub(super) fn install( builtins.set_symbol_unscopables(unscopables); builtins.set_symbol_prototype(prototype); - let constructor = install_function(heap, builtins, "Symbol", 0, constructor::); + let constructor = install_constructor_function(heap, builtins, "Symbol", 0, constructor::); builtins.set_constructor_prototype(heap, constructor, prototype); let symbol_for = install_function(heap, builtins, "for", 1, symbol_for::); define_native_property(heap, constructor, "for", symbol_for); + let key_for = install_function(heap, builtins, "keyFor", 1, key_for::); + define_native_property(heap, constructor, "keyFor", key_for); for (name, value) in [ ("iterator", iterator), ("asyncIterator", async_iterator), ("hasInstance", has_instance), ("match", match_symbol), ("replace", replace_symbol), + ("matchAll", match_all_symbol), + ("search", search_symbol), + ("split", split_symbol), + ("isConcatSpreadable", is_concat_spreadable), + ("toPrimitive", to_primitive), ("toStringTag", to_string_tag), ("species", species), ("dispose", dispose), @@ -56,11 +75,13 @@ pub(super) fn install( } let to_string = install_function(heap, builtins, "toString", 0, to_string::); - let value_of = install_function(heap, builtins, "valueOf", 0, value_of::); + let value_of_method = install_function(heap, builtins, "valueOf", 0, value_of::); let description = install_function(heap, builtins, "get description", 0, description::); define_data(heap, prototype, "toString", to_string); - define_data(heap, prototype, "valueOf", value_of); - let symbol_tag = allocate_literal_string(heap, "Symbol"); + define_data(heap, prototype, "valueOf", value_of_method); + let to_primitive_method = + install_function(heap, builtins, "[Symbol.toPrimitive]", 1, value_of::); + define_to_string_tag(heap, prototype, to_string_tag, "Symbol"); let HeapEntry::Object { properties, .. } = &mut heap[heap_index(prototype)] else { unreachable!() }; @@ -74,8 +95,13 @@ pub(super) fn install( }, ); properties.insert( - PropertyKey::Symbol(heap_index(to_string_tag) as u32), - builtin_property(symbol_tag), + PropertyKey::Symbol(heap_index(to_primitive) as u32), + Property::Data { + value: to_primitive_method, + writable: false, + enumerable: false, + configurable: true, + }, ); globals.insert(EcmaString::encode("Symbol"), constructor); @@ -90,10 +116,6 @@ fn symbol(heap: &mut Vec, description: &str) -> Value { ) } -fn allocate_literal_string(heap: &mut Vec, text: &str) -> Value { - super::super::push(heap, HeapEntry::String(EcmaString::encode(text))) -} - fn define_native_property(heap: &mut [HeapEntry], object: Value, name: &str, value: Value) { let HeapEntry::NativeFunction { properties, .. } = &mut heap[heap_index(object)] else { unreachable!() @@ -132,7 +154,7 @@ fn constructor( .first() .copied() .filter(|value| *value != Value::UNDEFINED) - .map(|value| machine.to_string(value)) + .map(|value| machine.coerce_string_observable(value)) .transpose()? .unwrap_or_default(); let symbol = machine @@ -147,7 +169,8 @@ fn symbol_for( args: &[Value], _constructing: bool, ) -> Result { - let key = machine.to_string(args.first().copied().unwrap_or(Value::UNDEFINED))?; + let key = + machine.coerce_string_observable(args.first().copied().unwrap_or(Value::UNDEFINED))?; if let Some(existing) = machine.intrinsics.symbol_registry.get(&key).copied() { return Ok(BuiltinOutcome::Value(existing)); } @@ -178,16 +201,51 @@ fn symbol_for( Ok(BuiltinOutcome::Value(symbol)) } +/// `Symbol.keyFor`: a symbol's key is its description only when the global +/// registry holds that very symbol under it. +fn key_for( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let symbol = args.first().copied().unwrap_or(Value::UNDEFINED); + let index = machine.runtime_slot(symbol).map_err(EvalFailure::Runtime)?; + let Some(HeapEntry::Symbol { description }) = index.map(|index| &machine.heap[index]) else { + return Err(type_error("Symbol.keyFor requires a symbol")); + }; + let description = description.clone(); + if machine.intrinsics.symbol_registry.get(&description) == Some(&symbol) { + return Ok(BuiltinOutcome::Value(allocate_string( + machine, + description, + )?)); + } + Ok(BuiltinOutcome::Value(Value::UNDEFINED)) +} + +/// `ThisSymbolValue`: a symbol, or the symbol a Symbol wrapper object holds. +fn this_symbol_value( + machine: &Machine<'_, H>, + value: Value, +) -> Result { + let symbol = machine.unbox_primitive_or_self(value)?; + let index = machine.runtime_slot(symbol).map_err(EvalFailure::Runtime)?; + match index.map(|index| &machine.heap[index]) { + Some(HeapEntry::Symbol { .. }) => Ok(symbol), + _ => Err(type_error("Symbol method called on incompatible receiver")), + } +} + fn symbol_description( machine: &Machine<'_, H>, value: Value, ) -> Result { - let Some(index) = machine.runtime_slot(value).map_err(EvalFailure::Runtime)? else { - return Err(type_error("Symbol method called on incompatible receiver")); - }; - match &machine.heap[index] { - HeapEntry::Symbol { description } => Ok(description.clone()), - _ => Err(type_error("Symbol method called on incompatible receiver")), + let symbol = this_symbol_value(machine, value)?; + let index = machine.runtime_slot(symbol).map_err(EvalFailure::Runtime)?; + match index.map(|index| &machine.heap[index]) { + Some(HeapEntry::Symbol { description }) => Ok(description.clone()), + _ => unreachable!("this_symbol_value returned a symbol"), } } @@ -230,8 +288,7 @@ fn value_of( _args: &[Value], _constructing: bool, ) -> Result { - symbol_description(machine, this)?; - Ok(BuiltinOutcome::Value(this)) + Ok(BuiltinOutcome::Value(this_symbol_value(machine, this)?)) } #[cfg(test)] @@ -242,7 +299,7 @@ mod tests { use crate::PropertyMap; use crate::ThrowOrigin; use crate::intrinsics::{BuiltinDef, BuiltinOutcome, BuiltinTable, native_function}; - use bamts_bytecode::{FunctionId, ModuleId}; + use bamts_bytecode::{FunctionId, ModuleId, UnaryOp}; fn well_known_symbol_builtin(builtins: &BuiltinTable, name: &str) -> Option { match name { @@ -251,6 +308,7 @@ mod tests { "hasInstance" => None, "match" => Some(builtins.symbol_match()), "replace" => Some(builtins.symbol_replace()), + "toPrimitive" => Some(builtins.symbol_to_primitive()), "toStringTag" => Some(builtins.symbol_to_string_tag()), "species" => Some(builtins.symbol_species()), "dispose" => Some(builtins.symbol_dispose()), @@ -260,6 +318,128 @@ mod tests { } } + fn echo_hint( + _: &mut Machine<'_, TestHost>, + _: Value, + args: &[Value], + _: bool, + ) -> Result { + Ok(BuiltinOutcome::Value(args[0])) + } + + fn return_this( + _: &mut Machine<'_, TestHost>, + this: Value, + _: &[Value], + _: bool, + ) -> Result { + Ok(BuiltinOutcome::Value(this)) + } + + fn with_to_primitive(machine: &mut Machine<'_, TestHost>, method: Value) -> Value { + let object = ordinary_object(machine); + let key = machine + .to_property_key(machine.intrinsics.builtins.symbol_to_primitive()) + .unwrap(); + machine.set_data_property_key(object, key, method).unwrap(); + object + } + + #[test] + fn to_primitive_method_receives_the_hint_of_each_coercion() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let id = machine.intrinsics.builtins.register(BuiltinDef { + name: "echo hint", + length: 1, + handler: echo_hint, + }); + let method = native_function(&mut machine.heap, id, "echo hint", 1); + let object = with_to_primitive(&mut machine, method); + + let string = machine.coerce_primitive_observable(object, true).unwrap(); + let number = machine.coerce_primitive_observable(object, false).unwrap(); + let default = machine.coerce_primitive_default(object).unwrap(); + for (value, hint) in [(string, "string"), (number, "number"), (default, "default")] { + assert!( + machine + .string_value(value) + .is_some_and(|text| text.eq_ascii(hint)), + "expected hint {hint}" + ); + } + } + + fn return_seven( + _: &mut Machine<'_, TestHost>, + _: Value, + _: &[Value], + _: bool, + ) -> Result { + Ok(BuiltinOutcome::Value(Value::int32(7))) + } + + #[test] + fn unary_plus_coerces_an_object_through_to_primitive() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let id = machine.intrinsics.builtins.register(BuiltinDef { + name: "return seven", + length: 0, + handler: return_seven, + }); + let method = native_function(&mut machine.heap, id, "return seven", 0); + let object = with_to_primitive(&mut machine, method); + assert_eq!( + machine.eval_unary(UnaryOp::Plus, object).unwrap(), + Value::int32(7) + ); + } + + #[test] + fn to_primitive_rejects_object_results_and_non_callable_methods() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let id = machine.intrinsics.builtins.register(BuiltinDef { + name: "return this", + length: 0, + handler: return_this, + }); + let method = native_function(&mut machine.heap, id, "return this", 0); + for method in [method, Value::int32(1)] { + let object = with_to_primitive(&mut machine, method); + assert!(matches!( + machine.coerce_primitive_observable(object, false), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + } + + #[test] + fn a_null_to_primitive_method_falls_back_to_the_ordinary_methods() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let object = with_to_primitive(&mut machine, Value::NULL); + let primitive = machine.coerce_primitive_observable(object, true).unwrap(); + assert!( + machine + .string_value(primitive) + .is_some_and(|text| text.eq_ascii("[object Object]")) + ); + } + + #[test] + fn symbols_are_primitives_not_objects() { + let module = blank_program(""); + let mut host = TestHost; + let machine = Machine::new(&module, &mut host, Limits::default()); + assert!(!machine.is_object(machine.intrinsics.builtins.symbol_iterator())); + } + #[test] fn well_known_symbols_are_installed_with_expected_descriptors() { let module = blank_program(""); @@ -274,6 +454,11 @@ mod tests { ("iterator", "Symbol.iterator"), ("asyncIterator", "Symbol.asyncIterator"), ("hasInstance", "Symbol.hasInstance"), + ("isConcatSpreadable", "Symbol.isConcatSpreadable"), + ("matchAll", "Symbol.matchAll"), + ("search", "Symbol.search"), + ("split", "Symbol.split"), + ("toPrimitive", "Symbol.toPrimitive"), ("toStringTag", "Symbol.toStringTag"), ("species", "Symbol.species"), ("dispose", "Symbol.dispose"), @@ -327,6 +512,99 @@ mod tests { } } + #[test] + fn symbol_has_construct_but_new_throws_from_the_function_body() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let symbol = machine.intrinsics.global("Symbol").unwrap(); + assert!( + machine.is_constructor(symbol).unwrap(), + "an extends clause needs Symbol to have [[Construct]]" + ); + assert!(matches!( + machine.construct_value(symbol, &[]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "Symbol is not a constructor" + })) + )); + let result = machine.call_value(symbol, Value::UNDEFINED, &[]).unwrap(); + assert!( + !machine.is_object(result), + "a plain call still returns a primitive symbol" + ); + } + + #[test] + fn key_for_answers_only_for_the_registered_symbol() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let symbol_constructor = machine.intrinsics.global("Symbol").unwrap(); + let symbol_for = machine + .get_named_property(symbol_constructor, "for") + .unwrap(); + let key_for = machine + .get_named_property(symbol_constructor, "keyFor") + .unwrap(); + let key = machine + .allocate(HeapEntry::String(EcmaString::encode("shared"))) + .unwrap(); + let registered = machine + .call_value(symbol_for, symbol_constructor, &[key]) + .unwrap(); + let unregistered = machine + .call_value(symbol_constructor, Value::UNDEFINED, &[key]) + .unwrap(); + + let found = machine + .call_value(key_for, symbol_constructor, &[registered]) + .unwrap(); + assert!( + machine + .string_value(found) + .is_some_and(|text| text.eq_ascii("shared")) + ); + assert_eq!( + machine + .call_value(key_for, symbol_constructor, &[unregistered]) + .unwrap(), + Value::UNDEFINED, + "a same-named symbol outside the registry has no key" + ); + assert!(matches!( + machine.call_value(key_for, symbol_constructor, &[key]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + + #[test] + fn symbol_prototype_methods_accept_wrapper_objects() { + let module = blank_program(""); + let mut host = TestHost; + let mut machine = Machine::new(&module, &mut host, Limits::default()); + let symbol = machine.intrinsics.builtins.symbol_iterator(); + let wrapper = machine.value_to_object(symbol).unwrap(); + + let key = machine + .to_property_key(machine.intrinsics.builtins.symbol_to_primitive()) + .unwrap(); + let to_primitive = machine.get_property_key(wrapper, &key).unwrap(); + assert_eq!( + machine.call_value(to_primitive, wrapper, &[]).unwrap(), + symbol + ); + assert_eq!( + machine.coerce_primitive_default(wrapper).unwrap(), + symbol, + "ToPrimitive reaches the symbol through the prototype method" + ); + assert!(matches!( + machine.call_value(to_primitive, Value::int32(1), &[]), + Err(EvalFailure::Throw(ThrowOrigin::TypeError { .. })) + )); + } + #[test] fn failed_symbol_for_charge_releases_symbol_and_leaves_budget_unchanged() { let module = blank_program(""); @@ -480,7 +758,7 @@ mod tests { .unwrap(); let key = has_instance_property_key(&mut machine); machine - .set_data_property_key(constructor, key, handler) + .create_data_property_key(constructor, key, handler) .unwrap(); assert!( @@ -551,7 +829,7 @@ mod tests { .unwrap(); let key = has_instance_property_key(&mut machine); machine - .set_data_property_key(constructor, key, handler) + .create_data_property_key(constructor, key, handler) .unwrap(); assert!( @@ -572,7 +850,7 @@ mod tests { // A present but non-callable @@hasInstance is a TypeError per GetMethod. let key = has_instance_property_key(&mut machine); machine - .set_data_property_key(constructor, key, Value::int32(42)) + .create_data_property_key(constructor, key, Value::int32(42)) .unwrap(); let value = ordinary_object(&mut machine); diff --git a/crates/bamts-runtime/src/builtins/temporal/duration.rs b/crates/bamts-runtime/src/builtins/temporal/duration.rs new file mode 100644 index 00000000..e1ebd117 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/duration.rs @@ -0,0 +1,535 @@ +//! `Temporal.Duration`. + +use std::cmp::Ordering; + +use bamts_native::Value; + +use super::super::intl::duration_to_locale_string; +use super::fields::{ + RelativeTo, providers, relative_to_option, to_integer_if_integral, to_partial_duration, + to_temporal_duration, +}; +use super::options::{ + ALL_UNITS, TIME_UNITS, UnitOption, fractional_second_digits_option, get_options_object, + rounding_increment_option, rounding_mode_option, seconds_precision, unit_argument, + unit_valued_option, validate_unit_value, +}; +use super::{ + Class, OrThrow, arg, construct, create, install_class, number_outcome, range_error, + require_new, string_outcome, temporal_record, type_error, value_of_throws, +}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::instant_duration::{ + Duration, Overflow, RoundingMode, Unit, validate_rounding_increment, +}; +use crate::temporal::relative::{ + DifferenceRounding, InternalDuration, add_time_duration, compare_with_days, date_duration_days, + difference_plain_date_time_with_rounding, difference_plain_date_time_with_total, + difference_zoned_date_time_with_rounding, difference_zoned_date_time_with_total, + plain_relative_endpoints, round_time_duration, temporal_duration_from_internal, +}; +use crate::temporal::zoned_date_time::add_zoned_date_time; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: TemporalKind::Duration, + name: "Duration", + tag: "Temporal.Duration", + length: 0, + constructor: constructor::, + statics: &[("from", 1, from::), ("compare", 2, compare::)], + getters: &[ + ("years", "get years", years::), + ("months", "get months", months::), + ("weeks", "get weeks", weeks::), + ("days", "get days", days::), + ("hours", "get hours", hours::), + ("minutes", "get minutes", minutes::), + ("seconds", "get seconds", seconds::), + ("milliseconds", "get milliseconds", milliseconds::), + ("microseconds", "get microseconds", microseconds::), + ("nanoseconds", "get nanoseconds", nanoseconds::), + ("sign", "get sign", sign::), + ("blank", "get blank", blank::), + ], + methods: &[ + ("with", 1, with::), + ("negated", 0, negated::), + ("abs", 0, abs::), + ("add", 1, add::), + ("subtract", 1, subtract::), + ("round", 1, round::), + ("total", 1, total::), + ("toString", 0, to_string::), + ("toJSON", 0, to_json::), + ("toLocaleString", 0, to_locale_string::), + ("valueOf", 0, value_of_throws::), + ], + }, + ); +} + +fn this_duration( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result { + match temporal_record(machine, this)? { + Some(TemporalObject::Duration(value)) => Ok(value), + _ => Err(type_error(machine, "receiver is not a Temporal.Duration")), + } +} + +fn duration_outcome( + machine: &mut Machine<'_, H>, + value: Duration, +) -> Result { + create(machine, TemporalObject::Duration(value)).map(BuiltinOutcome::Value) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "Duration")?; + let mut fields = [0_i128; 10]; + for (index, field) in fields.iter_mut().enumerate() { + let value = arg(args, index); + if value != Value::UNDEFINED { + *field = to_integer_if_integral(machine, value)?; + } + } + let value = Duration::new(fields).or_throw(machine)?; + construct(machine, TemporalObject::Duration(value)) +} + +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = to_temporal_duration(machine, arg(args, 0))?; + duration_outcome(machine, value) +} + +fn compare( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let one = to_temporal_duration(machine, arg(args, 0))?; + let two = to_temporal_duration(machine, arg(args, 1))?; + let options = get_options_object(machine, arg(args, 2))?; + let relative_to = relative_to_option(machine, options)?; + if one == two { + return number_outcome(0.0); + } + let largest_one = one.default_largest_unit(); + let largest_two = two.default_largest_unit(); + let internal_one = InternalDuration::from_duration(&one); + let internal_two = InternalDuration::from_duration(&two); + if let RelativeTo::Zoned(zoned) = &relative_to + && (largest_one <= Unit::Day || largest_two <= Unit::Day) + { + let core = providers(); + let after = |internal| { + add_zoned_date_time( + &core, + zoned.to_instant(), + zoned.time_zone_id(), + zoned.calendar_id(), + internal, + Overflow::Constrain, + ) + }; + let after_one = after(internal_one).or_throw(machine)?; + let after_two = after(internal_two).or_throw(machine)?; + return ordering_outcome(after_one.cmp(&after_two)); + } + let (days_one, days_two) = if largest_one < Unit::Day || largest_two < Unit::Day { + let RelativeTo::Plain(date, calendar) = &relative_to else { + return Err(range_error( + machine, + "comparing calendar units requires relativeTo", + )); + }; + let core = providers(); + ( + date_duration_days(&core, &internal_one.date, *date, calendar).or_throw(machine)?, + date_duration_days(&core, &internal_two.date, *date, calendar).or_throw(machine)?, + ) + } else { + (one.days, two.days) + }; + let ordering = + compare_with_days(&internal_one, days_one, &internal_two, days_two).or_throw(machine)?; + ordering_outcome(ordering) +} + +fn ordering_outcome(ordering: Ordering) -> Result { + number_outcome(match ordering { + Ordering::Less => -1.0, + Ordering::Equal => 0.0, + Ordering::Greater => 1.0, + }) +} + +macro_rules! field_getter { + ($($name:ident),+) => { + $(fn $name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + number_outcome(this_duration(machine, this)?.$name as f64) + })+ + }; +} + +field_getter!( + years, + months, + weeks, + days, + hours, + minutes, + seconds, + milliseconds, + microseconds, + nanoseconds +); + +fn sign( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + number_outcome(this_duration(machine, this)?.sign() as f64) +} + +fn blank( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = this_duration(machine, this)?; + Ok(BuiltinOutcome::Value(Value::boolean(value.is_zero()))) +} + +fn with( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = this_duration(machine, this)?; + let partial = to_partial_duration(machine, arg(args, 0))?; + let mut fields = value.fields(); + for (field, replacement) in fields.iter_mut().zip(partial) { + if let Some(replacement) = replacement { + *field = replacement; + } + } + let result = Duration::new(fields).or_throw(machine)?; + duration_outcome(machine, result) +} + +fn negated( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = this_duration(machine, this)?.negated(); + duration_outcome(machine, value) +} + +fn abs( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = this_duration(machine, this)?.abs(); + duration_outcome(machine, value) +} + +/// AddDurations: only durations of time and days can be summed without a +/// starting point, and days count as 24 hours. +fn add_durations( + machine: &mut Machine<'_, H>, + one: Duration, + other: Value, + subtract: bool, +) -> Result { + let mut two = to_temporal_duration(machine, other)?; + if subtract { + two = two.negated(); + } + let largest_unit = one + .default_largest_unit() + .larger(two.default_largest_unit()); + if largest_unit < Unit::Day { + return Err(range_error( + machine, + "adding calendar units requires a starting point", + )); + } + let time = add_time_duration( + one.total_nanoseconds_with_24_hour_days(), + two.total_nanoseconds_with_24_hour_days(), + ) + .or_throw(machine)?; + let result = temporal_duration_from_internal( + InternalDuration::new(Duration::default(), time), + largest_unit, + ) + .or_throw(machine)?; + duration_outcome(machine, result) +} + +fn add( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = this_duration(machine, this)?; + add_durations(machine, value, arg(args, 0), false) +} + +fn subtract( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = this_duration(machine, this)?; + add_durations(machine, value, arg(args, 0), true) +} + +fn round( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let duration = this_duration(machine, this)?; + let round_to = unit_argument(machine, arg(args, 0))?; + let options = round_to.options; + let largest = unit_valued_option(machine, options, "largestUnit")?; + let relative_to = relative_to_option(machine, options)?; + let increment = rounding_increment_option(machine, options)?; + let mode = rounding_mode_option(machine, options, RoundingMode::HalfExpand)?; + let smallest = round_to.unit(machine, "smallestUnit")?; + validate_unit_value(machine, largest, ALL_UNITS, Some(UnitOption::Auto))?; + validate_unit_value(machine, smallest, ALL_UNITS, None)?; + let smallest_present = smallest != UnitOption::Unset; + let smallest_unit = match smallest { + UnitOption::Unit(unit) => unit, + _ => Unit::Nanosecond, + }; + let existing_largest = duration.default_largest_unit(); + let default_largest = existing_largest.larger(smallest_unit); + let largest_present = largest != UnitOption::Unset; + let largest_unit = match largest { + UnitOption::Unit(unit) => unit, + _ => default_largest, + }; + if !smallest_present && !largest_present { + return Err(range_error( + machine, + "at least one of smallestUnit or largestUnit is required", + )); + } + if largest_unit.larger(smallest_unit) != largest_unit { + return Err(range_error( + machine, + "smallestUnit must not be larger than largestUnit", + )); + } + if smallest_unit.max_rounding_increment().is_some() { + validate_rounding_increment(smallest_unit, increment, false).or_throw(machine)?; + } + if increment > 1 && largest_unit != smallest_unit && smallest_unit <= Unit::Day { + return Err(range_error( + machine, + "roundingIncrement above 1 requires smallestUnit to equal largestUnit \ + for a date unit", + )); + } + let rounding = DifferenceRounding { + largest_unit, + smallest_unit, + increment, + mode, + }; + let core = providers(); + let result = match relative_to { + RelativeTo::Zoned(zoned) => { + let origin = zoned.epoch_nanoseconds(); + let target = add_zoned_date_time( + &core, + zoned.to_instant(), + zoned.time_zone_id(), + zoned.calendar_id(), + InternalDuration::from_duration(&duration), + Overflow::Constrain, + ) + .or_throw(machine)?; + let internal = difference_zoned_date_time_with_rounding( + &core, + origin, + target.epoch_nanoseconds(), + zoned.time_zone_id(), + zoned.calendar_id(), + rounding, + ) + .or_throw(machine)?; + let balance_to = if largest_unit <= Unit::Day { + Unit::Hour + } else { + largest_unit + }; + temporal_duration_from_internal(internal, balance_to) + } + RelativeTo::Plain(date, calendar) => { + let (start, target) = + plain_relative_endpoints(&core, date, &calendar, &duration).or_throw(machine)?; + let internal = + difference_plain_date_time_with_rounding(&core, start, target, &calendar, rounding) + .or_throw(machine)?; + temporal_duration_from_internal(internal, largest_unit) + } + RelativeTo::None => { + if existing_largest < Unit::Day || largest_unit < Unit::Day { + return Err(range_error( + machine, + "rounding calendar units requires relativeTo", + )); + } + duration.round(smallest_unit, largest_unit, increment, mode) + } + } + .or_throw(machine)?; + duration_outcome(machine, result) +} + +fn total( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let duration = this_duration(machine, this)?; + let total_of = unit_argument(machine, arg(args, 0))?; + let relative_to = relative_to_option(machine, total_of.options)?; + let unit = total_of.required_unit(machine, "unit")?; + validate_unit_value(machine, unit, ALL_UNITS, None)?; + let UnitOption::Unit(unit) = unit else { + return Err(range_error(machine, "unit is required")); + }; + let core = providers(); + let result = match relative_to { + RelativeTo::Zoned(zoned) => { + let origin = zoned.epoch_nanoseconds(); + let target = add_zoned_date_time( + &core, + zoned.to_instant(), + zoned.time_zone_id(), + zoned.calendar_id(), + InternalDuration::from_duration(&duration), + Overflow::Constrain, + ) + .or_throw(machine)?; + difference_zoned_date_time_with_total( + &core, + origin, + target.epoch_nanoseconds(), + zoned.time_zone_id(), + zoned.calendar_id(), + unit, + ) + } + RelativeTo::Plain(date, calendar) => { + let (start, target) = + plain_relative_endpoints(&core, date, &calendar, &duration).or_throw(machine)?; + difference_plain_date_time_with_total(&core, start, target, &calendar, unit) + } + RelativeTo::None => duration.total_as_f64(unit), + } + .or_throw(machine)?; + number_outcome(result) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let duration = this_duration(machine, this)?; + let options = get_options_object(machine, arg(args, 0))?; + let digits = fractional_second_digits_option(machine, options)?; + let mode = rounding_mode_option(machine, options, RoundingMode::Trunc)?; + let smallest = unit_valued_option(machine, options, "smallestUnit")?; + validate_unit_value(machine, smallest, TIME_UNITS, None)?; + if matches!(smallest, UnitOption::Unit(Unit::Hour | Unit::Minute)) { + return Err(range_error( + machine, + "smallestUnit must be second or smaller", + )); + } + let precision = seconds_precision(machine, smallest, digits)?; + if precision.unit == Unit::Nanosecond && precision.increment == 1 { + return string_outcome( + machine, + &duration.format_with_precision(precision.precision), + ); + } + let internal = InternalDuration::from_duration(&duration); + let time = round_time_duration(internal.time, precision.increment, precision.unit, mode) + .or_throw(machine)?; + let rounded = temporal_duration_from_internal( + InternalDuration::new(internal.date, time), + duration.default_largest_unit().larger(Unit::Second), + ) + .or_throw(machine)?; + string_outcome(machine, &rounded.format_with_precision(precision.precision)) +} + +fn to_json( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let text = this_duration(machine, this)?.format(); + string_outcome(machine, &text) +} + +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let duration = this_duration(machine, this)?; + duration_to_locale_string(machine, &duration, arg(args, 0), arg(args, 1)) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/fields.rs b/crates/bamts-runtime/src/builtins/temporal/fields.rs new file mode 100644 index 00000000..3d8cf5f2 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/fields.rs @@ -0,0 +1,1228 @@ +//! Conversions from JavaScript values to Temporal records: the `To*` abstract +//! operations and the calendar field protocol. Property reads and their +//! conversions happen here, in the sorted order test262 observes. + +use bamts_native::Value; + +use super::super::value_number; +use super::options::{ + Options, disambiguation_option, get_options_object, offset_option, overflow_option, +}; +use super::{OrThrow, is_object, range_error, temporal_record, type_error}; +use crate::temporal::TemporalObject; +use crate::temporal::calendar::{self, Calendar, Calendars, MonthCode, MonthField}; +use crate::temporal::instant_duration::{Duration, Instant, Overflow}; +use crate::temporal::parse::{ + UtcOffsetRecord, parse_calendar_string, parse_date_time_string, parse_date_time_utc_offset, + parse_month_day_string, parse_time_zone_string, parse_year_month_string, +}; +use crate::temporal::plain_types::{ + MONTH_DAY_REFERENCE_YEAR, PlainDate, PlainDateTime, PlainMonthDay, PlainTime, PlainYearMonth, + regulate_iso_date, +}; +use crate::temporal::zoned_date_time::{ + CalendarDateFields, CalendarId, Disambiguation, FixedOffsetTimeZones, OffsetMatch, + OffsetOption, ResolveOptions, TemporalProviders, TimeZoneId, UtcOffsetNanoseconds, + ZonedDateTime, start_of_day, +}; +use crate::{EvalFailure, Host, Machine}; + +/// The providers behind every Temporal operation in this runtime. +pub(super) fn providers() -> TemporalProviders<'static> { + TemporalProviders::new(&FixedOffsetTimeZones, &Calendars) +} + +/// CalendarISOToDate. +pub(super) fn calendar_date( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + date: PlainDate, +) -> Result { + providers() + .calendar + .fields(calendar, date) + .or_throw(machine) +} + +fn to_number(machine: &mut Machine<'_, H>, value: Value) -> Result { + Ok(value_number(machine.coerce_number_observable(value)?)) +} + +/// ToIntegerWithTruncation. +pub(super) fn to_integer_with_truncation( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let number = to_number(machine, value)?; + if !number.is_finite() { + return Err(range_error(machine, "value must be a finite number")); + } + Ok(number.trunc() + 0.0) +} + +/// ToPositiveIntegerWithTruncation. +fn to_positive_integer_with_truncation( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let integer = to_integer_with_truncation(machine, value)?; + if integer <= 0.0 { + return Err(range_error(machine, "value must be a positive integer")); + } + Ok(integer) +} + +/// ToIntegerIfIntegral. A magnitude past every valid Duration field clamps, +/// staying invalid for the range check that runs after all fields are read. +pub(super) fn to_integer_if_integral( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let number = to_number(machine, value)?; + if !number.is_finite() || number.trunc() != number { + return Err(range_error(machine, "value must be an integer")); + } + Ok(number.clamp(-1e30, 1e30) as i128) +} + +/// A string argument's text, or the `TypeError` for anything else. +fn require_string( + machine: &mut Machine<'_, H>, + value: Value, + what: &str, +) -> Result { + match machine.string_value(value) { + Some(text) => Ok(text.to_utf8_lossy()), + None => Err(type_error(machine, format!("{what} must be a string"))), + } +} + +/// CanonicalizeCalendar of a parsed annotation, ISO when there is none. +fn parsed_calendar( + machine: &mut Machine<'_, H>, + parsed: &crate::temporal::parse::ParsedIsoDateTime, +) -> Result { + parsed.calendar_id().or_throw(machine) +} + +/// The `calendar` constructor argument: a string naming a calendar exactly. +pub(super) fn calendar_argument( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + if value == Value::UNDEFINED { + return Ok(CalendarId::iso8601()); + } + let text = require_string(machine, value, "calendar")?; + CalendarId::canonicalize(&text).or_throw(machine) +} + +/// ToTemporalCalendarIdentifier. +pub(super) fn to_calendar_identifier( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + if let Some(record) = temporal_record(machine, value)? + && let Some(calendar) = record.calendar() + { + return Ok(calendar); + } + let text = require_string(machine, value, "calendar")?; + let identifier = parse_calendar_string(&text).or_throw(machine)?; + CalendarId::canonicalize(&identifier).or_throw(machine) +} + +/// GetTemporalCalendarIdentifierWithISODefault. +fn calendar_with_iso_default( + machine: &mut Machine<'_, H>, + item: Value, +) -> Result { + let calendar = machine.get_named_property(item, "calendar")?; + if calendar == Value::UNDEFINED { + Ok(CalendarId::iso8601()) + } else { + to_calendar_identifier(machine, calendar) + } +} + +/// ToTemporalTimeZoneIdentifier. +pub(super) fn to_time_zone_identifier( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + if let Some(TemporalObject::ZonedDateTime(zoned)) = temporal_record(machine, value)? { + return Ok(zoned.time_zone_id().clone()); + } + let text = require_string(machine, value, "time zone")?; + let parsed = parse_time_zone_string(&text).or_throw(machine)?; + TimeZoneId::from_parsed(parsed).or_throw(machine) +} + +/// A set of calendar fields, named as `PrepareCalendarFields` lists them. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub(super) struct FieldNames(u16); + +impl FieldNames { + pub(super) const NONE: Self = Self(0); + pub(super) const DAY: Self = Self(1 << 0); + const HOUR: Self = Self(1 << 1); + const MICROSECOND: Self = Self(1 << 2); + const MILLISECOND: Self = Self(1 << 3); + const MINUTE: Self = Self(1 << 4); + const MONTH: Self = Self(1 << 5); + const MONTH_CODE: Self = Self(1 << 6); + const NANOSECOND: Self = Self(1 << 7); + pub(super) const OFFSET: Self = Self(1 << 8); + const SECOND: Self = Self(1 << 9); + pub(super) const TIME_ZONE: Self = Self(1 << 10); + pub(super) const YEAR: Self = Self(1 << 11); + const ERA: Self = Self(1 << 12); + const ERA_YEAR: Self = Self(1 << 13); + + /// «year, month, month-code, day» + pub(super) const DATE: Self = + Self(Self::YEAR.0 | Self::MONTH.0 | Self::MONTH_CODE.0 | Self::DAY.0); + /// «year, month, month-code» + pub(super) const YEAR_MONTH: Self = Self(Self::YEAR.0 | Self::MONTH.0 | Self::MONTH_CODE.0); + /// «hour, minute, second, millisecond, microsecond, nanosecond» + pub(super) const TIME: Self = Self( + Self::HOUR.0 + | Self::MINUTE.0 + | Self::SECOND.0 + | Self::MILLISECOND.0 + | Self::MICROSECOND.0 + | Self::NANOSECOND.0, + ); + + pub(super) const fn with(self, other: Self) -> Self { + Self(self.0 | other.0) + } + + const fn contains(self, other: Self) -> bool { + self.0 & other.0 != 0 + } +} + +/// Calendar Fields Record. Absent fields are `None`; time fields default to +/// zero when read. Numbers stay `f64` until resolution, so an enormous year is +/// rejected there, after every property has been read. +#[derive(Clone, Debug, Default, PartialEq)] +pub(super) struct CalendarFields { + pub(super) era: Option, + pub(super) era_year: Option, + pub(super) year: Option, + pub(super) month: Option, + pub(super) month_code: Option, + pub(super) day: Option, + pub(super) hour: Option, + pub(super) minute: Option, + pub(super) second: Option, + pub(super) millisecond: Option, + pub(super) microsecond: Option, + pub(super) nanosecond: Option, + pub(super) offset: Option, + pub(super) time_zone: Option, +} + +impl CalendarFields { + /// ISODateToFields of type `date`. + pub(super) fn from_date( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + date: PlainDate, + ) -> Result { + let fields = calendar_date(machine, calendar, date)?; + Ok(Self { + year: Some(f64::from(fields.year)), + month_code: Some(fields.month_code.to_string()), + day: Some(f64::from(fields.day)), + ..Self::default() + }) + } + + /// ISODateToFields of type `year-month`. + pub(super) fn from_year_month( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + value: PlainYearMonth, + ) -> Result { + let fields = calendar_date(machine, calendar, value.iso_date())?; + Ok(Self { + year: Some(f64::from(fields.year)), + month_code: Some(fields.month_code.to_string()), + ..Self::default() + }) + } + + /// ISODateToFields of type `month-day`. + pub(super) fn from_month_day( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + value: PlainMonthDay, + ) -> Result { + let fields = calendar_date(machine, calendar, value.iso_date())?; + Ok(Self { + month_code: Some(fields.month_code.to_string()), + day: Some(f64::from(fields.day)), + ..Self::default() + }) + } + + pub(super) fn with_time(mut self, time: PlainTime) -> Self { + self.hour = Some(f64::from(time.hour)); + self.minute = Some(f64::from(time.minute)); + self.second = Some(f64::from(time.second)); + self.millisecond = Some(f64::from(time.millisecond)); + self.microsecond = Some(f64::from(time.microsecond)); + self.nanosecond = Some(f64::from(time.nanosecond)); + self + } + + /// CalendarMergeFields: `month` and `monthCode` replace each other as one + /// field, and so do `year`, `era`, and `eraYear`, so a new value cannot + /// conflict with an inherited one. + pub(super) fn merged_with(mut self, additional: Self) -> Self { + if additional.month.is_some() || additional.month_code.is_some() { + self.month = None; + self.month_code = None; + } + if additional.year.is_some() || additional.era.is_some() || additional.era_year.is_some() { + self.year = None; + self.era = None; + self.era_year = None; + } + macro_rules! merge { + ($($field:ident),+) => { + $(if additional.$field.is_some() { + self.$field = additional.$field; + })+ + }; + } + merge!( + era, + era_year, + year, + month, + month_code, + day, + hour, + minute, + second, + millisecond, + microsecond, + nanosecond, + offset, + time_zone + ); + self + } +} + +/// ToMonthCode. +fn to_month_code( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let primitive = machine.coerce_primitive_observable(value, true)?; + let text = require_string(machine, primitive, "monthCode")?; + let bytes = text.as_bytes(); + let well_formed = matches!(bytes.len(), 3 | 4) + && bytes[0] == b'M' + && bytes[1].is_ascii_digit() + && bytes[2].is_ascii_digit() + && (bytes.len() == 3 || bytes[3] == b'L') + && (bytes.len() == 4 || &text[1..3] != "00"); + if well_formed { + Ok(text) + } else { + Err(range_error(machine, format!("invalid month code {text}"))) + } +} + +/// ToOffsetString, keeping the value it validates. +fn to_offset_nanoseconds( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let primitive = machine.coerce_primitive_observable(value, true)?; + let text = require_string(machine, primitive, "offset")?; + parse_date_time_utc_offset(&text).or_throw(machine) +} + +/// PrepareCalendarFields with a list of required fields, possibly empty. +pub(super) fn prepare_calendar_fields( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + item: Value, + names: FieldNames, + required: FieldNames, +) -> Result { + read_calendar_fields(machine, calendar, item, names, required).map(|(fields, _)| fields) +} + +/// PrepareCalendarFields with `partial`: at least one listed field must exist. +pub(super) fn prepare_partial_fields( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + item: Value, + names: FieldNames, +) -> Result { + let (fields, any) = read_calendar_fields(machine, calendar, item, names, FieldNames::NONE)?; + if any { + Ok(fields) + } else { + Err(type_error(machine, "no recognized properties were present")) + } +} + +/// The ISO calendar's field list has no `era`; every other calendar reads +/// `era` and `eraYear` wherever it reads `year`. Reads go in sorted property-name +/// order and each value converts as soon as it is read, so a throwing `valueOf` +/// stops every later read. Also reports whether any field was present. +fn read_calendar_fields( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + item: Value, + names: FieldNames, + required: FieldNames, +) -> Result<(CalendarFields, bool), EvalFailure> { + let mut fields = CalendarFields::default(); + let mut any = false; + let names = if names.contains(FieldNames::YEAR) && !calendar.is_iso8601() { + names.with(FieldNames::ERA).with(FieldNames::ERA_YEAR) + } else { + names + }; + let table = [ + ("day", FieldNames::DAY), + ("era", FieldNames::ERA), + ("eraYear", FieldNames::ERA_YEAR), + ("hour", FieldNames::HOUR), + ("microsecond", FieldNames::MICROSECOND), + ("millisecond", FieldNames::MILLISECOND), + ("minute", FieldNames::MINUTE), + ("month", FieldNames::MONTH), + ("monthCode", FieldNames::MONTH_CODE), + ("nanosecond", FieldNames::NANOSECOND), + ("offset", FieldNames::OFFSET), + ("second", FieldNames::SECOND), + ("timeZone", FieldNames::TIME_ZONE), + ("year", FieldNames::YEAR), + ]; + for (name, field) in table { + if !names.contains(field) { + continue; + } + let value = machine.get_named_property(item, name)?; + if value == Value::UNDEFINED { + if required.contains(field) { + return Err(type_error(machine, format!("{name} is required"))); + } + continue; + } + any = true; + match field { + FieldNames::DAY => { + fields.day = Some(to_positive_integer_with_truncation(machine, value)?); + } + FieldNames::MONTH => { + fields.month = Some(to_positive_integer_with_truncation(machine, value)?); + } + FieldNames::MONTH_CODE => fields.month_code = Some(to_month_code(machine, value)?), + FieldNames::ERA => { + fields.era = Some(machine.coerce_string_observable(value)?.to_utf8_lossy()); + } + FieldNames::ERA_YEAR => { + fields.era_year = Some(to_integer_with_truncation(machine, value)?); + } + FieldNames::OFFSET => fields.offset = Some(to_offset_nanoseconds(machine, value)?), + FieldNames::TIME_ZONE => { + fields.time_zone = Some(to_time_zone_identifier(machine, value)?); + } + FieldNames::YEAR => fields.year = Some(to_integer_with_truncation(machine, value)?), + FieldNames::HOUR => fields.hour = Some(to_integer_with_truncation(machine, value)?), + FieldNames::MICROSECOND => { + fields.microsecond = Some(to_integer_with_truncation(machine, value)?); + } + FieldNames::MILLISECOND => { + fields.millisecond = Some(to_integer_with_truncation(machine, value)?); + } + FieldNames::MINUTE => fields.minute = Some(to_integer_with_truncation(machine, value)?), + FieldNames::NANOSECOND => { + fields.nanosecond = Some(to_integer_with_truncation(machine, value)?); + } + _ => fields.second = Some(to_integer_with_truncation(machine, value)?), + } + } + Ok((fields, any)) +} + +/// An ISO year as the core stores it. Any year this far out is outside the +/// representable range, whatever overflow says. +pub(super) fn iso_year( + machine: &mut Machine<'_, H>, + year: f64, +) -> Result { + if year.abs() > 1_000_000.0 { + return Err(range_error(machine, "year outside the representable range")); + } + Ok(year as i32) +} + +/// CalendarResolveFields for the ISO calendar's month: `monthCode` must name a +/// real ISO month and agree with `month` when both are given. +fn resolve_month( + machine: &mut Machine<'_, H>, + fields: &CalendarFields, +) -> Result { + let Some(code) = &fields.month_code else { + return match fields.month { + Some(month) => Ok(month), + None => Err(type_error(machine, "month or monthCode is required")), + }; + }; + let number = (code.len() == 3) + .then(|| code[1..].parse::().ok()) + .flatten() + .filter(|month| (1..=12).contains(month)); + let Some(number) = number else { + return Err(range_error( + machine, + format!("{code} is not a month of the ISO calendar"), + )); + }; + let number = f64::from(number); + if fields.month.is_some_and(|month| month != number) { + return Err(range_error(machine, "month and monthCode do not agree")); + } + Ok(number) +} + +/// CalendarResolveFields for a date: the year, month, and day it will regulate. +pub(super) fn resolved_date_fields( + machine: &mut Machine<'_, H>, + fields: &CalendarFields, +) -> Result<(i32, i64, i64), EvalFailure> { + let Some(year) = fields.year else { + return Err(type_error(machine, "year is required")); + }; + let Some(day) = fields.day else { + return Err(type_error(machine, "day is required")); + }; + let month = resolve_month(machine, fields)?; + Ok((iso_year(machine, year)?, month as i64, day as i64)) +} + +/// Which fields a resolution needs. +#[derive(Clone, Copy, Eq, PartialEq)] +enum Shape { + Date, + YearMonth, + MonthDay, +} + +/// CalendarResolveFields outside the ISO calendar. Every `TypeError` for a +/// missing field comes before any `RangeError` for a field's value. Returns the +/// year, if the record names one, the month, and the day, 1 when not needed. +fn resolve_calendar_fields( + machine: &mut Machine<'_, H>, + calendar: Calendar, + fields: &CalendarFields, + shape: Shape, +) -> Result<(Option, MonthField, f64), EvalFailure> { + if fields.era.is_some() != fields.era_year.is_some() { + return Err(type_error(machine, "era and eraYear go together")); + } + if shape != Shape::MonthDay && fields.year.is_none() && fields.era.is_none() { + return Err(type_error(machine, "year is required")); + } + if fields.month.is_none() && fields.month_code.is_none() { + return Err(type_error(machine, "month or monthCode is required")); + } + if shape == Shape::MonthDay + && fields.month.is_some() + && fields.year.is_none() + && fields.era.is_none() + { + return Err(type_error(machine, "month needs a year")); + } + let day = match (fields.day, shape) { + (Some(day), _) => day, + (None, Shape::YearMonth) => 1.0, + (None, _) => return Err(type_error(machine, "day is required")), + }; + let mut year = fields.year; + if let (Some(era), Some(era_year)) = (&fields.era, fields.era_year) { + let Some(from_era) = calendar.year_of_era(era, era_year.clamp(-1e9, 1e9) as i64) else { + return Err(range_error( + machine, + format!("{era} is not an era of the {} calendar", calendar.id()), + )); + }; + let from_era = from_era as f64; + if year.is_some_and(|year| year != from_era) { + return Err(range_error(machine, "year and eraYear do not agree")); + } + year = Some(from_era); + } + let Some(text) = &fields.month_code else { + let month = fields.month.expect("one of month and monthCode is present"); + return Ok((year, MonthField::Ordinal(month), day)); + }; + let code = MonthCode::parse(text).filter(|code| calendar.has_month_code(*code)); + let Some(code) = code else { + return Err(range_error( + machine, + format!("{text} is not a month of the {} calendar", calendar.id()), + )); + }; + if let (Some(year), Some(month)) = (year, fields.month) + && !calendar::month_matches(calendar, year, month, code) + { + return Err(range_error(machine, "month and monthCode do not agree")); + } + Ok((year, MonthField::Code(code), day)) +} + +/// CalendarDateFromFields. +pub(super) fn date_from_fields( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + fields: &CalendarFields, + overflow: Overflow, +) -> Result { + if let Some(kind) = Calendar::from_id(calendar) { + let (year, month, day) = resolve_calendar_fields(machine, kind, fields, Shape::Date)?; + let year = year.expect("a date's year was required"); + return calendar::date_from_fields(kind, year, month, day, overflow).or_throw(machine); + } + let (year, month, day) = resolved_date_fields(machine, fields)?; + PlainDate::new(year, month, day, overflow).or_throw(machine) +} + +/// CalendarYearMonthFromFields. +pub(super) fn year_month_from_fields( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + fields: &CalendarFields, + overflow: Overflow, +) -> Result { + if let Some(kind) = Calendar::from_id(calendar) { + let (year, month, _) = resolve_calendar_fields(machine, kind, fields, Shape::YearMonth)?; + let year = year.expect("a year-month's year was required"); + return calendar::year_month_from_fields(kind, year, month, overflow).or_throw(machine); + } + let Some(year) = fields.year else { + return Err(type_error(machine, "year is required")); + }; + let month = resolve_month(machine, fields)?; + let year = iso_year(machine, year)?; + PlainYearMonth::new(year, month as i64, overflow).or_throw(machine) +} + +/// CalendarMonthDayFromFields. A `year`, when given, only decides which days +/// the month has, and only its leap-ness matters: the result carries the +/// reference year, so no year is too large to consult. +pub(super) fn month_day_from_fields( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + fields: &CalendarFields, + overflow: Overflow, +) -> Result { + if let Some(kind) = Calendar::from_id(calendar) { + let (year, month, day) = resolve_calendar_fields(machine, kind, fields, Shape::MonthDay)?; + return calendar::month_day_from_fields(kind, year, month, day, overflow).or_throw(machine); + } + let Some(day) = fields.day else { + return Err(type_error(machine, "day is required")); + }; + let month = resolve_month(machine, fields)?; + let year = fields.year.map_or(MONTH_DAY_REFERENCE_YEAR, |year| { + year.rem_euclid(400.0) as i32 + 2000 + }); + let (_, month, day) = + regulate_iso_date(year, month as i64, day as i64, overflow).or_throw(machine)?; + PlainMonthDay::new(month.into(), day.into(), Overflow::Reject).or_throw(machine) +} + +/// RegulateTime over a fields record; absent fields are zero. +pub(super) fn time_from_fields( + machine: &mut Machine<'_, H>, + fields: &CalendarFields, + overflow: Overflow, +) -> Result { + let field = |value: Option| value.unwrap_or(0.0) as i64; + PlainTime::new( + field(fields.hour), + field(fields.minute), + field(fields.second), + field(fields.millisecond), + field(fields.microsecond), + field(fields.nanosecond), + overflow, + ) + .or_throw(machine) +} + +/// InterpretTemporalDateTimeFields. The result is a record, with no limit of +/// its own; whoever creates an object from it applies that object's limit. +pub(super) fn date_time_from_fields( + machine: &mut Machine<'_, H>, + calendar: &CalendarId, + fields: &CalendarFields, + overflow: Overflow, +) -> Result { + let date = date_from_fields(machine, calendar, fields, overflow)?; + let time = time_from_fields(machine, fields, overflow)?; + Ok(PlainDateTime { date, time }) +} + +/// IsPartialTemporalObject. +pub(super) fn require_partial_temporal_object( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result<(), EvalFailure> { + if !is_object(machine, value)? { + return Err(type_error(machine, "argument must be an object")); + } + if let Some(record) = temporal_record(machine, value)? + && !matches!( + record, + TemporalObject::Instant(_) | TemporalObject::Duration(_) + ) + { + return Err(type_error( + machine, + "argument must not be a Temporal object", + )); + } + for name in ["calendar", "timeZone"] { + let property = machine.get_named_property(value, name)?; + if property != Value::UNDEFINED { + return Err(type_error( + machine, + format!("argument must not have a {name} property"), + )); + } + } + Ok(()) +} + +/// Reads and validates the options of a `from` whose argument needs no fields, +/// discarding the value: the read itself is the observable part. +fn read_overflow(machine: &mut Machine<'_, H>, options: Value) -> Result<(), EvalFailure> { + let resolved = get_options_object(machine, options)?; + overflow_option(machine, resolved)?; + Ok(()) +} + +/// ToTemporalDate. `options` is `undefined` when the caller has none. +pub(super) fn to_temporal_date( + machine: &mut Machine<'_, H>, + item: Value, + options: Value, +) -> Result<(PlainDate, CalendarId), EvalFailure> { + if is_object(machine, item)? { + match temporal_record(machine, item)? { + Some(TemporalObject::PlainDate { value, calendar }) => { + read_overflow(machine, options)?; + return Ok((value, calendar)); + } + Some(TemporalObject::ZonedDateTime(zoned)) => { + let local = zoned + .to_plain_date_time(&FixedOffsetTimeZones) + .or_throw(machine)?; + read_overflow(machine, options)?; + return Ok((local.date, zoned.calendar_id().clone())); + } + Some(TemporalObject::PlainDateTime { value, calendar }) => { + read_overflow(machine, options)?; + return Ok((value.date, calendar)); + } + _ => {} + } + let calendar = calendar_with_iso_default(machine, item)?; + let fields = + prepare_calendar_fields(machine, &calendar, item, FieldNames::DATE, FieldNames::NONE)?; + let resolved = get_options_object(machine, options)?; + let overflow = overflow_option(machine, resolved)?; + let date = date_from_fields(machine, &calendar, &fields, overflow)?; + return Ok((date, calendar)); + } + let text = require_string(machine, item, "date")?; + let parsed = parse_date_time_string(&text, false).or_throw(machine)?; + let calendar = parsed_calendar(machine, &parsed)?; + read_overflow(machine, options)?; + let date = PlainDate::from_parsed(&parsed).or_throw(machine)?; + Ok((date, calendar)) +} + +/// ToTemporalDateTime. +pub(super) fn to_temporal_date_time( + machine: &mut Machine<'_, H>, + item: Value, + options: Value, +) -> Result<(PlainDateTime, CalendarId), EvalFailure> { + if is_object(machine, item)? { + match temporal_record(machine, item)? { + Some(TemporalObject::PlainDateTime { value, calendar }) => { + read_overflow(machine, options)?; + return Ok((value, calendar)); + } + Some(TemporalObject::ZonedDateTime(zoned)) => { + let local = zoned + .to_plain_date_time(&FixedOffsetTimeZones) + .or_throw(machine)?; + read_overflow(machine, options)?; + return Ok((local, zoned.calendar_id().clone())); + } + Some(TemporalObject::PlainDate { value, calendar }) => { + read_overflow(machine, options)?; + let local = + PlainDateTime::from_parts(value, PlainTime::midnight()).or_throw(machine)?; + return Ok((local, calendar)); + } + _ => {} + } + let calendar = calendar_with_iso_default(machine, item)?; + let names = FieldNames::DATE.with(FieldNames::TIME); + let fields = prepare_calendar_fields(machine, &calendar, item, names, FieldNames::NONE)?; + let resolved = get_options_object(machine, options)?; + let overflow = overflow_option(machine, resolved)?; + let value = date_time_from_fields(machine, &calendar, &fields, overflow)?; + let value = PlainDateTime::from_parts(value.date, value.time).or_throw(machine)?; + return Ok((value, calendar)); + } + let text = require_string(machine, item, "date-time")?; + let parsed = parse_date_time_string(&text, false).or_throw(machine)?; + let calendar = parsed_calendar(machine, &parsed)?; + read_overflow(machine, options)?; + let value = PlainDateTime::from_parsed(&parsed).or_throw(machine)?; + Ok((value, calendar)) +} + +/// ToTemporalTime. +pub(super) fn to_temporal_time( + machine: &mut Machine<'_, H>, + item: Value, + options: Value, +) -> Result { + if is_object(machine, item)? { + match temporal_record(machine, item)? { + Some(TemporalObject::PlainTime(value)) => { + read_overflow(machine, options)?; + return Ok(value); + } + Some(TemporalObject::PlainDateTime { value, .. }) => { + read_overflow(machine, options)?; + return Ok(value.time); + } + Some(TemporalObject::ZonedDateTime(zoned)) => { + let time = zoned + .to_plain_time(&FixedOffsetTimeZones) + .or_throw(machine)?; + read_overflow(machine, options)?; + return Ok(time); + } + _ => {} + } + let fields = + prepare_partial_fields(machine, &CalendarId::iso8601(), item, FieldNames::TIME)?; + let resolved = get_options_object(machine, options)?; + let overflow = overflow_option(machine, resolved)?; + return time_from_fields(machine, &fields, overflow); + } + let text = require_string(machine, item, "time")?; + let time = PlainTime::parse(&text).or_throw(machine)?; + read_overflow(machine, options)?; + Ok(time) +} + +/// ToTemporalYearMonth. +pub(super) fn to_temporal_year_month( + machine: &mut Machine<'_, H>, + item: Value, + options: Value, +) -> Result<(PlainYearMonth, CalendarId), EvalFailure> { + if is_object(machine, item)? { + if let Some(TemporalObject::PlainYearMonth { value, calendar }) = + temporal_record(machine, item)? + { + read_overflow(machine, options)?; + return Ok((value, calendar)); + } + let calendar = calendar_with_iso_default(machine, item)?; + let fields = prepare_calendar_fields( + machine, + &calendar, + item, + FieldNames::YEAR_MONTH, + FieldNames::NONE, + )?; + let resolved = get_options_object(machine, options)?; + let overflow = overflow_option(machine, resolved)?; + let value = year_month_from_fields(machine, &calendar, &fields, overflow)?; + return Ok((value, calendar)); + } + let text = require_string(machine, item, "year-month")?; + let parsed = parse_year_month_string(&text).or_throw(machine)?; + let calendar = parsed_calendar(machine, &parsed)?; + read_overflow(machine, options)?; + let value = PlainYearMonth::from_parsed(&parsed).or_throw(machine)?; + let Some(kind) = Calendar::from_id(&calendar) else { + return Ok((value, calendar)); + }; + let date = PlainDate { + year: parsed.year, + month: parsed.month, + day: parsed.day, + }; + let value = calendar::year_month_of(kind, date).or_throw(machine)?; + Ok((value, calendar)) +} + +/// ToTemporalMonthDay. +pub(super) fn to_temporal_month_day( + machine: &mut Machine<'_, H>, + item: Value, + options: Value, +) -> Result<(PlainMonthDay, CalendarId), EvalFailure> { + if is_object(machine, item)? { + if let Some(TemporalObject::PlainMonthDay { value, calendar }) = + temporal_record(machine, item)? + { + read_overflow(machine, options)?; + return Ok((value, calendar)); + } + let calendar = calendar_with_iso_default(machine, item)?; + let fields = + prepare_calendar_fields(machine, &calendar, item, FieldNames::DATE, FieldNames::NONE)?; + let resolved = get_options_object(machine, options)?; + let overflow = overflow_option(machine, resolved)?; + let value = month_day_from_fields(machine, &calendar, &fields, overflow)?; + return Ok((value, calendar)); + } + let text = require_string(machine, item, "month-day")?; + let parsed = parse_month_day_string(&text).or_throw(machine)?; + let calendar = parsed_calendar(machine, &parsed)?; + read_overflow(machine, options)?; + let value = PlainMonthDay::from_parsed(&parsed).or_throw(machine)?; + let Some(kind) = Calendar::from_id(&calendar) else { + return Ok((value, calendar)); + }; + let date = PlainDate::from_parsed(&parsed).or_throw(machine)?; + let value = calendar::month_day_of(kind, date).or_throw(machine)?; + Ok((value, calendar)) +} + +/// How `offset` information in the input applies to a wall-clock time. +#[derive(Clone, Copy, Debug, PartialEq)] +pub(super) enum OffsetBehaviour { + Option(i128), + Exact, + Wall, +} + +/// InterpretISODateTimeOffset. +pub(super) fn interpret_offset( + machine: &mut Machine<'_, H>, + local: PlainDateTime, + behaviour: OffsetBehaviour, + time_zone: TimeZoneId, + calendar: CalendarId, + resolve: ResolveOptions, +) -> Result { + let (supplied, offset) = match behaviour { + OffsetBehaviour::Wall => (None, OffsetOption::Ignore), + OffsetBehaviour::Exact => (Some(UtcOffsetNanoseconds::UTC), OffsetOption::Use), + OffsetBehaviour::Option(nanoseconds) => { + let nanoseconds = i64::try_from(nanoseconds) + .map_err(|_| range_error(machine, "UTC offset out of range"))?; + ( + Some(UtcOffsetNanoseconds::new(nanoseconds).or_throw(machine)?), + resolve.offset, + ) + } + }; + ZonedDateTime::from_local( + &providers(), + local, + time_zone, + calendar, + supplied, + ResolveOptions { offset, ..resolve }, + ) + .or_throw(machine) +} + +/// ToTemporalZonedDateTime. +pub(super) fn to_temporal_zoned_date_time( + machine: &mut Machine<'_, H>, + item: Value, + options: Value, +) -> Result { + if is_object(machine, item)? { + if let Some(TemporalObject::ZonedDateTime(zoned)) = temporal_record(machine, item)? { + let resolved = get_options_object(machine, options)?; + disambiguation_option(machine, resolved)?; + offset_option(machine, resolved, OffsetOption::Reject)?; + overflow_option(machine, resolved)?; + return Ok(zoned); + } + let calendar = calendar_with_iso_default(machine, item)?; + let names = FieldNames::DATE + .with(FieldNames::TIME) + .with(FieldNames::OFFSET) + .with(FieldNames::TIME_ZONE); + let fields = + prepare_calendar_fields(machine, &calendar, item, names, FieldNames::TIME_ZONE)?; + let resolved = get_options_object(machine, options)?; + let disambiguation = disambiguation_option(machine, resolved)?; + let offset = offset_option(machine, resolved, OffsetOption::Reject)?; + let overflow = overflow_option(machine, resolved)?; + let local = date_time_from_fields(machine, &calendar, &fields, overflow)?; + let behaviour = fields + .offset + .map_or(OffsetBehaviour::Wall, OffsetBehaviour::Option); + let time_zone = fields.time_zone.clone().expect("timeZone is required"); + return interpret_offset( + machine, + local, + behaviour, + time_zone, + calendar, + ResolveOptions { + disambiguation, + offset, + offset_match: OffsetMatch::Exactly, + }, + ); + } + let text = require_string(machine, item, "zoned date-time")?; + let parsed = parse_date_time_string(&text, true).or_throw(machine)?; + let annotation = parsed + .time_zone + .clone() + .expect("the zoned goal requires one"); + let time_zone = time_zone_from_annotation(machine, &annotation)?; + let calendar = parsed_calendar(machine, &parsed)?; + let resolved = get_options_object(machine, options)?; + let disambiguation = disambiguation_option(machine, resolved)?; + let offset = offset_option(machine, resolved, OffsetOption::Reject)?; + overflow_option(machine, resolved)?; + zoned_from_parsed( + machine, + &parsed, + time_zone, + calendar, + disambiguation, + offset, + ) +} + +fn time_zone_from_annotation( + machine: &mut Machine<'_, H>, + annotation: &str, +) -> Result { + let parsed = parse_time_zone_string(annotation).or_throw(machine)?; + TimeZoneId::from_parsed(parsed).or_throw(machine) +} + +/// The instant a parsed zoned string names: `Z` is exact, a numeric offset is +/// checked to the minute unless it has seconds, and a bare date means the +/// start of that day. +fn zoned_from_parsed( + machine: &mut Machine<'_, H>, + parsed: &crate::temporal::parse::ParsedIsoDateTime, + time_zone: TimeZoneId, + calendar: CalendarId, + disambiguation: Disambiguation, + offset: OffsetOption, +) -> Result { + let date = PlainDate::from_parsed(parsed).or_throw(machine)?; + if !parsed.has_time { + let instant = start_of_day(&FixedOffsetTimeZones, &time_zone, date).or_throw(machine)?; + return Ok(ZonedDateTime::from_instant(instant, time_zone, calendar)); + } + let local = PlainDateTime::from_parsed(parsed).or_throw(machine)?; + let (behaviour, offset_match) = match parsed.offset { + Some(UtcOffsetRecord::Zulu) => (OffsetBehaviour::Exact, OffsetMatch::Minutes), + Some(UtcOffsetRecord::Numeric { + nanoseconds, + sub_minute, + }) => ( + OffsetBehaviour::Option(nanoseconds), + if sub_minute { + OffsetMatch::Exactly + } else { + OffsetMatch::Minutes + }, + ), + None => (OffsetBehaviour::Wall, OffsetMatch::Minutes), + }; + interpret_offset( + machine, + local, + behaviour, + time_zone, + calendar, + ResolveOptions { + disambiguation, + offset, + offset_match, + }, + ) +} + +/// ToTemporalInstant. +pub(super) fn to_temporal_instant( + machine: &mut Machine<'_, H>, + item: Value, +) -> Result { + let mut primitive = item; + if is_object(machine, item)? { + match temporal_record(machine, item)? { + Some(TemporalObject::Instant(value)) => return Ok(value), + Some(TemporalObject::ZonedDateTime(zoned)) => return Ok(zoned.to_instant()), + _ => {} + } + primitive = machine.coerce_primitive_observable(item, true)?; + } + let text = require_string(machine, primitive, "instant")?; + Instant::parse(&text).or_throw(machine) +} + +/// ToTemporalDuration. +pub(in crate::intrinsics::builtins) fn to_temporal_duration( + machine: &mut Machine<'_, H>, + item: Value, +) -> Result { + if is_object(machine, item)? { + if let Some(TemporalObject::Duration(value)) = temporal_record(machine, item)? { + return Ok(value); + } + let partial = to_partial_duration(machine, item)?; + return Duration::new(partial.map(|field| field.unwrap_or(0))).or_throw(machine); + } + let text = require_string(machine, item, "duration")?; + Duration::parse(&text).or_throw(machine) +} + +/// ToTemporalPartialDurationRecord, largest field first. Properties are read in +/// sorted order, each converting immediately. +pub(super) fn to_partial_duration( + machine: &mut Machine<'_, H>, + item: Value, +) -> Result<[Option; 10], EvalFailure> { + if !is_object(machine, item)? { + return Err(type_error(machine, "duration-like must be an object")); + } + let mut fields = [None; 10]; + let sorted = [ + ("days", 3), + ("hours", 4), + ("microseconds", 8), + ("milliseconds", 7), + ("minutes", 5), + ("months", 1), + ("nanoseconds", 9), + ("seconds", 6), + ("weeks", 2), + ("years", 0), + ]; + for (name, slot) in sorted { + let value = machine.get_named_property(item, name)?; + if value != Value::UNDEFINED { + fields[slot] = Some(to_integer_if_integral(machine, value)?); + } + } + if fields.iter().all(Option::is_none) { + return Err(type_error( + machine, + "duration-like has no recognized properties", + )); + } + Ok(fields) +} + +/// Result of GetTemporalRelativeToOption. +#[derive(Clone, Debug)] +pub(super) enum RelativeTo { + None, + Plain(PlainDate, CalendarId), + Zoned(ZonedDateTime), +} + +/// GetTemporalRelativeToOption. +pub(super) fn relative_to_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = match options { + Some(object) => machine.get_named_property(object, "relativeTo")?, + None => Value::UNDEFINED, + }; + if value == Value::UNDEFINED { + return Ok(RelativeTo::None); + } + if is_object(machine, value)? { + match temporal_record(machine, value)? { + Some(TemporalObject::ZonedDateTime(zoned)) => return Ok(RelativeTo::Zoned(zoned)), + Some(TemporalObject::PlainDate { value, calendar }) => { + return Ok(RelativeTo::Plain(value, calendar)); + } + Some(TemporalObject::PlainDateTime { value, calendar }) => { + return Ok(RelativeTo::Plain(value.date, calendar)); + } + _ => {} + } + let calendar = calendar_with_iso_default(machine, value)?; + let names = FieldNames::DATE + .with(FieldNames::TIME) + .with(FieldNames::OFFSET) + .with(FieldNames::TIME_ZONE); + let fields = prepare_calendar_fields(machine, &calendar, value, names, FieldNames::NONE)?; + let local = date_time_from_fields(machine, &calendar, &fields, Overflow::Constrain)?; + let Some(time_zone) = fields.time_zone.clone() else { + return Ok(RelativeTo::Plain(local.date, calendar)); + }; + let behaviour = fields + .offset + .map_or(OffsetBehaviour::Wall, OffsetBehaviour::Option); + return interpret_offset( + machine, + local, + behaviour, + time_zone, + calendar, + ResolveOptions { + disambiguation: Disambiguation::Compatible, + offset: OffsetOption::Reject, + offset_match: OffsetMatch::Exactly, + }, + ) + .map(RelativeTo::Zoned); + } + let Some(text) = machine.string_value(value).map(|text| text.to_utf8_lossy()) else { + return Err(type_error(machine, "relativeTo must be a string or object")); + }; + let parsed = parse_date_time_string(&text, true) + .or_else(|_| parse_date_time_string(&text, false)) + .or_throw(machine)?; + let Some(annotation) = parsed.time_zone.clone() else { + let calendar = parsed_calendar(machine, &parsed)?; + let date = PlainDate::from_parsed(&parsed).or_throw(machine)?; + return Ok(RelativeTo::Plain(date, calendar)); + }; + let time_zone = time_zone_from_annotation(machine, &annotation)?; + let calendar = parsed_calendar(machine, &parsed)?; + zoned_from_parsed( + machine, + &parsed, + time_zone, + calendar, + Disambiguation::Compatible, + OffsetOption::Reject, + ) + .map(RelativeTo::Zoned) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/instant.rs b/crates/bamts-runtime/src/builtins/temporal/instant.rs new file mode 100644 index 00000000..08f98130 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/instant.rs @@ -0,0 +1,402 @@ +//! `Temporal.Instant`. + +use bamts_native::Value; + +use super::super::bigint::to_bigint; +use super::super::intl::temporal_to_locale_string; +use super::super::value_number; +use super::fields::{ + providers, to_temporal_duration, to_temporal_instant, to_time_zone_identifier, +}; +use super::options::{ + DifferencePolicy, Options, TIME_UNITS, UnitOption, difference_settings, + fractional_second_digits_option, get_options_object, rounding_increment_option, + rounding_mode_option, seconds_precision, unit_argument, unit_valued_option, validate_increment, + validate_unit_value, +}; +use super::{ + Class, OrThrow, arg, bigint_outcome, bigint_to_i128, construct, create, install_class, + number_outcome, range_error, require_new, string_outcome, temporal_record, type_error, + value_of_throws, +}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::instant_duration::{Instant, NS_PER_DAY, RoundingMode, Unit}; +use crate::temporal::now_rounding::{ + format_iso_date_time, format_utc_offset_rounded, round_epoch_nanoseconds, +}; +use crate::temporal::zoned_date_time::{ + CalendarId, TimeZoneId, ZonedDateTime, iso_date_time_for_instant, +}; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: TemporalKind::Instant, + name: "Instant", + tag: "Temporal.Instant", + length: 1, + constructor: constructor::, + statics: &[ + ("from", 1, from::), + ("fromEpochMilliseconds", 1, from_epoch_milliseconds::), + ("fromEpochNanoseconds", 1, from_epoch_nanoseconds::), + ("compare", 2, compare::), + ], + getters: &[ + ( + "epochMilliseconds", + "get epochMilliseconds", + epoch_milliseconds::, + ), + ( + "epochNanoseconds", + "get epochNanoseconds", + epoch_nanoseconds::, + ), + ], + methods: &[ + ("add", 1, add::), + ("subtract", 1, subtract::), + ("until", 1, until::), + ("since", 1, since::), + ("round", 1, round::), + ("equals", 1, equals::), + ("toString", 0, to_string::), + ("toLocaleString", 0, to_locale_string::), + ("toJSON", 0, to_json::), + ("valueOf", 0, value_of_throws::), + ("toZonedDateTimeISO", 1, to_zoned_date_time_iso::), + ], + }, + ); +} + +fn this_instant( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result { + match temporal_record(machine, this)? { + Some(TemporalObject::Instant(value)) => Ok(value), + _ => Err(type_error(machine, "receiver is not a Temporal.Instant")), + } +} + +fn instant_outcome( + machine: &mut Machine<'_, H>, + value: Instant, +) -> Result { + create(machine, TemporalObject::Instant(value)).map(BuiltinOutcome::Value) +} + +/// ToBigInt, then the range check of CreateTemporalInstant. +fn instant_from_bigint( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let bigint = to_bigint(machine, value)?; + let nanoseconds = bigint_to_i128(machine, bigint)?; + Instant::from_epoch_nanoseconds(nanoseconds).or_throw(machine) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "Instant")?; + let value = instant_from_bigint(machine, arg(args, 0))?; + construct(machine, TemporalObject::Instant(value)) +} + +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = to_temporal_instant(machine, arg(args, 0))?; + instant_outcome(machine, value) +} + +fn from_epoch_milliseconds( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let number = value_number(machine.coerce_number_observable(arg(args, 0))?); + if !number.is_finite() || number.trunc() != number { + return Err(range_error( + machine, + "epoch milliseconds must be an integer", + )); + } + // A saturated value is still out of range, as the original was. + let value = Instant::from_epoch_milliseconds(number as i64).or_throw(machine)?; + instant_outcome(machine, value) +} + +fn from_epoch_nanoseconds( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = instant_from_bigint(machine, arg(args, 0))?; + instant_outcome(machine, value) +} + +fn compare( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let one = to_temporal_instant(machine, arg(args, 0))?; + let two = to_temporal_instant(machine, arg(args, 1))?; + number_outcome(f64::from(one.cmp(&two) as i8)) +} + +fn epoch_milliseconds( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + number_outcome(this_instant(machine, this)?.epoch_milliseconds() as f64) +} + +fn epoch_nanoseconds( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = this_instant(machine, this)?.epoch_nanoseconds(); + bigint_outcome(machine, value) +} + +fn add_duration( + machine: &mut Machine<'_, H>, + instant: Instant, + other: Value, + subtract: bool, +) -> Result { + let duration = to_temporal_duration(machine, other)?; + let value = if subtract { + instant.subtract(&duration) + } else { + instant.add(&duration) + } + .or_throw(machine)?; + instant_outcome(machine, value) +} + +fn add( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let instant = this_instant(machine, this)?; + add_duration(machine, instant, arg(args, 0), false) +} + +fn subtract( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let instant = this_instant(machine, this)?; + add_duration(machine, instant, arg(args, 0), true) +} + +/// DifferenceTemporalInstant. +fn difference( + machine: &mut Machine<'_, H>, + since: bool, + instant: Instant, + other: Value, + options: Value, +) -> Result { + let other = to_temporal_instant(machine, other)?; + let options = get_options_object(machine, options)?; + let settings = difference_settings( + machine, + since, + options, + DifferencePolicy { + group: TIME_UNITS, + disallowed: &[], + fallback_smallest: Unit::Nanosecond, + smallest_largest_default: Unit::Second, + }, + )?; + let value = instant + .until( + other, + settings.largest_unit, + settings.smallest_unit, + settings.rounding_increment, + settings.rounding_mode, + ) + .or_throw(machine)?; + let value = if since { value.negated() } else { value }; + create(machine, TemporalObject::Duration(value)).map(BuiltinOutcome::Value) +} + +fn until( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let instant = this_instant(machine, this)?; + difference(machine, false, instant, arg(args, 0), arg(args, 1)) +} + +fn since( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let instant = this_instant(machine, this)?; + difference(machine, true, instant, arg(args, 0), arg(args, 1)) +} + +fn round( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let instant = this_instant(machine, this)?; + let round_to = unit_argument(machine, arg(args, 0))?; + let increment = rounding_increment_option(machine, round_to.options)?; + let mode = rounding_mode_option(machine, round_to.options, RoundingMode::HalfExpand)?; + let smallest = round_to.required_unit(machine, "smallestUnit")?; + validate_unit_value(machine, smallest, TIME_UNITS, None)?; + let UnitOption::Unit(unit) = smallest else { + return Err(range_error(machine, "smallestUnit is required")); + }; + let unit_ns = unit.ns_per().unwrap_or(1); + validate_increment(machine, increment, NS_PER_DAY / unit_ns, true)?; + let value = instant.round(unit, increment, mode).or_throw(machine)?; + instant_outcome(machine, value) +} + +fn equals( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let instant = this_instant(machine, this)?; + let other = to_temporal_instant(machine, arg(args, 0))?; + Ok(BuiltinOutcome::Value(Value::boolean(instant == other))) +} + +/// TemporalInstantToString: `Z` without a zone, else the zone's rounded offset. +fn instant_text( + machine: &mut Machine<'_, H>, + instant: Instant, + time_zone: Option<&TimeZoneId>, + precision: crate::temporal::instant_duration::Precision, +) -> Result { + let Some(time_zone) = time_zone else { + return Ok(instant.format(precision)); + }; + let core = providers(); + let local = iso_date_time_for_instant(core.time_zone, time_zone, instant.epoch_nanoseconds()) + .or_throw(machine)?; + let offset = core + .time_zone + .offset_nanoseconds_for(time_zone, instant) + .or_throw(machine)?; + let offset = format_utc_offset_rounded(offset).or_throw(machine)?; + Ok(format!( + "{}{offset}", + format_iso_date_time(&local, precision) + )) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let instant = this_instant(machine, this)?; + let options: Options = get_options_object(machine, arg(args, 0))?; + let digits = fractional_second_digits_option(machine, options)?; + let mode = rounding_mode_option(machine, options, RoundingMode::Trunc)?; + let smallest = unit_valued_option(machine, options, "smallestUnit")?; + let time_zone = match options { + Some(object) => machine.get_named_property(object, "timeZone")?, + None => Value::UNDEFINED, + }; + validate_unit_value(machine, smallest, TIME_UNITS, None)?; + if smallest == UnitOption::Unit(Unit::Hour) { + return Err(range_error(machine, "smallestUnit must not be hour")); + } + let time_zone = if time_zone == Value::UNDEFINED { + None + } else { + Some(to_time_zone_identifier(machine, time_zone)?) + }; + let precision = seconds_precision(machine, smallest, digits)?; + let rounded = round_epoch_nanoseconds( + instant.epoch_nanoseconds(), + precision.unit, + precision.increment, + mode, + ) + .and_then(Instant::from_epoch_nanoseconds) + .or_throw(machine)?; + let text = instant_text(machine, rounded, time_zone.as_ref(), precision.precision)?; + string_outcome(machine, &text) +} + +fn to_json( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let text = this_instant(machine, this)?.to_string(); + string_outcome(machine, &text) +} + +fn to_zoned_date_time_iso( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let instant = this_instant(machine, this)?; + let time_zone = to_time_zone_identifier(machine, arg(args, 0))?; + let value = ZonedDateTime::from_instant(instant, time_zone, CalendarId::iso8601()); + create(machine, TemporalObject::ZonedDateTime(value)).map(BuiltinOutcome::Value) +} + +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + this_instant(machine, this)?; + temporal_to_locale_string(machine, this, arg(args, 0), arg(args, 1)) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/mod.rs b/crates/bamts-runtime/src/builtins/temporal/mod.rs new file mode 100644 index 00000000..a90c7688 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/mod.rs @@ -0,0 +1,422 @@ +//! The JavaScript binding of the `Temporal` proposal. +//! +//! Algorithms live in `crate::temporal`; this module owns what only JavaScript +//! can observe: argument conversion, the order options and properties are read +//! in, receiver checks, and the shape of the installed objects. + +/// Defines the calendar accessors shared by the classes that hold a date. +/// `$this` reads the receiver's date and calendar, and is the brand check. +macro_rules! date_accessors { + ($this:ident) => { + fn calendar_id( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let (_, calendar) = $this(machine, this)?; + string_outcome(machine, calendar.as_str()) + } + + /// The calendar's view of the receiver's date. + fn calendar_fields( + machine: &mut Machine<'_, H>, + this: Value, + ) -> Result { + let (date, calendar) = $this(machine, this)?; + super::fields::calendar_date(machine, &calendar, date) + } + + fn era( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let fields = calendar_fields(machine, this)?; + super::optional_string_outcome(machine, fields.era.as_deref()) + } + + fn month_code( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let fields = calendar_fields(machine, this)?; + string_outcome(machine, &fields.month_code) + } + + fn in_leap_year( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let fields = calendar_fields(machine, this)?; + Ok(BuiltinOutcome::Value(Value::boolean(fields.in_leap_year))) + } + + date_number_accessors! { + calendar_fields, + era_year: |fields| fields.era_year.map(f64::from), + year: |fields| Some(f64::from(fields.year)), + month: |fields| Some(f64::from(fields.month)), + day: |fields| Some(f64::from(fields.day)), + day_of_week: |fields| Some(f64::from(fields.day_of_week)), + day_of_year: |fields| Some(f64::from(fields.day_of_year)), + week_of_year: |fields| fields.week_of_year.map(f64::from), + year_of_week: |fields| fields.year_of_week.map(f64::from), + days_in_week: |fields| Some(f64::from(fields.days_in_week)), + days_in_month: |fields| Some(f64::from(fields.days_in_month)), + days_in_year: |fields| Some(f64::from(fields.days_in_year)), + months_in_year: |fields| Some(f64::from(fields.months_in_year)), + } + }; +} + +macro_rules! date_number_accessors { + ($this:ident, $($name:ident: |$fields:ident| $value:expr),+ $(,)?) => { + $(fn $name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let $fields = $this(machine, this)?; + super::optional_number_outcome($value) + })+ + }; +} + +/// The accessor table `date_accessors!` defines the functions for. +macro_rules! date_getter_table { + ($H:ident) => { + [ + ("calendarId", "get calendarId", calendar_id::<$H>), + ("era", "get era", era::<$H>), + ("eraYear", "get eraYear", era_year::<$H>), + ("year", "get year", year::<$H>), + ("month", "get month", month::<$H>), + ("monthCode", "get monthCode", month_code::<$H>), + ("day", "get day", day::<$H>), + ("dayOfWeek", "get dayOfWeek", day_of_week::<$H>), + ("dayOfYear", "get dayOfYear", day_of_year::<$H>), + ("weekOfYear", "get weekOfYear", week_of_year::<$H>), + ("yearOfWeek", "get yearOfWeek", year_of_week::<$H>), + ("daysInWeek", "get daysInWeek", days_in_week::<$H>), + ("daysInMonth", "get daysInMonth", days_in_month::<$H>), + ("daysInYear", "get daysInYear", days_in_year::<$H>), + ("monthsInYear", "get monthsInYear", months_in_year::<$H>), + ("inLeapYear", "get inLeapYear", in_leap_year::<$H>), + ] + }; +} + +mod duration; +mod fields; +mod instant; +mod now; +mod options; +mod plain_date; +mod plain_date_time; +mod plain_month_day; +mod plain_time; +mod plain_year_month; +mod zoned_date_time; + +pub(in crate::intrinsics::builtins) use fields::to_temporal_duration; + +use std::collections::BTreeMap; + +use bamts_bytecode::EcmaString; +use bamts_native::Value; + +use super::{allocate_string, define_data, install_constructor_function, install_function}; +use crate::intrinsics::{BuiltinHandler, BuiltinOutcome, BuiltinTable}; +use crate::temporal::instant_duration::{TemporalError, TemporalErrorKind}; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine, Property, PropertyKey, PropertyMap}; + +/// Name, `length`, and implementation of one installed function. +pub(super) type Method = (&'static str, u32, BuiltinHandler); + +/// Property name, function name, and implementation of one accessor. +pub(super) type Getter = (&'static str, &'static str, BuiltinHandler); + +/// The installed shape of one Temporal class. +struct Class<'a, H: Host> { + kind: TemporalKind, + name: &'static str, + tag: &'static str, + length: u32, + constructor: BuiltinHandler, + statics: &'a [Method], + getters: &'a [Getter], + methods: &'a [Method], +} + +fn install_class( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, + class: &Class<'_, H>, +) { + let prototype = builtins.temporal_prototype(class.kind); + let constructor = + install_constructor_function(heap, builtins, class.name, class.length, class.constructor); + builtins.set_constructor_prototype(heap, constructor, prototype); + for &(name, length, handler) in class.statics { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, constructor, name, function); + } + for &(property, name, handler) in class.getters { + let getter = install_function(heap, builtins, name, 0, handler); + define_getter(heap, prototype, property, getter); + } + for &(name, length, handler) in class.methods { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, prototype, name, function); + } + let tag_symbol = builtins.symbol_to_string_tag(); + super::define_to_string_tag(heap, prototype, tag_symbol, class.tag); + define_data(heap, namespace, class.name, constructor); +} + +pub(in crate::intrinsics::builtins) fn define_getter( + heap: &mut [HeapEntry], + object: Value, + name: &str, + getter: Value, +) { + let Some(bamts_native::Decoded::HeapRef(id)) = object.decode() else { + panic!("accessor target is a heap object"); + }; + let HeapEntry::Object { properties, .. } = &mut heap[id.slot() as usize - 1] else { + panic!("accessor target is an ordinary object"); + }; + properties.insert( + PropertyKey::Named(EcmaString::encode(name)), + Property::Accessor { + getter: Some(getter), + setter: None, + enumerable: false, + configurable: true, + }, + ); +} + +pub(super) fn install( + heap: &mut Vec, + globals: &mut BTreeMap, + builtins: &mut BuiltinTable, +) { + let namespace = super::super::ordinary_prototype(heap, builtins.object_prototype()); + let tag_symbol = builtins.symbol_to_string_tag(); + super::define_to_string_tag(heap, namespace, tag_symbol, "Temporal"); + // Classes refer to each other's instances, so every prototype exists before + // any class is populated. + for kind in TemporalKind::ALL { + let prototype = super::super::ordinary_prototype(heap, builtins.object_prototype()); + builtins.set_temporal_prototype(kind, prototype); + } + duration::install(heap, builtins, namespace); + instant::install(heap, builtins, namespace); + plain_time::install(heap, builtins, namespace); + plain_date::install(heap, builtins, namespace); + plain_date_time::install(heap, builtins, namespace); + plain_year_month::install(heap, builtins, namespace); + plain_month_day::install(heap, builtins, namespace); + zoned_date_time::install(heap, builtins, namespace); + now::install(heap, builtins, namespace); + globals.insert(EcmaString::encode("Temporal"), namespace); +} + +/// Maps a core error to the ECMAScript exception it stands for. Grammar +/// failures are `RangeError`s, and every message survives to the catch site. +pub(super) fn throw(machine: &mut Machine<'_, H>, error: TemporalError) -> EvalFailure { + let name = match error.kind { + TemporalErrorKind::Type => "TypeError", + TemporalErrorKind::Range | TemporalErrorKind::Syntax => "RangeError", + }; + let id = machine + .intrinsics + .builtins + .id_named(name) + .expect("error constructor is installed"); + machine.throw_error(id, error.message) +} + +/// Lifts a core result into the binding's error channel. +pub(super) trait OrThrow { + fn or_throw(self, machine: &mut Machine<'_, H>) -> Result; +} + +impl OrThrow for Result { + fn or_throw(self, machine: &mut Machine<'_, H>) -> Result { + self.map_err(|error| throw(machine, error)) + } +} + +pub(super) fn range_error( + machine: &mut Machine<'_, H>, + message: impl Into, +) -> EvalFailure { + throw(machine, TemporalError::range(message)) +} + +pub(super) fn type_error( + machine: &mut Machine<'_, H>, + message: impl Into, +) -> EvalFailure { + throw(machine, TemporalError::type_error(message)) +} + +/// `Type(value) is Object`. `Machine::is_object` also answers true for a +/// Symbol, which every Temporal "must be an object" check has to reject. +pub(super) fn is_object( + machine: &Machine<'_, H>, + value: Value, +) -> Result { + Ok(!machine.coercion_is_primitive(value)?) +} + +/// The Temporal record `value` carries, if it is a Temporal instance. +pub(in crate::intrinsics::builtins) fn temporal_record( + machine: &Machine<'_, H>, + value: Value, +) -> Result, EvalFailure> { + let Some(index) = machine.runtime_slot(value).map_err(EvalFailure::Runtime)? else { + return Ok(None); + }; + Ok(match &machine.heap[index] { + HeapEntry::Temporal { value, .. } => Some((**value).clone()), + _ => None, + }) +} + +/// Allocates an instance with the class's own prototype. +pub(super) fn create( + machine: &mut Machine<'_, H>, + value: TemporalObject, +) -> Result { + let prototype = machine.intrinsics.builtins.temporal_prototype(value.kind()); + allocate(machine, value, prototype) +} + +/// Allocates the result of `new`, honouring `new.target`. The proposal reads +/// `prototype` last, after every argument has been validated, and an object +/// whose `prototype` is not an object gets the class's own prototype rather +/// than `Object.prototype`. +pub(super) fn construct( + machine: &mut Machine<'_, H>, + value: TemporalObject, +) -> Result { + let new_target = machine.current_new_target(); + let prototype = machine.get_named_property(new_target, "prototype")?; + let prototype = if is_object(machine, prototype)? { + prototype + } else { + machine.intrinsics.builtins.temporal_prototype(value.kind()) + }; + allocate(machine, value, prototype).map(BuiltinOutcome::Value) +} + +fn allocate( + machine: &mut Machine<'_, H>, + value: TemporalObject, + prototype: Value, +) -> Result { + machine + .allocate(HeapEntry::Temporal { + value: Box::new(value), + properties: PropertyMap::default(), + prototype: Some(prototype), + extensible: true, + }) + .map_err(EvalFailure::Runtime) +} + +/// Throws unless this call came through `new`. +pub(super) fn require_new( + machine: &mut Machine<'_, H>, + constructing: bool, + name: &str, +) -> Result<(), EvalFailure> { + if constructing { + Ok(()) + } else { + Err(type_error( + machine, + format!("Temporal.{name} constructor requires new"), + )) + } +} + +pub(super) fn string_outcome( + machine: &mut Machine<'_, H>, + text: &str, +) -> Result { + allocate_string(machine, EcmaString::encode(text)).map(BuiltinOutcome::Value) +} + +pub(super) fn number_outcome(value: f64) -> Result { + Ok(BuiltinOutcome::Value(crate::number_value(value))) +} + +/// A calendar answer that some calendars leave `undefined`. +pub(super) fn optional_number_outcome(value: Option) -> Result { + value.map_or(Ok(BuiltinOutcome::Value(Value::UNDEFINED)), number_outcome) +} + +/// A calendar answer that some calendars leave `undefined`. +pub(super) fn optional_string_outcome( + machine: &mut Machine<'_, H>, + value: Option<&str>, +) -> Result { + match value { + Some(text) => string_outcome(machine, text), + None => Ok(BuiltinOutcome::Value(Value::UNDEFINED)), + } +} + +/// `valueOf` on every Temporal class: comparison must be explicit. +pub(super) fn value_of_throws( + machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + Err(type_error( + machine, + "Temporal objects are not comparable with valueOf; use compare or equals", + )) +} + +/// The value of a BigInt the caller has already produced with ToBigInt. One +/// too large for `i128` is out of every Temporal range. +pub(super) fn bigint_to_i128( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let text = super::bigint::bigint_from_value(machine, value)? + .expect("ToBigInt produces a BigInt") + .to_string_radix(10); + text.parse() + .map_err(|_| range_error(machine, "BigInt outside the Temporal range")) +} + +pub(super) fn bigint_outcome( + machine: &mut Machine<'_, H>, + value: i128, +) -> Result { + machine + .allocate(HeapEntry::BigInt(value.to_string())) + .map(BuiltinOutcome::Value) + .map_err(EvalFailure::Runtime) +} + +/// The argument at `index`, `undefined` when absent. +pub(super) fn arg(args: &[Value], index: usize) -> Value { + args.get(index).copied().unwrap_or(Value::UNDEFINED) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/now.rs b/crates/bamts-runtime/src/builtins/temporal/now.rs new file mode 100644 index 00000000..2ce9b4c2 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/now.rs @@ -0,0 +1,239 @@ +//! `Temporal.Now`. + +use std::convert::Infallible; + +use bamts_native::Value; + +use super::super::{define_data, install_function}; +use super::fields::to_time_zone_identifier; +use super::{Method, arg, create, string_outcome, throw}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::TemporalObject; +use crate::temporal::instant_duration::{Instant, NS_PER_MILLISECOND, TemporalError}; +use crate::temporal::now_rounding::{ + CalendarProjectionProvider, ClockSnapshot, HostClock, NowError, NowTimeZoneProvider, + TemporalNow, UtcOffsetNanoseconds, +}; +use crate::temporal::parse::parse_time_zone_identifier; +use crate::temporal::plain_types::PlainDate; +use crate::temporal::zoned_date_time::{ + CalendarId, FixedOffsetTimeZones, TimeZoneId, TimeZoneProvider, ZonedDateTime, +}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + let now = super::super::super::ordinary_prototype(heap, builtins.object_prototype()); + let methods: [Method; 6] = [ + ("instant", 0, instant::), + ("timeZoneId", 0, time_zone_id::), + ("zonedDateTimeISO", 0, zoned_date_time_iso::), + ("plainDateTimeISO", 0, plain_date_time_iso::), + ("plainDateISO", 0, plain_date_iso::), + ("plainTimeISO", 0, plain_time_iso::), + ]; + for (name, length, handler) in methods { + let function = install_function(heap, builtins, name, length, handler); + define_data(heap, now, name, function); + } + let tag_symbol = builtins.symbol_to_string_tag(); + super::super::define_to_string_tag(heap, now, tag_symbol, "Temporal.Now"); + define_data(heap, namespace, "Now", now); +} + +/// One reading of the host clock. The host has millisecond resolution. +struct HostReading(ClockSnapshot); + +impl HostClock for HostReading { + type Error = Infallible; + + fn snapshot(&mut self) -> Result { + Ok(self.0) + } +} + +/// The host's zone, and the offsets of every zone available without a database. +struct AvailableZones { + system: TimeZoneId, +} + +impl NowTimeZoneProvider for AvailableZones { + type Error = TemporalError; + type Identifier = TimeZoneId; + + fn system_time_zone_id(&mut self) -> Result { + Ok(self.system.clone()) + } + + fn offset_nanoseconds( + &mut self, + time_zone: &TimeZoneId, + instant: Instant, + ) -> Result { + FixedOffsetTimeZones.offset_nanoseconds_for(time_zone, instant) + } +} + +/// `Temporal.Now` produces ISO values, whose projection is the ISO date itself. +struct IsoProjection; + +impl CalendarProjectionProvider for IsoProjection { + type Error = Infallible; + type Identifier = CalendarId; + type Projection = PlainDate; + + fn iso8601_identifier(&self) -> CalendarId { + CalendarId::iso8601() + } + + fn project_iso_date( + &mut self, + _calendar: &CalendarId, + iso_date: PlainDate, + ) -> Result { + Ok(iso_date) + } +} + +type HostNow = TemporalNow; + +/// SystemTimeZoneIdentifier. A host zone needing a time zone database cannot +/// be represented here, and the proposal's answer for an implementation that +/// only knows UTC is `UTC`. +fn system_time_zone(machine: &Machine<'_, H>) -> TimeZoneId { + machine + .host + .env("TZ") + .and_then(|zone| parse_time_zone_identifier(zone).ok()) + .and_then(|zone| TimeZoneId::from_parsed(zone).ok()) + .unwrap_or_else(TimeZoneId::utc) +} + +fn host_now(machine: &mut Machine<'_, H>) -> HostNow { + let nanoseconds = i128::from(machine.host.now_ms()) * NS_PER_MILLISECOND; + TemporalNow::new( + HostReading(ClockSnapshot::clamped(nanoseconds)), + AvailableZones { + system: system_time_zone(machine), + }, + IsoProjection, + ) +} + +fn now_failure( + machine: &mut Machine<'_, H>, + failure: NowError, +) -> EvalFailure { + match failure { + NowError::Temporal(error) | NowError::TimeZone(error) => throw(machine, error), + NowError::Clock(never) | NowError::Calendar(never) => match never {}, + } +} + +/// The zone argument of the `*ISO` methods: absent means the host's. +fn time_zone_argument( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result, EvalFailure> { + if value == Value::UNDEFINED { + Ok(None) + } else { + to_time_zone_identifier(machine, value).map(Some) + } +} + +fn instant( + machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = host_now(machine) + .instant() + .map_err(|failure| now_failure(machine, failure))?; + create(machine, TemporalObject::Instant(value)).map(BuiltinOutcome::Value) +} + +fn time_zone_id( + machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let time_zone = host_now(machine) + .time_zone_id() + .map_err(|failure| now_failure(machine, failure))?; + string_outcome(machine, time_zone.as_str()) +} + +fn zoned_date_time_iso( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time_zone = time_zone_argument(machine, arg(args, 0))?; + let projected = host_now(machine) + .zoned_date_time_iso(time_zone) + .map_err(|failure| now_failure(machine, failure))?; + let value = ZonedDateTime::from_instant( + projected.instant(), + projected.time_zone_id().clone(), + projected.calendar_id().clone(), + ); + create(machine, TemporalObject::ZonedDateTime(value)).map(BuiltinOutcome::Value) +} + +fn plain_date_time_iso( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time_zone = time_zone_argument(machine, arg(args, 0))?; + let value = host_now(machine) + .plain_date_time_iso(time_zone) + .map_err(|failure| now_failure(machine, failure))?; + create( + machine, + TemporalObject::PlainDateTime { + value, + calendar: CalendarId::iso8601(), + }, + ) + .map(BuiltinOutcome::Value) +} + +fn plain_date_iso( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time_zone = time_zone_argument(machine, arg(args, 0))?; + let mut now = host_now(machine); + let projected = now + .zoned_date_time_iso(time_zone) + .map_err(|failure| now_failure(machine, failure))?; + let calendar = projected.calendar_id().clone(); + let value = now + .project_calendar(&projected, &calendar) + .map_err(|failure| now_failure(machine, failure))?; + create(machine, TemporalObject::PlainDate { value, calendar }).map(BuiltinOutcome::Value) +} + +fn plain_time_iso( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time_zone = time_zone_argument(machine, arg(args, 0))?; + let value = host_now(machine) + .plain_time_iso(time_zone) + .map_err(|failure| now_failure(machine, failure))?; + create(machine, TemporalObject::PlainTime(value)).map(BuiltinOutcome::Value) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/options.rs b/crates/bamts-runtime/src/builtins/temporal/options.rs new file mode 100644 index 00000000..1d51302b --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/options.rs @@ -0,0 +1,495 @@ +//! Option reading. Every function here performs exactly the property reads and +//! conversions its abstract operation does; callers sequence them in the +//! proposal's alphabetical order, which test262 observes through Proxies. + +use bamts_native::{Decoded, Value}; + +use super::{OrThrow, is_object, range_error, type_error}; +use crate::temporal::instant_duration::{ + Overflow, RoundingMode, Unit, UnitCategory, validate_rounding_increment, +}; +use crate::temporal::now_rounding::{ + FractionalSecondDigits, SecondsStringPrecision, ShowCalendar, ShowOffset, ShowTimeZone, +}; +use crate::temporal::zoned_date_time::{Disambiguation, OffsetOption}; +use crate::{EvalFailure, Host, Machine}; + +/// A resolved options argument. `None` stands for the empty null-prototype +/// object of `GetOptionsObject(undefined)`, which no read can observe. +pub(super) type Options = Option; + +/// GetOptionsObject. +pub(super) fn get_options_object( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + if value == Value::UNDEFINED { + Ok(None) + } else if is_object(machine, value)? { + Ok(Some(value)) + } else { + Err(type_error(machine, "options must be an object")) + } +} + +fn read( + machine: &mut Machine<'_, H>, + options: Options, + name: &str, +) -> Result { + match options { + Some(object) => machine.get_named_property(object, name), + None => Ok(Value::UNDEFINED), + } +} + +/// GetOption with type string, no default, and a closed set of values. +fn string_option( + machine: &mut Machine<'_, H>, + options: Options, + name: &str, + allowed: &[&str], +) -> Result, EvalFailure> { + let value = read(machine, options, name)?; + if value == Value::UNDEFINED { + return Ok(None); + } + let text = machine.coerce_string_observable(value)?.to_utf8_lossy(); + if allowed.contains(&text.as_str()) { + Ok(Some(text)) + } else { + Err(range_error( + machine, + format!("{text} is not a valid value for {name}"), + )) + } +} + +/// GetTemporalOverflowOption. +pub(super) fn overflow_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = string_option(machine, options, "overflow", &["constrain", "reject"])?; + Ok(value + .as_deref() + .and_then(Overflow::from_name) + .unwrap_or_default()) +} + +/// GetTemporalDisambiguationOption. +pub(super) fn disambiguation_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = string_option( + machine, + options, + "disambiguation", + &["compatible", "earlier", "later", "reject"], + )?; + Ok(match value.as_deref() { + Some("earlier") => Disambiguation::Earlier, + Some("later") => Disambiguation::Later, + Some("reject") => Disambiguation::Reject, + _ => Disambiguation::Compatible, + }) +} + +/// GetTemporalOffsetOption. +pub(super) fn offset_option( + machine: &mut Machine<'_, H>, + options: Options, + fallback: OffsetOption, +) -> Result { + let value = string_option( + machine, + options, + "offset", + &["prefer", "use", "ignore", "reject"], + )?; + Ok(match value.as_deref() { + Some("prefer") => OffsetOption::Prefer, + Some("use") => OffsetOption::Use, + Some("ignore") => OffsetOption::Ignore, + Some("reject") => OffsetOption::Reject, + _ => fallback, + }) +} + +/// GetTemporalShowCalendarNameOption. +pub(super) fn show_calendar_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = string_option( + machine, + options, + "calendarName", + &["auto", "always", "never", "critical"], + )?; + Ok(match value.as_deref() { + Some("always") => ShowCalendar::Always, + Some("never") => ShowCalendar::Never, + Some("critical") => ShowCalendar::Critical, + _ => ShowCalendar::Auto, + }) +} + +/// GetTemporalShowTimeZoneNameOption. +pub(super) fn show_time_zone_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = string_option( + machine, + options, + "timeZoneName", + &["auto", "never", "critical"], + )?; + Ok(match value.as_deref() { + Some("never") => ShowTimeZone::Never, + Some("critical") => ShowTimeZone::Critical, + _ => ShowTimeZone::Auto, + }) +} + +/// GetTemporalShowOffsetOption. +pub(super) fn show_offset_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = string_option(machine, options, "offset", &["auto", "never"])?; + Ok(if value.as_deref() == Some("never") { + ShowOffset::Never + } else { + ShowOffset::Auto + }) +} + +/// GetDirectionOption, which has no default. +pub(super) fn direction_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + match string_option(machine, options, "direction", &["next", "previous"])? { + Some(direction) => Ok(direction), + None => Err(range_error(machine, "direction is required")), + } +} + +/// GetRoundingModeOption. +pub(super) fn rounding_mode_option( + machine: &mut Machine<'_, H>, + options: Options, + fallback: RoundingMode, +) -> Result { + let value = string_option( + machine, + options, + "roundingMode", + &[ + "ceil", + "floor", + "expand", + "trunc", + "halfCeil", + "halfFloor", + "halfExpand", + "halfTrunc", + "halfEven", + ], + )?; + Ok(value + .as_deref() + .and_then(RoundingMode::from_name) + .unwrap_or(fallback)) +} + +/// GetRoundingIncrementOption. +pub(super) fn rounding_increment_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = read(machine, options, "roundingIncrement")?; + if value == Value::UNDEFINED { + return Ok(1); + } + let increment = super::fields::to_integer_with_truncation(machine, value)?; + if !(1.0..=1e9).contains(&increment) { + return Err(range_error(machine, "roundingIncrement is out of range")); + } + Ok(increment as i128) +} + +/// ValidateTemporalRoundingIncrement for an explicit dividend. +pub(super) fn validate_increment( + machine: &mut Machine<'_, H>, + increment: i128, + dividend: i128, + inclusive: bool, +) -> Result<(), EvalFailure> { + let maximum = if inclusive { dividend } else { dividend - 1 }; + if increment > maximum || dividend % increment != 0 { + return Err(range_error( + machine, + format!("roundingIncrement {increment} does not divide {dividend}"), + )); + } + Ok(()) +} + +/// GetTemporalFractionalSecondDigitsOption. +pub(super) fn fractional_second_digits_option( + machine: &mut Machine<'_, H>, + options: Options, +) -> Result { + let value = read(machine, options, "fractionalSecondDigits")?; + if value == Value::UNDEFINED { + return Ok(FractionalSecondDigits::Auto); + } + let number = match value.decode() { + Some(Decoded::Int32(number)) => f64::from(number as i32), + Some(Decoded::Number(number)) => number, + _ => { + let text = machine.coerce_string_observable(value)?.to_utf8_lossy(); + if text == "auto" { + return Ok(FractionalSecondDigits::Auto); + } + return Err(range_error( + machine, + format!("{text} is not a valid value for fractionalSecondDigits"), + )); + } + }; + let digits = number.floor(); + if !digits.is_finite() || digits < 0.0 { + return Err(range_error( + machine, + "fractionalSecondDigits must be from 0 to 9", + )); + } + FractionalSecondDigits::digits(digits.min(255.0) as u8).or_throw(machine) +} + +/// The value of a unit option before it is validated against a unit group. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub(super) enum UnitOption { + Unset, + Auto, + Unit(Unit), +} + +const UNIT_NAMES: [&str; 21] = [ + "year", + "years", + "month", + "months", + "week", + "weeks", + "day", + "days", + "hour", + "hours", + "minute", + "minutes", + "second", + "seconds", + "millisecond", + "milliseconds", + "microsecond", + "microseconds", + "nanosecond", + "nanoseconds", + "auto", +]; + +/// GetTemporalUnitValuedOption. +pub(super) fn unit_valued_option( + machine: &mut Machine<'_, H>, + options: Options, + name: &str, +) -> Result { + Ok(match string_option(machine, options, name, &UNIT_NAMES)? { + None => UnitOption::Unset, + Some(text) if text == "auto" => UnitOption::Auto, + Some(text) => Unit::from_name(&text).map_or(UnitOption::Unset, UnitOption::Unit), + }) +} + +/// The argument of `round` and `total`: a string is shorthand for the one +/// required unit option. +pub(super) struct UnitArgument { + pub(super) options: Options, + shorthand: Option, +} + +/// Reads such an argument. `undefined` is not an empty options object here. +pub(super) fn unit_argument( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + if value == Value::UNDEFINED { + return Err(type_error(machine, "a unit or options object is required")); + } + if let Some(text) = machine.string_value(value) { + return Ok(UnitArgument { + options: None, + shorthand: Some(text.to_utf8_lossy()), + }); + } + Ok(UnitArgument { + options: get_options_object(machine, value)?, + shorthand: None, + }) +} + +impl UnitArgument { + /// The string form of the argument, for an option that is not a unit. + pub(super) fn shorthand(&self) -> Option<&str> { + self.shorthand.as_deref() + } + + /// The unit under `key`, unset when neither form supplies one. + pub(super) fn unit( + &self, + machine: &mut Machine<'_, H>, + key: &str, + ) -> Result { + let Some(text) = &self.shorthand else { + return unit_valued_option(machine, self.options, key); + }; + if text == "auto" { + return Ok(UnitOption::Auto); + } + match Unit::from_name(text) { + Some(unit) => Ok(UnitOption::Unit(unit)), + None => Err(range_error(machine, format!("{text} is not a valid unit"))), + } + } + + /// The same, for an operation that cannot proceed without it. + pub(super) fn required_unit( + &self, + machine: &mut Machine<'_, H>, + key: &str, + ) -> Result { + match self.unit(machine, key)? { + UnitOption::Unset => Err(range_error(machine, format!("{key} is required"))), + value => Ok(value), + } + } +} + +/// The units an option may name: one category, or both. +pub(super) type UnitGroup = Option; +pub(super) const DATE_UNITS: UnitGroup = Some(UnitCategory::Date); +pub(super) const TIME_UNITS: UnitGroup = Some(UnitCategory::Time); +pub(super) const ALL_UNITS: UnitGroup = None; + +/// ValidateTemporalUnitValue. `extra` is a unit allowed outside the group. +pub(super) fn validate_unit_value( + machine: &mut Machine<'_, H>, + value: UnitOption, + group: UnitGroup, + extra: Option, +) -> Result<(), EvalFailure> { + if value == UnitOption::Unset || Some(value) == extra { + return Ok(()); + } + let valid = match value { + UnitOption::Unit(unit) => group.is_none_or(|category| category == unit.category()), + _ => false, + }; + if valid { + Ok(()) + } else { + Err(range_error(machine, "unit is not allowed in this context")) + } +} + +/// Result of GetDifferenceSettings. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub(super) struct DifferenceSettings { + pub(super) largest_unit: Unit, + pub(super) smallest_unit: Unit, + pub(super) rounding_increment: i128, + pub(super) rounding_mode: RoundingMode, +} + +/// What varies between the classes' `GetDifferenceSettings` calls. +#[derive(Clone, Copy)] +pub(super) struct DifferencePolicy<'a> { + pub(super) group: UnitGroup, + pub(super) disallowed: &'a [Unit], + pub(super) fallback_smallest: Unit, + pub(super) smallest_largest_default: Unit, +} + +/// GetDifferenceSettings. `since` negates the rounding mode: the operation +/// computes in the `until` direction and negates afterward. +pub(super) fn difference_settings( + machine: &mut Machine<'_, H>, + since: bool, + options: Options, + policy: DifferencePolicy<'_>, +) -> Result { + let largest = unit_valued_option(machine, options, "largestUnit")?; + let rounding_increment = rounding_increment_option(machine, options)?; + let mode = rounding_mode_option(machine, options, RoundingMode::Trunc)?; + let smallest = unit_valued_option(machine, options, "smallestUnit")?; + validate_unit_value(machine, largest, policy.group, Some(UnitOption::Auto))?; + if let UnitOption::Unit(unit) = largest + && policy.disallowed.contains(&unit) + { + return Err(range_error( + machine, + format!("largestUnit {} is not allowed here", unit.name()), + )); + } + validate_unit_value(machine, smallest, policy.group, None)?; + let smallest_unit = match smallest { + UnitOption::Unit(unit) => unit, + _ => policy.fallback_smallest, + }; + if policy.disallowed.contains(&smallest_unit) { + return Err(range_error( + machine, + format!("smallestUnit {} is not allowed here", smallest_unit.name()), + )); + } + let default_largest = policy.smallest_largest_default.larger(smallest_unit); + let largest_unit = match largest { + UnitOption::Unit(unit) => unit, + _ => default_largest, + }; + if largest_unit.larger(smallest_unit) != largest_unit { + return Err(range_error( + machine, + "smallestUnit must not be larger than largestUnit", + )); + } + if smallest_unit.max_rounding_increment().is_some() { + validate_rounding_increment(smallest_unit, rounding_increment, false).or_throw(machine)?; + } + Ok(DifferenceSettings { + largest_unit, + smallest_unit, + rounding_increment, + rounding_mode: if since { mode.negated() } else { mode }, + }) +} + +/// ToSecondsStringPrecisionRecord. `smallestUnit` takes precedence, and a +/// `fractionalSecondDigits` it overrides has already been read and validated. +pub(super) fn seconds_precision( + machine: &mut Machine<'_, H>, + smallest: UnitOption, + digits: FractionalSecondDigits, +) -> Result { + let unit = match smallest { + UnitOption::Unit(unit) => Some(unit), + _ => None, + }; + SecondsStringPrecision::resolve(unit, digits).or_throw(machine) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/plain_date.rs b/crates/bamts-runtime/src/builtins/temporal/plain_date.rs new file mode 100644 index 00000000..33f545b4 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/plain_date.rs @@ -0,0 +1,417 @@ +//! `Temporal.PlainDate`. + +use bamts_native::Value; + +use super::super::intl::temporal_to_locale_string; +use super::fields::{ + CalendarFields, FieldNames, calendar_argument, date_from_fields, iso_year, + prepare_partial_fields, providers, require_partial_temporal_object, resolved_date_fields, + to_calendar_identifier, to_integer_with_truncation, to_temporal_date, to_temporal_duration, + to_temporal_time, to_time_zone_identifier, +}; +use super::options::{ + DATE_UNITS, DifferencePolicy, difference_settings, get_options_object, overflow_option, + show_calendar_option, +}; +use super::{ + Class, OrThrow, arg, construct, create, install_class, is_object, number_outcome, range_error, + require_new, string_outcome, temporal_record, type_error, value_of_throws, +}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::calendar::{self, Calendar}; +use crate::temporal::instant_duration::{Instant, Overflow, Unit}; +use crate::temporal::now_rounding::{ShowCalendar, format_calendar_annotation}; +use crate::temporal::plain_types::{PlainDate, PlainDateTime, PlainTime}; +use crate::temporal::relative::{ + DifferenceRounding, date_duration_without_time, difference_plain_date, +}; +use crate::temporal::zoned_date_time::{ + CalendarDateFields, CalendarId, FixedOffsetTimeZones, ZonedDateTime, epoch_nanoseconds_for, + start_of_day, +}; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: TemporalKind::PlainDate, + name: "PlainDate", + tag: "Temporal.PlainDate", + length: 3, + constructor: constructor::, + statics: &[("from", 1, from::), ("compare", 2, compare::)], + getters: &date_getter_table!(H), + methods: &[ + ("toPlainYearMonth", 0, to_plain_year_month::), + ("toPlainMonthDay", 0, to_plain_month_day::), + ("add", 1, add::), + ("subtract", 1, subtract::), + ("with", 1, with::), + ("withCalendar", 1, with_calendar::), + ("until", 1, until::), + ("since", 1, since::), + ("equals", 1, equals::), + ("toPlainDateTime", 0, to_plain_date_time::), + ("toZonedDateTime", 1, to_zoned_date_time::), + ("toString", 0, to_string::), + ("toLocaleString", 0, to_locale_string::), + ("toJSON", 0, to_json::), + ("valueOf", 0, value_of_throws::), + ], + }, + ); +} + +fn this_date( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result<(PlainDate, CalendarId), EvalFailure> { + match temporal_record(machine, this)? { + Some(TemporalObject::PlainDate { value, calendar }) => Ok((value, calendar)), + _ => Err(type_error(machine, "receiver is not a Temporal.PlainDate")), + } +} + +/// The date behind the shared calendar accessors. +fn accessor_date( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result<(PlainDate, CalendarId), EvalFailure> { + this_date(machine, this) +} + +date_accessors!(accessor_date); + +fn date_outcome( + machine: &mut Machine<'_, H>, + value: PlainDate, + calendar: CalendarId, +) -> Result { + create(machine, TemporalObject::PlainDate { value, calendar }).map(BuiltinOutcome::Value) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "PlainDate")?; + let year = to_integer_with_truncation(machine, arg(args, 0))?; + let month = to_integer_with_truncation(machine, arg(args, 1))?; + let day = to_integer_with_truncation(machine, arg(args, 2))?; + let calendar = calendar_argument(machine, arg(args, 3))?; + let year = iso_year(machine, year)?; + let value = + PlainDate::new(year, month as i64, day as i64, Overflow::Reject).or_throw(machine)?; + construct(machine, TemporalObject::PlainDate { value, calendar }) +} + +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = to_temporal_date(machine, arg(args, 0), arg(args, 1))?; + date_outcome(machine, value, calendar) +} + +fn compare( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (one, _) = to_temporal_date(machine, arg(args, 0), Value::UNDEFINED)?; + let (two, _) = to_temporal_date(machine, arg(args, 1), Value::UNDEFINED)?; + number_outcome(f64::from(one.cmp(&two) as i8)) +} + +fn to_plain_year_month( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let value = match Calendar::from_id(&calendar) { + Some(kind) => calendar::year_month_of(kind, date).or_throw(machine)?, + None => date.to_plain_year_month(), + }; + create(machine, TemporalObject::PlainYearMonth { value, calendar }).map(BuiltinOutcome::Value) +} + +fn to_plain_month_day( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let value = match Calendar::from_id(&calendar) { + Some(kind) => calendar::month_day_of(kind, date).or_throw(machine)?, + None => date.to_plain_month_day(), + }; + create(machine, TemporalObject::PlainMonthDay { value, calendar }).map(BuiltinOutcome::Value) +} + +/// AddDurationToDate: time folds into whole days first. +fn add_duration( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + subtract: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let duration = to_temporal_duration(machine, arg(args, 0))?; + let date_duration = date_duration_without_time(&duration).or_throw(machine)?; + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let date_duration = if subtract { + date_duration.negated() + } else { + date_duration + }; + let value = providers() + .calendar + .date_add(&calendar, date, &date_duration, overflow) + .or_throw(machine)?; + date_outcome(machine, value, calendar) +} + +fn add( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + add_duration(machine, this, args, false) +} + +fn subtract( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + add_duration(machine, this, args, true) +} + +fn with( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let like = arg(args, 0); + require_partial_temporal_object(machine, like)?; + let partial = prepare_partial_fields(machine, &calendar, like, FieldNames::DATE)?; + let fields = CalendarFields::from_date(machine, &calendar, date)?.merged_with(partial); + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let value = if calendar.is_iso8601() { + let (year, month, day) = resolved_date_fields(machine, &fields)?; + date.with(Some(year), Some(month), Some(day), overflow) + .or_throw(machine)? + } else { + date_from_fields(machine, &calendar, &fields, overflow)? + }; + date_outcome(machine, value, calendar) +} + +fn with_calendar( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date, _) = this_date(machine, this)?; + let calendar = to_calendar_identifier(machine, arg(args, 0))?; + date_outcome(machine, date, calendar) +} + +/// DifferenceTemporalPlainDate. +fn difference( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + since: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let (other, other_calendar) = to_temporal_date(machine, arg(args, 0), Value::UNDEFINED)?; + if calendar != other_calendar { + return Err(range_error(machine, "calendars must match")); + } + let options = get_options_object(machine, arg(args, 1))?; + let settings = difference_settings( + machine, + since, + options, + DifferencePolicy { + group: DATE_UNITS, + disallowed: &[], + fallback_smallest: Unit::Day, + smallest_largest_default: Unit::Day, + }, + )?; + let value = if calendar.is_iso8601() { + date.until( + other, + settings.largest_unit, + settings.smallest_unit, + settings.rounding_increment, + settings.rounding_mode, + ) + } else { + difference_plain_date( + &providers(), + date, + other, + &calendar, + DifferenceRounding { + largest_unit: settings.largest_unit, + smallest_unit: settings.smallest_unit, + increment: settings.rounding_increment, + mode: settings.rounding_mode, + }, + ) + } + .or_throw(machine)?; + let value = if since { value.negated() } else { value }; + create(machine, TemporalObject::Duration(value)).map(BuiltinOutcome::Value) +} + +fn until( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + difference(machine, this, args, false) +} + +fn since( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + difference(machine, this, args, true) +} + +fn equals( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let (other, other_calendar) = to_temporal_date(machine, arg(args, 0), Value::UNDEFINED)?; + let equal = date == other && calendar == other_calendar; + Ok(BuiltinOutcome::Value(Value::boolean(equal))) +} + +fn to_plain_date_time( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let time = arg(args, 0); + let time = if time == Value::UNDEFINED { + PlainTime::midnight() + } else { + to_temporal_time(machine, time, Value::UNDEFINED)? + }; + let value = date.to_plain_date_time(time).or_throw(machine)?; + create(machine, TemporalObject::PlainDateTime { value, calendar }).map(BuiltinOutcome::Value) +} + +fn to_zoned_date_time( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let item = arg(args, 0); + let mut plain_time = Value::UNDEFINED; + let time_zone = if is_object(machine, item)? { + let like = machine.get_named_property(item, "timeZone")?; + if like == Value::UNDEFINED { + to_time_zone_identifier(machine, item)? + } else { + let time_zone = to_time_zone_identifier(machine, like)?; + plain_time = machine.get_named_property(item, "plainTime")?; + time_zone + } + } else { + to_time_zone_identifier(machine, item)? + }; + let instant = if plain_time == Value::UNDEFINED { + start_of_day(&FixedOffsetTimeZones, &time_zone, date).or_throw(machine)? + } else { + let time = to_temporal_time(machine, plain_time, Value::UNDEFINED)?; + let local = PlainDateTime::from_parts(date, time).or_throw(machine)?; + let nanoseconds = + epoch_nanoseconds_for(&FixedOffsetTimeZones, &time_zone, local).or_throw(machine)?; + Instant::from_epoch_nanoseconds(nanoseconds).or_throw(machine)? + }; + let value = ZonedDateTime::from_instant(instant, time_zone, calendar); + create(machine, TemporalObject::ZonedDateTime(value)).map(BuiltinOutcome::Value) +} + +fn date_text( + machine: &mut Machine<'_, H>, + date: PlainDate, + calendar: &CalendarId, + show: ShowCalendar, +) -> Result { + let annotation = format_calendar_annotation(calendar.as_str(), show).or_throw(machine)?; + Ok(format!("{}{annotation}", date.format())) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let options = get_options_object(machine, arg(args, 0))?; + let show = show_calendar_option(machine, options)?; + let text = date_text(machine, date, &calendar, show)?; + string_outcome(machine, &text) +} + +fn to_json( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (date, calendar) = this_date(machine, this)?; + let text = date_text(machine, date, &calendar, ShowCalendar::Auto)?; + string_outcome(machine, &text) +} + +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + this_date(machine, this)?; + temporal_to_locale_string(machine, this, arg(args, 0), arg(args, 1)) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/plain_date_time.rs b/crates/bamts-runtime/src/builtins/temporal/plain_date_time.rs new file mode 100644 index 00000000..69eb85a0 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/plain_date_time.rs @@ -0,0 +1,508 @@ +//! `Temporal.PlainDateTime`. + +use bamts_native::Value; + +use super::super::intl::temporal_to_locale_string; +use super::fields::{ + CalendarFields, FieldNames, calendar_argument, date_time_from_fields, iso_year, + prepare_partial_fields, providers, require_partial_temporal_object, to_calendar_identifier, + to_integer_with_truncation, to_temporal_date_time, to_temporal_duration, to_temporal_time, + to_time_zone_identifier, +}; +use super::options::{ + ALL_UNITS, DifferencePolicy, TIME_UNITS, UnitOption, difference_settings, + disambiguation_option, fractional_second_digits_option, get_options_object, overflow_option, + rounding_increment_option, rounding_mode_option, show_calendar_option, unit_argument, + unit_valued_option, validate_unit_value, +}; +use super::{ + Class, Getter, OrThrow, arg, construct, create, install_class, number_outcome, range_error, + require_new, string_outcome, temporal_record, type_error, value_of_throws, +}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::instant_duration::{ + Overflow, Precision, RoundingMode, Unit, validate_rounding_increment, +}; +use crate::temporal::now_rounding::{ + ShowCalendar, ToStringRoundingOptions, serialize_plain_date_time, +}; +use crate::temporal::plain_types::{PlainDate, PlainDateTime, PlainTime}; +use crate::temporal::relative::DifferenceRounding; +use crate::temporal::zoned_date_time::{ + CalendarDateFields, CalendarId, ResolveOptions, ZonedDateTime, +}; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + let date_getters: [Getter; 16] = date_getter_table!(H); + let time_getters: [Getter; 6] = [ + ("hour", "get hour", hour::), + ("minute", "get minute", minute::), + ("second", "get second", second::), + ("millisecond", "get millisecond", millisecond::), + ("microsecond", "get microsecond", microsecond::), + ("nanosecond", "get nanosecond", nanosecond::), + ]; + let getters: Vec> = date_getters.iter().chain(&time_getters).copied().collect(); + install_class( + heap, + builtins, + namespace, + &Class { + kind: TemporalKind::PlainDateTime, + name: "PlainDateTime", + tag: "Temporal.PlainDateTime", + length: 3, + constructor: constructor::, + statics: &[("from", 1, from::), ("compare", 2, compare::)], + getters: &getters, + methods: &[ + ("with", 1, with::), + ("withPlainTime", 0, with_plain_time::), + ("withCalendar", 1, with_calendar::), + ("add", 1, add::), + ("subtract", 1, subtract::), + ("until", 1, until::), + ("since", 1, since::), + ("round", 1, round::), + ("equals", 1, equals::), + ("toString", 0, to_string::), + ("toLocaleString", 0, to_locale_string::), + ("toJSON", 0, to_json::), + ("valueOf", 0, value_of_throws::), + ("toZonedDateTime", 1, to_zoned_date_time::), + ("toPlainDate", 0, to_plain_date::), + ("toPlainTime", 0, to_plain_time::), + ], + }, + ); +} + +fn this_date_time( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result<(PlainDateTime, CalendarId), EvalFailure> { + match temporal_record(machine, this)? { + Some(TemporalObject::PlainDateTime { value, calendar }) => Ok((value, calendar)), + _ => Err(type_error( + machine, + "receiver is not a Temporal.PlainDateTime", + )), + } +} + +/// The date behind the shared calendar accessors. +fn accessor_date( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result<(PlainDate, CalendarId), EvalFailure> { + let (value, calendar) = this_date_time(machine, this)?; + Ok((value.date, calendar)) +} + +date_accessors!(accessor_date); + +macro_rules! time_getter { + ($($name:ident),+) => { + $(fn $name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + number_outcome(f64::from(this_date_time(machine, this)?.0.time.$name)) + })+ + }; +} + +time_getter!(hour, minute, second, millisecond, microsecond, nanosecond); + +fn date_time_outcome( + machine: &mut Machine<'_, H>, + value: PlainDateTime, + calendar: CalendarId, +) -> Result { + create(machine, TemporalObject::PlainDateTime { value, calendar }).map(BuiltinOutcome::Value) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "PlainDateTime")?; + let year = to_integer_with_truncation(machine, arg(args, 0))?; + let month = to_integer_with_truncation(machine, arg(args, 1))?; + let day = to_integer_with_truncation(machine, arg(args, 2))?; + let mut time = [0_f64; 6]; + for (index, field) in time.iter_mut().enumerate() { + let value = arg(args, index + 3); + if value != Value::UNDEFINED { + *field = to_integer_with_truncation(machine, value)?; + } + } + let calendar = calendar_argument(machine, arg(args, 9))?; + let year = iso_year(machine, year)?; + let value = PlainDateTime::new( + year, + month as i64, + day as i64, + time.map(|field| field as i64), + Overflow::Reject, + ) + .or_throw(machine)?; + construct(machine, TemporalObject::PlainDateTime { value, calendar }) +} + +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = to_temporal_date_time(machine, arg(args, 0), arg(args, 1))?; + date_time_outcome(machine, value, calendar) +} + +fn compare( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (one, _) = to_temporal_date_time(machine, arg(args, 0), Value::UNDEFINED)?; + let (two, _) = to_temporal_date_time(machine, arg(args, 1), Value::UNDEFINED)?; + number_outcome(f64::from(one.cmp(&two) as i8)) +} + +fn with( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let like = arg(args, 0); + require_partial_temporal_object(machine, like)?; + let names = FieldNames::DATE.with(FieldNames::TIME); + let partial = prepare_partial_fields(machine, &calendar, like, names)?; + let fields = CalendarFields::from_date(machine, &calendar, date_time.date)? + .with_time(date_time.time) + .merged_with(partial); + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let value = date_time_from_fields(machine, &calendar, &fields, overflow)?; + let value = PlainDateTime::from_parts(value.date, value.time).or_throw(machine)?; + date_time_outcome(machine, value, calendar) +} + +fn with_plain_time( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let like = arg(args, 0); + let time = if like == Value::UNDEFINED { + PlainTime::midnight() + } else { + to_temporal_time(machine, like, Value::UNDEFINED)? + }; + let value = PlainDateTime::from_parts(date_time.date, time).or_throw(machine)?; + date_time_outcome(machine, value, calendar) +} + +fn with_calendar( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, _) = this_date_time(machine, this)?; + let calendar = to_calendar_identifier(machine, arg(args, 0))?; + date_time_outcome(machine, date_time, calendar) +} + +/// AddDurationToDateTime. +fn add_duration( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + subtract: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let duration = to_temporal_duration(machine, arg(args, 0))?; + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let value = if calendar.is_iso8601() { + if subtract { + date_time.subtract(&duration, overflow) + } else { + date_time.add(&duration, overflow) + } + } else { + let duration = if subtract { + duration.negated() + } else { + duration + }; + date_time.add_in(providers().calendar, &calendar, &duration, overflow) + } + .or_throw(machine)?; + date_time_outcome(machine, value, calendar) +} + +fn add( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + add_duration(machine, this, args, false) +} + +fn subtract( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + add_duration(machine, this, args, true) +} + +/// DifferenceTemporalPlainDateTime. +fn difference( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + since: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let (other, other_calendar) = to_temporal_date_time(machine, arg(args, 0), Value::UNDEFINED)?; + if calendar != other_calendar { + return Err(range_error(machine, "calendars must match")); + } + let options = get_options_object(machine, arg(args, 1))?; + let settings = difference_settings( + machine, + since, + options, + DifferencePolicy { + group: ALL_UNITS, + disallowed: &[], + fallback_smallest: Unit::Nanosecond, + smallest_largest_default: Unit::Day, + }, + )?; + let value = if calendar.is_iso8601() { + date_time.until( + other, + settings.largest_unit, + settings.smallest_unit, + settings.rounding_increment, + settings.rounding_mode, + ) + } else { + date_time.until_in( + &providers(), + &calendar, + other, + DifferenceRounding { + largest_unit: settings.largest_unit, + smallest_unit: settings.smallest_unit, + increment: settings.rounding_increment, + mode: settings.rounding_mode, + }, + ) + } + .or_throw(machine)?; + let value = if since { value.negated() } else { value }; + create(machine, TemporalObject::Duration(value)).map(BuiltinOutcome::Value) +} + +fn until( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + difference(machine, this, args, false) +} + +fn since( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + difference(machine, this, args, true) +} + +fn round( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let round_to = unit_argument(machine, arg(args, 0))?; + let increment = rounding_increment_option(machine, round_to.options)?; + let mode = rounding_mode_option(machine, round_to.options, RoundingMode::HalfExpand)?; + let smallest = round_to.required_unit(machine, "smallestUnit")?; + validate_unit_value( + machine, + smallest, + TIME_UNITS, + Some(UnitOption::Unit(Unit::Day)), + )?; + let UnitOption::Unit(unit) = smallest else { + return Err(range_error(machine, "smallestUnit is required")); + }; + if unit == Unit::Day { + if increment != 1 { + return Err(range_error( + machine, + "roundingIncrement must be 1 when rounding to day", + )); + } + } else { + validate_rounding_increment(unit, increment, false).or_throw(machine)?; + } + let value = date_time.round(unit, increment, mode).or_throw(machine)?; + date_time_outcome(machine, value, calendar) +} + +fn equals( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let (other, other_calendar) = to_temporal_date_time(machine, arg(args, 0), Value::UNDEFINED)?; + let equal = date_time == other && calendar == other_calendar; + Ok(BuiltinOutcome::Value(Value::boolean(equal))) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let options = get_options_object(machine, arg(args, 0))?; + let show = show_calendar_option(machine, options)?; + let digits = fractional_second_digits_option(machine, options)?; + let rounding_mode = rounding_mode_option(machine, options, RoundingMode::Trunc)?; + let smallest = unit_valued_option(machine, options, "smallestUnit")?; + validate_unit_value(machine, smallest, TIME_UNITS, None)?; + if smallest == UnitOption::Unit(Unit::Hour) { + return Err(range_error(machine, "smallestUnit must not be hour")); + } + let smallest_unit = match smallest { + UnitOption::Unit(unit) => Some(unit), + _ => None, + }; + let text = serialize_plain_date_time( + &date_time, + calendar.as_str(), + show, + ToStringRoundingOptions { + smallest_unit, + fractional_second_digits: digits, + rounding_mode, + }, + ) + .or_throw(machine)?; + string_outcome(machine, &text) +} + +fn to_json( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let text = crate::temporal::now_rounding::format_calendar_annotation( + calendar.as_str(), + ShowCalendar::Auto, + ) + .or_throw(machine)?; + let text = format!("{}{text}", date_time.format(Precision::Auto)); + string_outcome(machine, &text) +} + +fn to_zoned_date_time( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + let time_zone = to_time_zone_identifier(machine, arg(args, 0))?; + let options = get_options_object(machine, arg(args, 1))?; + let disambiguation = disambiguation_option(machine, options)?; + let value = ZonedDateTime::from_local( + &providers(), + date_time, + time_zone, + calendar, + None, + ResolveOptions { + disambiguation, + ..ResolveOptions::default() + }, + ) + .or_throw(machine)?; + create(machine, TemporalObject::ZonedDateTime(value)).map(BuiltinOutcome::Value) +} + +fn to_plain_date( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, calendar) = this_date_time(machine, this)?; + create( + machine, + TemporalObject::PlainDate { + value: date_time.to_plain_date(), + calendar, + }, + ) + .map(BuiltinOutcome::Value) +} + +fn to_plain_time( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (date_time, _) = this_date_time(machine, this)?; + create( + machine, + TemporalObject::PlainTime(date_time.to_plain_time()), + ) + .map(BuiltinOutcome::Value) +} + +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + this_date_time(machine, this)?; + temporal_to_locale_string(machine, this, arg(args, 0), arg(args, 1)) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/plain_month_day.rs b/crates/bamts-runtime/src/builtins/temporal/plain_month_day.rs new file mode 100644 index 00000000..9c8b0a48 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/plain_month_day.rs @@ -0,0 +1,262 @@ +//! `Temporal.PlainMonthDay`. + +use bamts_native::Value; + +use super::super::intl::temporal_to_locale_string; +use super::fields::{ + CalendarFields, FieldNames, calendar_argument, calendar_date, date_from_fields, iso_year, + month_day_from_fields, prepare_calendar_fields, prepare_partial_fields, + require_partial_temporal_object, to_integer_with_truncation, to_temporal_month_day, +}; +use super::options::{get_options_object, overflow_option, show_calendar_option}; +use super::{ + Class, OrThrow, arg, construct, create, install_class, is_object, number_outcome, require_new, + string_outcome, temporal_record, type_error, value_of_throws, +}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::instant_duration::{Overflow, format_iso_date}; +use crate::temporal::now_rounding::{ShowCalendar, format_calendar_annotation}; +use crate::temporal::plain_types::{MONTH_DAY_REFERENCE_YEAR, PlainMonthDay}; +use crate::temporal::zoned_date_time::CalendarId; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: TemporalKind::PlainMonthDay, + name: "PlainMonthDay", + tag: "Temporal.PlainMonthDay", + length: 2, + constructor: constructor::, + statics: &[("from", 1, from::)], + getters: &[ + ("calendarId", "get calendarId", calendar_id::), + ("monthCode", "get monthCode", month_code::), + ("day", "get day", day::), + ], + methods: &[ + ("with", 1, with::), + ("equals", 1, equals::), + ("toString", 0, to_string::), + ("toLocaleString", 0, to_locale_string::), + ("toJSON", 0, to_json::), + ("valueOf", 0, value_of_throws::), + ("toPlainDate", 1, to_plain_date::), + ], + }, + ); +} + +fn this_month_day( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result<(PlainMonthDay, CalendarId), EvalFailure> { + match temporal_record(machine, this)? { + Some(TemporalObject::PlainMonthDay { value, calendar }) => Ok((value, calendar)), + _ => Err(type_error( + machine, + "receiver is not a Temporal.PlainMonthDay", + )), + } +} + +fn month_day_outcome( + machine: &mut Machine<'_, H>, + value: PlainMonthDay, + calendar: CalendarId, +) -> Result { + create(machine, TemporalObject::PlainMonthDay { value, calendar }).map(BuiltinOutcome::Value) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "PlainMonthDay")?; + let month = to_integer_with_truncation(machine, arg(args, 0))?; + let day = to_integer_with_truncation(machine, arg(args, 1))?; + let calendar = calendar_argument(machine, arg(args, 2))?; + let reference = arg(args, 3); + let year = if reference == Value::UNDEFINED { + f64::from(MONTH_DAY_REFERENCE_YEAR) + } else { + to_integer_with_truncation(machine, reference)? + }; + let year = iso_year(machine, year)?; + let value = + PlainMonthDay::with_reference_year(month as i64, day as i64, year, Overflow::Reject) + .or_throw(machine)?; + construct(machine, TemporalObject::PlainMonthDay { value, calendar }) +} + +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = to_temporal_month_day(machine, arg(args, 0), arg(args, 1))?; + month_day_outcome(machine, value, calendar) +} + +fn calendar_id( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (_, calendar) = this_month_day(machine, this)?; + string_outcome(machine, calendar.as_str()) +} + +fn month_code( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_month_day(machine, this)?; + if calendar.is_iso8601() { + return string_outcome(machine, &value.month_code()); + } + let fields = calendar_date(machine, &calendar, value.iso_date())?; + string_outcome(machine, &fields.month_code) +} + +fn day( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_month_day(machine, this)?; + let fields = calendar_date(machine, &calendar, value.iso_date())?; + number_outcome(f64::from(fields.day)) +} + +fn with( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_month_day(machine, this)?; + let like = arg(args, 0); + require_partial_temporal_object(machine, like)?; + let partial = prepare_partial_fields(machine, &calendar, like, FieldNames::DATE)?; + let fields = CalendarFields::from_month_day(machine, &calendar, value)?.merged_with(partial); + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let result = month_day_from_fields(machine, &calendar, &fields, overflow)?; + month_day_outcome(machine, result, calendar) +} + +fn equals( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_month_day(machine, this)?; + let (other, other_calendar) = to_temporal_month_day(machine, arg(args, 0), Value::UNDEFINED)?; + let equal = value.equals(&other) && calendar == other_calendar; + Ok(BuiltinOutcome::Value(Value::boolean(equal))) +} + +/// TemporalMonthDayToString: the reference year is only shown when a calendar +/// annotation makes it part of the round trip. +fn month_day_text( + machine: &mut Machine<'_, H>, + value: PlainMonthDay, + calendar: &CalendarId, + show: ShowCalendar, +) -> Result { + let body = if matches!(show, ShowCalendar::Always | ShowCalendar::Critical) + || !calendar.is_iso8601() + { + format_iso_date(value.reference_year(), value.month, value.day) + } else { + value.format() + }; + let annotation = format_calendar_annotation(calendar.as_str(), show).or_throw(machine)?; + Ok(format!("{body}{annotation}")) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_month_day(machine, this)?; + let options = get_options_object(machine, arg(args, 0))?; + let show = show_calendar_option(machine, options)?; + let text = month_day_text(machine, value, &calendar, show)?; + string_outcome(machine, &text) +} + +fn to_json( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_month_day(machine, this)?; + let text = month_day_text(machine, value, &calendar, ShowCalendar::Auto)?; + string_outcome(machine, &text) +} + +fn to_plain_date( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_month_day(machine, this)?; + let item = arg(args, 0); + if !is_object(machine, item)? { + return Err(type_error(machine, "argument must be an object")); + } + let input = + prepare_calendar_fields(machine, &calendar, item, FieldNames::YEAR, FieldNames::NONE)?; + let date = if calendar.is_iso8601() { + let Some(year) = input.year else { + return Err(type_error(machine, "year is required")); + }; + let year = iso_year(machine, year)?; + value + .to_plain_date(year, Overflow::Constrain) + .or_throw(machine)? + } else { + let fields = CalendarFields::from_month_day(machine, &calendar, value)?.merged_with(input); + date_from_fields(machine, &calendar, &fields, Overflow::Constrain)? + }; + create( + machine, + TemporalObject::PlainDate { + value: date, + calendar, + }, + ) + .map(BuiltinOutcome::Value) +} + +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + this_month_day(machine, this)?; + temporal_to_locale_string(machine, this, arg(args, 0), arg(args, 1)) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/plain_time.rs b/crates/bamts-runtime/src/builtins/temporal/plain_time.rs new file mode 100644 index 00000000..c6c8d4a2 --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/plain_time.rs @@ -0,0 +1,323 @@ +//! `Temporal.PlainTime`. + +use bamts_native::Value; + +use super::super::intl::temporal_to_locale_string; +use super::fields::{ + FieldNames, prepare_partial_fields, require_partial_temporal_object, time_from_fields, + to_integer_with_truncation, to_temporal_duration, to_temporal_time, +}; +use super::options::{ + DifferencePolicy, TIME_UNITS, UnitOption, difference_settings, fractional_second_digits_option, + get_options_object, overflow_option, rounding_increment_option, rounding_mode_option, + unit_argument, unit_valued_option, validate_unit_value, +}; +use super::{ + Class, OrThrow, arg, construct, create, install_class, number_outcome, range_error, + require_new, string_outcome, temporal_record, type_error, value_of_throws, +}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::instant_duration::{ + Overflow, Precision, RoundingMode, Unit, validate_rounding_increment, +}; +use crate::temporal::now_rounding::{ToStringRoundingOptions, serialize_plain_time}; +use crate::temporal::plain_types::PlainTime; +use crate::temporal::zoned_date_time::CalendarId; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: TemporalKind::PlainTime, + name: "PlainTime", + tag: "Temporal.PlainTime", + length: 0, + constructor: constructor::, + statics: &[("from", 1, from::), ("compare", 2, compare::)], + getters: &[ + ("hour", "get hour", hour::), + ("minute", "get minute", minute::), + ("second", "get second", second::), + ("millisecond", "get millisecond", millisecond::), + ("microsecond", "get microsecond", microsecond::), + ("nanosecond", "get nanosecond", nanosecond::), + ], + methods: &[ + ("add", 1, add::), + ("subtract", 1, subtract::), + ("with", 1, with::), + ("until", 1, until::), + ("since", 1, since::), + ("round", 1, round::), + ("equals", 1, equals::), + ("toString", 0, to_string::), + ("toLocaleString", 0, to_locale_string::), + ("toJSON", 0, to_json::), + ("valueOf", 0, value_of_throws::), + ], + }, + ); +} + +fn this_time(machine: &mut Machine<'_, H>, this: Value) -> Result { + match temporal_record(machine, this)? { + Some(TemporalObject::PlainTime(value)) => Ok(value), + _ => Err(type_error(machine, "receiver is not a Temporal.PlainTime")), + } +} + +fn time_outcome( + machine: &mut Machine<'_, H>, + value: PlainTime, +) -> Result { + create(machine, TemporalObject::PlainTime(value)).map(BuiltinOutcome::Value) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "PlainTime")?; + let mut fields = [0_f64; 6]; + for (index, field) in fields.iter_mut().enumerate() { + let value = arg(args, index); + if value != Value::UNDEFINED { + *field = to_integer_with_truncation(machine, value)?; + } + } + let [hour, minute, second, millisecond, microsecond, nanosecond] = fields.map(|v| v as i64); + let value = PlainTime::new( + hour, + minute, + second, + millisecond, + microsecond, + nanosecond, + Overflow::Reject, + ) + .or_throw(machine)?; + construct(machine, TemporalObject::PlainTime(value)) +} + +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = to_temporal_time(machine, arg(args, 0), arg(args, 1))?; + time_outcome(machine, value) +} + +fn compare( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let one = to_temporal_time(machine, arg(args, 0), Value::UNDEFINED)?; + let two = to_temporal_time(machine, arg(args, 1), Value::UNDEFINED)?; + number_outcome(f64::from(one.cmp(&two) as i8)) +} + +macro_rules! field_getter { + ($($name:ident),+) => { + $(fn $name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + number_outcome(f64::from(this_time(machine, this)?.$name)) + })+ + }; +} + +field_getter!(hour, minute, second, millisecond, microsecond, nanosecond); + +fn add( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time = this_time(machine, this)?; + let duration = to_temporal_duration(machine, arg(args, 0))?; + time_outcome(machine, time.add(&duration)) +} + +fn subtract( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time = this_time(machine, this)?; + let duration = to_temporal_duration(machine, arg(args, 0))?; + time_outcome(machine, time.subtract(&duration)) +} + +fn with( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time = this_time(machine, this)?; + let like = arg(args, 0); + require_partial_temporal_object(machine, like)?; + let partial = prepare_partial_fields(machine, &CalendarId::iso8601(), like, FieldNames::TIME)?; + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let fields = super::fields::CalendarFields::default() + .with_time(time) + .merged_with(partial); + let value = time_from_fields(machine, &fields, overflow)?; + time_outcome(machine, value) +} + +/// DifferenceTemporalPlainTime. +fn difference( + machine: &mut Machine<'_, H>, + since: bool, + time: PlainTime, + other: Value, + options: Value, +) -> Result { + let other = to_temporal_time(machine, other, Value::UNDEFINED)?; + let options = get_options_object(machine, options)?; + let settings = difference_settings( + machine, + since, + options, + DifferencePolicy { + group: TIME_UNITS, + disallowed: &[], + fallback_smallest: Unit::Nanosecond, + smallest_largest_default: Unit::Hour, + }, + )?; + let value = time + .until( + other, + settings.largest_unit, + settings.smallest_unit, + settings.rounding_increment, + settings.rounding_mode, + ) + .or_throw(machine)?; + let value = if since { value.negated() } else { value }; + create(machine, TemporalObject::Duration(value)).map(BuiltinOutcome::Value) +} + +fn until( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time = this_time(machine, this)?; + difference(machine, false, time, arg(args, 0), arg(args, 1)) +} + +fn since( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time = this_time(machine, this)?; + difference(machine, true, time, arg(args, 0), arg(args, 1)) +} + +fn round( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time = this_time(machine, this)?; + let round_to = unit_argument(machine, arg(args, 0))?; + let increment = rounding_increment_option(machine, round_to.options)?; + let mode = rounding_mode_option(machine, round_to.options, RoundingMode::HalfExpand)?; + let smallest = round_to.required_unit(machine, "smallestUnit")?; + validate_unit_value(machine, smallest, TIME_UNITS, None)?; + let UnitOption::Unit(unit) = smallest else { + return Err(range_error(machine, "smallestUnit is required")); + }; + validate_rounding_increment(unit, increment, false).or_throw(machine)?; + let value = time.round(unit, increment, mode).or_throw(machine)?; + time_outcome(machine, value) +} + +fn equals( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time = this_time(machine, this)?; + let other = to_temporal_time(machine, arg(args, 0), Value::UNDEFINED)?; + Ok(BuiltinOutcome::Value(Value::boolean(time == other))) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let time = this_time(machine, this)?; + let options = get_options_object(machine, arg(args, 0))?; + let digits = fractional_second_digits_option(machine, options)?; + let rounding_mode = rounding_mode_option(machine, options, RoundingMode::Trunc)?; + let smallest = unit_valued_option(machine, options, "smallestUnit")?; + validate_unit_value(machine, smallest, TIME_UNITS, None)?; + if smallest == UnitOption::Unit(Unit::Hour) { + return Err(range_error(machine, "smallestUnit must not be hour")); + } + let smallest_unit = match smallest { + UnitOption::Unit(unit) => Some(unit), + _ => None, + }; + let text = serialize_plain_time( + &time, + ToStringRoundingOptions { + smallest_unit, + fractional_second_digits: digits, + rounding_mode, + }, + ) + .or_throw(machine)?; + string_outcome(machine, &text) +} + +fn to_json( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let text = this_time(machine, this)?.format(Precision::Auto); + string_outcome(machine, &text) +} + +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + this_time(machine, this)?; + temporal_to_locale_string(machine, this, arg(args, 0), arg(args, 1)) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/plain_year_month.rs b/crates/bamts-runtime/src/builtins/temporal/plain_year_month.rs new file mode 100644 index 00000000..4fafdbca --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/plain_year_month.rs @@ -0,0 +1,463 @@ +//! `Temporal.PlainYearMonth`. + +use bamts_native::Value; + +use super::super::intl::temporal_to_locale_string; +use super::fields::{ + CalendarFields, FieldNames, calendar_argument, calendar_date, date_from_fields, iso_year, + prepare_calendar_fields, prepare_partial_fields, providers, require_partial_temporal_object, + to_integer_with_truncation, to_temporal_duration, to_temporal_year_month, + year_month_from_fields, +}; +use super::options::{ + DATE_UNITS, DifferencePolicy, difference_settings, get_options_object, overflow_option, + show_calendar_option, +}; +use super::{ + Class, OrThrow, arg, construct, create, install_class, is_object, number_outcome, + optional_number_outcome, optional_string_outcome, range_error, require_new, string_outcome, + temporal_record, type_error, value_of_throws, +}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::calendar::{self, Calendar}; +use crate::temporal::instant_duration::{Duration, Overflow, Unit, format_iso_date}; +use crate::temporal::now_rounding::{ShowCalendar, format_calendar_annotation}; +use crate::temporal::plain_types::PlainYearMonth; +use crate::temporal::relative::{DifferenceRounding, difference_plain_year_month}; +use crate::temporal::zoned_date_time::{CalendarDateFields, CalendarId}; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + install_class( + heap, + builtins, + namespace, + &Class { + kind: TemporalKind::PlainYearMonth, + name: "PlainYearMonth", + tag: "Temporal.PlainYearMonth", + length: 2, + constructor: constructor::, + statics: &[("from", 1, from::), ("compare", 2, compare::)], + getters: &[ + ("calendarId", "get calendarId", calendar_id::), + ("era", "get era", era::), + ("eraYear", "get eraYear", era_year::), + ("year", "get year", year::), + ("month", "get month", month::), + ("monthCode", "get monthCode", month_code::), + ("daysInYear", "get daysInYear", days_in_year::), + ("daysInMonth", "get daysInMonth", days_in_month::), + ("monthsInYear", "get monthsInYear", months_in_year::), + ("inLeapYear", "get inLeapYear", in_leap_year::), + ], + methods: &[ + ("with", 1, with::), + ("add", 1, add::), + ("subtract", 1, subtract::), + ("until", 1, until::), + ("since", 1, since::), + ("equals", 1, equals::), + ("toString", 0, to_string::), + ("toLocaleString", 0, to_locale_string::), + ("toJSON", 0, to_json::), + ("valueOf", 0, value_of_throws::), + ("toPlainDate", 1, to_plain_date::), + ], + }, + ); +} + +fn this_year_month( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result<(PlainYearMonth, CalendarId), EvalFailure> { + match temporal_record(machine, this)? { + Some(TemporalObject::PlainYearMonth { value, calendar }) => Ok((value, calendar)), + _ => Err(type_error( + machine, + "receiver is not a Temporal.PlainYearMonth", + )), + } +} + +fn year_month_outcome( + machine: &mut Machine<'_, H>, + value: PlainYearMonth, + calendar: CalendarId, +) -> Result { + create(machine, TemporalObject::PlainYearMonth { value, calendar }).map(BuiltinOutcome::Value) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "PlainYearMonth")?; + let year = to_integer_with_truncation(machine, arg(args, 0))?; + let month = to_integer_with_truncation(machine, arg(args, 1))?; + let calendar = calendar_argument(machine, arg(args, 2))?; + let reference = arg(args, 3); + let day = if reference == Value::UNDEFINED { + 1.0 + } else { + to_integer_with_truncation(machine, reference)? + }; + let year = iso_year(machine, year)?; + if !(1.0..=12.0).contains(&month) || !(1.0..=31.0).contains(&day) { + return Err(range_error(machine, "invalid ISO month or reference day")); + } + let value = + PlainYearMonth::with_reference_day(year, month as u8, day as u8).or_throw(machine)?; + construct(machine, TemporalObject::PlainYearMonth { value, calendar }) +} + +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = to_temporal_year_month(machine, arg(args, 0), arg(args, 1))?; + year_month_outcome(machine, value, calendar) +} + +fn compare( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (one, _) = to_temporal_year_month(machine, arg(args, 0), Value::UNDEFINED)?; + let (two, _) = to_temporal_year_month(machine, arg(args, 1), Value::UNDEFINED)?; + number_outcome(f64::from(one.cmp(&two) as i8)) +} + +fn calendar_id( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (_, calendar) = this_year_month(machine, this)?; + string_outcome(machine, calendar.as_str()) +} + +/// The calendar's view of the receiver's date. +fn calendar_fields( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + calendar_date(machine, &calendar, value.iso_date()) +} + +fn era( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let fields = calendar_fields(machine, this)?; + optional_string_outcome(machine, fields.era.as_deref()) +} + +fn era_year( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let fields = calendar_fields(machine, this)?; + optional_number_outcome(fields.era_year.map(f64::from)) +} + +fn month_code( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let fields = calendar_fields(machine, this)?; + string_outcome(machine, &fields.month_code) +} + +fn in_leap_year( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let fields = calendar_fields(machine, this)?; + Ok(BuiltinOutcome::Value(Value::boolean(fields.in_leap_year))) +} + +macro_rules! number_getter { + ($($name:ident: |$fields:ident| $number:expr),+ $(,)?) => { + $(fn $name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let $fields = calendar_fields(machine, this)?; + number_outcome($number) + })+ + }; +} + +number_getter! { + year: |fields| f64::from(fields.year), + month: |fields| f64::from(fields.month), + days_in_year: |fields| f64::from(fields.days_in_year), + months_in_year: |fields| f64::from(fields.months_in_year), +} + +fn days_in_month( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + if calendar.is_iso8601() { + return number_outcome(f64::from(value.days_in_month())); + } + let fields = calendar_date(machine, &calendar, value.iso_date())?; + number_outcome(f64::from(fields.days_in_month)) +} + +fn with( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + let like = arg(args, 0); + require_partial_temporal_object(machine, like)?; + let partial = prepare_partial_fields(machine, &calendar, like, FieldNames::YEAR_MONTH)?; + let fields = CalendarFields::from_year_month(machine, &calendar, value)?.merged_with(partial); + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let result = year_month_from_fields(machine, &calendar, &fields, overflow)?; + year_month_outcome(machine, result, calendar) +} + +/// AddDurationToYearMonth. +fn add_duration( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + subtract: bool, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + let duration = to_temporal_duration(machine, arg(args, 0))?; + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let result = match Calendar::from_id(&calendar) { + Some(kind) if subtract => { + calendar::add_to_year_month(kind, value, &duration.negated(), overflow) + } + Some(kind) => calendar::add_to_year_month(kind, value, &duration, overflow), + None if subtract => value.subtract(&duration, overflow), + None => value.add(&duration, overflow), + } + .or_throw(machine)?; + year_month_outcome(machine, result, calendar) +} + +fn add( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + add_duration(machine, this, args, false) +} + +fn subtract( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + add_duration(machine, this, args, true) +} + +/// DifferenceTemporalPlainYearMonth. +fn difference( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + since: bool, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + let (other, other_calendar) = to_temporal_year_month(machine, arg(args, 0), Value::UNDEFINED)?; + if calendar != other_calendar { + return Err(range_error(machine, "calendars must match")); + } + let options = get_options_object(machine, arg(args, 1))?; + let settings = difference_settings( + machine, + since, + options, + DifferencePolicy { + group: DATE_UNITS, + disallowed: &[Unit::Week, Unit::Day], + fallback_smallest: Unit::Month, + smallest_largest_default: Unit::Year, + }, + )?; + let result = match Calendar::from_id(&calendar) { + Some(_) if value == other => Ok(Duration::default()), + Some(kind) => calendar::first_of_month(kind, value).and_then(|one| { + difference_plain_year_month( + &providers(), + one, + calendar::first_of_month(kind, other)?, + &calendar, + DifferenceRounding { + largest_unit: settings.largest_unit, + smallest_unit: settings.smallest_unit, + increment: settings.rounding_increment, + mode: settings.rounding_mode, + }, + ) + }), + None => value.until( + other, + settings.largest_unit, + settings.smallest_unit, + settings.rounding_increment, + settings.rounding_mode, + ), + } + .or_throw(machine)?; + let result = if since { result.negated() } else { result }; + create(machine, TemporalObject::Duration(result)).map(BuiltinOutcome::Value) +} + +fn until( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + difference(machine, this, args, false) +} + +fn since( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + difference(machine, this, args, true) +} + +fn equals( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + let (other, other_calendar) = to_temporal_year_month(machine, arg(args, 0), Value::UNDEFINED)?; + let equal = value == other && calendar == other_calendar; + Ok(BuiltinOutcome::Value(Value::boolean(equal))) +} + +/// TemporalYearMonthToString: the reference day is only shown when a calendar +/// annotation makes it part of the round trip. +fn year_month_text( + machine: &mut Machine<'_, H>, + value: PlainYearMonth, + calendar: &CalendarId, + show: ShowCalendar, +) -> Result { + let body = if matches!(show, ShowCalendar::Always | ShowCalendar::Critical) + || !calendar.is_iso8601() + { + format_iso_date(value.year, value.month, value.reference_day()) + } else { + value.format() + }; + let annotation = format_calendar_annotation(calendar.as_str(), show).or_throw(machine)?; + Ok(format!("{body}{annotation}")) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + let options = get_options_object(machine, arg(args, 0))?; + let show = show_calendar_option(machine, options)?; + let text = year_month_text(machine, value, &calendar, show)?; + string_outcome(machine, &text) +} + +fn to_json( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + let text = year_month_text(machine, value, &calendar, ShowCalendar::Auto)?; + string_outcome(machine, &text) +} + +fn to_plain_date( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let (value, calendar) = this_year_month(machine, this)?; + let item = arg(args, 0); + if !is_object(machine, item)? { + return Err(type_error(machine, "argument must be an object")); + } + let input = + prepare_calendar_fields(machine, &calendar, item, FieldNames::DAY, FieldNames::NONE)?; + let date = if calendar.is_iso8601() { + let Some(day) = input.day else { + return Err(type_error(machine, "day is required")); + }; + value + .to_plain_date(day as i64, Overflow::Constrain) + .or_throw(machine)? + } else { + let fields = CalendarFields::from_year_month(machine, &calendar, value)?.merged_with(input); + date_from_fields(machine, &calendar, &fields, Overflow::Constrain)? + }; + create( + machine, + TemporalObject::PlainDate { + value: date, + calendar, + }, + ) + .map(BuiltinOutcome::Value) +} + +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + this_year_month(machine, this)?; + temporal_to_locale_string(machine, this, arg(args, 0), arg(args, 1)) +} diff --git a/crates/bamts-runtime/src/builtins/temporal/zoned_date_time.rs b/crates/bamts-runtime/src/builtins/temporal/zoned_date_time.rs new file mode 100644 index 00000000..83ac957c --- /dev/null +++ b/crates/bamts-runtime/src/builtins/temporal/zoned_date_time.rs @@ -0,0 +1,704 @@ +//! `Temporal.ZonedDateTime`. + +use bamts_native::Value; + +use super::super::bigint::to_bigint; +use super::super::intl::temporal_to_locale_string; +use super::fields::{ + CalendarFields, FieldNames, OffsetBehaviour, calendar_argument, date_time_from_fields, + interpret_offset, prepare_partial_fields, providers, require_partial_temporal_object, + to_calendar_identifier, to_temporal_duration, to_temporal_time, to_temporal_zoned_date_time, + to_time_zone_identifier, +}; +use super::options::{ + ALL_UNITS, DifferencePolicy, TIME_UNITS, UnitOption, difference_settings, direction_option, + disambiguation_option, fractional_second_digits_option, get_options_object, offset_option, + overflow_option, rounding_increment_option, rounding_mode_option, seconds_precision, + show_calendar_option, show_offset_option, show_time_zone_option, unit_argument, + unit_valued_option, validate_unit_value, +}; +use super::{ + Class, Getter, OrThrow, arg, bigint_outcome, bigint_to_i128, construct, create, install_class, + number_outcome, range_error, require_new, string_outcome, temporal_record, type_error, + value_of_throws, +}; +use crate::intrinsics::{BuiltinOutcome, BuiltinTable}; +use crate::temporal::instant_duration::{ + Instant, NS_PER_HOUR, RoundingMode, Unit, UnitCategory, exact_ratio_to_f64, + validate_rounding_increment, +}; +use crate::temporal::parse::parse_time_zone_identifier; +use crate::temporal::plain_types::{PlainDate, PlainDateTime}; +use crate::temporal::zoned_date_time::{ + ArithmeticOptions, CalendarDateFields, CalendarId, DifferenceOptions, FixedOffsetTimeZones, + OffsetMatch, OffsetOption, ResolveOptions, RoundOptions, TimeZoneId, ToStringOptions, + TransitionDirection, ZonedDateTime, epoch_nanoseconds_for, format_utc_offset, start_of_day, +}; +use crate::temporal::{TemporalKind, TemporalObject}; +use crate::{EvalFailure, HeapEntry, Host, Machine}; + +pub(super) fn install( + heap: &mut Vec, + builtins: &mut BuiltinTable, + namespace: Value, +) { + let date_getters: [Getter; 16] = date_getter_table!(H); + let other_getters: [Getter; 11] = [ + ("timeZoneId", "get timeZoneId", time_zone_id::), + ("hour", "get hour", hour::), + ("minute", "get minute", minute::), + ("second", "get second", second::), + ("millisecond", "get millisecond", millisecond::), + ("microsecond", "get microsecond", microsecond::), + ("nanosecond", "get nanosecond", nanosecond::), + ( + "epochMilliseconds", + "get epochMilliseconds", + epoch_milliseconds::, + ), + ( + "epochNanoseconds", + "get epochNanoseconds", + epoch_nanoseconds::, + ), + ("hoursInDay", "get hoursInDay", hours_in_day::), + ( + "offsetNanoseconds", + "get offsetNanoseconds", + offset_nanoseconds::, + ), + ]; + let offset: Getter = ("offset", "get offset", offset::); + let getters: Vec> = date_getters + .iter() + .chain(&other_getters) + .chain(std::iter::once(&offset)) + .copied() + .collect(); + install_class( + heap, + builtins, + namespace, + &Class { + kind: TemporalKind::ZonedDateTime, + name: "ZonedDateTime", + tag: "Temporal.ZonedDateTime", + length: 2, + constructor: constructor::, + statics: &[("from", 1, from::), ("compare", 2, compare::)], + getters: &getters, + methods: &[ + ("with", 1, with::), + ("withPlainTime", 0, with_plain_time::), + ("withTimeZone", 1, with_time_zone::), + ("withCalendar", 1, with_calendar::), + ("add", 1, add::), + ("subtract", 1, subtract::), + ("until", 1, until::), + ("since", 1, since::), + ("round", 1, round::), + ("equals", 1, equals::), + ("toString", 0, to_string::), + ("toLocaleString", 0, to_locale_string::), + ("toJSON", 0, to_json::), + ("valueOf", 0, value_of_throws::), + ("startOfDay", 0, start_of_day_method::), + ("getTimeZoneTransition", 1, get_time_zone_transition::), + ("toInstant", 0, to_instant::), + ("toPlainDate", 0, to_plain_date::), + ("toPlainTime", 0, to_plain_time::), + ("toPlainDateTime", 0, to_plain_date_time::), + ], + }, + ); +} + +fn this_zoned( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result { + match temporal_record(machine, this)? { + Some(TemporalObject::ZonedDateTime(value)) => Ok(value), + _ => Err(type_error( + machine, + "receiver is not a Temporal.ZonedDateTime", + )), + } +} + +fn zoned_outcome( + machine: &mut Machine<'_, H>, + value: ZonedDateTime, +) -> Result { + create(machine, TemporalObject::ZonedDateTime(value)).map(BuiltinOutcome::Value) +} + +fn local_date_time( + machine: &mut Machine<'_, H>, + zoned: &ZonedDateTime, +) -> Result { + zoned + .to_plain_date_time(&FixedOffsetTimeZones) + .or_throw(machine) +} + +/// The date behind the shared calendar accessors. +fn accessor_date( + machine: &mut Machine<'_, H>, + this: Value, +) -> Result<(PlainDate, CalendarId), EvalFailure> { + let zoned = this_zoned(machine, this)?; + let local = local_date_time(machine, &zoned)?; + Ok((local.date, zoned.calendar_id().clone())) +} + +date_accessors!(accessor_date); + +macro_rules! time_getter { + ($($name:ident),+) => { + $(fn $name( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let zoned = this_zoned(machine, this)?; + let fields = zoned.fields(&providers()).or_throw(machine)?; + number_outcome(f64::from(fields.$name())) + })+ + }; +} + +time_getter!(hour, minute, second, millisecond, microsecond, nanosecond); + +/// The zone argument of the constructor, which must be exactly an identifier; +/// the ISO strings `ToTemporalTimeZoneIdentifier` accepts are not allowed here. +fn time_zone_argument( + machine: &mut Machine<'_, H>, + value: Value, +) -> Result { + let Some(text) = machine.string_value(value) else { + return Err(type_error(machine, "time zone must be a string")); + }; + let parsed = parse_time_zone_identifier(&text.to_utf8_lossy()).or_throw(machine)?; + TimeZoneId::from_parsed(parsed).or_throw(machine) +} + +fn constructor( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + constructing: bool, +) -> Result { + require_new(machine, constructing, "ZonedDateTime")?; + let bigint = to_bigint(machine, arg(args, 0))?; + let nanoseconds = bigint_to_i128(machine, bigint)?; + let instant = Instant::from_epoch_nanoseconds(nanoseconds).or_throw(machine)?; + let time_zone = time_zone_argument(machine, arg(args, 1))?; + let calendar = calendar_argument(machine, arg(args, 2))?; + construct( + machine, + TemporalObject::ZonedDateTime(ZonedDateTime::from_instant(instant, time_zone, calendar)), + ) +} + +fn from( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let value = to_temporal_zoned_date_time(machine, arg(args, 0), arg(args, 1))?; + zoned_outcome(machine, value) +} + +fn compare( + machine: &mut Machine<'_, H>, + _this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let one = to_temporal_zoned_date_time(machine, arg(args, 0), Value::UNDEFINED)?; + let two = to_temporal_zoned_date_time(machine, arg(args, 1), Value::UNDEFINED)?; + number_outcome(f64::from(ZonedDateTime::compare(&one, &two) as i8)) +} + +fn time_zone_id( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + string_outcome(machine, zoned.time_zone_id().as_str()) +} + +fn epoch_milliseconds( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + number_outcome(this_zoned(machine, this)?.epoch_milliseconds() as f64) +} + +fn epoch_nanoseconds( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = this_zoned(machine, this)?.epoch_nanoseconds(); + bigint_outcome(machine, value) +} + +fn hours_in_day( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let length = zoned + .day_length_nanoseconds(&providers()) + .or_throw(machine)?; + number_outcome(exact_ratio_to_f64(length, NS_PER_HOUR)) +} + +fn current_offset( + machine: &mut Machine<'_, H>, + zoned: &ZonedDateTime, +) -> Result { + let fields = zoned.fields(&providers()).or_throw(machine)?; + Ok(fields.offset) +} + +fn offset_nanoseconds( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let offset = current_offset(machine, &zoned)?; + number_outcome(offset.get() as f64) +} + +fn offset( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let offset = current_offset(machine, &zoned)?; + string_outcome(machine, &format_utc_offset(offset)) +} + +fn with( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let like = arg(args, 0); + require_partial_temporal_object(machine, like)?; + let local = local_date_time(machine, &zoned)?; + let current = current_offset(machine, &zoned)?; + let calendar = zoned.calendar_id().clone(); + let mut base = CalendarFields::from_date(machine, &calendar, local.date)?.with_time(local.time); + base.offset = Some(i128::from(current.get())); + let names = FieldNames::DATE + .with(FieldNames::TIME) + .with(FieldNames::OFFSET); + let partial = prepare_partial_fields(machine, &calendar, like, names)?; + let fields = base.merged_with(partial); + let options = get_options_object(machine, arg(args, 1))?; + let disambiguation = disambiguation_option(machine, options)?; + let offset = offset_option(machine, options, OffsetOption::Prefer)?; + let overflow = overflow_option(machine, options)?; + let resolved = date_time_from_fields(machine, &calendar, &fields, overflow)?; + let value = interpret_offset( + machine, + resolved, + OffsetBehaviour::Option(fields.offset.unwrap_or(0)), + zoned.time_zone_id().clone(), + calendar, + ResolveOptions { + disambiguation, + offset, + offset_match: OffsetMatch::Exactly, + }, + )?; + zoned_outcome(machine, value) +} + +fn with_plain_time( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let like = arg(args, 0); + let local = local_date_time(machine, &zoned)?; + let time_zone = zoned.time_zone_id().clone(); + let calendar = zoned.calendar_id().clone(); + let instant = if like == Value::UNDEFINED { + start_of_day(&FixedOffsetTimeZones, &time_zone, local.date).or_throw(machine)? + } else { + let time = to_temporal_time(machine, like, Value::UNDEFINED)?; + let target = PlainDateTime::from_parts(local.date, time).or_throw(machine)?; + let nanoseconds = + epoch_nanoseconds_for(&FixedOffsetTimeZones, &time_zone, target).or_throw(machine)?; + Instant::from_epoch_nanoseconds(nanoseconds).or_throw(machine)? + }; + zoned_outcome( + machine, + ZonedDateTime::from_instant(instant, time_zone, calendar), + ) +} + +fn with_time_zone( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let time_zone = to_time_zone_identifier(machine, arg(args, 0))?; + zoned_outcome(machine, zoned.with_time_zone(time_zone)) +} + +fn with_calendar( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let calendar = to_calendar_identifier(machine, arg(args, 0))?; + zoned_outcome(machine, zoned.with_calendar(calendar)) +} + +/// AddDurationToZonedDateTime. +fn add_duration( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + subtract: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let duration = to_temporal_duration(machine, arg(args, 0))?; + let options = get_options_object(machine, arg(args, 1))?; + let overflow = overflow_option(machine, options)?; + let core = providers(); + let arithmetic = ArithmeticOptions { overflow }; + let value = if subtract { + zoned.subtract(&core, &duration, arithmetic) + } else { + zoned.add(&core, &duration, arithmetic) + } + .or_throw(machine)?; + zoned_outcome(machine, value) +} + +fn add( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + add_duration(machine, this, args, false) +} + +fn subtract( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + add_duration(machine, this, args, true) +} + +/// DifferenceTemporalZonedDateTime. The settings are read with `since` false +/// because `ZonedDateTime::since` negates the rounding mode itself. +fn difference( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + since: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let other = to_temporal_zoned_date_time(machine, arg(args, 0), Value::UNDEFINED)?; + if zoned.calendar_id() != other.calendar_id() { + return Err(range_error(machine, "calendars must match")); + } + let options = get_options_object(machine, arg(args, 1))?; + let settings = difference_settings( + machine, + false, + options, + DifferencePolicy { + group: ALL_UNITS, + disallowed: &[], + fallback_smallest: Unit::Nanosecond, + smallest_largest_default: Unit::Hour, + }, + )?; + if settings.largest_unit.category() == UnitCategory::Date + && zoned.time_zone_id() != other.time_zone_id() + { + return Err(range_error( + machine, + "date units require matching time zones", + )); + } + let core = providers(); + let difference_options = DifferenceOptions { + largest_unit: settings.largest_unit, + smallest_unit: settings.smallest_unit, + rounding_increment: settings.rounding_increment, + rounding_mode: settings.rounding_mode, + }; + let value = if since { + zoned.since(&core, &other, difference_options) + } else { + zoned.until(&core, &other, difference_options) + } + .or_throw(machine)?; + create(machine, TemporalObject::Duration(value)).map(BuiltinOutcome::Value) +} + +fn until( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + difference(machine, this, args, false) +} + +fn since( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + difference(machine, this, args, true) +} + +fn round( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let round_to = unit_argument(machine, arg(args, 0))?; + let increment = rounding_increment_option(machine, round_to.options)?; + let mode = rounding_mode_option(machine, round_to.options, RoundingMode::HalfExpand)?; + let smallest = round_to.required_unit(machine, "smallestUnit")?; + validate_unit_value( + machine, + smallest, + TIME_UNITS, + Some(UnitOption::Unit(Unit::Day)), + )?; + let UnitOption::Unit(unit) = smallest else { + return Err(range_error(machine, "smallestUnit is required")); + }; + if unit == Unit::Day { + if increment != 1 { + return Err(range_error( + machine, + "roundingIncrement must be 1 when rounding to day", + )); + } + } else { + validate_rounding_increment(unit, increment, false).or_throw(machine)?; + } + if unit == Unit::Nanosecond && increment == 1 { + return zoned_outcome(machine, zoned); + } + let value = zoned + .round( + &providers(), + RoundOptions { + smallest_unit: unit, + rounding_increment: increment, + rounding_mode: mode, + }, + ) + .or_throw(machine)?; + zoned_outcome(machine, value) +} + +fn equals( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let other = to_temporal_zoned_date_time(machine, arg(args, 0), Value::UNDEFINED)?; + Ok(BuiltinOutcome::Value(Value::boolean(zoned.equals(&other)))) +} + +fn to_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let options = get_options_object(machine, arg(args, 0))?; + let show_calendar = show_calendar_option(machine, options)?; + let digits = fractional_second_digits_option(machine, options)?; + let show_offset = show_offset_option(machine, options)?; + let mode = rounding_mode_option(machine, options, RoundingMode::Trunc)?; + let smallest = unit_valued_option(machine, options, "smallestUnit")?; + let show_time_zone = show_time_zone_option(machine, options)?; + validate_unit_value(machine, smallest, TIME_UNITS, None)?; + if smallest == UnitOption::Unit(Unit::Hour) { + return Err(range_error(machine, "smallestUnit must not be hour")); + } + let precision = seconds_precision(machine, smallest, digits)?; + let text = zoned + .format( + &providers(), + ToStringOptions { + precision: precision.precision, + show_calendar, + show_time_zone, + show_offset, + round: Some(RoundOptions { + smallest_unit: precision.unit, + rounding_increment: precision.increment, + rounding_mode: mode, + }), + }, + ) + .or_throw(machine)?; + string_outcome(machine, &text) +} + +fn to_json( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let text = this_zoned(machine, this)? + .to_json(&providers()) + .or_throw(machine)?; + string_outcome(machine, &text) +} + +fn start_of_day_method( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let value = zoned.start_of_day(&providers()).or_throw(machine)?; + zoned_outcome(machine, value) +} + +fn get_time_zone_transition( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let direction = unit_argument(machine, arg(args, 0))?; + let direction = match direction.shorthand() { + Some(text) => direction_text(machine, text)?, + None => direction_option(machine, direction.options)?, + }; + let direction = if direction == "next" { + TransitionDirection::Next + } else { + TransitionDirection::Previous + }; + let transition = zoned + .get_time_zone_transition(&FixedOffsetTimeZones, direction) + .or_throw(machine)?; + match transition { + Some(value) => zoned_outcome(machine, value), + None => Ok(BuiltinOutcome::Value(Value::NULL)), + } +} + +fn direction_text( + machine: &mut Machine<'_, H>, + text: &str, +) -> Result { + if matches!(text, "next" | "previous") { + Ok(text.to_owned()) + } else { + Err(range_error(machine, format!("{text} is not a direction"))) + } +} + +fn to_instant( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let value = this_zoned(machine, this)?.to_instant(); + create(machine, TemporalObject::Instant(value)).map(BuiltinOutcome::Value) +} + +fn to_plain_date( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let value = zoned + .to_plain_date(&FixedOffsetTimeZones) + .or_throw(machine)?; + let calendar = zoned.calendar_id().clone(); + create(machine, TemporalObject::PlainDate { value, calendar }).map(BuiltinOutcome::Value) +} + +fn to_plain_time( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let value = zoned + .to_plain_time(&FixedOffsetTimeZones) + .or_throw(machine)?; + create(machine, TemporalObject::PlainTime(value)).map(BuiltinOutcome::Value) +} + +fn to_plain_date_time( + machine: &mut Machine<'_, H>, + this: Value, + _args: &[Value], + _constructing: bool, +) -> Result { + let zoned = this_zoned(machine, this)?; + let value = local_date_time(machine, &zoned)?; + let calendar = zoned.calendar_id().clone(); + create(machine, TemporalObject::PlainDateTime { value, calendar }).map(BuiltinOutcome::Value) +} + +fn to_locale_string( + machine: &mut Machine<'_, H>, + this: Value, + args: &[Value], + _constructing: bool, +) -> Result { + this_zoned(machine, this)?; + temporal_to_locale_string(machine, this, arg(args, 0), arg(args, 1)) +} diff --git a/crates/bamts-runtime/src/builtins/typedarray_all.rs b/crates/bamts-runtime/src/builtins/typedarray_all.rs index 303e1f9b..be8f9fa9 100644 --- a/crates/bamts-runtime/src/builtins/typedarray_all.rs +++ b/crates/bamts-runtime/src/builtins/typedarray_all.rs @@ -379,7 +379,6 @@ fn view_fields(machine: &Machine<'_, H>, view: Value) -> Result( machine: &Machine<'_, H>, view: Value, @@ -551,15 +549,17 @@ fn numeric_value(machine: &mut Machine<'_, H>, value: Value) -> Result< } } -fn integer_or_infinity( +/// ToIntegerOrInfinity. Adding `+0` turns the `-0` that truncating a small +/// negative fraction yields into the `+0` the operation is defined to return. +pub(crate) fn integer_or_infinity( machine: &mut Machine<'_, H>, value: Value, ) -> Result { let number = numeric_value(machine, value)?; - Ok(if number.is_nan() || number == 0.0 { + Ok(if number.is_nan() { 0.0 } else { - number.trunc() + number.trunc() + 0.0 }) } @@ -2023,7 +2023,7 @@ fn for_each( let this_arg = args.get(1).copied().unwrap_or(Value::UNDEFINED); for index in 0..length { let value = read_element(machine, this, index)?; - machine.call_value( + machine.call_callback( callback, this_arg, &[value, crate::number_value(index as f64), this], @@ -2198,7 +2198,7 @@ fn reduce_impl>( }; for index in indices { let value = read_element(machine, this, index)?; - accumulator = machine.call_value( + accumulator = machine.call_callback( callback, Value::UNDEFINED, &[accumulator, value, crate::number_value(index as f64), this], diff --git a/crates/bamts-runtime/src/builtins/weakref_finalization.rs b/crates/bamts-runtime/src/builtins/weakref_finalization.rs index 22b26f19..785188c4 100644 --- a/crates/bamts-runtime/src/builtins/weakref_finalization.rs +++ b/crates/bamts-runtime/src/builtins/weakref_finalization.rs @@ -368,11 +368,8 @@ impl<'a, H: Host> Machine<'a, H> { /// Clears dead weak handles and extracts holdings after the collector's /// ephemeron fixed point. This queues candidates, not jobs: the host chooses /// when (or whether) to call `schedule_finalization_cleanup_jobs`. - pub(crate) fn process_weak_targets_after_marking(&mut self, marks: &[bool]) { - for index in 0..self.heap.len() { - if !marks.get(index).copied().unwrap_or(false) { - continue; - } + pub(crate) fn process_weak_targets_after_marking(&mut self, marks: &[bool], holders: &[usize]) { + for &index in holders { let mut released = 0usize; let mut queue_registry = false; match &mut self.heap[index] { diff --git a/crates/bamts-runtime/src/gc.rs b/crates/bamts-runtime/src/gc.rs index e92c9caa..2ec8a50d 100644 --- a/crates/bamts-runtime/src/gc.rs +++ b/crates/bamts-runtime/src/gc.rs @@ -3,8 +3,17 @@ use super::*; #[derive(Default)] pub(super) struct GcState { marks: Vec, + /// Runtime slots that outlived the last collection. Allocation never + /// reuses a slot, so a collection that walked the heap would pay for every + /// allocation the program ever made. + survivors: Vec, + /// The heap length at the last sweep; every later slot is unswept. + swept_len: usize, work: Vec, weak_collections: Vec, + /// Marked `WeakRef`s and `FinalizationRegistry`s, found while marking so + /// that clearing them need not search the heap. + weak_holders: Vec, pending: bool, byte_watermark: usize, slot_watermark: usize, @@ -98,15 +107,16 @@ impl GcState { } fn collect(&mut self, machine: &mut Machine<'_, H>) { - self.marks.resize(machine.heap.len(), false); - self.marks.fill(false); + self.reset_marks(machine); self.work.clear(); self.weak_collections.clear(); + self.weak_holders.clear(); self.trace_roots(machine); self.drain(&machine.heap); self.ephemeron_fixed_point(&machine.heap); - machine.process_weak_targets_after_marking(&self.marks); + self.weak_holders.sort_unstable(); + machine.process_weak_targets_after_marking(&self.marks, &self.weak_holders); self.purge_weak_entries(machine); self.sweep(machine); self.pending = false; @@ -119,6 +129,20 @@ impl GcState { ); } + /// Clears the last collection's marks, which are all on intrinsics or + /// survivors, and accounts for a rolled-back clone shortening the heap. + fn reset_marks(&mut self, machine: &Machine<'_, H>) { + for &index in &self.survivors { + self.marks[index] = false; + } + let intrinsics = self.marks.len().min(machine.intrinsic_slots); + self.marks[..intrinsics].fill(false); + let heap_len = machine.heap.len(); + self.marks.resize(heap_len, false); + self.survivors.retain(|&index| index < heap_len); + self.swept_len = self.swept_len.min(heap_len); + } + fn trace_roots(&mut self, machine: &Machine<'_, H>) { for index in 0..machine.intrinsic_slots { mark_index(&machine.heap, &mut self.marks, &mut self.work, index); @@ -151,6 +175,8 @@ impl GcState { } self.mark_value(&machine.heap, machine.current_new_target); self.mark_values(&machine.heap, &machine.kept_alive); + self.mark_values(&machine.heap, &machine.builtin_temporaries); + self.mark_values(&machine.heap, &machine.callback_results); if let Some(resume) = &machine.pending_generator_resume { trace_generator_resume(resume, &machine.heap, &mut self.marks, &mut self.work); } @@ -161,9 +187,15 @@ impl GcState { for job in &machine.microtasks { trace_microtask(&job.job, &machine.heap, &mut self.marks, &mut self.work); } + for job in &machine.running_jobs { + trace_microtask(job, &machine.heap, &mut self.marks, &mut self.work); + } for ®istry in &machine.finalization_cleanup_queue { self.mark_value(&machine.heap, registry); } + for waiter in machine.async_waiters.values() { + self.mark_value(&machine.heap, waiter.promise); + } for timer in machine.timers.values() { self.mark_value(&machine.heap, timer.callback); self.mark_values(&machine.heap, &timer.arguments); @@ -225,6 +257,12 @@ impl GcState { fn drain(&mut self, heap: &[HeapEntry]) { while let Some(index) = self.work.pop() { + if matches!( + heap[index], + HeapEntry::WeakRef { .. } | HeapEntry::FinalizationRegistry { .. } + ) { + self.weak_holders.push(index); + } trace_entry( &heap[index], index, @@ -309,16 +347,23 @@ impl GcState { } } - fn sweep(&self, machine: &mut Machine<'_, H>) { - for index in machine.intrinsic_slots..machine.heap.len() { - if self.marks[index] || matches!(machine.heap[index], HeapEntry::Vacant) { - continue; + fn sweep(&mut self, machine: &mut Machine<'_, H>) { + let unswept = self.swept_len.max(machine.intrinsic_slots)..machine.heap.len(); + self.survivors.extend(unswept); + let marks = &self.marks; + self.survivors.retain(|&index| { + if marks[index] { + return true; } - machine.heap[index] = HeapEntry::Vacant; - let charge = std::mem::take(&mut machine.slot_bytes[index]); - machine.heap_bytes -= charge; - machine.vacant_count += 1; - } + if !matches!(machine.heap[index], HeapEntry::Vacant) { + machine.heap[index] = HeapEntry::Vacant; + let charge = std::mem::take(&mut machine.slot_bytes[index]); + machine.heap_bytes -= charge; + machine.vacant_count += 1; + } + false + }); + self.swept_len = machine.heap.len(); } } @@ -738,6 +783,11 @@ fn trace_entry( prototype, .. } + | HeapEntry::Temporal { + properties, + prototype, + .. + } | HeapEntry::Timeout { properties, prototype, @@ -748,6 +798,17 @@ fn trace_entry( prototype, .. } => trace_properties_and_prototype(properties, *prototype, heap, marks, work), + HeapEntry::Intl { + bound, + properties, + prototype, + .. + } => { + if let Some(bound) = bound { + mark_value(heap, marks, work, *bound); + } + trace_properties_and_prototype(properties, *prototype, heap, marks, work); + } HeapEntry::TypedArray { buffer, properties, diff --git a/crates/bamts-runtime/src/gc/stress_hardening.rs b/crates/bamts-runtime/src/gc/stress_hardening.rs index 5ec837fc..1afffac5 100644 --- a/crates/bamts-runtime/src/gc/stress_hardening.rs +++ b/crates/bamts-runtime/src/gc/stress_hardening.rs @@ -331,6 +331,43 @@ fn weak_ref_target_is_kept_alive_for_exactly_one_job() { assert_eq!(stress.call_method(weak_ref, "deref", &[]), Value::UNDEFINED); } +#[test] +fn collections_follow_survivors_without_revisiting_vacant_slots() { + let program = blank_program(); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + let mut stress = GcStress::new(&mut machine); + + let early = stress.object(); + stress.root("early", early); + let early_garbage = stress.object(); + stress.collect(); + stress.assert_live(early); + stress.assert_dead(early_garbage); + let vacant_after_first = stress.machine.vacant_count; + + // A slot from before the last sweep stays tracked, and a later one is + // picked up, whichever way each one goes. + let late = stress.object(); + stress.root("late", late); + let late_garbage = stress.object(); + stress.collect(); + stress.assert_live(early); + stress.assert_live(late); + stress.assert_dead(late_garbage); + assert_eq!(stress.machine.vacant_count, vacant_after_first + 1); + + stress.unroot("early"); + stress.collect(); + stress.assert_dead(early); + stress.assert_live(late); + assert_eq!(stress.machine.vacant_count, vacant_after_first + 2); + + // Nothing is reclaimed twice once it has gone. + stress.collect(); + assert_eq!(stress.machine.vacant_count, vacant_after_first + 2); +} + #[test] fn allocation_pressure_requests_collection_and_recomputes_watermarks() { let program = blank_program(); diff --git a/crates/bamts-runtime/src/host_objects.rs b/crates/bamts-runtime/src/host_objects.rs index af00883d..26ab5348 100644 --- a/crates/bamts-runtime/src/host_objects.rs +++ b/crates/bamts-runtime/src/host_objects.rs @@ -85,6 +85,8 @@ pub(crate) fn install( crate::intrinsics::builtins::define_frozen_data(heap, global_this, "Infinity", *value); } else if name.eq_ascii("NaN") { crate::intrinsics::builtins::define_frozen_data(heap, global_this, "NaN", *value); + } else if name.eq_ascii("undefined") { + crate::intrinsics::builtins::define_frozen_data(heap, global_this, "undefined", *value); } else { define_global_data(heap, global_this, name.clone(), *value); } @@ -411,6 +413,8 @@ impl Machine<'_, H> { HeapEntry::Object { properties, .. } | HeapEntry::Script { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::Generator { properties, .. } diff --git a/crates/bamts-runtime/src/intl/canonical_locales.rs b/crates/bamts-runtime/src/intl/canonical_locales.rs index add95c0d..bb2474bd 100644 --- a/crates/bamts-runtime/src/intl/canonical_locales.rs +++ b/crates/bamts-runtime/src/intl/canonical_locales.rs @@ -1,8 +1,9 @@ //! Cross-service ECMA-402 adapter contracts. //! //! Implements `Intl.getCanonicalLocales`, `Intl.supportedValuesOf`, and the -//! shared constructor/call/options plumbing every Intl service constructor -//! uses. All locale parsing and canonicalization delegate to the C10.1 +//! shared constructor/call contracts every Intl service constructor uses. +//! Options are not read here: the JavaScript binding reads them from live +//! values, in the order the specification makes observable. All locale parsing and canonicalization delegate to the C10.1 //! negotiation module through [`LocaleDataProvider`]; all //! implementation-defined value inventories (calendar, collation, currency, //! numberingSystem, timeZone, unit) arrive through the injected @@ -17,16 +18,13 @@ //! Specification sources: //! - //! - -//! - -//! - -//! - use std::collections::{BTreeMap, BTreeSet}; use std::error::Error; use std::fmt; use super::locale_negotiation::{ - JsErrorKind, LocaleDataProvider, LocaleError, LocaleMatcher, canonicalize_unicode_locale_id, + JsErrorKind, LocaleDataProvider, LocaleError, canonicalize_unicode_locale_id, }; /// The ECMAScript error class surfaced by a cross-service adapter failure. @@ -51,13 +49,19 @@ impl IntlError { /// Builds a `TypeError` failure. #[must_use] pub fn type_error(message: impl Into) -> Self { - Self { kind: IntlErrorKind::TypeError, message: message.into() } + Self { + kind: IntlErrorKind::TypeError, + message: message.into(), + } } /// Builds a `RangeError` failure. #[must_use] pub fn range_error(message: impl Into) -> Self { - Self { kind: IntlErrorKind::RangeError, message: message.into() } + Self { + kind: IntlErrorKind::RangeError, + message: message.into(), + } } /// Converts a C10.1 locale failure, preserving its JS error class. @@ -119,9 +123,9 @@ pub enum IntlCallForm { pub fn require_constructor_call(name: &str, form: IntlCallForm) -> Result<(), IntlError> { match form { IntlCallForm::Construct => Ok(()), - IntlCallForm::Apply => { - Err(IntlError::type_error(format!("Constructor {name} requires 'new'"))) - } + IntlCallForm::Apply => Err(IntlError::type_error(format!( + "Constructor {name} requires 'new'" + ))), } } @@ -134,9 +138,9 @@ pub fn require_constructor_call(name: &str, form: IntlCallForm) -> Result<(), In pub fn require_ordinary_call(name: &str, form: IntlCallForm) -> Result<(), IntlError> { match form { IntlCallForm::Apply => Ok(()), - IntlCallForm::Construct => { - Err(IntlError::type_error(format!("{name} is not a constructor"))) - } + IntlCallForm::Construct => Err(IntlError::type_error(format!( + "{name} is not a constructor" + ))), } } @@ -176,9 +180,7 @@ pub fn get_canonical_locales( ) -> Result, IntlError> { match argument { LocaleListArgument::Undefined | LocaleListArgument::Primitive => Ok(Vec::new()), - LocaleListArgument::Null => { - Err(IntlError::type_error("invalid locales argument: null")) - } + LocaleListArgument::Null => Err(IntlError::type_error("invalid locales argument: null")), LocaleListArgument::Text(text) => Ok(vec![canonicalize_single_tag(text, provider)?]), LocaleListArgument::Elements(elements) => { let mut seen = BTreeSet::new(); @@ -414,9 +416,9 @@ fn is_currency_code(value: &str) -> bool { /// `_`, `-`, `+`, and `/`. IANA zone names contain no `.` runs. fn is_time_zone_name(value: &str) -> bool { !value.is_empty() - && value.bytes().all(|byte| { - byte.is_ascii_alphanumeric() || matches!(byte, b'_' | b'-' | b'+' | b'/') - }) + && value + .bytes() + .all(|byte| byte.is_ascii_alphanumeric() || matches!(byte, b'_' | b'-' | b'+' | b'/')) } /// A well-formed simple or compound Unicode unit identifier: `-`-separated @@ -424,300 +426,14 @@ fn is_time_zone_name(value: &str) -> bool { /// denominator separator. fn is_unit_identifier(value: &str) -> bool { !value.is_empty() - && value.split('-').all(|part| { - !part.is_empty() && part.bytes().all(|byte| byte.is_ascii_alphanumeric()) - }) + && value + .split('-') + .all(|part| !part.is_empty() && part.bytes().all(|byte| byte.is_ascii_alphanumeric())) && value.matches("-per-").count() <= 1 && !value.starts_with("per-") && !value.ends_with("-per") } -/// A JavaScript property bag backing an `options` argument. The host -/// implements observable `[[Get]]` semantics; a throwing getter surfaces as -/// the returned error. Each shared option reader performs exactly one `get` -/// per call so the observable read order equals the caller's option order. -pub trait OptionsPropertySource { - /// Returns the property value, `IntlValue::Undefined` when absent. - /// - /// # Errors - /// Propagates a throwing getter as the host-provided error. - fn get(&mut self, key: &str) -> Result; -} - -/// The typed `options` argument accepted by ECMA-402 `GetOptionsObject`. -pub enum OptionsArgument<'a> { - /// `undefined`: replaced by a null-prototype empty object. - Undefined, - /// `null` or a boolean/number/string/symbol/bigint primitive. - NotAnObject, - /// A JavaScript object delegate. - Object(&'a mut dyn OptionsPropertySource), -} - -/// Applies ECMA-402 `GetOptionsObject`. Returns `None` for the synthetic -/// null-prototype object created from `undefined` (every property read on it -/// is `undefined`, and prototype chains are never consulted). -/// -/// # Errors -/// Returns `TypeError` when the argument is `null` or a primitive. -pub fn get_options_object( - argument: OptionsArgument<'_>, -) -> Result, IntlError> { - match argument { - OptionsArgument::Undefined => Ok(None), - OptionsArgument::Object(source) => Ok(Some(source)), - OptionsArgument::NotAnObject => { - Err(IntlError::type_error("options must be an object or undefined")) - } - } -} - -/// ECMA-402 `GetOption` with `type` string: a single `Get`, then `ToString`, -/// then validation against `values` when non-empty; `undefined` yields the -/// fallback. -/// -/// # Errors -/// Propagates getter errors and returns `RangeError` when a defined, -/// coerced value is not an element of `values`. -pub fn get_string_option( - options: &mut dyn OptionsPropertySource, - name: &str, - values: &[&str], - fallback: Option<&str>, -) -> Result, IntlError> { - let value = options.get(name)?; - if matches!(value, IntlValue::Undefined) { - return Ok(fallback.map(str::to_owned)); - } - let text = to_string(&value); - if !values.is_empty() && !values.contains(&text.as_str()) { - return Err(IntlError::range_error(format!("invalid option value: {text}"))); - } - Ok(Some(text)) -} - -/// ECMA-402 `GetOption` with `type` boolean: a single `Get`, then -/// `ToBoolean`; `undefined` yields the fallback. Boolean options carry an -/// empty `values` list per spec, so no membership check applies. -/// -/// # Errors -/// Propagates getter errors. -pub fn get_boolean_option( - options: &mut dyn OptionsPropertySource, - name: &str, - fallback: Option, -) -> Result, IntlError> { - let value = options.get(name)?; - if matches!(value, IntlValue::Undefined) { - return Ok(fallback); - } - Ok(Some(to_boolean(&value))) -} - -/// ECMA-402 `GetNumberOption`: a single `Get`, then `ToNumber`; `NaN` and -/// values outside `[minimum, maximum]` throw `RangeError`; the result is -/// floored. -/// -/// # Errors -/// Propagates getter errors and returns `RangeError` for `NaN` or -/// out-of-range defined values. -pub fn get_number_option( - options: &mut dyn OptionsPropertySource, - name: &str, - minimum: f64, - maximum: f64, - fallback: Option, -) -> Result, IntlError> { - let value = options.get(name)?; - if matches!(value, IntlValue::Undefined) { - return Ok(fallback); - } - let number = to_number(&value); - if number.is_nan() || number < minimum || number > maximum { - return Err(IntlError::range_error("invalid option value")); - } - Ok(Some(number.floor())) -} - -/// ECMA-402 `DefaultNumberOption`: a single `Get`, then `ToNumber`; `NaN` -/// and out-of-range values collapse to `otherwise` instead of throwing; the -/// result is floored. -/// -/// # Errors -/// Propagates getter errors. -pub fn get_default_number_option( - options: &mut dyn OptionsPropertySource, - name: &str, - minimum: f64, - maximum: f64, - otherwise: f64, -) -> Result { - let value = options.get(name)?; - if matches!(value, IntlValue::Undefined) { - return Ok(otherwise); - } - let number = to_number(&value); - if number.is_nan() || number < minimum || number > maximum { - return Ok(otherwise); - } - Ok(number.floor()) -} - -/// Shared `localeMatcher` option read, mapping its text to the C10.1 matcher. -/// Per each service's spec order this is read immediately after any leading -/// `usage` option and before service data options. -/// -/// # Errors -/// Propagates getter errors and returns `RangeError` for other spellings. -pub fn get_locale_matcher_option( - options: &mut dyn OptionsPropertySource, -) -> Result { - let value = - get_string_option(options, "localeMatcher", &["lookup", "best fit"], Some("best fit"))?; - Ok(match value.as_deref() { - Some("lookup") => LocaleMatcher::Lookup, - _ => LocaleMatcher::BestFit, - }) -} - -/// ECMA-262 `ToString` for the represented primitives; `Object` holds the -/// host-provided coercion result. -fn to_string(value: &IntlValue) -> String { - match value { - IntlValue::Undefined => "undefined".to_owned(), - IntlValue::Null => "null".to_owned(), - IntlValue::Boolean(boolean) => boolean.to_string(), - IntlValue::Number(number) => number_to_string(*number), - IntlValue::String(text) | IntlValue::Object(text) => text.clone(), - } -} - -fn number_to_string(number: f64) -> String { - if number.is_nan() { - return "NaN".to_owned(); - } - // Both zeroes print "0"; keeps the option-string surface integral. - if number == 0.0 { - return "0".to_owned(); - } - if number.is_infinite() { - return if number > 0.0 { "Infinity" } else { "-Infinity" }.to_owned(); - } - // Integral values take the shortest exact decimal form. Exponential-edge - // differences from ECMA-262 above 1e21 can only leak into error message - // text, never into a validated option value. - if number.fract() == 0.0 && number.abs() < 1e15 { - return format!("{number:.0}"); - } - format!("{number}") -} - -/// ECMA-262 `ToBoolean` for the represented primitives. -fn to_boolean(value: &IntlValue) -> bool { - match value { - IntlValue::Undefined | IntlValue::Null => false, - IntlValue::Boolean(boolean) => *boolean, - IntlValue::Number(number) => *number != 0.0 && !number.is_nan(), - IntlValue::String(text) => !text.is_empty(), - IntlValue::Object(_) => true, - } -} - -/// ECMA-262 `ToNumber` for the represented primitives; `Object` carries the -/// host-provided primitive string, which is then parsed. -fn to_number(value: &IntlValue) -> f64 { - match value { - IntlValue::Undefined => f64::NAN, - IntlValue::Null => 0.0, - IntlValue::Boolean(boolean) => f64::from(u8::from(*boolean)), - IntlValue::Number(number) => *number, - IntlValue::String(text) | IntlValue::Object(text) => string_to_number(text), - } -} - -/// ECMA-262 `StringNumericLiteral` parse: whitespace-trimmed decimal with -/// optional sign and exponent, `Infinity` with optional sign, and unsigned -/// `0x`/`0o`/`0b` integer forms; everything else is `NaN`. -fn string_to_number(text: &str) -> f64 { - let trimmed = text.trim_matches(is_ecma_whitespace); - if trimmed.is_empty() { - return 0.0; - } - match trimmed { - "Infinity" | "+Infinity" => return f64::INFINITY, - "-Infinity" => return f64::NEG_INFINITY, - _ => {} - } - // Rust's float parser accepts spellings that are not ECMA-262 - // StringNumericLiterals ("inf", "infinity", "NaN" in any case); force - // them to the JavaScript result. - if trimmed.eq_ignore_ascii_case("nan") - || trimmed.trim_start_matches(['-', '+']).eq_ignore_ascii_case("inf") - || trimmed.trim_start_matches(['-', '+']).eq_ignore_ascii_case("infinity") - { - return f64::NAN; - } - let (negative, digits) = match trimmed.strip_prefix('-') { - Some(rest) => (true, rest), - None => (false, trimmed.strip_prefix('+').unwrap_or(trimmed)), - }; - if let Some(prefixed) = parse_prefixed_radix(digits) { - return if negative { -prefixed } else { prefixed }; - } - trimmed.parse::().unwrap_or(f64::NAN) -} - -/// Parses `0x`/`0X`, `0o`/`0O`, and `0b`/`0B` integer literals, returning -/// `None` when the text carries no recognized prefix so the caller can fall -/// through to the decimal parser, and `Some(NaN)` for malformed bodies. -fn parse_prefixed_radix(digits: &str) -> Option { - let (radix, body) = [ - (["0x", "0X"], 16_u32), - (["0o", "0O"], 8_u32), - (["0b", "0B"], 2_u32), - ] - .into_iter() - .find_map(|(prefixes, radix)| { - prefixes - .into_iter() - .find_map(|prefix| digits.strip_prefix(prefix).map(|rest| (radix, rest))) - })?; - if body.is_empty() || !body.bytes().all(|byte| byte.is_ascii_hexdigit()) { - return Some(f64::NAN); - } - if radix != 16 && !body.bytes().all(|byte| byte.is_ascii_digit() && byte < b'0' + radix as u8) - { - return Some(f64::NAN); - } - let mut value = 0.0_f64; - for byte in body.bytes() { - let digit = (byte as char).to_digit(radix)?; - value = value.mul_add(f64::from(radix), f64::from(digit)); - } - Some(value) -} - -/// ECMA-262 WhiteSpace ∪ LineTerminator. -fn is_ecma_whitespace(ch: char) -> bool { - matches!( - ch, - '\t' | '\n' - | '\u{000B}' - | '\u{000C}' - | '\r' - | ' ' - | '\u{00A0}' - | '\u{1680}' - | '\u{2000}'..='\u{200A}' - | '\u{2028}' - | '\u{2029}' - | '\u{202F}' - | '\u{205F}' - | '\u{3000}' - | '\u{FEFF}' - ) -} - #[cfg(test)] mod tests { use super::*; @@ -729,52 +445,66 @@ mod tests { fn inventory() -> MapSupportedValues { MapSupportedValues::new() - .with_values(SupportedValuesKey::Calendar, vec![ - "gregory".into(), - "ROC".into(), - "Buddhist".into(), - "gregory".into(), - ]) - .with_values(SupportedValuesKey::Collation, vec![ - "search".into(), - "emoji".into(), - "standard".into(), - "EMOJI".into(), - "big5han".into(), - ]) - .with_values(SupportedValuesKey::Currency, vec![ - "thb".into(), - "usd".into(), - "EUR".into(), - "USD".into(), - ]) - .with_values(SupportedValuesKey::NumberingSystem, vec![ - "thai".into(), - "Latn".into(), - "thai".into(), - ]) - .with_values(SupportedValuesKey::TimeZone, vec![ - "Europe/Paris".into(), - "UTC".into(), - "America/New_York".into(), - "Europe/Paris".into(), - ]) - .with_values(SupportedValuesKey::Unit, vec![ - "square-meter".into(), - "celsius".into(), - "liter-per-100kilometer".into(), - "celsius".into(), - ]) + .with_values( + SupportedValuesKey::Calendar, + vec![ + "gregory".into(), + "ROC".into(), + "Buddhist".into(), + "gregory".into(), + ], + ) + .with_values( + SupportedValuesKey::Collation, + vec![ + "search".into(), + "emoji".into(), + "standard".into(), + "EMOJI".into(), + "big5han".into(), + ], + ) + .with_values( + SupportedValuesKey::Currency, + vec!["thb".into(), "usd".into(), "EUR".into(), "USD".into()], + ) + .with_values( + SupportedValuesKey::NumberingSystem, + vec!["thai".into(), "Latn".into(), "thai".into()], + ) + .with_values( + SupportedValuesKey::TimeZone, + vec![ + "Europe/Paris".into(), + "UTC".into(), + "America/New_York".into(), + "Europe/Paris".into(), + ], + ) + .with_values( + SupportedValuesKey::Unit, + vec![ + "square-meter".into(), + "celsius".into(), + "liter-per-100kilometer".into(), + "celsius".into(), + ], + ) } #[test] fn canonical_locales_argument_forms() { let data = MapLocaleData::new(); - assert_eq!(tag_list(&LocaleListArgument::Undefined, &data), Vec::::new()); - assert_eq!(tag_list(&LocaleListArgument::Primitive, &data), Vec::::new()); assert_eq!( - get_canonical_locales(&LocaleListArgument::Null, &data) - .map_err(|error| error.kind), + tag_list(&LocaleListArgument::Undefined, &data), + Vec::::new() + ); + assert_eq!( + tag_list(&LocaleListArgument::Primitive, &data), + Vec::::new() + ); + assert_eq!( + get_canonical_locales(&LocaleListArgument::Null, &data).map_err(|error| error.kind), Err(IntlErrorKind::TypeError), ); // A single string argument is wrapped, not iterated character-wise. @@ -803,7 +533,11 @@ mod tests { #[test] fn canonical_list_rejects_non_string_elements() { let data = MapLocaleData::new(); - for element in [IntlValue::Null, IntlValue::Boolean(true), IntlValue::Number(3.0)] { + for element in [ + IntlValue::Null, + IntlValue::Boolean(true), + IntlValue::Number(3.0), + ] { let error = get_canonical_locales(&LocaleListArgument::Elements(vec![element]), &data) .expect_err("accepted a non-string element"); assert_eq!(error.kind, IntlErrorKind::TypeError); @@ -816,19 +550,23 @@ mod tests { for tag in ["", "en_US", "en--US", "en-u", "-en"] { let error = get_canonical_locales(&LocaleListArgument::Text(tag.to_owned()), &data) .expect_err("accepted a malformed tag"); - assert_eq!(error.kind, IntlErrorKind::RangeError, "wrong class for {tag}"); + assert_eq!( + error.kind, + IntlErrorKind::RangeError, + "wrong class for {tag}" + ); } } #[test] fn canonical_list_is_deterministic_across_calls() { let data = MapLocaleData::new().with_language_alias("sh", "sr-Latn"); - let elements = vec![IntlValue::String("SH".into()), IntlValue::String("sh".into())]; + let elements = vec![ + IntlValue::String("SH".into()), + IntlValue::String("sh".into()), + ]; let argument = LocaleListArgument::Elements(elements); - assert_eq!( - tag_list(&argument, &data), - tag_list(&argument, &data), - ); + assert_eq!(tag_list(&argument, &data), tag_list(&argument, &data),); assert_eq!(tag_list(&argument, &data), vec!["sr-Latn".to_owned()]); } @@ -836,28 +574,27 @@ mod tests { fn supported_values_returns_each_key_sorted_unique_canonical() { let data = inventory(); assert_eq!( - supported_values_of("calendar", &data) - .unwrap_or_else(|error| panic!("{error}")), - vec!["buddhist".to_owned(), "gregory".to_owned(), "roc".to_owned()], + supported_values_of("calendar", &data).unwrap_or_else(|error| panic!("{error}")), + vec![ + "buddhist".to_owned(), + "gregory".to_owned(), + "roc".to_owned() + ], ); assert_eq!( - supported_values_of("collation", &data) - .unwrap_or_else(|error| panic!("{error}")), + supported_values_of("collation", &data).unwrap_or_else(|error| panic!("{error}")), vec!["big5han".to_owned(), "emoji".to_owned()], ); assert_eq!( - supported_values_of("currency", &data) - .unwrap_or_else(|error| panic!("{error}")), + supported_values_of("currency", &data).unwrap_or_else(|error| panic!("{error}")), vec!["EUR".to_owned(), "THB".to_owned(), "USD".to_owned()], ); assert_eq!( - supported_values_of("numberingSystem", &data) - .unwrap_or_else(|error| panic!("{error}")), + supported_values_of("numberingSystem", &data).unwrap_or_else(|error| panic!("{error}")), vec!["latn".to_owned(), "thai".to_owned()], ); assert_eq!( - supported_values_of("timeZone", &data) - .unwrap_or_else(|error| panic!("{error}")), + supported_values_of("timeZone", &data).unwrap_or_else(|error| panic!("{error}")), vec![ "America/New_York".to_owned(), "Europe/Paris".to_owned(), @@ -865,8 +602,7 @@ mod tests { ], ); assert_eq!( - supported_values_of("unit", &data) - .unwrap_or_else(|error| panic!("{error}")), + supported_values_of("unit", &data).unwrap_or_else(|error| panic!("{error}")), vec![ "celsius".to_owned(), "liter-per-100kilometer".to_owned(), @@ -880,7 +616,11 @@ mod tests { let data = MapSupportedValues::new(); for key in ["", "Calendar", "calendars", "ca", "timezone", "units"] { let error = supported_values_of(key, &data).expect_err("accepted an invalid key"); - assert_eq!(error.kind, IntlErrorKind::RangeError, "wrong class for {key}"); + assert_eq!( + error.kind, + IntlErrorKind::RangeError, + "wrong class for {key}" + ); assert_eq!(error.message, "Invalid key", "wrong message for {key}"); } } @@ -900,7 +640,11 @@ mod tests { let data = MapSupportedValues::new().with_values(key, vec![entry.to_owned()]); let error = supported_values_of(key.name(), &data) .expect_err("accepted a malformed inventory entry"); - assert_eq!(error.kind, IntlErrorKind::RangeError, "wrong class for {entry}"); + assert_eq!( + error.kind, + IntlErrorKind::RangeError, + "wrong class for {entry}" + ); assert!( error.message.contains(key.name()) && error.message.contains(entry), "message {error:?} names neither key nor value", @@ -910,11 +654,10 @@ mod tests { #[test] fn collation_exclusions_apply_before_sort_and_dedup() { - let data = MapSupportedValues::new().with_values(SupportedValuesKey::Collation, vec![ - "standard".into(), - "search".into(), - "EMO".into(), - ]); + let data = MapSupportedValues::new().with_values( + SupportedValuesKey::Collation, + vec!["standard".into(), "search".into(), "EMO".into()], + ); assert_eq!( supported_values_of("collation", &data).unwrap_or_else(|error| panic!("{error}")), vec!["emo".to_owned()], @@ -924,7 +667,14 @@ mod tests { #[test] fn supported_values_is_deterministic_across_calls() { let data = inventory(); - for key in ["calendar", "collation", "currency", "numberingSystem", "timeZone", "unit"] { + for key in [ + "calendar", + "collation", + "currency", + "numberingSystem", + "timeZone", + "unit", + ] { assert_eq!( supported_values_of(key, &data), supported_values_of(key, &data), @@ -932,198 +682,6 @@ mod tests { } } - struct RecordingSource { - values: BTreeMap, - reads: Vec, - } - - impl RecordingSource { - fn new(values: &[(&str, IntlValue)]) -> Self { - Self { - values: values - .iter() - .map(|(key, value)| ((*key).to_owned(), value.clone())) - .collect(), - reads: Vec::new(), - } - } - } - - impl OptionsPropertySource for RecordingSource { - fn get(&mut self, key: &str) -> Result { - self.reads.push(key.to_owned()); - Ok(self.values.get(key).cloned().unwrap_or(IntlValue::Undefined)) - } - } - - struct ThrowingSource; - - impl OptionsPropertySource for ThrowingSource { - fn get(&mut self, key: &str) -> Result { - Err(IntlError::type_error(format!("getter for {key} threw"))) - } - } - - #[test] - fn options_read_happens_once_per_option_in_caller_order() { - let mut source = RecordingSource::new(&[ - ("usage", IntlValue::String("sort".into())), - ("localeMatcher", IntlValue::String("lookup".into())), - ("numeric", IntlValue::Boolean(false)), - ]); - let usage = get_string_option(&mut source, "usage", &["sort", "search"], None) - .unwrap_or_else(|error| panic!("{error}")); - let matcher = get_locale_matcher_option(&mut source) - .unwrap_or_else(|error| panic!("{error}")); - let numeric = get_boolean_option(&mut source, "numeric", None) - .unwrap_or_else(|error| panic!("{error}")); - assert_eq!(usage, Some("sort".to_owned())); - assert_eq!(matcher, LocaleMatcher::Lookup); - assert_eq!(numeric, Some(false)); - assert_eq!( - source.reads, - vec!["usage".to_owned(), "localeMatcher".to_owned(), "numeric".to_owned()], - ); - } - - #[test] - fn string_option_fallback_validation_and_coercions() { - let mut missing = RecordingSource::new(&[]); - assert_eq!( - get_string_option(&mut missing, "localeMatcher", &["lookup", "best fit"], Some("best fit")) - .unwrap_or_else(|error| panic!("{error}")), - Some("best fit".to_owned()), - ); - let mut wrong = RecordingSource::new(&[("localeMatcher", IntlValue::String("fuzzy".into()))]); - let error = get_string_option(&mut wrong, "localeMatcher", &["lookup", "best fit"], None) - .expect_err("accepted an out-of-list value"); - assert_eq!(error.kind, IntlErrorKind::RangeError); - // `null` coerces to the string "null" and fails list validation. - let mut null = RecordingSource::new(&[("sensitivity", IntlValue::Null)]); - let error = get_string_option( - &mut null, - "sensitivity", - &["base", "accent", "case", "variant"], - None, - ) - .expect_err("accepted null for a valued option"); - assert_eq!(error.kind, IntlErrorKind::RangeError); - // Boolean values stringify before list validation. - let mut boolean = RecordingSource::new(&[("caseFirst", IntlValue::Boolean(true))]); - assert_eq!( - get_string_option(&mut boolean, "caseFirst", &[], None) - .unwrap_or_else(|error| panic!("{error}")), - Some("true".to_owned()), - ); - } - - #[test] - fn boolean_option_applies_to_boolean_rules() { - let cases: [(IntlValue, bool); 5] = [ - (IntlValue::Null, false), - (IntlValue::Number(0.0), false), - (IntlValue::Number(f64::NAN), false), - (IntlValue::String("x".into()), true), - (IntlValue::Object("".into()), true), - ]; - for (value, expected) in cases { - let mut source = RecordingSource::new(&[("pinGroup", value)]); - assert_eq!( - get_boolean_option(&mut source, "pinGroup", Some(false)) - .unwrap_or_else(|error| panic!("{error}")), - Some(expected), - ); - } - let mut missing = RecordingSource::new(&[]); - assert_eq!( - get_boolean_option(&mut missing, "pinGroup", Some(true)) - .unwrap_or_else(|error| panic!("{error}")), - Some(true), - ); - } - - #[test] - fn number_options_follow_spec_edges() { - let mut text = RecordingSource::new(&[("minimumIntegerDigits", IntlValue::String("3".into()))]); - assert_eq!( - get_number_option(&mut text, "minimumIntegerDigits", 1.0, 21.0, None) - .unwrap_or_else(|error| panic!("{error}")), - Some(3.0), - ); - let mut outstanding = RecordingSource::new(&[("maximumFractionDigits", IntlValue::Number(20.9))]); - assert_eq!( - get_number_option(&mut outstanding, "maximumFractionDigits", 0.0, 20.0, None) - .map_err(|error| error.kind), - Err(IntlErrorKind::RangeError), - ); - let mut fractional = RecordingSource::new(&[("roundingIncrement", IntlValue::Number(10.9))]); - assert_eq!( - get_number_option(&mut fractional, "roundingIncrement", 1.0, 20.0, None) - .unwrap_or_else(|error| panic!("{error}")), - Some(10.0), - ); - let mut absent = RecordingSource::new(&[]); - assert_eq!( - get_number_option(&mut absent, "minimumIntegerDigits", 1.0, 21.0, Some(1.0)) - .unwrap_or_else(|error| panic!("{error}")), - Some(1.0), - ); - let mut bad = RecordingSource::new(&[("sign", IntlValue::String("abc".into()))]); - let error = get_number_option(&mut bad, "sign", 1.0, 5.0, None) - .expect_err("accepted a NaN option value"); - assert_eq!(error.kind, IntlErrorKind::RangeError); - // DefaultNumberOption collapses NaN and out-of-range to `otherwise`. - let mut nan = RecordingSource::new(&[("signDisplay", IntlValue::String("xyz".into()))]); - assert_eq!( - get_default_number_option(&mut nan, "signDisplay", 1.0, 5.0, 1.0) - .unwrap_or_else(|error| panic!("{error}")), - 1.0, - ); - let mut high = RecordingSource::new(&[("digits", IntlValue::Number(99.0))]); - assert_eq!( - get_default_number_option(&mut high, "digits", 1.0, 2.0, 1.0) - .unwrap_or_else(|error| panic!("{error}")), - 1.0, - ); - } - - #[test] - fn string_numeric_coercions_match_ecma_262() { - let cases: [(&str, f64); 9] = [ - ("0x10", 16.0), - ("0b101", 5.0), - ("0o17", 15.0), - ("", 0.0), - (" 1.5\t", 1.5), - ("-0", -0.0), - ("Infinity", f64::INFINITY), - ("abc", f64::NAN), - ("inf", f64::NAN), - ]; - for (text, expected) in cases { - let mut source = RecordingSource::new(&[("value", IntlValue::String(text.to_owned()))]); - let result = get_default_number_option(&mut source, "value", f64::NEG_INFINITY, f64::INFINITY, f64::NAN); - if expected.is_nan() { - // DefaultNumberOption maps NaN to `otherwise` (NaN): compare NaN-ness. - assert!(result.map(f64::is_nan) == Ok(true), "wrong result for {text:?}"); - } else { - assert_eq!(result, Ok(expected), "wrong result for {text:?}"); - } - } - // `null` coerces to +0. - let mut source = RecordingSource::new(&[("value", IntlValue::Null)]); - assert_eq!( - get_default_number_option(&mut source, "value", f64::NEG_INFINITY, f64::INFINITY, f64::NAN), - Ok(0.0), - ); - // Object values already carry their coerced string. - let mut source = RecordingSource::new(&[("value", IntlValue::Object("2".into()))]); - assert_eq!( - get_default_number_option(&mut source, "value", f64::NEG_INFINITY, f64::INFINITY, f64::NAN), - Ok(2.0), - ); - } - #[test] fn constructor_and_plain_call_contracts() { assert!(require_constructor_call("Collator", IntlCallForm::Construct).is_ok()); @@ -1136,32 +694,4 @@ mod tests { .expect_err("accepted constructing a plain function"); assert_eq!(error.kind, IntlErrorKind::TypeError); } - - #[test] - fn get_options_object_accepts_only_object_or_undefined() { - let empty = get_options_object(OptionsArgument::Undefined) - .unwrap_or_else(|error| panic!("{error}")); - assert!(empty.is_none()); - let error = get_options_object(OptionsArgument::NotAnObject) - .expect_err("accepted a non-object options argument"); - assert_eq!(error.kind, IntlErrorKind::TypeError); - let mut source = RecordingSource::new(&[("style", IntlValue::String("currency".into()))]); - let object = get_options_object(OptionsArgument::Object(&mut source)) - .unwrap_or_else(|error| panic!("{error}")); - let object = object.expect("object options vanished"); - assert_eq!( - get_string_option(object, "style", &["decimal", "currency", "percent"], None) - .unwrap_or_else(|error| panic!("{error}")), - Some("currency".to_owned()), - ); - } - - #[test] - fn getter_errors_propagate_out_of_option_readers() { - let mut source = ThrowingSource; - let error = get_boolean_option(&mut source, "numeric", None) - .expect_err("swallowed a getter error"); - assert_eq!(error.kind, IntlErrorKind::TypeError); - assert!(error.message.contains("numeric")); - } } diff --git a/crates/bamts-runtime/src/intl/collation.rs b/crates/bamts-runtime/src/intl/collation.rs new file mode 100644 index 00000000..448210ab --- /dev/null +++ b/crates/bamts-runtime/src/intl/collation.rs @@ -0,0 +1,143 @@ +//! A root-locale collation built on Unicode properties. +//! +//! There are no DUCET tables here, so this approximates the Unicode Collation +//! Algorithm rather than implementing it: strings compare on the NFD of their +//! text, by the four levels and class order the algorithm prescribes, with +//! letters of one case-folded base ordered by code point. Canonically +//! equivalent strings are therefore equal, and ASCII orders as it should. + +use std::cmp::Ordering; + +use icu_properties::CodePointMapData; +use icu_properties::props::GeneralCategory; +use unicode_normalization::UnicodeNormalization; + +use super::collator::{CollatorCaseFirst, CollatorCompareRequest, CollatorSensitivity}; +use super::locale_data::decimal_digit_value; + +/// Primary classes, in the order the collation orders them. +const WHITESPACE: u32 = 0; +const PUNCTUATION: u32 = 1; +const SYMBOL: u32 = 2; +const DIGIT: u32 = 3; +const LETTER: u32 = 4; + +/// Numbers of different lengths sort by length when `numeric` is on. Lengths +/// are shifted past the digit values so the two weights cannot be confused. +const NUMERIC_LENGTH_BASE: u32 = 10; + +#[derive(Default)] +struct SortKey { + primary: Vec<(u32, u32)>, + secondary: Vec, + tertiary: Vec, +} + +fn scalars(units: &[u16]) -> String { + char::decode_utf16(units.iter().copied()) + .map(|unit| unit.unwrap_or(char::REPLACEMENT_CHARACTER)) + .collect() +} + +fn folded(character: char) -> char { + character.to_lowercase().next().unwrap_or(character) +} + +fn primary_class(character: char, category: GeneralCategory) -> Option { + use GeneralCategory as Gc; + if character.is_whitespace() { + return Some(WHITESPACE); + } + match category { + Gc::Control | Gc::Format | Gc::Unassigned | Gc::Surrogate => None, + Gc::ConnectorPunctuation + | Gc::DashPunctuation + | Gc::OpenPunctuation + | Gc::ClosePunctuation + | Gc::InitialPunctuation + | Gc::FinalPunctuation + | Gc::OtherPunctuation => Some(PUNCTUATION), + Gc::MathSymbol | Gc::CurrencySymbol | Gc::ModifierSymbol | Gc::OtherSymbol => Some(SYMBOL), + Gc::DecimalNumber => Some(DIGIT), + _ => Some(LETTER), + } +} + +fn sort_key(units: &[u16], ignore_punctuation: bool, numeric: bool) -> SortKey { + let categories = CodePointMapData::::new(); + let text = scalars(units); + let mut key = SortKey::default(); + let characters: Vec = text.nfd().collect(); + let mut index = 0; + while let Some(&character) = characters.get(index) { + let category = categories.get(character); + index += 1; + if matches!( + category, + GeneralCategory::NonspacingMark | GeneralCategory::EnclosingMark + ) { + key.secondary.push(u32::from(character) + 1); + continue; + } + let Some(class) = primary_class(character, category) else { + continue; + }; + if ignore_punctuation && matches!(class, WHITESPACE | PUNCTUATION) { + continue; + } + if class == DIGIT + && numeric + && let Some(first) = decimal_digit_value(character) + { + let mut digits = vec![first]; + while let Some(digit) = characters.get(index).copied().and_then(decimal_digit_value) { + digits.push(digit); + index += 1; + } + let leading = digits.iter().take_while(|digit| **digit == 0).count(); + let significant = &digits[leading.min(digits.len() - 1)..]; + let length = u32::try_from(significant.len()).unwrap_or(u32::MAX); + key.primary + .push((DIGIT, NUMERIC_LENGTH_BASE.saturating_add(length))); + key.primary + .extend(significant.iter().map(|digit| (DIGIT, *digit))); + key.secondary.push(0); + key.tertiary.push(0); + continue; + } + let weight = match decimal_digit_value(character) { + Some(digit) if class == DIGIT => digit, + _ => u32::from(folded(character)), + }; + key.primary.push((class, weight)); + key.secondary.push(0); + key.tertiary.push(u8::from(character.is_uppercase())); + } + key +} + +/// Compares two exact JavaScript strings under the requested options. +pub(super) fn compare(request: &CollatorCompareRequest<'_>) -> Ordering { + let options = request.options; + let left = sort_key(request.left, options.ignore_punctuation, options.numeric); + let right = sort_key(request.right, options.ignore_punctuation, options.numeric); + let primary = left.primary.cmp(&right.primary); + if primary != Ordering::Equal { + return primary; + } + let accents = left.secondary.cmp(&right.secondary); + let case = || { + let ordering = left.tertiary.cmp(&right.tertiary); + if options.case_first == CollatorCaseFirst::Upper { + ordering.reverse() + } else { + ordering + } + }; + match options.sensitivity { + CollatorSensitivity::Base => Ordering::Equal, + CollatorSensitivity::Accent => accents, + CollatorSensitivity::Case => case(), + CollatorSensitivity::Variant => accents.then_with(case), + } +} diff --git a/crates/bamts-runtime/src/intl/collator.rs b/crates/bamts-runtime/src/intl/collator.rs index a2aa8417..31261b33 100644 --- a/crates/bamts-runtime/src/intl/collator.rs +++ b/crates/bamts-runtime/src/intl/collator.rs @@ -44,18 +44,26 @@ impl CollatorError { /// Creates a JavaScript `TypeError` result. #[must_use] pub fn type_error(message: impl Into) -> Self { - Self { kind: CollatorErrorKind::TypeError, message: message.into() } + Self { + kind: CollatorErrorKind::TypeError, + message: message.into(), + } } /// Creates a JavaScript `RangeError` result. #[must_use] pub fn range_error(message: impl Into) -> Self { - Self { kind: CollatorErrorKind::RangeError, message: message.into() } + Self { + kind: CollatorErrorKind::RangeError, + message: message.into(), + } } } impl From for CollatorError { - fn from(error: LocaleError) -> Self { Self::range_error(error.to_string()) } + fn from(error: LocaleError) -> Self { + Self::range_error(error.to_string()) + } } impl fmt::Display for CollatorError { @@ -231,11 +239,7 @@ pub struct CollatorCompareRequest<'a> { /// when an injected provider deliberately has no tailoring for a request. pub trait CollatorDataProvider: LocaleDataProvider { /// Returns defaults from the resolved sort/search locale-data record. - fn locale_defaults( - &self, - data_locale: &str, - usage: CollatorUsage, - ) -> CollatorLocaleDefaults; + fn locale_defaults(&self, data_locale: &str, usage: CollatorUsage) -> CollatorLocaleDefaults; /// Compares two exact JavaScript strings, or selects the UTF-16 fallback. fn compare(&self, request: CollatorCompareRequest<'_>) -> Option; @@ -314,18 +318,12 @@ impl Collator { ))); } let numeric = read_boolean_option(&mut options, "numeric")?; - let case_first = read_string_option( - &mut options, - "caseFirst", - &["upper", "lower", "false"], - )?; + let case_first = + read_string_option(&mut options, "caseFirst", &["upper", "lower", "false"])?; let mut resolution_options = BTreeMap::new(); if let Some(value) = collation { - resolution_options.insert( - "co".to_owned(), - canonicalize_uvalue(provider, "co", &value), - ); + resolution_options.insert("co".to_owned(), canonicalize_uvalue(provider, "co", &value)); } if let Some(value) = numeric { resolution_options.insert( @@ -401,12 +399,7 @@ impl Collator { /// Compares exact JavaScript UTF-16 strings and normalizes the provider's /// implementation-defined number to `-1`, `0`, or `1`. #[must_use] - pub fn compare( - &self, - provider: &dyn CollatorDataProvider, - left: &[u16], - right: &[u16], - ) -> i8 { + pub fn compare(&self, provider: &dyn CollatorDataProvider, left: &[u16], right: &[u16]) -> i8 { let request = CollatorCompareRequest { options: CollatorCompareOptions { data_locale: &self.data_locale, @@ -525,13 +518,12 @@ fn parse_sensitivity(value: &str) -> CollatorSensitivity { } } -fn canonicalize_uvalue( - provider: &dyn LocaleDataProvider, - key: &str, - value: &str, -) -> String { +fn canonicalize_uvalue(provider: &dyn LocaleDataProvider, key: &str, value: &str) -> String { let lower = value.to_ascii_lowercase(); - provider.unicode_type_alias(key, &lower).unwrap_or(&lower).to_ascii_lowercase() + provider + .unicode_type_alias(key, &lower) + .unwrap_or(&lower) + .to_ascii_lowercase() } fn is_unicode_type(value: &str) -> bool { @@ -552,8 +544,8 @@ const fn ordering_number(ordering: Ordering) -> i8 { #[cfg(test)] mod tests { - use super::*; use super::super::locale_negotiation::LanguageId; + use super::*; use std::cell::{Cell, RefCell}; #[derive(Clone, Debug, Eq, PartialEq)] @@ -572,12 +564,14 @@ mod tests { impl Options { fn string(mut self, key: &'static str, value: &str) -> Self { - self.values.insert(key, CoercedCollatorOption::String(value.to_owned())); + self.values + .insert(key, CoercedCollatorOption::String(value.to_owned())); self } fn boolean(mut self, key: &'static str, value: bool) -> Self { - self.values.insert(key, CoercedCollatorOption::Boolean(value)); + self.values + .insert(key, CoercedCollatorOption::Boolean(value)); self } } @@ -682,7 +676,9 @@ mod tests { } impl LocaleDataProvider for TestProvider { - fn available_locales(&self) -> &[String] { &self.available } + fn available_locales(&self) -> &[String] { + &self.available + } fn unicode_type_alias(&self, key: &str, value: &str) -> Option<&str> { self.type_aliases @@ -708,7 +704,9 @@ mod tests { }) } - fn fallback_locale(&self) -> Option<&str> { self.fallback.as_deref() } + fn fallback_locale(&self) -> Option<&str> { + self.fallback.as_deref() + } } impl CollatorDataProvider for TestProvider { @@ -717,7 +715,9 @@ mod tests { data_locale: &str, usage: CollatorUsage, ) -> CollatorLocaleDefaults { - self.defaults_calls.borrow_mut().push((data_locale.to_owned(), usage)); + self.defaults_calls + .borrow_mut() + .push((data_locale.to_owned(), usage)); CollatorLocaleDefaults { sensitivity: CollatorSensitivity::Base, ignore_punctuation: usage == CollatorUsage::Search, @@ -754,11 +754,7 @@ mod tests { request.options.sensitivity, CollatorSensitivity::Case | CollatorSensitivity::Variant ) && request.options.case_first != CollatorCaseFirst::False - && let Some(ordering) = compare_case_variants( - &left, - &right, - request.options.case_first, - ) + && let Some(ordering) = compare_case_variants(&left, &right, request.options.case_first) { return ordering; } @@ -784,9 +780,10 @@ mod tests { .map(ascii_lower) .collect(), CollatorSensitivity::Accent => decomposed.into_iter().map(ascii_lower).collect(), - CollatorSensitivity::Case => { - decomposed.into_iter().filter(|unit| *unit != 0x0301).collect() - } + CollatorSensitivity::Case => decomposed + .into_iter() + .filter(|unit| *unit != 0x0301) + .collect(), CollatorSensitivity::Variant => decomposed, } } @@ -800,7 +797,7 @@ mod tests { } const fn is_ascii_punctuation(unit: u16) -> bool { - (unit >= 0x21 && unit <= 0x2f) + (unit >= 0x20 && unit <= 0x2f) || (unit >= 0x3a && unit <= 0x40) || (unit >= 0x5b && unit <= 0x60) || (unit >= 0x7b && unit <= 0x7e) @@ -864,11 +861,11 @@ mod tests { (left.len() - left_index).cmp(&(right.len() - right_index)) } - fn digit_run_end(units: &[u16], start: usize) -> usize { const fn is_ascii_digit_unit(unit: u16) -> bool { unit >= b'0' as u16 && unit <= b'9' as u16 } + fn digit_run_end(units: &[u16], start: usize) -> usize { let mut end = start; while end < units.len() && is_ascii_digit_unit(units[end]) { end += 1; @@ -881,7 +878,9 @@ mod tests { first_nonzero.map_or(&units[units.len()..], |index| &units[index..]) } - fn utf16(value: &str) -> Vec { value.encode_utf16().collect() } + fn utf16(value: &str) -> Vec { + value.encode_utf16().collect() + } #[test] fn constructor_reads_hostile_options_in_specification_order_once() { @@ -950,12 +949,8 @@ mod tests { #[test] fn supported_unicode_keywords_survive_and_provider_defaults_fill_absent_keys() { let provider = TestProvider::populated(); - let collator = Collator::new( - &["de-u-kn-kf-upper-co-phonebk".to_owned()], - None, - &provider, - ) - .expect("locale extension is supported"); + let collator = Collator::new(&["de-u-kn-kf-upper-co-phonebk".to_owned()], None, &provider) + .expect("locale extension is supported"); let resolved = collator.resolved_options(); assert_eq!(resolved.locale, "de-u-co-phonebk-kf-upper-kn"); @@ -963,9 +958,9 @@ mod tests { assert!(resolved.numeric); assert_eq!(resolved.case_first, CollatorCaseFirst::Upper); - let default = Collator::new(&["de".to_owned()], None, &provider) - .expect("provider has a default locale record") - .resolved_options(); + let collator = Collator::new(&["de".to_owned()], None, &provider) + .expect("provider has a default locale record"); + let default = collator.resolved_options(); assert_eq!(default.collation, "default"); assert!(!default.numeric); assert_eq!(default.case_first, CollatorCaseFirst::False); @@ -974,19 +969,21 @@ mod tests { #[test] fn sort_and_search_use_their_required_default_sources() { let provider = TestProvider::populated(); - let sort = Collator::new(&["en".to_owned()], None, &provider) - .expect("sort defaults resolve"); + let sort = + Collator::new(&["en".to_owned()], None, &provider).expect("sort defaults resolve"); let mut search_options = Options::default().string("usage", "search"); - let search = Collator::new( - &["en".to_owned()], - Some(&mut search_options), - &provider, - ) - .expect("search defaults resolve"); + let search = Collator::new(&["en".to_owned()], Some(&mut search_options), &provider) + .expect("search defaults resolve"); - assert_eq!(sort.resolved_options().sensitivity, CollatorSensitivity::Variant); + assert_eq!( + sort.resolved_options().sensitivity, + CollatorSensitivity::Variant + ); assert!(!sort.resolved_options().ignore_punctuation); - assert_eq!(search.resolved_options().sensitivity, CollatorSensitivity::Base); + assert_eq!( + search.resolved_options().sensitivity, + CollatorSensitivity::Base + ); assert!(search.resolved_options().ignore_punctuation); assert_eq!( *provider.defaults_calls.borrow(), @@ -1001,32 +998,24 @@ mod tests { fn locale_and_option_failures_keep_error_classes_and_short_circuit_getters() { let provider = TestProvider::populated(); let mut untouched = Options::default(); - let invalid_locale = Collator::new( - &["en--US".to_owned()], - Some(&mut untouched), - &provider, - ) - .expect_err("ill-formed locales are rejected"); + let invalid_locale = Collator::new(&["en--US".to_owned()], Some(&mut untouched), &provider) + .expect_err("ill-formed locales are rejected"); assert_eq!(invalid_locale.kind, CollatorErrorKind::RangeError); assert!(untouched.events.is_empty()); - let mut null_options = Options { fail_coercion: true, ..Options::default() }; - let null_error = Collator::new( - &["en".to_owned()], - Some(&mut null_options), - &provider, - ) - .expect_err("null ToObject is a TypeError"); + let mut null_options = Options { + fail_coercion: true, + ..Options::default() + }; + let null_error = Collator::new(&["en".to_owned()], Some(&mut null_options), &provider) + .expect_err("null ToObject is a TypeError"); assert_eq!(null_error.kind, CollatorErrorKind::TypeError); assert_eq!(null_options.events, vec![OptionEvent::Coerce]); let mut invalid_collation = Options::default().string("collation", "ab"); - let range_error = Collator::new( - &["en".to_owned()], - Some(&mut invalid_collation), - &provider, - ) - .expect_err("a Unicode type subtag needs at least three characters"); + let range_error = + Collator::new(&["en".to_owned()], Some(&mut invalid_collation), &provider) + .expect_err("a Unicode type subtag needs at least three characters"); assert_eq!(range_error.kind, CollatorErrorKind::RangeError); assert!(range_error.message.contains("collation")); assert_eq!( @@ -1039,15 +1028,17 @@ mod tests { ] ); - let mut hostile = Options { fail_property: Some("caseFirst"), ..Options::default() }; - let hostile_error = Collator::new( - &["en".to_owned()], - Some(&mut hostile), - &provider, - ) - .expect_err("getter failure propagates"); + let mut hostile = Options { + fail_property: Some("caseFirst"), + ..Options::default() + }; + let hostile_error = Collator::new(&["en".to_owned()], Some(&mut hostile), &provider) + .expect_err("getter failure propagates"); assert_eq!(hostile_error.kind, CollatorErrorKind::TypeError); - assert_eq!(hostile.events.last(), Some(&OptionEvent::Get("caseFirst", CollatorOptionType::String))); + assert_eq!( + hostile.events.last(), + Some(&OptionEvent::Get("caseFirst", CollatorOptionType::String)) + ); assert!(!hostile.events.iter().any(|event| matches!( event, OptionEvent::Get("sensitivity" | "ignorePunctuation", _) @@ -1076,21 +1067,21 @@ mod tests { ] ); - let best_fit = supported_locales_of(&locales, None, &provider) - .expect("best-fit filtering succeeds"); + let best_fit = + supported_locales_of(&locales, None, &provider).expect("best-fit filtering succeeds"); assert_eq!(best_fit, vec!["sv-SE", "zh-HK", "en-US"]); } #[test] fn supported_locales_validate_locales_before_touching_options() { let provider = TestProvider::populated(); - let mut options = Options { fail_coercion: true, ..Options::default() }; - let error = supported_locales_of( - &["not_a_locale".to_owned()], - Some(&mut options), - &provider, - ) - .expect_err("locale failure precedes options coercion"); + let mut options = Options { + fail_coercion: true, + ..Options::default() + }; + let error = + supported_locales_of(&["not_a_locale".to_owned()], Some(&mut options), &provider) + .expect_err("locale failure precedes options coercion"); assert_eq!(error.kind, CollatorErrorKind::RangeError); assert!(options.events.is_empty()); } @@ -1102,14 +1093,13 @@ mod tests { .boolean("numeric", true) .string("caseFirst", "lower") .boolean("ignorePunctuation", true); - let collator = Collator::new( - &["en".to_owned()], - Some(&mut options), - &provider, - ) - .expect("comparison options resolve"); + let collator = Collator::new(&["en".to_owned()], Some(&mut options), &provider) + .expect("comparison options resolve"); - assert_eq!(collator.compare(&provider, &utf16("item-2"), &utf16("item 10")), -1); + assert_eq!( + collator.compare(&provider, &utf16("item-2"), &utf16("item 10")), + -1 + ); assert_eq!(collator.compare(&provider, &utf16("a"), &utf16("A")), -1); assert_eq!(collator.compare(&provider, &utf16("a-b"), &utf16("ab")), 0); @@ -1117,14 +1107,18 @@ mod tests { assert_eq!(calls.len(), 3); assert!(calls.iter().all(|call| call.numeric)); assert!(calls.iter().all(|call| call.ignore_punctuation)); - assert!(calls.iter().all(|call| call.case_first == CollatorCaseFirst::Lower)); + assert!( + calls + .iter() + .all(|call| call.case_first == CollatorCaseFirst::Lower) + ); } #[test] fn canonical_equivalence_remains_equal_without_a_code_unit_tiebreak() { let provider = TestProvider::populated(); - let collator = Collator::new(&["en".to_owned()], None, &provider) - .expect("default collator resolves"); + let collator = + Collator::new(&["en".to_owned()], None, &provider).expect("default collator resolves"); let composed = vec![0x00e9]; let decomposed = vec!['e' as u16, 0x0301]; @@ -1139,8 +1133,8 @@ mod tests { fn utf16_fallback_is_total_and_preserves_lone_surrogates() { let provider = TestProvider::populated(); provider.defer_compare.set(true); - let collator = Collator::new(&["en".to_owned()], None, &provider) - .expect("default collator resolves"); + let collator = + Collator::new(&["en".to_owned()], None, &provider).expect("default collator resolves"); assert_eq!(collator.compare(&provider, &[0xd800], &[0xdc00]), -1); assert_eq!(collator.compare(&provider, &[0xd800], &[0xd800]), 0); @@ -1161,12 +1155,8 @@ mod tests { .string("caseFirst", "upper") .string("sensitivity", "accent") .boolean("ignorePunctuation", false); - let collator = Collator::new( - &["de".to_owned()], - Some(&mut options), - &provider, - ) - .expect("collator resolves"); + let collator = Collator::new(&["de".to_owned()], Some(&mut options), &provider) + .expect("collator resolves"); let left = utf16("A2"); let right = utf16("a10"); @@ -1187,8 +1177,8 @@ mod tests { #[test] fn absent_provider_data_fails_instead_of_reading_an_ambient_locale() { let provider = TestProvider::empty(); - let error = Collator::new(&[], None, &provider) - .expect_err("no provider locale means no Collator"); + let error = + Collator::new(&[], None, &provider).expect_err("no provider locale means no Collator"); assert_eq!(error.kind, CollatorErrorKind::RangeError); assert_eq!(error.message, "no available locale"); diff --git a/crates/bamts-runtime/src/intl/date_data.rs b/crates/bamts-runtime/src/intl/date_data.rs new file mode 100644 index 00000000..9ebfe20d --- /dev/null +++ b/crates/bamts-runtime/src/intl/date_data.rs @@ -0,0 +1,795 @@ +//! English date and time data: patterns, names, and calendar dates. + +use super::date_time_format::{ + CalendarDateTime, DateTimeComponents, DateTimeFormatDataProvider, DateTimePattern, + DateTimeRangePattern, DateTimeStyle, HourCycle, HourCyclePreferences, IntegerFormatRequest, + MonthStyle, NumericStyle, PatternField, PatternItem, PatternRequest, ProviderError, + RangeDifference, RangePatternItem, RangePatternRequest, RangeSource, TextField, + TextFormatRequest, TextWidth, TimeZoneNameStyle, +}; +use super::locale_data::{IntlData, numbering_system_digits, time_zone}; +use crate::temporal::calendar::Calendar; + +const NANOSECONDS_PER_MILLISECOND: i128 = 1_000_000; +const MILLISECONDS_PER_DAY: i128 = 86_400_000; + +const WEEKDAYS: [&str; 7] = [ + "Monday", + "Tuesday", + "Wednesday", + "Thursday", + "Friday", + "Saturday", + "Sunday", +]; +const MONTHS: [&str; 12] = [ + "January", + "February", + "March", + "April", + "May", + "June", + "July", + "August", + "September", + "October", + "November", + "December", +]; +const COPTIC_MONTHS: [&str; 13] = [ + "Tout", + "Baba", + "Hator", + "Kiahk", + "Toba", + "Amshir", + "Baramhat", + "Baramouda", + "Bashans", + "Paona", + "Epep", + "Mesra", + "Nasie", +]; +const ETHIOPIC_MONTHS: [&str; 13] = [ + "Meskerem", "Tekemt", "Hedar", "Tahsas", "Ter", "Yekatit", "Megabit", "Miazia", "Genbot", + "Sene", "Hamle", "Nehasse", "Pagumen", +]; +const INDIAN_MONTHS: [&str; 12] = [ + "Chaitra", + "Vaisakha", + "Jyaistha", + "Asadha", + "Sravana", + "Bhadra", + "Asvina", + "Kartika", + "Agrahayana", + "Pausa", + "Magha", + "Phalguna", +]; +const ISLAMIC_MONTHS: [&str; 12] = [ + "Muharram", + "Safar", + "Rabi\u{02BB} I", + "Rabi\u{02BB} II", + "Jumada I", + "Jumada II", + "Rajab", + "Sha\u{02BB}ban", + "Ramadan", + "Shawwal", + "Dhu\u{02BB}l-Qi\u{02BB}dah", + "Dhu\u{02BB}l-Hijjah", +]; +const PERSIAN_MONTHS: [&str; 12] = [ + "Farvardin", + "Ordibehesht", + "Khordad", + "Tir", + "Mordad", + "Shahrivar", + "Mehr", + "Aban", + "Azar", + "Dey", + "Bahman", + "Esfand", +]; +/// Hebrew months by month code, M01 to M12, without the leap month. +const HEBREW_MONTHS: [&str; 12] = [ + "Tishri", "Heshvan", "Kislev", "Tevet", "Shevat", "Adar", "Nisan", "Iyar", "Sivan", "Tamuz", + "Av", "Elul", +]; + +fn literal(text: &str) -> PatternItem { + PatternItem::Literal(text.to_owned()) +} + +fn field(field: PatternField) -> PatternItem { + PatternItem::Field(field) +} + +/// Appends `items` to `pattern` after `separator`, when both have something. +fn join(pattern: &mut Vec, separator: &str, items: Vec) { + if !pattern.is_empty() && !items.is_empty() { + pattern.push(literal(separator)); + } + pattern.extend(items); +} + +/// The date half of a pattern, in English order. +fn date_items(components: &DateTimeComponents) -> Vec { + let month = components + .month + .map(|style| field(PatternField::Month(style))); + let day = components.day.map(|style| field(PatternField::Day(style))); + let year = components + .year + .map(|style| field(PatternField::Year(style))); + let textual = matches!( + components.month, + Some(MonthStyle::Narrow | MonthStyle::Short | MonthStyle::Long) + ); + let mut items = Vec::new(); + if let Some(width) = components.weekday { + items.push(field(PatternField::Weekday(width))); + } + let mut body = Vec::new(); + if textual { + body.extend(month); + join(&mut body, " ", day.clone().into_iter().collect()); + let separator = if day.is_some() { ", " } else { " " }; + join(&mut body, separator, year.into_iter().collect()); + } else { + body.extend(month); + join(&mut body, "/", day.into_iter().collect()); + join(&mut body, "/", year.into_iter().collect()); + } + join(&mut items, ", ", body); + if let Some(width) = components.era { + join(&mut items, " ", vec![field(PatternField::Era(width))]); + } + items +} + +/// The time half of a pattern. Minutes and seconds following a larger field +/// always take two digits, and a 24-hour hour does too, as English writes them. +fn time_items(components: &DateTimeComponents, cycle: HourCycle) -> Vec { + let twelve_hour = cycle.is_twelve_hour(); + let mut items = Vec::new(); + let mut leading = true; + if let Some(style) = components.hour { + let style = if twelve_hour { + style + } else { + NumericStyle::TwoDigit + }; + items.push(field(PatternField::Hour(style))); + leading = false; + } + if let Some(style) = components.minute { + let style = if leading && components.second.is_none() { + style + } else { + NumericStyle::TwoDigit + }; + join(&mut items, ":", vec![field(PatternField::Minute(style))]); + leading = false; + } + if let Some(style) = components.second { + let style = if leading { + style + } else { + NumericStyle::TwoDigit + }; + join(&mut items, ":", vec![field(PatternField::Second(style))]); + } + if let Some(digits) = components.fractional_second_digits { + join( + &mut items, + ".", + vec![field(PatternField::FractionalSecond(digits))], + ); + } + if let Some(width) = components.day_period { + join(&mut items, " ", vec![field(PatternField::DayPeriod(width))]); + } else if components.hour.is_some() && twelve_hour { + items.push(literal(" ")); + items.push(field(PatternField::AmPm)); + } + items +} + +/// What the pattern for `components` resolves each field to, for +/// `resolvedOptions`. +fn resolved_components(components: &DateTimeComponents, cycle: HourCycle) -> DateTimeComponents { + let mut resolved = components.clone(); + if resolved.hour.is_some() && !cycle.is_twelve_hour() { + resolved.hour = Some(NumericStyle::TwoDigit); + } + if resolved.minute.is_some() && (resolved.hour.is_some() || resolved.second.is_some()) { + resolved.minute = Some(NumericStyle::TwoDigit); + } + if resolved.second.is_some() && resolved.minute.is_some() { + resolved.second = Some(NumericStyle::TwoDigit); + } + resolved +} + +fn components_pattern(components: &DateTimeComponents, cycle: HourCycle) -> DateTimePattern { + let mut items = date_items(components); + join(&mut items, ", ", time_items(components, cycle)); + if let Some(style) = components.time_zone_name { + join( + &mut items, + " ", + vec![field(PatternField::TimeZoneName(style))], + ); + } + DateTimePattern { + components: resolved_components(components, cycle), + items, + } +} + +fn date_style_items(style: DateTimeStyle) -> Vec { + let (month, day, year) = ( + match style { + DateTimeStyle::Full | DateTimeStyle::Long => MonthStyle::Long, + DateTimeStyle::Medium => MonthStyle::Short, + DateTimeStyle::Short => MonthStyle::Numeric, + }, + NumericStyle::Numeric, + if style == DateTimeStyle::Short { + NumericStyle::TwoDigit + } else { + NumericStyle::Numeric + }, + ); + date_items(&DateTimeComponents { + weekday: (style == DateTimeStyle::Full).then_some(TextWidth::Long), + month: Some(month), + day: Some(day), + year: Some(year), + ..DateTimeComponents::default() + }) +} + +fn time_style_items(style: DateTimeStyle, cycle: HourCycle) -> Vec { + let mut items = time_items( + &DateTimeComponents { + hour: Some(NumericStyle::Numeric), + minute: Some(NumericStyle::TwoDigit), + second: (style != DateTimeStyle::Short).then_some(NumericStyle::TwoDigit), + ..DateTimeComponents::default() + }, + cycle, + ); + let name = match style { + DateTimeStyle::Full => Some(TimeZoneNameStyle::Long), + DateTimeStyle::Long => Some(TimeZoneNameStyle::Short), + DateTimeStyle::Medium | DateTimeStyle::Short => None, + }; + if let Some(name) = name { + join( + &mut items, + " ", + vec![field(PatternField::TimeZoneName(name))], + ); + } + items +} + +fn style_pattern( + date: Option, + time: Option, + cycle: HourCycle, +) -> DateTimePattern { + let mut items = date.map_or_else(Vec::new, date_style_items); + let separator = if matches!(date, Some(DateTimeStyle::Full | DateTimeStyle::Long)) { + " at " + } else { + ", " + }; + if let Some(style) = time { + join(&mut items, separator, time_style_items(style, cycle)); + } + DateTimePattern { + components: DateTimeComponents::default(), + items, + } +} + +/// Where a pattern field sits among the differences a range can have. +fn difference_of(item: &PatternItem) -> Option { + let PatternItem::Field(field) = item else { + return None; + }; + Some(match field { + PatternField::Era(_) => RangeDifference::Era, + PatternField::Year(_) | PatternField::RelatedYear(_) | PatternField::YearName(_) => { + RangeDifference::Year + } + PatternField::Month(_) => RangeDifference::Month, + PatternField::Day(_) | PatternField::Weekday(_) => RangeDifference::Day, + PatternField::DayPeriod(_) | PatternField::AmPm => RangeDifference::DayPeriod, + PatternField::Hour(_) => RangeDifference::Hour, + PatternField::Minute(_) => RangeDifference::Minute, + PatternField::Second(_) => RangeDifference::Second, + PatternField::FractionalSecond(_) => RangeDifference::FractionalSecond, + PatternField::TimeZoneName(_) => return None, + }) +} + +const fn difference_rank(difference: RangeDifference) -> u8 { + match difference { + RangeDifference::Era => 0, + RangeDifference::Year => 1, + RangeDifference::Month => 2, + RangeDifference::Day => 3, + RangeDifference::DayPeriod => 4, + RangeDifference::Hour => 5, + RangeDifference::Minute => 6, + RangeDifference::Second => 7, + RangeDifference::FractionalSecond => 8, + } +} + +fn is_date_difference(difference: RangeDifference) -> bool { + difference_rank(difference) <= difference_rank(RangeDifference::Day) +} + +const RANGE_SEPARATOR: &str = " \u{2013} "; + +fn sourced(items: &[PatternItem], source: RangeSource) -> Vec { + items + .iter() + .map(|item| RangePatternItem { + source, + item: item.clone(), + }) + .collect() +} + +/// Both ends written out in full, around the separator. +fn full_range(items: &[PatternItem]) -> DateTimeRangePattern { + let mut result = sourced(items, RangeSource::StartRange); + result.push(RangePatternItem { + source: RangeSource::Shared, + item: literal(RANGE_SEPARATOR), + }); + result.extend(sourced(items, RangeSource::EndRange)); + DateTimeRangePattern { items: result } +} + +/// A range that shares what cannot differ: every field coarser than +/// `difference`. Numeric dates are always repeated whole, and so is a date +/// next to a time when the dates differ. +fn range_pattern(pattern: &DateTimePattern, difference: RangeDifference) -> DateTimeRangePattern { + let items = &pattern.items; + let has_time = items + .iter() + .any(|item| difference_of(item).is_some_and(|kind| !is_date_difference(kind))); + let has_date = items + .iter() + .any(|item| difference_of(item).is_some_and(is_date_difference)); + let numeric_month = items.iter().any(|item| { + matches!( + item, + PatternItem::Field(PatternField::Month( + MonthStyle::Numeric | MonthStyle::TwoDigit + )) + ) + }); + let date_differs = is_date_difference(difference); + if date_differs && (numeric_month || (has_date && has_time)) { + return full_range(items); + } + let rank = difference_rank(difference); + let varies = + |item: &PatternItem| difference_of(item).is_some_and(|kind| difference_rank(kind) >= rank); + let (Some(first), Some(last)) = ( + items.iter().position(varies), + items.iter().rposition(varies), + ) else { + return full_range(items); + }; + let mut result = sourced(&items[..first], RangeSource::Shared); + result.extend(sourced(&items[first..=last], RangeSource::StartRange)); + result.push(RangePatternItem { + source: RangeSource::Shared, + item: literal(RANGE_SEPARATOR), + }); + result.extend(sourced(&items[first..=last], RangeSource::EndRange)); + result.extend(sourced(&items[last + 1..], RangeSource::Shared)); + DateTimeRangePattern { items: result } +} + +/// Days since the Unix epoch to a proleptic Gregorian date. +fn civil_from_days(days: i64) -> (i64, u8, u8) { + let shifted = days + 719_468; + let era = shifted.div_euclid(146_097); + let day_of_era = shifted.rem_euclid(146_097); + let year_of_era = + (day_of_era - day_of_era / 1460 + day_of_era / 36_524 - day_of_era / 146_096) / 365; + let day_of_year = day_of_era - (365 * year_of_era + year_of_era / 4 - year_of_era / 100); + let shifted_month = (5 * day_of_year + 2) / 153; + let day = day_of_year - (153 * shifted_month + 2) / 5 + 1; + let month = if shifted_month < 10 { + shifted_month + 3 + } else { + shifted_month - 9 + }; + let year = year_of_era + era * 400 + i64::from(month <= 2); + (year, month as u8, day as u8) +} + +fn local_days(local_epoch_nanoseconds: i128) -> i64 { + local_epoch_nanoseconds + .div_euclid(NANOSECONDS_PER_MILLISECOND) + .div_euclid(MILLISECONDS_PER_DAY) as i64 +} + +fn gregorian_date_time(local_epoch_nanoseconds: i128) -> CalendarDateTime { + let milliseconds = local_epoch_nanoseconds.div_euclid(NANOSECONDS_PER_MILLISECOND); + let days = milliseconds.div_euclid(MILLISECONDS_PER_DAY); + let of_day = milliseconds.rem_euclid(MILLISECONDS_PER_DAY) as u32; + let (year, month, day) = civil_from_days(days as i64); + let (era, display_year) = if year <= 0 { + ("bce", 1 - year) + } else { + ("ce", year) + }; + CalendarDateTime { + era: Some(era.to_owned()), + year: display_year as i32, + related_year: None, + year_name: None, + month, + month_code: format!("M{month:02}"), + day, + weekday: ((days + 3).rem_euclid(7) + 1) as u8, + hour: (of_day / 3_600_000) as u8, + minute: (of_day / 60_000 % 60) as u8, + second: (of_day / 1000 % 60) as u8, + millisecond: (of_day % 1000) as u16, + } +} + +/// A calendar's view of a local time; the weekday and time of day are the same +/// in every calendar. +fn calendar_view(calendar: Calendar, local_epoch_nanoseconds: i128) -> CalendarDateTime { + let date = calendar.date(local_days(local_epoch_nanoseconds)); + let (era, year) = calendar.era(date); + CalendarDateTime { + era: Some(era.to_owned()), + year: year as i32, + month: date.month, + month_code: calendar.month_code(date.year, date.month).render(), + day: date.day, + ..gregorian_date_time(local_epoch_nanoseconds) + } +} + +/// The name of a Hebrew month: Adar is Adar II in a year that has Adar I. +fn hebrew_month_name(local: &CalendarDateTime) -> &'static str { + match local.month_code.as_str() { + "M05L" => "Adar I", + "M06" if local.month == 7 => "Adar II", + code => { + let number = code[1..].parse::().unwrap_or(1); + HEBREW_MONTHS[number - 1] + } + } +} + +fn month_name(calendar: &str, local: &CalendarDateTime) -> &'static str { + let names: &[&'static str] = match calendar { + "coptic" => &COPTIC_MONTHS, + "ethioaa" | "ethiopic" => ÐIOPIC_MONTHS, + "hebrew" => return hebrew_month_name(local), + "indian" => &INDIAN_MONTHS, + "islamic-civil" | "islamic-tbla" => &ISLAMIC_MONTHS, + "persian" => &PERSIAN_MONTHS, + _ => &MONTHS, + }; + names[usize::from(local.month) - 1] +} + +/// Calendars other than Gregorian show their era wherever they show a year. +fn show_calendar_era(calendar: &str, pattern: &mut DateTimePattern) { + if matches!(calendar, "gregory" | "iso8601") { + return; + } + let has_era = |item: &PatternItem| matches!(item, PatternItem::Field(PatternField::Era(_))); + let is_year = |item: &PatternItem| matches!(item, PatternItem::Field(PatternField::Year(_))); + if pattern.items.iter().any(has_era) { + return; + } + if let Some(year) = pattern.items.iter().position(is_year) { + pattern.items.insert(year + 1, literal(" ")); + pattern + .items + .insert(year + 2, field(PatternField::Era(TextWidth::Short))); + } +} + +fn day_period_text(hour: u8, width: TextWidth) -> &'static str { + match hour { + 6..=11 => "in the morning", + 12 if width == TextWidth::Narrow => "n", + 12 => "noon", + 13..=17 => "in the afternoon", + 18..=20 => "in the evening", + _ => "at night", + } +} + +fn era_text(era: &str, width: TextWidth) -> &'static str { + match (era, width) { + ("bce", TextWidth::Long) => "Before Christ", + ("bce", TextWidth::Short) => "BC", + ("bce", TextWidth::Narrow) => "B", + ("be", _) => "BE", + ("am", _) => "AM", + ("aa", _) => "AA", + ("shaka", _) => "Saka", + ("ah", _) => "AH", + ("bh", _) => "BH", + ("ap", _) => "AP", + ("roc", _) => "Minguo", + ("broc", _) => "Before R.O.C.", + (_, TextWidth::Long) => "Anno Domini", + (_, TextWidth::Short) => "AD", + (_, TextWidth::Narrow) => "A", + } +} + +fn abbreviated(name: &str, width: TextWidth) -> String { + match width { + TextWidth::Long => name.to_owned(), + TextWidth::Short => name.chars().take(3).collect(), + TextWidth::Narrow => name.chars().take(1).collect(), + } +} + +/// The offset of a time zone identifier, in minutes. +fn zone_offset_minutes(identifier: &str) -> Option { + if let Some((_, hours)) = time_zone(identifier) { + return Some(i32::from(hours) * 60); + } + let (sign, body) = match identifier.as_bytes().first()? { + b'+' => (1, &identifier[1..]), + b'-' => (-1, &identifier[1..]), + _ => return None, + }; + let (hours, minutes) = body.split_once(':')?; + Some(sign * (hours.parse::().ok()? * 60 + minutes.parse::().ok()?)) +} + +fn time_zone_text(identifier: &str, style: TimeZoneNameStyle) -> String { + if identifier == "UTC" { + match style { + TimeZoneNameStyle::Short => return String::from("UTC"), + TimeZoneNameStyle::Long => return String::from("Coordinated Universal Time"), + _ => {} + } + } + let offset = zone_offset_minutes(identifier).unwrap_or(0); + if offset == 0 { + return String::from("GMT"); + } + let sign = if offset < 0 { '-' } else { '+' }; + let (hours, minutes) = (offset.abs() / 60, offset.abs() % 60); + match style { + TimeZoneNameStyle::Long + | TimeZoneNameStyle::LongOffset + | TimeZoneNameStyle::LongGeneric => format!("GMT{sign}{hours:02}:{minutes:02}"), + _ if minutes == 0 => format!("GMT{sign}{hours}"), + _ => format!("GMT{sign}{hours}:{minutes:02}"), + } +} + +impl DateTimeFormatDataProvider for IntlData { + fn default_time_zone(&self) -> Result { + Ok(String::from("UTC")) + } + + fn canonicalize_time_zone(&self, identifier: &str) -> Result, ProviderError> { + Ok(time_zone(identifier).map(|(name, _)| name.to_owned())) + } + + fn hour_cycle_preferences( + &self, + _data_locale: &str, + ) -> Result { + Ok(HourCyclePreferences { + default: HourCycle::H12, + twelve_hour: HourCycle::H12, + twenty_four_hour: HourCycle::H23, + }) + } + + fn select_pattern( + &self, + request: &PatternRequest<'_>, + ) -> Result { + let mut pattern = if request.date_style.is_some() || request.time_style.is_some() { + style_pattern(request.date_style, request.time_style, request.hour_cycle) + } else { + components_pattern(request.components, request.hour_cycle) + }; + show_calendar_era(request.calendar, &mut pattern); + Ok(pattern) + } + + fn select_range_pattern( + &self, + request: &RangePatternRequest<'_>, + ) -> Result, ProviderError> { + Ok(Some(match request.largest_difference { + Some(difference) => range_pattern(request.pattern, difference), + None => full_range(&request.pattern.items), + })) + } + + fn named_time_zone_offset_nanoseconds( + &self, + identifier: &str, + _epoch_nanoseconds: i128, + ) -> Result { + time_zone(identifier) + .map(|(_, hours)| i64::from(hours) * 3_600_000_000_000) + .ok_or_else(|| ProviderError::missing("time zone", identifier)) + } + + fn calendar_date_time( + &self, + calendar: &str, + local_epoch_nanoseconds: i128, + ) -> Result { + match calendar { + "gregory" | "iso8601" => Ok(gregorian_date_time(local_epoch_nanoseconds)), + other => Calendar::from_name(other) + .map(|calendar| calendar_view(calendar, local_epoch_nanoseconds)) + .ok_or_else(|| ProviderError::missing("calendar", other)), + } + } + + fn format_integer(&self, request: &IntegerFormatRequest<'_>) -> Result { + let digits = numbering_system_digits(request.numbering_system) + .ok_or_else(|| ProviderError::missing("numbering system", request.numbering_system))?; + let text = format!( + "{:0>width$}", + request.value, + width = usize::from(request.minimum_digits) + ); + Ok(text + .chars() + .map(|character| { + character + .to_digit(10) + .map_or(character, |digit| digits[digit as usize]) + }) + .collect()) + } + + fn style_components( + &self, + date: Option, + time: Option, + ) -> DateTimeComponents { + let mut components = DateTimeComponents::default(); + if let Some(style) = date { + components.weekday = (style == DateTimeStyle::Full).then_some(TextWidth::Long); + components.month = Some(match style { + DateTimeStyle::Full | DateTimeStyle::Long => MonthStyle::Long, + DateTimeStyle::Medium => MonthStyle::Short, + DateTimeStyle::Short => MonthStyle::Numeric, + }); + components.day = Some(NumericStyle::Numeric); + components.year = Some(if style == DateTimeStyle::Short { + NumericStyle::TwoDigit + } else { + NumericStyle::Numeric + }); + } + if let Some(style) = time { + components.hour = Some(NumericStyle::Numeric); + components.minute = Some(NumericStyle::TwoDigit); + components.second = (style != DateTimeStyle::Short).then_some(NumericStyle::TwoDigit); + } + components + } + + fn format_text(&self, request: &TextFormatRequest<'_>) -> Result { + let local = request.local; + Ok(match request.field { + TextField::Weekday(width) => { + abbreviated(WEEKDAYS[usize::from(local.weekday) - 1], width) + } + TextField::Era(width) => { + era_text(local.era.as_deref().unwrap_or("ce"), width).to_owned() + } + TextField::YearName(_) => local.year_name.clone().unwrap_or_default(), + TextField::Month(width) => abbreviated(month_name(request.calendar, local), width), + TextField::DayPeriod(width) => day_period_text(local.hour, width).to_owned(), + TextField::AmPm => String::from(if local.hour < 12 { "AM" } else { "PM" }), + TextField::TimeZoneName(style) => time_zone_text(request.time_zone, style), + }) + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const NANOSECONDS_PER_DAY: i128 = 86_400_000_000_000; + + fn view(calendar: Calendar, days: i128) -> CalendarDateTime { + calendar_view(calendar, days * NANOSECONDS_PER_DAY) + } + + #[test] + fn calendar_view_shows_the_era_year_and_keeps_the_weekday() { + // 2024-03-26, a Tuesday. + let days = 19_808; + let gregorian = gregorian_date_time(days * NANOSECONDS_PER_DAY); + assert_eq!(gregorian.weekday, 2); + + let buddhist = view(Calendar::Buddhist, days); + assert_eq!((buddhist.year, buddhist.month, buddhist.day), (2567, 3, 26)); + assert_eq!(buddhist.era.as_deref(), Some("be")); + assert_eq!(buddhist.weekday, gregorian.weekday); + + let islamic = view(Calendar::IslamicCivil, days); + assert_eq!((islamic.year, islamic.month), (1445, 9)); + assert_eq!(month_name("islamic-civil", &islamic), "Ramadan"); + + // Before 1912 the ROC year counts down from the era's start. + let before_roc = view(Calendar::Roc, -30_000); + assert_eq!(before_roc.era.as_deref(), Some("broc")); + assert!(before_roc.year > 0); + } + + #[test] + fn hebrew_month_names_follow_the_month_code() { + let calendar = Calendar::Hebrew; + let named = |year, month| { + let days = calendar.epoch_days(year, month, 1); + let local = view(calendar, i128::from(days)); + month_name("hebrew", &local) + }; + // 5784 is a leap year, 5783 is not. + assert_eq!(named(5784, 6), "Adar I"); + assert_eq!(named(5784, 7), "Adar II"); + assert_eq!(named(5784, 8), "Nisan"); + assert_eq!(named(5783, 6), "Adar"); + assert_eq!(named(5783, 7), "Nisan"); + assert_eq!(named(5783, 12), "Elul"); + } + + #[test] + fn only_calendars_other_than_gregorian_add_an_era() { + let year_pattern = || DateTimePattern { + components: DateTimeComponents::default(), + items: vec![field(PatternField::Year(NumericStyle::Numeric))], + }; + let mut plain = year_pattern(); + show_calendar_era("gregory", &mut plain); + assert_eq!(plain.items.len(), 1); + show_calendar_era("iso8601", &mut plain); + assert_eq!(plain.items.len(), 1); + + let mut with_era = year_pattern(); + show_calendar_era("buddhist", &mut with_era); + assert_eq!( + with_era.items[2], + field(PatternField::Era(TextWidth::Short)) + ); + show_calendar_era("buddhist", &mut with_era); + assert_eq!(with_era.items.len(), 3, "an era is added only once"); + } +} diff --git a/crates/bamts-runtime/src/intl/date_time_format.rs b/crates/bamts-runtime/src/intl/date_time_format.rs index a6d97b38..5d497410 100644 --- a/crates/bamts-runtime/src/intl/date_time_format.rs +++ b/crates/bamts-runtime/src/intl/date_time_format.rs @@ -36,12 +36,18 @@ pub enum ProviderError { impl ProviderError { #[must_use] pub fn missing(kind: &'static str, key: impl Into) -> Self { - Self::MissingData { kind, key: key.into() } + Self::MissingData { + kind, + key: key.into(), + } } #[must_use] pub fn invalid(kind: &'static str, detail: impl Into) -> Self { - Self::InvalidData { kind, detail: detail.into() } + Self::InvalidData { + kind, + detail: detail.into(), + } } } @@ -76,7 +82,10 @@ impl DateTimeFormatError { } fn range(option: &'static str, value: impl Into) -> Self { - Self::Range { option, value: value.into() } + Self::Range { + option, + value: value.into(), + } } } @@ -102,11 +111,15 @@ impl Error for DateTimeFormatError { } impl From for DateTimeFormatError { - fn from(value: LocaleError) -> Self { Self::Locale(value) } + fn from(value: LocaleError) -> Self { + Self::Locale(value) + } } impl From for DateTimeFormatError { - fn from(value: ProviderError) -> Self { Self::Provider(value) } + fn from(value: ProviderError) -> Self { + Self::Provider(value) + } } /// ECMA-402 hour-cycle identifiers. @@ -140,7 +153,9 @@ impl HourCycle { } #[must_use] - pub const fn is_twelve_hour(self) -> bool { matches!(self, Self::H11 | Self::H12) } + pub const fn is_twelve_hour(self) -> bool { + matches!(self, Self::H11 | Self::H12) + } } /// Provider preferences used when `hour12` suppresses the `hc` option. @@ -244,26 +259,21 @@ impl DateTimeComponents { Ok(()) } - fn has_any(&self) -> bool { - self.weekday.is_some() - || self.era.is_some() - || self.year.is_some() - || self.month.is_some() - || self.day.is_some() - || self.day_period.is_some() - || self.hour.is_some() - || self.minute.is_some() - || self.second.is_some() - || self.fractional_second_digits.is_some() - || self.time_zone_name.is_some() + /// Whether every component is unset, which a style needs of them. + fn is_empty(&self) -> bool { + !self.has_any() && self.era.is_none() && self.time_zone_name.is_none() } + /// Whether anything in the `any` required list is set. `era` and + /// `timeZoneName` qualify a date or time but do not ask for one. + #[must_use] + pub fn has_any(&self) -> bool { + self.has_date() || self.has_time() + } + + /// Whether anything in the `date` required list is set. fn has_date(&self) -> bool { - self.weekday.is_some() - || self.era.is_some() - || self.year.is_some() - || self.month.is_some() - || self.day.is_some() + self.weekday.is_some() || self.year.is_some() || self.month.is_some() || self.day.is_some() } fn has_time(&self) -> bool { @@ -344,16 +354,25 @@ pub struct CalendarDateTime { impl CalendarDateTime { fn validate(&self) -> Result<(), ProviderError> { if !(1..=13).contains(&self.month) { - return Err(ProviderError::invalid("calendar", "month must be in 1..=13")); + return Err(ProviderError::invalid( + "calendar", + "month must be in 1..=13", + )); } if self.month_code.is_empty() { - return Err(ProviderError::invalid("calendar", "month_code must not be empty")); + return Err(ProviderError::invalid( + "calendar", + "month_code must not be empty", + )); } if !(1..=31).contains(&self.day) { return Err(ProviderError::invalid("calendar", "day must be in 1..=31")); } if !(1..=7).contains(&self.weekday) { - return Err(ProviderError::invalid("calendar", "weekday must be in 1..=7")); + return Err(ProviderError::invalid( + "calendar", + "weekday must be in 1..=7", + )); } if self.hour > 23 || self.minute > 59 || self.second > 59 || self.millisecond > 999 { return Err(ProviderError::invalid( @@ -375,6 +394,8 @@ pub enum PatternField { Month(MonthStyle), Day(NumericStyle), DayPeriod(TextWidth), + /// The AM or PM of a 12-hour clock. + AmPm, Hour(NumericStyle), Minute(NumericStyle), Second(NumericStyle), @@ -398,7 +419,10 @@ pub struct DateTimePattern { impl DateTimePattern { fn validate(&self) -> Result<(), ProviderError> { if self.items.is_empty() { - return Err(ProviderError::invalid("pattern", "pattern must not be empty")); + return Err(ProviderError::invalid( + "pattern", + "pattern must not be empty", + )); } for item in &self.items { if let PatternItem::Field(PatternField::FractionalSecond(digits)) = item @@ -414,39 +438,57 @@ impl DateTimePattern { } fn has_hour(&self) -> bool { - self.items.iter().any(|item| matches!(item, PatternItem::Field(PatternField::Hour(_)))) + self.items + .iter() + .any(|item| matches!(item, PatternItem::Field(PatternField::Hour(_)))) } fn has_field(&self, difference: RangeDifference) -> bool { self.items.iter().any(|item| { matches!( (difference, item), - (RangeDifference::Era, PatternItem::Field(PatternField::Era(_))) - | ( - RangeDifference::Year, - PatternItem::Field( - PatternField::Year(_) - | PatternField::RelatedYear(_) - | PatternField::YearName(_) - ) - ) - | (RangeDifference::Month, PatternItem::Field(PatternField::Month(_))) - | (RangeDifference::Day, PatternItem::Field(PatternField::Day(_))) - | (RangeDifference::DayPeriod, PatternItem::Field(PatternField::DayPeriod(_))) - | (RangeDifference::Hour, PatternItem::Field(PatternField::Hour(_))) - | (RangeDifference::Minute, PatternItem::Field(PatternField::Minute(_))) - | (RangeDifference::Second, PatternItem::Field(PatternField::Second(_))) - | ( - RangeDifference::FractionalSecond, - PatternItem::Field(PatternField::FractionalSecond(_)) + ( + RangeDifference::Era, + PatternItem::Field(PatternField::Era(_)) + ) | ( + RangeDifference::Year, + PatternItem::Field( + PatternField::Year(_) + | PatternField::RelatedYear(_) + | PatternField::YearName(_) ) + ) | ( + RangeDifference::Month, + PatternItem::Field(PatternField::Month(_)) + ) | ( + RangeDifference::Day, + PatternItem::Field(PatternField::Day(_)) + ) | ( + RangeDifference::DayPeriod, + PatternItem::Field(PatternField::DayPeriod(_) | PatternField::AmPm) + ) | ( + RangeDifference::Hour, + PatternItem::Field(PatternField::Hour(_)) + ) | ( + RangeDifference::Minute, + PatternItem::Field(PatternField::Minute(_)) + ) | ( + RangeDifference::Second, + PatternItem::Field(PatternField::Second(_)) + ) | ( + RangeDifference::FractionalSecond, + PatternItem::Field(PatternField::FractionalSecond(_)) + ) ) }) } - fn day_period_width(&self) -> Option { + fn day_period_field(&self) -> Option { self.items.iter().find_map(|item| match item { - PatternItem::Field(PatternField::DayPeriod(width)) => Some(*width), + PatternItem::Field(PatternField::DayPeriod(width)) => { + Some(TextField::DayPeriod(*width)) + } + PatternItem::Field(PatternField::AmPm) => Some(TextField::AmPm), PatternItem::Literal(_) | PatternItem::Field(_) => None, }) } @@ -480,7 +522,10 @@ pub struct DateTimeRangePattern { impl DateTimeRangePattern { fn validate(&self) -> Result<(), ProviderError> { if self.items.is_empty() { - return Err(ProviderError::invalid("range pattern", "pattern must not be empty")); + return Err(ProviderError::invalid( + "range pattern", + "pattern must not be empty", + )); } for item in &self.items { if let PatternItem::Field(PatternField::FractionalSecond(digits)) = &item.item @@ -549,6 +594,7 @@ pub enum TextField { YearName(TextWidth), Month(TextWidth), DayPeriod(TextWidth), + AmPm, TimeZoneName(TimeZoneNameStyle), } @@ -576,8 +622,10 @@ pub trait DateTimeFormatDataProvider: LocaleDataProvider { data_locale: &str, ) -> Result; - fn select_pattern(&self, request: &PatternRequest<'_>) - -> Result; + fn select_pattern( + &self, + request: &PatternRequest<'_>, + ) -> Result; /// Returns a difference-specific range pattern or `None` when the caller /// must request the mandatory default pattern with `largest_difference=None`. @@ -600,18 +648,28 @@ pub trait DateTimeFormatDataProvider: LocaleDataProvider { local_epoch_nanoseconds: i128, ) -> Result; - fn format_integer( - &self, - request: &IntegerFormatRequest<'_>, - ) -> Result; + fn format_integer(&self, request: &IntegerFormatRequest<'_>) -> Result; fn format_text(&self, request: &TextFormatRequest<'_>) -> Result; + + /// The components a `dateStyle` and `timeStyle` amount to, for a Temporal + /// type that can show only some of them. + fn style_components( + &self, + _date: Option, + _time: Option, + ) -> DateTimeComponents { + DateTimeComponents::default() + } } #[derive(Clone, Debug, Eq, PartialEq)] enum CanonicalTimeZone { Named(String), - Offset { identifier: String, nanoseconds: i64 }, + Offset { + identifier: String, + nanoseconds: i64, + }, } impl CanonicalTimeZone { @@ -695,19 +753,119 @@ pub struct DateTimeFormat { resolved: ResolvedDateTimeFormatOptions, time_zone: CanonicalTimeZone, pattern: DateTimePattern, + /// The cycle in effect, whether or not the main pattern has an hour. + hour_cycle: HourCycle, + format_matcher: FormatMatcher, + /// The components as requested, before defaults. + requested: DateTimeComponents, +} + +/// A Temporal type with its own formatter, per CreateDateTimeFormat. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum TemporalFormatKind { + PlainDate, + PlainYearMonth, + PlainMonthDay, + PlainTime, + PlainDateTime, + Instant, +} + +/// The components of `base` a Temporal type can show, or `None` when it can +/// show none of them. +fn filter_components( + base: &DateTimeComponents, + kind: TemporalFormatKind, +) -> Option { + let date = |result: &mut DateTimeComponents, weekday: bool, day: bool, era: bool| { + if weekday { + result.weekday = base.weekday; + } + if era { + result.era = base.era; + } + result.year = base.year; + result.month = base.month; + if day { + result.day = base.day; + } + }; + let time = |result: &mut DateTimeComponents| { + result.day_period = base.day_period; + result.hour = base.hour; + result.minute = base.minute; + result.second = base.second; + result.fractional_second_digits = base.fractional_second_digits; + }; + let mut result = DateTimeComponents::default(); + match kind { + TemporalFormatKind::PlainDate => date(&mut result, true, true, true), + TemporalFormatKind::PlainYearMonth => date(&mut result, false, false, true), + TemporalFormatKind::PlainMonthDay => { + date(&mut result, false, true, false); + result.year = None; + } + TemporalFormatKind::PlainTime => time(&mut result), + TemporalFormatKind::PlainDateTime => { + date(&mut result, true, true, true); + time(&mut result); + } + TemporalFormatKind::Instant => result = base.clone(), + } + Some(result) +} + +/// The fields a Temporal type falls back on when no requested one applies. +fn default_components(kind: TemporalFormatKind, components: &mut DateTimeComponents) { + use TemporalFormatKind as Kind; + if matches!( + kind, + Kind::PlainDate | Kind::PlainYearMonth | Kind::PlainDateTime | Kind::Instant + ) { + components.year = Some(NumericStyle::Numeric); + } + if kind != Kind::PlainTime { + components.month = Some(MonthStyle::Numeric); + } + if matches!( + kind, + Kind::PlainDate | Kind::PlainMonthDay | Kind::PlainDateTime | Kind::Instant + ) { + components.day = Some(NumericStyle::Numeric); + } + if matches!(kind, Kind::PlainTime | Kind::PlainDateTime | Kind::Instant) { + components.add_time_defaults(); + } +} + +fn shows_required(components: &DateTimeComponents, kind: TemporalFormatKind) -> bool { + use TemporalFormatKind as Kind; + match kind { + Kind::PlainDate => components.has_date(), + Kind::PlainYearMonth => components.year.is_some() || components.month.is_some(), + Kind::PlainMonthDay => components.month.is_some() || components.day.is_some(), + Kind::PlainTime => components.has_time(), + Kind::PlainDateTime | Kind::Instant => components.has_any(), + } } +/// Which components a formatter must end up with, or take the defaults. #[derive(Clone, Copy)] -enum RequiredComponents { +pub enum RequiredComponents { Date, Time, + YearMonth, + MonthDay, Any, } +/// Which components the defaults add. #[derive(Clone, Copy)] -enum DefaultComponents { +pub enum DefaultComponents { Date, Time, + YearMonth, + MonthDay, All, } @@ -730,7 +888,9 @@ impl DateTimeFormat { ) } - fn try_new_with_mode( + /// Constructs a formatter for the given component requirements, as + /// `Date.prototype.toLocaleDateString` and its peers do. + pub fn try_new_with_mode( locales: &[String], options: &DateTimeFormatOptions, required: RequiredComponents, @@ -754,20 +914,20 @@ impl DateTimeFormat { canonicalize_unicode_type("calendar", "ca", calendar, provider)?, ); } - if options.hour12.is_none() - && let Some(hour_cycle) = options.hour_cycle - { - locale_options.insert("hc".to_owned(), hour_cycle.as_str().to_owned()); + match (options.hour12, options.hour_cycle) { + // hour12 overrides any requested cycle, and the extension with it. + (Some(_), _) => { + locale_options.insert("hc".to_owned(), String::new()); + } + (None, Some(hour_cycle)) => { + locale_options.insert("hc".to_owned(), hour_cycle.as_str().to_owned()); + } + (None, None) => {} } if let Some(numbering_system) = &options.numbering_system { locale_options.insert( "nu".to_owned(), - canonicalize_unicode_type( - "numberingSystem", - "nu", - numbering_system, - provider, - )?, + canonicalize_unicode_type("numberingSystem", "nu", numbering_system, provider)?, ); } @@ -781,7 +941,9 @@ impl DateTimeFormat { )?; let calendar = resolved_keyword(&locale.values, "ca")?; let numbering_system = resolved_keyword(&locale.values, "nu")?; - let preferences = provider.hour_cycle_preferences(&locale.data_locale)?.validate()?; + let preferences = provider + .hour_cycle_preferences(&locale.data_locale)? + .validate()?; let hour_cycle = match options.hour12 { Some(true) => preferences.twelve_hour, Some(false) => preferences.twenty_four_hour, @@ -839,11 +1001,121 @@ impl DateTimeFormat { components: resolved_components, }; - Ok(Self { resolved, time_zone, pattern }) + Ok(Self { + resolved, + time_zone, + pattern, + hour_cycle, + format_matcher: options.format_matcher, + requested: options.components.clone(), + }) + } + + /// GetDateTimeFormat for a Temporal type: the pattern that type uses, or + /// `None` when the options and the type do not overlap. + fn temporal_pattern( + &self, + kind: TemporalFormatKind, + provider: &dyn DateTimeFormatDataProvider, + ) -> Result, DateTimeFormatError> { + let (date_style, time_style) = (self.resolved.date_style, self.resolved.time_style); + let components = if date_style.is_some() || time_style.is_some() { + if matches!(kind, TemporalFormatKind::Instant) { + return Ok(Some(self.pattern.clone())); + } + let needs_date = matches!( + kind, + TemporalFormatKind::PlainDate + | TemporalFormatKind::PlainYearMonth + | TemporalFormatKind::PlainMonthDay + ); + if (needs_date && date_style.is_none()) + || (kind == TemporalFormatKind::PlainTime && time_style.is_none()) + { + return Ok(None); + } + filter_components(&provider.style_components(date_style, time_style), kind) + } else { + let mut filtered = filter_components(&self.requested, kind); + if let Some(components) = &mut filtered + && !shows_required(components, kind) + { + if kind != TemporalFormatKind::Instant && self.requested.has_any() { + return Ok(None); + } + default_components(kind, components); + } + filtered + }; + let Some(components) = components else { + return Ok(None); + }; + let pattern = provider.select_pattern(&PatternRequest { + locale: &self.resolved.locale, + data_locale: &self.resolved.data_locale, + calendar: &self.resolved.calendar, + numbering_system: &self.resolved.numbering_system, + time_zone: self.time_zone.identifier(), + hour_cycle: self.hour_cycle, + format_matcher: self.format_matcher, + date_style: None, + time_style: None, + components: &components, + })?; + pattern.validate()?; + Ok(Some(pattern)) + } + + /// The zone a Temporal type renders in. Only an Instant has a place on + /// the time line; the plain types show their fields as they are. + fn temporal_zone(&self, kind: TemporalFormatKind) -> CanonicalTimeZone { + if kind == TemporalFormatKind::Instant { + self.time_zone.clone() + } else { + CanonicalTimeZone::Offset { + identifier: String::from("+00:00"), + nanoseconds: 0, + } + } + } + + /// Partitions a Temporal value, given as epoch nanoseconds. `Ok(None)` + /// means the type has no format here, which is a TypeError. + pub fn format_temporal_to_parts( + &self, + kind: TemporalFormatKind, + epoch_nanoseconds: i128, + provider: &dyn DateTimeFormatDataProvider, + ) -> Result>, DateTimeFormatError> { + let Some(pattern) = self.temporal_pattern(kind, provider)? else { + return Ok(None); + }; + let zone = self.temporal_zone(kind); + let local = self.local_time_in(&zone, epoch_nanoseconds, provider)?; + self.render_pattern(&pattern.items, epoch_nanoseconds, &local, provider) + .map(Some) + } + + /// Partitions a range of two values of one Temporal type. + pub fn format_temporal_range_to_parts( + &self, + kind: TemporalFormatKind, + start: i128, + end: i128, + provider: &dyn DateTimeFormatDataProvider, + ) -> Result>, DateTimeFormatError> { + let Some(pattern) = self.temporal_pattern(kind, provider)? else { + return Ok(None); + }; + let zone = self.temporal_zone(kind); + self.range_parts(&pattern, &zone, start, end, provider) + .map(Some) } #[must_use] - pub const fn resolved_options(&self) -> &ResolvedDateTimeFormatOptions { &self.resolved } + pub const fn resolved_options(&self) -> &ResolvedDateTimeFormatOptions { + &self.resolved + } pub fn format( &self, @@ -870,13 +1142,12 @@ impl DateTimeFormat { end_epoch_milliseconds: f64, provider: &dyn DateTimeFormatDataProvider, ) -> Result { - let parts = self.format_range_to_parts( - start_epoch_milliseconds, - end_epoch_milliseconds, - provider, - )?; + let parts = + self.format_range_to_parts(start_epoch_milliseconds, end_epoch_milliseconds, provider)?; let mut result = String::new(); - for part in parts { result.push_str(&part.value); } + for part in parts { + result.push_str(&part.value); + } Ok(result) } @@ -888,10 +1159,23 @@ impl DateTimeFormat { ) -> Result, DateTimeFormatError> { let start_instant = time_clip_to_nanoseconds(start_epoch_milliseconds)?; let end_instant = time_clip_to_nanoseconds(end_epoch_milliseconds)?; - let start_local = self.to_local_time(start_instant, provider)?; - let end_local = self.to_local_time(end_instant, provider)?; + let zone = self.time_zone.clone(); + self.range_parts(&self.pattern, &zone, start_instant, end_instant, provider) + } + + fn range_parts( + &self, + pattern: &DateTimePattern, + zone: &CanonicalTimeZone, + start_instant: i128, + end_instant: i128, + provider: &dyn DateTimeFormatDataProvider, + ) -> Result, DateTimeFormatError> { + let start_local = self.local_time_in(zone, start_instant, provider)?; + let end_local = self.local_time_in(zone, end_instant, provider)?; let difference = self.largest_range_difference( + pattern, start_instant, &start_local, end_instant, @@ -900,7 +1184,7 @@ impl DateTimeFormat { )?; let Some(difference) = difference else { return Ok(self - .render_pattern(&self.pattern.items, start_instant, &start_local, provider)? + .render_pattern(&pattern.items, start_instant, &start_local, provider)? .into_iter() .map(|part| DateTimeRangePart { part_type: part.part_type, @@ -916,8 +1200,8 @@ impl DateTimeFormat { calendar: &self.resolved.calendar, numbering_system: &self.resolved.numbering_system, time_zone: self.time_zone.identifier(), - hour_cycle: self.resolved.hour_cycle.unwrap_or(HourCycle::H23), - pattern: &self.pattern, + hour_cycle: self.hour_cycle, + pattern, largest_difference, }; let range_pattern = match provider.select_range_pattern(&request(Some(difference)))? { @@ -949,11 +1233,20 @@ impl DateTimeFormat { epoch_nanoseconds: i128, provider: &dyn DateTimeFormatDataProvider, ) -> Result { - let offset = match &self.time_zone { + self.local_time_in(&self.time_zone, epoch_nanoseconds, provider) + } + + fn local_time_in( + &self, + zone: &CanonicalTimeZone, + epoch_nanoseconds: i128, + provider: &dyn DateTimeFormatDataProvider, + ) -> Result { + let offset = match zone { CanonicalTimeZone::Offset { nanoseconds, .. } => i128::from(*nanoseconds), CanonicalTimeZone::Named(identifier) => { - let offset = provider - .named_time_zone_offset_nanoseconds(identifier, epoch_nanoseconds)?; + let offset = + provider.named_time_zone_offset_nanoseconds(identifier, epoch_nanoseconds)?; let offset = i128::from(offset); if !(-NANOS_PER_DAY..NANOS_PER_DAY).contains(&offset) { return Err(ProviderError::invalid( @@ -996,7 +1289,9 @@ impl DateTimeFormat { ) -> Result { let (part_type, value) = match item { PatternItem::Literal(value) => (DateTimePartType::Literal, value.clone()), - PatternItem::Field(field) => self.render_field(field, epoch_nanoseconds, local, provider)?, + PatternItem::Field(field) => { + self.render_field(field, epoch_nanoseconds, local, provider)? + } }; Ok(DateTimePart { part_type, value }) } @@ -1041,13 +1336,15 @@ impl DateTimeFormat { PatternField::Weekday(width) => { Ok((DateTimePartType::Weekday, text(TextField::Weekday(width))?)) } - PatternField::Era(width) => { - Ok((DateTimePartType::Era, text(TextField::Era(width))?)) + PatternField::Era(width) => Ok((DateTimePartType::Era, text(TextField::Era(width))?)), + PatternField::Year(style) => { + let year = i64::from(local.year); + let year = match style { + NumericStyle::Numeric => year, + NumericStyle::TwoDigit => year.rem_euclid(100), + }; + Ok((DateTimePartType::Year, integer(year, numeric_width(style))?)) } - PatternField::Year(style) => Ok(( - DateTimePartType::Year, - integer(i64::from(local.year), numeric_width(style))?, - )), PatternField::RelatedYear(style) => { let year = local.related_year.ok_or_else(|| { ProviderError::missing("calendar related year", &self.resolved.calendar) @@ -1057,18 +1354,17 @@ impl DateTimeFormat { integer(i64::from(year), numeric_width(style))?, )) } - PatternField::YearName(width) => { - Ok((DateTimePartType::YearName, text(TextField::YearName(width))?)) - } + PatternField::YearName(width) => Ok(( + DateTimePartType::YearName, + text(TextField::YearName(width))?, + )), PatternField::Month(style) => match style { - MonthStyle::Numeric => Ok(( - DateTimePartType::Month, - integer(i64::from(local.month), 1)?, - )), - MonthStyle::TwoDigit => Ok(( - DateTimePartType::Month, - integer(i64::from(local.month), 2)?, - )), + MonthStyle::Numeric => { + Ok((DateTimePartType::Month, integer(i64::from(local.month), 1)?)) + } + MonthStyle::TwoDigit => { + Ok((DateTimePartType::Month, integer(i64::from(local.month), 2)?)) + } MonthStyle::Narrow => Ok(( DateTimePartType::Month, text(TextField::Month(TextWidth::Narrow))?, @@ -1090,18 +1386,22 @@ impl DateTimeFormat { DateTimePartType::DayPeriod, text(TextField::DayPeriod(width))?, )), + PatternField::AmPm => Ok((DateTimePartType::DayPeriod, text(TextField::AmPm)?)), PatternField::Hour(style) => { - let cycle = self.resolved.hour_cycle.ok_or_else(|| { - ProviderError::invalid("pattern", "hour field has no resolved hour cycle") - })?; - let hour = match cycle { + let hour = match self.hour_cycle { HourCycle::H11 => local.hour % 12, HourCycle::H12 => { let hour = local.hour % 12; if hour == 0 { 12 } else { hour } } HourCycle::H23 => local.hour, - HourCycle::H24 => if local.hour == 0 { 24 } else { local.hour }, + HourCycle::H24 => { + if local.hour == 0 { + 24 + } else { + local.hour + } + } }; Ok(( DateTimePartType::Hour, @@ -1143,6 +1443,7 @@ impl DateTimeFormat { fn largest_range_difference( &self, + pattern: &DateTimePattern, start_instant: i128, start: &CalendarDateTime, end_instant: i128, @@ -1161,7 +1462,7 @@ impl DateTimeFormat { RangeDifference::FractionalSecond, ]; for difference in differences { - if !self.pattern.has_field(difference) { + if !pattern.has_field(difference) { continue; } let equal = match difference { @@ -1176,7 +1477,7 @@ impl DateTimeFormat { } RangeDifference::Day => start.day == end.day, RangeDifference::DayPeriod => { - let width = self.pattern.day_period_width().ok_or_else(|| { + let period = pattern.day_period_field().ok_or_else(|| { ProviderError::invalid("pattern", "missing day-period width") })?; let format = |instant, local| { @@ -1188,7 +1489,7 @@ impl DateTimeFormat { time_zone: self.time_zone.identifier(), epoch_nanoseconds: instant, local, - field: TextField::DayPeriod(width), + field: period, }) }; format(start_instant, start)? == format(end_instant, end)? @@ -1197,7 +1498,7 @@ impl DateTimeFormat { RangeDifference::Minute => start.minute == end.minute, RangeDifference::Second => start.second == end.second, RangeDifference::FractionalSecond => { - let digits = self.pattern.fractional_second_digits().ok_or_else(|| { + let digits = pattern.fractional_second_digits().ok_or_else(|| { ProviderError::invalid("pattern", "missing fractional-second width") })?; let divisor = match digits { @@ -1209,7 +1510,9 @@ impl DateTimeFormat { start.millisecond / divisor == end.millisecond / divisor } }; - if !equal { return Ok(Some(difference)); } + if !equal { + return Ok(Some(difference)); + } } Ok(None) } @@ -1221,15 +1524,19 @@ fn resolve_component_options( defaults: DefaultComponents, ) -> Result { if (options.date_style.is_some() || options.time_style.is_some()) - && options.components.has_any() + && !options.components.is_empty() { return Err(DateTimeFormatError::Type { message: "dateStyle/timeStyle may not be combined with explicit components", }); } - if matches!(required, RequiredComponents::Date) && options.time_style.is_some() { + if matches!( + required, + RequiredComponents::Date | RequiredComponents::YearMonth | RequiredComponents::MonthDay + ) && options.time_style.is_some() + { return Err(DateTimeFormatError::Type { - message: "timeStyle is not permitted by Date.prototype.toLocaleDateString", + message: "timeStyle is not permitted when formatting only a date", }); } if matches!(required, RequiredComponents::Time) && options.date_style.is_some() { @@ -1245,6 +1552,8 @@ fn resolve_component_options( let need_defaults = match required { RequiredComponents::Date => !components.has_date(), RequiredComponents::Time => !components.has_time(), + RequiredComponents::YearMonth => components.year.is_none() && components.month.is_none(), + RequiredComponents::MonthDay => components.month.is_none() && components.day.is_none(), RequiredComponents::Any => !components.has_any(), }; if need_defaults { @@ -1254,6 +1563,14 @@ fn resolve_component_options( if matches!(defaults, DefaultComponents::Time | DefaultComponents::All) { components.add_time_defaults(); } + if matches!(defaults, DefaultComponents::YearMonth) { + components.year = Some(NumericStyle::Numeric); + components.month = Some(MonthStyle::Numeric); + } + if matches!(defaults, DefaultComponents::MonthDay) { + components.month = Some(MonthStyle::Numeric); + components.day = Some(NumericStyle::Numeric); + } } Ok(components) } @@ -1265,14 +1582,17 @@ fn canonicalize_unicode_type( provider: &dyn DateTimeFormatDataProvider, ) -> Result { if value.is_empty() - || value - .split('-') - .any(|part| !(3..=8).contains(&part.len()) || !part.bytes().all(|byte| byte.is_ascii_alphanumeric())) + || value.split('-').any(|part| { + !(3..=8).contains(&part.len()) || !part.bytes().all(|byte| byte.is_ascii_alphanumeric()) + }) { return Err(DateTimeFormatError::range(option, value)); } let canonical = value.to_ascii_lowercase(); - Ok(provider.unicode_type_alias(key, &canonical).unwrap_or(&canonical).to_owned()) + Ok(provider + .unicode_type_alias(key, &canonical) + .unwrap_or(&canonical) + .to_owned()) } fn resolved_keyword( @@ -1286,6 +1606,12 @@ fn resolved_keyword( .ok_or_else(|| ProviderError::missing("locale keyword", key).into()) } +/// Whether a `timeZone` option names something the provider can use. +#[must_use] +pub fn is_valid_time_zone(value: &str, provider: &dyn DateTimeFormatDataProvider) -> bool { + canonicalize_time_zone(value, provider).is_ok() +} + fn canonicalize_time_zone( value: &str, provider: &dyn DateTimeFormatDataProvider, @@ -1312,66 +1638,36 @@ fn canonicalize_time_zone( Ok(CanonicalTimeZone::Named(canonical)) } +/// A time zone offset: a sign, then `hh`, `hhmm`, or `hh:mm`. fn parse_offset_minutes(value: &str) -> Result, DateTimeFormatError> { let (sign, body) = if let Some(body) = value.strip_prefix('+') { (1, body) - } else if let Some(body) = value.strip_prefix('-').or_else(|| value.strip_prefix('−')) { + } else if let Some(body) = value.strip_prefix('-') { (-1, body) } else { return Ok(None); }; - - let (main, fraction) = match body.find(['.', ',']) { - Some(index) => (&body[..index], Some(&body[index + 1..])), - None => (body, None), + let invalid = || DateTimeFormatError::range("timeZone", value); + let (hours, minutes) = match body.len() { + 2 => (body, "00"), + 4 => body.split_at(2), + 5 if body.as_bytes()[2] == b':' => (&body[..2], &body[3..]), + _ => return Err(invalid()), }; - if fraction.is_some_and(|digits| { - digits.is_empty() - || digits.len() > 9 - || !digits.bytes().all(|byte| byte.is_ascii_digit()) - }) { - return Err(DateTimeFormatError::range("timeZone", value)); - } - - let mut fields = [0_u8; 3]; - let count = if main.contains(':') { - let parts: Vec<_> = main.split(':').collect(); - if !(1..=3).contains(&parts.len()) - || parts.iter().any(|part| part.len() != 2) - || parts.iter().any(|part| !part.bytes().all(|byte| byte.is_ascii_digit())) - { - return Err(DateTimeFormatError::range("timeZone", value)); - } - for (index, part) in parts.iter().enumerate() { - fields[index] = part.parse().map_err(|_| DateTimeFormatError::range("timeZone", value))?; - } - parts.len() - } else { - if !matches!(main.len(), 2 | 4 | 6) - || !main.bytes().all(|byte| byte.is_ascii_digit()) - { - return Err(DateTimeFormatError::range("timeZone", value)); - } - let count = main.len() / 2; - for (index, field) in fields.iter_mut().take(count).enumerate() { - *field = main[index * 2..index * 2 + 2] - .parse() - .map_err(|_| DateTimeFormatError::range("timeZone", value))?; - } - count - }; - - if fields[0] > 23 || fields[1] > 59 || fields[2] > 59 { - return Err(DateTimeFormatError::range("timeZone", value)); - } - if fraction.is_some() && count < 3 { - return Err(DateTimeFormatError::range("timeZone", value)); + if !hours + .bytes() + .chain(minutes.bytes()) + .all(|byte| byte.is_ascii_digit()) + { + return Err(invalid()); } - if fields[2] != 0 || fraction.is_some_and(|digits| digits.bytes().any(|byte| byte != b'0')) { - return Err(DateTimeFormatError::range("timeZone", value)); + let hours: i32 = hours.parse().map_err(|_| invalid())?; + let minutes: i32 = minutes.parse().map_err(|_| invalid())?; + if hours > 23 || minutes > 59 { + return Err(invalid()); } - let minutes = i32::from(fields[0]) * 60 + i32::from(fields[1]); - Ok(Some(if minutes == 0 { 0 } else { sign * minutes })) + let total = hours * 60 + minutes; + Ok(Some(if total == 0 { 0 } else { sign * total })) } fn time_clip_to_nanoseconds(value: f64) -> Result { @@ -1388,7 +1684,9 @@ fn is_invalid_date_value(value: f64) -> bool { fn concatenate_parts(parts: &[DateTimePart]) -> String { let capacity = parts.iter().map(|part| part.value.len()).sum(); let mut result = String::with_capacity(capacity); - for part in parts { result.push_str(&part.value); } + for part in parts { + result.push_str(&part.value); + } result } @@ -1462,15 +1760,8 @@ fn date_adapter( if is_invalid_date_value(epoch_milliseconds) { return Ok("Invalid Date".to_owned()); } - DateTimeFormat::try_new_with_mode( - locales, - options, - required, - defaults, - provider, - host_locale, - )? - .format(epoch_milliseconds, provider) + DateTimeFormat::try_new_with_mode(locales, options, required, defaults, provider, host_locale)? + .format(epoch_milliseconds, provider) } #[cfg(test)] @@ -1486,7 +1777,9 @@ mod tests { struct FixtureHost; impl HostLocaleHook for FixtureHost { - fn preferred_locales(&self) -> Vec { vec!["en".to_owned()] } + fn preferred_locales(&self) -> Vec { + vec!["en".to_owned()] + } } struct FixtureProvider { @@ -1513,9 +1806,13 @@ mod tests { } } - fn record(&self, entry: impl Into) { self.log.borrow_mut().push(entry.into()); } + fn record(&self, entry: impl Into) { + self.log.borrow_mut().push(entry.into()); + } - fn take_log(&self) -> Vec { std::mem::take(&mut *self.log.borrow_mut()) } + fn take_log(&self) -> Vec { + std::mem::take(&mut *self.log.borrow_mut()) + } } impl LocaleDataProvider for FixtureProvider { @@ -1548,7 +1845,11 @@ mod tests { } fn add_likely_subtags(&self, locale: &LanguageId) -> Option { - self.record(format!("locale:likely:{}.{}", locale.language, locale.region.as_deref().unwrap_or(""))); + self.record(format!( + "locale:likely:{}.{}", + locale.language, + locale.region.as_deref().unwrap_or("") + )); Some(LanguageId { language: locale.language.clone(), script: Some("Latn".to_owned()), @@ -1618,13 +1919,17 @@ mod tests { date_items.push(PatternItem::Field(PatternField::Year(style))); } if let Some(style) = components.month { - if !date_items.is_empty() && !matches!(date_items.last(), Some(PatternItem::Literal(_))) { + if !date_items.is_empty() + && !matches!(date_items.last(), Some(PatternItem::Literal(_))) + { date_items.push(PatternItem::Literal("/".to_owned())); } date_items.push(PatternItem::Field(PatternField::Month(style))); } if let Some(style) = components.day { - if !date_items.is_empty() && !matches!(date_items.last(), Some(PatternItem::Literal(_))) { + if !date_items.is_empty() + && !matches!(date_items.last(), Some(PatternItem::Literal(_))) + { date_items.push(PatternItem::Literal("/".to_owned())); } date_items.push(PatternItem::Field(PatternField::Day(style))); @@ -1635,11 +1940,15 @@ mod tests { time_items.push(PatternItem::Field(PatternField::Hour(style))); } if let Some(style) = components.minute { - if !time_items.is_empty() { time_items.push(PatternItem::Literal(":".to_owned())); } + if !time_items.is_empty() { + time_items.push(PatternItem::Literal(":".to_owned())); + } time_items.push(PatternItem::Field(PatternField::Minute(style))); } if let Some(style) = components.second { - if !time_items.is_empty() { time_items.push(PatternItem::Literal(":".to_owned())); } + if !time_items.is_empty() { + time_items.push(PatternItem::Literal(":".to_owned())); + } time_items.push(PatternItem::Field(PatternField::Second(style))); } if let Some(digits) = components.fractional_second_digits { @@ -1647,11 +1956,15 @@ mod tests { time_items.push(PatternItem::Field(PatternField::FractionalSecond(digits))); } if let Some(width) = components.day_period { - if !time_items.is_empty() { time_items.push(PatternItem::Literal(" ".to_owned())); } + if !time_items.is_empty() { + time_items.push(PatternItem::Literal(" ".to_owned())); + } time_items.push(PatternItem::Field(PatternField::DayPeriod(width))); } if let Some(style) = components.time_zone_name { - if !time_items.is_empty() { time_items.push(PatternItem::Literal(" ".to_owned())); } + if !time_items.is_empty() { + time_items.push(PatternItem::Literal(" ".to_owned())); + } time_items.push(PatternItem::Field(PatternField::TimeZoneName(style))); } @@ -1687,8 +2000,10 @@ mod tests { let Some(difference) = request.largest_difference else { return Ok(Some(simple_hour_range())); }; - Ok(matches!(difference, RangeDifference::Hour | RangeDifference::Minute) - .then(simple_hour_range)) + Ok( + matches!(difference, RangeDifference::Hour | RangeDifference::Minute) + .then(simple_hour_range), + ) } fn named_time_zone_offset_nanoseconds( @@ -1724,7 +2039,11 @@ mod tests { let (era, year) = match calendar { "gregory" => ( Some(if gregorian_year >= 1 { "ce" } else { "bce" }.to_owned()), - if gregorian_year >= 1 { gregorian_year } else { 1 - gregorian_year }, + if gregorian_year >= 1 { + gregorian_year + } else { + 1 - gregorian_year + }, ), "buddhist" => (Some("be".to_owned()), gregorian_year + 543), _ => return Err(ProviderError::missing("calendar", calendar)), @@ -1755,7 +2074,10 @@ mod tests { )); let negative = request.value < 0; let magnitude = request.value.unsigned_abs(); - let mut value = format!("{magnitude:0width$}", width = usize::from(request.minimum_digits)); + let mut value = format!( + "{magnitude:0width$}", + width = usize::from(request.minimum_digits) + ); if request.numbering_system == "arab" { const DIGITS: [char; 10] = ['٠', '١', '٢', '٣', '٤', '٥', '٦', '٧', '٨', '٩']; value = value @@ -1768,16 +2090,21 @@ mod tests { request.numbering_system, )); } - if negative { value.insert(0, '-'); } + if negative { + value.insert(0, '-'); + } Ok(value) } fn format_text(&self, request: &TextFormatRequest<'_>) -> Result { - self.record(format!("text:{:?}:{}", request.field, request.epoch_nanoseconds)); + self.record(format!( + "text:{:?}:{}", + request.field, request.epoch_nanoseconds + )); let value = match request.field { TextField::Weekday(_) => ["Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun"] [usize::from(request.local.weekday - 1)] - .to_owned(), + .to_owned(), TextField::Era(_) => match request.local.era.as_deref() { Some("ce") => "AD".to_owned(), Some("bce") => "BC".to_owned(), @@ -1791,11 +2118,22 @@ mod tests { .clone() .ok_or_else(|| ProviderError::missing("year name", request.calendar))?, TextField::Month(_) => [ - "January", "February", "March", "April", "May", "June", "July", - "August", "September", "October", "November", "December", "Undecimber", + "January", + "February", + "March", + "April", + "May", + "June", + "July", + "August", + "September", + "October", + "November", + "December", + "Undecimber", ][usize::from(request.local.month - 1)] .to_owned(), - TextField::DayPeriod(_) => { + TextField::DayPeriod(_) | TextField::AmPm => { if request.local.hour < 12 { "AM" } else { "PM" }.to_owned() } TextField::TimeZoneName(_) => match request.time_zone { @@ -1842,8 +2180,7 @@ mod tests { let era = if z >= 0 { z } else { z - 146_096 } / 146_097; let day_of_era = z - era * 146_097; let year_of_era = - (day_of_era - day_of_era / 1_460 + day_of_era / 36_524 - day_of_era / 146_096) - / 365; + (day_of_era - day_of_era / 1_460 + day_of_era / 36_524 - day_of_era / 146_096) / 365; let mut year = year_of_era + era * 400; let day_of_year = day_of_era - (365 * year_of_era + year_of_era / 4 - year_of_era / 100); let month_prime = (5 * day_of_year + 2) / 153; @@ -1862,14 +2199,17 @@ mod tests { fn utc_milliseconds(year: i32, month: u8, day: u8, hour: u8, minute: u8) -> f64 { let adjusted_year = i64::from(year) - i64::from(month <= 2); - let era = if adjusted_year >= 0 { adjusted_year } else { adjusted_year - 399 } / 400; + let era = if adjusted_year >= 0 { + adjusted_year + } else { + adjusted_year - 399 + } / 400; let year_of_era = adjusted_year - era * 400; let adjusted_month = i64::from(month) + if month > 2 { -3 } else { 9 }; let day_of_year = (153 * adjusted_month + 2) / 5 + i64::from(day) - 1; let day_of_era = year_of_era * 365 + year_of_era / 4 - year_of_era / 100 + day_of_year; let days = era * 146_097 + day_of_era - 719_468; - (days * 86_400_000 + i64::from(hour) * 3_600_000 + i64::from(minute) * 60_000) - as f64 + (days * 86_400_000 + i64::from(hour) * 3_600_000 + i64::from(minute) * 60_000) as f64 } #[test] @@ -1918,14 +2258,14 @@ mod tests { let provider = FixtureProvider::new(); let host = FixtureHost; for time_zone in ["Mars/Phobos", "+24:00", "+01:00:01", "+1:00"] { - let error = DateTimeFormat::try_new( - &[], - &options_with_time_zone(time_zone), - &provider, - &host, - ) - .expect_err("invalid zone must fail"); - assert_eq!(error.kind(), DateTimeFormatErrorKind::RangeError, "{time_zone}"); + let error = + DateTimeFormat::try_new(&[], &options_with_time_zone(time_zone), &provider, &host) + .expect_err("invalid zone must fail"); + assert_eq!( + error.kind(), + DateTimeFormatErrorKind::RangeError, + "{time_zone}" + ); } for (calendar, numbering_system) in [(Some("a"), None), (None, Some("ab"))] { let options = DateTimeFormatOptions { @@ -1941,21 +2281,13 @@ mod tests { ); } - let offset = DateTimeFormat::try_new( - &[], - &options_with_time_zone("-2359"), - &provider, - &host, - ) - .expect("valid offset zone"); + let offset = + DateTimeFormat::try_new(&[], &options_with_time_zone("-2359"), &provider, &host) + .expect("valid offset zone"); assert_eq!(offset.resolved_options().time_zone, "-23:59"); - let zero = DateTimeFormat::try_new( - &[], - &options_with_time_zone("-00:00:00.000"), - &provider, - &host, - ) - .expect("negative zero canonicalizes"); + let zero = + DateTimeFormat::try_new(&[], &options_with_time_zone("-00:00"), &provider, &host) + .expect("negative zero canonicalizes"); assert_eq!(zero.resolved_options().time_zone, "+00:00"); } @@ -2008,9 +2340,12 @@ mod tests { }, ..DateTimeFormatOptions::default() }; - let format = DateTimeFormat::try_new(&[], &options, &provider, &host) - .expect("hour formatter"); - assert_eq!(format.format(midnight, &provider).expect("midnight"), midnight_expected); + let format = + DateTimeFormat::try_new(&[], &options, &provider, &host).expect("hour formatter"); + assert_eq!( + format.format(midnight, &provider).expect("midnight"), + midnight_expected + ); assert_eq!(format.format(noon, &provider).expect("noon"), noon_expected); } @@ -2045,8 +2380,8 @@ mod tests { }, ..DateTimeFormatOptions::default() }; - let format = DateTimeFormat::try_new(&[], &options, &provider, &host) - .expect("time formatter"); + let format = + DateTimeFormat::try_new(&[], &options, &provider, &host).expect("time formatter"); assert_eq!( format.format(-1.9, &provider).expect("negative epoch"), "23:59:59.999" @@ -2067,19 +2402,31 @@ mod tests { }, ..DateTimeFormatOptions::default() }; - let format = DateTimeFormat::try_new(&[], &options, &provider, &host) - .expect("transition formatter"); + let format = + DateTimeFormat::try_new(&[], &options, &provider, &host).expect("transition formatter"); provider.take_log(); let spring_before = (SPRING_TRANSITION_SECONDS - 1_800) as f64 * 1_000.0; let spring_after = (SPRING_TRANSITION_SECONDS + 1_800) as f64 * 1_000.0; - assert_eq!(format.format(spring_before, &provider).expect("before gap"), "01:30 EST"); - assert_eq!(format.format(spring_after, &provider).expect("after gap"), "03:30 EDT"); + assert_eq!( + format.format(spring_before, &provider).expect("before gap"), + "01:30 EST" + ); + assert_eq!( + format.format(spring_after, &provider).expect("after gap"), + "03:30 EDT" + ); let fall_before = (FALL_TRANSITION_SECONDS - 1_800) as f64 * 1_000.0; let fall_after = (FALL_TRANSITION_SECONDS + 1_800) as f64 * 1_000.0; - assert_eq!(format.format(fall_before, &provider).expect("first fold"), "01:30 EDT"); - assert_eq!(format.format(fall_after, &provider).expect("second fold"), "01:30 EST"); + assert_eq!( + format.format(fall_before, &provider).expect("first fold"), + "01:30 EDT" + ); + assert_eq!( + format.format(fall_after, &provider).expect("second fold"), + "01:30 EST" + ); let log = provider.take_log(); for milliseconds in [spring_before, spring_after, fall_before, fall_after] { @@ -2103,8 +2450,8 @@ mod tests { }, ..DateTimeFormatOptions::default() }; - let format = DateTimeFormat::try_new(&[], &options, &provider, &host) - .expect("range formatter"); + let format = + DateTimeFormat::try_new(&[], &options, &provider, &host).expect("range formatter"); let start = utc_milliseconds(2024, 1, 2, 1, 0); let end = utc_milliseconds(2024, 1, 2, 3, 0); let parts = format @@ -2130,41 +2477,60 @@ mod tests { }, ] ); - assert_eq!(format.format_range(start, end, &provider).expect("range"), "1–3"); + assert_eq!( + format.format_range(start, end, &provider).expect("range"), + "1–3" + ); provider.take_log(); let collapsed = format .format_range_to_parts(start, start, &provider) .expect("collapsed range"); - assert!(collapsed.iter().all(|part| part.source == RangeSource::Shared)); - assert!(!provider.take_log().iter().any(|entry| entry.starts_with("range:"))); + assert!( + collapsed + .iter() + .all(|part| part.source == RangeSource::Shared) + ); + assert!( + !provider + .take_log() + .iter() + .any(|entry| entry.starts_with("range:")) + ); } #[test] fn non_finite_format_is_range_error_but_invalid_date_adapter_is_literal() { let provider = FixtureProvider::new(); let host = FixtureHost; - let format = DateTimeFormat::try_new( - &[], - &options_with_time_zone("UTC"), - &provider, - &host, - ) - .expect("formatter"); + let format = DateTimeFormat::try_new(&[], &options_with_time_zone("UTC"), &provider, &host) + .expect("formatter"); for value in [f64::NAN, f64::INFINITY, MAX_TIME_MILLISECONDS + 1.0] { assert_eq!( - format.format(value, &provider).expect_err("invalid time").kind(), + format + .format(value, &provider) + .expect_err("invalid time") + .kind(), DateTimeFormatErrorKind::RangeError ); } provider.take_log(); assert_eq!( - date_to_locale_string(f64::NAN, &[], &DateTimeFormatOptions::default(), &provider, &host) - .expect("invalid Date is a string"), + date_to_locale_string( + f64::NAN, + &[], + &DateTimeFormatOptions::default(), + &provider, + &host + ) + .expect("invalid Date is a string"), "Invalid Date" ); - assert!(provider.take_log().is_empty(), "invalid Date must not access Intl data"); + assert!( + provider.take_log().is_empty(), + "invalid Date must not access Intl data" + ); } #[test] @@ -2203,7 +2569,10 @@ mod tests { let second_log = provider.take_log(); assert_eq!(first, second); assert_eq!(first_log, second_log); - assert_eq!(first_log[0], format!("offset:UTC:{}", time_clip_to_nanoseconds(instant).unwrap())); + assert_eq!( + first_log[0], + format!("offset:UTC:{}", time_clip_to_nanoseconds(instant).unwrap()) + ); assert!(first_log[1].starts_with("calendar:gregory:")); } } diff --git a/crates/bamts-runtime/src/intl/duration_format.rs b/crates/bamts-runtime/src/intl/duration_format.rs new file mode 100644 index 00000000..3de38b60 --- /dev/null +++ b/crates/bamts-runtime/src/intl/duration_format.rs @@ -0,0 +1,343 @@ +//! ECMA-402 `Intl.DurationFormat` mechanics. + +use super::locale_data::IntlData; +use super::number_data::NoHostLocale; +use super::number_format::{ + IntlMathematicalValue, NumberFormat, NumberFormatError, NumberFormatOptions, NumberPart, + UseGroupingOption, +}; +use super::plural_rules::{IntlError, ListFormat, ListFormatOptions, ListPartKind}; + +/// The units a duration has, largest first, as their option names. +pub const UNITS: [&str; 10] = [ + "years", + "months", + "weeks", + "days", + "hours", + "minutes", + "seconds", + "milliseconds", + "microseconds", + "nanoseconds", +]; + +const SECONDS: usize = 6; + +/// How one unit appears. `Fractional` joins the unit above it as a fraction. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum UnitStyle { + Long, + Short, + Narrow, + Numeric, + TwoDigit, + Fractional, +} + +impl UnitStyle { + /// The style's option spelling. A fractional unit reports as numeric. + #[must_use] + pub const fn as_str(self) -> &'static str { + match self { + Self::Long => "long", + Self::Short => "short", + Self::Narrow => "narrow", + Self::Numeric | Self::Fractional => "numeric", + Self::TwoDigit => "2-digit", + } + } + + /// Reads an option value. + #[must_use] + pub fn parse(value: &str) -> Self { + match value { + "long" => Self::Long, + "short" => Self::Short, + "narrow" => Self::Narrow, + "numeric" => Self::Numeric, + _ => Self::TwoDigit, + } + } + + const fn is_numeric(self) -> bool { + matches!(self, Self::Numeric | Self::TwoDigit | Self::Fractional) + } +} + +/// The resolved style and display of one unit. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct UnitOptions { + pub style: UnitStyle, + pub always: bool, +} + +/// The default style and display of a unit that has no option of its own. +#[must_use] +pub fn default_unit_options( + unit: usize, + digital: bool, + base: UnitStyle, + previous: Option, +) -> (UnitStyle, bool) { + let is_hms = (4..=SECONDS).contains(&unit); + if digital { + ( + if is_hms || unit > SECONDS { + UnitStyle::Numeric + } else { + UnitStyle::Short + }, + is_hms, + ) + } else if previous.is_some_and(UnitStyle::is_numeric) { + (UnitStyle::Numeric, unit == 5 || unit == SECONDS) + } else { + (base, false) + } +} + +/// ValidateDurationUnitStyle. +#[must_use] +pub fn is_valid_unit_style(style: UnitStyle, always: bool, previous: Option) -> bool { + !(always && style == UnitStyle::Fractional + || previous == Some(UnitStyle::Fractional) && style != UnitStyle::Fractional + || matches!(previous, Some(UnitStyle::Numeric | UnitStyle::TwoDigit)) + && !style.is_numeric()) +} + +/// The numeric style a unit settles on after its neighbours are known. +#[must_use] +pub fn settled_unit_style(unit: usize, style: UnitStyle, previous: Option) -> UnitStyle { + if (unit == 5 || unit == SECONDS) + && matches!(previous, Some(UnitStyle::Numeric | UnitStyle::TwoDigit)) + { + UnitStyle::TwoDigit + } else { + style + } +} + +/// One part of a formatted duration. +#[derive(Clone, Debug, Eq, PartialEq)] +pub struct DurationPart { + pub part_type: &'static str, + pub value: String, + pub unit: Option<&'static str>, +} + +/// A failure inside the number or list formatters a duration leans on. +#[derive(Clone, Debug)] +pub enum DurationFormatError { + Number(NumberFormatError), + List(IntlError), +} + +impl std::fmt::Display for DurationFormatError { + fn fmt(&self, formatter: &mut std::fmt::Formatter<'_>) -> std::fmt::Result { + match self { + Self::Number(error) => error.fmt(formatter), + Self::List(error) => error.fmt(formatter), + } + } +} + +/// Resolved `Intl.DurationFormat` slots. +#[derive(Clone, Debug)] +pub struct DurationFormat { + pub locale: String, + pub numbering_system: String, + pub style: String, + pub units: [UnitOptions; 10], + pub fractional_digits: Option, +} + +/// An exact decimal for a value `digits` places below its unit, in the +/// `sign * integer.fraction` form the number formatter parses. +fn scaled_decimal(value: i128, digits: u32) -> String { + let divisor = 10_i128.pow(digits); + let magnitude = value.unsigned_abs(); + let integer = magnitude / divisor.unsigned_abs(); + let fraction = magnitude % divisor.unsigned_abs(); + let sign = if value < 0 { "-" } else { "" }; + format!( + "{sign}{integer}.{fraction:0width$}", + width = digits as usize + ) +} + +impl DurationFormat { + /// The number formatter of one unit, given the parts of its options that + /// depend on the position in the output. + fn number_format( + &self, + unit: usize, + style: UnitStyle, + fractional: bool, + sign_hidden: bool, + data: &'static IntlData, + ) -> Result, DurationFormatError> { + let mut options = NumberFormatOptions { + locale_matcher: Some(String::from("best fit")), + numbering_system: Some(self.numbering_system.clone()), + ..NumberFormatOptions::default() + }; + if fractional { + options.maximum_fraction_digits = Some(self.fractional_digits.map_or(9.0, f64::from)); + options.minimum_fraction_digits = Some(self.fractional_digits.map_or(0.0, f64::from)); + options.rounding_mode = Some(String::from("trunc")); + } + if sign_hidden { + options.sign_display = Some(String::from("never")); + } + if style == UnitStyle::TwoDigit { + options.minimum_integer_digits = Some(2.0); + } + if style.is_numeric() { + options.use_grouping = Some(UseGroupingOption::Boolean(false)); + } else { + options.style = Some(String::from("unit")); + options.unit = Some(UNITS[unit].trim_end_matches('s').to_owned()); + options.unit_display = Some(style.as_str().to_owned()); + } + NumberFormat::new( + std::slice::from_ref(&self.locale), + &options, + data, + &NoHostLocale, + ) + .map_err(DurationFormatError::Number) + } + + /// PartitionDurationFormatPattern over the ten fields of a duration. + /// + /// # Errors + /// Reports a failure of the number or list formatters. + pub fn partition( + &self, + fields: &[i128; 10], + data: &'static IntlData, + ) -> Result, DurationFormatError> { + let negative = fields.iter().any(|field| *field < 0); + let mut groups: Vec> = Vec::new(); + let mut need_separator = false; + let mut sign_shown = false; + for (unit, options) in self.units.iter().enumerate() { + let singular = UNITS[unit].trim_end_matches('s'); + let mut value = fields[unit]; + // A numeric seconds or sub-second unit takes in everything below it. + let next_numeric = + (SECONDS..9).contains(&unit) && self.units[unit + 1].style == UnitStyle::Fractional; + let mut exact = None; + if next_numeric { + let digits = 3 * (9 - unit as u32); + let mut total = fields[9]; + for (index, factor) in [(8, 3), (7, 6), (6, 9)] { + if index >= unit { + total += fields[index] * 10_i128.pow(factor); + } + } + if fields[unit + 1..].iter().all(|field| *field == 0) { + value = fields[unit]; + } else { + value = total; + exact = Some(scaled_decimal(total, digits)); + } + } + let minutes_needed = unit == 5 + && need_separator + && (self.units[SECONDS].always + || fields[SECONDS..].iter().any(|field| *field != 0)); + if value != 0 || options.always || minutes_needed { + let hide_sign = sign_shown; + sign_shown = true; + let number = + self.number_format(unit, options.style, next_numeric, hide_sign, data)?; + let parts = + self.unit_parts(&number, value, exact.as_deref(), negative && !hide_sign)?; + let tagged = parts.into_iter().map(|part| DurationPart { + part_type: part.part_type.as_str(), + value: part.value, + unit: Some(singular), + }); + if need_separator { + let list = groups.last_mut().expect("a separator follows a group"); + list.push(DurationPart { + part_type: "literal", + value: String::from(":"), + unit: None, + }); + list.extend(tagged); + } else { + groups.push(tagged.collect()); + need_separator = options.style.is_numeric(); + } + } + if next_numeric { + break; + } + } + self.join_groups(groups, data) + } + + fn unit_parts( + &self, + number: &NumberFormat<'static>, + value: i128, + exact: Option<&str>, + keep_negative_zero: bool, + ) -> Result, DurationFormatError> { + let mathematical = match exact { + Some(text) => IntlMathematicalValue::from_decimal_string(text), + None if value == 0 && keep_negative_zero => Ok(IntlMathematicalValue::from_f64(-0.0)), + None => IntlMathematicalValue::from_bigint(&value.to_string()), + } + .map_err(DurationFormatError::Number)?; + number + .format_to_parts(&mathematical) + .map_err(DurationFormatError::Number) + } + + fn join_groups( + &self, + groups: Vec>, + data: &'static IntlData, + ) -> Result, DurationFormatError> { + let style = if self.style == "digital" { + "short" + } else { + self.style.as_str() + }; + let list = ListFormat::try_new( + std::slice::from_ref(&self.locale), + &ListFormatOptions { + locale_matcher: Some(String::from("best fit")), + list_type: Some(String::from("unit")), + style: Some(style.to_owned()), + }, + data, + &NoHostLocale, + ) + .map_err(DurationFormatError::List)?; + let strings: Vec = groups + .iter() + .map(|parts| parts.iter().map(|part| part.value.as_str()).collect()) + .collect(); + let formatted = list + .format_to_parts(&strings) + .map_err(DurationFormatError::List)?; + let mut groups = groups.into_iter(); + let mut result = Vec::new(); + for part in formatted { + match part.kind { + ListPartKind::Element => result.extend(groups.next().into_iter().flatten()), + ListPartKind::Literal => result.push(DurationPart { + part_type: "literal", + value: part.value, + unit: None, + }), + } + } + Ok(result) + } +} diff --git a/crates/bamts-runtime/src/intl/locale_aliases.rs b/crates/bamts-runtime/src/intl/locale_aliases.rs new file mode 100644 index 00000000..a7aeaf21 --- /dev/null +++ b/crates/bamts-runtime/src/intl/locale_aliases.rs @@ -0,0 +1,1197 @@ +//! Locale identifier aliases from CLDR `supplementalMetadata.xml` and the BCP 47 +//! extension data files, version 36.1 with the sign-language tags of the IANA +//! registry, in the form UTS #35 canonicalization consumes them. +//! +//! Every table is sorted by its key so lookups can binary-search. + +/// Whole language identifiers replaced as a unit. +pub(super) const TAG_ALIASES: &[(&str, &str)] = &[ + ("art-lojban", "jbo"), + ("cel-gaulish", "xtg"), + ("sgn-BR", "bzs"), + ("sgn-CO", "csn"), + ("sgn-DE", "gsg"), + ("sgn-DK", "dsl"), + ("sgn-ES", "ssp"), + ("sgn-FR", "fsl"), + ("sgn-GB", "bfi"), + ("sgn-GH", "gse"), + ("sgn-GR", "gss"), + ("sgn-IE", "isg"), + ("sgn-IT", "ise"), + ("sgn-JP", "jsl"), + ("sgn-MX", "mfs"), + ("sgn-NI", "ncs"), + ("sgn-NL", "dse"), + ("sgn-NO", "nsi"), + ("sgn-PT", "psr"), + ("sgn-SE", "swl"), + ("sgn-US", "ase"), + ("sgn-ZA", "sfs"), + ("zh-guoyu", "zh"), + ("zh-hakka", "hak"), + ("zh-xiang", "hsn"), +]; + +/// Language subtags replaced by a language and perhaps a script or region. +pub(super) const LANGUAGE_ALIASES: &[(&str, &str)] = &[ + ("aam", "aas"), + ("aar", "aa"), + ("abk", "ab"), + ("adp", "dz"), + ("afr", "af"), + ("aju", "jrb"), + ("aka", "ak"), + ("alb", "sq"), + ("als", "sq"), + ("amh", "am"), + ("ara", "ar"), + ("arb", "ar"), + ("arg", "an"), + ("arm", "hy"), + ("asd", "snz"), + ("asm", "as"), + ("aue", "ktz"), + ("ava", "av"), + ("ave", "ae"), + ("aym", "ay"), + ("ayr", "ay"), + ("ayx", "nun"), + ("aze", "az"), + ("azj", "az"), + ("bak", "ba"), + ("bam", "bm"), + ("baq", "eu"), + ("bcc", "bal"), + ("bcl", "bik"), + ("bel", "be"), + ("ben", "bn"), + ("bgm", "bcg"), + ("bh", "bho"), + ("bih", "bho"), + ("bis", "bi"), + ("bjd", "drl"), + ("bod", "bo"), + ("bos", "bs"), + ("bre", "br"), + ("bul", "bg"), + ("bur", "my"), + ("bxk", "luy"), + ("bxr", "bua"), + ("cat", "ca"), + ("ccq", "rki"), + ("ces", "cs"), + ("cha", "ch"), + ("che", "ce"), + ("chi", "zh"), + ("chu", "cu"), + ("chv", "cv"), + ("cjr", "mom"), + ("cka", "cmr"), + ("cld", "syr"), + ("cmk", "xch"), + ("cmn", "zh"), + ("cnr", "sr-ME"), + ("cor", "kw"), + ("cos", "co"), + ("coy", "pij"), + ("cqu", "quh"), + ("cre", "cr"), + ("cwd", "cr"), + ("cym", "cy"), + ("cze", "cs"), + ("dan", "da"), + ("deu", "de"), + ("dgo", "doi"), + ("dhd", "mwr"), + ("dik", "din"), + ("diq", "zza"), + ("dit", "dif"), + ("div", "dv"), + ("drh", "mn"), + ("drw", "fa-AF"), + ("dut", "nl"), + ("dzo", "dz"), + ("ekk", "et"), + ("ell", "el"), + ("emk", "man"), + ("eng", "en"), + ("epo", "eo"), + ("esk", "ik"), + ("est", "et"), + ("eus", "eu"), + ("ewe", "ee"), + ("fao", "fo"), + ("fas", "fa"), + ("fat", "ak"), + ("fij", "fj"), + ("fin", "fi"), + ("fra", "fr"), + ("fre", "fr"), + ("fry", "fy"), + ("fuc", "ff"), + ("ful", "ff"), + ("gav", "dev"), + ("gaz", "om"), + ("gbo", "grb"), + ("geo", "ka"), + ("ger", "de"), + ("gfx", "vaj"), + ("ggn", "gvr"), + ("gla", "gd"), + ("gle", "ga"), + ("glg", "gl"), + ("glv", "gv"), + ("gno", "gon"), + ("gre", "el"), + ("grn", "gn"), + ("gti", "nyc"), + ("gug", "gn"), + ("guj", "gu"), + ("guv", "duz"), + ("gya", "gba"), + ("hat", "ht"), + ("hau", "ha"), + ("hbs", "sr-Latn"), + ("hdn", "hai"), + ("hea", "hmn"), + ("heb", "he"), + ("her", "hz"), + ("him", "srx"), + ("hin", "hi"), + ("hmo", "ho"), + ("hrr", "jal"), + ("hrv", "hr"), + ("hun", "hu"), + ("hye", "hy"), + ("ibi", "opa"), + ("ibo", "ig"), + ("ice", "is"), + ("ido", "io"), + ("iii", "ii"), + ("ike", "iu"), + ("iku", "iu"), + ("ile", "ie"), + ("ilw", "gal"), + ("in", "id"), + ("ina", "ia"), + ("ind", "id"), + ("ipk", "ik"), + ("isl", "is"), + ("ita", "it"), + ("iw", "he"), + ("jav", "jv"), + ("jeg", "oyb"), + ("ji", "yi"), + ("jpn", "ja"), + ("jw", "jv"), + ("kal", "kl"), + ("kan", "kn"), + ("kas", "ks"), + ("kat", "ka"), + ("kau", "kr"), + ("kaz", "kk"), + ("kgc", "tdf"), + ("kgh", "kml"), + ("khk", "mn"), + ("khm", "km"), + ("kik", "ki"), + ("kin", "rw"), + ("kir", "ky"), + ("kmr", "ku"), + ("knc", "kr"), + ("kng", "kg"), + ("knn", "kok"), + ("koj", "kwv"), + ("kom", "kv"), + ("kon", "kg"), + ("kor", "ko"), + ("kpv", "kv"), + ("krm", "bmf"), + ("ktr", "dtp"), + ("kua", "kj"), + ("kur", "ku"), + ("kvs", "gdj"), + ("kwq", "yam"), + ("kxe", "tvd"), + ("kzj", "dtp"), + ("kzt", "dtp"), + ("lao", "lo"), + ("lat", "la"), + ("lav", "lv"), + ("lbk", "bnc"), + ("lii", "raq"), + ("lim", "li"), + ("lin", "ln"), + ("lit", "lt"), + ("llo", "ngt"), + ("lmm", "rmx"), + ("ltz", "lb"), + ("lub", "lu"), + ("lug", "lg"), + ("lvs", "lv"), + ("mac", "mk"), + ("mah", "mh"), + ("mal", "ml"), + ("mao", "mi"), + ("mar", "mr"), + ("may", "ms"), + ("meg", "cir"), + ("mhr", "chm"), + ("mkd", "mk"), + ("mlg", "mg"), + ("mlt", "mt"), + ("mnk", "man"), + ("mo", "ro"), + ("mol", "ro"), + ("mon", "mn"), + ("mri", "mi"), + ("msa", "ms"), + ("mst", "mry"), + ("mup", "raj"), + ("mwj", "vaj"), + ("mya", "my"), + ("myd", "aog"), + ("myt", "mry"), + ("nad", "xny"), + ("nau", "na"), + ("nav", "nv"), + ("nbl", "nr"), + ("ncp", "kdz"), + ("nde", "nd"), + ("ndo", "ng"), + ("nep", "ne"), + ("nld", "nl"), + ("nno", "nn"), + ("nns", "nbr"), + ("nnx", "ngv"), + ("no", "nb"), + ("nob", "nb"), + ("nor", "nb"), + ("npi", "ne"), + ("nts", "pij"), + ("nya", "ny"), + ("oci", "oc"), + ("ojg", "oj"), + ("oji", "oj"), + ("ori", "or"), + ("orm", "om"), + ("ory", "or"), + ("oss", "os"), + ("oun", "vaj"), + ("pan", "pa"), + ("pbu", "ps"), + ("pcr", "adx"), + ("per", "fa"), + ("pes", "fa"), + ("pli", "pi"), + ("plt", "mg"), + ("pmc", "huw"), + ("pmu", "phr"), + ("pnb", "lah"), + ("pol", "pl"), + ("por", "pt"), + ("ppa", "bfy"), + ("ppr", "lcq"), + ("prs", "fa-AF"), + ("pry", "prt"), + ("pus", "ps"), + ("puz", "pub"), + ("que", "qu"), + ("quz", "qu"), + ("rmy", "rom"), + ("roh", "rm"), + ("ron", "ro"), + ("rum", "ro"), + ("run", "rn"), + ("rus", "ru"), + ("sag", "sg"), + ("san", "sa"), + ("sca", "hle"), + ("scc", "sr"), + ("scr", "hr"), + ("sh", "sr-Latn"), + ("sin", "si"), + ("skk", "oyb"), + ("slk", "sk"), + ("slo", "sk"), + ("slv", "sl"), + ("sme", "se"), + ("smo", "sm"), + ("sna", "sn"), + ("snd", "sd"), + ("som", "so"), + ("sot", "st"), + ("spa", "es"), + ("spy", "kln"), + ("sqi", "sq"), + ("src", "sc"), + ("srd", "sc"), + ("srp", "sr"), + ("ssw", "ss"), + ("sun", "su"), + ("swa", "sw"), + ("swc", "sw-CD"), + ("swe", "sv"), + ("swh", "sw"), + ("tah", "ty"), + ("tam", "ta"), + ("tat", "tt"), + ("tdu", "dtp"), + ("tel", "te"), + ("tgk", "tg"), + ("tgl", "fil"), + ("tha", "th"), + ("thc", "tpo"), + ("thx", "oyb"), + ("tib", "bo"), + ("tie", "ras"), + ("tir", "ti"), + ("tkk", "twm"), + ("tl", "fil"), + ("tlw", "weo"), + ("tmp", "tyj"), + ("tne", "kak"), + ("tnf", "fa-AF"), + ("ton", "to"), + ("tsf", "taj"), + ("tsn", "tn"), + ("tso", "ts"), + ("ttq", "tmh"), + ("tuk", "tk"), + ("tur", "tr"), + ("tw", "ak"), + ("twi", "ak"), + ("uig", "ug"), + ("ukr", "uk"), + ("umu", "del"), + ("uok", "ema"), + ("urd", "ur"), + ("uzb", "uz"), + ("uzn", "uz"), + ("ven", "ve"), + ("vie", "vi"), + ("vol", "vo"), + ("wel", "cy"), + ("wln", "wa"), + ("wol", "wo"), + ("xba", "cax"), + ("xho", "xh"), + ("xia", "acn"), + ("xkh", "waw"), + ("xpe", "kpe"), + ("xsj", "suj"), + ("xsl", "den"), + ("ybd", "rki"), + ("ydd", "yi"), + ("yid", "yi"), + ("yma", "lrr"), + ("ymt", "mtm"), + ("yor", "yo"), + ("yos", "zom"), + ("yuu", "yug"), + ("zai", "zap"), + ("zha", "za"), + ("zho", "zh"), + ("zsm", "ms"), + ("zul", "zu"), + ("zyb", "za"), +]; + +/// Region subtags with one replacement. +pub(super) const REGION_ALIASES: &[(&str, &str)] = &[ + ("004", "AF"), + ("008", "AL"), + ("010", "AQ"), + ("012", "DZ"), + ("016", "AS"), + ("020", "AD"), + ("024", "AO"), + ("028", "AG"), + ("031", "AZ"), + ("032", "AR"), + ("036", "AU"), + ("040", "AT"), + ("044", "BS"), + ("048", "BH"), + ("050", "BD"), + ("051", "AM"), + ("052", "BB"), + ("056", "BE"), + ("060", "BM"), + ("062", "034"), + ("064", "BT"), + ("068", "BO"), + ("070", "BA"), + ("072", "BW"), + ("074", "BV"), + ("076", "BR"), + ("084", "BZ"), + ("086", "IO"), + ("090", "SB"), + ("092", "VG"), + ("096", "BN"), + ("100", "BG"), + ("104", "MM"), + ("108", "BI"), + ("112", "BY"), + ("116", "KH"), + ("120", "CM"), + ("124", "CA"), + ("132", "CV"), + ("136", "KY"), + ("140", "CF"), + ("144", "LK"), + ("148", "TD"), + ("152", "CL"), + ("156", "CN"), + ("158", "TW"), + ("162", "CX"), + ("166", "CC"), + ("170", "CO"), + ("174", "KM"), + ("175", "YT"), + ("178", "CG"), + ("180", "CD"), + ("184", "CK"), + ("188", "CR"), + ("191", "HR"), + ("192", "CU"), + ("196", "CY"), + ("203", "CZ"), + ("204", "BJ"), + ("208", "DK"), + ("212", "DM"), + ("214", "DO"), + ("218", "EC"), + ("222", "SV"), + ("226", "GQ"), + ("230", "ET"), + ("231", "ET"), + ("232", "ER"), + ("233", "EE"), + ("234", "FO"), + ("238", "FK"), + ("239", "GS"), + ("242", "FJ"), + ("246", "FI"), + ("248", "AX"), + ("249", "FR"), + ("250", "FR"), + ("254", "GF"), + ("258", "PF"), + ("260", "TF"), + ("262", "DJ"), + ("266", "GA"), + ("268", "GE"), + ("270", "GM"), + ("275", "PS"), + ("276", "DE"), + ("278", "DE"), + ("280", "DE"), + ("288", "GH"), + ("292", "GI"), + ("296", "KI"), + ("300", "GR"), + ("304", "GL"), + ("308", "GD"), + ("312", "GP"), + ("316", "GU"), + ("320", "GT"), + ("324", "GN"), + ("328", "GY"), + ("332", "HT"), + ("334", "HM"), + ("336", "VA"), + ("340", "HN"), + ("344", "HK"), + ("348", "HU"), + ("352", "IS"), + ("356", "IN"), + ("360", "ID"), + ("364", "IR"), + ("368", "IQ"), + ("372", "IE"), + ("376", "IL"), + ("380", "IT"), + ("384", "CI"), + ("388", "JM"), + ("392", "JP"), + ("398", "KZ"), + ("400", "JO"), + ("404", "KE"), + ("408", "KP"), + ("410", "KR"), + ("414", "KW"), + ("417", "KG"), + ("418", "LA"), + ("422", "LB"), + ("426", "LS"), + ("428", "LV"), + ("430", "LR"), + ("434", "LY"), + ("438", "LI"), + ("440", "LT"), + ("442", "LU"), + ("446", "MO"), + ("450", "MG"), + ("454", "MW"), + ("458", "MY"), + ("462", "MV"), + ("466", "ML"), + ("470", "MT"), + ("474", "MQ"), + ("478", "MR"), + ("480", "MU"), + ("484", "MX"), + ("492", "MC"), + ("496", "MN"), + ("498", "MD"), + ("499", "ME"), + ("500", "MS"), + ("504", "MA"), + ("508", "MZ"), + ("512", "OM"), + ("516", "NA"), + ("520", "NR"), + ("524", "NP"), + ("528", "NL"), + ("531", "CW"), + ("533", "AW"), + ("534", "SX"), + ("535", "BQ"), + ("540", "NC"), + ("548", "VU"), + ("554", "NZ"), + ("558", "NI"), + ("562", "NE"), + ("566", "NG"), + ("570", "NU"), + ("574", "NF"), + ("578", "NO"), + ("580", "MP"), + ("581", "UM"), + ("583", "FM"), + ("584", "MH"), + ("585", "PW"), + ("586", "PK"), + ("591", "PA"), + ("598", "PG"), + ("600", "PY"), + ("604", "PE"), + ("608", "PH"), + ("612", "PN"), + ("616", "PL"), + ("620", "PT"), + ("624", "GW"), + ("626", "TL"), + ("630", "PR"), + ("634", "QA"), + ("638", "RE"), + ("642", "RO"), + ("643", "RU"), + ("646", "RW"), + ("652", "BL"), + ("654", "SH"), + ("659", "KN"), + ("660", "AI"), + ("662", "LC"), + ("663", "MF"), + ("666", "PM"), + ("670", "VC"), + ("674", "SM"), + ("678", "ST"), + ("682", "SA"), + ("686", "SN"), + ("688", "RS"), + ("690", "SC"), + ("694", "SL"), + ("702", "SG"), + ("703", "SK"), + ("704", "VN"), + ("705", "SI"), + ("706", "SO"), + ("710", "ZA"), + ("716", "ZW"), + ("720", "YE"), + ("724", "ES"), + ("728", "SS"), + ("729", "SD"), + ("732", "EH"), + ("736", "SD"), + ("740", "SR"), + ("744", "SJ"), + ("748", "SZ"), + ("752", "SE"), + ("756", "CH"), + ("760", "SY"), + ("762", "TJ"), + ("764", "TH"), + ("768", "TG"), + ("772", "TK"), + ("776", "TO"), + ("780", "TT"), + ("784", "AE"), + ("788", "TN"), + ("792", "TR"), + ("795", "TM"), + ("796", "TC"), + ("798", "TV"), + ("800", "UG"), + ("804", "UA"), + ("807", "MK"), + ("818", "EG"), + ("826", "GB"), + ("830", "JE"), + ("831", "GG"), + ("832", "JE"), + ("833", "IM"), + ("834", "TZ"), + ("840", "US"), + ("850", "VI"), + ("854", "BF"), + ("858", "UY"), + ("860", "UZ"), + ("862", "VE"), + ("876", "WF"), + ("882", "WS"), + ("886", "YE"), + ("887", "YE"), + ("891", "RS"), + ("894", "ZM"), + ("958", "AA"), + ("959", "QM"), + ("960", "QN"), + ("962", "QP"), + ("963", "QQ"), + ("964", "QR"), + ("965", "QS"), + ("966", "QT"), + ("967", "EU"), + ("968", "QV"), + ("969", "QW"), + ("970", "QX"), + ("971", "QY"), + ("972", "QZ"), + ("973", "XA"), + ("974", "XB"), + ("975", "XC"), + ("976", "XD"), + ("977", "XE"), + ("978", "XF"), + ("979", "XG"), + ("980", "XH"), + ("981", "XI"), + ("982", "XJ"), + ("983", "XK"), + ("984", "XL"), + ("985", "XM"), + ("986", "XN"), + ("987", "XO"), + ("988", "XP"), + ("989", "XQ"), + ("990", "XR"), + ("991", "XS"), + ("992", "XT"), + ("993", "XU"), + ("994", "XV"), + ("995", "XW"), + ("996", "XX"), + ("997", "XY"), + ("998", "XZ"), + ("999", "ZZ"), + ("BU", "MM"), + ("CS", "RS"), + ("CT", "KI"), + ("DD", "DE"), + ("DY", "BJ"), + ("FQ", "AQ"), + ("FX", "FR"), + ("HV", "BF"), + ("JT", "UM"), + ("MI", "UM"), + ("NH", "VU"), + ("NQ", "AQ"), + ("PU", "UM"), + ("PZ", "PA"), + ("QU", "EU"), + ("RH", "ZW"), + ("TP", "TL"), + ("UK", "GB"), + ("VD", "VN"), + ("WK", "UM"), + ("YD", "YE"), + ("YU", "RS"), + ("ZR", "CD"), +]; + +/// Region subtags with several replacements: the region, the default, and the +/// choice for each `language` or `language-Script` that prefers another. +pub(super) type RegionChoices = &'static [(&'static str, &'static str)]; + +pub(super) const COMPLEX_REGION_ALIASES: &[(&str, &str, RegionChoices)] = &[ + ( + "172", + "RU", + &[ + ("ab", "GE"), + ("az", "AZ"), + ("be", "BY"), + ("crh", "UA"), + ("gag", "MD"), + ("got", "UA"), + ("hy", "AM"), + ("ji", "UA"), + ("ka", "GE"), + ("kaa", "UZ"), + ("kk", "KZ"), + ("ku-Yezi", "GE"), + ("ky", "KG"), + ("os", "GE"), + ("rue", "UA"), + ("sog", "UZ"), + ("tg", "TJ"), + ("tk", "TM"), + ("tkr", "AZ"), + ("tly", "AZ"), + ("ttt", "AZ"), + ("ug-Cyrl", "KZ"), + ("uk", "UA"), + ("und-Armn", "AM"), + ("und-Chrs", "UZ"), + ("und-Geor", "GE"), + ("und-Goth", "UA"), + ("und-Sogd", "UZ"), + ("und-Sogo", "UZ"), + ("und-Yezi", "GE"), + ("uz", "UZ"), + ("xco", "UZ"), + ("xmf", "GE"), + ], + ), + ("200", "CZ", &[("sk", "SK")]), + ("530", "CW", &[("vic", "SX")]), + ("532", "CW", &[("vic", "SX")]), + ( + "536", + "SA", + &[ + ("akk", "IQ"), + ("ckb", "IQ"), + ("ku-Arab", "IQ"), + ("mis", "IQ"), + ("syr", "IQ"), + ("und-Hatr", "IQ"), + ("und-Syrc", "IQ"), + ("und-Xsux", "IQ"), + ], + ), + ("582", "FM", &[("mh", "MH"), ("pau", "PW")]), + ( + "810", + "RU", + &[ + ("ab", "GE"), + ("az", "AZ"), + ("be", "BY"), + ("crh", "UA"), + ("et", "EE"), + ("gag", "MD"), + ("got", "UA"), + ("hy", "AM"), + ("ji", "UA"), + ("ka", "GE"), + ("kaa", "UZ"), + ("kk", "KZ"), + ("ku-Yezi", "GE"), + ("ky", "KG"), + ("lt", "LT"), + ("ltg", "LV"), + ("lv", "LV"), + ("os", "GE"), + ("rue", "UA"), + ("sgs", "LT"), + ("sog", "UZ"), + ("tg", "TJ"), + ("tk", "TM"), + ("tkr", "AZ"), + ("tly", "AZ"), + ("ttt", "AZ"), + ("ug-Cyrl", "KZ"), + ("uk", "UA"), + ("und-Armn", "AM"), + ("und-Chrs", "UZ"), + ("und-Geor", "GE"), + ("und-Goth", "UA"), + ("und-Sogd", "UZ"), + ("und-Sogo", "UZ"), + ("und-Yezi", "GE"), + ("uz", "UZ"), + ("vro", "EE"), + ("xco", "UZ"), + ("xmf", "GE"), + ], + ), + ( + "890", + "RS", + &[("bs", "BA"), ("hr", "HR"), ("mk", "MK"), ("sl", "SI")], + ), + ("AN", "CW", &[("vic", "SX")]), + ( + "NT", + "SA", + &[ + ("akk", "IQ"), + ("ckb", "IQ"), + ("ku-Arab", "IQ"), + ("mis", "IQ"), + ("syr", "IQ"), + ("und-Hatr", "IQ"), + ("und-Syrc", "IQ"), + ("und-Xsux", "IQ"), + ], + ), + ("PC", "FM", &[("mh", "MH"), ("pau", "PW")]), + ( + "SU", + "RU", + &[ + ("ab", "GE"), + ("az", "AZ"), + ("be", "BY"), + ("crh", "UA"), + ("et", "EE"), + ("gag", "MD"), + ("got", "UA"), + ("hy", "AM"), + ("ji", "UA"), + ("ka", "GE"), + ("kaa", "UZ"), + ("kk", "KZ"), + ("ku-Yezi", "GE"), + ("ky", "KG"), + ("lt", "LT"), + ("ltg", "LV"), + ("lv", "LV"), + ("os", "GE"), + ("rue", "UA"), + ("sgs", "LT"), + ("sog", "UZ"), + ("tg", "TJ"), + ("tk", "TM"), + ("tkr", "AZ"), + ("tly", "AZ"), + ("ttt", "AZ"), + ("ug-Cyrl", "KZ"), + ("uk", "UA"), + ("und-Armn", "AM"), + ("und-Chrs", "UZ"), + ("und-Geor", "GE"), + ("und-Goth", "UA"), + ("und-Sogd", "UZ"), + ("und-Sogo", "UZ"), + ("und-Yezi", "GE"), + ("uz", "UZ"), + ("vro", "EE"), + ("xco", "UZ"), + ("xmf", "GE"), + ], + ), +]; + +/// What a variant subtag turns into. +#[derive(Clone, Copy)] +pub(super) enum VariantAlias { + Language(&'static str), + Region(&'static str), + Variant(&'static str), +} + +pub(super) const VARIANT_ALIASES: &[(&str, VariantAlias)] = &[ + ("aaland", VariantAlias::Region("AX")), + ("arevela", VariantAlias::Language("hy")), + ("arevmda", VariantAlias::Language("hyw")), + ("heploc", VariantAlias::Variant("alalc97")), + ("polytoni", VariantAlias::Variant("polyton")), +]; + +/// Unicode extension values: key, deprecated value, preferred value. +pub(super) const UNICODE_TYPE_ALIASES: &[(&str, &str, &str)] = &[ + ("ca", "ethiopic-amete-alem", "ethioaa"), + ("ca", "islamicc", "islamic-civil"), + ("kb", "yes", "true"), + ("kc", "yes", "true"), + ("kh", "yes", "true"), + ("kk", "yes", "true"), + ("kn", "yes", "true"), + ("ks", "primary", "level1"), + ("ks", "tertiary", "level3"), + ("ms", "imperial", "uksystem"), + ("rg", "cn11", "cnbj"), + ("rg", "cn12", "cntj"), + ("rg", "cn13", "cnhe"), + ("rg", "cn14", "cnsx"), + ("rg", "cn15", "cnmn"), + ("rg", "cn21", "cnln"), + ("rg", "cn22", "cnjl"), + ("rg", "cn23", "cnhl"), + ("rg", "cn31", "cnsh"), + ("rg", "cn32", "cnjs"), + ("rg", "cn33", "cnzj"), + ("rg", "cn34", "cnah"), + ("rg", "cn35", "cnfj"), + ("rg", "cn36", "cnjx"), + ("rg", "cn37", "cnsd"), + ("rg", "cn41", "cnha"), + ("rg", "cn42", "cnhb"), + ("rg", "cn43", "cnhn"), + ("rg", "cn44", "cngd"), + ("rg", "cn45", "cngx"), + ("rg", "cn46", "cnhi"), + ("rg", "cn50", "cncq"), + ("rg", "cn51", "cnsc"), + ("rg", "cn52", "cngz"), + ("rg", "cn53", "cnyn"), + ("rg", "cn54", "cnxz"), + ("rg", "cn61", "cnsn"), + ("rg", "cn62", "cngs"), + ("rg", "cn63", "cnqh"), + ("rg", "cn64", "cnnx"), + ("rg", "cn65", "cnxj"), + ("rg", "cz10a", "cz110"), + ("rg", "cz10b", "cz111"), + ("rg", "cz10c", "cz112"), + ("rg", "cz10d", "cz113"), + ("rg", "cz10e", "cz114"), + ("rg", "cz10f", "cz115"), + ("rg", "cz611", "cz663"), + ("rg", "cz612", "cz632"), + ("rg", "cz613", "cz633"), + ("rg", "cz614", "cz634"), + ("rg", "cz615", "cz635"), + ("rg", "cz621", "cz641"), + ("rg", "cz622", "cz642"), + ("rg", "cz623", "cz643"), + ("rg", "cz624", "cz644"), + ("rg", "cz626", "cz646"), + ("rg", "cz627", "cz647"), + ("rg", "czjc", "cz31"), + ("rg", "czjm", "cz64"), + ("rg", "czka", "cz41"), + ("rg", "czkr", "cz52"), + ("rg", "czli", "cz51"), + ("rg", "czmo", "cz80"), + ("rg", "czol", "cz71"), + ("rg", "czpa", "cz53"), + ("rg", "czpl", "cz32"), + ("rg", "czpr", "cz10"), + ("rg", "czst", "cz20"), + ("rg", "czus", "cz42"), + ("rg", "czvy", "cz63"), + ("rg", "czzl", "cz72"), + ("rg", "fra", "frges"), + ("rg", "frb", "frnaq"), + ("rg", "frc", "frara"), + ("rg", "frd", "frbfc"), + ("rg", "fre", "frbre"), + ("rg", "frf", "frcvl"), + ("rg", "frg", "frges"), + ("rg", "frh", "frcor"), + ("rg", "fri", "frbfc"), + ("rg", "frj", "fridf"), + ("rg", "frk", "frocc"), + ("rg", "frl", "frnaq"), + ("rg", "frm", "frges"), + ("rg", "frn", "frocc"), + ("rg", "fro", "frhdf"), + ("rg", "frp", "frnor"), + ("rg", "frq", "frnor"), + ("rg", "frr", "frpdl"), + ("rg", "frs", "frhdf"), + ("rg", "frt", "frnaq"), + ("rg", "fru", "frpac"), + ("rg", "frv", "frara"), + ("rg", "laxn", "laxs"), + ("rg", "lud", "lucl"), + ("rg", "lug", "luec"), + ("rg", "lul", "luca"), + ("rg", "mrnkc", "mr13"), + ("rg", "no23", "no50"), + ("rg", "nzn", "nzauk"), + ("rg", "nzs", "nzcan"), + ("rg", "omba", "ombj"), + ("rg", "omsh", "omsj"), + ("rg", "plds", "pl02"), + ("rg", "plkp", "pl04"), + ("rg", "pllb", "pl08"), + ("rg", "plld", "pl10"), + ("rg", "pllu", "pl06"), + ("rg", "plma", "pl12"), + ("rg", "plmz", "pl14"), + ("rg", "plop", "pl16"), + ("rg", "plpd", "pl20"), + ("rg", "plpk", "pl18"), + ("rg", "plpm", "pl22"), + ("rg", "plsk", "pl26"), + ("rg", "plsl", "pl24"), + ("rg", "plwn", "pl28"), + ("rg", "plwp", "pl30"), + ("rg", "plzp", "pl32"), + ("rg", "tteto", "tttob"), + ("rg", "ttrcm", "ttmrc"), + ("rg", "ttwto", "tttob"), + ("rg", "twkhq", "twkhh"), + ("rg", "twtnq", "twtnn"), + ("rg", "twtpq", "twnwt"), + ("rg", "twtxq", "twtxg"), + ("sd", "cn11", "cnbj"), + ("sd", "cn12", "cntj"), + ("sd", "cn13", "cnhe"), + ("sd", "cn14", "cnsx"), + ("sd", "cn15", "cnmn"), + ("sd", "cn21", "cnln"), + ("sd", "cn22", "cnjl"), + ("sd", "cn23", "cnhl"), + ("sd", "cn31", "cnsh"), + ("sd", "cn32", "cnjs"), + ("sd", "cn33", "cnzj"), + ("sd", "cn34", "cnah"), + ("sd", "cn35", "cnfj"), + ("sd", "cn36", "cnjx"), + ("sd", "cn37", "cnsd"), + ("sd", "cn41", "cnha"), + ("sd", "cn42", "cnhb"), + ("sd", "cn43", "cnhn"), + ("sd", "cn44", "cngd"), + ("sd", "cn45", "cngx"), + ("sd", "cn46", "cnhi"), + ("sd", "cn50", "cncq"), + ("sd", "cn51", "cnsc"), + ("sd", "cn52", "cngz"), + ("sd", "cn53", "cnyn"), + ("sd", "cn54", "cnxz"), + ("sd", "cn61", "cnsn"), + ("sd", "cn62", "cngs"), + ("sd", "cn63", "cnqh"), + ("sd", "cn64", "cnnx"), + ("sd", "cn65", "cnxj"), + ("sd", "cz10a", "cz110"), + ("sd", "cz10b", "cz111"), + ("sd", "cz10c", "cz112"), + ("sd", "cz10d", "cz113"), + ("sd", "cz10e", "cz114"), + ("sd", "cz10f", "cz115"), + ("sd", "cz611", "cz663"), + ("sd", "cz612", "cz632"), + ("sd", "cz613", "cz633"), + ("sd", "cz614", "cz634"), + ("sd", "cz615", "cz635"), + ("sd", "cz621", "cz641"), + ("sd", "cz622", "cz642"), + ("sd", "cz623", "cz643"), + ("sd", "cz624", "cz644"), + ("sd", "cz626", "cz646"), + ("sd", "cz627", "cz647"), + ("sd", "czjc", "cz31"), + ("sd", "czjm", "cz64"), + ("sd", "czka", "cz41"), + ("sd", "czkr", "cz52"), + ("sd", "czli", "cz51"), + ("sd", "czmo", "cz80"), + ("sd", "czol", "cz71"), + ("sd", "czpa", "cz53"), + ("sd", "czpl", "cz32"), + ("sd", "czpr", "cz10"), + ("sd", "czst", "cz20"), + ("sd", "czus", "cz42"), + ("sd", "czvy", "cz63"), + ("sd", "czzl", "cz72"), + ("sd", "fra", "frges"), + ("sd", "frb", "frnaq"), + ("sd", "frc", "frara"), + ("sd", "frd", "frbfc"), + ("sd", "fre", "frbre"), + ("sd", "frf", "frcvl"), + ("sd", "frg", "frges"), + ("sd", "frh", "frcor"), + ("sd", "fri", "frbfc"), + ("sd", "frj", "fridf"), + ("sd", "frk", "frocc"), + ("sd", "frl", "frnaq"), + ("sd", "frm", "frges"), + ("sd", "frn", "frocc"), + ("sd", "fro", "frhdf"), + ("sd", "frp", "frnor"), + ("sd", "frq", "frnor"), + ("sd", "frr", "frpdl"), + ("sd", "frs", "frhdf"), + ("sd", "frt", "frnaq"), + ("sd", "fru", "frpac"), + ("sd", "frv", "frara"), + ("sd", "laxn", "laxs"), + ("sd", "lud", "lucl"), + ("sd", "lug", "luec"), + ("sd", "lul", "luca"), + ("sd", "mrnkc", "mr13"), + ("sd", "no23", "no50"), + ("sd", "nzn", "nzauk"), + ("sd", "nzs", "nzcan"), + ("sd", "omba", "ombj"), + ("sd", "omsh", "omsj"), + ("sd", "plds", "pl02"), + ("sd", "plkp", "pl04"), + ("sd", "pllb", "pl08"), + ("sd", "plld", "pl10"), + ("sd", "pllu", "pl06"), + ("sd", "plma", "pl12"), + ("sd", "plmz", "pl14"), + ("sd", "plop", "pl16"), + ("sd", "plpd", "pl20"), + ("sd", "plpk", "pl18"), + ("sd", "plpm", "pl22"), + ("sd", "plsk", "pl26"), + ("sd", "plsl", "pl24"), + ("sd", "plwn", "pl28"), + ("sd", "plwp", "pl30"), + ("sd", "plzp", "pl32"), + ("sd", "tteto", "tttob"), + ("sd", "ttrcm", "ttmrc"), + ("sd", "ttwto", "tttob"), + ("sd", "twkhq", "twkhh"), + ("sd", "twtnq", "twtnn"), + ("sd", "twtpq", "twnwt"), + ("sd", "twtxq", "twtxg"), + ("tz", "aqams", "nzakl"), + ("tz", "cnckg", "cnsha"), + ("tz", "cnhrb", "cnsha"), + ("tz", "cnkhg", "cnurc"), + ("tz", "cuba", "cuhav"), + ("tz", "egypt", "egcai"), + ("tz", "eire", "iedub"), + ("tz", "est", "papty"), + ("tz", "gmt0", "gmt"), + ("tz", "hongkong", "hkhkg"), + ("tz", "hst", "ushnl"), + ("tz", "iceland", "isrey"), + ("tz", "iran", "irthr"), + ("tz", "israel", "jeruslm"), + ("tz", "jamaica", "jmkin"), + ("tz", "japan", "jptyo"), + ("tz", "libya", "lytip"), + ("tz", "mst", "usphx"), + ("tz", "navajo", "usden"), + ("tz", "poland", "plwaw"), + ("tz", "portugal", "ptlis"), + ("tz", "prc", "cnsha"), + ("tz", "roc", "twtpe"), + ("tz", "rok", "krsel"), + ("tz", "turkey", "trist"), + ("tz", "uct", "utc"), + ("tz", "usnavajo", "usden"), + ("tz", "zulu", "utc"), +]; + +/// Transformed extension values: key, deprecated value, preferred value. +pub(super) const TRANSFORM_TYPE_ALIASES: &[(&str, &str, &str)] = + &[("d0", "name", "charname"), ("m0", "names", "prprname")]; diff --git a/crates/bamts-runtime/src/intl/locale_data.rs b/crates/bamts-runtime/src/intl/locale_data.rs new file mode 100644 index 00000000..950f1489 --- /dev/null +++ b/crates/bamts-runtime/src/intl/locale_data.rs @@ -0,0 +1,1054 @@ +//! The locale data this runtime ships. +//! +//! There is no CLDR here. Identifier aliases are complete (see +//! [`super::locale_aliases`]); everything else is a small table of well-known +//! facts about languages and regions, and only `en` and `en-US` are available +//! locales. A locale outside that set is reported as unsupported through the +//! ordinary `supportedLocalesOf` and fallback paths. + +use std::cmp::Ordering; +use std::sync::LazyLock; + +use super::canonical_locales::SupportedValuesInventory; +use super::collation; +use super::collator::{ + CollatorCompareRequest, CollatorDataProvider, CollatorLocaleDefaults, CollatorSensitivity, + CollatorUsage, +}; +use super::locale_aliases::{ + COMPLEX_REGION_ALIASES, LANGUAGE_ALIASES, REGION_ALIASES, TAG_ALIASES, TRANSFORM_TYPE_ALIASES, + UNICODE_TYPE_ALIASES, VARIANT_ALIASES, VariantAlias, +}; +use super::locale_negotiation::{LanguageId, LanguageTag, LocaleDataProvider}; +use super::number_format::SANCTIONED_UNITS; + +/// The locale used when a request matches nothing available. +pub const DEFAULT_LOCALE: &str = "en-US"; + +/// `language[-Script][-REGION]` to the full identifier it most likely means. +const LIKELY_SUBTAGS: &[(&str, &str)] = &[ + ("aa", "aa-Latn-ET"), + ("aae", "aae-Latn-IT"), + ("af", "af-Latn-ZA"), + ("am", "am-Ethi-ET"), + ("ar", "ar-Arab-EG"), + ("as", "as-Beng-IN"), + ("ast", "ast-Latn-ES"), + ("az", "az-Latn-AZ"), + ("az-Arab", "az-Arab-IR"), + ("az-IQ", "az-Arab-IQ"), + ("az-IR", "az-Arab-IR"), + ("az-RU", "az-Cyrl-RU"), + ("be", "be-Cyrl-BY"), + ("bg", "bg-Cyrl-BG"), + ("bn", "bn-Beng-BD"), + ("bo", "bo-Tibt-CN"), + ("br", "br-Latn-FR"), + ("bs", "bs-Latn-BA"), + ("ca", "ca-Latn-ES"), + ("ceb", "ceb-Latn-PH"), + ("cs", "cs-Latn-CZ"), + ("cy", "cy-Latn-GB"), + ("da", "da-Latn-DK"), + ("de", "de-Latn-DE"), + ("dv", "dv-Thaa-MV"), + ("el", "el-Grek-GR"), + ("en", "en-Latn-US"), + ("en-Shaw", "en-Shaw-GB"), + ("eo", "eo-Latn-001"), + ("es", "es-Latn-ES"), + ("et", "et-Latn-EE"), + ("eu", "eu-Latn-ES"), + ("fa", "fa-Arab-IR"), + ("fi", "fi-Latn-FI"), + ("fil", "fil-Latn-PH"), + ("fo", "fo-Latn-FO"), + ("fr", "fr-Latn-FR"), + ("fy", "fy-Latn-NL"), + ("ga", "ga-Latn-IE"), + ("gd", "gd-Latn-GB"), + ("gl", "gl-Latn-ES"), + ("gsw", "gsw-Latn-CH"), + ("gu", "gu-Gujr-IN"), + ("ha", "ha-Latn-NG"), + ("hak", "hak-Hans-CN"), + ("he", "he-Hebr-IL"), + ("hi", "hi-Deva-IN"), + ("hr", "hr-Latn-HR"), + ("hsn", "hsn-Hans-CN"), + ("hu", "hu-Latn-HU"), + ("hy", "hy-Armn-AM"), + ("hyw", "hyw-Armn-AM"), + ("id", "id-Latn-ID"), + ("ig", "ig-Latn-NG"), + ("is", "is-Latn-IS"), + ("it", "it-Latn-IT"), + ("ja", "ja-Jpan-JP"), + ("jbo", "jbo-Latn-001"), + ("jv", "jv-Latn-ID"), + ("ka", "ka-Geor-GE"), + ("kk", "kk-Cyrl-KZ"), + ("km", "km-Khmr-KH"), + ("kn", "kn-Knda-IN"), + ("ko", "ko-Kore-KR"), + ("ku", "ku-Latn-TR"), + ("ku-Arab", "ku-Arab-IQ"), + ("ky", "ky-Cyrl-KG"), + ("la", "la-Latn-VA"), + ("lb", "lb-Latn-LU"), + ("lo", "lo-Laoo-LA"), + ("lt", "lt-Latn-LT"), + ("lv", "lv-Latn-LV"), + ("mg", "mg-Latn-MG"), + ("mi", "mi-Latn-NZ"), + ("mk", "mk-Cyrl-MK"), + ("ml", "ml-Mlym-IN"), + ("mn", "mn-Cyrl-MN"), + ("mn-CN", "mn-Mong-CN"), + ("mn-Mong", "mn-Mong-CN"), + ("mr", "mr-Deva-IN"), + ("ms", "ms-Latn-MY"), + ("ms-Arab", "ms-Arab-MY"), + ("mt", "mt-Latn-MT"), + ("my", "my-Mymr-MM"), + ("nan", "nan-Hans-CN"), + ("nb", "nb-Latn-NO"), + ("ne", "ne-Deva-NP"), + ("nl", "nl-Latn-NL"), + ("nn", "nn-Latn-NO"), + ("no", "no-Latn-NO"), + ("nv", "nv-Latn-US"), + ("or", "or-Orya-IN"), + ("pa", "pa-Guru-IN"), + ("pa-Arab", "pa-Arab-PK"), + ("pa-PK", "pa-Arab-PK"), + ("pap", "pap-Latn-CW"), + ("pl", "pl-Latn-PL"), + ("ps", "ps-Arab-AF"), + ("pt", "pt-Latn-BR"), + ("rm", "rm-Latn-CH"), + ("ro", "ro-Latn-RO"), + ("ru", "ru-Cyrl-RU"), + ("rw", "rw-Latn-RW"), + ("sa", "sa-Deva-IN"), + ("sco", "sco-Latn-GB"), + ("sd", "sd-Arab-PK"), + ("sd-Deva", "sd-Deva-IN"), + ("sd-IN", "sd-Deva-IN"), + ("si", "si-Sinh-LK"), + ("sk", "sk-Latn-SK"), + ("sl", "sl-Latn-SI"), + ("so", "so-Latn-SO"), + ("sq", "sq-Latn-AL"), + ("sr", "sr-Cyrl-RS"), + ("sr-Latn", "sr-Latn-RS"), + ("sr-ME", "sr-Latn-ME"), + ("sr-RO", "sr-Latn-RO"), + ("sr-TR", "sr-Latn-TR"), + ("sv", "sv-Latn-SE"), + ("sw", "sw-Latn-TZ"), + ("ta", "ta-Taml-IN"), + ("te", "te-Telu-IN"), + ("tg", "tg-Cyrl-TJ"), + ("th", "th-Thai-TH"), + ("ti", "ti-Ethi-ET"), + ("tk", "tk-Latn-TM"), + ("tl", "tl-Latn-PH"), + ("tr", "tr-Latn-TR"), + ("tt", "tt-Cyrl-RU"), + ("ug", "ug-Arab-CN"), + ("uk", "uk-Cyrl-UA"), + ("und", "en-Latn-US"), + ("und-001", "en-Latn-001"), + ("und-150", "en-Latn-150"), + ("und-419", "es-Latn-419"), + ("und-AE", "ar-Arab-AE"), + ("und-AF", "fa-Arab-AF"), + ("und-AM", "hy-Armn-AM"), + ("und-AR", "es-Latn-AR"), + ("und-AT", "de-Latn-AT"), + ("und-AU", "en-Latn-AU"), + ("und-AW", "nl-Latn-AW"), + ("und-AZ", "az-Latn-AZ"), + ("und-Arab", "ar-Arab-EG"), + ("und-Armn", "hy-Armn-AM"), + ("und-BD", "bn-Beng-BD"), + ("und-BE", "nl-Latn-BE"), + ("und-BG", "bg-Cyrl-BG"), + ("und-BR", "pt-Latn-BR"), + ("und-BY", "be-Cyrl-BY"), + ("und-Beng", "bn-Beng-BD"), + ("und-CA", "en-Latn-CA"), + ("und-CH", "de-Latn-CH"), + ("und-CL", "es-Latn-CL"), + ("und-CN", "zh-Hans-CN"), + ("und-CO", "es-Latn-CO"), + ("und-CW", "pap-Latn-CW"), + ("und-CZ", "cs-Latn-CZ"), + ("und-Cyrl", "ru-Cyrl-RU"), + ("und-Cyrl-RO", "bg-Cyrl-RO"), + ("und-DE", "de-Latn-DE"), + ("und-DK", "da-Latn-DK"), + ("und-Deva", "hi-Deva-IN"), + ("und-EG", "ar-Arab-EG"), + ("und-ES", "es-Latn-ES"), + ("und-ET", "am-Ethi-ET"), + ("und-Ethi", "am-Ethi-ET"), + ("und-FI", "fi-Latn-FI"), + ("und-FR", "fr-Latn-FR"), + ("und-GB", "en-Latn-GB"), + ("und-GE", "ka-Geor-GE"), + ("und-GR", "el-Grek-GR"), + ("und-Geor", "ka-Geor-GE"), + ("und-Grek", "el-Grek-GR"), + ("und-Gujr", "gu-Gujr-IN"), + ("und-Guru", "pa-Guru-IN"), + ("und-HK", "zh-Hant-HK"), + ("und-HU", "hu-Latn-HU"), + ("und-Hans", "zh-Hans-CN"), + ("und-Hant", "zh-Hant-TW"), + ("und-Hebr", "he-Hebr-IL"), + ("und-Hira", "ja-Hira-JP"), + ("und-ID", "id-Latn-ID"), + ("und-IE", "en-Latn-IE"), + ("und-IL", "he-Hebr-IL"), + ("und-IN", "hi-Deva-IN"), + ("und-IQ", "ar-Arab-IQ"), + ("und-IR", "fa-Arab-IR"), + ("und-IT", "it-Latn-IT"), + ("und-JP", "ja-Jpan-JP"), + ("und-Jpan", "ja-Jpan-JP"), + ("und-KE", "sw-Latn-KE"), + ("und-KH", "km-Khmr-KH"), + ("und-KR", "ko-Kore-KR"), + ("und-KZ", "ru-Cyrl-KZ"), + ("und-Kana", "ja-Kana-JP"), + ("und-Khmr", "km-Khmr-KH"), + ("und-Knda", "kn-Knda-IN"), + ("und-Kore", "ko-Kore-KR"), + ("und-LA", "lo-Laoo-LA"), + ("und-Laoo", "lo-Laoo-LA"), + ("und-Latn", "en-Latn-US"), + ("und-MM", "my-Mymr-MM"), + ("und-MN", "mn-Cyrl-MN"), + ("und-MX", "es-Latn-MX"), + ("und-MY", "ms-Latn-MY"), + ("und-Mlym", "ml-Mlym-IN"), + ("und-Mong", "mn-Mong-CN"), + ("und-Mymr", "my-Mymr-MM"), + ("und-NG", "en-Latn-NG"), + ("und-NL", "nl-Latn-NL"), + ("und-NO", "nb-Latn-NO"), + ("und-NP", "ne-Deva-NP"), + ("und-NZ", "en-Latn-NZ"), + ("und-Orya", "or-Orya-IN"), + ("und-PE", "es-Latn-PE"), + ("und-PH", "fil-Latn-PH"), + ("und-PK", "ur-Arab-PK"), + ("und-PL", "pl-Latn-PL"), + ("und-PT", "pt-Latn-PT"), + ("und-RO", "ro-Latn-RO"), + ("und-RS", "sr-Cyrl-RS"), + ("und-RU", "ru-Cyrl-RU"), + ("und-SA", "ar-Arab-SA"), + ("und-SE", "sv-Latn-SE"), + ("und-SG", "en-Latn-SG"), + ("und-SK", "sk-Latn-SK"), + ("und-Shaw", "en-Shaw-GB"), + ("und-Sinh", "si-Sinh-LK"), + ("und-TH", "th-Thai-TH"), + ("und-TR", "tr-Latn-TR"), + ("und-TW", "zh-Hant-TW"), + ("und-TZ", "sw-Latn-TZ"), + ("und-Taml", "ta-Taml-IN"), + ("und-Telu", "te-Telu-IN"), + ("und-Thaa", "dv-Thaa-MV"), + ("und-Thai", "th-Thai-TH"), + ("und-Tibt", "bo-Tibt-CN"), + ("und-UA", "uk-Cyrl-UA"), + ("und-US", "en-Latn-US"), + ("und-VE", "es-Latn-VE"), + ("und-VN", "vi-Latn-VN"), + ("und-ZA", "en-Latn-ZA"), + ("ur", "ur-Arab-PK"), + ("ur-IN", "ur-Arab-IN"), + ("uz", "uz-Latn-UZ"), + ("uz-AF", "uz-Arab-AF"), + ("uz-Arab", "uz-Arab-AF"), + ("uz-CN", "uz-Cyrl-CN"), + ("vi", "vi-Latn-VN"), + ("wuu", "wuu-Hans-CN"), + ("xh", "xh-Latn-ZA"), + ("yi", "yi-Hebr-UA"), + ("yo", "yo-Latn-NG"), + ("yue", "yue-Hant-HK"), + ("zh", "zh-Hans-CN"), + ("zh-AU", "zh-Hant-AU"), + ("zh-BN", "zh-Hant-BN"), + ("zh-GB", "zh-Hant-GB"), + ("zh-GF", "zh-Hant-GF"), + ("zh-HK", "zh-Hant-HK"), + ("zh-Hant", "zh-Hant-TW"), + ("zh-ID", "zh-Hant-ID"), + ("zh-MO", "zh-Hant-MO"), + ("zh-PA", "zh-Hant-PA"), + ("zh-PF", "zh-Hant-PF"), + ("zh-PH", "zh-Hant-PH"), + ("zh-SR", "zh-Hant-SR"), + ("zh-TH", "zh-Hant-TH"), + ("zh-TW", "zh-Hant-TW"), + ("zh-US", "zh-Hant-US"), + ("zh-VN", "zh-Hant-VN"), + ("zu", "zu-Latn-ZA"), +]; + +/// Numbering systems whose ten digits are consecutive code points, with their +/// zero. `hanidec`, the one other system with a simple mapping, is not. +const CONTIGUOUS_NUMBERING_SYSTEMS: &[(&str, char)] = &[ + ("adlm", '\u{1E950}'), + ("ahom", '\u{11730}'), + ("arab", '\u{660}'), + ("arabext", '\u{6F0}'), + ("bali", '\u{1B50}'), + ("beng", '\u{9E6}'), + ("bhks", '\u{11C50}'), + ("brah", '\u{11066}'), + ("cakm", '\u{11136}'), + ("cham", '\u{AA50}'), + ("deva", '\u{966}'), + ("diak", '\u{11950}'), + ("fullwide", '\u{FF10}'), + ("gara", '\u{10D40}'), + ("gong", '\u{11DA0}'), + ("gonm", '\u{11D50}'), + ("gujr", '\u{AE6}'), + ("gukh", '\u{16130}'), + ("guru", '\u{A66}'), + ("hmng", '\u{16B50}'), + ("hmnp", '\u{1E140}'), + ("java", '\u{A9D0}'), + ("kali", '\u{A900}'), + ("kawi", '\u{11F50}'), + ("khmr", '\u{17E0}'), + ("knda", '\u{CE6}'), + ("krai", '\u{16D70}'), + ("lana", '\u{1A80}'), + ("lanatham", '\u{1A90}'), + ("laoo", '\u{ED0}'), + ("latn", '\u{30}'), + ("lepc", '\u{1C40}'), + ("limb", '\u{1946}'), + ("mathbold", '\u{1D7CE}'), + ("mathdbl", '\u{1D7D8}'), + ("mathmono", '\u{1D7F6}'), + ("mathsanb", '\u{1D7EC}'), + ("mathsans", '\u{1D7E2}'), + ("mlym", '\u{D66}'), + ("modi", '\u{11650}'), + ("mong", '\u{1810}'), + ("mroo", '\u{16A60}'), + ("mtei", '\u{ABF0}'), + ("mymr", '\u{1040}'), + ("mymrepka", '\u{116DA}'), + ("mymrpao", '\u{116D0}'), + ("mymrshan", '\u{1090}'), + ("mymrtlng", '\u{A9F0}'), + ("nagm", '\u{1E4F0}'), + ("newa", '\u{11450}'), + ("nkoo", '\u{7C0}'), + ("olck", '\u{1C50}'), + ("onao", '\u{1E5F1}'), + ("orya", '\u{B66}'), + ("osma", '\u{104A0}'), + ("outlined", '\u{1CCF0}'), + ("rohg", '\u{10D30}'), + ("saur", '\u{A8D0}'), + ("segment", '\u{1FBF0}'), + ("shrd", '\u{111D0}'), + ("sind", '\u{112F0}'), + ("sinh", '\u{DE6}'), + ("sora", '\u{110F0}'), + ("sund", '\u{1BB0}'), + ("sunu", '\u{11BF0}'), + ("takr", '\u{116C0}'), + ("talu", '\u{19D0}'), + ("tamldec", '\u{BE6}'), + ("telu", '\u{C66}'), + ("thai", '\u{E50}'), + ("tibt", '\u{F20}'), + ("tirh", '\u{114D0}'), + ("tnsa", '\u{16AC0}'), + ("tols", '\u{11DE0}'), + ("vaii", '\u{A620}'), + ("wara", '\u{118E0}'), + ("wcho", '\u{1E2F0}'), +]; + +/// The digits of `hanidec`. +const HANIDEC_DIGITS: [char; 10] = [ + '\u{3007}', '\u{4E00}', '\u{4E8C}', '\u{4E09}', '\u{56DB}', '\u{4E94}', '\u{516D}', '\u{4E03}', + '\u{516B}', '\u{4E5D}', +]; + +/// The ten digits of a numbering system with a simple digit mapping. +#[must_use] +pub fn numbering_system_digits(name: &str) -> Option<[char; 10]> { + if name == "hanidec" { + return Some(HANIDEC_DIGITS); + } + let zero = lookup(CONTIGUOUS_NUMBERING_SYSTEMS, name)?; + Some(std::array::from_fn(|digit| { + char::from_u32(u32::from(zero) + digit as u32).expect("digits are assigned code points") + })) +} + +/// The value of a decimal digit of any numbering system this runtime knows. +#[must_use] +pub fn decimal_digit_value(character: char) -> Option { + if let Some(digit) = character.to_digit(10) { + return Some(digit); + } + let code = u32::from(character); + CONTIGUOUS_NUMBERING_SYSTEMS.iter().find_map(|(_, zero)| { + code.checked_sub(u32::from(*zero)) + .filter(|digit| *digit < 10) + }) +} + +/// ISO 4217 codes. +const CURRENCIES: &[&str] = &[ + "AED", "AFN", "ALL", "AMD", "ANG", "AOA", "ARS", "AUD", "AWG", "AZN", "BAM", "BBD", "BDT", + "BGN", "BHD", "BIF", "BMD", "BND", "BOB", "BOV", "BRL", "BSD", "BTN", "BWP", "BYN", "BZD", + "CAD", "CDF", "CHE", "CHF", "CHW", "CLF", "CLP", "CNY", "COP", "COU", "CRC", "CUC", "CUP", + "CVE", "CZK", "DJF", "DKK", "DOP", "DZD", "EGP", "ERN", "ETB", "EUR", "FJD", "FKP", "GBP", + "GEL", "GHS", "GIP", "GMD", "GNF", "GTQ", "GYD", "HKD", "HNL", "HTG", "HUF", "IDR", "ILS", + "INR", "IQD", "IRR", "ISK", "JMD", "JOD", "JPY", "KES", "KGS", "KHR", "KMF", "KPW", "KRW", + "KWD", "KYD", "KZT", "LAK", "LBP", "LKR", "LRD", "LSL", "LYD", "MAD", "MDL", "MGA", "MKD", + "MMK", "MNT", "MOP", "MRU", "MUR", "MVR", "MWK", "MXN", "MXV", "MYR", "MZN", "NAD", "NGN", + "NIO", "NOK", "NPR", "NZD", "OMR", "PAB", "PEN", "PGK", "PHP", "PKR", "PLN", "PYG", "QAR", + "RON", "RSD", "RUB", "RWF", "SAR", "SBD", "SCR", "SDG", "SEK", "SGD", "SHP", "SLE", "SLL", + "SOS", "SRD", "SSP", "STN", "SVC", "SYP", "SZL", "THB", "TJS", "TMT", "TND", "TOP", "TRY", + "TTD", "TWD", "TZS", "UAH", "UGX", "USD", "USN", "UYI", "UYU", "UYW", "UZS", "VED", "VES", + "VND", "VUV", "WST", "XAF", "XAG", "XAU", "XBA", "XBB", "XBC", "XBD", "XCD", "XCG", "XDR", + "XOF", "XPD", "XPF", "XPT", "XSU", "XTS", "XUA", "XXX", "YER", "ZAR", "ZMW", "ZWG", "ZWL", +]; + +/// Time zones this runtime can compute offsets for: UTC and the fixed `Etc` +/// zones. There is no time zone database, so no zone with transitions. +const TIME_ZONES: [(&str, i8); 27] = [ + ("Etc/GMT+1", -1), + ("Etc/GMT+10", -10), + ("Etc/GMT+11", -11), + ("Etc/GMT+12", -12), + ("Etc/GMT+2", -2), + ("Etc/GMT+3", -3), + ("Etc/GMT+4", -4), + ("Etc/GMT+5", -5), + ("Etc/GMT+6", -6), + ("Etc/GMT+7", -7), + ("Etc/GMT+8", -8), + ("Etc/GMT+9", -9), + ("Etc/GMT-1", 1), + ("Etc/GMT-10", 10), + ("Etc/GMT-11", 11), + ("Etc/GMT-12", 12), + ("Etc/GMT-13", 13), + ("Etc/GMT-14", 14), + ("Etc/GMT-2", 2), + ("Etc/GMT-3", 3), + ("Etc/GMT-4", 4), + ("Etc/GMT-5", 5), + ("Etc/GMT-6", 6), + ("Etc/GMT-7", 7), + ("Etc/GMT-8", 8), + ("Etc/GMT-9", 9), + ("UTC", 0), +]; + +/// The links to UTC, which keep their own names: an identifier is reported as +/// written, in its canonical case, rather than replaced by its target. +const UTC_LINKS: [&str; 10] = [ + "Etc/GMT", + "Etc/Greenwich", + "Etc/UCT", + "Etc/UTC", + "Etc/Universal", + "Etc/Zulu", + "GMT", + "UCT", + "Universal", + "Zulu", +]; + +/// The time zone a case-insensitive identifier names and its offset in hours. +#[must_use] +pub fn time_zone(identifier: &str) -> Option<(&'static str, i8)> { + if let Some(link) = UTC_LINKS + .iter() + .find(|link| link.eq_ignore_ascii_case(identifier)) + { + return Some((link, 0)); + } + TIME_ZONES + .iter() + .find(|(name, _)| name.eq_ignore_ascii_case(identifier)) + .copied() +} + +fn likely_entry(key: &str) -> Option { + let index = LIKELY_SUBTAGS + .binary_search_by(|(entry, _)| (*entry).cmp(key)) + .ok()?; + LanguageTag::parse(LIKELY_SUBTAGS[index].1) + .ok() + .map(|tag| tag.id) +} + +fn join_id(language: &str, script: Option<&str>, region: Option<&str>) -> String { + let mut key = language.to_owned(); + for part in [script, region].into_iter().flatten() { + key.push('-'); + key.push_str(part); + } + key +} + +/// The UTS #35 Add Likely Subtags lookup. Subtags the input already has win. +#[must_use] +pub fn add_likely_subtags(locale: &LanguageId) -> Option { + let language = locale.language.as_str(); + let script = locale.script.as_deref(); + let region = locale.region.as_deref(); + let candidates = [ + Some(join_id(language, script, region)), + Some(join_id(language, None, region)), + Some(join_id(language, script, None)), + Some(join_id(language, None, None)), + script.map(|script| join_id("und", Some(script), None)), + ]; + let found = candidates + .iter() + .flatten() + .find_map(|key| likely_entry(key))?; + Some(LanguageId { + language: if language == "und" { + found.language + } else { + locale.language.clone() + }, + script: locale.script.clone().or(found.script), + region: locale.region.clone().or(found.region), + variants: locale.variants.clone(), + }) +} + +/// The UTS #35 Remove Likely Subtags algorithm, favouring the region. +#[must_use] +pub fn remove_likely_subtags(locale: &LanguageId) -> Option { + let maximal = add_likely_subtags(locale)?; + let trial = |script: Option<&String>, region: Option<&String>| LanguageId { + language: maximal.language.clone(), + script: script.cloned(), + region: region.cloned(), + variants: Vec::new(), + }; + let target = (&maximal.language, &maximal.script, &maximal.region); + for candidate in [ + trial(None, None), + trial(None, maximal.region.as_ref()), + trial(maximal.script.as_ref(), None), + ] { + if add_likely_subtags(&candidate).is_some_and(|expanded| { + (&expanded.language, &expanded.script, &expanded.region) == target + }) { + return Some(LanguageId { + variants: locale.variants.clone(), + ..candidate + }); + } + } + Some(maximal) +} + +/// `-u-rg` and `-u-sd` values start with the region they belong to. +fn keyword_region(tag: &LanguageTag, key: &str) -> Option { + let value = tag.unicode.as_ref()?.keywords.get(key)?; + let region = value.get(..2)?; + region + .bytes() + .all(|byte| byte.is_ascii_alphabetic()) + .then(|| region.to_ascii_uppercase()) +} + +/// RegionPreference: the region that speaks for a locale. Every table below +/// covers all regions by way of a default, so the first signal present decides. +fn preferred_region(tag: &LanguageTag) -> String { + keyword_region(tag, "rg") + .or_else(|| tag.id.region.clone()) + .or_else(|| keyword_region(tag, "sd")) + .or_else(|| add_likely_subtags(&tag.id).and_then(|maximal| maximal.region)) + .unwrap_or_else(|| String::from("001")) +} + +/// Hour cycles for language-region pairs and regions, preferred first. Regions +/// outside the table use the 24-hour clock first, as the world does. +const HOUR_CYCLES: &[(&str, &[&str])] = &[ + ("AE", TWELVE_HOUR), + ("AU", TWELVE_HOUR), + ("BD", TWELVE_HOUR), + ("CA", TWELVE_HOUR), + ("CO", TWELVE_HOUR), + ("DO", TWELVE_HOUR), + ("EG", TWELVE_HOUR), + ("ET", TWELVE_HOUR), + ("GR", TWELVE_HOUR), + ("IN", TWELVE_HOUR), + ("IQ", TWELVE_HOUR), + ("JO", TWELVE_HOUR), + ("KR", TWELVE_HOUR), + ("KW", TWELVE_HOUR), + ("LB", TWELVE_HOUR), + ("MX", TWELVE_HOUR), + ("MY", TWELVE_HOUR), + ("NZ", TWELVE_HOUR), + ("OM", TWELVE_HOUR), + ("PA", TWELVE_HOUR), + ("PH", TWELVE_HOUR), + ("PK", TWELVE_HOUR), + ("PR", TWELVE_HOUR), + ("QA", TWELVE_HOUR), + ("SA", TWELVE_HOUR), + ("SG", TWELVE_HOUR), + ("SY", TWELVE_HOUR), + ("US", TWELVE_HOUR), + ("VE", TWELVE_HOUR), + ("YE", TWELVE_HOUR), + ("ZA", TWELVE_HOUR), + ("fr-CA", TWENTY_FOUR_HOUR), +]; +const TWELVE_HOUR: &[&str] = &["h12", "h23"]; +const TWENTY_FOUR_HOUR: &[&str] = &["h23", "h12"]; + +/// HourCyclesOfLocale without its keyword step. +#[must_use] +pub fn hour_cycles(tag: &LanguageTag) -> &'static [&'static str] { + let region = preferred_region(tag); + let paired = format!("{}-{region}", tag.id.language); + lookup(HOUR_CYCLES, &paired) + .or_else(|| lookup(HOUR_CYCLES, ®ion)) + .unwrap_or(TWENTY_FOUR_HOUR) +} + +/// Calendars in use by region, preferred first. Only calendars this runtime +/// implements are named; other regions use `gregory`. +const CALENDARS_BY_REGION: &[(&str, &[&str])] = &[ + ( + "AF", + &["persian", "gregory", "islamic-civil", "islamic-tbla"], + ), + ( + "EG", + &["gregory", "coptic", "islamic-civil", "islamic-tbla"], + ), + ("ET", &["gregory", "ethiopic"]), + ( + "IL", + &["gregory", "hebrew", "islamic-civil", "islamic-tbla"], + ), + ("IN", &["gregory", "indian"]), + ( + "IR", + &["persian", "gregory", "islamic-civil", "islamic-tbla"], + ), + ("JP", &["gregory", "japanese"]), + ("SA", &["gregory", "islamic-civil", "islamic-tbla"]), + ("TH", &["buddhist", "gregory"]), + ("TW", &["gregory", "roc"]), +]; + +/// CalendarsOfLocale without its keyword step. +#[must_use] +pub fn calendars_of(tag: &LanguageTag) -> &'static [&'static str] { + lookup(CALENDARS_BY_REGION, &preferred_region(tag)).unwrap_or(&["gregory"]) +} + +/// Regions whose week starts on Sunday, Saturday, or Friday. Every other region +/// starts on Monday. +const SUNDAY_FIRST: &[&str] = &[ + "AG", "AS", "AU", "BD", "BR", "BS", "BT", "BW", "BZ", "CA", "CN", "CO", "DM", "DO", "ET", "GT", + "GU", "HK", "HN", "ID", "IL", "IN", "JM", "JP", "KE", "KH", "KR", "LA", "MH", "MM", "MO", "MT", + "MX", "MZ", "NI", "NP", "PA", "PE", "PH", "PK", "PR", "PT", "PY", "SA", "SG", "SV", "TH", "TT", + "TW", "UM", "US", "VE", "VI", "WS", "YE", "ZA", "ZW", +]; +const SATURDAY_FIRST: &[&str] = &[ + "AE", "AF", "BH", "DJ", "DZ", "EG", "IQ", "IR", "JO", "KW", "LY", "OM", "QA", "SD", "SY", +]; + +/// Weekends that are not Saturday and Sunday, as ISO weekday numbers. +const WEEKENDS: &[(&str, &[u8])] = &[ + ("AE", FRIDAY_SATURDAY), + ("AF", &[4, 5]), + ("BH", FRIDAY_SATURDAY), + ("DZ", FRIDAY_SATURDAY), + ("EG", FRIDAY_SATURDAY), + ("IL", FRIDAY_SATURDAY), + ("IN", &[7]), + ("IQ", FRIDAY_SATURDAY), + ("IR", &[5]), + ("JO", FRIDAY_SATURDAY), + ("KW", FRIDAY_SATURDAY), + ("LY", FRIDAY_SATURDAY), + ("OM", FRIDAY_SATURDAY), + ("QA", FRIDAY_SATURDAY), + ("SA", FRIDAY_SATURDAY), + ("SD", FRIDAY_SATURDAY), + ("SY", FRIDAY_SATURDAY), + ("UG", &[7]), + ("YE", FRIDAY_SATURDAY), +]; +const FRIDAY_SATURDAY: &[u8] = &[5, 6]; + +/// WeekInfoOfLocale: the first day and weekend, as ISO weekday numbers. +#[must_use] +pub fn week_info(tag: &LanguageTag) -> (u8, &'static [u8]) { + let region = preferred_region(tag); + let region = region.as_str(); + let first_day = if SUNDAY_FIRST.contains(®ion) { + 7 + } else if SATURDAY_FIRST.contains(®ion) { + 6 + } else if region == "MV" { + 5 + } else { + 1 + }; + (first_day, lookup(WEEKENDS, region).unwrap_or(&[6, 7])) +} + +/// Time zones in use by region, sorted. Regions outside the table report none. +const TIME_ZONES_BY_REGION: &[(&str, &[&str])] = &[ + ("AR", &["America/Argentina/Buenos_Aires"]), + ("AT", &["Europe/Vienna"]), + ("BE", &["Europe/Brussels"]), + ("CH", &["Europe/Zurich"]), + ("CN", &["Asia/Shanghai", "Asia/Urumqi"]), + ("DE", &["Europe/Berlin", "Europe/Busingen"]), + ("ES", &["Africa/Ceuta", "Atlantic/Canary", "Europe/Madrid"]), + ("FR", &["Europe/Paris"]), + ("GB", &["Europe/London"]), + ("IE", &["Europe/Dublin"]), + ("IL", &["Asia/Jerusalem"]), + ("IN", &["Asia/Kolkata"]), + ("IT", &["Europe/Rome"]), + ("JP", &["Asia/Tokyo"]), + ("KR", &["Asia/Seoul"]), + ("NL", &["Europe/Amsterdam"]), + ("NO", &["Europe/Oslo"]), + ("SE", &["Europe/Stockholm"]), + ("TH", &["Asia/Bangkok"]), + ("TW", &["Asia/Taipei"]), + ( + "US", + &[ + "America/Adak", + "America/Anchorage", + "America/Boise", + "America/Chicago", + "America/Denver", + "America/Detroit", + "America/Indiana/Indianapolis", + "America/Juneau", + "America/Kentucky/Louisville", + "America/Los_Angeles", + "America/New_York", + "America/Phoenix", + "Pacific/Honolulu", + ], + ), +]; + +/// TimeZonesOfLocale, `None` without a region subtag. +#[must_use] +pub fn time_zones_of(tag: &LanguageTag) -> Option<&'static [&'static str]> { + let region = tag.id.region.as_deref()?; + Some(lookup(TIME_ZONES_BY_REGION, region).unwrap_or(&[])) +} + +/// TextDirectionOfLocale: whether the locale's script runs right to left. +#[must_use] +pub fn is_right_to_left(tag: &LanguageTag) -> bool { + const RIGHT_TO_LEFT: &[&str] = &[ + "Adlm", "Arab", "Hebr", "Mand", "Nkoo", "Rohg", "Samr", "Syrc", "Thaa", "Yezi", + ]; + let script = tag + .id + .script + .clone() + .or_else(|| add_likely_subtags(&tag.id).and_then(|maximal| maximal.script)); + script.is_some_and(|script| RIGHT_TO_LEFT.contains(&script.as_str())) +} + +/// NumberingSystemsOfLocale without its keyword step. +#[must_use] +pub fn numbering_systems_of(tag: &LanguageTag) -> &'static [&'static str] { + match tag.id.language.as_str() { + "as" | "bn" => &["beng"], + "fa" | "ps" => &["arabext"], + "mr" | "ne" => &["deva"], + _ => &["latn"], + } +} + +/// Reads a sorted `(key, value)` table. +fn lookup(table: &[(&str, T)], key: &str) -> Option { + table + .binary_search_by(|(entry, _)| (*entry).cmp(key)) + .ok() + .map(|index| table[index].1) +} + +fn triple<'a>(table: &'a [(&str, &str, &'a str)], key: &str, value: &str) -> Option<&'a str> { + table + .binary_search_by(|(entry_key, entry_value, _)| { + (*entry_key, *entry_value).cmp(&(key, value)) + }) + .ok() + .map(|index| table[index].2) +} + +/// The collations available everywhere: those of the root locale. +pub const COLLATIONS: &[&str] = &["emoji", "eor"]; + +/// The locale data behind every Intl service. +pub struct IntlData { + available: Vec, + calendars: Vec, + collations: Vec, + currencies: Vec, + numbering_systems: Vec, + time_zones: Vec, + units: Vec, + collation_keywords: Vec, + case_first_keywords: Vec, + numeric_keywords: Vec, + numbering_keywords: Vec, + calendar_keywords: Vec, + hour_cycle_keywords: Vec, +} + +/// The one instance services hold a reference to. +pub static DATA: LazyLock = LazyLock::new(|| { + let owned = |values: &[&str]| values.iter().map(|value| (*value).to_owned()).collect(); + let mut numbering_systems: Vec = CONTIGUOUS_NUMBERING_SYSTEMS + .iter() + .map(|(name, _)| (*name).to_owned()) + .collect(); + numbering_systems.push(String::from("hanidec")); + numbering_systems.sort(); + let mut numbering_keywords = vec![String::from("latn")]; + numbering_keywords.extend( + numbering_systems + .iter() + .filter(|name| *name != "latn") + .cloned(), + ); + IntlData { + available: owned(&["en", "en-US"]), + calendars: owned(&[ + "buddhist", + "coptic", + "ethioaa", + "ethiopic", + "gregory", + "hebrew", + "indian", + "islamic-civil", + "islamic-tbla", + "iso8601", + "persian", + "roc", + ]), + collations: owned(COLLATIONS), + currencies: owned(CURRENCIES), + numbering_systems, + time_zones: TIME_ZONES + .iter() + .map(|(name, _)| (*name).to_owned()) + .collect(), + units: owned(&SANCTIONED_UNITS), + collation_keywords: std::iter::once(String::new()) + .chain(COLLATIONS.iter().map(|value| (*value).to_owned())) + .collect(), + case_first_keywords: owned(&["false", "lower", "upper"]), + numeric_keywords: owned(&["false", "true"]), + numbering_keywords, + calendar_keywords: owned(&[ + "gregory", + "buddhist", + "coptic", + "ethioaa", + "ethiopic", + "hebrew", + "indian", + "islamic-civil", + "islamic-tbla", + "iso8601", + "persian", + "roc", + ]), + hour_cycle_keywords: owned(&["h12", "h11", "h23", "h24"]), + } +}); + +impl SupportedValuesInventory for IntlData { + fn supported_calendars(&self) -> &[String] { + &self.calendars + } + fn supported_collations(&self) -> &[String] { + &self.collations + } + fn supported_currencies(&self) -> &[String] { + &self.currencies + } + fn supported_numbering_systems(&self) -> &[String] { + &self.numbering_systems + } + fn supported_time_zones(&self) -> &[String] { + &self.time_zones + } + fn supported_units(&self) -> &[String] { + &self.units + } +} + +impl CollatorDataProvider for IntlData { + fn locale_defaults(&self, _data_locale: &str, _usage: CollatorUsage) -> CollatorLocaleDefaults { + CollatorLocaleDefaults { + sensitivity: CollatorSensitivity::Variant, + ignore_punctuation: false, + } + } + fn compare(&self, request: CollatorCompareRequest<'_>) -> Option { + Some(collation::compare(&request)) + } +} + +impl LocaleDataProvider for IntlData { + fn available_locales(&self) -> &[String] { + &self.available + } + fn language_alias(&self, language: &str) -> Option<&str> { + lookup(TAG_ALIASES, language).or_else(|| lookup(LANGUAGE_ALIASES, language)) + } + fn region_alias(&self, region: &str) -> Option<&str> { + lookup(REGION_ALIASES, region) + } + fn complex_region_alias( + &self, + region: &str, + language: &str, + script: Option<&str>, + ) -> Option<&str> { + let index = COMPLEX_REGION_ALIASES + .binary_search_by(|(entry, _, _)| (*entry).cmp(region)) + .ok()?; + let (_, default, choices) = COMPLEX_REGION_ALIASES[index]; + let scripted = script.and_then(|script| { + lookup(choices, &join_id(language, Some(script), None)) + .or_else(|| lookup(choices, &join_id("und", Some(script), None))) + }); + Some( + scripted + .or_else(|| lookup(choices, language)) + .unwrap_or(default), + ) + } + fn variant_alias(&self, variant: &str) -> Option<&str> { + match lookup(VARIANT_ALIASES, variant)? { + VariantAlias::Variant(replacement) => Some(replacement), + VariantAlias::Language(_) | VariantAlias::Region(_) => None, + } + } + fn variant_language_alias(&self, variant: &str) -> Option<&str> { + match lookup(VARIANT_ALIASES, variant)? { + VariantAlias::Language(replacement) => Some(replacement), + VariantAlias::Region(_) | VariantAlias::Variant(_) => None, + } + } + fn variant_region_alias(&self, variant: &str) -> Option<&str> { + match lookup(VARIANT_ALIASES, variant)? { + VariantAlias::Region(replacement) => Some(replacement), + VariantAlias::Language(_) | VariantAlias::Variant(_) => None, + } + } + fn unicode_type_alias(&self, key: &str, value: &str) -> Option<&str> { + triple(UNICODE_TYPE_ALIASES, key, value) + } + fn transform_type_alias(&self, key: &str, value: &str) -> Option<&str> { + triple(TRANSFORM_TYPE_ALIASES, key, value) + } + fn key_values(&self, _data_locale: &str, key: &str) -> &[String] { + match key { + "ca" => &self.calendar_keywords, + "co" => &self.collation_keywords, + "hc" => &self.hour_cycle_keywords, + "kf" => &self.case_first_keywords, + "kn" => &self.numeric_keywords, + "nu" => &self.numbering_keywords, + _ => &[], + } + } + fn add_likely_subtags(&self, locale: &LanguageId) -> Option { + add_likely_subtags(locale) + } + fn fallback_locale(&self) -> Option<&str> { + Some(DEFAULT_LOCALE) + } +} + +/// Whether a table in this file is sorted the way its lookups assume. +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn lookup_tables_are_sorted_by_their_keys() { + for table in [ + TAG_ALIASES, + LANGUAGE_ALIASES, + REGION_ALIASES, + LIKELY_SUBTAGS, + ] { + assert!(table.windows(2).all(|pair| pair[0].0 < pair[1].0)); + } + assert!( + COMPLEX_REGION_ALIASES + .windows(2) + .all(|pair| pair[0].0 < pair[1].0) + ); + for (_, _, choices) in COMPLEX_REGION_ALIASES { + assert!(choices.windows(2).all(|pair| pair[0].0 < pair[1].0)); + } + assert!(VARIANT_ALIASES.windows(2).all(|pair| pair[0].0 < pair[1].0)); + for table in [UNICODE_TYPE_ALIASES, TRANSFORM_TYPE_ALIASES] { + assert!( + table + .windows(2) + .all(|pair| (pair[0].0, pair[0].1) < (pair[1].0, pair[1].1)) + ); + } + } + + #[test] + fn likely_subtags_keep_what_the_input_already_has() { + let maximize = |tag: &str| { + let id = LanguageTag::parse(tag).unwrap().id; + add_likely_subtags(&id).unwrap().render() + }; + assert_eq!(maximize("en"), "en-Latn-US"); + assert_eq!(maximize("en-Arab"), "en-Arab-US"); + assert_eq!(maximize("und-419"), "es-Latn-419"); + assert_eq!(maximize("und-Cyrl-RO"), "bg-Cyrl-RO"); + assert_eq!(maximize("und-AQ"), "en-Latn-AQ"); + } +} diff --git a/crates/bamts-runtime/src/intl/locale_negotiation.rs b/crates/bamts-runtime/src/intl/locale_negotiation.rs index 020db219..197e8884 100644 --- a/crates/bamts-runtime/src/intl/locale_negotiation.rs +++ b/crates/bamts-runtime/src/intl/locale_negotiation.rs @@ -64,7 +64,9 @@ pub struct LanguageId { } impl LanguageId { - fn render(&self) -> String { + /// The identifier in canonical subtag order. + #[must_use] + pub fn render(&self) -> String { let mut out = self.language.clone(); if let Some(script) = &self.script { out.push('-'); @@ -120,6 +122,20 @@ impl LanguageTag { /// # Errors /// Returns [`LocaleError`] for malformed grammar or duplicate subtags. pub fn parse(input: &str) -> Result { + Self::parse_with(input, false) + } + + /// Parses as `CanonicalizeUnicodeLocaleId` reads its input: a repeated + /// Unicode attribute or keyword is structurally valid, and the first + /// occurrence stands. + /// + /// # Errors + /// Returns [`LocaleError`] for malformed grammar or other duplicate subtags. + pub fn parse_first_keyword(input: &str) -> Result { + Self::parse_with(input, true) + } + + fn parse_with(input: &str, keep_first: bool) -> Result { if input.is_empty() || !input.is_ascii() || input.starts_with('-') || input.ends_with('-') { return Err(LocaleError::InvalidLanguageTag(input.to_owned())); } @@ -166,11 +182,14 @@ impl LanguageTag { } let body = &parts[start..index]; if singleton == 'u' { - unicode = Some(parse_unicode_extension(body, input)?); + unicode = Some(parse_unicode_extension(body, input, keep_first)?); } else if singleton == 't' { extensions.insert(singleton, parse_transformed_extension(body, input)?); } else { - if body.iter().any(|part| !(2..=8).contains(&part.len()) || !is_alnum(part)) { + if body + .iter() + .any(|part| !(2..=8).contains(&part.len()) || !is_alnum(part)) + { return Err(LocaleError::InvalidLanguageTag(input.to_owned())); } extensions.insert( @@ -180,7 +199,12 @@ impl LanguageTag { } } id.normalize(); - let mut tag = Self { id, unicode, extensions, private_use }; + let mut tag = Self { + id, + unicode, + extensions, + private_use, + }; tag.normalize(); Ok(tag) } @@ -204,6 +228,30 @@ impl LanguageTag { } } + /// Whether `value` matches `unicode_language_subtag`. + #[must_use] + pub fn is_language_subtag(value: &str) -> bool { + is_language(value) + } + + /// Whether `value` matches `unicode_script_subtag`. + #[must_use] + pub fn is_script_subtag(value: &str) -> bool { + is_script(&value) + } + + /// Whether `value` matches `unicode_region_subtag`. + #[must_use] + pub fn is_region_subtag(value: &str) -> bool { + is_region(&value) + } + + /// Whether `value` matches `unicode_variant_subtag`. + #[must_use] + pub fn is_variant_subtag(value: &str) -> bool { + is_variant(value) + } + fn without_unicode(&self) -> Self { let mut result = self.clone(); result.unicode = None; @@ -267,14 +315,20 @@ fn parse_language_id( return Err(LocaleError::InvalidLanguageTag(original.to_owned())); } let mut index = start + 1; - let script = parts.get(index).filter(|value| is_script(value)).map(|value| { - index += 1; - titlecase(value) - }); - let region = parts.get(index).filter(|value| is_region(value)).map(|value| { - index += 1; - value.to_ascii_uppercase() - }); + let script = parts + .get(index) + .filter(|value| is_script(value)) + .map(|value| { + index += 1; + titlecase(value) + }); + let region = parts + .get(index) + .filter(|value| is_region(value)) + .map(|value| { + index += 1; + value.to_ascii_uppercase() + }); let mut variants = Vec::new(); let mut seen = BTreeSet::new(); while let Some(part) = parts.get(index) { @@ -288,15 +342,22 @@ fn parse_language_id( variants.push(variant); index += 1; } - Ok((LanguageId { - language: language.to_ascii_lowercase(), - script, - region, - variants, - }, index)) + Ok(( + LanguageId { + language: language.to_ascii_lowercase(), + script, + region, + variants, + }, + index, + )) } -fn parse_unicode_extension(parts: &[&str], original: &str) -> Result { +fn parse_unicode_extension( + parts: &[&str], + original: &str, + keep_first: bool, +) -> Result { let mut extension = UnicodeExtension::default(); let mut seen_attributes = BTreeSet::new(); let mut index = 0; @@ -305,10 +366,11 @@ fn parse_unicode_extension(parts: &[&str], original: &str) -> Result Result Result>() - .join("-"), - ); + if !repeated { + extension.keywords.insert( + key, + parts[start..index] + .iter() + .map(|part| part.to_ascii_lowercase()) + .collect::>() + .join("-"), + ); + } } extension.attributes.sort(); if extension.attributes.is_empty() && extension.keywords.is_empty() { @@ -388,7 +453,12 @@ fn parse_transformed_extension(parts: &[&str], original: &str) -> Result Result bool { ((2..=3).contains(&value.len()) || (5..=8).contains(&value.len())) && is_alpha(value) } -fn is_script(value: &&str) -> bool { value.len() == 4 && is_alpha(value) } +fn is_script(value: &&str) -> bool { + value.len() == 4 && is_alpha(value) +} fn is_region(value: &&str) -> bool { - (value.len() == 2 && is_alpha(value)) || (value.len() == 3 && value.bytes().all(|byte| byte.is_ascii_digit())) + (value.len() == 2 && is_alpha(value)) + || (value.len() == 3 && value.bytes().all(|byte| byte.is_ascii_digit())) } fn is_variant(value: &str) -> bool { ((5..=8).contains(&value.len()) && is_alnum(value)) || (value.len() == 4 && value.as_bytes()[0].is_ascii_digit() && is_alnum(value)) } fn is_unicode_key(value: &str) -> bool { - value.len() == 2 && value.as_bytes()[0].is_ascii_alphanumeric() && value.as_bytes()[1].is_ascii_alphabetic() + value.len() == 2 + && value.as_bytes()[0].is_ascii_alphanumeric() + && value.as_bytes()[1].is_ascii_alphabetic() +} +fn is_transform_key(value: &str) -> bool { + let bytes = value.as_bytes(); + bytes.len() == 2 && bytes[0].is_ascii_alphabetic() && bytes[1].is_ascii_digit() +} +fn is_alpha(value: &str) -> bool { + value.bytes().all(|byte| byte.is_ascii_alphabetic()) +} +fn is_alnum(value: &str) -> bool { + value.bytes().all(|byte| byte.is_ascii_alphanumeric()) } -fn is_alpha(value: &str) -> bool { value.bytes().all(|byte| byte.is_ascii_alphabetic()) } -fn is_alnum(value: &str) -> bool { value.bytes().all(|byte| byte.is_ascii_alphanumeric()) } fn titlecase(value: &str) -> String { let mut result = value.to_ascii_lowercase(); result[0..1].make_ascii_uppercase(); @@ -429,23 +512,63 @@ pub trait LocaleDataProvider { /// Canonical, deduplicated locales supported by the active Intl service. fn available_locales(&self) -> &[String]; /// Preferred replacement for a language subtag or language identifier. - fn language_alias(&self, _language: &str) -> Option<&str> { None } + fn language_alias(&self, _language: &str) -> Option<&str> { + None + } /// Preferred replacement for a script subtag. - fn script_alias(&self, _script: &str) -> Option<&str> { None } + fn script_alias(&self, _script: &str) -> Option<&str> { + None + } /// Preferred replacement for a region subtag. - fn region_alias(&self, _region: &str) -> Option<&str> { None } + fn region_alias(&self, _region: &str) -> Option<&str> { + None + } /// Preferred replacement for a variant subtag. - fn variant_alias(&self, _variant: &str) -> Option<&str> { None } + fn variant_alias(&self, _variant: &str) -> Option<&str> { + None + } + /// Language a variant subtag stands for; the variant is dropped. + fn variant_language_alias(&self, _variant: &str) -> Option<&str> { + None + } + /// Region a variant subtag stands for; the variant is dropped. + fn variant_region_alias(&self, _variant: &str) -> Option<&str> { + None + } + /// Replacement for a region that split into several, chosen by the + /// language and script it is used with. + fn complex_region_alias( + &self, + _region: &str, + _language: &str, + _script: Option<&str>, + ) -> Option<&str> { + None + } + /// Preferred replacement for a transformed extension value. + fn transform_type_alias(&self, _key: &str, _value: &str) -> Option<&str> { + None + } /// Preferred replacement for a Unicode extension key. - fn unicode_key_alias(&self, _key: &str) -> Option<&str> { None } + fn unicode_key_alias(&self, _key: &str) -> Option<&str> { + None + } /// Preferred replacement for a Unicode extension type. - fn unicode_type_alias(&self, _key: &str, _value: &str) -> Option<&str> { None } + fn unicode_type_alias(&self, _key: &str, _value: &str) -> Option<&str> { + None + } /// Supported values for a relevant extension key. The first value is the default. - fn key_values(&self, _data_locale: &str, _key: &str) -> &[String] { &[] } + fn key_values(&self, _data_locale: &str, _key: &str) -> &[String] { + &[] + } /// Maximizes a language identifier with provider likely-subtags data. - fn add_likely_subtags(&self, _locale: &LanguageId) -> Option { None } + fn add_likely_subtags(&self, _locale: &LanguageId) -> Option { + None + } /// Canonical last-resort locale, which must be in [`Self::available_locales`]. - fn fallback_locale(&self) -> Option<&str> { None } + fn fallback_locale(&self) -> Option<&str> { + None + } } /// Supplies a host-preferred default locale without granting environment access @@ -473,46 +596,66 @@ pub struct MapLocaleData { impl MapLocaleData { /// Creates an empty provider. It intentionally contains no CLDR guesses. #[must_use] - pub fn new() -> Self { Self::default() } + pub fn new() -> Self { + Self::default() + } /// Replaces the supported locale set, sorting and deduplicating it. #[must_use] pub fn with_available(mut self, mut locales: Vec) -> Self { - locales.sort(); locales.dedup(); self.available = locales; self + locales.sort(); + locales.dedup(); + self.available = locales; + self } /// Adds a language or compound-language alias. #[must_use] pub fn with_language_alias(mut self, from: &str, to: &str) -> Self { - self.language_aliases.insert(from.to_ascii_lowercase(), to.to_owned()); self + self.language_aliases + .insert(from.to_ascii_lowercase(), to.to_owned()); + self } /// Adds a script alias. #[must_use] pub fn with_script_alias(mut self, from: &str, to: &str) -> Self { - self.script_aliases.insert(titlecase(from), to.to_owned()); self + self.script_aliases.insert(titlecase(from), to.to_owned()); + self } /// Adds a region alias. #[must_use] pub fn with_region_alias(mut self, from: &str, to: &str) -> Self { - self.region_aliases.insert(from.to_ascii_uppercase(), to.to_owned()); self + self.region_aliases + .insert(from.to_ascii_uppercase(), to.to_owned()); + self } /// Adds a variant alias. #[must_use] pub fn with_variant_alias(mut self, from: &str, to: &str) -> Self { - self.variant_aliases.insert(from.to_ascii_lowercase(), to.to_owned()); self + self.variant_aliases + .insert(from.to_ascii_lowercase(), to.to_owned()); + self } /// Adds a Unicode extension key alias. #[must_use] pub fn with_key_alias(mut self, from: &str, to: &str) -> Self { - self.key_aliases.insert(from.to_ascii_lowercase(), to.to_ascii_lowercase()); self + self.key_aliases + .insert(from.to_ascii_lowercase(), to.to_ascii_lowercase()); + self } /// Adds a Unicode extension type alias. #[must_use] pub fn with_type_alias(mut self, key: &str, from: &str, to: &str) -> Self { - self.type_aliases.insert((key.to_ascii_lowercase(), from.to_ascii_lowercase()), to.to_ascii_lowercase()); self + self.type_aliases.insert( + (key.to_ascii_lowercase(), from.to_ascii_lowercase()), + to.to_ascii_lowercase(), + ); + self } /// Defines supported key values; the first value is the locale-data default. #[must_use] pub fn with_key_values(mut self, locale: &str, key: &str, values: Vec) -> Self { - self.key_values.insert((locale.to_owned(), key.to_ascii_lowercase()), values); self + self.key_values + .insert((locale.to_owned(), key.to_ascii_lowercase()), values); + self } /// Adds one likely-subtags expansion. /// @@ -529,41 +672,201 @@ impl MapLocaleData { } /// Defines the provider's canonical last-resort locale. #[must_use] - pub fn with_fallback(mut self, locale: &str) -> Self { self.fallback = Some(locale.to_owned()); self } + pub fn with_fallback(mut self, locale: &str) -> Self { + self.fallback = Some(locale.to_owned()); + self + } } impl LocaleDataProvider for MapLocaleData { - fn available_locales(&self) -> &[String] { &self.available } - fn language_alias(&self, value: &str) -> Option<&str> { self.language_aliases.get(value).map(String::as_str) } - fn script_alias(&self, value: &str) -> Option<&str> { self.script_aliases.get(value).map(String::as_str) } - fn region_alias(&self, value: &str) -> Option<&str> { self.region_aliases.get(value).map(String::as_str) } - fn variant_alias(&self, value: &str) -> Option<&str> { self.variant_aliases.get(value).map(String::as_str) } - fn unicode_key_alias(&self, value: &str) -> Option<&str> { self.key_aliases.get(value).map(String::as_str) } + fn available_locales(&self) -> &[String] { + &self.available + } + fn language_alias(&self, value: &str) -> Option<&str> { + self.language_aliases.get(value).map(String::as_str) + } + fn script_alias(&self, value: &str) -> Option<&str> { + self.script_aliases.get(value).map(String::as_str) + } + fn region_alias(&self, value: &str) -> Option<&str> { + self.region_aliases.get(value).map(String::as_str) + } + fn variant_alias(&self, value: &str) -> Option<&str> { + self.variant_aliases.get(value).map(String::as_str) + } + fn unicode_key_alias(&self, value: &str) -> Option<&str> { + self.key_aliases.get(value).map(String::as_str) + } fn unicode_type_alias(&self, key: &str, value: &str) -> Option<&str> { - self.type_aliases.get(&(key.to_owned(), value.to_owned())).map(String::as_str) + self.type_aliases + .get(&(key.to_owned(), value.to_owned())) + .map(String::as_str) } fn key_values(&self, locale: &str, key: &str) -> &[String] { - self.key_values.get(&(locale.to_owned(), key.to_owned())).map_or(&[], Vec::as_slice) + self.key_values + .get(&(locale.to_owned(), key.to_owned())) + .map_or(&[], Vec::as_slice) } fn add_likely_subtags(&self, locale: &LanguageId) -> Option { let candidates = [ locale.render(), - LanguageId { language: locale.language.clone(), script: None, region: locale.region.clone(), variants: Vec::new() }.render(), - LanguageId { language: locale.language.clone(), script: locale.script.clone(), region: None, variants: Vec::new() }.render(), + LanguageId { + language: locale.language.clone(), + script: None, + region: locale.region.clone(), + variants: Vec::new(), + } + .render(), + LanguageId { + language: locale.language.clone(), + script: locale.script.clone(), + region: None, + variants: Vec::new(), + } + .render(), locale.language.clone(), ]; for candidate in candidates { if let Some(expanded) = self.likely_subtags.get(&candidate) { let mut result = expanded.clone(); - if locale.script.is_some() { result.script.clone_from(&locale.script); } - if locale.region.is_some() { result.region.clone_from(&locale.region); } + if locale.script.is_some() { + result.script.clone_from(&locale.script); + } + if locale.region.is_some() { + result.region.clone_from(&locale.region); + } result.variants.clone_from(&locale.variants); return Some(result); } } None } - fn fallback_locale(&self) -> Option<&str> { self.fallback.as_deref() } + fn fallback_locale(&self) -> Option<&str> { + self.fallback.as_deref() + } +} + +/// Applies the UTS #35 alias rules to a language identifier. Whole-identifier +/// aliases are tried first, because `art-lojban` and `sgn-GR` name one language +/// even though their parts are ordinary subtags. +fn canonicalize_language_id( + id: &mut LanguageId, + provider: &dyn LocaleDataProvider, +) -> Result<(), LocaleError> { + apply_language_aliases(id, provider)?; + if let Some(script) = id.script.clone() + && let Some(alias) = provider.script_alias(&script) + { + id.script = Some(titlecase(alias)); + } + if let Some(region) = id.region.clone() { + let replacement = provider + .complex_region_alias(®ion, &id.language, id.script.as_deref()) + .or_else(|| provider.region_alias(®ion)); + if let Some(alias) = replacement { + id.region = alias + .split_ascii_whitespace() + .next() + .map(str::to_ascii_uppercase); + } + } + let variants = std::mem::take(&mut id.variants); + let combined = variants.iter().any(|variant| variant == "hepburn") + && variants.iter().any(|variant| variant == "heploc"); + for variant in variants { + if combined && variant == "hepburn" { + continue; + } + if let Some(language) = provider.variant_language_alias(&variant) { + id.language = language.to_owned(); + } else if let Some(region) = provider.variant_region_alias(&variant) { + id.region = Some(region.to_ascii_uppercase()); + } else if let Some(alias) = provider.variant_alias(&variant) { + id.variants.push(alias.to_ascii_lowercase()); + } else { + id.variants.push(variant); + } + } + id.variants.sort(); + id.variants.dedup(); + Ok(()) +} + +fn apply_language_aliases( + id: &mut LanguageId, + provider: &dyn LocaleDataProvider, +) -> Result<(), LocaleError> { + for index in 0..id.variants.len() { + let whole = format!("{}-{}", id.language, id.variants[index]); + if let Some(alias) = provider.language_alias(&whole) { + let alias = LanguageTag::parse(alias)?; + id.variants.remove(index); + adopt_language(id, alias.id); + return Ok(()); + } + } + if let Some(region) = &id.region { + let whole = format!("{}-{region}", id.language); + if let Some(alias) = provider.language_alias(&whole) { + let alias = LanguageTag::parse(alias)?; + id.region = None; + adopt_language(id, alias.id); + return Ok(()); + } + } + if let Some(alias) = provider.language_alias(&id.language) { + let alias = LanguageTag::parse(alias)?; + adopt_language(id, alias.id); + } + Ok(()) +} + +fn adopt_language(id: &mut LanguageId, replacement: LanguageId) { + id.language = replacement.language; + if id.script.is_none() { + id.script = replacement.script; + } + if id.region.is_none() { + id.region = replacement.region; + } + id.variants.extend(replacement.variants); +} + +/// Canonicalizes the body of a `-t-` extension: its language identifier by the +/// same rules as the main one, in lowercase, and each field's value. +fn canonicalize_transformed_extension( + body: &[String], + provider: &dyn LocaleDataProvider, +) -> Result, LocaleError> { + let parts: Vec<&str> = body.iter().map(String::as_str).collect(); + let original = body.join("-"); + let mut index = 0; + let mut result = Vec::new(); + if parts.first().is_some_and(|part| is_language(part)) { + let (mut id, next) = parse_language_id(&parts, 0, &original)?; + index = next; + canonicalize_language_id(&mut id, provider)?; + id.normalize(); + result.extend( + id.render() + .to_ascii_lowercase() + .split('-') + .map(str::to_owned), + ); + } + while index < parts.len() { + let key = parts[index]; + index += 1; + let start = index; + while index < parts.len() && !is_transform_key(parts[index]) { + index += 1; + } + let value = parts[start..index].join("-"); + result.push(key.to_owned()); + let value = provider.transform_type_alias(key, &value).unwrap_or(&value); + result.extend(value.split('-').map(str::to_owned)); + } + Ok(result) } /// Canonicalizes a Unicode locale identifier using provider alias data. @@ -574,37 +877,32 @@ pub fn canonicalize_unicode_locale_id( locale: &str, provider: &dyn LocaleDataProvider, ) -> Result { - let mut tag = LanguageTag::parse(locale)?; - if let Some(alias) = provider.language_alias(&tag.id.language) { - let replacement = LanguageTag::parse(alias)?; - tag.id.language = replacement.id.language; - if tag.id.script.is_none() { tag.id.script = replacement.id.script; } - if tag.id.region.is_none() { tag.id.region = replacement.id.region; } - tag.id.variants.extend(replacement.id.variants); - } - if let Some(script) = tag.id.script.clone() - && let Some(alias) = provider.script_alias(&script) - { - tag.id.script = Some(titlecase(alias)); - } - if let Some(region) = tag.id.region.clone() - && let Some(alias) = provider.region_alias(®ion) - { - tag.id.region = alias.split_ascii_whitespace().next().map(str::to_ascii_uppercase); - } - for variant in &mut tag.id.variants { - if let Some(alias) = provider.variant_alias(variant) { *variant = alias.to_ascii_lowercase(); } - } + let mut tag = LanguageTag::parse_first_keyword(locale)?; + canonicalize_language_id(&mut tag.id, provider)?; if let Some(extension) = &mut tag.unicode { let old = std::mem::take(&mut extension.keywords); for (key, value) in old { - let canonical_key = provider.unicode_key_alias(&key).unwrap_or(&key).to_ascii_lowercase(); - let canonical_value = provider.unicode_type_alias(&canonical_key, &value).unwrap_or(&value).to_ascii_lowercase(); - if extension.keywords.insert(canonical_key.clone(), canonical_value).is_some() { + let canonical_key = provider + .unicode_key_alias(&key) + .unwrap_or(&key) + .to_ascii_lowercase(); + let canonical_value = provider + .unicode_type_alias(&canonical_key, &value) + .unwrap_or(&value) + .to_ascii_lowercase(); + if extension + .keywords + .insert(canonical_key.clone(), canonical_value) + .is_some() + { return Err(LocaleError::DuplicateSubtag(canonical_key)); } } } + if let Some(body) = tag.extensions.get(&'t') { + let canonical = canonicalize_transformed_extension(body, provider)?; + tag.extensions.insert('t', canonical); + } tag.normalize(); LanguageTag::parse(&tag.render()).map(|validated| validated.render()) } @@ -621,16 +919,21 @@ pub fn canonicalize_locale_list( let mut result = Vec::new(); for locale in locales { let canonical = canonicalize_unicode_locale_id(locale, provider)?; - if seen.insert(canonical.clone()) { result.push(canonical); } + if seen.insert(canonical.clone()) { + result.push(canonical); + } } Ok(result) } /// Locale matching strategy selected by `localeMatcher`. #[derive(Clone, Copy, Debug, Eq, PartialEq)] -pub enum LocaleMatcher { /// RFC 4647 lookup matching. - Lookup, /// Provider-driven best-fit matching. - BestFit } +pub enum LocaleMatcher { + /// RFC 4647 lookup matching. + Lookup, + /// Provider-driven best-fit matching. + BestFit, +} /// The locale and requested Unicode extension selected by a matcher. #[derive(Clone, Debug, Eq, PartialEq)] @@ -644,11 +947,17 @@ pub struct MatchResult { fn best_available_locale(available: &[String], candidate: &str) -> Option { let mut candidate = candidate.to_owned(); loop { - if available.binary_search(&candidate).is_ok() || available.iter().any(|item| item == &candidate) { + if available.binary_search(&candidate).is_ok() + || available.iter().any(|item| item == &candidate) + { return Some(candidate); } let position = candidate.rfind('-')?; - let truncate = if position >= 2 && candidate.as_bytes()[position - 2] == b'-' { position - 2 } else { position }; + let truncate = if position >= 2 && candidate.as_bytes()[position - 2] == b'-' { + position - 2 + } else { + position + }; candidate.truncate(truncate); } } @@ -663,8 +972,14 @@ pub fn lookup_matcher( ) -> Result { for requested_locale in canonicalize_locale_list(requested, provider)? { let tag = LanguageTag::parse(&requested_locale)?; - if let Some(locale) = best_available_locale(provider.available_locales(), &tag.without_unicode().render()) { - return Ok(MatchResult { locale, extension: tag.unicode }); + if let Some(locale) = best_available_locale( + provider.available_locales(), + &tag.without_unicode().render(), + ) { + return Ok(MatchResult { + locale, + extension: tag.unicode, + }); } } fallback_match(provider) @@ -673,11 +988,26 @@ pub fn lookup_matcher( fn fallback_match(provider: &dyn LocaleDataProvider) -> Result { if let Some(locale) = provider.fallback_locale() { let locale = canonicalize_unicode_locale_id(locale, provider)?; - if provider.available_locales().iter().any(|available| available == &locale) { - return Ok(MatchResult { locale, extension: None }); + if provider + .available_locales() + .iter() + .any(|available| available == &locale) + { + return Ok(MatchResult { + locale, + extension: None, + }); } } - provider.available_locales().first().cloned().map(|locale| MatchResult { locale, extension: None }).ok_or(LocaleError::NoAvailableLocale) + provider + .available_locales() + .first() + .cloned() + .map(|locale| MatchResult { + locale, + extension: None, + }) + .ok_or(LocaleError::NoAvailableLocale) } /// Applies provider-driven best-fit matching. If likely-subtags data is absent, @@ -703,6 +1033,15 @@ fn best_fit_available( provider: &dyn LocaleDataProvider, ) -> Result, LocaleError> { let requested_tag = LanguageTag::parse(requested_locale)?; + if let Some(locale) = best_available_locale( + provider.available_locales(), + &requested_tag.without_unicode().render(), + ) { + return Ok(Some(MatchResult { + locale, + extension: requested_tag.unicode, + })); + } if let Some(max_requested) = provider.add_likely_subtags(&requested_tag.id) { let mut best: Option<(u8, String)> = None; for available in provider.available_locales() { @@ -723,14 +1062,20 @@ fn best_fit_available( } } if let Some((_, locale)) = best { - return Ok(Some(MatchResult { locale, extension: requested_tag.unicode })); + return Ok(Some(MatchResult { + locale, + extension: requested_tag.unicode, + })); } } Ok(best_available_locale( provider.available_locales(), &requested_tag.without_unicode().render(), ) - .map(|locale| MatchResult { locale, extension: requested_tag.unicode })) + .map(|locale| MatchResult { + locale, + extension: requested_tag.unicode, + })) } /// Returns the requested locales supported by a service, preserving requested @@ -749,11 +1094,17 @@ pub fn supported_locales( let matched = match matcher { LocaleMatcher::Lookup => { let tag = LanguageTag::parse(&locale)?; - best_available_locale(provider.available_locales(), &tag.without_unicode().render()).is_some() + best_available_locale( + provider.available_locales(), + &tag.without_unicode().render(), + ) + .is_some() } LocaleMatcher::BestFit => best_fit_available(&locale, provider)?.is_some(), }; - if matched { result.push(locale); } + if matched { + result.push(locale); + } } Ok(result) } @@ -793,28 +1144,60 @@ pub fn resolve_locale( let supported = provider.key_values(&matched.locale, &key); let mut value = supported.first().cloned().unwrap_or_default(); let mut requested_supported = false; - if let Some(requested) = matched.extension.as_ref().and_then(|extension| extension.keywords.get(&key)) { - let requested = if requested.is_empty() { "true" } else { requested }; - if supported.iter().any(|supported_value| supported_value == requested) { + if let Some(requested) = matched + .extension + .as_ref() + .and_then(|extension| extension.keywords.get(&key)) + { + let requested = if requested.is_empty() { + "true" + } else { + requested + }; + if supported + .iter() + .any(|supported_value| supported_value == requested) + { value = requested.to_owned(); requested_supported = true; } } - if let Some(option) = options.get(&key) - && supported.iter().any(|supported_value| supported_value == option) - { - if option != &value { requested_supported = false; } - value.clone_from(option); + if let Some(option) = options.get(&key) { + if option.is_empty() { + requested_supported = false; + } else if supported + .iter() + .any(|supported_value| supported_value == option) + { + if option != &value { + requested_supported = false; + } + value.clone_from(option); + } } if requested_supported { - supported_keywords.insert(key.clone(), if value == "true" { String::new() } else { value.clone() }); + supported_keywords.insert( + key.clone(), + if value == "true" { + String::new() + } else { + value.clone() + }, + ); } values.insert(key, value); } if !supported_keywords.is_empty() { - result_tag.unicode = Some(UnicodeExtension { attributes: Vec::new(), keywords: supported_keywords }); + result_tag.unicode = Some(UnicodeExtension { + attributes: Vec::new(), + keywords: supported_keywords, + }); } - Ok(ResolvedLocale { locale: result_tag.render(), data_locale: matched.locale, values }) + Ok(ResolvedLocale { + locale: result_tag.render(), + data_locale: matched.locale, + values, + }) } /// Resolves the injected host preference against provider locales, then uses @@ -830,7 +1213,10 @@ pub fn default_locale( if !preferred.is_empty() { for locale in canonicalize_locale_list(&preferred, provider)? { let tag = LanguageTag::parse(&locale)?; - if let Some(found) = best_available_locale(provider.available_locales(), &tag.without_unicode().render()) { + if let Some(found) = best_available_locale( + provider.available_locales(), + &tag.without_unicode().render(), + ) { return Ok(found); } } @@ -844,7 +1230,15 @@ mod tests { fn provider() -> MapLocaleData { MapLocaleData::new() - .with_available(vec!["en".into(), "en-GB".into(), "he-IL".into(), "sr-Cyrl".into(), "sr-Latn".into(), "zh-Hans".into(), "zh-Hant".into()]) + .with_available(vec![ + "en".into(), + "en-GB".into(), + "he-IL".into(), + "sr-Cyrl".into(), + "sr-Latn".into(), + "zh-Hans".into(), + "zh-Hant".into(), + ]) .with_language_alias("iw", "he") .with_language_alias("sh", "sr-Latn") .with_region_alias("BU", "MM") @@ -857,24 +1251,50 @@ mod tests { #[test] fn canonical_aliases_come_only_from_provider() { - assert_eq!(canonicalize_unicode_locale_id("IW-il-u-ca-islamicc", &provider()), Ok("he-IL-u-ca-islamic-civil".into())); - assert_eq!(canonicalize_unicode_locale_id("iw-IL", &MapLocaleData::new()), Ok("iw-IL".into())); - assert_eq!(canonicalize_unicode_locale_id("sh-Cyrl", &provider()), Ok("sr-Cyrl".into())); + assert_eq!( + canonicalize_unicode_locale_id("IW-il-u-ca-islamicc", &provider()), + Ok("he-IL-u-ca-islamic-civil".into()) + ); + assert_eq!( + canonicalize_unicode_locale_id("iw-IL", &MapLocaleData::new()), + Ok("iw-IL".into()) + ); + assert_eq!( + canonicalize_unicode_locale_id("sh-Cyrl", &provider()), + Ok("sr-Cyrl".into()) + ); } #[test] fn structural_ordering_and_true_are_canonical() { - let value = canonicalize_unicode_locale_id("EN-latn-us-1996-u-FOO-kn-true-ca-buddhist-a-ZZZ", &MapLocaleData::new()); - assert_eq!(value, Ok("en-Latn-US-1996-a-zzz-u-foo-ca-buddhist-kn".into())); + let value = canonicalize_unicode_locale_id( + "EN-latn-us-1996-u-FOO-kn-true-ca-buddhist-a-ZZZ", + &MapLocaleData::new(), + ); + assert_eq!( + value, + Ok("en-Latn-US-1996-a-zzz-u-foo-ca-buddhist-kn".into()) + ); let once = value.unwrap_or_default(); - assert_eq!(canonicalize_unicode_locale_id(&once, &MapLocaleData::new()), Ok(once)); + assert_eq!( + canonicalize_unicode_locale_id(&once, &MapLocaleData::new()), + Ok(once) + ); } #[test] fn invalid_tags_are_rejected() { for invalid in [ - "", "e", "en--US", "en-US-", "en-u", "en-x", "en-abcdefghi", - "i-klingon", "en-GB-oed", "en-t-a0", + "", + "e", + "en--US", + "en-US-", + "en-u", + "en-x", + "en-abcdefghi", + "i-klingon", + "en-GB-oed", + "en-t-a0", ] { assert!(LanguageTag::parse(invalid).is_err(), "accepted {invalid}"); } @@ -890,15 +1310,29 @@ mod tests { "en-t-es-1996-1996-h0-hybrid", "en-t-h0-hybrid-h0-ascii", ] { - assert!(matches!(LanguageTag::parse(invalid), Err(LocaleError::DuplicateSubtag(_))), "wrong result for {invalid}"); + assert!( + matches!( + LanguageTag::parse(invalid), + Err(LocaleError::DuplicateSubtag(_)) + ), + "wrong result for {invalid}" + ); } } #[test] fn lookup_truncates_and_drops_singleton_sequences() { - let data = MapLocaleData::new().with_available(vec!["de".into()]).with_fallback("de"); - assert_eq!(lookup_matcher(&["de-DE-x-goethe".into()], &data).map(|result| result.locale), Ok("de".into())); - assert_eq!(lookup_matcher(&["de-a-foo".into()], &data).map(|result| result.locale), Ok("de".into())); + let data = MapLocaleData::new() + .with_available(vec!["de".into()]) + .with_fallback("de"); + assert_eq!( + lookup_matcher(&["de-DE-x-goethe".into()], &data).map(|result| result.locale), + Ok("de".into()) + ); + assert_eq!( + lookup_matcher(&["de-a-foo".into()], &data).map(|result| result.locale), + Ok("de".into()) + ); } #[test] @@ -906,23 +1340,44 @@ mod tests { let data = provider(); let requested = vec!["en-u-ca-buddhist-kn".into()]; let keys = vec!["ca".into(), "kn".into()]; - let extension = resolve_locale(&requested, &BTreeMap::new(), &keys, LocaleMatcher::Lookup, &data).unwrap_or_else(|error| panic!("{error}")); + let extension = resolve_locale( + &requested, + &BTreeMap::new(), + &keys, + LocaleMatcher::Lookup, + &data, + ) + .unwrap_or_else(|error| panic!("{error}")); assert_eq!(extension.locale, "en-u-ca-buddhist-kn"); - assert_eq!(extension.values.get("ca").map(String::as_str), Some("buddhist")); + assert_eq!( + extension.values.get("ca").map(String::as_str), + Some("buddhist") + ); assert_eq!(extension.values.get("kn").map(String::as_str), Some("true")); let options = BTreeMap::from([("ca".into(), "gregory".into())]); - let overridden = resolve_locale(&requested, &options, &keys, LocaleMatcher::Lookup, &data).unwrap_or_else(|error| panic!("{error}")); + let overridden = resolve_locale(&requested, &options, &keys, LocaleMatcher::Lookup, &data) + .unwrap_or_else(|error| panic!("{error}")); assert_eq!(overridden.locale, "en-u-kn"); - assert_eq!(overridden.values.get("ca").map(String::as_str), Some("gregory")); + assert_eq!( + overridden.values.get("ca").map(String::as_str), + Some("gregory") + ); } struct Hook(Vec); - impl HostLocaleHook for Hook { fn preferred_locales(&self) -> Vec { self.0.clone() } } + impl HostLocaleHook for Hook { + fn preferred_locales(&self) -> Vec { + self.0.clone() + } + } #[test] fn default_locale_uses_hook_then_provider_fallback() { let data = provider(); - assert_eq!(default_locale(&Hook(vec!["en-GB-u-ca-buddhist".into()]), &data), Ok("en-GB".into())); + assert_eq!( + default_locale(&Hook(vec!["en-GB-u-ca-buddhist".into()]), &data), + Ok("en-GB".into()) + ); assert_eq!(default_locale(&Hook(Vec::new()), &data), Ok("en".into())); } @@ -931,7 +1386,10 @@ mod tests { let requested = vec!["EN-gb-u-ca-gregory".into(), "fr-FR".into(), "en".into()]; let first = supported_locales(&requested, LocaleMatcher::Lookup, &provider()); assert_eq!(first, Ok(vec!["en-GB-u-ca-gregory".into(), "en".into()])); - assert_eq!(first, supported_locales(&requested, LocaleMatcher::Lookup, &provider())); + assert_eq!( + first, + supported_locales(&requested, LocaleMatcher::Lookup, &provider()) + ); assert_eq!( supported_locales(&requested, LocaleMatcher::BestFit, &provider()), Ok(vec!["en-GB-u-ca-gregory".into(), "en".into()]), @@ -943,11 +1401,22 @@ mod tests { let data = MapLocaleData::new() .with_available(vec!["zh-Hans".into(), "zh-Hant".into()]) .with_fallback("zh-Hans") - .with_likely_subtag("zh-TW", "zh-Hant-TW").unwrap_or_else(|error| panic!("{error}")) - .with_likely_subtag("zh-Hant", "zh-Hant-TW").unwrap_or_else(|error| panic!("{error}")) - .with_likely_subtag("zh-Hans", "zh-Hans-CN").unwrap_or_else(|error| panic!("{error}")); - assert_eq!(best_fit_matcher(&["zh-TW".into()], &data).map(|result| result.locale), Ok("zh-Hant".into())); - let no_likely = MapLocaleData::new().with_available(vec!["en".into()]).with_fallback("en"); - assert_eq!(best_fit_matcher(&["en-US".into()], &no_likely), lookup_matcher(&["en-US".into()], &no_likely)); + .with_likely_subtag("zh-TW", "zh-Hant-TW") + .unwrap_or_else(|error| panic!("{error}")) + .with_likely_subtag("zh-Hant", "zh-Hant-TW") + .unwrap_or_else(|error| panic!("{error}")) + .with_likely_subtag("zh-Hans", "zh-Hans-CN") + .unwrap_or_else(|error| panic!("{error}")); + assert_eq!( + best_fit_matcher(&["zh-TW".into()], &data).map(|result| result.locale), + Ok("zh-Hant".into()) + ); + let no_likely = MapLocaleData::new() + .with_available(vec!["en".into()]) + .with_fallback("en"); + assert_eq!( + best_fit_matcher(&["en-US".into()], &no_likely), + lookup_matcher(&["en-US".into()], &no_likely) + ); } } diff --git a/crates/bamts-runtime/src/intl/mod.rs b/crates/bamts-runtime/src/intl/mod.rs index c9657b46..becd6b57 100644 --- a/crates/bamts-runtime/src/intl/mod.rs +++ b/crates/bamts-runtime/src/intl/mod.rs @@ -2,16 +2,24 @@ use std::fmt; -pub mod locale_negotiation; pub mod canonical_locales; +mod collation; pub mod collator; +mod date_data; pub mod date_time_format; +pub mod duration_format; +mod locale_aliases; +pub mod locale_data; +pub mod locale_negotiation; +pub mod number_data; pub mod number_format; pub mod plural_rules; +mod segmentation; +pub mod service_data; #[derive(Clone, Copy, Debug, Eq, PartialEq)] #[repr(u8)] -pub(crate) enum IntlKind { +pub enum IntlKind { Collator, NumberFormat, DateTimeFormat, @@ -21,20 +29,37 @@ pub(crate) enum IntlKind { Segmenter, DisplayNames, Locale, + DurationFormat, + Segments, + SegmentIterator, } impl IntlKind { - pub(crate) const COUNT: usize = 9; + pub const COUNT: usize = 12; + pub const ALL: [Self; Self::COUNT] = [ + Self::Collator, + Self::NumberFormat, + Self::DateTimeFormat, + Self::PluralRules, + Self::RelativeTimeFormat, + Self::ListFormat, + Self::Segmenter, + Self::DisplayNames, + Self::Locale, + Self::DurationFormat, + Self::Segments, + Self::SegmentIterator, + ]; - pub(crate) const fn index(self) -> usize { + pub const fn index(self) -> usize { self as usize } } #[derive(Clone)] -pub(crate) enum IntlObject { +pub enum IntlObject { Collator(collator::Collator), - NumberFormat(number_format::NumberFormat<'static>), + NumberFormat(Box>), DateTimeFormat(date_time_format::DateTimeFormat), PluralRules(plural_rules::PluralRules<'static>), RelativeTimeFormat(plural_rules::RelativeTimeFormat<'static>), @@ -42,10 +67,28 @@ pub(crate) enum IntlObject { Segmenter(plural_rules::Segmenter<'static>), DisplayNames(plural_rules::DisplayNames<'static>), Locale(locale_negotiation::LanguageTag), + DurationFormat(Box), + Segments(Box), + SegmentIterator(Box), +} + +/// The state of a `Segments` object: the text and how it broke up. +#[derive(Clone, Debug)] +pub struct SegmentsRecord { + pub text: plural_rules::SegmentedText, + pub granularity: plural_rules::SegmenterGranularity, +} + +/// The state of a Segment Iterator: what it walks and where it is. +#[derive(Clone, Debug)] +pub struct SegmentIteratorRecord { + pub text: plural_rules::SegmentedText, + pub granularity: plural_rules::SegmenterGranularity, + pub next_record: usize, } impl IntlObject { - pub(crate) const fn kind(&self) -> IntlKind { + pub const fn kind(&self) -> IntlKind { match self { Self::Collator(_) => IntlKind::Collator, Self::NumberFormat(_) => IntlKind::NumberFormat, @@ -56,12 +99,18 @@ impl IntlObject { Self::Segmenter(_) => IntlKind::Segmenter, Self::DisplayNames(_) => IntlKind::DisplayNames, Self::Locale(_) => IntlKind::Locale, + Self::DurationFormat(_) => IntlKind::DurationFormat, + Self::Segments(_) => IntlKind::Segments, + Self::SegmentIterator(_) => IntlKind::SegmentIterator, } } } impl fmt::Debug for IntlObject { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { - formatter.debug_tuple("IntlObject").field(&self.kind()).finish() + formatter + .debug_tuple("IntlObject") + .field(&self.kind()) + .finish() } } diff --git a/crates/bamts-runtime/src/intl/number_data.rs b/crates/bamts-runtime/src/intl/number_data.rs new file mode 100644 index 00000000..30a22e5d --- /dev/null +++ b/crates/bamts-runtime/src/intl/number_data.rs @@ -0,0 +1,424 @@ +//! English number data: symbols, patterns, currencies, and units. + +use super::locale_data::{IntlData, numbering_system_digits}; +use super::number_format::{ + CompactDisplay, CompactPattern, CurrencyDisplay, CurrencySign, GroupingSizes, + NumberFormatDataProvider, NumberPatternRequest, NumberPatternSet, NumberStyle, NumberSymbols, + PluralCategory, PluralOperands, RangePatterns, UnitDisplay, +}; + +/// Currencies with other than two minor units, from ISO 4217. +const CURRENCY_MINOR_UNITS: &[(&str, u8)] = &[ + ("BHD", 3), + ("BIF", 0), + ("CLF", 4), + ("CLP", 0), + ("DJF", 0), + ("GNF", 0), + ("IQD", 3), + ("ISK", 0), + ("JOD", 3), + ("JPY", 0), + ("KMF", 0), + ("KRW", 0), + ("KWD", 3), + ("LYD", 3), + ("OMR", 3), + ("PYG", 0), + ("RWF", 0), + ("TND", 3), + ("UGX", 0), + ("UYI", 0), + ("UYW", 4), + ("VND", 0), + ("VUV", 0), + ("XAF", 0), + ("XOF", 0), + ("XPF", 0), +]; + +/// English currency symbols: code, symbol, narrow symbol. +const CURRENCY_SYMBOLS: &[(&str, &str, &str)] = &[ + ("AUD", "A$", "$"), + ("BRL", "R$", "R$"), + ("CAD", "CA$", "$"), + ("CNY", "CN\u{a5}", "\u{a5}"), + ("EUR", "\u{20ac}", "\u{20ac}"), + ("GBP", "\u{a3}", "\u{a3}"), + ("HKD", "HK$", "$"), + ("ILS", "\u{20aa}", "\u{20aa}"), + ("INR", "\u{20b9}", "\u{20b9}"), + ("JPY", "\u{a5}", "\u{a5}"), + ("KRW", "\u{20a9}", "\u{20a9}"), + ("MXN", "MX$", "$"), + ("NZD", "NZ$", "$"), + ("TWD", "NT$", "$"), + ("USD", "$", "$"), + ("VND", "\u{20ab}", "\u{20ab}"), + ("XAF", "FCFA", "FCFA"), + ("XCD", "EC$", "$"), + ("XOF", "F\u{202f}CFA", "F\u{202f}CFA"), + ("XPF", "CFPF", "CFPF"), +]; + +/// English currency names: code, singular, plural. +pub(super) const CURRENCY_NAMES: &[(&str, &str, &str)] = &[ + ("AUD", "Australian dollar", "Australian dollars"), + ("BRL", "Brazilian real", "Brazilian reals"), + ("CAD", "Canadian dollar", "Canadian dollars"), + ("CHF", "Swiss franc", "Swiss francs"), + ("CNY", "Chinese yuan", "Chinese yuan"), + ("EUR", "euro", "euros"), + ("GBP", "British pound", "British pounds"), + ("INR", "Indian rupee", "Indian rupees"), + ("JPY", "Japanese yen", "Japanese yen"), + ("KRW", "South Korean won", "South Korean won"), + ("MXN", "Mexican peso", "Mexican pesos"), + ("RUB", "Russian ruble", "Russian rubles"), + ("USD", "US dollar", "US dollars"), +]; + +/// English simple units: identifier, short, narrow, long singular, long plural. +pub(super) const UNITS: &[(&str, &str, &str, &str, &str)] = &[ + ("acre", "ac", "ac", "acre", "acres"), + ("bit", "bit", "bit", "bit", "bits"), + ("byte", "byte", "B", "byte", "bytes"), + ( + "celsius", + "\u{b0}C", + "\u{b0}C", + "degree Celsius", + "degrees Celsius", + ), + ("centimeter", "cm", "cm", "centimeter", "centimeters"), + ("day", "day", "d", "day", "days"), + ("degree", "deg", "\u{b0}", "degree", "degrees"), + ( + "fahrenheit", + "\u{b0}F", + "\u{b0}", + "degree Fahrenheit", + "degrees Fahrenheit", + ), + ( + "fluid-ounce", + "fl oz", + "fl oz", + "fluid ounce", + "fluid ounces", + ), + ("foot", "ft", "\u{2032}", "foot", "feet"), + ("gallon", "gal", "gal", "gallon", "gallons"), + ("gigabit", "Gb", "Gb", "gigabit", "gigabits"), + ("gigabyte", "GB", "GB", "gigabyte", "gigabytes"), + ("gram", "g", "g", "gram", "grams"), + ("hectare", "ha", "ha", "hectare", "hectares"), + ("hour", "hr", "h", "hour", "hours"), + ("inch", "in", "\u{2033}", "inch", "inches"), + ("kilobit", "kb", "kb", "kilobit", "kilobits"), + ("kilobyte", "kB", "kB", "kilobyte", "kilobytes"), + ("kilogram", "kg", "kg", "kilogram", "kilograms"), + ("kilometer", "km", "km", "kilometer", "kilometers"), + ("liter", "L", "L", "liter", "liters"), + ("megabit", "Mb", "Mb", "megabit", "megabits"), + ("megabyte", "MB", "MB", "megabyte", "megabytes"), + ("meter", "m", "m", "meter", "meters"), + ( + "microsecond", + "\u{3bc}s", + "\u{3bc}s", + "microsecond", + "microseconds", + ), + ("mile", "mi", "mi", "mile", "miles"), + ( + "mile-scandinavian", + "smi", + "smi", + "mile-scandinavian", + "miles-scandinavian", + ), + ("milliliter", "mL", "mL", "milliliter", "milliliters"), + ("millimeter", "mm", "mm", "millimeter", "millimeters"), + ("millisecond", "ms", "ms", "millisecond", "milliseconds"), + ("minute", "min", "m", "minute", "minutes"), + ("month", "mth", "m", "month", "months"), + ("nanosecond", "ns", "ns", "nanosecond", "nanoseconds"), + ("ounce", "oz", "oz", "ounce", "ounces"), + ("percent", "%", "%", "percent", "percent"), + ("petabyte", "PB", "PB", "petabyte", "petabytes"), + ("pound", "lb", "#", "pound", "pounds"), + ("second", "sec", "s", "second", "seconds"), + ("stone", "st", "st", "stone", "stones"), + ("terabit", "Tb", "Tb", "terabit", "terabits"), + ("terabyte", "TB", "TB", "terabyte", "terabytes"), + ("week", "wk", "w", "week", "weeks"), + ("yard", "yd", "yd", "yard", "yards"), + ("year", "yr", "y", "year", "years"), +]; + +/// Short forms whose plural is not the singular. +const SHORT_PLURALS: &[(&str, &str)] = &[ + ("day", "days"), + ("month", "mths"), + ("week", "wks"), + ("year", "yrs"), +]; + +/// Abbreviations of time units after a slash, where the short forms give way. +const PER_UNITS: &[(&str, &str)] = &[ + ("day", "d"), + ("hour", "h"), + ("month", "m"), + ("second", "s"), + ("week", "w"), + ("year", "y"), +]; + +/// Compound units English abbreviates as a whole. +const WHOLE_UNITS: &[(&str, &str)] = &[("mile-per-gallon", "mpg"), ("mile-per-hour", "mph")]; + +/// Units whose abbreviation attaches to the number without a space. +fn attaches(unit: &str, display: UnitDisplay) -> bool { + matches!(unit, "celsius" | "fahrenheit" | "percent") && display != UnitDisplay::Long + || display == UnitDisplay::Narrow +} + +fn simple_unit(unit: &str, display: UnitDisplay, category: PluralCategory) -> Option { + let index = UNITS.binary_search_by(|entry| entry.0.cmp(unit)).ok()?; + let (_, short, narrow, singular, plural) = UNITS[index]; + let one = category == PluralCategory::One; + Some( + match display { + UnitDisplay::Short if one => short, + UnitDisplay::Short => SHORT_PLURALS + .iter() + .find(|(name, _)| *name == unit) + .map_or(short, |(_, plural)| plural), + UnitDisplay::Narrow => narrow, + UnitDisplay::Long if one => singular, + UnitDisplay::Long => plural, + } + .to_owned(), + ) +} + +fn unit_text(unit: &str, display: UnitDisplay, category: PluralCategory) -> Option { + let Some((numerator, denominator)) = unit.split_once("-per-") else { + return simple_unit(unit, display, category); + }; + if display != UnitDisplay::Long + && let Some((_, whole)) = WHOLE_UNITS.iter().find(|(name, _)| *name == unit) + { + return Some((*whole).to_owned()); + } + let top = simple_unit(numerator, display, category)?; + let bottom = match PER_UNITS.iter().find(|(name, _)| *name == denominator) { + Some((_, per)) if display != UnitDisplay::Long => (*per).to_owned(), + _ => simple_unit(denominator, display, PluralCategory::One)?, + }; + Some(if display == UnitDisplay::Long { + format!("{top} per {bottom}") + } else { + format!("{top}/{bottom}") + }) +} + +fn signed(pattern: &str) -> NumberPatternSet { + NumberPatternSet { + zero: pattern.to_owned(), + positive: format!("{{plusSign}}{pattern}"), + negative: format!("{{minusSign}}{pattern}"), + } +} + +/// A prefix that ends in a letter is set off from the number. +fn currency_prefix_pattern(currency: &str, display: CurrencyDisplay) -> String { + let known = CURRENCY_SYMBOLS.iter().any(|entry| entry.0 == currency); + let letters = display == CurrencyDisplay::Code || !known; + if letters { + String::from("{currencyPrefix}\u{a0}{number}") + } else { + String::from("{currencyPrefix}{number}") + } +} + +impl NumberFormatDataProvider for IntlData { + fn number_symbols(&self, _data_locale: &str, numbering_system: &str) -> Option { + Some(NumberSymbols { + digits: numbering_system_digits(numbering_system)?, + decimal: String::from("."), + group: String::from(","), + plus_sign: String::from("+"), + minus_sign: String::from("-"), + percent_sign: String::from("%"), + infinity: String::from("\u{221e}"), + nan: String::from("NaN"), + exponent_separator: String::from("E"), + approximately_sign: String::from("~"), + }) + } + + fn grouping_sizes(&self, _data_locale: &str, _numbering_system: &str) -> Option { + Some(GroupingSizes { + primary: 3, + secondary: 3, + minimum_grouping_digits: 1, + }) + } + + fn number_patterns( + &self, + _data_locale: &str, + request: NumberPatternRequest<'_>, + ) -> Option { + Some(match request.style { + NumberStyle::Decimal => signed("{number}"), + NumberStyle::Percent => signed("{number}{percentSign}"), + NumberStyle::Currency => { + let display = request.currency_display?; + if display == CurrencyDisplay::Name { + return Some(signed("{number} {currencySuffix}")); + } + let pattern = currency_prefix_pattern(request.currency?, display); + let mut set = signed(&pattern); + if request.currency_sign == Some(CurrencySign::Accounting) { + set.negative = format!("({pattern})"); + } + set + } + NumberStyle::Unit => { + let unit = request.unit?; + let display = request.unit_display?; + signed(if attaches(unit, display) { + "{number}{unitSuffix}" + } else { + "{number} {unitSuffix}" + }) + } + }) + } + + fn scientific_pattern(&self, _data_locale: &str) -> Option { + Some(String::from( + "{number}{scientificSeparator}{scientificExponent}", + )) + } + + fn compact_exponent( + &self, + _data_locale: &str, + _display: CompactDisplay, + magnitude: i32, + ) -> Option { + Some(if magnitude < 3 { + 0 + } else { + (magnitude / 3).min(4) * 3 + }) + } + + fn compact_pattern( + &self, + _data_locale: &str, + display: CompactDisplay, + exponent: i32, + _category: PluralCategory, + ) -> Option { + let (short, long) = match exponent { + 3 => ("K", "thousand"), + 6 => ("M", "million"), + 9 => ("B", "billion"), + 12 => ("T", "trillion"), + _ => return None, + }; + Some(match display { + CompactDisplay::Short => CompactPattern { + pattern: String::from("{number}{compactSymbol}"), + display: short.to_owned(), + }, + CompactDisplay::Long => CompactPattern { + pattern: String::from("{number} {compactName}"), + display: long.to_owned(), + }, + }) + } + + fn currency_minor_units(&self, currency: &str) -> Option { + CURRENCY_MINOR_UNITS + .binary_search_by(|(code, _)| (*code).cmp(currency)) + .ok() + .map(|index| CURRENCY_MINOR_UNITS[index].1) + } + + fn currency_display( + &self, + _data_locale: &str, + currency: &str, + display: CurrencyDisplay, + category: PluralCategory, + ) -> Option { + match display { + CurrencyDisplay::Code => Some(currency.to_owned()), + CurrencyDisplay::Symbol | CurrencyDisplay::NarrowSymbol => { + let (_, symbol, narrow) = + CURRENCY_SYMBOLS.iter().find(|entry| entry.0 == currency)?; + Some( + if display == CurrencyDisplay::Symbol { + symbol + } else { + narrow + } + .to_string(), + ) + } + CurrencyDisplay::Name => { + let (_, singular, plural) = + CURRENCY_NAMES.iter().find(|entry| entry.0 == currency)?; + Some( + if category == PluralCategory::One { + singular + } else { + plural + } + .to_string(), + ) + } + } + } + + fn unit_display( + &self, + _data_locale: &str, + unit: &str, + display: UnitDisplay, + category: PluralCategory, + ) -> Option { + unit_text(unit, display, category) + } + + fn plural_category(&self, _data_locale: &str, operands: &PluralOperands) -> PluralCategory { + if operands.integer_digits == "1" && operands.visible_fraction_digits == 0 { + PluralCategory::One + } else { + PluralCategory::Other + } + } + + fn range_patterns(&self, _data_locale: &str, _numbering_system: &str) -> Option { + Some(RangePatterns { + separator: String::from("\u{2013}"), + approximate: String::from("{approximatelySign}{number}"), + }) + } +} + +/// The host locale hook of this runtime: no preference, so the provider's +/// default stands. +pub struct NoHostLocale; + +impl super::locale_negotiation::HostLocaleHook for NoHostLocale { + fn preferred_locales(&self) -> Vec { + Vec::new() + } +} diff --git a/crates/bamts-runtime/src/intl/number_format.rs b/crates/bamts-runtime/src/intl/number_format.rs index 96e48540..3ab64eea 100644 --- a/crates/bamts-runtime/src/intl/number_format.rs +++ b/crates/bamts-runtime/src/intl/number_format.rs @@ -16,7 +16,7 @@ use std::error::Error; use std::fmt; use super::locale_negotiation::{ - default_locale, resolve_locale, HostLocaleHook, LocaleDataProvider, LocaleError, LocaleMatcher, + HostLocaleHook, LocaleDataProvider, LocaleError, LocaleMatcher, default_locale, resolve_locale, }; /// The failure category visible to a runtime adapter. @@ -43,19 +43,31 @@ pub struct NumberFormatError { impl NumberFormatError { fn range(message: impl Into) -> Self { - Self { kind: NumberFormatErrorKind::RangeError, message: message.into() } + Self { + kind: NumberFormatErrorKind::RangeError, + message: message.into(), + } } fn type_error(message: impl Into) -> Self { - Self { kind: NumberFormatErrorKind::TypeError, message: message.into() } + Self { + kind: NumberFormatErrorKind::TypeError, + message: message.into(), + } } fn missing(message: impl Into) -> Self { - Self { kind: NumberFormatErrorKind::MissingData, message: message.into() } + Self { + kind: NumberFormatErrorKind::MissingData, + message: message.into(), + } } fn invalid_data(message: impl Into) -> Self { - Self { kind: NumberFormatErrorKind::InvalidData, message: message.into() } + Self { + kind: NumberFormatErrorKind::InvalidData, + message: message.into(), + } } } @@ -245,10 +257,18 @@ impl RoundingMode { const fn unsigned(self, negative: bool) -> UnsignedRoundingMode { match (self, negative) { - (Self::Ceil, false) | (Self::Floor, true) | (Self::Expand, _) => UnsignedRoundingMode::Infinity, - (Self::Ceil, true) | (Self::Floor, false) | (Self::Trunc, _) => UnsignedRoundingMode::Zero, - (Self::HalfCeil, false) | (Self::HalfFloor, true) | (Self::HalfExpand, _) => UnsignedRoundingMode::HalfInfinity, - (Self::HalfCeil, true) | (Self::HalfFloor, false) | (Self::HalfTrunc, _) => UnsignedRoundingMode::HalfZero, + (Self::Ceil, false) | (Self::Floor, true) | (Self::Expand, _) => { + UnsignedRoundingMode::Infinity + } + (Self::Ceil, true) | (Self::Floor, false) | (Self::Trunc, _) => { + UnsignedRoundingMode::Zero + } + (Self::HalfCeil, false) | (Self::HalfFloor, true) | (Self::HalfExpand, _) => { + UnsignedRoundingMode::HalfInfinity + } + (Self::HalfCeil, true) | (Self::HalfFloor, false) | (Self::HalfTrunc, _) => { + UnsignedRoundingMode::HalfZero + } (Self::HalfEven, _) => UnsignedRoundingMode::HalfEven, } } @@ -436,10 +456,19 @@ pub struct PluralOperands { pub trait NumberFormatDataProvider: LocaleDataProvider { fn number_symbols(&self, data_locale: &str, numbering_system: &str) -> Option; fn grouping_sizes(&self, data_locale: &str, numbering_system: &str) -> Option; - fn number_patterns(&self, data_locale: &str, request: NumberPatternRequest<'_>) -> Option; + fn number_patterns( + &self, + data_locale: &str, + request: NumberPatternRequest<'_>, + ) -> Option; fn scientific_pattern(&self, data_locale: &str) -> Option; /// ILD `ComputeExponentForMagnitude` result for compact notation. - fn compact_exponent(&self, data_locale: &str, display: CompactDisplay, magnitude: i32) -> Option; + fn compact_exponent( + &self, + data_locale: &str, + display: CompactDisplay, + magnitude: i32, + ) -> Option; fn compact_pattern( &self, data_locale: &str, @@ -581,7 +610,7 @@ pub struct ResolvedNumberFormatOptions { /// Exact finite decimal magnitude, normalized without leading or trailing zeroes. #[derive(Clone, Debug, Eq, PartialEq)] -struct Decimal { +pub struct Decimal { digits: Vec, /// Value is `digits × 10^exponent`. exponent: i32, @@ -589,7 +618,10 @@ struct Decimal { impl Decimal { fn zero() -> Self { - Self { digits: vec![0], exponent: 0 } + Self { + digits: vec![0], + exponent: 0, + } } fn new(mut digits: Vec, mut exponent: i32) -> Result { @@ -635,7 +667,10 @@ impl Decimal { .exponent .checked_add(places) .ok_or_else(|| NumberFormatError::range("decimal exponent is out of range"))?; - Ok(Self { digits: self.digits.clone(), exponent }) + Ok(Self { + digits: self.digits.clone(), + exponent, + }) } } @@ -667,19 +702,28 @@ impl IntlMathematicalValue { } let negative = value.is_sign_negative(); let text = value.abs().to_string(); - let magnitude = parse_decimal_magnitude(&text).expect("finite f64 renders as a decimal literal"); - Self::Finite { negative, magnitude } + let magnitude = + parse_decimal_magnitude(&text).expect("finite f64 renders as a decimal literal"); + Self::Finite { + negative, + magnitude, + } } /// Parses an exact BigInt decimal representation. pub fn from_bigint(value: &str) -> Result { let (negative, body) = split_sign(value); if body.is_empty() || !body.bytes().all(|byte| byte.is_ascii_digit()) { - return Err(NumberFormatError::range("invalid BigInt decimal representation")); + return Err(NumberFormatError::range( + "invalid BigInt decimal representation", + )); } let digits = body.bytes().map(|byte| byte - b'0').collect(); let magnitude = Decimal::new(digits, 0)?; - Ok(Self::Finite { negative: negative && !magnitude.is_zero(), magnitude }) + Ok(Self::Finite { + negative: negative && !magnitude.is_zero(), + magnitude, + }) } /// Parses an exact decimal literal for a future JavaScript String adapter. @@ -689,12 +733,18 @@ impl IntlMathematicalValue { if negative && magnitude.is_zero() { return Ok(Self::NegativeZero); } - Ok(Self::Finite { negative, magnitude }) + Ok(Self::Finite { + negative, + magnitude, + }) } fn shifted(&self, places: i32) -> Result { match self { - Self::Finite { negative, magnitude } => Ok(Self::Finite { + Self::Finite { + negative, + magnitude, + } => Ok(Self::Finite { negative: *negative, magnitude: magnitude.shifted(places)?, }), @@ -793,7 +843,10 @@ struct QuantumRound { } fn normalize_integer(mut digits: Vec) -> Vec { - let first = digits.iter().position(|digit| *digit != 0).unwrap_or(digits.len()); + let first = digits + .iter() + .position(|digit| *digit != 0) + .unwrap_or(digits.len()); if first == digits.len() { return vec![0]; } @@ -802,7 +855,9 @@ fn normalize_integer(mut digits: Vec) -> Vec { } fn mod_small(digits: &[u8], divisor: u32) -> u32 { - digits.iter().fold(0, |remainder, digit| (remainder * 10 + u32::from(*digit)) % divisor) + digits.iter().fold(0, |remainder, digit| { + (remainder * 10 + u32::from(*digit)) % divisor + }) } fn add_small(mut digits: Vec, mut addend: u32) -> Vec { @@ -960,18 +1015,26 @@ fn round_to_quantum( upper } } - UnsignedRoundingMode::Zero | UnsignedRoundingMode::Infinity => unreachable!(), + UnsignedRoundingMode::Zero | UnsignedRoundingMode::Infinity => { + unreachable!() + } }, } } } }; let rounded = Decimal::new(chosen.clone(), quantum_exponent)?; - Ok(QuantumRound { integer: chosen, rounded }) + Ok(QuantumRound { + integer: chosen, + rounded, + }) } fn digits_to_ascii(digits: &[u8]) -> String { - digits.iter().map(|digit| char::from(b'0' + *digit)).collect() + digits + .iter() + .map(|digit| char::from(b'0' + *digit)) + .collect() } fn trim_fraction(mut formatted: String, maximum: u8, minimum: u8) -> String { @@ -1029,7 +1092,7 @@ fn to_raw_precision( mode: UnsignedRoundingMode, ) -> Result { let precision = usize::from(maximum_precision); - let (mut coefficient, mut exponent, rounded_value) = if value.is_zero() { + let (mut coefficient, exponent, rounded_value) = if value.is_zero() { (vec![0; precision], 0, Decimal::zero()) } else { let mut exponent = value.magnitude()?; @@ -1041,7 +1104,9 @@ fn to_raw_precision( let mut coefficient = rounded.integer; while coefficient.len() > precision { if coefficient.last() != Some(&0) { - return Err(NumberFormatError::invalid_data("precision carry is not a power of ten")); + return Err(NumberFormatError::invalid_data( + "precision carry is not a power of ten", + )); } coefficient.pop(); exponent = exponent @@ -1055,7 +1120,11 @@ fn to_raw_precision( .checked_sub(i32::from(maximum_precision)) .and_then(|result| result.checked_add(1)) .ok_or_else(|| NumberFormatError::range("rounding magnitude is out of range"))?; - (coefficient.clone(), exponent, Decimal::new(coefficient, final_quantum)?) + ( + coefficient.clone(), + exponent, + Decimal::new(coefficient, final_quantum)?, + ) }; let mut formatted = digits_to_ascii(&coefficient); @@ -1139,12 +1208,18 @@ impl<'a> NumberFormat<'a> { let locale_matcher = match options.locale_matcher.as_deref().unwrap_or("best fit") { "lookup" => LocaleMatcher::Lookup, "best fit" => LocaleMatcher::BestFit, - value => return Err(NumberFormatError::range(format!("invalid localeMatcher: {value}"))), + value => { + return Err(NumberFormatError::range(format!( + "invalid localeMatcher: {value}" + ))); + } }; let numbering_option = match &options.numbering_system { Some(value) => { if !is_unicode_type(value) { - return Err(NumberFormatError::range(format!("invalid numberingSystem: {value}"))); + return Err(NumberFormatError::range(format!( + "invalid numberingSystem: {value}" + ))); } Some(value.to_ascii_lowercase()) } @@ -1175,32 +1250,43 @@ impl<'a> NumberFormat<'a> { } let style = parse_style(options.style.as_deref())?; - let currency = options.currency.as_deref().map(validate_currency).transpose()?; + let currency = options + .currency + .as_deref() + .map(validate_currency) + .transpose()?; if style == NumberStyle::Currency && currency.is_none() { - return Err(NumberFormatError::type_error("currency is required when style is currency")); + return Err(NumberFormatError::type_error( + "currency is required when style is currency", + )); } let currency_display = parse_currency_display(options.currency_display.as_deref())?; let currency_sign = parse_currency_sign(options.currency_sign.as_deref())?; let unit = options.unit.as_deref().map(validate_unit).transpose()?; if style == NumberStyle::Unit && unit.is_none() { - return Err(NumberFormatError::type_error("unit is required when style is unit")); + return Err(NumberFormatError::type_error( + "unit is required when style is unit", + )); } let unit_display = parse_unit_display(options.unit_display.as_deref())?; let notation = parse_notation(options.notation.as_deref())?; - let (minimum_fraction_default, maximum_fraction_default) = if style == NumberStyle::Currency - && notation == Notation::Standard - { - let digits = provider.currency_minor_units(currency.as_deref().expect("currency style validated")).unwrap_or(2); - if digits > 100 { - return Err(NumberFormatError::invalid_data("currency minor-unit digits exceed 100")); - } - (digits, digits) - } else if style == NumberStyle::Percent { - (0, 0) - } else { - (0, 3) - }; + let (minimum_fraction_default, maximum_fraction_default) = + if style == NumberStyle::Currency && notation == Notation::Standard { + let digits = provider + .currency_minor_units(currency.as_deref().expect("currency style validated")) + .unwrap_or(2); + if digits > 100 { + return Err(NumberFormatError::invalid_data( + "currency minor-unit digits exceed 100", + )); + } + (digits, digits) + } else if style == NumberStyle::Percent { + (0, 0) + } else { + (0, 3) + }; let digits = resolve_digit_options( options, minimum_fraction_default, @@ -1209,24 +1295,35 @@ impl<'a> NumberFormat<'a> { )?; let compact_display_value = parse_compact_display(options.compact_display.as_deref())?; let compact_display = (notation == Notation::Compact).then_some(compact_display_value); - let default_grouping = if notation == Notation::Compact { UseGrouping::Min2 } else { UseGrouping::Auto }; + let default_grouping = if notation == Notation::Compact { + UseGrouping::Min2 + } else { + UseGrouping::Auto + }; let use_grouping = resolve_grouping(options.use_grouping.as_ref(), default_grouping)?; let sign_display = parse_sign_display(options.sign_display.as_deref())?; let symbols = provider .number_symbols(&resolved.data_locale, &numbering_system) - .ok_or_else(|| NumberFormatError::missing(format!( - "missing symbols for {}/{}", - resolved.data_locale, numbering_system - )))?; + .ok_or_else(|| { + NumberFormatError::missing(format!( + "missing symbols for {}/{}", + resolved.data_locale, numbering_system + )) + })?; let grouping = provider .grouping_sizes(&resolved.data_locale, &numbering_system) - .ok_or_else(|| NumberFormatError::missing(format!( - "missing grouping data for {}/{}", - resolved.data_locale, numbering_system - )))?; - if grouping.primary == 0 || grouping.secondary == 0 || grouping.minimum_grouping_digits == 0 { - return Err(NumberFormatError::invalid_data("grouping sizes must be non-zero")); + .ok_or_else(|| { + NumberFormatError::missing(format!( + "missing grouping data for {}/{}", + resolved.data_locale, numbering_system + )) + })?; + if grouping.primary == 0 || grouping.secondary == 0 || grouping.minimum_grouping_digits == 0 + { + return Err(NumberFormatError::invalid_data( + "grouping sizes must be non-zero", + )); } let request = NumberPatternRequest { style, @@ -1240,7 +1337,8 @@ impl<'a> NumberFormat<'a> { .number_patterns(&resolved.data_locale, request) .ok_or_else(|| NumberFormatError::missing("missing number pattern"))?; validate_number_patterns(&patterns)?; - let scientific_pattern = if matches!(notation, Notation::Scientific | Notation::Engineering) { + let scientific_pattern = if matches!(notation, Notation::Scientific | Notation::Engineering) + { let pattern = provider .scientific_pattern(&resolved.data_locale) .ok_or_else(|| NumberFormatError::missing("missing scientific notation pattern"))?; @@ -1260,10 +1358,10 @@ impl<'a> NumberFormat<'a> { data_locale: resolved.data_locale, numbering_system, style, - currency: (style == NumberStyle::Currency).then_some(currency.expect("currency style validated")), + currency: currency.filter(|_| style == NumberStyle::Currency), currency_display: (style == NumberStyle::Currency).then_some(currency_display), currency_sign: (style == NumberStyle::Currency).then_some(currency_sign), - unit: (style == NumberStyle::Unit).then_some(unit.expect("unit style validated")), + unit: unit.filter(|_| style == NumberStyle::Unit), unit_display: (style == NumberStyle::Unit).then_some(unit_display), minimum_integer_digits: digits.minimum_integer, minimum_fraction_digits: digits.minimum_fraction, @@ -1336,7 +1434,10 @@ impl<'a> NumberFormat<'a> { } /// Partitions an exact BigInt decimal representation. - pub fn format_bigint_to_parts(&self, value: &str) -> Result, NumberFormatError> { + pub fn format_bigint_to_parts( + &self, + value: &str, + ) -> Result, NumberFormatError> { self.format_to_parts(&IntlMathematicalValue::from_bigint(value)?) } @@ -1362,41 +1463,76 @@ impl<'a> NumberFormat<'a> { .collect()) } - /// ECMA-402 `PartitionNumberRangePattern` with the conforming trivial - /// `CollapseNumberRange` implementation explicitly permitted by the spec. + /// ECMA-402 `PartitionNumberRangePattern`. `CollapseNumberRange` removes an + /// affix the two ends share when it is longer than one code point, so + /// `+$2.90–3.10` shares its `+$` and `$3 – $5` repeats its `$`. Any affix + /// left on an inner side spaces out the separator. pub fn format_range_to_parts( &self, start: &IntlMathematicalValue, end: &IntlMathematicalValue, ) -> Result, NumberFormatError> { if start.is_nan() || end.is_nan() { - return Err(NumberFormatError::range("number range endpoints must not be NaN")); + return Err(NumberFormatError::range( + "number range endpoints must not be NaN", + )); } let start_parts = self.format_to_parts(start)?; let end_parts = self.format_to_parts(end)?; if concatenate_parts(&start_parts) == concatenate_parts(&end_parts) { return self.approximate_range(start_parts); } + let (start_prefix, start_number, start_suffix) = split_number(&start_parts); + let (end_prefix, end_number, end_suffix) = split_number(&end_parts); + let shared_prefix = collapsible(start_prefix, end_prefix); + let shared_suffix = collapsible(start_suffix, end_suffix); + let spaced = (!shared_prefix && !(start_prefix.is_empty() && end_prefix.is_empty())) + || (!shared_suffix && !(start_suffix.is_empty() && end_suffix.is_empty())); + let separator = if spaced { + format!(" {} ", self.range_patterns.separator) + } else { + self.range_patterns.separator.clone() + }; let mut result = Vec::with_capacity(start_parts.len() + end_parts.len() + 1); - result.extend(start_parts.into_iter().map(|part| NumberRangePart { - part_type: part.part_type, - value: part.value, - source: RangeSource::StartRange, - })); - result.push(NumberRangePart { - part_type: NumberPartType::Literal, - value: self.range_patterns.separator.clone(), - source: RangeSource::Shared, - }); - result.extend(end_parts.into_iter().map(|part| NumberRangePart { - part_type: part.part_type, - value: part.value, - source: RangeSource::EndRange, - })); + let mut push = |parts: &[NumberPart], source| { + result.extend(parts.iter().map(|part| NumberRangePart { + part_type: part.part_type, + value: part.value.clone(), + source, + })); + }; + if shared_prefix { + push(start_prefix, RangeSource::Shared); + } else { + push(start_prefix, RangeSource::StartRange); + } + push(start_number, RangeSource::StartRange); + if !shared_suffix { + push(start_suffix, RangeSource::StartRange); + } + push( + &[NumberPart { + part_type: NumberPartType::Literal, + value: separator, + }], + RangeSource::Shared, + ); + if !shared_prefix { + push(end_prefix, RangeSource::EndRange); + } + push(end_number, RangeSource::EndRange); + if shared_suffix { + push(start_suffix, RangeSource::Shared); + } else { + push(end_suffix, RangeSource::EndRange); + } Ok(result) } - fn prepare_format(&self, value: &IntlMathematicalValue) -> Result { + fn prepare_format( + &self, + value: &IntlMathematicalValue, + ) -> Result { match value { IntlMathematicalValue::Nan => Ok(PreparedFormat { rounded: IntlMathematicalValue::Nan, @@ -1422,7 +1558,11 @@ impl<'a> NumberFormat<'a> { let exponent = self.compute_exponent(&scaled_percent)?; let scaled = scaled_percent.shifted(-exponent)?; let (rounded, ascii) = self.format_numeric_to_string(&scaled)?; - Ok(PreparedFormat { rounded, ascii, exponent }) + Ok(PreparedFormat { + rounded, + ascii, + exponent, + }) } } } @@ -1433,43 +1573,57 @@ impl<'a> NumberFormat<'a> { ) -> Result<(IntlMathematicalValue, String), NumberFormatError> { let (negative, magnitude) = match value { IntlMathematicalValue::NegativeZero => (true, Decimal::zero()), - IntlMathematicalValue::Finite { negative, magnitude } => (*negative, magnitude.clone()), + IntlMathematicalValue::Finite { + negative, + magnitude, + } => (*negative, magnitude.clone()), IntlMathematicalValue::PositiveInfinity | IntlMathematicalValue::NegativeInfinity | IntlMathematicalValue::Nan => { - return Err(NumberFormatError::invalid_data("non-finite value reached rounding")); + return Err(NumberFormatError::invalid_data( + "non-finite value reached rounding", + )); } }; let mode = self.rounding_mode.unsigned(negative); let result = match self.rounding_type { RoundingType::SignificantDigits => to_raw_precision( &magnitude, - self.minimum_significant_digits.expect("significant rounding has minimum"), - self.maximum_significant_digits.expect("significant rounding has maximum"), + self.minimum_significant_digits + .expect("significant rounding has minimum"), + self.maximum_significant_digits + .expect("significant rounding has maximum"), mode, )?, RoundingType::FractionDigits => to_raw_fixed( &magnitude, - self.minimum_fraction_digits.expect("fraction rounding has minimum"), - self.maximum_fraction_digits.expect("fraction rounding has maximum"), + self.minimum_fraction_digits + .expect("fraction rounding has minimum"), + self.maximum_fraction_digits + .expect("fraction rounding has maximum"), self.rounding_increment, mode, )?, RoundingType::MorePrecision | RoundingType::LessPrecision => { let significant = to_raw_precision( &magnitude, - self.minimum_significant_digits.expect("priority rounding has significant minimum"), - self.maximum_significant_digits.expect("priority rounding has significant maximum"), + self.minimum_significant_digits + .expect("priority rounding has significant minimum"), + self.maximum_significant_digits + .expect("priority rounding has significant maximum"), mode, )?; let fraction = to_raw_fixed( &magnitude, - self.minimum_fraction_digits.expect("priority rounding has fraction minimum"), - self.maximum_fraction_digits.expect("priority rounding has fraction maximum"), + self.minimum_fraction_digits + .expect("priority rounding has fraction minimum"), + self.maximum_fraction_digits + .expect("priority rounding has fraction maximum"), self.rounding_increment, mode, )?; - let fixed_is_more_precise = fraction.rounding_magnitude < significant.rounding_magnitude; + let fixed_is_more_precise = + fraction.rounding_magnitude < significant.rounding_magnitude; if (self.rounding_type == RoundingType::MorePrecision && fixed_is_more_precise) || (self.rounding_type == RoundingType::LessPrecision && !fixed_is_more_precise) { @@ -1480,22 +1634,32 @@ impl<'a> NumberFormat<'a> { } }; let mut ascii = result.formatted; - if self.trailing_zero_display == TrailingZeroDisplay::StripIfInteger && result.rounded.is_integer() { - if let Some(index) = ascii.find('.') { - ascii.truncate(index); - } + if self.trailing_zero_display == TrailingZeroDisplay::StripIfInteger + && result.rounded.is_integer() + && let Some(index) = ascii.find('.') + { + ascii.truncate(index); } if result.integer_digits_count < usize::from(self.minimum_integer_digits) { - ascii.insert_str(0, &"0".repeat(usize::from(self.minimum_integer_digits) - result.integer_digits_count)); + ascii.insert_str( + 0, + &"0".repeat(usize::from(self.minimum_integer_digits) - result.integer_digits_count), + ); } let rounded = if negative { if result.rounded.is_zero() { IntlMathematicalValue::NegativeZero } else { - IntlMathematicalValue::Finite { negative: true, magnitude: result.rounded } + IntlMathematicalValue::Finite { + negative: true, + magnitude: result.rounded, + } } } else { - IntlMathematicalValue::Finite { negative: false, magnitude: result.rounded } + IntlMathematicalValue::Finite { + negative: false, + magnitude: result.rounded, + } }; Ok((rounded, ascii)) } @@ -1515,7 +1679,9 @@ impl<'a> NumberFormat<'a> { }; let (rounded, _) = self.format_numeric_to_string(&scaled)?; let new_magnitude = match rounded { - IntlMathematicalValue::Finite { magnitude, .. } if magnitude.is_zero() => return Ok(exponent), + IntlMathematicalValue::Finite { magnitude, .. } if magnitude.is_zero() => { + return Ok(exponent); + } IntlMathematicalValue::Finite { magnitude, .. } => magnitude.magnitude()?, IntlMathematicalValue::NegativeZero => return Ok(exponent), _ => return Err(NumberFormatError::invalid_data("non-finite exponent probe")), @@ -1543,13 +1709,18 @@ impl<'a> NumberFormat<'a> { self.compact_display.expect("compact display is resolved"), magnitude, ) - .ok_or_else(|| NumberFormatError::missing(format!( - "missing compact exponent for magnitude {magnitude}" - ))), + .ok_or_else(|| { + NumberFormatError::missing(format!( + "missing compact exponent for magnitude {magnitude}" + )) + }), } } - fn partition_outer_pattern(&self, prepared: &PreparedFormat) -> Result, NumberFormatError> { + fn partition_outer_pattern( + &self, + prepared: &PreparedFormat, + ) -> Result, NumberFormatError> { let pattern = self.pattern_for_value(&prepared.rounded); let operands = operands_from_ascii(&prepared.ascii, prepared.exponent); let category = match prepared.rounded { @@ -1562,13 +1733,27 @@ impl<'a> NumberFormat<'a> { let mut result = Vec::new(); for token in tokens { match token { - PatternToken::Literal(value) => push_part(&mut result, NumberPartType::Literal, value), + PatternToken::Literal(value) => { + push_part(&mut result, NumberPartType::Literal, value) + } PatternToken::Placeholder(name) => match name.as_str() { "number" => result.extend(self.partition_notation(prepared, category)?), - "plusSign" => push_part(&mut result, NumberPartType::PlusSign, self.symbols.plus_sign.clone()), - "minusSign" => push_part(&mut result, NumberPartType::MinusSign, self.symbols.minus_sign.clone()), + "plusSign" => push_part( + &mut result, + NumberPartType::PlusSign, + self.symbols.plus_sign.clone(), + ), + "minusSign" => push_part( + &mut result, + NumberPartType::MinusSign, + self.symbols.minus_sign.clone(), + ), "percentSign" if self.style == NumberStyle::Percent => { - push_part(&mut result, NumberPartType::PercentSign, self.symbols.percent_sign.clone()); + push_part( + &mut result, + NumberPartType::PercentSign, + self.symbols.percent_sign.clone(), + ); } "currencyCode" if self.style == NumberStyle::Currency => { push_part( @@ -1596,9 +1781,11 @@ impl<'a> NumberFormat<'a> { self.unit_display.expect("unit style"), category, ) - .ok_or_else(|| NumberFormatError::missing(format!( - "missing display name for unit {unit}" - )))?; + .ok_or_else(|| { + NumberFormatError::missing(format!( + "missing display name for unit {unit}" + )) + })?; push_part(&mut result, NumberPartType::Unit, value); } _ => push_part(&mut result, NumberPartType::Unknown, name), @@ -1615,17 +1802,26 @@ impl<'a> NumberFormat<'a> { ) -> Result, NumberFormatError> { match prepared.rounded { IntlMathematicalValue::Nan => { - return Ok(vec![NumberPart { part_type: NumberPartType::Nan, value: prepared.ascii.clone() }]); + return Ok(vec![NumberPart { + part_type: NumberPartType::Nan, + value: prepared.ascii.clone(), + }]); } IntlMathematicalValue::PositiveInfinity | IntlMathematicalValue::NegativeInfinity => { - return Ok(vec![NumberPart { part_type: NumberPartType::Infinity, value: prepared.ascii.clone() }]); + return Ok(vec![NumberPart { + part_type: NumberPartType::Infinity, + value: prepared.ascii.clone(), + }]); } _ => {} } let (pattern, compact_display) = match self.notation { Notation::Standard => (String::from("{number}"), None), Notation::Scientific | Notation::Engineering => ( - self.scientific_pattern.as_ref().expect("scientific pattern resolved").clone(), + self.scientific_pattern + .as_ref() + .expect("scientific pattern resolved") + .clone(), None, ), Notation::Compact if prepared.exponent == 0 => (String::from("{number}"), None), @@ -1638,10 +1834,12 @@ impl<'a> NumberFormat<'a> { prepared.exponent, category, ) - .ok_or_else(|| NumberFormatError::missing(format!( - "missing compact pattern for exponent {}", - prepared.exponent - )))?; + .ok_or_else(|| { + NumberFormatError::missing(format!( + "missing compact pattern for exponent {}", + prepared.exponent + )) + })?; validate_pattern(&compact.pattern)?; (compact.pattern, Some(compact.display)) } @@ -1650,14 +1848,18 @@ impl<'a> NumberFormat<'a> { let mut result = Vec::new(); for token in tokens { match token { - PatternToken::Literal(value) => push_part(&mut result, NumberPartType::Literal, value), + PatternToken::Literal(value) => { + push_part(&mut result, NumberPartType::Literal, value) + } PatternToken::Placeholder(name) => match name.as_str() { "number" => result.extend(self.partition_ascii_number(&prepared.ascii)), "compactSymbol" | "compactName" => push_part( &mut result, NumberPartType::Compact, compact_display.clone().ok_or_else(|| { - NumberFormatError::invalid_data("compact placeholder without compact display") + NumberFormatError::invalid_data( + "compact placeholder without compact display", + ) })?, ), "scientificSeparator" => push_part( @@ -1690,18 +1892,36 @@ impl<'a> NumberFormat<'a> { } fn partition_ascii_number(&self, ascii: &str) -> Vec { - let (integer, fraction) = ascii.split_once('.').map_or((ascii, None), |(left, right)| (left, Some(right))); + let (integer, fraction) = ascii + .split_once('.') + .map_or((ascii, None), |(left, right)| (left, Some(right))); let groups = grouped_slices(integer, self.use_grouping, self.grouping); let mut result = Vec::with_capacity(groups.len() * 2 + usize::from(fraction.is_some())); for (index, group) in groups.iter().enumerate() { - push_part(&mut result, NumberPartType::Integer, self.transliterate_digits(group)); + push_part( + &mut result, + NumberPartType::Integer, + self.transliterate_digits(group), + ); if index + 1 < groups.len() { - push_part(&mut result, NumberPartType::Group, self.symbols.group.clone()); + push_part( + &mut result, + NumberPartType::Group, + self.symbols.group.clone(), + ); } } if let Some(fraction) = fraction { - push_part(&mut result, NumberPartType::Decimal, self.symbols.decimal.clone()); - push_part(&mut result, NumberPartType::Fraction, self.transliterate_digits(fraction)); + push_part( + &mut result, + NumberPartType::Decimal, + self.symbols.decimal.clone(), + ); + push_part( + &mut result, + NumberPartType::Fraction, + self.transliterate_digits(fraction), + ); } result } @@ -1712,7 +1932,11 @@ impl<'a> NumberFormat<'a> { .map(|character| { character .to_digit(10) - .and_then(|digit| self.symbols.digits.get(usize::try_from(digit).expect("digit index"))) + .and_then(|digit| { + self.symbols + .digits + .get(usize::try_from(digit).expect("digit index")) + }) .copied() .unwrap_or(character) }) @@ -1724,23 +1948,23 @@ impl<'a> NumberFormat<'a> { match self.sign_display { SignDisplay::Never => &self.patterns.zero, SignDisplay::Auto => match category { - ValueCategory::PositiveNonZero | ValueCategory::PositiveZero => &self.patterns.zero, - ValueCategory::NegativeNonZero | ValueCategory::NegativeZero => &self.patterns.negative, + ValueCategory::Positive | ValueCategory::ZeroPositive => &self.patterns.zero, + ValueCategory::Negative | ValueCategory::ZeroNegative => &self.patterns.negative, }, SignDisplay::Always => match category { - ValueCategory::PositiveNonZero | ValueCategory::PositiveZero => &self.patterns.positive, - ValueCategory::NegativeNonZero | ValueCategory::NegativeZero => &self.patterns.negative, + ValueCategory::Positive | ValueCategory::ZeroPositive => &self.patterns.positive, + ValueCategory::Negative | ValueCategory::ZeroNegative => &self.patterns.negative, }, SignDisplay::ExceptZero => match category { - ValueCategory::PositiveZero | ValueCategory::NegativeZero => &self.patterns.zero, - ValueCategory::PositiveNonZero => &self.patterns.positive, - ValueCategory::NegativeNonZero => &self.patterns.negative, + ValueCategory::ZeroPositive | ValueCategory::ZeroNegative => &self.patterns.zero, + ValueCategory::Positive => &self.patterns.positive, + ValueCategory::Negative => &self.patterns.negative, }, SignDisplay::Negative => match category { - ValueCategory::NegativeNonZero => &self.patterns.negative, - ValueCategory::PositiveNonZero | ValueCategory::PositiveZero | ValueCategory::NegativeZero => { - &self.patterns.zero - } + ValueCategory::Negative => &self.patterns.negative, + ValueCategory::Positive + | ValueCategory::ZeroPositive + | ValueCategory::ZeroNegative => &self.patterns.zero, }, } } @@ -1794,23 +2018,26 @@ struct PreparedFormat { #[derive(Clone, Copy, Debug, Eq, PartialEq)] enum ValueCategory { - PositiveNonZero, - PositiveZero, - NegativeNonZero, - NegativeZero, + Positive, + ZeroPositive, + Negative, + ZeroNegative, } fn value_category(value: &IntlMathematicalValue) -> ValueCategory { match value { - IntlMathematicalValue::Nan => ValueCategory::PositiveZero, - IntlMathematicalValue::PositiveInfinity => ValueCategory::PositiveNonZero, - IntlMathematicalValue::NegativeInfinity => ValueCategory::NegativeNonZero, - IntlMathematicalValue::NegativeZero => ValueCategory::NegativeZero, - IntlMathematicalValue::Finite { negative: true, magnitude } if !magnitude.is_zero() => { - ValueCategory::NegativeNonZero + IntlMathematicalValue::Nan => ValueCategory::ZeroPositive, + IntlMathematicalValue::PositiveInfinity => ValueCategory::Positive, + IntlMathematicalValue::NegativeInfinity => ValueCategory::Negative, + IntlMathematicalValue::NegativeZero => ValueCategory::ZeroNegative, + IntlMathematicalValue::Finite { + negative: true, + magnitude, + } if !magnitude.is_zero() => ValueCategory::Negative, + IntlMathematicalValue::Finite { magnitude, .. } if magnitude.is_zero() => { + ValueCategory::ZeroPositive } - IntlMathematicalValue::Finite { magnitude, .. } if magnitude.is_zero() => ValueCategory::PositiveZero, - IntlMathematicalValue::Finite { .. } => ValueCategory::PositiveNonZero, + IntlMathematicalValue::Finite { .. } => ValueCategory::Positive, } } @@ -1820,6 +2047,39 @@ fn push_part(parts: &mut Vec, part_type: NumberPartType, value: Stri } } +/// Splits parts into what precedes the number, the number, and what follows. +fn split_number(parts: &[NumberPart]) -> (&[NumberPart], &[NumberPart], &[NumberPart]) { + let is_number = |part: &NumberPart| { + matches!( + part.part_type, + NumberPartType::Integer + | NumberPartType::Group + | NumberPartType::Decimal + | NumberPartType::Fraction + | NumberPartType::Nan + | NumberPartType::Infinity + | NumberPartType::Compact + | NumberPartType::ExponentInteger + | NumberPartType::ExponentMinusSign + | NumberPartType::ExponentSeparator + ) + }; + let Some(first) = parts.iter().position(is_number) else { + return (parts, &[], &[]); + }; + let last = parts.iter().rposition(is_number).unwrap_or(first) + 1; + (&parts[..first], &parts[first..last], &parts[last..]) +} + +fn collapsible(start: &[NumberPart], end: &[NumberPart]) -> bool { + start == end + && start + .iter() + .map(|part| part.value.chars().count()) + .sum::() + > 1 +} + fn concatenate_parts(parts: &[NumberPart]) -> String { parts.iter().map(|part| part.value.as_str()).collect() } @@ -1850,13 +2110,18 @@ fn grouped_slices(integer: &str, use_grouping: UseGrouping, grouping: GroupingSi fn operands_from_ascii(ascii: &str, exponent: i32) -> PluralOperands { let (integer, fraction) = ascii.split_once('.').unwrap_or((ascii, "")); let integer = integer.trim_start_matches('0'); - let integer_digits = if integer.is_empty() { String::from("0") } else { integer.to_owned() }; + let integer_digits = if integer.is_empty() { + String::from("0") + } else { + integer.to_owned() + }; let without_trailing = fraction.trim_end_matches('0').to_owned(); PluralOperands { integer_digits, fraction_digits: fraction.to_owned(), fraction_digits_without_trailing_zeros: without_trailing.clone(), - visible_fraction_digits: u8::try_from(fraction.len()).expect("fraction option is at most 100"), + visible_fraction_digits: u8::try_from(fraction.len()) + .expect("fraction option is at most 100"), visible_fraction_digits_without_trailing_zeros: u8::try_from(without_trailing.len()) .expect("fraction option is at most 100"), exponent, @@ -1895,13 +2160,19 @@ fn number_option( fn resolve_digit_options( options: &NumberFormatOptions, - mut minimum_fraction_default: u8, + minimum_fraction_default: u8, mut maximum_fraction_default: u8, notation: Notation, ) -> Result { let minimum_integer = u8::try_from( - number_option("minimumIntegerDigits", options.minimum_integer_digits, 1, 21, Some(1))? - .expect("fallback provided"), + number_option( + "minimumIntegerDigits", + options.minimum_integer_digits, + 1, + 21, + Some(1), + )? + .expect("fallback provided"), ) .expect("minimum integer range fits u8"); let mut minimum_fraction = options.minimum_fraction_digits; @@ -1916,13 +2187,16 @@ fn resolve_digit_options( Some(1), )? .expect("fallback provided"); - const ALLOWED_INCREMENTS: [u16; 15] = [1, 2, 5, 10, 20, 25, 50, 100, 200, 250, 500, 1000, 2000, 2500, 5000]; + const ALLOWED_INCREMENTS: [u16; 15] = [ + 1, 2, 5, 10, 20, 25, 50, 100, 200, 250, 500, 1000, 2000, 2500, 5000, + ]; if !ALLOWED_INCREMENTS.contains(&rounding_increment) { return Err(NumberFormatError::range("invalid roundingIncrement")); } let rounding_mode = parse_rounding_mode(options.rounding_mode.as_deref())?; let requested_priority = parse_rounding_priority(options.rounding_priority.as_deref())?; - let trailing_zero_display = parse_trailing_zero_display(options.trailing_zero_display.as_deref())?; + let trailing_zero_display = + parse_trailing_zero_display(options.trailing_zero_display.as_deref())?; if rounding_increment != 1 { maximum_fraction_default = minimum_fraction_default; } @@ -1940,8 +2214,14 @@ fn resolve_digit_options( let (minimum_significant_resolved, maximum_significant_resolved) = if need_significant { if has_significant { let minimum = u8::try_from( - number_option("minimumSignificantDigits", minimum_significant, 1, 21, Some(1))? - .expect("fallback provided"), + number_option( + "minimumSignificantDigits", + minimum_significant, + 1, + 21, + Some(1), + )? + .expect("fallback provided"), ) .expect("significant digit range fits u8"); let maximum = u8::try_from( @@ -1965,11 +2245,27 @@ fn resolve_digit_options( let (minimum_fraction_resolved, maximum_fraction_resolved) = if need_fraction { if has_fraction { - let minimum = number_option("minimumFractionDigits", minimum_fraction.take(), 0, 100, None)?; - let maximum = number_option("maximumFractionDigits", maximum_fraction.take(), 0, 100, None)?; + let minimum = number_option( + "minimumFractionDigits", + minimum_fraction.take(), + 0, + 100, + None, + )?; + let maximum = number_option( + "maximumFractionDigits", + maximum_fraction.take(), + 0, + 100, + None, + )?; let (minimum, maximum) = match (minimum, maximum) { - (None, Some(maximum)) => (u16::from(minimum_fraction_default).min(maximum), maximum), - (Some(minimum), None) => (minimum, u16::from(maximum_fraction_default).max(minimum)), + (None, Some(maximum)) => { + (u16::from(minimum_fraction_default).min(maximum), maximum) + } + (Some(minimum), None) => { + (minimum, u16::from(maximum_fraction_default).max(minimum)) + } (Some(minimum), Some(maximum)) if minimum <= maximum => (minimum, maximum), (Some(_), Some(_)) => { return Err(NumberFormatError::range( @@ -1983,7 +2279,10 @@ fn resolve_digit_options( Some(u8::try_from(maximum).expect("fraction range fits u8")), ) } else { - (Some(minimum_fraction_default), Some(maximum_fraction_default)) + ( + Some(minimum_fraction_default), + Some(maximum_fraction_default), + ) } } else { (None, None) @@ -1997,7 +2296,14 @@ fn resolve_digit_options( rounding_type, computed_priority, ) = if !need_significant && !need_fraction { - (Some(0), Some(0), Some(1), Some(2), RoundingType::MorePrecision, RoundingPriority::MorePrecision) + ( + Some(0), + Some(0), + Some(1), + Some(2), + RoundingType::MorePrecision, + RoundingPriority::MorePrecision, + ) } else if requested_priority == RoundingPriority::MorePrecision { ( minimum_fraction_resolved, @@ -2079,7 +2385,9 @@ fn parse_currency_display(value: Option<&str>) -> Result Ok(CurrencyDisplay::Symbol), "narrowSymbol" => Ok(CurrencyDisplay::NarrowSymbol), "name" => Ok(CurrencyDisplay::Name), - value => Err(NumberFormatError::range(format!("invalid currencyDisplay: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid currencyDisplay: {value}" + ))), } } @@ -2087,7 +2395,9 @@ fn parse_currency_sign(value: Option<&str>) -> Result Ok(CurrencySign::Standard), "accounting" => Ok(CurrencySign::Accounting), - value => Err(NumberFormatError::range(format!("invalid currencySign: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid currencySign: {value}" + ))), } } @@ -2096,7 +2406,9 @@ fn parse_unit_display(value: Option<&str>) -> Result Ok(UnitDisplay::Short), "narrow" => Ok(UnitDisplay::Narrow), "long" => Ok(UnitDisplay::Long), - value => Err(NumberFormatError::range(format!("invalid unitDisplay: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid unitDisplay: {value}" + ))), } } @@ -2106,7 +2418,9 @@ fn parse_notation(value: Option<&str>) -> Result { "scientific" => Ok(Notation::Scientific), "engineering" => Ok(Notation::Engineering), "compact" => Ok(Notation::Compact), - value => Err(NumberFormatError::range(format!("invalid notation: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid notation: {value}" + ))), } } @@ -2114,7 +2428,9 @@ fn parse_compact_display(value: Option<&str>) -> Result Ok(CompactDisplay::Short), "long" => Ok(CompactDisplay::Long), - value => Err(NumberFormatError::range(format!("invalid compactDisplay: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid compactDisplay: {value}" + ))), } } @@ -2125,7 +2441,9 @@ fn parse_sign_display(value: Option<&str>) -> Result Ok(SignDisplay::Always), "exceptZero" => Ok(SignDisplay::ExceptZero), "negative" => Ok(SignDisplay::Negative), - value => Err(NumberFormatError::range(format!("invalid signDisplay: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid signDisplay: {value}" + ))), } } @@ -2140,7 +2458,9 @@ fn parse_rounding_mode(value: Option<&str>) -> Result Ok(RoundingMode::HalfExpand), "halfTrunc" => Ok(RoundingMode::HalfTrunc), "halfEven" => Ok(RoundingMode::HalfEven), - value => Err(NumberFormatError::range(format!("invalid roundingMode: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid roundingMode: {value}" + ))), } } @@ -2149,15 +2469,21 @@ fn parse_rounding_priority(value: Option<&str>) -> Result Ok(RoundingPriority::Auto), "morePrecision" => Ok(RoundingPriority::MorePrecision), "lessPrecision" => Ok(RoundingPriority::LessPrecision), - value => Err(NumberFormatError::range(format!("invalid roundingPriority: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid roundingPriority: {value}" + ))), } } -fn parse_trailing_zero_display(value: Option<&str>) -> Result { +fn parse_trailing_zero_display( + value: Option<&str>, +) -> Result { match value.unwrap_or("auto") { "auto" => Ok(TrailingZeroDisplay::Auto), "stripIfInteger" => Ok(TrailingZeroDisplay::StripIfInteger), - value => Err(NumberFormatError::range(format!("invalid trailingZeroDisplay: {value}"))), + value => Err(NumberFormatError::range(format!( + "invalid trailingZeroDisplay: {value}" + ))), } } @@ -2174,19 +2500,23 @@ fn resolve_grouping( "auto" => Ok(UseGrouping::Auto), "always" => Ok(UseGrouping::Always), "true" | "false" => Ok(fallback), - _ => Err(NumberFormatError::range(format!("invalid useGrouping: {value}"))), + _ => Err(NumberFormatError::range(format!( + "invalid useGrouping: {value}" + ))), }, } } fn validate_currency(value: &str) -> Result { if value.len() != 3 || !value.bytes().all(|byte| byte.is_ascii_alphabetic()) { - return Err(NumberFormatError::range(format!("invalid currency code: {value}"))); + return Err(NumberFormatError::range(format!( + "invalid currency code: {value}" + ))); } Ok(value.to_ascii_uppercase()) } -const SANCTIONED_UNITS: [&str; 45] = [ +pub const SANCTIONED_UNITS: [&str; 45] = [ "acre", "bit", "byte", @@ -2234,15 +2564,22 @@ const SANCTIONED_UNITS: [&str; 45] = [ "year", ]; -fn validate_unit(value: &str) -> Result { - let valid = SANCTIONED_UNITS.contains(&value) +/// IsWellFormedUnitIdentifier. +#[must_use] +pub fn is_well_formed_unit(value: &str) -> bool { + SANCTIONED_UNITS.contains(&value) || value .split_once("-per-") .is_some_and(|(numerator, denominator)| { SANCTIONED_UNITS.contains(&numerator) && SANCTIONED_UNITS.contains(&denominator) - }); - if !valid { - return Err(NumberFormatError::range(format!("invalid unit identifier: {value}"))); + }) +} + +fn validate_unit(value: &str) -> Result { + if !is_well_formed_unit(value) { + return Err(NumberFormatError::range(format!( + "invalid unit identifier: {value}" + ))); } Ok(value.to_owned()) } @@ -2250,7 +2587,8 @@ fn validate_unit(value: &str) -> Result { fn is_unicode_type(value: &str) -> bool { !value.is_empty() && value.split('-').all(|subtag| { - (3..=8).contains(&subtag.len()) && subtag.bytes().all(|byte| byte.is_ascii_alphanumeric()) + (3..=8).contains(&subtag.len()) + && subtag.bytes().all(|byte| byte.is_ascii_alphanumeric()) }) } @@ -2278,7 +2616,9 @@ fn partition_pattern(pattern: &str) -> Result, NumberFormatErr .bytes() .all(|byte| byte.is_ascii_alphanumeric() || byte == b'_') { - return Err(NumberFormatError::invalid_data("invalid pattern placeholder")); + return Err(NumberFormatError::invalid_data( + "invalid pattern placeholder", + )); } result.push(PatternToken::Placeholder(name.to_owned())); cursor = end + 1; @@ -2296,8 +2636,13 @@ fn validate_pattern(pattern: &str) -> Result<(), NumberFormatError> { fn validate_number_patterns(patterns: &NumberPatternSet) -> Result<(), NumberFormatError> { for pattern in [&patterns.zero, &patterns.positive, &patterns.negative] { let tokens = partition_pattern(pattern)?; - if !tokens.iter().any(|token| matches!(token, PatternToken::Placeholder(name) if name == "number")) { - return Err(NumberFormatError::invalid_data("number pattern lacks {number}")); + if !tokens + .iter() + .any(|token| matches!(token, PatternToken::Placeholder(name) if name == "number")) + { + return Err(NumberFormatError::invalid_data( + "number pattern lacks {number}", + )); } } Ok(()) @@ -2308,8 +2653,13 @@ fn validate_range_patterns(patterns: &RangePatterns) -> Result<(), NumberFormatE return Err(NumberFormatError::invalid_data("range separator is empty")); } let tokens = partition_pattern(&patterns.approximate)?; - if !tokens.iter().any(|token| matches!(token, PatternToken::Placeholder(name) if name == "number")) { - return Err(NumberFormatError::invalid_data("approximate pattern lacks {number}")); + if !tokens + .iter() + .any(|token| matches!(token, PatternToken::Placeholder(name) if name == "number")) + { + return Err(NumberFormatError::invalid_data( + "approximate pattern lacks {number}", + )); } Ok(()) } @@ -2394,7 +2744,11 @@ mod tests { } impl NumberFormatDataProvider for TestProvider { - fn number_symbols(&self, _data_locale: &str, numbering_system: &str) -> Option { + fn number_symbols( + &self, + _data_locale: &str, + numbering_system: &str, + ) -> Option { let digits = match numbering_system { "latn" => ['0', '1', '2', '3', '4', '5', '6', '7', '8', '9'], "deva" => ['०', '१', '२', '३', '४', '५', '६', '७', '८', '९'], @@ -2414,15 +2768,31 @@ mod tests { }) } - fn grouping_sizes(&self, data_locale: &str, _numbering_system: &str) -> Option { + fn grouping_sizes( + &self, + data_locale: &str, + _numbering_system: &str, + ) -> Option { Some(if data_locale == "hi" { - GroupingSizes { primary: 3, secondary: 2, minimum_grouping_digits: 1 } + GroupingSizes { + primary: 3, + secondary: 2, + minimum_grouping_digits: 1, + } } else { - GroupingSizes { primary: 3, secondary: 3, minimum_grouping_digits: 1 } + GroupingSizes { + primary: 3, + secondary: 3, + minimum_grouping_digits: 1, + } }) } - fn number_patterns(&self, _data_locale: &str, request: NumberPatternRequest<'_>) -> Option { + fn number_patterns( + &self, + _data_locale: &str, + request: NumberPatternRequest<'_>, + ) -> Option { let affixes = match request.style { NumberStyle::Decimal => ("{number}", "{plusSign}{number}", "{minusSign}{number}"), NumberStyle::Percent => ( @@ -2430,11 +2800,15 @@ mod tests { "{plusSign}{number}{percentSign}", "{minusSign}{number}{percentSign}", ), - NumberStyle::Currency if request.currency_sign == Some(CurrencySign::Accounting) => ( - "{currencyPrefix}{number}", - "{plusSign}{currencyPrefix}{number}", - "({currencyPrefix}{number})", - ), + NumberStyle::Currency + if request.currency_sign == Some(CurrencySign::Accounting) => + { + ( + "{currencyPrefix}{number}", + "{plusSign}{currencyPrefix}{number}", + "({currencyPrefix}{number})", + ) + } NumberStyle::Currency => ( "{currencyPrefix}{number}", "{plusSign}{currencyPrefix}{number}", @@ -2454,11 +2828,22 @@ mod tests { } fn scientific_pattern(&self, _data_locale: &str) -> Option { - Some(String::from("{number}{scientificSeparator}{scientificExponent}")) + Some(String::from( + "{number}{scientificSeparator}{scientificExponent}", + )) } - fn compact_exponent(&self, _data_locale: &str, _display: CompactDisplay, magnitude: i32) -> Option { - Some(if magnitude < 3 { 0 } else { (magnitude / 3).min(4) * 3 }) + fn compact_exponent( + &self, + _data_locale: &str, + _display: CompactDisplay, + magnitude: i32, + ) -> Option { + Some(if magnitude < 3 { + 0 + } else { + (magnitude / 3).min(4) * 3 + }) } fn compact_pattern( @@ -2520,11 +2905,13 @@ mod tests { "CLF" => Some(String::from("UF")), _ => None, }, - CurrencyDisplay::Name if currency == "USD" => Some(if category == PluralCategory::One { - String::from("US dollar") - } else { - String::from("US dollars") - }), + CurrencyDisplay::Name if currency == "USD" => { + Some(if category == PluralCategory::One { + String::from("US dollar") + } else { + String::from("US dollars") + }) + } CurrencyDisplay::Name => None, } } @@ -2549,14 +2936,21 @@ mod tests { } fn plural_category(&self, _data_locale: &str, operands: &PluralOperands) -> PluralCategory { - if operands.integer_digits == "1" && operands.visible_fraction_digits == 0 && operands.exponent == 0 { + if operands.integer_digits == "1" + && operands.visible_fraction_digits == 0 + && operands.exponent == 0 + { PluralCategory::One } else { PluralCategory::Other } } - fn range_patterns(&self, _data_locale: &str, _numbering_system: &str) -> Option { + fn range_patterns( + &self, + _data_locale: &str, + _numbering_system: &str, + ) -> Option { Some(RangePatterns { separator: String::from("–"), approximate: String::from("{approximatelySign}{number}"), @@ -2582,9 +2976,17 @@ mod tests { #[test] fn finite_non_finite_percent_and_negative_zero() { let decimal = formatter(NumberFormatOptions::default()).expect("formatter"); - assert_eq!(decimal.format_number(12_345.678).expect("finite"), "12,345.678"); + assert_eq!( + decimal.format_number(12_345.678).expect("finite"), + "12,345.678" + ); assert_eq!(decimal.format_number(f64::INFINITY).expect("infinity"), "∞"); - assert_eq!(decimal.format_number(f64::NEG_INFINITY).expect("negative infinity"), "−∞"); + assert_eq!( + decimal + .format_number(f64::NEG_INFINITY) + .expect("negative infinity"), + "−∞" + ); assert_eq!(decimal.format_number(f64::NAN).expect("NaN"), "NaN"); assert_eq!(decimal.format_number(-0.0).expect("negative zero"), "−0"); @@ -2611,9 +3013,21 @@ mod tests { ..NumberFormatOptions::default() }) .expect("sign formatter"); - assert_eq!(formatter.format_number(-0.0).expect("-0"), negative_zero, "{mode}"); - assert_eq!(formatter.format_number(0.0).expect("0"), positive_zero, "{mode}"); - assert_eq!(formatter.format_number(-1.0).expect("-1"), negative, "{mode}"); + assert_eq!( + formatter.format_number(-0.0).expect("-0"), + negative_zero, + "{mode}" + ); + assert_eq!( + formatter.format_number(0.0).expect("0"), + positive_zero, + "{mode}" + ); + assert_eq!( + formatter.format_number(-1.0).expect("-1"), + negative, + "{mode}" + ); assert_eq!(formatter.format_number(1.0).expect("1"), positive, "{mode}"); } } @@ -2627,7 +3041,12 @@ mod tests { .expect("large bigint"), "123,456,789,012,345,678,901,234,567,890" ); - assert_eq!(formatter.format_bigint("-100000000000000000001").expect("negative bigint"), "−100,000,000,000,000,000,001"); + assert_eq!( + formatter + .format_bigint("-100000000000000000001") + .expect("negative bigint"), + "−100,000,000,000,000,000,001" + ); } #[test] @@ -2645,7 +3064,11 @@ mod tests { ..NumberFormatOptions::default() }) .expect("currency formatter"); - assert_eq!(formatter.format_number(1.23456).expect("currency"), expected, "{currency}"); + assert_eq!( + formatter.format_number(1.23456).expect("currency"), + expected, + "{currency}" + ); } let accounting = formatter(NumberFormatOptions { style: Some(String::from("currency")), @@ -2654,7 +3077,10 @@ mod tests { ..NumberFormatOptions::default() }) .expect("accounting formatter"); - assert_eq!(accounting.format_number(-2.0).expect("accounting"), "($2.00)"); + assert_eq!( + accounting.format_number(-2.0).expect("accounting"), + "($2.00)" + ); } #[test] @@ -2746,7 +3172,10 @@ mod tests { #[test] fn number_shortest_decimal_not_binary_expansion_is_rounded() { let formatter = formatter(fixed_options(2.0, "halfExpand")).expect("fixed formatter"); - assert_eq!(formatter.format_number(1.005).expect("shortest decimal"), "1.01"); + assert_eq!( + formatter.format_number(1.005).expect("shortest decimal"), + "1.01" + ); assert_eq!(formatter.format_number(0.1 + 0.2).expect("sum"), "0.30"); } @@ -2799,18 +3228,36 @@ mod tests { ..NumberFormatOptions::default() }) .expect("scientific"); - assert_eq!(scientific.format_number(12_345.0).expect("scientific"), "1.235E4"); + assert_eq!( + scientific.format_number(12_345.0).expect("scientific"), + "1.235E4" + ); let scientific_parts = scientific.format_number_to_parts(0.0123).expect("parts"); - assert!(scientific_parts.iter().any(|part| part.part_type == NumberPartType::ExponentSeparator)); - assert!(scientific_parts.iter().any(|part| part.part_type == NumberPartType::ExponentMinusSign)); - assert!(scientific_parts.iter().any(|part| part.part_type == NumberPartType::ExponentInteger)); + assert!( + scientific_parts + .iter() + .any(|part| part.part_type == NumberPartType::ExponentSeparator) + ); + assert!( + scientific_parts + .iter() + .any(|part| part.part_type == NumberPartType::ExponentMinusSign) + ); + assert!( + scientific_parts + .iter() + .any(|part| part.part_type == NumberPartType::ExponentInteger) + ); let engineering = formatter(NumberFormatOptions { notation: Some(String::from("engineering")), ..NumberFormatOptions::default() }) .expect("engineering"); - assert_eq!(engineering.format_number(12_345.0).expect("engineering"), "12.345E3"); + assert_eq!( + engineering.format_number(12_345.0).expect("engineering"), + "12.345E3" + ); let compact = formatter(NumberFormatOptions { notation: Some(String::from("compact")), @@ -2818,12 +3265,17 @@ mod tests { }) .expect("compact"); assert_eq!(compact.format_number(1_234_567.0).expect("compact"), "1.2M"); - assert_eq!(compact.format_number(999_500.0).expect("compact carry"), "1M"); - assert!(compact - .format_number_to_parts(1_200.0) - .expect("compact parts") - .iter() - .any(|part| part.part_type == NumberPartType::Compact && part.value == "K")); + assert_eq!( + compact.format_number(999_500.0).expect("compact carry"), + "1M" + ); + assert!( + compact + .format_number_to_parts(1_200.0) + .expect("compact parts") + .iter() + .any(|part| part.part_type == NumberPartType::Compact && part.value == "K") + ); } #[test] @@ -2834,7 +3286,9 @@ mod tests { ..NumberFormatOptions::default() }) .expect("currency"); - let parts = currency.format_number_to_parts(-1_234.5).expect("currency parts"); + let parts = currency + .format_number_to_parts(-1_234.5) + .expect("currency parts"); assert_eq!( parts.iter().map(|part| part.part_type).collect::>(), vec![ @@ -2869,12 +3323,34 @@ mod tests { .expect("formatter"); let start = IntlMathematicalValue::from_f64(1.2); let end = IntlMathematicalValue::from_f64(2.3); - let parts = formatter.format_range_to_parts(&start, &end).expect("range parts"); - assert!(parts.iter().any(|part| part.source == RangeSource::StartRange)); - assert!(parts.iter().any(|part| part.source == RangeSource::EndRange)); - assert!(parts.iter().any(|part| part.source == RangeSource::Shared && part.value == "–")); - assert_eq!(formatter.format_range(&start, &end).expect("range"), "1.2–2.3"); - assert_eq!(formatter.format_range(&end, &start).expect("reversed range"), "2.3–1.2"); + let parts = formatter + .format_range_to_parts(&start, &end) + .expect("range parts"); + assert!( + parts + .iter() + .any(|part| part.source == RangeSource::StartRange) + ); + assert!( + parts + .iter() + .any(|part| part.source == RangeSource::EndRange) + ); + assert!( + parts + .iter() + .any(|part| part.source == RangeSource::Shared && part.value == "–") + ); + assert_eq!( + formatter.format_range(&start, &end).expect("range"), + "1.2–2.3" + ); + assert_eq!( + formatter + .format_range(&end, &start) + .expect("reversed range"), + "2.3–1.2" + ); let equal = formatter .format_range_to_parts( @@ -2883,11 +3359,23 @@ mod tests { ) .expect("approximately equal"); assert!(equal.iter().all(|part| part.source == RangeSource::Shared)); - assert_eq!(equal.first().map(|part| part.part_type), Some(NumberPartType::ApproximatelySign)); - assert_eq!(equal.iter().map(|part| part.value.as_str()).collect::(), "≈1.2"); + assert_eq!( + equal.first().map(|part| part.part_type), + Some(NumberPartType::ApproximatelySign) + ); + assert_eq!( + equal + .iter() + .map(|part| part.value.as_str()) + .collect::(), + "≈1.2" + ); let nan_error = formatter - .format_range(&IntlMathematicalValue::Nan, &IntlMathematicalValue::from_f64(1.0)) + .format_range( + &IntlMathematicalValue::Nan, + &IntlMathematicalValue::from_f64(1.0), + ) .expect_err("NaN endpoint"); assert_eq!(nan_error.kind, NumberFormatErrorKind::RangeError); } @@ -2897,14 +3385,22 @@ mod tests { let provider = TestProvider::new(); let formatter = NumberFormat::new( &[String::from("hi-u-nu-latn")], - &NumberFormatOptions { numbering_system: Some(String::from("deva")), ..NumberFormatOptions::default() }, + &NumberFormatOptions { + numbering_system: Some(String::from("deva")), + ..NumberFormatOptions::default() + }, &provider, &TestHost, ) .expect("resolved formatter"); assert_eq!(formatter.resolved_options().locale, "hi"); assert_eq!(formatter.resolved_options().numbering_system, "deva"); - assert_eq!(formatter.format_number(12_345_678.0).expect("Indian grouping"), "१,२३,४५,६७८"); + assert_eq!( + formatter + .format_number(12_345_678.0) + .expect("Indian grouping"), + "१,२३,४५,६७८" + ); let no_group = NumberFormat::new( &[String::from("en")], @@ -2916,22 +3412,29 @@ mod tests { &TestHost, ) .expect("no grouping"); - assert_eq!(no_group.format_number(12_345.0).expect("ungrouped"), "12345"); + assert_eq!( + no_group.format_number(12_345.0).expect("ungrouped"), + "12345" + ); } #[test] fn provider_determinism_and_no_hidden_english_fallback() { - let options = NumberFormatOptions { maximum_fraction_digits: Some(1.0), ..NumberFormatOptions::default() }; + let options = NumberFormatOptions { + maximum_fraction_digits: Some(1.0), + ..NumberFormatOptions::default() + }; let first_provider = TestProvider::new().with_decimal("·"); let second_provider = TestProvider::new().with_decimal("·"); let first = NumberFormat::new(&[String::from("en")], &options, &first_provider, &TestHost) .expect("first formatter") .format_number(12.5) .expect("first output"); - let second = NumberFormat::new(&[String::from("en")], &options, &second_provider, &TestHost) - .expect("second formatter") - .format_number(12.5) - .expect("second output"); + let second = + NumberFormat::new(&[String::from("en")], &options, &second_provider, &TestHost) + .expect("second formatter") + .format_number(12.5) + .expect("second output"); assert_eq!(first, "12·5"); assert_eq!(first, second); } @@ -2940,8 +3443,14 @@ mod tests { fn free_to_locale_string_entry_points_share_the_core() { let provider = TestProvider::new(); assert_eq!( - number_to_locale_string(1_234.5, &[], &NumberFormatOptions::default(), &provider, &TestHost) - .expect("Number toLocaleString"), + number_to_locale_string( + 1_234.5, + &[], + &NumberFormatOptions::default(), + &provider, + &TestHost + ) + .expect("Number toLocaleString"), "1,234.5" ); assert_eq!( diff --git a/crates/bamts-runtime/src/intl/plural_rules.rs b/crates/bamts-runtime/src/intl/plural_rules.rs index a3a99d9d..b4ff97e4 100644 --- a/crates/bamts-runtime/src/intl/plural_rules.rs +++ b/crates/bamts-runtime/src/intl/plural_rules.rs @@ -656,11 +656,7 @@ pub trait IntlServiceDataProvider { fn locale_data(&self) -> &dyn LocaleDataProvider; /// Ordered plural categories exposed by `resolvedOptions()`. - fn plural_categories( - &self, - _locale: &str, - _rule_type: PluralRuleType, - ) -> &[PluralCategory] { + fn plural_categories(&self, _locale: &str, _rule_type: PluralRuleType) -> &[PluralCategory] { &[] } @@ -776,6 +772,10 @@ pub struct PluralRulesOptions { pub rounding_mode: Option, /// `trailingZeroDisplay`. pub trailing_zero_display: Option, + /// `notation`. + pub notation: Option, + /// `compactDisplay`. + pub compact_display: Option, } /// PluralRules rounding-priority mode. @@ -931,8 +931,12 @@ impl DigitOptions { TrailingZeroDisplay::parse(options.trailing_zero_display.as_deref())?; let has_significant = options.minimum_significant_digits.is_some() || options.maximum_significant_digits.is_some(); + let has_fraction = + options.minimum_fraction_digits.is_some() || options.maximum_fraction_digits.is_some(); + let compact = options.notation.as_deref() == Some("compact"); let need_significant = priority != RoundingPriority::Auto || has_significant; - let need_fraction = priority != RoundingPriority::Auto || !has_significant; + let need_fraction = + priority != RoundingPriority::Auto || !(has_significant || (compact && !has_fraction)); let (minimum_fraction_digits, maximum_fraction_digits) = if need_fraction { fraction_digit_bounds(options)? @@ -961,6 +965,21 @@ impl DigitOptions { value: rounding_increment.to_string(), }); } + if !need_significant && !need_fraction { + return Ok(Self { + minimum_integer_digits, + minimum_fraction_digits: 0, + maximum_fraction_digits: 0, + minimum_significant_digits: 1, + maximum_significant_digits: 2, + has_significant_digits: true, + rounding_increment, + rounding_mode, + rounding_priority: RoundingPriority::MorePrecision, + rounding_type: RoundingType::MorePrecision, + trailing_zero_display, + }); + } let rounding_type = match priority { RoundingPriority::MorePrecision => RoundingType::MorePrecision, RoundingPriority::LessPrecision => RoundingType::LessPrecision, @@ -1088,7 +1107,9 @@ fn resolve_service_locale( keys: &[String], ) -> Result { let requested = requested_with_default(requested, host, provider)?; - Ok(resolve_locale(&requested, options, keys, matcher, provider)?) + Ok(resolve_locale( + &requested, options, keys, matcher, provider, + )?) } /// The exact values reported by `PluralRules.prototype.resolvedOptions`. @@ -1118,6 +1139,10 @@ pub struct PluralRulesResolvedOptions { pub trailing_zero_display: TrailingZeroDisplay, /// Provider-defined observable category order. pub plural_categories: Vec, + /// Resolved notation. + pub notation: String, + /// Resolved compact display, for compact notation. + pub compact_display: Option, } /// ECMA-402 PluralRules service core. @@ -1129,6 +1154,8 @@ pub struct PluralRules<'a> { rule_type: PluralRuleType, digits: DigitOptions, categories: Vec, + notation: String, + compact_display: Option, } impl<'a> PluralRules<'a> { @@ -1165,6 +1192,16 @@ impl<'a> PluralRules<'a> { rule_type, digits, categories, + notation: options + .notation + .clone() + .unwrap_or_else(|| String::from("standard")), + compact_display: (options.notation.as_deref() == Some("compact")).then(|| { + options + .compact_display + .clone() + .unwrap_or_else(|| String::from("short")) + }), }) } @@ -1205,16 +1242,16 @@ impl<'a> PluralRules<'a> { pub fn resolved_options(&self) -> PluralRulesResolvedOptions { let fraction_active = matches!( self.digits.rounding_type, - RoundingType::FractionDigits | RoundingType::MorePrecision | RoundingType::LessPrecision + RoundingType::FractionDigits + | RoundingType::MorePrecision + | RoundingType::LessPrecision ); PluralRulesResolvedOptions { locale: self.locale.clone(), plural_type: self.rule_type, minimum_integer_digits: self.digits.minimum_integer_digits, - minimum_fraction_digits: fraction_active - .then_some(self.digits.minimum_fraction_digits), - maximum_fraction_digits: fraction_active - .then_some(self.digits.maximum_fraction_digits), + minimum_fraction_digits: fraction_active.then_some(self.digits.minimum_fraction_digits), + maximum_fraction_digits: fraction_active.then_some(self.digits.maximum_fraction_digits), minimum_significant_digits: self .digits .has_significant_digits @@ -1228,6 +1265,8 @@ impl<'a> PluralRules<'a> { rounding_priority: self.digits.rounding_priority, trailing_zero_display: self.digits.trailing_zero_display, plural_categories: self.categories.clone(), + notation: self.notation.clone(), + compact_display: self.compact_display.clone(), } } @@ -1333,11 +1372,7 @@ fn round_finite(value: f64, options: DigitOptions) -> RoundedDecimal { } } -fn round_fraction( - exact: &ExactDecimal, - negative: bool, - options: DigitOptions, -) -> RoundedDecimal { +fn round_fraction(exact: &ExactDecimal, negative: bool, options: DigitOptions) -> RoundedDecimal { let magnitude = -i32::from(options.maximum_fraction_digits); let coefficient = quantize( exact, @@ -1399,13 +1434,7 @@ fn quantize( let has_remainder = integer_remainder != 0 || has_fraction; let relation = compare_half(integer_remainder, &fraction, increment); let quotient_is_odd = quotient.last().is_some_and(|digit| digit % 2 == 1); - if should_increment( - mode, - negative, - has_remainder, - relation, - quotient_is_odd, - ) { + if should_increment(mode, negative, has_remainder, relation, quotient_is_odd) { add_one(&mut quotient); } multiply_small("ient, increment) @@ -1503,8 +1532,7 @@ fn should_increment( RoundingMode::HalfExpand => relation != HalfRelation::Below, RoundingMode::HalfTrunc => relation == HalfRelation::Above, RoundingMode::HalfEven => { - relation == HalfRelation::Above - || (relation == HalfRelation::Equal && quotient_is_odd) + relation == HalfRelation::Above || (relation == HalfRelation::Equal && quotient_is_odd) } } } @@ -1801,12 +1829,9 @@ impl<'a> RelativeTimeFormat<'a> { && value.fract() == 0.0 && value >= i64::MIN as f64 && value <= i64::MAX as f64 - && let Some(term) = self.provider.relative_time_auto( - &self.data_locale, - self.style, - unit, - value as i64, - ) + && let Some(term) = + self.provider + .relative_time_auto(&self.data_locale, self.style, unit, value as i64) { return Ok(vec![RelativeTimePart { kind: RelativeTimePartKind::Literal, @@ -1884,7 +1909,11 @@ impl<'a> RelativeTimeFormat<'a> { fn validate_formatted_number(number: &FormattedNumber) -> Result<(), IntlError> { if number.parts.is_empty() - || number.parts.iter().map(|part| part.value.as_str()).collect::() + || number + .parts + .iter() + .map(|part| part.value.as_str()) + .collect::() != number.value { return Err(IntlError::InvalidData( @@ -1894,11 +1923,7 @@ fn validate_formatted_number(number: &FormattedNumber) -> Result<(), IntlError> Ok(()) } -fn push_relative_literal( - parts: &mut Vec, - literal: &str, - index: &mut usize, -) { +fn push_relative_literal(parts: &mut Vec, literal: &str, index: &mut usize) { if literal.is_empty() { return; } @@ -1911,7 +1936,10 @@ fn push_relative_literal( *index += utf16_len(literal); } -fn one_placeholder<'a>(pattern: &'a str, placeholder: &str) -> Result<(&'a str, &'a str), IntlError> { +fn one_placeholder<'a>( + pattern: &'a str, + placeholder: &str, +) -> Result<(&'a str, &'a str), IntlError> { let (prefix, suffix) = pattern .split_once(placeholder) .ok_or(IntlError::InvalidData("pattern is missing a placeholder"))?; @@ -2563,41 +2591,17 @@ fn canonical_calendar_code( fn canonical_date_time_field(code: &str) -> Result { const FIELDS: &[&str] = &[ "era", - "era-short", - "era-narrow", "year", - "year-short", - "year-narrow", "quarter", - "quarter-short", - "quarter-narrow", "month", - "month-short", - "month-narrow", "weekOfYear", - "weekOfYear-short", - "weekOfYear-narrow", "weekday", - "weekday-short", - "weekday-narrow", "day", - "day-short", - "day-narrow", "dayPeriod", - "dayPeriod-short", - "dayPeriod-narrow", "hour", - "hour-short", - "hour-narrow", "minute", - "minute-short", - "minute-narrow", "second", - "second-short", - "second-narrow", - "zone", - "zone-short", - "zone-narrow", + "timeZoneName", ]; if FIELDS.contains(&code) { Ok(code.to_owned()) @@ -2628,9 +2632,10 @@ fn is_unicode_language_id(value: &str) -> bool { return false; } let mut index = 0; - if rest.get(index).is_some_and(|part| { - part.len() == 4 && part.bytes().all(|byte| byte.is_ascii_alphabetic()) - }) { + if rest + .get(index) + .is_some_and(|part| part.len() == 4 && part.bytes().all(|byte| byte.is_ascii_alphabetic())) + { index += 1; } if rest.get(index).is_some_and(|part| is_region_subtag(part)) { @@ -2659,8 +2664,7 @@ fn is_region_subtag(value: &str) -> bool { fn is_unicode_type(value: &str) -> bool { !value.is_empty() && value.split('-').all(|part| { - (3..=8).contains(&part.len()) - && part.bytes().all(|byte| byte.is_ascii_alphanumeric()) + (3..=8).contains(&part.len()) && part.bytes().all(|byte| byte.is_ascii_alphanumeric()) }) } diff --git a/crates/bamts-runtime/src/intl/segmentation.rs b/crates/bamts-runtime/src/intl/segmentation.rs new file mode 100644 index 00000000..795ed14f --- /dev/null +++ b/crates/bamts-runtime/src/intl/segmentation.rs @@ -0,0 +1,369 @@ +//! UAX #29 text segmentation over UTF-16, from Unicode property data. + +use icu_properties::props::{ + ExtendedPictographic, GeneralCategory, GraphemeClusterBreak, SentenceBreak, WordBreak, +}; +use icu_properties::{CodePointMapData, CodePointSetData}; + +use super::plural_rules::{ProviderSegmentBoundary, SegmenterGranularity}; + +/// One code point and where it ends. A lone surrogate is a code point here. +struct Scalar { + character: char, + end: usize, +} + +fn scalars(input: &[u16]) -> Vec { + let mut result = Vec::with_capacity(input.len()); + let mut end = 0; + for unit in char::decode_utf16(input.iter().copied()) { + let (character, width) = match unit { + Ok(character) => (character, character.len_utf16()), + Err(_) => ('\u{fffd}', 1), + }; + end += width; + result.push(Scalar { character, end }); + } + result +} + +/// The grapheme cluster boundaries after each code point that ends a cluster. +fn grapheme_breaks(text: &[Scalar]) -> Vec { + use GraphemeClusterBreak as Gcb; + let classes = CodePointMapData::::new(); + let pictographic = CodePointSetData::new::(); + let mut breaks = Vec::new(); + let mut pictographic_run = false; + let mut indicators = 0_usize; + for (index, scalar) in text.iter().enumerate() { + let class = classes.get(scalar.character); + if pictographic.contains(scalar.character) { + pictographic_run = true; + } else if class != Gcb::Extend && class != Gcb::ZWJ { + pictographic_run = false; + } + indicators = if class == Gcb::RegionalIndicator { + indicators + 1 + } else { + 0 + }; + let Some(next) = text.get(index + 1) else { + breaks.push(index); + break; + }; + let after = classes.get(next.character); + let joined = match (class, after) { + (Gcb::CR, Gcb::LF) => true, + (Gcb::Control | Gcb::CR | Gcb::LF, _) | (_, Gcb::Control | Gcb::CR | Gcb::LF) => false, + (Gcb::L, Gcb::L | Gcb::V | Gcb::LV | Gcb::LVT) + | (Gcb::LV | Gcb::V, Gcb::V | Gcb::T) + | (Gcb::LVT | Gcb::T, Gcb::T) => true, + (_, Gcb::Extend | Gcb::ZWJ | Gcb::SpacingMark) | (Gcb::Prepend, _) => true, + (Gcb::ZWJ, _) if pictographic_run && pictographic.contains(next.character) => true, + (Gcb::RegionalIndicator, Gcb::RegionalIndicator) => indicators % 2 == 1, + _ => false, + }; + if !joined { + breaks.push(index); + } + } + breaks +} + +fn ignorable_word(class: WordBreak) -> bool { + matches!( + class, + WordBreak::Extend | WordBreak::Format | WordBreak::ZWJ + ) +} + +fn hard_word(class: WordBreak) -> bool { + matches!(class, WordBreak::CR | WordBreak::LF | WordBreak::Newline) +} + +fn letter(class: WordBreak) -> bool { + matches!(class, WordBreak::ALetter | WordBreak::HebrewLetter) +} + +/// Whether the word rules keep positions `index` and `index + 1` together. +fn word_joined(classes: &[WordBreak], pictographic: &[bool], index: usize) -> bool { + use WordBreak as Wb; + let (here, next) = (classes[index], classes[index + 1]); + if here == Wb::CR && next == Wb::LF { + return true; + } + if hard_word(here) || hard_word(next) { + return false; + } + if here == Wb::ZWJ && pictographic[index + 1] { + return true; + } + if here == Wb::WSegSpace && next == Wb::WSegSpace { + return true; + } + if ignorable_word(next) { + return true; + } + // Later rules see through Extend, Format and ZWJ. + let significant = |mut position: usize| { + while ignorable_word(classes[position]) && position > 0 { + position -= 1; + } + position + }; + let before = significant(index); + let left = classes[before]; + let previous = (before > 0).then(|| classes[significant(before - 1)]); + let mut following = index + 2; + while following < classes.len() && ignorable_word(classes[following]) { + following += 1; + } + let after = classes.get(following).copied(); + match (left, next) { + (l, r) if letter(l) && letter(r) => true, + (l, Wb::MidLetter | Wb::MidNumLet | Wb::SingleQuote) + if letter(l) && after.is_some_and(letter) => + { + true + } + (Wb::MidLetter | Wb::MidNumLet | Wb::SingleQuote, r) + if letter(r) && previous.is_some_and(letter) => + { + true + } + (Wb::HebrewLetter, Wb::SingleQuote) => true, + (Wb::HebrewLetter, Wb::DoubleQuote) if after == Some(Wb::HebrewLetter) => true, + (Wb::DoubleQuote, Wb::HebrewLetter) if previous == Some(Wb::HebrewLetter) => true, + (Wb::Numeric, Wb::Numeric) => true, + (l, Wb::Numeric) if letter(l) => true, + (Wb::Numeric, r) if letter(r) => true, + (Wb::MidNum | Wb::MidNumLet | Wb::SingleQuote, Wb::Numeric) + if previous == Some(Wb::Numeric) => + { + true + } + (Wb::Numeric, Wb::MidNum | Wb::MidNumLet | Wb::SingleQuote) + if after == Some(Wb::Numeric) => + { + true + } + (Wb::Katakana, Wb::Katakana) => true, + (l, Wb::ExtendNumLet) + if letter(l) || matches!(l, Wb::Numeric | Wb::Katakana | Wb::ExtendNumLet) => + { + true + } + (Wb::ExtendNumLet, r) if letter(r) || matches!(r, Wb::Numeric | Wb::Katakana) => true, + (Wb::RegionalIndicator, Wb::RegionalIndicator) => { + let mut count = 0; + let mut position = before; + loop { + if classes[position] == Wb::RegionalIndicator { + count += 1; + } else if !ignorable_word(classes[position]) { + break; + } + if position == 0 { + break; + } + position -= 1; + } + count % 2 == 1 + } + _ => false, + } +} + +fn word_breaks(text: &[Scalar]) -> Vec { + let map = CodePointMapData::::new(); + let extended = CodePointSetData::new::(); + let classes: Vec = text.iter().map(|s| map.get(s.character)).collect(); + let pictographic: Vec = text + .iter() + .map(|s| extended.contains(s.character)) + .collect(); + (0..text.len()) + .filter(|index| index + 1 == text.len() || !word_joined(&classes, &pictographic, *index)) + .collect() +} + +fn ignorable_sentence(class: SentenceBreak) -> bool { + matches!(class, SentenceBreak::Extend | SentenceBreak::Format) +} + +fn paragraph_separator(class: SentenceBreak) -> bool { + matches!( + class, + SentenceBreak::Sep | SentenceBreak::CR | SentenceBreak::LF + ) +} + +/// The rules SB6 to SB11, for a boundary before position `next`. `significant` +/// lists the code points before it that are not Extend or Format, nearest last. +fn sentence_joined(significant: &[SentenceBreak], classes: &[SentenceBreak], next: usize) -> bool { + use SentenceBreak as Sb; + let mut position = significant.len(); + let mut take_while = |test: &dyn Fn(Sb) -> bool| { + let start = position; + while position > 0 && test(significant[position - 1]) { + position -= 1; + } + start - position + }; + let spaces = take_while(&|class| class == Sb::Sp); + let closes = take_while(&|class| class == Sb::Close); + let Some(&terminator) = position + .checked_sub(1) + .and_then(|last| significant.get(last)) + else { + return true; + }; + if !matches!(terminator, Sb::ATerm | Sb::STerm) { + return true; + } + let before_terminator = position.checked_sub(2).map(|index| significant[index]); + let upcoming: Vec = classes[next..] + .iter() + .copied() + .filter(|class| !ignorable_sentence(*class)) + .collect(); + let first = upcoming.first().copied(); + if terminator == Sb::ATerm { + if spaces == 0 && closes == 0 && first == Some(Sb::Numeric) { + return true; + } + if spaces == 0 + && closes == 0 + && first == Some(Sb::Upper) + && matches!(before_terminator, Some(Sb::Upper | Sb::Lower)) + { + return true; + } + let lower_ahead = upcoming.iter().copied().find(|class| { + matches!( + *class, + Sb::OLetter + | Sb::Upper + | Sb::Lower + | Sb::Sep + | Sb::CR + | Sb::LF + | Sb::ATerm + | Sb::STerm + ) + }) == Some(Sb::Lower); + if lower_ahead { + return true; + } + } + if matches!(first, Some(Sb::SContinue | Sb::ATerm | Sb::STerm)) { + return true; + } + if spaces == 0 && matches!(first, Some(Sb::Close | Sb::Sp)) { + return true; + } + if spaces > 0 && matches!(first, Some(Sb::Sp)) { + return true; + } + spaces > 0 && first.is_some_and(paragraph_separator) + || closes > 0 && first.is_some_and(paragraph_separator) +} + +fn sentence_breaks(text: &[Scalar]) -> Vec { + use SentenceBreak as Sb; + let map = CodePointMapData::::new(); + let classes: Vec = text.iter().map(|s| map.get(s.character)).collect(); + let mut breaks = Vec::new(); + let mut significant: Vec = Vec::new(); + for index in 0..text.len() { + let class = classes[index]; + if !ignorable_sentence(class) || significant.is_empty() { + significant.push(class); + } + if index + 1 == text.len() { + breaks.push(index); + break; + } + let next = classes[index + 1]; + if class == Sb::CR && next == Sb::LF { + continue; + } + if paragraph_separator(class) { + breaks.push(index); + significant.clear(); + continue; + } + if ignorable_sentence(next) { + continue; + } + if !sentence_joined(&significant, &classes, index + 1) { + breaks.push(index); + significant.clear(); + } + } + breaks +} + +/// Whether a word segment has a letter or digit in it, as opposed to being +/// whitespace or punctuation. +fn is_word_like(text: &[Scalar]) -> bool { + let categories = CodePointMapData::::new(); + text.iter().any(|scalar| { + matches!( + categories.get(scalar.character), + GeneralCategory::UppercaseLetter + | GeneralCategory::LowercaseLetter + | GeneralCategory::TitlecaseLetter + | GeneralCategory::ModifierLetter + | GeneralCategory::OtherLetter + | GeneralCategory::DecimalNumber + | GeneralCategory::LetterNumber + | GeneralCategory::OtherNumber + ) + }) +} + +/// Segment ends for `input`, in UTF-16 code units. +pub(super) fn boundaries( + granularity: SegmenterGranularity, + input: &[u16], +) -> Vec { + let text = scalars(input); + let breaks = match granularity { + SegmenterGranularity::Grapheme => grapheme_breaks(&text), + SegmenterGranularity::Word => word_breaks(&text), + SegmenterGranularity::Sentence => sentence_breaks(&text), + }; + let mut start = 0; + breaks + .into_iter() + .map(|index| { + let boundary = ProviderSegmentBoundary { + end: text[index].end, + is_word_like: is_word_like(&text[start..=index]), + }; + start = index + 1; + boundary + }) + .collect() +} + +#[cfg(test)] +mod tests { + use super::*; + + fn ends(granularity: SegmenterGranularity, text: &str) -> Vec { + let units: Vec = text.encode_utf16().collect(); + boundaries(granularity, &units) + .iter() + .map(|boundary| boundary.end) + .collect() + } + + #[test] + fn words_break_around_spaces_and_keep_numbers_and_contractions_whole() { + assert_eq!(ends(SegmenterGranularity::Word, "123 "), [3, 4]); + assert_eq!(ends(SegmenterGranularity::Word, "can't stop"), [5, 6, 10]); + assert_eq!(ends(SegmenterGranularity::Word, "3.14"), [4]); + } +} diff --git a/crates/bamts-runtime/src/intl/service_data.rs b/crates/bamts-runtime/src/intl/service_data.rs new file mode 100644 index 00000000..741e6e72 --- /dev/null +++ b/crates/bamts-runtime/src/intl/service_data.rs @@ -0,0 +1,394 @@ +//! English data for PluralRules, RelativeTimeFormat, ListFormat, Segmenter, and +//! DisplayNames. + +use std::collections::BTreeMap; +use std::sync::LazyLock; + +use super::locale_data::{IntlData, numbering_system_digits}; +use super::locale_negotiation::LocaleDataProvider; +use super::number_data::CURRENCY_NAMES; +use super::plural_rules::{ + DisplayNamesStyle, DisplayNamesType, FormattedNumber, IntlServiceDataProvider, LanguageDisplay, + ListPatternPosition, ListStyle, ListType, NumberInput, NumberPart, NumberPartKind, + NumberSpecial, PluralCategory, PluralOperands, PluralRuleType, ProviderSegmentBoundary, + RelativeTimeStyle, RelativeTimeTense, RelativeTimeUnit, SegmenterGranularity, +}; +use super::segmentation; + +const CARDINAL: [PluralCategory; 2] = [PluralCategory::One, PluralCategory::Other]; +const ORDINAL: [PluralCategory; 4] = [ + PluralCategory::One, + PluralCategory::Two, + PluralCategory::Few, + PluralCategory::Other, +]; + +/// The English ordinal rule, on the integer digits. +fn ordinal(integer: &str) -> PluralCategory { + let last = |count: usize| { + integer + .get(integer.len().saturating_sub(count)..) + .unwrap_or(integer) + }; + match (last(1), last(2)) { + ("1", tens) if tens != "11" => PluralCategory::One, + ("2", tens) if tens != "12" => PluralCategory::Two, + ("3", tens) if tens != "13" => PluralCategory::Few, + _ => PluralCategory::Other, + } +} + +const fn style_index(style: RelativeTimeStyle) -> usize { + match style { + RelativeTimeStyle::Long => 0, + RelativeTimeStyle::Short => 1, + RelativeTimeStyle::Narrow => 2, + } +} + +/// Unit names by style: singular and plural, long then short, then the narrow +/// suffix, which attaches to the number. +const RELATIVE_UNITS: [(RelativeTimeUnit, [&str; 5]); 8] = [ + ( + RelativeTimeUnit::Second, + ["second", "seconds", "sec.", "sec.", "s"], + ), + ( + RelativeTimeUnit::Minute, + ["minute", "minutes", "min.", "min.", "min"], + ), + (RelativeTimeUnit::Hour, ["hour", "hours", "hr.", "hr.", "h"]), + (RelativeTimeUnit::Day, ["day", "days", "day", "days", "d"]), + (RelativeTimeUnit::Week, ["week", "weeks", "wk.", "wk.", "w"]), + ( + RelativeTimeUnit::Month, + ["month", "months", "mo.", "mo.", "mo"], + ), + ( + RelativeTimeUnit::Quarter, + ["quarter", "quarters", "qtr.", "qtrs.", "q"], + ), + (RelativeTimeUnit::Year, ["year", "years", "yr.", "yr.", "y"]), +]; + +/// Patterns by style, unit, tense, and whether the count is singular. +static RELATIVE_PATTERNS: LazyLock> = + LazyLock::new(|| { + let mut patterns = BTreeMap::new(); + for (unit, (_, names)) in RELATIVE_UNITS.iter().enumerate() { + for future in [false, true] { + for one in [false, true] { + let long = names[usize::from(!one)]; + let short = names[2 + usize::from(!one)]; + let spaced = |name: &str| { + if future { + format!("in {{0}} {name}") + } else { + format!("{{0}} {name} ago") + } + }; + let narrow = if future { + format!("in {{0}}{}", names[4]) + } else { + format!("{{0}}{} ago", names[4]) + }; + patterns.insert((0, unit, future, one), spaced(long)); + patterns.insert((1, unit, future, one), spaced(short)); + patterns.insert((2, unit, future, one), narrow); + } + } + } + patterns + }); + +/// The words for the current, previous, and next unit. Clock units have only +/// the current one. +fn relative_term(unit: RelativeTimeUnit, offset: i64) -> Option<&'static str> { + use RelativeTimeUnit as Unit; + Some(match (unit, offset) { + (Unit::Second, 0) => "now", + (Unit::Minute, 0) => "this minute", + (Unit::Hour, 0) => "this hour", + (Unit::Day, -1) => "yesterday", + (Unit::Day, 0) => "today", + (Unit::Day, 1) => "tomorrow", + (Unit::Week, -1) => "last week", + (Unit::Week, 0) => "this week", + (Unit::Week, 1) => "next week", + (Unit::Month, -1) => "last month", + (Unit::Month, 0) => "this month", + (Unit::Month, 1) => "next month", + (Unit::Quarter, -1) => "last quarter", + (Unit::Quarter, 0) => "this quarter", + (Unit::Quarter, 1) => "next quarter", + (Unit::Year, -1) => "last year", + (Unit::Year, 0) => "this year", + (Unit::Year, 1) => "next year", + _ => return None, + }) +} + +/// English names by code, the small set a display name is commonly asked for. +const LANGUAGE_NAMES: &[(&str, &str)] = &[ + ("ar", "Arabic"), + ("de", "German"), + ("en", "English"), + ("en-GB", "British English"), + ("en-US", "American English"), + ("es", "Spanish"), + ("fr", "French"), + ("hi", "Hindi"), + ("it", "Italian"), + ("ja", "Japanese"), + ("ko", "Korean"), + ("nl", "Dutch"), + ("pt", "Portuguese"), + ("ru", "Russian"), + ("zh", "Chinese"), +]; +const REGION_NAMES: &[(&str, &str)] = &[ + ("BR", "Brazil"), + ("CA", "Canada"), + ("CN", "China"), + ("DE", "Germany"), + ("ES", "Spain"), + ("FR", "France"), + ("GB", "United Kingdom"), + ("IN", "India"), + ("IT", "Italy"), + ("JP", "Japan"), + ("KR", "South Korea"), + ("MX", "Mexico"), + ("RU", "Russia"), + ("US", "United States"), +]; +const SCRIPT_NAMES: &[(&str, &str)] = &[ + ("Arab", "Arabic"), + ("Cyrl", "Cyrillic"), + ("Deva", "Devanagari"), + ("Grek", "Greek"), + ("Hans", "Simplified Han"), + ("Hant", "Traditional Han"), + ("Hebr", "Hebrew"), + ("Jpan", "Japanese"), + ("Kore", "Korean"), + ("Latn", "Latin"), +]; +const CALENDAR_NAMES: &[(&str, &str)] = &[ + ("buddhist", "Buddhist Calendar"), + ("coptic", "Coptic Calendar"), + ("ethioaa", "Ethiopic Amete Alem Calendar"), + ("ethiopic", "Ethiopic Calendar"), + ("gregory", "Gregorian Calendar"), + ("hebrew", "Hebrew Calendar"), + ("indian", "Indian National Calendar"), + ("islamic-civil", "Hijri Calendar (tabular, civil epoch)"), + ( + "islamic-tbla", + "Hijri Calendar (tabular, astronomical epoch)", + ), + ("iso8601", "ISO-8601 Calendar"), + ("japanese", "Japanese Calendar"), + ("persian", "Persian Calendar"), + ("roc", "Minguo Calendar"), +]; +const DATE_TIME_FIELD_NAMES: &[(&str, &str)] = &[ + ("day", "day"), + ("dayPeriod", "AM/PM"), + ("era", "era"), + ("hour", "hour"), + ("minute", "minute"), + ("month", "month"), + ("quarter", "quarter"), + ("second", "second"), + ("timeZoneName", "time zone"), + ("weekOfYear", "week"), + ("weekday", "day of the week"), + ("year", "year"), +]; + +/// Writes an ASCII decimal with grouping into `parts`, in `numbering_system`. +fn number_parts(input: &NumberInput, digits: &[char; 10]) -> Vec { + let transliterate = |ascii: &str| -> String { + ascii + .chars() + .map(|character| { + character + .to_digit(10) + .map_or(character, |digit| digits[digit as usize]) + }) + .collect() + }; + let mut parts = Vec::new(); + let mut push = |kind, value: String| parts.push(NumberPart { kind, value }); + let (integer, fraction) = input + .ascii + .split_once('.') + .map_or((input.ascii.as_str(), None), |(left, right)| { + (left, Some(right)) + }); + let leading = match integer.len() % 3 { + 0 => 3, + length => length, + }; + let mut start = 0; + let mut end = leading.min(integer.len()); + while start < integer.len() { + if start > 0 { + push(NumberPartKind::Group, String::from(",")); + } + push(NumberPartKind::Integer, transliterate(&integer[start..end])); + start = end; + end += 3; + } + if let Some(fraction) = fraction { + push(NumberPartKind::Decimal, String::from(".")); + push(NumberPartKind::Fraction, transliterate(fraction)); + } + parts +} + +impl IntlServiceDataProvider for IntlData { + fn locale_data(&self) -> &dyn LocaleDataProvider { + self + } + + fn plural_categories(&self, _locale: &str, rule_type: PluralRuleType) -> &[PluralCategory] { + match rule_type { + PluralRuleType::Cardinal => &CARDINAL, + PluralRuleType::Ordinal => &ORDINAL, + } + } + + fn plural_category( + &self, + _locale: &str, + rule_type: PluralRuleType, + operands: &PluralOperands, + ) -> Option { + Some(match rule_type { + PluralRuleType::Cardinal if operands.i == "1" && operands.v == 0 => PluralCategory::One, + PluralRuleType::Cardinal => PluralCategory::Other, + PluralRuleType::Ordinal if operands.v == 0 => ordinal(&operands.i), + PluralRuleType::Ordinal => PluralCategory::Other, + }) + } + + fn plural_range( + &self, + _locale: &str, + _start: PluralCategory, + end: PluralCategory, + ) -> Option { + Some(end) + } + + fn format_number( + &self, + _locale: &str, + numbering_system: &str, + input: &NumberInput, + ) -> Option { + let digits = numbering_system_digits(numbering_system)?; + let parts = match input.special { + Some(NumberSpecial::Nan) => vec![NumberPart { + kind: NumberPartKind::Nan, + value: String::from("NaN"), + }], + Some(NumberSpecial::Infinity) => vec![NumberPart { + kind: NumberPartKind::Infinity, + value: String::from("\u{221e}"), + }], + None => number_parts(input, &digits), + }; + let value = parts.iter().map(|part| part.value.as_str()).collect(); + Some(FormattedNumber { value, parts }) + } + + fn relative_time_auto( + &self, + _locale: &str, + _style: RelativeTimeStyle, + unit: RelativeTimeUnit, + offset: i64, + ) -> Option<&str> { + relative_term(unit, offset) + } + + fn relative_time_pattern( + &self, + _locale: &str, + style: RelativeTimeStyle, + unit: RelativeTimeUnit, + tense: RelativeTimeTense, + category: PluralCategory, + ) -> Option<&str> { + let unit = RELATIVE_UNITS.iter().position(|(name, _)| *name == unit)?; + RELATIVE_PATTERNS + .get(&( + style_index(style), + unit, + tense == RelativeTimeTense::Future, + category == PluralCategory::One, + )) + .map(String::as_str) + } + + fn list_pattern( + &self, + _locale: &str, + list_type: ListType, + style: ListStyle, + position: ListPatternPosition, + ) -> Option<&str> { + use ListPatternPosition as Position; + Some(match (list_type, style, position) { + (ListType::Unit, ListStyle::Narrow, _) => "{0} {1}", + (ListType::Unit, _, _) + | (_, _, Position::Start | Position::Middle) + | (ListType::Conjunction, ListStyle::Narrow, _) => "{0}, {1}", + (ListType::Disjunction, _, Position::Pair) => "{0} or {1}", + (ListType::Disjunction, _, Position::End) => "{0}, or {1}", + (ListType::Conjunction, ListStyle::Long, Position::Pair) => "{0} and {1}", + (ListType::Conjunction, ListStyle::Long, Position::End) => "{0}, and {1}", + (ListType::Conjunction, ListStyle::Short, Position::Pair) => "{0} & {1}", + (ListType::Conjunction, ListStyle::Short, Position::End) => "{0}, & {1}", + }) + } + + fn segment_boundaries( + &self, + _locale: &str, + granularity: SegmenterGranularity, + input: &[u16], + ) -> Option> { + Some(segmentation::boundaries(granularity, input)) + } + + fn display_name( + &self, + _locale: &str, + _style: DisplayNamesStyle, + name_type: DisplayNamesType, + _language_display: LanguageDisplay, + code: &str, + ) -> Option<&str> { + let table = match name_type { + DisplayNamesType::Language => LANGUAGE_NAMES, + DisplayNamesType::Region => REGION_NAMES, + DisplayNamesType::Script => SCRIPT_NAMES, + DisplayNamesType::Calendar => CALENDAR_NAMES, + DisplayNamesType::DateTimeField => DATE_TIME_FIELD_NAMES, + DisplayNamesType::Currency => { + return CURRENCY_NAMES + .iter() + .find(|entry| entry.0 == code) + .map(|entry| entry.1); + } + }; + table + .iter() + .find(|(name, _)| *name == code) + .map(|(_, text)| *text) + } +} diff --git a/crates/bamts-runtime/src/intrinsics.rs b/crates/bamts-runtime/src/intrinsics.rs index c0bf309d..b3a64d07 100644 --- a/crates/bamts-runtime/src/intrinsics.rs +++ b/crates/bamts-runtime/src/intrinsics.rs @@ -42,6 +42,16 @@ pub(crate) struct BuiltinDef { pub(crate) length: u32, pub(crate) handler: BuiltinHandler, } +/// The builtin iterators with a prototype of their own. Each `next` serves only +/// its family, so the discriminant also indexes their prototypes. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub(crate) enum IteratorFamily { + Array, + Map, + Set, + String, +} + pub(crate) struct BuiltinTable { defs: Vec>, /// Index-aligned constructability flags. `register` defaults to `false` @@ -55,12 +65,21 @@ pub(crate) struct BuiltinTable { number_prototype: Value, boolean_prototype: Value, error_prototypes: Vec<(BuiltinId, Value)>, + error_brand: Option, + throw_type_error: Option, + function_reified_brand: Option, symbol_iterator: Option, symbol_async_iterator: Option, symbol_to_string_tag: Option, symbol_match: Option, symbol_replace: Option, + symbol_match_all: Option, + symbol_search: Option, + symbol_split: Option, + symbol_is_concat_spreadable: Option, + symbol_has_instance: Option, symbol_species: Option, + symbol_to_primitive: Option, symbol_dispose: Option, symbol_async_dispose: Option, symbol_unscopables: Option, @@ -70,8 +89,15 @@ pub(crate) struct BuiltinTable { object_to_string: Option, regexp_prototype: Option, iterator_prototype: Option, + iterator_family_prototypes: Option<[Value; 4]>, + iterator_helper_prototype: Option, + wrap_for_valid_iterator_prototype: Option, + regexp_string_iterator_prototype: Option, async_iterator_prototype: Option, generator_prototype: Option, + generator_function_prototype: Option, + async_function_prototype: Option, + async_generator_function_prototype: Option, async_generator_prototype: Option, async_from_sync_iterator_prototype: Option, promise_resolver_targets: Option<(Value, Value)>, @@ -84,10 +110,15 @@ pub(crate) struct BuiltinTable { arraybuffer_constructor: Option, arraybuffer_prototype: Option, dataview_prototype: Option, + sharedarraybuffer_constructor: Option, sharedarraybuffer_prototype: Option, + wait_async_timeout: Option, disposable_stack_prototype: Option, async_disposable_stack_prototype: Option, date_prototype: Option, + temporal_prototypes: [Option; crate::temporal::TemporalKind::COUNT], + intl_prototypes: [Option; crate::intl::IntlKind::COUNT], + intl_fallback_symbol: Option, weak_ref_prototype: Option, finalization_registry_prototype: Option, map_prototype: Option, @@ -122,12 +153,21 @@ impl BuiltinTable { number_prototype, boolean_prototype, error_prototypes: Vec::new(), + error_brand: None, + throw_type_error: None, + function_reified_brand: None, symbol_iterator: None, symbol_async_iterator: None, symbol_to_string_tag: None, symbol_match: None, symbol_replace: None, + symbol_match_all: None, + symbol_search: None, + symbol_split: None, + symbol_is_concat_spreadable: None, + symbol_has_instance: None, symbol_species: None, + symbol_to_primitive: None, symbol_dispose: None, symbol_async_dispose: None, symbol_unscopables: None, @@ -137,8 +177,15 @@ impl BuiltinTable { object_to_string: None, regexp_prototype: None, iterator_prototype: None, + iterator_family_prototypes: None, + iterator_helper_prototype: None, + wrap_for_valid_iterator_prototype: None, + regexp_string_iterator_prototype: None, async_iterator_prototype: None, generator_prototype: None, + generator_function_prototype: None, + async_function_prototype: None, + async_generator_function_prototype: None, async_generator_prototype: None, async_from_sync_iterator_prototype: None, promise_resolver_targets: None, @@ -151,10 +198,15 @@ impl BuiltinTable { arraybuffer_constructor: None, arraybuffer_prototype: None, dataview_prototype: None, + sharedarraybuffer_constructor: None, sharedarraybuffer_prototype: None, + wait_async_timeout: None, disposable_stack_prototype: None, async_disposable_stack_prototype: None, date_prototype: None, + temporal_prototypes: [None; crate::temporal::TemporalKind::COUNT], + intl_prototypes: [None; crate::intl::IntlKind::COUNT], + intl_fallback_symbol: None, weak_ref_prototype: None, finalization_registry_prototype: None, map_prototype: None, @@ -244,6 +296,39 @@ impl BuiltinTable { pub(crate) fn symbol_replace(&self) -> Value { self.symbol_replace.expect("Symbol builtins install first") } + pub(crate) fn set_symbol_match_all(&mut self, symbol: Value) { + self.symbol_match_all = Some(symbol); + } + pub(crate) fn set_symbol_search(&mut self, symbol: Value) { + self.symbol_search = Some(symbol); + } + pub(crate) fn set_symbol_split(&mut self, symbol: Value) { + self.symbol_split = Some(symbol); + } + pub(crate) fn set_symbol_is_concat_spreadable(&mut self, symbol: Value) { + self.symbol_is_concat_spreadable = Some(symbol); + } + pub(crate) fn set_symbol_has_instance(&mut self, symbol: Value) { + self.symbol_has_instance = Some(symbol); + } + pub(crate) fn symbol_has_instance(&self) -> Value { + self.symbol_has_instance + .expect("Symbol builtins install first") + } + pub(crate) fn symbol_is_concat_spreadable(&self) -> Value { + self.symbol_is_concat_spreadable + .expect("Symbol builtins install first") + } + pub(crate) fn symbol_match_all(&self) -> Value { + self.symbol_match_all + .expect("Symbol builtins install first") + } + pub(crate) fn symbol_search(&self) -> Value { + self.symbol_search.expect("Symbol builtins install first") + } + pub(crate) fn symbol_split(&self) -> Value { + self.symbol_split.expect("Symbol builtins install first") + } pub(crate) fn set_symbol_dispose(&mut self, symbol: Value) { self.symbol_dispose = Some(symbol); } @@ -264,6 +349,13 @@ impl BuiltinTable { self.symbol_unscopables .expect("Symbol builtins install first") } + pub(crate) fn set_symbol_to_primitive(&mut self, symbol: Value) { + self.symbol_to_primitive = Some(symbol); + } + pub(crate) fn symbol_to_primitive(&self) -> Value { + self.symbol_to_primitive + .expect("Symbol builtins install first") + } pub(crate) fn set_symbol_species(&mut self, symbol: Value) { self.symbol_species = Some(symbol); } @@ -313,6 +405,32 @@ impl BuiltinTable { self.regexp_prototype .expect("RegExp builtins install their prototype") } + pub(crate) fn set_iterator_family_prototypes(&mut self, prototypes: [Value; 4]) { + self.iterator_family_prototypes = Some(prototypes); + } + pub(crate) fn iterator_family_prototype(&self, family: IteratorFamily) -> Value { + self.iterator_family_prototypes + .expect("collection builtins install the iterator prototypes")[family as usize] + } + pub(crate) fn set_iterator_helper_prototypes(&mut self, helper: Value, wrap: Value) { + self.iterator_helper_prototype = Some(helper); + self.wrap_for_valid_iterator_prototype = Some(wrap); + } + pub(crate) fn iterator_helper_prototype(&self) -> Value { + self.iterator_helper_prototype + .expect("Iterator builtins install the helper prototype") + } + pub(crate) fn wrap_for_valid_iterator_prototype(&self) -> Value { + self.wrap_for_valid_iterator_prototype + .expect("Iterator builtins install the wrapper prototype") + } + pub(crate) fn set_regexp_string_iterator_prototype(&mut self, prototype: Value) { + self.regexp_string_iterator_prototype = Some(prototype); + } + pub(crate) fn regexp_string_iterator_prototype(&self) -> Value { + self.regexp_string_iterator_prototype + .expect("RegExp builtins install the string iterator prototype") + } pub(crate) fn set_iterator_prototype(&mut self, prototype: Value) { self.iterator_prototype = Some(prototype); } @@ -327,6 +445,28 @@ impl BuiltinTable { self.async_iterator_prototype .expect("async iterator builtins install their prototype") } + pub(crate) fn set_function_kind_prototypes( + &mut self, + generator: Value, + asynchronous: Value, + async_generator: Value, + ) { + self.generator_function_prototype = Some(generator); + self.async_function_prototype = Some(asynchronous); + self.async_generator_function_prototype = Some(async_generator); + } + pub(crate) fn generator_function_prototype(&self) -> Value { + self.generator_function_prototype + .expect("Function builtins install the generator function prototype") + } + pub(crate) fn async_function_prototype(&self) -> Value { + self.async_function_prototype + .expect("Function builtins install the async function prototype") + } + pub(crate) fn async_generator_function_prototype(&self) -> Value { + self.async_generator_function_prototype + .expect("Function builtins install the async generator function prototype") + } pub(crate) fn set_generator_prototype(&mut self, prototype: Value) { self.generator_prototype = Some(prototype); } @@ -408,6 +548,27 @@ impl BuiltinTable { pub(crate) fn set_arraybuffer_prototype(&mut self, prototype: Value) { self.arraybuffer_prototype = Some(prototype); } + pub(crate) fn set_wait_async_timeout(&mut self, callback: Value) { + self.wait_async_timeout = Some(callback); + } + pub(crate) fn wait_async_timeout(&self) -> Value { + self.wait_async_timeout + .expect("Atomics installs before runtime execution") + } + pub(crate) fn set_sharedarraybuffer_constructor(&mut self, constructor: Value) { + self.sharedarraybuffer_constructor = Some(constructor); + } + pub(crate) fn sharedarraybuffer_constructor(&self) -> Value { + self.sharedarraybuffer_constructor + .expect("SharedArrayBuffer installs before runtime execution") + } + pub(crate) fn set_sharedarraybuffer_prototype(&mut self, prototype: Value) { + self.sharedarraybuffer_prototype = Some(prototype); + } + pub(crate) fn sharedarraybuffer_prototype(&self) -> Value { + self.sharedarraybuffer_prototype + .expect("SharedArrayBuffer installs before runtime execution") + } pub(crate) fn arraybuffer_prototype(&self) -> Value { self.arraybuffer_prototype .expect("ArrayBuffer builtins install their prototype") @@ -433,6 +594,30 @@ impl BuiltinTable { self.async_disposable_stack_prototype .expect("AsyncDisposableStack builtins install their prototype") } + pub(crate) fn set_temporal_prototype( + &mut self, + kind: crate::temporal::TemporalKind, + prototype: Value, + ) { + self.temporal_prototypes[kind.index()] = Some(prototype); + } + pub(crate) fn temporal_prototype(&self, kind: crate::temporal::TemporalKind) -> Value { + self.temporal_prototypes[kind.index()].expect("Temporal builtins install their prototypes") + } + /// `%Intl%.[[FallbackSymbol]]`. + pub(crate) fn set_intl_fallback_symbol(&mut self, symbol: Value) { + self.intl_fallback_symbol = Some(symbol); + } + pub(crate) fn intl_fallback_symbol(&self) -> Value { + self.intl_fallback_symbol + .expect("Intl builtins install the fallback symbol") + } + pub(crate) fn set_intl_prototype(&mut self, kind: crate::intl::IntlKind, prototype: Value) { + self.intl_prototypes[kind.index()] = Some(prototype); + } + pub(crate) fn intl_prototype(&self, kind: crate::intl::IntlKind) -> Value { + self.intl_prototypes[kind.index()].expect("Intl builtins install their prototypes") + } pub(crate) fn set_date_prototype(&mut self, prototype: Value) { self.date_prototype = Some(prototype); } @@ -574,7 +759,9 @@ impl BuiltinTable { match &mut heap[prototype_index] { HeapEntry::Object { properties, .. } | HeapEntry::Array { properties, .. } - | HeapEntry::Date { properties, .. } => { + | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } => { properties.insert( crate::PropertyKey::Named(EcmaString::encode("constructor")), constructor_property, @@ -619,6 +806,36 @@ impl BuiltinTable { }; self.error_prototypes.push((id, prototype)); } + pub(crate) fn set_function_reified_brand(&mut self, brand: Value) { + self.function_reified_brand = Some(brand); + } + + /// Marks a function whose `length` and `name` are stored properties. + pub(crate) fn function_reified_key(&self) -> crate::PropertyKey { + crate::PropertyKey::Private(heap_index( + self.function_reified_brand + .expect("Object builtins install the brand"), + ) as u32) + } + + pub(crate) fn set_throw_type_error(&mut self, thrower: Value) { + self.throw_type_error = Some(thrower); + } + + pub(crate) fn throw_type_error(&self) -> Value { + self.throw_type_error + .expect("Object builtins install %ThrowTypeError%") + } + + pub(crate) fn set_error_brand(&mut self, brand: Value) { + self.error_brand = Some(brand); + } + /// The hidden key standing in for the `[[ErrorData]]` internal slot. + pub(crate) fn error_brand_key(&self) -> crate::PropertyKey { + crate::PropertyKey::Private(heap_index( + self.error_brand.expect("Error builtins install the brand"), + ) as u32) + } pub(crate) fn id_named(&self, name: &str) -> Option { self.defs .iter() @@ -636,12 +853,21 @@ impl BuiltinTable { number_prototype, boolean_prototype, error_prototypes, + error_brand, + throw_type_error, + function_reified_brand, symbol_iterator, symbol_async_iterator, symbol_to_string_tag, symbol_match, symbol_replace, + symbol_match_all, + symbol_search, + symbol_split, + symbol_is_concat_spreadable, + symbol_has_instance, symbol_species, + symbol_to_primitive, symbol_dispose, symbol_async_dispose, symbol_unscopables, @@ -651,8 +877,15 @@ impl BuiltinTable { object_to_string, regexp_prototype, iterator_prototype, + iterator_family_prototypes, + iterator_helper_prototype, + wrap_for_valid_iterator_prototype, + regexp_string_iterator_prototype, async_iterator_prototype, generator_prototype, + generator_function_prototype, + async_function_prototype, + async_generator_function_prototype, async_generator_prototype, async_from_sync_iterator_prototype, promise_resolver_targets, @@ -665,10 +898,15 @@ impl BuiltinTable { arraybuffer_constructor, arraybuffer_prototype, dataview_prototype, + sharedarraybuffer_constructor, sharedarraybuffer_prototype, + wait_async_timeout, disposable_stack_prototype, async_disposable_stack_prototype, date_prototype, + temporal_prototypes, + intl_prototypes, + intl_fallback_symbol, weak_ref_prototype, finalization_registry_prototype, map_prototype, @@ -697,13 +935,25 @@ impl BuiltinTable { for (_, value) in error_prototypes { visit(*value); } + for value in iterator_family_prototypes.iter().flatten() { + visit(*value); + } for value in [ + *error_brand, + *throw_type_error, + *function_reified_brand, *symbol_iterator, *symbol_async_iterator, *symbol_to_string_tag, *symbol_match, *symbol_replace, + *symbol_match_all, + *symbol_search, + *symbol_split, + *symbol_is_concat_spreadable, + *symbol_has_instance, *symbol_species, + *symbol_to_primitive, *symbol_dispose, *symbol_async_dispose, *symbol_unscopables, @@ -713,8 +963,14 @@ impl BuiltinTable { *object_to_string, *regexp_prototype, *iterator_prototype, + *iterator_helper_prototype, + *wrap_for_valid_iterator_prototype, + *regexp_string_iterator_prototype, *async_iterator_prototype, *generator_prototype, + *generator_function_prototype, + *async_function_prototype, + *async_generator_function_prototype, *async_generator_prototype, *async_from_sync_iterator_prototype, *promise_all_target, @@ -724,7 +980,9 @@ impl BuiltinTable { *arraybuffer_constructor, *arraybuffer_prototype, *dataview_prototype, + *sharedarraybuffer_constructor, *sharedarraybuffer_prototype, + *wait_async_timeout, *disposable_stack_prototype, *async_disposable_stack_prototype, *date_prototype, @@ -750,6 +1008,9 @@ impl BuiltinTable { for value in typedarray_constructors .iter() .chain(typedarray_prototypes.iter()) + .chain(temporal_prototypes.iter()) + .chain(intl_prototypes.iter()) + .chain(std::iter::once(intl_fallback_symbol)) .flatten() { visit(*value); @@ -772,7 +1033,11 @@ pub(crate) struct Intrinsics { pub(crate) builtins: BuiltinTable, } impl Intrinsics { - pub(crate) fn initialize(heap: &mut Vec, timers_available: bool) -> Self { + pub(crate) fn initialize( + heap: &mut Vec, + timers_available: bool, + script_compiler: bool, + ) -> Self { let object_prototype = push( heap, HeapEntry::Object { @@ -793,9 +1058,10 @@ impl Intrinsics { length_writable: true, }, ); - let string_prototype = ordinary_prototype(heap, object_prototype); - let number_prototype = ordinary_prototype(heap, object_prototype); - let boolean_prototype = ordinary_prototype(heap, object_prototype); + let empty_string = push(heap, HeapEntry::String(EcmaString::encode(""))); + let string_prototype = wrapper_prototype(heap, object_prototype, empty_string); + let number_prototype = wrapper_prototype(heap, object_prototype, Value::int32(0)); + let boolean_prototype = wrapper_prototype(heap, object_prototype, Value::FALSE); let mut globals = BTreeMap::new(); let mut builtins = BuiltinTable::new( object_prototype, @@ -805,7 +1071,13 @@ impl Intrinsics { number_prototype, boolean_prototype, ); - builtins::install(heap, &mut globals, &mut builtins, timers_available); + builtins::install( + heap, + &mut globals, + &mut builtins, + timers_available, + script_compiler, + ); crate::host_objects::install(heap, &mut globals, &mut builtins); Self { globals, @@ -867,6 +1139,22 @@ impl Intrinsics { builtins.for_each_value(visit); } } +/// A prototype that is itself a wrapper around `primitive`. +fn wrapper_prototype( + heap: &mut Vec, + object_prototype: Value, + primitive: Value, +) -> Value { + push( + heap, + HeapEntry::Object { + properties: PropertyMap::default(), + prototype: Some(object_prototype), + extensible: true, + boxed_primitive: Some(primitive), + }, + ) +} fn ordinary_prototype(heap: &mut Vec, object_prototype: Value) -> Value { push( heap, @@ -878,32 +1166,41 @@ fn ordinary_prototype(heap: &mut Vec, object_prototype: Value) -> Val }, ) } +/// The `length` and `name` every built-in function owns, in that order. +pub(crate) fn native_function_properties( + heap: &mut Vec, + name: &'static str, + length: u32, +) -> PropertyMap { + let name_value = push(heap, HeapEntry::String(EcmaString::encode(name))); + function_properties(length, name_value) +} +/// `length` then `name`, the order a built-in function creates them in. +pub(crate) fn function_properties(length: u32, name: Value) -> PropertyMap { + let mut properties = PropertyMap::default(); + for (key, value) in [ + ("length", crate::number_value(f64::from(length))), + ("name", name), + ] { + properties.insert( + crate::PropertyKey::Named(EcmaString::encode(key)), + crate::Property::Data { + value, + writable: false, + enumerable: false, + configurable: true, + }, + ); + } + properties +} pub(crate) fn native_function( heap: &mut Vec, id: BuiltinId, name: &'static str, length: u32, ) -> Value { - let name_value = push(heap, HeapEntry::String(EcmaString::encode(name))); - let mut properties = PropertyMap::default(); - properties.insert( - crate::PropertyKey::Named(EcmaString::encode("length")), - crate::Property::Data { - value: crate::number_value(f64::from(length)), - writable: false, - enumerable: false, - configurable: true, - }, - ); - properties.insert( - crate::PropertyKey::Named(EcmaString::encode("name")), - crate::Property::Data { - value: name_value, - writable: false, - enumerable: false, - configurable: true, - }, - ); + let properties = native_function_properties(heap, name, length); push( heap, HeapEntry::native_function(NativeCallable::Builtin(id), properties, None), @@ -944,7 +1241,8 @@ impl<'a, H: Host> Machine<'a, H> { let handler = self.intrinsics.builtins.get(id).handler; let previous_id = self.current_builtin_id.replace(id); let previous_new_target = std::mem::replace(&mut self.current_new_target, new_target); - let outcome = handler(self, this_value, arguments, constructing); + let outcome = + self.with_temporaries(|machine| handler(machine, this_value, arguments, constructing)); self.current_builtin_id = previous_id; self.current_new_target = previous_new_target; outcome @@ -969,38 +1267,37 @@ impl<'a, H: Host> Machine<'a, H> { HeapEntry::BigInt(_) => "BigInt", HeapEntry::PrivateName { .. } => "Symbol", HeapEntry::Date { .. } => "Date", - HeapEntry::Object { .. } if self.is_error_object(index)? => "Error", - HeapEntry::Proxy { .. } => "Object", + HeapEntry::Object { + boxed_primitive: Some(primitive), + .. + } => match self.object_to_string_tag(*primitive)? { + tag @ ("Boolean" | "Number" | "String") => tag, + _ => "Object", + }, + HeapEntry::Object { properties, .. } + if properties + .get(&self.intrinsics.builtins.error_brand_key()) + .is_some() => + { + "Error" + } + HeapEntry::Proxy { .. } if builtins::is_array(self, value)? => "Array", + HeapEntry::Proxy { .. } if self.is_callable(value)? => "Function", HeapEntry::ProxyRevoker { .. } => "Function", _ => "Object", }) } } } - fn is_error_object(&self, mut index: usize) -> Result { - for _ in 0..=self.heap.len() { - let value = Value::heap_ref( - bamts_native::SlotId::from_parts( - crate::RUNTIME_HEAP_SEGMENT, - u32::try_from(index + 1).expect("heap index fits in u32"), - ) - .expect("heap index is nonzero"), - ); - if self - .intrinsics - .builtins - .error_prototypes - .iter() - .any(|(_, prototype)| *prototype == value) - { - return Ok(true); - } - match self.prototype_index(index)? { - Some(next) => index = next, - None => return Ok(false), - } + /// The own `message` data property of a thrown object. Like constructor + /// resolution, this runs no user code, so an accessor yields nothing. + pub(crate) fn resolve_thrown_message(&self, value: Value) -> Option { + let index = self.runtime_slot(value).ok().flatten()?; + match self.own_get_ascii(index, "message")? { + Found::Value(message) => self.string_value(message), + Found::Text(message) => Some(message), + _ => None, } - Ok(false) } /// Resolves the constructor name of a thrown value by walking its @@ -1173,6 +1470,58 @@ mod tests { #[derive(Default)] struct TestHost; impl Host for TestHost {} + #[test] + fn wrapper_prototypes_hold_their_default_primitive() { + let program = module(); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + let (number, boolean, string) = ( + machine.intrinsics.number_prototype, + machine.intrinsics.boolean_prototype, + machine.intrinsics.string_prototype, + ); + assert_eq!( + machine.unbox_primitive_or_self(number).unwrap(), + Value::int32(0) + ); + assert_eq!( + machine.unbox_primitive_or_self(boolean).unwrap(), + Value::FALSE + ); + assert_eq!( + machine.get_named_property(string, "length").unwrap(), + Value::int32(0) + ); + } + #[test] + fn built_in_tags_follow_the_object_rather_than_its_prototype_chain() { + let program = module(); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + let wrapped = machine.value_to_object(Value::int32(1)).unwrap(); + assert_eq!(machine.object_to_string_tag(wrapped).unwrap(), "Number"); + let flag = machine.value_to_object(Value::TRUE).unwrap(); + assert_eq!(machine.object_to_string_tag(flag).unwrap(), "Boolean"); + let error_prototype = machine.intrinsics.global("Error").unwrap(); + let error_prototype = machine + .get_named_property(error_prototype, "prototype") + .unwrap(); + assert_eq!( + machine.object_to_string_tag(error_prototype).unwrap(), + "Object" + ); + let array = machine + .construct_value(machine.intrinsics.global("Array").unwrap(), &[]) + .unwrap(); + let handler = machine + .construct_value(machine.intrinsics.global("Object").unwrap(), &[]) + .unwrap(); + let proxy_constructor = machine.intrinsics.global("Proxy").unwrap(); + let proxy = machine + .construct_value(proxy_constructor, &[array, handler]) + .unwrap(); + assert_eq!(machine.object_to_string_tag(proxy).unwrap(), "Array"); + } fn module() -> Program { let code = Module::new( vec![Constant::String(EcmaString::encode(""))], @@ -1228,7 +1577,7 @@ mod tests { Value::int32(5), Value::int32(6), ); - let roots: Vec = (101..=145).map(Value::int32).collect(); + let roots: Vec = (101..=147).map(Value::int32).collect(); table.typedarray_constructor = Some(roots[0]); table.promise_capability_executor = Some(roots[1]); table.promise_finally_value = Some(roots[2]); @@ -1254,6 +1603,8 @@ mod tests { table.typedarray_constructors[offset] = Some(roots[21 + offset]); table.typedarray_prototypes[offset] = Some(roots[33 + offset]); } + table.sharedarraybuffer_constructor = Some(roots[45]); + table.wait_async_timeout = Some(roots[46]); let mut visited = Vec::new(); table.for_each_value(|value| visited.push(value)); for root in roots { @@ -1743,8 +2094,6 @@ mod tests { let machine = Machine::new(&module, &mut host, Limits::default()); let names = [ "eval", - "Intl", - "Iterator", "AsyncIterator", "URL", "URLSearchParams", @@ -1784,8 +2133,8 @@ mod tests { "ArrayBuffer is installed before globals" ); assert!( - machine.intrinsics.global("SharedArrayBuffer").is_none(), - "SharedArrayBuffer waits for atomics install" + machine.intrinsics.global("SharedArrayBuffer").is_some(), + "SharedArrayBuffer is installed by the atomics surface" ); let _ = machine.intrinsics.builtins.arraybuffer_constructor(); let _ = machine.intrinsics.builtins.arraybuffer_prototype(); diff --git a/crates/bamts-runtime/src/lib.rs b/crates/bamts-runtime/src/lib.rs index 2584b5d1..0f163372 100644 --- a/crates/bamts-runtime/src/lib.rs +++ b/crates/bamts-runtime/src/lib.rs @@ -26,8 +26,7 @@ use std::collections::BTreeSet; use std::collections::VecDeque; use std::error::Error; use std::fmt; -use std::sync::atomic::{AtomicUsize, Ordering as AtomicOrdering}; -use std::sync::{Arc, Condvar, Mutex}; +use std::sync::Arc; use crate::vm::explicit_resource::DisposalStackState; use crate::vm::generator_async::GeneratorCompletion; @@ -44,10 +43,12 @@ use intrinsics::builtins::typedarray_all::{self, ElementKind, LengthSlot}; mod external_modules; mod gc; mod host_objects; +pub mod intl; mod intrinsics; mod native; #[cfg(test)] mod proxy_reflect_integration; +mod temporal; mod vm; pub(crate) use intrinsics::builtins; @@ -56,6 +57,8 @@ pub use native::{ run_linked_program_tiered, run_linked_program_with_cancel, }; +/// Callback results a builtin keeps rooted, for the locals it still holds. +const RECENT_CALLBACK_RESULTS: usize = 32; const RUNTIME_HEAP_SEGMENT: u16 = 1; /// The observable result of a terminated execution. @@ -96,6 +99,8 @@ pub struct Limits { pub max_microtasks: usize, /// Ceiling on live timers armed at once. pub max_timers: usize, + /// This agent's `[[CanBlock]]`: whether `Atomics.wait` may suspend it. + pub agent_can_suspend: bool, } impl Default for Limits { @@ -111,6 +116,7 @@ impl Default for Limits { max_dynamic_modules: 1 << 10, max_microtasks: 1 << 20, max_timers: 1 << 20, + agent_can_suspend: true, } } } @@ -330,6 +336,9 @@ pub enum RuntimeErrorKind { /// names the constructor, or when the value is not an object with a /// resolvable constructor. constructor_name: Option, + /// The thrown object's own `message` data property, read at the same + /// boundary, so a failure can say more than which constructor threw. + message: Option, }, FuelExhausted { limit: u64, @@ -422,6 +431,7 @@ impl fmt::Display for RuntimeError { value, origin, constructor_name, + .. } => match constructor_name { Some(name) => write!( formatter, @@ -684,6 +694,14 @@ enum IterationKind { Key, Value, Entry, + /// A `%StringIteratorPrototype%` instance; `source` is its code points. + StringCodePoint, + /// A `%RegExpStringIteratorPrototype%` instance; `source` is its hidden state. + RegExpString, + /// An `Iterator` helper; `source` is its hidden state. + IteratorHelper(intrinsics::builtins::iterator_helpers::Operation), + /// A `%WrapForValidIteratorPrototype%` instance; `source` is its record. + WrapForValid, } #[derive(Clone, Copy, Debug)] @@ -1219,6 +1237,24 @@ enum HeapEntry { prototype: Option, extensible: bool, }, + /// Instances of the `Temporal` classes. The boxed record holds no `Value`, + /// so tracing needs only the properties and prototype. + Temporal { + value: Box, + properties: PropertyMap, + prototype: Option, + extensible: bool, + }, + /// Instances of the `Intl` classes. The boxed record holds no `Value`; + /// `bound` is the instance's one cached function, the getter-created + /// `format` or `compare`. + Intl { + value: Box, + bound: Option, + properties: PropertyMap, + prototype: Option, + extensible: bool, + }, /// Entries retain insertion positions. Deletion tombstones an entry so /// live iterators keep a stable order cursor; the index retains only /// integer positions and therefore introduces no GC roots. @@ -1450,10 +1486,14 @@ impl HeapEntry { ) .saturating_add(properties.charge_bytes()) .saturating_add(1), - Self::NativeFunction { callable, .. } => match callable { + Self::NativeFunction { + callable, + properties, + .. + } => properties.charge_bytes().saturating_add(match callable { NativeCallable::Builtin(_) => 1, NativeCallable::Bound(bound) => bound.arguments.len().saturating_add(1), - }, + }), Self::Proxy { .. } => 1, Self::ProxyRevoker { properties, .. } => properties.charge_bytes().saturating_add(1), Self::Generator { state, .. } => match state { @@ -1512,6 +1552,8 @@ impl HeapEntry { | Self::ExternalModuleNamespace { .. } | Self::ProcessEnv { .. } | Self::Date { .. } + | Self::Temporal { .. } + | Self::Intl { .. } | Self::BuiltinIterator { .. } | Self::Iterator { .. } | Self::Promise { .. } @@ -1784,40 +1826,6 @@ struct TimerRecord { interval_delay: Option, } -#[allow(dead_code)] -pub(crate) enum ExternalJob { - ResolveWaitAsync { promise: u64, result: &'static str }, -} - -pub(crate) struct ExternalInbox { - queue: Mutex>, - signal: Condvar, - pending: AtomicUsize, -} - -#[allow(dead_code)] -impl ExternalInbox { - fn new() -> Arc { - Arc::new(Self { - queue: Mutex::new(VecDeque::new()), - signal: Condvar::new(), - pending: AtomicUsize::new(0), - }) - } - - pub(crate) fn reserve(&self) { - self.pending.fetch_add(1, AtomicOrdering::AcqRel); - } - - pub(crate) fn push(&self, job: ExternalJob) { - self.queue - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) - .push_back(job); - self.signal.notify_one(); - } -} - /// The production bytecode interpreter. pub struct Machine<'a, H: Host> { program: Option<&'a Program>, @@ -1826,9 +1834,33 @@ pub struct Machine<'a, H: Host> { limits: Limits, gc: gc::GcState, native_roots: Vec>, + /// Recently compiled patterns. `exec` runs once per match of a global search, + /// so recompiling on every call would make it cost a parse per match. + /// Objects being joined, innermost last. A cycle passes through native code + /// on every level, so no activation limit would stop it. + array_join_stack: Vec, + regexp_cache: Vec<( + EcmaString, + EcmaString, + std::rc::Rc, + )>, /// Values kept strongly reachable for the current ECMAScript job by /// WeakRef construction/deref. Cleared exactly once at the outer job seam. kept_alive: Vec, + /// What running builtins hold in Rust locals: everything they allocated or + /// got back from user code. Calling user code can reach a collection, so + /// these stay rooted until the builtin returns. + builtin_temporaries: Vec, + /// The latest results of `call_callback`, for loops that use each result + /// once: `reduce` holds one at a time, and keeping them all would hold every + /// intermediate. + callback_results: Vec, + /// Where the running builtin's results start in `callback_results`. + results_floor: usize, + /// A builtin is running, and not user code it called. + in_builtin: bool, + /// Microtask jobs that have left the queue and are still running. + running_jobs: Vec, frames: Vec, heap: Vec, slot_bytes: Vec, @@ -1864,7 +1896,9 @@ pub struct Machine<'a, H: Host> { async_boundaries: Vec, pending_async_suspend: Option<(Value, SuspendedActivation)>, microtasks: VecDeque, - external_inbox: Arc, + /// Every pending `Atomics.waitAsync`, by the id in its `Waiter::Async`. + async_waiters: BTreeMap, + next_async_waiter: u64, finalization_cleanup_queue: VecDeque, microtask_drain_active: bool, next_timer_id: Option, @@ -1885,6 +1919,8 @@ pub struct Machine<'a, H: Host> { registry: ModuleRegistry, /// First machine-wide module ID reserved for host-compiled script modules. dynamic_base: usize, + /// The last global a read found no binding for, which names it in a report. + missing_global: Option, /// Host-compiled programs retained for the machine lifetime so their /// closures remain executable after the originating Script object dies. dynamic: Vec, @@ -2060,26 +2096,13 @@ pub fn run_with_telemetry( (outcome, telemetry) } -/// Best-effort process peak resident set size, in KiB, from -/// `/proc/self/status` `VmHWM`. Returns `None` when the file or field is -/// unavailable (non-Linux hosts, sandboxes). +/// Best-effort peak memory of this process, in KiB: the resident-set peak on +/// Linux and the physical-footprint peak on macOS. Returns `None` when the host +/// cannot say, such as a sandbox without `/proc`. fn peak_rss_kb() -> Option { - let content = std::fs::read_to_string("/proc/self/status").ok()?; - parse_peak_rss_kb(&content) -} - -/// Parses the process peak resident set size, in KiB, from a -/// `/proc/self/status`-shaped document. Selects the `VmHWM:` field (the peak -/// resident set) rather than `VmRSS:` (the current resident set), returning -/// `None` when the field or its numeric value is missing. -fn parse_peak_rss_kb(status: &str) -> Option { - status.lines().find_map(|line| { - line.strip_prefix("VmHWM:")? - .split_whitespace() - .next()? - .parse() - .ok() - }) + bamts_platform::peak_memory_bytes(std::process::id()) + .ok() + .map(|bytes| bytes / 1024) } impl<'a, H: Host> Machine<'a, H> { @@ -2143,11 +2166,12 @@ impl<'a, H: Host> Machine<'a, H> { let live_registers = frame.registers.len(); let mut heap = Vec::new(); let timers_available = host.timers().is_some(); - let mut intrinsics = intrinsics::Intrinsics::::initialize(&mut heap, timers_available); + let script_compiler = host.script_compiler().is_some(); + let mut intrinsics = + intrinsics::Intrinsics::::initialize(&mut heap, timers_available, script_compiler); let global_object = intrinsics .global("globalThis") .expect("host objects install globalThis"); - let script_compiler = host.script_compiler().is_some(); let installed_external = external_modules::install( &mut heap, &mut intrinsics.builtins, @@ -2207,7 +2231,14 @@ impl<'a, H: Host> Machine<'a, H> { gc: gc::GcState::new(&limits), limits, native_roots: Vec::new(), + regexp_cache: Vec::new(), + array_join_stack: Vec::new(), kept_alive: Vec::new(), + builtin_temporaries: Vec::new(), + callback_results: Vec::new(), + results_floor: 0, + in_builtin: false, + running_jobs: Vec::new(), fuel, frames: vec![frame], heap, @@ -2226,7 +2257,8 @@ impl<'a, H: Host> Machine<'a, H> { async_boundaries: Vec::new(), pending_async_suspend: None, microtasks: VecDeque::new(), - external_inbox: ExternalInbox::new(), + async_waiters: BTreeMap::new(), + next_async_waiter: 0, finalization_cleanup_queue: VecDeque::new(), microtask_drain_active: false, next_timer_id: Some(1), @@ -2269,6 +2301,7 @@ impl<'a, H: Host> Machine<'a, H> { }, dynamic_base: program.map_or(1, |program| program.modules().len()), dynamic: Vec::new(), + missing_global: None, template_objects, template_sites: BTreeMap::new(), current_builtin_id: None, @@ -2394,6 +2427,7 @@ impl<'a, H: Host> Machine<'a, H> { value: exception.value, origin: exception.origin, constructor_name: None, + message: None, })); } } @@ -2614,54 +2648,6 @@ impl<'a, H: Host> Machine<'a, H> { !self.timers.is_empty() } - #[allow(dead_code)] - pub(crate) fn external_inbox(&self) -> Arc { - Arc::clone(&self.external_inbox) - } - - fn drain_external_jobs(&mut self) -> Result<(), RuntimeError> { - loop { - let job = self - .external_inbox - .queue - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()) - .pop_front(); - let Some(job) = job else { - return Ok(()); - }; - self.external_inbox - .pending - .fetch_sub(1, AtomicOrdering::AcqRel); - match job { - ExternalJob::ResolveWaitAsync { promise, result } => { - let value = self - .allocate(HeapEntry::String(EcmaString::encode(result))) - .map_err(|kind| self.checkpoint_error(kind))?; - self.fulfill_promise(Value::from_bits(promise), value) - .map_err(|kind| self.checkpoint_error(kind))?; - } - } - } - } - - fn wait_for_external_job(&self) { - let queue = self - .external_inbox - .queue - .lock() - .unwrap_or_else(|poisoned| poisoned.into_inner()); - drop( - self.external_inbox - .signal - .wait_while(queue, |queue| { - queue.is_empty() - && self.external_inbox.pending.load(AtomicOrdering::Acquire) != 0 - }) - .unwrap_or_else(|poisoned| poisoned.into_inner()), - ); - } - /// Drives the automatic event loop to quiescence after synchronous /// evaluation. /// @@ -2681,7 +2667,6 @@ impl<'a, H: Host> Machine<'a, H> { /// exhausted work fails with [`RuntimeErrorKind::FuelExhausted`]. pub(crate) fn run_to_quiescence(&mut self) -> Result<(), RuntimeError> { loop { - self.drain_external_jobs()?; self.drain_microtasks_automatic()?; self.check_cancel() .map_err(|kind| self.checkpoint_error(kind))?; @@ -2705,14 +2690,11 @@ impl<'a, H: Host> Machine<'a, H> { value: exception.value, origin: exception.origin, constructor_name: None, + message: None, })); } continue; } - if self.external_inbox.pending.load(AtomicOrdering::Acquire) != 0 { - self.wait_for_external_job(); - continue; - } return Ok(()); } } @@ -2842,6 +2824,13 @@ impl<'a, H: Host> Machine<'a, H> { let Some(live_record) = self.timers.get(&id) else { return Ok(()); }; + // A bare number comes from a script, and an internal timeout is not its + // to cancel. Only the runtime holds that timer's handle. + if request_generation.is_none() + && live_record.callback == self.intrinsics.builtins.wait_async_timeout() + { + return Ok(()); + } if let Some(request_generation) = request_generation && live_record.generation != request_generation { @@ -2906,7 +2895,6 @@ impl<'a, H: Host> Machine<'a, H> { /// throws settle their derived promises. Throws from `queueMicrotask` /// callbacks are returned in [`MicrotaskDrain::uncaught`]. pub fn drain_microtasks(&mut self) -> Result { - self.drain_external_jobs()?; self.drain_microtasks_core(MicrotaskExceptionPolicy::CollectAndContinue) } @@ -2923,6 +2911,7 @@ impl<'a, H: Host> Machine<'a, H> { value: exception.value, origin: exception.origin, constructor_name: None, + message: None, })) } @@ -2959,7 +2948,9 @@ impl<'a, H: Host> Machine<'a, H> { report.executed = report.executed.saturating_add(1); let is_finalization_cleanup = matches!(&queued.job, MicrotaskJob::FinalizationCleanup { .. }); + self.running_jobs.push(queued.job.clone()); let job_result = self.execute_microtask_job(queued.job); + self.running_jobs.pop(); self.clear_kept_alive_for_job(); let Some(exception) = job_result.map_err(|kind| self.checkpoint_error(kind))? else { @@ -3308,6 +3299,7 @@ impl<'a, H: Host> Machine<'a, H> { value, origin, constructor_name: None, + message: None, }, ); self.store_async_module_completion(pending.record, Some(Err(error.clone()))) @@ -3486,13 +3478,16 @@ impl<'a, H: Host> Machine<'a, H> { target: Value, record: Value, ) -> Result { + let name = self + .allocate(HeapEntry::String(EcmaString::encode(""))) + .map_err(EvalFailure::Runtime)?; self.allocate(HeapEntry::native_function( NativeCallable::Bound(Box::new(BoundCallable { target, this_value: Value::UNDEFINED, arguments: vec![record], })), - PropertyMap::default(), + intrinsics::function_properties(1, name), None, )) .map_err(EvalFailure::Runtime) @@ -3928,6 +3923,7 @@ impl<'a, H: Host> Machine<'a, H> { }, ); } + properties.insert(self.intrinsics.builtins.error_brand_key(), error_brand()); self.allocate(HeapEntry::Object { properties, prototype: Some(self.intrinsics.error_prototype(id)), @@ -4397,9 +4393,9 @@ impl<'a, H: Host> Machine<'a, H> { } } - /// Populates the `constructor_name` field of a - /// [`RuntimeErrorKind::UncaughtThrow`] when the throw originated from - /// guest bytecode ([`ThrowOrigin::Bytecode`]). Engine-originated throws + /// Populates the `message` field of a [`RuntimeErrorKind::UncaughtThrow`], + /// and its `constructor_name` field when the throw originated from guest + /// bytecode ([`ThrowOrigin::Bytecode`]). Engine-originated throws /// (`TypeError`, `RangeError`, etc.) already name their constructor in /// the [`ThrowOrigin`] variant and are left as `None`. /// @@ -4412,10 +4408,21 @@ impl<'a, H: Host> Machine<'a, H> { value, origin, constructor_name, + message, } = &mut error.kind else { return; }; + if message.is_none() { + *message = match origin { + ThrowOrigin::ReferenceError { + operation: "global is not defined", + } => self.missing_global.as_ref().map(|name| { + EcmaString::encode(&format!("{} is not defined", name.to_utf8_lossy())) + }), + _ => self.resolve_thrown_message(*value), + }; + } if *origin != ThrowOrigin::Bytecode { return; } @@ -5079,6 +5086,7 @@ impl<'a, H: Host> Machine<'a, H> { value, origin, constructor_name: None, + message: None, }, ), Err(EvalFailure::Runtime(kind)) => self.program_error(module, kind), @@ -5736,14 +5744,17 @@ impl<'a, H: Host> Machine<'a, H> { captures, } => match self.read_captures(frame_index, captures.get(), function) { Ok(captures) => { + let properties = self + .closure_properties(module_id, function) + .map_err(|kind| self.error_at(kind, function_index, pc))?; let value = self .allocate(HeapEntry::Function { module: module_id, function, captures, context: self.frames[frame_index].context, - properties: PropertyMap::default(), - prototype: Some(self.intrinsics.function_prototype), + properties, + prototype: Some(self.closure_prototype(module_id, function)), extensible: true, }) .map_err(|kind| self.error_at(kind, function_index, pc))?; @@ -5755,7 +5766,7 @@ impl<'a, H: Host> Machine<'a, H> { Instruction::GetProperty { dst, object, key } => { let object = self.read_register(frame_index, object.get()); let key_value = self.read_register(frame_index, key.get()); - let key = match self.to_property_key(key_value) { + let key = match self.computed_property_key(object, key_value) { Ok(key) => key, Err(failure) => { self.resolve_failure(failure, pc)?; @@ -5774,7 +5785,7 @@ impl<'a, H: Host> Machine<'a, H> { let object = self.read_register(frame_index, object.get()); let value = self.read_register(frame_index, value.get()); let key_value = self.read_register(frame_index, key.get()); - let key = match self.to_property_key(key_value) { + let key = match self.computed_property_key(object, key_value) { Ok(key) => key, Err(failure) => { self.resolve_failure(failure, pc)?; @@ -5783,6 +5794,9 @@ impl<'a, H: Host> Machine<'a, H> { }; match self.internal_set(object, key, value, object) { Ok(true) => self.frames[frame_index].pc = pc + 1, + Ok(false) if self.is_sloppy(module_id, function_index) => { + self.frames[frame_index].pc = pc + 1; + } Ok(false) => { self.resolve_failure( EvalFailure::Throw(ThrowOrigin::TypeError { @@ -5797,14 +5811,17 @@ impl<'a, H: Host> Machine<'a, H> { Instruction::DeleteProperty { dst, object, key } => { let object = self.read_register(frame_index, object.get()); let key_value = self.read_register(frame_index, key.get()); - let key = match self.to_property_key(key_value) { + let key = match self.computed_property_key(object, key_value) { Ok(key) => key, Err(failure) => { self.resolve_failure(failure, pc)?; continue; } }; - match self.internal_delete(object, &key) { + let deleted = self.internal_delete(object, &key).and_then(|deleted| { + self.deletion_result(module_id, function_index, deleted) + }); + match deleted { Ok(deleted) => { self.write_register(frame_index, dst.get(), Value::boolean(deleted)); self.frames[frame_index].pc = pc + 1; @@ -5821,7 +5838,7 @@ impl<'a, H: Host> Machine<'a, H> { let object = self.read_register(frame_index, object.get()); let accessor = self.read_register(frame_index, accessor.get()); let key_value = self.read_register(frame_index, key.get()); - let key = match self.to_property_key(key_value) { + let key = match self.observable_property_key(key_value) { Ok(key) => key, Err(failure) => { self.resolve_failure(failure, pc)?; @@ -5837,7 +5854,7 @@ impl<'a, H: Host> Machine<'a, H> { let object = self.read_register(frame_index, object.get()); let value = self.read_register(frame_index, value.get()); let key_value = self.read_register(frame_index, key.get()); - let key = match self.to_property_key(key_value) { + let key = match self.observable_property_key(key_value) { Ok(key) => key, Err(failure) => { self.resolve_failure(failure, pc)?; @@ -5958,7 +5975,8 @@ impl<'a, H: Host> Machine<'a, H> { }, Instruction::StoreGlobal { name, value } => { let value = self.read_register(frame_index, value.get()); - match self.store_global(module_id, name, value) { + let sloppy = self.is_sloppy(module_id, function_index); + match self.store_global(module_id, name, value, sloppy) { Ok(()) => self.frames[frame_index].pc = pc + 1, Err(failure) => self.resolve_failure(failure, pc)?, } @@ -5977,9 +5995,15 @@ impl<'a, H: Host> Machine<'a, H> { } } Instruction::LoadThis { dst } => { - let value = self.frames[frame_index].this_value; - self.write_register(frame_index, dst.get(), value); - self.frames[frame_index].pc = pc + 1; + let this_value = self.frames[frame_index].this_value; + match self.bind_this(module_id, function_index, this_value) { + Ok(value) => { + self.frames[frame_index].this_value = value; + self.write_register(frame_index, dst.get(), value); + self.frames[frame_index].pc = pc + 1; + } + Err(failure) => self.resolve_failure(failure, pc)?, + } } Instruction::LoadArguments { dst } => { let value = self.materialize_arguments(frame_index, function_index, pc)?; @@ -6326,7 +6350,7 @@ impl<'a, H: Host> Machine<'a, H> { let home = self.read_register(frame_index, home.get()); let receiver = self.read_register(frame_index, receiver.get()); let key_value = self.read_register(frame_index, key.get()); - let key = match self.to_property_key(key_value) { + let key = match self.observable_property_key(key_value) { Ok(key) => key, Err(failure) => { self.resolve_failure(failure, pc)?; @@ -6350,7 +6374,7 @@ impl<'a, H: Host> Machine<'a, H> { let home = self.read_register(frame_index, home.get()); let receiver = self.read_register(frame_index, receiver.get()); let key_value = self.read_register(frame_index, key.get()); - let key = match self.to_property_key(key_value) { + let key = match self.observable_property_key(key_value) { Ok(key) => key, Err(failure) => { self.resolve_failure(failure, pc)?; @@ -6484,7 +6508,11 @@ impl<'a, H: Host> Machine<'a, H> { let id = SlotId::from_parts(RUNTIME_HEAP_SEGMENT, slot) .expect("runtime segment and one-based slot are nonzero"); self.request_garbage_collection(); - Ok(Value::heap_ref(id)) + let value = Value::heap_ref(id); + if self.in_builtin { + self.builtin_temporaries.push(value); + } + Ok(value) } fn live_runtime_slots(&self) -> usize { @@ -6695,19 +6723,194 @@ impl<'a, H: Host> Machine<'a, H> { return Ok(Some(value)); } let name = self.constant_text(module, name).to_owned(); - let key = PropertyKey::Named(name); + let key = PropertyKey::Named(name.clone()); let global = self.context_global.unwrap_or(self.global_object); if self.internal_has_property(global, &key)? { return self.get_property_key(global, &key).map(Some); } + self.missing_global = Some(name); Ok(None) } + /// `ToPropertyKey` for a property reference. A null or undefined base throws + /// here, before the key converts, so that conversion runs no user code. + pub(crate) fn computed_property_key( + &mut self, + object: Value, + key: Value, + ) -> Result { + if matches!(object.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "access a property of null or undefined", + })); + } + self.observable_property_key(key) + } + + /// The `[[Prototype]]` a new closure starts with, which depends on its kind. + pub(crate) fn closure_prototype(&self, module: ModuleId, function: FunctionId) -> Value { + let flags = self.module_code(module).functions()[function.get() as usize].flags(); + let builtins = &self.intrinsics.builtins; + match (flags.is_async, flags.is_generator) { + (false, false) => self.intrinsics.function_prototype, + (false, true) => builtins.generator_function_prototype(), + (true, false) => builtins.async_function_prototype(), + (true, true) => builtins.async_generator_function_prototype(), + } + } + + /// The own properties a new closure starts with. A generator function owns + /// the `prototype` object its kind requires, which the compiler emits only + /// for constructors; every other property of a closure is materialized on + /// demand. + pub(crate) fn closure_properties( + &mut self, + module: ModuleId, + function: FunctionId, + ) -> Result { + let flags = self.module_code(module).functions()[function.get() as usize].flags(); + let mut properties = PropertyMap::default(); + if flags.is_generator { + let builtins = &self.intrinsics.builtins; + let parent = if flags.is_async { + builtins.async_generator_prototype() + } else { + builtins.generator_prototype() + }; + let value = self.allocate(HeapEntry::Object { + properties: PropertyMap::default(), + prototype: Some(parent), + extensible: true, + boxed_primitive: None, + })?; + properties.insert( + PropertyKey::Named(EcmaString::encode("prototype")), + Property::Data { + value, + writable: true, + enumerable: false, + configurable: false, + }, + ); + } + Ok(properties) + } + + /// A function's `length` and `name` come from its code until something could + /// tell that from stored properties: a descriptor read, a key list, a delete or + /// a redefinition stores them first, ahead of whatever is already stored. + pub(crate) fn reify_function_properties(&mut self, index: usize) -> Result<(), EvalFailure> { + let reified = self.intrinsics.builtins.function_reified_key(); + let name_key = PropertyKey::Named(EcmaString::encode("name")); + let length_key = PropertyKey::Named(EcmaString::encode("length")); + let HeapEntry::Function { + module, + function, + properties, + .. + } = &self.heap[index] + else { + return Ok(()); + }; + if properties.contains_key(&reified) { + return Ok(()); + } + let metadata = &self.module_code(*module).functions()[function.get() as usize]; + let length = (!properties.contains_key(&length_key)) + .then(|| number_value(metadata.parameter_count() as f64)); + let name = (!properties.contains_key(&name_key)).then(|| { + metadata + .name() + .map(|id| self.constant_text(*module, id).clone()) + .unwrap_or_default() + }); + let name = name + .map(|text| self.allocate(HeapEntry::String(text))) + .transpose() + .map_err(EvalFailure::Runtime)?; + let mut stored = PropertyMap::default(); + for (key, value) in [(length_key, length), (name_key, name)] { + if let Some(value) = value { + stored.insert( + key, + Property::Data { + value, + writable: false, + enumerable: false, + configurable: true, + }, + ); + } + } + stored.insert( + reified, + Property::Data { + value: Value::TRUE, + writable: false, + enumerable: false, + configurable: false, + }, + ); + self.charge_slot(index, stored.charge_bytes()) + .map_err(EvalFailure::Runtime)?; + let HeapEntry::Function { properties, .. } = &mut self.heap[index] else { + unreachable!("function checked above"); + }; + for (key, property) in std::mem::take(properties).0 { + stored.insert(key, property); + } + *properties = stored; + Ok(()) + } + + /// Whether a function is non-strict ECMAScript code. + pub(crate) fn is_sloppy(&self, module: ModuleId, function: usize) -> bool { + self.module_code(module).functions()[function] + .flags() + .is_sloppy + } + + /// `OrdinaryCallBindThis` for the running function: a sloppy function sees + /// the global object for a missing receiver and a wrapper for a primitive. + /// Callers keep the result so a wrapper keeps its identity. Doing this at + /// the first read cannot be observed, because `ToObject` runs no user code. + pub(crate) fn bind_this( + &mut self, + module: ModuleId, + function: usize, + this_value: Value, + ) -> Result { + if !self.is_sloppy(module, function) { + return Ok(this_value); + } + if this_value == Value::UNDEFINED || this_value == Value::NULL { + return Ok(self.context_global.unwrap_or(self.global_object)); + } + self.value_to_object(this_value) + } + + /// The value of a `delete` the running function performed. Strict code + /// must not silently keep a property it asked to remove. + pub(crate) fn deletion_result( + &self, + module: ModuleId, + function: usize, + deleted: bool, + ) -> Result { + if deleted || self.is_sloppy(module, function) { + return Ok(deleted); + } + Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "delete of an unconfigurable property", + })) + } + pub(crate) fn store_global( &mut self, module: ModuleId, name: ConstantId, value: Value, + sloppy: bool, ) -> Result<(), EvalFailure> { let cell = self .registry @@ -6734,7 +6937,17 @@ impl<'a, H: Host> Machine<'a, H> { } else { let name = self.constant_text(module, name).to_owned(); let global = self.context_global.unwrap_or(self.global_object); - self.set_data_property_key(global, PropertyKey::Named(name), value)?; + let key = PropertyKey::Named(name); + let stored = self.internal_set(global, key.clone(), value, global)?; + // This instruction also carries global declarations, whose failure + // always throws. Only an assignment can fail on an existing + // property, and sloppy code ignores that; a missing property on a + // non-extensible global is taken to be a declaration. + if !stored && !(sloppy && self.has_own_property_key(global, &key)?) { + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "assign to read only property", + })); + } } Ok(()) } @@ -6841,22 +7054,6 @@ impl<'a, H: Host> Machine<'a, H> { }) } - fn bound_target(&self, mut value: Value) -> Result { - loop { - let Some(index) = self.runtime_slot(value)? else { - return Ok(value); - }; - let HeapEntry::NativeFunction { - callable: NativeCallable::Bound(bound), - .. - } = &self.heap[index] - else { - return Ok(value); - }; - value = bound.target; - } - } - /// Materializes a constant into an ABI value, interning strings and bigints /// into the slot heap. Shared with the native engine. pub(crate) fn load_constant_value( @@ -6966,6 +7163,25 @@ impl<'a, H: Host> Machine<'a, H> { } } + /// A strict arguments object poisons `callee`; a sloppy one would name the + /// function, which a frame does not hold. + pub(crate) fn arguments_properties(&self, sloppy: bool) -> PropertyMap { + let mut properties = PropertyMap::default(); + if !sloppy { + let thrower = self.intrinsics.builtins.throw_type_error(); + properties.insert( + PropertyKey::Named(EcmaString::encode("callee")), + Property::Accessor { + getter: Some(thrower), + setter: Some(thrower), + enumerable: false, + configurable: false, + }, + ); + } + properties + } + pub(crate) fn materialize_arguments( &mut self, frame: usize, @@ -6976,10 +7192,12 @@ impl<'a, H: Host> Machine<'a, H> { return Ok(existing); } let args = self.frames[frame].args.clone(); + let properties = + self.arguments_properties(self.is_sloppy(self.frames[frame].module, function)); let value = self .allocate(HeapEntry::Array { elements: args, - properties: PropertyMap::default(), + properties, prototype: Some(self.intrinsics.array_prototype), extensible: true, length_writable: true, @@ -7124,16 +7342,112 @@ impl<'a, H: Host> Machine<'a, H> { f: impl FnOnce(&mut Machine<'a, H>) -> Result, ) -> Result { self.enter_proxy_dispatch()?; - let result = f(self); + let result = self.with_temporaries(f); self.leave_proxy_dispatch(); result } + /// Runs native code that holds values in Rust locals across user code, + /// keeping what it allocates and what user code returns to it rooted. + pub(crate) fn with_temporaries(&mut self, body: impl FnOnce(&mut Machine<'a, H>) -> T) -> T { + if self.in_builtin { + return body(self); + } + self.in_builtin = true; + let mark = self.builtin_temporaries.len(); + let floor = std::mem::replace(&mut self.results_floor, self.callback_results.len()); + let result = body(self); + self.callback_results.truncate(self.results_floor); + self.results_floor = floor; + self.builtin_temporaries.truncate(mark); + self.in_builtin = false; + result + } + + /// Roots values for native code outside a builtin; `release_temporaries` + /// with the returned mark ends that. + pub(crate) fn hold_temporaries(&mut self, values: &[Value]) -> usize { + let mark = self.builtin_temporaries.len(); + self.builtin_temporaries.extend_from_slice(values); + mark + } + + pub(crate) fn release_temporaries(&mut self, mark: usize) { + self.builtin_temporaries.truncate(mark); + } + + /// Keeps a value a builtin just received until that builtin returns. + pub(crate) fn keep_temporary(&mut self, value: Value) { + if self.in_builtin && matches!(value.decode(), Some(Decoded::HeapRef(_))) { + self.builtin_temporaries.push(value); + } + } + + fn keep_callback_result(&mut self, value: Value) { + if !self.in_builtin || !matches!(value.decode(), Some(Decoded::HeapRef(_))) { + return; + } + if self.callback_results.len() - self.results_floor == RECENT_CALLBACK_RESULTS { + self.callback_results.remove(self.results_floor); + } + self.callback_results.push(value); + } + + /// Runs user code for a builtin, which holds the result in a Rust local. + fn call_user_code( + &mut self, + call: impl FnOnce(&mut Machine<'a, H>) -> Result, + keep: fn(&mut Self, Value), + ) -> Result { + let in_builtin = std::mem::replace(&mut self.in_builtin, false); + let result = call(self); + self.in_builtin = in_builtin; + match &result { + Ok(value) + | Err(EvalFailure::ThrowValue(value) | EvalFailure::ThrowValueOrigin { value, .. }) => { + keep(self, *value); + } + Err(_) => {} + } + result + } + + /// `call_value` for a loop over a user callback that is done with each result + /// before the next call: `forEach`, `map`, `reduce` and their like. Only + /// such a loop may use this; anything collecting results needs `call_value`. + pub(crate) fn call_callback( + &mut self, + callee: Value, + this_value: Value, + arguments: &[Value], + ) -> Result { + self.call_user_code( + |machine| machine.dispatch_call(callee, this_value, arguments), + Self::keep_callback_result, + ) + } + /// Answers whether `value` has `[[Construct]]`. Builtin callees consult the /// realm's registration metadata (`BuiltinTable::is_constructable`), bound /// callees recurse into their target, runtime closures consult the /// compiler-emitted `FunctionFlags::is_constructable` bit, and proxies /// answer from the snapshot `ProxyCreate` took of their target. + /// Whether `value` is a non-strict `function`, the one kind of function that + /// the specification lets an implementation give `caller` and `arguments`. + pub(crate) fn is_sloppy_ordinary_function(&self, value: Value) -> Result { + let Some(index) = self.runtime_slot(value).map_err(EvalFailure::Runtime)? else { + return Ok(false); + }; + let HeapEntry::Function { + module, function, .. + } = &self.heap[index] + else { + return Ok(false); + }; + let flags = self.module_code(*module).functions()[function.get() as usize].flags(); + Ok(flags.is_sloppy && flags.is_constructable && !flags.is_async && !flags.is_generator) + } + pub(crate) fn is_constructor(&self, value: Value) -> Result { let Some(index) = self.runtime_slot(value).map_err(EvalFailure::Runtime)? else { return Ok(false); @@ -7186,37 +7500,24 @@ impl<'a, H: Host> Machine<'a, H> { let flags = self.module_code(target.module).functions() [target.function.get() as usize] .flags(); - if flags.is_async && flags.is_generator { - let generator = self - .create_async_generator(GeneratorStart { - target, - captures, - context, - this_value, - new_target, - args: arguments.as_ref().to_vec(), - }) - .map_err(|kind| self.error_here_at(kind, call_pc))?; - if let Some(register) = destination { - self.write_register(self.frames.len() - 1, register, generator); - } - return Ok(()); - } - if flags.is_generator && !flags.is_async { - let generator = self - .create_generator(GeneratorStart { - target, - captures, - context, - this_value, - new_target, - args: arguments.as_ref().to_vec(), - }) - .map_err(|kind| self.error_here_at(kind, call_pc))?; - if let Some(register) = destination { - self.write_register(self.frames.len() - 1, register, generator); - } - return Ok(()); + if flags.is_generator { + let start = GeneratorStart { + target, + captures, + context, + this_value, + new_target, + args: arguments.as_ref().to_vec(), + }; + return match self.start_generator(start, flags) { + Ok(generator) => { + if let Some(register) = destination { + self.write_register(self.frames.len() - 1, register, generator); + } + Ok(()) + } + Err(failure) => self.resolve_failure(failure, call_pc), + }; } if flags.is_async && !flags.is_generator { return match crate::vm::generator_async::start_async_function( @@ -7367,6 +7668,11 @@ impl<'a, H: Host> Machine<'a, H> { Err(failure) => self.resolve_failure(failure, call_pc), }; } + match self.is_constructor(callee) { + Ok(true) => {} + Ok(false) => return self.throw_type("construct", call_pc), + Err(failure) => return self.resolve_failure(failure, call_pc), + } let index = match self.runtime_slot(callee) { Ok(Some(index)) => index, Ok(None) => return self.throw_type("construct", call_pc), @@ -7674,13 +7980,56 @@ impl<'a, H: Host> Machine<'a, H> { self.box_primitive(value) } - pub(crate) fn unbox_primitive_or_self(&self, value: Value) -> Result { - let Some(index) = self.runtime_slot(value).map_err(EvalFailure::Runtime)? else { - return Ok(value); + /// The string inside a String exotic object. + fn boxed_string(&self, index: usize) -> Option<&EcmaString> { + let HeapEntry::Object { + boxed_primitive: Some(primitive), + .. + } = &self.heap[index] + else { + return None; }; - match self.heap[index] { - HeapEntry::Object { - boxed_primitive: Some(primitive), + let slot = self.runtime_slot(*primitive).ok().flatten()?; + match &self.heap[slot] { + HeapEntry::String(text) => Some(text), + _ => None, + } + } + + /// `StringGetOwnProperty`, plus the `length` every String exotic object owns. + pub(crate) fn string_exotic_property( + &mut self, + index: usize, + key: &PropertyKey, + ) -> Result, EvalFailure> { + let found = self + .boxed_string(index) + .and_then(|text| string_own_get(text, key)); + let (value, enumerable) = match found { + Some(Found::Value(length)) => (length, false), + Some(Found::Text(unit)) => { + let unit = self + .allocate(HeapEntry::String(unit)) + .map_err(EvalFailure::Runtime)?; + (unit, true) + } + _ => return Ok(None), + }; + Ok(Some(Property::Data { + value, + writable: false, + enumerable, + configurable: false, + })) + } + + pub(crate) fn unbox_primitive_or_self(&self, value: Value) -> Result { + let Some(index) = self.runtime_slot(value).map_err(EvalFailure::Runtime)? else { + return Ok(value); + }; + match self.heap[index] { + HeapEntry::Object { + boxed_primitive: Some(primitive), .. } => Ok(primitive), _ => Ok(value), @@ -7740,6 +8089,7 @@ impl<'a, H: Host> Machine<'a, H> { configurable: true, }, ); + properties.insert(self.intrinsics.builtins.error_brand_key(), error_brand()); match self.allocate(HeapEntry::Object { properties, prototype: Some(self.intrinsics.error_prototype(id)), @@ -7767,6 +8117,18 @@ impl<'a, H: Host> Machine<'a, H> { callee: Value, arguments: &[Value], new_target: Value, + ) -> Result { + self.call_user_code( + |machine| machine.dispatch_construct(callee, arguments, new_target), + Self::keep_temporary, + ) + } + + fn dispatch_construct( + &mut self, + callee: Value, + arguments: &[Value], + new_target: Value, ) -> Result { match self.callee_kind(callee).map_err(EvalFailure::Runtime)? { CalleeKind::Builtin { id } => { @@ -7798,9 +8160,11 @@ impl<'a, H: Host> Machine<'a, H> { } let object = self.allocate_constructed_receiver(new_target)?; let stop_depth = self.frames.len(); - let return_to = self.frames.last().map(|frame| ReturnTo { + // Native code calling back has no interpreter frame here, and + // the construction still has to answer with `object`. + let return_to = Some(ReturnTo { destination: None, - call_pc: frame.pc, + call_pc: self.frames.last().map_or(0, |frame| frame.pc), constructed: Some(object), }); self.push_frame(ActivationRequest { @@ -7822,7 +8186,10 @@ impl<'a, H: Host> Machine<'a, H> { Ok(None) => self.last_completion.take().ok_or(EvalFailure::Runtime( RuntimeErrorKind::InvalidValue { value: object }, )), - Ok(Some(execution)) => Ok(execution.value), + // With no activation to return to, `complete_frame` hands back + // the raw completion, so the receiver rule applies here. + Ok(Some(execution)) if self.is_object(execution.value) => Ok(execution.value), + Ok(Some(_)) => Ok(object), Err(error) => { self.unwind_frames_to(stop_depth); match error.kind { @@ -7867,6 +8234,18 @@ impl<'a, H: Host> Machine<'a, H> { callee: Value, this_value: Value, arguments: &[Value], + ) -> Result { + self.call_user_code( + |machine| machine.dispatch_call(callee, this_value, arguments), + Self::keep_temporary, + ) + } + + fn dispatch_call( + &mut self, + callee: Value, + this_value: Value, + arguments: &[Value], ) -> Result { let mut callee = callee; let mut this_value = this_value; @@ -7903,29 +8282,16 @@ impl<'a, H: Host> Machine<'a, H> { let flags = self.module_code(target.module).functions() [target.function.get() as usize] .flags(); - if flags.is_async && flags.is_generator { - return self - .create_async_generator(GeneratorStart { - target, - captures, - context, - this_value, - new_target: Value::UNDEFINED, - args: arguments.as_ref().to_vec(), - }) - .map_err(EvalFailure::Runtime); - } - if flags.is_generator && !flags.is_async { - return self - .create_generator(GeneratorStart { - target, - context, - captures, - this_value, - new_target: Value::UNDEFINED, - args: arguments.as_ref().to_vec(), - }) - .map_err(EvalFailure::Runtime); + if flags.is_generator { + let start = GeneratorStart { + target, + context, + captures, + this_value, + new_target: Value::UNDEFINED, + args: arguments.as_ref().to_vec(), + }; + return self.start_generator(start, flags); } if flags.is_async && !flags.is_generator { return crate::vm::generator_async::start_async_function( @@ -8089,6 +8455,7 @@ impl<'a, H: Host> Machine<'a, H> { value, origin, constructor_name: None, + message: None, }, site_module, site_function, @@ -8126,6 +8493,7 @@ impl<'a, H: Host> Machine<'a, H> { value, origin, constructor_name: None, + message: None, }, site_module, site_function, @@ -8423,10 +8791,6 @@ impl<'a, H: Host> Machine<'a, H> { fn own_get_ascii(&self, index: usize, name: &str) -> Option { debug_assert!(name.is_ascii()); - let slot = |value| self.runtime_slot(value).ok().flatten(); - if slot(self.intrinsics.object_prototype) == Some(index) && name == "toString" { - return Some(Found::Value(self.intrinsics.object_to_string())); - } match &self.heap[index] { HeapEntry::Object { properties, @@ -8450,6 +8814,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::Script { properties, .. } | HeapEntry::NativeFunction { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::Promise { properties, .. } @@ -8497,6 +8863,9 @@ impl<'a, H: Host> Machine<'a, H> { if let Some(found) = property_lookup_ascii(properties, name) { return Some(found); } + if properties.contains_key(&self.intrinsics.builtins.function_reified_key()) { + return None; + } let metadata = &self.module_code(*module).functions()[function.get() as usize]; if name == "length" { return Some(Found::Value( @@ -8536,33 +8905,7 @@ impl<'a, H: Host> Machine<'a, H> { self.registry.cells[cell.0].value }))) } - HeapEntry::RegExp { - pattern, - flags, - properties, - .. - } => { - if let Some(found) = property_lookup_ascii(properties, name) { - return Some(found); - } - let flag = |unit| { - Found::Value(Value::boolean(flags.as_units().contains(&u16::from(unit)))) - }; - match name { - "source" => Some(Found::Text(crate::intrinsics::builtins::canonical_source( - pattern, - ))), - "flags" => Some(Found::Text(flags.clone())), - "global" => Some(flag(b'g')), - "ignoreCase" => Some(flag(b'i')), - "multiline" => Some(flag(b'm')), - "sticky" => Some(flag(b'y')), - "unicode" => Some(flag(b'u')), - "dotAll" => Some(flag(b's')), - "lastIndex" => Some(Found::Value(Value::int32(0))), - _ => None, - } - } + HeapEntry::RegExp { properties, .. } => property_lookup_ascii(properties, name), HeapEntry::HashState { update, digest, .. } => match name { "update" => Some(Found::Value(*update)), "digest" => Some(Found::Value(*digest)), @@ -8586,12 +8929,6 @@ impl<'a, H: Host> Machine<'a, H> { /// Looks up an own property of the heap entry at `index`, returning `None` /// when the key is absent so the caller may continue up the prototype chain. fn own_get(&self, index: usize, key: &PropertyKey) -> Option { - if let PropertyKey::Named(name) = key { - let slot = |value| self.runtime_slot(value).ok().flatten(); - if slot(self.intrinsics.object_prototype) == Some(index) && name.eq_ascii("toString") { - return Some(Found::Value(self.intrinsics.object_to_string())); - } - } match &self.heap[index] { HeapEntry::Object { properties, @@ -8614,6 +8951,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::AsyncFromSync { properties, .. } | HeapEntry::Script { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::Promise { properties, .. } @@ -8663,7 +9002,9 @@ impl<'a, H: Host> Machine<'a, H> { if let Some(found) = property_lookup(properties, key) { return Some(found); } - if let PropertyKey::Named(name) = key { + if let PropertyKey::Named(name) = key + && !properties.contains_key(&self.intrinsics.builtins.function_reified_key()) + { let metadata = &self.module_code(*module).functions()[function.get() as usize]; if name.eq_ascii("length") { return Some(Found::Value( @@ -8701,53 +9042,7 @@ impl<'a, H: Host> Machine<'a, H> { }))) } HeapEntry::NativeFunction { properties, .. } => property_lookup(properties, key), - HeapEntry::RegExp { - pattern, - flags, - properties, - .. - } => { - if let Some(found) = property_lookup(properties, key) { - return Some(found); - } - if let PropertyKey::Named(name) = key { - let flag = |ascii: &str| { - Found::Value(Value::boolean( - flags.as_units().contains(&u16::from(ascii.as_bytes()[0])), - )) - }; - if name.eq_ascii("source") { - return Some(Found::Text(crate::intrinsics::builtins::canonical_source( - pattern, - ))); - } - if name.eq_ascii("flags") { - return Some(Found::Text(flags.clone())); - } - if name.eq_ascii("global") { - return Some(flag("g")); - } - if name.eq_ascii("ignoreCase") { - return Some(flag("i")); - } - if name.eq_ascii("multiline") { - return Some(flag("m")); - } - if name.eq_ascii("sticky") { - return Some(flag("y")); - } - if name.eq_ascii("unicode") { - return Some(flag("u")); - } - if name.eq_ascii("dotAll") { - return Some(flag("s")); - } - if name.eq_ascii("lastIndex") { - return Some(Found::Value(Value::int32(0))); - } - } - None - } + HeapEntry::RegExp { properties, .. } => property_lookup(properties, key), HeapEntry::HashState { update, digest, .. } => { let PropertyKey::Named(name) = key else { return None; @@ -8825,6 +9120,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::TypedArray { properties, .. } @@ -8856,6 +9153,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::Function { prototype, .. } | HeapEntry::RegExp { prototype, .. } | HeapEntry::Date { prototype, .. } + | HeapEntry::Temporal { prototype, .. } + | HeapEntry::Intl { prototype, .. } | HeapEntry::BuiltinIterator { prototype, .. } | HeapEntry::Collection { prototype, .. } | HeapEntry::TypedArray { prototype, .. } @@ -8938,6 +9237,10 @@ impl<'a, H: Host> Machine<'a, H> { value: Value, receiver: Value, ) -> Result { + let wrapper = self.value_to_object(object)?; + if wrapper != object { + return self.resolve_set(wrapper, key, value, receiver); + } let index = match self.runtime_slot(object).map_err(EvalFailure::Runtime)? { Some(index) => index, None => { @@ -8947,9 +9250,7 @@ impl<'a, H: Host> Machine<'a, H> { } }; if matches!(self.heap[index], HeapEntry::ModuleNamespace { .. }) { - return Err(EvalFailure::Throw(ThrowOrigin::TypeError { - operation: "assign to module namespace", - })); + return Ok(false); } if matches!(self.heap[index], HeapEntry::ProcessEnv { .. }) { let PropertyKey::Named(name) = &key else { @@ -9047,12 +9348,9 @@ impl<'a, H: Host> Machine<'a, H> { ) .map(|_| true); } - let index = self - .runtime_slot(receiver) - .map_err(EvalFailure::Runtime)? - .ok_or(EvalFailure::Throw(ThrowOrigin::TypeError { - operation: "set property on primitive", - }))?; + let Some(index) = self.object_slot(receiver)? else { + return Ok(false); + }; self.set_own_data(index, key, value)?; return Ok(true); }; @@ -9077,16 +9375,32 @@ impl<'a, H: Host> Machine<'a, H> { } // Ordinary receiver: the stored-property fast path keeps the exact // attributes and handles Array element / TypedArray virtual writes. - let index = self - .runtime_slot(receiver) - .map_err(EvalFailure::Runtime)? - .ok_or(EvalFailure::Throw(ThrowOrigin::TypeError { - operation: "set property on primitive", - }))?; + let Some(index) = self.object_slot(receiver)? else { + return Ok(false); + }; + if self.is_array_length_key(index, &key) { + let descriptor = crate::builtins::property_descriptor::PropertyDescriptor { + value: Some(value), + ..Default::default() + }; + return self.array_define_length(index, descriptor); + } self.set_own_data(index, key, value)?; Ok(true) } + /// The heap slot of `value` when it is an object. A primitive receiver is + /// valid in `[[Set]]`, but it can hold nothing. + fn object_slot(&self, value: Value) -> Result, EvalFailure> { + let slot = self.runtime_slot(value).map_err(EvalFailure::Runtime)?; + Ok(slot.filter(|index| { + !matches!( + self.heap[*index], + HeapEntry::String(_) | HeapEntry::BigInt(_) | HeapEntry::Symbol { .. } + ) + })) + } + pub(crate) fn set_super( &mut self, home: Value, @@ -9165,9 +9479,110 @@ impl<'a, H: Host> Machine<'a, H> { key: &PropertyKey, ) -> Result { let base = self.super_base(home)?; + if base == Value::NULL { + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "get super property through null prototype", + })); + } self.internal_get(base, key, receiver) } + pub(crate) fn is_array_length_key(&self, index: usize, key: &PropertyKey) -> bool { + matches!(key, PropertyKey::Named(name) if name.eq_ascii("length")) + && matches!(self.heap[index], HeapEntry::Array { .. }) + } + + /// `ToUint32` and then `ToNumber`, which both run user code and must agree. + fn array_length_value(&mut self, value: Value) -> Result { + let first = self.coerce_number_observable(value)?; + let second = self.coerce_number_observable(value)?; + let length = numeric_f64(first).map_or(0, to_uint32); + if numeric_f64(second) == Some(f64::from(length)) { + Ok(length as usize) + } else { + Err(EvalFailure::Throw(ThrowOrigin::RangeError { + operation: "set array length", + })) + } + } + + /// `ArraySetLength`. A non-configurable element stops a truncation there, + /// which is reported as `false` after the elements above it have gone. + pub(crate) fn array_define_length( + &mut self, + index: usize, + descriptor: crate::builtins::property_descriptor::PropertyDescriptor, + ) -> Result { + let length = descriptor + .value + .map(|value| self.array_length_value(value)) + .transpose()?; + let HeapEntry::Array { + elements, + length_writable, + .. + } = &self.heap[index] + else { + unreachable!("array_define_length requires an array"); + }; + let (old_length, writable) = (elements.len(), *length_writable); + if descriptor.is_accessor() + || descriptor.enumerable == Some(true) + || descriptor.configurable == Some(true) + || (!writable + && (descriptor.writable == Some(true) + || length.is_some_and(|length| length != old_length))) + { + return Ok(false); + } + let reached = match length { + Some(length) if length != old_length => self.resize_array(index, length)?, + _ => true, + }; + if descriptor.writable == Some(false) + && let HeapEntry::Array { + length_writable, .. + } = &mut self.heap[index] + { + *length_writable = false; + } + Ok(reached) + } + + /// Resizes with the heap accounting, and says whether `length` was reached. + fn resize_array(&mut self, index: usize, length: usize) -> Result { + let old_length_and_bytes = |entry: &HeapEntry| match entry { + HeapEntry::Array { + elements, + properties, + .. + } => (elements.len(), properties.charge_bytes()), + _ => unreachable!("resize_array requires an array"), + }; + let (old_length, old_property_bytes) = old_length_and_bytes(&self.heap[index]); + let growth = length + .saturating_sub(old_length) + .saturating_mul(std::mem::size_of::()); + self.charge_slot(index, growth) + .map_err(EvalFailure::Runtime)?; + let HeapEntry::Array { + elements, + properties, + .. + } = &mut self.heap[index] + else { + unreachable!("resize_array requires an array"); + }; + let reached = apply_array_length(elements, properties, length); + let (new_length, new_property_bytes) = old_length_and_bytes(&self.heap[index]); + let released = old_length + .saturating_sub(new_length) + .saturating_mul(std::mem::size_of::()) + .saturating_add(old_property_bytes.saturating_sub(new_property_bytes)); + self.refund_slot(index, released); + Ok(reached) + } + fn set_own_data( &mut self, index: usize, @@ -9185,57 +9600,18 @@ impl<'a, H: Host> Machine<'a, H> { typedarray_all::write_element(self, view, offset, value)?; return Ok(()); } - if matches!(&key, PropertyKey::Named(name) if name.eq_ascii("length")) - && matches!(self.heap[index], HeapEntry::Array { .. }) - { - let length = - exact_array_length(value).ok_or(EvalFailure::Throw(ThrowOrigin::RangeError { - operation: "set array length", - }))?; - let (old_length, old_property_bytes, length_writable) = match &self.heap[index] { - HeapEntry::Array { - elements, - properties, - length_writable, - .. - } => (elements.len(), properties.charge_bytes(), *length_writable), - _ => unreachable!("array checked above"), + if self.is_array_length_key(index, &key) { + let descriptor = crate::builtins::property_descriptor::PropertyDescriptor { + value: Some(value), + ..Default::default() }; - if !length_writable { - return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + return if self.array_define_length(index, descriptor)? { + Ok(()) + } else { + Err(EvalFailure::Throw(ThrowOrigin::TypeError { operation: "set array length", - })); - } - let growth = length - .saturating_sub(old_length) - .saturating_mul(std::mem::size_of::()); - self.charge_slot(index, growth) - .map_err(EvalFailure::Runtime)?; - let result = { - let HeapEntry::Array { - elements, - properties, - .. - } = &mut self.heap[index] - else { - unreachable!("array checked above"); - }; - apply_array_length(elements, properties, length, "set array length") - }; - let (new_length, new_property_bytes) = match &self.heap[index] { - HeapEntry::Array { - elements, - properties, - .. - } => (elements.len(), properties.charge_bytes()), - _ => unreachable!("array checked above"), + })) }; - let released = old_length - .saturating_sub(new_length) - .saturating_mul(std::mem::size_of::()) - .saturating_add(old_property_bytes.saturating_sub(new_property_bytes)); - self.refund_slot(index, released); - return result; } if let HeapEntry::Array { elements, @@ -9296,6 +9672,16 @@ impl<'a, H: Host> Machine<'a, H> { extensible, .. } + | HeapEntry::Temporal { + properties, + extensible, + .. + } + | HeapEntry::Intl { + properties, + extensible, + .. + } | HeapEntry::BuiltinIterator { properties, extensible, @@ -9397,6 +9783,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::TypedArray { properties, .. } @@ -9480,6 +9868,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::TypedArray { properties, .. } @@ -9533,7 +9923,14 @@ impl<'a, H: Host> Machine<'a, H> { "set array index", )?; } - elements[offset] = value; + let key = PropertyKey::Named(name); + if let Some(Property::Data { value: stored, .. }) = + properties.get_mut(&key) + { + *stored = value; + } else { + elements[offset] = value; + } } else { let key = PropertyKey::Named(name); let descriptor = match properties.get(&key) { @@ -9658,6 +10055,16 @@ impl<'a, H: Host> Machine<'a, H> { extensible, .. } + | HeapEntry::Temporal { + properties, + extensible, + .. + } + | HeapEntry::Intl { + properties, + extensible, + .. + } | HeapEntry::BuiltinIterator { properties, extensible, @@ -9748,6 +10155,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::TypedArray { properties, .. } @@ -9762,21 +10171,25 @@ impl<'a, H: Host> Machine<'a, H> { _ => unreachable!("validated object cannot change heap entry kind"), }; match properties.get_mut(&key) { - Some(Property::Accessor { getter, setter, .. }) => match kind { - AccessorKind::Getter => *getter = Some(accessor), - AccessorKind::Setter => *setter = Some(accessor), - }, + Some(Property::Accessor { getter, setter, .. }) => { + if kind.is_getter() { + *getter = Some(accessor); + } else { + *setter = Some(accessor); + } + } Some(Property::Data { .. }) | None => { - let (getter, setter) = match kind { - AccessorKind::Getter => (Some(accessor), None), - AccessorKind::Setter => (None, Some(accessor)), + let (getter, setter) = if kind.is_getter() { + (Some(accessor), None) + } else { + (None, Some(accessor)) }; properties.insert( key, Property::Accessor { getter, setter, - enumerable: true, + enumerable: kind.is_enumerable(), configurable: true, }, ); @@ -9802,6 +10215,13 @@ impl<'a, H: Host> Machine<'a, H> { } match self.runtime_slot(object).map_err(EvalFailure::Runtime)? { Some(index) => { + if self + .boxed_string(index) + .is_some_and(|text| string_own_get(text, key).is_some()) + { + return Ok(false); + } + self.reify_function_properties(index)?; if let PropertyKey::Named(name) = key && let Some(typed_index) = typedarray_all::typed_array_index(name) && matches!(self.heap[index], HeapEntry::TypedArray { .. }) @@ -9822,6 +10242,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::TypedArray { properties, .. } @@ -9872,13 +10294,15 @@ impl<'a, H: Host> Machine<'a, H> { 0 } } + // An environment variable is always configurable, so this + // succeeds whether or not the variable was set, as in Node. HeapEntry::ProcessEnv { .. } => { - let PropertyKey::Named(name) = key else { - return Ok(true); - }; - return Ok(name - .to_utf8_strict() - .is_ok_and(|name| self.host.delete_env(&name))); + if let PropertyKey::Named(name) = key + && let Ok(name) = name.to_utf8_strict() + { + self.host.delete_env(&name); + } + return Ok(true); } HeapEntry::String(_) | HeapEntry::BigInt(_) @@ -10004,17 +10428,23 @@ impl<'a, H: Host> Machine<'a, H> { } fn array_extend(&mut self, array: Value, iterable: Value) -> Result<(), EvalFailure> { - let iterator = self.create_iterator(iterable, IteratorKind::Sync)?; - loop { - let (done, value) = self.iterator_next(iterator)?; - if done { - return Ok(()); + self.with_temporaries(|machine| { + let iterator = machine.create_iterator(iterable, IteratorKind::Sync)?; + loop { + let (done, value) = machine.iterator_next(iterator)?; + if done { + return Ok(()); + } + machine.array_push(array, value)?; } - self.array_push(array, value)?; - } + }) } fn object_spread(&mut self, target: Value, source: Value) -> Result<(), EvalFailure> { + self.with_temporaries(|machine| machine.spread_properties(target, source)) + } + + fn spread_properties(&mut self, target: Value, source: Value) -> Result<(), EvalFailure> { let target_index = match self.runtime_slot(target).map_err(EvalFailure::Runtime)? { Some(index) if matches!( @@ -10247,6 +10677,12 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::Date { prototype: slot, .. } + | HeapEntry::Temporal { + prototype: slot, .. + } + | HeapEntry::Intl { + prototype: slot, .. + } | HeapEntry::BuiltinIterator { prototype: slot, .. } @@ -10299,19 +10735,91 @@ impl<'a, H: Host> Machine<'a, H> { } } - pub(crate) fn create_generator( + /// Calls a generator function. When its parameters can run code, the + /// compiler ended its prologue with a `Suspend`, and running to that point + /// now lets a throw from a parameter reach the caller. + pub(crate) fn start_generator( &mut self, start: GeneratorStart, - ) -> Result { - self.allocate(HeapEntry::Generator { - state: GeneratorState::SuspendedStart(start), - properties: PropertyMap::default(), - prototype: Some(self.intrinsics.builtins.generator_prototype()), - extensible: true, - }) + flags: bamts_bytecode::FunctionFlags, + ) -> Result { + if flags.is_async { + let generator = self + .create_async_generator(start) + .map_err(EvalFailure::Runtime)?; + if flags.binds_at_call { + self.run_async_generator_prologue(generator)?; + } + return Ok(generator); + } + let generator = self.create_generator(start).map_err(EvalFailure::Runtime)?; + if flags.binds_at_call { + self.resume_generator(generator, GeneratorCompletion::Normal(Value::UNDEFINED))?; + } + Ok(generator) } - fn resume_generator( + /// Runs an async generator to the `Suspend` that ends its prologue. No + /// request is queued yet, so the stop settles nothing; the first request + /// resumes it the way any later one resumes a `yield`. + fn run_async_generator_prologue(&mut self, generator: Value) -> Result<(), EvalFailure> { + let AsyncGeneratorState::SuspendedStart(start) = + self.take_async_generator_state(generator)? + else { + unreachable!("a new async generator is suspended at its start") + }; + let stop_depth = self.frames.len(); + let return_to = self.frames.last().map(|frame| ReturnTo { + destination: None, + call_pc: frame.pc, + constructed: None, + }); + let pushed = self + .push_frame(ActivationRequest { + target: start.target, + captures: &start.captures, + context: start.context, + this_value: start.this_value, + new_target: start.new_target, + arguments: &start.args, + return_to, + }) + .map_err(|error| EvalFailure::Runtime(error.kind)); + let step = pushed.and_then(|()| self.drive_async_generator_activation(stop_depth, None)); + match step { + Ok(AsyncGeneratorStep::Yield { activation, .. }) => self + .replace_async_generator_state( + generator, + AsyncGeneratorState::SuspendedYield(activation), + ) + .map_err(EvalFailure::Runtime), + Ok(AsyncGeneratorStep::Throw { value, origin }) => { + self.complete_async_generator(generator)?; + Err(EvalFailure::ThrowValueOrigin { value, origin }) + } + Ok(AsyncGeneratorStep::Await { .. } | AsyncGeneratorStep::Return(_)) => { + unreachable!("parameters cannot await, and the prologue ends at a Suspend") + } + Err(failure) => { + self.complete_async_generator(generator)?; + Err(failure) + } + } + } + + pub(crate) fn create_generator( + &mut self, + start: GeneratorStart, + ) -> Result { + self.allocate(HeapEntry::Generator { + state: GeneratorState::SuspendedStart(start), + properties: PropertyMap::default(), + prototype: Some(self.intrinsics.builtins.generator_prototype()), + extensible: true, + }) + } + + fn resume_generator( &mut self, generator: Value, completion: GeneratorCompletion, @@ -11754,6 +12262,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::Promise { properties, .. } @@ -11794,9 +12304,11 @@ impl<'a, H: Host> Machine<'a, H> { indices.push((offset, key)); } indices.sort_unstable_by_key(|(offset, _)| *offset); + let length = PropertyKey::Named(EcmaString::encode("length")); Ok(indices .into_iter() .map(|(_, key)| key) + .chain(std::iter::once(length)) .chain(suffix) .collect()) } @@ -11851,6 +12363,12 @@ impl<'a, H: Host> Machine<'a, H> { .internal_get_own_property(src, key)? .is_some_and(|descriptor| descriptor.enumerable == Some(true))); } + if self + .boxed_string(index) + .is_some_and(|text| string_has_index(text, key)) + { + return Ok(true); + } Ok(match &self.heap[index] { HeapEntry::Array { elements, @@ -11868,9 +12386,7 @@ impl<'a, H: Host> Machine<'a, H> { }, Property::enumerable, ), - HeapEntry::String(text) => key.as_string().is_some_and(|name| { - array_index(name).is_some_and(|offset| (offset as usize) < text.len_units()) - }), + HeapEntry::String(text) => string_has_index(text, key), HeapEntry::TypedArray { properties, .. } => properties.get(key).map_or_else( || { key.as_string() @@ -11895,6 +12411,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { properties, .. } | HeapEntry::RegExp { properties, .. } | HeapEntry::Date { properties, .. } + | HeapEntry::Temporal { properties, .. } + | HeapEntry::Intl { properties, .. } | HeapEntry::BuiltinIterator { properties, .. } | HeapEntry::Collection { properties, .. } | HeapEntry::Promise { properties, .. } @@ -12150,6 +12668,7 @@ impl<'a, H: Host> Machine<'a, H> { return Ok((true, Value::UNDEFINED)); } let value = self.get_named_property(result, "value")?; + self.keep_temporary(value); Ok((false, value)) } @@ -12446,8 +12965,9 @@ impl<'a, H: Host> Machine<'a, H> { self.allocate(HeapEntry::String(text)) .map_err(EvalFailure::Runtime) } - UnaryOp::Plus => self.to_number(operand), + UnaryOp::Plus => self.coerce_number_observable(operand), UnaryOp::Negate => { + let operand = self.coerce_primitive_observable(operand, false)?; if let Some(value) = intrinsics::builtins::bigint::unary_minus(self, operand)? { return Ok(value); } @@ -12456,6 +12976,7 @@ impl<'a, H: Host> Machine<'a, H> { Ok(number_value(-number)) } UnaryOp::BitwiseNot => { + let operand = self.coerce_primitive_observable(operand, false)?; if let Some(value) = intrinsics::builtins::bigint::bitwise_not(self, operand)? { return Ok(value); } @@ -12464,6 +12985,21 @@ impl<'a, H: Host> Machine<'a, H> { )) } UnaryOp::LogicalNot => Ok(Value::boolean(!self.truthy(operand))), + UnaryOp::ToString => { + if self.string_value(operand).is_some() { + return Ok(operand); + } + let text = self.coerce_string_observable(operand)?; + self.allocate(HeapEntry::String(text)) + .map_err(EvalFailure::Runtime) + } + UnaryOp::ToPropertyKey => { + if !self.is_object(operand) { + return Ok(operand); + } + let key = self.observable_property_key(operand)?; + intrinsics::builtins::reflect::property_key_value(self, key) + } } } @@ -12472,6 +13008,20 @@ impl<'a, H: Host> Machine<'a, H> { op: BinaryOp, left: Value, right: Value, + ) -> Result { + // Converting one operand can run user code while the other's primitive + // is only a Rust local. + if self.is_object(left) || self.is_object(right) { + return self.with_temporaries(|machine| machine.binary_values(op, left, right)); + } + self.binary_values(op, left, right) + } + + fn binary_values( + &mut self, + op: BinaryOp, + left: Value, + right: Value, ) -> Result { match op { BinaryOp::StrictEqual => Ok(Value::boolean(self.strict_equal(left, right))), @@ -12501,7 +13051,7 @@ impl<'a, H: Host> Machine<'a, H> { } BinaryOp::InstanceOf => self.instance_of(left, right).map(Value::boolean), BinaryOp::In => { - let key = self.to_property_key(left)?; + let key = self.observable_property_key(left)?; self.internal_has_property(right, &key).map(Value::boolean) } BinaryOp::Add => self.add(left, right), @@ -12565,6 +13115,10 @@ impl<'a, H: Host> Machine<'a, H> { left: Value, right: Value, ) -> Result { + // ToNumeric, left first: an object operand runs its `valueOf` here, and + // the operator only ever sees primitives after this. + let left = self.coerce_primitive_observable(left, false)?; + let right = self.coerce_primitive_observable(right, false)?; if let Some(result) = intrinsics::builtins::bigint::binary_op(self, bigint_binary_op(op), left, right)? { @@ -12577,7 +13131,7 @@ impl<'a, H: Host> Machine<'a, H> { BinaryOp::Multiply => number_value(left * right), BinaryOp::Divide => Value::number(left / right), BinaryOp::Remainder => Value::number(left % right), - BinaryOp::Exponent => Value::number(left.powf(right)), + BinaryOp::Exponent => Value::number(number_exponentiate(left, right)), BinaryOp::BitAnd => Value::int32((to_int32(left) & to_int32(right)) as u32), BinaryOp::BitOr => Value::int32((to_int32(left) | to_int32(right)) as u32), BinaryOp::BitXor => Value::int32((to_int32(left) ^ to_int32(right)) as u32), @@ -12609,21 +13163,57 @@ impl<'a, H: Host> Machine<'a, H> { } pub(crate) fn coerce_primitive_default(&mut self, value: Value) -> Result { - let prefer_string = self - .runtime_slot(value) - .map_err(EvalFailure::Runtime)? - .is_some_and(|index| matches!(self.heap[index], HeapEntry::Date { .. })); - self.coerce_primitive_observable(value, prefer_string) + self.coerce_primitive_with_hint(value, "default", false) } pub(crate) fn coerce_primitive_observable( &mut self, value: Value, prefer_string: bool, + ) -> Result { + let hint = if prefer_string { "string" } else { "number" }; + self.coerce_primitive_with_hint(value, hint, prefer_string) + } + + /// ToPrimitive. `hint` is what a `Symbol.toPrimitive` method receives, and + /// `prefer_string` orders the two ordinary methods when there is none. + fn coerce_primitive_with_hint( + &mut self, + value: Value, + hint: &str, + prefer_string: bool, ) -> Result { if self.coercion_is_primitive(value)? { return Ok(value); } + let key = self.to_property_key(self.intrinsics.builtins.symbol_to_primitive())?; + let exotic = self.get_property_key(value, &key)?; + if !matches!(exotic.decode(), Some(Decoded::Undefined | Decoded::Null)) { + if !self.is_callable(exotic)? { + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "Symbol.toPrimitive is not a function", + })); + } + let hint = self + .allocate(HeapEntry::String(EcmaString::encode(hint))) + .map_err(EvalFailure::Runtime)?; + let primitive = self.call_value(exotic, value, &[hint])?; + if self.coercion_is_primitive(primitive)? { + return Ok(primitive); + } + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "Symbol.toPrimitive returned an object", + })); + } + self.ordinary_to_primitive(value, prefer_string) + } + + /// `OrdinaryToPrimitive`: the two ordinary methods, in hint order. + pub(crate) fn ordinary_to_primitive( + &mut self, + value: Value, + prefer_string: bool, + ) -> Result { let methods = if prefer_string { ["toString", "valueOf"] } else { @@ -12700,6 +13290,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::NativeFunction { .. } | HeapEntry::RegExp { .. } | HeapEntry::Date { .. } + | HeapEntry::Temporal { .. } + | HeapEntry::Intl { .. } | HeapEntry::BuiltinIterator { .. } | HeapEntry::Collection { .. } | HeapEntry::TypedArray { .. } @@ -12756,6 +13348,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::PrivateName { .. } | HeapEntry::RegExp { .. } | HeapEntry::Date { .. } + | HeapEntry::Temporal { .. } + | HeapEntry::Intl { .. } | HeapEntry::BuiltinIterator { .. } | HeapEntry::Collection { .. } | HeapEntry::TypedArray { .. } @@ -12815,6 +13409,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::HashState { .. } | HeapEntry::RegExp { .. } | HeapEntry::Date { .. } + | HeapEntry::Temporal { .. } + | HeapEntry::Intl { .. } | HeapEntry::BuiltinIterator { .. } | HeapEntry::Collection { .. } | HeapEntry::TypedArray { .. } @@ -12924,14 +13520,19 @@ impl<'a, H: Host> Machine<'a, H> { /// prototype-chain check. A present-but-non-callable handler is a /// TypeError, matching `GetMethod`. fn instance_of(&mut self, value: Value, constructor: Value) -> Result { - let constructor = self - .bound_target(constructor) - .map_err(EvalFailure::Runtime)?; + if !self.is_object(constructor) { + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "instanceof", + })); + } let key = self.has_instance_key()?; let handler = self.get_property_key(constructor, &key)?; if matches!(handler.decode(), Some(Decoded::Undefined | Decoded::Null)) { - // No `@@hasInstance`: ordinary prototype-chain check, byte-for-byte - // the pre-symbol `instanceof` behavior. + if !self.is_callable(constructor)? { + return Err(EvalFailure::Throw(ThrowOrigin::TypeError { + operation: "instanceof", + })); + } return self.ordinary_has_instance(value, constructor); } // GetMethod: a present, non-nullish handler must be callable. @@ -12946,18 +13547,29 @@ impl<'a, H: Host> Machine<'a, H> { /// `OrdinaryHasInstance` (ECMA-262 §7.3.20): walks `value`'s prototype /// chain for the constructor's own `prototype` object, matching by heap - /// identity. This is the fallback for a right-hand side that defines no - /// callable `Symbol.hasInstance` and is unchanged from the original - /// `instanceof` implementation. + /// identity. A bound function defers to `instanceof` on its target, and a + /// non-callable receiver is simply not a constructor of anything. fn ordinary_has_instance( &mut self, value: Value, constructor: Value, ) -> Result { if !self.is_callable(constructor)? { - return Err(EvalFailure::Throw(ThrowOrigin::TypeError { - operation: "instanceof", - })); + return Ok(false); + } + if let Some(index) = self + .runtime_slot(constructor) + .map_err(EvalFailure::Runtime)? + && let HeapEntry::NativeFunction { + callable: NativeCallable::Bound(bound), + .. + } = &self.heap[index] + { + let target = bound.target; + return self.instance_of(value, target); + } + if !self.is_object(value) { + return Ok(false); } let prototype_key = PropertyKey::Named(EcmaString::encode("prototype")); let prototype = self.internal_get(constructor, &prototype_key, constructor)?; @@ -13056,6 +13668,8 @@ impl<'a, H: Host> Machine<'a, H> { | HeapEntry::AsyncFromSync { .. } | HeapEntry::Script { .. } | HeapEntry::Date { .. } + | HeapEntry::Temporal { .. } + | HeapEntry::Intl { .. } | HeapEntry::BuiltinIterator { .. } | HeapEntry::Collection { .. } | HeapEntry::TypedArray { .. } @@ -13169,6 +13783,8 @@ impl<'a, H: Host> Machine<'a, H> { self.heap[index], HeapEntry::String(_) | HeapEntry::BigInt(_) + | HeapEntry::Symbol { .. } + | HeapEntry::PrivateName { .. } | HeapEntry::PromiseResolver { .. } | HeapEntry::PromiseAll { .. } | HeapEntry::AsyncActivation { .. } @@ -13255,11 +13871,25 @@ fn numeric_f64(value: Value) -> Option { } } +/// What an error object stores under the `[[ErrorData]]` brand key. +fn error_brand() -> Property { + Property::Data { + value: Value::TRUE, + writable: false, + enumerable: false, + configurable: false, + } +} + +/// The compact encoding for a whole Number. `-0` stays a double: Int32 has no +/// negative zero, and `1 / -0` and `Object.is` can both tell the difference. fn number_value(number: f64) -> Value { + // An int32 has no negative zero, so that one integral value stays a double. if number.is_finite() && number.fract() == 0.0 && number >= f64::from(i32::MIN) && number <= f64::from(i32::MAX) + && !(number == 0.0 && number.is_sign_negative()) { Value::int32(number as i32 as u32) } else { @@ -13267,20 +13897,21 @@ fn number_value(number: f64) -> Value { } } +/// `Number::exponentiate`, which differs from `powf` for a base of 1 or -1 and a +/// NaN or infinite exponent. +pub(crate) fn number_exponentiate(base: f64, exponent: f64) -> f64 { + if exponent.is_nan() || (base.abs() == 1.0 && exponent.is_infinite()) { + f64::NAN + } else { + base.powf(exponent) + } +} + fn parse_number(text: &EcmaString) -> f64 { let Ok(text) = text.to_utf8_strict() else { return f64::NAN; }; - parse_number_utf8(&text) -} - -fn parse_number_utf8(text: &str) -> f64 { - let trimmed = text.trim(); - if trimmed.is_empty() { - 0.0 - } else { - trimmed.parse::().unwrap_or(f64::NAN) - } + intrinsics::builtins::string_to_number(&text) } fn to_uint32(number: f64) -> u32 { @@ -13326,6 +13957,12 @@ fn array_index(key: &EcmaString) -> Option { (index != u32::MAX).then_some(index) } +fn string_has_index(text: &EcmaString, key: &PropertyKey) -> bool { + key.as_string().is_some_and(|name| { + array_index(name).is_some_and(|offset| (offset as usize) < text.len_units()) + }) +} + fn string_own_get(text: &EcmaString, key: &PropertyKey) -> Option { let PropertyKey::Named(name) = key else { return None; @@ -13354,15 +13991,16 @@ fn exact_array_length(value: Value) -> Option { } } +/// Resizes to `length`, stopping above the highest non-configurable element, and +/// says whether the whole of `length` was reached. pub(crate) fn apply_array_length( elements: &mut Vec, properties: &mut PropertyMap, length: usize, - operation: &'static str, -) -> Result<(), EvalFailure> { +) -> bool { if length >= elements.len() { elements.resize(length, Value::HOLE); - return Ok(()); + return true; } let blocked = properties .iter() @@ -13381,10 +14019,7 @@ pub(crate) fn apply_array_length( .is_none_or(|offset| (offset as usize) < effective_length) }); elements.resize(effective_length, Value::HOLE); - if blocked.is_some() { - return Err(EvalFailure::Throw(ThrowOrigin::TypeError { operation })); - } - Ok(()) + blocked.is_none() } pub(crate) fn array_set_length( @@ -13399,7 +14034,11 @@ pub(crate) fn array_set_length( if !length_writable { return Err(EvalFailure::Throw(ThrowOrigin::TypeError { operation })); } - apply_array_length(elements, properties, length, operation) + if apply_array_length(elements, properties, length) { + Ok(()) + } else { + Err(EvalFailure::Throw(ThrowOrigin::TypeError { operation })) + } } fn bigint_binary_op(op: BinaryOp) -> intrinsics::builtins::bigint::BigIntBinaryOp { @@ -13429,6 +14068,8 @@ pub(crate) fn unary_from_selector(op: u32) -> Option { 3 => Some(UnaryOp::Negate), 4 => Some(UnaryOp::BitwiseNot), 5 => Some(UnaryOp::LogicalNot), + 6 => Some(UnaryOp::ToString), + 7 => Some(UnaryOp::ToPropertyKey), _ => None, } } @@ -13474,6 +14115,8 @@ pub(crate) fn accessor_from_selector(kind: u32) -> Option { match kind { 0 => Some(AccessorKind::Getter), 1 => Some(AccessorKind::Setter), + 2 => Some(AccessorKind::ClassGetter), + 3 => Some(AccessorKind::ClassSetter), _ => None, } } @@ -13535,6 +14178,7 @@ mod tests { is_async: false, is_generator: true, is_constructable: false, + ..FunctionFlags::default() }, code, handlers, @@ -13556,6 +14200,7 @@ mod tests { is_async: true, is_generator: false, is_constructable: false, + ..FunctionFlags::default() }, code, handlers, @@ -13577,6 +14222,7 @@ mod tests { is_async: true, is_generator: true, is_constructable: false, + ..FunctionFlags::default() }, code, handlers, @@ -13667,9 +14313,8 @@ mod tests { assert_eq!(plain, outcome); } - #[cfg(target_os = "linux")] #[test] - fn run_with_telemetry_reports_peak_rss_on_linux() { + fn run_with_telemetry_reports_peak_rss() { let program = verified( vec![], vec![function(0, 0, vec![Instruction::Halt], Vec::new())], @@ -13678,27 +14323,10 @@ mod tests { let (_, telemetry) = super::run_with_telemetry(&program, &mut host, &Limits::default()); assert!( telemetry.rss_kb > 0, - "peak RSS should be read from /proc on Linux" + "peak RSS should be read from the host" ); } - #[test] - fn parse_peak_rss_kb_selects_vmhwm_not_vmrss() { - let status = "Name: bamts-runtime -VmRSS: 64000 kB -VmHWM: 32000 kB -VmPeak: 64000 kB"; - assert_eq!(super::parse_peak_rss_kb(status), Some(32_000)); - } - - #[test] - fn parse_peak_rss_kb_returns_none_when_only_vmrss_present() { - let status = "Name: bamts-runtime -VmRSS: 64000 kB -VmPeak: 64000 kB"; - assert_eq!(super::parse_peak_rss_kb(status), None); - } - fn namespace_descriptor_entry() -> Function { function( 0, @@ -14199,6 +14827,52 @@ VmPeak: 64000 kB"; )); } + #[test] + fn a_class_accessor_is_hidden_and_stays_hidden_when_its_other_half_arrives() { + with_machine(Limits::default(), |machine| { + let object = machine + .allocate(HeapEntry::Object { + properties: PropertyMap::default(), + prototype: Some(machine.intrinsics.object_prototype), + extensible: true, + boxed_primitive: None, + }) + .unwrap(); + let index = machine.runtime_slot(object).unwrap().unwrap(); + let key = PropertyKey::Named(EcmaString::encode("value")); + let enumerable = |machine: &Machine<'_, TestHost>| match &machine.heap[index] { + HeapEntry::Object { properties, .. } => properties + .get(&PropertyKey::Named(EcmaString::encode("value"))) + .map(Property::enumerable), + _ => None, + }; + machine + .define_accessor( + object, + key.clone(), + Value::int32(1), + AccessorKind::ClassGetter, + ) + .unwrap(); + assert_eq!(enumerable(machine), Some(false)); + machine + .define_accessor(object, key, Value::int32(2), AccessorKind::ClassSetter) + .unwrap(); + assert_eq!(enumerable(machine), Some(false)); + let HeapEntry::Object { properties, .. } = &machine.heap[index] else { + panic!("expected an object"); + }; + assert!(matches!( + properties.get(&PropertyKey::Named(EcmaString::encode("value"))), + Some(Property::Accessor { + getter: Some(_), + setter: Some(_), + .. + }) + )); + }); + } + #[test] fn define_accessor_charges_only_net_growth_after_validation() { with_machine(Limits::default(), |machine| { @@ -19203,6 +19877,55 @@ VmPeak: 64000 kB"; )); } + #[test] + fn a_failed_global_store_is_silent_only_in_sloppy_code() { + fn store_to_read_only_global(is_sloppy: bool) -> Result { + let entry = Function::new( + None, + 0, + 0, + 1, + FunctionFlags { + is_sloppy, + ..FunctionFlags::default() + }, + vec![ + Instruction::LoadConst { + dst: reg(0), + constant: cid(1), + }, + Instruction::StoreGlobal { + name: cid(0), + value: reg(0), + }, + Instruction::Return { value: reg(0) }, + ], + vec![], + ); + let program = verified( + vec![ + Constant::String(EcmaString::encode("undefined")), + Constant::Int32(1), + ], + vec![entry], + ); + let mut host = TestHost; + Machine::new(&program, &mut host, Limits::default()) + .run() + .map(|execution| execution.value) + .map_err(|error| error.kind) + } + + assert_eq!(store_to_read_only_global(true), Ok(Value::int32(1))); + assert!(matches!( + store_to_read_only_global(false), + Err(RuntimeErrorKind::UncaughtThrow { + origin: ThrowOrigin::TypeError { .. }, + .. + }) + )); + } + #[test] fn globals_store_load_and_typeof_undeclared() { // StoreGlobal x=5; TypeOfGlobal y (undeclared) -> "undefined"; @@ -19516,6 +20239,7 @@ VmPeak: 64000 kB"; operation: "global is not defined", }, constructor_name: None, + message: Some(EcmaString::encode("missing is not defined")), } ); } @@ -19554,6 +20278,7 @@ VmPeak: 64000 kB"; value: Value::UNDEFINED, origin: ThrowOrigin::TypeError { operation: "call" }, constructor_name: None, + message: None, } ); } @@ -19664,12 +20389,12 @@ VmPeak: 64000 kB"; } #[test] - fn thrown_plain_error_reports_constructor_name() { - // `throw new Error()` — the base Error constructor. + fn a_builtin_with_no_construct_cannot_be_called_with_new() { + // `new Math.abs()`. let module = verified( vec![ - Constant::String(EcmaString::encode("Error")), - Constant::Undefined, + Constant::String(EcmaString::encode("Math")), + Constant::String(EcmaString::encode("abs")), ], vec![function( 0, @@ -19683,13 +20408,18 @@ VmPeak: 64000 kB"; dst: reg(1), constant: cid(1), }, - Instruction::CreateArray { dst: reg(2) }, + Instruction::GetProperty { + dst: reg(2), + object: reg(0), + key: reg(1), + }, + Instruction::CreateArray { dst: reg(3) }, Instruction::Construct { - dst: reg(3), - callee: reg(0), - arguments: reg(2), + dst: reg(4), + callee: reg(2), + arguments: reg(3), }, - Instruction::Throw { value: reg(3) }, + Instruction::Return { value: reg(4) }, ], Vec::new(), )], @@ -19698,45 +20428,164 @@ VmPeak: 64000 kB"; let error = Machine::new(&module, &mut host, Limits::default()) .run() .unwrap_err(); - match error.kind { + assert!(matches!( + error.kind, RuntimeErrorKind::UncaughtThrow { - origin, - constructor_name, + origin: ThrowOrigin::TypeError { + operation: "construct" + }, .. - } => { - assert_eq!(origin, ThrowOrigin::Bytecode); - assert_eq!( - constructor_name.as_ref().map(|s| s.to_utf8_lossy()), - Some("Error".to_owned()) - ); } - other => panic!("expected UncaughtThrow, got {other:?}"), - } + )); } #[test] - fn thrown_engine_type_error_has_no_constructor_name() { - // Engine-originated throws (ThrowOrigin::TypeError) already name - // their constructor in the origin variant. The constructor_name - // field must remain None so the oracle does not double-resolve. + fn a_thrown_error_reports_its_message_and_a_plain_value_reports_none() { + // `throw new Error("boom")`. let module = verified( - vec![Constant::Int32(0)], + vec![ + Constant::String(EcmaString::encode("Error")), + Constant::String(EcmaString::encode("boom")), + ], vec![function( 0, - 3, + 4, vec![ - Instruction::LoadConst { + Instruction::LoadGlobal { dst: reg(0), - constant: cid(0), + name: cid(0), }, - Instruction::CreateArray { dst: reg(1) }, - Instruction::Call { - dst: reg(2), - callee: reg(0), - this_value: reg(0), - arguments: reg(1), + Instruction::LoadConst { + dst: reg(1), + constant: cid(1), }, - Instruction::Return { value: reg(2) }, + Instruction::CreateArray { dst: reg(2) }, + Instruction::ArrayPush { + array: reg(2), + value: reg(1), + }, + Instruction::Construct { + dst: reg(3), + callee: reg(0), + arguments: reg(2), + }, + Instruction::Throw { value: reg(3) }, + ], + Vec::new(), + )], + ); + let mut host = TestHost; + let error = Machine::new(&module, &mut host, Limits::default()) + .run() + .unwrap_err(); + let RuntimeErrorKind::UncaughtThrow { message, .. } = error.kind else { + panic!("expected an uncaught throw"); + }; + assert_eq!( + message.map(|text| text.to_utf8_lossy()).as_deref(), + Some("boom") + ); + + // `throw "boom"`: a thrown string has no `message` property. + let module = verified( + vec![Constant::String(EcmaString::encode("boom"))], + vec![function( + 0, + 1, + vec![ + Instruction::LoadConst { + dst: reg(0), + constant: cid(0), + }, + Instruction::Throw { value: reg(0) }, + ], + Vec::new(), + )], + ); + let mut host = TestHost; + let error = Machine::new(&module, &mut host, Limits::default()) + .run() + .unwrap_err(); + let RuntimeErrorKind::UncaughtThrow { message, .. } = error.kind else { + panic!("expected an uncaught throw"); + }; + assert_eq!(message, None); + } + + #[test] + fn thrown_plain_error_reports_constructor_name() { + // `throw new Error()` — the base Error constructor. + let module = verified( + vec![ + Constant::String(EcmaString::encode("Error")), + Constant::Undefined, + ], + vec![function( + 0, + 5, + vec![ + Instruction::LoadGlobal { + dst: reg(0), + name: cid(0), + }, + Instruction::LoadConst { + dst: reg(1), + constant: cid(1), + }, + Instruction::CreateArray { dst: reg(2) }, + Instruction::Construct { + dst: reg(3), + callee: reg(0), + arguments: reg(2), + }, + Instruction::Throw { value: reg(3) }, + ], + Vec::new(), + )], + ); + let mut host = TestHost; + let error = Machine::new(&module, &mut host, Limits::default()) + .run() + .unwrap_err(); + match error.kind { + RuntimeErrorKind::UncaughtThrow { + origin, + constructor_name, + .. + } => { + assert_eq!(origin, ThrowOrigin::Bytecode); + assert_eq!( + constructor_name.as_ref().map(|s| s.to_utf8_lossy()), + Some("Error".to_owned()) + ); + } + other => panic!("expected UncaughtThrow, got {other:?}"), + } + } + + #[test] + fn thrown_engine_type_error_has_no_constructor_name() { + // Engine-originated throws (ThrowOrigin::TypeError) already name + // their constructor in the origin variant. The constructor_name + // field must remain None so the oracle does not double-resolve. + let module = verified( + vec![Constant::Int32(0)], + vec![function( + 0, + 3, + vec![ + Instruction::LoadConst { + dst: reg(0), + constant: cid(0), + }, + Instruction::CreateArray { dst: reg(1) }, + Instruction::Call { + dst: reg(2), + callee: reg(0), + this_value: reg(0), + arguments: reg(1), + }, + Instruction::Return { value: reg(2) }, ], Vec::new(), )], @@ -21300,6 +22149,322 @@ VmPeak: 64000 kB"; assert!(machine.frames.is_empty()); } + #[test] + fn computed_property_keys_run_user_conversion() { + fn key_text( + machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + let text = machine + .allocate(HeapEntry::String(EcmaString::encode("converted"))) + .map_err(EvalFailure::Runtime)?; + Ok(intrinsics::BuiltinOutcome::Value(text)) + } + let program = verified( + vec![ + Constant::String(EcmaString::encode("converted")), + Constant::String(EcmaString::encode("toString")), + Constant::Int32(7), + ], + vec![function( + 0, + 6, + vec![ + Instruction::CreateObject { dst: reg(0) }, + Instruction::LoadConst { + dst: reg(1), + constant: cid(0), + }, + Instruction::LoadConst { + dst: reg(2), + constant: cid(2), + }, + Instruction::SetProperty { + object: reg(0), + key: reg(1), + value: reg(2), + }, + Instruction::LoadGlobal { + dst: reg(3), + name: cid(1), + }, + Instruction::GetProperty { + dst: reg(4), + object: reg(0), + key: reg(3), + }, + Instruction::Return { value: reg(4) }, + ], + Vec::new(), + )], + ); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + let id = machine + .intrinsics + .builtins + .register(intrinsics::BuiltinDef { + name: "key text", + length: 0, + handler: key_text::, + }); + let to_string = intrinsics::native_function(&mut machine.heap, id, "key text", 0); + let key = machine + .allocate(HeapEntry::Object { + properties: PropertyMap::default(), + prototype: Some(machine.intrinsics.object_prototype), + boxed_primitive: None, + extensible: true, + }) + .unwrap(); + machine + .set_data_property(key, "toString", to_string) + .unwrap(); + machine.test_set_global("toString", key); + assert_eq!(machine.run().unwrap().value, Value::int32(7)); + } + + #[test] + fn closure_prototypes_follow_the_function_kind() { + let plain = function(0, 1, vec![Instruction::Halt], Vec::new()); + let kinded = |is_async, is_generator| { + Function::new( + None, + 0, + 0, + 1, + FunctionFlags { + is_async, + is_generator, + ..FunctionFlags::default() + }, + vec![Instruction::Halt], + Vec::new(), + ) + }; + let program = verified( + Vec::new(), + vec![ + plain, + kinded(false, true), + kinded(true, false), + kinded(true, true), + ], + ); + let mut host = TestHost; + let machine = Machine::new(&program, &mut host, Limits::default()); + let kind = |index| machine.closure_prototype(ModuleId::new(0), FunctionId::new(index)); + let builtins = &machine.intrinsics.builtins; + assert_eq!(kind(0), machine.intrinsics.function_prototype); + assert_eq!(kind(1), builtins.generator_function_prototype()); + assert_eq!(kind(2), builtins.async_function_prototype()); + assert_eq!(kind(3), builtins.async_generator_function_prototype()); + assert!( + !machine.is_callable(kind(1)).unwrap(), + "only %Function.prototype% is itself callable" + ); + } + + #[test] + fn a_generator_closure_owns_a_prototype_inheriting_from_its_kind() { + let generator = Function::new( + None, + 0, + 0, + 1, + FunctionFlags { + is_generator: true, + ..FunctionFlags::default() + }, + vec![Instruction::Halt], + Vec::new(), + ); + let program = verified( + Vec::new(), + vec![ + function(0, 1, vec![Instruction::Halt], Vec::new()), + generator, + ], + ); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + let module = ModuleId::new(0); + assert!( + machine + .closure_properties(module, FunctionId::new(0)) + .unwrap() + .0 + .is_empty() + ); + let properties = machine + .closure_properties(module, FunctionId::new(1)) + .unwrap(); + let Some(Property::Data { + value, + writable: true, + enumerable: false, + configurable: false, + }) = properties.get(&PropertyKey::Named(EcmaString::encode("prototype"))) + else { + panic!("a generator function's prototype is a writable data property"); + }; + assert_eq!( + machine.internal_get_prototype_of(*value).unwrap(), + Some(machine.intrinsics.builtins.generator_prototype()) + ); + } + + #[test] + fn arithmetic_converts_object_operands_left_first() { + fn seven( + _machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + Ok(intrinsics::BuiltinOutcome::Value(Value::int32(7))) + } + fn fail( + _machine: &mut Machine<'_, H>, + _this: Value, + _args: &[Value], + _constructing: bool, + ) -> Result { + Err(EvalFailure::ThrowValue(Value::int32(1))) + } + let program = verified( + Vec::new(), + vec![function(0, 1, vec![Instruction::Halt], Vec::new())], + ); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + let mut object_with = |name, handler| { + let id = machine + .intrinsics + .builtins + .register(intrinsics::BuiltinDef { + name, + length: 0, + handler, + }); + let value_of = intrinsics::native_function(&mut machine.heap, id, name, 0); + let object = machine + .allocate(HeapEntry::Object { + properties: PropertyMap::default(), + prototype: Some(machine.intrinsics.object_prototype), + boxed_primitive: None, + extensible: true, + }) + .unwrap(); + machine + .set_data_property(object, "valueOf", value_of) + .unwrap(); + object + }; + let number = object_with("seven", seven::); + let thrower = object_with("fail", fail::); + assert_eq!( + machine + .numeric_binary(BinaryOp::Multiply, number, Value::int32(2)) + .unwrap(), + Value::int32(14) + ); + assert_eq!( + machine.eval_unary(UnaryOp::Negate, number).unwrap(), + Value::int32(-7i32 as u32) + ); + assert_eq!( + machine.eval_unary(UnaryOp::BitwiseNot, number).unwrap(), + Value::int32(-8i32 as u32) + ); + assert!( + matches!( + machine.numeric_binary(BinaryOp::Subtract, thrower, Value::int32(1)), + Err(EvalFailure::ThrowValue(_)) + ), + "a conversion failure propagates" + ); + } + + #[test] + fn number_value_keeps_the_sign_of_zero() { + assert_eq!(number_value(0.0), Value::int32(0)); + assert!(matches!( + number_value(-0.0).decode(), + Some(Decoded::Number(zero)) if zero == 0.0 && zero.is_sign_negative() + )); + assert_eq!(number_value(-1.0), Value::int32(-1i32 as u32)); + } + + #[test] + fn construct_with_no_calling_activation_applies_the_receiver_rule() { + // The native tiers keep no interpreter frame, so a constructor entered + // from a builtin has nothing to return to. A primitive completion must + // still yield the receiver, and an object completion must replace it. + let program = verified( + vec![Constant::Int32(7)], + vec![ + function(0, 1, vec![Instruction::Halt], Vec::new()), + function( + 0, + 1, + vec![ + Instruction::LoadConst { + dst: reg(0), + constant: cid(0), + }, + Instruction::Return { value: reg(0) }, + ], + Vec::new(), + ), + function( + 0, + 1, + vec![ + Instruction::CreateArray { dst: reg(0) }, + Instruction::Return { value: reg(0) }, + ], + Vec::new(), + ), + ], + ); + let mut host = TestHost; + let mut machine = Machine::new(&program, &mut host, Limits::default()); + machine.frames.clear(); + machine.live_registers = 0; + let mut constructor = |index: u32| { + machine + .allocate(HeapEntry::Function { + module: ModuleId::new(0), + function: FunctionId::new(index), + captures: Vec::new(), + context: None, + properties: PropertyMap::default(), + prototype: Some(machine.intrinsics.function_prototype), + extensible: true, + }) + .unwrap() + }; + let returns_primitive = constructor(1); + let returns_object = constructor(2); + + let receiver = machine.construct_value(returns_primitive, &[]).unwrap(); + let index = machine.runtime_slot(receiver).unwrap().unwrap(); + assert!( + matches!(machine.heap[index], HeapEntry::Object { .. }), + "a primitive completion yields the constructed receiver" + ); + let replaced = machine.construct_value(returns_object, &[]).unwrap(); + let index = machine.runtime_slot(replaced).unwrap().unwrap(); + assert!( + matches!(machine.heap[index], HeapEntry::Array { .. }), + "an object completion replaces the receiver" + ); + assert!(machine.frames.is_empty()); + } + #[test] fn native_callback_throw_uncaught_at_outer_call_site() { let entry = function( @@ -21387,6 +22552,7 @@ VmPeak: 64000 kB"; value: Value::int32(7), origin: ThrowOrigin::Bytecode, constructor_name: None, + message: None, } ); assert!(machine.callback_boundaries.is_empty()); @@ -26597,6 +27763,7 @@ VmPeak: 64000 kB"; value: Value::int32(7), origin: ThrowOrigin::Bytecode, constructor_name: None, + message: None, } ); // The later job stays queued and unrun. @@ -26644,6 +27811,7 @@ VmPeak: 64000 kB"; value: Value::int32(7), origin: ThrowOrigin::Bytecode, constructor_name: None, + message: None, } ); // The exception converts before any drain, so the queued microtask and diff --git a/crates/bamts-runtime/src/native.rs b/crates/bamts-runtime/src/native.rs index 5a21a71a..5d085bcf 100644 --- a/crates/bamts-runtime/src/native.rs +++ b/crates/bamts-runtime/src/native.rs @@ -89,6 +89,8 @@ fn unary_to_selector(op: bamts_bytecode::UnaryOp) -> u32 { UnaryOp::Negate => 3, UnaryOp::BitwiseNot => 4, UnaryOp::LogicalNot => 5, + UnaryOp::ToString => 6, + UnaryOp::ToPropertyKey => 7, } } @@ -149,6 +151,8 @@ fn accessor_to_selector(kind: bamts_bytecode::AccessorKind) -> u32 { match kind { AccessorKind::Getter => 0, AccessorKind::Setter => 1, + AccessorKind::ClassGetter => 2, + AccessorKind::ClassSetter => 3, } } @@ -158,6 +162,7 @@ fn descriptor_slot_to_selector(slot: bamts_bytecode::DescriptorSlot) -> u32 { DescriptorSlot::Value => 0, DescriptorSlot::Getter => 1, DescriptorSlot::Setter => 2, + DescriptorSlot::Entire => 4, } } @@ -167,6 +172,7 @@ fn descriptor_slot_from_selector(selector: u32) -> Option Some(DescriptorSlot::Value), 1 => Some(DescriptorSlot::Getter), 2 => Some(DescriptorSlot::Setter), + 4 => Some(DescriptorSlot::Entire), _ => None, } } @@ -565,6 +571,7 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { value, origin, constructor_name: None, + message: None, }, site.function.get() as usize, fault_pc, @@ -576,6 +583,7 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { value, origin, constructor_name: None, + message: None, }, function, pc, @@ -1868,26 +1876,23 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { let handle = self.code_ref(target.module); let flags = handle.code(target.module).functions()[index].flags(); if flags.is_async && flags.is_generator { - let created = self - .machine - .borrow_mut() - .create_async_generator(GeneratorStart { - target, - captures: captures.to_vec(), - context, - this_value: this, - new_target, - args: args.to_vec(), - }); - return match created { + // An async generator's prologue runs on the shared Machine, like + // the rest of its body. + let start = GeneratorStart { + target, + captures: captures.to_vec(), + context, + this_value: this, + new_target, + args: args.to_vec(), + }; + let started = self.machine.borrow_mut().start_generator(start, flags); + return match started { Ok(generator) => InvokeOutcome::Value(generator), - Err(kind) => { - self.pending_fatal_kind.set(Some(kind)); - InvokeOutcome::Fatal - } + Err(failure) => self.failure_outcome(failure), }; } - if flags.is_generator && !flags.is_async { + if flags.is_generator { let created = self.machine.borrow_mut().create_generator(GeneratorStart { target, captures: captures.to_vec(), @@ -1896,13 +1901,21 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { new_target, args: args.to_vec(), }); - return match created { - Ok(generator) => InvokeOutcome::Value(generator), + let generator = match created { + Ok(generator) => generator, Err(kind) => { self.pending_fatal_kind.set(Some(kind)); - InvokeOutcome::Fatal + return InvokeOutcome::Fatal; } }; + if flags.binds_at_call { + let completion = GeneratorCompletion::Normal(Value::UNDEFINED); + match self.resume_generator(generator, completion) { + InvokeOutcome::Value(_) => {} + other => return other, + } + } + return InvokeOutcome::Value(generator); } if flags.is_async && !flags.is_generator { // Async calls route through the shared reference Machine state @@ -2340,12 +2353,20 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { InvokeOutcome::Value(close) => close, outcome => return map_outcome(outcome), }; + if matches!(close.decode(), Some(Decoded::Undefined | Decoded::Null)) { + return Ok(Value::UNDEFINED); + } let callable = match self.machine.borrow().is_callable(close) { Ok(callable) => callable, Err(failure) => return map_outcome(self.failure_outcome(failure)), }; if !callable { - return Ok(Value::UNDEFINED); + return map_outcome(InvokeOutcome::Threw( + Value::UNDEFINED, + ThrowOrigin::TypeError { + operation: "iterator return method is not callable", + }, + )); } frame.set_register(called_reg, Value::TRUE); map_outcome(self.invoke_callee(close, target, &[], Value::UNDEFINED)) @@ -2397,11 +2418,23 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { } fn array_extend_active(&self, array: Value, iterable: Value) -> InvokeOutcome { + // The iteration runs user code, and only these locals hold its operands. + let mark = self + .machine + .borrow_mut() + .hold_temporaries(&[array, iterable]); + let outcome = self.drain_iterable_into(array, iterable); + self.machine.borrow_mut().release_temporaries(mark); + outcome + } + + fn drain_iterable_into(&self, array: Value, iterable: Value) -> InvokeOutcome { let iterator = match self.get_iterator_active(iterable, bamts_bytecode::IteratorKind::Sync) { InvokeOutcome::Value(iterator) => iterator, other => return other, }; + self.machine.borrow_mut().hold_temporaries(&[iterator]); loop { let (done, value) = match self.iterator_next_active(iterator) { Ok(step) => step, @@ -3155,6 +3188,7 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { value, origin, constructor_name: None, + message: None, }, function.get() as usize, 0, @@ -3211,6 +3245,45 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { } } + /// The function the innermost activation runs. + fn running_function(&self) -> crate::RuntimeFunction { + self.activations + .borrow() + .last() + .expect("a helper runs inside an activation") + .target + } + + fn running_is_sloppy(&self) -> bool { + let target = self.running_function(); + self.machine + .borrow() + .is_sloppy(target.module, target.function.get() as usize) + } + + fn load_this(&self) -> HelperResult { + let target = self.running_function(); + let this_value = self + .activations + .borrow() + .last() + .map_or(Value::UNDEFINED, |activation| activation.this_value); + let bound = self.machine.borrow_mut().bind_this( + target.module, + target.function.get() as usize, + this_value, + ); + match bound { + Ok(value) => { + if let Some(activation) = self.activations.borrow_mut().last_mut() { + activation.this_value = value; + } + HelperResult::normal(value) + } + Err(failure) => self.fail(failure), + } + } + fn fail(&self, failure: EvalFailure) -> HelperResult { match failure { EvalFailure::Throw(origin) => { @@ -3290,9 +3363,13 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { return HelperResult::normal(existing); } let prototype = self.machine.borrow().intrinsics.array_prototype; + let properties = self + .machine + .borrow() + .arguments_properties(self.running_is_sloppy()); let allocated = self.machine.borrow_mut().allocate(HeapEntry::Array { elements: args, - properties: PropertyMap::default(), + properties, prototype: Some(prototype), extensible: true, length_writable: true, @@ -3339,6 +3416,22 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { } let kind = self.machine.borrow().callee_kind(callee); match kind { + Ok(CalleeKind::Builtin { id }) + if !self + .machine + .borrow() + .intrinsics + .builtins + .is_constructable(id) => + { + self.pending_throw.set(Some(PendingThrow { + value: Value::UNDEFINED, + origin: ThrowOrigin::TypeError { + operation: "construct", + }, + })); + HelperResult::throw(Value::UNDEFINED) + } Ok(CalleeKind::Builtin { id }) => { let result = self.machine.borrow_mut().call_builtin_with_new_target( id, @@ -3392,7 +3485,7 @@ impl<'m, 'h, H: Host> NativeEngine<'m, 'h, H> { let handle = self.code_ref(target.module); handle.code(target.module).functions()[target.function.get() as usize].flags() }; - if flags.is_async && !flags.is_generator { + if !flags.is_constructable { self.pending_throw.set(Some(PendingThrow { value: Value::UNDEFINED, origin: ThrowOrigin::TypeError { @@ -3915,13 +4008,22 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { let context = self.machine.borrow().context_global; match materialized { Ok(captures) => { - let prototype = self.machine.borrow().intrinsics.function_prototype; + let prototype = + self.machine.borrow().closure_prototype(module, function); + let properties = self + .machine + .borrow_mut() + .closure_properties(module, function); + let properties = match properties { + Ok(properties) => properties, + Err(kind) => break 'result self.fatal(kind), + }; self.allocated(HeapEntry::Function { module, function, captures, context, - properties: PropertyMap::default(), + properties, prototype: Some(prototype), extensible: true, }) @@ -3931,7 +4033,7 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { } HelperCall::GetProperty { object, key } => { let key = { - let coerced = self.machine.borrow().to_property_key(key); + let coerced = self.machine.borrow_mut().computed_property_key(object, key); match coerced { Ok(key) => key, Err(failure) => break 'result self.fail(failure), @@ -3945,7 +4047,7 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { } HelperCall::SetProperty { object, key, value } => { let key = { - let coerced = self.machine.borrow().to_property_key(key); + let coerced = self.machine.borrow_mut().computed_property_key(object, key); match coerced { Ok(key) => key, Err(failure) => break 'result self.fail(failure), @@ -3957,6 +4059,9 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { .internal_set(object, key, value, object); match outcome { Ok(true) => HelperResult::normal(Value::UNDEFINED), + Ok(false) if self.running_is_sloppy() => { + HelperResult::normal(Value::UNDEFINED) + } Ok(false) => { self.pending_throw.set(Some(PendingThrow { value: Value::UNDEFINED, @@ -3971,13 +4076,21 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { } HelperCall::DeleteProperty { object, key } => { let key = { - let coerced = self.machine.borrow().to_property_key(key); + let coerced = self.machine.borrow_mut().computed_property_key(object, key); match coerced { Ok(key) => key, Err(failure) => break 'result self.fail(failure), } }; let deleted = self.machine.borrow_mut().internal_delete(object, &key); + let target = self.running_function(); + let deleted = deleted.and_then(|deleted| { + self.machine.borrow().deletion_result( + target.module, + target.function.get() as usize, + deleted, + ) + }); match deleted { Ok(deleted) => HelperResult::normal(Value::boolean(deleted)), Err(failure) => self.fail(failure), @@ -3998,7 +4111,7 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { } }; let key = { - let coerced = self.machine.borrow().to_property_key(key); + let coerced = self.machine.borrow_mut().observable_property_key(key); match coerced { Ok(key) => key, Err(failure) => break 'result self.fail(failure), @@ -4015,7 +4128,7 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { } HelperCall::DefineDataProperty { object, key, value } => { let key = { - let coerced = self.machine.borrow().to_property_key(key); + let coerced = self.machine.borrow_mut().observable_property_key(key); match coerced { Ok(key) => key, Err(failure) => break 'result self.fail(failure), @@ -4092,7 +4205,7 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { receiver, key, } => { - let key = match self.machine.borrow().to_property_key(key) { + let key = match self.machine.borrow_mut().observable_property_key(key) { Ok(key) => key, Err(failure) => break 'result self.fail(failure), }; @@ -4105,7 +4218,7 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { key, value, } => { - let key = match self.machine.borrow().to_property_key(key) { + let key = match self.machine.borrow_mut().observable_property_key(key) { Ok(key) => key, Err(failure) => break 'result self.fail(failure), }; @@ -4288,10 +4401,12 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { } } HelperCall::StoreGlobal { name, value } => { + let sloppy = self.running_is_sloppy(); let stored = self.machine.borrow_mut().store_global( module, ConstantId::new(name), value, + sloppy, ); match stored { Ok(()) => HelperResult::normal(Value::UNDEFINED), @@ -4310,12 +4425,7 @@ impl<'m, 'h, H: Host> NativeOps for NativeEngine<'m, 'h, H> { }; self.allocated(HeapEntry::String(text)) } - HelperCall::LoadThis => HelperResult::normal( - self.activations - .borrow() - .last() - .map_or(Value::UNDEFINED, |activation| activation.this_value), - ), + HelperCall::LoadThis => self.load_this(), HelperCall::LoadArguments => self.load_arguments(), HelperCall::LoadNewTarget => HelperResult::normal( self.activations @@ -5524,6 +5634,7 @@ mod tests { is_async: true, is_generator: false, is_constructable: false, + ..FunctionFlags::default() }, code, Vec::new(), @@ -8743,6 +8854,7 @@ mod tests { is_async: false, is_generator: true, is_constructable: false, + ..FunctionFlags::default() }, code, Vec::new(), @@ -9605,6 +9717,7 @@ mod tests { value: Value::UNDEFINED, origin: ThrowOrigin::TypeError { operation }, constructor_name: None, + message: None, } ); }; @@ -9889,6 +10002,7 @@ mod tests { operation: "global is not defined", }, constructor_name: None, + message: Some(EcmaString::encode("missing_global is not defined")), } ); }; diff --git a/crates/bamts-runtime/src/proxy_reflect_integration.rs b/crates/bamts-runtime/src/proxy_reflect_integration.rs index a5dfbe5e..78a24564 100755 --- a/crates/bamts-runtime/src/proxy_reflect_integration.rs +++ b/crates/bamts-runtime/src/proxy_reflect_integration.rs @@ -2247,8 +2247,8 @@ fn it18_constructability_expected_table_sweep() { with_machine(|machine| { // Expected [[Construct]] presence per installed global, derived from // the B1 production registration census: every - // install_constructor_function site is true; every plain function, - // namespace object, and BigInt/Symbol is false. + // install_constructor_function site is true; every plain function + // and namespace object is false. const CONSTRUCTABLE: &[&str] = &[ "Object", "Array", @@ -2289,10 +2289,10 @@ fn it18_constructability_expected_table_sweep() { "Float64Array", "BigInt64Array", "BigUint64Array", - ]; - const NON_CONSTRUCTABLE: &[&str] = &[ "BigInt", "Symbol", + ]; + const NON_CONSTRUCTABLE: &[&str] = &[ "Reflect", "Math", "JSON", diff --git a/crates/bamts-runtime/src/regexp.rs b/crates/bamts-runtime/src/regexp.rs index 655c62fb..465e59dd 100644 --- a/crates/bamts-runtime/src/regexp.rs +++ b/crates/bamts-runtime/src/regexp.rs @@ -1,4 +1,4 @@ -use std::collections::{BTreeMap, HashSet}; +use std::collections::HashSet; use std::ops::Range; use bamts_bytecode::EcmaString; @@ -13,7 +13,7 @@ pub(crate) const STEP_BUDGET: usize = 100_000; /// Identity of a backtracking state: input position plus capture bindings. Two /// states sharing one continue identically, so a repetition level keeps only the /// first — that is what stops equivalent alternatives multiplying per level. -type StateKey = (usize, Vec>); +type StateKey = (usize, usize, Vec>); #[derive(Clone, Copy, Debug, Default, Eq, PartialEq)] pub(crate) struct Flags { @@ -23,6 +23,8 @@ pub(crate) struct Flags { pub(crate) dot_all: bool, pub(crate) unicode: bool, pub(crate) sticky: bool, + /// Inside a lookbehind, which reads from right to left. Never set by a flag. + backward: bool, } impl Flags { @@ -132,7 +134,8 @@ impl RegexError { pub(crate) struct Match { pub(crate) range: Range, pub(crate) captures: Vec>>, - pub(crate) named: BTreeMap>>, + /// Each group name once, in source order, with the group that participated. + pub(crate) named: Vec<(String, Option>)>, } #[derive(Clone, Debug)] @@ -140,17 +143,22 @@ pub(crate) struct Regex { expression: Node, flags: Flags, capture_count: usize, - names: BTreeMap, + names: Vec<(String, Vec)>, } impl Regex { pub(crate) fn compile(pattern: &EcmaString, flags: &EcmaString) -> Result { let flags = Flags::parse(flags)?; - let mut parser = Parser::new(pattern.as_units(), flags.unicode); + let mut parser = Parser::new(pattern.as_units(), flags); let expression = parser.parse_disjunction(None)?; if parser.peek().is_some() { return Err(parser.error("unexpected token")); } + for name in &parser.referenced_names { + if !parser.names.iter().any(|(known, _)| known == name) { + return Err(parser.error("invalid named reference")); + } + } Ok(Self { expression, flags, @@ -179,7 +187,7 @@ impl Regex { captures: vec![None; self.capture_count + 1], }; if let Some(mut matched) = self - .match_node(&self.expression, input, state)? + .match_node(self.flags, &self.expression, input, state)? .into_iter() .next() { @@ -187,7 +195,12 @@ impl Regex { let named = self .names .iter() - .map(|(name, index)| (name.clone(), matched.captures[*index].clone())) + .map(|(name, indices)| { + let participant = indices + .iter() + .find_map(|index| matched.captures[*index].clone()); + (name.clone(), participant) + }) .collect(); return Ok(Some(Match { range: position..matched.position, @@ -204,60 +217,61 @@ impl Regex { fn match_node( &self, + flags: Flags, node: &Node, input: &[u16], state: State, ) -> Result, RegexError> { match node { - Node::Sequence(nodes) => self.match_sequence(nodes, input, state), + Node::Sequence(nodes) => self.match_sequence(flags, nodes, input, state), Node::Alternation(branches) => { let mut out = Vec::new(); for branch in branches { - out.extend(self.match_node(branch, input, state.clone())?); + out.extend(self.match_node(flags, branch, input, state.clone())?); } Ok(out) } Node::Literal(expected) => { - if state.position == input.len() { + let Some((actual, width)) = adjacent_code_point(input, state.position, flags) + else { return Ok(Vec::new()); - } - let (actual, width) = next_code_point(input, state.position, self.flags.unicode); + }; Ok(self - .code_point_eq(actual, *expected) - .then(|| state.advanced(width)) + .code_point_eq(flags, actual, *expected) + .then(|| state.moved(width, flags.backward)) .into_iter() .collect()) } Node::Dot => { - if state.position == input.len() { + let Some((actual, width)) = adjacent_code_point(input, state.position, flags) + else { return Ok(Vec::new()); - } - let (actual, width) = next_code_point(input, state.position, self.flags.unicode); - Ok((self.flags.dot_all || !is_line_terminator(actual)) - .then(|| state.advanced(width)) + }; + Ok((flags.dot_all || !is_line_terminator(actual)) + .then(|| state.moved(width, flags.backward)) .into_iter() .collect()) } Node::Class(class) => { - if state.position == input.len() { + let Some((actual, width)) = adjacent_code_point(input, state.position, flags) + else { return Ok(Vec::new()); - } - let (actual, width) = next_code_point(input, state.position, self.flags.unicode); + }; Ok(class - .matches(actual, self.flags.ignore_case, self.flags.unicode) - .then(|| state.advanced(width)) + .matches(actual, flags.ignore_case, flags.unicode) + .then(|| state.moved(width, flags.backward)) .into_iter() .collect()) } Node::Start => Ok((state.position == 0 - || (self.flags.multiline + || (flags.multiline && state.position > 0 && is_line_terminator(u32::from(input[state.position - 1])))) .then_some(state) .into_iter() .collect()), Node::End => Ok((state.position == input.len() - || (self.flags.multiline + || (flags.multiline && input .get(state.position) .is_some_and(|value| is_line_terminator(u32::from(*value))))) @@ -269,7 +283,7 @@ impl Regex { false } else { let low = input[state.position - 1]; - let value = if self.flags.unicode + let value = if flags.unicode && (0xdc00..=0xdfff).contains(&low) && state.position >= 2 && (0xd800..=0xdbff).contains(&input[state.position - 2]) @@ -278,13 +292,11 @@ impl Regex { } else { u32::from(low) }; - is_word(value, self.flags.ignore_case, self.flags.unicode) + is_word(value, flags.ignore_case, flags.unicode) }; let right = (state.position < input.len()) - .then(|| next_code_point(input, state.position, self.flags.unicode).0) - .is_some_and(|value| { - is_word(value, self.flags.ignore_case, self.flags.unicode) - }); + .then(|| next_code_point(input, state.position, flags.unicode).0) + .is_some_and(|value| is_word(value, flags.ignore_case, flags.unicode)); Ok(((left != right) == *positive) .then_some(state) .into_iter() @@ -293,57 +305,53 @@ impl Regex { Node::Group { index, body } => { let begin = state.position; Ok(self - .match_node(body, input, state)? + .match_node(flags, body, input, state)? .into_iter() .map(|mut matched| { if let Some(index) = index { - matched.captures[*index] = Some(begin..matched.position); + matched.captures[*index] = Some(if flags.backward { + matched.position..begin + } else { + begin..matched.position + }); } matched }) .collect()) } + Node::Modified { flags, body } => self.match_node(*flags, body, input, state), Node::BackReference(index) => { let Some(range) = state.captures.get(*index).and_then(Clone::clone) else { return Ok(vec![state]); }; - Ok(self.match_backreference(input, range, state)) + Ok(self.match_backreference(flags, input, range, state)) + } + Node::NamedBackReference(name) => { + let Some((_, indices)) = self.names.iter().find(|(known, _)| known == name) else { + return Ok(Vec::new()); + }; + match indices + .iter() + .find_map(|index| state.captures[*index].clone()) + { + Some(range) => Ok(self.match_backreference(flags, input, range, state)), + None => Ok(vec![state]), + } } - Node::NamedBackReference(name) => match self.names.get(name) { - Some(index) => Ok(self.match_node(&Node::BackReference(*index), input, state)?), - None => Ok(Vec::new()), - }, Node::Look { body, behind, positive, } => { - let candidates = if *behind { - let mut candidates = Vec::new(); - let mut begin = 0; - loop { - let mut initial = state.clone(); - initial.position = begin; - candidates.extend( - self.match_node(body, input, initial)? - .into_iter() - .filter(|matched| matched.position == state.position), - ); - if begin == state.position { - break; - } - begin += next_code_point(input, begin, self.flags.unicode).1; - if begin > state.position { - break; - } - } - candidates - } else { - self.match_node(body, input, state.clone())? + let direction = Flags { + backward: *behind, + ..flags }; + let candidates = self.match_node(direction, body, input, state.clone())?; Ok(if *positive { candidates .into_iter() + .take(1) .map(|mut matched| { matched.position = state.position; matched @@ -359,32 +367,54 @@ impl Regex { } } - fn match_backreference(&self, input: &[u16], range: Range, state: State) -> Vec { - let mut captured = range.start; - let mut candidate = state.position; + fn match_backreference( + &self, + flags: Flags, + input: &[u16], + range: Range, + state: State, + ) -> Vec { + let width = range.len(); + let start = if flags.backward { + state.position.checked_sub(width) + } else { + Some(state.position).filter(|position| position + width <= input.len()) + }; + let Some(start) = start else { + return Vec::new(); + }; + let (mut captured, mut candidate) = (range.start, start); while captured < range.end { if candidate >= input.len() { return Vec::new(); } - let (left, left_width) = next_code_point(input, captured, self.flags.unicode); - let (right, right_width) = next_code_point(input, candidate, self.flags.unicode); - if captured + left_width > range.end || !self.code_point_eq(left, right) { + let (left, left_width) = next_code_point(input, captured, flags.unicode); + let (right, right_width) = next_code_point(input, candidate, flags.unicode); + if captured + left_width > range.end || !self.code_point_eq(flags, left, right) { return Vec::new(); } captured += left_width; candidate += right_width; } - let width = candidate - state.position; - vec![state.advanced(width)] + if candidate != start + width { + return Vec::new(); + } + vec![state.moved(width, flags.backward)] } fn match_sequence( &self, + flags: Flags, nodes: &[Node], input: &[u16], state: State, ) -> Result, RegexError> { - let Some((first, rest)) = nodes.split_first() else { + let split = if flags.backward { + nodes.split_last() + } else { + nodes.split_first() + }; + let Some((first, rest)) = split else { return Ok(vec![state]); }; if let Node::Repeat { @@ -392,93 +422,132 @@ impl Regex { min, max, greedy, + captures, } = first { - // Deduplicate states within each repetition level by (position, - // captures). Equivalent alternatives like (a|a|a) produce k - // identical states at every level — without dedup these multiply to - // k^n states, each cloning its captures Vec. Two states that share - // position and captures yield identical continuations, so collapsing - // duplicates is lossless. - // - // The hash set is allocated once and cleared per level so its bucket - // capacity is reused — ordinary patterns (1 state per level) pay one - // allocation for the whole repeat, not one per level. Dedup is kept - // even for single-source levels because the body itself can be an - // alternation (e.g. (a|a)*) whose branches yield duplicate - // (position, captures) states from one prior — skipping would let - // those duplicates through and reintroduce the multiplication. - // - // The step budget remains as a backstop for non-dedupable blowups - // (e.g. ((a)|(a))* where branches set different capture groups). - // On exhaustion it returns a distinct error rather than an empty - // result, so a caller can tell an aborted match from a real non-match. - let mut levels = vec![vec![state]]; - let limit = max.unwrap_or(input.len().saturating_add(*min).saturating_add(1)); - let mut total_states: usize = 1; - let mut seen: HashSet = HashSet::new(); - for count in 0..limit { - let mut next = Vec::new(); - seen.clear(); - for prior in &levels[count] { - for matched in self.match_node(body, input, prior.clone())? { - if matched.position == prior.position && count + 1 >= *min { - continue; - } - let key = ( - matched.position, - matched - .captures - .iter() - .map(|c| c.as_ref().map(|r| (r.start, r.end))) - .collect(), - ); - if seen.insert(key) { - next.push(matched); - } - } - } - if next.is_empty() { - break; - } - total_states += next.len(); - if total_states > STEP_BUDGET { - return Err(RegexError::budget_exhausted()); - } - levels.push(next); + self.match_repeat( + flags, + &Repeat { + body, + min: *min, + max: *max, + greedy: *greedy, + captures, + }, + rest, + input, + state, + ) + } else { + let mut result = Vec::new(); + for matched in self.match_node(flags, first, input, state)? { + result.extend(self.match_sequence(flags, rest, input, matched)?); } - let counts: Box> = if *greedy { - Box::new((*min..levels.len()).rev()) + Ok(result) + } + } + + /// `RepeatMatcher` as a depth-first walk with its own stack, because the number + /// of iterations follows the input. Results stay in priority order: a greedy + /// repeat tries another iteration before the continuation, and a lazy one after. + /// + /// A state already expanded cannot do better the second time, since everything + /// it reaches was reached earlier at a higher priority. Skipping it keeps + /// equivalent alternatives such as (a|a)* linear; the step budget is the backstop + /// for states that differ only in their captures, as in ((a)|(a))*. It grows with + /// the input, since a plain repeat over a long string takes a step per character. + fn match_repeat( + &self, + flags: Flags, + repeat: &Repeat<'_>, + rest: &[Node], + input: &[u16], + state: State, + ) -> Result, RegexError> { + enum Step { + Expand { state: State, count: usize }, + Continue(State), + } + let mut result = Vec::new(); + let mut steps = 0; + let mut seen: HashSet = HashSet::new(); + let mut stack = vec![Step::Expand { state, count: 0 }]; + while let Some(step) = stack.pop() { + let (state, count) = match step { + Step::Continue(state) => { + result.extend(self.match_sequence(flags, rest, input, state)?); + continue; + } + Step::Expand { state, count } => (state, count), + }; + let tracked = if repeat.max.is_some() { + count } else { - Box::new(*min..levels.len()) + count.min(repeat.min) }; - let mut result = Vec::new(); - for count in counts { - // Move ownership instead of cloning: each level is visited - // exactly once, so the deep copy of every captures allocation - // on each iteration was pure waste. - for candidate in std::mem::take(&mut levels[count]) { - result.extend(self.match_sequence(rest, input, candidate)?); - } + let key = ( + tracked, + state.position, + state + .captures + .iter() + .map(|capture| capture.as_ref().map(|range| (range.start, range.end))) + .collect(), + ); + if !seen.insert(key) { + continue; } - Ok(result) - } else { - let mut result = Vec::new(); - for matched in self.match_node(first, input, state)? { - result.extend(self.match_sequence(rest, input, matched)?); + steps += 1; + if steps > STEP_BUDGET.max(input.len().saturating_mul(2)) { + return Err(RegexError::budget_exhausted()); + } + let mut iterations = Vec::new(); + if repeat.max.is_none_or(|max| count < max) { + let mut attempt = state.clone(); + attempt.captures[repeat.captures.clone()].fill(None); + iterations = self.match_node(flags, repeat.body, input, attempt)?; + iterations + .retain(|matched| count < repeat.min || matched.position != state.position); + } + let stop = (count >= repeat.min).then_some(Step::Continue(state)); + let next = iterations.into_iter().rev().map(|state| Step::Expand { + state, + count: count + 1, + }); + if repeat.greedy { + stack.extend(stop); + stack.extend(next); + } else { + stack.extend(next); + stack.extend(stop); } - Ok(result) } + Ok(result) } - fn code_point_eq(&self, left: u32, right: u32) -> bool { + fn code_point_eq(&self, flags: Flags, left: u32, right: u32) -> bool { left == right - || (self.flags.ignore_case - && canonicalize(left, self.flags.unicode) - == canonicalize(right, self.flags.unicode)) + || (flags.ignore_case + && canonicalize(left, flags.unicode) == canonicalize(right, flags.unicode)) } } +/// The code point the matcher reads next, on whichever side of `position` that is. +fn adjacent_code_point(input: &[u16], position: usize, flags: Flags) -> Option<(u32, usize)> { + if !flags.backward { + return (position < input.len()).then(|| next_code_point(input, position, flags.unicode)); + } + let low = *input.get(position.checked_sub(1)?)?; + if flags.unicode + && (0xdc00..=0xdfff).contains(&low) + && position >= 2 + && (0xd800..=0xdbff).contains(&input[position - 2]) + { + return Some((combine_surrogates(input[position - 2], low), 2)); + } + Some((u32::from(low), 1)) +} + pub(crate) fn next_code_point(input: &[u16], position: usize, unicode: bool) -> (u32, usize) { let first = input[position]; if unicode @@ -493,6 +562,15 @@ pub(crate) fn next_code_point(input: &[u16], position: usize, unicode: bool) -> (u32::from(first), 1) } +/// A quantified atom, as `Regex::match_repeat` reads it. +struct Repeat<'a> { + body: &'a Node, + min: usize, + max: Option, + greedy: bool, + captures: &'a Range, +} + #[derive(Clone, Debug)] struct State { position: usize, @@ -500,8 +578,12 @@ struct State { } impl State { - fn advanced(mut self, count: usize) -> Self { - self.position += count; + fn moved(mut self, count: usize, backward: bool) -> Self { + if backward { + self.position -= count; + } else { + self.position += count; + } self } } @@ -520,6 +602,11 @@ enum Node { index: Option, body: Box, }, + /// `(?ims-ims:...)`: its body runs under these flags instead. + Modified { + flags: Flags, + body: Box, + }, BackReference(usize), NamedBackReference(String), Look { @@ -532,6 +619,8 @@ enum Node { min: usize, max: Option, greedy: bool, + /// The groups inside `body`, which each iteration starts without. + captures: Range, }, } @@ -579,12 +668,8 @@ impl ClassItem { Self::NotDigit => !(value <= 0x7f && (value as u8).is_ascii_digit()), Self::Word => is_word(value, ignore_case, unicode), Self::NotWord => !is_word(value, ignore_case, unicode), - Self::Space => { - char::from_u32(value).is_some_and(char::is_whitespace) || value == 0xfeff - } - Self::NotSpace => { - !(char::from_u32(value).is_some_and(char::is_whitespace) || value == 0xfeff) - } + Self::Space => is_space(value), + Self::NotSpace => !is_space(value), } } } @@ -751,6 +836,11 @@ fn is_word(value: u32, ignore_case: bool, unicode: bool) -> bool { value <= 0x7f && ((value as u8).is_ascii_alphanumeric() || value == u32::from(b'_')) } +/// `WhiteSpace` and `LineTerminator`. Rust's own set adds U+0085 and omits U+FEFF. +fn is_space(value: u32) -> bool { + value != 0x85 && (value == 0xfeff || char::from_u32(value).is_some_and(char::is_whitespace)) +} + fn is_line_terminator(value: u32) -> bool { matches!(value, 0x0a | 0x0d | 0x2028 | 0x2029) } @@ -996,30 +1086,48 @@ struct Parser<'a> { units: &'a [u16], position: usize, capture_count: usize, - names: BTreeMap, + names: Vec<(String, Vec)>, + /// Names that could already have matched at this point of the pattern. A + /// repeat within them is an error; a repeat in another alternative is not. + reachable_names: Vec, unicode: bool, + flags: Flags, + /// The pattern's capturing groups, including those not parsed yet. + total_captures: usize, + has_named_groups: bool, + referenced_names: Vec, } impl<'a> Parser<'a> { - fn new(units: &'a [u16], unicode: bool) -> Self { + fn new(units: &'a [u16], flags: Flags) -> Self { + let (total_captures, has_named_groups) = scan_groups(units); Self { units, position: 0, capture_count: 0, - names: BTreeMap::new(), - unicode, + names: Vec::new(), + reachable_names: Vec::new(), + unicode: flags.unicode, + flags, + total_captures, + has_named_groups, + referenced_names: Vec::new(), } } fn parse_disjunction(&mut self, terminator: Option) -> Result { + let reachable = self.reachable_names.len(); + let mut branch_names = Vec::new(); let mut branches = Vec::new(); loop { branches.push(Node::Sequence(self.parse_sequence(terminator)?)); + branch_names.extend(self.reachable_names.drain(reachable..)); if self.peek() != Some(u16::from(b'|')) { break; } self.position += 1; } + self.reachable_names.extend(branch_names); if branches.len() == 1 { Ok(branches.pop().expect("one branch")) } else { @@ -1033,8 +1141,22 @@ impl<'a> Parser<'a> { if Some(token) == terminator || token == u16::from(b'|') { break; } + let first_capture = self.capture_count + 1; let atom = self.parse_atom()?; - nodes.push(self.parse_quantifier(atom)?); + let assertion = match &atom { + Node::Start | Node::End | Node::WordBoundary(_) => true, + Node::Look { behind, .. } => *behind || self.unicode, + _ => false, + }; + if assertion && self.at_quantifier()? { + return Err(self.error("nothing to repeat")); + } + let quantified = self.parse_quantifier(atom, first_capture)?; + let repeated = matches!(quantified, Node::Repeat { .. }); + nodes.push(quantified); + if repeated && self.at_quantifier()? { + return Err(self.error("nothing to repeat")); + } } Ok(nodes) } @@ -1043,6 +1165,14 @@ impl<'a> Parser<'a> { let token = self .next() .ok_or_else(|| self.error("expected regular expression atom"))?; + if token == 0x7b && !self.unicode { + self.position -= 1; + let quantifier = self.at_quantifier()?; + self.position += 1; + if quantifier { + return Err(self.error("nothing to repeat")); + } + } match token { 0x2e => Ok(Node::Dot), 0x5e => Ok(Node::Start), @@ -1052,6 +1182,7 @@ impl<'a> Parser<'a> { 0x5c => self.parse_escape(false), 0x29 => Err(self.error("unmatched ')'")), 0x2a | 0x2b | 0x3f => Err(self.error("nothing to repeat")), + 0x7b | 0x7d | 0x5d if self.unicode => Err(self.error("lone bracket in Unicode mode")), _ => { let start = self.position - 1; let (value, width) = next_code_point(self.units, start, self.unicode); @@ -1064,6 +1195,8 @@ impl<'a> Parser<'a> { fn parse_group(&mut self) -> Result { let mut index = None; let mut look = None; + let outer_flags = self.flags; + let mut modified = false; if self.peek() == Some(u16::from(b'?')) { self.position += 1; match self.next() { @@ -1080,15 +1213,24 @@ impl<'a> Parser<'a> { look = Some((true, false)); } _ => { - let name = self.take_until(0x3e)?; - if name.is_empty() || self.names.contains_key(&name) { + let name = self.parse_group_name()?; + if self.reachable_names.contains(&name) { return Err(self.error("invalid duplicate capture group name")); } self.capture_count += 1; index = Some(self.capture_count); - self.names.insert(name, self.capture_count); + self.reachable_names.push(name.clone()); + match self.names.iter_mut().find(|(known, _)| *known == name) { + Some((_, indices)) => indices.push(self.capture_count), + None => self.names.push((name, vec![self.capture_count])), + } } }, + Some(0x69 | 0x6d | 0x73 | 0x2d) => { + self.position -= 1; + self.flags = self.parse_modifiers()?; + modified = true; + } _ => return Err(self.error("invalid group")), } } else { @@ -1099,7 +1241,13 @@ impl<'a> Parser<'a> { if self.next() != Some(0x29) { return Err(self.error("unterminated group")); } - Ok(if let Some((behind, positive)) = look { + let body_flags = std::mem::replace(&mut self.flags, outer_flags); + Ok(if modified { + Node::Modified { + flags: body_flags, + body: Box::new(body), + } + } else if let Some((behind, positive)) = look { Node::Look { body: Box::new(body), behind, @@ -1113,6 +1261,29 @@ impl<'a> Parser<'a> { }) } + /// The flags inside `(?ims-ims:`. Each letter may appear once, in either + /// list, and at least one must appear. + fn parse_modifiers(&mut self) -> Result { + let mut flags = self.flags; + let mut seen = Vec::new(); + let mut enable = true; + loop { + match self.next() { + Some(0x3a) if !seen.is_empty() => return Ok(flags), + Some(0x2d) if enable => enable = false, + Some(unit @ (0x69 | 0x6d | 0x73)) if !seen.contains(&unit) => { + seen.push(unit); + match unit { + 0x69 => flags.ignore_case = enable, + 0x6d => flags.multiline = enable, + _ => flags.dot_all = enable, + } + } + _ => return Err(self.error("invalid group")), + } + } + } + fn parse_class(&mut self) -> Result { let negated = if self.peek() == Some(0x5e) { self.position += 1; @@ -1121,13 +1292,11 @@ impl<'a> Parser<'a> { false }; let mut items = Vec::new(); - let mut first = true; while let Some(token) = self.peek() { - if token == 0x5d && !first { + if token == 0x5d { self.position += 1; return Ok(CharacterClass { negated, items }); } - first = false; let left = self.parse_class_item()?; if self.peek() == Some(0x2d) && self.units.get(self.position + 1) != Some(&0x5d) { self.position += 1; @@ -1181,10 +1350,16 @@ impl<'a> Parser<'a> { 0x53 => Ok(class_node(ClassItem::NotSpace)), 0x62 => Ok(Node::WordBoundary(true)), 0x42 => Ok(Node::WordBoundary(false)), - 0x6b if self.next() == Some(0x3c) => { - Ok(Node::NamedBackReference(self.take_until(0x3e)?)) + 0x6b if self.unicode || self.has_named_groups => { + if self.next() != Some(0x3c) { + return Err(self.error("invalid named reference")); + } + let name = self.parse_group_name()?; + self.referenced_names.push(name.clone()); + Ok(Node::NamedBackReference(name)) } value if value <= 0x7f && (value as u8).is_ascii_digit() && value != 0x30 => { + let checkpoint = self.position; let mut number = (value - 0x30) as usize; while let Some(digit) = self.peek().and_then(decimal_digit) { self.position += 1; @@ -1193,7 +1368,14 @@ impl<'a> Parser<'a> { .and_then(|number| number.checked_add(digit)) .ok_or_else(|| self.error("backreference number is too large"))?; } - Ok(Node::BackReference(number)) + if number <= self.total_captures { + return Ok(Node::BackReference(number)); + } + if self.unicode { + return Err(self.error("invalid back reference")); + } + self.position = checkpoint; + self.escape_character(value, false).map(Node::Literal) } value => self.escape_character(value, false).map(Node::Literal), } @@ -1206,8 +1388,16 @@ impl<'a> Parser<'a> { 0x74 => Ok(0x09), 0x66 => Ok(0x0c), 0x76 => Ok(0x0b), - 0x30 => Ok(0), - 0x78 => self.hex_escape(2), + 0x30 if self.unicode && self.peek().and_then(decimal_digit).is_some() => { + Err(self.error("invalid decimal escape")) + } + 0x30 if self.unicode || !self.peek().is_some_and(is_octal_digit) => Ok(0), + 0x30..=0x37 if !self.unicode => Ok(self.legacy_octal_escape(escaped)), + 0x63 => self.control_escape(in_class), + 0x78 => match self.hex_escape(2) { + Err(_) if !self.unicode => Ok(0x78), + result => result, + }, 0x75 if self.unicode && self.peek() == Some(0x7b) => { self.position += 1; let digits = self.take_until(0x7d)?; @@ -1246,6 +1436,43 @@ impl<'a> Parser<'a> { } } + /// `\c`. Without a letter, Annex B reads the backslash as itself and leaves the + /// `c` to be read next, and a class also takes a digit or `_`. + fn control_escape(&mut self, in_class: bool) -> Result { + let letter = self.peek().filter(|unit| { + u8::try_from(*unit).is_ok_and(|byte| { + byte.is_ascii_alphabetic() + || (in_class && !self.unicode && (byte.is_ascii_digit() || byte == b'_')) + }) + }); + match letter { + Some(letter) => { + self.position += 1; + Ok(u32::from(letter) % 32) + } + None if !self.unicode => { + self.position -= 1; + Ok(0x5c) + } + None => Err(self.error("invalid control escape")), + } + } + + /// Annex B `LegacyOctalEscapeSequence`: at most three digits, up to `\377`. + fn legacy_octal_escape(&mut self, first: u16) -> u32 { + let mut value = u32::from(first) - 0x30; + for _ in 0..2 { + match self.peek().filter(|unit| is_octal_digit(*unit)) { + Some(unit) if value * 8 + u32::from(unit) - 0x30 <= 0o377 => { + value = value * 8 + u32::from(unit) - 0x30; + self.position += 1; + } + _ => break, + } + } + value + } + fn hex_escape(&mut self, count: usize) -> Result { if self.position + count > self.units.len() { return Err(self.error("invalid hexadecimal escape")); @@ -1261,7 +1488,7 @@ impl<'a> Parser<'a> { Ok(value) } - fn parse_quantifier(&mut self, atom: Node) -> Result { + fn parse_quantifier(&mut self, atom: Node, first_capture: usize) -> Result { let Some(token) = self.peek() else { return Ok(atom); }; @@ -1279,26 +1506,10 @@ impl<'a> Parser<'a> { (0, Some(1)) } 0x7b => { - let checkpoint = self.position; - self.position += 1; - let Some(minimum) = self.parse_decimal() else { - self.position = checkpoint; + let Some(bounds) = self.parse_braced_bounds()? else { return Ok(atom); }; - match self.next() { - Some(0x7d) => (minimum, Some(minimum)), - Some(0x2c) => { - let maximum = self.parse_decimal(); - if self.next() != Some(0x7d) { - return Err(self.error("invalid quantifier")); - } - if maximum.is_some_and(|value| value < minimum) { - return Err(self.error("quantifier range out of order")); - } - (minimum, maximum) - } - _ => return Err(self.error("invalid quantifier")), - } + bounds } _ => return Ok(atom), }; @@ -1313,9 +1524,51 @@ impl<'a> Parser<'a> { min, max, greedy, + captures: first_capture..self.capture_count + 1, }) } + /// `{n}`, `{n,}` or `{n,m}` at the cursor. Anything else leaves the cursor + /// there: it is a literal `{` under Annex B and an error in Unicode mode. + fn parse_braced_bounds(&mut self) -> Result)>, RegexError> { + let checkpoint = self.position; + self.position += 1; + let bounds = self.parse_decimal().and_then(|minimum| match self.next() { + Some(0x7d) => Some((minimum, Some(minimum))), + Some(0x2c) => { + let maximum = self.parse_decimal(); + (self.next() == Some(0x7d)).then_some((minimum, maximum)) + } + _ => None, + }); + let Some((minimum, maximum)) = bounds else { + self.position = checkpoint; + return if self.unicode { + Err(self.error("invalid quantifier")) + } else { + Ok(None) + }; + }; + if maximum.is_some_and(|maximum| maximum < minimum) { + return Err(self.error("quantifier range out of order")); + } + Ok(Some((minimum, maximum))) + } + + /// Whether a quantifier starts here, without consuming it. + fn at_quantifier(&mut self) -> Result { + match self.peek() { + Some(0x2a | 0x2b | 0x3f) => Ok(true), + Some(0x7b) => { + let checkpoint = self.position; + let bounds = self.parse_braced_bounds()?; + self.position = checkpoint; + Ok(bounds.is_some()) + } + _ => Ok(false), + } + } + fn parse_decimal(&mut self) -> Option { let begin = self.position; let mut value = 0usize; @@ -1326,6 +1579,70 @@ impl<'a> Parser<'a> { (self.position > begin).then_some(value) } + /// `GroupName` up to and including `>`. Escapes name a code point, and each + /// code point must fit an identifier. + fn parse_group_name(&mut self) -> Result { + let mut name = String::new(); + loop { + let invalid = |parser: &Self| parser.error("invalid capture group name"); + let unit = self.next().ok_or_else(|| invalid(self))?; + if unit == 0x3e { + break; + } + let value = if unit == 0x5c { + if self.next() != Some(0x75) { + return Err(invalid(self)); + } + self.group_name_escape()? + } else { + let start = self.position - 1; + let (value, width) = next_code_point(self.units, start, true); + self.position = start + width; + value + }; + let character = char::from_u32(value).ok_or_else(|| invalid(self))?; + let allowed = matches!(character, '$' | '_') + || character.is_alphabetic() + || (!name.is_empty() + && (character.is_alphanumeric() + || matches!(character, '\u{200c}' | '\u{200d}'))); + if !allowed { + return Err(invalid(self)); + } + name.push(character); + } + if name.is_empty() { + return Err(self.error("invalid capture group name")); + } + Ok(name) + } + + /// The code point after `\u` in a group name, which reads as in Unicode mode + /// whatever the pattern's flags are. + fn group_name_escape(&mut self) -> Result { + if self.peek() == Some(0x7b) { + self.position += 1; + let digits = self.take_until(0x7d)?; + return u32::from_str_radix(&digits, 16) + .ok() + .filter(|value| *value <= 0x10ffff) + .ok_or_else(|| self.error("invalid Unicode escape")); + } + let first = self.hex_escape(4)?; + if (0xd800..=0xdbff).contains(&first) && self.units.get(self.position) == Some(&0x5c) { + let checkpoint = self.position; + self.position += 1; + if self.next() == Some(0x75) + && let Ok(second) = self.hex_escape(4) + && (0xdc00..=0xdfff).contains(&second) + { + return Ok(combine_surrogates(first as u16, second as u16)); + } + self.position = checkpoint; + } + Ok(first) + } + fn take_until(&mut self, terminator: u16) -> Result { let begin = self.position; while self.peek().is_some_and(|token| token != terminator) { @@ -1361,6 +1678,34 @@ fn class_node(item: ClassItem) -> Node { }) } +fn is_octal_digit(unit: u16) -> bool { + (0x30..=0x37).contains(&unit) +} + +/// Counts the capturing groups in a pattern, and says whether any is named. +fn scan_groups(units: &[u16]) -> (usize, bool) { + let (mut count, mut named, mut in_class, mut position) = (0, false, false, 0); + while let Some(unit) = units.get(position).copied() { + match unit { + 0x5c => position += 1, + 0x5b => in_class = true, + 0x5d => in_class = false, + 0x28 if !in_class => match units.get(position + 1) { + Some(0x3f) => { + let name = units.get(position + 2) == Some(&0x3c) + && !matches!(units.get(position + 3), Some(0x3d | 0x21)); + count += usize::from(name); + named |= name; + } + _ => count += 1, + }, + _ => {} + } + position += 1; + } + (count, named) +} + fn decimal_digit(unit: u16) -> Option { (unit <= 0x7f) .then_some(unit as u8) @@ -1513,6 +1858,172 @@ mod tests { ); } + #[test] + fn modifiers_change_flags_only_inside_their_group() { + assert_eq!(ranges("(?i:a)b", "", "Ab"), [Some(0..2)]); + assert!(regex("(?i:a)b", "").exec(&text("AB"), 0).unwrap().is_none()); + assert!( + regex("(?-i:a)b", "i") + .exec(&text("Ab"), 0) + .unwrap() + .is_none() + ); + assert_eq!(ranges("(?-i:a)b", "i", "aB"), [Some(0..2)]); + assert_eq!(ranges("(?s:.).", "", "\na"), [Some(0..2)]); + assert_eq!(ranges("(?m:^b)", "", "a\nb"), [Some(2..3)]); + assert_eq!(ranges("(?i:(?-i:a)b)", "", "aB"), [Some(0..2)]); + for pattern in [ + "(?-:a)", + "(?ii:a)", + "(?i-i:a)", + "(?g:a)", + "(?i--s:a)", + "(?I:a)", + ] { + assert!( + Regex::compile(&text(pattern), &text("")).is_err(), + "{pattern}" + ); + } + } + + #[test] + fn a_group_name_may_repeat_only_across_alternatives() { + let regex = regex("(?a)|(?b)", ""); + let matched = regex.exec(&text("b"), 0).unwrap().unwrap(); + assert_eq!(matched.named, [("year".to_owned(), Some(0..1))]); + for pattern in ["(?a)(?b)", "(?(?a))", "(?:(?a)|b)(?c)"] { + assert!( + Regex::compile(&text(pattern), &text("")).is_err(), + "{pattern}" + ); + } + } + + #[test] + fn named_groups_keep_source_order() { + let matched = regex("(?a)(?b)", "") + .exec(&text("ab"), 0) + .unwrap() + .unwrap(); + let names: Vec<_> = matched + .named + .iter() + .map(|(name, _)| name.as_str()) + .collect(); + assert_eq!(names, ["z", "b"]); + } + + #[test] + fn annex_b_escapes_read_literally_outside_unicode_mode() { + assert_eq!(ranges("\\x", "", "x"), [Some(0..1)]); + assert_eq!(ranges("\\c", "", "\\c"), [Some(0..2)]); + assert_eq!(ranges("\\ca", "", "\u{1}"), [Some(0..1)]); + assert_eq!(ranges("[\\c1]", "", "\u{11}"), [Some(0..1)]); + assert_eq!(ranges("[\\12-\\14]", "", "\n"), [Some(0..1)]); + assert_eq!(ranges("\\1", "", "\u{1}"), [Some(0..1)]); + assert_eq!(ranges("a{1,b}", "", "a{1,b}"), [Some(0..6)]); + assert_eq!(ranges("\\kx)", "") + .exec(&text("x"), 0) + .unwrap() + .unwrap(); + assert_eq!(matched.named[0].0, "Ab"); + for pattern in ["\\k(?a)", "(?<1a>x)", "(?<>x)"] { + assert!( + Regex::compile(&text(pattern), &text("")).is_err(), + "{pattern}" + ); + } + } + + #[test] + fn a_repeat_explores_each_alternative_before_giving_back_an_iteration() { + assert_eq!( + ranges("(aa|aabaac|ba|b|c)*", "", "aabaac"), + [Some(0..4), Some(2..4)] + ); + assert_eq!( + ranges("^([a-z]+)*[a-z]$", "", "abc"), + [Some(0..3), Some(0..2)] + ); + } + + #[test] + fn a_lookahead_keeps_only_its_first_match() { + assert_eq!( + ranges("(?=(a+))a*b\\1", "", "baaabac"), + [Some(3..6), Some(3..4)] + ); + } + + #[test] + fn whitespace_follows_the_ecmascript_set() { + assert_eq!(ranges("\\s", "", "\u{feff}"), [Some(0..1)]); + assert!(regex("\\s", "").exec(&text("\u{85}"), 0).unwrap().is_none()); + } + + #[test] + fn a_lookbehind_reads_its_body_from_right_to_left() { + assert_eq!( + ranges("(?<=(\\d+)(\\d+))$", "", "1053"), + [Some(4..4), Some(0..1), Some(1..4)] + ); + assert_eq!( + ranges("(?<=([abc])+)def", "", "abcdef"), + [Some(3..6), Some(0..1)] + ); + assert_eq!(ranges("(?<=\\1(a))b", "", "aab"), [Some(2..3), Some(1..2)]); + assert!( + regex("(?a)|(?b)){2}\\k", "", "abb"), + [Some(0..3), None, Some(1..2)] + ); + } + #[test] fn lazy_and_sticky_matching() { let lazy = regex("a.*?b", "s"); diff --git a/crates/bamts-runtime/src/temporal/calendar.rs b/crates/bamts-runtime/src/temporal/calendar.rs new file mode 100644 index 00000000..8768c4b4 --- /dev/null +++ b/crates/bamts-runtime/src/temporal/calendar.rs @@ -0,0 +1,1124 @@ +//! Calendars other than ISO 8601 that have arithmetic rules. +//! +//! A date keeps its ISO fields; these calendars interpret them. Every calendar +//! here is a pair of conversions to and from a day count, plus the shape of its +//! years. `chinese`, `dangi`, and the observational Islamic calendars need +//! astronomical tables this runtime does not have, so they are not here. + +use std::sync::Arc; + +use super::instant_duration::{ + Duration, Overflow, TemporalError, TemporalResult, Unit, epoch_days_from_ymd, + ymd_from_epoch_days, +}; +use super::plain_types::{PlainDate, PlainMonthDay, PlainYearMonth, iso_day_of_week}; +use super::relative::iso_date_until; +use super::zoned_date_time::{ + CalendarDateFields, CalendarId, CalendarProvider, IsoCalendarProvider, +}; + +/// Rata Die of the Unix epoch. +const RATA_DIE_OF_UNIX_EPOCH: i64 = 719_163; + +const COPTIC_EPOCH: i64 = 103_605; +const ETHIOPIC_EPOCH: i64 = 2796; +const ETHIOAA_YEAR_OFFSET: i64 = 5500; +const ISLAMIC_CIVIL_EPOCH: i64 = 227_015; +const ISLAMIC_TBLA_EPOCH: i64 = 227_014; +const HEBREW_EPOCH: i64 = -1_373_427; +const PERSIAN_EPOCH: i64 = 226_896; +const INDIAN_YEAR_OFFSET: i64 = 78; + +/// Calendars reach this far at most, in any year numbering. Anything past it is +/// out of range for every date, which keeps the arithmetic in `i64`. +const YEAR_LIMIT: i64 = 1_000_000; + +/// How far back a month-day's reference search looks. Every combination of +/// leap rules the calendars have recurs within this many years. +const REFERENCE_SEARCH_YEARS: i64 = 60; + +/// Identifiers `CanonicalizeCalendar` accepts, with their canonical form. +const CALENDAR_ALIASES: [(&str, &str); 2] = [ + ("ethiopic-amete-alem", "ethioaa"), + ("islamicc", "islamic-civil"), +]; + +/// The canonical identifier of a supported calendar, ASCII case-insensitively. +#[must_use] +pub fn canonical_calendar(identifier: &str) -> Option<&'static str> { + let lower = identifier.to_ascii_lowercase(); + if lower == "iso8601" { + return Some("iso8601"); + } + if let Some((_, canonical)) = CALENDAR_ALIASES.iter().find(|(alias, _)| *alias == lower) { + return Some(canonical); + } + Calendar::ALL + .iter() + .map(|calendar| calendar.id()) + .find(|id| *id == lower) +} + +/// The non-ISO calendars. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub enum Calendar { + Buddhist, + Coptic, + Ethioaa, + Ethiopic, + Gregory, + Hebrew, + Indian, + IslamicCivil, + IslamicTbla, + Persian, + Roc, +} + +/// A month code: its number and whether it is a leap month. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct MonthCode { + pub number: u8, + pub leap: bool, +} + +impl MonthCode { + /// Reads a well-formed `Mnn` or `MnnL`. + #[must_use] + pub fn parse(text: &str) -> Option { + let leap = text.ends_with('L'); + let digits = text.strip_prefix('M')?; + let digits = digits.strip_suffix('L').unwrap_or(digits); + (digits.len() == 2) + .then(|| digits.parse().ok()) + .flatten() + .map(|number| Self { number, leap }) + } + + /// The code's text. + #[must_use] + pub fn render(self) -> String { + format!("M{:02}{}", self.number, if self.leap { "L" } else { "" }) + } +} + +/// A calendar date: arithmetic year, ordinal month, day. +#[derive(Clone, Copy, Debug, Eq, PartialEq)] +pub struct CalendarDate { + pub year: i64, + pub month: u8, + pub day: u8, +} + +const fn gregorian_leap(year: i64) -> bool { + year % 4 == 0 && (year % 100 != 0 || year % 400 == 0) +} + +fn iso_days_in_month(year: i64, month: u8) -> u8 { + match month { + 2 if gregorian_leap(year) => 29, + 2 => 28, + 4 | 6 | 9 | 11 => 30, + _ => 31, + } +} + +fn iso_days(year: i64, month: u8, day: u8) -> i64 { + // Only the year's position in the 400-year cycle matters past `i32`. + let cycles = year.div_euclid(400); + let within = year.rem_euclid(400) as i32; + cycles * 146_097 + epoch_days_from_ymd(within, month, day) +} + +fn iso_from_days(days: i64) -> (i64, u8, u8) { + let cycles = days.div_euclid(146_097); + let (year, month, day) = ymd_from_epoch_days(days.rem_euclid(146_097)); + (i64::from(year) + cycles * 400, month, day) +} + +/// The ordinal month and day of a day of the year, counted from zero. +fn month_of_year(mut remaining: i64, days_in_month: impl Fn(u8) -> u8) -> (u8, u8) { + let mut month = 1; + loop { + let length = i64::from(days_in_month(month)); + if remaining < length { + return (month, remaining as u8 + 1); + } + remaining -= length; + month += 1; + } +} + +fn coptic_like_days(epoch: i64, year: i64, month: u8, day: u8) -> i64 { + epoch - 1 + 365 * (year - 1) + year.div_euclid(4) + 30 * (i64::from(month) - 1) + i64::from(day) + - RATA_DIE_OF_UNIX_EPOCH +} + +fn coptic_like_date(epoch: i64, days: i64) -> (i64, u8, u8) { + let rata_die = days + RATA_DIE_OF_UNIX_EPOCH; + let year = (4 * (rata_die - epoch) + 1463).div_euclid(1461); + let start = coptic_like_days(epoch, year, 1, 1); + let month = (days - start).div_euclid(30) as u8 + 1; + let day = days - coptic_like_days(epoch, year, month, 1) + 1; + (year, month, day as u8) +} + +fn islamic_days(epoch: i64, year: i64, month: u8, day: u8) -> i64 { + let month_index = i64::from(month); + epoch - 1 + + (year - 1) * 354 + + (3 + 11 * year).div_euclid(30) + + 29 * (month_index - 1) + + month_index.div_euclid(2) + + i64::from(day) + - RATA_DIE_OF_UNIX_EPOCH +} + +fn islamic_date(epoch: i64, days: i64) -> (i64, u8, u8) { + let rata_die = days + RATA_DIE_OF_UNIX_EPOCH; + let year = (30 * (rata_die - epoch) + 10_646).div_euclid(10_631); + let prior = days - islamic_days(epoch, year, 1, 1); + let month = ((11 * prior + 330).div_euclid(325) as u8).min(12); + let day = days - islamic_days(epoch, year, month, 1) + 1; + (year, month, day as u8) +} + +const fn islamic_leap(year: i64) -> bool { + (14 + 11 * year).rem_euclid(30) < 11 +} + +/// Days before Tishri 1 of `year`, counting from the Hebrew epoch. +fn hebrew_elapsed_days(year: i64) -> i64 { + let months = (235 * year - 234).div_euclid(19); + let parts = 12_084 + 13_753 * months; + let days = 29 * months + parts.div_euclid(25_920); + if (3 * (days + 1)).rem_euclid(7) < 3 { + days + 1 + } else { + days + } +} + +fn hebrew_new_year(year: i64) -> i64 { + let (previous, current, next) = ( + hebrew_elapsed_days(year - 1), + hebrew_elapsed_days(year), + hebrew_elapsed_days(year + 1), + ); + let delay = if next - current == 356 { + 2 + } else if current - previous == 382 { + 1 + } else { + 0 + }; + HEBREW_EPOCH + current + delay +} + +const fn hebrew_leap(year: i64) -> bool { + (7 * year + 1).rem_euclid(19) < 7 +} + +fn hebrew_year_length(year: i64) -> i64 { + hebrew_new_year(year + 1) - hebrew_new_year(year) +} + +fn hebrew_days_in_month(year: i64, month: u8) -> u8 { + let length = hebrew_year_length(year); + let leap = hebrew_leap(year); + let month = if !leap && month >= 6 { + month + 1 + } else { + month + }; + match month { + 1 | 5 | 6 | 8 | 10 | 12 => 30, + 2 if length.rem_euclid(10) == 5 => 30, + 3 if length.rem_euclid(10) != 3 => 30, + _ => 29, + } +} + +fn hebrew_days(year: i64, month: u8, day: u8) -> i64 { + let before: i64 = (1..month) + .map(|earlier| i64::from(hebrew_days_in_month(year, earlier))) + .sum(); + hebrew_new_year(year) + before + i64::from(day) - 1 - RATA_DIE_OF_UNIX_EPOCH +} + +fn hebrew_date(days: i64) -> (i64, u8, u8) { + let rata_die = days + RATA_DIE_OF_UNIX_EPOCH; + let mut year = ((rata_die - HEBREW_EPOCH) * 10_000).div_euclid(3_652_468) + 1; + while hebrew_new_year(year) > rata_die { + year -= 1; + } + while hebrew_new_year(year + 1) <= rata_die { + year += 1; + } + let (month, day) = month_of_year(rata_die - hebrew_new_year(year), |month| { + hebrew_days_in_month(year, month) + }); + (year, month, day) +} + +const fn persian_leap(year: i64) -> bool { + (25 * year + 11).rem_euclid(33) < 8 +} + +fn persian_days_in_month(year: i64, month: u8) -> u8 { + match month { + 1..=6 => 31, + 7..=11 => 30, + _ if persian_leap(year) => 30, + _ => 29, + } +} + +fn persian_new_year(year: i64) -> i64 { + PERSIAN_EPOCH - 1 + 365 * (year - 1) + (8 * year + 21).div_euclid(33) - RATA_DIE_OF_UNIX_EPOCH +} + +fn persian_days(year: i64, month: u8, day: u8) -> i64 { + let month_index = i64::from(month) - 1; + let before = if month <= 7 { + 31 * month_index + } else { + 30 * month_index + 6 + }; + persian_new_year(year) + before + i64::from(day) - 1 +} + +fn persian_date(days: i64) -> (i64, u8, u8) { + let mut year = ((days - persian_new_year(1)) * 10_000).div_euclid(3_652_425) + 1; + while persian_new_year(year) > days { + year -= 1; + } + while persian_new_year(year + 1) <= days { + year += 1; + } + let (month, day) = month_of_year(days - persian_new_year(year), |month| { + persian_days_in_month(year, month) + }); + (year, month, day) +} + +fn indian_days_in_month(year: i64, month: u8) -> u8 { + match month { + 1 if gregorian_leap(year + INDIAN_YEAR_OFFSET) => 31, + 1 | 7..=12 => 30, + _ => 31, + } +} + +fn indian_new_year(year: i64) -> i64 { + let gregorian = year + INDIAN_YEAR_OFFSET; + iso_days( + gregorian, + 3, + if gregorian_leap(gregorian) { 21 } else { 22 }, + ) +} + +fn indian_days(year: i64, month: u8, day: u8) -> i64 { + let before: i64 = (1..month) + .map(|earlier| i64::from(indian_days_in_month(year, earlier))) + .sum(); + indian_new_year(year) + before + i64::from(day) - 1 +} + +fn indian_date(days: i64) -> (i64, u8, u8) { + let (iso_year, _, _) = iso_from_days(days); + let mut year = iso_year - INDIAN_YEAR_OFFSET; + if days < indian_new_year(year) { + year -= 1; + } + let (month, day) = month_of_year(days - indian_new_year(year), |month| { + indian_days_in_month(year, month) + }); + (year, month, day) +} + +impl Calendar { + const ALL: [Self; 11] = [ + Self::Buddhist, + Self::Coptic, + Self::Ethioaa, + Self::Ethiopic, + Self::Gregory, + Self::Hebrew, + Self::Indian, + Self::IslamicCivil, + Self::IslamicTbla, + Self::Persian, + Self::Roc, + ]; + + /// The calendar behind a canonical identifier; ISO has none. + #[must_use] + pub fn from_id(calendar: &CalendarId) -> Option { + Self::from_name(calendar.as_str()) + } + + /// The calendar of a canonical name. + #[must_use] + pub fn from_name(name: &str) -> Option { + Self::ALL + .iter() + .copied() + .find(|candidate| candidate.id() == name) + } + + #[must_use] + pub const fn id(self) -> &'static str { + match self { + Self::Buddhist => "buddhist", + Self::Coptic => "coptic", + Self::Ethioaa => "ethioaa", + Self::Ethiopic => "ethiopic", + Self::Gregory => "gregory", + Self::Hebrew => "hebrew", + Self::Indian => "indian", + Self::IslamicCivil => "islamic-civil", + Self::IslamicTbla => "islamic-tbla", + Self::Persian => "persian", + Self::Roc => "roc", + } + } + + /// The offset from ISO years, for the calendars laid over ISO dates. + const fn iso_year_offset(self) -> Option { + match self { + Self::Gregory => Some(0), + Self::Buddhist => Some(543), + Self::Roc => Some(-1911), + _ => None, + } + } + + const fn islamic_epoch(self) -> i64 { + if matches!(self, Self::IslamicCivil) { + ISLAMIC_CIVIL_EPOCH + } else { + ISLAMIC_TBLA_EPOCH + } + } + + #[must_use] + pub fn months_in_year(self, year: i64) -> u8 { + match self { + Self::Coptic | Self::Ethiopic | Self::Ethioaa => 13, + Self::Hebrew if hebrew_leap(year) => 13, + _ => 12, + } + } + + #[must_use] + pub fn days_in_month(self, year: i64, month: u8) -> u8 { + if let Some(offset) = self.iso_year_offset() { + return iso_days_in_month(year - offset, month); + } + match self { + Self::Coptic | Self::Ethiopic | Self::Ethioaa => match month { + 13 if year.rem_euclid(4) == 3 => 6, + 13 => 5, + _ => 30, + }, + Self::Hebrew => hebrew_days_in_month(year, month), + Self::Persian => persian_days_in_month(year, month), + Self::Indian => indian_days_in_month(year, month), + _ => { + if month % 2 == 1 || (month == 12 && islamic_leap(year)) { + 30 + } else { + 29 + } + } + } + } + + #[must_use] + pub fn in_leap_year(self, year: i64) -> bool { + if let Some(offset) = self.iso_year_offset() { + return gregorian_leap(year - offset); + } + match self { + Self::Coptic | Self::Ethiopic | Self::Ethioaa => year.rem_euclid(4) == 3, + Self::Hebrew => hebrew_leap(year), + Self::Persian => persian_leap(year), + Self::Indian => gregorian_leap(year + INDIAN_YEAR_OFFSET), + _ => islamic_leap(year), + } + } + + #[must_use] + pub fn days_in_year(self, year: i64) -> u16 { + if self == Self::Hebrew { + return hebrew_year_length(year) as u16; + } + (1..=self.months_in_year(year)) + .map(|month| u16::from(self.days_in_month(year, month))) + .sum() + } + + /// Whether any day of `year` is a representable date. + fn has_representable_days(self, year: i64) -> bool { + let first = self.epoch_days(year, 1, 1); + let next = self.epoch_days(year + 1, 1, 1); + PlainDate::from_epoch_days(first).is_ok() || PlainDate::from_epoch_days(next - 1).is_ok() + } + + /// Days since the Unix epoch of a calendar date. The date must exist. + #[must_use] + pub fn epoch_days(self, year: i64, month: u8, day: u8) -> i64 { + if let Some(offset) = self.iso_year_offset() { + return iso_days(year - offset, month, day); + } + match self { + Self::Coptic => coptic_like_days(COPTIC_EPOCH, year, month, day), + Self::Ethiopic => coptic_like_days(ETHIOPIC_EPOCH, year, month, day), + Self::Ethioaa => { + coptic_like_days(ETHIOPIC_EPOCH, year - ETHIOAA_YEAR_OFFSET, month, day) + } + Self::Hebrew => hebrew_days(year, month, day), + Self::Persian => persian_days(year, month, day), + Self::Indian => indian_days(year, month, day), + _ => islamic_days(self.islamic_epoch(), year, month, day), + } + } + + /// The calendar date of a day count. + #[must_use] + pub fn date(self, days: i64) -> CalendarDate { + let (year, month, day) = if let Some(offset) = self.iso_year_offset() { + let (year, month, day) = iso_from_days(days); + (year + offset, month, day) + } else { + match self { + Self::Coptic => coptic_like_date(COPTIC_EPOCH, days), + Self::Ethiopic => coptic_like_date(ETHIOPIC_EPOCH, days), + Self::Ethioaa => { + let (year, month, day) = coptic_like_date(ETHIOPIC_EPOCH, days); + (year + ETHIOAA_YEAR_OFFSET, month, day) + } + Self::Hebrew => hebrew_date(days), + Self::Persian => persian_date(days), + Self::Indian => indian_date(days), + _ => islamic_date(self.islamic_epoch(), days), + } + }; + CalendarDate { year, month, day } + } + + /// The code of an ordinal month. Hebrew's sixth month in a leap year is + /// Adar I, the leap month, and pushes the rest along by one. + #[must_use] + pub fn month_code(self, year: i64, month: u8) -> MonthCode { + if self == Self::Hebrew && hebrew_leap(year) { + return match month { + 1..=5 => MonthCode { + number: month, + leap: false, + }, + 6 => MonthCode { + number: 5, + leap: true, + }, + _ => MonthCode { + number: month - 1, + leap: false, + }, + }; + } + MonthCode { + number: month, + leap: false, + } + } + + /// Whether a month code exists in some year of this calendar. + #[must_use] + pub fn has_month_code(self, code: MonthCode) -> bool { + let last = if matches!(self, Self::Coptic | Self::Ethiopic | Self::Ethioaa) { + 13 + } else { + 12 + }; + if code.leap { + self == Self::Hebrew && code.number == 5 + } else { + (1..=last).contains(&code.number) + } + } + + /// The ordinal month of a code in `year`, if the year has it. + #[must_use] + pub fn ordinal_month(self, year: i64, code: MonthCode) -> Option { + if !self.has_month_code(code) { + return None; + } + if self == Self::Hebrew { + let leap = hebrew_leap(year); + return match (code.leap, leap) { + (true, true) => Some(6), + (true, false) => None, + (false, true) if code.number >= 6 => Some(code.number + 1), + (false, _) => Some(code.number), + }; + } + Some(code.number) + } + + /// The ordinal month a code lands on in `year`. A leap month a year lacks + /// falls to the month it stands in for, or is an error. + fn ordinal_month_or_constrain( + self, + year: i64, + code: MonthCode, + overflow: Overflow, + ) -> TemporalResult { + match self.ordinal_month(year, code) { + Some(month) => Ok(month), + None if overflow == Overflow::Constrain => Ok(self + .ordinal_month( + year, + MonthCode { + number: code.number + 1, + leap: false, + }, + ) + .unwrap_or(code.number)), + None => Err(TemporalError::range(format!( + "month {} does not exist in year {year}", + code.render() + ))), + } + } + + /// The era and era year of a date, for the calendars that have eras. + #[must_use] + pub fn era(self, date: CalendarDate) -> (&'static str, i64) { + let CalendarDate { year, .. } = date; + match self { + Self::Buddhist => ("be", year), + Self::Coptic | Self::Hebrew => ("am", year), + Self::Ethioaa => ("aa", year), + Self::Ethiopic if year > 0 => ("am", year), + Self::Ethiopic => ("aa", year + ETHIOAA_YEAR_OFFSET), + Self::Indian => ("shaka", year), + Self::Persian => ("ap", year), + Self::IslamicCivil | Self::IslamicTbla if year > 0 => ("ah", year), + Self::IslamicCivil | Self::IslamicTbla => ("bh", 1 - year), + Self::Roc if year > 0 => ("roc", year), + Self::Roc => ("broc", 1 - year), + Self::Gregory if year > 0 => ("ce", year), + Self::Gregory => ("bce", 1 - year), + } + } + + /// The arithmetic year of an era year, or `None` for an era the calendar + /// does not have. + #[must_use] + pub fn year_of_era(self, era: &str, era_year: i64) -> Option { + let single = |name: &str| (era == name).then_some(era_year); + match self { + Self::Buddhist => single("be"), + Self::Coptic | Self::Hebrew => single("am"), + Self::Ethioaa => single("aa"), + Self::Indian => single("shaka"), + Self::Persian => single("ap"), + Self::Ethiopic => match era { + "am" => Some(era_year), + "aa" => Some(era_year - ETHIOAA_YEAR_OFFSET), + _ => None, + }, + Self::IslamicCivil | Self::IslamicTbla => match era { + "ah" => Some(era_year), + "bh" => Some(1 - era_year), + _ => None, + }, + Self::Roc => match era { + "roc" | "minguo" => Some(era_year), + "broc" | "before-roc" | "minguo-qian" => Some(1 - era_year), + _ => None, + }, + Self::Gregory => match era { + "ce" | "ad" => Some(era_year), + "bce" | "bc" => Some(1 - era_year), + _ => None, + }, + } + } + + /// The day count of `year`, an ordinal month, and a day, kept within the + /// month as `overflow` says. It may lie past the representable range; only + /// a finished date has to be inside it. + fn days_of(self, year: i64, month: u8, day: i64, overflow: Overflow) -> TemporalResult { + let last = i64::from(self.days_in_month(year, month)); + if day > last && overflow == Overflow::Reject { + return Err(TemporalError::range(format!( + "day {day} does not exist in month {month} of year {year}" + ))); + } + Ok(self.epoch_days(year, month, day.clamp(1, last) as u8)) + } + + /// The year and ordinal month `months` after a month. + fn add_months(self, mut year: i64, mut month: i64, months: i64) -> TemporalResult<(i64, u8)> { + month += months; + while month < 1 { + year -= 1; + check_year(year)?; + month += i64::from(self.months_in_year(year)); + } + while month > i64::from(self.months_in_year(year)) { + month -= i64::from(self.months_in_year(year)); + year += 1; + check_year(year)?; + } + Ok((year, month as u8)) + } + + /// CalendarDateAdd. + fn add( + self, + iso_date: PlainDate, + duration: &Duration, + overflow: Overflow, + ) -> TemporalResult { + let date = self.date(iso_date.epoch_days()); + let year = i64::try_from(duration.years) + .ok() + .and_then(|years| date.year.checked_add(years)) + .filter(|year| year.abs() < YEAR_LIMIT) + .ok_or_else(out_of_range)?; + let code = self.month_code(date.year, date.month); + let month = self.ordinal_month_or_constrain(year, code, overflow)?; + let months = i64::try_from(duration.months).map_err(|_| out_of_range())?; + if months.abs() > 13 * YEAR_LIMIT { + return Err(out_of_range()); + } + let (year, month) = self.add_months(year, i64::from(month), months)?; + let start = self.days_of(year, month, i64::from(date.day), overflow)?; + let total = i128::from(start) + duration.weeks * 7 + duration.days; + PlainDate::from_epoch_days(i64::try_from(total).map_err(|_| out_of_range())?) + } + + /// Whether a first date moved by `years` lands past `target`. The code is + /// compared as it stands, before any constraining to the year. + fn years_surpass( + self, + sign: i64, + first: CalendarDate, + years: i64, + target: CalendarDate, + ) -> TemporalResult { + let year = first.year + years; + let code = self.month_code(first.year, first.month); + let target_code = self.month_code(target.year, target.month); + let as_written = (year, (code.number, code.leap), first.day).cmp(&( + target.year, + (target_code.number, target_code.leap), + target.day, + )); + if as_written as i64 * sign > 0 { + return Ok(true); + } + let month = self.ordinal_month_or_constrain(year, code, Overflow::Constrain)?; + Ok(month_surpasses(sign, year, month, first.day, target)) + } + + /// CalendarDateUntil. + fn until(self, one: PlainDate, two: PlainDate, largest_unit: Unit) -> TemporalResult { + let (one_days, two_days) = (one.epoch_days(), two.epoch_days()); + let sign = (two_days - one_days).signum(); + if sign == 0 { + return Ok(Duration::default()); + } + let first = self.date(one_days); + let target = self.date(two_days); + let mut years = 0; + if largest_unit == Unit::Year { + years = target.year - first.year; + while self.years_surpass(sign, first, years, target)? { + years -= sign; + } + } + let code = self.month_code(first.year, first.month); + let year = first.year + years; + let month = self.ordinal_month_or_constrain(year, code, Overflow::Constrain)?; + let mut position = (year, month); + let mut months = 0; + if matches!(largest_unit, Unit::Year | Unit::Month) { + loop { + let next = self.add_months(position.0, i64::from(position.1), sign)?; + if month_surpasses(sign, next.0, next.1, first.day, target) { + break; + } + position = next; + months += sign; + } + } + let day = first.day.min(self.days_in_month(position.0, position.1)); + let mut days = two_days - self.epoch_days(position.0, position.1, day); + let mut weeks = 0; + if largest_unit == Unit::Week { + weeks = days / 7; + days %= 7; + } + Duration::from_date(years.into(), months.into(), weeks.into(), days.into()) + } + + fn fields(self, iso_date: PlainDate) -> CalendarDateFields { + let days = iso_date.epoch_days(); + let date = self.date(days); + let (era, era_year) = self.era(date); + let start_of_year = self.epoch_days(date.year, 1, 1); + CalendarDateFields { + era: Some(Arc::from(era)), + era_year: Some(era_year as i32), + year: date.year as i32, + month: date.month, + month_code: Arc::from(self.month_code(date.year, date.month).render()), + day: date.day, + day_of_week: iso_day_of_week(iso_date.year, iso_date.month, iso_date.day), + day_of_year: (days - start_of_year + 1) as u16, + week_of_year: None, + year_of_week: None, + days_in_week: 7, + days_in_month: self.days_in_month(date.year, date.month), + days_in_year: self.days_in_year(date.year), + months_in_year: self.months_in_year(date.year), + in_leap_year: self.in_leap_year(date.year), + } + } +} + +/// Whether a year, ordinal month, and unconstrained day land past `target`. +fn month_surpasses(sign: i64, year: i64, month: u8, day: u8, target: CalendarDate) -> bool { + let ordering = (year, month, day).cmp(&(target.year, target.month, target.day)); + ordering as i64 * sign > 0 +} + +fn out_of_range() -> TemporalError { + TemporalError::range("date outside the representable range") +} + +fn check_year(year: i64) -> TemporalResult<()> { + if year.abs() < YEAR_LIMIT { + Ok(()) + } else { + Err(out_of_range()) + } +} + +/// What a fields record says about a month. +#[derive(Clone, Copy, Debug, PartialEq)] +pub enum MonthField { + Ordinal(f64), + Code(MonthCode), +} + +/// The day count of resolved fields: CalendarDateToISO, before the limits. +/// +/// # Errors +/// `RangeError` for a month, day, or year that does not exist under `overflow`. +fn days_from_fields( + calendar: Calendar, + year: f64, + month: MonthField, + day: f64, + overflow: Overflow, +) -> TemporalResult { + if year.abs() >= YEAR_LIMIT as f64 { + return Err(out_of_range()); + } + let year = year as i64; + let month = match month { + MonthField::Code(code) => calendar.ordinal_month_or_constrain(year, code, overflow)?, + MonthField::Ordinal(ordinal) => { + let last = calendar.months_in_year(year); + if ordinal > f64::from(last) && overflow == Overflow::Reject { + return Err(TemporalError::range(format!( + "month {ordinal} does not exist in year {year}" + ))); + } + ordinal.min(f64::from(last)) as u8 + } + }; + calendar.days_of(year, month, day.min(1000.0) as i64, overflow) +} + +/// CalendarDateFromFields. +/// +/// # Errors +/// `RangeError` for a date that does not exist under `overflow` or lies past the +/// representable range. +pub fn date_from_fields( + calendar: Calendar, + year: f64, + month: MonthField, + day: f64, + overflow: Overflow, +) -> TemporalResult { + PlainDate::from_epoch_days(days_from_fields(calendar, year, month, day, overflow)?) +} + +/// CalendarYearMonthFromFields: the month's first day is the reference. +/// +/// # Errors +/// `RangeError` for a month that does not exist under `overflow` or lies past +/// the representable range. +pub fn year_month_from_fields( + calendar: Calendar, + year: f64, + month: MonthField, + overflow: Overflow, +) -> TemporalResult { + let days = days_from_fields(calendar, year, month, 1.0, overflow)?; + let (year, month, day) = ymd_from_epoch_days(days); + PlainYearMonth::with_reference_day(year, month, day) +} + +/// CalendarMonthDayFromFields. A year decides only what the month and day +/// constrain to, but it has to be a year some date falls in; without one, +/// `month` alone names nothing. +/// +/// # Errors +/// `TypeError` for an ordinal month with no year; `RangeError` for a year past +/// the representable range, or a month or day that does not exist under +/// `overflow`. +pub fn month_day_from_fields( + calendar: Calendar, + year: Option, + month: MonthField, + day: f64, + overflow: Overflow, +) -> TemporalResult { + match (year, month) { + (Some(year), month) => { + if !calendar.has_representable_days(year as i64) { + return Err(out_of_range()); + } + let days = days_from_fields(calendar, year, month, day, overflow)?; + let date = calendar.date(days); + month_day_reference( + calendar, + calendar.month_code(date.year, date.month), + f64::from(date.day), + overflow, + ) + } + (None, MonthField::Code(code)) => month_day_reference(calendar, code, day, overflow), + (None, MonthField::Ordinal(_)) => Err(TemporalError::type_error( + "month needs a year to name a month", + )), + } +} + +/// Whether `month` and `code` name the same month of `year`, the check for a +/// fields record that gives both. +#[must_use] +pub fn month_matches(calendar: Calendar, year: f64, month: f64, code: MonthCode) -> bool { + year.abs() >= YEAR_LIMIT as f64 + || calendar + .ordinal_month(year as i64, code) + .is_none_or(|ordinal| f64::from(ordinal) == month) +} + +/// CalendarMonthDayToISOReferenceDate: the latest ISO date no later than the end +/// of 1972 that has this month code and day. A day past the longest such month +/// is constrained to it or is an error. +fn month_day_reference( + calendar: Calendar, + code: MonthCode, + day: f64, + overflow: Overflow, +) -> TemporalResult { + let limit = iso_days(1972, 12, 31); + let start = calendar.date(limit).year; + let months = (start - REFERENCE_SEARCH_YEARS..=start) + .rev() + .filter_map(|year| { + calendar + .ordinal_month(year, code) + .map(|month| (year, month, calendar.days_in_month(year, month))) + }); + let longest = months.clone().map(|(_, _, length)| length).max(); + let Some(longest) = longest else { + return Err(TemporalError::range(format!( + "{} is not a month of the {} calendar", + code.render(), + calendar.id() + ))); + }; + if day > f64::from(longest) && overflow == Overflow::Reject { + return Err(TemporalError::range(format!( + "day {day} does not exist in {}", + code.render() + ))); + } + let day = day.min(f64::from(longest)) as u8; + months + .filter(|(_, _, length)| *length >= day) + .map(|(year, month, _)| calendar.epoch_days(year, month, day)) + .find(|days| *days <= limit) + .ok_or_else(out_of_range) + .and_then(|days| { + let (year, month, day) = ymd_from_epoch_days(days); + PlainMonthDay::with_reference_year(month.into(), day.into(), year, Overflow::Reject) + }) +} + +/// The year-month holding an ISO date, as CalendarYearMonthFromFields sees it. +/// +/// # Errors +/// `RangeError` when that month is past the representable range. +pub fn year_month_of(calendar: Calendar, iso_date: PlainDate) -> TemporalResult { + let date = calendar.date(iso_date.epoch_days()); + let (year, month, day) = ymd_from_epoch_days(calendar.epoch_days(date.year, date.month, 1)); + PlainYearMonth::with_reference_day(year, month, day) +} + +/// The month-day of an ISO date, as CalendarMonthDayFromFields sees it. +/// +/// # Errors +/// `RangeError` when no reference date exists. +pub fn month_day_of(calendar: Calendar, iso_date: PlainDate) -> TemporalResult { + let date = calendar.date(iso_date.epoch_days()); + month_day_reference( + calendar, + calendar.month_code(date.year, date.month), + f64::from(date.day), + Overflow::Constrain, + ) +} + +/// The first day of a year-month's calendar month. +/// +/// # Errors +/// `RangeError` when that day is past the representable range. +pub fn first_of_month(calendar: Calendar, value: PlainYearMonth) -> TemporalResult { + let date = calendar.date(value.iso_date().epoch_days()); + PlainDate::from_epoch_days(calendar.epoch_days(date.year, date.month, 1)) +} + +/// AddDurationToYearMonth. Only years and months move a year-month. +/// +/// # Errors +/// `RangeError` for a smaller unit, a month that does not exist under +/// `overflow`, or a result past the representable range. +pub fn add_to_year_month( + calendar: Calendar, + value: PlainYearMonth, + duration: &Duration, + overflow: Overflow, +) -> TemporalResult { + if duration.default_largest_unit() > Unit::Month && !duration.is_zero() { + return Err(TemporalError::range( + "a year-month can only change by whole years and months", + )); + } + let moved = calendar.add(first_of_month(calendar, value)?, duration, overflow)?; + year_month_of(calendar, moved) +} + +/// The calendars this runtime has, ISO 8601 among them. +#[derive(Debug, Clone, Copy, Default)] +pub struct Calendars; + +impl CalendarProvider for Calendars { + fn fields( + &self, + calendar: &CalendarId, + iso_date: PlainDate, + ) -> TemporalResult { + match Calendar::from_id(calendar) { + Some(calendar) => Ok(calendar.fields(iso_date)), + None => IsoCalendarProvider.fields(calendar, iso_date), + } + } + + fn date_add( + &self, + calendar: &CalendarId, + iso_date: PlainDate, + duration: &Duration, + overflow: Overflow, + ) -> TemporalResult { + match Calendar::from_id(calendar) { + Some(calendar) => calendar.add(iso_date, duration, overflow), + None => IsoCalendarProvider.date_add(calendar, iso_date, duration, overflow), + } + } + + fn date_until( + &self, + calendar: &CalendarId, + one: PlainDate, + two: PlainDate, + largest_unit: Unit, + ) -> TemporalResult { + match Calendar::from_id(calendar) { + Some(calendar) => calendar.until(one, two, largest_unit), + None => iso_date_until(one, two, largest_unit), + } + } +} + +#[cfg(test)] +mod tests { + use super::*; + + const ALL: [Calendar; 11] = Calendar::ALL; + + #[test] + fn dates_round_trip_through_day_counts() { + for calendar in ALL { + for days in (-1_000_000..1_000_000).step_by(997) { + let date = calendar.date(days); + assert_eq!( + calendar.epoch_days(date.year, date.month, date.day), + days, + "{} {date:?}", + calendar.id() + ); + assert!(date.month <= calendar.months_in_year(date.year)); + assert!(date.day >= 1); + assert!(date.day <= calendar.days_in_month(date.year, date.month)); + } + } + } + + #[test] + fn known_new_years_land_on_known_days() { + let date = |calendar: Calendar, year, month, day| { + let (iso_year, iso_month, iso_day) = + iso_from_days(calendar.epoch_days(year, month, day)); + (iso_year, iso_month, iso_day) + }; + assert_eq!(date(Calendar::Hebrew, 5784, 1, 1), (2023, 9, 16)); + assert_eq!(date(Calendar::Persian, 1403, 1, 1), (2024, 3, 20)); + assert_eq!(date(Calendar::Indian, 1946, 1, 1), (2024, 3, 21)); + assert_eq!(date(Calendar::Coptic, 1741, 1, 1), (2024, 9, 11)); + assert_eq!(date(Calendar::Ethiopic, 2017, 1, 1), (2024, 9, 11)); + assert_eq!(date(Calendar::IslamicCivil, 1446, 1, 1), (2024, 7, 8)); + } + + #[test] + fn hebrew_leap_years_shift_the_codes_after_adar_i() { + let calendar = Calendar::Hebrew; + assert_eq!(calendar.month_code(5779, 6).render(), "M05L"); + assert_eq!(calendar.month_code(5779, 7).render(), "M06"); + assert_eq!(calendar.month_code(5780, 6).render(), "M06"); + assert_eq!( + calendar.ordinal_month( + 5780, + MonthCode { + number: 5, + leap: true + } + ), + None + ); + } +} diff --git a/crates/bamts-runtime/src/temporal/instant_duration.rs b/crates/bamts-runtime/src/temporal/instant_duration.rs index 6757e874..1fefbebc 100644 --- a/crates/bamts-runtime/src/temporal/instant_duration.rs +++ b/crates/bamts-runtime/src/temporal/instant_duration.rs @@ -3,11 +3,15 @@ //! Grounded in ECMA-262 + the Temporal proposal (tc39/proposal-temporal, ECMA-402 //! aligned option semantics). All arithmetic is exact checked `i128`; no floats //! participate in any date/time computation. No ambient host access (no clock, -//! no environment, no time zone database) — zoned/tzdb work is the jiff-backed -//! C11.3 boundary and is intentionally absent here. +//! no environment, no time zone database). use std::fmt; +use super::now_rounding::round_and_balance_duration; +use super::parse::{ + ParsedIsoDateTime, UtcOffsetRecord, parse_duration_string, parse_instant_string, +}; + // --------------------------------------------------------------------------- // Errors // --------------------------------------------------------------------------- @@ -34,13 +38,22 @@ pub struct TemporalError { impl TemporalError { pub fn range(message: impl Into) -> Self { - Self { kind: TemporalErrorKind::Range, message: message.into() } + Self { + kind: TemporalErrorKind::Range, + message: message.into(), + } } pub fn type_error(message: impl Into) -> Self { - Self { kind: TemporalErrorKind::Type, message: message.into() } + Self { + kind: TemporalErrorKind::Type, + message: message.into(), + } } pub fn syntax(message: impl Into) -> Self { - Self { kind: TemporalErrorKind::Syntax, message: message.into() } + Self { + kind: TemporalErrorKind::Syntax, + message: message.into(), + } } } @@ -108,7 +121,11 @@ impl Unit { ]; pub fn category(self) -> UnitCategory { - if self <= Unit::Day { UnitCategory::Date } else { UnitCategory::Time } + if self <= Unit::Day { + UnitCategory::Date + } else { + UnitCategory::Time + } } /// Nanoseconds per unit for time units (and Day, defined as exactly 24h in @@ -245,7 +262,7 @@ impl Overflow { /// combined with smallestUnit precision resolution). #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] pub enum Precision { - /// Emit exactly the digits needed (trailing zeros trimmed in groups of 3). + /// Emit exactly the digits needed (every trailing zero trimmed). #[default] Auto, /// Truncate the seconds part entirely (smallestUnit: "minute"). @@ -269,55 +286,154 @@ impl Precision { // Rounding core // --------------------------------------------------------------------------- +/// Which neighbour `ApplyUnsignedRoundingMode` selects for a magnitude strictly +/// between two adjacent multiples. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum UnsignedRounding { + Zero, + Infinity, + HalfZero, + HalfInfinity, + HalfEven, +} + +/// GetUnsignedRoundingMode. +fn unsigned_rounding(mode: RoundingMode, is_negative: bool) -> UnsignedRounding { + match (mode, is_negative) { + (RoundingMode::Ceil, false) | (RoundingMode::Floor, true) | (RoundingMode::Expand, _) => { + UnsignedRounding::Infinity + } + (RoundingMode::Ceil, true) | (RoundingMode::Floor, false) | (RoundingMode::Trunc, _) => { + UnsignedRounding::Zero + } + (RoundingMode::HalfCeil, false) + | (RoundingMode::HalfFloor, true) + | (RoundingMode::HalfExpand, _) => UnsignedRounding::HalfInfinity, + (RoundingMode::HalfCeil, true) + | (RoundingMode::HalfFloor, false) + | (RoundingMode::HalfTrunc, _) => UnsignedRounding::HalfZero, + (RoundingMode::HalfEven, _) => UnsignedRounding::HalfEven, + } +} + +/// ApplyUnsignedRoundingMode: whether a non-exact magnitude with this +/// `remainder` above the lower quotient rounds up to the next multiple. +fn rounds_up( + remainder: u128, + increment: u128, + lower_quotient: u128, + mode: UnsignedRounding, +) -> bool { + match mode { + UnsignedRounding::Zero => false, + UnsignedRounding::Infinity => true, + UnsignedRounding::HalfZero + | UnsignedRounding::HalfInfinity + | UnsignedRounding::HalfEven => { + // `distance_up` is positive because the magnitude is not an exact + // multiple, so this compares the two neighbours without doubling. + let distance_up = increment - remainder; + match remainder.cmp(&distance_up) { + std::cmp::Ordering::Less => false, + std::cmp::Ordering::Greater => true, + std::cmp::Ordering::Equal => match mode { + UnsignedRounding::HalfInfinity => true, + UnsignedRounding::HalfEven => lower_quotient % 2 == 1, + _ => false, + }, + } + } + } +} + +/// Whether a value lying `remainder / divisor` of the way from one multiple to +/// the next rounds to the farther one. `lower_quotient` is the nearer multiple's +/// count and `is_negative` the sign of the value being rounded. The fraction +/// must be strictly between 0 and 1. +pub fn rounds_past_fraction( + remainder: u128, + divisor: u128, + lower_quotient: u128, + mode: RoundingMode, + is_negative: bool, +) -> bool { + rounds_up( + remainder, + divisor, + lower_quotient, + unsigned_rounding(mode, is_negative), + ) +} + /// RoundNumberToIncrement on exact integers: rounds `x` to a multiple of -/// `increment` (> 0) under `mode`. Errors only on i128 overflow at the extreme -/// edge of the domain. +/// `increment` (> 0) under `mode`. The direction is decided on the magnitude, +/// so negative ties follow the mode's sign-aware meaning. Errors only on i128 +/// overflow at the extreme edge of the domain. pub fn round_to_increment(x: i128, increment: i128, mode: RoundingMode) -> TemporalResult { debug_assert!(increment > 0); - let quotient = x.div_euclid(increment); - let remainder = x.rem_euclid(increment); // 0 <= remainder < increment + let is_negative = x < 0; + let divisor = increment.unsigned_abs(); + let magnitude = x.unsigned_abs(); + let lower = magnitude / divisor; + let remainder = magnitude % divisor; if remainder == 0 { return Ok(x); } - // `floor` result is quotient; `ceil` result is quotient + 1. - let is_negative = x < 0; - let double_rem = remainder.checked_mul(2).ok_or_else(|| overflow_err("round"))?; - let round_up = match mode { - RoundingMode::Ceil => true, - RoundingMode::Floor => false, - RoundingMode::Expand => !is_negative, - RoundingMode::Trunc => is_negative, - RoundingMode::HalfCeil => double_rem >= increment, - RoundingMode::HalfFloor => double_rem > increment, - RoundingMode::HalfExpand => { - if is_negative { double_rem <= increment } else { double_rem >= increment } - } - RoundingMode::HalfTrunc => { - if is_negative { double_rem >= increment } else { double_rem > increment } - } - RoundingMode::HalfEven => { - if double_rem > increment { - true - } else if double_rem < increment { - false - } else { - quotient.rem_euclid(2) == 1 - } - } - }; - let q = if round_up { - quotient.checked_add(1).ok_or_else(|| overflow_err("round"))? + let up = rounds_up( + remainder, + divisor, + lower, + unsigned_rounding(mode, is_negative), + ); + let quotient = if up { lower + 1 } else { lower }; + let quotient = i128::try_from(quotient).map_err(|_| overflow_err("round"))?; + let signed = if is_negative { -quotient } else { quotient }; + signed + .checked_mul(increment) + .ok_or_else(|| overflow_err("round")) +} + +/// RoundNumberToIncrementAsIfPositive: rounds an epoch value, whose direction +/// is interpreted as if it were positive even before 1970. Euclidean division +/// keeps the lower neighbour the numerically smaller one. +pub fn round_to_increment_as_if_positive( + x: i128, + increment: i128, + mode: RoundingMode, +) -> TemporalResult { + debug_assert!(increment > 0); + let lower = x.div_euclid(increment); + let remainder = x.rem_euclid(increment); + if remainder == 0 { + return Ok(x); + } + let up = rounds_up( + remainder.unsigned_abs(), + increment.unsigned_abs(), + lower.rem_euclid(2).unsigned_abs(), + unsigned_rounding(mode, false), + ); + let quotient = if up { + lower.checked_add(1).ok_or_else(|| overflow_err("round"))? } else { - quotient + lower }; - q.checked_mul(increment).ok_or_else(|| overflow_err("round")) + quotient + .checked_mul(increment) + .ok_or_else(|| overflow_err("round")) } /// ValidateTemporalRoundingIncrement: `increment` must be >= 1, and when the /// unit is bounded, `increment` must be < max and divide max evenly. -pub fn validate_rounding_increment(unit: Unit, increment: i128, inclusive: bool) -> TemporalResult<()> { +pub fn validate_rounding_increment( + unit: Unit, + increment: i128, + inclusive: bool, +) -> TemporalResult<()> { if increment < 1 { - return Err(TemporalError::range(format!("roundingIncrement must be >= 1, got {increment}"))); + return Err(TemporalError::range(format!( + "roundingIncrement must be >= 1, got {increment}" + ))); } if let Some(max) = unit.max_rounding_increment() { let bound = if inclusive { max } else { max - 1 }; @@ -376,13 +492,9 @@ impl Instant { self.epoch_ns.div_euclid(NS_PER_MILLISECOND) as i64 } - pub fn epoch_seconds(self) -> i64 { - self.epoch_ns.div_euclid(NS_PER_SECOND) as i64 - } - /// AddDurationToInstant (sign = +1). Date units are rejected: an instant /// has no calendar. - pub fn add(self, duration: &Duration) -> TemporalResult { + pub fn add(&self, duration: &Duration) -> TemporalResult { self.add_signed(duration, 1) } @@ -391,7 +503,8 @@ impl Instant { } fn add_signed(self, duration: &Duration, sign: i128) -> TemporalResult { - if duration.years != 0 || duration.months != 0 || duration.weeks != 0 || duration.days != 0 { + if duration.years != 0 || duration.months != 0 || duration.weeks != 0 || duration.days != 0 + { return Err(TemporalError::range( "Instant arithmetic does not accept date units (years/months/weeks/days)", )); @@ -399,7 +512,10 @@ impl Instant { let delta = duration.time_total_nanoseconds(); let shifted = self .epoch_ns - .checked_add(sign.checked_mul(delta).ok_or_else(|| overflow_err("Instant.add"))?) + .checked_add( + sign.checked_mul(delta) + .ok_or_else(|| overflow_err("Instant.add"))?, + ) .ok_or_else(|| overflow_err("Instant.add"))?; Instant::from_epoch_nanoseconds(shifted) } @@ -414,7 +530,9 @@ impl Instant { increment: i128, mode: RoundingMode, ) -> TemporalResult { - if largest_unit.category() == UnitCategory::Date || smallest_unit.category() == UnitCategory::Date { + if largest_unit.category() == UnitCategory::Date + || smallest_unit.category() == UnitCategory::Date + { return Err(TemporalError::range( "Instant difference supports time units only (hour..nanosecond)", )); @@ -426,23 +544,16 @@ impl Instant { } validate_rounding_increment(smallest_unit, increment, false)?; let diff = other.epoch_ns - self.epoch_ns; // both within ±8.64e21: no overflow - let unit_ns = smallest_unit.ns_per().ok_or_else(|| overflow_err("until"))?; - let step = unit_ns.checked_mul(increment).ok_or_else(|| overflow_err("until"))?; + let unit_ns = smallest_unit + .ns_per() + .ok_or_else(|| overflow_err("until"))?; + let step = unit_ns + .checked_mul(increment) + .ok_or_else(|| overflow_err("until"))?; let rounded = round_to_increment(diff, step, mode)?; Duration::from_time_nanoseconds(rounded, largest_unit) } - pub fn since( - self, - other: Instant, - largest_unit: Unit, - smallest_unit: Unit, - increment: i128, - mode: RoundingMode, - ) -> TemporalResult { - other.until(self, largest_unit, smallest_unit, increment, mode) - } - /// Temporal.Instant.prototype.round. `unit` must be a time unit; the /// increment must divide one day's worth of that unit evenly. pub fn round(self, unit: Unit, increment: i128, mode: RoundingMode) -> TemporalResult { @@ -460,26 +571,18 @@ impl Instant { ))); } let step = unit_ns * increment; // <= NS_PER_DAY: no overflow - let rounded = round_to_increment(self.epoch_ns, step, mode)?; + let rounded = round_to_increment_as_if_positive(self.epoch_ns, step, mode)?; Instant::from_epoch_nanoseconds(rounded) } - /// ParseTemporalInstantString: full date-time with a required UTC offset - /// (`Z`, `z`, or numeric ±hh[:mm[:ss[.f{1,9}]]]), optional annotations. - pub fn parse(text: &str) -> TemporalResult { - let parsed = parse_iso_datetime_string(text)?; + /// The exact time an already-parsed Instant string names. Calendar + /// annotations are ignored here, critical or not: an Instant has no + /// calendar for one to apply to. + pub fn from_parsed(parsed: &ParsedIsoDateTime) -> TemporalResult { let offset_ns = match parsed.offset { - Some(UtcOffsetRecord::Zulu) => 0, - Some(UtcOffsetRecord::Numeric(ns)) => ns, - None => { - return Err(TemporalError::syntax( - "Instant string requires a UTC offset (Z or ±hh:mm)", - )) - } + Some(UtcOffsetRecord::Numeric { nanoseconds, .. }) => nanoseconds, + _ => 0, }; - if !parsed.has_time { - return Err(TemporalError::syntax("Instant string requires a time component")); - } let days = epoch_days_checked(parsed.year, parsed.month, parsed.day)?; let ns = (days as i128) .checked_mul(NS_PER_DAY) @@ -489,6 +592,11 @@ impl Instant { Instant::from_epoch_nanoseconds(ns) } + /// ParseTemporalInstantString. + pub fn parse(text: &str) -> TemporalResult { + Instant::from_parsed(&parse_instant_string(text)?) + } + /// TemporalInstantToString with UTC ("Z") output. pub fn format(self, precision: Precision) -> String { let days = self.epoch_ns.div_euclid(NS_PER_DAY); @@ -513,42 +621,50 @@ impl fmt::Display for Instant { // --------------------------------------------------------------------------- /// |years|, |months|, |weeks| must each be < 2^32 (IsValidDuration). -const CALENDAR_FIELD_LIMIT: i64 = 1 << 32; -/// Normalized time duration bound: |days·86400 + h·3600 + … + s| ≤ 2^53 − 1 -/// seconds with the sub-second remainder attached (IsValidDuration step 6). -const MAX_TIME_DURATION_SECONDS: i128 = (1_i128 << 53) - 1; - -/// Ten-field Temporal.Duration. All fields are exact integers; a duration is -/// valid only when every field shares one sign (or is zero) and magnitudes are -/// within the spec bounds. +const CALENDAR_FIELD_LIMIT: i128 = 1 << 32; +/// A time duration's magnitude must stay below 2^53 seconds, in nanoseconds. +pub const MAX_TIME_DURATION: i128 = (1_i128 << 53) * NS_PER_SECOND; + +/// Ten-field Temporal.Duration. Fields hold exact integers that a Number can +/// represent: `Duration::new` rounds each through a double, as the proposal's +/// `𝔽(value)` does, so a field reads back as the same Number the object reports. +/// `i128` is required because a valid `nanoseconds` reaches about 9e24, past +/// the range of `i64`. A duration is valid only when every field shares one +/// sign (or is zero) and magnitudes are within the spec bounds. #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] pub struct Duration { - pub years: i64, - pub months: i64, - pub weeks: i64, - pub days: i64, - pub hours: i64, - pub minutes: i64, - pub seconds: i64, - pub milliseconds: i64, - pub microseconds: i64, - pub nanoseconds: i64, + pub years: i128, + pub months: i128, + pub weeks: i128, + pub days: i128, + pub hours: i128, + pub minutes: i128, + pub seconds: i128, + pub milliseconds: i128, + pub microseconds: i128, + pub nanoseconds: i128, +} + +/// The integer a Number holds after `𝔽(value)`. +fn float64_representable(value: i128) -> i128 { + value as f64 as i128 } impl Duration { - #[allow(clippy::too_many_arguments)] // mirrors the ten-argument spec constructor - pub fn new( - years: i64, - months: i64, - weeks: i64, - days: i64, - hours: i64, - minutes: i64, - seconds: i64, - milliseconds: i64, - microseconds: i64, - nanoseconds: i64, - ) -> TemporalResult { + /// CreateTemporalDuration, fields largest to smallest. + pub fn new(fields: [i128; 10]) -> TemporalResult { + let [ + years, + months, + weeks, + days, + hours, + minutes, + seconds, + milliseconds, + microseconds, + nanoseconds, + ] = fields.map(float64_representable); let d = Duration { years, months, @@ -565,18 +681,17 @@ impl Duration { Ok(d) } - pub fn from_time( - hours: i64, - minutes: i64, - seconds: i64, - milliseconds: i64, - microseconds: i64, - nanoseconds: i64, + /// CreateDateDurationRecord. + pub fn from_date( + years: i128, + months: i128, + weeks: i128, + days: i128, ) -> TemporalResult { - Duration::new(0, 0, 0, 0, hours, minutes, seconds, milliseconds, microseconds, nanoseconds) + Duration::new([years, months, weeks, days, 0, 0, 0, 0, 0, 0]) } - fn fields(&self) -> [i64; 10] { + pub fn fields(&self) -> [i128; 10] { [ self.years, self.months, @@ -591,9 +706,20 @@ impl Duration { ] } + /// The date portion alone, with every time field zero. + pub fn date_part(&self) -> Duration { + Duration { + years: self.years, + months: self.months, + weeks: self.weeks, + days: self.days, + ..Duration::default() + } + } + /// IsValidDuration. fn validate(&self) -> TemporalResult<()> { - let mut sign = 0_i64; + let mut sign = 0_i128; for v in self.fields() { let s = v.signum(); if s != 0 { @@ -605,25 +731,23 @@ impl Duration { sign = s; } } - for name_value in [ + for (name, value) in [ ("years", self.years), ("months", self.months), ("weeks", self.weeks), ] { - if name_value.1.abs() >= CALENDAR_FIELD_LIMIT { + if value.abs() >= CALENDAR_FIELD_LIMIT { return Err(TemporalError::range(format!( - "|{}| must be < 2^32, got {}", - name_value.0, name_value.1 + "|{name}| must be < 2^32, got {value}" ))); } } - // Normalized seconds bound, computed exactly in i128. - let total_ns = self.time_total_nanoseconds_with_days()?; - let bound = MAX_TIME_DURATION_SECONDS - .checked_mul(NS_PER_SECOND) - .and_then(|v| v.checked_add(NS_PER_SECOND - 1)) - .ok_or_else(|| overflow_err("Duration"))?; - if total_ns.abs() > bound { + // Every field carries one sign, so overflow in this sum can only mean + // a magnitude far beyond the limit. + let total_ns = self + .checked_time_total_nanoseconds() + .ok_or_else(|| TemporalError::range("Duration time total exceeds 2^53 seconds"))?; + if total_ns.abs() >= MAX_TIME_DURATION { return Err(TemporalError::range( "Duration time total exceeds 2^53 seconds", )); @@ -631,8 +755,23 @@ impl Duration { Ok(()) } + fn checked_time_total_nanoseconds(&self) -> Option { + let terms = [ + (self.days, NS_PER_DAY), + (self.hours, NS_PER_HOUR), + (self.minutes, NS_PER_MINUTE), + (self.seconds, NS_PER_SECOND), + (self.milliseconds, NS_PER_MILLISECOND), + (self.microseconds, NS_PER_MICROSECOND), + (self.nanoseconds, 1), + ]; + terms.iter().try_fold(0_i128, |total, &(value, unit)| { + total.checked_add(value.checked_mul(unit)?) + }) + } + /// DurationSign. - pub fn sign(&self) -> i64 { + pub fn sign(&self) -> i128 { for v in self.fields() { if v != 0 { return v.signum(); @@ -661,25 +800,36 @@ impl Duration { } pub fn abs(&self) -> Duration { - if self.sign() < 0 { self.negated() } else { *self } + if self.sign() < 0 { + self.negated() + } else { + *self + } + } + + /// DefaultTemporalLargestUnit: the largest unit with a non-zero field, + /// `nanosecond` for a blank duration. + pub fn default_largest_unit(&self) -> Unit { + self.fields() + .iter() + .position(|&value| value != 0) + .map_or(Unit::Nanosecond, |index| Unit::ALL[index]) } - /// Total nanoseconds of the time portion (hours and smaller). Exact: - /// worst case magnitudes fit i128 with vast headroom. + /// Total nanoseconds of the time portion (hours and smaller). Exact: a + /// validated duration keeps this below `MAX_TIME_DURATION`. pub fn time_total_nanoseconds(&self) -> i128 { - (self.hours as i128) * NS_PER_HOUR - + (self.minutes as i128) * NS_PER_MINUTE - + (self.seconds as i128) * NS_PER_SECOND - + (self.milliseconds as i128) * NS_PER_MILLISECOND - + (self.microseconds as i128) * NS_PER_MICROSECOND - + (self.nanoseconds as i128) + self.hours * NS_PER_HOUR + + self.minutes * NS_PER_MINUTE + + self.seconds * NS_PER_SECOND + + self.milliseconds * NS_PER_MILLISECOND + + self.microseconds * NS_PER_MICROSECOND + + self.nanoseconds } - fn time_total_nanoseconds_with_days(&self) -> TemporalResult { - (self.days as i128) - .checked_mul(NS_PER_DAY) - .and_then(|d| d.checked_add(self.time_total_nanoseconds())) - .ok_or_else(|| overflow_err("Duration")) + /// Time portion plus days taken as exactly 24 hours. + pub fn total_nanoseconds_with_24_hour_days(&self) -> i128 { + self.days * NS_PER_DAY + self.time_total_nanoseconds() } /// BalanceTimeDuration: build a duration from total nanoseconds, carrying @@ -687,39 +837,28 @@ impl Duration { pub fn from_time_nanoseconds(total_ns: i128, largest_unit: Unit) -> TemporalResult { if largest_unit < Unit::Day { return Err(TemporalError::range( - "balancing calendar units requires a relativeTo (C11.3 boundary)", + "balancing calendar units requires a relativeTo", )); } let sign: i128 = if total_ns < 0 { -1 } else { 1 }; let mut rest = total_ns.abs(); - let mut out = Duration::default(); + let mut fields = [0_i128; 10]; let order = [ - (Unit::Day, NS_PER_DAY), - (Unit::Hour, NS_PER_HOUR), - (Unit::Minute, NS_PER_MINUTE), - (Unit::Second, NS_PER_SECOND), - (Unit::Millisecond, NS_PER_MILLISECOND), - (Unit::Microsecond, NS_PER_MICROSECOND), + (3, Unit::Day, NS_PER_DAY), + (4, Unit::Hour, NS_PER_HOUR), + (5, Unit::Minute, NS_PER_MINUTE), + (6, Unit::Second, NS_PER_SECOND), + (7, Unit::Millisecond, NS_PER_MILLISECOND), + (8, Unit::Microsecond, NS_PER_MICROSECOND), ]; - for (unit, ns) in order { + for (slot, unit, ns) in order { if largest_unit <= unit { - let amount = rest / ns; + fields[slot] = rest / ns * sign; rest %= ns; - let field = i64::try_from(amount * sign).map_err(|_| overflow_err("balance"))?; - match unit { - Unit::Day => out.days = field, - Unit::Hour => out.hours = field, - Unit::Minute => out.minutes = field, - Unit::Second => out.seconds = field, - Unit::Millisecond => out.milliseconds = field, - Unit::Microsecond => out.microseconds = field, - _ => unreachable!("order contains only day..microsecond"), - } } } - out.nanoseconds = i64::try_from(rest * sign).map_err(|_| overflow_err("balance"))?; - out.validate()?; - Ok(out) + fields[9] = rest * sign; + Duration::new(fields) } /// Temporal.Duration.prototype.round without relativeTo: valid only when @@ -734,12 +873,12 @@ impl Duration { ) -> TemporalResult { if self.years != 0 || self.months != 0 || self.weeks != 0 { return Err(TemporalError::range( - "rounding a duration with calendar units requires relativeTo (C11.3 boundary)", + "rounding a duration with calendar units requires relativeTo", )); } if smallest_unit < Unit::Day || largest_unit < Unit::Day { return Err(TemporalError::range( - "rounding to calendar units requires relativeTo (C11.3 boundary)", + "rounding to calendar units requires relativeTo", )); } if smallest_unit < largest_unit { @@ -747,12 +886,13 @@ impl Duration { "smallestUnit must not be larger than largestUnit", )); } - validate_rounding_increment(smallest_unit, increment, false)?; - let total = self.time_total_nanoseconds_with_days()?; - let unit_ns = smallest_unit.ns_per().ok_or_else(|| overflow_err("round"))?; - let step = unit_ns.checked_mul(increment).ok_or_else(|| overflow_err("round"))?; - let rounded = round_to_increment(total, step, mode)?; - Duration::from_time_nanoseconds(rounded, largest_unit) + round_and_balance_duration( + self.total_nanoseconds_with_24_hour_days(), + smallest_unit, + largest_unit, + increment, + mode, + ) } /// Temporal.Duration.prototype.total without relativeTo (unit must be Day @@ -761,154 +901,40 @@ impl Duration { pub fn total_exact(&self, unit: Unit) -> TemporalResult<(i128, i128, i128)> { if self.years != 0 || self.months != 0 || self.weeks != 0 { return Err(TemporalError::range( - "totaling a duration with calendar units requires relativeTo (C11.3 boundary)", + "totaling a duration with calendar units requires relativeTo", )); } let unit_ns = unit.ns_per().ok_or_else(|| { TemporalError::range("total unit must be day..nanosecond without relativeTo") })?; - let total = self.time_total_nanoseconds_with_days()?; - Ok((total.div_euclid(unit_ns), total.rem_euclid(unit_ns), unit_ns)) + let total = self.total_nanoseconds_with_24_hour_days(); + Ok(( + total.div_euclid(unit_ns), + total.rem_euclid(unit_ns), + unit_ns, + )) } - /// Presentation-only float conversion of `total_exact` (the JS `total()` - /// return value). Never feeds back into arithmetic. + /// The JS `total()` return value: the exact quotient rounded once to the + /// nearest Number. Never feeds back into arithmetic. pub fn total_as_f64(&self, unit: Unit) -> TemporalResult { - let (q, r, unit_ns) = self.total_exact(unit)?; - Ok(q as f64 + (r as f64) / (unit_ns as f64)) + let (quotient, remainder, unit_ns) = self.total_exact(unit)?; + Ok(exact_ratio_to_f64(quotient * unit_ns + remainder, unit_ns)) } /// ParseTemporalDurationString (ISO 8601-2 duration). pub fn parse(text: &str) -> TemporalResult { - let bytes = text.as_bytes(); - let mut pos = 0_usize; - let sign: i64 = match bytes.first() { - Some(b'-') => { - pos += 1; - -1 - } - Some(b'+') => { - pos += 1; - 1 - } - _ => 1, - }; - if !matches!(bytes.get(pos), Some(b'P') | Some(b'p')) { - return Err(TemporalError::syntax("duration string must contain 'P'")); - } - pos += 1; - - let mut d = Duration::default(); - let mut any = false; - let mut fraction_seen = false; - - // Date part designators, in mandatory order. - let date_slots: [(u8, u8); 4] = [(b'Y', 0), (b'M', 1), (b'W', 2), (b'D', 3)]; - let mut slot_idx = 0; - while pos < bytes.len() && !matches!(bytes[pos], b'T' | b't') { - let (value, digits, frac) = parse_duration_number(bytes, &mut pos)?; - if frac.is_some() { - return Err(TemporalError::syntax("fractions are only allowed on time components")); - } - if digits == 0 { - return Err(TemporalError::syntax("expected digits in duration")); - } - let designator = *bytes - .get(pos) - .ok_or_else(|| TemporalError::syntax("missing duration designator"))?; - pos += 1; - let upper = designator.to_ascii_uppercase(); - let slot = date_slots - .iter() - .position(|&(c, _)| c == upper) - .ok_or_else(|| TemporalError::syntax("invalid date designator in duration"))?; - if slot < slot_idx { - return Err(TemporalError::syntax("duration designators out of order")); - } - slot_idx = slot + 1; - let field = value.checked_mul(sign as i128).ok_or_else(|| overflow_err("parse"))?; - let field = i64::try_from(field).map_err(|_| overflow_err("Duration.parse"))?; - match upper { - b'Y' => d.years = field, - b'M' => d.months = field, - b'W' => d.weeks = field, - _ => d.days = field, - } - any = true; - } - - if pos < bytes.len() { - // Time part. - pos += 1; // consume 'T' - let mut time_any = false; - let mut time_slot = 0; // H=0, M=1, S=2 - while pos < bytes.len() { - if fraction_seen { - return Err(TemporalError::syntax( - "no components may follow a fractional component", - )); - } - let (value, digits, frac) = parse_duration_number(bytes, &mut pos)?; - if digits == 0 { - return Err(TemporalError::syntax("expected digits in duration")); - } - let designator = *bytes - .get(pos) - .ok_or_else(|| TemporalError::syntax("missing duration designator"))?; - pos += 1; - let (slot, unit_ns): (usize, i128) = match designator.to_ascii_uppercase() { - b'H' => (0, NS_PER_HOUR), - b'M' => (1, NS_PER_MINUTE), - b'S' => (2, NS_PER_SECOND), - _ => return Err(TemporalError::syntax("invalid time designator in duration")), - }; - if slot < time_slot { - return Err(TemporalError::syntax("duration designators out of order")); - } - time_slot = slot + 1; - let whole = value.checked_mul(sign as i128).ok_or_else(|| overflow_err("parse"))?; - match slot { - 0 => d.hours = i64::try_from(whole).map_err(|_| overflow_err("Duration.parse"))?, - 1 => d.minutes = i64::try_from(whole).map_err(|_| overflow_err("Duration.parse"))?, - _ => d.seconds = i64::try_from(whole).map_err(|_| overflow_err("Duration.parse"))?, - } - if let Some(frac_ns_of_unit) = frac { - fraction_seen = true; - // frac_ns_of_unit is the fraction scaled to 10^-9 of one - // unit; convert exactly to nanoseconds of that unit. - let frac_ns = frac_ns_of_unit - .checked_mul(unit_ns / NS_PER_SECOND * (sign as i128)) - .ok_or_else(|| overflow_err("parse"))?; - // Distribute into ms/us/ns exactly. - let extra = if unit_ns == NS_PER_SECOND { - frac_ns_of_unit * (sign as i128) - } else { - frac_ns - }; - d.milliseconds = i64::try_from(extra / NS_PER_MILLISECOND) - .map_err(|_| overflow_err("Duration.parse"))?; - d.microseconds = i64::try_from((extra % NS_PER_MILLISECOND) / NS_PER_MICROSECOND) - .map_err(|_| overflow_err("Duration.parse"))?; - d.nanoseconds = i64::try_from(extra % NS_PER_MICROSECOND) - .map_err(|_| overflow_err("Duration.parse"))?; - } - time_any = true; - any = true; - } - if !time_any { - return Err(TemporalError::syntax("'T' must be followed by a time component")); - } - } - - if !any { - return Err(TemporalError::syntax("duration must contain at least one component")); - } - d.validate()?; - Ok(d) + Duration::new(parse_duration_string(text)?) } /// TemporalDurationToString (auto precision). pub fn format(&self) -> String { + self.format_with_precision(Precision::Auto) + } + + /// TemporalDurationToString. Sub-second fields carry exactly into seconds, + /// so a field past 1000 still serializes in canonical form. + pub fn format_with_precision(&self, precision: Precision) -> String { let sign = self.sign(); let a = self.abs(); let mut out = String::new(); @@ -916,28 +942,29 @@ impl Duration { out.push('-'); } out.push('P'); - if a.years != 0 { - out.push_str(&format!("{}Y", a.years)); - } - if a.months != 0 { - out.push_str(&format!("{}M", a.months)); - } - if a.weeks != 0 { - out.push_str(&format!("{}W", a.weeks)); - } - if a.days != 0 { - out.push_str(&format!("{}D", a.days)); + for (value, designator) in [ + (a.years, 'Y'), + (a.months, 'M'), + (a.weeks, 'W'), + (a.days, 'D'), + ] { + if value != 0 { + out.push_str(&format!("{value}{designator}")); + } } - let sub_ns = - a.milliseconds as i128 * NS_PER_MILLISECOND + a.microseconds as i128 * NS_PER_MICROSECOND - + a.nanoseconds as i128; - // Carry sub-second overflow into seconds for canonical output. - let extra_seconds = (sub_ns / NS_PER_SECOND) as i64; - let frac_ns = (sub_ns % NS_PER_SECOND) as u64; - let seconds = a.seconds + extra_seconds; - let has_time = a.hours != 0 || a.minutes != 0 || seconds != 0 || frac_ns != 0; + let total_seconds_ns = a.seconds * NS_PER_SECOND + + a.milliseconds * NS_PER_MILLISECOND + + a.microseconds * NS_PER_MICROSECOND + + a.nanoseconds; + let seconds = total_seconds_ns / NS_PER_SECOND; + let fraction = total_seconds_ns % NS_PER_SECOND; let date_empty = a.years == 0 && a.months == 0 && a.weeks == 0 && a.days == 0; - if has_time || date_empty { + let zero_clock = a.hours == 0 && a.minutes == 0; + let show_seconds = seconds != 0 + || (date_empty && zero_clock) + || !matches!(precision, Precision::Auto) + || fraction != 0; + if !zero_clock || show_seconds { out.push('T'); if a.hours != 0 { out.push_str(&format!("{}H", a.hours)); @@ -945,16 +972,9 @@ impl Duration { if a.minutes != 0 { out.push_str(&format!("{}M", a.minutes)); } - if seconds != 0 || frac_ns != 0 || (a.hours == 0 && a.minutes == 0 && date_empty) { + if show_seconds { out.push_str(&seconds.to_string()); - if frac_ns != 0 { - let mut frac = format!("{frac_ns:09}"); - while frac.ends_with('0') { - frac.pop(); - } - out.push('.'); - out.push_str(&frac); - } + out.push_str(&format_fractional_seconds(fraction, precision)); out.push('S'); } } @@ -962,57 +982,52 @@ impl Duration { } } -impl fmt::Display for Duration { - fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { - f.write_str(&self.format()) +/// FormatFractionalSeconds: the leading `.` and digits, or nothing when none +/// are called for. `fraction_ns` is below one second. +pub fn format_fractional_seconds(fraction_ns: i128, precision: Precision) -> String { + let digits = format!("{fraction_ns:09}"); + match precision { + Precision::Digits(0) | Precision::Minute => String::new(), + Precision::Digits(count) => format!(".{}", &digits[..usize::from(count)]), + Precision::Auto if fraction_ns == 0 => String::new(), + Precision::Auto => format!(".{}", digits.trim_end_matches('0')), } } -/// Parses digits (and optional `.`/`,` fraction of up to 9 digits) at `pos`. -/// Returns (whole value, digit count, optional fraction scaled to 10^-9). -fn parse_duration_number( - bytes: &[u8], - pos: &mut usize, -) -> TemporalResult<(i128, usize, Option)> { - let mut value: i128 = 0; - let mut digits = 0_usize; - while let Some(&b) = bytes.get(*pos) { - if !b.is_ascii_digit() { - break; - } - value = value - .checked_mul(10) - .and_then(|v| v.checked_add((b - b'0') as i128)) - .ok_or_else(|| overflow_err("Duration.parse"))?; - digits += 1; - *pos += 1; - } - let mut fraction = None; - if matches!(bytes.get(*pos), Some(b'.') | Some(b',')) { - *pos += 1; - let mut frac: i128 = 0; - let mut frac_digits = 0_usize; - while let Some(&b) = bytes.get(*pos) { - if !b.is_ascii_digit() { - break; - } - if frac_digits < 9 { - frac = frac * 10 + (b - b'0') as i128; - frac_digits += 1; - } else { - return Err(TemporalError::syntax("fraction exceeds 9 digits")); - } - *pos += 1; - } - if frac_digits == 0 { - return Err(TemporalError::syntax("fraction requires at least one digit")); - } - for _ in frac_digits..9 { - frac *= 10; - } - fraction = Some(frac); +/// The Number nearest `numerator / denominator` (`denominator` > 0), rounded +/// once from the exact quotient. Dividing two rounded doubles would round +/// twice, which is observable in `total()`. +pub fn exact_ratio_to_f64(numerator: i128, denominator: i128) -> f64 { + debug_assert!(denominator > 0); + if numerator == 0 { + return 0.0; + } + let divisor = denominator.unsigned_abs(); + let dividend = numerator.unsigned_abs(); + let mut quotient = dividend / divisor; + let mut remainder = dividend % divisor; + let mut exponent = 0_i32; + // Binary long division until the quotient holds more bits than a double + // keeps, leaving `remainder` as the sticky information below them. + while quotient < (1_u128 << 64) && remainder != 0 { + let doubled = remainder << 1; + let bit = doubled >= divisor; + quotient = (quotient << 1) | u128::from(bit); + remainder = if bit { doubled - divisor } else { doubled }; + exponent -= 1; + } + let excess = (128 - quotient.leading_zeros()).saturating_sub(64); + let kept = quotient >> excess; + let lost = quotient & ((1_u128 << excess) - 1); + let sticky = u64::from(lost != 0 || remainder != 0); + let magnitude = (kept as u64 | sticky) as f64 * 2_f64.powi(exponent + excess as i32); + if numerator < 0 { -magnitude } else { magnitude } +} + +impl fmt::Display for Duration { + fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { + f.write_str(&self.format()) } - Ok((value, digits, fraction)) } // --------------------------------------------------------------------------- @@ -1101,16 +1116,10 @@ pub fn format_time_ns(time_ns: i128, precision: Precision) -> String { Precision::Auto => { out.push_str(&format!(":{second:02}")); if frac != 0 { - let mut digits = format!("{frac:09}"); - while digits.len() > 3 && digits[digits.len() - 3..].bytes().all(|b| b == b'0') { - digits.truncate(digits.len() - 3); - } - while digits.ends_with('0') && digits.len() % 3 != 0 { - // keep groups of 3 per spec auto precision - break; - } + let digits = format!("{frac:09}"); + let digits = digits.trim_end_matches('0'); out.push('.'); - out.push_str(&digits); + out.push_str(digits); } out } @@ -1126,326 +1135,6 @@ pub fn format_time_ns(time_ns: i128, precision: Precision) -> String { } } -// --------------------------------------------------------------------------- -// ISO 8601 string parsing (shared with plain_types) -// --------------------------------------------------------------------------- - -/// UTC offset in a parsed string. -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub enum UtcOffsetRecord { - /// `Z` / `z`. - Zulu, - /// Numeric offset in nanoseconds east of UTC. - Numeric(i128), -} - -/// Result of parsing an ISO date[-time] string with annotations. -#[derive(Debug, Clone, PartialEq, Eq)] -pub struct ParsedIsoDateTime { - pub year: i32, - pub month: u8, - pub day: u8, - pub has_time: bool, - /// Nanoseconds since midnight; 0 when `has_time` is false. - pub time_ns: i128, - pub offset: Option, - /// Bracketed time zone annotation (e.g. `UTC`, `+01:00`, `America/New_York`), - /// unresolved: resolution is the jiff/tzdb C11.3 boundary. - pub time_zone: Option, - /// `[u-ca=…]` value if present. - pub calendar: Option, -} - -struct Cursor<'a> { - bytes: &'a [u8], - pos: usize, -} - -impl<'a> Cursor<'a> { - fn new(text: &'a str) -> Self { - Cursor { bytes: text.as_bytes(), pos: 0 } - } - fn peek(&self) -> Option { - self.bytes.get(self.pos).copied() - } - fn bump(&mut self) -> Option { - let b = self.peek()?; - self.pos += 1; - Some(b) - } - fn eat(&mut self, b: u8) -> bool { - if self.peek() == Some(b) { - self.pos += 1; - true - } else { - false - } - } - fn digits(&mut self, count: usize) -> TemporalResult { - let mut value: u32 = 0; - for _ in 0..count { - match self.bump() { - Some(b) if b.is_ascii_digit() => value = value * 10 + (b - b'0') as u32, - _ => return Err(TemporalError::syntax("expected digit")), - } - } - Ok(value) - } - fn done(&self) -> bool { - self.pos >= self.bytes.len() - } -} - -fn parse_iso_year(c: &mut Cursor) -> TemporalResult { - match c.peek() { - Some(b'+') | Some(b'-') => { - let neg = c.bump() == Some(b'-'); - let v = c.digits(6)? as i32; - if neg && v == 0 { - return Err(TemporalError::syntax("-000000 is not a valid extended year")); - } - Ok(if neg { -v } else { v }) - } - _ => Ok(c.digits(4)? as i32), - } -} - -/// Parses `hh[:mm[:ss[.f{1,9}]]]` (with or without separators, but -/// consistently) returning nanoseconds since midnight. -fn parse_iso_time(c: &mut Cursor) -> TemporalResult { - let hour = c.digits(2)?; - if hour > 23 { - return Err(TemporalError::syntax("hour out of range")); - } - let mut ns = hour as i128 * NS_PER_HOUR; - let sep = c.peek() == Some(b':'); - if sep { - c.pos += 1; - } - if matches!(c.peek(), Some(b) if b.is_ascii_digit()) { - let minute = c.digits(2)?; - if minute > 59 { - return Err(TemporalError::syntax("minute out of range")); - } - ns += minute as i128 * NS_PER_MINUTE; - let had_sep = c.peek() == Some(b':'); - if had_sep != sep && had_sep { - return Err(TemporalError::syntax("inconsistent time separators")); - } - if had_sep { - c.pos += 1; - } - if (had_sep || !sep) && matches!(c.peek(), Some(b) if b.is_ascii_digit()) { - if !had_sep && sep { - return Err(TemporalError::syntax("inconsistent time separators")); - } - let second = c.digits(2)?; - if second > 59 { - return Err(TemporalError::syntax("second out of range")); - } - ns += second as i128 * NS_PER_SECOND; - if matches!(c.peek(), Some(b'.') | Some(b',')) { - c.pos += 1; - let mut frac: i128 = 0; - let mut count = 0; - while let Some(b) = c.peek() { - if !b.is_ascii_digit() { - break; - } - if count >= 9 { - return Err(TemporalError::syntax("fraction exceeds 9 digits")); - } - frac = frac * 10 + (b - b'0') as i128; - count += 1; - c.pos += 1; - } - if count == 0 { - return Err(TemporalError::syntax("fraction requires at least one digit")); - } - for _ in count..9 { - frac *= 10; - } - ns += frac; - } - } - } else if sep { - return Err(TemporalError::syntax("expected minutes after ':'")); - } - Ok(ns) -} - -fn parse_utc_offset(c: &mut Cursor) -> TemporalResult> { - match c.peek() { - Some(b'Z') | Some(b'z') => { - c.pos += 1; - Ok(Some(UtcOffsetRecord::Zulu)) - } - Some(b'+') | Some(b'-') => { - let neg = c.bump() == Some(b'-'); - let hour = c.digits(2)?; - if hour > 23 { - return Err(TemporalError::syntax("offset hour out of range")); - } - let mut ns = hour as i128 * NS_PER_HOUR; - let sep = c.eat(b':'); - if matches!(c.peek(), Some(b) if b.is_ascii_digit()) { - let minute = c.digits(2)?; - if minute > 59 { - return Err(TemporalError::syntax("offset minute out of range")); - } - ns += minute as i128 * NS_PER_MINUTE; - let sep2 = c.eat(b':'); - if sep2 && !sep { - return Err(TemporalError::syntax("inconsistent offset separators")); - } - if (sep2 || !sep) && matches!(c.peek(), Some(b) if b.is_ascii_digit()) { - let second = c.digits(2)?; - if second > 59 { - return Err(TemporalError::syntax("offset second out of range")); - } - ns += second as i128 * NS_PER_SECOND; - if matches!(c.peek(), Some(b'.') | Some(b',')) { - c.pos += 1; - let mut frac: i128 = 0; - let mut count = 0; - while let Some(b) = c.peek() { - if !b.is_ascii_digit() { - break; - } - if count >= 9 { - return Err(TemporalError::syntax("offset fraction exceeds 9 digits")); - } - frac = frac * 10 + (b - b'0') as i128; - count += 1; - c.pos += 1; - } - if count == 0 { - return Err(TemporalError::syntax("offset fraction requires digits")); - } - for _ in count..9 { - frac *= 10; - } - ns += frac; - } - } - } else if sep { - return Err(TemporalError::syntax("expected offset minutes after ':'")); - } - Ok(Some(UtcOffsetRecord::Numeric(if neg { -ns } else { ns }))) - } - _ => Ok(None), - } -} - -fn parse_annotations( - c: &mut Cursor, - time_zone: &mut Option, - calendar: &mut Option, -) -> TemporalResult<()> { - let mut calendar_seen = false; - while c.eat(b'[') { - let critical = c.eat(b'!'); - let start = c.pos; - while let Some(b) = c.peek() { - if b == b']' { - break; - } - c.pos += 1; - } - if !c.eat(b']') { - return Err(TemporalError::syntax("unterminated annotation")); - } - let body = std::str::from_utf8(&c.bytes[start..c.pos - 1]) - .map_err(|_| TemporalError::syntax("annotation is not valid UTF-8"))?; - if body.is_empty() { - return Err(TemporalError::syntax("empty annotation")); - } - if let Some(value) = body.strip_prefix("u-ca=") { - if value.is_empty() { - return Err(TemporalError::syntax("empty u-ca annotation value")); - } - if calendar_seen { - if critical { - return Err(TemporalError::range("duplicate critical u-ca annotation")); - } - continue; // first annotation wins per spec - } - calendar_seen = true; - if value != "iso8601" && critical { - return Err(TemporalError::range(format!( - "unsupported critical calendar annotation '{value}'" - ))); - } - if value != "iso8601" { - return Err(TemporalError::range(format!( - "only the iso8601 calendar is supported, got '{value}'" - ))); - } - *calendar = Some(value.to_string()); - } else if body.contains('=') { - // Unknown key=value annotation: ignored unless critical. - if critical { - return Err(TemporalError::range(format!( - "unknown critical annotation '{body}'" - ))); - } - } else { - // Time zone annotation: first bracketed non key=value item only, - // and it must come before any other annotation. - if time_zone.is_some() || calendar_seen { - return Err(TemporalError::syntax( - "time zone annotation must be the first annotation", - )); - } - *time_zone = Some(body.to_string()); - } - } - Ok(()) -} - -/// Parses `date[T time[offset]][annotations]`. Central grammar for Instant and -/// the plain types (which apply their own presence rules on the result). -pub fn parse_iso_datetime_string(text: &str) -> TemporalResult { - let mut c = Cursor::new(text); - let year = parse_iso_year(&mut c)?; - let dash = c.eat(b'-'); - let month = c.digits(2)? as u8; - if dash && !c.eat(b'-') { - return Err(TemporalError::syntax("inconsistent date separators")); - } - if !dash && c.peek() == Some(b'-') { - return Err(TemporalError::syntax("inconsistent date separators")); - } - let day = c.digits(2)? as u8; - if !(1..=12).contains(&month) { - return Err(TemporalError::range(format!("month {month} out of range"))); - } - if day < 1 || day > days_in_month(year, month) { - return Err(TemporalError::range(format!("day {day} out of range"))); - } - - let mut has_time = false; - let mut time_ns = 0_i128; - let mut offset = None; - if matches!(c.peek(), Some(b'T') | Some(b't') | Some(b' ')) { - c.pos += 1; - has_time = true; - time_ns = parse_iso_time(&mut c)?; - offset = parse_utc_offset(&mut c)?; - } - - let mut time_zone = None; - let mut calendar = None; - parse_annotations(&mut c, &mut time_zone, &mut calendar)?; - if !c.done() { - return Err(TemporalError::syntax(format!( - "unexpected trailing characters at position {}", - c.pos - ))); - } - Ok(ParsedIsoDateTime { year, month, day, has_time, time_ns, offset, time_zone, calendar }) -} - // --------------------------------------------------------------------------- // Tests // --------------------------------------------------------------------------- @@ -1469,15 +1158,18 @@ mod tests { #[test] fn instant_add_overflow_at_boundary() { let max = Instant::from_epoch_nanoseconds(INSTANT_NS_MAX).unwrap(); - let one_ns = Duration::from_time(0, 0, 0, 0, 0, 1).unwrap(); + let one_ns = Duration::new([0, 0, 0, 0, 0, 0, 0, 0, 0, 1]).unwrap(); assert_eq!(max.add(&one_ns).unwrap_err().kind, TemporalErrorKind::Range); - assert_eq!(max.subtract(&one_ns).unwrap().epoch_nanoseconds(), INSTANT_NS_MAX - 1); + assert_eq!( + max.subtract(&one_ns).unwrap().epoch_nanoseconds(), + INSTANT_NS_MAX - 1 + ); } #[test] fn instant_rejects_date_units() { let i = Instant::from_epoch_nanoseconds(0).unwrap(); - let d = Duration::new(0, 0, 0, 1, 0, 0, 0, 0, 0, 0).unwrap(); + let d = Duration::new([0, 0, 0, 1, 0, 0, 0, 0, 0, 0]).unwrap(); assert_eq!(i.add(&d).unwrap_err().kind, TemporalErrorKind::Range); } @@ -1485,7 +1177,6 @@ mod tests { fn instant_epoch_milliseconds_floors_negative() { let i = Instant::from_epoch_nanoseconds(-1).unwrap(); assert_eq!(i.epoch_milliseconds(), -1); - assert_eq!(i.epoch_seconds(), -1); } // ---- Rounding, including negative operands ---- @@ -1498,12 +1189,30 @@ mod tests { assert_eq!(round_to_increment(-7, 3, RoundingMode::Trunc).unwrap(), -6); assert_eq!(round_to_increment(-7, 3, RoundingMode::Expand).unwrap(), -9); // Half cases at exact midpoint: x = -3, increment 2. - assert_eq!(round_to_increment(-3, 2, RoundingMode::HalfExpand).unwrap(), -4); - assert_eq!(round_to_increment(-3, 2, RoundingMode::HalfTrunc).unwrap(), -2); - assert_eq!(round_to_increment(-3, 2, RoundingMode::HalfCeil).unwrap(), -2); - assert_eq!(round_to_increment(-3, 2, RoundingMode::HalfFloor).unwrap(), -4); - assert_eq!(round_to_increment(-3, 2, RoundingMode::HalfEven).unwrap(), -4); - assert_eq!(round_to_increment(-1, 2, RoundingMode::HalfEven).unwrap(), 0); + assert_eq!( + round_to_increment(-3, 2, RoundingMode::HalfExpand).unwrap(), + -4 + ); + assert_eq!( + round_to_increment(-3, 2, RoundingMode::HalfTrunc).unwrap(), + -2 + ); + assert_eq!( + round_to_increment(-3, 2, RoundingMode::HalfCeil).unwrap(), + -2 + ); + assert_eq!( + round_to_increment(-3, 2, RoundingMode::HalfFloor).unwrap(), + -4 + ); + assert_eq!( + round_to_increment(-3, 2, RoundingMode::HalfEven).unwrap(), + -4 + ); + assert_eq!( + round_to_increment(-1, 2, RoundingMode::HalfEven).unwrap(), + 0 + ); } #[test] @@ -1555,7 +1264,7 @@ mod tests { fn instant_parse_annotations() { let i = Instant::parse("1970-01-01T00:00:00Z[UTC][u-ca=iso8601]").unwrap(); assert_eq!(i.epoch_nanoseconds(), 0); - assert!(Instant::parse("1970-01-01T00:00:00Z[!u-ca=japanese]").is_err()); + assert!(Instant::parse("1970-01-01T00:00:00Z[!u-ca=japanese]").is_ok()); assert!(Instant::parse("1970-01-01T00:00:00Z[!unknown=x]").is_err()); // Non-critical unknown annotation ignored. assert!(Instant::parse("1970-01-01T00:00:00Z[unknown=x]").is_ok()); @@ -1578,7 +1287,10 @@ mod tests { let i = Instant::from_epoch_nanoseconds(1_500_000_000).unwrap(); assert_eq!(i.format(Precision::Auto), "1970-01-01T00:00:01.5Z"); assert_eq!(i.format(Precision::Digits(0)), "1970-01-01T00:00:01Z"); - assert_eq!(i.format(Precision::Digits(6)), "1970-01-01T00:00:01.500000Z"); + assert_eq!( + i.format(Precision::Digits(6)), + "1970-01-01T00:00:01.500000Z" + ); assert_eq!(i.format(Precision::Minute), "1970-01-01T00:00Z"); } @@ -1602,40 +1314,43 @@ mod tests { #[test] fn instant_until_rejects_date_units() { let a = Instant::from_epoch_nanoseconds(0).unwrap(); - assert!(a.until(a, Unit::Day, Unit::Nanosecond, 1, RoundingMode::Trunc).is_err()); + assert!( + a.until(a, Unit::Day, Unit::Nanosecond, 1, RoundingMode::Trunc) + .is_err() + ); } // ---- Duration validation ---- #[test] fn duration_mixed_sign_rejected() { - let err = Duration::new(1, 0, 0, -1, 0, 0, 0, 0, 0, 0).unwrap_err(); + let err = Duration::new([1, 0, 0, -1, 0, 0, 0, 0, 0, 0]).unwrap_err(); assert_eq!(err.kind, TemporalErrorKind::Range); - let err = Duration::new(0, 0, 0, 0, 1, 0, 0, 0, 0, -1).unwrap_err(); + let err = Duration::new([0, 0, 0, 0, 1, 0, 0, 0, 0, -1]).unwrap_err(); assert_eq!(err.kind, TemporalErrorKind::Range); - assert!(Duration::new(0, 0, 0, 0, -1, -30, 0, 0, 0, 0).is_ok()); + assert!(Duration::new([0, 0, 0, 0, -1, -30, 0, 0, 0, 0]).is_ok()); } #[test] fn duration_calendar_field_limit() { - let limit = 1_i64 << 32; - assert!(Duration::new(limit, 0, 0, 0, 0, 0, 0, 0, 0, 0).is_err()); - assert!(Duration::new(limit - 1, 0, 0, 0, 0, 0, 0, 0, 0, 0).is_ok()); + let limit = 1_i128 << 32; + assert!(Duration::new([limit, 0, 0, 0, 0, 0, 0, 0, 0, 0]).is_err()); + assert!(Duration::new([limit - 1, 0, 0, 0, 0, 0, 0, 0, 0, 0]).is_ok()); } #[test] fn duration_time_limit_2_pow_53_seconds() { - let max_s = (1_i64 << 53) - 1; - assert!(Duration::new(0, 0, 0, 0, 0, 0, max_s, 0, 0, 999_999_999).is_ok()); - assert!(Duration::new(0, 0, 0, 0, 0, 0, max_s + 1, 0, 0, 0).is_err()); + let max_s = (1_i128 << 53) - 1; + assert!(Duration::new([0, 0, 0, 0, 0, 0, max_s, 0, 0, 999_999_999]).is_ok()); + assert!(Duration::new([0, 0, 0, 0, 0, 0, max_s + 1, 0, 0, 0]).is_err()); // Days participate in the time bound. let max_days = max_s / 86_400; - assert!(Duration::new(0, 0, 0, max_days + 1, 0, 0, 0, 0, 0, 0).is_err()); + assert!(Duration::new([0, 0, 0, max_days + 1, 0, 0, 0, 0, 0, 0]).is_err()); } #[test] fn duration_sign_and_negate() { - let d = Duration::new(0, 0, 0, 0, 0, 0, 0, 0, 0, -5).unwrap(); + let d = Duration::new([0, 0, 0, 0, 0, 0, 0, 0, 0, -5]).unwrap(); assert_eq!(d.sign(), -1); assert_eq!(d.negated().sign(), 1); assert_eq!(d.abs().nanoseconds, 5); @@ -1659,24 +1374,35 @@ mod tests { #[test] fn duration_round_time_only() { - let d = Duration::from_time(0, 90, 0, 0, 0, 0).unwrap(); - let r = d.round(Unit::Hour, Unit::Hour, 1, RoundingMode::HalfExpand).unwrap(); + let d = Duration::new([0, 0, 0, 0, 0, 90, 0, 0, 0, 0]).unwrap(); + let r = d + .round(Unit::Hour, Unit::Hour, 1, RoundingMode::HalfExpand) + .unwrap(); assert_eq!(r.hours, 2); - let neg = d.negated().round(Unit::Hour, Unit::Hour, 1, RoundingMode::HalfExpand).unwrap(); + let neg = d + .negated() + .round(Unit::Hour, Unit::Hour, 1, RoundingMode::HalfExpand) + .unwrap(); assert_eq!(neg.hours, -2); - let cal = Duration::new(1, 0, 0, 0, 0, 0, 0, 0, 0, 0).unwrap(); - assert!(cal.round(Unit::Hour, Unit::Hour, 1, RoundingMode::Trunc).is_err()); - assert!(d.round(Unit::Month, Unit::Month, 1, RoundingMode::Trunc).is_err()); + let cal = Duration::new([1, 0, 0, 0, 0, 0, 0, 0, 0, 0]).unwrap(); + assert!( + cal.round(Unit::Hour, Unit::Hour, 1, RoundingMode::Trunc) + .is_err() + ); + assert!( + d.round(Unit::Month, Unit::Month, 1, RoundingMode::Trunc) + .is_err() + ); } #[test] fn duration_total_exact_and_float() { - let d = Duration::from_time(1, 30, 0, 0, 0, 0).unwrap(); + let d = Duration::new([0, 0, 0, 0, 1, 30, 0, 0, 0, 0]).unwrap(); let (q, r, unit) = d.total_exact(Unit::Hour).unwrap(); assert_eq!((q, r, unit), (1, NS_PER_HOUR / 2, NS_PER_HOUR)); assert_eq!(d.total_as_f64(Unit::Hour).unwrap(), 1.5); assert_eq!(d.total_as_f64(Unit::Minute).unwrap(), 90.0); - let cal = Duration::new(0, 1, 0, 0, 0, 0, 0, 0, 0, 0).unwrap(); + let cal = Duration::new([0, 1, 0, 0, 0, 0, 0, 0, 0, 0]).unwrap(); assert!(cal.total_exact(Unit::Day).is_err()); } @@ -1686,7 +1412,9 @@ mod tests { fn duration_parse_basic_and_sign() { let d = Duration::parse("P1Y2M3W4DT5H6M7S").unwrap(); assert_eq!( - (d.years, d.months, d.weeks, d.days, d.hours, d.minutes, d.seconds), + ( + d.years, d.months, d.weeks, d.days, d.hours, d.minutes, d.seconds + ), (1, 2, 3, 4, 5, 6, 7) ); let n = Duration::parse("-PT1H").unwrap(); @@ -1701,7 +1429,7 @@ mod tests { let n = Duration::parse("-PT0.000000001S").unwrap(); assert_eq!(n.nanoseconds, -1); let h = Duration::parse("PT0.5H").unwrap(); - assert_eq!((h.hours, h.milliseconds), (0, 1_800_000)); + assert_eq!((h.hours, h.minutes, h.milliseconds), (0, 30, 0)); assert!(Duration::parse("PT1.5H30M").is_err()); // fraction must be last assert!(Duration::parse("PT1.1234567890S").is_err()); // >9 digits } @@ -1720,7 +1448,7 @@ mod tests { #[test] fn duration_format_canonical() { assert_eq!(Duration::default().format(), "PT0S"); - let d = Duration::new(1, 0, 0, 2, 3, 0, 4, 500, 0, 0).unwrap(); + let d = Duration::new([1, 0, 0, 2, 3, 0, 4, 500, 0, 0]).unwrap(); assert_eq!(d.format(), "P1Y2DT3H4.5S"); assert_eq!(Duration::parse("-PT1H").unwrap().format(), "-PT1H"); // Round trip. @@ -1752,7 +1480,7 @@ mod tests { (-1, 12, 31), ] { let days = epoch_days_from_ymd(y, m, d); - assert_eq!(ymd_from_epoch_days(days), (y, m as u8, d as u8), "case {y}-{m}-{d}"); + assert_eq!(ymd_from_epoch_days(days), (y, m, d), "case {y}-{m}-{d}"); } assert_eq!(epoch_days_from_ymd(1970, 1, 1), 0); assert_eq!(epoch_days_from_ymd(1969, 12, 31), -1); diff --git a/crates/bamts-runtime/src/temporal/mod.rs b/crates/bamts-runtime/src/temporal/mod.rs index fbc63132..05db3d40 100644 --- a/crates/bamts-runtime/src/temporal/mod.rs +++ b/crates/bamts-runtime/src/temporal/mod.rs @@ -1,13 +1,16 @@ //! ECMA-402 Temporal primitives. +pub mod calendar; pub mod instant_duration; pub mod now_rounding; +pub mod parse; pub mod plain_types; +pub mod relative; pub mod zoned_date_time; #[derive(Clone, Copy, Debug, Eq, PartialEq)] #[repr(u8)] -pub(crate) enum TemporalKind { +pub enum TemporalKind { Instant, Duration, PlainDate, @@ -21,13 +24,24 @@ pub(crate) enum TemporalKind { impl TemporalKind { pub(crate) const COUNT: usize = 8; + pub(crate) const ALL: [Self; Self::COUNT] = [ + Self::Instant, + Self::Duration, + Self::PlainDate, + Self::PlainTime, + Self::PlainDateTime, + Self::PlainYearMonth, + Self::PlainMonthDay, + Self::ZonedDateTime, + ]; + pub(crate) const fn index(self) -> usize { self as usize } } #[derive(Clone, Debug)] -pub(crate) enum TemporalObject { +pub enum TemporalObject { Instant(instant_duration::Instant), Duration(instant_duration::Duration), PlainDate { @@ -51,6 +65,18 @@ pub(crate) enum TemporalObject { } impl TemporalObject { + /// The `[[Calendar]]` slot, for the classes that have one. + pub(crate) fn calendar(&self) -> Option { + match self { + Self::Instant(_) | Self::Duration(_) | Self::PlainTime(_) => None, + Self::PlainDate { calendar, .. } + | Self::PlainDateTime { calendar, .. } + | Self::PlainYearMonth { calendar, .. } + | Self::PlainMonthDay { calendar, .. } => Some(calendar.clone()), + Self::ZonedDateTime(value) => Some(value.calendar_id().clone()), + } + } + pub(crate) const fn kind(&self) -> TemporalKind { match self { Self::Instant(_) => TemporalKind::Instant, diff --git a/crates/bamts-runtime/src/temporal/now_rounding.rs b/crates/bamts-runtime/src/temporal/now_rounding.rs index f419918f..f93bf109 100644 --- a/crates/bamts-runtime/src/temporal/now_rounding.rs +++ b/crates/bamts-runtime/src/temporal/now_rounding.rs @@ -1,6 +1,6 @@ //! Deterministic Temporal.Now host boundary, exact rounding, and serialization. //! -//! A host must inject civil and monotonic clock readings, time-zone data, and +//! A host must inject civil clock readings, time-zone data, and //! calendar projection data. This module never reads ambient process or machine //! state. It implements the host-facing operations in Temporal sections 2 and //! 11 together with the serialization operations shared by Temporal and Intl. @@ -14,79 +14,46 @@ use std::fmt; use super::instant_duration::{ - Duration, INSTANT_NS_MAX, INSTANT_NS_MIN, Instant, NS_PER_DAY, NS_PER_HOUR, - NS_PER_MICROSECOND, NS_PER_MILLISECOND, NS_PER_MINUTE, NS_PER_SECOND, Overflow, Precision, - RoundingMode, TemporalError, TemporalResult, Unit, format_iso_date, format_time_ns, - round_to_increment, validate_rounding_increment, ymd_from_epoch_days, + Duration, INSTANT_NS_MAX, INSTANT_NS_MIN, Instant, NS_PER_DAY, NS_PER_HOUR, NS_PER_MICROSECOND, + NS_PER_MILLISECOND, NS_PER_MINUTE, NS_PER_SECOND, Overflow, Precision, RoundingMode, + TemporalError, TemporalResult, Unit, format_iso_date, format_time_ns, round_to_increment, + round_to_increment_as_if_positive, validate_rounding_increment, ymd_from_epoch_days, }; use super::plain_types::{PlainDate, PlainDateTime, PlainTime}; +pub use super::zoned_date_time::UtcOffsetNanoseconds; +use super::zoned_date_time::format_utc_offset; -/// A validated UTC offset. Temporal offsets are strictly between -24 and +24 hours. -#[derive(Clone, Copy, Debug, Eq, Hash, Ord, PartialEq, PartialOrd)] -pub struct UtcOffsetNanoseconds(i64); - -impl UtcOffsetNanoseconds { - /// Validates the range required by `GetOffsetNanosecondsFor`. - pub fn new(nanoseconds: i64) -> TemporalResult { - let day = i64::try_from(NS_PER_DAY) - .map_err(|_| TemporalError::range("Temporal day does not fit an offset"))?; - if !(-day < nanoseconds && nanoseconds < day) { - return Err(TemporalError::range(format!( - "UTC offset {nanoseconds}ns must be strictly between -24h and +24h" - ))); - } - Ok(Self(nanoseconds)) - } - - #[must_use] - pub const fn as_nanoseconds(self) -> i64 { - self.0 - } -} - -/// One atomic host clock observation. -/// -/// The civil reading is visible to Temporal. The monotonic reading is retained -/// so an embedding can order snapshots without substituting it for civil time. +/// One host clock observation. #[derive(Clone, Copy, Debug, Eq, PartialEq)] pub struct ClockSnapshot { instant: Instant, - monotonic_nanoseconds: u128, } impl ClockSnapshot { /// Builds a snapshot from a host reading that already satisfies the /// `HostSystemUTCEpochNanoseconds` range contract. - pub fn new(epoch_nanoseconds: i128, monotonic_nanoseconds: u128) -> TemporalResult { + pub fn new(epoch_nanoseconds: i128) -> TemporalResult { Ok(Self { instant: Instant::from_epoch_nanoseconds(epoch_nanoseconds)?, - monotonic_nanoseconds, }) } /// Implements the specification's default host clamping behavior. #[must_use] - pub fn clamped(epoch_nanoseconds: i128, monotonic_nanoseconds: u128) -> Self { - let epoch_nanoseconds = epoch_nanoseconds.clamp(INSTANT_NS_MIN, INSTANT_NS_MAX); - let instant = match Instant::from_epoch_nanoseconds(epoch_nanoseconds) { - Ok(instant) => instant, + pub fn clamped(epoch_nanoseconds: i128) -> Self { + match Self::new(epoch_nanoseconds.clamp(INSTANT_NS_MIN, INSTANT_NS_MAX)) { + Ok(snapshot) => snapshot, Err(_) => unreachable!("clamped epoch nanoseconds are always a valid Instant"), - }; - Self { instant, monotonic_nanoseconds } + } } #[must_use] pub const fn instant(self) -> Instant { self.instant } - - #[must_use] - pub const fn monotonic_nanoseconds(self) -> u128 { - self.monotonic_nanoseconds - } } -/// Injected atomic civil/monotonic clock capability. +/// Injected civil clock capability. pub trait HostClock { type Error; @@ -166,6 +133,17 @@ where pub type NowResult = Result>; +/// The outcome of projecting one clock snapshot into a zone. +pub type ZonedNowResult = NowResult< + NowZonedProjection< + ::Identifier, + ::Identifier, + >, + ::Error, + ::Error, + ::Error, +>; + /// One clock snapshot projected into a time zone and tagged with an ISO calendar. /// /// C11.4 can construct its `ZonedDateTime` directly from `instant`, `time_zone`, @@ -175,16 +153,10 @@ pub struct NowZonedProjection { snapshot: ClockSnapshot, time_zone: TimeZoneId, calendar: CalendarId, - offset: UtcOffsetNanoseconds, iso_date_time: PlainDateTime, } impl NowZonedProjection { - #[must_use] - pub const fn snapshot(&self) -> ClockSnapshot { - self.snapshot - } - #[must_use] pub const fn instant(&self) -> Instant { self.snapshot.instant() @@ -200,11 +172,6 @@ impl NowZonedProjection { &self.calendar } - #[must_use] - pub const fn offset(&self) -> UtcOffsetNanoseconds { - self.offset - } - #[must_use] pub const fn iso_date_time(&self) -> PlainDateTime { self.iso_date_time @@ -226,10 +193,14 @@ where { #[must_use] pub const fn new(clock: Clock, time_zones: TimeZones, calendars: Calendars) -> Self { - Self { clock, time_zones, calendars } + Self { + clock, + time_zones, + calendars, + } } - /// Captures civil and monotonic readings atomically through the host. + /// Captures one civil reading through the host. pub fn capture( &mut self, ) -> NowResult { @@ -247,23 +218,23 @@ where pub fn time_zone_id( &mut self, ) -> NowResult { - self.time_zones.system_time_zone_id().map_err(NowError::TimeZone) + self.time_zones + .system_time_zone_id() + .map_err(NowError::TimeZone) } /// Captures once and projects the instant into an explicit or system zone. pub fn zoned_date_time_iso( &mut self, time_zone: Option, - ) -> NowResult< - NowZonedProjection, - Clock::Error, - TimeZones::Error, - Calendars::Error, - > { + ) -> ZonedNowResult { // SystemDateTime resolves the zone before observing the civil clock. let time_zone = match time_zone { Some(time_zone) => time_zone, - None => self.time_zones.system_time_zone_id().map_err(NowError::TimeZone)?, + None => self + .time_zones + .system_time_zone_id() + .map_err(NowError::TimeZone)?, }; let snapshot = self.capture()?; let offset = self @@ -275,7 +246,6 @@ where snapshot, time_zone, calendar: self.calendars.iso8601_identifier(), - offset, iso_date_time, }) } @@ -288,14 +258,6 @@ where Ok(self.zoned_date_time_iso(time_zone)?.iso_date_time()) } - /// `Temporal.Now.plainDateISO()`. - pub fn plain_date_iso( - &mut self, - time_zone: Option, - ) -> NowResult { - Ok(self.zoned_date_time_iso(time_zone)?.iso_date_time().date) - } - /// `Temporal.Now.plainTimeISO()`. pub fn plain_time_iso( &mut self, @@ -324,7 +286,7 @@ pub fn iso_date_time_for( ) -> TemporalResult { let local_ns = instant .epoch_nanoseconds() - .checked_add(i128::from(offset.as_nanoseconds())) + .checked_add(i128::from(offset.get())) .ok_or_else(|| TemporalError::range("time-zone projection overflow"))?; let epoch_days = local_ns.div_euclid(NS_PER_DAY); let time_ns = local_ns.rem_euclid(NS_PER_DAY); @@ -341,13 +303,14 @@ pub fn iso_date_time_for( year, i64::from(month), i64::from(day), - i64::try_from(hour).map_err(|_| TemporalError::range("hour overflow"))?, - i64::try_from(minute).map_err(|_| TemporalError::range("minute overflow"))?, - i64::try_from(second).map_err(|_| TemporalError::range("second overflow"))?, - i64::try_from(millisecond) - .map_err(|_| TemporalError::range("millisecond overflow"))?, - i64::try_from(microsecond).map_err(|_| TemporalError::range("microsecond overflow"))?, - i64::try_from(nanosecond).map_err(|_| TemporalError::range("nanosecond overflow"))?, + [ + i64::try_from(hour).map_err(|_| TemporalError::range("hour overflow"))?, + i64::try_from(minute).map_err(|_| TemporalError::range("minute overflow"))?, + i64::try_from(second).map_err(|_| TemporalError::range("second overflow"))?, + i64::try_from(millisecond).map_err(|_| TemporalError::range("millisecond overflow"))?, + i64::try_from(microsecond).map_err(|_| TemporalError::range("microsecond overflow"))?, + i64::try_from(nanosecond).map_err(|_| TemporalError::range("nanosecond overflow"))?, + ], Overflow::Reject, ) } @@ -378,49 +341,15 @@ pub fn round_epoch_nanoseconds( mode: RoundingMode, ) -> TemporalResult { validate_rounding_increment(smallest_unit, increment, false)?; - let unit_ns = smallest_unit.ns_per().ok_or_else(|| { - TemporalError::range("epoch rounding requires day or a time unit") - })?; + let unit_ns = smallest_unit + .ns_per() + .ok_or_else(|| TemporalError::range("epoch rounding requires day or a time unit"))?; let step = unit_ns .checked_mul(increment) .ok_or_else(|| TemporalError::range("rounding increment overflow"))?; round_to_increment_as_if_positive(value, step, mode) } -fn round_to_increment_as_if_positive( - value: i128, - increment: i128, - mode: RoundingMode, -) -> TemporalResult { - let quotient = value.div_euclid(increment); - let remainder = value.rem_euclid(increment); - if remainder == 0 { - return Ok(value); - } - let half = increment / 2; - let tie = increment % 2 == 0 && remainder == half; - let above_half = remainder > half && !tie; - let round_up = match mode { - RoundingMode::Ceil | RoundingMode::Expand => true, - RoundingMode::Floor | RoundingMode::Trunc => false, - RoundingMode::HalfCeil | RoundingMode::HalfExpand => above_half || tie, - RoundingMode::HalfFloor | RoundingMode::HalfTrunc => above_half, - RoundingMode::HalfEven => { - above_half || (tie && quotient.rem_euclid(2) == 1) - } - }; - let quotient = if round_up { - quotient - .checked_add(1) - .ok_or_else(|| TemporalError::range("epoch rounding overflow"))? - } else { - quotient - }; - quotient - .checked_mul(increment) - .ok_or_else(|| TemporalError::range("epoch rounding overflow")) -} - /// Rounds exact time nanoseconds and balances them into a shared `Duration`. pub fn round_and_balance_duration( total_nanoseconds: i128, @@ -583,28 +512,30 @@ pub enum ShowOffset { } /// Formats a canonical calendar annotation, including its own critical flag. -pub fn format_calendar_annotation( - calendar: &str, - show: ShowCalendar, -) -> TemporalResult { +pub fn format_calendar_annotation(calendar: &str, show: ShowCalendar) -> TemporalResult { if show == ShowCalendar::Never || (show == ShowCalendar::Auto && calendar == "iso8601") { return Ok(String::new()); } validate_calendar_identifier(calendar)?; - let critical = if show == ShowCalendar::Critical { "!" } else { "" }; + let critical = if show == ShowCalendar::Critical { + "!" + } else { + "" + }; Ok(format!("[{critical}u-ca={calendar}]")) } /// Formats a time-zone annotation, including its own critical flag. -pub fn format_time_zone_annotation( - time_zone: &str, - show: ShowTimeZone, -) -> TemporalResult { +pub fn format_time_zone_annotation(time_zone: &str, show: ShowTimeZone) -> TemporalResult { if show == ShowTimeZone::Never { return Ok(String::new()); } validate_time_zone_annotation(time_zone)?; - let critical = if show == ShowTimeZone::Critical { "!" } else { "" }; + let critical = if show == ShowTimeZone::Critical { + "!" + } else { + "" + }; Ok(format!("[{critical}{time_zone}]")) } @@ -627,8 +558,7 @@ fn validate_calendar_identifier(calendar: &str) -> TemporalResult<()> { fn validate_time_zone_annotation(time_zone: &str) -> TemporalResult<()> { let valid = !time_zone.is_empty() && time_zone.bytes().all(|byte| { - byte.is_ascii_alphanumeric() - || matches!(byte, b'/' | b'_' | b'-' | b'+' | b':' | b'.') + byte.is_ascii_alphanumeric() || matches!(byte, b'/' | b'_' | b'-' | b'+' | b':' | b'.') }); if !valid { return Err(TemporalError::range(format!( @@ -638,40 +568,15 @@ fn validate_time_zone_annotation(time_zone: &str) -> TemporalResult<()> { Ok(()) } -/// Formats an exact UTC offset as `±HH:MM[:SS[.fffffffff]]`. -#[must_use] -pub fn format_utc_offset(offset: UtcOffsetNanoseconds) -> String { - let value = offset.as_nanoseconds(); - let sign = if value < 0 { '-' } else { '+' }; - let absolute = i128::from(value).abs(); - let hour = absolute / NS_PER_HOUR; - let minute = (absolute / NS_PER_MINUTE) % 60; - let second = (absolute / NS_PER_SECOND) % 60; - let fraction = absolute % NS_PER_SECOND; - let mut result = format!("{sign}{hour:02}:{minute:02}"); - if second != 0 || fraction != 0 { - result.push_str(&format!(":{second:02}")); - if fraction != 0 { - let mut digits = format!("{fraction:09}"); - while digits.ends_with('0') { - digits.pop(); - } - result.push('.'); - result.push_str(&digits); - } - } - result -} - /// Formats a named-zone date-time offset after half-expand minute rounding. pub fn format_utc_offset_rounded(offset: UtcOffsetNanoseconds) -> TemporalResult { let rounded = round_to_increment( - i128::from(offset.as_nanoseconds()), + i128::from(offset.get()), NS_PER_MINUTE, RoundingMode::HalfExpand, )?; - let rounded = i64::try_from(rounded) - .map_err(|_| TemporalError::range("rounded UTC offset overflow"))?; + let rounded = + i64::try_from(rounded).map_err(|_| TemporalError::range("rounded UTC offset overflow"))?; Ok(format_utc_offset(UtcOffsetNanoseconds::new(rounded)?)) } @@ -680,7 +585,10 @@ pub fn format_utc_offset_rounded(offset: UtcOffsetNanoseconds) -> TemporalResult pub fn format_iso_date_time(value: &PlainDateTime, precision: Precision) -> String { let mut result = format_iso_date(value.date.year, value.date.month, value.date.day); result.push('T'); - result.push_str(&format_time_ns(value.time.to_nanoseconds_of_day(), precision)); + result.push_str(&format_time_ns( + value.time.to_nanoseconds_of_day(), + precision, + )); result } @@ -782,7 +690,6 @@ mod tests { #[derive(Clone, Copy, Debug, Eq, PartialEq)] enum ZoneFailure { - System, Offset, } @@ -859,9 +766,7 @@ mod tests { } } - fn clock( - readings: Vec>, - ) -> (FakeClock, Rc>) { + fn clock(readings: Vec>) -> (FakeClock, Rc>) { let calls = Rc::new(Cell::new(0)); ( FakeClock { @@ -915,8 +820,14 @@ mod tests { (RoundingMode::HalfEven, 4, -4), ]; for (mode, positive, negative) in cases { - assert_eq!(round_nanoseconds(5, Unit::Nanosecond, 2, mode), Ok(positive)); - assert_eq!(round_nanoseconds(-5, Unit::Nanosecond, 2, mode), Ok(negative)); + assert_eq!( + round_nanoseconds(5, Unit::Nanosecond, 2, mode), + Ok(positive) + ); + assert_eq!( + round_nanoseconds(-5, Unit::Nanosecond, 2, mode), + Ok(negative) + ); } assert_eq!( round_nanoseconds(-7, Unit::Nanosecond, 2, RoundingMode::HalfEven), @@ -949,10 +860,7 @@ mod tests { fn increment_validation_and_overflow_are_typed_errors() { assert!(round_nanoseconds(1, Unit::Second, 0, RoundingMode::Trunc).is_err()); assert!(round_nanoseconds(1, Unit::Second, 7, RoundingMode::Trunc).is_err()); - assert!( - round_nanoseconds(i128::MAX, Unit::Day, i128::MAX, RoundingMode::Ceil) - .is_err() - ); + assert!(round_nanoseconds(i128::MAX, Unit::Day, i128::MAX, RoundingMode::Ceil).is_err()); assert!( round_and_balance_duration( i128::MIN, @@ -1013,10 +921,7 @@ mod tests { }) ); assert_eq!( - SecondsStringPrecision::resolve( - Some(Unit::Minute), - FractionalSecondDigits::Digits(9), - ), + SecondsStringPrecision::resolve(Some(Unit::Minute), FractionalSecondDigits::Digits(9),), Ok(SecondsStringPrecision { precision: Precision::Minute, unit: Unit::Minute, @@ -1025,18 +930,14 @@ mod tests { ); assert!(FractionalSecondDigits::digits(10).is_err()); assert!( - SecondsStringPrecision::resolve( - Some(Unit::Hour), - FractionalSecondDigits::Auto, - ) - .is_err() + SecondsStringPrecision::resolve(Some(Unit::Hour), FractionalSecondDigits::Auto,) + .is_err() ); } #[test] fn precision_serialization_trims_or_fixes_fraction_and_omits_seconds() { - let time = PlainTime::new(1, 2, 3, 123, 400, 0, Overflow::Reject) - .expect("valid time"); + let time = PlainTime::new(1, 2, 3, 123, 400, 0, Overflow::Reject).expect("valid time"); assert_eq!( serialize_plain_time(&time, ToStringRoundingOptions::default()), Ok("01:02:03.1234".to_owned()) @@ -1082,21 +983,9 @@ mod tests { Ok("[!Test/Zone]".to_owned()) ); - let date_time = PlainDateTime::new( - 1970, - 1, - 1, - 1, - 2, - 3, - 123, - 400, - 0, - Overflow::Reject, - ) - .expect("valid date-time"); - let offset = UtcOffsetNanoseconds::new(90 * 60 * 1_000_000_000) - .expect("valid offset"); + let date_time = PlainDateTime::new(1970, 1, 1, [1, 2, 3, 123, 400, 0], Overflow::Reject) + .expect("valid date-time"); + let offset = UtcOffsetNanoseconds::new(90 * 60 * 1_000_000_000).expect("valid offset"); assert_eq!( serialize_zoned_date_time(ZonedSerialization { iso_date_time: &date_time, @@ -1120,8 +1009,8 @@ mod tests { .expect("valid offset"); assert_eq!(format_utc_offset(exact), "+01:02:03.4"); - let negative_tie = UtcOffsetNanoseconds::new(-((62 * 60 + 30) * 1_000_000_000)) - .expect("valid offset"); + let negative_tie = + UtcOffsetNanoseconds::new(-((62 * 60 + 30) * 1_000_000_000)).expect("valid offset"); assert_eq!( format_utc_offset_rounded(negative_tie), Ok("-01:03".to_owned()) @@ -1134,15 +1023,16 @@ mod tests { let calls = Rc::new(Cell::new(0)); let clock = FakeClock { readings: vec![ - Ok(ClockSnapshot::new(0, 41).expect("valid snapshot")), - Ok(ClockSnapshot::new(NS_PER_DAY, 42).expect("valid snapshot")), + Ok(ClockSnapshot::new(0).expect("valid snapshot")), + Ok(ClockSnapshot::new(NS_PER_DAY).expect("valid snapshot")), ], cursor: 0, calls: Rc::clone(&calls), }; - let (zones, system_calls, offset_calls) = time_zones(Ok( - UtcOffsetNanoseconds::new(90 * 60 * 1_000_000_000).expect("valid offset"), - )); + let (zones, system_calls, offset_calls) = time_zones(Ok(UtcOffsetNanoseconds::new( + 90 * 60 * 1_000_000_000, + ) + .expect("valid offset"))); let (calendars, calendar_calls) = calendars(false); let mut now = TemporalNow::new(clock, zones, calendars); @@ -1150,10 +1040,10 @@ mod tests { assert_eq!(calls.get(), 1); assert_eq!(system_calls.get(), 1); assert_eq!(offset_calls.get(), 1); - assert_eq!(projected.snapshot().monotonic_nanoseconds(), 41); + assert_eq!(projected.instant().epoch_nanoseconds(), 0); assert_eq!( projected.iso_date_time(), - PlainDateTime::new(1970, 1, 1, 1, 30, 0, 0, 0, 0, Overflow::Reject) + PlainDateTime::new(1970, 1, 1, [1, 30, 0, 0, 0, 0], Overflow::Reject) .expect("valid expected date-time") ); assert_eq!(projected.calendar_id(), "iso8601"); @@ -1164,61 +1054,64 @@ mod tests { Ok("hebrew:1970-01-01".to_owned()) ); assert_eq!(calendar_calls.get(), 1); - assert_eq!(calls.get(), 1, "calendar projection must not recapture time"); + assert_eq!( + calls.get(), + 1, + "calendar projection must not recapture time" + ); } #[test] fn injected_readings_prove_no_ambient_clock_dependency() { - let (clock_a, _) = clock(vec![Ok( - ClockSnapshot::new(-1, 10).expect("valid snapshot"), - )]); - let (zones_a, _, _) = time_zones(Ok( - UtcOffsetNanoseconds::new(0).expect("valid offset"), - )); + let (clock_a, _) = clock(vec![Ok(ClockSnapshot::new(-1).expect("valid snapshot"))]); + let (zones_a, _, _) = time_zones(Ok(UtcOffsetNanoseconds::new(0).expect("valid offset"))); let (calendars_a, _) = calendars(false); let mut now_a = TemporalNow::new(clock_a, zones_a, calendars_a); - let (clock_b, _) = clock(vec![Ok( - ClockSnapshot::new(1, 20).expect("valid snapshot"), - )]); - let (zones_b, _, _) = time_zones(Ok( - UtcOffsetNanoseconds::new(0).expect("valid offset"), - )); + let (clock_b, _) = clock(vec![Ok(ClockSnapshot::new(1).expect("valid snapshot"))]); + let (zones_b, _, _) = time_zones(Ok(UtcOffsetNanoseconds::new(0).expect("valid offset"))); let (calendars_b, _) = calendars(false); let mut now_b = TemporalNow::new(clock_b, zones_b, calendars_b); - assert_eq!(now_a.instant().expect("first injected clock").epoch_nanoseconds(), -1); - assert_eq!(now_b.instant().expect("second injected clock").epoch_nanoseconds(), 1); + assert_eq!( + now_a + .instant() + .expect("first injected clock") + .epoch_nanoseconds(), + -1 + ); + assert_eq!( + now_b + .instant() + .expect("second injected clock") + .epoch_nanoseconds(), + 1 + ); } #[test] fn provider_failures_remain_typed_and_clock_is_not_retried() { let (failed_clock, clock_calls) = clock(vec![Err(ClockFailure::Unavailable)]); - let (zones, _, _) = time_zones(Ok( - UtcOffsetNanoseconds::new(0).expect("valid offset"), - )); - let (calendars, _) = calendars(false); - let mut now = TemporalNow::new(failed_clock, zones, calendars); - assert_eq!(now.instant(), Err(NowError::Clock(ClockFailure::Unavailable))); + let (zones, _, _) = time_zones(Ok(UtcOffsetNanoseconds::new(0).expect("valid offset"))); + let (first_calendars, _) = calendars(false); + let mut now = TemporalNow::new(failed_clock, zones, first_calendars); + assert_eq!( + now.instant(), + Err(NowError::Clock(ClockFailure::Unavailable)) + ); assert_eq!(clock_calls.get(), 1); - let (good_clock, _) = clock(vec![Ok( - ClockSnapshot::new(0, 0).expect("valid snapshot"), - )]); + let (good_clock, _) = clock(vec![Ok(ClockSnapshot::new(0).expect("valid snapshot"))]); let (failed_zones, _, _) = time_zones(Err(ZoneFailure::Offset)); - let (calendars, _) = calendars(false); - let mut now = TemporalNow::new(good_clock, failed_zones, calendars); + let (second_calendars, _) = calendars(false); + let mut now = TemporalNow::new(good_clock, failed_zones, second_calendars); assert_eq!( now.zoned_date_time_iso(Some("UTC".to_owned())), Err(NowError::TimeZone(ZoneFailure::Offset)) ); - let (good_clock, _) = clock(vec![Ok( - ClockSnapshot::new(0, 0).expect("valid snapshot"), - )]); - let (zones, _, _) = time_zones(Ok( - UtcOffsetNanoseconds::new(0).expect("valid offset"), - )); + let (good_clock, _) = clock(vec![Ok(ClockSnapshot::new(0).expect("valid snapshot"))]); + let (zones, _, _) = time_zones(Ok(UtcOffsetNanoseconds::new(0).expect("valid offset"))); let (failed_calendars, _) = calendars(true); let mut now = TemporalNow::new(good_clock, zones, failed_calendars); let projected = now @@ -1233,13 +1126,13 @@ mod tests { #[test] fn clock_clamping_and_boundary_zone_projection_are_exact() { assert_eq!( - ClockSnapshot::clamped(i128::MAX, 1) + ClockSnapshot::clamped(i128::MAX) .instant() .epoch_nanoseconds(), INSTANT_NS_MAX ); assert_eq!( - ClockSnapshot::clamped(i128::MIN, 2) + ClockSnapshot::clamped(i128::MIN) .instant() .epoch_nanoseconds(), INSTANT_NS_MIN diff --git a/crates/bamts-runtime/src/temporal/parse.rs b/crates/bamts-runtime/src/temporal/parse.rs new file mode 100644 index 00000000..9d256b93 --- /dev/null +++ b/crates/bamts-runtime/src/temporal/parse.rs @@ -0,0 +1,914 @@ +//! Strict RFC 9557 / Temporal ISO 8601 grammar. +//! +//! Each public entry point recognises exactly one goal symbol from the Temporal +//! grammar (`TemporalInstantString`, `TemporalDateTimeString`, ...). A string +//! either matches a goal completely or the parse fails; there is no partial +//! acceptance and no retry on a synthetic prefix. Every failure surfaces as a +//! `RangeError` at the JavaScript boundary. + +use super::instant_duration::{ + NS_PER_HOUR, NS_PER_MICROSECOND, NS_PER_MILLISECOND, NS_PER_MINUTE, NS_PER_SECOND, + TemporalError, TemporalResult, days_in_month, +}; +use super::zoned_date_time::CalendarId; + +/// UTC offset in a parsed string. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub enum UtcOffsetRecord { + /// `Z` / `z`. + Zulu, + /// Numeric offset in nanoseconds east of UTC. + Numeric { + nanoseconds: i128, + /// Whether the text carried a seconds component. Such an offset never + /// matches a time zone's offset by rounding to the minute. + sub_minute: bool, + }, +} + +/// Result of parsing an ISO date[-time] string with annotations. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ParsedIsoDateTime { + pub year: i32, + pub month: u8, + pub day: u8, + pub has_time: bool, + /// Nanoseconds since midnight; 0 when `has_time` is false. + pub time_ns: i128, + pub offset: Option, + /// Bracketed time zone annotation, unresolved. + pub time_zone: Option, + /// `[u-ca=…]` value if present. + pub calendar: Option, +} + +impl ParsedIsoDateTime { + /// The string's calendar, canonicalized; ISO when it carries none. + pub fn calendar_id(&self) -> TemporalResult { + self.calendar + .as_deref() + .map_or(Ok(CalendarId::iso8601()), CalendarId::canonicalize) + } +} + +/// The time zone a string names, before availability is decided. +#[derive(Debug, Clone, PartialEq, Eq)] +pub enum ParsedTimeZone { + Named(String), + OffsetMinutes(i32), +} + +struct Cursor<'a> { + bytes: &'a [u8], + pos: usize, +} + +impl<'a> Cursor<'a> { + fn new(text: &'a str) -> Self { + Cursor { + bytes: text.as_bytes(), + pos: 0, + } + } + + fn peek(&self) -> Option { + self.bytes.get(self.pos).copied() + } + + fn eat(&mut self, byte: u8) -> bool { + if self.peek() == Some(byte) { + self.pos += 1; + true + } else { + false + } + } + + fn eat_any(&mut self, set: &[u8]) -> bool { + match self.peek() { + Some(byte) if set.contains(&byte) => { + self.pos += 1; + true + } + _ => false, + } + } + + fn peek_digit(&self) -> bool { + self.peek().is_some_and(|byte| byte.is_ascii_digit()) + } + + fn digits(&mut self, count: usize) -> Option { + let mut value = 0_u32; + for _ in 0..count { + let byte = self.peek().filter(u8::is_ascii_digit)?; + value = value * 10 + u32::from(byte - b'0'); + self.pos += 1; + } + Some(value) + } + + fn done(&self) -> bool { + self.pos >= self.bytes.len() + } +} + +fn date_year(c: &mut Cursor<'_>) -> Option { + match c.peek() { + Some(sign @ (b'+' | b'-')) => { + c.pos += 1; + let value = i32::try_from(c.digits(6)?).ok()?; + if sign == b'-' { + // `-000000` is the one spelling of year zero the grammar forbids. + (value != 0).then_some(-value) + } else { + Some(value) + } + } + _ => i32::try_from(c.digits(4)?).ok(), + } +} + +fn date_month(c: &mut Cursor<'_>) -> Option { + let month = u8::try_from(c.digits(2)?).ok()?; + (1..=12).contains(&month).then_some(month) +} + +fn date_spec(c: &mut Cursor<'_>) -> Option<(i32, u8, u8)> { + let year = date_year(c)?; + let extended = c.eat(b'-'); + let month = date_month(c)?; + if extended && !c.eat(b'-') { + return None; + } + let day = u8::try_from(c.digits(2)?).ok()?; + (1..=days_in_month(year, month)) + .contains(&day) + .then_some((year, month, day)) +} + +fn year_month_spec(c: &mut Cursor<'_>) -> Option<(i32, u8)> { + let year = date_year(c)?; + c.eat(b'-'); + Some((year, date_month(c)?)) +} + +fn month_day_spec(c: &mut Cursor<'_>) -> Option<(u8, u8)> { + if c.eat(b'-') && !c.eat(b'-') { + return None; + } + let month = date_month(c)?; + c.eat(b'-'); + let day = u8::try_from(c.digits(2)?).ok()?; + // Validity is decided against a leap year: 02-29 is a real month-day. + (1..=days_in_month(2000, month)) + .contains(&day) + .then_some((month, day)) +} + +fn two_digits_at_most(c: &mut Cursor<'_>, max: u32) -> Option { + let value = c.digits(2)?; + (value <= max).then_some(value) +} + +/// `TemporalDecimalFraction`: returns nanoseconds, or `Some(0)` when absent. +fn fraction_nanoseconds(c: &mut Cursor<'_>) -> Option { + if !c.eat_any(b".,") { + return Some(0); + } + let mut value = 0_i128; + let mut count = 0; + while count < 9 && c.peek_digit() { + value = value * 10 + i128::from(c.digits(1)?); + count += 1; + } + if count == 0 { + return None; + } + for _ in count..9 { + value *= 10; + } + Some(value) +} + +/// `TimeSpec`, in either the extended or the basic form but never a mixture. +/// Leap seconds are accepted and clamp to `:59`, as the proposal requires. +fn time_spec(c: &mut Cursor<'_>) -> Option { + let hour = i128::from(two_digits_at_most(c, 23)?); + let mut ns = hour * NS_PER_HOUR; + let extended = c.eat(b':'); + if !extended && !c.peek_digit() { + return Some(ns); + } + ns += i128::from(two_digits_at_most(c, 59)?) * NS_PER_MINUTE; + if extended { + if !c.eat(b':') { + return Some(ns); + } + } else if !c.peek_digit() { + return Some(ns); + } + let second = two_digits_at_most(c, 60)?.min(59); + ns += i128::from(second) * NS_PER_SECOND; + Some(ns + fraction_nanoseconds(c)?) +} + +/// `UTCOffset`: minute precision, or seconds and a fraction when `sub_minute`. +fn utc_offset(c: &mut Cursor<'_>, sub_minute: bool) -> Option { + let negative = match c.peek()? { + b'+' => false, + b'-' => true, + _ => return None, + }; + c.pos += 1; + let mut ns = i128::from(two_digits_at_most(c, 23)?) * NS_PER_HOUR; + let extended = c.eat(b':'); + let mut has_seconds = false; + if extended || c.peek_digit() { + ns += i128::from(two_digits_at_most(c, 59)?) * NS_PER_MINUTE; + let wants_seconds = if extended { + sub_minute && c.eat(b':') + } else { + sub_minute && c.peek_digit() + }; + if wants_seconds { + ns += i128::from(two_digits_at_most(c, 59)?) * NS_PER_SECOND; + ns += fraction_nanoseconds(c)?; + has_seconds = true; + } + } + Some(UtcOffsetRecord::Numeric { + nanoseconds: if negative { -ns } else { ns }, + sub_minute: has_seconds, + }) +} + +/// `DateTimeUTCOffset[Z]`: an absent offset is `Some(None)`. +fn date_time_offset(c: &mut Cursor<'_>, allow_zulu: bool) -> Option> { + match c.peek() { + Some(b'Z' | b'z') if allow_zulu => { + c.pos += 1; + Some(Some(UtcOffsetRecord::Zulu)) + } + Some(b'+' | b'-') => utc_offset(c, true).map(Some), + _ => Some(None), + } +} + +fn is_iana_leading(byte: u8) -> bool { + byte.is_ascii_alphabetic() || matches!(byte, b'.' | b'_') +} + +fn is_iana_continuing(byte: u8) -> bool { + is_iana_leading(byte) || byte.is_ascii_digit() || matches!(byte, b'-' | b'+') +} + +/// `TimeZoneIANAName`: `/`-separated parts, none of which is `.` or `..`. +fn iana_name(c: &mut Cursor<'_>) -> Option { + let start = c.pos; + loop { + let part = c.pos; + if !c.peek().is_some_and(is_iana_leading) { + return None; + } + c.pos += 1; + while c.peek().is_some_and(is_iana_continuing) { + c.pos += 1; + } + if matches!(&c.bytes[part..c.pos], b"." | b"..") { + return None; + } + if !c.eat(b'/') { + break; + } + } + std::str::from_utf8(&c.bytes[start..c.pos]) + .ok() + .map(str::to_owned) +} + +/// `TimeZoneAnnotation`. A bracket that is not one leaves the cursor untouched +/// so the caller can read it as a key-value annotation. +fn time_zone_annotation(c: &mut Cursor<'_>) -> Option { + let start = c.pos; + let parsed = (|| { + if !c.eat(b'[') { + return None; + } + c.eat(b'!'); + let body = c.pos; + match c.peek()? { + b'+' | b'-' => { + utc_offset(c, false)?; + } + _ => { + iana_name(c)?; + } + } + let end = c.pos; + if !c.eat(b']') { + return None; + } + std::str::from_utf8(&c.bytes[body..end]) + .ok() + .map(str::to_owned) + })(); + if parsed.is_none() { + c.pos = start; + } + parsed +} + +fn annotation_key<'a>(c: &mut Cursor<'a>) -> Option<&'a [u8]> { + let start = c.pos; + if !c + .peek() + .is_some_and(|byte| byte.is_ascii_lowercase() || byte == b'_') + { + return None; + } + c.pos += 1; + while c.peek().is_some_and(|byte| { + byte.is_ascii_lowercase() || byte.is_ascii_digit() || matches!(byte, b'_' | b'-') + }) { + c.pos += 1; + } + Some(&c.bytes[start..c.pos]) +} + +fn annotation_value<'a>(c: &mut Cursor<'a>) -> Option<&'a [u8]> { + let start = c.pos; + loop { + let component = c.pos; + while c.peek().is_some_and(|byte| byte.is_ascii_alphanumeric()) { + c.pos += 1; + } + if c.pos == component { + return None; + } + if !c.eat(b'-') { + return Some(&c.bytes[start..c.pos]); + } + } +} + +/// `Annotations`, applying the proposal's calendar rules: the first `u-ca` wins, +/// a repeat is an error once a critical flag is involved, and an unknown key is +/// ignored unless it is critical. +fn annotations(c: &mut Cursor<'_>) -> TemporalResult> { + let mut calendar = None; + let mut calendar_was_critical = false; + while c.eat(b'[') { + let critical = c.eat(b'!'); + let parsed = (|| { + let key = annotation_key(c)?; + if !c.eat(b'=') { + return None; + } + let value = annotation_value(c)?; + c.eat(b']').then_some((key, value)) + })(); + let Some((key, value)) = parsed else { + return Err(TemporalError::syntax("malformed annotation")); + }; + if key == b"u-ca" { + if calendar.is_none() { + calendar = std::str::from_utf8(value).ok().map(str::to_owned); + calendar_was_critical = critical; + } else if critical || calendar_was_critical { + return Err(TemporalError::range( + "a repeated calendar annotation must not be critical", + )); + } + } else if critical { + return Err(TemporalError::range("unknown critical annotation")); + } + } + Ok(calendar) +} + +fn annotated_date_time( + text: &str, + zoned: bool, + time_required: bool, +) -> TemporalResult { + let syntax = || TemporalError::syntax("invalid ISO 8601 date-time string"); + let mut c = Cursor::new(text); + let (year, month, day) = date_spec(&mut c).ok_or_else(syntax)?; + let mut has_time = false; + let mut time_ns = 0; + let mut offset = None; + if c.eat_any(b"Tt ") { + has_time = true; + time_ns = time_spec(&mut c).ok_or_else(syntax)?; + offset = date_time_offset(&mut c, zoned).ok_or_else(syntax)?; + } else if time_required { + return Err(syntax()); + } + let time_zone = time_zone_annotation(&mut c); + if zoned && time_zone.is_none() { + return Err(syntax()); + } + let calendar = annotations(&mut c)?; + if !c.done() { + return Err(syntax()); + } + Ok(ParsedIsoDateTime { + year, + month, + day, + has_time, + time_ns, + offset, + time_zone, + calendar, + }) +} + +/// `TemporalDateTimeString[Zoned]`. +pub fn parse_date_time_string(text: &str, zoned: bool) -> TemporalResult { + annotated_date_time(text, zoned, false) +} + +/// `TemporalInstantString`: a time and an offset are both required. +pub fn parse_instant_string(text: &str) -> TemporalResult { + let syntax = || TemporalError::syntax("invalid Instant string"); + let mut c = Cursor::new(text); + let (year, month, day) = date_spec(&mut c).ok_or_else(syntax)?; + if !c.eat_any(b"Tt ") { + return Err(TemporalError::syntax( + "Instant string requires a time component", + )); + } + let time_ns = time_spec(&mut c).ok_or_else(syntax)?; + let offset = date_time_offset(&mut c, true).ok_or_else(syntax)?; + if offset.is_none() { + return Err(TemporalError::syntax( + "Instant string requires a UTC offset (Z or ±hh:mm)", + )); + } + let time_zone = time_zone_annotation(&mut c); + let calendar = annotations(&mut c)?; + if !c.done() { + return Err(syntax()); + } + Ok(ParsedIsoDateTime { + year, + month, + day, + has_time: true, + time_ns, + offset, + time_zone, + calendar, + }) +} + +fn parse_whole(text: &str, rule: impl FnOnce(&mut Cursor<'_>) -> bool) -> bool { + let mut c = Cursor::new(text); + rule(&mut c) && c.done() +} + +/// A time without the `T` designator must not also read as a month-day or a +/// year-month; those strings need the designator to mean a time. +fn is_ambiguous_time(text: &str) -> bool { + parse_whole(text, |c| month_day_spec(c).is_some()) + || parse_whole(text, |c| year_month_spec(c).is_some()) +} + +fn annotated_time(text: &str) -> TemporalResult { + let syntax = || TemporalError::syntax("invalid time string"); + let mut c = Cursor::new(text); + let designated = c.eat_any(b"Tt"); + let start = c.pos; + let time_ns = time_spec(&mut c).ok_or_else(syntax)?; + let offset = date_time_offset(&mut c, false).ok_or_else(syntax)?; + if !designated + && std::str::from_utf8(&c.bytes[start..c.pos]) + .ok() + .is_some_and(is_ambiguous_time) + { + return Err(TemporalError::syntax( + "time string needs a T designator to be unambiguous", + )); + } + let time_zone = time_zone_annotation(&mut c); + let calendar = annotations(&mut c)?; + if !c.done() { + return Err(syntax()); + } + Ok(ParsedIsoDateTime { + year: 0, + month: 1, + day: 1, + has_time: true, + time_ns, + offset, + time_zone, + calendar, + }) +} + +/// `TemporalTimeString`. +pub fn parse_time_string(text: &str) -> TemporalResult { + annotated_date_time(text, false, true).or_else(|_| annotated_time(text)) +} + +/// A bare year-month or month-day may only carry the ISO calendar: without a +/// day or year there is nothing a different calendar could interpret. +fn require_iso_annotation(calendar: &Option) -> TemporalResult<()> { + match calendar { + Some(value) if !value.eq_ignore_ascii_case("iso8601") => Err(TemporalError::range( + "a string without a full date must use the ISO calendar", + )), + _ => Ok(()), + } +} + +fn annotated_year_month(text: &str) -> TemporalResult { + let mut c = Cursor::new(text); + let (year, month) = + year_month_spec(&mut c).ok_or_else(|| TemporalError::syntax("invalid year-month"))?; + let time_zone = time_zone_annotation(&mut c); + let calendar = annotations(&mut c)?; + if !c.done() { + return Err(TemporalError::syntax("invalid year-month")); + } + require_iso_annotation(&calendar)?; + Ok(ParsedIsoDateTime { + year, + month, + day: 1, + has_time: false, + time_ns: 0, + offset: None, + time_zone, + calendar, + }) +} + +/// `TemporalYearMonthString`. +pub fn parse_year_month_string(text: &str) -> TemporalResult { + annotated_year_month(text).or_else(|bare| match annotated_date_time(text, false, false) { + Ok(full) => Ok(full), + Err(full) => Err(prefer_semantic_error(bare, full)), + }) +} + +fn annotated_month_day(text: &str) -> TemporalResult { + let mut c = Cursor::new(text); + let (month, day) = + month_day_spec(&mut c).ok_or_else(|| TemporalError::syntax("invalid month-day"))?; + let time_zone = time_zone_annotation(&mut c); + let calendar = annotations(&mut c)?; + if !c.done() { + return Err(TemporalError::syntax("invalid month-day")); + } + require_iso_annotation(&calendar)?; + Ok(ParsedIsoDateTime { + year: 1972, + month, + day, + has_time: false, + time_ns: 0, + offset: None, + time_zone, + calendar, + }) +} + +/// `TemporalMonthDayString`. +pub fn parse_month_day_string(text: &str) -> TemporalResult { + annotated_month_day(text).or_else(|bare| match annotated_date_time(text, false, false) { + Ok(full) => Ok(full), + Err(full) => Err(prefer_semantic_error(bare, full)), + }) +} + +fn prefer_semantic_error(first: TemporalError, second: TemporalError) -> TemporalError { + use super::instant_duration::TemporalErrorKind::Syntax; + if first.kind == Syntax { second } else { first } +} + +/// `ParseDateTimeUTCOffset`: the value of an `offset` property. +pub fn parse_date_time_utc_offset(text: &str) -> TemporalResult { + let mut c = Cursor::new(text); + match utc_offset(&mut c, true) { + Some(UtcOffsetRecord::Numeric { nanoseconds, .. }) if c.done() => Ok(nanoseconds), + _ => Err(TemporalError::syntax("invalid UTC offset string")), + } +} + +/// `ParseTimeZoneIdentifier`. +pub fn parse_time_zone_identifier(text: &str) -> TemporalResult { + let invalid = || TemporalError::range("invalid time zone identifier"); + let mut c = Cursor::new(text); + match c.peek() { + Some(b'+' | b'-') => match utc_offset(&mut c, false) { + Some(UtcOffsetRecord::Numeric { nanoseconds, .. }) if c.done() => { + let minutes = i32::try_from(nanoseconds / NS_PER_MINUTE).map_err(|_| invalid())?; + Ok(ParsedTimeZone::OffsetMinutes(minutes)) + } + _ => Err(invalid()), + }, + _ => match iana_name(&mut c) { + Some(name) if c.done() => Ok(ParsedTimeZone::Named(name)), + _ => Err(invalid()), + }, + } +} + +/// `ParseTemporalTimeZoneString`: an identifier, or any ISO string naming one +/// through its annotation, `Z`, or numeric offset, in that order of preference. +pub fn parse_time_zone_string(text: &str) -> TemporalResult { + if let Ok(identifier) = parse_time_zone_identifier(text) { + return Ok(identifier); + } + let parsed = parse_any_iso_string(text)?; + if let Some(annotation) = &parsed.time_zone { + return parse_time_zone_identifier(annotation); + } + match parsed.offset { + Some(UtcOffsetRecord::Zulu) => Ok(ParsedTimeZone::Named("UTC".to_owned())), + Some(UtcOffsetRecord::Numeric { + nanoseconds, + sub_minute: false, + }) => { + let minutes = i32::try_from(nanoseconds / NS_PER_MINUTE) + .map_err(|_| TemporalError::range("invalid UTC offset"))?; + Ok(ParsedTimeZone::OffsetMinutes(minutes)) + } + _ => Err(TemporalError::range( + "string does not name a time zone with minute precision", + )), + } +} + +/// Every goal `ParseTemporalTimeZoneString` and `ParseTemporalCalendarString` +/// try, in the proposal's order. +fn parse_any_iso_string(text: &str) -> TemporalResult { + annotated_date_time(text, true, false) + .or_else(|_| annotated_date_time(text, false, false)) + .or_else(|_| parse_instant_string(text)) + .or_else(|_| parse_time_string(text)) + .or_else(|_| parse_month_day_string(text)) + .or_else(|_| parse_year_month_string(text)) +} + +/// `ParseTemporalCalendarString`: the annotation of an ISO string, `iso8601` +/// for one without an annotation, or a bare calendar identifier. +pub fn parse_calendar_string(text: &str) -> TemporalResult { + if let Ok(parsed) = parse_any_iso_string(text) { + return Ok(parsed.calendar.unwrap_or_else(|| "iso8601".to_owned())); + } + let mut c = Cursor::new(text); + match annotation_value(&mut c) { + Some(_) if c.done() => Ok(text.to_owned()), + _ => Err(TemporalError::syntax("invalid calendar string")), + } +} + +/// Exact nanoseconds in `whole.fraction` of one `unit_ns`, where `fraction` is +/// scaled to nine digits. +fn fractional_nanoseconds(fraction: i128, unit_ns: i128) -> i128 { + fraction * (unit_ns / NS_PER_SECOND) +} + +/// Splits sub-unit nanoseconds into the Duration fields below `unit_ns`. +fn distribute_fraction(mut rest: i128, unit_ns: i128) -> [i128; 5] { + let mut fields = [0; 5]; + let steps = [ + (0, NS_PER_MINUTE), + (1, NS_PER_SECOND), + (2, NS_PER_MILLISECOND), + (3, NS_PER_MICROSECOND), + ]; + for (slot, size) in steps { + if size < unit_ns { + fields[slot] = rest / size; + rest %= size; + } + } + fields[4] = rest; + fields +} + +/// `ParseTemporalDurationString`. Returns the ten fields, signed. Only the +/// smallest time component may carry a fraction, and it spills downward into +/// the smaller fields exactly. +pub fn parse_duration_string(text: &str) -> TemporalResult<[i128; 10]> { + let syntax = || TemporalError::syntax("invalid duration string"); + let overflow = || TemporalError::range("duration component out of range"); + let mut c = Cursor::new(text); + let negative = match c.peek() { + Some(b'-') => { + c.pos += 1; + true + } + Some(b'+') => { + c.pos += 1; + false + } + _ => false, + }; + if !c.eat_any(b"Pp") { + return Err(syntax()); + } + let mut fields = [0_i128; 10]; + let mut any = false; + + let whole_number = |c: &mut Cursor<'_>| -> TemporalResult { + let start = c.pos; + let mut value = 0_i128; + while c.peek_digit() { + let digit = i128::from(c.digits(1).ok_or_else(syntax)?); + value = value + .checked_mul(10) + .and_then(|scaled| scaled.checked_add(digit)) + .ok_or_else(overflow)?; + } + if c.pos == start { + return Err(syntax()); + } + Ok(value) + }; + + let mut next_date_slot = 0; + while !c.done() && !matches!(c.peek(), Some(b'T' | b't')) { + let value = whole_number(&mut c)?; + let slot = match c.peek().map(|byte| byte.to_ascii_uppercase()) { + Some(b'Y') => 0, + Some(b'M') => 1, + Some(b'W') => 2, + Some(b'D') => 3, + _ => return Err(syntax()), + }; + c.pos += 1; + if slot < next_date_slot { + return Err(syntax()); + } + next_date_slot = slot + 1; + fields[slot] = value; + any = true; + } + + if c.eat_any(b"Tt") { + let mut next_time_slot = 0; + let mut any_time = false; + while !c.done() { + let value = whole_number(&mut c)?; + let before_fraction = c.pos; + let fraction = fraction_nanoseconds(&mut c).ok_or_else(syntax)?; + let has_fraction = c.pos != before_fraction; + let (slot, unit_ns) = match c.peek().map(|byte| byte.to_ascii_uppercase()) { + Some(b'H') => (4, NS_PER_HOUR), + Some(b'M') => (5, NS_PER_MINUTE), + Some(b'S') => (6, NS_PER_SECOND), + _ => return Err(syntax()), + }; + c.pos += 1; + if slot < next_time_slot { + return Err(syntax()); + } + next_time_slot = slot + 1; + fields[slot] = value; + any = true; + any_time = true; + if has_fraction { + let [minutes, seconds, milliseconds, microseconds, nanoseconds] = + distribute_fraction(fractional_nanoseconds(fraction, unit_ns), unit_ns); + if slot == 4 { + fields[5] = minutes; + } + if slot <= 5 { + fields[6] = seconds; + } + fields[7] = milliseconds; + fields[8] = microseconds; + fields[9] = nanoseconds; + if !c.done() { + return Err(TemporalError::syntax( + "no component may follow a fractional component", + )); + } + } + } + if !any_time { + return Err(syntax()); + } + } + + if !any || !c.done() { + return Err(syntax()); + } + if negative { + for field in &mut fields { + *field = -*field; + } + } + Ok(fields) +} + +#[cfg(test)] +mod tests { + use super::*; + + #[test] + fn date_time_goal_rejects_a_utc_designator_unless_zoned() { + assert!(parse_date_time_string("2020-01-01T00:00Z", false).is_err()); + assert!(parse_date_time_string("2020-01-01T00:00Z[UTC]", true).is_ok()); + assert!(parse_date_time_string("2020-01-01T00:00+01:00", false).is_ok()); + assert!(parse_date_time_string("2020-01-01T00:00", true).is_err()); + } + + #[test] + fn year_zero_has_exactly_one_negative_spelling_forbidden() { + assert!(parse_date_time_string("-000000-01-01", false).is_err()); + assert!(parse_date_time_string("+000000-01-01", false).is_ok()); + assert!(parse_date_time_string("0000-01-01", false).is_ok()); + } + + #[test] + fn separators_may_not_mix_between_extended_and_basic_forms() { + assert!(parse_date_time_string("2020-0101", false).is_err()); + assert!(parse_date_time_string("202001-01", false).is_err()); + assert!(parse_date_time_string("20200101T121314", false).is_ok()); + assert!(parse_date_time_string("2020-01-01T12:1314", false).is_err()); + assert!(parse_date_time_string("2020-01-01T1213:14", false).is_err()); + } + + #[test] + fn leap_second_clamps_and_fraction_stops_at_nine_digits() { + let parsed = parse_date_time_string("2016-12-31T23:59:60", false).unwrap(); + assert_eq!(parsed.time_ns % NS_PER_MINUTE, 59 * NS_PER_SECOND); + assert!(parse_date_time_string("2020-01-01T00:00:00.123456789", false).is_ok()); + assert!(parse_date_time_string("2020-01-01T00:00:00.1234567890", false).is_err()); + assert!(parse_date_time_string("2020-01-01T00:00:61", false).is_err()); + assert!(parse_date_time_string("2020-01-01T24:00", false).is_err()); + } + + #[test] + fn calendar_annotations_follow_first_wins_and_critical_rules() { + let first = "2020-01-01[u-ca=iso8601][u-ca=hebrew]"; + assert_eq!( + parse_date_time_string(first, false).unwrap().calendar, + Some("iso8601".to_owned()) + ); + assert!(parse_date_time_string("2020-01-01[!u-ca=iso8601][u-ca=hebrew]", false).is_err()); + assert!(parse_date_time_string("2020-01-01[u-ca=iso8601][!u-ca=hebrew]", false).is_err()); + assert!(parse_date_time_string("2020-01-01[foo=bar]", false).is_ok()); + assert!(parse_date_time_string("2020-01-01[!foo=bar]", false).is_err()); + assert!(parse_date_time_string("2020-01-01[FOO=bar]", false).is_err()); + } + + #[test] + fn time_zone_annotation_must_lead_and_be_unique() { + assert!(parse_date_time_string("2020-01-01[UTC][UTC]", false).is_err()); + assert!(parse_date_time_string("2020-01-01[u-ca=iso8601][UTC]", false).is_err()); + assert!(parse_date_time_string("2020-01-01[+01:00:30]", false).is_err()); + assert_eq!( + parse_date_time_string("2020-01-01[!Foo/Bar_Baz-1]", false) + .unwrap() + .time_zone, + Some("Foo/Bar_Baz-1".to_owned()) + ); + } + + #[test] + fn bare_times_that_read_as_dates_need_the_designator() { + for ambiguous in ["2021-12", "1214", "0229", "1130", "12-14", "202112"] { + assert!(parse_time_string(ambiguous).is_err(), "{ambiguous}"); + assert!( + parse_time_string(&format!("T{ambiguous}")).is_ok(), + "T{ambiguous}" + ); + } + for unambiguous in ["2021-13", "1314", "0230", "1131", "12-15:00"] { + assert!(parse_time_string(unambiguous).is_ok(), "{unambiguous}"); + } + } + + #[test] + fn instant_goal_requires_both_time_and_offset() { + assert!(parse_instant_string("2020-01-01T00:00Z").is_ok()); + assert!(parse_instant_string("2020-01-01T00:00+01:00:30.5").is_ok()); + assert!(parse_instant_string("2020-01-01T00:00").is_err()); + assert!(parse_instant_string("2020-01-01Z").is_err()); + } + + #[test] + fn duration_fractions_spill_exactly_and_end_the_string() { + let fields = parse_duration_string("PT1.5H").unwrap(); + assert_eq!(fields, [0, 0, 0, 0, 1, 30, 0, 0, 0, 0]); + let fields = parse_duration_string("-PT0.000000001H").unwrap(); + assert_eq!(fields, [0, 0, 0, 0, 0, 0, 0, 0, -3, -600]); + let fields = parse_duration_string("PT1.123456789S").unwrap(); + assert_eq!(fields, [0, 0, 0, 0, 0, 0, 1, 123, 456, 789]); + assert!(parse_duration_string("PT1.5H1M").is_err()); + assert!(parse_duration_string("P1.5Y").is_err()); + assert!(parse_duration_string("P").is_err()); + assert!(parse_duration_string("PT").is_err()); + assert!(parse_duration_string("P1YT").is_err()); + assert!(parse_duration_string("P1M1Y").is_err()); + } +} diff --git a/crates/bamts-runtime/src/temporal/plain_types.rs b/crates/bamts-runtime/src/temporal/plain_types.rs index 69309a12..da63e07f 100644 --- a/crates/bamts-runtime/src/temporal/plain_types.rs +++ b/crates/bamts-runtime/src/temporal/plain_types.rs @@ -3,16 +3,24 @@ //! //! All arithmetic is exact checked integer work on proleptic-Gregorian //! rata-die days plus nanoseconds-of-day; no floats, no ambient host access. -//! Zoned/time-zone operations are excluded — that is the jiff C11.3 boundary. use std::fmt; use super::instant_duration::{ - days_in_month, epoch_days_from_ymd, format_iso_date, format_time_ns, is_leap_year, - parse_iso_datetime_string, round_to_increment, validate_rounding_increment, ymd_from_epoch_days, - Duration, Instant, Overflow, Precision, RoundingMode, TemporalError, TemporalResult, Unit, - INSTANT_NS_MAX, NS_PER_DAY, NS_PER_HOUR, NS_PER_MICROSECOND, NS_PER_MILLISECOND, - NS_PER_MINUTE, NS_PER_SECOND, + Duration, INSTANT_NS_MAX, NS_PER_DAY, NS_PER_HOUR, NS_PER_MICROSECOND, NS_PER_MILLISECOND, + NS_PER_MINUTE, NS_PER_SECOND, Overflow, Precision, RoundingMode, TemporalError, TemporalResult, + Unit, UnitCategory, days_in_month, days_in_year, epoch_days_from_ymd, format_iso_date, + format_time_ns, is_leap_year, round_to_increment, validate_rounding_increment, + ymd_from_epoch_days, +}; +use super::parse::{ParsedIsoDateTime, parse_time_string}; +use super::relative::{ + DifferenceRounding, InternalDuration, difference_plain_date, + difference_plain_date_time_with_rounding, difference_plain_year_month, difference_time, + round_time_duration, temporal_duration_from_internal, +}; +use super::zoned_date_time::{ + CalendarId, CalendarProvider, FixedOffsetTimeZones, IsoCalendarProvider, TemporalProviders, }; // --------------------------------------------------------------------------- @@ -46,7 +54,12 @@ fn reject_iso_date(year: i32, month: u8, day: u8) -> TemporalResult<()> { } /// RegulateISODate: apply overflow policy to possibly-out-of-range fields. -pub fn regulate_iso_date(year: i32, month: i64, day: i64, overflow: Overflow) -> TemporalResult<(i32, u8, u8)> { +pub fn regulate_iso_date( + year: i32, + month: i64, + day: i64, + overflow: Overflow, +) -> TemporalResult<(i32, u8, u8)> { match overflow { Overflow::Reject => { let month = u8::try_from(month) @@ -87,9 +100,8 @@ pub fn iso_week_of_year(year: i32, month: u8, day: u8) -> (i32, u8) { let dow = i64::from(iso_day_of_week(year, month, day)); let thursday = days + 4 - dow; let (week_year, _, _) = ymd_from_epoch_days(thursday); - let first_thursday = epoch_days_from_ymd(week_year, 1, 4) - - (iso_day_of_week(week_year, 1, 4) as i64) - + 3; + let first_thursday = + epoch_days_from_ymd(week_year, 1, 4) - (iso_day_of_week(week_year, 1, 4) as i64) + 3; let week = (thursday - first_thursday) / 7 + 1; (week_year, week as u8) } @@ -167,7 +179,7 @@ impl PlainTime { } /// Nanoseconds since midnight. - pub fn to_nanoseconds_of_day(&self) -> i128 { + pub fn to_nanoseconds_of_day(self) -> i128 { self.hour as i128 * NS_PER_HOUR + self.minute as i128 * NS_PER_MINUTE + self.second as i128 * NS_PER_SECOND @@ -176,7 +188,7 @@ impl PlainTime { + self.nanosecond as i128 } - fn from_nanoseconds_of_day(ns: i128) -> PlainTime { + pub fn from_nanoseconds_of_day(ns: i128) -> PlainTime { debug_assert!((0..NS_PER_DAY).contains(&ns)); let hour = ns / NS_PER_HOUR; let minute = (ns / NS_PER_MINUTE) % 60; @@ -196,19 +208,19 @@ impl PlainTime { pub fn add_signed_nanoseconds(&self, delta_ns: i128) -> (i128, PlainTime) { let total = self.to_nanoseconds_of_day() + delta_ns; let carry = total.div_euclid(NS_PER_DAY); - (carry, PlainTime::from_nanoseconds_of_day(total.rem_euclid(NS_PER_DAY))) + ( + carry, + PlainTime::from_nanoseconds_of_day(total.rem_euclid(NS_PER_DAY)), + ) } - pub fn add(&self, duration: &Duration) -> TemporalResult { - if duration.years != 0 || duration.months != 0 || duration.weeks != 0 || duration.days != 0 { - return Err(TemporalError::range( - "PlainTime arithmetic does not accept date units", - )); - } - Ok(self.add_signed_nanoseconds(duration.time_total_nanoseconds()).1) + /// AddDurationToTime: only the time portion moves a wall-clock time. + pub fn add(&self, duration: &Duration) -> PlainTime { + self.add_signed_nanoseconds(duration.time_total_nanoseconds()) + .1 } - pub fn subtract(&self, duration: &Duration) -> TemporalResult { + pub fn subtract(&self, duration: &Duration) -> PlainTime { self.add(&duration.negated()) } @@ -221,40 +233,42 @@ impl PlainTime { increment: i128, mode: RoundingMode, ) -> TemporalResult { - if largest_unit.category() == super::instant_duration::UnitCategory::Date - || smallest_unit.category() == super::instant_duration::UnitCategory::Date + if largest_unit.category() == UnitCategory::Date + || smallest_unit.category() == UnitCategory::Date { return Err(TemporalError::range( "PlainTime difference supports time units only", )); } if smallest_unit < largest_unit { - return Err(TemporalError::range("smallestUnit must not be larger than largestUnit")); + return Err(TemporalError::range( + "smallestUnit must not be larger than largestUnit", + )); } validate_rounding_increment(smallest_unit, increment, false)?; - let diff = other.to_nanoseconds_of_day() - self.to_nanoseconds_of_day(); - let unit_ns = smallest_unit.ns_per().ok_or(TemporalErrorKindRange)?; - let step = unit_ns.checked_mul(increment).ok_or(TemporalErrorKindRange)?; - let rounded = round_to_increment(diff, step, mode)?; - Duration::from_time_nanoseconds(rounded, largest_unit.max(Unit::Hour)) + let time = round_time_duration( + difference_time(*self, other), + increment, + smallest_unit, + mode, + )?; + temporal_duration_from_internal( + InternalDuration::new(Duration::default(), time), + largest_unit, + ) } - pub fn since( + /// RoundTime on the whole day's nanoseconds. Every valid increment divides + /// the next larger unit, so this agrees with rounding that unit's portion. + pub fn round( &self, - other: PlainTime, - largest_unit: Unit, - smallest_unit: Unit, + unit: Unit, increment: i128, mode: RoundingMode, - ) -> TemporalResult { - other.until(*self, largest_unit, smallest_unit, increment, mode) - } - - pub fn round(&self, unit: Unit, increment: i128, mode: RoundingMode) -> TemporalResult { - let unit_ns = match unit { - Unit::Day => NS_PER_DAY, - _ => unit.ns_per().ok_or(TemporalErrorKindRange)?, - }; + ) -> TemporalResult { + let unit_ns = unit + .ns_per() + .ok_or_else(|| TemporalError::range("rounding a time requires a time unit"))?; if increment < 1 { return Err(TemporalError::range("roundingIncrement must be >= 1")); } @@ -266,72 +280,15 @@ impl PlainTime { ))); } let rounded = round_to_increment(self.to_nanoseconds_of_day(), unit_ns * increment, mode)?; - Ok(PlainTime::from_nanoseconds_of_day(rounded.rem_euclid(NS_PER_DAY))) - } - - /// `withTime`-style field replacement; `None` keeps the current value. - pub fn with( - &self, - hour: Option, - minute: Option, - second: Option, - millisecond: Option, - microsecond: Option, - nanosecond: Option, - overflow: Overflow, - ) -> TemporalResult { - PlainTime::new( - hour.unwrap_or(self.hour.into()), - minute.unwrap_or(self.minute.into()), - second.unwrap_or(self.second.into()), - millisecond.unwrap_or(self.millisecond.into()), - microsecond.unwrap_or(self.microsecond.into()), - nanosecond.unwrap_or(self.nanosecond.into()), - overflow, - ) + Ok(PlainTime::from_nanoseconds_of_day( + rounded.rem_euclid(NS_PER_DAY), + )) } - /// ParseTemporalTimeString. Accepts bare `T?HH:MM[:SS[.f{1,9}]]` (no - /// offset/annotations allowed on a bare time) or a full date-time from - /// which the time portion is taken. + /// ParseTemporalTimeString. pub fn parse(text: &str) -> TemporalResult { - let trimmed = text.trim_start_matches(|c| c == 'T' || c == 't').trim_start(); - // Full date-time form: date component first. - if trimmed.len() > 2 && (trimmed.as_bytes()[4.min(trimmed.len() - 1)] == b'-' - || trimmed.starts_with(['+', '-']) - || trimmed.chars().take(4).all(|ch| ch.is_ascii_digit())) - { - if let Ok(full) = parse_iso_datetime_string(text) { - if !full.has_time { - return Err(TemporalError::syntax("PlainTime string requires a time component")); - } - return PlainTime::from_nanoseconds_of_day(full.time_ns).into_ok(); - } - } - match parse_iso_datetime_string(text) { - Ok(full) => { - if !full.has_time { - Err(TemporalError::syntax("PlainTime string requires a time component")) - } else { - Ok(PlainTime::from_nanoseconds_of_day(full.time_ns)) - } - } - Err(_date_err) => { - // Bare time form, possibly with a leading T. - let bare = if !trimmed.is_empty() && (text.as_bytes().first() == Some(&b'T') - || text.as_bytes().first() == Some(&b't')) - { - &text[1..] - } else { - text - }; - parse_bare_time_string(bare) - } - } - } - - fn into_ok(self) -> TemporalResult { - Ok(self) + let parsed = parse_time_string(text)?; + Ok(PlainTime::from_nanoseconds_of_day(parsed.time_ns)) } /// TemporalTimeToString. @@ -346,30 +303,6 @@ impl fmt::Display for PlainTime { } } -fn parse_bare_time_string(text: &str) -> TemporalResult { - // Reuse the datetime grammar on a synthetic epoch date; reject offsets - // and annotations that would leak zone semantics into a plain time. - let synthetic = format!("2000-01-01T{text}"); - let full = parse_iso_datetime_string(&synthetic) - .map_err(|e| TemporalError::syntax(format!("invalid time string: {}", e.message)))?; - if full.offset.is_some() || full.time_zone.is_some() || full.calendar.is_some() { - return Err(TemporalError::range( - "PlainTime string must not contain a UTC offset, time zone, or calendar", - )); - } - Ok(PlainTime::from_nanoseconds_of_day(full.time_ns)) -} - -const TemporalErrorKindRange: TemporalError = TemporalError::const_range(); - -impl TemporalError { - const fn const_range() -> TemporalError { - TemporalError { kind: TemporalErrorKind::Range, message: String::new() } - } -} - -use super::instant_duration::TemporalErrorKind; - // --------------------------------------------------------------------------- // PlainDate // --------------------------------------------------------------------------- @@ -392,10 +325,14 @@ impl PlainDate { format_iso_date(y, m, d) ))); } - Ok(PlainDate { year: y, month: m, day: d }) + Ok(PlainDate { + year: y, + month: m, + day: d, + }) } - fn from_validated(year: i32, month: u8, day: u8) -> TemporalResult { + pub fn from_validated(year: i32, month: u8, day: u8) -> TemporalResult { reject_iso_date(year, month, day)?; if !iso_date_within_limits(year, month, day) { return Err(TemporalError::range("date outside ISO representable range")); @@ -403,11 +340,11 @@ impl PlainDate { Ok(PlainDate { year, month, day }) } - fn epoch_days(&self) -> i64 { + pub fn epoch_days(&self) -> i64 { epoch_days_from_ymd(self.year, self.month, self.day) } - fn from_epoch_days(days: i64) -> TemporalResult { + pub fn from_epoch_days(days: i64) -> TemporalResult { let (y, m, d) = ymd_from_epoch_days(days); PlainDate::from_validated(y, m, d) } @@ -429,7 +366,7 @@ impl PlainDate { } pub fn days_in_year(&self) -> u16 { - if is_leap_year(self.year) { 366 } else { 365 } + days_in_year(self.year) } pub fn months_in_year(&self) -> u8 { @@ -465,10 +402,10 @@ impl PlainDate { Ok(d) } - pub fn subtract(&self, duration: &Duration, overflow: Overflow) -> TemporalResult { - self.add(&duration.negated(), overflow) - } - + /// CalendarDateAdd for the ISO calendar. Years and months move first and + /// land under `overflow`, then weeks and days move on the day line, so the + /// intermediate year may lie far outside the representable range as long as + /// the final date does not. fn add_iso_date(&self, duration: &Duration, overflow: Overflow) -> TemporalResult { if duration.hours != 0 || duration.minutes != 0 @@ -481,32 +418,34 @@ impl PlainDate { "PlainDate arithmetic does not accept time units", )); } - // 1. Add years and months by constraining into the target month. - let total_months = i64::from(self.year) * 12 + i64::from(self.month) - 1 + let out_of_range = || TemporalError::range("date arithmetic outside the supported range"); + let total_months = i128::from(self.year) * 12 + i128::from(self.month) - 1 + duration.years * 12 + duration.months; - let new_year_i = total_months.div_euclid(12); - let new_month_i = total_months.rem_euclid(12) + 1; - let new_year = i32::try_from(new_year_i) - .map_err(|_| TemporalError::range("year outside i32 range after arithmetic"))?; - let (y, m, d) = regulate_iso_date( - new_year, - new_month_i, - i64::from(self.day), - overflow, - )?; - // 2. Add weeks and days on the day timeline (exact). - let days = epoch_days_from_ymd(y, m, d) - .checked_add((duration.weeks as i64).checked_mul(7).and_then(|w| w.checked_add(duration.days)).ok_or_else(|| { - TemporalError::range("Duration week/day arithmetic overflow") - })?) - .ok_or_else(|| TemporalError::range("date arithmetic overflow"))?; - PlainDate::from_epoch_days(days) - } - - /// DifferenceISODate: exact years/months/weeks/days (no rounding beyond - /// integer truncation of the largest unit pair), rounded under `mode` to - /// `smallest_unit` when the difference has a remainder in a smaller unit. + let year = i64::try_from(total_months.div_euclid(12)).map_err(|_| out_of_range())?; + let month = u8::try_from(total_months.rem_euclid(12) + 1).map_err(|_| out_of_range())?; + let longest = i64::from(days_in_unbounded_month(year, month)); + let day = match overflow { + Overflow::Constrain => self.day.min(longest as u8), + Overflow::Reject if i64::from(self.day) > longest => { + return Err(TemporalError::range(format!( + "day {} out of range for the resulting month", + self.day + ))); + } + Overflow::Reject => self.day, + }; + let shift = duration.weeks * 7 + duration.days; + let days = unbounded_epoch_days(year, month, day) + shift; + // ±10^8 days bound every representable date, so anything past this + // cannot be a date and must not reach the i32 year conversion. + if days.abs() > 200_000_000 { + return Err(out_of_range()); + } + PlainDate::from_epoch_days(days as i64) + } + + /// DifferenceTemporalPlainDate with `until` direction. pub fn until( &self, other: PlainDate, @@ -514,84 +453,65 @@ impl PlainDate { smallest_unit: Unit, increment: i128, mode: RoundingMode, - overflow: Overflow, ) -> TemporalResult { - if !matches!(largest_unit, Unit::Year | Unit::Month | Unit::Week | Unit::Day) - || !matches!(smallest_unit, Unit::Year | Unit::Month | Unit::Week | Unit::Day) - { + if !matches!( + largest_unit, + Unit::Year | Unit::Month | Unit::Week | Unit::Day + ) || !matches!( + smallest_unit, + Unit::Year | Unit::Month | Unit::Week | Unit::Day + ) { return Err(TemporalError::range( "PlainDate difference units must be year..day", )); } if smallest_unit < largest_unit { - return Err(TemporalError::range("smallestUnit must not be larger than largestUnit")); + return Err(TemporalError::range( + "smallestUnit must not be larger than largestUnit", + )); } if increment < 1 { return Err(TemporalError::range("roundingIncrement must be >= 1")); } - let sign_flip = self > &other; - let (a, b) = if sign_flip { (other, *self) } else { (*self, other) }; - let whole = difference_iso_date_whole(&a, &b, largest_unit)?; - let mut result = whole; - // Round into smallest/increment if the residual day remainder exists. - if smallest_unit != largest_unit || increment != 1 { - if smallest_unit == Unit::Day { - // Rounding in days: recompute whole-largest part then add rounded days. - let mut days_only = whole; - days_only.weeks = 0; - days_only.days = 0; - let rebased = a.add_iso_date(&days_only, Overflow::Constrain)?; - let remainder_days = b.epoch_days() - rebased.epoch_days(); - let rounded_days = round_to_increment(i128::from(remainder_days), increment, mode)?; - let mut rounded = days_only; - rounded.days = days_only.days.checked_add(i64::try_from(rounded_days).map_err(|_| TemporalError::range("overflow"))?).ok_or_else(|| TemporalError::range("overflow"))?; - // Balance day remainder into weeks when largest == week. - if largest_unit == Unit::Week { - let weeks = rounded.days / 7; - rounded.days -= weeks * 7; - let weeks = rounded.weeks.checked_add(weeks).ok_or_else(|| TemporalError::range("overflow"))?; - rounded.weeks = weeks; - } - result = rounded; - } else { - // Calendar rounding beyond day granularity requires relativeTo - // semantics handled here at whole-unit precision: truncate. - } - } - if sign_flip { - result = result.negated(); - } - Ok(result) - } - - pub fn since( - &self, - other: PlainDate, - largest_unit: Unit, - smallest_unit: Unit, - increment: i128, - mode: RoundingMode, - overflow: Overflow, - ) -> TemporalResult { - other.until(*self, largest_unit, smallest_unit, increment, mode, overflow) + difference_plain_date( + &iso_providers(), + *self, + other, + &CalendarId::iso8601(), + DifferenceRounding { + largest_unit, + smallest_unit, + increment, + mode, + }, + ) } - pub fn to_plain_date_time(&self, time: PlainTime) -> TemporalResult { - PlainDateTime::from_parts(*self, time) + pub fn to_plain_date_time(self, time: PlainTime) -> TemporalResult { + PlainDateTime::from_parts(self, time) } - pub fn to_plain_year_month(&self) -> PlainYearMonth { - PlainYearMonth { year: self.year, month: self.month, reference_day: 1 } + pub fn to_plain_year_month(self) -> PlainYearMonth { + PlainYearMonth { + year: self.year, + month: self.month, + reference_day: 1, + } } - pub fn to_plain_month_day(&self) -> PlainMonthDay { - PlainMonthDay { year: self.year, month: self.month, day: self.day } + pub fn to_plain_month_day(self) -> PlainMonthDay { + PlainMonthDay { + year: MONTH_DAY_REFERENCE_YEAR, + month: self.month, + day: self.day, + } } - /// ParseTemporalDateString. - pub fn parse(text: &str) -> TemporalResult { - let full = parse_iso_datetime_string(text)?; - PlainDate::from_validated(full.year, full.month, full.day) + /// The date a parsed string names, checked against the limits. The + /// proposal reads options between the parse and this step, so the two stay + /// separate. + pub fn from_parsed(parsed: &ParsedIsoDateTime) -> TemporalResult { + PlainDate::from_validated(parsed.year, parsed.month, parsed.day) } /// TemporalDateToString ([u-ca=iso8601] shown only in showCriticalAlways @@ -607,85 +527,23 @@ impl fmt::Display for PlainDate { } } -/// Whole-unit date difference used by `PlainDate::until`: computes the -/// largest number of `largest_unit` steps from `start` that does not pass -/// `end`, then the residual days/weeks below that unit. -fn difference_iso_date_whole( - start: &PlainDate, - end: &PlainDate, - largest_unit: Unit, -) -> TemporalResult { - if start == end { - return Ok(Duration::default()); - } - let end_days = end.epoch_days(); - match largest_unit { - Unit::Year | Unit::Month => { - // Greedy forward fill: candidate = start + k units. - // Bounds on iteration: total month span + 1. - let months_total = (i64::from(end.year) - i64::from(start.year)) * 12 - + (i64::from(end.month) - i64::from(start.month)); - let steps = match largest_unit { - Unit::Year => months_total.div_euclid(12), - _ => months_total, - }; - let mut best: i64 = 0; - let mut best_date = *start; - // Binary search in [0, steps+1] for the largest fitting k. - let mut lo: i64 = 0; - let mut hi: i64 = steps.saturating_add(2); - while lo <= hi { - let mid = lo.midpoint(hi); - let candidate = if largest_unit == Unit::Year { - Duration::new(mid, 0, 0, 0, 0, 0, 0, 0, 0, 0)? - } else { - Duration::new(0, mid, 0, 0, 0, 0, 0, 0, 0, 0)? - }; - match start.add_iso_date(&candidate, Overflow::Constrain) { - Ok(date) if date <= *end => { - best = mid; - best_date = date; - lo = mid + 1; - } - _ => hi = mid - 1, - } - } - let remainder_days = end_days - best_date.epoch_days(); - // Decompose remainder into months+days if largest==year? Spec - // balancing: difference with largestUnit year yields years+months+days. - match largest_unit { - Unit::Year => { - let extra_months = (i64::from(end.year) * 12 + i64::from(end.month)) - - (i64::from(best_date.year) * 12 + i64::from(best_date.month)); - // Keep months in [0, 11]: binary search a fitting month count. - let mut m_lo = 0_i64; - let mut m_hi = extra_months.max(0); - let mut m_best = 0_i64; - let mut m_date = best_date; - while m_lo <= m_hi { - let mid = m_lo.midpoint(m_hi); - let candidate = Duration::new(0, mid, 0, 0, 0, 0, 0, 0, 0, 0)?; - match best_date.add_iso_date(&candidate, Overflow::Constrain) { - Ok(date) if date <= *end => { - m_best = mid; - m_date = date; - m_lo = mid + 1; - } - _ => m_hi = mid - 1, - } - } - let days_left = end_days - m_date.epoch_days(); - Duration::new(best, m_best, 0, days_left, 0, 0, 0, 0, 0, 0) - } - _ => Duration::new(0, best, 0, remainder_days, 0, 0, 0, 0, 0, 0), - } - } - Unit::Week => { - let days = end_days - start.epoch_days(); - Duration::new(0, 0, days / 7, days % 7, 0, 0, 0, 0, 0, 0) - } - _ => Duration::new(0, 0, 0, end_days - start.epoch_days(), 0, 0, 0, 0, 0, 0), - } +/// Days in a month of a year that may lie far outside the representable range. +fn days_in_unbounded_month(year: i64, month: u8) -> u8 { + days_in_month(((year % 400) + 400) as i32 + 2000, month) +} + +/// ISODateToEpochDays for a year that may lie far outside the representable +/// range: whole 400-year cycles move on the day line first. +fn unbounded_epoch_days(year: i64, month: u8, day: u8) -> i128 { + const DAYS_PER_CYCLE: i128 = 146_097; + let cycles = year.div_euclid(400); + let base = year.rem_euclid(400) as i32; + i128::from(cycles) * DAYS_PER_CYCLE + i128::from(epoch_days_from_ymd(base, month, day)) +} + +/// The ISO-only providers behind the plain types' arithmetic. +fn iso_providers() -> TemporalProviders<'static> { + TemporalProviders::new(&FixedOffsetTimeZones, &IsoCalendarProvider) } // --------------------------------------------------------------------------- @@ -700,62 +558,85 @@ pub struct PlainDateTime { } impl PlainDateTime { - #[allow(clippy::too_many_arguments)] + /// `time` is hour through nanosecond. pub fn new( year: i32, month: i64, day: i64, - hour: i64, - minute: i64, - second: i64, - millisecond: i64, - microsecond: i64, - nanosecond: i64, + time: [i64; 6], overflow: Overflow, ) -> TemporalResult { + let [hour, minute, second, millisecond, microsecond, nanosecond] = time; let date = PlainDate::new(year, month, day, overflow)?; - let time = PlainTime::new(hour, minute, second, millisecond, microsecond, nanosecond, overflow)?; + let time = PlainTime::new( + hour, + minute, + second, + millisecond, + microsecond, + nanosecond, + overflow, + )?; PlainDateTime::from_parts(date, time) } pub fn from_parts(date: PlainDate, time: PlainTime) -> TemporalResult { let days = date.epoch_days(); if !iso_datetime_within_limits(days, time.to_nanoseconds_of_day()) { - return Err(TemporalError::range("date-time outside ISO representable range")); + return Err(TemporalError::range( + "date-time outside ISO representable range", + )); } Ok(PlainDateTime { date, time }) } /// AddDuration: time part first, carry whole days into the date. pub fn add(&self, duration: &Duration, overflow: Overflow) -> TemporalResult { - let (carry, new_time) = self.time.add_signed_nanoseconds(duration.time_total_nanoseconds()); - let carry_days = i64::try_from(carry) - .map_err(|_| TemporalError::range("date-time arithmetic overflow"))?; - let date_shift = Duration::new( + self.add_in( + &IsoCalendarProvider, + &CalendarId::iso8601(), + duration, + overflow, + ) + } + + /// `add` under a calendar. + pub fn add_in( + &self, + provider: &dyn CalendarProvider, + calendar: &CalendarId, + duration: &Duration, + overflow: Overflow, + ) -> TemporalResult { + let (carry, new_time) = self + .time + .add_signed_nanoseconds(duration.time_total_nanoseconds()); + let date_shift = Duration::new([ duration.years, duration.months, duration.weeks, - duration - .days - .checked_add(carry_days) - .ok_or_else(|| TemporalError::range("date-time arithmetic overflow"))?, + duration.days + carry, 0, 0, 0, 0, 0, 0, - )?; - let new_date = self.date.add(&date_shift, overflow)?; + ])?; + let new_date = provider.date_add(calendar, self.date, &date_shift, overflow)?; PlainDateTime::from_parts(new_date, new_time) } - pub fn subtract(&self, duration: &Duration, overflow: Overflow) -> TemporalResult { + pub fn subtract( + &self, + duration: &Duration, + overflow: Overflow, + ) -> TemporalResult { self.add(&duration.negated(), overflow) } - /// DifferenceTemporalPlainDateTime: exact integer difference balanced to - /// `largest_unit` (date or time) and rounded to `smallest_unit`. + /// DifferenceTemporalPlainDateTime: balanced to `largest_unit` and rounded + /// to `smallest_unit`, with `until` direction. pub fn until( &self, other: PlainDateTime, @@ -764,132 +645,85 @@ impl PlainDateTime { increment: i128, mode: RoundingMode, ) -> TemporalResult { - if smallest_unit < largest_unit { - return Err(TemporalError::range("smallestUnit must not be larger than largestUnit")); - } - if increment < 1 { - return Err(TemporalError::range("roundingIncrement must be >= 1")); - } - // Total nanosecond difference (date days + time), exact in i128. - let total_ns = self.total_ns_until(&other); - if largest_unit.category() == super::instant_duration::UnitCategory::Date - && largest_unit != Unit::Day - { - // Calendar granularity: difference dates, then fold time remainder. - let date_diff = self.date.until( - other.date, + self.until_in( + &iso_providers(), + &CalendarId::iso8601(), + other, + DifferenceRounding { largest_unit, - Unit::Day, - 1, - RoundingMode::Trunc, - Overflow::Constrain, - )?; - // Rebase: start + date_diff, difference that datetime against end. - let rebased = self.add(&date_diff, Overflow::Constrain)?; - let residual_ns = rebased.total_ns_until(&other); - let time_part = Duration::from_time_nanoseconds(residual_ns, Unit::Hour)?; - return Duration::new( - date_diff.years + time_part.years, - date_diff.months + time_part.months, - date_diff.weeks + time_part.weeks, - date_diff.days + time_part.days, - time_part.hours, - time_part.minutes, - time_part.seconds, - time_part.milliseconds, - time_part.microseconds, - time_part.nanoseconds, - ); - } - // Day-or-smaller granularity: pure balanced ns difference. - let unit_ns = smallest_unit.ns_per().ok_or(TemporalErrorKindRange)?; - validate_rounding_increment(smallest_unit, increment, false)?; - let step = unit_ns.checked_mul(increment).ok_or(TemporalErrorKindRange)?; - let rounded = round_to_increment(total_ns, step, mode)?; - Duration::from_time_nanoseconds(rounded, largest_unit.max(Unit::Day)) + smallest_unit, + increment, + mode, + }, + ) } - fn total_ns_until(&self, other: &PlainDateTime) -> i128 { - let day_delta = i128::from(other.date.epoch_days() - self.date.epoch_days()); - day_delta * NS_PER_DAY + (other.time.to_nanoseconds_of_day() - self.time.to_nanoseconds_of_day()) + /// `until` under a calendar. + pub fn until_in( + &self, + providers: &TemporalProviders<'_>, + calendar: &CalendarId, + other: PlainDateTime, + rounding: DifferenceRounding, + ) -> TemporalResult { + if rounding.smallest_unit < rounding.largest_unit { + return Err(TemporalError::range( + "smallestUnit must not be larger than largestUnit", + )); + } + if rounding.increment < 1 { + return Err(TemporalError::range("roundingIncrement must be >= 1")); + } + let internal = + difference_plain_date_time_with_rounding(providers, *self, other, calendar, rounding)?; + temporal_duration_from_internal(internal, rounding.largest_unit) } - pub fn since( + /// RoundISODateTime: rounds the time of day, whose sign is always positive, + /// and carries a rounded-up midnight into the date. + pub fn round( &self, - other: PlainDateTime, - largest_unit: Unit, - smallest_unit: Unit, + unit: Unit, increment: i128, mode: RoundingMode, - ) -> TemporalResult { - other.until(*self, largest_unit, smallest_unit, increment, mode) - } - - /// RoundTimeToDays handles `day` rounding: returns next midnight carry. - pub fn round(&self, unit: Unit, increment: i128, mode: RoundingMode) -> TemporalResult { - let (day_length_ns, rounded_from_epoch) = match unit { - Unit::Day => { - if increment != 1 { - return Err(TemporalError::range( - "roundingIncrement must be 1 when smallestUnit is day", - )); - } - (NS_PER_DAY, round_to_increment( - i128::from(self.date.epoch_days()) * NS_PER_DAY + self.time.to_nanoseconds_of_day(), - NS_PER_DAY, - mode, - )?) + ) -> TemporalResult { + let unit_ns = unit + .ns_per() + .ok_or_else(|| TemporalError::range("rounding requires day or a time unit"))?; + if increment < 1 { + return Err(TemporalError::range("roundingIncrement must be >= 1")); + } + if unit == Unit::Day { + if increment != 1 { + return Err(TemporalError::range( + "roundingIncrement must be 1 when smallestUnit is day", + )); } - _ => { - let unit_ns = unit.ns_per().ok_or(TemporalErrorKindRange)?; - if increment < 1 { - return Err(TemporalError::range("roundingIncrement must be >= 1")); - } - let per_day = NS_PER_DAY / unit_ns; - if increment > per_day || per_day % increment != 0 { - return Err(TemporalError::range(format!( - "roundingIncrement {increment} must evenly divide {per_day} {}s per day", - unit.name() - ))); - } - let step = unit_ns * increment; - let day_start_ns = i128::from(self.date.epoch_days()) * NS_PER_DAY; - let rounded_time = round_to_increment(self.time.to_nanoseconds_of_day(), step, mode)?; - (NS_PER_DAY, day_start_ns + rounded_time) + } else { + let per_day = NS_PER_DAY / unit_ns; + if increment > per_day || per_day % increment != 0 { + return Err(TemporalError::range(format!( + "roundingIncrement {increment} must evenly divide {per_day} {}s per day", + unit.name() + ))); } - }; - let _ = day_length_ns; - let days = rounded_from_epoch.div_euclid(NS_PER_DAY); - let time_ns = rounded_from_epoch.rem_euclid(NS_PER_DAY); - let date_days = i64::try_from(days) + } + let rounded = + round_to_increment(self.time.to_nanoseconds_of_day(), unit_ns * increment, mode)?; + let carry = i64::try_from(rounded.div_euclid(NS_PER_DAY)) .map_err(|_| TemporalError::range("date-time round overflow"))?; - let date = PlainDate::from_epoch_days(date_days)?; - let time = PlainTime::from_nanoseconds_of_day(time_ns); + let date = PlainDate::from_epoch_days(self.date.epoch_days() + carry)?; + let time = PlainTime::from_nanoseconds_of_day(rounded.rem_euclid(NS_PER_DAY)); PlainDateTime::from_parts(date, time) } - pub fn with( - &self, - date_delta: impl FnOnce(&PlainDate) -> TemporalResult, - time: Option, - ) -> TemporalResult { - let date = date_delta(&self.date)?; - PlainDateTime::from_parts(date, time.unwrap_or(self.time)) - } - - /// ParseTemporalDateTimeString. - pub fn parse(text: &str) -> TemporalResult { - let full = parse_iso_datetime_string(text)?; - let date = PlainDate::from_validated(full.year, full.month, full.day)?; - let time = if full.has_time { - PlainTime::from_nanoseconds_of_day(full.time_ns) - } else { - PlainTime::midnight() - }; - let dt = PlainDateTime::from_parts(date, time)?; - // A datetime parse is also the WithinLimits check only; the Instant - // boundary minus one day is permitted (spec uses half-open bounds). - Ok(dt) + /// The date-time a parsed string names, checked against the limits. A + /// string without a time means midnight. + pub fn from_parsed(parsed: &ParsedIsoDateTime) -> TemporalResult { + PlainDateTime::from_parts( + PlainDate::from_parsed(parsed)?, + PlainTime::from_nanoseconds_of_day(parsed.time_ns), + ) } /// toString: `YYYY-MM-DDTHH:MM:SS.f{auto}` (calendar omitted when iso8601). @@ -900,20 +734,13 @@ impl PlainDateTime { out } - pub fn to_plain_date(&self) -> PlainDate { + pub fn to_plain_date(self) -> PlainDate { self.date } - pub fn to_plain_time(&self) -> PlainTime { + pub fn to_plain_time(self) -> PlainTime { self.time } - - /// Epoch-nanosecond equivalent treating this wall time as UTC — used by - /// exact cross-checks; NOT a zoned conversion. - pub fn to_utc_instant_unchecked(&self) -> TemporalResult { - let ns = i128::from(self.date.epoch_days()) * NS_PER_DAY + self.time.to_nanoseconds_of_day(); - Instant::from_epoch_nanoseconds(ns) - } } impl fmt::Display for PlainDateTime { @@ -926,7 +753,16 @@ impl fmt::Display for PlainDateTime { // PlainYearMonth / PlainMonthDay // --------------------------------------------------------------------------- -/// A year/month pair (ISO calendar), internally anchored at day 1. +/// ISOYearMonthWithinLimits. +pub fn iso_year_month_within_limits(year: i32, month: u8) -> bool { + match year { + -271_821 => month >= 4, + 275_760 => month <= 9, + _ => (-271_821..=275_760).contains(&year), + } +} + +/// A year/month pair (ISO calendar) with the reference day it was built with. #[derive(Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash)] pub struct PlainYearMonth { pub year: i32, @@ -937,42 +773,67 @@ pub struct PlainYearMonth { impl PlainYearMonth { pub fn new(year: i32, month: i64, overflow: Overflow) -> TemporalResult { let (y, m, _) = regulate_iso_date(year, month, 1, overflow)?; - if !iso_date_within_limits(y, m, 1) { - return Err(TemporalError::range("year-month outside ISO representable range")); - } - Ok(PlainYearMonth { year: y, month: m, reference_day: 1 }) + PlainYearMonth::with_reference_day(y, m, 1) } - fn as_date(&self) -> TemporalResult { - PlainDate::from_validated(self.year, self.month, self.reference_day) + /// CreateTemporalYearMonth: the reference day need only exist in the month. + pub fn with_reference_day(year: i32, month: u8, day: u8) -> TemporalResult { + reject_iso_date(year, month, day)?; + if !iso_year_month_within_limits(year, month) { + return Err(TemporalError::range( + "year-month outside ISO representable range", + )); + } + Ok(PlainYearMonth { + year, + month, + reference_day: day, + }) } - pub fn days_in_month(&self) -> u8 { - days_in_month(self.year, self.month) + pub fn reference_day(&self) -> u8 { + self.reference_day } - pub fn days_in_year(&self) -> u16 { - if is_leap_year(self.year) { 366 } else { 365 } + /// The ISO date a year-month holds. Unlike a `PlainDate`, it may lie before + /// the first representable date, in the first representable month. + pub fn iso_date(&self) -> PlainDate { + PlainDate { + year: self.year, + month: self.month, + day: self.reference_day, + } } - pub fn in_leap_year(&self) -> bool { - is_leap_year(self.year) + fn first_of_month(&self) -> TemporalResult { + PlainDate::from_validated(self.year, self.month, 1) } - pub fn months_in_year(&self) -> u8 { - 12 + pub fn days_in_month(&self) -> u8 { + days_in_month(self.year, self.month) } + /// AddDurationToYearMonth. Only years and months move a year-month; a unit + /// below that has no meaning without a day. pub fn add(&self, duration: &Duration, overflow: Overflow) -> TemporalResult { - let date = self.as_date()?.add(duration, overflow)?; + if duration.default_largest_unit() > Unit::Month && !duration.is_zero() { + return Err(TemporalError::range( + "a year-month can only change by whole years and months", + )); + } + let date = self.first_of_month()?.add(duration, overflow)?; PlainYearMonth::new(date.year, date.month.into(), overflow) } - pub fn subtract(&self, duration: &Duration, overflow: Overflow) -> TemporalResult { + pub fn subtract( + &self, + duration: &Duration, + overflow: Overflow, + ) -> TemporalResult { self.add(&duration.negated(), overflow) } - /// Difference at month granularity (spec: year/month/day units only). + /// DifferenceTemporalPlainYearMonth with `until` direction. pub fn until( &self, other: PlainYearMonth, @@ -989,64 +850,48 @@ impl PlainYearMonth { )); } if smallest_unit < largest_unit { - return Err(TemporalError::range("smallestUnit must not be larger than largestUnit")); + return Err(TemporalError::range( + "smallestUnit must not be larger than largestUnit", + )); } if increment < 1 { return Err(TemporalError::range("roundingIncrement must be >= 1")); } - let months = (i64::from(other.year) - i64::from(self.year)) * 12 - + (i64::from(other.month) - i64::from(self.month)); - match smallest_unit { - Unit::Year => { - let rounded = round_to_increment(i128::from(months), 12 * increment, mode)?; - let years = i64::try_from(rounded.div_euclid(12)) - .map_err(|_| TemporalError::range("overflow"))?; - Duration::new(years, 0, 0, 0, 0, 0, 0, 0, 0, 0) - } - _ => { - let rounded = round_to_increment(i128::from(months), increment, mode)?; - match largest_unit { - Unit::Year => Duration::new( - i64::try_from(rounded.div_euclid(12)).map_err(|_| TemporalError::range("overflow"))?, - i64::try_from(rounded.rem_euclid(12)).map_err(|_| TemporalError::range("overflow"))?, - 0, 0, 0, 0, 0, 0, 0, 0, - ), - _ => Duration::new(0, i64::try_from(rounded).map_err(|_| TemporalError::range("overflow"))?, 0, 0, 0, 0, 0, 0, 0, 0), - } - } + if (self.year, self.month, self.reference_day) + == (other.year, other.month, other.reference_day) + { + return Ok(Duration::default()); } - } - - pub fn since( - &self, - other: PlainYearMonth, - largest_unit: Unit, - smallest_unit: Unit, - increment: i128, - mode: RoundingMode, - ) -> TemporalResult { - other.until(*self, largest_unit, smallest_unit, increment, mode) + difference_plain_year_month( + &iso_providers(), + self.first_of_month()?, + other.first_of_month()?, + &CalendarId::iso8601(), + DifferenceRounding { + largest_unit, + smallest_unit, + increment, + mode, + }, + ) } /// toPlainDate({ day }) cross-type operation. - pub fn to_plain_date(&self, day: i64, overflow: Overflow) -> TemporalResult { + pub fn to_plain_date(self, day: i64, overflow: Overflow) -> TemporalResult { PlainDate::new(self.year, self.month.into(), day, overflow) } - /// ParseTemporalYearMonthString: `YYYY-MM` or full date-time (fields taken). - pub fn parse(text: &str) -> TemporalResult { - if let Ok(full) = parse_iso_datetime_string(text) { - return PlainYearMonth::new(full.year, full.month.into(), Overflow::Reject); - } - // Bare year-month form (first two date components). - let synthetic = format!("{text}-01"); - let full = parse_iso_datetime_string(&synthetic) - .map_err(|e| TemporalError::syntax(format!("invalid year-month string: {}", e.message)))?; - PlainYearMonth::new(full.year, full.month.into(), Overflow::Reject) + /// The year-month a parsed string names. The string's day is discarded + /// for the canonical reference day. + pub fn from_parsed(parsed: &ParsedIsoDateTime) -> TemporalResult { + PlainYearMonth::new(parsed.year, parsed.month.into(), Overflow::Reject) } + /// TemporalYearMonthToString for the ISO calendar: the reference day is not + /// part of the value's text unless a calendar annotation requires it. pub fn format(&self) -> String { - format_iso_date(self.year, self.month, self.reference_day) + let date = format_iso_date(self.year, self.month, 1); + date[..date.len() - 3].to_owned() } } @@ -1079,7 +924,24 @@ impl PlainMonthDay { overflow: Overflow, ) -> TemporalResult { let date = PlainDate::new(reference_year, month, day, overflow)?; - Ok(PlainMonthDay { year: date.year, month: date.month, day: date.day }) + Ok(PlainMonthDay { + year: date.year, + month: date.month, + day: date.day, + }) + } + + pub fn reference_year(&self) -> i32 { + self.year + } + + /// The ISO date a month-day holds. + pub fn iso_date(&self) -> PlainDate { + PlainDate { + year: self.year, + month: self.month, + day: self.day, + } } /// Temporal monthCode (e.g. "M02", "M02L" for leap months in other @@ -1090,36 +952,20 @@ impl PlainMonthDay { /// toPlainDate({ year }) cross-type operation; rejects nonexistent dates /// (e.g. 02-29 + year 2023) unless overflow constrains. - pub fn to_plain_date(&self, year: i32, overflow: Overflow) -> TemporalResult { + pub fn to_plain_date(self, year: i32, overflow: Overflow) -> TemporalResult { PlainDate::new(year, self.month.into(), self.day.into(), overflow) } + /// Temporal.PlainMonthDay.prototype.equals: the reference year is part of + /// the value being compared. pub fn equals(&self, other: &PlainMonthDay) -> bool { - (self.year, self.month, self.day) == (other.year, other.month, other.day) - || (self.month, self.day) == (other.month, other.day) - } - - /// ParseTemporalMonthDayString: `MM-DD`, `--MM-DD`, or full date-time. - pub fn parse(text: &str) -> TemporalResult { - if let Ok(full) = parse_iso_datetime_string(text) { - return PlainMonthDay::with_reference_year( - full.month.into(), - full.day.into(), - MONTH_DAY_REFERENCE_YEAR, - Overflow::Reject, - ); - } - let bare = text.strip_prefix("--").unwrap_or(text); - let synthetic = format!("{MONTH_DAY_REFERENCE_YEAR:04}-{bare}"); - match parse_iso_datetime_string(&synthetic) { - Ok(full) => PlainMonthDay::with_reference_year( - full.month.into(), - full.day.into(), - full.year, - Overflow::Reject, - ), - Err(e) => Err(TemporalError::syntax(format!("invalid month-day string: {}", e.message))), - } + self == other + } + + /// The month-day a parsed string names. The string's year is discarded for + /// the reference year. + pub fn from_parsed(parsed: &ParsedIsoDateTime) -> TemporalResult { + PlainMonthDay::new(parsed.month.into(), parsed.day.into(), Overflow::Reject) } pub fn format(&self) -> String { @@ -1140,6 +986,36 @@ impl fmt::Display for PlainMonthDay { #[cfg(test)] mod tests { use super::*; + use crate::temporal::instant_duration::Instant; + use crate::temporal::parse::{ + parse_date_time_string, parse_month_day_string, parse_year_month_string, + }; + use crate::temporal::zoned_date_time::local_epoch_nanoseconds; + + // The string paths of `from`, without the options read between their steps. + fn parse_date(text: &str) -> TemporalResult { + let parsed = parse_date_time_string(text, false)?; + parsed.calendar_id()?; + PlainDate::from_parsed(&parsed) + } + + fn parse_date_time(text: &str) -> TemporalResult { + let parsed = parse_date_time_string(text, false)?; + parsed.calendar_id()?; + PlainDateTime::from_parsed(&parsed) + } + + fn parse_year_month(text: &str) -> TemporalResult { + let parsed = parse_year_month_string(text)?; + parsed.calendar_id()?; + PlainYearMonth::from_parsed(&parsed) + } + + fn parse_month_day(text: &str) -> TemporalResult { + let parsed = parse_month_day_string(text)?; + parsed.calendar_id()?; + PlainMonthDay::from_parsed(&parsed) + } // ---- ISO helpers ---- @@ -1171,7 +1047,14 @@ mod tests { fn time_new_overflow_policies() { let t = PlainTime::new(25, 61, 61, 1_500, 2_000, 3_000, Overflow::Constrain).unwrap(); assert_eq!( - (t.hour, t.minute, t.second, t.millisecond, t.microsecond, t.nanosecond), + ( + t.hour, + t.minute, + t.second, + t.millisecond, + t.microsecond, + t.nanosecond + ), (23, 59, 59, 999, 999, 999) ); for bad in [ @@ -1187,20 +1070,24 @@ mod tests { #[test] fn time_add_carry_across_midnight() { let t = PlainTime::new(23, 30, 0, 0, 0, 0, Overflow::Reject).unwrap(); - let d = Duration::from_time(1, 0, 0, 0, 0, 0).unwrap(); + let d = Duration::new([0, 0, 0, 0, 1, 0, 0, 0, 0, 0]).unwrap(); let (carry, new_time) = t.add_signed_nanoseconds(d.time_total_nanoseconds()); assert_eq!(carry, 1); assert_eq!((new_time.hour, new_time.minute), (0, 30)); - assert_eq!(t.add(&d).unwrap().hour, 0); + assert_eq!(t.add(&d).hour, 0); } #[test] fn time_until_rounding() { let a = PlainTime::new(10, 0, 0, 0, 0, 0, Overflow::Reject).unwrap(); let b = PlainTime::new(12, 30, 0, 0, 0, 0, Overflow::Reject).unwrap(); - let d = a.until(b, Unit::Hour, Unit::Nanosecond, 1, RoundingMode::HalfExpand).unwrap(); + let d = a + .until(b, Unit::Hour, Unit::Nanosecond, 1, RoundingMode::HalfExpand) + .unwrap(); assert_eq!((d.hours, d.minutes), (2, 30)); - let neg = b.until(a, Unit::Hour, Unit::Nanosecond, 1, RoundingMode::HalfExpand).unwrap(); + let neg = b + .until(a, Unit::Hour, Unit::Nanosecond, 1, RoundingMode::HalfExpand) + .unwrap(); assert_eq!(neg.sign(), -1); assert_eq!((neg.hours, neg.minutes), (-2, -30)); } @@ -1211,7 +1098,10 @@ mod tests { let down = t.round(Unit::Second, 1, RoundingMode::Trunc).unwrap(); assert_eq!(down, PlainTime::midnight()); let up = t.round(Unit::Second, 1, RoundingMode::Ceil).unwrap(); - assert_eq!(up, PlainTime::new(0, 0, 1, 0, 0, 0, Overflow::Reject).unwrap()); + assert_eq!( + up, + PlainTime::new(0, 0, 1, 0, 0, 0, Overflow::Reject).unwrap() + ); } #[test] @@ -1220,7 +1110,10 @@ mod tests { assert_eq!(t.format(Precision::Auto), "13:47:22.123456789"); assert_eq!(t.format(Precision::Digits(3)), "13:47:22.123"); assert_eq!(t.format(Precision::Minute), "13:47"); - assert_eq!(PlainTime::parse("T08:30").unwrap().format(Precision::Auto), "08:30:00"); + assert_eq!( + PlainTime::parse("T08:30").unwrap().format(Precision::Auto), + "08:30:00" + ); assert!(PlainTime::parse("24:00").is_err()); assert!(PlainTime::parse("08:30Z").is_err()); // no offset allowed assert!(PlainTime::parse("08:30:00.1234567890").is_err()); @@ -1244,20 +1137,25 @@ mod tests { #[test] fn date_leap_day_arithmetic() { let leap = PlainDate::new(2000, 2, 28, Overflow::Reject).unwrap(); - let one_day = Duration::new(0, 0, 0, 1, 0, 0, 0, 0, 0, 0).unwrap(); + let one_day = Duration::new([0, 0, 0, 1, 0, 0, 0, 0, 0, 0]).unwrap(); let next = leap.add(&one_day, Overflow::Constrain).unwrap(); assert_eq!((next.year, next.month, next.day), (2000, 2, 29)); - let non_leap = PlainDate::new(1900, 2, 28, Overflow::Reject).unwrap() - .add(&one_day, Overflow::Constrain).unwrap(); + let non_leap = PlainDate::new(1900, 2, 28, Overflow::Reject) + .unwrap() + .add(&one_day, Overflow::Constrain) + .unwrap(); assert_eq!((non_leap.year, non_leap.month, non_leap.day), (1900, 3, 1)); } #[test] fn date_add_year_constrains_feb29() { let leap = PlainDate::new(2020, 2, 29, Overflow::Reject).unwrap(); - let one_year = Duration::new(1, 0, 0, 0, 0, 0, 0, 0, 0, 0).unwrap(); + let one_year = Duration::new([1, 0, 0, 0, 0, 0, 0, 0, 0, 0]).unwrap(); let constrained = leap.add(&one_year, Overflow::Constrain).unwrap(); - assert_eq!((constrained.year, constrained.month, constrained.day), (2021, 2, 28)); + assert_eq!( + (constrained.year, constrained.month, constrained.day), + (2021, 2, 28) + ); assert!(leap.add(&one_year, Overflow::Reject).is_err()); } @@ -1274,16 +1172,16 @@ mod tests { let start = PlainDate::new(2019, 2, 25, Overflow::Reject).unwrap(); let end = PlainDate::new(2020, 3, 10, Overflow::Reject).unwrap(); let d = start - .until(end, Unit::Year, Unit::Day, 1, RoundingMode::Trunc, Overflow::Constrain) + .until(end, Unit::Year, Unit::Day, 1, RoundingMode::Trunc) .unwrap(); - assert_eq!((d.years, d.months, d.days), (1, 0, 13)); + assert_eq!((d.years, d.months, d.days), (1, 0, 14)); // Reconstruct: start + d == end. let back = start.add(&d, Overflow::Constrain).unwrap(); assert_eq!(back, end); let md = start - .until(end, Unit::Month, Unit::Day, 1, RoundingMode::Trunc, Overflow::Constrain) + .until(end, Unit::Month, Unit::Day, 1, RoundingMode::Trunc) .unwrap(); - assert_eq!((md.months, md.days), (12, 13)); + assert_eq!((md.months, md.days), (12, 14)); assert_eq!(start.add(&md, Overflow::Constrain).unwrap(), end); } @@ -1292,7 +1190,7 @@ mod tests { let start = PlainDate::new(2019, 2, 25, Overflow::Reject).unwrap(); let end = PlainDate::new(2020, 3, 10, Overflow::Reject).unwrap(); let d = end - .until(start, Unit::Year, Unit::Day, 1, RoundingMode::Trunc, Overflow::Constrain) + .until(start, Unit::Year, Unit::Day, 1, RoundingMode::Trunc) .unwrap(); assert_eq!((d.years, d.months, d.days), (-1, 0, -13)); assert_eq!(d.sign(), -1); @@ -1302,39 +1200,51 @@ mod tests { fn date_until_weeks_and_days() { let a = PlainDate::new(2020, 1, 1, Overflow::Reject).unwrap(); let b = PlainDate::new(2020, 1, 17, Overflow::Reject).unwrap(); - let w = a.until(b, Unit::Week, Unit::Day, 1, RoundingMode::Trunc, Overflow::Constrain).unwrap(); + let w = a + .until(b, Unit::Week, Unit::Day, 1, RoundingMode::Trunc) + .unwrap(); assert_eq!((w.weeks, w.days), (2, 2)); } #[test] fn date_rejects_time_units_in_arithmetic() { let d = PlainDate::new(2020, 1, 1, Overflow::Reject).unwrap(); - let t = Duration::from_time(1, 0, 0, 0, 0, 0).unwrap(); + let t = Duration::new([0, 0, 0, 0, 1, 0, 0, 0, 0, 0]).unwrap(); assert!(d.add(&t, Overflow::Constrain).is_err()); } + #[test] + fn critical_unknown_calendar_is_rejected() { + assert!(parse_date("2000-01-01[!u-ca=discordian]").is_err()); + } + #[test] fn date_parse_extended_years_round_trip() { - let min = PlainDate::parse("-271821-04-19").unwrap(); + let min = parse_date("-271821-04-19").unwrap(); assert_eq!(min.format(), "-271821-04-19"); - let max = PlainDate::parse("+275760-09-13").unwrap(); + let max = parse_date("+275760-09-13").unwrap(); assert_eq!(max.format(), "+275760-09-13"); - assert!(PlainDate::parse("-000000-01-01").is_err()); - assert!(PlainDate::parse("+275760-09-14").is_err()); - assert!(PlainDate::parse("2000-02-30").is_err()); + assert!(parse_date("-000000-01-01").is_err()); + assert!(parse_date("+275760-09-14").is_err()); + assert!(parse_date("2000-02-30").is_err()); // Annotations tolerated; critical unknown rejected. - assert!(PlainDate::parse("2000-01-01[u-ca=iso8601]").is_ok()); - assert!(PlainDate::parse("2000-01-01[!u-ca=japanese]").is_err()); + assert!(parse_date("2000-01-01[u-ca=iso8601]").is_ok()); + assert!(parse_date("2000-01-01[!u-ca=japanese]").is_err()); // Full datetime string: date portion taken. - let dt = PlainDate::parse("2000-05-15T12:30:00Z").unwrap(); + let dt = parse_date("2000-05-15T12:30:00").unwrap(); assert_eq!(dt.format(), "2000-05-15"); + // A UTC designator names an exact time, not a wall-clock date. + assert!(parse_date("2000-05-15T12:30:00Z").is_err()); } #[test] fn date_with_and_equality() { let d = PlainDate::new(2000, 2, 29, Overflow::Reject).unwrap(); let next_year = d.with(Some(2001), None, None, Overflow::Constrain).unwrap(); - assert_eq!((next_year.year, next_year.month, next_year.day), (2001, 2, 28)); + assert_eq!( + (next_year.year, next_year.month, next_year.day), + (2001, 2, 28) + ); assert!(d.with(Some(2001), None, None, Overflow::Reject).is_err()); assert_ne!(d, next_year); assert_eq!(d.with(None, None, None, Overflow::Constrain).unwrap(), d); @@ -1344,57 +1254,67 @@ mod tests { #[test] fn datetime_add_carries_across_midnight() { - let dt = PlainDateTime::new(2020, 12, 31, 23, 30, 0, 0, 0, 0, Overflow::Reject).unwrap(); - let d = Duration::from_time(1, 0, 0, 0, 0, 0).unwrap(); + let dt = PlainDateTime::new(2020, 12, 31, [23, 30, 0, 0, 0, 0], Overflow::Reject).unwrap(); + let d = Duration::new([0, 0, 0, 0, 1, 0, 0, 0, 0, 0]).unwrap(); let next = dt.add(&d, Overflow::Constrain).unwrap(); assert_eq!(next.format(Precision::Auto), "2021-01-01T00:30:00"); } #[test] fn datetime_subtract_negative_carries() { - let dt = PlainDateTime::new(2020, 1, 1, 0, 15, 0, 0, 0, 0, Overflow::Reject).unwrap(); - let d = Duration::from_time(1, 0, 0, 0, 0, 0).unwrap(); + let dt = PlainDateTime::new(2020, 1, 1, [0, 15, 0, 0, 0, 0], Overflow::Reject).unwrap(); + let d = Duration::new([0, 0, 0, 0, 1, 0, 0, 0, 0, 0]).unwrap(); let prev = dt.subtract(&d, Overflow::Constrain).unwrap(); assert_eq!(prev.format(Precision::Auto), "2019-12-31T23:15:00"); } #[test] fn datetime_until_time_granularity() { - let a = PlainDateTime::new(2020, 1, 1, 0, 0, 0, 0, 0, 0, Overflow::Reject).unwrap(); - let b = PlainDateTime::new(2020, 1, 3, 6, 45, 0, 0, 0, 0, Overflow::Reject).unwrap(); - let d = a.until(b, Unit::Day, Unit::Nanosecond, 1, RoundingMode::Trunc).unwrap(); + let a = PlainDateTime::new(2020, 1, 1, [0, 0, 0, 0, 0, 0], Overflow::Reject).unwrap(); + let b = PlainDateTime::new(2020, 1, 3, [6, 45, 0, 0, 0, 0], Overflow::Reject).unwrap(); + let d = a + .until(b, Unit::Day, Unit::Nanosecond, 1, RoundingMode::Trunc) + .unwrap(); assert_eq!((d.days, d.hours, d.minutes), (2, 6, 45)); - let hours = a.until(b, Unit::Hour, Unit::Nanosecond, 1, RoundingMode::Trunc).unwrap(); + let hours = a + .until(b, Unit::Hour, Unit::Nanosecond, 1, RoundingMode::Trunc) + .unwrap(); assert_eq!(hours.hours, 54); } #[test] fn datetime_until_month_granularity() { - let a = PlainDateTime::new(2020, 1, 15, 10, 0, 0, 0, 0, 0, Overflow::Reject).unwrap(); - let b = PlainDateTime::new(2020, 3, 20, 12, 30, 0, 0, 0, 0, Overflow::Reject).unwrap(); - let d = a.until(b, Unit::Month, Unit::Nanosecond, 1, RoundingMode::Trunc).unwrap(); + let a = PlainDateTime::new(2020, 1, 15, [10, 0, 0, 0, 0, 0], Overflow::Reject).unwrap(); + let b = PlainDateTime::new(2020, 3, 20, [12, 30, 0, 0, 0, 0], Overflow::Reject).unwrap(); + let d = a + .until(b, Unit::Month, Unit::Nanosecond, 1, RoundingMode::Trunc) + .unwrap(); assert_eq!((d.months, d.days, d.hours, d.minutes), (2, 5, 2, 30)); } #[test] fn datetime_round_to_day() { - let dt = PlainDateTime::new(2020, 5, 10, 13, 0, 0, 0, 0, 0, Overflow::Reject).unwrap(); + let dt = PlainDateTime::new(2020, 5, 10, [13, 0, 0, 0, 0, 0], Overflow::Reject).unwrap(); let up = dt.round(Unit::Day, 1, RoundingMode::HalfExpand).unwrap(); assert_eq!(up.format(Precision::Auto), "2020-05-11T00:00:00"); - let noon = PlainDateTime::new(2020, 5, 10, 12, 0, 0, 0, 0, 0, Overflow::Reject).unwrap(); + let noon = PlainDateTime::new(2020, 5, 10, [12, 0, 0, 0, 0, 0], Overflow::Reject).unwrap(); assert_eq!( - noon.round(Unit::Day, 1, RoundingMode::HalfExpand).unwrap().format(Precision::Auto), + noon.round(Unit::Day, 1, RoundingMode::HalfExpand) + .unwrap() + .format(Precision::Auto), "2020-05-11T00:00:00" ); assert_eq!( - noon.round(Unit::Day, 1, RoundingMode::HalfTrunc).unwrap().format(Precision::Auto), + noon.round(Unit::Day, 1, RoundingMode::HalfTrunc) + .unwrap() + .format(Precision::Auto), "2020-05-10T00:00:00" ); } #[test] fn datetime_round_time_units() { - let dt = PlainDateTime::parse("2020-05-10T13:47:22.123456789").unwrap(); + let dt = parse_date_time("2020-05-10T13:47:22.123456789").unwrap(); let r = dt.round(Unit::Minute, 15, RoundingMode::Floor).unwrap(); assert_eq!(r.format(Precision::Auto), "2020-05-10T13:45:00"); assert!(dt.round(Unit::Minute, 7, RoundingMode::Floor).is_err()); @@ -1403,19 +1323,28 @@ mod tests { #[test] fn datetime_limits() { // Exact instant bound minus one day is allowed at midnight+ε of next. - assert!(PlainDateTime::new(275_760, 9, 13, 23, 59, 59, 999, 999, 999, Overflow::Reject).is_ok()); - assert!(PlainDateTime::new(275_760, 9, 14, 0, 0, 0, 0, 0, 0, Overflow::Reject).is_err()); + assert!( + PlainDateTime::new( + 275_760, + 9, + 13, + [23, 59, 59, 999, 999, 999], + Overflow::Reject + ) + .is_ok() + ); + assert!(PlainDateTime::new(275_760, 9, 14, [0, 0, 0, 0, 0, 0], Overflow::Reject).is_err()); } #[test] fn datetime_parse_format() { - let dt = PlainDateTime::parse("2000-02-29T08:30:00.000000001").unwrap(); + let dt = parse_date_time("2000-02-29T08:30:00.000000001").unwrap(); assert_eq!(dt.format(Precision::Auto), "2000-02-29T08:30:00.000000001"); - let date_only = PlainDateTime::parse("1999-12-31").unwrap(); + let date_only = parse_date_time("1999-12-31").unwrap(); assert_eq!(date_only.format(Precision::Auto), "1999-12-31T00:00:00"); assert_eq!(date_only.to_plain_date().format(), "1999-12-31"); // UTC offset present but ignored for wall-clock read-out. - let with_offset = PlainDateTime::parse("2000-01-01T01:00:00+01:00").unwrap(); + let with_offset = parse_date_time("2000-01-01T01:00:00+01:00").unwrap(); assert_eq!(with_offset.format(Precision::Auto), "2000-01-01T01:00:00"); } @@ -1427,14 +1356,19 @@ mod tests { let time = PlainTime::new(12, 0, 0, 0, 0, 0, Overflow::Reject).unwrap(); let dt = date.to_plain_date_time(time).unwrap(); assert_eq!(dt.format(Precision::Auto), "2000-02-29T12:00:00"); - assert_eq!(dt.to_utc_instant_unchecked().unwrap().format(Precision::Auto), "2000-02-29T12:00:00Z"); + assert_eq!( + Instant::from_epoch_nanoseconds(local_epoch_nanoseconds(dt)) + .unwrap() + .format(Precision::Auto), + "2000-02-29T12:00:00Z" + ); } #[test] fn cross_type_year_month_and_month_day() { let date = PlainDate::new(2024, 2, 29, Overflow::Reject).unwrap(); let ym = date.to_plain_year_month(); - assert_eq!(ym.format(), "2024-02-01"); + assert_eq!(ym.format(), "2024-02"); assert_eq!(ym.days_in_month(), 29); assert_eq!(ym.to_plain_date(29, Overflow::Reject).unwrap(), date); let md = date.to_plain_month_day(); @@ -1442,33 +1376,42 @@ mod tests { assert_eq!(md.month_code(), "M02"); // Feb 29 + non-leap year: reject unless constrained. assert!(md.to_plain_date(2023, Overflow::Reject).is_err()); - assert_eq!(md.to_plain_date(2023, Overflow::Constrain).unwrap().format(), "2023-02-28"); + assert_eq!( + md.to_plain_date(2023, Overflow::Constrain) + .unwrap() + .format(), + "2023-02-28" + ); } #[test] fn year_month_arithmetic_and_difference() { let ym = PlainYearMonth::new(2020, 11, Overflow::Reject).unwrap(); - let three = Duration::new(0, 14, 0, 0, 0, 0, 0, 0, 0, 0).unwrap(); + let three = Duration::new([0, 14, 0, 0, 0, 0, 0, 0, 0, 0]).unwrap(); let later = ym.add(&three, Overflow::Constrain).unwrap(); - assert_eq!(later.format(), "2022-01-01"); - let d = ym.until(later, Unit::Month, Unit::Month, 1, RoundingMode::Trunc).unwrap(); + assert_eq!(later.format(), "2022-01"); + let d = ym + .until(later, Unit::Month, Unit::Month, 1, RoundingMode::Trunc) + .unwrap(); assert_eq!(d.months, 14); - let yd = ym.until(later, Unit::Year, Unit::Year, 1, RoundingMode::Trunc).unwrap(); + let yd = ym + .until(later, Unit::Year, Unit::Year, 1, RoundingMode::Trunc) + .unwrap(); assert_eq!((yd.years, yd.months), (1, 0)); - assert_eq!( - PlainYearMonth::parse("2020-11").unwrap().format(), - "2020-11-01" + assert_eq!(parse_year_month("2020-11").unwrap().format(), "2020-11"); + assert!( + ym.until(later, Unit::Day, Unit::Day, 1, RoundingMode::Trunc) + .is_err() ); - assert!(ym.until(later, Unit::Day, Unit::Day, 1, RoundingMode::Trunc).is_err()); } #[test] fn month_day_parse_forms() { - let md = PlainMonthDay::parse("02-29").unwrap(); + let md = parse_month_day("02-29").unwrap(); assert_eq!(md.format(), "02-29"); - let dashed = PlainMonthDay::parse("--12-25").unwrap(); + let dashed = parse_month_day("--12-25").unwrap(); assert_eq!(dashed.format(), "12-25"); - assert!(PlainMonthDay::parse("02-30").is_err()); + assert!(parse_month_day("02-30").is_err()); // Equality ignores reference year. let other = PlainMonthDay::with_reference_year(2, 29, 1972, Overflow::Reject).unwrap(); assert!(md.equals(&other)); diff --git a/crates/bamts-runtime/src/temporal/relative.rs b/crates/bamts-runtime/src/temporal/relative.rs new file mode 100644 index 00000000..608343a6 --- /dev/null +++ b/crates/bamts-runtime/src/temporal/relative.rs @@ -0,0 +1,1114 @@ +//! Calendar-relative duration arithmetic. +//! +//! A duration only has a length relative to a starting point: months differ, +//! and with a time zone so do days. This module holds the operations that need +//! such a starting point: `CalendarDateUntil`, the `Difference*` family, and +//! `RoundRelativeDuration` with its nudging machinery. Epoch arithmetic is exact +//! `i128` nanoseconds; rounding decides on exact ratios and never on floats. + +use std::cmp::Ordering; + +use super::instant_duration::{ + Duration, Instant, MAX_TIME_DURATION, NS_PER_DAY, NS_PER_HOUR, Overflow, RoundingMode, + TemporalError, TemporalResult, Unit, UnitCategory, days_in_month, epoch_days_from_ymd, + exact_ratio_to_f64, round_to_increment, rounds_past_fraction, +}; +use super::now_rounding::round_nanoseconds; +use super::plain_types::{PlainDate, PlainDateTime, PlainTime}; +use super::zoned_date_time::{ + CalendarId, TemporalProviders, TimeZoneId, epoch_nanoseconds_for, iso_date_time_for_instant, + local_epoch_nanoseconds, +}; + +/// `Internal Duration Record`: a date duration plus a normalized time duration. +/// The two parts always agree in sign. `date` has every time field zero. +#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] +pub struct InternalDuration { + pub date: Duration, + pub time: i128, +} + +impl InternalDuration { + /// CombineDateAndTimeDuration. + pub fn new(date: Duration, time: i128) -> Self { + Self { date, time } + } + + /// InternalDurationSign. + pub fn sign(&self) -> i128 { + match self.date.sign() { + 0 => self.time.signum(), + sign => sign, + } + } + + /// ToInternalDurationRecord. + pub fn from_duration(duration: &Duration) -> Self { + Self::new(duration.date_part(), duration.time_total_nanoseconds()) + } + + /// ToInternalDurationRecordWith24HourDays. + pub fn from_duration_with_24_hour_days(duration: &Duration) -> Self { + let mut date = duration.date_part(); + date.days = 0; + Self::new(date, duration.total_nanoseconds_with_24_hour_days()) + } +} + +/// How a difference is rounded; `largest_unit` also bounds its balancing. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +pub struct DifferenceRounding { + pub largest_unit: Unit, + pub smallest_unit: Unit, + pub increment: i128, + pub mode: RoundingMode, +} + +fn limit_error() -> TemporalError { + TemporalError::range("value outside the limits of Temporal time durations") +} + +/// Rejects a time duration of 2^53 seconds or more. +fn checked_time_duration(time: i128) -> TemporalResult { + if time.abs() >= MAX_TIME_DURATION { + return Err(limit_error()); + } + Ok(time) +} + +/// AddTimeDuration. +pub fn add_time_duration(one: i128, two: i128) -> TemporalResult { + checked_time_duration(one + two) +} + +/// Add24HourDaysToTimeDuration. +fn add_24_hour_days(time: i128, days: i128) -> TemporalResult { + let shift = days.checked_mul(NS_PER_DAY).ok_or_else(limit_error)?; + checked_time_duration(time.checked_add(shift).ok_or_else(limit_error)?) +} + +/// RoundTimeDuration: `unit` is a time unit or day. +pub fn round_time_duration( + time: i128, + increment: i128, + unit: Unit, + mode: RoundingMode, +) -> TemporalResult { + checked_time_duration(round_nanoseconds(time, unit, increment, mode)?) +} + +/// RoundTimeDurationToIncrement. +fn round_time_duration_to_increment( + time: i128, + step: i128, + mode: RoundingMode, +) -> TemporalResult { + checked_time_duration(round_to_increment(time, step, mode)?) +} + +/// TotalTimeDuration: the exact quotient, rounded once to a Number. +pub fn total_time_duration(time: i128, unit: Unit) -> TemporalResult { + let unit_ns = unit + .ns_per() + .ok_or_else(|| TemporalError::range("time total requires day or a time unit"))?; + Ok(exact_ratio_to_f64(time, unit_ns)) +} + +/// ToDateDurationRecordWithoutTime: time folds into whole days, truncating. +pub fn date_duration_without_time(duration: &Duration) -> TemporalResult { + let internal = InternalDuration::from_duration_with_24_hour_days(duration); + let mut date = internal.date; + date.days = internal.time / NS_PER_DAY; + Duration::new(date.fields()) +} + +/// TemporalDurationFromInternal. Time balances up to `largest_unit`, with day +/// the largest bucket it can reach. +pub fn temporal_duration_from_internal( + internal: InternalDuration, + largest_unit: Unit, +) -> TemporalResult { + let time = Duration::from_time_nanoseconds(internal.time, largest_unit.max(Unit::Day))?; + let date = internal.date; + Duration::new([ + date.years, + date.months, + date.weeks, + date.days + time.days, + time.hours, + time.minutes, + time.seconds, + time.milliseconds, + time.microseconds, + time.nanoseconds, + ]) +} + +/// CheckISODaysRange. +pub fn check_iso_days_range(date: PlainDate) -> TemporalResult<()> { + if epoch_days_from_ymd(date.year, date.month, date.day).abs() > 100_000_000 { + return Err(TemporalError::range("date is too far from the epoch")); + } + Ok(()) +} + +/// BalanceISOYearMonth on an unbounded year and a 1-based month. +fn balance_year_month(year: i64, month: i64) -> (i64, i64) { + ( + year + (month - 1).div_euclid(12), + (month - 1).rem_euclid(12) + 1, + ) +} + +/// ISODateSurpasses: whether (year, month, day), compared field by field and +/// before any clamping, lies past `target` in the direction of `sign`. +fn iso_date_surpasses(sign: i64, year: i64, month: i64, day: i64, target: PlainDate) -> bool { + let (target_year, target_month, target_day) = ( + i64::from(target.year), + i64::from(target.month), + i64::from(target.day), + ); + let difference = if year != target_year { + year - target_year + } else if month != target_month { + month - target_month + } else { + day - target_day + }; + sign * difference > 0 +} + +/// CalendarDateUntil for the ISO calendar. The walk starts at `one` in the +/// direction of `two`, so month-length asymmetry falls on the same side for +/// both signs of the result; this is why it is not `until` with swapped ends. +pub fn iso_date_until( + one: PlainDate, + two: PlainDate, + largest_unit: Unit, +) -> TemporalResult { + let sign: i64 = match two.cmp(&one) { + Ordering::Equal => return Ok(Duration::default()), + Ordering::Greater => 1, + Ordering::Less => -1, + }; + let (one_year, one_month, one_day) = ( + i64::from(one.year), + i64::from(one.month), + i64::from(one.day), + ); + let mut years = 0_i64; + let mut months = 0_i64; + if matches!(largest_unit, Unit::Year | Unit::Month) { + let mut candidate_years = i64::from(two.year) - one_year; + if candidate_years != 0 { + candidate_years -= sign; + } + while !iso_date_surpasses(sign, one_year + candidate_years, one_month, one_day, two) { + years = candidate_years; + candidate_years += sign; + } + let mut candidate_months = sign; + loop { + let (year, month) = balance_year_month(one_year + years, one_month + candidate_months); + if iso_date_surpasses(sign, year, month, one_day, two) { + break; + } + months = candidate_months; + candidate_months += sign; + } + if largest_unit == Unit::Month { + months += years * 12; + years = 0; + } + } + let (year, month) = balance_year_month(one_year + years, one_month + months); + let year = i32::try_from(year).map_err(|_| limit_error())?; + let month = u8::try_from(month).map_err(|_| limit_error())?; + let day = one.day.min(days_in_month(year, month)); + let constrained_days = epoch_days_from_ymd(year, month, day); + let mut days = epoch_days_from_ymd(two.year, two.month, two.day) - constrained_days; + let mut weeks = 0; + if largest_unit == Unit::Week { + weeks = days / 7; + days %= 7; + } + Duration::from_date(years.into(), months.into(), weeks.into(), days.into()) +} + +/// The fixed context of one relative operation. +#[derive(Clone, Copy)] +pub struct RelativeOrigin<'a> { + pub providers: &'a TemporalProviders<'a>, + pub date_time: PlainDateTime, + pub time_zone: Option<&'a TimeZoneId>, + pub calendar: &'a CalendarId, +} + +impl RelativeOrigin<'_> { + fn date_add(&self, duration: &Duration) -> TemporalResult { + self.providers.calendar.date_add( + self.calendar, + self.date_time.date, + duration, + Overflow::Constrain, + ) + } + + /// The exact time of `date` at the origin's wall-clock time. Without a time + /// zone this is plain arithmetic and may lie past the Instant range, which + /// `CalendarDateAdd` has already bounded by date. + fn epoch_nanoseconds_at(&self, date: PlainDate) -> TemporalResult { + let date_time = PlainDateTime { + date, + time: self.date_time.time, + }; + match self.time_zone { + Some(time_zone) => { + epoch_nanoseconds_for(self.providers.time_zone, time_zone, date_time) + } + None => Ok(local_epoch_nanoseconds(date_time)), + } + } +} + +struct NudgeWindow { + r1: i128, + start_epoch_ns: i128, + end_epoch_ns: i128, + start_duration: Duration, + end_duration: Duration, +} + +/// ComputeNudgeWindow: the two candidate multiples of `increment` around the +/// duration, and the exact times they land on. +fn compute_nudge_window( + sign: i128, + duration: &InternalDuration, + origin_epoch_ns: i128, + origin: &RelativeOrigin<'_>, + increment: i128, + unit: Unit, + additional_shift: bool, +) -> TemporalResult { + let date = duration.date; + let window = |truncated: i128| { + let r1 = if additional_shift { + truncated + increment * sign + } else { + truncated + }; + (r1, r1 + increment * sign) + }; + let (r1, start_duration, end_duration) = match unit { + Unit::Year => { + let truncated = round_to_increment(date.years, increment, RoundingMode::Trunc)?; + let (r1, r2) = window(truncated); + ( + r1, + Duration::from_date(r1, 0, 0, 0)?, + Duration::from_date(r2, 0, 0, 0)?, + ) + } + Unit::Month => { + let truncated = round_to_increment(date.months, increment, RoundingMode::Trunc)?; + let (r1, r2) = window(truncated); + ( + r1, + Duration::from_date(date.years, r1, 0, 0)?, + Duration::from_date(date.years, r2, 0, 0)?, + ) + } + Unit::Week => { + let years_months = Duration::from_date(date.years, date.months, 0, 0)?; + let weeks_start = origin.date_add(&years_months)?; + let weeks_end = origin.providers.calendar.date_add( + origin.calendar, + weeks_start, + &Duration::from_date(0, 0, 0, date.days)?, + Overflow::Constrain, + )?; + let until = origin.providers.calendar.date_until( + origin.calendar, + weeks_start, + weeks_end, + Unit::Week, + )?; + let truncated = + round_to_increment(date.weeks + until.weeks, increment, RoundingMode::Trunc)?; + let (r1, r2) = window(truncated); + ( + r1, + Duration::from_date(date.years, date.months, r1, 0)?, + Duration::from_date(date.years, date.months, r2, 0)?, + ) + } + _ => { + let truncated = round_to_increment(date.days, increment, RoundingMode::Trunc)?; + let (r1, r2) = window(truncated); + ( + r1, + Duration::from_date(date.years, date.months, date.weeks, r1)?, + Duration::from_date(date.years, date.months, date.weeks, r2)?, + ) + } + }; + // Only an empty start duration is the origin itself; with larger units + // present, r1 = 0 still lies after them. The origin is used as given rather + // than recomputed because a wall-clock round trip may not reproduce it. + let start_epoch_ns = if start_duration.is_zero() { + origin_epoch_ns + } else { + origin.epoch_nanoseconds_at(origin.date_add(&start_duration)?)? + }; + let end_epoch_ns = origin.epoch_nanoseconds_at(origin.date_add(&end_duration)?)?; + Ok(NudgeWindow { + r1, + start_epoch_ns, + end_epoch_ns, + start_duration, + end_duration, + }) +} + +fn window_contains(sign: i128, window: &NudgeWindow, dest_epoch_ns: i128) -> bool { + if sign > 0 { + window.start_epoch_ns <= dest_epoch_ns && dest_epoch_ns <= window.end_epoch_ns + } else { + window.end_epoch_ns <= dest_epoch_ns && dest_epoch_ns <= window.start_epoch_ns + } +} + +struct NudgeResult { + duration: InternalDuration, + nudged_epoch_ns: i128, + did_expand: bool, +} + +struct CalendarNudge { + result: NudgeResult, + /// The exact unit count `numerator / denominator`, for `total()`. + total_numerator: i128, + total_denominator: i128, +} + +/// The exact times a relative operation starts from and is rounding toward. +#[derive(Clone, Copy)] +struct Endpoints { + origin_epoch_ns: i128, + dest_epoch_ns: i128, +} + +/// NudgeToCalendarUnit. +fn nudge_to_calendar_unit( + sign: i128, + duration: &InternalDuration, + endpoints: Endpoints, + origin: &RelativeOrigin<'_>, + increment: i128, + unit: Unit, + mode: RoundingMode, +) -> TemporalResult { + let Endpoints { + origin_epoch_ns, + dest_epoch_ns, + } = endpoints; + let mut did_expand = false; + let mut window = compute_nudge_window( + sign, + duration, + origin_epoch_ns, + origin, + increment, + unit, + false, + )?; + if !window_contains(sign, &window, dest_epoch_ns) { + window = compute_nudge_window( + sign, + duration, + origin_epoch_ns, + origin, + increment, + unit, + true, + )?; + if !window_contains(sign, &window, dest_epoch_ns) { + return Err(TemporalError::range( + "calendar arithmetic could not bracket the rounding target", + )); + } + did_expand = true; + } + let span = window.end_epoch_ns - window.start_epoch_ns; + let progress = dest_epoch_ns - window.start_epoch_ns; + if span == 0 { + return Err(TemporalError::range( + "calendar arithmetic produced an empty rounding window", + )); + } + // total = r1 + (progress / span) × increment × sign, over a positive + // denominator. `progress` and `span` share a sign, so the fraction is in + // [0, 1] and the comparison below is on magnitudes. + let total_denominator = span.abs(); + let total_numerator = window.r1 * total_denominator + progress.abs() * increment * sign; + let expanded = progress.abs() == span.abs() + || (progress != 0 + && rounds_past_fraction( + progress.unsigned_abs(), + span.unsigned_abs(), + (window.r1.abs() / increment).unsigned_abs(), + mode, + sign < 0, + )); + let (nudged, nudged_epoch_ns) = if expanded { + did_expand = true; + (window.end_duration, window.end_epoch_ns) + } else { + (window.start_duration, window.start_epoch_ns) + }; + Ok(CalendarNudge { + result: NudgeResult { + duration: InternalDuration::new(nudged, 0), + nudged_epoch_ns, + did_expand, + }, + total_numerator, + total_denominator, + }) +} + +/// NudgeToZonedTime: rounds a time unit inside one wall-clock day of a time zone +/// and detects a result that passes the end of that day. +fn nudge_to_zoned_time( + sign: i128, + duration: &InternalDuration, + origin: &RelativeOrigin<'_>, + increment: i128, + unit: Unit, + mode: RoundingMode, +) -> TemporalResult { + let start = origin.date_add(&duration.date)?; + let end = origin.providers.calendar.date_add( + origin.calendar, + start, + &Duration::from_date(0, 0, 0, sign)?, + Overflow::Constrain, + )?; + let start_epoch_ns = origin.epoch_nanoseconds_at(start)?; + let end_epoch_ns = origin.epoch_nanoseconds_at(end)?; + let day_span = end_epoch_ns - start_epoch_ns; + if day_span.signum() != sign { + return Err(TemporalError::range("a wall-clock day has no length")); + } + let unit_ns = unit + .ns_per() + .ok_or_else(|| TemporalError::range("zoned rounding requires a time unit"))?; + let step = increment * unit_ns; + let mut rounded = round_time_duration_to_increment(duration.time, step, mode)?; + let beyond_day = add_time_duration(rounded, -day_span)?; + let (did_expand, day_delta, nudged_epoch_ns) = if beyond_day.signum() == -sign { + (false, 0, start_epoch_ns + rounded) + } else { + rounded = round_time_duration_to_increment(beyond_day, step, mode)?; + (true, sign, end_epoch_ns + rounded) + }; + let mut date = duration.date; + date.days += day_delta; + Ok(NudgeResult { + duration: InternalDuration::new(Duration::new(date.fields())?, rounded), + nudged_epoch_ns, + did_expand, + }) +} + +/// NudgeToDayOrTime: days are exactly 24 hours here. +fn nudge_to_day_or_time( + duration: &InternalDuration, + dest_epoch_ns: i128, + rounding: DifferenceRounding, +) -> TemporalResult { + let time = add_24_hour_days(duration.time, duration.date.days)?; + let unit_ns = rounding + .smallest_unit + .ns_per() + .ok_or_else(|| TemporalError::range("time rounding requires day or a time unit"))?; + let rounded = + round_time_duration_to_increment(time, unit_ns * rounding.increment, rounding.mode)?; + let difference = add_time_duration(rounded, -time)?; + let whole_days = time / NS_PER_DAY; + let rounded_whole_days = rounded / NS_PER_DAY; + let day_delta = rounded_whole_days - whole_days; + let did_expand = day_delta.signum() == time.signum(); + let nudged_epoch_ns = dest_epoch_ns + difference; + let (days, remainder) = if rounding.largest_unit.category() == UnitCategory::Date { + ( + rounded_whole_days, + add_time_duration(rounded, -(rounded_whole_days * 24 * NS_PER_HOUR))?, + ) + } else { + (0, rounded) + }; + let mut date = duration.date; + date.days = days; + Ok(NudgeResult { + duration: InternalDuration::new(Duration::new(date.fields())?, remainder), + nudged_epoch_ns, + did_expand, + }) +} + +/// BubbleRelativeDuration: after rounding carried into a larger unit, walks +/// upward and absorbs each unit the nudged time has reached. +fn bubble_relative_duration( + sign: i128, + mut duration: InternalDuration, + nudged_epoch_ns: i128, + origin: &RelativeOrigin<'_>, + largest_unit: Unit, + smallest_unit: Unit, +) -> TemporalResult { + if smallest_unit == largest_unit { + return Ok(duration); + } + for unit in [Unit::Week, Unit::Month, Unit::Year] { + if unit >= smallest_unit || unit < largest_unit { + continue; + } + if unit == Unit::Week && largest_unit != Unit::Week { + continue; + } + let date = duration.date; + let end_duration = match unit { + Unit::Year => Duration::from_date(date.years + sign, 0, 0, 0)?, + Unit::Month => Duration::from_date(date.years, date.months + sign, 0, 0)?, + _ => Duration::from_date(date.years, date.months, date.weeks + sign, 0)?, + }; + let end_epoch_ns = origin.epoch_nanoseconds_at(origin.date_add(&end_duration)?)?; + if (nudged_epoch_ns - end_epoch_ns).signum() == -sign { + break; + } + duration = InternalDuration::new(end_duration, 0); + } + Ok(duration) +} + +/// RoundRelativeDuration. +fn round_relative_duration( + duration: InternalDuration, + endpoints: Endpoints, + origin: &RelativeOrigin<'_>, + rounding: DifferenceRounding, +) -> TemporalResult { + let smallest_unit = rounding.smallest_unit; + let irregular = + smallest_unit < Unit::Day || (origin.time_zone.is_some() && smallest_unit == Unit::Day); + let sign = if duration.sign() < 0 { -1 } else { 1 }; + let nudge = if irregular { + nudge_to_calendar_unit( + sign, + &duration, + endpoints, + origin, + rounding.increment, + smallest_unit, + rounding.mode, + )? + .result + } else if origin.time_zone.is_some() { + nudge_to_zoned_time( + sign, + &duration, + origin, + rounding.increment, + smallest_unit, + rounding.mode, + )? + } else { + nudge_to_day_or_time(&duration, endpoints.dest_epoch_ns, rounding)? + }; + if nudge.did_expand && smallest_unit != Unit::Week { + return bubble_relative_duration( + sign, + nudge.duration, + nudge.nudged_epoch_ns, + origin, + rounding.largest_unit, + smallest_unit.larger(Unit::Day), + ); + } + Ok(nudge.duration) +} + +/// TotalRelativeDuration. +fn total_relative_duration( + duration: InternalDuration, + endpoints: Endpoints, + origin: &RelativeOrigin<'_>, + unit: Unit, +) -> TemporalResult { + if unit < Unit::Day || (origin.time_zone.is_some() && unit == Unit::Day) { + let sign = if duration.sign() < 0 { -1 } else { 1 }; + let nudge = nudge_to_calendar_unit( + sign, + &duration, + endpoints, + origin, + 1, + unit, + RoundingMode::Trunc, + )?; + return Ok(exact_ratio_to_f64( + nudge.total_numerator, + nudge.total_denominator, + )); + } + total_time_duration(add_24_hour_days(duration.time, duration.date.days)?, unit) +} + +/// CombineISODateAndTimeRecord at midnight. A record has no limit of its own: +/// the first representable date's midnight lies just before the range. +fn midnight_record(date: PlainDate) -> PlainDateTime { + PlainDateTime { + date, + time: PlainTime::midnight(), + } +} + +/// DifferenceTime. +pub fn difference_time(one: PlainTime, two: PlainTime) -> i128 { + two.to_nanoseconds_of_day() - one.to_nanoseconds_of_day() +} + +/// DifferenceISODateTime. +fn difference_iso_date_time( + providers: &TemporalProviders<'_>, + one: PlainDateTime, + two: PlainDateTime, + calendar: &CalendarId, + largest_unit: Unit, +) -> TemporalResult { + let mut time = difference_time(one.time, two.time); + let time_sign = time.signum(); + // CompareISODate(one, two): equal to the time's sign exactly when the time + // difference runs against the date difference and must borrow a day. + let date_sign = match one.date.cmp(&two.date) { + Ordering::Less => -1, + Ordering::Equal => 0, + Ordering::Greater => 1, + }; + let mut adjusted = two.date; + if time_sign == date_sign { + adjusted = providers.calendar.date_add( + calendar, + adjusted, + &Duration::from_date(0, 0, 0, time_sign)?, + Overflow::Constrain, + )?; + time = add_24_hour_days(time, -time_sign)?; + } + let date_largest = largest_unit.larger(Unit::Day); + let mut date = providers + .calendar + .date_until(calendar, one.date, adjusted, date_largest)?; + if largest_unit != date_largest { + time = add_24_hour_days(time, date.days)?; + date.days = 0; + } + Ok(InternalDuration::new(date, time)) +} + +/// DifferencePlainDateTimeWithRounding. +pub fn difference_plain_date_time_with_rounding( + providers: &TemporalProviders<'_>, + one: PlainDateTime, + two: PlainDateTime, + calendar: &CalendarId, + rounding: DifferenceRounding, +) -> TemporalResult { + if one == two { + return Ok(InternalDuration::default()); + } + PlainDateTime::from_parts(one.date, one.time)?; + PlainDateTime::from_parts(two.date, two.time)?; + let difference = + difference_iso_date_time(providers, one, two, calendar, rounding.largest_unit)?; + if rounding.smallest_unit == Unit::Nanosecond && rounding.increment == 1 { + return Ok(difference); + } + let origin = RelativeOrigin { + providers, + date_time: one, + time_zone: None, + calendar, + }; + let endpoints = Endpoints { + origin_epoch_ns: local_epoch_nanoseconds(one), + dest_epoch_ns: local_epoch_nanoseconds(two), + }; + round_relative_duration(difference, endpoints, &origin, rounding) +} + +/// DifferencePlainDateTimeWithTotal. +pub fn difference_plain_date_time_with_total( + providers: &TemporalProviders<'_>, + one: PlainDateTime, + two: PlainDateTime, + calendar: &CalendarId, + unit: Unit, +) -> TemporalResult { + if one == two { + return Ok(0.0); + } + PlainDateTime::from_parts(one.date, one.time)?; + PlainDateTime::from_parts(two.date, two.time)?; + let difference = difference_iso_date_time(providers, one, two, calendar, unit)?; + if unit == Unit::Nanosecond { + return total_time_duration(difference.time, unit); + } + let origin = RelativeOrigin { + providers, + date_time: one, + time_zone: None, + calendar, + }; + let endpoints = Endpoints { + origin_epoch_ns: local_epoch_nanoseconds(one), + dest_epoch_ns: local_epoch_nanoseconds(two), + }; + total_relative_duration(difference, endpoints, &origin, unit) +} + +/// The date-only difference behind `PlainDate.prototype.until`. +pub fn difference_plain_date( + providers: &TemporalProviders<'_>, + one: PlainDate, + two: PlainDate, + calendar: &CalendarId, + rounding: DifferenceRounding, +) -> TemporalResult { + if one == two { + return Ok(Duration::default()); + } + let date = providers + .calendar + .date_until(calendar, one, two, rounding.largest_unit)?; + let mut duration = InternalDuration::new(date, 0); + if rounding.smallest_unit != Unit::Day || rounding.increment != 1 { + let start = midnight_record(one); + let end = midnight_record(two); + let origin = RelativeOrigin { + providers, + date_time: start, + time_zone: None, + calendar, + }; + let endpoints = Endpoints { + origin_epoch_ns: local_epoch_nanoseconds(start), + dest_epoch_ns: local_epoch_nanoseconds(end), + }; + duration = round_relative_duration(duration, endpoints, &origin, rounding)?; + } + temporal_duration_from_internal(duration, Unit::Day) +} + +/// The year-month difference behind `PlainYearMonth.prototype.until`: both ends +/// are read at the first of their month, and only years and months survive. +pub fn difference_plain_year_month( + providers: &TemporalProviders<'_>, + one: PlainDate, + two: PlainDate, + calendar: &CalendarId, + rounding: DifferenceRounding, +) -> TemporalResult { + let until = providers + .calendar + .date_until(calendar, one, two, rounding.largest_unit)?; + let years_months = Duration::from_date(until.years, until.months, 0, 0)?; + let mut duration = InternalDuration::new(years_months, 0); + if rounding.smallest_unit != Unit::Month || rounding.increment != 1 { + let start = midnight_record(one); + let end = midnight_record(two); + let origin = RelativeOrigin { + providers, + date_time: start, + time_zone: None, + calendar, + }; + let endpoints = Endpoints { + origin_epoch_ns: local_epoch_nanoseconds(start), + dest_epoch_ns: local_epoch_nanoseconds(end), + }; + duration = round_relative_duration(duration, endpoints, &origin, rounding)?; + } + temporal_duration_from_internal(duration, Unit::Day) +} + +/// DifferenceInstant. +pub fn difference_instant( + one: i128, + two: i128, + increment: i128, + unit: Unit, + mode: RoundingMode, +) -> TemporalResult { + let time = round_time_duration(two - one, increment, unit, mode)?; + Ok(InternalDuration::new(Duration::default(), time)) +} + +/// DifferenceZonedDateTime. +fn difference_zoned_date_time( + providers: &TemporalProviders<'_>, + start_ns: i128, + end_ns: i128, + time_zone: &TimeZoneId, + calendar: &CalendarId, + largest_unit: Unit, +) -> TemporalResult { + if start_ns == end_ns { + return Ok(InternalDuration::default()); + } + let start = iso_date_time_for_instant(providers.time_zone, time_zone, start_ns)?; + let end = iso_date_time_for_instant(providers.time_zone, time_zone, end_ns)?; + if start.date == end.date { + return Ok(InternalDuration::new( + Duration::default(), + end_ns - start_ns, + )); + } + let sign = (end_ns - start_ns).signum(); + let max_day_correction = if sign == 1 { 2 } else { 1 }; + let mut day_correction = 0; + if difference_time(start.time, end.time).signum() == -sign { + day_correction += 1; + } + while day_correction <= max_day_correction { + let intermediate_date = providers.calendar.date_add( + calendar, + end.date, + &Duration::from_date(0, 0, 0, day_correction * -sign)?, + Overflow::Constrain, + )?; + let intermediate = PlainDateTime::from_parts(intermediate_date, start.time)?; + let intermediate_ns = epoch_nanoseconds_for(providers.time_zone, time_zone, intermediate)?; + let time = end_ns - intermediate_ns; + if sign != -time.signum() { + let date = providers.calendar.date_until( + calendar, + start.date, + intermediate_date, + largest_unit.larger(Unit::Day), + )?; + return Ok(InternalDuration::new(date, time)); + } + day_correction += 1; + } + Err(TemporalError::range( + "time zone data prevented a zoned difference from converging", + )) +} + +/// DifferenceZonedDateTimeWithRounding. +pub fn difference_zoned_date_time_with_rounding( + providers: &TemporalProviders<'_>, + start_ns: i128, + end_ns: i128, + time_zone: &TimeZoneId, + calendar: &CalendarId, + rounding: DifferenceRounding, +) -> TemporalResult { + if rounding.largest_unit.category() == UnitCategory::Time { + return difference_instant( + start_ns, + end_ns, + rounding.increment, + rounding.smallest_unit, + rounding.mode, + ); + } + let difference = difference_zoned_date_time( + providers, + start_ns, + end_ns, + time_zone, + calendar, + rounding.largest_unit, + )?; + if rounding.smallest_unit == Unit::Nanosecond && rounding.increment == 1 { + return Ok(difference); + } + let origin = RelativeOrigin { + providers, + date_time: iso_date_time_for_instant(providers.time_zone, time_zone, start_ns)?, + time_zone: Some(time_zone), + calendar, + }; + let endpoints = Endpoints { + origin_epoch_ns: start_ns, + dest_epoch_ns: end_ns, + }; + round_relative_duration(difference, endpoints, &origin, rounding) +} + +/// DifferenceZonedDateTimeWithTotal. +pub fn difference_zoned_date_time_with_total( + providers: &TemporalProviders<'_>, + start_ns: i128, + end_ns: i128, + time_zone: &TimeZoneId, + calendar: &CalendarId, + unit: Unit, +) -> TemporalResult { + if unit.category() == UnitCategory::Time { + return total_time_duration(end_ns - start_ns, unit); + } + let difference = + difference_zoned_date_time(providers, start_ns, end_ns, time_zone, calendar, unit)?; + let origin = RelativeOrigin { + providers, + date_time: iso_date_time_for_instant(providers.time_zone, time_zone, start_ns)?, + time_zone: Some(time_zone), + calendar, + }; + let endpoints = Endpoints { + origin_epoch_ns: start_ns, + dest_epoch_ns: end_ns, + }; + total_relative_duration(difference, endpoints, &origin, unit) +} + +/// The two ends of a duration laid against a plain `relativeTo`: midnight of +/// that date, and the wall-clock time the whole duration reaches. +pub fn plain_relative_endpoints( + providers: &TemporalProviders<'_>, + date: PlainDate, + calendar: &CalendarId, + duration: &Duration, +) -> TemporalResult<(PlainDateTime, PlainDateTime)> { + let internal = InternalDuration::from_duration_with_24_hour_days(duration); + let (carry, time) = PlainTime::midnight().add_signed_nanoseconds(internal.time); + let mut date_duration = internal.date; + date_duration.days = carry; + let date_duration = Duration::new(date_duration.fields())?; + let target = + providers + .calendar + .date_add(calendar, date, &date_duration, Overflow::Constrain)?; + Ok((midnight_record(date), PlainDateTime { date: target, time })) +} + +/// DateDurationDays: a date duration's whole length in days from `date`. +pub fn date_duration_days( + providers: &TemporalProviders<'_>, + duration: &Duration, + date: PlainDate, + calendar: &CalendarId, +) -> TemporalResult { + let without_days = Duration::from_date(duration.years, duration.months, duration.weeks, 0)?; + if without_days.sign() == 0 { + return Ok(duration.days); + } + let later = providers + .calendar + .date_add(calendar, date, &without_days, Overflow::Constrain)?; + Ok(duration.days + i128::from(later.epoch_days() - date.epoch_days())) +} + +/// CompareTimeDuration after folding `days` into each side as 24-hour days. +pub fn compare_with_days( + one: &InternalDuration, + days_one: i128, + two: &InternalDuration, + days_two: i128, +) -> TemporalResult { + Ok(add_24_hour_days(one.time, days_one)?.cmp(&add_24_hour_days(two.time, days_two)?)) +} + +/// AddInstant. +pub fn add_instant(epoch_ns: i128, time: i128) -> TemporalResult { + Instant::from_epoch_nanoseconds(epoch_ns + time) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::temporal::zoned_date_time::IsoCalendarProvider; + + fn date(year: i32, month: i64, day: i64) -> PlainDate { + PlainDate::new(year, month, day, Overflow::Reject).unwrap() + } + + #[test] + fn date_until_walks_from_the_receiver_in_both_directions() { + let start = date(2019, 2, 25); + let end = date(2020, 3, 10); + let forward = iso_date_until(start, end, Unit::Year).unwrap(); + assert_eq!((forward.years, forward.months, forward.days), (1, 0, 14)); + let backward = iso_date_until(end, start, Unit::Year).unwrap(); + assert_eq!( + (backward.years, backward.months, backward.days), + (-1, 0, -13) + ); + } + + #[test] + fn date_until_compares_the_unclamped_day_of_month() { + // January 31 plus one month is not yet past February 28: the unclamped + // day 31 is, so the month does not count. + let until = iso_date_until(date(2021, 1, 31), date(2021, 2, 28), Unit::Month).unwrap(); + assert_eq!((until.months, until.days), (0, 28)); + } + + #[test] + fn date_until_reports_weeks_only_when_asked() { + let until = iso_date_until(date(2020, 1, 1), date(2020, 1, 17), Unit::Week).unwrap(); + assert_eq!((until.weeks, until.days), (2, 2)); + let days = iso_date_until(date(2020, 1, 1), date(2020, 1, 17), Unit::Day).unwrap(); + assert_eq!((days.weeks, days.days), (0, 16)); + } + + #[test] + fn time_duration_sum_rejects_exactly_two_to_the_53_seconds() { + assert!(add_time_duration(MAX_TIME_DURATION - 1, 0).is_ok()); + assert!(add_time_duration(MAX_TIME_DURATION - 1, 1).is_err()); + assert!(add_time_duration(-MAX_TIME_DURATION, 0).is_err()); + } + + #[test] + fn date_duration_without_time_truncates_toward_zero() { + let negative = Duration::new([0, 0, 0, -1, -25, 0, 0, 0, 0, 0]).unwrap(); + assert_eq!(date_duration_without_time(&negative).unwrap().days, -2); + } + + #[test] + fn calendar_rounding_rounds_on_the_exact_position_in_the_window() { + let provider = IsoCalendarProvider; + let zones = crate::temporal::zoned_date_time::FixedOffsetTimeZones; + let providers = TemporalProviders::new(&zones, &provider); + let calendar = CalendarId::iso8601(); + let rounding = DifferenceRounding { + largest_unit: Unit::Year, + smallest_unit: Unit::Month, + increment: 1, + mode: RoundingMode::HalfExpand, + }; + // 15 days into a 31-day January rounds down, 16 rounds up. + let down = difference_plain_date( + &providers, + date(2021, 1, 1), + date(2021, 1, 16), + &calendar, + rounding, + ) + .unwrap(); + assert_eq!((down.months, down.days), (0, 0)); + let up = difference_plain_date( + &providers, + date(2021, 1, 1), + date(2021, 1, 17), + &calendar, + rounding, + ) + .unwrap(); + assert_eq!((up.months, up.days), (1, 0)); + } +} diff --git a/crates/bamts-runtime/src/temporal/zoned_date_time.rs b/crates/bamts-runtime/src/temporal/zoned_date_time.rs index 67d85845..a773f942 100644 --- a/crates/bamts-runtime/src/temporal/zoned_date_time.rs +++ b/crates/bamts-runtime/src/temporal/zoned_date_time.rs @@ -7,15 +7,22 @@ use std::{fmt, sync::Arc}; +use super::calendar::canonical_calendar; use super::instant_duration::{ - Duration, INSTANT_NS_MAX, INSTANT_NS_MIN, Instant, NS_PER_DAY, NS_PER_HOUR, - NS_PER_MICROSECOND, NS_PER_MILLISECOND, NS_PER_MINUTE, NS_PER_SECOND, Overflow, - Precision, RoundingMode, TemporalError, TemporalResult, Unit, UnitCategory, - epoch_days_from_ymd, format_iso_date, format_time_ns, round_to_increment, + Duration, INSTANT_NS_MAX, INSTANT_NS_MIN, Instant, NS_PER_DAY, NS_PER_HOUR, NS_PER_MICROSECOND, + NS_PER_MILLISECOND, NS_PER_MINUTE, NS_PER_SECOND, Overflow, Precision, RoundingMode, + TemporalError, TemporalResult, Unit, UnitCategory, epoch_days_from_ymd, round_to_increment, validate_rounding_increment, ymd_from_epoch_days, }; -use super::plain_types::{ - PlainDate, PlainDateTime, PlainTime, iso_day_of_week, iso_day_of_year, iso_week_of_year, +use super::now_rounding::{ + ShowCalendar, ShowOffset, ShowTimeZone, ZonedSerialization, round_epoch_nanoseconds, + serialize_zoned_date_time, +}; +use super::parse::{ParsedTimeZone, parse_time_zone_identifier}; +use super::plain_types::{PlainDate, PlainDateTime, PlainTime}; +use super::relative::{ + DifferenceRounding, InternalDuration, add_instant, check_iso_days_range, + difference_zoned_date_time_with_rounding, iso_date_until, temporal_duration_from_internal, }; /// A canonical time-zone identifier. Canonicalization and availability are a @@ -28,9 +35,9 @@ impl TimeZoneId { pub fn new(identifier: impl AsRef) -> TemporalResult { let identifier = identifier.as_ref(); if identifier.is_empty() - || identifier - .bytes() - .any(|byte| byte.is_ascii_control() || byte.is_ascii_whitespace() || matches!(byte, b'[' | b']')) + || identifier.bytes().any(|byte| { + byte.is_ascii_control() || byte.is_ascii_whitespace() || matches!(byte, b'[' | b']') + }) { return Err(TemporalError::range("invalid time-zone identifier")); } @@ -41,11 +48,35 @@ impl TimeZoneId { Self(Arc::from("UTC")) } + /// Resolves a parsed time zone to the identifier the provider serves: + /// FormatOffsetTimeZoneIdentifier for an offset, and + /// GetAvailableNamedTimeZoneIdentifier for a name, of which only `UTC` is + /// available without a time zone database. + pub fn from_parsed(parsed: ParsedTimeZone) -> TemporalResult { + match parsed { + ParsedTimeZone::OffsetMinutes(minutes) => { + let sign = if minutes < 0 { '-' } else { '+' }; + let absolute = minutes.unsigned_abs(); + Self::new(format!("{sign}{:02}:{:02}", absolute / 60, absolute % 60)) + } + ParsedTimeZone::Named(name) if name.eq_ignore_ascii_case("UTC") => Ok(Self::utc()), + ParsedTimeZone::Named(name) => Err(TemporalError::range(format!( + "time zone {name} is not available" + ))), + } + } + pub fn as_str(&self) -> &str { &self.0 } } +impl AsRef for TimeZoneId { + fn as_ref(&self) -> &str { + self.as_str() + } +} + impl fmt::Display for TimeZoneId { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.write_str(self.as_str()) @@ -58,22 +89,17 @@ impl fmt::Display for TimeZoneId { pub struct CalendarId(Arc); impl CalendarId { - pub fn new(identifier: impl AsRef) -> TemporalResult { - let identifier = identifier.as_ref(); - if identifier.is_empty() - || !identifier - .bytes() - .all(|byte| byte.is_ascii_alphanumeric() || byte == b'-') - { - return Err(TemporalError::range("invalid calendar identifier")); - } - Ok(Self(Arc::from(identifier.to_ascii_lowercase()))) - } - pub fn iso8601() -> Self { Self(Arc::from("iso8601")) } + /// CanonicalizeCalendar. + pub fn canonicalize(identifier: &str) -> TemporalResult { + canonical_calendar(identifier) + .map(|canonical| Self(Arc::from(canonical))) + .ok_or_else(|| TemporalError::range(format!("calendar {identifier} is not supported"))) + } + pub fn as_str(&self) -> &str { &self.0 } @@ -83,6 +109,12 @@ impl CalendarId { } } +impl AsRef for CalendarId { + fn as_ref(&self) -> &str { + self.as_str() + } +} + impl fmt::Display for CalendarId { fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result { f.write_str(self.as_str()) @@ -97,7 +129,8 @@ impl UtcOffsetNanoseconds { pub const UTC: Self = Self(0); pub fn new(nanoseconds: i64) -> TemporalResult { - if !(-(NS_PER_DAY as i64)..NS_PER_DAY as i64).contains(&nanoseconds) { + let day = NS_PER_DAY as i64; + if nanoseconds <= -day || nanoseconds >= day { return Err(TemporalError::range( "time-zone offset must be strictly between -24h and +24h", )); @@ -190,6 +223,59 @@ pub trait TimeZoneProvider { ) -> TemporalResult>; } +/// The zones available without a time zone database: `UTC` and the fixed +/// offsets, each of which has one offset for all time and no transitions. +#[derive(Debug, Clone, Copy, Default)] +pub struct FixedOffsetTimeZones; + +impl FixedOffsetTimeZones { + /// The constant offset of an identifier this provider produced. + fn offset_of(time_zone: &TimeZoneId) -> TemporalResult { + if time_zone.as_str() == "UTC" { + return Ok(UtcOffsetNanoseconds::UTC); + } + match parse_time_zone_identifier(time_zone.as_str())? { + ParsedTimeZone::OffsetMinutes(minutes) => { + UtcOffsetNanoseconds::new(i64::from(minutes) * NS_PER_MINUTE as i64) + } + ParsedTimeZone::Named(name) => Err(TemporalError::range(format!( + "time zone {name} requires an unavailable time zone database" + ))), + } + } +} + +impl TimeZoneProvider for FixedOffsetTimeZones { + fn possible_instants_for( + &self, + time_zone: &TimeZoneId, + local: PlainDateTime, + ) -> TemporalResult> { + let offset = Self::offset_of(time_zone)?; + let instant = Instant::from_epoch_nanoseconds( + local_epoch_nanoseconds(local) - i128::from(offset.get()), + )?; + Ok(vec![instant]) + } + + fn offset_nanoseconds_for( + &self, + time_zone: &TimeZoneId, + _instant: Instant, + ) -> TemporalResult { + Self::offset_of(time_zone) + } + + fn transition( + &self, + time_zone: &TimeZoneId, + _instant: Instant, + _direction: TransitionDirection, + ) -> TemporalResult> { + Self::offset_of(time_zone).map(|_| None) + } +} + #[derive(Debug, Clone, PartialEq, Eq)] pub struct CalendarDateFields { pub era: Option>, @@ -219,16 +305,6 @@ pub trait CalendarProvider { iso_date: PlainDate, ) -> TemporalResult; - fn date_from_fields( - &self, - calendar: &CalendarId, - current: PlainDate, - year: Option, - month: Option, - day: Option, - overflow: Overflow, - ) -> TemporalResult; - fn date_add( &self, calendar: &CalendarId, @@ -269,7 +345,7 @@ impl CalendarProvider for IsoCalendarProvider { iso_date: PlainDate, ) -> TemporalResult { Self::require_iso(calendar)?; - let (year_of_week, week_of_year) = iso_week_of_year(iso_date.year, iso_date.month, iso_date.day); + let (year_of_week, week_of_year) = iso_date.week_of_year(); Ok(CalendarDateFields { era: None, era_year: None, @@ -277,31 +353,18 @@ impl CalendarProvider for IsoCalendarProvider { month: iso_date.month, month_code: Arc::from(format!("M{:02}", iso_date.month)), day: iso_date.day, - day_of_week: iso_day_of_week(iso_date.year, iso_date.month, iso_date.day), - day_of_year: iso_day_of_year(iso_date.year, iso_date.month, iso_date.day), + day_of_week: iso_date.day_of_week(), + day_of_year: iso_date.day_of_year(), week_of_year: Some(week_of_year), year_of_week: Some(year_of_week), days_in_week: 7, days_in_month: iso_date.days_in_month(), days_in_year: iso_date.days_in_year(), - months_in_year: 12, + months_in_year: iso_date.months_in_year(), in_leap_year: iso_date.in_leap_year(), }) } - fn date_from_fields( - &self, - calendar: &CalendarId, - current: PlainDate, - year: Option, - month: Option, - day: Option, - overflow: Overflow, - ) -> TemporalResult { - Self::require_iso(calendar)?; - current.with(year, month, day, overflow) - } - fn date_add( &self, calendar: &CalendarId, @@ -321,14 +384,7 @@ impl CalendarProvider for IsoCalendarProvider { largest_unit: Unit, ) -> TemporalResult { Self::require_iso(calendar)?; - one.until( - two, - largest_unit, - Unit::Day, - 1, - RoundingMode::Trunc, - Overflow::Constrain, - ) + iso_date_until(one, two, largest_unit) } } @@ -341,11 +397,11 @@ pub struct TemporalProviders<'a> { } impl<'a> TemporalProviders<'a> { - pub fn new( - time_zone: &'a dyn TimeZoneProvider, - calendar: &'a dyn CalendarProvider, - ) -> Self { - Self { time_zone, calendar } + pub fn new(time_zone: &'a dyn TimeZoneProvider, calendar: &'a dyn CalendarProvider) -> Self { + Self { + time_zone, + calendar, + } } } @@ -367,33 +423,18 @@ impl Default for ResolveOptions { } #[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub struct WithOptions { - pub disambiguation: Disambiguation, - pub offset: OffsetOption, +pub struct ArithmeticOptions { pub overflow: Overflow, } -impl Default for WithOptions { +impl Default for ArithmeticOptions { fn default() -> Self { Self { - disambiguation: Disambiguation::Compatible, - offset: OffsetOption::Prefer, overflow: Overflow::Constrain, } } } -#[derive(Debug, Clone, Copy, PartialEq, Eq)] -pub struct ArithmeticOptions { - pub overflow: Overflow, -} - -impl Default for ArithmeticOptions { - fn default() -> Self { - Self { overflow: Overflow::Constrain } - } -} - #[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct DifferenceOptions { pub largest_unit: Unit, @@ -414,6 +455,15 @@ impl Default for DifferenceOptions { } impl DifferenceOptions { + fn rounding(self) -> DifferenceRounding { + DifferenceRounding { + largest_unit: self.largest_unit, + smallest_unit: self.smallest_unit, + increment: self.rounding_increment, + mode: self.rounding_mode, + } + } + fn validate(self) -> TemporalResult<()> { if self.smallest_unit < self.largest_unit { return Err(TemporalError::range( @@ -424,11 +474,7 @@ impl DifferenceOptions { return Err(TemporalError::range("roundingIncrement must be >= 1")); } if self.smallest_unit.category() == UnitCategory::Time { - validate_rounding_increment( - self.smallest_unit, - self.rounding_increment, - false, - )?; + validate_rounding_increment(self.smallest_unit, self.rounding_increment, false)?; } Ok(()) } @@ -451,35 +497,6 @@ impl Default for RoundOptions { } } -#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] -pub struct ZonedDateTimeFields { - pub year: Option, - pub month: Option, - pub day: Option, - pub hour: Option, - pub minute: Option, - pub second: Option, - pub millisecond: Option, - pub microsecond: Option, - pub nanosecond: Option, - pub offset: Option, -} - -impl ZonedDateTimeFields { - fn has_any(self) -> bool { - self.year.is_some() - || self.month.is_some() - || self.day.is_some() - || self.hour.is_some() - || self.minute.is_some() - || self.second.is_some() - || self.millisecond.is_some() - || self.microsecond.is_some() - || self.nanosecond.is_some() - || self.offset.is_some() - } -} - #[derive(Debug, Clone, PartialEq, Eq)] pub struct ZonedFields { pub date_time: PlainDateTime, @@ -533,12 +550,12 @@ impl ZonedDateTime { }) } - pub fn from_instant( - instant: Instant, - time_zone: TimeZoneId, - calendar: CalendarId, - ) -> Self { - Self { instant, time_zone, calendar } + pub fn from_instant(instant: Instant, time_zone: TimeZoneId, calendar: CalendarId) -> Self { + Self { + instant, + time_zone, + calendar, + } } pub fn from_local( @@ -585,7 +602,11 @@ impl ZonedDateTime { .offset_nanoseconds_for(&self.time_zone, self.instant)?; let date_time = plain_date_time_for(self.instant, offset)?; let calendar = providers.calendar.fields(&self.calendar, date_time.date)?; - Ok(ZonedFields { date_time, calendar, offset }) + Ok(ZonedFields { + date_time, + calendar, + offset, + }) } pub fn to_plain_date_time( @@ -603,63 +624,6 @@ impl ZonedDateTime { Ok(self.to_plain_date_time(time_zone)?.time) } - pub fn with( - &self, - providers: &TemporalProviders<'_>, - replacements: ZonedDateTimeFields, - options: WithOptions, - ) -> TemporalResult { - if !replacements.has_any() { - return Err(TemporalError::type_error( - "ZonedDateTime.with requires at least one field", - )); - } - let current_offset = providers - .time_zone - .offset_nanoseconds_for(&self.time_zone, self.instant)?; - let current = plain_date_time_for(self.instant, current_offset)?; - let date = providers.calendar.date_from_fields( - &self.calendar, - current.date, - replacements.year, - replacements.month, - replacements.day, - options.overflow, - )?; - let time = PlainTime::new( - replacements.hour.unwrap_or(i64::from(current.time.hour)), - replacements.minute.unwrap_or(i64::from(current.time.minute)), - replacements.second.unwrap_or(i64::from(current.time.second)), - replacements - .millisecond - .unwrap_or(i64::from(current.time.millisecond)), - replacements - .microsecond - .unwrap_or(i64::from(current.time.microsecond)), - replacements - .nanosecond - .unwrap_or(i64::from(current.time.nanosecond)), - options.overflow, - )?; - let local = PlainDateTime::from_parts(date, time)?; - let instant = resolve_local( - providers.time_zone, - &self.time_zone, - local, - Some(replacements.offset.unwrap_or(current_offset)), - ResolveOptions { - disambiguation: options.disambiguation, - offset: options.offset, - offset_match: OffsetMatch::Exactly, - }, - )?; - Ok(Self::from_instant( - instant, - self.time_zone.clone(), - self.calendar.clone(), - )) - } - pub fn with_time_zone(&self, time_zone: TimeZoneId) -> Self { Self::from_instant(self.instant, time_zone, self.calendar.clone()) } @@ -674,55 +638,14 @@ impl ZonedDateTime { duration: &Duration, options: ArithmeticOptions, ) -> TemporalResult { - if duration.years == 0 - && duration.months == 0 - && duration.weeks == 0 - && duration.days == 0 - { - let instant = self.instant.add(duration)?; - return Ok(Self::from_instant( - instant, - self.time_zone.clone(), - self.calendar.clone(), - )); - } - - let local = self.to_plain_date_time(providers.time_zone)?; - let date_duration = Duration::new( - duration.years, - duration.months, - duration.weeks, - duration.days, - 0, - 0, - 0, - 0, - 0, - 0, - )?; - let added_date = providers.calendar.date_add( + let instant = add_zoned_date_time( + providers, + self.instant, + &self.time_zone, &self.calendar, - local.date, - &date_duration, + InternalDuration::from_duration(duration), options.overflow, )?; - let intermediate_local = PlainDateTime::from_parts(added_date, local.time)?; - let intermediate = resolve_local( - providers.time_zone, - &self.time_zone, - intermediate_local, - None, - ResolveOptions { - disambiguation: Disambiguation::Compatible, - offset: OffsetOption::Ignore, - offset_match: OffsetMatch::Exactly, - }, - )?; - let time_duration = Duration::from_time_nanoseconds( - duration.time_total_nanoseconds(), - Unit::Hour, - )?; - let instant = intermediate.add(&time_duration)?; Ok(Self::from_instant( instant, self.time_zone.clone(), @@ -739,6 +662,7 @@ impl ZonedDateTime { self.add(providers, &duration.negated(), options) } + /// DifferenceTemporalZonedDateTime with `until` direction. pub fn until( &self, providers: &TemporalProviders<'_>, @@ -751,32 +675,45 @@ impl ZonedDateTime { "ZonedDateTime difference requires equal calendars", )); } - if options.largest_unit.category() == UnitCategory::Time { - return self.instant.until( - other.instant, - options.largest_unit, - options.smallest_unit, - options.rounding_increment, - options.rounding_mode, - ); - } - if self.time_zone != other.time_zone { + if options.largest_unit.category() == UnitCategory::Date + && self.time_zone != other.time_zone + { return Err(TemporalError::range( "date-unit ZonedDateTime difference requires equal time zones", )); } - - let unrounded = difference_zoned_date_units(providers, self, other, options.largest_unit)?; - round_zoned_difference(providers, self, other, unrounded, options) + if options.largest_unit.category() == UnitCategory::Date && self.instant == other.instant { + return Ok(Duration::default()); + } + let internal = difference_zoned_date_time_with_rounding( + providers, + self.instant.epoch_nanoseconds(), + other.instant.epoch_nanoseconds(), + &self.time_zone, + &self.calendar, + options.rounding(), + )?; + let balance_to = if options.largest_unit.category() == UnitCategory::Date { + Unit::Hour + } else { + options.largest_unit + }; + temporal_duration_from_internal(internal, balance_to) } + /// `since` is `until` from the same receiver, negated, with the rounding + /// direction negated to match. pub fn since( &self, providers: &TemporalProviders<'_>, other: &Self, options: DifferenceOptions, ) -> TemporalResult { - other.until(providers, self, options) + let negated = DifferenceOptions { + rounding_mode: options.rounding_mode.negated(), + ..options + }; + Ok(self.until(providers, other, negated)?.negated()) } pub fn round( @@ -802,11 +739,7 @@ impl ZonedDateTime { return self.round_to_day(providers, options.rounding_mode); } - validate_rounding_increment( - options.smallest_unit, - options.rounding_increment, - false, - )?; + validate_rounding_increment(options.smallest_unit, options.rounding_increment, false)?; let current_offset = providers .time_zone .offset_nanoseconds_for(&self.time_zone, self.instant)?; @@ -841,7 +774,7 @@ impl ZonedDateTime { ) -> TemporalResult { let local = self.to_plain_date_time(providers.time_zone)?; let start = start_of_day(providers.time_zone, &self.time_zone, local.date)?; - let one_day = Duration::new(0, 0, 0, 1, 0, 0, 0, 0, 0, 0)?; + let one_day = Duration::new([0, 0, 0, 1, 0, 0, 0, 0, 0, 0])?; let tomorrow = providers.calendar.date_add( &self.calendar, local.date, @@ -856,18 +789,16 @@ impl ZonedDateTime { "provider returned a non-positive or inconsistent calendar day", )); } - let choose_end = choose_upper_boundary(progress, length, 0, mode); + let rounded = round_to_increment(progress, length, mode)?; + let instant = Instant::from_epoch_nanoseconds(start.epoch_nanoseconds() + rounded)?; Ok(Self::from_instant( - if choose_end { end } else { start }, + instant, self.time_zone.clone(), self.calendar.clone(), )) } - pub fn start_of_day( - &self, - providers: &TemporalProviders<'_>, - ) -> TemporalResult { + pub fn start_of_day(&self, providers: &TemporalProviders<'_>) -> TemporalResult { let date = self.to_plain_date(providers.time_zone)?; let instant = start_of_day(providers.time_zone, &self.time_zone, date)?; Ok(Self::from_instant( @@ -883,7 +814,7 @@ impl ZonedDateTime { ) -> TemporalResult { let local = self.to_plain_date_time(providers.time_zone)?; let start = start_of_day(providers.time_zone, &self.time_zone, local.date)?; - let one_day = Duration::new(0, 0, 0, 1, 0, 0, 0, 0, 0, 0)?; + let one_day = Duration::new([0, 0, 0, 1, 0, 0, 0, 0, 0, 0])?; let tomorrow = providers.calendar.date_add( &self.calendar, local.date, @@ -893,7 +824,9 @@ impl ZonedDateTime { let end = start_of_day(providers.time_zone, &self.time_zone, tomorrow)?; let length = end.epoch_nanoseconds() - start.epoch_nanoseconds(); if length <= 0 { - return Err(TemporalError::range("provider returned a non-positive calendar day")); + return Err(TemporalError::range( + "provider returned a non-positive calendar day", + )); } Ok(length) } @@ -903,7 +836,8 @@ impl ZonedDateTime { provider: &dyn TimeZoneProvider, direction: TransitionDirection, ) -> TemporalResult> { - let Some(transition) = provider.transition(&self.time_zone, self.instant, direction)? else { + let Some(transition) = provider.transition(&self.time_zone, self.instant, direction)? + else { return Ok(None); }; let valid_direction = match direction { @@ -942,150 +876,111 @@ impl ZonedDateTime { one.instant.cmp(&two.instant) } + /// TemporalZonedDateTimeToString. Rounding applies to the exact time, and + /// the offset is read after it, so a rounded time can never show an offset + /// that belongs to a moment on the other side of a transition. pub fn format( &self, providers: &TemporalProviders<'_>, options: ToStringOptions, ) -> TemporalResult { - let rounded = if let Some(round) = options.round { - self.round(providers, round)? - } else { - self.clone() + let rounded = match options.round { + Some(round) => Self::new( + round_epoch_nanoseconds( + self.epoch_nanoseconds(), + round.smallest_unit, + round.rounding_increment, + round.rounding_mode, + )?, + self.time_zone.clone(), + self.calendar.clone(), + )?, + None => self.clone(), }; let fields = rounded.fields(providers)?; - let mut result = format_iso_date( - fields.date_time.date.year, - fields.date_time.date.month, - fields.date_time.date.day, - ); - result.push('T'); - result.push_str(&format_time_ns( - fields.date_time.time.to_nanoseconds_of_day(), - options.precision, - )); - if options.show_offset == ShowOffset::Auto { - result.push_str(&format_utc_offset(fields.offset)); - } - match options.show_time_zone { - ShowAnnotation::Never => {} - ShowAnnotation::Critical => { - result.push_str("[!"); - result.push_str(rounded.time_zone.as_str()); - result.push(']'); - } - ShowAnnotation::Auto | ShowAnnotation::Always => { - result.push('['); - result.push_str(rounded.time_zone.as_str()); - result.push(']'); - } - } - let show_calendar = match options.show_calendar { - ShowAnnotation::Never => false, - ShowAnnotation::Auto => !rounded.calendar.is_iso8601(), - ShowAnnotation::Always | ShowAnnotation::Critical => true, - }; - if show_calendar { - result.push('['); - if options.show_calendar == ShowAnnotation::Critical { - result.push('!'); - } - result.push_str("u-ca="); - result.push_str(rounded.calendar.as_str()); - result.push(']'); - } - Ok(result) + serialize_zoned_date_time(ZonedSerialization { + iso_date_time: &fields.date_time, + offset: fields.offset, + time_zone: rounded.time_zone.as_str(), + calendar: rounded.calendar.as_str(), + precision: options.precision, + show_offset: options.show_offset, + show_time_zone: options.show_time_zone, + show_calendar: options.show_calendar, + }) } pub fn to_json(&self, providers: &TemporalProviders<'_>) -> TemporalResult { self.format(providers, ToStringOptions::default()) } - - pub fn to_json_with( - &self, - providers: &TemporalProviders<'_>, - adapter: &A, - ) -> TemporalResult { - adapter.from_temporal_string(self.to_json(providers)?) - } - - pub fn to_locale_string( - &self, - providers: &TemporalProviders<'_>, - adapter: &dyn ZonedDateTimeLocaleAdapter, - locales: &[&str], - options: &LocaleFormatOptions, - ) -> TemporalResult { - let fields = self.fields(providers)?; - adapter.format(self, &fields, locales, options) - } -} - -#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] -pub enum ShowAnnotation { - #[default] - Auto, - Always, - Never, - Critical, } #[derive(Debug, Clone, Copy, PartialEq, Eq, Default)] -pub enum ShowOffset { - #[default] - Auto, - Never, -} - -#[derive(Debug, Clone, Copy, PartialEq, Eq)] pub struct ToStringOptions { pub precision: Precision, - pub show_calendar: ShowAnnotation, - pub show_time_zone: ShowAnnotation, + pub show_calendar: ShowCalendar, + pub show_time_zone: ShowTimeZone, pub show_offset: ShowOffset, pub round: Option, } -impl Default for ToStringOptions { - fn default() -> Self { - Self { - precision: Precision::Auto, - show_calendar: ShowAnnotation::Auto, - show_time_zone: ShowAnnotation::Auto, - show_offset: ShowOffset::Auto, - round: None, - } - } -} - -#[derive(Debug, Clone, PartialEq, Eq, Default)] -pub struct LocaleFormatOptions { - pub calendar: Option, - pub numbering_system: Option>, - pub date_style: Option>, - pub time_style: Option>, +/// AddZonedDateTime: the date part moves in wall-clock terms and the time part +/// in exact time, in that order. +pub fn add_zoned_date_time( + providers: &TemporalProviders<'_>, + instant: Instant, + time_zone: &TimeZoneId, + calendar: &CalendarId, + duration: InternalDuration, + overflow: Overflow, +) -> TemporalResult { + if duration.date.sign() == 0 { + return add_instant(instant.epoch_nanoseconds(), duration.time); + } + let local = local_date_time(providers.time_zone, time_zone, instant)?; + let added = providers + .calendar + .date_add(calendar, local.date, &duration.date, overflow)?; + let intermediate = PlainDateTime::from_parts(added, local.time)?; + let intermediate_ns = epoch_nanoseconds_for(providers.time_zone, time_zone, intermediate)?; + add_instant(intermediate_ns, duration.time) } -/// ECMA-402 adapter boundary. The core resolves fields exactly once before -/// calling the adapter, so locale data cannot change Temporal arithmetic. -pub trait ZonedDateTimeLocaleAdapter { - fn format( - &self, - value: &ZonedDateTime, - fields: &ZonedFields, - locales: &[&str], - options: &LocaleFormatOptions, - ) -> TemporalResult; +/// GetEpochNanosecondsFor with `compatible` disambiguation. +pub fn epoch_nanoseconds_for( + provider: &dyn TimeZoneProvider, + time_zone: &TimeZoneId, + local: PlainDateTime, +) -> TemporalResult { + let instant = resolve_local( + provider, + time_zone, + local, + None, + ResolveOptions { + disambiguation: Disambiguation::Compatible, + offset: OffsetOption::Ignore, + offset_match: OffsetMatch::Exactly, + }, + )?; + Ok(instant.epoch_nanoseconds()) } -/// Runtime JSON adapter boundary for converting the normative Temporal string -/// into the host runtime's string/value representation. -pub trait ZonedDateTimeJsonAdapter { - type Output; - - fn from_temporal_string(&self, value: String) -> TemporalResult; +/// GetISODateTimeFor. +pub fn iso_date_time_for_instant( + provider: &dyn TimeZoneProvider, + time_zone: &TimeZoneId, + epoch_nanoseconds: i128, +) -> TemporalResult { + local_date_time( + provider, + time_zone, + Instant::from_epoch_nanoseconds(epoch_nanoseconds)?, + ) } -fn local_epoch_nanoseconds(local: PlainDateTime) -> i128 { +/// GetUTCEpochNanoseconds: the wall time read as if it were UTC. +pub fn local_epoch_nanoseconds(local: PlainDateTime) -> i128 { i128::from(epoch_days_from_ymd( local.date.year, local.date.month, @@ -1168,14 +1063,17 @@ fn resolve_local( supplied_offset: Option, options: ResolveOptions, ) -> TemporalResult { - if let Some(offset) = supplied_offset { - if options.offset == OffsetOption::Use { - return Instant::from_epoch_nanoseconds( - local_epoch_nanoseconds(local) - i128::from(offset.get()), - ); - } + if let Some(offset) = supplied_offset + && options.offset == OffsetOption::Use + { + return Instant::from_epoch_nanoseconds( + local_epoch_nanoseconds(local) - i128::from(offset.get()), + ); } + if supplied_offset.is_some() && options.offset != OffsetOption::Ignore { + check_iso_days_range(local.date)?; + } let possible = possible_instants_checked(provider, time_zone, local)?; if let Some(supplied) = supplied_offset && options.offset != OffsetOption::Ignore @@ -1183,16 +1081,14 @@ fn resolve_local( let naive = local_epoch_nanoseconds(local); for &candidate in &possible { let candidate_offset = naive - candidate.epoch_nanoseconds(); - let matches = match options.offset_match { - OffsetMatch::Exactly => candidate_offset == i128::from(supplied.get()), - OffsetMatch::Minutes => { - round_to_increment( + let supplied_ns = i128::from(supplied.get()); + let matches = candidate_offset == supplied_ns + || (options.offset_match == OffsetMatch::Minutes + && round_to_increment( candidate_offset, NS_PER_MINUTE, RoundingMode::HalfExpand, - )? == i128::from(supplied.get()) - } - }; + )? == supplied_ns); if matches { return Ok(candidate); } @@ -1238,10 +1134,14 @@ fn disambiguate_possible( // validation below rejects inconsistent provider data. let naive = local_epoch_nanoseconds(local); let before_probe = Instant::from_epoch_nanoseconds( - naive.saturating_sub(NS_PER_DAY).clamp(INSTANT_NS_MIN, INSTANT_NS_MAX), + naive + .saturating_sub(NS_PER_DAY) + .clamp(INSTANT_NS_MIN, INSTANT_NS_MAX), )?; let after_probe = Instant::from_epoch_nanoseconds( - naive.saturating_add(NS_PER_DAY).clamp(INSTANT_NS_MIN, INSTANT_NS_MAX), + naive + .saturating_add(NS_PER_DAY) + .clamp(INSTANT_NS_MIN, INSTANT_NS_MAX), )?; let offset_before = provider.offset_nanoseconds_for(time_zone, before_probe)?; let offset_after = provider.offset_nanoseconds_for(time_zone, after_probe)?; @@ -1272,7 +1172,8 @@ fn disambiguate_possible( } } -fn start_of_day( +/// GetStartOfDay. +pub fn start_of_day( provider: &dyn TimeZoneProvider, time_zone: &TimeZoneId, date: PlainDate, @@ -1291,228 +1192,19 @@ fn start_of_day( ) } -fn combine_duration(date: Duration, time_nanoseconds: i128) -> TemporalResult { - let time = Duration::from_time_nanoseconds(time_nanoseconds, Unit::Hour)?; - Duration::new( - date.years, - date.months, - date.weeks, - date.days, - time.hours, - time.minutes, - time.seconds, - time.milliseconds, - time.microseconds, - time.nanoseconds, - ) -} - -fn difference_zoned_date_units( - providers: &TemporalProviders<'_>, - start: &ZonedDateTime, - end: &ZonedDateTime, - largest_unit: Unit, -) -> TemporalResult { - if start.instant == end.instant { - return Ok(Duration::default()); - } - if end.instant < start.instant { - return Ok(difference_zoned_date_units(providers, end, start, largest_unit)?.negated()); - } - - let start_local = start.to_plain_date_time(providers.time_zone)?; - let end_local = end.to_plain_date_time(providers.time_zone)?; - if start_local.date == end_local.date { - return Duration::from_time_nanoseconds( - end.instant.epoch_nanoseconds() - start.instant.epoch_nanoseconds(), - Unit::Hour, - ); - } - - let wall_time_difference = end_local.time.to_nanoseconds_of_day() - - start_local.time.to_nanoseconds_of_day(); - let mut day_correction = if wall_time_difference < 0 { 1 } else { 0 }; - while day_correction <= 2 { - let correction = Duration::new(0, 0, 0, -day_correction, 0, 0, 0, 0, 0, 0)?; - let intermediate_date = providers.calendar.date_add( - &start.calendar, - end_local.date, - &correction, - Overflow::Constrain, - )?; - let intermediate_local = PlainDateTime::from_parts(intermediate_date, start_local.time)?; - let intermediate = resolve_local( - providers.time_zone, - &start.time_zone, - intermediate_local, - None, - ResolveOptions { - disambiguation: Disambiguation::Compatible, - offset: OffsetOption::Ignore, - offset_match: OffsetMatch::Exactly, - }, - )?; - let time_nanoseconds = end.instant.epoch_nanoseconds() - intermediate.epoch_nanoseconds(); - if time_nanoseconds >= 0 { - let date = providers.calendar.date_until( - &start.calendar, - start_local.date, - intermediate_date, - largest_unit.max(Unit::Day), - )?; - return combine_duration(date, time_nanoseconds); - } - day_correction += 1; - } - Err(TemporalError::range( - "time-zone provider prevented ZonedDateTime difference convergence", - )) -} - -fn round_zoned_difference( - providers: &TemporalProviders<'_>, - start: &ZonedDateTime, - end: &ZonedDateTime, - unrounded: Duration, - options: DifferenceOptions, -) -> TemporalResult { - if options.smallest_unit == Unit::Nanosecond && options.rounding_increment == 1 { - return Ok(unrounded); - } - if options.smallest_unit.category() == UnitCategory::Time { - let step = options - .smallest_unit - .ns_per() - .and_then(|unit| unit.checked_mul(options.rounding_increment)) - .ok_or_else(|| TemporalError::range("difference rounding increment overflow"))?; - let rounded_time = round_to_increment( - unrounded.time_total_nanoseconds(), - step, - options.rounding_mode, - )?; - let candidate_duration = combine_duration(unrounded, rounded_time)?; - let rounded_end = start.add( - providers, - &candidate_duration, - ArithmeticOptions::default(), - )?; - return difference_zoned_date_units( - providers, - start, - &rounded_end, - options.largest_unit, - ); - } - - let sign = if end.instant > start.instant { 1 } else { -1 }; - let whole = match options.smallest_unit { - Unit::Year => unrounded.years, - Unit::Month => unrounded.years * 12 + unrounded.months, - Unit::Week => unrounded.weeks, - Unit::Day => unrounded.days, - _ => unreachable!(), - }; - let increment = i64::try_from(options.rounding_increment) - .map_err(|_| TemporalError::range("calendar rounding increment overflow"))?; - let toward_count = whole / increment * increment; - let away_count = toward_count - .checked_add(sign * increment) - .ok_or_else(|| TemporalError::range("calendar rounding overflow"))?; - let toward_duration = duration_from_unit_count(options.smallest_unit, toward_count)?; - let away_duration = duration_from_unit_count(options.smallest_unit, away_count)?; - let toward = start.add(providers, &toward_duration, ArithmeticOptions::default())?; - let away = start.add(providers, &away_duration, ArithmeticOptions::default())?; - let toward_distance = (end.epoch_nanoseconds() - toward.epoch_nanoseconds()).abs(); - let away_distance = (away.epoch_nanoseconds() - end.epoch_nanoseconds()).abs(); - let choose_away = choose_away_boundary( - toward_distance, - away_distance, - i128::from(toward_count / increment), - sign, - options.rounding_mode, - ); - difference_zoned_date_units( - providers, - start, - if choose_away { &away } else { &toward }, - options.largest_unit, - ) -} - -fn duration_from_unit_count(unit: Unit, count: i64) -> TemporalResult { - match unit { - Unit::Year => Duration::new(count, 0, 0, 0, 0, 0, 0, 0, 0, 0), - Unit::Month => Duration::new(0, count, 0, 0, 0, 0, 0, 0, 0, 0), - Unit::Week => Duration::new(0, 0, count, 0, 0, 0, 0, 0, 0, 0), - Unit::Day => Duration::new(0, 0, 0, count, 0, 0, 0, 0, 0, 0), - _ => Err(TemporalError::range("calendar unit required")), - } -} - -fn choose_upper_boundary( - progress: i128, - length: i128, - lower_quotient: i128, - mode: RoundingMode, -) -> bool { - choose_away_boundary(progress, length - progress, lower_quotient, 1, mode) -} - -fn choose_away_boundary( - toward_distance: i128, - away_distance: i128, - toward_quotient: i128, - sign: i64, - mode: RoundingMode, -) -> bool { - match mode { - RoundingMode::Trunc => false, - RoundingMode::Expand => true, - RoundingMode::Ceil => sign > 0, - RoundingMode::Floor => sign < 0, - RoundingMode::HalfExpand - | RoundingMode::HalfTrunc - | RoundingMode::HalfCeil - | RoundingMode::HalfFloor - | RoundingMode::HalfEven => match toward_distance.cmp(&away_distance) { - std::cmp::Ordering::Less => false, - std::cmp::Ordering::Greater => true, - std::cmp::Ordering::Equal => match mode { - RoundingMode::HalfExpand => true, - RoundingMode::HalfTrunc => false, - RoundingMode::HalfCeil => sign > 0, - RoundingMode::HalfFloor => sign < 0, - RoundingMode::HalfEven => toward_quotient.rem_euclid(2) != 0, - _ => unreachable!(), - }, - }, - } -} - #[cfg(test)] mod tests { use std::cell::RefCell; - use super::*; use super::super::instant_duration::TemporalErrorKind; + use super::*; - fn local( - year: i32, - month: i64, - day: i64, - hour: i64, - minute: i64, - ) -> PlainDateTime { + fn local(year: i32, month: i64, day: i64, hour: i64, minute: i64) -> PlainDateTime { PlainDateTime::new( year, month, day, - hour, - minute, - 0, - 0, - 0, - 0, + [hour, minute, 0, 0, 0, 0], Overflow::Reject, ) .unwrap() @@ -1540,7 +1232,9 @@ mod tests { impl RecordingEastern { fn new() -> Self { - Self { calls: RefCell::new(Vec::new()) } + Self { + calls: RefCell::new(Vec::new()), + } } fn spring() -> i128 { @@ -1573,10 +1267,7 @@ mod tests { let candidate_ns = naive - i128::from(offset); if Self::raw_offset(candidate_ns) == offset { let candidate = Instant::from_epoch_nanoseconds(candidate_ns)?; - if plain_date_time_for( - candidate, - UtcOffsetNanoseconds::new(offset)?, - )? == local + if plain_date_time_for(candidate, UtcOffsetNanoseconds::new(offset)?)? == local { result.push(candidate); } @@ -1625,8 +1316,7 @@ mod tests { #[derive(Debug, Clone, PartialEq, Eq)] enum CalendarCall { Fields(PlainDate), - FromFields(PlainDate), - Add(PlainDate, [i64; 4]), + Add(PlainDate, [i128; 4]), Until(PlainDate, PlainDate, Unit), } @@ -1636,7 +1326,9 @@ mod tests { impl RecordingCalendar { fn new() -> Self { - Self { calls: RefCell::new(Vec::new()) } + Self { + calls: RefCell::new(Vec::new()), + } } } @@ -1650,21 +1342,6 @@ mod tests { IsoCalendarProvider.fields(calendar, iso_date) } - fn date_from_fields( - &self, - calendar: &CalendarId, - current: PlainDate, - year: Option, - month: Option, - day: Option, - overflow: Overflow, - ) -> TemporalResult { - self.calls.borrow_mut().push(CalendarCall::FromFields(current)); - IsoCalendarProvider.date_from_fields( - calendar, current, year, month, day, overflow, - ) - } - fn date_add( &self, calendar: &CalendarId, @@ -1674,7 +1351,12 @@ mod tests { ) -> TemporalResult { self.calls.borrow_mut().push(CalendarCall::Add( iso_date, - [duration.years, duration.months, duration.weeks, duration.days], + [ + duration.years, + duration.months, + duration.weeks, + duration.days, + ], )); IsoCalendarProvider.date_add(calendar, iso_date, duration, overflow) } @@ -1834,9 +1516,7 @@ mod tests { epoch(2024, 11, 3, 7, 30) ); assert_eq!( - resolve(mismatch, OffsetOption::Reject) - .unwrap_err() - .kind, + resolve(mismatch, OffsetOption::Reject).unwrap_err().kind, TemporalErrorKind::Range ); } @@ -1848,8 +1528,18 @@ mod tests { let providers = providers(&tz, &cal); let value = ZonedDateTime::new(-1, zone(), calendar()).unwrap(); let fields = value.fields(&providers).unwrap(); - assert_eq!((fields.calendar.year, fields.calendar.month, fields.calendar.day), (1969, 12, 31)); - assert_eq!((fields.hour(), fields.minute(), fields.second()), (18, 59, 59)); + assert_eq!( + ( + fields.calendar.year, + fields.calendar.month, + fields.calendar.day + ), + (1969, 12, 31) + ); + assert_eq!( + (fields.hour(), fields.minute(), fields.second()), + (18, 59, 59) + ); assert_eq!(fields.nanosecond(), 999); assert_eq!(value.epoch_milliseconds(), -1); } @@ -1865,8 +1555,8 @@ mod tests { Disambiguation::Compatible, ) .unwrap(); - let one_day = Duration::new(0, 0, 0, 1, 0, 0, 0, 0, 0, 0).unwrap(); - let hours_24 = Duration::new(0, 0, 0, 0, 24, 0, 0, 0, 0, 0).unwrap(); + let one_day = Duration::new([0, 0, 0, 1, 0, 0, 0, 0, 0, 0]).unwrap(); + let hours_24 = Duration::new([0, 0, 0, 0, 24, 0, 0, 0, 0, 0]).unwrap(); let by_date = start .add(&providers, &one_day, ArithmeticOptions::default()) .unwrap(); @@ -1881,8 +1571,14 @@ mod tests { by_time.epoch_nanoseconds() - start.epoch_nanoseconds(), 24 * NS_PER_HOUR ); - assert_eq!(by_date.to_plain_date_time(&tz).unwrap(), local(2024, 3, 10, 12, 0)); - assert_eq!(by_time.to_plain_date_time(&tz).unwrap(), local(2024, 3, 10, 13, 0)); + assert_eq!( + by_date.to_plain_date_time(&tz).unwrap(), + local(2024, 3, 10, 12, 0) + ); + assert_eq!( + by_time.to_plain_date_time(&tz).unwrap(), + local(2024, 3, 10, 13, 0) + ); } #[test] @@ -1925,7 +1621,18 @@ mod tests { assert_eq!(days.days, 1); assert_eq!(days.time_total_nanoseconds(), 0); assert_eq!(hours.hours, 23); - assert_eq!(end.since(&providers, &start, DifferenceOptions { largest_unit: Unit::Day, ..DifferenceOptions::default() }).unwrap(), days); + assert_eq!( + end.since( + &providers, + &start, + DifferenceOptions { + largest_unit: Unit::Day, + ..DifferenceOptions::default() + } + ) + .unwrap(), + days + ); } #[test] @@ -1949,7 +1656,10 @@ mod tests { }, ) .unwrap(); - assert_eq!(rounded.to_plain_date_time(&tz).unwrap(), local(2024, 3, 10, 3, 0)); + assert_eq!( + rounded.to_plain_date_time(&tz).unwrap(), + local(2024, 3, 10, 3, 0) + ); assert_eq!( value.day_length_nanoseconds(&providers).unwrap(), 23 * NS_PER_HOUR @@ -2008,8 +1718,8 @@ mod tests { #[test] fn protocol_errors_propagate_without_reclassification() { - let calendar = RecordingCalendar::new(); - let providers = TemporalProviders::new(&FailingProvider, &calendar); + let recording = RecordingCalendar::new(); + let providers = TemporalProviders::new(&FailingProvider, &recording); let error = ZonedDateTime::from_local( &providers, local(2024, 1, 1, 0, 0), diff --git a/crates/bamts-runtime/src/vm.rs b/crates/bamts-runtime/src/vm.rs index 03593480..119d0a2a 100644 --- a/crates/bamts-runtime/src/vm.rs +++ b/crates/bamts-runtime/src/vm.rs @@ -38,15 +38,15 @@ pub(crate) fn install( extensible: true, }, ); - let script = register(heap, builtins, "Script", 1, script_constructor::); - let run_in_this_context = register( + let script = register_constructor(heap, builtins, "Script", 1, script_constructor::); + let run_in_this_context = register_constructor( heap, builtins, "runInThisContext", 2, run_in_this_context::, ); - let run_in_new_context = register( + let run_in_new_context = register_constructor( heap, builtins, "runInNewContext", @@ -129,6 +129,23 @@ fn register( intrinsics::native_function(heap, id, name, length) } +/// Registers a function `new` can reach. Node's `runIn*Context` are plain +/// functions, so unlike `Script` they work with or without it. +fn register_constructor( + heap: &mut Vec, + builtins: &mut BuiltinTable, + name: &'static str, + length: u32, + handler: intrinsics::BuiltinHandler, +) -> Value { + let id = builtins.register_constructor(BuiltinDef { + name, + length, + handler, + }); + intrinsics::native_function(heap, id, name, length) +} + fn type_error(operation: &'static str) -> EvalFailure { EvalFailure::Throw(ThrowOrigin::TypeError { operation }) } diff --git a/crates/bamts-runtime/src/vm/esm_eval.rs b/crates/bamts-runtime/src/vm/esm_eval.rs index e62c644e..0aff6aa3 100644 --- a/crates/bamts-runtime/src/vm/esm_eval.rs +++ b/crates/bamts-runtime/src/vm/esm_eval.rs @@ -183,6 +183,7 @@ mod tests { value: Value::UNDEFINED, origin: ThrowOrigin::Bytecode, constructor_name: None, + message: None, }) } diff --git a/crates/bamts-runtime/src/vm/generator_async.rs b/crates/bamts-runtime/src/vm/generator_async.rs index 08a98dba..c98c604a 100644 --- a/crates/bamts-runtime/src/vm/generator_async.rs +++ b/crates/bamts-runtime/src/vm/generator_async.rs @@ -479,6 +479,7 @@ mod tests { is_async: false, is_generator: true, is_constructable: false, + ..FunctionFlags::default() }, vec![ Instruction::LoadConst { @@ -620,6 +621,7 @@ mod tests { is_async: true, is_generator: false, is_constructable: false, + ..FunctionFlags::default() }, vec![ Instruction::LoadConst { @@ -672,6 +674,7 @@ mod tests { is_async: true, is_generator: false, is_constructable: false, + ..FunctionFlags::default() }, vec![ Instruction::LoadConst { @@ -744,6 +747,7 @@ mod tests { is_async: true, is_generator: false, is_constructable: false, + ..FunctionFlags::default() }, vec![ Instruction::Await { @@ -805,6 +809,7 @@ mod tests { is_async: false, is_generator: true, is_constructable: false, + ..FunctionFlags::default() }, vec![ Instruction::Suspend { @@ -876,6 +881,7 @@ mod tests { is_async: true, is_generator: false, is_constructable: false, + ..FunctionFlags::default() }, vec![ Instruction::CreateArray { dst: reg(0) }, @@ -975,6 +981,7 @@ mod tests { is_async: true, is_generator: true, is_constructable: false, + ..FunctionFlags::default() }, vec![ Instruction::LoadConst { diff --git a/crates/bamts-verification/Cargo.toml b/crates/bamts-verification/Cargo.toml index 7c60a2f0..f24e5e31 100644 --- a/crates/bamts-verification/Cargo.toml +++ b/crates/bamts-verification/Cargo.toml @@ -17,6 +17,7 @@ bamts-cli.workspace = true bamts-codegen.workspace = true bamts-compiler.workspace = true bamts-native.workspace = true +bamts-platform.workspace = true bamts-node = { workspace = true, features = ["script-compiler"] } bamts-runtime.workspace = true libc = "=0.2.189" diff --git a/crates/bamts-verification/probe/support.rs b/crates/bamts-verification/probe/support.rs new file mode 100644 index 00000000..318983e0 --- /dev/null +++ b/crates/bamts-verification/probe/support.rs @@ -0,0 +1,133 @@ +//! Freestanding support linked with the target-cell probe object. +//! +//! This is the whole runtime of a target-cell image: the two helpers the probe +//! calls, and an entry point that runs it and turns what it saw into an exit +//! status. `toolchain measure` compiles it for the target; cargo never sees it. +//! `@INT32_42@` is replaced with the boxed value of the measuring build. + +#![no_std] + +const INT32_42: u64 = @INT32_42@; + +const NORMAL: u32 = 0; +const FATAL_TRAP: u32 = 3; + +/// Layout of `bamts_native::ShadowFrame`. +#[repr(C)] +pub struct ShadowFrame { + previous: *mut ShadowFrame, + bytecode_pc: u32, + module_id: u32, + handles: *mut u64, + handle_len: u16, + padding: [u8; 6], +} + +static mut FUEL: u64 = 0; + +unsafe extern "C" { + fn bamts_m0_fn_0(frame: *mut ShadowFrame, out: *mut u64) -> u32; +} + +/// # Safety +/// +/// `out` must be writable. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn bamts_load_constant( + _frame: *mut ShadowFrame, + constant: u32, + out: *mut u64, +) -> u32 { + if constant != 1 { + return FATAL_TRAP; + } + unsafe { out.write(INT32_42) }; + NORMAL +} + +/// # Safety +/// +/// Called only from the single thread `bamts_probe_run` runs on. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn bamts_consume_fuel( + _frame: *mut ShadowFrame, + amount: u32, + _out: *mut u64, +) -> u32 { + unsafe { FUEL = FUEL.wrapping_add(u64::from(amount)) }; + NORMAL +} + +/// Runs the probe. Zero means it returned `int32:42` and charged fuel. +/// +/// # Safety +/// +/// Called once, on a single thread, from the image's entry point. +#[unsafe(no_mangle)] +pub unsafe extern "C" fn bamts_probe_run() -> i32 { + let mut register = 0u64; + let mut out = 0u64; + let mut frame = ShadowFrame { + previous: core::ptr::null_mut(), + bytecode_pc: 0, + module_id: 0, + handles: &raw mut register, + handle_len: 1, + padding: [0; 6], + }; + let tag = unsafe { bamts_m0_fn_0(&raw mut frame, &raw mut out) }; + if tag != NORMAL { + 10 + } else if out != INT32_42 { + 11 + } else if unsafe { FUEL } == 0 { + 12 + } else { + 0 + } +} + +#[cfg(all(target_os = "linux", target_arch = "x86_64"))] +core::arch::global_asm!( + ".globl _start", + "_start:", + "xor ebp, ebp", + "and rsp, -16", + "call bamts_probe_run", + "mov edi, eax", + "mov eax, 231", + "syscall", +); + +#[cfg(all(target_os = "linux", target_arch = "aarch64"))] +core::arch::global_asm!( + ".globl _start", + "_start:", + "bl bamts_probe_run", + "mov x8, #94", + "svc #0", +); + +#[cfg(all(target_os = "linux", target_arch = "riscv64"))] +core::arch::global_asm!( + ".globl _start", + "_start:", + "call bamts_probe_run", + "li a7, 94", + "ecall", +); + +#[cfg(all(target_os = "linux", target_arch = "s390x"))] +core::arch::global_asm!( + ".globl _start", + "_start:", + "aghi %r15, -160", + "brasl %r14, bamts_probe_run", + "svc 248", +); + +#[cfg(not(target_os = "linux"))] +#[unsafe(no_mangle)] +pub extern "C" fn main() -> i32 { + unsafe { bamts_probe_run() } +} diff --git a/crates/bamts-verification/src/bin/bh1_jit.rs b/crates/bamts-verification/src/bin/bh1_jit.rs index b3d23082..a1a85c6c 100644 --- a/crates/bamts-verification/src/bin/bh1_jit.rs +++ b/crates/bamts-verification/src/bin/bh1_jit.rs @@ -1,3 +1,4 @@ +#[cfg(unix)] use std::{ env, ffi::OsString, @@ -6,18 +7,22 @@ use std::{ process, }; +#[cfg(unix)] use bamts_verification::jit_runner::{ Attempt, HeadSha, JitError, ProvisionArgs, PullRequestNumber, RepoName, Result, RunId, RunnerGroupId, provision, run_internal_launcher, }; +#[cfg(unix)] const USAGE: &str = "usage: bh1_jit provision --repo owner/name --pr N --run ID --attempt N --head-sha 40hex --runner-group-id ID [--runner-parent PATH]"; +#[cfg(unix)] enum Command { Provision(ProvisionArgs), InternalLaunch, } +#[cfg(unix)] fn main() { if let Err(error) = run() { let _ = writeln!(io::stderr().lock(), "{error}"); @@ -25,6 +30,7 @@ fn main() { } } +#[cfg(unix)] fn run() -> Result<()> { match parse_command(env::args_os())? { Command::Provision(args) => { @@ -39,6 +45,7 @@ fn run() -> Result<()> { Ok(()) } +#[cfg(unix)] fn parse_command(mut arguments: impl Iterator) -> Result { let _program = arguments.next(); let command = next_utf8(&mut arguments, "command")?; @@ -54,6 +61,7 @@ fn parse_command(mut arguments: impl Iterator) -> Result) -> Result { let mut repo: Option = None; let mut pull_request: Option = None; @@ -101,6 +109,7 @@ fn parse_provision(arguments: &mut impl Iterator) -> Result, subject: &str) -> Result { arguments .next() @@ -109,6 +118,7 @@ fn next_utf8(arguments: &mut impl Iterator, subject: &str) -> R .map_err(|_| JitError::usage(format!("{subject} is not valid UTF-8; {USAGE}"))) } +#[cfg(unix)] fn set_once(slot: &mut Option, value: T, flag: &str) -> Result<()> { if slot.replace(value).is_some() { return Err(JitError::usage(format!("duplicate flag `{flag}`; {USAGE}"))); @@ -116,10 +126,12 @@ fn set_once(slot: &mut Option, value: T, flag: &str) -> Result<()> { Ok(()) } +#[cfg(unix)] fn required(value: Option, flag: &str) -> Result { value.ok_or_else(|| JitError::usage(format!("missing required flag `{flag}`; {USAGE}"))) } +#[cfg(unix)] #[cfg(test)] mod tests { use super::*; @@ -231,3 +243,11 @@ mod tests { ); } } + +/// The JIT runner provisioning drives unix process groups, file modes, and +/// locks, so it exists only on unix hosts. +#[cfg(not(unix))] +fn main() { + eprintln!("bh1_jit is not supported on this platform: it needs a unix host"); + std::process::exit(1); +} diff --git a/crates/bamts-verification/src/bin/runtime_lane_worker.rs b/crates/bamts-verification/src/bin/runtime_lane_worker.rs index d83f4a05..c86ee1c2 100644 --- a/crates/bamts-verification/src/bin/runtime_lane_worker.rs +++ b/crates/bamts-verification/src/bin/runtime_lane_worker.rs @@ -155,7 +155,7 @@ fn execute_obligation(workspace: &Path, request: &LaneRequest) -> LaneOutcome { let deadline = Duration::from_millis(request.timeout_ms()); let scratch = workspace.join("target/tmp/runtime-worker"); let runner = match backend_runner(mode, scratch) { - Ok(runner) => runner, + Ok(runner) => runner.for_test_file(&test_file), Err(reason) => { return LaneOutcome::BlockingFail { detail: reason.to_owned(), diff --git a/crates/bamts-verification/src/bin/ts_lane_worker.rs b/crates/bamts-verification/src/bin/ts_lane_worker.rs index b978ae78..eb22d9c5 100644 --- a/crates/bamts-verification/src/bin/ts_lane_worker.rs +++ b/crates/bamts-verification/src/bin/ts_lane_worker.rs @@ -1,7 +1,17 @@ +//! Compiler lane worker. +//! +//! With `BAMTS_LANE_WORKER_REQUEST` set, it answers that one request and exits. +//! With `BAMTS_LANE_WORKER_SERVE` set instead, it prints [`LANE_WORKER_READY`], +//! then answers one request path per line of standard input and prints +//! [`LANE_WORKER_DONE`] with the request ID after each response is written. +//! A serving worker authenticates the snapshot of a binding once and keeps it +//! for the later requests of that binding. A failure that ends a one-request +//! worker ends a serving worker the same way, so the parent records the same row. + use std::{ env, fs, fs::OpenOptions, - io::Write, + io::{BufRead, Write}, path::{Path, PathBuf}, }; @@ -9,8 +19,14 @@ use bamts_verification::{ ErrorCode, Result, VerificationError, check_cells::{CheckContext, baseline_groups}, facets::load_diagnostic_code_map, - lane::{LANE_WORKER_REQUEST, LaneOutcome, LaneRequest, LaneResponse}, - suite::{DEFAULT_SNAPSHOT_REL, load_bound_compiler_snapshot, observe_compiler_lane}, + historical::{is_historical, observe_historical_lane}, + lane::{ + LANE_WORKER_DONE, LANE_WORKER_READY, LANE_WORKER_REQUEST, LANE_WORKER_SERVE, LaneBinding, + LaneOutcome, LaneRequest, LaneResponse, + }, + suite::{ + CompilerSnapshot, DEFAULT_SNAPSHOT_REL, load_bound_compiler_snapshot, observe_compiler_lane, + }, }; fn main() { @@ -21,14 +37,50 @@ fn main() { } fn run() -> Result<()> { - let request_path = env::var_os(LANE_WORKER_REQUEST).ok_or_else(|| { - VerificationError::new( - ErrorCode::Usage, - format!("{LANE_WORKER_REQUEST} is not set"), - ) - })?; - let request_path = PathBuf::from(request_path); - let request_bytes = fs::read(&request_path).map_err(|error| { + if let Some(request_path) = env::var_os(LANE_WORKER_REQUEST) { + return answer(&PathBuf::from(request_path), &mut None).map(|_| ()); + } + if env::var_os(LANE_WORKER_SERVE).is_some() { + return serve(); + } + Err(VerificationError::new( + ErrorCode::Usage, + format!("{LANE_WORKER_REQUEST} is not set"), + )) +} + +/// The authenticated snapshot of one binding and the check context built from it. +struct Prepared { + binding: LaneBinding, + snapshot: CompilerSnapshot, + ctx: CheckContext, +} + +fn serve() -> Result<()> { + let mut prepared = None; + say(LANE_WORKER_READY)?; + for line in std::io::stdin().lock().lines() { + let line = line.map_err(|error| { + VerificationError::new(ErrorCode::Io, format!("standard input: {error}")) + })?; + let request_id = answer(Path::new(&line), &mut prepared)?; + // The empty line ends any line that the compile above left open. + say(&format!("\n{LANE_WORKER_DONE} {request_id}"))?; + } + Ok(()) +} + +/// Prints one line to standard output and flushes it, so the parent reads it now. +fn say(line: &str) -> Result<()> { + let mut stdout = std::io::stdout(); + writeln!(stdout, "{line}") + .and_then(|()| stdout.flush()) + .map_err(|error| VerificationError::new(ErrorCode::Io, format!("standard output: {error}"))) +} + +/// Answers the request at `request_path` and returns its request ID. +fn answer(request_path: &Path, prepared: &mut Option) -> Result { + let request_bytes = fs::read(request_path).map_err(|error| { VerificationError::new( ErrorCode::Io, format!("{}: {error}", request_path.display()), @@ -47,8 +99,8 @@ fn run() -> Result<()> { detail: "compiler worker request is missing snapshot binding".to_owned(), } } else { - match observe_from_workspace(&workspace, &snapshot_root, &request) { - Ok(observation) => observation.lane_outcome(), + match observe_from_workspace(&workspace, &snapshot_root, prepared, &request) { + Ok(outcome) => outcome, Err(error) => LaneOutcome::BlockingFail { detail: error.to_string(), }, @@ -64,20 +116,41 @@ fn run() -> Result<()> { VerificationError::new(ErrorCode::Json, format!("encode lane response: {error}")) })?; let response_path = request_path.with_extension("response.json"); - write_atomically(&response_path, &encoded) + write_atomically(&response_path, &encoded)?; + Ok(request.request_id()) } fn observe_from_workspace( workspace: &Path, snapshot_root: &Path, + prepared: &mut Option, request: &LaneRequest, -) -> Result { - let snapshot = load_bound_compiler_snapshot(snapshot_root, request.binding())?; - let ctx = CheckContext { - code_map: load_diagnostic_code_map(workspace)?, - baseline_groups: baseline_groups(&snapshot.index), - }; - Ok(observe_compiler_lane(&snapshot, &ctx, request)) +) -> Result { + if prepared + .as_ref() + .is_none_or(|current| ¤t.binding != request.binding()) + { + // Let go of the old snapshot before the new one is loaded beside it. + *prepared = None; + let snapshot = load_bound_compiler_snapshot(snapshot_root, request.binding())?; + let ctx = CheckContext { + code_map: load_diagnostic_code_map(workspace)?, + baseline_groups: baseline_groups(&snapshot.index), + }; + *prepared = Some(Prepared { + binding: request.binding().clone(), + snapshot, + ctx, + }); + } + let current = prepared + .as_ref() + .expect("the snapshot of this binding was prepared above"); + if is_historical(request.key().catalog()) { + observe_historical_lane(workspace, ¤t.snapshot, ¤t.ctx, request) + } else { + Ok(observe_compiler_lane(¤t.snapshot, ¤t.ctx, request).lane_outcome()) + } } fn write_atomically(path: &Path, bytes: &[u8]) -> Result<()> { @@ -129,7 +202,6 @@ fn write_atomically(path: &Path, bytes: &[u8]) -> Result<()> { #[cfg(test)] mod tests { use super::*; - use bamts_verification::lane::LaneBinding; #[test] fn worker_requires_compiler_snapshot_binding() { diff --git a/crates/bamts-verification/src/catalog.rs b/crates/bamts-verification/src/catalog.rs index ecc46f3a..c017d097 100644 --- a/crates/bamts-verification/src/catalog.rs +++ b/crates/bamts-verification/src/catalog.rs @@ -7,7 +7,7 @@ //! rows are labeled for exact policy exclusion rather than guessed away. use crate::{ - ErrorCode, Result, VerificationError, + ErrorCode, Result, VerificationError, fsync, schema::{ CATALOG_NAMES, Catalog, CatalogSource, MANIFEST_PATH, TYPESCRIPT_SUITE_SOURCE, VerificationManifest, catalog_source_from_pin, encode_manifest, identifiers_sha256, @@ -21,7 +21,7 @@ use sha2::{Digest, Sha256}; use std::{ collections::{BTreeMap, BTreeSet}, fmt, - fs::{self, File, OpenOptions}, + fs::{self, OpenOptions}, io::Write, path::{Component, Path}, sync::atomic::{AtomicU64, Ordering}, @@ -1096,9 +1096,7 @@ fn replace_file_atomically(path: &Path, bytes: &[u8]) -> Result<()> { file.sync_all() .map_err(|error| io_error(&temporary, &error))?; fs::rename(&temporary, path).map_err(|error| io_error(path, &error))?; - File::open(parent) - .and_then(|directory| directory.sync_all()) - .map_err(|error| io_error(parent, &error)) + fsync::sync_directory(parent).map_err(|error| io_error(parent, &error)) })(); if result.is_err() { let _ = fs::remove_file(&temporary); diff --git a/crates/bamts-verification/src/check_cells.rs b/crates/bamts-verification/src/check_cells.rs index 2a5edbaf..d910ade5 100644 --- a/crates/bamts-verification/src/check_cells.rs +++ b/crates/bamts-verification/src/check_cells.rs @@ -36,6 +36,7 @@ use bamts_compiler::program::{ use bamts_compiler::project::build_mode::{ BUILD_INFO_SCHEMA, BuildInfo, canonical_json, source_signature, }; +use bamts_compiler::project::canonical::canonicalize; use bamts_compiler::project::resolution::{ ModuleResolutionKind, ResolutionCache, ResolutionHost, TraceResolutionServices, resolve_module_name_with_trace, @@ -222,6 +223,20 @@ impl CheckedCase { .filter_map(|(unit, index)| index.map(|index| (unit, &self.output.modules()[index]))) } + /// `(unit, output)` pairs in the order upstream's `.types` and `.symbols` + /// documents list them. A last unit that names its dependencies is the one + /// the harness compiles, so it comes first and the others follow. + pub fn units_in_document_order(&self) -> Vec<(&CaseUnit, &FrontendOutput)> { + let mut units: Vec<_> = self.reached_units().collect(); + let last_is_root = units.last().is_some_and(|(unit, _)| { + unit.text.contains("require(") || unit.text.contains("reference path") + }); + if last_is_root && let Some(last) = units.pop() { + units.insert(0, last); + } + units + } + /// `(unit, output)` pairs for every reached unit, sorted by ascending /// module index. `ProgramFrontendOutput::modules()` is dependency-first /// (dependencies before dependents), matching upstream's output section @@ -987,6 +1002,7 @@ fn build_tsconfig(pragmas: &CasePragmas) -> String { | "resolvejsonmodule" | "noemit" | "noemithelpers" + | "removecomments" | "declaration" | "alwaysstrict" | "exactoptionalpropertytypes" @@ -1019,10 +1035,12 @@ fn build_tsconfig(pragmas: &CasePragmas) -> String { "resolvejsonmodule" => "resolveJsonModule", "moduleresolution" => "moduleResolution", "noemithelpers" => "noEmitHelpers", + "removecomments" => "removeComments", "noemit" => "noEmit", "alwaysstrict" => "alwaysStrict", "usedefineforclassfields" => "useDefineForClassFields", "importhelpers" => "importHelpers", + "verbatimmodulesyntax" => "verbatimModuleSyntax", "exactoptionalpropertytypes" => "exactOptionalPropertyTypes", "nouncheckedindexedaccess" => "noUncheckedIndexedAccess", "strictpropertyinitialization" => "strictPropertyInitialization", @@ -1124,7 +1142,8 @@ pub fn compile_case_frontend( .map_err(|error| CaseCompileError::Load(format!("config: {error}")))?; let options = config.options().clone(); let loader = ProgramLoader::new(&root, config.options()) - .map_err(|error| CaseCompileError::Load(format!("loader: {error}")))?; + .map_err(|error| CaseCompileError::Load(format!("loader: {error}")))? + .tolerating_unresolved_imports(); let program = loader .load(Path::new(ENTRY)) .map_err(|error| CaseCompileError::Load(error.to_string()))?; @@ -1163,7 +1182,7 @@ fn pair_units_with_modules( units .iter() .map(|unit| { - let canonical = fs::canonicalize(root.join(&unit.virtual_path)).ok(); + let canonical = canonicalize(root.join(&unit.virtual_path)).ok(); let index = canonical.and_then(|canonical| { program .modules() @@ -1524,7 +1543,7 @@ struct TypeAnnotation { #[must_use] pub fn emit_types_baseline(case: &CheckedCase, logical_path: &str) -> String { let mut out = format!("//// [{logical_path}] ////\n\n"); - for (unit, output) in case.reached_units() { + for (unit, output) in case.units_in_document_order() { if is_config_unit_name(&unit.display_name) { continue; } @@ -1812,9 +1831,10 @@ fn resolve_stem_baseline( pub fn check_types( snapshot: &(impl SnapshotAssets + ?Sized), case: &CheckedCase, + logical_path: &str, baseline_path: &str, ) -> Result { - let emitted = emit_types_baseline(case, &case.entry); + let emitted = emit_types_baseline(case, logical_path); let bytes = read_verified_snapshot_asset(snapshot, baseline_path)?; let expected = String::from_utf8_lossy(&bytes).into_owned(); let verdict = compare_types(&expected, &emitted); @@ -1874,16 +1894,17 @@ pub(crate) fn observe_types( artifact: None, }; }; - let outcome = match check_types(snapshot, &case, &baseline_path) { - Ok(outcome) => outcome, - Err(error) => { - return CompilerCheckObservation { - class: FailureClass::HarnessError, - detail: error.to_string(), - artifact: None, - }; - } - }; + let outcome = + match check_types(snapshot, &case, &index_entry.logical_path, &baseline_path) { + Ok(outcome) => outcome, + Err(error) => { + return CompilerCheckObservation { + class: FailureClass::HarnessError, + detail: error.to_string(), + artifact: None, + }; + } + }; match outcome.verdict { FacetVerdict::Pass => CompilerCheckObservation { class: FailureClass::Pass, @@ -2008,8 +2029,10 @@ pub fn emit_javascript_baseline( // the document carries no `.js` sections at all; its `result.DTS` is empty // unless declaration emit is on. The section set follows those two maps. let mut outputs = 0usize; + // The echo above keeps split order; the output sections follow emit order. + let emitted = case.reached_units_in_module_order(); if !options.emit_declaration_only() { - for (unit, output) in &units { + for (unit, output) in &emitted { let name = unit_basename(&unit.virtual_path); // A `.d.ts` input is never a JavaScript output: upstream's output // mapping skips declaration files outright. @@ -2037,7 +2060,7 @@ pub fn emit_javascript_baseline( } if options.declaration() || options.emit_declaration_only() { let mut declarations = Vec::new(); - for (unit, output) in &units { + for (unit, output) in &emitted { let name = unit_basename(&unit.virtual_path); if is_declaration_input_name(&name) { continue; @@ -2497,6 +2520,8 @@ pub fn emit_source_map_baseline( prog.import_helpers(), ); options.no_emit_helpers = check.no_emit_helpers(); + options.remove_comments = check.remove_comments(); + options.verbatim_module_syntax = prog.verbatim_module_syntax(); match prog.jsx_routing_decision(ProgramOutputKind::JavaScript) { JsxRoutingDecision::Emit | JsxRoutingDecision::TransformAndEmit => { options.jsx = prog.jsx(); @@ -3064,7 +3089,7 @@ pub(crate) fn observe_trace( #[must_use] pub fn emit_symbols_baseline(case: &CheckedCase, logical_path: &str) -> String { let mut out = format!("//// [{logical_path}] ////\n\n"); - let units: Vec<_> = case.reached_units().collect(); + let units = case.units_in_document_order(); let mut declaration_anchors: HashMap<(String, SymbolKind), Vec> = HashMap::new(); for (unit, output) in &units { @@ -3149,6 +3174,7 @@ pub fn emit_symbols_baseline(case: &CheckedCase, logical_path: &str) -> String { emit_unit_symbols( output.semantic_model(), output.source_file(), + &unit.display_name, §ion, &declaration_anchors, &mut out, @@ -3176,6 +3202,7 @@ struct SymbolRecord { fn emit_unit_symbols( model: &SemanticModel, source_file: &bamts_compiler::syntax::SourceFile, + heading: &str, section: &str, declaration_anchors: &HashMap<(String, SymbolKind), Vec>, out: &mut String, @@ -3337,7 +3364,7 @@ fn emit_unit_symbols( by_line.entry(record.line).or_default().push(record); } - out.push_str(&format!("=== {section} ===\n")); + out.push_str(&format!("=== {heading} ===\n")); for (line_index, raw_line) in source.as_str().split('\n').enumerate() { let line = raw_line.strip_suffix('\r').unwrap_or(raw_line); out.push_str(line); @@ -3636,9 +3663,10 @@ pub fn resolve_symbols_baseline(groups: &BaselineGroups, logical_path: &str) -> pub fn check_symbols( snapshot: &(impl SnapshotAssets + ?Sized), case: &CheckedCase, + logical_path: &str, baseline_path: &str, ) -> Result { - let emitted = emit_symbols_baseline(case, &case.entry); + let emitted = emit_symbols_baseline(case, logical_path); let bytes = read_verified_snapshot_asset(snapshot, baseline_path)?; let expected = String::from_utf8_lossy(&bytes).into_owned(); let verdict = compare_symbols(&expected, &emitted); @@ -3704,16 +3732,17 @@ pub(crate) fn observe_symbols( artifact: None, }; }; - let outcome = match check_symbols(snapshot, &case, &baseline_path) { - Ok(outcome) => outcome, - Err(error) => { - return CompilerCheckObservation { - class: FailureClass::HarnessError, - detail: error.to_string(), - artifact: None, - }; - } - }; + let outcome = + match check_symbols(snapshot, &case, &index_entry.logical_path, &baseline_path) { + Ok(outcome) => outcome, + Err(error) => { + return CompilerCheckObservation { + class: FailureClass::HarnessError, + detail: error.to_string(), + artifact: None, + }; + } + }; match outcome.verdict { FacetVerdict::Pass => CompilerCheckObservation { class: FailureClass::Pass, diff --git a/crates/bamts-verification/src/completion.rs b/crates/bamts-verification/src/completion.rs index 473a9a95..e2a8b661 100644 --- a/crates/bamts-verification/src/completion.rs +++ b/crates/bamts-verification/src/completion.rs @@ -19,7 +19,9 @@ use serde::{ de::{IgnoredAny, MapAccess, SeqAccess, Visitor}, }; -use crate::{ErrorCode, VerificationError, leaf_evidence, schema::sha256_hex}; +use crate::{ + ErrorCode, VerificationError, leaf_evidence, rebuild::LEDGER_SCHEMA, schema::sha256_hex, +}; const PROGRAM_PATH: &str = "verification/completion-program.toml"; const COMPLETENESS_LEDGER_PATH: &str = "proof/completeness-ledger.json"; @@ -28,7 +30,10 @@ const SPECIAL_ASPECT_LEAVES: [&str; 11] = [ "A1.1", "A1.2", "A1.3", "A1.4", "A1.5", "A2.1", "A2.2", "A2.3", "A2.6", "A2.7", "F2.4", ]; -// These recovery ledgers are not part of the frozen 105-leaf product authority. +/// Leaves the contract table freezes; a table with any other count is stale. +const FROZEN_LEAF_COUNT: usize = 114; + +// These recovery ledgers are not part of the frozen product authority. const LEGACY_INTEGRATION_LEDGER_IDS: [&str; 14] = [ "clean-port-authority-evidence", "clean-port-authority-root", @@ -268,7 +273,7 @@ impl CompletionReport { Some("RUNTIME COMPLETE modes=3 blocking=0 external_blocked=0".into()) } CompletionScope::Root(id) if id == "native" => { - Some("NATIVE COMPLETE formal=PASS targets=28/28 benchmarks=9/9".into()) + Some("NATIVE COMPLETE formal=PASS targets=49/49 benchmarks=9/9".into()) } CompletionScope::Root(id) if id == "package" => { Some("PACKAGE COMPLETE exports=13 cli=PASS api=PASS".into()) @@ -570,7 +575,7 @@ fn validate_program(root: &Path, p: &Program) -> Vec { let authority_path = root.join(".outline/type-script-7.0.2-completion.md"); let require_authority = !cfg!(test) || authority_path.exists(); let (authority, strict_authority) = completion_authority_leaves(root); - if require_authority && (!strict_authority || authority.len() != 105) { + if require_authority && (!strict_authority || authority.len() != FROZEN_LEAF_COUNT) { out.push(CompletionFailure::StaleEvidence { id: format!("completion-authority-leaf-count={}", authority.len()), }); @@ -1160,7 +1165,7 @@ impl<'de> Visitor<'de> for LedgerVisitor<'_, '_> { while let Some(key) = map.next_key::()? { match key.as_str() { "schema" => { - schema_current = map.next_value::()? == "bamti.completeness-ledger/v1"; + schema_current = map.next_value::()? == LEDGER_SCHEMA; } "manifest_sha256" => { manifest_current = map.next_value::()? == self.scan.expected_manifest; @@ -1354,18 +1359,22 @@ fn replace_atomically(path: &Path, bytes: &[u8]) -> Result<(), CompletionError> #[cfg(test)] mod tests { use super::*; - use std::time::{SystemTime, UNIX_EPOCH}; + use std::sync::atomic::{AtomicU64, Ordering}; struct Scratch(PathBuf); impl Scratch { fn new(name: &str) -> Self { + // The clock cannot keep these apart: macOS reads it in microseconds, + // and `fixture` gives every test the same label. A collision does not + // fail, because `create_dir_all` accepts an existing directory, so + // two tests would share one and one would delete it under the other. + static NEXT: AtomicU64 = AtomicU64::new(0); let p = std::env::temp_dir().join(format!( "bamts-completion-{name}-{}-{}", process::id(), - SystemTime::now() - .duration_since(UNIX_EPOCH) - .unwrap() - .as_nanos() + NEXT.fetch_add(1, Ordering::Relaxed) )); + // A directory left by an earlier process with the same id is stale. + let _ = fs::remove_dir_all(&p); fs::create_dir_all(p.join("verification")).unwrap(); fs::create_dir_all(p.join(".outline/gates")).unwrap(); fs::create_dir_all(p.join("proof")).unwrap(); @@ -1377,6 +1386,24 @@ mod tests { let _ = fs::remove_dir_all(&self.0); } } + + #[test] + fn scratch_directories_made_together_never_share_a_path() { + const THREADS: usize = 32; + let start = std::sync::Arc::new(std::sync::Barrier::new(THREADS)); + let workers: Vec<_> = (0..THREADS) + .map(|_| { + let start = std::sync::Arc::clone(&start); + std::thread::spawn(move || { + start.wait(); + Scratch::new("state") + }) + }) + .collect(); + let scratches: Vec<_> = workers.into_iter().map(|w| w.join().unwrap()).collect(); + let distinct: std::collections::BTreeSet<_> = scratches.iter().map(|s| &s.0).collect(); + assert_eq!(distinct.len(), THREADS); + } fn fixture(state: &str, public: Option<&str>, mutation: bool) -> Scratch { let scratch = Scratch::new("state"); fs::create_dir_all(scratch.0.join("src")).unwrap(); @@ -1458,7 +1485,7 @@ mod tests { fs::write( scratch.0.join(COMPLETENESS_LEDGER_PATH), format!( - "{{\"schema\":\"bamti.completeness-ledger/v1\",\"manifest_sha256\":\"{}\",\"rows\":[{{\"id\":\"test:case\",\"state\":\"{state}\"{surface}}}]}}", + "{{\"schema\":\"bamti.completeness-ledger/v2\",\"manifest_sha256\":\"{}\",\"rows\":[{{\"id\":\"test:case\",\"state\":\"{state}\"{surface}}}]}}", sha256_hex(manifest) ), ) @@ -1724,24 +1751,62 @@ mod tests { ); } - #[test] - fn frozen_authority_rejects_joint_leaf_removal() { - let fixture = fixture("PASS", None, true); + fn write_authority_table(root: &Path, leaves: usize) { let mut authority = String::from( "---\nrelease: typescript-7.0.2\nproduct: bamti\n---\n\ | Leaf | Wave | Cluster | Deliverable | Owned paths | Gate ledger |\n\ |---|---|---|---|---|---|\n", ); - for index in 1..=105 { + for index in 1..=leaves { authority.push_str(&format!( "| X{index} | W0 | X | d | `src/x.rs` | `.outline/gates/X{index}.md` |\n" )); } fs::write( - fixture.0.join(".outline/type-script-7.0.2-completion.md"), + root.join(".outline/type-script-7.0.2-completion.md"), authority, ) .unwrap(); + } + + fn leaf_count_failures(leaves: usize) -> Vec { + let fixture = fixture("PASS", None, true); + write_authority_table(&fixture.0, leaves); + match verify_completion( + &fixture.0, + &CompletionScope::Leaf("X1".into()), + CompletionAspect::Contract, + ) { + Err(CompletionError::Invalid(failures)) => failures, + other => panic!("a table that disagrees with the program must be invalid: {other:?}"), + } + } + + fn reports_stale_leaf_count(failures: &[CompletionFailure], leaves: usize) -> bool { + failures.iter().any(|failure| { + matches!(failure, CompletionFailure::StaleEvidence { id } + if *id == format!("completion-authority-leaf-count={leaves}")) + }) + } + + #[test] + fn authority_table_must_hold_exactly_the_frozen_leaf_count() { + for leaves in [FROZEN_LEAF_COUNT - 1, FROZEN_LEAF_COUNT + 1] { + assert!( + reports_stale_leaf_count(&leaf_count_failures(leaves), leaves), + "{leaves} leaves must be reported as a stale count" + ); + } + assert!( + !reports_stale_leaf_count(&leaf_count_failures(FROZEN_LEAF_COUNT), FROZEN_LEAF_COUNT), + "the frozen count itself must not be reported as stale" + ); + } + + #[test] + fn frozen_authority_rejects_joint_leaf_removal() { + let fixture = fixture("PASS", None, true); + write_authority_table(&fixture.0, FROZEN_LEAF_COUNT); let error = verify_completion( &fixture.0, &CompletionScope::Leaf("X1".into()), diff --git a/crates/bamts-verification/src/corpus.rs b/crates/bamts-verification/src/corpus.rs index a0e1a0b2..26709b5b 100644 --- a/crates/bamts-verification/src/corpus.rs +++ b/crates/bamts-verification/src/corpus.rs @@ -24,7 +24,7 @@ use std::{ collections::BTreeSet, env, - ffi::OsString, + ffi::{OsStr, OsString}, fs, io::{ErrorKind, Read}, path::{Component, Path, PathBuf}, @@ -1748,6 +1748,7 @@ pub(crate) fn run_process( for (key, value) in environment { command.env(key, value); } + forward_host_runtime_env(&mut command); command .current_dir(cwd) .args(args) @@ -1803,7 +1804,9 @@ pub(crate) fn run_process( Ok(OracleOutcome { timed_out, - exit_code: status.code(), + // A child the harness killed has no exit code of its own: Unix reports + // the signal, and Windows would report the 1 that `TerminateProcess` sets. + exit_code: if timed_out { None } else { status.code() }, signal: termination_signal(&status), stdout, stdout_truncated, @@ -1879,16 +1882,34 @@ pub(crate) fn drain_stream( } #[cfg(unix)] -fn termination_signal(status: &ExitStatus) -> Option { +pub(crate) fn termination_signal(status: &ExitStatus) -> Option { use std::os::unix::process::ExitStatusExt; status.signal() } #[cfg(not(unix))] -fn termination_signal(_status: &ExitStatus) -> Option { +pub(crate) fn termination_signal(_status: &ExitStatus) -> Option { None } +/// Windows resolves hosts, loads system libraries and seeds randomness through +/// `SystemRoot`; a child started without it fails (`curl` cannot resolve any +/// name). It also finds its temporary directory only through `TEMP` and `TMP`; +/// without them `std::env::temp_dir()` is the system directory, which an +/// unprivileged process cannot write. These variables are host plumbing and +/// stay out of `NORMALIZED_ENV`, which evidence records byte for byte. +#[cfg(windows)] +pub(crate) fn forward_host_runtime_env(command: &mut Command) { + for key in ["SystemRoot", "windir", "TEMP", "TMP"] { + if let Some(value) = env::var_os(key) { + command.env(key, value); + } + } +} + +#[cfg(not(windows))] +pub(crate) fn forward_host_runtime_env(_command: &mut Command) {} + fn locate_node() -> Result { let path = env::var_os("PATH").ok_or_else(|| { VerificationError::new( @@ -1896,19 +1917,17 @@ fn locate_node() -> Result { "PATH is unset; cannot locate the Node oracle", ) })?; - for dir in env::split_paths(&path) { - if dir.as_os_str().is_empty() { - continue; - } - let candidate = dir.join("node"); - if is_executable_file(&candidate) { - return Ok(candidate); - } - } - Err(VerificationError::new( - ErrorCode::ToolMissing, - "cannot locate `node` on PATH", - )) + find_node_in(&path).ok_or_else(|| { + VerificationError::new(ErrorCode::ToolMissing, "cannot locate `node` on PATH") + }) +} + +fn find_node_in(search_path: &OsStr) -> Option { + let file_name = if cfg!(windows) { "node.exe" } else { "node" }; + env::split_paths(search_path) + .filter(|dir| !dir.as_os_str().is_empty()) + .map(|dir| dir.join(file_name)) + .find(|candidate| is_executable_file(candidate)) } #[cfg(unix)] @@ -2141,7 +2160,11 @@ fn is_lower_hex(value: &str, length: usize) -> bool { .all(|byte| byte.is_ascii_digit() || matches!(byte, b'a'..=b'f')) } -fn spawn_error(label: &str, program: &Path, error: &std::io::Error) -> VerificationError { +pub(crate) fn spawn_error( + label: &str, + program: &Path, + error: &std::io::Error, +) -> VerificationError { let code = if error.kind() == ErrorKind::NotFound { ErrorCode::ToolMissing } else { @@ -2153,18 +2176,18 @@ fn spawn_error(label: &str, program: &Path, error: &std::io::Error) -> Verificat ) } -fn pipe_error(label: &str, stream: &str) -> VerificationError { +pub(crate) fn pipe_error(label: &str, stream: &str) -> VerificationError { VerificationError::new( ErrorCode::Io, format!("{label} {stream} pipe was not captured"), ) } -fn wait_error(label: &str, error: std::io::Error) -> VerificationError { +pub(crate) fn wait_error(label: &str, error: std::io::Error) -> VerificationError { VerificationError::new(ErrorCode::Io, format!("cannot wait on {label}: {error}")) } -fn thread_error(label: &str, stream: &str) -> VerificationError { +pub(crate) fn thread_error(label: &str, stream: &str) -> VerificationError { VerificationError::new( ErrorCode::ToolFailed, format!("{label} {stream} reader thread panicked"), @@ -2246,6 +2269,34 @@ mod tests { .expect("oracle run") } + #[test] + fn node_is_found_by_the_file_name_the_platform_gives_it() { + let decoys = scratch("node-decoys"); + let installed = scratch("node-installed"); + let (native, foreign) = if cfg!(windows) { + ("node.exe", "node") + } else { + ("node", "node.exe") + }; + fs::write(decoys.join(foreign), b"").expect("write foreign-named file"); + let binary = installed.join(native); + fs::write(&binary, b"").expect("write stand-in binary"); + #[cfg(unix)] + { + use std::os::unix::fs::PermissionsExt; + fs::set_permissions(&binary, fs::Permissions::from_mode(0o755)) + .expect("mark stand-in executable"); + } + + let search = env::join_paths([&decoys, &installed]).expect("join search path"); + assert_eq!(find_node_in(&search), Some(binary)); + let search = env::join_paths([&decoys]).expect("join search path"); + assert_eq!(find_node_in(&search), None); + + fs::remove_dir_all(decoys).expect("remove decoys"); + fs::remove_dir_all(installed).expect("remove installed"); + } + // ---- execution modes -------------------------------------------------- #[test] @@ -2871,6 +2922,27 @@ mod tests { let _ = fs::remove_dir_all(&dir); } + #[test] + #[cfg(windows)] + fn windows_children_keep_the_host_system_root() { + let dir = scratch("system-root"); + let outcome = run_process( + "SystemRoot probe", + Path::new("cmd.exe"), + &dir, + &[], + &["/D", "/C", "echo %SystemRoot%"].map(OsString::from), + &OracleLimits { + timeout: Duration::from_secs(10), + max_output_bytes: DEFAULT_MAX_OUTPUT_BYTES, + }, + ) + .expect("run cmd.exe"); + let expected = env::var("SystemRoot").expect("host SystemRoot"); + assert_eq!(String::from_utf8_lossy(&outcome.stdout).trim(), expected); + let _ = fs::remove_dir_all(&dir); + } + #[test] fn oracle_bounds_output_without_deadlock() { let dir = scratch("bound"); diff --git a/crates/bamts-verification/src/evidence.rs b/crates/bamts-verification/src/evidence.rs index 47175f31..1a98943e 100644 --- a/crates/bamts-verification/src/evidence.rs +++ b/crates/bamts-verification/src/evidence.rs @@ -294,6 +294,12 @@ impl ExecutionBinding { Ok(binding) } + /// The commit the run was built from. + #[must_use] + pub fn source_sha(&self) -> &str { + &self.source_sha + } + #[cfg(test)] pub(crate) fn local_for_tests() -> Self { Self { @@ -521,6 +527,12 @@ impl EvidenceRow { pub fn artifacts(&self) -> &BTreeMap { &self.artifacts } + + /// The diagnostic text recorded with the state; empty on a clean `PASS`. + #[must_use] + pub fn detail(&self) -> &str { + &self.detail + } } /// Footer that closes a stream. diff --git a/crates/bamts-verification/src/facets.rs b/crates/bamts-verification/src/facets.rs index 0d0dc198..333864ff 100644 --- a/crates/bamts-verification/src/facets.rs +++ b/crates/bamts-verification/src/facets.rs @@ -672,11 +672,128 @@ fn compare_structural( FacetVerdict::Pass } else { FacetVerdict::Fail { - reason: format!("{facet} structural mismatch after formatting/order normalization"), + reason: format!( + "{facet} structural mismatch {}", + describe_difference(&expected_norm, &actual_norm) + ), } } } +/// Longest excerpt of a differing line kept in a reason. +const EXCERPT_CHARS: usize = 100; + +/// `[category]: where`, naming the first difference between two normalized +/// outputs. The category is a plain word, so rows that differ the same way group +/// together. The lines themselves sit in backticks, so a family ignores them. +fn describe_difference(expected: &[String], actual: &[String]) -> String { + let common = expected + .iter() + .zip(actual) + .take_while(|(left, right)| left == right) + .count(); + match (expected.get(common), actual.get(common)) { + (None, Some(extra)) => format!( + "[extra-lines]: actual has {} extra line(s) from line {}: `{}`", + actual.len() - common, + common + 1, + excerpt(extra) + ), + (Some(missing), None) => format!( + "[missing-lines]: actual ends at line {common}; expected continues with `{}`", + excerpt(missing) + ), + (Some(left), Some(right)) => { + let mut sorted_expected = expected.to_vec(); + let mut sorted_actual = actual.to_vec(); + sorted_expected.sort(); + sorted_actual.sort(); + let category = if sorted_expected == sorted_actual { + "order".to_owned() + } else { + line_difference(left, right) + }; + format!( + "[{category}]: line {} expected `{}` actual `{}`", + common + 1, + excerpt(left), + excerpt(right) + ) + } + // The slices differ, so one of them has a line at `common`. + (None, None) => "[text]: outputs differ".to_owned(), + } +} + +/// How two lines at the same position differ, from the most to the least +/// specific. Lines that differ in kind are named by the pair, such as +/// `comment-vs-binding`: a baseline comment that the output replaced with code. +fn line_difference(expected: &str, actual: &str) -> String { + let unquoted = |text: &str| text.replace('\'', "\""); + let unpunctuated = |text: &str| text.trim_end_matches([';', ',']).to_owned(); + if expected.split_whitespace().eq(actual.split_whitespace()) { + return "spacing".to_owned(); + } + if expected.eq_ignore_ascii_case(actual) { + return "case".to_owned(); + } + if unquoted(expected) == unquoted(actual) { + return "quote-style".to_owned(); + } + if unpunctuated(expected) == unpunctuated(actual) { + return "punctuation".to_owned(); + } + let (left, right) = (line_kind(expected), line_kind(actual)); + if left == right { + "text".to_owned() + } else { + format!("{left}-vs-{right}") + } +} + +/// The kind of JavaScript line, from its first token. A small closed set, so +/// the pairs a mismatch can name stay few. +fn line_kind(line: &str) -> &'static str { + const KINDS: [(&str, &[&str]); 10] = [ + ("comment", &["//", "/*", "*"]), + ("import", &["import "]), + ("export", &["export "]), + ("declare", &["declare "]), + ("binding", &["var ", "let ", "const "]), + ("function", &["function ", "async function "]), + ("class", &["class ", "abstract class "]), + ("strict", &["\"use strict\"", "'use strict'"]), + ("brace", &["}", "{", ")", "]"]), + ( + "cjs", + &[ + "exports.", + "module.exports", + "Object.defineProperty(exports", + ], + ), + ]; + let line = line.trim(); + if line.is_empty() { + return "blank"; + } + KINDS + .iter() + .find(|(_, prefixes)| prefixes.iter().any(|prefix| line.starts_with(prefix))) + .map_or("other", |(kind, _)| kind) +} + +/// A line cut to a readable length, with backticks replaced so it can sit in one. +fn excerpt(line: &str) -> String { + let line = line.trim().replace('`', "'"); + if line.chars().count() <= EXCERPT_CHARS { + return line; + } + let mut cut: String = line.chars().take(EXCERPT_CHARS).collect(); + cut.push('…'); + cut +} + /// Declared identity of one element in a declaration-set container. /// /// Only a simple declared name is a provably distinct identity. Every other @@ -1230,7 +1347,8 @@ fn canonical_class_member(member: &ClassMemberNode, file: &SourceFile) -> (DeclI ClassMember::Constructor(_) | ClassMember::StaticBlock(_) | ClassMember::IndexSignature(_) - | ClassMember::Missing(_) => DeclIdentity::Anonymous, + | ClassMember::Missing(_) + | ClassMember::Semicolon => DeclIdentity::Anonymous, ClassMember::Method(method) => property_identity(&method.name, file), ClassMember::Property(property) => property_identity(&property.name, file), ClassMember::AutoAccessor(accessor) => property_identity(&accessor.name, file), @@ -1263,9 +1381,10 @@ fn canonical_class_member(member: &ClassMemberNode, file: &SourceFile) -> (DeclI children.push(&*index.type_annotation.data().type_node); rebuild_children(file, member.range(), children) } - ClassMember::Constructor(_) | ClassMember::StaticBlock(_) | ClassMember::Missing(_) => { - slice_node(file, member.range()) - } + ClassMember::Constructor(_) + | ClassMember::StaticBlock(_) + | ClassMember::Missing(_) + | ClassMember::Semicolon => slice_node(file, member.range()), }; (identity, raw) } @@ -1988,6 +2107,127 @@ mod tests { use super::*; use crate::corpus::{OracleOutcome, Provenance}; + fn js_reason(expected: &str, actual: &str) -> String { + match compare_js_emit(expected, actual) { + FacetVerdict::Fail { reason } => reason, + other => panic!("a difference must fail, got {other:?}"), + } + } + + #[test] + fn a_mismatch_names_its_category_and_the_first_differing_line() { + let cases: [(&str, &str, &str, &[&str]); 12] = [ + ( + "a\nb", + "a\nb\nc\nd", + "extra-lines", + &["2 extra line(s) from line 3", "`c`"], + ), + ( + "a\nb\nc", + "a\nb", + "missing-lines", + &["ends at line 2", "`c`"], + ), + ( + "a\nb\nc", + "a\nc\nb", + "order", + &["line 2", "expected `b`", "actual `c`"], + ), + ( + "var x = 1;", + "var x = 1;", + "spacing", + &["line 1", "expected `var x = 1;`"], + ), + ("let s = 'a';", "let s = \"a\";", "quote-style", &["line 1"]), + ( + "f()", + "f();", + "punctuation", + &["expected `f()`", "actual `f();`"], + ), + ("Foo();", "foo();", "case", &["line 1"]), + ( + "ok\na = 1;", + "ok\na = 2;", + "text", + &["line 2", "expected `a = 1;`", "actual `a = 2;`"], + ), + ( + "// note\nok", + "var x = 1;\nok", + "comment-vs-binding", + &["line 1", "expected `// note`"], + ), + ( + "class C {", + "var C = (function () {", + "class-vs-binding", + &["line 1"], + ), + ( + "exports.a = 1;", + "var a = 1;", + "cjs-vs-binding", + &["actual `var a = 1;`"], + ), + ( + "var x = 1;", + "export var x = 1;", + "binding-vs-export", + &["line 1"], + ), + ]; + for (expected, actual, category, mentions) in cases { + let reason = js_reason(expected, actual); + assert!( + reason.starts_with(&format!("javascript structural mismatch [{category}]: ")), + "{category}: {reason}" + ); + for mention in mentions { + assert!(reason.contains(mention), "{category}: {reason}"); + } + } + } + + #[test] + fn a_mismatch_stays_a_failure_and_equal_output_still_passes() { + assert!(compare_js_emit("a = 1;", "a = 2;").is_fail()); + assert!(compare_js_emit("a = 1;\r\n", "a = 1; \n").is_pass()); + } + + #[test] + fn a_reason_excerpts_long_lines_and_never_nests_backticks() { + let long = "x".repeat(400); + let reason = js_reason("ok", &format!("ok\n{long}")); + assert!(reason.len() < 200, "{} bytes: {reason}", reason.len()); + let tick = js_reason("a `b` c", "a `d` c"); + assert_eq!(tick.matches('`').count() % 2, 0, "{tick}"); + assert!(!tick.contains("`b`"), "{tick}"); + } + + #[test] + fn rows_that_differ_in_category_not_in_content_share_a_family() { + use crate::sweep_report::family_name; + let family = + |expected: &str, actual: &str| family_name("javascript", &js_reason(expected, actual)); + assert_eq!( + family("a = 1;", "a = 2;"), + family("return f(x);", "return g(y);") + ); + assert_eq!(family("f()", "f();"), family("g(1, 2)", "g(1, 2);")); + assert_ne!(family("a = 1;", "a = 2;"), family("f()", "f();")); + assert_eq!( + family("// a", "var x = 1;"), + family("// b c", "let y = f(z);") + ); + assert_ne!(family("// a", "var x = 1;"), family("// a", "class X {")); + assert_ne!(family("// a", "var x = 1;"), family("var x = 1;", "// a")); + assert_ne!(family("a\nb", "a\nb\nc"), family("a\nb\nc", "a\nb")); + } + fn empty_map_json(entries: &[(&str, DiagnosticMappingStatus, Option)]) -> String { let evidence = "fixture evidence"; let mut items = Vec::new(); diff --git a/crates/bamts-verification/src/fsync.rs b/crates/bamts-verification/src/fsync.rs new file mode 100644 index 00000000..81e7370c --- /dev/null +++ b/crates/bamts-verification/src/fsync.rs @@ -0,0 +1,140 @@ +//! Flushes that mean the same thing on every host. +//! +//! Windows flushes a file only through a handle with write access, and refuses +//! to open a directory as a plain file, so `File::open(..).sync_all()` fails +//! there with "access denied". + +use std::{fs::File, io, path::Path}; + +/// Flushes the file at `path` to storage. Other hosts flush through a read-only +/// handle, which also covers files whose mode forbids writing. +#[cfg(not(windows))] +pub(crate) fn sync_file(path: &Path) -> io::Result<()> { + File::open(path)?.sync_all() +} + +/// Flushes the file at `path` to storage through a handle with write access. +/// +/// A file with the read-only attribute refuses such a handle. The attribute is +/// lifted for the flush and put back afterwards, whether or not the flush worked. +#[cfg(windows)] +pub(crate) fn sync_file(path: &Path) -> io::Result<()> { + let flush = || { + std::fs::OpenOptions::new() + .read(true) + .write(true) + .open(path)? + .sync_all() + }; + match flush() { + Err(denied) if denied.kind() == io::ErrorKind::PermissionDenied && is_read_only(path) => { + set_read_only(path, false)?; + let flushed = flush(); + let restored = set_read_only(path, true); + flushed.and(restored) + } + flushed => flushed, + } +} + +#[cfg(windows)] +fn is_read_only(path: &Path) -> bool { + std::fs::metadata(path).is_ok_and(|metadata| metadata.permissions().readonly()) +} + +/// Sets or clears the read-only attribute. Only Windows reaches this code: on +/// Unix, clearing the flag would make the file writable for everyone. +#[cfg(windows)] +fn set_read_only(path: &Path, read_only: bool) -> io::Result<()> { + let mut permissions = std::fs::metadata(path)?.permissions(); + permissions.set_readonly(read_only); + std::fs::set_permissions(path, permissions) +} + +/// Flushes the directory entry list at `path`. +/// +/// Windows is skipped. `std` cannot open a directory there; a handle opened with +/// `FILE_FLAG_BACKUP_SEMANTICS` for writing can be flushed on NTFS, but this code +/// does not do that. NTFS journals the rename, which is not a promise that the +/// rename is on disk after a power loss, so a rename that has returned is +/// slightly less durable on Windows than on other hosts. +pub(crate) fn sync_directory(path: &Path) -> io::Result<()> { + if cfg!(windows) { + return Ok(()); + } + File::open(path)?.sync_all() +} + +#[cfg(test)] +mod tests { + use std::{env, fs}; + + use super::{sync_directory, sync_file}; + + fn scratch(tag: &str) -> std::path::PathBuf { + let dir = env::temp_dir().join(format!("bamts-fsync-{tag}-{}", std::process::id())); + fs::create_dir_all(&dir).expect("create scratch dir"); + dir + } + + #[test] + fn a_written_file_and_its_directory_flush() { + let dir = scratch("written"); + let file = dir.join("data"); + fs::write(&file, b"payload").expect("write file"); + + sync_file(&file).expect("flush file"); + sync_directory(&dir).expect("flush directory"); + + fs::remove_dir_all(dir).expect("remove scratch dir"); + } + + #[test] + fn a_missing_file_is_an_error_not_a_silent_success() { + let dir = scratch("missing"); + + assert_eq!( + sync_file(&dir.join("absent")) + .expect_err("missing file") + .kind(), + std::io::ErrorKind::NotFound + ); + + fs::remove_dir_all(dir).expect("remove scratch dir"); + } + + #[test] + #[cfg(windows)] + fn a_file_with_the_read_only_attribute_flushes_and_stays_read_only() { + let dir = scratch("attribute"); + let file = dir.join("data"); + fs::write(&file, b"payload").expect("write file"); + super::set_read_only(&file, true).expect("set the read-only attribute"); + + sync_file(&file).expect("flush read-only file"); + + let read_only = fs::metadata(&file) + .expect("metadata") + .permissions() + .readonly(); + super::set_read_only(&file, false).expect("clear the attribute for cleanup"); + assert!(read_only, "the attribute is restored after the flush"); + + fs::remove_dir_all(dir).expect("remove scratch dir"); + } + + #[test] + #[cfg(unix)] + fn a_file_whose_mode_forbids_writing_still_flushes() { + use std::os::unix::fs::PermissionsExt; + + let dir = scratch("readonly"); + let file = dir.join("data"); + fs::write(&file, b"payload").expect("write file"); + fs::set_permissions(&file, fs::Permissions::from_mode(0o444)).expect("drop write bit"); + + sync_file(&file).expect("flush read-only file"); + + fs::remove_dir_all(dir).expect("remove scratch dir"); + } +} diff --git a/crates/bamts-verification/src/historical.rs b/crates/bamts-verification/src/historical.rs new file mode 100644 index 00000000..76777125 --- /dev/null +++ b/crates/bamts-verification/src/historical.rs @@ -0,0 +1,244 @@ +//! Historical TypeScript catalogs: which cells keep a measurement, and from where. +//! +//! The contract keeps TypeScript 6.0 and 5.9 inputs only while the stable 7.0.2 +//! oracle still accepts them, and they never act as oracles: a historical tree +//! supplies inputs, never expected output. +//! +//! A historical cell takes the 7.0.2 measurement when 7.0.2 declares the same +//! input, byte for byte, and its receipt row says so. Every 6.0.2 input qualifies. +//! Any other cell has nothing to be compared with until it is run against the +//! oracle, so it stays a blocking row that names why, and never reads as a pass. + +use std::{ + collections::BTreeMap, + fs, + io::ErrorKind, + path::{Path, PathBuf}, +}; + +use crate::{ + ErrorCode, Result, VerificationError, + check_cells::CheckContext, + lane::{LaneOutcome, LaneRequest}, + shard::ObligationKey, + suite::{COMPILER_CATALOG, SnapshotAssets, observe_compiler_lane}, +}; + +/// Catalogs measured through their 7.0.2 twin. +pub const HISTORICAL_CATALOGS: [&str; 2] = ["typescript-6.0.2", "typescript-5.9.3"]; + +/// Support files every `project/` case reads, relative to an authority tree. +const PROJECT_SUPPORT: &str = "tests/cases/projects"; + +/// What a reused row records about where its measurement came from. +const REUSED_FROM: &str = "typescript-7.0.2 cell with byte-identical input"; + +#[must_use] +pub fn is_historical(catalog: &str) -> bool { + HISTORICAL_CATALOGS.contains(&catalog) +} + +/// Measures one historical compiler cell. +/// +/// # Errors +/// +/// Returns an error when the cell's own input cannot be read, which means the +/// authority tree is damaged rather than that the cell failed. +pub fn observe_historical_lane( + workspace: &Path, + snapshot: &(impl SnapshotAssets + ?Sized), + ctx: &CheckContext, + request: &LaneRequest, +) -> Result { + let key = request.key(); + if let Some(reason) = reuse_blocker(workspace, key.catalog(), key.case())? { + return Ok(LaneOutcome::BlockingFail { + detail: format!("historical cell has no oracle comparison: {reason}"), + }); + } + let stable = ObligationKey::new( + COMPILER_CATALOG, + key.case(), + key.configuration(), + key.mode(), + key.platform(), + )?; + let outcome = observe_compiler_lane(snapshot, ctx, &request.with_key(stable)?).lane_outcome(); + Ok(match outcome { + LaneOutcome::Completed { artifacts } => LaneOutcome::Reused { + artifacts, + source: REUSED_FROM.to_owned(), + }, + other => other, + }) +} + +/// Why this case cannot take the 7.0.2 measurement, or `None` when it can. +fn reuse_blocker(workspace: &Path, catalog: &str, case: &str) -> Result> { + let Some((runner, relative)) = case.split_once('/') else { + return Err(VerificationError::new( + ErrorCode::Schema, + format!("case `{case}` has no authority path"), + )); + }; + let input = read_file(&authority_tree(workspace, catalog).join(relative))?; + let stable_tree = authority_tree(workspace, COMPILER_CATALOG); + let stable = match fs::read(stable_tree.join(relative)) { + Ok(bytes) => bytes, + Err(error) if error.kind() == ErrorKind::NotFound => { + return Ok(Some("7.0.2 has no such input".to_owned())); + } + Err(error) => return Err(io_error(&stable_tree.join(relative), &error)), + }; + if input != stable { + return Ok(Some("its input differs from 7.0.2".to_owned())); + } + if runner == "project" { + let own = read_tree(&authority_tree(workspace, catalog).join(PROJECT_SUPPORT))?; + if own != read_tree(&stable_tree.join(PROJECT_SUPPORT))? { + return Ok(Some("its project files differ from 7.0.2".to_owned())); + } + } + Ok(None) +} + +fn authority_tree(workspace: &Path, catalog: &str) -> PathBuf { + workspace.join(format!("target/authority/{catalog}-tests")) +} + +fn read_file(path: &Path) -> Result> { + fs::read(path).map_err(|error| io_error(path, &error)) +} + +/// Every file under `root` by relative path. Finder's folder files are not input. +fn read_tree(root: &Path) -> Result>> { + let mut files = BTreeMap::new(); + let mut pending = vec![root.to_path_buf()]; + while let Some(directory) = pending.pop() { + for entry in fs::read_dir(&directory).map_err(|error| io_error(&directory, &error))? { + let path = entry.map_err(|error| io_error(&directory, &error))?.path(); + if path.is_dir() { + pending.push(path); + } else if path.file_name().is_some_and(|name| name != ".DS_Store") { + let relative = path.strip_prefix(root).unwrap_or(&path).to_path_buf(); + files.insert(relative, read_file(&path)?); + } + } + } + Ok(files) +} + +fn io_error(path: &Path, error: &std::io::Error) -> VerificationError { + VerificationError::new(ErrorCode::Io, format!("{}: {error}", path.display())) +} + +#[cfg(test)] +mod tests { + use super::*; + + const HISTORICAL: &str = "typescript-5.9.3"; + const CASE: &str = "compiler/tests/cases/compiler/a.ts"; + const PROJECT: &str = "project/tests/cases/project/a.json"; + + fn workspace(name: &str) -> PathBuf { + let root = + std::env::temp_dir().join(format!("bamts-historical-{}-{name}", std::process::id())); + let _ = fs::remove_dir_all(&root); + fs::create_dir_all(&root).expect("scratch"); + root + } + + fn write(root: &Path, catalog: &str, relative: &str, text: &str) { + let path = authority_tree(root, catalog).join(relative); + fs::create_dir_all(path.parent().expect("parent")).expect("directories"); + fs::write(path, text).expect("write"); + } + + fn both(root: &Path, relative: &str, text: &str) { + write(root, HISTORICAL, relative, text); + write(root, COMPILER_CATALOG, relative, text); + } + + #[test] + fn identical_input_takes_the_stable_measurement() { + let root = workspace("identical"); + both(&root, "tests/cases/compiler/a.ts", "let a = 1;"); + assert_eq!(reuse_blocker(&root, HISTORICAL, CASE).expect("reads"), None); + fs::remove_dir_all(root).expect("remove scratch"); + } + + #[test] + fn one_changed_byte_blocks_reuse() { + let root = workspace("changed"); + write(&root, HISTORICAL, "tests/cases/compiler/a.ts", "let a = 1;"); + write( + &root, + COMPILER_CATALOG, + "tests/cases/compiler/a.ts", + "let a = 2;", + ); + let reason = reuse_blocker(&root, HISTORICAL, CASE).expect("reads"); + assert_eq!(reason.as_deref(), Some("its input differs from 7.0.2")); + fs::remove_dir_all(root).expect("remove scratch"); + } + + #[test] + fn an_input_7_0_2_dropped_cannot_be_reused() { + let root = workspace("dropped"); + write(&root, HISTORICAL, "tests/cases/compiler/a.ts", "let a = 1;"); + let reason = reuse_blocker(&root, HISTORICAL, CASE).expect("reads"); + assert_eq!(reason.as_deref(), Some("7.0.2 has no such input")); + fs::remove_dir_all(root).expect("remove scratch"); + } + + #[test] + fn a_missing_historical_input_is_an_error_not_a_failed_cell() { + let root = workspace("damaged"); + let error = reuse_blocker(&root, HISTORICAL, CASE).expect_err("damaged tree"); + assert_eq!(error.code(), ErrorCode::Io); + fs::remove_dir_all(root).expect("remove scratch"); + } + + #[test] + fn a_project_case_also_needs_its_support_files_identical() { + let root = workspace("project"); + both(&root, "tests/cases/project/a.json", "{}"); + both(&root, "tests/cases/projects/a/a.ts", "let a = 1;"); + assert_eq!( + reuse_blocker(&root, HISTORICAL, PROJECT).expect("reads"), + None + ); + + write( + &root, + HISTORICAL, + "tests/cases/projects/.DS_Store", + "finder", + ); + assert_eq!( + reuse_blocker(&root, HISTORICAL, PROJECT).expect("reads"), + None + ); + + write( + &root, + HISTORICAL, + "tests/cases/projects/a/a.ts", + "let a = 2;", + ); + let reason = reuse_blocker(&root, HISTORICAL, PROJECT).expect("reads"); + assert_eq!( + reason.as_deref(), + Some("its project files differ from 7.0.2") + ); + fs::remove_dir_all(root).expect("remove scratch"); + } + + #[test] + fn only_the_two_historical_catalogs_are_historical() { + assert!(is_historical("typescript-6.0.2")); + assert!(is_historical("typescript-5.9.3")); + assert!(!is_historical(COMPILER_CATALOG)); + assert!(!is_historical("test262")); + } +} diff --git a/crates/bamts-verification/src/jit_runner.rs b/crates/bamts-verification/src/jit_runner.rs index 315102b7..631b352e 100644 --- a/crates/bamts-verification/src/jit_runner.rs +++ b/crates/bamts-verification/src/jit_runner.rs @@ -30,6 +30,7 @@ const RUNNER_URL: &str = "https://github.com/actions/runner/releases/download/v2 const RUNNER_SHA256: &str = "04cf0be1aff4c3ec3554466c39124ca250e3effd8873bb7e8d68535aa9505d5d"; const MAX_API_BYTES: usize = 8 * 1024 * 1024; const MAX_DOWNLOAD_BYTES: usize = 512 * 1024 * 1024; +#[cfg(any(target_os = "linux", test))] const MAX_JIT_BYTES: usize = 1024 * 1024; const WORKFLOW_PATH: &str = ".github/workflows/ci.yml"; const PERF_JOB: &str = "perf-bh1"; @@ -2003,6 +2004,7 @@ if not authorized: )) } +#[cfg(any(target_os = "linux", test))] fn read_jit_secret(mut input: impl Read) -> Result { let bytes = read_bounded(&mut input, MAX_JIT_BYTES).map_err(|_| { JitError::new( @@ -2024,6 +2026,7 @@ fn read_jit_secret(mut input: impl Read) -> Result { }) } +#[cfg(target_os = "linux")] pub fn run_internal_launcher() -> Result<()> { let secret = read_jit_secret(io::stdin().lock())?; #[allow(unsafe_code)] @@ -2050,6 +2053,15 @@ pub fn run_internal_launcher() -> Result<()> { )) } +// Refuses before reading the secret: the launcher needs Linux process hardening. +#[cfg(not(target_os = "linux"))] +pub fn run_internal_launcher() -> Result<()> { + Err(JitError::new( + JitErrorCode::Provision, + "internal launcher is supported only on Linux", + )) +} + #[cfg(test)] mod tests { use super::*; diff --git a/crates/bamts-verification/src/lane.rs b/crates/bamts-verification/src/lane.rs index d2a699a3..21255fcb 100644 --- a/crates/bamts-verification/src/lane.rs +++ b/crates/bamts-verification/src/lane.rs @@ -6,20 +6,31 @@ //! signal, protocol error, and worker crash become blocking receipts, and the //! lane continues with later cases. Process execution reuses the corpus //! normalized environment, bounded pipe drain, process-group timeout, and -//! whole-group termination. +//! whole-group termination. [`ProcessExecutor`] starts one worker per +//! request; [`ServingExecutor`] keeps one worker for many requests and records +//! a worker that exits or misses a deadline the way [`ProcessExecutor`] does. use std::{ collections::{BTreeMap, BTreeSet}, ffi::OsString, fs::{self, OpenOptions}, - io::Write, + io::{BufRead, BufReader, ErrorKind, Read, Write}, path::{Path, PathBuf}, + process::{Child, ChildStdin, Command, Stdio}, + sync::{ + Arc, Mutex, PoisonError, + mpsc::{self, Receiver, RecvTimeoutError}, + }, + thread::{self, JoinHandle}, time::{Duration, Instant}, }; use crate::{ ErrorCode, Result, VerificationError, - corpus::{OracleLimits, normalized_env, run_process}, + corpus::{ + OracleLimits, forward_host_runtime_env, normalized_env, pipe_error, run_process, + spawn_error, terminate_process_group, termination_signal, thread_error, wait_error, + }, evidence::{EvidenceRow, TerminalState, WorkingDirectoryPolicy}, shard::{ObligationKey, require_token}, }; @@ -29,6 +40,17 @@ use sha2::{Digest, Sha256}; /// Environment variable naming the worker request JSON path. pub const LANE_WORKER_REQUEST: &str = "BAMTS_LANE_WORKER_REQUEST"; +/// Environment variable that asks a lane worker to answer one request path per +/// line of standard input, instead of the one request in [`LANE_WORKER_REQUEST`]. +pub const LANE_WORKER_SERVE: &str = "BAMTS_LANE_WORKER_SERVE"; + +/// The first line a serving worker prints, before it reads any request. +pub const LANE_WORKER_READY: &str = "bamts-lane-worker ready 1"; + +/// What a serving worker prints, followed by a space and the request ID, once +/// it has written the response to that request. +pub const LANE_WORKER_DONE: &str = "bamts-lane-worker done"; + /// Version of the compiler snapshot handoff binding. pub const COMPILER_SNAPSHOT_BINDING_VERSION: u32 = 1; @@ -267,6 +289,19 @@ pub struct LaneRequest { } impl LaneRequest { + /// The same request for another key, keeping its run binding and identity. + pub fn with_key(&self, key: ObligationKey) -> Result { + Self::new( + self.binding.clone(), + self.request_id, + key, + self.observables.clone(), + self.argv.clone(), + self.working_directory, + self.timeout_ms, + ) + } + pub fn new( binding: LaneBinding, request_id: u64, @@ -350,6 +385,12 @@ pub enum LaneOutcome { Completed { artifacts: BTreeMap, }, + /// Completed, with the result measured on another cell. Derives the same + /// row as [`Self::Completed`], and the row records `source`. + Reused { + artifacts: BTreeMap, + source: String, + }, BlockingFail { detail: String, }, @@ -439,9 +480,15 @@ impl LaneResponse { /// What the parent observed about the worker process itself. #[derive(Debug, Clone, PartialEq, Eq)] pub enum ProcessObservation { - Exited { code: i32 }, + Exited { + code: i32, + }, TimedOut, - Signaled { number: i32 }, + Signaled { + number: i32, + }, + /// A serving worker said that it wrote the response, and keeps running. + Answered, } /// Raw executor result. `response_body` is untrusted until the parent parses it. @@ -490,65 +537,74 @@ impl ProcessExecutor { } } -impl LaneExecutor for ProcessExecutor { - fn run(&mut self, request: &LaneRequest) -> Result { - request.validate()?; - fs::create_dir_all(&self.work_dir).map_err(|error| { - VerificationError::new( - ErrorCode::Io, - format!("{}: {error}", self.work_dir.display()), - ) - })?; - let request_path = self.work_dir.join(format!( - "lane-{}-{}.json", - request.binding().run_id(), - request.request_id() +/// Writes `request` where its worker reads it, and returns the request and the +/// response paths. A response that an earlier run left behind is refused. +fn write_request_file(work_dir: &Path, request: &LaneRequest) -> Result<(PathBuf, PathBuf)> { + request.validate()?; + fs::create_dir_all(work_dir).map_err(|error| { + VerificationError::new(ErrorCode::Io, format!("{}: {error}", work_dir.display())) + })?; + let request_path = work_dir.join(format!( + "lane-{}-{}.json", + request.binding().run_id(), + request.request_id() + )); + let response_path = request_path.with_extension("response.json"); + if response_path.try_exists().map_err(|error| { + VerificationError::new( + ErrorCode::Io, + format!("{}: {error}", response_path.display()), + ) + })? { + return Err(VerificationError::new( + ErrorCode::Replay, + format!( + "refusing retained lane response `{}`", + response_path.display() + ), )); - let response_path = request_path.with_extension("response.json"); - if response_path.try_exists().map_err(|error| { - VerificationError::new( - ErrorCode::Io, - format!("{}: {error}", response_path.display()), - ) - })? { - return Err(VerificationError::new( - ErrorCode::Replay, - format!( - "refusing retained lane response `{}`", - response_path.display() - ), - )); - } - let encoded = serde_json::to_vec(request).map_err(|error| { - VerificationError::new(ErrorCode::Json, format!("encode lane request: {error}")) - })?; - let mut request_file = OpenOptions::new() - .create_new(true) - .write(true) - .open(&request_path) - .map_err(|error| { - VerificationError::new( - ErrorCode::Io, - format!("{}: {error}", request_path.display()), - ) - })?; - request_file.write_all(&encoded).map_err(|error| { - VerificationError::new( - ErrorCode::Io, - format!("{}: {error}", request_path.display()), - ) - })?; - request_file.sync_all().map_err(|error| { + } + let encoded = serde_json::to_vec(request).map_err(|error| { + VerificationError::new(ErrorCode::Json, format!("encode lane request: {error}")) + })?; + let mut request_file = OpenOptions::new() + .create_new(true) + .write(true) + .open(&request_path) + .map_err(|error| { VerificationError::new( ErrorCode::Io, format!("{}: {error}", request_path.display()), ) })?; + request_file.write_all(&encoded).map_err(|error| { + VerificationError::new( + ErrorCode::Io, + format!("{}: {error}", request_path.display()), + ) + })?; + request_file.sync_all().map_err(|error| { + VerificationError::new( + ErrorCode::Io, + format!("{}: {error}", request_path.display()), + ) + })?; + Ok((request_path, response_path)) +} - let mut environment = normalized_env(); - if let Ok(path) = std::env::var("PATH") { - environment.push(("PATH".to_owned(), path)); - } +/// The environment of every lane worker: the normalized corpus environment and `PATH`. +fn worker_environment() -> Vec<(String, String)> { + let mut environment = normalized_env(); + if let Ok(path) = std::env::var("PATH") { + environment.push(("PATH".to_owned(), path)); + } + environment +} + +impl LaneExecutor for ProcessExecutor { + fn run(&mut self, request: &LaneRequest) -> Result { + let (request_path, response_path) = write_request_file(&self.work_dir, request)?; + let mut environment = worker_environment(); environment.push(( LANE_WORKER_REQUEST.to_owned(), request_path.to_string_lossy().into_owned(), @@ -585,6 +641,326 @@ impl LaneExecutor for ProcessExecutor { } } +const SERVING_LABEL: &str = "serving lane worker"; + +/// How long a serving worker may take to exit after its input closes. +const SERVING_STOP_GRACE: Duration = Duration::from_secs(5); + +/// How often a serving worker that is stopping is asked whether it has exited. +const SERVING_STOP_POLL: Duration = Duration::from_millis(5); + +/// Sends every request to one long-lived worker that serves requests (see +/// [`LANE_WORKER_SERVE`]). The worker starts with the first request, and each +/// request has its own deadline. A worker that exits, or misses a deadline and +/// is terminated, gives its request the observation that [`ProcessExecutor`] +/// gives such a worker, and the next request starts a new worker. Environment, +/// process group, and termination are those of [`ProcessExecutor`]. +#[derive(Debug)] +pub struct ServingExecutor { + program: PathBuf, + args: Vec, + cwd: PathBuf, + work_dir: PathBuf, + max_output_bytes: usize, + worker: Option, + workers_started: u64, +} + +impl ServingExecutor { + pub fn new( + program: impl Into, + args: Vec, + cwd: impl Into, + work_dir: impl Into, + ) -> Self { + Self { + program: program.into(), + args, + cwd: cwd.into(), + work_dir: work_dir.into(), + max_output_bytes: DEFAULT_MAX_OUTPUT_BYTES, + worker: None, + workers_started: 0, + } + } + + /// Bounds the standard error kept for one request. + pub fn with_max_output_bytes(mut self, cap: usize) -> Self { + self.max_output_bytes = cap; + self + } + + /// How many worker processes this executor has started. + #[must_use] + pub fn workers_started(&self) -> u64 { + self.workers_started + } + + fn start_worker(&mut self) -> Result { + let mut command = Command::new(&self.program); + command.env_clear(); + for (key, value) in worker_environment() { + command.env(key, value); + } + command.env(LANE_WORKER_SERVE, "1"); + forward_host_runtime_env(&mut command); + command + .current_dir(&self.cwd) + .args(&self.args) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()); + #[cfg(unix)] + { + use std::os::unix::process::CommandExt; + command.process_group(0); + } + let mut child = command + .spawn() + .map_err(|error| spawn_error(SERVING_LABEL, &self.program, &error))?; + self.workers_started += 1; + let stdin = child + .stdin + .take() + .ok_or_else(|| pipe_error(SERVING_LABEL, "stdin"))?; + let stdout = child + .stdout + .take() + .ok_or_else(|| pipe_error(SERVING_LABEL, "stdout"))?; + let stderr = child + .stderr + .take() + .ok_or_else(|| pipe_error(SERVING_LABEL, "stderr"))?; + let (sender, lines) = mpsc::channel(); + let line_reader = thread::spawn(move || { + let mut reader = BufReader::new(stdout); + let mut line = Vec::new(); + loop { + line.clear(); + match reader.read_until(b'\n', &mut line) { + Ok(0) | Err(_) => break, + Ok(_) => { + let text = String::from_utf8_lossy(&line).trim_end().to_owned(); + if sender.send(text).is_err() { + break; + } + } + } + } + }); + let captured = Arc::new(Mutex::new(Vec::new())); + let sink = Arc::clone(&captured); + let cap = self.max_output_bytes; + let stderr_reader = thread::spawn(move || { + let mut stderr = stderr; + let mut chunk = [0_u8; 8192]; + loop { + match stderr.read(&mut chunk) { + Ok(0) => break, + Ok(read) => { + let mut kept = sink.lock().unwrap_or_else(PoisonError::into_inner); + let room = cap.saturating_sub(kept.len()); + kept.extend_from_slice(&chunk[..read.min(room)]); + } + Err(error) if error.kind() == ErrorKind::Interrupted => {} + Err(_) => break, + } + } + }); + Ok(ServingWorker { + child, + stdin, + lines, + stderr: captured, + readers: [line_reader, stderr_reader], + }) + } +} + +impl LaneExecutor for ServingExecutor { + fn run(&mut self, request: &LaneRequest) -> Result { + let (request_path, response_path) = write_request_file(&self.work_dir, request)?; + let started = Instant::now(); + let deadline = started + Duration::from_millis(request.timeout_ms()); + let mut worker = match self.worker.take() { + Some(worker) => worker, + None => { + let worker = self.start_worker()?; + match worker.await_line(|line| line == LANE_WORKER_READY, deadline) { + Heard::Line => worker, + heard => return worker.finish(heard, deadline, &response_path, started), + } + } + }; + worker.clear_stderr(); + let sent = writeln!(worker.stdin, "{}", request_path.to_string_lossy()) + .and_then(|()| worker.stdin.flush()); + let heard = if sent.is_err() { + Heard::Ended + } else { + let done = format!("{LANE_WORKER_DONE} {}", request.request_id()); + worker.await_line(|line| line == done, deadline) + }; + if heard != Heard::Line { + return worker.finish(heard, deadline, &response_path, started); + } + let result = LaneProcessResult { + observation: ProcessObservation::Answered, + response_body: fs::read(&response_path).ok(), + duration_ms: elapsed_ms(started), + detail: worker.stderr_text(), + }; + self.worker = Some(worker); + Ok(result) + } +} + +impl Drop for ServingExecutor { + fn drop(&mut self) { + if let Some(worker) = self.worker.take() { + worker.stop(); + } + } +} + +/// What a serving worker did while the parent waited for one line. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum Heard { + /// It printed the line. + Line, + /// Its output ended, because it exited or closed it. + Ended, + /// The deadline passed first. + Late, +} + +/// One running serving worker and the threads that read its output. +#[derive(Debug)] +struct ServingWorker { + child: Child, + stdin: ChildStdin, + lines: Receiver, + stderr: Arc>>, + readers: [JoinHandle<()>; 2], +} + +impl ServingWorker { + /// Waits for a line that `expected` accepts. Other lines are output of the + /// work itself, which a one-request worker also prints and nobody reads. + fn await_line(&self, expected: impl Fn(&str) -> bool, deadline: Instant) -> Heard { + loop { + let left = deadline.saturating_duration_since(Instant::now()); + match self.lines.recv_timeout(left) { + Ok(line) if expected(&line) => return Heard::Line, + Ok(_) => {} + Err(RecvTimeoutError::Timeout) => return Heard::Late, + Err(RecvTimeoutError::Disconnected) => return Heard::Ended, + } + } + } + + fn clear_stderr(&self) { + self.stderr + .lock() + .unwrap_or_else(PoisonError::into_inner) + .clear(); + } + + fn stderr_text(&self) -> String { + String::from_utf8_lossy(&self.stderr.lock().unwrap_or_else(PoisonError::into_inner)) + .into_owned() + } + + /// Ends a worker that exited or missed the deadline, and records its request + /// the way [`ProcessExecutor`] records a worker that did the same. + fn finish( + mut self, + heard: Heard, + deadline: Instant, + response_path: &Path, + started: Instant, + ) -> Result { + let mut timed_out = heard == Heard::Late; + // A worker whose output ended is exiting, and it has to exit by the deadline. + while !timed_out + && self + .child + .try_wait() + .map_err(|error| wait_error(SERVING_LABEL, error))? + .is_none() + { + if Instant::now() >= deadline { + timed_out = true; + } else { + thread::sleep(SERVING_STOP_POLL); + } + } + let termination_error = if timed_out { + terminate_process_group(&mut self.child, SERVING_LABEL).err() + } else { + None + }; + let status = self + .child + .wait() + .map_err(|error| wait_error(SERVING_LABEL, error))?; + drop(self.stdin); + let [line_reader, stderr_reader] = self.readers; + line_reader + .join() + .map_err(|_| thread_error(SERVING_LABEL, "stdout"))?; + stderr_reader + .join() + .map_err(|_| thread_error(SERVING_LABEL, "stderr"))?; + if let Some(error) = termination_error { + return Err(error); + } + let observation = if timed_out { + ProcessObservation::TimedOut + } else if let Some(number) = termination_signal(&status) { + ProcessObservation::Signaled { number } + } else { + ProcessObservation::Exited { + code: status.code().unwrap_or(1), + } + }; + let detail = + String::from_utf8_lossy(&self.stderr.lock().unwrap_or_else(PoisonError::into_inner)) + .into_owned(); + Ok(LaneProcessResult { + observation, + response_body: fs::read(response_path).ok(), + duration_ms: elapsed_ms(started), + detail, + }) + } + + /// Closes the worker's input, so it ends its loop and exits. A worker that + /// does not exit within [`SERVING_STOP_GRACE`] is terminated. + fn stop(mut self) { + drop(self.stdin); + let deadline = Instant::now() + SERVING_STOP_GRACE; + loop { + match self.child.try_wait() { + Ok(Some(_)) => break, + Ok(None) if Instant::now() < deadline => thread::sleep(SERVING_STOP_POLL), + _ => { + let _ = terminate_process_group(&mut self.child, SERVING_LABEL); + let _ = self.child.wait(); + break; + } + } + } + for reader in self.readers { + let _ = reader.join(); + } + } +} + +fn elapsed_ms(started: Instant) -> u64 { + u64::try_from(started.elapsed().as_millis()).unwrap_or(u64::MAX) +} + /// Runs every request, recording blocking evidence on failure and continuing. pub fn run_lane( executor: &mut E, @@ -624,7 +1000,7 @@ pub fn derive_row(request: &LaneRequest, process: LaneProcessResult) -> Result { + ProcessObservation::Exited { code: _ } | ProcessObservation::Answered => { match parse_response(process.response_body.as_deref()) { Err(detail) => (TerminalState::ProtocolError, BTreeMap::new(), detail), Ok(response) => derive_from_response(request, response), @@ -655,6 +1031,26 @@ fn parse_response(body: Option<&[u8]>) -> std::result::Result, + detail: String, +) -> (TerminalState, BTreeMap, String) { + let declared = request.observables(); + let observed: BTreeSet = artifacts.keys().cloned().collect(); + if &observed != declared { + return ( + TerminalState::ProtocolError, + BTreeMap::new(), + format!( + "observable set mismatch for `{}`: declared {declared:?}, observed {observed:?}", + request.key() + ), + ); + } + (TerminalState::Pass, artifacts, detail) +} + fn derive_from_response( request: &LaneRequest, response: LaneResponse, @@ -689,20 +1085,9 @@ fn derive_from_response( ); } match response.outcome { - LaneOutcome::Completed { artifacts } => { - let declared = request.observables(); - let observed: BTreeSet = artifacts.keys().cloned().collect(); - if &observed != declared { - return ( - TerminalState::ProtocolError, - BTreeMap::new(), - format!( - "observable set mismatch for `{}`: declared {declared:?}, observed {observed:?}", - request.key() - ), - ); - } - (TerminalState::Pass, artifacts, String::new()) + LaneOutcome::Completed { artifacts } => derive_pass(request, artifacts, String::new()), + LaneOutcome::Reused { artifacts, source } => { + derive_pass(request, artifacts, format!("reused {source}")) } LaneOutcome::BlockingFail { detail } => { (TerminalState::BlockingFail, BTreeMap::new(), detail) @@ -736,7 +1121,12 @@ fn derive_from_response( fn ignore_pass_attempt(body: Option<&[u8]>, cause: &str) -> String { match parse_response(body) { - Ok(response) if matches!(response.outcome(), LaneOutcome::Completed { .. }) => { + Ok(response) + if matches!( + response.outcome(), + LaneOutcome::Completed { .. } | LaneOutcome::Reused { .. } + ) => + { format!("{cause}: ignoring completed/PASS attempt from a failed worker") } _ => cause.to_string(), @@ -747,11 +1137,69 @@ fn ignore_pass_attempt(body: Option<&[u8]>, cause: &str) -> String { mod tests { use super::*; use crate::shard::ExecutionMode; + use crate::test_programs::windows_system_program; use std::collections::BTreeMap; const TEST_RUN_ID: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; const OTHER_RUN_ID: &str = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; + /// A fixture path written the Unix way, on the drive Windows needs to call it absolute. + fn abs(unix: &str) -> PathBuf { + if cfg!(windows) { + PathBuf::from(format!("C:{}", unix.replace('/', "\\"))) + } else { + PathBuf::from(unix) + } + } + + /// A worker that outlives a 200 ms timeout: `sleep 2` on Unix, and on Windows + /// `ping` with three echoes, which are one second apart. + fn sleeping_worker() -> (PathBuf, Vec) { + if cfg!(windows) { + let ping = windows_system_program("PING.EXE"); + let args = ["-n", "3", "127.0.0.1"].map(OsString::from).to_vec(); + (ping, args) + } else { + (PathBuf::from("/bin/sleep"), vec![OsString::from("2")]) + } + } + + /// A worker that exits with status 0 at once. + fn instant_worker() -> (PathBuf, Vec) { + if cfg!(windows) { + let args = ["/C", "exit", "0"].map(OsString::from).to_vec(); + (windows_system_program("cmd.exe"), args) + } else { + (PathBuf::from("/usr/bin/true"), Vec::new()) + } + } + + /// A serving worker that says it is ready and then runs `then`, a shell + /// command that is the same in `cmd.exe` and `sh`. + fn ready_worker(then: &str) -> (PathBuf, Vec) { + if cfg!(windows) { + let script = format!("echo {LANE_WORKER_READY}&{then}"); + let args = vec![OsString::from("/C"), OsString::from(script)]; + (windows_system_program("cmd.exe"), args) + } else { + let script = format!("echo '{LANE_WORKER_READY}'; {then}"); + let args = vec![OsString::from("-c"), OsString::from(script)]; + (PathBuf::from("/bin/sh"), args) + } + } + + fn scratch_dir(label: &str) -> PathBuf { + let dir = std::env::temp_dir().join(format!( + "bamts-lane-{label}-{}", + std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("time") + .as_nanos() + )); + fs::create_dir_all(&dir).expect("scratch"); + dir + } + struct ScriptedExecutor { results: Vec, index: usize, @@ -826,6 +1274,53 @@ mod tests { } } + fn reused( + observation: ProcessObservation, + key: &ObligationKey, + artifacts: BTreeMap, + ) -> LaneProcessResult { + let outcome = LaneOutcome::Reused { + artifacts, + source: "a twin cell".to_owned(), + }; + let response = + LaneResponse::new(test_binding(), 7, key.clone(), outcome).expect("response"); + LaneProcessResult { + observation, + response_body: Some(serde_json::to_vec(&response).expect("encode")), + duration_ms: 3, + detail: String::new(), + } + } + + #[test] + fn a_reused_result_passes_and_its_row_names_the_source() { + let good_key = key(ExecutionMode::Aot, "x86_64-unknown-linux-gnu", "case-0001"); + let request = request(7, good_key.clone()); + let exited = ProcessObservation::Exited { code: 0 }; + + let row = + derive_row(&request, reused(exited.clone(), &good_key, stdout_ok())).expect("derive"); + assert_eq!(row.state(), TerminalState::Pass); + assert_eq!(row.detail(), "reused a twin cell"); + + let row = derive_row(&request, reused(exited, &good_key, BTreeMap::new())).expect("derive"); + assert_eq!( + row.state(), + TerminalState::ProtocolError, + "observables still bind" + ); + + let late = reused(ProcessObservation::TimedOut, &good_key, stdout_ok()); + let row = derive_row(&request, late).expect("derive"); + assert_eq!( + row.state(), + TerminalState::Timeout, + "a late pass is not a pass" + ); + assert!(row.detail().contains("ignoring"), "{}", row.detail()); + } + #[test] fn parent_derives_pass_from_exact_response() { let good_key = key( @@ -900,7 +1395,7 @@ mod tests { "case-0001", ); let catalog = "typescript-7.0.2"; - let root = PathBuf::from("/canonical/ts-suite"); + let root = abs("/canonical/ts-suite"); let pin = b"pin"; let index = b"index"; let ledger = b"ledger"; @@ -948,7 +1443,7 @@ mod tests { #[test] fn snapshot_binding_matches_retained_bytes_not_sha256_alone() { - let root = PathBuf::from("/canonical/ts-suite"); + let root = abs("/canonical/ts-suite"); let bound = test_binding() .with_snapshot("typescript-7.0.2", &root, b"pin", b"index", b"ledger") .expect("bound"); @@ -1043,9 +1538,9 @@ mod tests { .as_nanos() )); fs::create_dir_all(&scratch).expect("scratch"); + let (program, args) = sleeping_worker(); let mut executor = - ProcessExecutor::new("/bin/sleep", vec![OsString::from("2")], "/", &scratch) - .with_max_output_bytes(64); + ProcessExecutor::new(program, args, abs("/"), &scratch).with_max_output_bytes(64); let item = key( ExecutionMode::Interpreter, "x86_64-unknown-linux-gnu", @@ -1078,8 +1573,9 @@ mod tests { .as_nanos() )); fs::create_dir_all(&scratch).expect("scratch"); + let (program, args) = instant_worker(); let mut executor = - ProcessExecutor::new("/bin/true", Vec::new(), "/", &scratch).with_max_output_bytes(64); + ProcessExecutor::new(program, args, abs("/"), &scratch).with_max_output_bytes(64); let request = request( 1, key( @@ -1109,8 +1605,9 @@ mod tests { fs::create_dir_all(&scratch).expect("scratch"); let retained = scratch.join(format!("lane-{TEST_RUN_ID}-1.response.json")); fs::write(&retained, b"stale").expect("retained response"); + let (program, args) = instant_worker(); let mut executor = - ProcessExecutor::new("/bin/true", Vec::new(), "/", &scratch).with_max_output_bytes(64); + ProcessExecutor::new(program, args, abs("/"), &scratch).with_max_output_bytes(64); let item = key( ExecutionMode::Interpreter, "x86_64-unknown-linux-gnu", @@ -1122,6 +1619,101 @@ mod tests { assert_eq!(error.code(), ErrorCode::Replay); } + #[test] + fn a_serving_worker_that_exits_is_recorded_like_an_exiting_worker_and_replaced() { + let scratch = scratch_dir("serving-exit"); + let (program, args) = ready_worker("exit 3"); + let mut executor = + ServingExecutor::new(program, args, &scratch, &scratch).with_max_output_bytes(64); + let mode = ExecutionMode::Interpreter; + let platform = "x86_64-unknown-linux-gnu"; + let first = executor + .run(&request(1, key(mode, platform, "case-first"))) + .expect("first"); + let second = executor + .run(&request(2, key(mode, platform, "case-second"))) + .expect("second"); + let started = executor.workers_started(); + drop(executor); + let _ = fs::remove_dir_all(&scratch); + assert_eq!(first.observation, ProcessObservation::Exited { code: 3 }); + assert_eq!(second.observation, ProcessObservation::Exited { code: 3 }); + assert_eq!(started, 2, "each request after an exit starts a new worker"); + let row = derive_row(&request(1, key(mode, platform, "case-first")), first).expect("row"); + assert_eq!(row.state(), TerminalState::WorkerCrash); + } + + #[test] + fn a_serving_worker_that_misses_the_deadline_is_terminated_and_times_out() { + let scratch = scratch_dir("serving-timeout"); + let (program, args) = ready_worker(if cfg!(windows) { + "ping -n 3 127.0.0.1>NUL" + } else { + "sleep 2" + }); + let mut executor = + ServingExecutor::new(program, args, &scratch, &scratch).with_max_output_bytes(64); + let request = LaneRequest::new( + test_binding(), + 1, + key( + ExecutionMode::Interpreter, + "x86_64-unknown-linux-gnu", + "case-silent", + ), + BTreeSet::from(["stdout".to_owned()]), + vec!["lane".to_owned()], + WorkingDirectoryPolicy::RepositoryRoot, + 200, + ) + .expect("short timeout"); + let result = executor.run(&request).expect("run"); + drop(executor); + let _ = fs::remove_dir_all(&scratch); + assert_eq!(result.observation, ProcessObservation::TimedOut); + let row = derive_row(&request, result).expect("row"); + assert_eq!(row.state(), TerminalState::Timeout); + } + + #[test] + fn a_serving_executor_refuses_a_retained_response() { + let scratch = scratch_dir("serving-replay"); + let retained = scratch.join(format!("lane-{TEST_RUN_ID}-1.response.json")); + fs::write(&retained, b"stale").expect("retained response"); + let (program, args) = ready_worker("exit 0"); + let mut executor = ServingExecutor::new(program, args, &scratch, &scratch); + let item = key( + ExecutionMode::Interpreter, + "x86_64-unknown-linux-gnu", + "case-replay", + ); + let error = executor + .run(&request(1, item)) + .expect_err("retained response"); + let started = executor.workers_started(); + drop(executor); + let _ = fs::remove_dir_all(&scratch); + assert_eq!(error.code(), ErrorCode::Replay); + assert_eq!(started, 0, "a refused request starts no worker"); + } + + #[test] + fn an_answered_request_derives_its_row_from_the_response() { + let item = key( + ExecutionMode::Interpreter, + "x86_64-unknown-linux-gnu", + "case-answered", + ); + let answered = LaneProcessResult { + observation: ProcessObservation::Answered, + ..success(1, &item) + }; + let row = derive_row(&request(1, item.clone()), answered).expect("row"); + let exited = derive_row(&request(1, item.clone()), success(1, &item)).expect("row"); + assert_eq!(row.state(), TerminalState::Pass); + assert_eq!(row, exited); + } + #[test] fn fresh_bindings_are_unpredictable() { let first = LaneBinding::fresh().expect("first"); diff --git a/crates/bamts-verification/src/ledger.rs b/crates/bamts-verification/src/ledger.rs index 8c69919a..62da09d0 100644 --- a/crates/bamts-verification/src/ledger.rs +++ b/crates/bamts-verification/src/ledger.rs @@ -17,7 +17,7 @@ use crate::schema::{ set_difference_detail, set_mismatch, sha256_hex, source_by_name, string_set, validate_digest, validate_manifest, validate_sha256, }; -use crate::{ErrorCode, Gate, GateReport, Result, VerificationError}; +use crate::{ErrorCode, Gate, GateReport, Result, VerificationError, rebuild::LEDGER_SCHEMA}; const TOOLCHAINS_PATH: &str = "formal/toolchains.toml"; const LEDGER_PATH: &str = "proof/completeness-ledger.json"; @@ -114,7 +114,6 @@ struct LedgerMetadata { } struct LedgerRow { - case: String, catalog: String, citation: String, id: String, @@ -162,7 +161,6 @@ impl LedgerState { for (field, value) in [ ("catalog", row.catalog.as_str()), - ("case", row.case.as_str()), ("id", row.id.as_str()), ("state", row.state.as_str()), ] { @@ -175,13 +173,13 @@ impl LedgerState { } } - let expected_id = format!("{}:{}", row.catalog, row.case); - if row.id != expected_id { + let case = row.id.strip_prefix(&format!("{}:", row.catalog)); + if !case.is_some_and(is_nonempty) { self.record_error(VerificationError::new( ErrorCode::Schema, format!( - "{LEDGER_PATH}: row id `{}` does not equal `{expected_id}`", - row.id + "{LEDGER_PATH}: row id `{}` names no case in catalog `{}`", + row.id, row.catalog ), )); return; @@ -487,11 +485,11 @@ fn stream_ledger( } fn validate_ledger_metadata(ledger: &LedgerMetadata, manifest_sha256: &str) -> Result<()> { - if ledger.schema != "bamti.completeness-ledger/v1" { + if ledger.schema != LEDGER_SCHEMA { return Err(VerificationError::new( ErrorCode::Schema, format!( - "{LEDGER_PATH}: expected schema `bamti.completeness-ledger/v1`, found `{}`", + "{LEDGER_PATH}: expected schema `{LEDGER_SCHEMA}`, found `{}`", ledger.schema ), )); @@ -735,7 +733,6 @@ impl<'de, 'a> Visitor<'de> for RowVisitor<'a> { where A: MapAccess<'de>, { - let mut case = None; let mut catalog = None; let mut citation = None; let mut id = None; @@ -751,7 +748,6 @@ impl<'de, 'a> Visitor<'de> for RowVisitor<'a> { } match key.as_str() { - "case" => read_ledger_string(&mut map, &mut case, "row", "case", self.state)?, "catalog" => { read_ledger_string(&mut map, &mut catalog, "row", "catalog", self.state)? } @@ -776,13 +772,8 @@ impl<'de, 'a> Visitor<'de> for RowVisitor<'a> { } let missing = [ - ("case", case.is_none()), ("catalog", catalog.is_none()), - ("citation", citation.is_none()), ("id", id.is_none()), - ("matcher", matcher.is_none()), - ("owner", owner.is_none()), - ("reason", reason.is_none()), ("state", state.is_none()), ] .into_iter() @@ -795,28 +786,17 @@ impl<'de, 'a> Visitor<'de> for RowVisitor<'a> { return Ok(()); } - let ( - Some(case), - Some(catalog), - Some(citation), - Some(id), - Some(matcher), - Some(owner), - Some(reason), - Some(state), - ) = (case, catalog, citation, id, matcher, owner, reason, state) - else { + let (Some(catalog), Some(id), Some(state)) = (catalog, id, state) else { return Ok(()); }; self.state.process_row(LedgerRow { - case, catalog, - citation, + citation: citation.unwrap_or_default(), id, - matcher, - owner, - reason, + matcher: matcher.unwrap_or_default(), + owner: owner.unwrap_or_default(), + reason: reason.unwrap_or_default(), state, }); Ok(()) @@ -1025,7 +1005,6 @@ mod tests { "BLOCKING_FAIL" }; json!({ - "case": case, "catalog": catalog, "citation": format!("https://example.invalid/{id}"), "id": id, @@ -1039,7 +1018,7 @@ mod tests { json!({ "manifest_sha256": manifest_sha256, "rows": rows, - "schema": "bamti.completeness-ledger/v1", + "schema": LEDGER_SCHEMA, "states": P0_STATES, }) } @@ -1162,13 +1141,73 @@ mod tests { ); } + #[test] + fn a_pass_row_needs_no_prose_and_a_non_pass_row_does() { + let bare = |ledger: &mut Value| { + let row = &mut ledger["rows"].as_array_mut().unwrap()[0]; + for field in ["citation", "matcher", "owner", "reason"] { + row.as_object_mut().unwrap().remove(field); + } + }; + let fixture_a = fixture(); + mutate_ledger(&fixture_a, |ledger| { + bare(ledger); + ledger["rows"][0]["state"] = json!("PASS"); + }); + verify_ledger_g0(&fixture_a.root).expect("a PASS row is three fields"); + + let fixture_b = fixture(); + mutate_ledger(&fixture_b, bare); + assert_eq!( + verify_ledger_g0(&fixture_b.root).unwrap_err().code(), + ErrorCode::Transition + ); + } + + #[test] + fn a_v1_ledger_is_not_read_as_v2() { + let schema = fixture(); + mutate_ledger(&schema, |ledger| { + ledger["schema"] = json!("bamti.completeness-ledger/v1"); + }); + assert_eq!( + verify_ledger_g0(&schema.root).unwrap_err().code(), + ErrorCode::Schema + ); + + let row = fixture(); + mutate_ledger(&row, |ledger| { + ledger["rows"][0]["case"] = json!("fixture"); + }); + assert_eq!( + verify_ledger_g0(&row.root).unwrap_err().code(), + ErrorCode::Schema + ); + } + + #[test] + fn a_row_id_must_name_a_case_in_its_catalog() { + for id in ["typescript-6.0.2:elsewhere", "typescript-7.0.2:"] { + let fixture = fixture(); + mutate_ledger(&fixture, |ledger| { + let row = &mut ledger["rows"].as_array_mut().unwrap()[0]; + row["catalog"] = json!("typescript-7.0.2"); + row["id"] = json!(id); + }); + assert_eq!( + verify_ledger_g0(&fixture.root).unwrap_err().code(), + ErrorCode::Schema, + "{id}" + ); + } + } + #[test] fn rejects_extra_ledger_rows() { let fixture = fixture(); mutate_ledger(&fixture, |ledger| { let rows = ledger["rows"].as_array_mut().unwrap(); let mut extra = rows[0].clone(); - extra["case"] = json!("case-extra"); extra["id"] = json!("typescript-7.0.2:case-extra"); rows.push(extra); }); diff --git a/crates/bamts-verification/src/lib.rs b/crates/bamts-verification/src/lib.rs index bed7ad50..1b2308d6 100644 --- a/crates/bamts-verification/src/lib.rs +++ b/crates/bamts-verification/src/lib.rs @@ -10,6 +10,9 @@ pub mod facets; pub mod fixtures; pub mod formal_bridge; pub mod formal_gates; +pub(crate) mod fsync; +pub mod historical; +#[cfg(unix)] pub mod jit_runner; pub mod lane; pub mod leaf_evidence; @@ -27,6 +30,11 @@ pub(crate) mod schema; pub mod shard; pub mod source_fetch; pub mod suite; +pub mod sweep_report; +#[cfg(test)] +mod test_programs; +pub mod toolchain_image; +pub mod toolchain_measure; pub mod toolchain_schema; pub mod ts_ledger; pub mod unicode; diff --git a/crates/bamts-verification/src/main.rs b/crates/bamts-verification/src/main.rs index cd39eb4f..25c75e53 100644 --- a/crates/bamts-verification/src/main.rs +++ b/crates/bamts-verification/src/main.rs @@ -29,9 +29,18 @@ use bamts_verification::{ rebuild::{RebuildMode, discover_receipts, rebuild_ledger}, shard::ShardSpec, source_fetch::{CommandBackend, materialize_named}, - suite::completion::{SuiteMergeRequest, SuiteRunRequest, merge_suite, run_suite}, + suite::completion::{ + SuiteMergeRequest, SuiteRecountRequest, SuiteRunRequest, merge_suite, recount_suite, + run_suite, + }, + sweep_report::{ + DiffRequest, FamiliesRequest, StatusRequest, diff as diff_receipts, families, + render_recount, status_report, + }, + toolchain_measure::measure_cell, toolchain_schema::{ blocking_obligations, catalog_identifiers, emit_probe_object, load_target_cells, + write_committed_record, }, unicode, workspace_guard::audit_workspace, @@ -80,6 +89,10 @@ enum Command { check: bool, }, ToolchainEvidence, + ToolchainMeasure { + target: String, + write: bool, + }, ToolchainObject { target: String, out: PathBuf, @@ -107,6 +120,14 @@ enum Command { }, SuiteRun(SuiteRunRequest), SuiteMerge(SuiteMergeRequest), + SuiteFamilies(FamiliesRequest), + SuiteDiff(DiffRequest), + SuiteStatus(StatusRequest), + SuiteRecount { + request: SuiteRecountRequest, + limit: usize, + fail_on_regression: bool, + }, } fn main() { @@ -152,6 +173,7 @@ fn dispatch(root: &Path, command: Command) -> Result<()> { Command::CompletionRegenerate { check } => regenerate_completion_command(root, check), Command::DiagnosticsRegenerate { check } => regenerate_diagnostics(root, check), Command::ToolchainEvidence => audit_toolchain_evidence(root), + Command::ToolchainMeasure { target, write } => measure_toolchain_cell(root, &target, write), Command::ToolchainObject { target, out } => emit_toolchain_object(&target, &out), Command::PerfStage0 => run_perf_stage0(), Command::PerfJit => run_perf_jit(), @@ -169,7 +191,41 @@ fn dispatch(root: &Path, command: Command) -> Result<()> { Command::ReplayCanary { faulty } => run_replay_canary_child(root, faulty), Command::SuiteRun(request) => run_suite(root, &request).map(|_| ()), Command::SuiteMerge(request) => merge_suite(root, &request).map(|_| ()), + Command::SuiteFamilies(request) => print_report(&families(&request)?), + Command::SuiteDiff(request) => { + let (report, regressions) = diff_receipts(&request)?; + print_report(&report)?; + fail_on_regressions(request.fail_on_regression, regressions) + } + Command::SuiteStatus(request) => print_report(&format!("{}\n", status_report(&request)?)), + Command::SuiteRecount { + request, + limit, + fail_on_regression, + } => { + let recount = recount_suite(root, &request)?; + let (report, regressions) = + render_recount(&request.family, &recount.before, &recount.after, limit); + print_report(&report)?; + fail_on_regressions(fail_on_regression, regressions) + } + } +} + +fn fail_on_regressions(enabled: bool, regressions: usize) -> Result<()> { + if enabled && regressions > 0 { + return Err(VerificationError::new( + ErrorCode::Transition, + format!("{regressions} row(s) regressed from PASS"), + )); } + Ok(()) +} + +fn print_report(report: &str) -> Result<()> { + write!(io::stdout().lock(), "{report}").map_err(|error| { + VerificationError::new(ErrorCode::Io, format!("cannot write report: {error}")) + }) } fn parse_command(mut arguments: impl Iterator) -> Result { @@ -277,11 +333,101 @@ fn parse_suite_command(arguments: &mut impl Iterator) -> Result }, })) } + "families" => { + let receipt = path_argument(arguments, "suite families receipt")?; + let flags = parse_flags(arguments, &["--top", "--family", "--root"], &[])?; + Ok(Command::SuiteFamilies(FamiliesRequest { + receipt, + top: count_flag(&flags.values, "--top", 20)?, + family: flags.values.get("--family").cloned(), + root: flags.values.get("--root").map(PathBuf::from), + })) + } + "diff" => { + let a = path_argument(arguments, "first receipt of suite diff")?; + let b = path_argument(arguments, "second receipt of suite diff")?; + let flags = parse_flags(arguments, &["--limit"], &["--fail-on-regression"])?; + Ok(Command::SuiteDiff(DiffRequest { + a, + b, + limit: count_flag(&flags.values, "--limit", 20)?, + fail_on_regression: flags.switches.contains("--fail-on-regression"), + })) + } + "status" => { + let flags = parse_flags(arguments, &["--receipts", "--out"], &[])?; + Ok(Command::SuiteStatus(StatusRequest { + receipts: PathBuf::from(required_flag( + &flags.values, + "--receipts", + "suite status", + )?), + out: PathBuf::from(required_flag(&flags.values, "--out", "suite status")?), + })) + } + "recount" => { + let flags = parse_flags( + arguments, + &[ + "--catalog", + "--runner", + "--baseline", + "--family", + "--limit", + "--guard", + ], + &["--fail-on-regression"], + )?; + let guard = count_flag(&flags.values, "--guard", 500)?; + let fail_on_regression = flags.switches.contains("--fail-on-regression"); + if fail_on_regression && guard == 0 { + return Err(usage( + "`--fail-on-regression` needs passing rows to watch, so `--guard` cannot be 0", + )); + } + Ok(Command::SuiteRecount { + request: SuiteRecountRequest { + catalog: required_flag(&flags.values, "--catalog", "suite recount")?, + runner: required_flag(&flags.values, "--runner", "suite recount")?, + baseline: PathBuf::from(required_flag( + &flags.values, + "--baseline", + "suite recount", + )?), + family: required_flag(&flags.values, "--family", "suite recount")?, + guard, + }, + limit: count_flag(&flags.values, "--limit", 20)?, + fail_on_regression, + }) + } _ if subcommand.starts_with("--") => Err(usage(format!("unknown flag `{subcommand}`"))), _ => Err(usage(format!("unknown suite subcommand `{subcommand}`"))), } } +/// A positional path. A word that looks like a flag is a missing path. +fn path_argument( + arguments: &mut impl Iterator, + description: &str, +) -> Result { + let value = required_argument(arguments, description)?; + if value.starts_with("--") { + return Err(usage(format!( + "missing {description}, found flag `{value}`" + ))); + } + Ok(PathBuf::from(value)) +} + +fn count_flag(values: &BTreeMap, flag: &str, default: usize) -> Result { + values.get(flag).map_or(Ok(default), |value| { + value + .parse::() + .map_err(|_| usage(format!("`{flag}` must be an unsigned integer"))) + }) +} + /// Parses low-level suite coordinates as zero-based `/`. /// `ts_conformance --shards` is intentionally a separate one-based surface. fn parse_shard(value: &str) -> Result { @@ -497,6 +643,14 @@ fn parse_toolchain_command(arguments: &mut impl Iterator) -> Re reject_extra_arguments(arguments, &[])?; Ok(Command::ToolchainEvidence) } + "measure" => { + let flags = parse_flags(arguments, &["--target"], &["--write"])?; + let target = required_flag(&flags.values, "--target", "toolchain measure")?; + Ok(Command::ToolchainMeasure { + target, + write: flags.switches.contains("--write"), + }) + } "object" => { let flags = parse_flags(arguments, &["--target", "--out"], &[])?; let target = required_flag(&flags.values, "--target", "toolchain object")?; @@ -1006,6 +1160,32 @@ fn audit_toolchain_evidence(root: &Path) -> Result<()> { catalog_identifiers().len(), )) } +fn measure_toolchain_cell(root: &Path, target: &str, write: bool) -> Result<()> { + let work = root.join("target/toolchain-measure").join(target); + let measured = measure_cell(target, &work)?; + for note in &measured.notes { + emit(&format!("NOTE {note}\n"))?; + } + for (obligation, evidence) in measured.record.obligations() { + emit(&format!( + "{obligation} {} {}\n", + evidence.status().as_str(), + evidence.reason() + ))?; + } + if write { + let path = write_committed_record(root, &measured.record)?; + emit(&format!("TOOLCHAIN MEASURE WROTE {}\n", path.display())) + } else if measured.record.is_closed() { + emit(&format!("TOOLCHAIN MEASURE PASS target={target}\n")) + } else { + Err(VerificationError::new( + ErrorCode::SetMismatch, + format!("target cell `{target}` did not close"), + )) + } +} + fn emit_toolchain_object(target: &str, out: &Path) -> Result<()> { let object = emit_probe_object(target, out)?; emit(&format!( @@ -1357,6 +1537,226 @@ mod tests { assert_eq!(request.publish, PublishMode::Check); } + #[test] + fn parses_suite_families_with_its_optional_flags() { + let Command::SuiteFamilies(plain) = + parse(&["bamts-verification", "suite", "families", "merged.jsonl"]).expect("plain") + else { + panic!("suite families variant") + }; + assert_eq!(plain.receipt, PathBuf::from("merged.jsonl")); + assert_eq!(plain.top, 20); + assert_eq!(plain.family, None); + assert_eq!(plain.root, None); + + let Command::SuiteFamilies(one) = parse(&[ + "bamts-verification", + "suite", + "families", + "merged.jsonl", + "--top", + "5", + "--family", + "types/mismatch", + "--root", + "target/authority/test262", + ]) + .expect("flags") else { + panic!("suite families variant") + }; + assert_eq!(one.top, 5); + assert_eq!(one.family.as_deref(), Some("types/mismatch")); + assert_eq!(one.root, Some(PathBuf::from("target/authority/test262"))); + } + + #[test] + fn parses_suite_diff_with_two_receipts_and_a_regression_switch() { + let Command::SuiteDiff(request) = parse(&[ + "bamts-verification", + "suite", + "diff", + "before.jsonl", + "after.jsonl", + "--limit", + "3", + "--fail-on-regression", + ]) + .expect("diff") else { + panic!("suite diff variant") + }; + assert_eq!(request.a, PathBuf::from("before.jsonl")); + assert_eq!(request.b, PathBuf::from("after.jsonl")); + assert_eq!(request.limit, 3); + assert!(request.fail_on_regression); + + let Command::SuiteDiff(defaults) = + parse(&["bamts-verification", "suite", "diff", "a.jsonl", "b.jsonl"]) + .expect("defaults") + else { + panic!("suite diff variant") + }; + assert_eq!(defaults.limit, 20); + assert!(!defaults.fail_on_regression); + } + + #[test] + fn parses_suite_recount_with_its_required_and_optional_flags() { + let Command::SuiteRecount { + request, + limit, + fail_on_regression, + } = parse(&[ + "bamts-verification", + "suite", + "recount", + "--catalog", + "typescript-7.0.2", + "--runner", + "compiler", + "--baseline", + "merged.jsonl", + "--family", + "types/mismatch", + "--limit", + "4", + "--fail-on-regression", + ]) + .expect("recount") + else { + panic!("suite recount variant") + }; + assert_eq!(request.catalog, "typescript-7.0.2"); + assert_eq!(request.runner, "compiler"); + assert_eq!(request.baseline, PathBuf::from("merged.jsonl")); + assert_eq!(request.family, "types/mismatch"); + assert_eq!(request.guard, 500); + assert_eq!(limit, 4); + assert!(fail_on_regression); + + let unguarded = parse(&[ + "bamts-verification", + "suite", + "recount", + "--catalog", + "test262", + "--runner", + "jit", + "--baseline", + "merged.jsonl", + "--family", + "types/mismatch", + "--guard", + "0", + "--fail-on-regression", + ]); + assert!( + unguarded.is_err_and(|error| error.code() == ErrorCode::Usage), + "a regression check with nothing to watch" + ); + + let missing = parse(&[ + "bamts-verification", + "suite", + "recount", + "--catalog", + "test262", + "--runner", + "jit", + "--baseline", + "m.jsonl", + ]) + .expect_err("a family is required"); + assert_eq!(missing.code(), ErrorCode::Usage); + assert!(missing.to_string().contains("--family")); + } + + #[test] + fn parses_suite_status_with_both_paths_and_requires_each() { + let Command::SuiteStatus(request) = parse(&[ + "bamts-verification", + "suite", + "status", + "--receipts", + "target/evidence-local/merged", + "--out", + "target/evidence-local/status.md", + ]) + .expect("status") else { + panic!("suite status variant") + }; + assert_eq!( + request.receipts, + PathBuf::from("target/evidence-local/merged") + ); + assert_eq!( + request.out, + PathBuf::from("target/evidence-local/status.md") + ); + + for (missing, arguments) in [ + ("--out", vec!["--receipts", "merged"]), + ("--receipts", vec!["--out", "status.md"]), + ] { + let mut command = vec!["bamts-verification", "suite", "status"]; + command.extend(arguments); + let error = parse(&command).expect_err("a required flag is missing"); + assert_eq!(error.code(), ErrorCode::Usage); + assert!(error.to_string().contains(missing), "{error}"); + } + } + + #[test] + fn rejects_malformed_suite_report_arguments() { + for arguments in [ + vec!["bamts-verification", "suite", "families"], + vec!["bamts-verification", "suite", "families", "--top", "5"], + vec![ + "bamts-verification", + "suite", + "families", + "r.jsonl", + "--top", + "many", + ], + vec![ + "bamts-verification", + "suite", + "families", + "r.jsonl", + "--top", + ], + vec!["bamts-verification", "suite", "diff", "only-one.jsonl"], + vec![ + "bamts-verification", + "suite", + "diff", + "a.jsonl", + "--limit", + "2", + ], + vec![ + "bamts-verification", + "suite", + "diff", + "a.jsonl", + "b.jsonl", + "--limit", + "x", + ], + vec![ + "bamts-verification", + "suite", + "diff", + "a.jsonl", + "b.jsonl", + "c.jsonl", + ], + ] { + let error = parse(&arguments).expect_err(&format!("{arguments:?}")); + assert_eq!(error.code(), ErrorCode::Usage, "{arguments:?}"); + } + } + #[test] fn rejects_malformed_suite_shard() { let error = parse(&[ diff --git a/crates/bamts-verification/src/oracles/test262.rs b/crates/bamts-verification/src/oracles/test262.rs index 2ca86528..457f1306 100644 --- a/crates/bamts-verification/src/oracles/test262.rs +++ b/crates/bamts-verification/src/oracles/test262.rs @@ -10,7 +10,8 @@ use bamts_bytecode::EcmaString; use bamts_compiler::diagnostic::Diagnostic; -use bamts_runtime::{RuntimeErrorKind, ThrowOrigin, run_linked_program}; +use bamts_compiler::lint::{LintLevel, LintOverride, LintProfile, LintTable}; +use bamts_runtime::{CancellationToken, RuntimeErrorKind, ThrowOrigin, run_linked_program}; use serde::Deserialize; use std::fs; use std::path::{Component, Path, PathBuf}; @@ -25,7 +26,32 @@ pub const DEFAULT_ASYNC_DEADLINE: Duration = Duration::from_millis(500); const HARNESS_ASSERT: &str = "assert.js"; const HARNESS_STA: &str = "sta.js"; const HARNESS_DONE: &str = "doneprintHandle.js"; +/// What names a file a test may import from its own directory. +const FIXTURE_MARKER: &str = "_FIXTURE"; const STRICT_PREFIX: &str = "\"use strict\";\n"; +/// The host's `$262`, for every test that gets a harness. A member exists only +/// where the engine truly does the work, so a test of a missing facility fails. +/// `INTERPRETING.md` makes it a writable, configurable, non-enumerable global, +/// and the methods capture what they use because tests replace builtins. +const HOST_PRELUDE: &str = "(function (global) {\n\ + var apply = Reflect.apply;\n\ + var transfer = ArrayBuffer.prototype.transfer;\n\ + var evaluate = global.eval;\n\ + Object.defineProperty(global, '$262', {\n\ + value: {\n\ + global: global,\n\ + detachArrayBuffer: function (buffer) { apply(transfer, buffer, []); return null; },\n\ + evalScript: function (source) { return evaluate(source); },\n\ + },\n\ + writable: true,\n\ + enumerable: false,\n\ + configurable: true,\n\ + });\n\ + })(globalThis);\n"; +/// Ends a module case. The compiler reads the goal from the tree, as `tsc` does, +/// so this is what makes a test with no `import` or `export` of its own a module. +/// It exports no name, so it cannot collide with one the test declares. +const MODULE_MARKER: &str = "\nexport {};\n"; #[derive(Debug, Clone, PartialEq, Eq)] pub enum OracleError { @@ -50,6 +76,8 @@ pub enum OracleError { ExpectationMismatch { expected: String, actual: String, + /// What the throw said, when it said anything. + message: String, }, /// The runner could not observe the script at all. Always blocking. BlockedRun { @@ -257,6 +285,8 @@ pub struct ExecutionPlan { pub harness: HarnessPlan, pub negative: Option, pub async_done: bool, + /// The `[[CanBlock]]` of the agent that must run the test. + pub can_block: bool, } #[derive(Debug, Clone, PartialEq, Eq)] @@ -280,6 +310,7 @@ pub struct RunRequest { pub script: ComposedScript, pub negative: Option, pub async_done: bool, + pub can_block: bool, pub deadline: Duration, } @@ -287,6 +318,8 @@ pub struct RunRequest { pub struct ThrownError { pub phase: NegativePhase, pub error_type: NegativeType, + /// What the throw said. Never part of a verdict. + pub message: String, } #[derive(Debug, Clone, Copy, PartialEq, Eq)] @@ -554,6 +587,7 @@ pub fn plan_execution( harness, negative: frontmatter.negative.clone(), async_done, + can_block: !frontmatter.has(Flag::CanBlockIsFalse), }) } @@ -563,11 +597,7 @@ pub fn compose_script( variant: &ExecutionVariant, harness_sources: &[HarnessSource], ) -> Result { - let early = plan - .negative - .as_ref() - .is_some_and(|negative| negative.phase.is_early()); - if early || variant.kind == SourceKind::Raw { + if variant.kind == SourceKind::Raw { return Ok(ComposedScript { bytes: parsed.source_bytes.clone(), test_offset: 0, @@ -576,8 +606,18 @@ pub fn compose_script( }); } - let mut prefix = Vec::new(); - if let HarnessPlan::Canonical { files, .. } = &plan.harness { + // `INTERPRETING.md`: the directive is the initial character sequence of the + // file and precedes every other text modification, so it leads the harness. + let mut bytes = Vec::new(); + if variant.kind == SourceKind::ScriptStrict { + bytes.extend_from_slice(STRICT_PREFIX.as_bytes()); + } + let early = plan + .negative + .as_ref() + .is_some_and(|negative| negative.phase.is_early()); + if !early && let HarnessPlan::Canonical { files, .. } = &plan.harness { + bytes.extend_from_slice(HOST_PRELUDE.as_bytes()); for file in files { let source = harness_sources .iter() @@ -586,22 +626,23 @@ pub fn compose_script( .ok_or_else(|| OracleError::MissingHarness { name: file.requested.clone(), })?; - prefix.extend_from_slice(source); + bytes.extend_from_slice(source); if !source.ends_with(b"\n") { - prefix.push(b'\n'); + bytes.push(b'\n'); } } } - if variant.kind == SourceKind::ScriptStrict { - prefix.extend_from_slice(STRICT_PREFIX.as_bytes()); + let test_offset = bytes.len(); + bytes.extend_from_slice(&parsed.source_bytes); + if variant.kind == SourceKind::Module { + bytes.extend_from_slice(MODULE_MARKER.as_bytes()); } - let test_offset = prefix.len(); - prefix.extend_from_slice(&parsed.source_bytes); + let untouched = bytes.len() == parsed.source_bytes.len(); Ok(ComposedScript { - bytes: prefix, + bytes, test_offset, kind: variant.kind, - untouched: false, + untouched, }) } @@ -629,6 +670,32 @@ pub fn judge_done(trace: &AsyncTrace) -> Result<(), DoneFailure> { } } +/// What a failed run reports. A mismatch reads as its cause, so runs that failed +/// the same way group together in a sweep report. +fn failure_detail(error: &OracleError) -> String { + match error { + OracleError::ExpectationMismatch { + expected, + actual, + message, + } if message.is_empty() => format!("expected {expected}, got {actual}"), + OracleError::ExpectationMismatch { + expected, + actual, + message, + } => format!("expected {expected}, got {actual}: {message}"), + other => format!("{other:?}"), + } +} + +/// The first diagnostic of a rejected parse, which is what the parser objected to. +fn parse_message(diagnostics: &[Diagnostic]) -> String { + diagnostics + .first() + .map(|diagnostic| format!("{} {}", diagnostic.code().as_str(), diagnostic.message())) + .unwrap_or_default() +} + pub fn judge_run(request: &RunRequest, outcome: &RunOutcome) -> Result<(), OracleError> { match outcome { RunOutcome::Async(trace) => { @@ -636,6 +703,7 @@ pub fn judge_run(request: &RunRequest, outcome: &RunOutcome) -> Result<(), Oracl return Err(OracleError::ExpectationMismatch { expected: "non-async".to_owned(), actual: "async".to_owned(), + message: String::new(), }); } match judge_done(trace) { @@ -644,6 +712,7 @@ pub fn judge_run(request: &RunRequest, outcome: &RunOutcome) -> Result<(), Oracl Some(expected) => Err(OracleError::ExpectationMismatch { expected: format!("{:?}/{}", expected.phase, expected.error_type.as_str()), actual: "async completion".to_owned(), + message: String::new(), }), }, Err(DoneFailure::Error) => match &request.negative { @@ -676,6 +745,7 @@ pub fn judge_run(request: &RunRequest, outcome: &RunOutcome) -> Result<(), Oracl trace.failure_value.as_deref().unwrap_or_default() ), }, + message: trace.failure_value.clone().unwrap_or_default(), }) } } @@ -698,16 +768,19 @@ pub fn judge_run(request: &RunRequest, outcome: &RunOutcome) -> Result<(), Oracl Err(OracleError::ExpectationMismatch { expected: format!("{:?}/{}", expected.phase, expected.error_type.as_str()), actual: format!("{:?}/{}", actual.phase, actual.error_type.as_str()), + message: actual.message.clone(), }) } } (Some(expected), None) => Err(OracleError::ExpectationMismatch { expected: format!("{:?}/{}", expected.phase, expected.error_type.as_str()), actual: "completed".to_owned(), + message: String::new(), }), (None, Some(actual)) => Err(OracleError::ExpectationMismatch { expected: "success".to_owned(), actual: format!("{:?}/{}", actual.phase, actual.error_type.as_str()), + message: actual.message.clone(), }), }, RunOutcome::Blocked { detail } => Err(OracleError::BlockedRun { @@ -744,13 +817,14 @@ pub fn evaluate_in_all_modes( script, negative: plan.negative.clone(), async_done: plan.async_done, + can_block: plan.can_block, deadline, }; let outcome = runner.run(&request); if let Err(error) = judge_run(&request, &outcome) { return Err(OracleError::ModeFailure(ModeFailure { mode, - detail: format!("{error:?}"), + detail: failure_detail(&error), })); } } @@ -793,13 +867,14 @@ pub fn evaluate_in_mode( script, negative: plan.negative.clone(), async_done: plan.async_done, + can_block: plan.can_block, deadline, }; let outcome = runner.run(&request); if let Err(error) = judge_run(&request, &outcome) { return Err(OracleError::ModeFailure(ModeFailure { mode, - detail: format!("{error:?}"), + detail: failure_detail(&error), })); } } @@ -839,16 +914,29 @@ pub fn evaluate_in_mode( pub struct InterpreterRunner { scratch: PathBuf, composed: AtomicU64, + origin: Option, } -/// Removes a materialized script on every exit path, including early `?`-less +/// Removes a materialized case on every exit path, including early `?`-less /// classification returns and panics. -struct MaterializedScript(PathBuf); +struct MaterializedCase(PathBuf); -impl Drop for MaterializedScript { +impl Drop for MaterializedCase { fn drop(&mut self) { - let _ = fs::remove_file(&self.0); + let _ = fs::remove_dir_all(&self.0); + } +} + +/// Copies the `_FIXTURE` files beside a test, which it imports by relative path. +fn copy_fixtures(from: &Path, to: &Path) -> std::io::Result<()> { + for entry in fs::read_dir(from)? { + let entry = entry?; + let name = entry.file_name(); + if name.to_string_lossy().contains(FIXTURE_MARKER) && entry.file_type()?.is_file() { + fs::copy(entry.path(), to.join(name))?; + } } + Ok(()) } impl InterpreterRunner { @@ -860,8 +948,38 @@ impl InterpreterRunner { Self { scratch: scratch.into(), composed: AtomicU64::new(0), + origin: None, } } + + /// Materializes each composed script as `test_file` would be found: under + /// its name, beside copies of its fixtures. Relative imports then resolve, + /// including a module test that imports itself. + #[must_use] + pub fn for_test_file(mut self, test_file: impl Into) -> Self { + self.origin = Some(test_file.into()); + self + } + + fn materialize(&self, request: &RunRequest, directory: &Path) -> std::io::Result { + fs::create_dir_all(directory)?; + let origin = self.origin.as_deref(); + if let Some(parent) = origin.and_then(Path::parent) + && request + .script + .bytes + .windows(FIXTURE_MARKER.len()) + .any(|window| window == FIXTURE_MARKER.as_bytes()) + { + copy_fixtures(parent, directory)?; + } + let name = origin + .and_then(Path::file_name) + .map_or_else(|| "composed.js".into(), std::ffi::OsStr::to_os_string); + let script = directory.join(name); + fs::write(&script, &request.script.bytes)?; + Ok(script) + } } impl Test262Runner for InterpreterRunner { @@ -875,20 +993,30 @@ impl Test262Runner for InterpreterRunner { }; } let unique = self.composed.fetch_add(1, Ordering::Relaxed); - let script = MaterializedScript( + let case = MaterializedCase( self.scratch - .join(format!("composed-{}-{unique}.js", std::process::id())), + .join(format!("case-{}-{unique}", std::process::id())), ); - if let Err(error) = fs::write(&script.0, &request.script.bytes) { - return RunOutcome::Blocked { - detail: format!( - "could not materialize composed script `{}`: {error}", - script.0.display() - ), - }; - } + let script = match self.materialize(request, &case.0) { + Ok(script) => script, + Err(error) => { + return RunOutcome::Blocked { + detail: format!( + "could not materialize composed script in `{}`: {error}", + case.0.display() + ), + }; + } + }; - let executable = match bamts::compile_source_file(&script.0) { + let lints = conformance_lints(); + let cancel = CancellationToken::new(); + let compiled = if request.script.kind == SourceKind::Module { + bamts::compile_source_file_with_lints(&script, &lints, &cancel) + } else { + bamts::compile_script_file(&script, &lints, &cancel) + }; + let executable = match compiled { Ok(executable) => executable, Err(bamts::Error::Diagnostics { diagnostics }) => { return parse_phase_outcome(&diagnostics); @@ -900,9 +1028,14 @@ impl Test262Runner for InterpreterRunner { } }; - let limits = interpreter_fuel(request.deadline); + let limits = bamts_runtime::Limits { + agent_can_suspend: request.can_block, + ..interpreter_fuel(request.deadline) + }; let started = Instant::now(); let mut host = bamts_node::NodeHost::new(); + // Local time needs a zone, and test262 accepts any of them. + host.set_env("TZ", "UTC"); // Dynamic `Function(...)` synthesis needs a compile provider; the // facade's ScriptCompiler is the same one `bamts::run_program` uses. host.set_script_compiler(Box::new(bamts::ScriptCompiler)); @@ -1008,6 +1141,16 @@ fn synthetic_runtime_error(error: bamts_runtime::NativeError) -> bamts_runtime:: /// JavaScript parse errors. Any other diagnostic means the oracle pipeline /// rejected source that Test262 considers valid, which is an observation /// failure and never a Test262 verdict. +/// The lints a conformance run compiles under. A lint is project policy, so +/// one must not refuse a program the language allows. +fn conformance_lints() -> LintTable { + let mut lints = LintTable::new(LintProfile::Default); + lints + .apply_cli([LintOverride::new("no-with", LintLevel::Allow, "test262")]) + .expect("the default profile forbids no rule"); + lints +} + fn parse_phase_outcome(diagnostics: &[Diagnostic]) -> RunOutcome { let parse_shaped = diagnostics.iter().any(|diagnostic| { let code = diagnostic.code().as_str(); @@ -1018,6 +1161,7 @@ fn parse_phase_outcome(diagnostics: &[Diagnostic]) -> RunOutcome { thrown: Some(ThrownError { phase: NegativePhase::Parse, error_type: NegativeType::SyntaxError, + message: parse_message(diagnostics), }), }; } @@ -1067,12 +1211,16 @@ fn observed_failure(error: &bamts_runtime::RuntimeError, deadline: Duration) -> RuntimeErrorKind::UncaughtThrow { origin, constructor_name, + message, .. } => match classify_thrown(*origin, constructor_name.as_ref()) { Some(error_type) => RunOutcome::Completed { thrown: Some(ThrownError { phase: NegativePhase::Runtime, error_type, + message: message + .as_ref() + .map_or_else(|| format!("{origin:?}"), EcmaString::to_utf8_lossy), }), }, None => { @@ -1275,6 +1423,24 @@ function $ERROR(message) {\n\ plan: &ExecutionPlan, sources: &[HarnessSource], variant: &ExecutionVariant, + ) -> Result<(), OracleError> { + run_variant_in( + ExecutionMode::Interpreter, + runner, + parsed, + plan, + sources, + variant, + ) + } + + fn run_variant_in( + mode: ExecutionMode, + runner: &InterpreterRunner, + parsed: &ParsedTest, + plan: &ExecutionPlan, + sources: &[HarnessSource], + variant: &ExecutionVariant, ) -> Result<(), OracleError> { // These real-engine fixtures check completion/error classification, // not host scheduling latency. Deadline policy is tested below with @@ -1282,11 +1448,12 @@ function $ERROR(message) {\n\ const SEMANTIC_FIXTURE_DEADLINE: Duration = Duration::from_secs(10); let script = compose_script(parsed, plan, variant, sources)?; let request = RunRequest { - mode: ExecutionMode::Interpreter, + mode, variant: variant.clone(), script, negative: plan.negative.clone(), async_done: plan.async_done, + can_block: plan.can_block, deadline: SEMANTIC_FIXTURE_DEADLINE, }; let outcome = runner.run(&request); @@ -1297,6 +1464,65 @@ function $ERROR(message) {\n\ format!("{FRONTMATTER_OPEN}\n{yaml}\n{FRONTMATTER_CLOSE}\n{body}") } + /// Runs `body` as a module test named `case.js` beside `value_FIXTURE.js`. + fn run_module_beside_fixture(name: &str, body: &str) -> (RunOutcome, usize) { + let root = + std::env::temp_dir().join(format!("bamts-test262-{}-{name}", std::process::id())); + let _ = fs::remove_dir_all(&root); + let tests = root.join("tests"); + let scratch = root.join("scratch"); + fs::create_dir_all(&tests).expect("tests directory"); + fs::write(tests.join("value_FIXTURE.js"), "export const value = 42;\n").expect("fixture"); + fs::write(tests.join("unrelated.js"), "throw 1;\n").expect("sibling"); + + let runner = InterpreterRunner::new(&scratch).for_test_file(tests.join("case.js")); + let request = RunRequest { + mode: ExecutionMode::Interpreter, + variant: ExecutionVariant { + kind: SourceKind::Module, + }, + script: ComposedScript { + bytes: body.as_bytes().to_vec(), + test_offset: 0, + kind: SourceKind::Module, + untouched: true, + }, + negative: None, + async_done: false, + can_block: true, + deadline: Duration::from_secs(10), + }; + let outcome = runner.run(&request); + let left = fs::read_dir(&scratch).map_or(0, |entries| entries.count()); + fs::remove_dir_all(root).expect("remove scratch"); + (outcome, left) + } + + #[test] + fn a_composed_module_finds_the_fixtures_beside_its_test() { + let (outcome, left) = run_module_beside_fixture( + "fixture", + "import { value } from './value_FIXTURE.js';\nif (value !== 42) { throw new Error('wrong'); }\n", + ); + assert!( + matches!(outcome, RunOutcome::Completed { thrown: None }), + "{outcome:?}" + ); + assert_eq!(left, 0, "a run leaves nothing behind"); + } + + #[test] + fn a_composed_module_can_import_itself_by_its_test_name() { + let (outcome, _) = run_module_beside_fixture( + "self", + "import * as self from './case.js';\nexport const mine = 1;\nif (self.mine !== 1) { throw new Error('wrong'); }\n", + ); + assert!( + matches!(outcome, RunOutcome::Completed { thrown: None }), + "{outcome:?}" + ); + } + fn harness_root() -> PathBuf { PathBuf::from("/pinned/harness") } @@ -1341,6 +1567,7 @@ function $ERROR(message) {\n\ thrown: Some(ThrownError { phase: negative.phase, error_type: negative.error_type, + message: String::new(), }), } } else { @@ -1358,6 +1585,7 @@ function $ERROR(message) {\n\ thrown: Some(ThrownError { phase: NegativePhase::Runtime, error_type: NegativeType::TypeError, + message: String::new(), }), } } else { @@ -1511,8 +1739,9 @@ function $ERROR(message) {\n\ let script = compose_script(&parsed, &plan, variant, &harness_sources()) .expect("compose negative"); if phase.is_early() { - assert!(script.untouched); - assert_eq!(script.bytes, parsed.source_bytes); + // No harness, though a module case keeps what makes it a module. + assert_eq!(script.untouched, variant.kind == SourceKind::Raw); + assert!(script.bytes.starts_with(&parsed.source_bytes)); assert_eq!(script.test_offset, 0); } let request = RunRequest { @@ -1521,6 +1750,7 @@ function $ERROR(message) {\n\ script, negative: plan.negative.clone(), async_done: plan.async_done, + can_block: plan.can_block, deadline: DEFAULT_ASYNC_DEADLINE, }; let outcome = PassRunner.run(&request); @@ -1566,7 +1796,7 @@ function $ERROR(message) {\n\ let strict_script = compose_script(&parsed, &plan, strict, &harness_sources()).expect("compose strict"); let before_test = &strict_script.bytes[..strict_script.test_offset]; - assert!(before_test.ends_with(STRICT_PREFIX.as_bytes())); + assert!(before_test.starts_with(STRICT_PREFIX.as_bytes())); assert_eq!( &strict_script.bytes[strict_script.test_offset..], parsed.source_bytes.as_slice() @@ -1764,15 +1994,11 @@ function $ERROR(message) {\n\ let plan = plan_execution(&parsed.frontmatter, &harness).unwrap(); let sources = load_harness_sources(&plan, &FileHarnessLoader).unwrap(); - // Strict frontmatter is preserved: the strict variant still composes - // the directive directly before the test body. + // The strict variant opens with the directive, ahead of the harness. assert_eq!(plan.variants.len(), 2); let strict = compose_script(&parsed, &plan, &plan.variants[1], &sources).unwrap(); assert_eq!(strict.kind, SourceKind::ScriptStrict); - assert_eq!( - &strict.bytes[strict.test_offset - STRICT_PREFIX.len()..strict.test_offset], - STRICT_PREFIX.as_bytes() - ); + assert!(strict.bytes.starts_with(STRICT_PREFIX.as_bytes())); let runner = backend_runner(ExecutionMode::Interpreter, &scratch).unwrap(); for variant in &plan.variants { @@ -1780,6 +2006,1081 @@ function $ERROR(message) {\n\ } } + /// Runs every variant of `source` against the small fixture harness. + fn run_fixture(label: &str, source: &str) -> Result<(), OracleError> { + run_fixture_in(ExecutionMode::Interpreter, label, source) + } + + fn run_fixture_in(mode: ExecutionMode, label: &str, source: &str) -> Result<(), OracleError> { + let harness = runner_scratch(&format!("{label}-harness")); + let scratch = runner_scratch(&format!("{label}-scratch")); + write_harness(&harness, "assert.js", ASSERT_JS); + write_harness(&harness, "sta.js", STA_JS); + write_harness(&harness, "doneprintHandle.js", DONEPRINT_JS); + let parsed = parse_test(source).unwrap(); + let plan = plan_execution(&parsed.frontmatter, &harness).unwrap(); + let sources = load_harness_sources(&plan, &FileHarnessLoader).unwrap(); + let runner = backend_runner(mode, &scratch).unwrap(); + for variant in &plan.variants { + run_variant_in(mode, &runner, &parsed, &plan, &sources, variant)?; + } + Ok(()) + } + + #[test] + fn can_block_is_false_runs_on_an_agent_that_cannot_suspend() { + let body = "var view = new Int32Array(new SharedArrayBuffer(8));\n\ + var threw = false;\n\ + try { Atomics.wait(view, 0, 0, 0); } catch (e) { threw = e instanceof TypeError; }\n\ + assert.sameValue(threw, true);\n"; + let source = wrap( + "description: agent cannot block\nflags: [CanBlockIsFalse]", + body, + ); + for mode in ExecutionMode::ALL { + run_fixture_in(mode, "can-block-false", &source).unwrap(); + } + } + + #[test] + fn a_case_with_no_can_block_flag_runs_on_an_agent_that_can_suspend() { + let body = "var view = new Int32Array(new SharedArrayBuffer(8));\n\ + assert.sameValue(Atomics.wait(view, 0, 0, 0), 'timed-out');\n"; + let source = wrap("description: agent can block", body); + run_fixture("can-block-default", &source).unwrap(); + } + + #[test] + fn an_object_pattern_collects_what_is_left() { + let source = wrap( + "description: object rest in an assignment pattern", + "var a, rest, holder = {};\n\ + ({ a, ...rest } = { a: 1, b: 2, c: 3 });\n\ + assert.sameValue(a, 1);\n\ + assert.sameValue(rest.a, undefined);\n\ + assert.sameValue(rest.b, 2);\n\ + assert.sameValue(rest.c, 3);\n\ + ({ ...holder.rest } = { d: 4 });\n\ + assert.sameValue(holder.rest.d, 4);", + ); + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + run_fixture_in(mode, &format!("object-rest-{mode:?}"), &source).unwrap(); + } + } + + #[test] + fn an_object_pattern_refuses_null_and_undefined() { + const THROWS: &str = "function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n"; + const CASES: [(&str, &str); 3] = [ + ( + "", + "assert.sameValue(throws(TypeError, function () { let {} = undefined; }), true);", + ), + ( + "", + "var a; assert.sameValue(throws(TypeError, function () { ({} = null); }), true);", + ), + ( + "", + "var read = 0; var key = function () { read += 1; return 'k'; }; assert.sameValue(throws(TypeError, function () { var { [key()]: x } = null; }), true); assert.sameValue(read, 0);", + ), + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, (flags, body)) in CASES.iter().enumerate() { + let source = wrap( + &format!("description: object patterns\n{flags}"), + &format!("{THROWS}{body}"), + ); + let label = format!("object-patterns-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} {body}: {error:?}"); + } + } + } + } + + #[test] + fn a_generator_binds_its_parameters_at_the_call() { + const THROWS: &str = "function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n"; + const CASES: [(&str, &str); 9] = [ + ( + "", + "function* g({}) {} assert.sameValue(throws(TypeError, function () { g(undefined); }), true);", + ), + ( + "", + "async function* g({}) {} assert.sameValue(throws(TypeError, function () { g(undefined); }), true);", + ), + ( + "", + "class C { static *m({}) {} } assert.sameValue(throws(TypeError, function () { C.m(undefined); }), true);", + ), + ( + "", + "function* g(a = (function () { throw new Test262Error(); })()) {} assert.sameValue(throws(Test262Error, function () { g(); }), true);", + ), + ( + "", + "var log = 0; function* g(a = log += 1) { log += 10; } var it = g(); assert.sameValue(log, 1); it.next(); assert.sameValue(log, 11);", + ), + ( + "", + "function* g(a = 1) { yield a; } var it = g(); assert.sameValue(it.next(5).value, 1); assert.sameValue(it.next().done, true);", + ), + ( + "", + "function* g(a = 1) { yield a; } var it = g(); assert.sameValue(it.return(7).value, 7); assert.sameValue(it.next().done, true);", + ), + ( + "", + "function* g(a = 1) { yield a; } var it = g(); assert.sameValue(throws(Test262Error, function () { it.throw(new Test262Error()); }), true); assert.sameValue(it.next().done, true);", + ), + ( + "flags: [async]", + "async function* g(a = 1) { yield a; } g().next().then(function (result) { assert.sameValue(result.value, 1); }).then($DONE, $DONE);", + ), + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, (flags, body)) in CASES.iter().enumerate() { + let source = wrap( + &format!("description: generator parameters\n{flags}"), + &format!("{THROWS}{body}"), + ); + let label = format!("generator-parameters-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} {body}: {error:?}"); + } + } + } + } + + #[test] + fn the_host_object_does_real_work() { + const CASES: [&str; 5] = [ + "var b = new ArrayBuffer(8); assert.sameValue($262.detachArrayBuffer(b), null); \ + assert.sameValue(b.detached, true); assert.sameValue(b.byteLength, 0);", + "var view = new Uint8Array(4); $262.detachArrayBuffer(view.buffer); \ + assert.sameValue(view.length, 0);", + "assert.sameValue($262.global, globalThis);", + "assert.sameValue($262.evalScript('1 + 2'), 3);", + "var d = Object.getOwnPropertyDescriptor(globalThis, '$262'); \ + assert.sameValue(d.enumerable, false); assert.sameValue(d.writable, true); \ + assert.sameValue(d.configurable, true);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: the host object", body); + let label = format!("host-object-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} {body}: {error:?}"); + } + } + } + } + + #[test] + fn wrapper_prototypes_report_their_own_primitive() { + const THROWS: &str = "function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n"; + const CASES: [&str; 7] = [ + "assert.sameValue('a'.toString(), 'a');", + "assert.sameValue('a'.valueOf(), 'a');", + "assert.sameValue(new String('a').toString(), 'a');", + "assert.sameValue(String.prototype.toString.call(new String('')), '');", + "assert.sameValue(throws(TypeError, function () { String.prototype.toString.call(1); }), true);", + "assert.sameValue(true.toString(), 'true'); assert.sameValue(new Boolean(false).toString(), 'false');", + "assert.sameValue(throws(TypeError, function () { Boolean.prototype.toString.call('a'); }), true);", + ]; + for (index, body) in CASES.iter().enumerate() { + let source = wrap( + "description: wrapper prototypes", + &format!("{THROWS}{body}"), + ); + if let Err(error) = run_fixture(&format!("wrapper-prototypes-{index}"), &source) { + panic!("{body}: {error:?}"); + } + } + } + + #[test] + fn super_reads_above_the_home_object() { + const CASES: [&str; 17] = [ + "class A { m() { return this.v; } } \ + class B extends A { v = 1; f() { return (() => super.m())(); } } \ + assert.sameValue(new B().f(), 1);", + "class A { static s() { return 'A'; } static t = 1; } \ + class B extends A { static s() { return 'B'; } static x = super.s(); \ + static y = super.t; static z; static { this.z = super.s(); } } \ + assert.sameValue(B.x, 'A'); assert.sameValue(B.y, 1); assert.sameValue(B.z, 'A');", + "class A {} class B extends A { m() { [super.x] = [1]; return this.x; } } \ + assert.sameValue(new B().m(), 1);", + "class A {} class B extends A { m() { for (super.x of [2]); return this.x; } } \ + assert.sameValue(new B().m(), 2);", + "class A {} class B extends A { m() { delete super.x; } } \ + var caught; try { new B().m(); } catch (error) { caught = error; } \ + assert.sameValue(caught instanceof ReferenceError, true);", + "class A { constructor() { this.a = 1; } } \ + class B extends A { f = 2; constructor() { (() => super())(); this.c = 3; } } \ + var b = new B(); assert.sameValue(b.a, 1); assert.sameValue(b.f, 2); \ + assert.sameValue(b.c, 3);", + "class A { m() { return 1; } } \ + class B extends A { *g() { yield super.m(); } async a() { return super.m(); } } \ + assert.sameValue(new B().g().next().value, 1);", + "var o = { m() { return super.x; } }; Object.setPrototypeOf(o, null); \ + var caught; try { o.m(); } catch (error) { caught = error; } \ + assert.sameValue(caught instanceof TypeError, true);", + "class A { m() { return 1; } } \ + class B extends A { m() { return super.m() + 1; } } \ + assert.sameValue(new B().m(), 2);", + "class A { get g() { return this.v; } } \ + class B extends A { constructor() { super(); this.v = 5; } h() { return super.g; } } \ + assert.sameValue(new B().h(), 5);", + "class A { m() { return 'a'; } } \ + class B extends A { constructor() { super(); this.r = super.m(); } } \ + assert.sameValue(new B().r, 'a');", + "class A { static s() { return 1; } } \ + class B extends A { static s() { return super.s() + 1; } } \ + assert.sameValue(B.s(), 2);", + "class A { m() { return 1; } } \ + class B extends A { m() { var f = () => super.m(); return f(); } } \ + assert.sameValue(new B().m(), 1);", + "var p = { m() { return 'p'; } }; \ + var o = { __proto__: p, m() { return super.m(); } }; \ + assert.sameValue(o.m(), 'p');", + "class A {} class B extends A { w() { super.x = 1; return this.x; } } \ + assert.sameValue(new B().w(), 1); \ + assert.sameValue(A.prototype.x, undefined);", + "class A { get x() { return 1; } set x(v) { this.y = v; } } \ + class B extends A { u() { super.x += 1; return this.y; } } \ + assert.sameValue(new B().u(), 2);", + "class A { m() { return 1; } } class B extends A { f = super.m(); } \ + assert.sameValue(new B().f, 1);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: super references", body); + let label = format!("super-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn a_read_write_reference_converts_its_key_once() { + const KEY: &str = "var n = 0;\n\ + var k = { toString() { n++; return 'a'; } };\n\ + function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n"; + const CASES: [&str; 6] = [ + "var o = { a: 1 }; o[k] += 1; assert.sameValue(n, 1); assert.sameValue(o.a, 2);", + "var o = { a: 1 }; o[k]++; assert.sameValue(n, 1); assert.sameValue(o.a, 2);", + "var o = { a: 0 }; o[k] ||= 5; assert.sameValue(n, 1); assert.sameValue(o.a, 5);", + "var o = null; assert.sameValue(throws(TypeError, function () { o[k] += 1; }), true); \ + assert.sameValue(n, 0);", + "var a = [1, 2]; a[1] += 1; a[0]--; assert.sameValue(a[1], 3); \ + assert.sameValue(a[0], 0);", + "var s = Symbol(); var o = { [s]: 1 }; o[s]++; assert.sameValue(o[s], 2);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: one key conversion", &format!("{KEY}{body}")); + let label = format!("key-once-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn super_called_in_an_arrow_finishes_the_constructor() { + const SETUP: &str = "var fields = 0;\n\ + class A { m() { return 7; } }\n\ + function refused(body) {\n\ + try { body(); } catch (error) { return error instanceof ReferenceError; }\n\ + return false;\n\ + }\n"; + const CASES: [&str; 8] = [ + "class B extends A { f = () => { return super.m(); }; g = () => { return; }; \ + h = () => { return 3; }; }\n\ + var b = new B; assert.sameValue(b.f(), 7); assert.sameValue(b.g(), undefined); \ + assert.sameValue(b.h(), 3);", + "class B extends A { constructor() { const init = () => { super(); return 5; }; \ + assert.sameValue(init(), 5); } }\n\ + new B;", + "function m() { const y = 1; class B extends A { v = y; #p = y + 1; \ + get p() { return this.#p; } constructor() { (() => super())(); } } return new B; }\n\ + var b = m(); assert.sameValue(b.v, 1); assert.sameValue(b.p, 2);", + "class B extends A { v = fields++; constructor() { super(); \ + assert.sameValue(refused(() => super()), true); } }\n\ + new B; assert.sameValue(fields, 1);", + "class B extends A { v = fields++; constructor() { const init = () => super(); \ + init(); assert.sameValue(refused(init), true); } }\n\ + new B; assert.sameValue(fields, 1);", + "var later; class B extends A { v = fields++; \ + constructor() { super(); later = () => super(); } }\n\ + new B; assert.sameValue(refused(later), true); assert.sameValue(fields, 1);", + "class B extends A { constructor(a = (() => super())()) { \ + assert.sameValue(this instanceof A, true); } }\n\ + new B;", + "class B extends A { v = fields++; constructor() { (() => super())(); \ + var caught; try { super(); } catch (error) { caught = error; } \ + assert.sameValue(caught instanceof ReferenceError, true); } }\n\ + new B; assert.sameValue(fields, 1);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: arrow super", &format!("{SETUP}{body}")); + let label = format!("arrow-super-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn a_field_initializer_sees_the_class_scope() { + const SETUP: &str = "function run() {\n\ + const y = 10;\n\ + class A {}\n"; + const CASES: [&str; 6] = [ + "class C { v = y; constructor(y) { assert.sameValue(this.v, 10); \ + assert.sameValue(y, 3); } } new C(3);", + "class C { v = y; constructor() { var y = 2; assert.sameValue(this.v, 10); \ + assert.sameValue(y, 2); } } new C;", + "class B extends A { v = y; constructor() { const y = 2; super(); \ + assert.sameValue(this.v, 10); assert.sameValue(y, 2); } } new B;", + "class B extends A { v = y; constructor() { const y = 2; ((y) => super())(3); \ + assert.sameValue(this.v, 10); assert.sameValue(y, 2); } } new B;", + "class B extends A { v = () => y; constructor(y) { super(); \ + assert.sameValue(this.v(), 10); assert.sameValue(y, 3); } } new B(3);", + "class C { v = y; w = class { m() { return y; } }; constructor() { \ + assert.sameValue(new this.w().m(), 10); } } new C;", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap( + "description: field scope", + &format!("{SETUP}{body}\n}}\nrun();"), + ); + let label = format!("field-scope-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn a_collection_during_a_callback_keeps_what_native_code_holds() { + const SETUP: &str = "var sink; var warm = false;\n\ + function gc() { var n = warm ? 12 : 44; warm = true; \ + for (var i = 0; i < n; i++) { sink = 'x'.repeat(1 << 20) + i; } }\n\ + gc();\n\ + function* items() { yield { x: 1 }; gc(); yield { x: 2 }; }\n"; + const CASES: [&str; 12] = [ + "var many = []; for (var i = 0; i < 64; i++) { many.push({ toJSON() { \ + if (i === 63) { gc(); } return { v: 'a' + 'b' }; } }); } \ + assert.sameValue(JSON.stringify(many).length > 64, true);", + "var strings = []; for (var i = 0; i < 64; i++) { strings.push(i); } \ + var doubled = Float64Array.from(strings, (v) => { if (v === 63) { gc(); } \ + return String(v) + '0'; }); assert.sameValue(doubled[1], 10);", + "var text = ''; for (var i = 0; i < 2000; i++) { text += 'a line of text\\n'; } \ + var lines = text.split('\\n'); \ + var joined = lines.reduce((sum, line) => sum + line + '\\n', ''); \ + assert.sameValue(joined.length > 2000, true);", + "var left = { valueOf() { return 'a' + 'b'; } }; \ + var right = { valueOf() { gc(); return 'y'; } }; \ + assert.sameValue(left + right, 'aby'); assert.sameValue(left < right, true);", + "var list = new Proxy([{ a: 3 }, { a: 1 }], { get(t, k) { gc(); \ + var v = t[k]; return typeof v === 'object' ? { a: v.a } : v; } }); \ + var ordered = Array.prototype.toSorted.call(list, (p, q) => { gc(); \ + return p.a - q.a; }); assert.sameValue(ordered[0].a, 1);", + "var seen = 0; Promise.resolve(1).then(() => { gc(); return { a: 1 }; })\ + .then((v) => { seen = v.a; });", + "var sorted = [{ a: 3 }, { a: 1 }].toSorted((p, q) => { gc(); return p.a - q.a; }); \ + assert.sameValue(sorted[0].a, 1);", + "var groups = Object.groupBy(items(), (v) => { gc(); return 'k' + v.x; }); \ + assert.sameValue(groups.k2[0].x, 2);", + "var spread = [...items()]; assert.sameValue(spread[1].x, 2);", + "function add(a, b) { return a.x + b.x; } \ + assert.sameValue(add(...items()), 3);", + "var parsed = JSON.parse('{\"a\":{\"b\":1},\"c\":2}', function (k, v) { \ + gc(); return v; }); assert.sameValue(parsed.a.b, 1);", + "var proxy = new Proxy({}, { get(t, k) { gc(); return { key: k }; } }); \ + assert.sameValue(proxy.a.key, 'a');", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: native roots", &format!("{SETUP}{body}")); + let label = format!("native-roots-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn a_class_is_constructed_only_with_new_and_defines_its_static_fields() { + const SETUP: &str = "function refused(body) {\n\ + try { body(); } catch (error) { return error instanceof TypeError; }\n\ + return false;\n\ + }\n\ + class A {}\n\ + class B extends A {}\n\ + class C extends A { constructor() { super(); } }\n"; + const CASES: [&str; 5] = [ + "assert.sameValue(refused(() => A()), true);", + "assert.sameValue(refused(() => B()), true);", + "assert.sameValue(refused(() => C.call({})), true);", + "class D { static name = 'x'; static length = 2; #p = 1; static #q = 3; \ + static q() { return D.#q; } get p() { return this.#p; } }\n\ + assert.sameValue(D.name, 'x'); assert.sameValue(D.length, 2); \ + assert.sameValue(D.q(), 3); assert.sameValue(new D().p, 1);", + "assert.sameValue(Math.abs(new Number(-1)), 1); \ + assert.sameValue(Math.pow({ valueOf() { return 2; } }, 3), 8);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: class calls", &format!("{SETUP}{body}")); + let label = format!("class-call-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn a_destructuring_target_is_evaluated_before_its_value() { + const LOG: &str = "var log = [];\n\ + function source() { log.push('source'); return { get p() { log.push('get'); } }; }\n\ + function target() { log.push('target'); return { set q(v) { log.push('set'); } }; }\n\ + function key() { log.push('key'); return 'q'; }\n\ + function sourceKey() { log.push('source-key'); \ + return { toString() { log.push('source-string'); return 'p'; } }; }\n\ + var empty = { [Symbol.iterator]() { return { next() { log.push('next'); \ + return { done: true }; } }; } };\n"; + const CASES: [(&str, &str); 7] = [ + ( + "({ [sourceKey()]: target()[key()] } = source());", + "source,source-key,source-string,target,key,get,set", + ), + ( + "var hits = 0, rest; var k = { toString() { hits++; return 'p'; } }; \ + ({ [k]: target().q, ...rest } = { p: 1, r: 2 }); log.push(hits, rest.r, rest.p);", + "target,set,1,2,", + ), + ( + "({ p: target()[key()] } = source());", + "source,target,key,get,set", + ), + ( + "({ p: target()[key()] = log.push('default') } = source());", + "source,target,key,get,default,set", + ), + ( + "({ ...target()[key()] } = source());", + "source,target,key,get,set", + ), + ("[target()[key()]] = empty;", "target,key,next,set"), + ("[...target()[key()]] = empty;", "target,key,next,set"), + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, (body, expected)) in CASES.iter().enumerate() { + let source = wrap( + "description: target before value", + &format!("{LOG}{body}\nassert.sameValue(log.join(), '{expected}');"), + ); + let label = format!("target-order-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn a_computed_literal_key_converts_before_its_value() { + let source = wrap( + "description: key order", + "var log = []; var key = { toString() { log.push('key'); return 'k'; } }; \ + function value() { log.push('value'); return 1; } \ + var o = { [key]: value() }; \ + assert.sameValue(log.join(), 'key,value'); assert.sameValue(o.k, 1);", + ); + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + let label = format!("key-order-{mode:?}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + + #[test] + fn closing_an_iterator_checks_its_return_method() { + const THROWS: &str = "function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n\ + function iterable(close) {\n\ + var it = { next() { return { done: false }; }, return: close };\n\ + return { [Symbol.iterator]() { return it; } };\n\ + }\n"; + const CASES: [&str; 3] = [ + "assert.sameValue(throws(TypeError, function () { \ + for (var x of iterable(1)) { break; } }), true);", + "for (var x of iterable(null)) { break; }", + "for (var x of iterable(undefined)) { break; }", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: iterator close", &format!("{THROWS}{body}")); + let label = format!("iterator-close-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn construction_behaves_the_same_in_every_tier() { + const THROWS: &str = "function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n"; + const CASES: [&str; 7] = [ + "assert.sameValue(throws(TypeError, function () { new (() => {}); }), true);", + "function* g() {} assert.sameValue(throws(TypeError, function () { new g(); }), true);", + "var o = { m() {} }; assert.sameValue(throws(TypeError, function () { new o.m(); }), true);", + "function F() { this.args = arguments; } \ + var r = Reflect.construct(F, [1, 2, 3]); \ + assert.sameValue(r instanceof F, true); assert.sameValue(r.args.length, 3);", + "function F() { return { x: 1 }; } \ + assert.sameValue(Reflect.construct(F, []).x, 1);", + "function F() { this.x = 1; } var B = F.bind(null); \ + assert.sameValue(new B().x, 1); \ + assert.sameValue(Reflect.construct(B, []).x, 1);", + "function F() { throw new RangeError('no'); } \ + assert.sameValue(throws(RangeError, function () { \ + Reflect.construct(F, []); }), true);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: construction", &format!("{THROWS}{body}")); + let label = format!("construction-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn an_object_literal_defines_its_own_properties() { + const CASES: [&str; 5] = [ + "Object.defineProperty(Object.prototype, 'x', { set() { throw new Error('set'); }, \ + configurable: true }); \ + var o = { x: 1 }; delete Object.prototype.x; \ + var d = Object.getOwnPropertyDescriptor(o, 'x'); \ + assert.sameValue(d.value, 1); assert.sameValue(d.enumerable, true); \ + assert.sameValue(d.writable, true); assert.sameValue(d.configurable, true);", + "Object.defineProperty(Object.prototype, '0', { value: 9, writable: false, \ + configurable: true }); var o = { 0: true }; delete Object.prototype[0]; \ + assert.sameValue(o[0], true);", + "var o = { get a() { return 1; }, a: 2 }; assert.sameValue(o.a, 2); \ + assert.sameValue(Object.getOwnPropertyDescriptor(o, 'a').writable, true);", + "var p = {}; var o = { __proto__: p }; assert.sameValue(Object.getPrototypeOf(o), p); \ + assert.sameValue(Object.keys(o).length, 0);", + "var p = {}; var o = { ['__proto__']: p }; \ + assert.sameValue(Object.getPrototypeOf(o), Object.prototype); \ + assert.sameValue(Object.keys(o)[0], '__proto__');", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: object literal definition", body); + let label = format!("object-literal-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn local_time_works_in_the_host() { + let source = wrap( + "description: local time", + "assert.sameValue(new Date(0).getTimezoneOffset(), 0);\n\ + assert.sameValue(new Date(0).getHours(), 0);", + ); + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + let label = format!("local-time-{mode:?}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + + #[test] + fn an_array_elision_leaves_a_hole() { + const CASES: [&str; 4] = [ + "var a = [0, , 2]; assert.sameValue(a.length, 3); \ + assert.sameValue(1 in a, false); assert.sameValue(a[2], 2);", + "var a = [,]; assert.sameValue(a.length, 1); assert.sameValue(0 in a, false);", + "var a = [1, , ]; assert.sameValue(a.length, 2); assert.sameValue(1 in a, false);", + "var a = [...[1, 2], , 4]; assert.sameValue(a.length, 4); \ + assert.sameValue(2 in a, false); assert.sameValue(a[3], 4);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: array elisions", body); + let label = format!("array-elision-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{label}: {error:?}"); + } + } + } + } + + #[test] + fn a_template_substitution_is_converted_with_to_string() { + const THROWS: &str = "function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n"; + const CASES: [&str; 4] = [ + "var o = { valueOf() { return 'V'; }, toString() { return 'T'; } }; \ + assert.sameValue(`${o}`, 'T');", + "assert.sameValue(`${1}${'a'}${null}${undefined}`, '1anullundefined');", + "assert.sameValue(throws(TypeError, function () { `${Symbol()}`; }), true);", + "var log = ''; var part = function (name) { return { toString() { log += name; \ + return name; } }; }; assert.sameValue(`${part('a')}-${part('b')}`, 'a-b'); \ + assert.sameValue(log, 'ab');", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: template literals", &format!("{THROWS}{body}")); + let label = format!("template-literal-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} {body}: {error:?}"); + } + } + } + } + + #[test] + fn an_anonymous_function_takes_the_name_it_initializes() { + const CASES: [&str; 20] = [ + "var f = function () {}; assert.sameValue(f.name, 'f');", + "var a = () => {}; assert.sameValue(a.name, 'a');", + "var c = class {}; assert.sameValue(c.name, 'c');", + "var g = function* () {}; assert.sameValue(g.name, 'g');", + "var { x = function () {} } = {}; assert.sameValue(x.name, 'x');", + "var [y = () => {}] = []; assert.sameValue(y.name, 'y');", + "function p(z = class {}) { return z; } assert.sameValue(p().name, 'z');", + "var h; h = function () {}; assert.sameValue(h.name, 'h');", + "var i; i ??= function () {}; assert.sameValue(i.name, 'i');", + "var j; j ||= () => {}; assert.sameValue(j.name, 'j');", + "var u; [u = function () {}] = []; assert.sameValue(u.name, 'u');", + "var v; ({ v = function () {} } = {}); assert.sameValue(v.name, 'v');", + "var o = { m: function () {}, 'n': () => {} }; assert.sameValue(o.m.name, 'm'); \ + assert.sameValue(o.n.name, 'n');", + "var paren = (function () {}); assert.sameValue(paren.name, 'paren');", + "var kept = function other() {}; assert.sameValue(kept.name, 'other');", + "var cover = (0, function () {}); assert.sameValue(cover.name, '');", + "var q; (q) = function () {}; assert.sameValue(q.name, '');", + "var s = class { static name() {} }; assert.sameValue(typeof s.name, 'function');", + "var inner = class { static c = class {}; }; assert.sameValue(inner.c.name, 'c');", + "var calls = 0; var [w = (calls += 1, undefined)] = []; assert.sameValue(calls, 1);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: function names", body); + let label = format!("function-names-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} {body}: {error:?}"); + } + } + } + } + + #[test] + fn negative_zero_survives_in_every_tier() { + const CASES: [&str; 12] = [ + "assert.sameValue(1 / Math.round(-0), -Infinity);", + "assert.sameValue(Math.round(0.49999999999999994), 0);", + "assert.sameValue(Math.round(-0.5) === 0, true); assert.sameValue(Math.round(2.5), 3); \ + assert.sameValue(Math.round(-2.5), -2); assert.sameValue(Math.round(4503599627370495.5), 4503599627370496);", + "assert.sameValue(1 / -0, -Infinity);", + "assert.sameValue(Object.is(-0, 0), false);", + "var z = 0; assert.sameValue(1 / -z, -Infinity);", + "var z = 0; assert.sameValue(1 / (z * -1), -Infinity);", + "var a = -1; var b = 0; assert.sameValue(1 / (a * b), -Infinity);", + "assert.sameValue(1 / Math.round(-0.4), -Infinity);", + "assert.sameValue(Object.is(Number('-0'), -0), true);", + "assert.sameValue(String(-0), '0'); assert.sameValue(-0 === 0, true);", + "var o = {}; o[-0] = 1; assert.sameValue(o['0'], 1);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: negative zero", body); + let label = format!("negative-zero-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} {body}: {error:?}"); + } + } + } + } + + #[test] + fn indirect_eval_runs_in_the_global_scope() { + const THROWS: &str = "function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n"; + const CASES: [&str; 12] = [ + "assert.sameValue((0, eval)('1 + 1'), 2);", + "assert.sameValue((0, eval)(1), 1);", + "assert.sameValue((0, eval)(), undefined);", + "var x = 1; assert.sameValue((0, eval)('x'), 1);", + "(0, eval)('var y = 2'); assert.sameValue(globalThis.y, 2);", + "(0, eval)('function g() { return 3; }'); assert.sameValue(globalThis.g(), 3);", + "assert.sameValue((0, eval)('this'), globalThis);", + "assert.sameValue((0, eval)('if (true) { 4; } else { 5; }'), 4);", + "assert.sameValue(throws(SyntaxError, function () { (0, eval)('('); }), true);", + "assert.sameValue(throws(TypeError, function () { new eval('1'); }), true);", + "assert.sameValue(throws(SyntaxError, function () { (0, eval)('\"use strict\"; with (a) {}'); }), true);", + "assert.sameValue(throws(SyntaxError, function () { Function('\"use strict\"; var public;'); }), true);", + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, body) in CASES.iter().enumerate() { + let source = wrap("description: indirect eval", &format!("{THROWS}{body}")); + let label = format!("indirect-eval-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} {body}: {error:?}"); + } + } + } + } + + #[test] + fn a_script_declares_on_the_global_object() { + const CASES: [(&str, &str); 11] = [ + ( + "", + "function f() { var x = 1; var x; return x; } assert.sameValue(f(), 1);", + ), + ("", "var x = 1; assert.sameValue(globalThis.x, 1);"), + ("", "function f() {} assert.sameValue(globalThis.f, f);"), + ("", "assert.sameValue(hoisted, undefined); var hoisted = 1;"), + ( + "", + "assert.sameValue(early(), 1); function early() { return 1; }", + ), + ("", "assert.sameValue(this, globalThis);"), + ("", "var undefined; assert.sameValue(undefined, void 0);"), + ("", "var Array; assert.sameValue(typeof Array, 'function');"), + ( + "", + "let y = 1; assert.sameValue(y, 1); assert.sameValue('y' in globalThis, false);", + ), + ( + "", + "var v = 1; (function () { v = 2; })(); assert.sameValue(globalThis.v, 2);", + ), + ( + "flags: [module]", + "var x = 1; assert.sameValue('x' in globalThis, false); assert.sameValue(this, undefined);", + ), + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, (flags, body)) in CASES.iter().enumerate() { + let source = wrap(&format!("description: script goal\n{flags}"), body); + let label = format!("script-goal-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} {body}: {error:?}"); + } + } + } + } + + #[test] + fn module_syntax_is_an_early_error_in_a_script() { + for (index, body) in ["export var x;", "import './composed.js';", "import.meta;"] + .iter() + .enumerate() + { + let source = wrap( + "description: module syntax in a script\n\ + negative:\n phase: parse\n type: SyntaxError", + body, + ); + run_fixture(&format!("script-goal-early-{index}"), &source) + .unwrap_or_else(|error| panic!("{body}: {error:?}")); + } + } + + #[test] + fn a_lint_cannot_refuse_a_valid_program() { + let source = wrap( + "description: a with statement is valid in sloppy code\nflags: [noStrict]", + "with ({ a: 1 }) { assert.sameValue(a, 1); }", + ); + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + run_fixture_in(mode, &format!("with-{mode:?}"), &source).unwrap(); + } + } + + #[test] + fn strict_and_sloppy_code_differ_the_same_way_in_every_tier() { + const THROWS: &str = "function throws(kind, body) {\n\ + try { body(); } catch (error) { return error instanceof kind; }\n\ + return false;\n\ + }\n"; + const CASES: [(&str, &str); 18] = [ + ( + "noStrict", + "assert.sameValue((function () { return this; })(), globalThis);", + ), + ( + "onlyStrict", + "assert.sameValue((function () { return this; })(), undefined);", + ), + ( + "noStrict", + "assert.sameValue((function () { return this; }).call(null), globalThis);", + ), + ( + "noStrict", + "assert.sameValue(typeof (function () { return this; }).call(1), 'object');", + ), + ( + "noStrict", + "var f = function () { return this === this; }; assert.sameValue(f.call(1), true);", + ), + ( + "onlyStrict", + "assert.sameValue((function () { return this; }).call(1), 1);", + ), + ( + "noStrict", + "assert.sameValue((function () { return (() => this)(); })(), globalThis);", + ), + ( + "noStrict", + "function *g() { yield this; } assert.sameValue(g().next().value, globalThis);", + ), + ( + "noStrict", + "function f() { 'use strict'; return this; } assert.sameValue(f(), undefined);", + ), + ( + "noStrict", + "class C { m() { return this; } } var m = new C().m; assert.sameValue(m(), undefined);", + ), + ( + "noStrict", + "var o = Object.freeze({ a: 1 }); o.a = 2; assert.sameValue(o.a, 1);", + ), + ( + "onlyStrict", + "var o = Object.freeze({ a: 1 }); assert.sameValue(throws(TypeError, function () { o.a = 2; }), true);", + ), + ( + "noStrict", + "'text'.x = 1; (1).x = 1; assert.sameValue((1).x, undefined);", + ), + ( + "onlyStrict", + "assert.sameValue(throws(TypeError, function () { 'text'.x = 1; }), true);", + ), + ( + "noStrict", + "undeclared = 1; assert.sameValue(undeclared, 1);", + ), + ( + "onlyStrict", + "assert.sameValue(throws(ReferenceError, function () { undeclared = 1; }), true);", + ), + ( + "noStrict", + "assert.sameValue(delete Object.prototype, false);", + ), + ( + "onlyStrict", + "assert.sameValue(throws(TypeError, function () { delete Object.prototype; }), true);", + ), + ]; + for mode in [ + ExecutionMode::Interpreter, + ExecutionMode::Jit, + ExecutionMode::Aot, + ] { + for (index, (flag, body)) in CASES.iter().enumerate() { + let source = wrap( + &format!("description: strictness\nflags: [{flag}]"), + &format!("{THROWS}{body}"), + ); + let label = format!("strictness-{mode:?}-{index}"); + if let Err(error) = run_fixture_in(mode, &label, &source) { + panic!("{mode:?} [{flag}] {body}: {error:?}"); + } + } + } + } + + #[test] + fn a_composed_test_is_compiled_as_strict_code() { + let source = wrap( + "description: reserved in strict code only, so this must not compile\n\ + flags: [onlyStrict]", + "var public;", + ); + assert!( + run_fixture("strict-compiled", &source).is_err(), + "the directive reaches the test past the harness" + ); + } + + #[test] + fn an_early_negative_is_parsed_under_its_own_flags() { + let strict = wrap( + "description: reserved in strict code only\n\ + flags: [onlyStrict]\n\ + negative:\n phase: parse\n type: SyntaxError", + "var public;", + ); + run_fixture("strict-parse", &strict).expect("rejected as strict code"); + let module = wrap( + "description: reserved in module code only\n\ + flags: [module]\n\ + negative:\n phase: parse\n type: SyntaxError", + "var await;", + ); + run_fixture("module-parse", &module).expect("rejected as module code"); + } + + #[test] + fn a_module_case_without_imports_has_the_module_goal() { + let source = wrap( + "description: reserved in module code only, so this must not compile\n\ + flags: [module]", + "var await;", + ); + assert!( + run_fixture("module-goal", &source).is_err(), + "a test with no import or export is still a module" + ); + } + #[test] fn interpreter_runner_judges_runtime_reference_error_fixture() { let harness = runner_scratch("neg-runtime-harness"); @@ -1824,7 +3125,15 @@ function $ERROR(message) {\n\ let runner = backend_runner(ExecutionMode::Interpreter, &scratch).unwrap(); for variant in &plan.variants { let script = compose_script(&parsed, &plan, variant, &sources).unwrap(); - assert!(script.untouched, "early negatives compose untouched"); + assert_eq!( + script.untouched, + variant.kind == SourceKind::ScriptNonStrict, + "an early negative takes no harness, only its strict directive" + ); + assert_eq!( + &script.bytes[script.test_offset..], + parsed.source_bytes.as_slice() + ); run_variant(&runner, &parsed, &plan, &sources, variant).unwrap(); } } @@ -1866,6 +3175,7 @@ function $ERROR(message) {\n\ }, negative, async_done: true, + can_block: true, deadline: Duration::from_millis(100), } } @@ -2051,7 +3361,9 @@ function $ERROR(message) {\n\ async_done_outcome(DoneEventKind::Error, Some("CustomError: not a known type")); match judge_run(&request, &outcome) { - Err(OracleError::ExpectationMismatch { expected, actual }) => { + Err(OracleError::ExpectationMismatch { + expected, actual, .. + }) => { assert_eq!(expected, "Runtime/Test262Error"); assert_eq!(actual, "Runtime/CustomError: not a known type"); } @@ -2171,7 +3483,7 @@ function $ERROR(message) {\n\ // this is an ExpectationMismatch, not a BlockedRun. assert!( matches!(&result, - Err(OracleError::ExpectationMismatch { expected, actual }) + Err(OracleError::ExpectationMismatch { expected, actual, .. }) if expected == "success" && actual == "Runtime/Test262Error" ), "a guest throw with a known constructor must be classified, not blocked: {result:?}" @@ -2179,6 +3491,45 @@ function $ERROR(message) {\n\ } } + #[test] + fn a_failed_run_reads_as_what_it_expected_and_what_the_throw_said() { + let mismatch = |message: &str| OracleError::ExpectationMismatch { + expected: "success".to_owned(), + actual: "Runtime/Test262Error".to_owned(), + message: message.to_owned(), + }; + assert_eq!( + failure_detail(&mismatch("Expected SameValue(«1», «2»)")), + "expected success, got Runtime/Test262Error: Expected SameValue(«1», «2»)" + ); + assert_eq!( + failure_detail(&mismatch("")), + "expected success, got Runtime/Test262Error" + ); + let blocked = OracleError::BlockedRun { + detail: "no verdict".to_owned(), + }; + assert!(failure_detail(&blocked).contains("no verdict")); + } + + #[test] + fn a_thrown_message_never_changes_a_verdict() { + let request = negative_async_request(Some(Negative { + phase: NegativePhase::Runtime, + error_type: NegativeType::TypeError, + })); + for message in ["", "anything at all"] { + let outcome = RunOutcome::Completed { + thrown: Some(ThrownError { + phase: NegativePhase::Runtime, + error_type: NegativeType::TypeError, + message: message.to_owned(), + }), + }; + assert!(judge_run(&request, &outcome).is_ok()); + } + } + #[test] fn fuel_exhaustion_blocks_the_run() { let harness = runner_scratch("fuel-harness"); @@ -2201,6 +3552,7 @@ function $ERROR(message) {\n\ script, negative: plan.negative.clone(), async_done: plan.async_done, + can_block: plan.can_block, deadline: Duration::from_millis(1), }; let outcome = runner.run(&request); @@ -2233,6 +3585,7 @@ function $ERROR(message) {\n\ }, negative: None, async_done: false, + can_block: true, deadline: Duration::from_millis(100), }); } diff --git a/crates/bamts-verification/src/oracles/test262_harness.rs b/crates/bamts-verification/src/oracles/test262_harness.rs index 6cf6ade4..381924ab 100644 --- a/crates/bamts-verification/src/oracles/test262_harness.rs +++ b/crates/bamts-verification/src/oracles/test262_harness.rs @@ -26,8 +26,6 @@ use super::test262::{ pub enum Capability { /// `$262.createRealm`: the machine exposes no realm factory. CreateRealm, - /// `$262.detachArrayBuffer`: `ArrayBuffer` is not implemented. - DetachArrayBuffer, /// `$262.agent.start`: the host exposes no agent scheduler. AgentStart, /// `$262.agent.broadcast`: requires a started agent. @@ -44,7 +42,6 @@ impl Capability { pub const fn as_str(self) -> &'static str { match self { Self::CreateRealm => "$262.createRealm", - Self::DetachArrayBuffer => "$262.detachArrayBuffer", Self::AgentStart => "$262.agent.start", Self::AgentBroadcast => "$262.agent.broadcast", Self::AgentGetReport => "$262.agent.getReport", @@ -284,6 +281,7 @@ pub fn judge_stage_outcome( let thrown = failure.map(|(stage, error_type)| ThrownError { phase: stage.phase(), error_type, + message: String::new(), }); judge_run(request, &RunOutcome::Completed { thrown }).map_err(HarnessError::Oracle) } @@ -508,6 +506,7 @@ mod tests { }, negative, async_done: false, + can_block: true, deadline: DEADLINE, } } @@ -843,10 +842,6 @@ mod tests { #[test] fn unsupported_capabilities_name_their_262_member() { assert_eq!(Capability::CreateRealm.as_str(), "$262.createRealm"); - assert_eq!( - Capability::DetachArrayBuffer.as_str(), - "$262.detachArrayBuffer" - ); assert_eq!(Capability::AgentSleep.as_str(), "$262.agent.sleep"); } diff --git a/crates/bamts-verification/src/perf.rs b/crates/bamts-verification/src/perf.rs index 7de7b361..13827d8d 100644 --- a/crates/bamts-verification/src/perf.rs +++ b/crates/bamts-verification/src/perf.rs @@ -283,7 +283,7 @@ impl ObservedConditions { } } -/// A live machine fingerprint read from `/proc` and `/sys`. +/// A live machine fingerprint read from the host. #[derive(Debug, Clone, PartialEq, Eq)] pub struct MachineFingerprint { /// `uname -r` kernel release. @@ -306,189 +306,21 @@ pub struct MachineFingerprint { pub swap_total_kib: u64, } -/// Parses the kernel release and build-version tail from `/proc/version`. -fn parse_proc_version(content: &str) -> Result<(String, String)> { - let mut tokens = content.split_whitespace(); - // Expected: "Linux version ..." - let release = tokens - .nth(2) - .ok_or_else(|| PerfError::harness("/proc/version missing kernel release"))? - .to_owned(); - let version = content - .find('#') - .map(|start| content[start..].trim_end().to_owned()) - .unwrap_or_default(); - Ok((release, version)) -} - -/// Parses `(cpu_model, sockets, cores_per_socket, microcode)` from cpuinfo. -fn parse_proc_cpuinfo(content: &str) -> Result<(String, u32, u32, String)> { - let mut model = None; - let mut microcode = None; - let mut cores_per_socket = None; - let mut sockets = BTreeSet::new(); - - for line in content.lines() { - let Some((key, value)) = line.split_once(':') else { - continue; - }; - let (key, value) = (key.trim(), value.trim()); - match key { - "model name" if model.is_none() => model = Some(value.to_owned()), - "microcode" if microcode.is_none() => microcode = Some(value.to_owned()), - "cpu cores" if cores_per_socket.is_none() => cores_per_socket = value.parse().ok(), - "physical id" => { - sockets.insert(value.to_owned()); - } - _ => {} - } - } - - let model = model.ok_or_else(|| PerfError::harness("/proc/cpuinfo missing model name"))?; - let microcode = - microcode.ok_or_else(|| PerfError::harness("/proc/cpuinfo missing microcode"))?; - let cores_per_socket = - cores_per_socket.ok_or_else(|| PerfError::harness("/proc/cpuinfo missing cpu cores"))?; - let socket_count = u32::try_from(sockets.len()) - .map_err(|_| PerfError::harness("/proc/cpuinfo socket count overflow"))? - .max(1); - Ok((model, socket_count, cores_per_socket, microcode)) -} - -/// Parses `SwapTotal` (KiB) from `/proc/meminfo`. -fn parse_proc_meminfo_swap(content: &str) -> Result { - for line in content.lines() { - if let Some(rest) = line.strip_prefix("SwapTotal:") { - let value = rest - .split_whitespace() - .next() - .ok_or_else(|| PerfError::harness("/proc/meminfo SwapTotal malformed"))?; - return value - .parse() - .map_err(|_| PerfError::harness("/proc/meminfo SwapTotal not a number")); - } - } - Err(PerfError::harness("/proc/meminfo missing SwapTotal")) -} - -fn parse_cpu_affinity(status: &str) -> Result { - status - .lines() - .find_map(|line| line.strip_prefix("Cpus_allowed_list:")) - .map(str::trim) - .filter(|value| !value.is_empty()) - .map(str::to_owned) - .ok_or_else(|| PerfError::harness("/proc/thread-self/status missing Cpus_allowed_list")) -} - -#[cfg(target_os = "linux")] -const NUMA_MASK_BITS: usize = 1024; -#[cfg(target_os = "linux")] -const NUMA_MASK_WORDS: usize = NUMA_MASK_BITS.div_ceil(libc::c_ulong::BITS as usize); - -#[cfg(target_os = "linux")] -fn format_memory_nodes(mask: &[libc::c_ulong]) -> String { - let mut ranges = Vec::new(); - let mut start = None; - for node in 0..NUMA_MASK_BITS { - let word = node / libc::c_ulong::BITS as usize; - let bit = node % libc::c_ulong::BITS as usize; - let is_set = mask[word] & (1 as libc::c_ulong) << bit != 0; - if is_set && start.is_none() { - start = Some(node); - } else if !is_set && start.is_some() { - let first = start.take().expect("checked above"); - let last = node - 1; - ranges.push(if first == last { - first.to_string() - } else { - format!("{first}-{last}") - }); - } - } - if let Some(first) = start { - let last = NUMA_MASK_BITS - 1; - ranges.push(if first == last { - first.to_string() - } else { - format!("{first}-{last}") - }); - } - ranges.join(",") -} - -#[cfg(target_os = "linux")] -fn decode_memory_policy(mode: libc::c_int, mask: &[libc::c_ulong]) -> Result<(String, String)> { - let mode_flags = libc::MPOL_F_STATIC_NODES | libc::MPOL_F_RELATIVE_NODES; - let base_mode = mode & !mode_flags; - let mut policy = match base_mode { - libc::MPOL_DEFAULT => "default".to_owned(), - libc::MPOL_PREFERRED => "preferred".to_owned(), - libc::MPOL_BIND => "bind".to_owned(), - libc::MPOL_INTERLEAVE => "interleave".to_owned(), - libc::MPOL_LOCAL => "local".to_owned(), - _ => { - return Err(PerfError::harness(format!( - "get_mempolicy(2) returned unsupported mode {mode}" - ))); - } - }; - if mode & libc::MPOL_F_STATIC_NODES != 0 { - policy.push_str("|static"); - } - if mode & libc::MPOL_F_RELATIVE_NODES != 0 { - policy.push_str("|relative"); - } - Ok((policy, format_memory_nodes(mask))) -} - -#[cfg(target_os = "linux")] -#[allow(unsafe_code)] -fn read_memory_policy() -> Result<(String, String)> { - let mut mode = 0 as libc::c_int; - let mut mask = [0 as libc::c_ulong; NUMA_MASK_WORDS]; - // SAFETY: get_mempolicy writes one c_int and at most NUMA_MASK_BITS into - // the two live buffers. flags=0 and addr=NULL request this thread's - // default policy, as required by get_mempolicy(2). - let status = unsafe { - libc::syscall( - libc::SYS_get_mempolicy, - std::ptr::addr_of_mut!(mode), - mask.as_mut_ptr(), - NUMA_MASK_BITS as libc::c_ulong, - std::ptr::null::(), - 0 as libc::c_ulong, - ) - }; - if status != 0 { - return Err(PerfError::harness(format!( - "get_mempolicy(2): {}", - std::io::Error::last_os_error() - ))); - } - decode_memory_policy(mode, &mask) -} - -#[cfg(not(target_os = "linux"))] -fn read_memory_policy() -> Result<(String, String)> { - Err(PerfError::harness( - "NUMA memory policy evidence requires Linux", - )) -} - fn read_observed_conditions(machine: &MachineFingerprint) -> Result { - let status = read_evidence("/proc/thread-self/status")?; - let cpu_affinity = parse_cpu_affinity(&status)?; - let (memory_policy, memory_nodes) = read_memory_policy()?; + let placement = bamts_platform::thread_placement().map_err(platform_error)?; Ok(ObservedConditions { governor: machine.governor.clone(), swap_total_kib: machine.swap_total_kib, - cpu_affinity, - memory_policy, - memory_nodes, + cpu_affinity: placement.cpu_affinity, + memory_policy: placement.memory_policy, + memory_nodes: placement.memory_nodes, }) } +fn platform_error(error: bamts_platform::PlatformError) -> PerfError { + PerfError::harness(error.to_string()) +} + fn require_conditions(observed: &ObservedConditions, expected: &HostConditions) -> Result<()> { if observed.satisfies(expected) { Ok(()) @@ -505,40 +337,26 @@ fn read_validated_host(host: &HostManifest) -> Result<(MachineFingerprint, Obser Ok((machine, observed)) } -/// Reads the live machine fingerprint from `/proc` and `/sys`. +/// Reads the live machine fingerprint from the host. /// /// # Errors -/// Returns [`PerfErrorCode::HarnessError`] if any evidence file cannot be read +/// Returns [`PerfErrorCode::HarnessError`] if any host fact cannot be read /// or parsed. pub fn read_machine_fingerprint() -> Result { - let version = read_evidence("/proc/version")?; - let cpuinfo = read_evidence("/proc/cpuinfo")?; - let meminfo = read_evidence("/proc/meminfo")?; - let governor = read_evidence("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor") - .map(|text| text.trim().to_owned()) - .unwrap_or_else(|_| "unknown".to_owned()); - - let (kernel_release, kernel_version) = parse_proc_version(&version)?; - let (cpu_model, sockets, cores_per_socket, microcode) = parse_proc_cpuinfo(&cpuinfo)?; - let swap_total_kib = parse_proc_meminfo_swap(&meminfo)?; - + let facts = bamts_platform::machine_facts().map_err(platform_error)?; Ok(MachineFingerprint { - kernel_release, - kernel_version, + kernel_release: facts.kernel_release, + kernel_version: facts.kernel_version, arch: std::env::consts::ARCH.to_owned(), - cpu_model, - sockets, - cores_per_socket, - microcode, - governor, - swap_total_kib, + cpu_model: facts.cpu_model, + sockets: facts.sockets, + cores_per_socket: facts.cores_per_socket, + microcode: facts.microcode, + governor: facts.governor, + swap_total_kib: facts.swap_total_kib, }) } -fn read_evidence(path: &str) -> Result { - fs::read_to_string(path).map_err(|error| PerfError::harness(format!("{path}: {error}"))) -} - // --------------------------------------------------------------------------- // Benchmark manifest schema // --------------------------------------------------------------------------- @@ -1025,30 +843,33 @@ pub fn measure(options: &MeasureOptions) -> Result { let mut check_samples = Vec::with_capacity(repeats as usize); let mut emit_samples = Vec::with_capacity(repeats as usize); let mut rss_samples = Vec::with_capacity(repeats as usize); - // Reset the peak RSS high-water mark before every repeat so each sample - // measures only that run's allocation peak, not the cumulative peak of - // every prior repeat and the harness. + // Open a peak-memory window around every repeat so each sample measures only + // that run's allocation peak, not the cumulative peak of every prior repeat + // and the harness. The wall time is taken inside the window, so it spans the + // same work it always did. for _ in 0..repeats { - reset_peak_rss()?; - let started = Instant::now(); - let (_report, telemetry) = run_suite_with_telemetry( - &options.workspace_root, - &options.snapshot_root, - &RunFilterOptions { - status: StatusFilter::All, - slice: Some(options.slice.clone()), - backends: BackendFilter::All, - shards: None, - }, - ) - .map_err(|error| PerfError::harness(format!("run_suite seam failed: {error}")))?; - wall_samples.push(started.elapsed().as_secs_f64() * 1_000.0); + let ((telemetry, wall_ms), peak) = measure_peak(|| { + let started = Instant::now(); + let (_report, telemetry) = run_suite_with_telemetry( + &options.workspace_root, + &options.snapshot_root, + &RunFilterOptions { + status: StatusFilter::All, + slice: Some(options.slice.clone()), + backends: BackendFilter::All, + shards: None, + }, + ) + .map_err(|error| PerfError::harness(format!("run_suite seam failed: {error}")))?; + Ok((telemetry, started.elapsed().as_secs_f64() * 1_000.0)) + })?; + wall_samples.push(wall_ms); scan_samples.push(telemetry.millis(Phase::Scan)); parse_samples.push(telemetry.millis(Phase::Parse)); bind_samples.push(telemetry.millis(Phase::Bind)); check_samples.push(telemetry.millis(Phase::Check)); emit_samples.push(telemetry.millis(Phase::Emit)); - rss_samples.push(read_peak_rss_bytes()? as f64); + rss_samples.push(peak as f64); } // `total` is the whole measured seam wall; the phase split is the compiler's @@ -1160,64 +981,17 @@ fn load_baseline(path: Option<&Path>, host: &HostManifest) -> Result Result<()> { - #[cfg(target_os = "linux")] - { - fs::write("/proc/self/clear_refs", "5") - .map_err(|error| PerfError::harness(format!("/proc/self/clear_refs: {error}"))) - } - - #[cfg(not(target_os = "linux"))] - { - Err(PerfError::harness( - "peak RSS reset requires /proc/self/clear_refs and is only available on Linux", - )) - } -} - -/// Reads the current process peak RSS high-water mark (`VmHWM`) in bytes. -/// -/// After a [`reset_peak_rss`] call this is the peak since that reset, so the -/// benchmark can attribute peak memory to each measured repeat. On non-Linux -/// hosts this returns a contextual error instead of a fake zero. -fn read_peak_rss_bytes() -> Result { - #[cfg(target_os = "linux")] - { - let content = fs::read_to_string("/proc/self/status") - .map_err(|error| PerfError::harness(format!("/proc/self/status: {error}")))?; - for line in content.lines() { - if let Some(rest) = line.strip_prefix("VmHWM:") { - let kib: u64 = rest - .split_whitespace() - .next() - .ok_or_else(|| PerfError::harness("/proc/self/status: VmHWM has no value"))? - .parse() - .map_err(|error| { - PerfError::harness(format!( - "/proc/self/status: VmHWM is not a number: {error}" - )) - })?; - return Ok(kib * 1024); - } - } - Err(PerfError::harness( - "/proc/self/status: VmHWM field not found", - )) - } - - #[cfg(not(target_os = "linux"))] - { - Err(PerfError::harness( - "peak RSS read requires /proc/self/status and is only available on Linux", - )) - } +/// The peak is the resident-set peak on Linux, reset when the window opens, and +/// the sampled physical-footprint peak on macOS, which can miss a spike shorter +/// than a millisecond. A window cannot overlap another on Linux. +fn measure_peak(work: impl FnOnce() -> Result) -> Result<(T, u64)> { + let window = bamts_platform::PeakWindow::start().map_err(platform_error)?; + let output = work()?; + let peak = window.finish().map_err(platform_error)?; + Ok((output, peak)) } fn write_result(path: &Path, result: &MeasureResult) -> Result<()> { @@ -1682,24 +1456,26 @@ fn invalid_conditions(observed: &ObservedConditions, expected: &HostConditions) ) } +/// Polls the peak memory of a child until it is gone, and returns the highest +/// value seen. Run it beside the `wait` that reaps the child: `ProcessGone` +/// comes only after the reap. fn poll_child_peak_rss(pid: u32) -> u64 { - let status_path = format!("/proc/{pid}/status"); + poll_peak(|| bamts_platform::peak_memory_bytes(pid)) +} + +fn poll_peak( + mut read: impl FnMut() -> std::result::Result, +) -> u64 { let mut peak = 0_u64; loop { - let Ok(status) = fs::read_to_string(&status_path) else { - return peak; - }; - for line in status.lines() { - if let Some(value) = line.strip_prefix("VmHWM:") { - if let Some(kib) = value - .split_whitespace() - .next() - .and_then(|v| v.parse::().ok()) - { - peak = peak.max(kib.saturating_mul(1024)); - } - break; - } + match read() { + Ok(bytes) => peak = peak.max(bytes), + // Not the end: Linux reads this for most of a child's exit, and + // macOS while a live child execs. + Err(bamts_platform::PlatformError::NoAddressSpace(_)) => {} + // Gone after the reap, or unreadable. Stop on every other error, so a + // persistent one cannot hang the thread that joins this poller. + Err(_) => return peak, } thread::sleep(Duration::from_millis(2)); } @@ -2505,7 +2281,7 @@ mod tests { assert_eq!(host.conditions.swap_total_kib, 403_979_000); assert_eq!( host.source.get("governor").map(String::as_str), - Some("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor") + Some(bamts_platform::LINUX_GOVERNOR_SOURCE) ); assert_eq!(host.conditions.cpu_affinity, "0-19"); assert_eq!(host.conditions.memory_policy, "bind"); @@ -2686,30 +2462,6 @@ mod tests { assert_eq!(error.code, PerfErrorCode::InvalidHost); } - #[test] - fn parse_proc_version_extracts_release_and_tail() { - let content = "Linux version 7.0.0-28-generic (buildd@lcy02) (gcc 15) #28-Ubuntu SMP PREEMPT_DYNAMIC Sun Jun 21 01:01:36 UTC 2026\n"; - let (release, version) = parse_proc_version(content).unwrap(); - assert_eq!(release, "7.0.0-28-generic"); - assert!(version.starts_with("#28-Ubuntu SMP PREEMPT_DYNAMIC")); - } - - #[test] - fn parse_proc_cpuinfo_counts_sockets_and_cores() { - let content = "model name\t: Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz\nmicrocode\t: 0x2007006\ncpu cores\t: 20\nphysical id\t: 0\n\nphysical id\t: 1\ncpu cores\t: 20\n"; - let (model, sockets, cores, micro) = parse_proc_cpuinfo(content).unwrap(); - assert_eq!(model, "Intel(R) Xeon(R) Gold 6138 CPU @ 2.00GHz"); - assert_eq!(sockets, 2); - assert_eq!(cores, 20); - assert_eq!(micro, "0x2007006"); - } - - #[test] - fn parse_proc_meminfo_reads_swap_total() { - let content = "MemTotal: 791180784 kB\nSwapTotal: 0 kB\n"; - assert_eq!(parse_proc_meminfo_swap(content).unwrap(), 0); - } - #[test] fn nearest_rank_uses_ceil_rank() { let samples = [10.0, 20.0, 30.0, 40.0]; @@ -2785,104 +2537,90 @@ mod tests { let _ = std::fs::remove_file(&path); } - #[cfg(target_os = "linux")] - fn parse_proc_status_value(rest: &str, field: &str) -> Result { - let kib: u64 = rest - .split_whitespace() - .next() - .ok_or_else(|| PerfError::harness(format!("/proc/self/status: {field} has no value")))? - .parse() - .map_err(|error| { - PerfError::harness(format!( - "/proc/self/status: {field} is not a number: {error}" - )) - })?; - Ok(kib * 1024) - } - - #[cfg(target_os = "linux")] - /// Reads the current `(VmHWM, VmRSS)` pair from a single `/proc/self/status` - /// snapshot so the test compares the two fields atomically. - fn read_hwm_and_rss() -> Result<(u64, u64)> { - let content = std::fs::read_to_string("/proc/self/status") - .map_err(|error| PerfError::harness(format!("/proc/self/status: {error}")))?; - let mut hwm = None; - let mut rss = None; - for line in content.lines() { - if let Some(rest) = line.strip_prefix("VmHWM:") { - hwm = Some(parse_proc_status_value(rest, "VmHWM")?); - } else if let Some(rest) = line.strip_prefix("VmRSS:") { - rss = Some(parse_proc_status_value(rest, "VmRSS")?); - } - } - match (hwm, rss) { - (Some(h), Some(r)) => Ok((h, r)), - (None, _) => Err(PerfError::harness("/proc/self/status: VmHWM not found")), - (_, None) => Err(PerfError::harness("/proc/self/status: VmRSS not found")), - } + #[test] + fn the_peak_window_reports_a_touched_allocation() { + const BYTES: usize = 64 * 1024 * 1024; + let (marker, peak) = measure_peak(|| { + let block = vec![1_u8; BYTES]; + // Hold it long enough for the 1 ms macOS sampler to see it. + thread::sleep(Duration::from_millis(150)); + assert_eq!(block.iter().filter(|byte| **byte == 1).count(), BYTES); + Ok("done") + }) + .unwrap(); + assert_eq!(marker, "done"); + assert!(peak >= BYTES as u64, "peak {peak} is below the allocation"); } - #[cfg(target_os = "linux")] #[test] - #[ignore = "requires Node 24.18.0 or /proc/self/clear_refs permission; external_blocked in this environment"] - fn reset_peak_rss_resets_hwm_to_current_rss() { - // Raise the high-water mark by faulting in a large allocation. - let (hwm_before, _rss_before) = - read_hwm_and_rss().expect("read /proc/self/status before allocation"); - let big = { - let mut v = vec![0u8; 64 * 1024 * 1024]; - for i in (0..v.len()).step_by(4096) { - v[i] = 1; - } - std::hint::black_box(v) - }; - let (hwm_during, _rss_during) = - read_hwm_and_rss().expect("read /proc/self/status during allocation"); - assert!(hwm_during >= hwm_before, "allocation must not lower VmHWM"); + fn a_failed_measurement_returns_the_work_error() { + let error = + measure_peak(|| -> Result<()> { Err(PerfError::harness("work failed")) }).unwrap_err(); + assert!(error.detail.contains("work failed")); + } - // Dropping the allocation returns the mapped pages to the OS on glibc, - // lowering current RSS while leaving the lifetime HWM high. - drop(big); + fn scripted(script: Vec>) -> u64 { + let mut reads = script.into_iter(); + poll_peak(|| reads.next().expect("the poller read past its script")) + } - reset_peak_rss().expect("reset VmHWM"); + #[test] + fn the_poller_keeps_the_highest_peak_until_the_child_is_gone() { + use bamts_platform::PlatformError::ProcessGone; + assert_eq!(scripted(vec![Ok(5), Ok(9), Ok(7), Err(ProcessGone(1))]), 9); + } - let (hwm_after, rss_after) = - read_hwm_and_rss().expect("read /proc/self/status after reset"); - assert!( - hwm_after > 0, - "VmHWM after reset must be positive, not fake" - ); - assert!( - hwm_after >= rss_after, - "VmHWM cannot be below current VmRSS (hwm={hwm_after}, rss={rss_after})" - ); - assert!( - hwm_after <= rss_after + 8 * 1024 * 1024, - "VmHWM after reset must be near current VmRSS (within test overhead); \ - got hwm={hwm_after}, rss={rss_after}" - ); + /// Nearly every Linux child reads this way for most of its exit, and macOS + /// reads it while a child execs. Neither is the end. + #[test] + fn the_poller_keeps_polling_through_a_missing_address_space() { + use bamts_platform::PlatformError::{NoAddressSpace, ProcessGone}; + let script = vec![ + Ok(5), + Err(NoAddressSpace(1)), + Err(NoAddressSpace(1)), + Ok(9), + Err(ProcessGone(1)), + ]; + assert_eq!(scripted(script), 9); } - #[cfg(not(target_os = "linux"))] + /// A persistent read error must end the poll, or `sampler.join()` would hang. #[test] - fn peak_rss_sampling_is_unsupported_outside_linux() { - // Resetting and reading VmHWM are platform-specific; on non-Linux they - // must fail with a typed, contextual error rather than return fake data. - let reset = - reset_peak_rss().expect_err("reset should fail on non-Linux with a contextual error"); - assert_eq!(reset.code, PerfErrorCode::HarnessError); - assert!( - reset.detail.contains("Linux"), - "reset error must name the platform" - ); + fn the_poller_stops_on_any_other_error() { + use bamts_platform::PlatformError::Read; + let broken = Read { + source: "peak".to_owned(), + detail: "denied".to_owned(), + }; + assert_eq!(scripted(vec![Ok(5), Err(broken.clone())]), 5); + assert_eq!(scripted(vec![Err(broken)]), 0); + } + + /// A child that outlives a few polls: `sleep 0.3` on Unix, and on Windows + /// `ping` with two echoes, which are one second apart. + fn short_lived_child() -> Command { + if cfg!(windows) { + let mut command = + Command::new(crate::test_programs::windows_system_program("PING.EXE")); + command + .args(["-n", "2", "127.0.0.1"]) + .stdout(std::process::Stdio::null()); + command + } else { + let mut command = Command::new("/bin/sh"); + command.args(["-c", "sleep 0.3"]); + command + } + } - let read = read_peak_rss_bytes() - .expect_err("read should fail on non-Linux with a contextual error"); - assert_eq!(read.code, PerfErrorCode::HarnessError); - assert!( - read.detail.contains("Linux"), - "read error must name the platform" - ); + #[test] + fn polling_beside_a_real_child_ends_when_it_is_reaped() { + let mut child = short_lived_child().spawn().unwrap(); + let pid = child.id(); + let poller = thread::spawn(move || poll_child_peak_rss(pid)); + child.wait().unwrap(); + assert!(poller.join().unwrap() > 0); } #[test] @@ -3168,24 +2906,4 @@ mod tests { assert!(!mismatch.satisfies(&expected)); } } - - #[test] - fn parses_thread_cpu_affinity() { - let status = - "Name:\tperf_budget\nCpus_allowed:\t00000000,000fffff\nCpus_allowed_list:\t0-19\n"; - assert_eq!(parse_cpu_affinity(status).unwrap(), "0-19"); - assert!(parse_cpu_affinity("Name:\tperf_budget\n").is_err()); - } - - #[cfg(target_os = "linux")] - #[test] - fn decodes_memory_policy_and_node_ranges() { - let mut mask = [0 as libc::c_ulong; NUMA_MASK_WORDS]; - mask[0] = 0b1_1111; - assert_eq!( - decode_memory_policy(libc::MPOL_BIND, &mask).unwrap(), - ("bind".to_owned(), "0-4".to_owned()) - ); - assert!(decode_memory_policy(-1, &mask).is_err()); - } } diff --git a/crates/bamts-verification/src/perf_jit.rs b/crates/bamts-verification/src/perf_jit.rs index 5c0db0f2..cc9243b0 100644 --- a/crates/bamts-verification/src/perf_jit.rs +++ b/crates/bamts-verification/src/perf_jit.rs @@ -103,10 +103,10 @@ pub struct ConditionsRecord { impl Conditions { pub fn capture() -> Result { Ok(Self { - governors: read_governors()?, - swap_total_kb: read_swap_total_kb()?, - cpus_allowed_list: read_cpus_allowed_list()?, - cpus_online: read_cpus_online()?, + governors: bamts_platform::cpu_governors()?, + swap_total_kb: bamts_platform::swap_total_kib()?, + cpus_allowed_list: bamts_platform::cpu_affinity()?, + cpus_online: bamts_platform::online_cpu_list()?, }) } @@ -156,52 +156,6 @@ impl Conditions { } } -fn read_governors() -> Result> { - let online = parse_cpu_list(&read_cpus_online()?)?; - let mut out = Vec::new(); - for entry in std::fs::read_dir("/sys/devices/system/cpu")? { - let entry = entry?; - let name = entry.file_name().to_string_lossy().to_string(); - if let Some(n) = name.strip_prefix("cpu") - && n.chars().all(|c| c.is_ascii_digit()) - { - let number: u32 = n.parse()?; - if online.contains(&number) { - let path = entry.path().join("cpufreq/scaling_governor"); - out.push(read_to_string(&path).unwrap_or_default().trim().to_owned()); - } - } - } - out.sort(); - Ok(out) -} - -fn read_swap_total_kb() -> Result { - let text = read_to_string("/proc/meminfo")?; - text.lines() - .find_map(|line| { - line.split_once("SwapTotal:") - .and_then(|(_, rest)| rest.split_whitespace().next()) - }) - .ok_or_else(|| anyhow!("SwapTotal not found in /proc/meminfo"))? - .parse() - .map_err(|e| anyhow!("SwapTotal: {e}")) -} - -fn read_cpus_allowed_list() -> Result { - let text = read_to_string("/proc/self/status")?; - text.lines() - .find_map(|line| line.split_once("Cpus_allowed_list:")) - .map(|(_, v)| v.trim().to_owned()) - .ok_or_else(|| anyhow!("Cpus_allowed_list not found in /proc/self/status")) -} - -fn read_cpus_online() -> Result { - Ok(read_to_string("/sys/devices/system/cpu/online")? - .trim() - .to_owned()) -} - fn parse_cpu_list(s: &str) -> Result> { let mut set = BTreeSet::new(); for part in s.split(',') { @@ -596,6 +550,31 @@ mod tests { verify_catalog().unwrap(); } + /// The conditions come from the platform crate on every host, so macOS can + /// read them too. The old readers opened `/sys` and `/proc` directly. + #[test] + fn conditions_come_from_the_platform_crate() { + let conditions = Conditions::capture().unwrap(); + // The same readers the harness uses: `thread_placement` also asks for the + // NUMA policy, which a sandbox that blocks `get_mempolicy` refuses. + assert_eq!( + conditions.governors, + bamts_platform::cpu_governors().unwrap() + ); + assert_eq!( + conditions.swap_total_kb, + bamts_platform::swap_total_kib().unwrap() + ); + assert_eq!( + conditions.cpus_allowed_list, + bamts_platform::cpu_affinity().unwrap() + ); + assert_eq!( + conditions.cpus_online, + bamts_platform::online_cpu_list().unwrap() + ); + } + #[test] fn conditions_detect_full_machine_affinity() { let c = Conditions { diff --git a/crates/bamts-verification/src/perf_stage0.rs b/crates/bamts-verification/src/perf_stage0.rs index 967178e8..79145b63 100644 --- a/crates/bamts-verification/src/perf_stage0.rs +++ b/crates/bamts-verification/src/perf_stage0.rs @@ -7,6 +7,8 @@ use anyhow::{Context, Result}; use bamts_cli::args::{ExecutionTarget, Mode}; use bamts_cli::driver; +use bamts_compiler::project::canonical::canonicalize; +use bamts_platform::{MemoryRegion, PlatformError}; use serde::{Deserialize, Serialize}; use serde_json::{Map, Value, json}; use sha2::{Digest, Sha256}; @@ -42,11 +44,11 @@ pub struct HostConditions { impl HostConditions { fn new() -> Result { Ok(Self { - governor: capture_governor()?, - swap_total_kib: capture_swap_total_kib()?, - cpu_affinity: capture_cpu_affinity()?, - mempolicy_mode: capture_mempolicy_mode(), - memory_nodes: capture_memory_nodes()?, + governor: bamts_platform::cpu_governor()?, + swap_total_kib: bamts_platform::swap_total_kib()?, + cpu_affinity: bamts_platform::cpu_affinity()?, + mempolicy_mode: bamts_platform::numa_policy_mode(), + memory_nodes: bamts_platform::numa_memory_nodes()?, }) } } @@ -255,13 +257,6 @@ impl RealEvidenceProvider<'_> { } } -#[derive(Debug, Clone)] -struct ProcessMapping { - identity: String, - permissions: String, - pathname: Option, -} - pub fn run() -> Value { let root = match repo_root() { Ok(root) => root, @@ -669,7 +664,7 @@ fn validate_event_stream(stdout: &[u8]) -> Result> { #[derive(Debug)] struct ChildMapSample { - mappings: Vec, + mappings: Vec, executable_mappings: usize, samples: usize, } @@ -695,8 +690,7 @@ fn capture_aot_child_maps(root: &Path, workload: &Path) -> Result Resu } fn sample_child_maps_with_observer( + child: Child, + executable: &Path, + timeout: Duration, + observed: impl FnMut(&mut Child, &[MemoryRegion]) -> Result<()>, +) -> Result { + sample_child_maps_from( + child, + executable, + timeout, + |child| process_maps_for(child.id()), + observed, + ) +} + +fn sample_child_maps_from( mut child: Child, executable: &Path, timeout: Duration, - mut observed: impl FnMut(&mut Child, &[ProcessMapping]) -> Result<()>, + mut read_maps: impl FnMut(&mut Child) -> Result>, + mut observed: impl FnMut(&mut Child, &[MemoryRegion]) -> Result<()>, ) -> Result { let stdout = child .stdout @@ -740,8 +750,21 @@ fn sample_child_maps_with_observer( Ok(None) => {} Err(error) => break Err(error), } - let mappings = match process_maps_for(child.id()) { + if std::time::Instant::now() >= deadline { + break Err(anyhow::anyhow!( + "controlled child did not exit before timeout" + )); + } + let mappings = match read_maps(&mut child) { Ok(mappings) => mappings, + // Neither answer is final. An exiting child gives them before it + // can be reaped, and a live child gives `NoAddressSpace` during an + // exec. Keep sampling: `try_wait` decides an exit, and a new image + // that outlives a poll still gets checked. + Err(error) if child_is_between_states(&error) => { + thread::sleep(Duration::from_millis(1)); + continue; + } Err(error) => match child .try_wait() .context("checking child after map-read failure") @@ -775,11 +798,6 @@ fn sample_child_maps_with_observer( for mapping in mappings { union.entry(mapping.identity.clone()).or_insert(mapping); } - if std::time::Instant::now() >= deadline { - break Err(anyhow::anyhow!( - "controlled child did not exit before timeout" - )); - } thread::sleep(Duration::from_millis(1)); }; @@ -833,103 +851,29 @@ fn join_pipe_reader(reader: thread::JoinHandle>>) -> io::Resu .map_err(|_| io::Error::other("controlled child pipe reader panicked"))? } -fn process_maps_for(pid: u32) -> Result> { - let text = fs::read_to_string(format!("/proc/{pid}/maps")) - .with_context(|| format!("reading /proc/{pid}/maps"))?; - parse_process_maps(&text) +fn process_maps_for(pid: u32) -> Result> { + bamts_platform::process_regions(pid) + .with_context(|| format!("reading the memory regions of process {pid}")) } -fn process_maps() -> Result> { +fn process_maps() -> Result> { process_maps_for(std::process::id()) } -fn parse_process_maps(text: &str) -> Result> { - text.lines() - .map(|line| { - let mut fields = line.split_whitespace(); - let range = fields.next().context("mapping lacks range")?; - let permissions = fields - .next() - .context("mapping lacks permissions")? - .to_owned(); - let offset = fields.next().context("mapping lacks offset")?; - let device = fields.next().context("mapping lacks device")?; - let inode = fields.next().context("mapping lacks inode")?; - let pathname = fields.next().map(str::to_owned); - Ok(ProcessMapping { - identity: format!( - "{range} {offset} {device} {inode} {}", - pathname.as_deref().unwrap_or("") - ), - permissions, - pathname, - }) - }) - .collect() +fn child_is_between_states(error: &anyhow::Error) -> bool { + matches!( + error.downcast_ref::(), + Some(PlatformError::ProcessGone(_) | PlatformError::NoAddressSpace(_)) + ) } -fn anonymous_executable_mappings(maps: &[ProcessMapping]) -> BTreeSet { +fn anonymous_executable_mappings(maps: &[MemoryRegion]) -> BTreeSet { maps.iter() .filter(|mapping| mapping.permissions.contains('x') && mapping.pathname.is_none()) .map(|mapping| mapping.identity.clone()) .collect() } -fn capture_governor() -> Result { - Ok( - fs::read_to_string("/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor")? - .trim() - .to_owned(), - ) -} -fn capture_swap_total_kib() -> Result { - let text = fs::read_to_string("/proc/meminfo")?; - text.lines() - .find_map(|line| line.strip_prefix("SwapTotal:")) - .and_then(|rest| rest.split_whitespace().next()) - .context("SwapTotal not found")? - .parse() - .context("parsing SwapTotal") -} -fn capture_cpu_affinity() -> Result { - let text = fs::read_to_string("/proc/self/status")?; - text.lines() - .find_map(|line| line.strip_prefix("Cpus_allowed_list:")) - .map(str::trim) - .map(str::to_owned) - .context("Cpus_allowed_list not found") -} -fn capture_mempolicy_mode() -> String { - Command::new("numactl") - .arg("--show") - .output() - .ok() - .filter(|output| output.status.success()) - .and_then(|output| String::from_utf8(output.stdout).ok()) - .and_then(|text| { - text.lines().find_map(|line| { - line.strip_prefix("policy:") - .map(str::trim) - .map(str::to_owned) - }) - }) - .unwrap_or_else(|| "unavailable".into()) -} -fn capture_memory_nodes() -> Result> { - if let Ok(output) = Command::new("numactl").arg("--show").output() - && output.status.success() - && let Ok(text) = String::from_utf8(output.stdout) - && let Some(nodes) = text.lines().find_map(|line| line.strip_prefix("nodebind:")) - { - return parse_whitespace_list(nodes.trim()); - } - let text = fs::read_to_string("/proc/self/status")?; - text.lines() - .find_map(|line| line.strip_prefix("Mems_allowed_list:")) - .map(str::trim) - .context("Mems_allowed_list not found") - .and_then(parse_range_list) -} fn parse_range_list(text: &str) -> Result> { anyhow::ensure!(!text.trim().is_empty(), "memory-node list is empty"); let mut nodes = Vec::new(); @@ -949,11 +893,6 @@ fn parse_range_list(text: &str) -> Result> { } Ok(nodes) } -fn parse_whitespace_list(text: &str) -> Result> { - text.split_whitespace() - .map(|token| token.parse().map_err(Into::into)) - .collect() -} #[cfg(test)] mod tests { @@ -972,6 +911,33 @@ mod tests { } } + /// The conditions come from the platform crate on every host, so macOS can + /// read them too. The old readers opened `/sys` and `/proc` directly. + #[test] + fn host_conditions_come_from_the_platform_crate() { + let conditions = HostConditions::new(); + // A Linux host without cpufreq has no governor, which stage 0 refuses + // rather than recording a stand-in. + if bamts_platform::cpu_governor().is_err() { + assert!(conditions.is_err()); + return; + } + let conditions = conditions.unwrap(); + assert_eq!(conditions.governor, bamts_platform::cpu_governor().unwrap()); + assert_eq!( + conditions.swap_total_kib, + bamts_platform::swap_total_kib().unwrap() + ); + assert_eq!( + conditions.cpu_affinity, + bamts_platform::cpu_affinity().unwrap() + ); + assert_eq!( + conditions.mempolicy_mode, + bamts_platform::numa_policy_mode() + ); + } + fn conditions() -> HostConditions { HostConditions { governor: "powersave".into(), @@ -1156,13 +1122,7 @@ requires="live map captured" #[test] fn continuous_sampler_accepts_safe_child_through_exit() { - let executable = fs::canonicalize("/bin/cat").unwrap(); - let child = Command::new(&executable) - .stdin(Stdio::piped()) - .stdout(Stdio::piped()) - .stderr(Stdio::piped()) - .spawn() - .unwrap(); + let (executable, child) = spawn_idle_cat(); let mut executable_samples = 0; let sample = sample_child_maps_with_observer( child, @@ -1192,9 +1152,167 @@ requires="live map captured" assert!(anonymous_executable_mappings(&sample.mappings).is_empty()); } + /// A program that stays alive until its standard input closes, then exits with + /// status 0: `cat` on Unix, `sort` from the system directory on Windows. + fn idle_program() -> PathBuf { + if cfg!(windows) { + crate::test_programs::windows_system_program("sort.exe") + } else { + PathBuf::from("/bin/cat") + } + } + + fn spawn_idle_cat() -> (PathBuf, Child) { + let executable = canonicalize(idle_program()).unwrap(); + let child = Command::new(&executable) + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .unwrap(); + (executable, child) + } + + /// `spawn` can return before the executable is mapped, so a fixture that + /// needs it observed reads real maps until it appears. + fn shows_executable(mappings: &[MemoryRegion], executable: &Path) -> bool { + mappings + .iter() + .any(|mapping| mapping.pathname.as_deref().map(Path::new) == Some(executable)) + } + + /// macOS gives both errors before an exiting child can be reaped. How often + /// a real read lands there depends on load, so these reads are scripted. #[test] - fn continuous_sampler_rejects_forbidden_mapping_added_after_executable_observation() { - let temp = TempDir::new("stage0-late-executable-map").unwrap(); + fn continuous_sampler_polls_a_child_that_reads_as_gone_until_it_exits() { + for error in [ + PlatformError::ProcessGone(1), + PlatformError::NoAddressSpace(1), + ] { + let (executable, child) = spawn_idle_cat(); + let mut mapped = false; + let sample = sample_child_maps_from( + child, + &executable, + Duration::from_secs(2), + |child| { + if mapped { + // EOF starts the exit, so `try_wait` still sees a live child. + drop(child.stdin.take()); + return Err(anyhow::Error::new(error.clone())); + } + let mappings = process_maps_for(child.id())?; + mapped = shows_executable(&mappings, &executable); + Ok(mappings) + }, + |_, _| Ok(()), + ) + .unwrap(); + assert!(sample.executable_mappings > 0); + } + } + + /// macOS also reports no address space while a live child execs, so that + /// answer must never end sampling: the new image still has to be checked. + #[test] + fn continuous_sampler_checks_the_image_a_child_execs_into() { + let (executable, child) = spawn_idle_cat(); + let mut mapped = false; + let mut execed = false; + let error = sample_child_maps_from( + child, + &executable, + Duration::from_secs(2), + |child| { + if execed { + return Ok(vec![MemoryRegion { + identity: "new image".to_owned(), + permissions: "rwxp".to_owned(), + pathname: None, + }]); + } + if mapped { + execed = true; + let error = PlatformError::NoAddressSpace(child.id()); + return Err(anyhow::Error::new(error)); + } + let mappings = process_maps_for(child.id())?; + mapped = shows_executable(&mappings, &executable); + Ok(mappings) + }, + |_, _| Ok(()), + ) + .unwrap_err(); + assert!(error.to_string().contains("RWX"), "{error:#}"); + } + + #[test] + fn continuous_sampler_times_out_on_a_child_that_stays_unreadable() { + let (executable, child) = spawn_idle_cat(); + let error = sample_child_maps_from( + child, + &executable, + Duration::from_millis(50), + |child| { + let error = PlatformError::NoAddressSpace(child.id()); + Err(anyhow::Error::new(error)) + }, + |_, _| Ok(()), + ) + .unwrap_err(); + assert!( + error.to_string().contains("did not exit before timeout"), + "{error:#}" + ); + } + + #[test] + fn a_reaped_child_is_recognised_through_the_error_context() { + let mut child = Command::new(idle_program()) + .stdin(Stdio::null()) + .spawn() + .unwrap(); + let pid = child.id(); + child.wait().unwrap(); + assert!(child_is_between_states(&process_maps_for(pid).unwrap_err())); + } + + /// Waits for one byte, maps a private page and makes it executable, then waits + /// for a second byte that never comes, as the Unix child does. + #[cfg(windows)] + const LATE_MAP_SOURCE_WINDOWS: &str = r#" +use std::io::Read; +#[link(name = "kernel32")] +unsafe extern "system" { + fn VirtualAlloc(address: *mut u8, size: usize, allocation: u32, protect: u32) -> *mut u8; + fn VirtualProtect(address: *mut u8, size: usize, protect: u32, old: *mut u32) -> i32; +} +fn main() { + let mut observed = [0_u8; 1]; + let mut stdin = std::io::stdin(); + if stdin.read(&mut observed).unwrap_or(0) != 1 { + std::process::exit(3); + } + unsafe { + // MEM_COMMIT | MEM_RESERVE, then PAGE_READWRITE -> PAGE_EXECUTE_READ. + let page = VirtualAlloc(std::ptr::null_mut(), 4096, 0x3000, 0x04); + let mut old = 0_u32; + if page.is_null() || VirtualProtect(page, 4096, 0x20, &mut old) == 0 { + std::process::exit(2); + } + } + // Keep the forbidden mapping live until the sampler rejects and kills us. + if stdin.read(&mut observed).unwrap_or(0) != 1 { + std::process::exit(4); + } + std::process::exit(5); +} +"#; + + /// Builds the child that maps an executable page once it is released: a C program + /// on Unix, where `cc` and `mmap` exist. + #[cfg(not(windows))] + fn compile_late_map_child(temp: &TempDir) -> PathBuf { let source = temp.path().join("late-map.c"); let executable = temp.path().join("late-map"); fs::write( @@ -1223,7 +1341,32 @@ int main(void) { .status() .unwrap(); assert!(compiled.success()); - let executable = fs::canonicalize(executable).unwrap(); + executable + } + + /// Builds the child that maps an executable page once it is released. Windows has + /// no `cc` and no `mmap`, so the child is a small Rust program that calls + /// `VirtualAlloc` and `VirtualProtect`, built with the `rustc` of this toolchain. + #[cfg(windows)] + fn compile_late_map_child(temp: &TempDir) -> PathBuf { + let source = temp.path().join("late_map.rs"); + let executable = temp.path().join("late-map.exe"); + fs::write(&source, LATE_MAP_SOURCE_WINDOWS).unwrap(); + let compiled = Command::new("rustc") + .arg("--edition=2021") + .arg(&source) + .arg("-o") + .arg(&executable) + .status() + .unwrap(); + assert!(compiled.success()); + executable + } + + #[test] + fn continuous_sampler_rejects_forbidden_mapping_added_after_executable_observation() { + let temp = TempDir::new("stage0-late-executable-map").unwrap(); + let executable = canonicalize(compile_late_map_child(&temp)).unwrap(); let child = Command::new(&executable) .stdin(Stdio::piped()) .stdout(Stdio::piped()) diff --git a/crates/bamts-verification/src/rebuild.rs b/crates/bamts-verification/src/rebuild.rs index 5f2b9ba0..157c8d85 100644 --- a/crates/bamts-verification/src/rebuild.rs +++ b/crates/bamts-verification/src/rebuild.rs @@ -11,6 +11,11 @@ //! run binding, or a missing receipt is a rejection, and a rejection never //! publishes. //! +//! One ledger row is one manifest identifier. A receipt names it by lane key, +//! and an identifier is proven in every mode its catalog declares: `test262` +//! needs the interpreter, JIT, and AOT to agree, so its row settles only when +//! all three have reported the same state. +//! //! Memory is one receipt row, one ledger row, and bounded per-obligation //! outcome state. The ledger document is never materialized as a value tree. //! Publication streams into a sibling temporary file and renames it only after @@ -41,7 +46,7 @@ use crate::{ validate_manifest, }, shard::{ExecutionMode, ObligationKey}, - suite::completion::current_run_bindings, + suite::completion::{current_run_bindings, logical_obligation, required_modes}, }; /// Canonical published ledger, relative to the repository root. @@ -109,7 +114,7 @@ pub(crate) fn is_generated_run_output(path: &[u8]) -> bool { } /// Schema tag the G0 ledger verifier requires. -pub const LEDGER_SCHEMA: &str = "bamti.completeness-ledger/v1"; +pub const LEDGER_SCHEMA: &str = "bamti.completeness-ledger/v2"; /// Closed ledger state vocabulary, in canonical declaration order. pub const LEDGER_STATES: [&str; 7] = [ @@ -184,12 +189,14 @@ impl LedgerProse { /// Why one obligation cannot be projected into a green ledger row. #[derive(Debug, Clone, PartialEq, Eq)] pub enum RejectionReason { - /// No receipt recorded this obligation. - MissingReceipt, + /// No receipt recorded this obligation in a mode its catalog requires. + MissingReceipt { mode: ExecutionMode }, /// A receipt named an obligation the manifest does not declare. UnknownObligation, /// Two receipts recorded the same obligation and run coordinates. DuplicateReceipt, + /// The modes of one obligation reported different states. + ModeDisagreement { states: String }, /// A PASS row did not declare exactly the identity's observables. ObservableMismatch { expected: String, actual: String }, /// A receipt declared an observable without a digest. @@ -213,11 +220,20 @@ pub enum RejectionReason { impl fmt::Display for RejectionReason { fn fmt(&self, formatter: &mut fmt::Formatter<'_>) -> fmt::Result { match self { - Self::MissingReceipt => formatter.write_str("no receipt recorded this obligation"), + Self::MissingReceipt { mode } => { + write!( + formatter, + "no {} receipt recorded this obligation", + mode.as_str() + ) + } Self::UnknownObligation => { formatter.write_str("receipt names an obligation the manifest does not declare") } Self::DuplicateReceipt => formatter.write_str("duplicate receipt for one obligation"), + Self::ModeDisagreement { states } => { + write!(formatter, "modes disagree on this obligation: {states}") + } Self::ObservableMismatch { expected, actual } => write!( formatter, "PASS declares observables `{actual}` instead of `{expected}`" @@ -302,10 +318,9 @@ impl RebuildReport { pub struct ObligationSlot { catalog: String, identifier: String, - configuration: String, - mode: ExecutionMode, + modes: Vec, platform: String, - declared_observable: Option, + observables: BTreeSet, classification: Option, } @@ -313,12 +328,22 @@ pub struct ObligationSlot { #[derive(Debug, Clone, Default, PartialEq, Eq)] pub struct ObligationIndex { slots: BTreeMap, + /// The row each lane cell proves. Mode and platform are not part of a cell: + /// they are checked against the slot, so a wrong one reads as stale. + rows: BTreeMap, +} + +/// A lane cell as one string. Key tokens cannot contain NUL, so this is injective. +fn lane_cell(key: &ObligationKey) -> String { + format!("{}\0{}\0{}", key.catalog(), key.case(), key.configuration()) } impl ObligationIndex { /// Builds the index from manifest row ids of the form `catalog:identifier`. pub fn from_row_ids(row_ids: &BTreeSet) -> Result { + let platform = bamts_platform::host_triple(); let mut slots = BTreeMap::new(); + let mut rows = BTreeMap::new(); for row_id in row_ids { let (catalog, identifier) = row_id.split_once(':').ok_or_else(|| { VerificationError::new( @@ -332,20 +357,36 @@ impl ObligationIndex { format!("manifest row `{row_id}` has an empty component"), )); } + let modes = required_modes(catalog)?; + let obligation = logical_obligation(catalog, identifier, modes[0], &platform)?; + if rows + .insert(lane_cell(&obligation.key), row_id.clone()) + .is_some() + { + return Err(VerificationError::new( + ErrorCode::SetMismatch, + format!("manifest row `{row_id}` shares a lane key with another row"), + )); + } slots.insert( row_id.clone(), ObligationSlot { catalog: catalog.to_owned(), identifier: identifier.to_owned(), - configuration: declared_configuration(identifier), - mode: ExecutionMode::Interpreter, - platform: native_platform(), - declared_observable: declared_observable(identifier), + modes, + platform: platform.clone(), + observables: obligation.observables, classification: None, }, ); } - Ok(Self { slots }) + Ok(Self { slots, rows }) + } + + /// The manifest row a receipt key proves, if it proves one. + fn resolve(&self, key: &ObligationKey) -> Option<(&str, &ObligationSlot)> { + let row_id = self.rows.get(&lane_cell(key))?; + Some((row_id, self.slots.get(row_id)?)) } fn attach_classifications( @@ -404,39 +445,6 @@ impl ObligationIndex { } } -/// A cell identity ends with `#observable`; other identities declare none. -fn declared_observable(identifier: &str) -> Option { - identifier - .rsplit_once('#') - .map(|(_, observable)| observable.to_owned()) - .filter(|observable| !observable.is_empty()) -} - -fn declared_configuration(identifier: &str) -> String { - identifier - .rsplit_once('#') - .and_then(|(prefix, _)| prefix.rsplit_once('#')) - .map_or_else( - || "default".to_owned(), - |(_, configuration)| configuration.to_owned(), - ) -} -fn native_platform() -> String { - let environment = if cfg!(target_env = "musl") { - "musl" - } else if cfg!(target_env = "msvc") { - "msvc" - } else { - "gnu" - }; - match std::env::consts::OS { - "linux" => format!("{}-unknown-linux-{environment}", std::env::consts::ARCH), - "macos" => format!("{}-apple-darwin", std::env::consts::ARCH), - "windows" => format!("{}-pc-windows-{environment}", std::env::consts::ARCH), - os => format!("{}-unknown-{os}", std::env::consts::ARCH), - } -} - /// Rebuilds the ledger from the manifest and the supplied receipts. pub fn rebuild_ledger( root: &Path, @@ -598,6 +606,21 @@ struct Outcome { rejection: Option, } +/// Every mode one obligation has reported in, including any it should not have. +type ModeOutcomes = BTreeMap; + +/// The state one row publishes. It needs every required mode to have reported, +/// nothing rejected, and all modes in agreement. +fn settled_state(slot: &ObligationSlot, recorded: &ModeOutcomes) -> Option { + let complete = slot.modes.iter().all(|mode| recorded.contains_key(mode)); + if !complete || recorded.values().any(|outcome| outcome.rejection.is_some()) { + return None; + } + let mut states = recorded.values().map(|outcome| outcome.state); + let first = states.next()?; + states.all(|state| state == first).then_some(first) +} + pub(crate) fn rebuild_projection( root: &Path, index: &ObligationIndex, @@ -621,7 +644,7 @@ pub(crate) fn rebuild_projection( } let ledger_path = root.join(LEDGER_PATH); - let mut outcomes: BTreeMap = BTreeMap::new(); + let mut outcomes: BTreeMap = BTreeMap::new(); let mut rejections: Vec = Vec::new(); let mut receipts_consumed = 0usize; let catalogs = index.catalogs(); @@ -688,29 +711,45 @@ pub(crate) fn rebuild_projection( reader.finish()?; } - for row_id in index.slots.keys() { - match outcomes.get_mut(row_id) { - None => rejections.push(RebuildRejection { + for (row_id, slot) in &index.slots { + let recorded = outcomes.entry(row_id.clone()).or_default(); + let before = rejections.len(); + let mut reject = |reason: RejectionReason| { + rejections.push(RebuildRejection { row_id: row_id.clone(), - reason: RejectionReason::MissingReceipt, - }), - Some(outcome) => { - if !outcome.state.is_pass() - && outcome.rejection.is_none() - && !prose.contains_key(row_id) - { - outcome.rejection = Some(RejectionReason::MissingPolicyProse { - state: outcome.state, - }); - } - if let Some(reason) = &outcome.rejection { - rejections.push(RebuildRejection { - row_id: row_id.clone(), - reason: reason.clone(), - }); - } + reason, + }); + }; + for mode in &slot.modes { + if !recorded.contains_key(mode) { + reject(RejectionReason::MissingReceipt { mode: *mode }); } } + for outcome in recorded.values_mut() { + if !outcome.state.is_pass() + && outcome.rejection.is_none() + && !prose.contains_key(row_id) + { + outcome.rejection = Some(RejectionReason::MissingPolicyProse { + state: outcome.state, + }); + } + if let Some(reason) = &outcome.rejection { + reject(reason.clone()); + } + } + if rejections.len() == before && settled_state(slot, recorded).is_none() { + let states: Vec = recorded + .iter() + .map(|(mode, outcome)| format!("{}={}", mode.as_str(), outcome.state.as_str())) + .collect(); + rejections.push(RebuildRejection { + row_id: row_id.clone(), + reason: RejectionReason::ModeDisagreement { + states: states.join(","), + }, + }); + } } let projection = Projection { @@ -759,14 +798,13 @@ pub(crate) fn rebuild_projection( fn record_outcome( index: &ObligationIndex, - outcomes: &mut BTreeMap, + outcomes: &mut BTreeMap, rejections: &mut Vec, row: &EvidenceRow, ) { - let row_id = row_id_for(row.key()); - let Some(slot) = index.slots.get(&row_id) else { + let Some((row_id, slot)) = index.resolve(row.key()) else { rejections.push(RebuildRejection { - row_id, + row_id: row_id_for(row.key()), reason: RejectionReason::UnknownObligation, }); return; @@ -778,7 +816,8 @@ fn record_outcome( row.observables(), row.artifacts(), ); - match outcomes.entry(row_id) { + let recorded = outcomes.entry(row_id.to_owned()).or_default(); + match recorded.entry(row.key().mode()) { Entry::Vacant(slot) => { slot.insert(Outcome { state: row.state(), @@ -802,8 +841,7 @@ fn admit( artifacts: &BTreeMap, ) -> Option { for (field, matches) in [ - ("configuration", key.configuration() == slot.configuration), - ("execution_mode", key.mode() == slot.mode), + ("execution_mode", slot.modes.contains(&key.mode())), ("platform", key.platform() == slot.platform), ] { if !matches { @@ -821,14 +859,11 @@ fn admit( }); } if state.is_pass() { - if let Some(declared) = &slot.declared_observable { - let expected: BTreeSet = BTreeSet::from([declared.clone()]); - if observables != &expected { - return Some(RejectionReason::ObservableMismatch { - expected: render_set(&expected), - actual: render_set(observables), - }); - } + if observables != &slot.observables { + return Some(RejectionReason::ObservableMismatch { + expected: render_set(&slot.observables), + actual: render_set(observables), + }); } for observable in observables { if !artifacts.contains_key(observable) { @@ -912,21 +947,27 @@ fn receipt_label(receipt: &Path) -> String { receipt.display().to_string() } +/// One published row. Prose exists only on non-PASS rows and the case is the +/// tail of `id`, so a PASS row is these three fields: at the manifest's full size +/// that keeps the ledger about half the size of writing every field. #[derive(Serialize)] struct LedgerRowOut<'a> { - case: &'a str, catalog: &'a str, + #[serde(skip_serializing_if = "str::is_empty")] citation: &'a str, id: &'a str, + #[serde(skip_serializing_if = "str::is_empty")] matcher: &'a str, + #[serde(skip_serializing_if = "str::is_empty")] owner: &'a str, + #[serde(skip_serializing_if = "str::is_empty")] reason: &'a str, state: &'a str, } struct Projection<'a> { index: &'a ObligationIndex, - outcomes: &'a BTreeMap, + outcomes: &'a BTreeMap, prose: &'a BTreeMap, manifest_sha256: &'a str, } @@ -1031,13 +1072,14 @@ impl Projection<'_> { let mut first = true; for (row_id, slot) in &self.index.slots { - let Some(outcome) = self.outcomes.get(row_id) else { + let Some(settled) = self + .outcomes + .get(row_id) + .and_then(|recorded| settled_state(slot, recorded)) + else { continue; }; - if outcome.rejection.is_some() { - continue; - } - let Some(state) = ledger_state(outcome.state) else { + let Some(state) = ledger_state(settled) else { continue; }; let empty = LedgerProse { @@ -1046,7 +1088,7 @@ impl Projection<'_> { reason: String::new(), matcher: String::new(), }; - let prose = if outcome.state.is_pass() { + let prose = if settled.is_pass() { &empty } else { let prose = self.prose.get(row_id).ok_or_else(|| { @@ -1061,7 +1103,6 @@ impl Projection<'_> { // One row is live at a time: encoded, written, and dropped. let row = LedgerRowOut { - case: &slot.identifier, catalog: &slot.catalog, citation: &prose.citation, id: row_id, @@ -1192,12 +1233,15 @@ mod tests { use crate::{ evidence::{PublishMode, ToolchainPin, WorkingDirectoryPolicy, publish_evidence}, shard::{ExecutionMode, ShardIdentity, ShardSpec}, + suite::completion::{SuiteRunner, logical_obligation, materialize_obligations}, }; const DIGEST: &str = "a1b2c3d4e5f60718293a4b5c6d7e8f90a1b2c3d4e5f60718293a4b5c6d7e8f90"; const OTHER: &str = "0f1e2d3c4b5a69788796a5b4c3d2e1f00f1e2d3c4b5a69788796a5b4c3d2e1f0"; const CATALOG: &str = "typescript-7.0.2"; - const PLATFORM: &str = "x86_64-unknown-linux-gnu"; + fn platform() -> String { + bamts_platform::host_triple() + } fn scratch(name: &str) -> PathBuf { let nonce = TEMP_NONCE.fetch_add(1, Ordering::Relaxed); let root = @@ -1219,15 +1263,15 @@ mod tests { ObligationIndex::from_row_ids(&row_ids).expect("index") } + /// The key a lane really writes, so a fixture cannot drift from a receipt. + fn lane_key(catalog: &str, identity: &str, mode: ExecutionMode) -> ObligationKey { + logical_obligation(catalog, identity, mode, &platform()) + .expect("logical obligation") + .key + } + fn key_for(identity: &str) -> ObligationKey { - ObligationKey::new( - CATALOG, - identity, - "default", - ExecutionMode::Interpreter, - PLATFORM, - ) - .expect("obligation key") + lane_key(CATALOG, identity, ExecutionMode::Aot) } fn toolchain_pin() -> ToolchainPin { ToolchainPin::new("rustc-1.97.1", DIGEST).expect("toolchain pin") @@ -1241,8 +1285,8 @@ mod tests { BTreeMap::from([(CATALOG.to_owned(), binding())]) } - const CATALOG_A: &str = "catalog-a"; - const CATALOG_B: &str = "catalog-b"; + const CATALOG_A: &str = "typescript-a"; + const CATALOG_B: &str = "typescript-b"; const THIRD: &str = "1111111111111111111111111111111111111111111111111111111111111111"; fn catalog_identity(catalog: &str, index: usize) -> String { @@ -1259,14 +1303,7 @@ mod tests { fn pass_row_for_catalog(catalog: &str, identity: &str) -> EvidenceRow { EvidenceRow::new( - ObligationKey::new( - catalog, - identity, - "default", - ExecutionMode::Interpreter, - PLATFORM, - ) - .expect("catalog key"), + lane_key(catalog, identity, ExecutionMode::Aot), vec!["bamts".to_owned()], WorkingDirectoryPolicy::RepositoryRoot, BTreeSet::from(["diagnostics".to_owned()]), @@ -1458,10 +1495,10 @@ mod tests { report.rejections ); assert!( - report - .rejections - .iter() - .any(|rejection| rejection.reason == RejectionReason::MissingReceipt), + report.rejections.iter().any(|rejection| rejection.reason + == RejectionReason::MissingReceipt { + mode: ExecutionMode::Aot + }), "the declared obligation is still unproven" ); @@ -1615,17 +1652,22 @@ mod tests { // Mode, platform, and normalization are bound by the obligation key and // the canonical environment: a receipt naming another mode or platform // projects to no declared obligation. - for key in [ - ObligationKey::new(CATALOG, &declared, "default", ExecutionMode::Jit, PLATFORM) - .expect("mode"), - ObligationKey::new( - CATALOG, - &declared, - "default", - ExecutionMode::Interpreter, - "aarch64-unknown-linux-gnu", - ) - .expect("platform"), + for (field, key) in [ + ( + "execution_mode", + lane_key(CATALOG, &declared, ExecutionMode::Jit), + ), + ( + "platform", + logical_obligation( + CATALOG, + &declared, + ExecutionMode::Aot, + &format!("{}-foreign", platform()), + ) + .expect("platform") + .key, + ), ] { let index = index_of(std::slice::from_ref(&declared)); let row = EvidenceRow::new( @@ -1644,7 +1686,14 @@ mod tests { let report = project(&root, &index, &[path], RebuildMode::Write).expect("coordinate"); // The obligation key carries mode and platform, so a divergent run // coordinate cannot satisfy the declared obligation. - assert!(!report.is_clean()); + assert!( + report.rejections.iter().any(|rejection| rejection.reason + == RejectionReason::StaleBinding { + field: field.to_owned() + }), + "{field}: {:?}", + report.rejections + ); } assert!(!root.join(LEDGER_PATH).exists()); @@ -1931,14 +1980,7 @@ mod tests { .iter() .map(|identity| { EvidenceRow::new( - ObligationKey::new( - CATALOG, - identity, - declared_configuration(identity), - ExecutionMode::Interpreter, - PLATFORM, - ) - .expect("key"), + key_for(identity), vec!["bamts".to_owned()], WorkingDirectoryPolicy::RepositoryRoot, BTreeSet::from(["diagnostics".to_owned()]), @@ -1963,17 +2005,6 @@ mod tests { // Bounded memory: exactly one ledger row is live at any time. assert_eq!(report.max_live_rows, 1); - // The published artifact must stay under GitHub's 100 MiB blob limit at - // the manifest's full logical size. - const LOGICAL_IDENTIFIERS: u64 = 248_802; - let per_row = report.bytes / 2_000; - let projected = per_row * LOGICAL_IDENTIFIERS; - assert!( - projected < MAX_LEDGER_BYTES, - "projected {projected} bytes for {LOGICAL_IDENTIFIERS} rows at {per_row} bytes/row \ - exceeds {MAX_LEDGER_BYTES}" - ); - // Compact encoding: no per-row pretty-printing reaches the artifact. let bytes = fs::read(&report.ledger_path).expect("published"); let text = String::from_utf8(bytes).expect("utf8"); @@ -1983,6 +2014,399 @@ mod tests { let _ = fs::remove_dir_all(&root); } + const TEST262_CASE: &str = "test262/test262/test/built-ins/Array/case.js#default#runtime"; + + fn test262_index() -> ObligationIndex { + multi_index(&[("test262", TEST262_CASE)]) + } + + fn test262_row(mode: ExecutionMode, state: TerminalState) -> EvidenceRow { + let pass = state == TerminalState::Pass; + EvidenceRow::new( + lane_key("test262", TEST262_CASE, mode), + vec!["bamts".to_owned()], + WorkingDirectoryPolicy::RepositoryRoot, + BTreeSet::from(["runtime".to_owned()]), + if pass { + BTreeMap::from([("runtime".to_owned(), DIGEST.to_owned())]) + } else { + BTreeMap::new() + }, + state, + 1, + "receipt", + ) + .expect("test262 row") + } + + /// One receipt per row, the way one lane writes one receipt per mode. + fn test262_receipts(root: &Path, rows: Vec) -> Vec { + rows.into_iter() + .enumerate() + .map(|(position, row)| { + let path = root.join(format!("test262-{position}.jsonl")); + write_receipt(&path, &[row], binding()); + path + }) + .collect() + } + + fn project_test262(root: &Path, receipts: &[PathBuf]) -> RebuildReport { + let expected = BTreeMap::from([("test262".to_owned(), binding())]); + project_with_bindings( + root, + &test262_index(), + receipts, + &expected, + RebuildMode::Write, + ) + .expect("test262 projection") + } + + #[test] + fn a_compiler_receipt_row_is_admitted() { + let root = scratch("compiler-row"); + let declared = identity(0, "diagnostics"); + let index = index_of(std::slice::from_ref(&declared)); + let lanes = materialize_obligations( + CATALOG, + std::slice::from_ref(&declared), + SuiteRunner::Compiler, + &platform(), + ) + .expect("lane keys"); + assert_eq!(lanes[0].key, key_for(&declared)); + + let receipt = root.join("compiler.jsonl"); + write_receipt(&receipt, &[pass_row(&declared, "diagnostics")], binding()); + let report = project(&root, &index, &[receipt], RebuildMode::Write).expect("rebuild"); + assert!(report.is_clean(), "{:?}", report.rejections); + let ledger: serde_json::Value = + serde_json::from_slice(&fs::read(&report.ledger_path).expect("ledger")).expect("json"); + assert_eq!( + ledger["rows"][0], + serde_json::json!({ + "catalog": CATALOG, + "id": format!("{CATALOG}:{declared}"), + "state": "PASS", + }), + "a PASS row carries nothing the id does not already say" + ); + let _ = fs::remove_dir_all(&root); + } + + #[test] + fn a_non_pass_row_keeps_its_policy_prose() { + let root = scratch("non-pass-row"); + let declared = identity(0, "diagnostics"); + let index = index_of(std::slice::from_ref(&declared)); + let receipt = root.join("blocked.jsonl"); + write_receipt( + &receipt, + &[row_with( + &declared, + TerminalState::ExternalBlocked, + &["diagnostics"], + &[], + )], + binding(), + ); + let row_id = format!("{CATALOG}:{declared}"); + let prose = BTreeMap::from([( + row_id.clone(), + LedgerProse { + citation: "citation".to_owned(), + owner: "owner".to_owned(), + reason: "reason".to_owned(), + matcher: "matcher".to_owned(), + }, + )]); + let report = rebuild_projection( + &root, + &index, + DIGEST, + &[receipt], + RebuildMode::Write, + &expected_bindings(), + &prose, + ) + .expect("rebuild"); + assert!(report.is_clean(), "{:?}", report.rejections); + let ledger: serde_json::Value = + serde_json::from_slice(&fs::read(&report.ledger_path).expect("ledger")).expect("json"); + assert_eq!( + ledger["rows"][0], + serde_json::json!({ + "catalog": CATALOG, + "citation": "citation", + "id": row_id, + "matcher": "matcher", + "owner": "owner", + "reason": "reason", + "state": "EXTERNAL_BLOCKED", + }) + ); + let _ = fs::remove_dir_all(&root); + } + + #[test] + fn a_test262_case_is_complete_only_in_all_three_modes() { + let root = scratch("three-modes"); + let partial = test262_receipts( + &root, + vec![ + test262_row(ExecutionMode::Interpreter, TerminalState::Pass), + test262_row(ExecutionMode::Jit, TerminalState::Pass), + ], + ); + let report = project_test262(&root, &partial); + assert_eq!( + report.rejections, + [RebuildRejection { + row_id: format!("test262:{TEST262_CASE}"), + reason: RejectionReason::MissingReceipt { + mode: ExecutionMode::Aot + }, + }] + ); + assert!( + !root.join(LEDGER_PATH).exists(), + "a JIT row without an AOT row leaves the case incomplete" + ); + + let all = test262_receipts( + &root, + ExecutionMode::ALL + .into_iter() + .map(|mode| test262_row(mode, TerminalState::Pass)) + .collect(), + ); + let report = project_test262(&root, &all); + assert!(report.is_clean(), "{:?}", report.rejections); + assert_eq!(report.receipts_consumed, 3); + let ledger: serde_json::Value = + serde_json::from_slice(&fs::read(&report.ledger_path).expect("ledger")).expect("json"); + assert_eq!(ledger["rows"].as_array().expect("rows").len(), 1); + assert_eq!(ledger["rows"][0]["state"], "PASS"); + let _ = fs::remove_dir_all(&root); + } + + #[test] + fn a_row_settles_only_when_every_required_mode_has_reported() { + let index = test262_index(); + let slot = index.slots.values().next().expect("one slot"); + let pass = Outcome { + state: TerminalState::Pass, + rejection: None, + }; + let mut recorded = ModeOutcomes::new(); + for mode in ExecutionMode::ALL { + assert_eq!(settled_state(slot, &recorded), None); + recorded.insert(mode, pass.clone()); + } + assert_eq!(settled_state(slot, &recorded), Some(TerminalState::Pass)); + + recorded.insert( + ExecutionMode::Jit, + Outcome { + state: TerminalState::Pass, + rejection: Some(RejectionReason::DuplicateReceipt), + }, + ); + assert_eq!(settled_state(slot, &recorded), None); + } + + #[test] + fn one_failing_mode_keeps_a_test262_case_out_of_the_ledger() { + let root = scratch("one-mode-fails"); + let receipts = test262_receipts( + &root, + vec![ + test262_row(ExecutionMode::Interpreter, TerminalState::Pass), + test262_row(ExecutionMode::Jit, TerminalState::Pass), + test262_row(ExecutionMode::Aot, TerminalState::BlockingFail), + ], + ); + let report = project_test262(&root, &receipts); + assert_eq!(report.rejections.len(), 1, "{:?}", report.rejections); + assert!(matches!( + report.rejections[0].reason, + RejectionReason::BlockingState { .. } + )); + assert!(!root.join(LEDGER_PATH).exists()); + let _ = fs::remove_dir_all(&root); + } + + #[test] + fn modes_that_disagree_do_not_settle_a_row() { + let root = scratch("disagreement"); + let receipts = test262_receipts( + &root, + vec![ + test262_row(ExecutionMode::Interpreter, TerminalState::Pass), + test262_row(ExecutionMode::Jit, TerminalState::Pass), + test262_row( + ExecutionMode::Aot, + TerminalState::InapplicableOutOfScopeHostFeature, + ), + ], + ); + let row_id = format!("test262:{TEST262_CASE}"); + let prose = BTreeMap::from([( + row_id.clone(), + LedgerProse { + citation: "citation".to_owned(), + owner: "owner".to_owned(), + reason: "reason".to_owned(), + matcher: "matcher".to_owned(), + }, + )]); + let report = rebuild_projection( + &root, + &test262_index(), + DIGEST, + &receipts, + RebuildMode::Write, + &BTreeMap::from([("test262".to_owned(), binding())]), + &prose, + ) + .expect("disagreement"); + assert_eq!(report.rejections.len(), 1, "{:?}", report.rejections); + assert_eq!(report.rejections[0].row_id, row_id); + assert!(matches!( + report.rejections[0].reason, + RejectionReason::ModeDisagreement { .. } + )); + assert!(!root.join(LEDGER_PATH).exists()); + let _ = fs::remove_dir_all(&root); + } + + #[test] + fn a_mode_is_a_duplicate_only_of_itself() { + let root = scratch("mode-duplicate"); + let receipts = test262_receipts( + &root, + vec![ + test262_row(ExecutionMode::Interpreter, TerminalState::Pass), + test262_row(ExecutionMode::Interpreter, TerminalState::Pass), + test262_row(ExecutionMode::Jit, TerminalState::Pass), + test262_row(ExecutionMode::Aot, TerminalState::Pass), + ], + ); + let report = project_test262(&root, &receipts); + assert_eq!(report.rejections.len(), 1, "{:?}", report.rejections); + assert_eq!( + report.rejections[0].reason, + RejectionReason::DuplicateReceipt + ); + let _ = fs::remove_dir_all(&root); + } + + #[test] + fn a_receipt_in_a_mode_the_catalog_does_not_run_cannot_hide_in_a_complete_row() { + let root = scratch("extra-mode"); + let declared = identity(0, "diagnostics"); + let index = index_of(std::slice::from_ref(&declared)); + let proper = root.join("proper.jsonl"); + write_receipt(&proper, &[pass_row(&declared, "diagnostics")], binding()); + let extra = root.join("extra.jsonl"); + let row = EvidenceRow::new( + lane_key(CATALOG, &declared, ExecutionMode::Jit), + vec!["bamts".to_owned()], + WorkingDirectoryPolicy::RepositoryRoot, + BTreeSet::from(["diagnostics".to_owned()]), + BTreeMap::from([("diagnostics".to_owned(), DIGEST.to_owned())]), + TerminalState::Pass, + 1, + "receipt", + ) + .expect("row"); + write_receipt(&extra, &[row], binding()); + let report = project(&root, &index, &[proper, extra], RebuildMode::Write).expect("rebuild"); + assert_eq!( + report + .rejections + .iter() + .map(|r| &r.reason) + .collect::>(), + [&RejectionReason::StaleBinding { + field: "execution_mode".to_owned() + }] + ); + assert!(!root.join(LEDGER_PATH).exists()); + let _ = fs::remove_dir_all(&root); + } + + /// The obligation universe of the committed manifest. + fn committed_index() -> ObligationIndex { + let root = Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."); + let path = root.join(MANIFEST_PATH); + let bytes = read_bytes(&path).expect("manifest"); + let manifest: VerificationManifest = parse_json(&path, &bytes).expect("manifest json"); + let row_ids: BTreeSet = manifest + .catalogs + .iter() + .flat_map(|catalog| { + catalog + .identifiers + .iter() + .map(|identifier| format!("{}:{identifier}", catalog.id)) + }) + .collect(); + let index = ObligationIndex::from_row_ids(&row_ids).expect("every identifier has a key"); + assert_eq!(index.len(), row_ids.len()); + index + } + + #[test] + fn a_full_ledger_of_the_committed_manifest_stays_under_ninety_megabytes() { + const TARGET_BYTES: u64 = 90_000_000; + let index = committed_index(); + let outcomes: BTreeMap = index + .slots + .iter() + .map(|(row_id, slot)| { + let recorded = slot.modes.iter().map(|mode| { + let outcome = Outcome { + state: TerminalState::Pass, + rejection: None, + }; + (*mode, outcome) + }); + (row_id.clone(), recorded.collect()) + }) + .collect(); + let prose = BTreeMap::new(); + let measured = Projection { + index: &index, + outcomes: &outcomes, + prose: &prose, + manifest_sha256: DIGEST, + } + .measure() + .expect("full ledger"); + assert!( + measured.bytes < TARGET_BYTES, + "a full ledger is {} bytes", + measured.bytes + ); + assert_eq!(measured.max_live_rows, 1); + } + + #[test] + fn every_manifest_identifier_is_reached_from_its_own_lane_key() { + let index = committed_index(); + for (row_id, slot) in &index.slots { + for mode in &slot.modes { + let obligation = + logical_obligation(&slot.catalog, &slot.identifier, *mode, &platform()) + .expect("lane key"); + let (reached, _) = index.resolve(&obligation.key).expect("key resolves"); + assert_eq!(reached, row_id); + } + } + } + #[test] fn publication_is_atomic() { let root = scratch("atomic"); diff --git a/crates/bamts-verification/src/source_fetch.rs b/crates/bamts-verification/src/source_fetch.rs index ad3f4742..c679c205 100644 --- a/crates/bamts-verification/src/source_fetch.rs +++ b/crates/bamts-verification/src/source_fetch.rs @@ -19,6 +19,7 @@ use sha2::{Digest, Sha256, Sha512}; use crate::{ ErrorCode, Result, VerificationError, corpus::{OracleLimits, normalized_env, run_process}, + fsync, schema::{SourcePin, base64_value, load_sources, required_source}, }; @@ -594,6 +595,9 @@ fn select_payload(extracted: &Path) -> Result { .map_err(|error| io_error(extracted, error))? .collect::, _>>() .map_err(|error| io_error(extracted, error))?; + // macOS can write this into any new directory, which must not hide the + // archive's single root and change the layout of the whole tree. + entries.retain(|entry| entry.file_name() != ".DS_Store"); if entries.is_empty() { return Err(VerificationError::new( ErrorCode::Schema, @@ -619,18 +623,14 @@ fn sync_tree(root: &Path) -> Result<()> { if metadata.is_dir() && !metadata.file_type().is_symlink() { sync_tree(&entry.path())?; } else if metadata.is_file() { - File::open(entry.path()) - .and_then(|file| file.sync_all()) - .map_err(|error| io_error(&entry.path(), error))?; + fsync::sync_file(&entry.path()).map_err(|error| io_error(&entry.path(), error))?; } } sync_directory(root) } fn sync_directory(path: &Path) -> Result<()> { - File::open(path) - .and_then(|directory| directory.sync_all()) - .map_err(|error| io_error(path, error)) + fsync::sync_directory(path).map_err(|error| io_error(path, error)) } fn ensure_parent(parent: &Path) -> Result { @@ -850,6 +850,16 @@ mod tests { assert_eq!(backend.fetches.get(), 1); } + #[test] + fn a_finder_file_does_not_hide_the_archive_root() { + let root = std::env::temp_dir().join(format!("select-payload-{}", std::process::id())); + let payload = root.join("source-1234"); + fs::create_dir_all(&payload).expect("payload"); + fs::write(root.join(".DS_Store"), b"finder").expect("finder file"); + assert_eq!(select_payload(&root).expect("payload"), payload); + fs::remove_dir_all(&root).expect("remove root"); + } + #[test] fn command_backend_argv_is_accepted_by_tools() { let scratch = Scratch::new("commands"); diff --git a/crates/bamts-verification/src/suite.rs b/crates/bamts-verification/src/suite.rs index 8854473c..72450345 100644 --- a/crates/bamts-verification/src/suite.rs +++ b/crates/bamts-verification/src/suite.rs @@ -17,6 +17,7 @@ use std::{ use bamts_compiler::diagnostic::Diagnostic; use bamts_compiler::parser::parse; use bamts_compiler::program::ProgramLoader; +use bamts_compiler::project::resolution_trace::display_path; use bamts_compiler::project::{ProjectConfig, ProjectRoot}; use bamts_compiler::scanner::scan; use bamts_compiler::source::{ScriptKind, SourceId, SourceText}; @@ -290,7 +291,6 @@ pub struct SuiteSnapshot { #[derive(Debug, Clone, PartialEq, Eq)] pub struct CompilerSnapshot { pub root: PathBuf, - pub metadata_digest: String, pub index: SuiteIndex, } @@ -2081,7 +2081,6 @@ pub fn load_bound_compiler_snapshot( validate_suite_index(&index)?; Ok(CompilerSnapshot { root: snapshot_root, - metadata_digest: sha256_hex_concat(&[&pin_bytes, &index_bytes, &ledger_bytes]), index, }) } @@ -2608,7 +2607,7 @@ pub fn materialize_from_extracted( }) } -const COMPILER_CATALOG: &str = "typescript-7.0.2"; +pub(crate) const COMPILER_CATALOG: &str = "typescript-7.0.2"; /// Exact compiler-lane observation of one catalog key through BamTS check cells. #[derive(Debug, Clone, PartialEq, Eq)] @@ -3172,7 +3171,7 @@ fn produce_project_trace( if !rendered.is_empty() { rendered.push('\n'); } - let physical_root = loaded.root().path().to_string_lossy().replace('\\', "/"); + let physical_root = display_path(loaded.root().path()); let produced = rendered .replace(&physical_root, fixture.project_root.trim_end_matches('/')) .into_bytes(); diff --git a/crates/bamts-verification/src/suite/completion.rs b/crates/bamts-verification/src/suite/completion.rs index 76411b8c..de170a83 100644 --- a/crates/bamts-verification/src/suite/completion.rs +++ b/crates/bamts-verification/src/suite/completion.rs @@ -42,14 +42,16 @@ use crate::{ EvidenceHeader, EvidenceReader, EvidenceRow, EvidenceWriter, ExecutionBinding, PublishMode, RunBinding, TerminalState, ToolchainPin, WorkingDirectoryPolicy, merge_shards, }, + fsync, lane::{ LaneBinding, LaneExecutor, LaneOutcome, LaneProcessResult, LaneRequest, LaneResponse, - ProcessExecutor, ProcessObservation, derive_row, + ProcessExecutor, ProcessObservation, ServingExecutor, derive_row, }, schema, shard::{ObligationKey, ShardIdentity, ShardSpec, validate_catalog}, suite::{DEFAULT_SNAPSHOT_REL, verify_snapshot}, - toolchain_schema::load_target_cells, + sweep_report::{ReportRow, read_receipt}, + toolchain_schema::load_committed_cells, }; /// Per-obligation wall-clock bound handed to the lane. @@ -232,19 +234,6 @@ fn runner_for_mode(catalog: &str, mode: crate::shard::ExecutionMode) -> Option String { - let arch = std::env::consts::ARCH; - match std::env::consts::OS { - "linux" => format!("{arch}-unknown-linux-gnu"), - "macos" => format!("{arch}-apple-darwin"), - "windows" => format!("{arch}-pc-windows-msvc"), - os => format!("{arch}-unknown-{os}"), - } -} - /// Selected catalog plus the digests that name the current manifest image. #[derive(Debug, Clone, PartialEq, Eq)] pub struct CatalogManifest { @@ -360,24 +349,10 @@ pub fn materialize_obligations( )); } let mode = runner.execution_mode(); - let mut obligations = Vec::with_capacity(identifiers.len()); - let authority_prefix = format!("{catalog}/"); - for identifier in identifiers { - let (case, configuration, observables) = parse_logical_identifier(catalog, identifier)?; - // TS/test262 cases are `runner/path` tokens after the authority - // prefix; every other family keeps its own identity segments. - let key = ObligationKey::new( - catalog, - case.strip_prefix(authority_prefix.as_str()) - .map(str::to_owned) - .unwrap_or(case.clone()), - configuration, - mode, - platform, - )?; - let _ = authority_prefix; - obligations.push(LogicalObligation { key, observables }); - } + let mut obligations = identifiers + .iter() + .map(|identifier| logical_obligation(catalog, identifier, mode, platform)) + .collect::>>()?; obligations.sort_by(|left, right| left.key.cmp(&right.key)); validate_catalog( &obligations @@ -388,6 +363,34 @@ pub fn materialize_obligations( Ok(obligations) } +/// The lane key that proves one manifest identifier in `mode` on `platform`, +/// with its declared observables. This is the one mapping between the two +/// names of an obligation: receipts carry the key, the manifest the identifier. +pub(crate) fn logical_obligation( + catalog: &str, + identifier: &str, + mode: crate::shard::ExecutionMode, + platform: &str, +) -> Result { + let (case, configuration, observables) = parse_logical_identifier(catalog, identifier)?; + // TS/test262 cases are `runner/path` tokens after the authority + // prefix; every other family keeps its own identity segments. + let case = case + .strip_prefix(&format!("{catalog}/")) + .map_or_else(|| case.clone(), str::to_owned); + let key = ObligationKey::new(catalog, case, configuration, mode, platform)?; + Ok(LogicalObligation { key, observables }) +} + +/// The modes every identifier of `catalog` must be proven in: one per runner +/// the catalog declares. +pub(crate) fn required_modes(catalog: &str) -> Result> { + Ok(allowed_runners(catalog)? + .iter() + .map(|runner| runner.execution_mode()) + .collect()) +} + /// Splits one logical identifier into〈case, configuration, observables〉. /// /// The returned `case` still carries the authority prefix for TS/test262 @@ -425,18 +428,14 @@ fn parse_logical_identifier( BTreeSet::from([(*observable).to_owned()]), )); } - if identifier.contains("::") { - let segments: Vec<&str> = identifier.split("::").collect(); - let [locator, name] = segments.as_slice() else { - return Err(reject()); - }; + if let Some((locator, name)) = identifier.split_once("::") { if locator.is_empty() || name.is_empty() { return Err(reject()); } return Ok(( - (*locator).to_owned(), - (*name).to_owned(), - BTreeSet::from([(*name).to_owned()]), + locator.to_owned(), + name.to_owned(), + BTreeSet::from([name.to_owned()]), )); } if identifier.is_empty() { @@ -516,7 +515,7 @@ fn load_obligation_classifications( /// unavailable adapter. An unavailable adapter is evidence, not a suite abort. enum SuiteAdapter { TargetCells(BTreeMap), - External(ProcessExecutor), + External(Box), Missing(String), } @@ -547,7 +546,7 @@ fn lane_program(root: &Path, runner: SuiteRunner) -> Option { impl SuiteAdapter { fn build(root: &Path, catalog: &str, runner: SuiteRunner) -> Result { if runner == SuiteRunner::Aot && catalog == "target-cells" { - return Ok(Self::TargetCells(load_target_cells(root)?)); + return Ok(Self::TargetCells(load_committed_cells(root)?)); } let variable = lane_adapter_variable(runner); @@ -570,10 +569,22 @@ impl SuiteAdapter { attempted.display() ))); }; - Ok(Self::External( - ProcessExecutor::new(program, Vec::new(), root, root.join("target/suite-lanes")) - .with_max_output_bytes(PROBE_OUTPUT_BYTES), - )) + let work_dir = root.join("target/suite-lanes"); + // The compiler worker serves many requests from one process, so it + // authenticates and parses the snapshot once instead of once per case. + // The other lane workers answer one request per process. + let executor: Box = if runner == SuiteRunner::Compiler { + Box::new( + ServingExecutor::new(program, Vec::new(), root, work_dir) + .with_max_output_bytes(PROBE_OUTPUT_BYTES), + ) + } else { + Box::new( + ProcessExecutor::new(program, Vec::new(), root, work_dir) + .with_max_output_bytes(PROBE_OUTPUT_BYTES), + ) + }; + Ok(Self::External(executor)) } /// Executes one obligation through an internal path. Result is a worker @@ -581,16 +592,14 @@ impl SuiteAdapter { fn evaluate(&self, request: &LaneRequest) -> Result { match self { Self::TargetCells(cells) => { - let record = cells.get(request.key().case()).ok_or_else(|| { - VerificationError::new( - ErrorCode::Schema, - format!( - "no target-cell record for `{}` obligation `{}`", - request.key().case(), - request.key() + let Some(record) = cells.get(request.key().case()) else { + return Ok(LaneOutcome::BlockingFail { + detail: format!( + "target cell `{}` has not been measured; no committed record", + request.key().case() ), - ) - })?; + }); + }; let obligation = record .obligations() .get(request.key().configuration()) @@ -749,12 +758,12 @@ fn classified_outcome(state: NonPassState, key: &ObligationKey) -> LaneOutcome { /// /// Abort error (manifest, adapter, digests, writer) leaves no receipt /// behind; per-obligation blocking outcomes never abort the loop. -pub fn run_suite(root: &Path, request: &SuiteRunRequest) -> Result { - let runner = resolve_runner(&request.catalog, &request.runner)?; - let platform = if request.platform.is_empty() { - default_platform() +/// The requested platform, or the host's when none is given. +fn canonical_platform(requested: &str) -> Result { + let platform = if requested.is_empty() { + bamts_platform::host_triple() } else { - request.platform.clone() + requested.to_owned() }; if platform.trim() != platform || platform.is_empty() { return Err(VerificationError::new( @@ -762,6 +771,23 @@ pub fn run_suite(root: &Path, request: &SuiteRunRequest) -> Result "suite platform must be a nonempty canonical token", )); } + Ok(platform) +} + +/// A fresh run identity; the compiler lane also binds the verified snapshot. +fn fresh_lane_binding(root: &Path, runner: SuiteRunner) -> Result { + let run = LaneBinding::fresh()?; + if runner == SuiteRunner::Compiler { + let verified = verify_snapshot(&root.join(DEFAULT_SNAPSHOT_REL))?; + verified.bind_compiler_lane(run) + } else { + Ok(run) + } +} + +pub fn run_suite(root: &Path, request: &SuiteRunRequest) -> Result { + let runner = resolve_runner(&request.catalog, &request.runner)?; + let platform = canonical_platform(&request.platform)?; let catalog = load_catalog_manifest(root, &request.catalog)?; let obligations = materialize_obligations(&request.catalog, &catalog.identifiers, runner, &platform)?; @@ -777,15 +803,7 @@ pub fn run_suite(root: &Path, request: &SuiteRunRequest) -> Result let binding = current_run_binding(root, &request.catalog)?; let header = EvidenceHeader::new(shard.clone(), binding, request.execution.clone())?; let members: Vec = request.shard.member_indices(keys.len()).collect(); - let lane_binding = { - let run = LaneBinding::fresh()?; - if runner == SuiteRunner::Compiler { - let verified = verify_snapshot(&root.join(DEFAULT_SNAPSHOT_REL))?; - verified.bind_compiler_lane(run)? - } else { - run - } - }; + let lane_binding = fresh_lane_binding(root, runner)?; let executor = SuiteExecutor { adapter, @@ -833,10 +851,7 @@ pub fn run_suite(root: &Path, request: &SuiteRunRequest) -> Result writer.write_row(&row)?; } writer.finish()?; - File::open(&temp) - .map_err(|error| io_path(&temp, error))? - .sync_all() - .map_err(|error| io_path(&temp, error)) + fsync::sync_file(&temp).map_err(|error| io_path(&temp, error)) })(); match outcome { Ok(()) => { @@ -862,6 +877,133 @@ pub fn run_suite(root: &Path, request: &SuiteRunRequest) -> Result } } +/// Reruns the obligations of one failure family named by an earlier receipt. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct SuiteRecountRequest { + pub catalog: String, + pub runner: String, + pub baseline: PathBuf, + pub family: String, + /// How many passing rows to rerun beside the family. A family holds only + /// failing rows, so these are the rows a fix can regress. + pub guard: usize, +} + +/// The rerun rows before and after, in the same order: the family, then the +/// passing rows that guard against a regression. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct SuiteRecount { + pub before: Vec, + pub after: Vec, +} + +/// `count` of `rows`, spread evenly so a sample reaches every part of a catalog +/// and the same receipt always gives the same sample. +fn evenly_spaced(rows: Vec, count: usize) -> Vec { + let total = rows.len(); + if count >= total { + return rows; + } + rows.into_iter() + .enumerate() + .filter(|(index, _)| (index + 1) * count / total != index * count / total) + .map(|(_, row)| row) + .collect() +} + +/// Reruns the rows of `family` and a sample of the passing rows, on the +/// platform the baseline recorded. +/// +/// This answers one question for a repair loop: did the fix convert rows and +/// regress none. Only a full sweep proves none regressed; the sample catches a +/// broad regression between sweeps. It writes no receipt. Every state comes from +/// the same lane executor and `derive_row` as a sweep, so a rerun cannot mint a +/// `PASS` that a sweep could not. +pub fn recount_suite(root: &Path, request: &SuiteRecountRequest) -> Result { + let (mut before, others): (Vec, Vec) = read_receipt(&request.baseline)? + .into_iter() + .partition(|row| row.family().as_deref() == Some(request.family.as_str())); + let Some(first) = before.first() else { + return Err(VerificationError::new( + ErrorCode::Usage, + format!( + "{}: no row belongs to family `{}`", + request.baseline.display(), + request.family + ), + )); + }; + let platform = canonical_platform(first.key().platform())?; + if before.iter().any(|row| row.key().platform() != platform) { + return Err(VerificationError::new( + ErrorCode::Usage, + format!("family `{}` spans more than one platform", request.family), + )); + } + let passing = others + .into_iter() + .filter(|row| row.state().is_pass() && row.key().platform() == platform) + .collect(); + before.extend(evenly_spaced(passing, request.guard)); + let runner = resolve_runner(&request.catalog, &request.runner)?; + let catalog = load_catalog_manifest(root, &request.catalog)?; + let obligations = + materialize_obligations(&request.catalog, &catalog.identifiers, runner, &platform)?; + let classifications = load_obligation_classifications( + root, + &request.catalog, + &catalog.identifiers, + &obligations, + )?; + let adapter = SuiteAdapter::build(root, &request.catalog, runner)?; + let lane_binding = fresh_lane_binding(root, runner)?; + let mut executor = SuiteExecutor { + adapter, + classifications, + }; + let mut after = Vec::with_capacity(before.len()); + for (ordinal, row) in before.iter().enumerate() { + // The catalog is strictly increasing by key, so a search finds the row. + let Ok(index) = obligations.binary_search_by(|entry| entry.key.cmp(row.key())) else { + return Err(VerificationError::new( + ErrorCode::SetMismatch, + format!( + "the baseline names `{}`, which the `{}` catalog does not contain", + row.key(), + request.catalog + ), + )); + }; + let obligation = &obligations[index]; + let lane_request = LaneRequest::new( + lane_binding.clone(), + (ordinal + 1) as u64, + obligation.key.clone(), + obligation.observables.clone(), + vec![ + "bamts-verification::suite::worker".to_owned(), + obligation.key.to_string(), + ], + WorkingDirectoryPolicy::RepositoryRoot, + CASE_TIMEOUT_MS, + )?; + let process = match executor.run(&lane_request) { + Ok(process) => process, + Err(error) => LaneProcessResult { + observation: ProcessObservation::Exited { code: 1 }, + response_body: None, + duration_ms: 0, + detail: error.to_string(), + }, + }; + after.push(ReportRow::from_evidence(&derive_row( + &lane_request, + process, + )?)); + } + Ok(SuiteRecount { before, after }) +} + /// Merges every shard document below `receipts_root` for the catalog. /// /// Discovery is bounded and rejects wrong catalog, mixed runner/platform, @@ -2256,6 +2398,71 @@ mod tests { ); } + #[test] + fn a_formal_name_may_itself_contain_the_separator() { + let (case, configuration, observables) = parse_logical_identifier( + "formal-redex", + "ecmascript/semantics.rkt::rule::property_get", + ) + .expect("the manifest declares identifiers of this shape"); + assert_eq!(case, "ecmascript/semantics.rkt"); + assert_eq!(configuration, "rule::property_get"); + assert_eq!( + observables, + BTreeSet::from(["rule::property_get".to_owned()]) + ); + } + + #[test] + fn test262_is_proven_in_all_three_modes_and_other_catalogs_in_one() { + use crate::shard::ExecutionMode as Mode; + assert_eq!( + required_modes("test262").expect("modes"), + [Mode::Interpreter, Mode::Jit, Mode::Aot] + ); + for catalog in [ + "typescript-7.0.2", + "formal-redex", + "target-cells", + "benchmarks", + ] { + assert_eq!(required_modes(catalog).expect("modes"), [Mode::Aot]); + } + } + + #[test] + fn an_unmeasured_target_cell_is_a_row_not_a_lane_abort() { + let key = ObligationKey::new( + "target-cells", + "x86_64-pc-windows-msvc", + "target-object", + ExecutionMode::Aot, + "ubuntu-latest", + ) + .expect("key"); + let binding = LaneBinding::unbound( + "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa", + ) + .expect("binding"); + let request = LaneRequest::new( + binding, + 7, + key, + BTreeSet::from(["target-object".to_owned()]), + vec!["bamts-verification::suite::worker".to_owned()], + WorkingDirectoryPolicy::RepositoryRoot, + CASE_TIMEOUT_MS, + ) + .expect("request"); + let outcome = SuiteAdapter::TargetCells(BTreeMap::new()) + .evaluate(&request) + .expect("an unmeasured cell is an outcome"); + assert!(matches!( + outcome, + LaneOutcome::BlockingFail { detail } if detail.contains("has not been measured") + )); + } + #[test] fn suite_never_mints_pass_without_exact_observables() { fn key(case: &str) -> ObligationKey { @@ -2694,6 +2901,176 @@ mod tests { assert_eq!(rows, 2); } + /// A baseline from a run with no adapter, with the detail of the middle row + /// changed so the rows fall into two failure families. + fn two_family_baseline(identifiers: &[&str]) -> (Scratch, PathBuf, String, String) { + let (scratch, _) = manifest_root("recount", identifiers); + let baseline = scratch.root.join("baseline.jsonl"); + run_suite( + &scratch.root, + &SuiteRunRequest { + catalog: "benchmarks".to_owned(), + shard: ShardSpec::unsharded(), + receipt: baseline.clone(), + runner: "perf".to_owned(), + platform: "ubuntu-latest".to_owned(), + execution: ExecutionBinding::local_for_tests(), + }, + ) + .expect("baseline run"); + // Every benchmark names its own observable, which would give each row a + // facet, and so a family, of its own. Share one so rows can group. + let mut text = fs::read_to_string(&baseline).expect("baseline text"); + for identifier in identifiers { + text = text.replace( + &format!("\"observables\":[\"{identifier}\"]"), + "\"observables\":[\"perf\"]", + ); + } + let detail = text + .lines() + .find_map(|line| { + let value: serde_json::Value = serde_json::from_str(line).ok()?; + value["body"]["detail"].as_str().map(str::to_owned) + }) + .expect("a row detail"); + let first = text.find(&detail).expect("first detail"); + let second = first + + text[first + detail.len()..] + .find(&detail) + .expect("second detail") + + detail.len(); + let edited = format!( + "{}beta failure{}", + &text[..second], + &text[second + detail.len()..] + ); + fs::write(&baseline, edited).expect("edited baseline"); + let rows = crate::sweep_report::read_receipt(&baseline).expect("readable baseline"); + let family_of = |index: usize| rows[index].family().expect("failing row"); + let (alpha, beta) = (family_of(0), family_of(1)); + assert_ne!(alpha, beta); + assert_eq!(family_of(2), alpha); + (scratch, baseline, alpha, beta) + } + + fn recount_request(baseline: &Path, family: &str) -> SuiteRecountRequest { + SuiteRecountRequest { + catalog: "benchmarks".to_owned(), + runner: "perf".to_owned(), + baseline: baseline.to_owned(), + family: family.to_owned(), + guard: 0, + } + } + + #[test] + fn recount_reruns_only_the_rows_of_the_named_family() { + let (scratch, baseline, alpha, beta) = two_family_baseline(&["jit.a", "jit.b", "jit.c"]); + + let one = recount_suite(&scratch.root, &recount_request(&baseline, &beta)) + .expect("recount of the one-row family"); + assert_eq!(one.before.len(), 1); + assert_eq!(one.after.len(), 1); + assert_eq!(one.before[0].key(), one.after[0].key()); + assert!( + one.before[0].key().case().ends_with("jit.b"), + "{:?}", + one.before[0].key() + ); + assert_eq!(one.after[0].key().platform(), "ubuntu-latest"); + + let two = recount_suite(&scratch.root, &recount_request(&baseline, &alpha)) + .expect("recount of the two-row family"); + let cases: Vec<&str> = two.after.iter().map(|row| row.key().case()).collect(); + assert_eq!(cases.len(), 2, "{cases:?}"); + assert!( + cases[0].ends_with("jit.a") && cases[1].ends_with("jit.c"), + "{cases:?}" + ); + assert!(two.after.iter().all(|row| !row.state().is_pass())); + } + + #[test] + fn recount_reruns_passing_rows_so_a_regression_can_be_seen() { + let (scratch, baseline, _, beta) = two_family_baseline(&["jit.a", "jit.b", "jit.c"]); + // These lanes fail every row, so a row recorded as passing regresses. + let edited: Vec = fs::read_to_string(&baseline) + .expect("baseline text") + .lines() + .map(|line| { + let mut value: serde_json::Value = serde_json::from_str(line).expect("json line"); + let is_row = value["body"]["key"].to_string().contains("jit.c"); + if is_row { + value["body"]["state"] = serde_json::Value::from("PASS"); + value["body"]["artifacts"] = serde_json::json!({"perf": "0".repeat(64)}); + } + value.to_string() + }) + .collect(); + fs::write(&baseline, edited.join("\n") + "\n").expect("edited baseline"); + + let mut request = recount_request(&baseline, &beta); + let unguarded = recount_suite(&scratch.root, &request).expect("recount without a guard"); + let (_, regressions) = + crate::sweep_report::render_recount(&beta, &unguarded.before, &unguarded.after, 5); + assert_eq!(unguarded.after.len(), 1); + assert_eq!(regressions, 0); + + request.guard = 5; + let guarded = recount_suite(&scratch.root, &request).expect("recount with a guard"); + let (report, regressions) = + crate::sweep_report::render_recount(&beta, &guarded.before, &guarded.after, 5); + assert_eq!(guarded.after.len(), 2, "{report}"); + assert_eq!(regressions, 1, "{report}"); + } + + #[test] + fn an_even_sample_has_the_exact_size_and_reaches_both_halves() { + let rows: Vec = (0..10) + .map(|index| { + let key = ObligationKey::new( + "benchmarks", + format!("case-{index}"), + "default", + ExecutionMode::Interpreter, + "ubuntu-latest", + ) + .expect("key"); + ReportRow::new(key, "perf", TerminalState::Pass, "") + }) + .collect(); + for count in 0..=12 { + let sample = evenly_spaced(rows.clone(), count); + assert_eq!(sample.len(), count.min(10)); + } + let cases: Vec = evenly_spaced(rows, 2) + .iter() + .map(|row| row.key().case().to_owned()) + .collect(); + assert_eq!(cases, ["case-4", "case-9"]); + } + + #[test] + fn recount_of_an_unknown_family_is_a_usage_error() { + let (scratch, baseline, ..) = two_family_baseline(&["jit.a", "jit.b", "jit.c"]); + let error = recount_suite( + &scratch.root, + &recount_request(&baseline, "perf/no-such-family"), + ) + .expect_err("unknown family"); + assert_eq!(error.code(), ErrorCode::Usage); + } + + #[test] + fn recount_refuses_a_baseline_the_catalog_no_longer_contains() { + let (_scratch, baseline, _, beta) = two_family_baseline(&["jit.a", "jit.b", "jit.c"]); + let (smaller, _) = manifest_root("recount-smaller", &["jit.a", "jit.c"]); + let error = recount_suite(&smaller.root, &recount_request(&baseline, &beta)) + .expect_err("jit.b is gone from the catalog"); + assert_eq!(error.code(), ErrorCode::SetMismatch); + } + /// `fourslash/…` obligations are language-service authority cases: the /// executor routes them to `INAPPLICABLE_LANGUAGE_SERVICE` before any lane /// adapter runs, while compiler/conformance cases stay routable. diff --git a/crates/bamts-verification/src/sweep_report.rs b/crates/bamts-verification/src/sweep_report.rs new file mode 100644 index 00000000..53828577 --- /dev/null +++ b/crates/bamts-verification/src/sweep_report.rs @@ -0,0 +1,1484 @@ +//! Read-only reports over evidence receipts. +//! +//! A sweep leaves one receipt per catalog and runner. These reports answer what +//! a repair loop asks of it: how many rows sit in each state, which failures +//! share a cause, and what changed between two receipts. Nothing here writes a +//! receipt or decides a state. + +use std::{ + collections::{BTreeMap, BTreeSet}, + fmt::Write as _, + fs, io, + path::{Path, PathBuf}, + process, +}; + +use serde::{Deserialize, Serialize}; + +use crate::{ + ErrorCode, Result, VerificationError, + evidence::{EvidenceReader, EvidenceRow, TerminalState}, + shard::{ObligationKey, ShardSpec}, +}; + +/// Characters of a quoted span that still count toward a family. The rest of a +/// long span is usually a path or an identifier that differs per case. +const QUOTED_KEEP: usize = 48; +/// Longest family handle, so it stays usable as one command-line argument. +const SLUG_MAX: usize = 96; + +/// One receipt row reduced to what the reports compare. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ReportRow { + key: ObligationKey, + facet: String, + state: TerminalState, + detail: String, +} + +impl ReportRow { + #[must_use] + pub fn new( + key: ObligationKey, + facet: impl Into, + state: TerminalState, + detail: impl Into, + ) -> Self { + Self { + key, + facet: facet.into(), + state, + detail: detail.into(), + } + } + + /// The facet is the set of observables the obligation checks, joined by `+`. + #[must_use] + pub fn from_evidence(row: &EvidenceRow) -> Self { + let facet = row + .observables() + .iter() + .map(String::as_str) + .collect::>() + .join("+"); + Self::new(row.key().clone(), facet, row.state(), row.detail()) + } + + #[must_use] + pub fn key(&self) -> &ObligationKey { + &self.key + } + + #[must_use] + pub fn facet(&self) -> &str { + &self.facet + } + + #[must_use] + pub fn state(&self) -> TerminalState { + self.state + } + + /// The failure family of a row that blocks completion, or `None` for a row + /// that passes or is excluded by policy. + /// + /// A family is `/`, where the slug is the diagnostic text with + /// the parts that differ per case removed. Rows in one family failed the + /// same way. The handle is only as precise as the text a worker records: + /// when the text is coarse, the family is too. + #[must_use] + pub fn family(&self) -> Option { + blocks_completion(self.state).then(|| family_name(&self.facet, &self.detail)) + } +} + +/// States a scoped root does not accept. A policy exclusion is not one of them. +const fn blocks_completion(state: TerminalState) -> bool { + !matches!( + state, + TerminalState::Pass + | TerminalState::InapplicableLanguageService + | TerminalState::InapplicableOutOfScopeHostFeature + | TerminalState::InapplicableV8Internal + ) +} + +/// The handle for rows of one facet that fail with this diagnostic text. +#[must_use] +pub fn family_name(facet: &str, detail: &str) -> String { + let slug = slugify(&normalize_detail(detail)); + let slug = if slug.is_empty() { "no-detail" } else { &slug }; + format!("{facet}/{slug}") +} + +/// Drops what differs per case: the mode wrapper test262 workers add, backtick +/// spans, the tail of long quoted spans, absolute paths, and the value of every +/// number. +fn normalize_detail(detail: &str) -> String { + let detail = strip_mode_wrapper(detail); + let mut out = String::with_capacity(detail.len()); + let mut chars = detail.chars().peekable(); + while let Some(character) = chars.next() { + match character { + '`' => { + for inner in chars.by_ref() { + if inner == '`' { + break; + } + } + } + '"' => { + out.push('"'); + let mut kept = 0; + while let Some(inner) = chars.next() { + if inner == '\\' { + chars.next(); + } else if inner == '"' { + break; + } else if kept < QUOTED_KEEP { + out.push(inner); + kept += 1; + } + } + out.push('"'); + } + digit if digit.is_ascii_digit() => { + while chars.next_if(char::is_ascii_digit).is_some() {} + out.push('N'); + } + other => out.push(other), + } + } + out.split_whitespace() + .filter(|word| !is_absolute_path(word)) + .collect::>() + .join(" ") +} + +/// A scratch directory or checkout path names the run, not the failure. +fn is_absolute_path(word: &str) -> bool { + let bytes = word.as_bytes(); + word.starts_with('/') + || (bytes.len() > 2 + && bytes[0].is_ascii_alphabetic() + && bytes[1] == b':' + && matches!(bytes[2], b'\\' | b'/')) +} + +/// test262 workers prefix their text with `test262 mode failure: `. The +/// mode would split one failure into three, so it is not part of the family. +fn strip_mode_wrapper(detail: &str) -> &str { + const MARKER: &str = " mode failure: "; + detail + .strip_prefix("test262 ") + .and_then(|rest| { + let end = rest.find(MARKER)?; + (end <= QUOTED_KEEP).then(|| &rest[end + MARKER.len()..]) + }) + .unwrap_or(detail) +} + +fn slugify(text: &str) -> String { + let mut slug = String::new(); + for character in text.chars() { + if character.is_ascii_alphanumeric() { + slug.push(character.to_ascii_lowercase()); + } else if !slug.ends_with('-') && !slug.is_empty() { + slug.push('-'); + } + } + slug.truncate(SLUG_MAX); + slug.trim_end_matches('-').to_owned() +} + +/// Reads and validates a whole receipt: header, strictly ordered rows, footer. +/// +/// The reader returns `None` only once the footer has matched the rows read and +/// nothing follows it, so a truncated or inconsistent receipt is an error here. +pub fn read_receipt(path: &Path) -> Result> { + let mut reader = EvidenceReader::open(path)?; + let mut rows = Vec::new(); + while let Some(row) = reader.next_row()? { + rows.push(ReportRow::from_evidence(&row)); + } + Ok(rows) +} + +/// Percentage with one decimal, from integers so the output is exact. +fn percent(part: usize, whole: usize) -> String { + if whole == 0 { + return "0.0%".to_owned(); + } + let tenths = part * 1000 / whole; + format!("{}.{}%", tenths / 10, tenths % 10) +} + +/// Counts of the rows that fail, largest first, ties by name. +fn family_counts(rows: &[ReportRow]) -> Vec<(String, usize)> { + let mut counts: BTreeMap = BTreeMap::new(); + for family in rows.iter().filter_map(ReportRow::family) { + *counts.entry(family).or_default() += 1; + } + let mut ordered: Vec<(String, usize)> = counts.into_iter().collect(); + ordered.sort_by(|left, right| right.1.cmp(&left.1).then_with(|| left.0.cmp(&right.0))); + ordered +} + +/// Counts by state, by facet, and by failure family. +#[must_use] +pub fn render_status(source: &str, rows: &[ReportRow], top: usize) -> String { + let mut states: BTreeMap = BTreeMap::new(); + let mut facets: BTreeMap<&str, (usize, usize)> = BTreeMap::new(); + let mut catalogs: BTreeMap<&str, ()> = BTreeMap::new(); + let mut modes: BTreeMap<&str, ()> = BTreeMap::new(); + for row in rows { + *states.entry(row.state).or_default() += 1; + let facet = facets.entry(&row.facet).or_default(); + facet.0 += 1; + facet.1 += usize::from(row.state.is_pass()); + catalogs.insert(row.key.catalog(), ()); + modes.insert(row.key.mode().as_str(), ()); + } + let join = |names: &BTreeMap<&str, ()>| names.keys().copied().collect::>().join(","); + + let mut out = String::new(); + let _ = writeln!(out, "receipt: {source}"); + let _ = writeln!( + out, + "rows: {} catalogs: {} modes: {}", + rows.len(), + join(&catalogs), + join(&modes) + ); + out.push_str("\nstates:\n"); + for (state, count) in &states { + let _ = writeln!( + out, + " {:<40} {count:>7} {:>6}", + state.as_str(), + percent(*count, rows.len()) + ); + } + out.push_str("\nfacets: pass total\n"); + for (facet, (total, passing)) in &facets { + let _ = writeln!(out, " {facet:<40} {passing:>7} {total:>7}"); + } + let families = family_counts(rows); + let _ = writeln!( + out, + "\nfamilies of rows that block completion ({} total, largest {} shown):", + families.len(), + top.min(families.len()) + ); + for (family, count) in families.iter().take(top) { + let _ = writeln!(out, " {count:>7} {family}"); + } + out +} + +/// One row of a family, with the size of its input when the file is found. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct FamilyCase { + pub size: Option, + pub state: TerminalState, + pub key: ObligationKey, +} + +/// Rows of `family`, smallest input first; rows whose file is not found last. +/// +/// A case path starts with the catalog's runner segment. Dropping it leaves the +/// path inside the authority tree at `root`. +#[must_use] +pub fn family_cases(rows: &[ReportRow], family: &str, root: Option<&Path>) -> Vec { + let mut cases: Vec = rows + .iter() + .filter(|row| row.family().as_deref() == Some(family)) + .map(|row| FamilyCase { + size: root.and_then(|root| input_size(root, row.key.case())), + state: row.state, + key: row.key.clone(), + }) + .collect(); + cases.sort_by(|left, right| { + let size = |case: &FamilyCase| (case.size.is_none(), case.size); + size(left) + .cmp(&size(right)) + .then_with(|| left.key.cmp(&right.key)) + }); + cases +} + +fn input_size(root: &Path, case: &str) -> Option { + let (_, inside) = case.split_once('/')?; + fs::metadata(root.join(inside)).ok().map(|meta| meta.len()) +} + +#[must_use] +pub fn render_family_cases(family: &str, cases: &[FamilyCase]) -> String { + let mut out = String::new(); + let _ = writeln!( + out, + "family: {family} ({} rows, smallest first)", + cases.len() + ); + for case in cases { + let size = case + .size + .map_or_else(|| "?".to_owned(), |bytes| bytes.to_string()); + let _ = writeln!( + out, + " {size:>9} {:<14} {}", + case.state.as_str(), + case.key + ); + } + out +} + +/// What changed between two receipts, by obligation identity. +#[derive(Debug, Clone, Default, PartialEq, Eq)] +pub struct Diff { + pub common: usize, + pub unchanged: usize, + pub only_in_a: Vec, + pub only_in_b: Vec, + pub changed: BTreeMap<(TerminalState, TerminalState), Vec>, +} + +impl Diff { + /// Rows that became `PASS`. + #[must_use] + pub fn conversions(&self) -> Vec<&ObligationKey> { + self.moves(|from, to| !from.is_pass() && to.is_pass()) + } + + /// Rows that stopped being `PASS`. + #[must_use] + pub fn regressions(&self) -> Vec<&ObligationKey> { + self.moves(|from, to| from.is_pass() && !to.is_pass()) + } + + fn moves(&self, wanted: impl Fn(TerminalState, TerminalState) -> bool) -> Vec<&ObligationKey> { + self.changed + .iter() + .filter(|((from, to), _)| wanted(*from, *to)) + .flat_map(|(_, keys)| keys) + .collect() + } +} + +#[must_use] +pub fn diff_rows(a: &[ReportRow], b: &[ReportRow]) -> Diff { + let after: BTreeMap<&ObligationKey, TerminalState> = + b.iter().map(|row| (&row.key, row.state)).collect(); + let mut seen = 0usize; + let mut diff = Diff::default(); + for row in a { + match after.get(&row.key) { + None => diff.only_in_a.push(row.key.clone()), + Some(&state) => { + seen += 1; + if state == row.state { + diff.unchanged += 1; + } else { + diff.changed + .entry((row.state, state)) + .or_default() + .push(row.key.clone()); + } + } + } + } + diff.common = seen; + let before: BTreeMap<&ObligationKey, ()> = a.iter().map(|row| (&row.key, ())).collect(); + diff.only_in_b = b + .iter() + .filter(|row| !before.contains_key(&row.key)) + .map(|row| row.key.clone()) + .collect(); + diff +} + +#[must_use] +pub fn render_diff(a: &str, b: &str, diff: &Diff, limit: usize) -> String { + let changed: usize = diff.changed.values().map(Vec::len).sum(); + let mut out = String::new(); + let _ = writeln!(out, "diff: {a} -> {b}"); + let _ = writeln!( + out, + "rows in both: {} only in first: {} only in second: {}", + diff.common, + diff.only_in_a.len(), + diff.only_in_b.len() + ); + let _ = writeln!( + out, + "unchanged: {} changed: {changed} conversions to PASS: {} regressions from PASS: {}", + diff.unchanged, + diff.conversions().len(), + diff.regressions().len() + ); + for ((from, to), keys) in &diff.changed { + let _ = writeln!( + out, + "\n{} -> {} ({})", + from.as_str(), + to.as_str(), + keys.len() + ); + for key in keys.iter().take(limit) { + let _ = writeln!(out, " {key}"); + } + if keys.len() > limit { + let _ = writeln!(out, " ... {} more", keys.len() - limit); + } + } + out +} + +/// The report of a family rerun, and how many of its rows stopped passing. +#[must_use] +pub fn render_recount( + family: &str, + before: &[ReportRow], + after: &[ReportRow], + limit: usize, +) -> (String, usize) { + let diff = diff_rows(before, after); + let guard = before.iter().filter(|row| row.state.is_pass()).count(); + let mut report = format!( + "recount of family `{family}`: {} row(s) rerun, {guard} of them passing rows kept as a \ + regression guard\n", + after.len() + ); + report.push_str(&render_diff("before", "after", &diff, limit)); + (report, diff.regressions().len()) +} + +const STATUS_SCHEMA: &str = "bamti.status/v1"; + +/// One receipt's place in the status report: which lane it measures, at which +/// commit, and how its rows fall across the states. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct LaneStatus { + pub file: String, + pub catalog: String, + pub mode: String, + pub platform: String, + /// `merged` for a complete receipt, otherwise `/`. + pub shard: String, + pub commit: String, + pub rows: usize, + pub states: BTreeMap, +} + +impl LaneStatus { + fn identity(&self) -> (&str, &str, &str, &str) { + (&self.catalog, &self.mode, &self.platform, &self.shard) + } + + fn count(&self, state: &str) -> usize { + self.states.get(state).copied().unwrap_or(0) + } +} + +/// What the status report keeps beside the Markdown, so the next report can say +/// what changed. +#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)] +#[serde(deny_unknown_fields)] +pub struct StatusFile { + pub schema: String, + pub lanes: Vec, +} + +/// Reads one receipt end to end and reduces it to a lane. +pub fn read_lane_status(path: &Path) -> Result { + let mut reader = EvidenceReader::open(path)?; + let commit = reader.header().execution().source_sha().to_owned(); + let spec = reader.header().shard().spec(); + let shard = if spec == ShardSpec::unsharded() { + "merged".to_owned() + } else { + format!("{}/{}", spec.index(), spec.count()) + }; + let mut lane: Option<(String, String, String)> = None; + let mut states: BTreeMap = BTreeMap::new(); + let mut rows = 0; + while let Some(row) = reader.next_row()? { + let key = row.key(); + let this = ( + key.catalog().to_owned(), + key.mode().as_str().to_owned(), + key.platform().to_owned(), + ); + match &lane { + None => lane = Some(this), + Some(first) if *first != this => { + return Err(VerificationError::new( + ErrorCode::Schema, + format!( + "{}: one receipt mixes catalogs, modes or platforms", + path.display() + ), + )); + } + Some(_) => {} + } + *states.entry(row.state().as_str().to_owned()).or_default() += 1; + rows += 1; + } + let Some((catalog, mode, platform)) = lane else { + return Err(VerificationError::new( + ErrorCode::Schema, + format!("{}: receipt carries no rows", path.display()), + )); + }; + Ok(LaneStatus { + file: path + .file_name() + .map_or_else(String::new, |name| name.to_string_lossy().into_owned()), + catalog, + mode, + platform, + shard, + commit, + rows, + states, + }) +} + +fn short(commit: &str) -> &str { + commit.get(..12).unwrap_or(commit) +} + +/// `+n`, `-n` or `0`. +fn signed_change(now: usize, before: usize) -> String { + match now.cmp(&before) { + std::cmp::Ordering::Greater => format!("+{}", now - before), + std::cmp::Ordering::Less => format!("-{}", before - now), + std::cmp::Ordering::Equal => "0".to_owned(), + } +} + +/// The Markdown report: one summary table, then each lane's counts and what +/// changed since the previous report for the same lane. It is a function of the +/// receipts and the previous report alone, so it carries no timestamp. +#[must_use] +pub fn render_status_report(lanes: &[LaneStatus], previous: &[LaneStatus]) -> String { + let before: BTreeMap<(&str, &str, &str, &str), &LaneStatus> = previous + .iter() + .map(|lane| (lane.identity(), lane)) + .collect(); + let mut out = String::from("# Sweep status\n\n"); + let _ = writeln!( + out, + "{} receipt(s). Counts come from the receipts alone; Change compares with the previous report for the same lane.\n", + lanes.len() + ); + out.push_str( + "| Catalog | Mode | Platform | Shard | Commit | Rows | PASS | Not PASS |\n|---|---|---|---|---|---:|---:|---:|\n", + ); + for lane in lanes { + let _ = writeln!( + out, + "| {} | {} | {} | {} | `{}` | {} | {} | {} |", + lane.catalog, + lane.mode, + lane.platform, + lane.shard, + short(&lane.commit), + lane.rows, + lane.count("PASS"), + lane.rows - lane.count("PASS") + ); + } + for lane in lanes { + let _ = writeln!( + out, + "\n## {} / {} / {} / shard {}\n", + lane.catalog, lane.mode, lane.platform, lane.shard + ); + let prior = before.get(&lane.identity()).copied(); + let _ = writeln!( + out, + "Commit `{}`, receipt `{}`, {} rows, {} PASS ({}).", + short(&lane.commit), + lane.file, + lane.rows, + lane.count("PASS"), + percent(lane.count("PASS"), lane.rows) + ); + let _ = match prior { + None => writeln!(out, "Previous report: none for this lane.\n"), + Some(prior) => writeln!(out, "Previous report: commit `{}`.\n", short(&prior.commit)), + }; + out.push_str("| State | Rows | Change |\n|---|---:|---:|\n"); + let mut names: BTreeSet<&str> = lane.states.keys().map(String::as_str).collect(); + if let Some(prior) = prior { + names.extend(prior.states.keys().map(String::as_str)); + } + let mut ordered: Vec<&str> = names.into_iter().collect(); + ordered.sort_by_key(|name| (*name != "PASS", *name)); + for name in ordered { + let change = prior.map_or_else( + || "new".to_owned(), + |prior| signed_change(lane.count(name), prior.count(name)), + ); + let _ = writeln!(out, "| {name} | {} | {change} |", lane.count(name)); + } + } + out +} + +/// `suite status`: write the report for every receipt in a directory. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct StatusRequest { + pub receipts: PathBuf, + pub out: PathBuf, +} + +/// Writes `out` and a `.json` sidecar beside it, and returns a one-line summary. +/// +/// An empty directory, an unreadable receipt, or two receipts for one lane is +/// an error, and nothing is written: a report must not read as complete when it +/// is not. +pub fn status_report(request: &StatusRequest) -> Result { + let sidecar = request.out.with_extension("json"); + if sidecar == request.out { + return Err(VerificationError::new( + ErrorCode::Usage, + format!( + "{}: the report cannot share a path with its `.json` sidecar", + request.out.display() + ), + )); + } + let io_error = |path: &Path, error: io::Error| { + VerificationError::new(ErrorCode::Io, format!("{}: {error}", path.display())) + }; + let mut paths: Vec = fs::read_dir(&request.receipts) + .map_err(|error| io_error(&request.receipts, error))? + .filter_map(|entry| entry.ok().map(|entry| entry.path())) + .filter(|path| { + path.extension() + .is_some_and(|extension| extension == "jsonl") + }) + .collect(); + paths.sort(); + if paths.is_empty() { + return Err(VerificationError::new( + ErrorCode::Usage, + format!( + "{}: no `.jsonl` receipts to report on", + request.receipts.display() + ), + )); + } + let mut lanes = paths + .iter() + .map(|path| read_lane_status(path)) + .collect::>>()?; + lanes.sort_by(|left, right| { + left.identity() + .cmp(&right.identity()) + .then_with(|| left.file.cmp(&right.file)) + }); + if let Some(pair) = lanes + .windows(2) + .find(|pair| pair[0].identity() == pair[1].identity()) + { + return Err(VerificationError::new( + ErrorCode::Duplicate, + format!( + "`{}` and `{}` are receipts for the same lane", + pair[0].file, pair[1].file + ), + )); + } + + let previous = match fs::read_to_string(&sidecar) { + Ok(text) => { + serde_json::from_str::(&text) + .map_err(|error| { + VerificationError::new( + ErrorCode::Json, + format!("{}: {error}", sidecar.display()), + ) + })? + .lanes + } + Err(error) if error.kind() == io::ErrorKind::NotFound => Vec::new(), + Err(error) => return Err(io_error(&sidecar, error)), + }; + let report = render_status_report(&lanes, &previous); + let file = StatusFile { + schema: STATUS_SCHEMA.to_owned(), + lanes, + }; + let json = serde_json::to_string_pretty(&file) + .map_err(|error| VerificationError::new(ErrorCode::Json, error.to_string()))?; + write_atomically(&request.out, report.as_bytes())?; + write_atomically(&sidecar, format!("{json}\n").as_bytes())?; + Ok(format!( + "STATUS REPORT {} from {} receipt(s)", + request.out.display(), + file.lanes.len() + )) +} + +fn write_atomically(path: &Path, bytes: &[u8]) -> Result<()> { + let io_error = |path: &Path, error: io::Error| { + VerificationError::new(ErrorCode::Io, format!("{}: {error}", path.display())) + }; + if let Some(parent) = path + .parent() + .filter(|parent| !parent.as_os_str().is_empty()) + { + fs::create_dir_all(parent).map_err(|error| io_error(parent, error))?; + } + let temp = path.with_extension(format!("tmp-{}", process::id())); + fs::write(&temp, bytes).map_err(|error| io_error(&temp, error))?; + fs::rename(&temp, path).map_err(|error| { + let _ = fs::remove_file(&temp); + io_error(path, error) + }) +} + +/// `suite families`: the status report, or the cases of one family. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct FamiliesRequest { + pub receipt: PathBuf, + pub top: usize, + pub family: Option, + pub root: Option, +} + +pub fn families(request: &FamiliesRequest) -> Result { + let rows = read_receipt(&request.receipt)?; + let source = request.receipt.display().to_string(); + Ok(match &request.family { + None => render_status(&source, &rows, request.top), + Some(family) => { + let cases = family_cases(&rows, family, request.root.as_deref()); + if cases.is_empty() { + return Err(VerificationError::new( + ErrorCode::Usage, + format!("{source}: no row belongs to family `{family}`"), + )); + } + render_family_cases(family, &cases) + } + }) +} + +/// `suite diff`: compares two receipts; optionally fails on any regression. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct DiffRequest { + pub a: PathBuf, + pub b: PathBuf, + pub limit: usize, + pub fail_on_regression: bool, +} + +/// The rendered report, and the number of regressions it found. +pub fn diff(request: &DiffRequest) -> Result<(String, usize)> { + let a = read_receipt(&request.a)?; + let b = read_receipt(&request.b)?; + let diff = diff_rows(&a, &b); + let report = render_diff( + &request.a.display().to_string(), + &request.b.display().to_string(), + &diff, + request.limit, + ); + Ok((report, diff.regressions().len())) +} + +#[cfg(test)] +mod tests { + use std::{ + collections::BTreeSet, + sync::atomic::{AtomicU64, Ordering}, + }; + + use super::*; + use crate::{ + evidence::{ + EvidenceHeader, ExecutionBinding, PublishMode, RunBinding, ToolchainPin, + WorkingDirectoryPolicy, publish_evidence, + }, + shard::{ExecutionMode, ShardIdentity, ShardSpec}, + }; + + const DIGEST: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + static NEXT: AtomicU64 = AtomicU64::new(0); + + struct Scratch(PathBuf); + + impl Scratch { + fn new() -> Self { + let root = std::env::temp_dir().join(format!( + "bamts-sweep-report-{}-{}", + std::process::id(), + NEXT.fetch_add(1, Ordering::Relaxed) + )); + let _ = fs::remove_dir_all(&root); + fs::create_dir_all(&root).unwrap(); + Self(root) + } + } + + impl Drop for Scratch { + fn drop(&mut self) { + let _ = fs::remove_dir_all(&self.0); + } + } + + fn key(case: &str) -> ObligationKey { + ObligationKey::new( + "typescript-7.0.2", + case, + "default#diagnostics", + ExecutionMode::Aot, + "aarch64-apple-darwin", + ) + .unwrap() + } + + fn row(case: &str, state: TerminalState, detail: &str) -> ReportRow { + ReportRow::new(key(case), "diagnostics", state, detail) + } + + fn failing(case: &str, detail: &str) -> ReportRow { + row(case, TerminalState::BlockingFail, detail) + } + + fn passing(case: &str) -> ReportRow { + row(case, TerminalState::Pass, "") + } + + #[test] + fn rows_that_differ_only_in_numbers_share_a_family() { + let one = failing("a", "diagnostic count mismatch: expected 2 actual 1"); + let two = failing("b", "diagnostic count mismatch: expected 17 actual 3"); + let other = failing("c", "diagnostic text mismatch"); + assert_eq!(one.family(), two.family()); + assert_ne!(one.family(), other.family()); + assert_eq!( + one.family().as_deref(), + Some("diagnostics/diagnostic-count-mismatch-expected-n-actual-n") + ); + } + + #[test] + fn a_test262_family_ignores_the_mode_but_not_the_failure_kind() { + let detail = |mode: &str, actual: &str| { + format!( + "test262 {mode} mode failure: ExpectationMismatch {{ expected: \"success\", actual: \"{actual}\" }}" + ) + }; + let interpreter = failing("a", &detail("interpreter", "Parse/SyntaxError")); + let jit = failing("b", &detail("jit", "Parse/SyntaxError")); + let thrown = failing("c", &detail("jit", "Runtime/Test262Error")); + assert_eq!(interpreter.family(), jit.family()); + assert_ne!(jit.family(), thrown.family()); + } + + #[test] + fn the_tail_of_a_long_quoted_span_does_not_split_a_family() { + let prefix = "compile pipeline failed: could not load program: cannot resolve module"; + let first = failing( + "a", + &format!("BlockedRun {{ detail: \"{prefix} one/x.js\" }}"), + ); + let second = failing( + "b", + &format!("BlockedRun {{ detail: \"{prefix} two/y.js\" }}"), + ); + assert_eq!(first.family(), second.family()); + let short = failing("c", "BlockedRun { detail: \"timeout\" }"); + assert_ne!(first.family(), short.family()); + } + + #[test] + fn an_absolute_path_in_the_text_does_not_split_a_family() { + let unix = |directory: &str| { + failing( + "a", + &format!("cannot resolve \"react\" from {directory}/tests/cases/file.tsx"), + ) + }; + assert_eq!( + unix("/private/var/folders/x1/T/cell-1").family(), + unix("/tmp/other-2").family() + ); + assert_eq!( + unix("/tmp/a").family().as_deref(), + Some("diagnostics/cannot-resolve-react-from") + ); + let windows = failing( + "b", + "cannot resolve \"react\" from C:\\Users\\run\\file.tsx", + ); + let drive = failing("c", "cannot resolve \"react\" from D:/work/file.tsx"); + assert_eq!(windows.family(), unix("/tmp/a").family()); + assert_eq!(drive.family(), unix("/tmp/a").family()); + let other_module = failing("d", "cannot resolve \"vue\" from /tmp/a/file.tsx"); + assert_ne!(other_module.family(), unix("/tmp/a").family()); + } + + #[test] + fn backtick_spans_and_escaped_quotes_are_dropped_from_the_family() { + let first = failing("a", "locked classification for `x/one` is stale"); + let second = failing("b", "locked classification for `y/two/three` is stale"); + assert_eq!(first.family(), second.family()); + let escaped = failing("c", "got \"a \\\" b\" here"); + assert_eq!( + escaped.family().as_deref(), + Some("diagnostics/got-a-b-here") + ); + } + + #[test] + fn only_states_that_block_completion_have_a_family() { + for state in [ + TerminalState::BlockingFail, + TerminalState::ExternalBlocked, + TerminalState::InapplicableCatalogError, + TerminalState::Timeout, + TerminalState::Signal, + TerminalState::ProtocolError, + TerminalState::WorkerCrash, + ] { + assert!(row("a", state, "x").family().is_some(), "{state:?}"); + } + for state in [ + TerminalState::Pass, + TerminalState::InapplicableLanguageService, + TerminalState::InapplicableOutOfScopeHostFeature, + TerminalState::InapplicableV8Internal, + ] { + assert!(row("a", state, "x").family().is_none(), "{state:?}"); + } + } + + #[test] + fn a_failure_with_no_text_still_has_a_family() { + assert_eq!( + failing("a", "").family().as_deref(), + Some("diagnostics/no-detail") + ); + } + + #[test] + fn status_lists_the_largest_family_first_and_honors_the_limit() { + let mut rows = vec![passing("p")]; + for index in 0..3 { + rows.push(failing(&format!("big{index}"), "big failure")); + } + rows.push(failing("small", "small failure")); + let report = render_status("r.jsonl", &rows, 1); + assert!(report.contains("rows: 5"), "{report}"); + assert!(report.contains("PASS"), "{report}"); + assert!(report.contains("BLOCKING_FAIL"), "{report}"); + assert!(report.contains("20.0%"), "{report}"); + assert!(report.contains("diagnostics/big-failure"), "{report}"); + assert!(!report.contains("diagnostics/small-failure"), "{report}"); + let all = render_status("r.jsonl", &rows, 10); + let big = all.find("big-failure").unwrap(); + let small = all.find("small-failure").unwrap(); + assert!(big < small, "{all}"); + } + + #[test] + fn a_receipt_diffed_against_itself_has_no_transitions() { + let rows = vec![passing("a"), failing("b", "x"), failing("c", "y")]; + let diff = diff_rows(&rows, &rows); + assert_eq!(diff.common, 3); + assert_eq!(diff.unchanged, 3); + assert!(diff.changed.is_empty()); + assert!(diff.only_in_a.is_empty() && diff.only_in_b.is_empty()); + assert!(diff.conversions().is_empty() && diff.regressions().is_empty()); + } + + #[test] + fn a_diff_names_conversions_regressions_and_unmatched_rows() { + let before = vec![ + failing("fixed", "x"), + passing("broken"), + failing("still", "x"), + failing("gone", "x"), + ]; + let after = vec![ + passing("fixed"), + failing("broken", "x"), + failing("still", "x"), + failing("new", "x"), + ]; + let diff = diff_rows(&before, &after); + assert_eq!(diff.common, 3); + assert_eq!(diff.unchanged, 1); + assert_eq!(diff.conversions(), vec![&key("fixed")]); + assert_eq!(diff.regressions(), vec![&key("broken")]); + assert_eq!(diff.only_in_a, vec![key("gone")]); + assert_eq!(diff.only_in_b, vec![key("new")]); + let report = render_diff("a", "b", &diff, 10); + assert!(report.contains("conversions to PASS: 1"), "{report}"); + assert!(report.contains("regressions from PASS: 1"), "{report}"); + assert!(report.contains("BLOCKING_FAIL -> PASS"), "{report}"); + } + + #[test] + fn a_transition_between_two_failing_states_is_not_a_regression() { + let before = vec![failing("a", "x")]; + let after = vec![row("a", TerminalState::Timeout, "x")]; + let diff = diff_rows(&before, &after); + assert_eq!(diff.changed.len(), 1); + assert!(diff.regressions().is_empty() && diff.conversions().is_empty()); + } + + #[test] + fn a_recount_report_names_the_family_and_counts_regressions() { + let before = vec![failing("a", "x"), passing("b")]; + let after = vec![passing("a"), failing("b", "x")]; + let (report, regressions) = render_recount("diagnostics/x", &before, &after, 5); + assert_eq!(regressions, 1, "{report}"); + assert!(report.contains("family `diagnostics/x`"), "{report}"); + assert!(report.contains("2 row(s) rerun"), "{report}"); + assert!(report.contains("conversions to PASS: 1"), "{report}"); + + let (clean, none) = render_recount("diagnostics/x", &before, &before, 5); + assert_eq!(none, 0, "{clean}"); + assert!(clean.contains("changed: 0"), "{clean}"); + } + + #[test] + fn the_report_limit_truncates_each_transition_list() { + let before: Vec = (0..5) + .map(|index| failing(&format!("c{index}"), "x")) + .collect(); + let after: Vec = (0..5).map(|index| passing(&format!("c{index}"))).collect(); + let report = render_diff("a", "b", &diff_rows(&before, &after), 2); + assert!(report.contains("... 3 more"), "{report}"); + } + + #[test] + fn family_cases_come_smallest_first_with_unknown_sizes_last() { + let scratch = Scratch::new(); + fs::create_dir_all(scratch.0.join("tests")).unwrap(); + fs::write(scratch.0.join("tests/big.ts"), "x".repeat(900)).unwrap(); + fs::write(scratch.0.join("tests/small.ts"), "x".repeat(10)).unwrap(); + let rows = vec![ + failing("compiler/tests/big.ts", "same"), + failing("compiler/tests/missing.ts", "same"), + failing("compiler/tests/small.ts", "same"), + failing("compiler/tests/other.ts", "different"), + ]; + let family = rows[0].family().unwrap(); + let cases = family_cases(&rows, &family, Some(&scratch.0)); + let order: Vec<&str> = cases.iter().map(|case| case.key.case()).collect(); + assert_eq!( + order, + [ + "compiler/tests/small.ts", + "compiler/tests/big.ts", + "compiler/tests/missing.ts" + ] + ); + assert_eq!(cases[0].size, Some(10)); + assert_eq!(cases[2].size, None); + assert!(family_cases(&rows, "diagnostics/none", Some(&scratch.0)).is_empty()); + } + + fn evidence_row(case: &str, state: TerminalState, detail: &str) -> EvidenceRow { + let mut artifacts = BTreeMap::new(); + if state.is_pass() { + artifacts.insert("diagnostics".to_owned(), DIGEST.to_owned()); + } + EvidenceRow::new( + key(case), + vec!["run".to_owned()], + WorkingDirectoryPolicy::RepositoryRoot, + BTreeSet::from(["diagnostics".to_owned()]), + artifacts, + state, + 1, + detail, + ) + .unwrap() + } + + fn write_receipt(path: &Path, rows: &[(&str, TerminalState, &str)]) { + let mut sorted: Vec<_> = rows.to_vec(); + sorted.sort_by_key(|(case, ..)| key(case)); + let catalog: Vec = sorted.iter().map(|(case, ..)| key(case)).collect(); + let shard = ShardIdentity::plan(ShardSpec::unsharded(), &catalog).unwrap(); + let pin = ToolchainPin::new("rustc-1.97.1", DIGEST).unwrap(); + let binding = RunBinding::new(DIGEST, DIGEST, DIGEST, DIGEST, pin).unwrap(); + let header = + EvidenceHeader::new(shard, binding, ExecutionBinding::local_for_tests()).unwrap(); + let evidence: Vec = sorted + .iter() + .map(|(case, state, detail)| evidence_row(case, *state, detail)) + .collect(); + publish_evidence(path, header, &evidence, PublishMode::Replace).unwrap(); + } + + #[test] + fn a_real_receipt_reads_back_with_its_facet_state_and_family() { + let scratch = Scratch::new(); + let path = scratch.0.join("r.jsonl"); + write_receipt( + &path, + &[ + ("a", TerminalState::Pass, ""), + ( + "b", + TerminalState::BlockingFail, + "types mismatch at line 12", + ), + ], + ); + let rows = read_receipt(&path).unwrap(); + assert_eq!(rows.len(), 2); + assert_eq!(rows[0].facet(), "diagnostics"); + assert!(rows[0].state().is_pass() && rows[0].family().is_none()); + assert_eq!( + rows[1].family().as_deref(), + Some("diagnostics/types-mismatch-at-line-n") + ); + } + + #[test] + fn a_truncated_receipt_is_rejected_not_reported_short() { + let scratch = Scratch::new(); + let path = scratch.0.join("r.jsonl"); + write_receipt( + &path, + &[ + ("a", TerminalState::Pass, ""), + ("b", TerminalState::Pass, ""), + ], + ); + let text = fs::read_to_string(&path).unwrap(); + let without_footer: Vec<&str> = text.lines().collect::>(); + let cut = without_footer[..without_footer.len() - 1].join("\n"); + fs::write(&path, cut).unwrap(); + assert!(read_receipt(&path).is_err()); + } + + #[test] + fn a_receipt_whose_footer_disagrees_with_its_rows_is_rejected() { + let scratch = Scratch::new(); + let path = scratch.0.join("r.jsonl"); + write_receipt( + &path, + &[ + ("a", TerminalState::Pass, ""), + ("b", TerminalState::Pass, ""), + ], + ); + let text = fs::read_to_string(&path).unwrap(); + assert_eq!(text.matches("\"row_count\":2").count(), 1, "{text}"); + fs::write(&path, text.replace("\"row_count\":2", "\"row_count\":9")).unwrap(); + assert!(read_receipt(&path).is_err()); + } + + /// A receipt for one lane. `shard` is `(index, count)`; only that shard's + /// members of the four-case catalog are written, as a real run does. + fn write_lane(path: &Path, commit: &str, shard: (u32, u32), states: [TerminalState; 4]) { + let cases = ["c0", "c1", "c2", "c3"]; + let catalog: Vec = cases.iter().map(|case| key(case)).collect(); + let spec = ShardSpec::new(shard.0, shard.1).unwrap(); + let identity = ShardIdentity::plan(spec, &catalog).unwrap(); + let pin = ToolchainPin::new("rustc-1.97.1", DIGEST).unwrap(); + let binding = RunBinding::new(DIGEST, DIGEST, DIGEST, DIGEST, pin).unwrap(); + let execution = + ExecutionBinding::new("Justfile", "run", 1, commit, "job", "Darwin-arm64", "rust") + .unwrap(); + let header = EvidenceHeader::new(identity, binding, execution).unwrap(); + let rows: Vec = spec + .member_indices(catalog.len()) + .map(|index| evidence_row(cases[index], states[index], "x")) + .collect(); + publish_evidence(path, header, &rows, PublishMode::Replace).unwrap(); + } + + const SHA_A: &str = "aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa"; + const SHA_B: &str = "bbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbbb"; + use TerminalState::{BlockingFail as Fail, Pass}; + + #[test] + fn a_lane_status_reads_its_commit_shard_and_counts_from_the_receipt() { + let scratch = Scratch::new(); + let merged = scratch.0.join("merged.jsonl"); + let shard = scratch.0.join("shard.jsonl"); + write_lane(&merged, SHA_A, (0, 1), [Pass, Pass, Fail, Fail]); + write_lane(&shard, SHA_B, (0, 2), [Pass, Pass, Fail, Fail]); + + let whole = read_lane_status(&merged).unwrap(); + assert_eq!(whole.commit, SHA_A); + assert_eq!(whole.shard, "merged"); + assert_eq!( + (whole.catalog.as_str(), whole.mode.as_str()), + ("typescript-7.0.2", "aot") + ); + assert_eq!(whole.platform, "aarch64-apple-darwin"); + assert_eq!(whole.rows, 4); + assert_eq!(whole.states.get("PASS"), Some(&2)); + assert_eq!(whole.states.get("BLOCKING_FAIL"), Some(&2)); + + let half = read_lane_status(&shard).unwrap(); + assert_eq!(half.shard, "0/2"); + assert_eq!(half.rows, 2, "shard 0/2 holds cases c0 and c2"); + assert_eq!(half.states.get("PASS"), Some(&1)); + } + + #[test] + fn the_report_covers_every_receipt_in_the_directory() { + let scratch = Scratch::new(); + let receipts = scratch.0.join("merged"); + fs::create_dir_all(&receipts).unwrap(); + write_lane( + &receipts.join("a.jsonl"), + SHA_A, + (0, 1), + [Pass, Pass, Pass, Fail], + ); + write_lane( + &receipts.join("b.jsonl"), + SHA_A, + (1, 2), + [Pass, Fail, Fail, Fail], + ); + fs::write(receipts.join("notes.txt"), "not a receipt").unwrap(); + let out = scratch.0.join("status.md"); + + let summary = status_report(&StatusRequest { + receipts, + out: out.clone(), + }) + .unwrap(); + assert!(summary.contains("2 receipt(s)"), "{summary}"); + let report = fs::read_to_string(&out).unwrap(); + assert!(report.contains("shard merged"), "{report}"); + assert!(report.contains("shard 1/2"), "{report}"); + assert!(report.contains(&SHA_A[..12]), "{report}"); + let sidecar = fs::read_to_string(scratch.0.join("status.json")).unwrap(); + let lanes: StatusFile = serde_json::from_str(&sidecar).unwrap(); + assert_eq!(lanes.lanes.len(), 2); + } + + #[test] + fn a_later_report_shows_the_change_since_the_previous_one() { + let scratch = Scratch::new(); + let receipts = scratch.0.join("merged"); + fs::create_dir_all(&receipts).unwrap(); + let out = scratch.0.join("status.md"); + let request = StatusRequest { + receipts: receipts.clone(), + out: out.clone(), + }; + + write_lane( + &receipts.join("a.jsonl"), + SHA_A, + (0, 1), + [Pass, Fail, Fail, Fail], + ); + status_report(&request).unwrap(); + let first = fs::read_to_string(&out).unwrap(); + assert!( + first.contains("new"), + "a first report marks the lane new: {first}" + ); + + write_lane( + &receipts.join("a.jsonl"), + SHA_B, + (0, 1), + [Pass, Pass, Pass, Fail], + ); + status_report(&request).unwrap(); + let second = fs::read_to_string(&out).unwrap(); + assert!( + second.contains(&SHA_A[..12]), + "names the previous commit: {second}" + ); + assert!(second.contains("| PASS | 3 | +2 |"), "{second}"); + assert!(second.contains("| BLOCKING_FAIL | 1 | -2 |"), "{second}"); + + status_report(&request).unwrap(); + let third = fs::read_to_string(&out).unwrap(); + assert!( + third.contains("| PASS | 3 | 0 |"), + "an unchanged lane shows no change: {third}" + ); + } + + #[test] + fn a_report_cannot_take_the_path_of_its_own_sidecar() { + let scratch = Scratch::new(); + let receipts = scratch.0.join("merged"); + fs::create_dir_all(&receipts).unwrap(); + write_lane( + &receipts.join("a.jsonl"), + SHA_A, + (0, 1), + [Pass, Pass, Fail, Fail], + ); + let out = scratch.0.join("status.json"); + fs::write(&out, "previous").unwrap(); + let error = status_report(&StatusRequest { + receipts, + out: out.clone(), + }) + .expect_err("the report and sidecar share a path"); + assert_eq!(error.code(), ErrorCode::Usage); + assert_eq!(fs::read_to_string(&out).unwrap(), "previous"); + } + + #[test] + fn a_truncated_receipt_is_refused_by_the_status_report() { + let scratch = Scratch::new(); + let receipts = scratch.0.join("merged"); + fs::create_dir_all(&receipts).unwrap(); + let path = receipts.join("a.jsonl"); + write_lane(&path, SHA_A, (0, 1), [Pass, Pass, Fail, Fail]); + let text = fs::read_to_string(&path).unwrap(); + let lines: Vec<&str> = text.lines().collect(); + fs::write(&path, lines[..lines.len() - 1].join("\n")).unwrap(); + let out = scratch.0.join("status.md"); + assert!( + status_report(&StatusRequest { + receipts, + out: out.clone() + }) + .is_err() + ); + assert!(!out.exists()); + } + + #[test] + fn one_receipt_that_mixes_modes_is_refused() { + let scratch = Scratch::new(); + let path = scratch.0.join("mixed.jsonl"); + let aot = key("c0"); + let jit = ObligationKey::new( + "typescript-7.0.2", + "c1", + "default#diagnostics", + ExecutionMode::Jit, + "aarch64-apple-darwin", + ) + .unwrap(); + let catalog = vec![aot.clone(), jit.clone()]; + let shard = ShardIdentity::plan(ShardSpec::unsharded(), &catalog).unwrap(); + let pin = ToolchainPin::new("rustc-1.97.1", DIGEST).unwrap(); + let binding = RunBinding::new(DIGEST, DIGEST, DIGEST, DIGEST, pin).unwrap(); + let header = + EvidenceHeader::new(shard, binding, ExecutionBinding::local_for_tests()).unwrap(); + let rows: Vec = [aot, jit] + .into_iter() + .map(|key| { + EvidenceRow::new( + key, + vec!["run".to_owned()], + WorkingDirectoryPolicy::RepositoryRoot, + BTreeSet::from(["diagnostics".to_owned()]), + BTreeMap::new(), + TerminalState::BlockingFail, + 1, + "x", + ) + .unwrap() + }) + .collect(); + publish_evidence(&path, header, &rows, PublishMode::Replace).unwrap(); + assert!(read_lane_status(&path).is_err()); + } + + #[test] + fn a_receipt_directory_with_no_receipts_or_a_bad_one_is_an_error() { + let scratch = Scratch::new(); + let empty = scratch.0.join("empty"); + fs::create_dir_all(&empty).unwrap(); + let out = scratch.0.join("status.md"); + let none = status_report(&StatusRequest { + receipts: empty, + out: out.clone(), + }); + assert!( + none.is_err(), + "an empty directory must not yield an empty report" + ); + + let bad = scratch.0.join("bad"); + fs::create_dir_all(&bad).unwrap(); + fs::write(bad.join("x.jsonl"), "{\"not\":\"a receipt\"}\n").unwrap(); + assert!( + status_report(&StatusRequest { + receipts: bad, + out: out.clone() + }) + .is_err() + ); + assert!(!out.exists(), "a failed report must not leave a file"); + } + + #[test] + fn the_request_functions_report_through_real_receipts() { + let scratch = Scratch::new(); + let (first, second) = (scratch.0.join("a.jsonl"), scratch.0.join("b.jsonl")); + write_receipt( + &first, + &[ + ("a", TerminalState::Pass, ""), + ("b", TerminalState::BlockingFail, "x"), + ], + ); + write_receipt( + &second, + &[ + ("a", TerminalState::BlockingFail, "x"), + ("b", TerminalState::BlockingFail, "x"), + ], + ); + + let (report, regressions) = diff(&DiffRequest { + a: first.clone(), + b: second.clone(), + limit: 5, + fail_on_regression: false, + }) + .unwrap(); + assert_eq!(regressions, 1, "{report}"); + + let (_, none) = diff(&DiffRequest { + a: first.clone(), + b: first.clone(), + limit: 5, + fail_on_regression: true, + }) + .unwrap(); + assert_eq!(none, 0); + + let status = families(&FamiliesRequest { + receipt: second.clone(), + top: 5, + family: None, + root: None, + }) + .unwrap(); + assert!(status.contains("diagnostics/x"), "{status}"); + + let missing = families(&FamiliesRequest { + receipt: second, + top: 5, + family: Some("diagnostics/not-there".to_owned()), + root: None, + }); + assert!(missing.is_err()); + } +} diff --git a/crates/bamts-verification/src/test_programs.rs b/crates/bamts-verification/src/test_programs.rs new file mode 100644 index 00000000..22288c08 --- /dev/null +++ b/crates/bamts-verification/src/test_programs.rs @@ -0,0 +1,9 @@ +//! Programs that every host has, for the tests that need a child process. + +use std::path::PathBuf; + +/// A program of the Windows system directory, such as `PING.EXE`. +pub(crate) fn windows_system_program(name: &str) -> PathBuf { + let root = std::env::var_os("SystemRoot").expect("SystemRoot is set on Windows"); + PathBuf::from(root).join("System32").join(name) +} diff --git a/crates/bamts-verification/src/toolchain_image.rs b/crates/bamts-verification/src/toolchain_image.rs new file mode 100644 index 00000000..67837a74 --- /dev/null +++ b/crates/bamts-verification/src/toolchain_image.rs @@ -0,0 +1,349 @@ +//! Reads what an object or linked image declares about itself. +//! +//! Target-cell evidence states facts about a file, so they are read from the +//! file's own headers rather than from the flags it was linked with. + +use crate::{ErrorCode, Result, VerificationError}; + +const PT_LOAD: u32 = 1; +const PT_DYNAMIC: u32 = 2; +const PT_GNU_STACK: u32 = 0x6474_e551; +const PT_GNU_RELRO: u32 = 0x6474_e552; +const ELF_WRITE: u32 = 2; +const ELF_EXECUTE: u32 = 1; +const ELF_PROGRAM_HEADER_BYTES: usize = 56; + +const MH_MAGIC_64: u32 = 0xfeed_facf; +const MH_OBJECT: u32 = 1; +const MH_EXECUTE: u32 = 2; +const MH_PIE: u32 = 0x0020_0000; +const LC_SEGMENT_64: u32 = 0x19; +const VM_PROT_WRITE: u32 = 2; +const VM_PROT_EXECUTE: u32 = 4; + +/// What one object or image declares in its headers. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct ImageFacts { + pub object_format: &'static str, + pub object_type: &'static str, + pub machine: &'static str, + /// Mapped regions that are both writable and executable. + pub writable_executable_regions: usize, + /// `key=value` evidence specific to this format, in evidence order. + pub format_evidence: Vec<(&'static str, String)>, +} + +/// Parses the headers of an ELF64 or Mach-O 64 file. +/// +/// # Errors +/// +/// Returns [`ErrorCode::Schema`] when `bytes` is not a complete 64-bit ELF or +/// Mach-O file of a known machine and type. +pub fn inspect_image(bytes: &[u8]) -> Result { + if bytes.starts_with(b"\x7fELF") { + inspect_elf(bytes) + } else if read_u32(bytes, 0, false)? == MH_MAGIC_64 { + inspect_macho(bytes) + } else { + Err(malformed("not an ELF64 or Mach-O 64 file")) + } +} + +fn inspect_elf(bytes: &[u8]) -> Result { + if bytes.get(4) != Some(&2) { + return Err(malformed("ELF file is not 64-bit")); + } + let big_endian = match bytes.get(5) { + Some(1) => false, + Some(2) => true, + _ => return Err(malformed("ELF file declares no byte order")), + }; + let object_type = match read_u16(bytes, 16, big_endian)? { + 1 => "relocatable", + 2 | 3 => "executable", + other => return Err(malformed(&format!("ELF type {other} is not supported"))), + }; + let machine = match read_u16(bytes, 18, big_endian)? { + 62 => "x86_64", + 183 => "aarch64", + 22 => "s390x", + 243 => "riscv64", + other => return Err(malformed(&format!("ELF machine {other} is not a target"))), + }; + + let table = usize::try_from(read_u64(bytes, 32, big_endian)?) + .map_err(|_| malformed("ELF program header offset does not fit"))?; + let entry_size = usize::from(read_u16(bytes, 54, big_endian)?); + let entries = usize::from(read_u16(bytes, 56, big_endian)?); + let table_end = table.checked_add(entries * entry_size); + if entries > 0 && (entry_size < ELF_PROGRAM_HEADER_BYTES || table_end > Some(bytes.len())) { + return Err(malformed("ELF program header table does not fit the file")); + } + let mut regions = 0; + let mut dynamic = false; + let mut relro = false; + let mut stack = None; + for index in 0..entries { + let header = table + index * entry_size; + let kind = read_u32(bytes, header, big_endian)?; + let flags = read_u32(bytes, header + 4, big_endian)?; + match kind { + PT_LOAD if flags & ELF_WRITE != 0 && flags & ELF_EXECUTE != 0 => regions += 1, + PT_DYNAMIC => dynamic = true, + PT_GNU_RELRO => relro = true, + PT_GNU_STACK => stack = Some(flags), + _ => {} + } + } + let stack_is_rw = stack == Some(6); + Ok(ImageFacts { + object_format: "elf", + object_type, + machine, + writable_executable_regions: regions, + format_evidence: vec![ + ("relro", relro.to_string()), + // With no dynamic section nothing is bound lazily, so all of it is bound now. + ("now", (!dynamic).to_string()), + ("noexecstack", stack_is_rw.to_string()), + ( + "gnu_stack", + if stack_is_rw { "RW" } else { "other" }.to_owned(), + ), + ( + "dynamic_section", + if dynamic { "present" } else { "absent" }.to_owned(), + ), + ], + }) +} + +fn inspect_macho(bytes: &[u8]) -> Result { + let machine = match read_u32(bytes, 4, false)? { + 0x0100_000c => "aarch64", + 0x0100_0007 => "x86_64", + other => return Err(malformed(&format!("Mach-O CPU {other:#x} is not a target"))), + }; + let object_type = match read_u32(bytes, 12, false)? { + MH_OBJECT => "relocatable", + MH_EXECUTE => "executable", + other => return Err(malformed(&format!("Mach-O type {other} is not supported"))), + }; + let flags = read_u32(bytes, 24, false)?; + + let mut regions = 0; + let mut command = 32; + for _ in 0..read_u32(bytes, 16, false)? { + let kind = read_u32(bytes, command, false)?; + let size = usize::try_from(read_u32(bytes, command + 4, false)?) + .map_err(|_| malformed("Mach-O load command size does not fit"))?; + if size < 8 { + return Err(malformed("Mach-O load command is shorter than its header")); + } + if kind == LC_SEGMENT_64 { + let protection = read_u32(bytes, command + 56, false)?; + if protection & VM_PROT_WRITE != 0 && protection & VM_PROT_EXECUTE != 0 { + regions += 1; + } + } + command += size; + } + Ok(ImageFacts { + object_format: "macho", + object_type, + machine, + writable_executable_regions: regions, + format_evidence: vec![("pie", (flags & MH_PIE != 0).to_string())], + }) +} + +fn slice(bytes: &[u8], offset: usize) -> Result<[u8; N]> { + offset + .checked_add(N) + .and_then(|end| bytes.get(offset..end)) + .and_then(|field| field.try_into().ok()) + .ok_or_else(|| malformed("file ends inside a header")) +} + +fn read_u16(bytes: &[u8], offset: usize, big_endian: bool) -> Result { + let field = slice(bytes, offset)?; + Ok(if big_endian { + u16::from_be_bytes(field) + } else { + u16::from_le_bytes(field) + }) +} + +fn read_u32(bytes: &[u8], offset: usize, big_endian: bool) -> Result { + let field = slice(bytes, offset)?; + Ok(if big_endian { + u32::from_be_bytes(field) + } else { + u32::from_le_bytes(field) + }) +} + +fn read_u64(bytes: &[u8], offset: usize, big_endian: bool) -> Result { + let field = slice(bytes, offset)?; + Ok(if big_endian { + u64::from_be_bytes(field) + } else { + u64::from_le_bytes(field) + }) +} + +fn malformed(detail: &str) -> VerificationError { + VerificationError::new(ErrorCode::Schema, format!("image headers: {detail}")) +} + +#[cfg(test)] +mod tests { + use super::*; + + /// A minimal ELF64 header with these program headers as (type, flags). + fn elf(big_endian: bool, machine: u16, kind: u16, headers: &[(u32, u32)]) -> Vec { + let u16b = |value: u16| { + if big_endian { + value.to_be_bytes() + } else { + value.to_le_bytes() + } + }; + let u32b = |value: u32| { + if big_endian { + value.to_be_bytes() + } else { + value.to_le_bytes() + } + }; + let u64b = |value: u64| { + if big_endian { + value.to_be_bytes() + } else { + value.to_le_bytes() + } + }; + let mut bytes = vec![0u8; 64]; + bytes[..4].copy_from_slice(b"\x7fELF"); + bytes[4] = 2; + bytes[5] = if big_endian { 2 } else { 1 }; + bytes[16..18].copy_from_slice(&u16b(kind)); + bytes[18..20].copy_from_slice(&u16b(machine)); + bytes[32..40].copy_from_slice(&u64b(64)); + bytes[54..56].copy_from_slice(&u16b(56)); + let count = u16::try_from(headers.len()).expect("header count"); + bytes[56..58].copy_from_slice(&u16b(count)); + for (kind, flags) in headers { + let mut header = [0u8; 56]; + header[..4].copy_from_slice(&u32b(*kind)); + header[4..8].copy_from_slice(&u32b(*flags)); + bytes.extend_from_slice(&header); + } + bytes + } + + /// A minimal Mach-O 64 header with one LC_SEGMENT_64 per `protections` entry. + fn macho(cpu: u32, kind: u32, flags: u32, protections: &[u32]) -> Vec { + let mut bytes = vec![0u8; 32]; + bytes[..4].copy_from_slice(&MH_MAGIC_64.to_le_bytes()); + bytes[4..8].copy_from_slice(&cpu.to_le_bytes()); + bytes[12..16].copy_from_slice(&kind.to_le_bytes()); + let count = u32::try_from(protections.len()).expect("command count"); + bytes[16..20].copy_from_slice(&count.to_le_bytes()); + bytes[24..28].copy_from_slice(&flags.to_le_bytes()); + for protection in protections { + let mut command = [0u8; 72]; + command[..4].copy_from_slice(&LC_SEGMENT_64.to_le_bytes()); + command[4..8].copy_from_slice(&72u32.to_le_bytes()); + command[56..60].copy_from_slice(&protection.to_le_bytes()); + bytes.extend_from_slice(&command); + } + bytes + } + + fn evidence(facts: &ImageFacts, key: &str) -> String { + facts + .format_evidence + .iter() + .find(|(name, _)| *name == key) + .map(|(_, value)| value.clone()) + .unwrap_or_default() + } + + #[test] + fn a_hardened_static_elf_reads_as_hardened() { + let headers = [ + (PT_LOAD, 5), + (PT_LOAD, 6), + (PT_GNU_RELRO, 4), + (PT_GNU_STACK, 6), + ]; + let facts = inspect_image(&elf(false, 62, 2, &headers)).expect("elf"); + assert_eq!(facts.object_format, "elf"); + assert_eq!(facts.object_type, "executable"); + assert_eq!(facts.machine, "x86_64"); + assert_eq!(facts.writable_executable_regions, 0); + assert_eq!(evidence(&facts, "relro"), "true"); + assert_eq!(evidence(&facts, "now"), "true"); + assert_eq!(evidence(&facts, "noexecstack"), "true"); + assert_eq!(evidence(&facts, "gnu_stack"), "RW"); + assert_eq!(evidence(&facts, "dynamic_section"), "absent"); + } + + #[test] + fn an_unhardened_elf_says_so() { + let headers = [(PT_LOAD, 7), (PT_DYNAMIC, 6), (PT_GNU_STACK, 7)]; + let facts = inspect_image(&elf(false, 183, 2, &headers)).expect("elf"); + assert_eq!(facts.machine, "aarch64"); + assert_eq!(facts.writable_executable_regions, 1); + assert_eq!(evidence(&facts, "relro"), "false"); + assert_eq!(evidence(&facts, "now"), "false"); + assert_eq!(evidence(&facts, "noexecstack"), "false"); + assert_eq!(evidence(&facts, "dynamic_section"), "present"); + } + + #[test] + fn a_missing_stack_header_is_not_a_safe_stack() { + let facts = inspect_image(&elf(false, 62, 2, &[(PT_LOAD, 5)])).expect("elf"); + assert_eq!(evidence(&facts, "noexecstack"), "false"); + } + + #[test] + fn big_endian_elf_reads_correctly() { + let facts = inspect_image(&elf(true, 22, 1, &[(PT_LOAD, 7)])).expect("s390x"); + assert_eq!(facts.machine, "s390x"); + assert_eq!(facts.object_type, "relocatable"); + assert_eq!(facts.writable_executable_regions, 1); + } + + #[test] + fn a_macho_executable_reads_its_pie_flag_and_regions() { + let image = macho(0x0100_000c, MH_EXECUTE, MH_PIE, &[5, 3]); + let facts = inspect_image(&image).expect("macho"); + assert_eq!(facts.object_format, "macho"); + assert_eq!(facts.object_type, "executable"); + assert_eq!(facts.machine, "aarch64"); + assert_eq!(facts.writable_executable_regions, 0); + assert_eq!(evidence(&facts, "pie"), "true"); + + let open = macho(0x0100_0007, MH_EXECUTE, 0, &[7]); + let facts = inspect_image(&open).expect("macho"); + assert_eq!(facts.machine, "x86_64"); + assert_eq!(facts.writable_executable_regions, 1); + assert_eq!(evidence(&facts, "pie"), "false"); + } + + #[test] + fn a_truncated_or_foreign_file_is_an_error() { + let mut truncated = elf(false, 62, 2, &[(PT_LOAD, 5)]); + truncated.truncate(100); + for bytes in [truncated, vec![0; 8], Vec::new()] { + assert_eq!( + inspect_image(&bytes).map_err(|error| error.code()), + Err(ErrorCode::Schema) + ); + } + let unknown = elf(false, 3, 2, &[]); + assert!(inspect_image(&unknown).is_err(), "32-bit x86 is no target"); + } +} diff --git a/crates/bamts-verification/src/toolchain_measure.rs b/crates/bamts-verification/src/toolchain_measure.rs new file mode 100644 index 00000000..a7fe61bd --- /dev/null +++ b/crates/bamts-verification/src/toolchain_measure.rs @@ -0,0 +1,659 @@ +//! Measures one target cell with real tools and real runs. +//! +//! Every value in a cell comes from something that happened here: the probe is +//! emitted twice through production codegen, linked twice with the committed +//! freestanding support, run natively or under the cell's emulator, and read back +//! from its own headers. What cannot be observed stays a gap for [`generate_cell`] +//! to classify. A measurement never fills one in. + +use std::{ + env, fs, + path::{Path, PathBuf}, + process::{Command, Output}, +}; + +use crate::{ + ErrorCode, Result, VerificationError, + schema::sha256_hex, + toolchain_image::{ImageFacts, inspect_image}, + toolchain_schema::{ + CellClass, HostObservation, RustcPin, TargetCellRecord, cell_class_for, emit_probe_object, + generate_cell, + }, +}; + +/// The Rust source of the image's runtime, with its value still to fill in. +const SUPPORT_SOURCE: &str = include_str!("../probe/support.rs"); + +/// Where Rosetta's runtime lives once it is installed. +const ROSETTA_RUNTIME: &str = "/Library/Apple/usr/libexec/oah/libRosettaRuntime"; + +/// One cell, and what stopped any obligation from being observed. +#[derive(Debug, Clone, PartialEq, Eq)] +pub struct CellMeasurement { + pub record: TargetCellRecord, + pub notes: Vec, +} + +/// Measures `target` from this host, using `work` for its files. +/// +/// # Errors +/// +/// Returns an error when `target` cannot be measured from this host at all, when +/// `rustc` cannot be read, or when `work` cannot be written. A step that fails +/// after that leaves a gap in the record and a line in the notes. +pub fn measure_cell(target: &str, work: &Path) -> Result { + fs::create_dir_all(work).map_err(|error| io_path(work, &error))?; + let (rustc, manifest) = read_rustc()?; + let class = cell_class_for(target, rustc.host())?; + let mut observation = HostObservation::new(rustc).with_toolchain_manifest(manifest); + let notes = match observe(target, class, work, &mut observation) { + Ok(()) => Vec::new(), + Err(error) => vec![error.to_string()], + }; + Ok(CellMeasurement { + record: generate_cell(target, &observation)?, + notes, + }) +} + +/// Records each obligation as it is observed, so a later step that fails leaves +/// what came before it in place. +fn observe( + target: &str, + class: CellClass, + work: &Path, + observed: &mut HostObservation, +) -> Result<()> { + let first = work.join("first"); + let second = work.join("second"); + for build in [&first, &second] { + fs::create_dir_all(build).map_err(|error| io_path(build, &error))?; + } + let probe = emit_probe_object(target, &first.join("probe.o"))?; + let repeat = emit_probe_object(target, &second.join("probe.o"))?; + let object_facts = inspect_image(&read_file(&first.join("probe.o"))?)?; + *observed = observed.clone().with_target_object( + target, + format!( + "command=bamts-verification toolchain object --target {target} --out probe.o; object_sha256={}; {}; bytes={}; entry={}; helpers={}", + probe.sha256, + identity_fields(&object_facts), + probe.byte_len, + probe.entry_symbol, + probe.required_helpers.join(","), + ), + ); + + let linker = find_linker(target).ok_or_else(|| { + VerificationError::new( + ErrorCode::ToolMissing, + format!("no linker for `{target}` was found on PATH"), + ) + })?; + *observed = observed.clone().with_linker(target, linker.identity()?); + compile_support(target, work)?; + let command = linker.link(&first)?; + linker.link(&second)?; + let image = read_file(&first.join("probe"))?; + let executable_sha256 = sha256_hex(&image); + let second_image = sha256_hex(&read_file(&second.join("probe"))?); + let mut facts = inspect_image(&image)?; + if let Some(signature) = linker.signature(&first)? { + facts.format_evidence.push(("signature", signature)); + } + *observed = observed.clone().with_hardened_link( + target, + format!( + "command={command}; executable_sha256={executable_sha256}; {}; {}; {}", + identity_fields(&facts), + format_fields(&facts), + linker.provenance()?, + ), + ); + + let inspector = env::current_exe().map_err(|error| { + VerificationError::new(ErrorCode::Io, format!("cannot name this program: {error}")) + })?; + *observed = observed.clone().with_execmem_guard( + target, + format!( + "command=bamts-verification toolchain measure --target {target}; executable_sha256={executable_sha256}; inspector_sha256={}; writable_executable_regions={}; {}", + sha256_hex(&read_file(&inspector)?), + facts.writable_executable_regions, + format_fields(&facts), + ), + ); + + if probe.sha256 == repeat.sha256 && executable_sha256 == second_image { + *observed = observed.clone().with_reproducible_build( + target, + format!( + "command=bamts-verification toolchain measure --target {target}; first_object_sha256={}; second_object_sha256={}; first_executable_sha256={executable_sha256}; second_executable_sha256={second_image}; byte_identical=true", + probe.sha256, repeat.sha256 + ), + ); + } + + let runner = Runner::for_class(class, target)?; + run_probe(&runner, &first)?; + let evidence = format!( + "command={}; executable_sha256={executable_sha256}; execution={}; exit=0; result=int32:42; fuel=positive{}", + runner.display("./probe"), + runner.execution(target), + runner.identity_fields()?, + ); + *observed = match class { + CellClass::NativeHost => observed.clone().with_native_execution(target, evidence), + CellClass::CrossQemu | CellClass::CrossRosetta => { + observed.clone().with_emulator(target, evidence) + } + }; + Ok(()) +} + +/// Runs the image. A clean exit is how it reports that it returned `int32:42` and +/// charged fuel, so anything else leaves the run unobserved. +fn run_probe(runner: &Runner, directory: &Path) -> Result<()> { + let output = run(runner.command("./probe").current_dir(directory))?; + if output.status.code() == Some(0) { + Ok(()) + } else { + Err(VerificationError::new( + ErrorCode::ToolFailed, + format!("the probe did not finish cleanly: {:?}", output.status), + )) + } +} + +fn identity_fields(facts: &ImageFacts) -> String { + format!( + "object_format={}; object_type={}; machine={}", + facts.object_format, facts.object_type, facts.machine + ) +} + +fn format_fields(facts: &ImageFacts) -> String { + facts + .format_evidence + .iter() + .map(|(key, value)| format!("{key}={value}")) + .collect::>() + .join("; ") +} + +/// The image's runtime, with the value it must see filled in. +fn support_source() -> String { + let int32_42 = bamts_native::Value::int32(42).to_bits(); + SUPPORT_SOURCE.replace("@INT32_42@", &format!("{int32_42:#x}")) +} + +/// Compiles the image's runtime for `target` into `work/support.o`. +fn compile_support(target: &str, work: &Path) -> Result<()> { + let source = work.join("support.rs"); + fs::write(&source, support_source()).map_err(|error| io_path(&source, &error))?; + require_success( + "rustc", + &run(Command::new("rustc") + .args(["--edition=2024", "--crate-type=lib", "--crate-name=support"]) + .args(["--emit=obj", "-Cpanic=abort", "-Copt-level=2"]) + .args(["-Ccodegen-units=1", "--target", target]) + .arg("-o") + .arg(work.join("support.o")) + .arg(&source))?, + ) +} + +/// The linker a target's image is built with. +enum Linker { + Lld { + path: PathBuf, + emulation: &'static str, + }, + Cc { + path: PathBuf, + arch: &'static str, + }, +} + +fn find_linker(target: &str) -> Option { + let elf = |emulation| find_program("ld.lld").map(|path| Linker::Lld { path, emulation }); + let macho = |arch| find_program("cc").map(|path| Linker::Cc { path, arch }); + match target { + "x86_64-unknown-linux-gnu" => elf("elf_x86_64"), + "aarch64-unknown-linux-gnu" => elf("aarch64linux"), + "s390x-unknown-linux-gnu" => elf("elf64_s390"), + "riscv64gc-unknown-linux-gnu" => elf("elf64lriscv"), + "aarch64-apple-darwin" => macho("arm64"), + "x86_64-apple-darwin" => macho("x86_64"), + _ => None, + } +} + +impl Linker { + const fn path(&self) -> &PathBuf { + match self { + Self::Lld { path, .. } | Self::Cc { path, .. } => path, + } + } + + fn version(&self) -> Result { + let output = run(Command::new(self.path()).arg("--version"))?; + require_success("linker", &output)?; + Ok(first_line(&output.stdout)) + } + + fn identity(&self) -> Result { + Ok(format!("linker={}", self.version()?)) + } + + fn provenance(&self) -> Result { + Ok(format!( + "linker_version={}; linker_sha256={}", + self.version()?, + sha256_hex(&read_file(self.path())?) + )) + } + + /// Links `directory/probe`, and returns the command as it should be replayed. + fn link(&self, directory: &Path) -> Result { + let mut command = Command::new(self.path()); + let replay = match self { + Self::Lld { emulation, .. } => { + let args = [ + "-m", + emulation, + "-static", + "-e", + "_start", + "-z", + "relro", + "-z", + "now", + "-z", + "noexecstack", + "-z", + "defs", + "--build-id=none", + "-o", + "probe", + "../support.o", + "probe.o", + ]; + command.args(args); + format!("ld.lld {}", args.join(" ")) + } + Self::Cc { arch, .. } => { + let args = ["-arch", arch, "-o", "probe", "../support.o", "probe.o"]; + command.args(args); + format!("cc {}", args.join(" ")) + } + }; + require_success("linker", &run(command.current_dir(directory))?)?; + if matches!(self, Self::Cc { .. }) { + require_success( + "codesign", + &run(Command::new("codesign") + .args([ + "--force", + "--sign", + "-", + "--identifier", + "bamts-probe", + "probe", + ]) + .current_dir(directory))?, + )?; + } + Ok(replay) + } + + /// The kind of signature `directory/probe` carries, on formats that sign. + fn signature(&self, directory: &Path) -> Result> { + if matches!(self, Self::Lld { .. }) { + return Ok(None); + } + require_success( + "codesign", + &run(Command::new("codesign") + .args(["--verify", "probe"]) + .current_dir(directory))?, + )?; + let output = run(Command::new("codesign") + .args(["--display", "--verbose=2", "probe"]) + .current_dir(directory))?; + let details = String::from_utf8_lossy(&output.stderr); + Ok(Some( + details + .lines() + .find_map(|line| line.strip_prefix("Signature=")) + .unwrap_or("none") + .to_owned(), + )) + } +} + +/// How a cell's image is started. +enum Runner { + Native, + Emulated { + program: PathBuf, + prefix: Vec, + version: String, + identity: PathBuf, + }, +} + +impl Runner { + fn for_class(class: CellClass, target: &str) -> Result { + match class { + CellClass::NativeHost => Ok(Self::Native), + CellClass::CrossRosetta => { + let identity = PathBuf::from(ROSETTA_RUNTIME); + let build = run(Command::new("sw_vers").arg("-buildVersion"))?; + require_success("sw_vers", &build)?; + read_file(&identity)?; + Ok(Self::Emulated { + program: PathBuf::from("/usr/bin/arch"), + prefix: vec!["-x86_64".to_owned()], + version: format!("rosetta on macOS {}", first_line(&build.stdout)), + identity, + }) + } + CellClass::CrossQemu => { + let name = format!("qemu-{}", qemu_arch(target)); + let program = find_program(&format!("{name}-static")) + .or_else(|| find_program(&name)) + .ok_or_else(|| { + VerificationError::new( + ErrorCode::ToolMissing, + format!("`{name}` was not found on PATH"), + ) + })?; + let output = run(Command::new(&program).arg("--version"))?; + require_success(&name, &output)?; + Ok(Self::Emulated { + identity: program.clone(), + program, + prefix: Vec::new(), + version: first_line(&output.stdout), + }) + } + } + } + + fn command(&self, image: &str) -> Command { + match self { + Self::Native => Command::new(image), + Self::Emulated { + program, prefix, .. + } => { + let mut command = Command::new(program); + command.args(prefix).arg(image); + command + } + } + } + + fn display(&self, image: &str) -> String { + match self { + Self::Native => image.to_owned(), + Self::Emulated { + program, prefix, .. + } => { + let mut words = vec![program.display().to_string()]; + words.extend(prefix.iter().cloned()); + words.push(image.to_owned()); + words.join(" ") + } + } + } + + fn execution(&self, target: &str) -> String { + match self { + Self::Native => "native".to_owned(), + Self::Emulated { identity, .. } if identity.ends_with("libRosettaRuntime") => { + "rosetta".to_owned() + } + Self::Emulated { .. } => format!("qemu-{}", qemu_arch(target)), + } + } + + fn identity_fields(&self) -> Result { + match self { + Self::Native => Ok(String::new()), + Self::Emulated { + version, identity, .. + } => Ok(format!( + "; emulator_version={version}; emulator_sha256={}", + sha256_hex(&read_file(identity)?) + )), + } + } +} + +fn qemu_arch(target: &str) -> &str { + let arch = target.split('-').next().unwrap_or(target); + arch.strip_suffix("gc").unwrap_or(arch) +} + +/// Reads `rustc -vV` and the identity of the binary behind it. +fn read_rustc() -> Result<(RustcPin, String)> { + let output = run(Command::new("rustc").arg("-vV"))?; + require_success("rustc", &output)?; + let pin = parse_rustc(&String::from_utf8_lossy(&output.stdout))?; + let sysroot = run(Command::new("rustc").args(["--print", "sysroot"]))?; + require_success("rustc", &sysroot)?; + let binary = PathBuf::from(first_line(&sysroot.stdout)).join("bin/rustc"); + let manifest = format!( + "command=rustc -vV; rustc_version={}; rustc_host={}; rustc_sha256={}", + pin.version(), + pin.host(), + sha256_hex(&read_file(&binary)?) + ); + Ok((pin, manifest)) +} + +fn parse_rustc(text: &str) -> Result { + let field = |name: &str| { + text.lines() + .find_map(|line| line.strip_prefix(name)?.strip_prefix(": ")) + .ok_or_else(|| { + VerificationError::new(ErrorCode::Schema, format!("`rustc -vV` has no `{name}`")) + }) + }; + RustcPin::new( + field("commit-hash")?, + field("commit-date")?, + field("host")?, + field("LLVM version")?, + field("release")?, + ) +} + +fn find_program(name: &str) -> Option { + env::split_paths(&env::var_os("PATH")?) + .map(|directory| directory.join(name)) + .find(|path| path.is_file()) +} + +fn first_line(bytes: &[u8]) -> String { + String::from_utf8_lossy(bytes) + .lines() + .next() + .unwrap_or_default() + .replace("; ", ", ") +} + +fn run(command: &mut Command) -> Result { + command.output().map_err(|error| { + VerificationError::new( + ErrorCode::ToolMissing, + format!("cannot run `{}`: {error}", command.get_program().display()), + ) + }) +} + +fn require_success(tool: &str, output: &Output) -> Result<()> { + if output.status.success() { + Ok(()) + } else { + Err(VerificationError::new( + ErrorCode::ToolFailed, + format!( + "`{tool}` failed: {}", + String::from_utf8_lossy(&output.stderr) + .chars() + .take(500) + .collect::() + ), + )) + } +} + +fn read_file(path: &Path) -> Result> { + fs::read(path).map_err(|error| io_path(path, &error)) +} + +fn io_path(path: &Path, error: &std::io::Error) -> VerificationError { + VerificationError::new(ErrorCode::Io, format!("{}: {error}", path.display())) +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::evidence::TerminalState; + + #[test] + fn rustc_version_output_becomes_a_pin() { + let pin = parse_rustc( + "rustc 1.97.1 (8bab26f4f 2026-07-14)\nbinary: rustc\ncommit-hash: 8bab26f4f68e0e26f0bb7960be334d5b520ea452\ncommit-date: 2026-07-14\nhost: aarch64-apple-darwin\nrelease: 1.97.1\nLLVM version: 22.1.6\n", + ) + .expect("pin"); + assert_eq!(pin.host(), "aarch64-apple-darwin"); + assert_eq!(pin.version(), "1.97.1"); + parse_rustc("rustc 1.97.1\n").expect_err("a field is missing"); + } + + #[test] + fn emulators_are_named_for_the_architecture_they_run() { + assert_eq!(qemu_arch("riscv64gc-unknown-linux-gnu"), "riscv64"); + assert_eq!(qemu_arch("s390x-unknown-linux-gnu"), "s390x"); + assert_eq!(qemu_arch("aarch64-unknown-linux-gnu"), "aarch64"); + } + + #[test] + fn the_support_source_expects_the_boxed_value_of_int32_42() { + let expected = bamts_native::Value::int32(42); + assert_eq!(expected.as_int32(), Some(42)); + let declaration = format!("const INT32_42: u64 = {:#x};", expected.to_bits()); + assert!(support_source().contains(&declaration)); + } + + #[test] + fn an_emulated_run_names_its_emulator_and_hashes_it() { + let identity = + std::env::temp_dir().join(format!("bamts-measure-{}-emulator", std::process::id())); + fs::write(&identity, "emulator").expect("write"); + let digest = sha256_hex(b"emulator"); + let runner = Runner::Emulated { + program: PathBuf::from("/usr/bin/qemu-s390x-static"), + prefix: Vec::new(), + version: "qemu-s390x 8.2".to_owned(), + identity: identity.clone(), + }; + assert_eq!( + runner.identity_fields().expect("identity"), + format!("; emulator_version=qemu-s390x 8.2; emulator_sha256={digest}") + ); + assert_eq!(runner.execution("s390x-unknown-linux-gnu"), "qemu-s390x"); + assert_eq!( + runner.display("./probe"), + "/usr/bin/qemu-s390x-static ./probe" + ); + fs::remove_file(identity).expect("remove"); + + let rosetta = Runner::Emulated { + program: PathBuf::from("/usr/bin/arch"), + prefix: vec!["-x86_64".to_owned()], + version: "rosetta".to_owned(), + identity: PathBuf::from(ROSETTA_RUNTIME), + }; + assert_eq!(rosetta.execution("x86_64-apple-darwin"), "rosetta"); + assert_eq!(rosetta.display("./probe"), "/usr/bin/arch -x86_64 ./probe"); + } + + #[test] + fn a_target_with_no_route_from_this_host_is_refused_not_guessed() { + let work = std::env::temp_dir().join(format!("bamts-measure-{}-norun", std::process::id())); + let foreign = if bamts_platform::host_triple().contains("windows") { + "aarch64-apple-darwin" + } else { + "x86_64-pc-windows-msvc" + }; + measure_cell(foreign, &work).expect_err("no route"); + let _ = fs::remove_dir_all(work); + } + + #[cfg(unix)] + #[test] + fn a_probe_that_exits_nonzero_is_not_an_observed_run() { + use std::os::unix::fs::PermissionsExt; + let directory = + std::env::temp_dir().join(format!("bamts-measure-{}-exit", std::process::id())); + fs::create_dir_all(&directory).expect("scratch"); + let probe = directory.join("probe"); + fs::write(&probe, "#!/bin/sh\nexit 11\n").expect("script"); + fs::set_permissions(&probe, fs::Permissions::from_mode(0o755)).expect("mode"); + let error = run_probe(&Runner::Native, &directory).expect_err("exit 11 is a failure"); + assert_eq!(error.code(), ErrorCode::ToolFailed); + fs::write(&probe, "#!/bin/sh\nexit 0\n").expect("script"); + run_probe(&Runner::Native, &directory).expect("exit 0 is the probe's success"); + fs::remove_dir_all(directory).expect("remove scratch"); + } + + #[test] + fn a_mac_cell_without_rosetta_stops_at_the_emulator() { + if bamts_platform::host_triple() != "aarch64-apple-darwin" + || Path::new(ROSETTA_RUNTIME).exists() + { + return; + } + let work = std::env::temp_dir().join(format!("bamts-measure-{}-mac", std::process::id())); + let measured = measure_cell("x86_64-apple-darwin", &work).expect("measure"); + let _ = fs::remove_dir_all(work); + assert_eq!(measured.notes.len(), 1, "{:?}", measured.notes); + let obligations = measured.record.obligations(); + assert_eq!( + obligations["target-functional"].status(), + TerminalState::ExternalBlocked + ); + assert_eq!( + obligations["target-functional"].missing_artifact(), + "rosetta" + ); + assert_eq!(obligations["hardened-link"].status(), TerminalState::Pass); + } + + #[test] + fn the_host_cell_measures_end_to_end_on_a_host_with_a_linker() { + let host = bamts_platform::host_triple(); + if find_linker(&host).is_none() { + return; + } + let work = std::env::temp_dir().join(format!("bamts-measure-{}-host", std::process::id())); + let measured = measure_cell(&host, &work).expect("measure"); + let _ = fs::remove_dir_all(work); + assert_eq!(measured.notes, Vec::::new()); + assert_eq!(measured.record.triple(), host); + assert_eq!( + measured.record.obligations()["target-functional"].status(), + TerminalState::Pass + ); + assert!(measured.record.is_closed(), "{:?}", measured.record); + measured + .record + .check_committable(Path::new("cell.json")) + .expect("a measured cell passes the committed rules"); + } +} diff --git a/crates/bamts-verification/src/toolchain_schema.rs b/crates/bamts-verification/src/toolchain_schema.rs index eb28d711..c1423d91 100644 --- a/crates/bamts-verification/src/toolchain_schema.rs +++ b/crates/bamts-verification/src/toolchain_schema.rs @@ -1,14 +1,20 @@ //! Target-cell evidence schema and `verification/toolchains/.json` generator. //! -//! The `target-cells` catalog is the cross-product of four triples and seven +//! The `target-cells` catalog is the cross-product of seven triples and seven //! obligations. Each record on disk is one triple. Every obligation is named, //! classified, and either observed or explicitly missing: a cell cannot skip an //! obligation it never looked for, and `PASS` cannot be written over a gap. //! +//! An obligation means the same thing on every target, and what proves it +//! depends on the target's object format: a hardened link is RELRO on ELF, an +//! ad-hoc signed PIE on Mach-O, and ASLR with DEP on COFF. +//! //! Committed E4 records are stricter than intermediate generator values: every //! obligation must be `PASS`, content-addressed, and backed by an explicit //! replay command. `BLOCKING_FAIL` and `EXTERNAL_BLOCKED` remain available only //! while collecting observations and are rejected by the committed-record loader. +//! A cell with no record has not been measured. It is reported as such and is +//! never closed, so [`load_target_cells`] refuses until all seven exist. use std::{ collections::{BTreeMap, BTreeSet}, @@ -32,34 +38,43 @@ use bamts_codegen::{TargetDescriptor, emit_for_target}; pub const TOOLCHAIN_DIR: &str = "verification/toolchains"; /// Schema tag bound into every target-cell record. -pub const TARGET_CELL_SCHEMA: &str = "bamti.target-cell/v1"; +pub const TARGET_CELL_SCHEMA: &str = "bamti.target-cell/v2"; /// Target cells closed by the E4 leaves, in catalog target order. -pub const TARGET_TRIPLES: [&str; 4] = [ +pub const TARGET_TRIPLES: [&str; 7] = [ "x86_64-unknown-linux-gnu", "aarch64-unknown-linux-gnu", "s390x-unknown-linux-gnu", "riscv64gc-unknown-linux-gnu", + "aarch64-apple-darwin", + "x86_64-apple-darwin", + "x86_64-pc-windows-msvc", ]; /// Catalog obligations for every target cell, in extractor order. pub const CELL_OBLIGATIONS: [&str; 7] = [ "toolchain-manifest", - "elf-object", + "target-object", "hardened-link", - "qemu-functional", + "target-functional", "native-runtime", "execmem-guard", "reproducible-build", ]; -const ENVIRONMENT_ARTIFACTS: [&str; 2] = ["cross-linker", "qemu-user"]; +const ENVIRONMENT_ARTIFACTS: [&str; 3] = ["cross-linker", "qemu-user", "rosetta"]; + +/// The pairs of hashes two independent builds must agree on. +const REPRODUCED_HASHES: [(&str, &str); 2] = [ + ("first_object_sha256", "second_object_sha256"), + ("first_executable_sha256", "second_executable_sha256"), +]; /// Schema tag bound into every target-cell record. #[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)] pub enum TargetCellSchema { - #[serde(rename = "bamti.target-cell/v1")] - V1, + #[serde(rename = "bamti.target-cell/v2")] + V2, } impl TargetCellSchema { @@ -75,6 +90,55 @@ impl TargetCellSchema { pub enum CellClass { NativeHost, CrossQemu, + CrossRosetta, +} + +impl CellClass { + /// The environment artifact that runs this cell's images, unless the host does. + const fn emulator(self) -> Option<&'static str> { + match self { + Self::NativeHost => None, + Self::CrossQemu => Some("qemu-user"), + Self::CrossRosetta => Some("rosetta"), + } + } +} + +/// The object format a target links, which decides what proves its obligations. +#[derive(Debug, Clone, Copy, PartialEq, Eq)] +enum ObjectFormat { + Elf, + MachO, + Coff, +} + +impl ObjectFormat { + fn of(triple: &str) -> Self { + if triple.ends_with("-apple-darwin") { + Self::MachO + } else if triple.ends_with("-pc-windows-msvc") { + Self::Coff + } else { + Self::Elf + } + } + + const fn name(self) -> &'static str { + match self { + Self::Elf => "elf", + Self::MachO => "macho", + Self::Coff => "coff", + } + } + + /// The evidence that shows a linked image is hardened on this format. + const fn hardening(self) -> &'static [(&'static str, &'static str)] { + match self { + Self::Elf => &[("relro", "true"), ("now", "true"), ("noexecstack", "true")], + Self::MachO => &[("pie", "true"), ("signature", "adhoc")], + Self::Coff => &[("aslr", "true"), ("dep", "true")], + } + } } /// The compiler identity a cell was evaluated against. @@ -271,26 +335,15 @@ impl TargetCellRecord { )); } self.rustc.validate(path)?; - match self.cell_class { - CellClass::NativeHost if self.rustc.host != self.triple => { - return Err(schema_error( - path, - format!( - "native-host cell `{}` was evaluated by a rustc hosted on `{}`", - self.triple, self.rustc.host - ), - )); - } - CellClass::CrossQemu if self.rustc.host == self.triple => { - return Err(schema_error( - path, - format!( - "cross-qemu cell `{}` is the rustc host and must be native_host", - self.triple - ), - )); - } - _ => {} + let measured_as = cell_class_for(&self.triple, &self.rustc.host)?; + if self.cell_class != measured_as { + return Err(schema_error( + path, + format!( + "cell `{}` evaluated by a rustc hosted on `{}` is {measured_as:?}, not {:?}", + self.triple, self.rustc.host, self.cell_class + ), + )); } let expected = obligation_set(); @@ -336,6 +389,16 @@ impl TargetCellRecord { .all(|obligation| obligation.status == TerminalState::Pass) } + /// Holds this record to the rules a committed record must meet. + /// + /// # Errors + /// + /// Returns a schema error naming the first rule the record breaks. + pub fn check_committable(&self, path: &Path) -> Result<()> { + self.validate(path)?; + self.validate_committed_evidence(path) + } + fn validate_committed_evidence(&self, path: &Path) -> Result<()> { if !self.is_closed() { return Err(schema_error( @@ -347,174 +410,119 @@ impl TargetCellRecord { )); } for (obligation, record) in &self.obligations { - require_evidence_fields( - path, - &self.triple, - obligation, - &record.evidence, - &["command"], - )?; - let required = match obligation.as_str() { - "toolchain-manifest" => &["rustc_version", "rustc_sha256"][..], - "elf-object" => &[ - "object_sha256", - "elf_type", - "elf_machine", - "entry", - "helpers", - ][..], - "hardened-link" => &[ - "executable_sha256", - "elf_type", - "elf_machine", - "linker_version", - "linker_sha256", - ][..], - "qemu-functional" | "native-runtime" => { - &["executable_sha256", "execution", "exit", "result", "fuel"][..] - } - "execmem-guard" => &[ - "executable_sha256", - "writable_executable_load_segments", - "gnu_stack", - "dynamic_section", - "readelf_sha256", - ][..], - "reproducible-build" => &[ - "first_object_sha256", - "second_object_sha256", - "byte_identical", - ][..], - _ => unreachable!("obligation set was validated"), - }; - let expected_machine = match self.triple.as_str() { - "x86_64-unknown-linux-gnu" => "Advanced Micro Devices X86-64", - "aarch64-unknown-linux-gnu" => "AArch64", - "s390x-unknown-linux-gnu" => "IBM S/390", - "riscv64gc-unknown-linux-gnu" => "RISC-V", - _ => unreachable!("target set was validated"), - }; - if matches!(obligation.as_str(), "elf-object" | "hardened-link") { - require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - "elf_machine", - expected_machine, - )?; + let triple = self.triple.as_str(); + let evidence = record.evidence.as_str(); + let rules = self.evidence_rules(obligation); + require_evidence_fields(path, triple, obligation, evidence, &["command"])?; + require_evidence_fields(path, triple, obligation, evidence, &rules.fields)?; + for (field, expected) in rules.values { + require_evidence_value(path, triple, obligation, evidence, field, expected)?; } - require_evidence_fields(path, &self.triple, obligation, &record.evidence, required)?; - if matches!(obligation.as_str(), "qemu-functional" | "native-runtime") { - match self.cell_class { - CellClass::NativeHost => require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - "execution", - "native", - )?, - CellClass::CrossQemu => { - require_evidence_fields( - path, - &self.triple, - obligation, - &record.evidence, - &["emulator_version", "emulator_sha256"], - )?; - let expected_execution = match self.triple.as_str() { - "aarch64-unknown-linux-gnu" => "qemu-aarch64", - "s390x-unknown-linux-gnu" => "qemu-s390x", - "riscv64gc-unknown-linux-gnu" => "qemu-riscv64", - _ => unreachable!("cross target set was validated"), - }; - require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - "execution", - expected_execution, - )?; - } + if obligation == "reproducible-build" { + require_equal_hashes(path, triple, obligation, evidence)?; + } + validate_evidence_hashes(path, triple, obligation, evidence)?; + } + Ok(()) + } + + /// What one obligation's evidence must state for this cell. + fn evidence_rules(&self, obligation: &str) -> EvidenceRules { + let format = ObjectFormat::of(&self.triple); + let image = |object_type| { + vec![ + ("object_format", format.name()), + ("object_type", object_type), + ("machine", machine_name(&self.triple)), + ] + }; + match obligation { + "toolchain-manifest" => EvidenceRules::fields(&["rustc_version", "rustc_sha256"]), + "target-object" => EvidenceRules { + fields: vec!["object_sha256", "entry", "helpers"], + values: image("relocatable"), + }, + "hardened-link" => { + let mut values = image("executable"); + values.extend_from_slice(format.hardening()); + EvidenceRules { + fields: vec!["executable_sha256", "linker_version", "linker_sha256"], + values, } - require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - "exit", - "0", - )?; - require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - "result", - "int32:42", - )?; } - if obligation == "elf-object" { - require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - "elf_type", - "REL", - )?; - } else if obligation == "hardened-link" { - require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - "elf_type", - "EXEC", - )?; - } else if obligation == "execmem-guard" { - for (field, expected) in [ - ("writable_executable_load_segments", "0"), - ("gnu_stack", "RW"), - ("dynamic_section", "absent"), - ] { - require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - field, - expected, - )?; + "target-functional" | "native-runtime" => self.execution_rules(), + "execmem-guard" => { + let mut values = vec![("writable_executable_regions", "0")]; + if format == ObjectFormat::Elf { + values.extend([("gnu_stack", "RW"), ("dynamic_section", "absent")]); } - } else if obligation == "reproducible-build" { - require_evidence_value( - path, - &self.triple, - obligation, - &record.evidence, - "byte_identical", - "true", - )?; - let first = - evidence_value(&record.evidence, "first_object_sha256").expect("required"); - let second = - evidence_value(&record.evidence, "second_object_sha256").expect("required"); - if first != second { - return Err(schema_error( - path, - format!( - "{}::{obligation} claims byte identity for unequal object hashes", - self.triple - ), - )); + EvidenceRules { + fields: vec!["executable_sha256", "inspector_sha256"], + values, } } - validate_evidence_hashes(path, &self.triple, obligation, &record.evidence)?; + "reproducible-build" => EvidenceRules { + fields: REPRODUCED_HASHES + .iter() + .flat_map(|(first, second)| [*first, *second]) + .collect(), + values: vec![("byte_identical", "true")], + }, + _ => unreachable!("obligation set was validated"), + } + } + + /// Rules shared by the two obligations that run a linked image. + fn execution_rules(&self) -> EvidenceRules { + let execution = match self.cell_class { + CellClass::NativeHost => "native", + CellClass::CrossRosetta => "rosetta", + CellClass::CrossQemu => match self.triple.as_str() { + "aarch64-unknown-linux-gnu" => "qemu-aarch64", + "s390x-unknown-linux-gnu" => "qemu-s390x", + "riscv64gc-unknown-linux-gnu" => "qemu-riscv64", + _ => unreachable!("cross target set was validated"), + }, + }; + let mut fields = vec!["executable_sha256"]; + if self.cell_class.emulator().is_some() { + fields.extend(["emulator_version", "emulator_sha256"]); + } + EvidenceRules { + fields, + values: vec![ + ("execution", execution), + ("exit", "0"), + ("result", "int32:42"), + ("fuel", "positive"), + ], } - Ok(()) + } +} + +/// The fields one obligation's evidence must carry, and the values some must hold. +struct EvidenceRules { + fields: Vec<&'static str>, + values: Vec<(&'static str, &'static str)>, +} + +impl EvidenceRules { + fn fields(fields: &[&'static str]) -> Self { + Self { + fields: fields.to_vec(), + values: Vec::new(), + } + } +} + +/// The architecture a linked image must declare, named the same on every format. +fn machine_name(triple: &str) -> &'static str { + match triple.split('-').next() { + Some("x86_64") => "x86_64", + Some("aarch64") => "aarch64", + Some("s390x") => "s390x", + Some("riscv64gc") => "riscv64", + _ => unreachable!("target set was validated"), } } @@ -567,6 +575,18 @@ fn require_evidence_value( } } +fn require_equal_hashes(path: &Path, triple: &str, obligation: &str, evidence: &str) -> Result<()> { + for (first, second) in REPRODUCED_HASHES { + if evidence_value(evidence, first) != evidence_value(evidence, second) { + return Err(schema_error( + path, + format!("{triple}::{obligation} claims byte identity for unequal `{first}`"), + )); + } + } + Ok(()) +} + fn validate_evidence_hashes( path: &Path, triple: &str, @@ -606,13 +626,14 @@ fn validate_evidence_hashes( #[derive(Debug, Clone, PartialEq, Eq)] pub struct HostObservation { rustc: RustcPin, - elf_objects: BTreeMap, + emulators: BTreeMap, execmem_guards: BTreeMap, hardened_links: BTreeMap, linkers: BTreeMap, native_executions: BTreeMap, - qemu_user: BTreeMap, reproducible_builds: BTreeMap, + target_objects: BTreeMap, + toolchain_manifest: Option, } impl HostObservation { @@ -620,16 +641,24 @@ impl HostObservation { pub fn new(rustc: RustcPin) -> Self { Self { rustc, - elf_objects: BTreeMap::new(), + emulators: BTreeMap::new(), execmem_guards: BTreeMap::new(), hardened_links: BTreeMap::new(), linkers: BTreeMap::new(), native_executions: BTreeMap::new(), - qemu_user: BTreeMap::new(), reproducible_builds: BTreeMap::new(), + target_objects: BTreeMap::new(), + toolchain_manifest: None, } } + /// Replaces the plain compiler identity with content-addressed evidence. + #[must_use] + pub fn with_toolchain_manifest(mut self, evidence: impl Into) -> Self { + self.toolchain_manifest = Some(evidence.into()); + self + } + #[must_use] pub fn with_linker(mut self, triple: impl Into, identity: impl Into) -> Self { self.linkers.insert(triple.into(), identity.into()); @@ -637,22 +666,18 @@ impl HostObservation { } #[must_use] - pub fn with_qemu_user( - mut self, - triple: impl Into, - identity: impl Into, - ) -> Self { - self.qemu_user.insert(triple.into(), identity.into()); + pub fn with_emulator(mut self, triple: impl Into, identity: impl Into) -> Self { + self.emulators.insert(triple.into(), identity.into()); self } #[must_use] - pub fn with_elf_object( + pub fn with_target_object( mut self, triple: impl Into, identity: impl Into, ) -> Self { - self.elf_objects.insert(triple.into(), identity.into()); + self.target_objects.insert(triple.into(), identity.into()); self } @@ -710,7 +735,7 @@ pub fn catalog_identifier(triple: &str, obligation: &str) -> String { format!("{triple}::{obligation}") } -/// The 28 catalog identifiers in sorted order. +/// The 49 catalog identifiers in sorted order. #[must_use] pub fn catalog_identifiers() -> Vec { let mut identifiers = Vec::with_capacity(TARGET_TRIPLES.len() * CELL_OBLIGATIONS.len()); @@ -725,9 +750,14 @@ pub fn catalog_identifiers() -> Vec { /// Class for `triple` given the rustc that evaluated it. /// +/// Every cell can be measured on its own host. The other routes are the ones that +/// can run its images: `qemu-user` on a Linux host for the three cross Linux cells, +/// and Rosetta on Apple Silicon for Intel macOS. +/// /// # Errors /// -/// Returns [`ErrorCode::Schema`] when `triple` is not a declared target cell. +/// Returns [`ErrorCode::Schema`] when `triple` is not a declared target cell, or +/// when a rustc on `rustc_host` has no way to run this cell's images. pub fn cell_class_for(triple: &str, rustc_host: &str) -> Result { if !TARGET_TRIPLES.contains(&triple) { return Err(VerificationError::new( @@ -735,10 +765,17 @@ pub fn cell_class_for(triple: &str, rustc_host: &str) -> Result { format!("`{triple}` is not a declared target cell"), )); } - if triple == rustc_host { - Ok(CellClass::NativeHost) - } else { - Ok(CellClass::CrossQemu) + match (triple, rustc_host) { + _ if triple == rustc_host => Ok(CellClass::NativeHost), + ("x86_64-apple-darwin", "aarch64-apple-darwin") => Ok(CellClass::CrossRosetta), + ( + "aarch64-unknown-linux-gnu" | "s390x-unknown-linux-gnu" | "riscv64gc-unknown-linux-gnu", + _, + ) if rustc_host.ends_with("-unknown-linux-gnu") => Ok(CellClass::CrossQemu), + _ => Err(VerificationError::new( + ErrorCode::Schema, + format!("cell `{triple}` cannot be measured from a rustc hosted on `{rustc_host}`"), + )), } } @@ -761,22 +798,25 @@ pub fn generate_cell(triple: &str, observation: &HostObservation) -> Result"))?; Ok(record) } -/// Build every declared target cell from one observation. +/// Build every target cell this observation's host can measure. /// /// # Errors /// /// Returns the first [`generate_cell`] failure. pub fn generate_cells(observation: &HostObservation) -> Result> { + let host = observation.rustc.host(); let mut cells = BTreeMap::new(); for triple in TARGET_TRIPLES { - cells.insert(triple.to_owned(), generate_cell(triple, observation)?); + if cell_class_for(triple, host).is_ok() { + cells.insert(triple.to_owned(), generate_cell(triple, observation)?); + } } Ok(cells) } @@ -799,6 +839,29 @@ pub fn encode_record(record: &TargetCellRecord) -> Result> { Ok(bytes) } +/// Writes `record` to its committed path, if it meets the committed rules. +/// +/// # Errors +/// +/// Returns a schema error for a record [`load_committed_cells`] would refuse, and +/// an I/O error when the file cannot be written. +pub fn write_committed_record( + root: &Path, + record: &TargetCellRecord, +) -> Result { + let path = root + .join(TOOLCHAIN_DIR) + .join(format!("{}.json", record.triple)); + record.check_committable(&path)?; + let bytes = encode_record(record)?; + let io = |error: std::io::Error| { + VerificationError::new(ErrorCode::Io, format!("{}: {error}", path.display())) + }; + std::fs::create_dir_all(root.join(TOOLCHAIN_DIR)).map_err(io)?; + std::fs::write(&path, bytes).map_err(io)?; + Ok(path) +} + /// Parse and validate one record. Duplicate JSON keys are rejected by the /// shared JSON reader so a shadowed status cannot survive into the record. pub fn parse_record(path: &Path, bytes: &[u8]) -> Result { @@ -906,10 +969,37 @@ pub fn emit_probe_object(target: &str, output: &Path) -> Result { /// Load every target cell. A missing file is an error, never an empty cell. pub fn load_target_cells(root: &Path) -> Result> { + let cells = load_committed_cells(root)?; + let unmeasured = unmeasured_cells(&cells); + if unmeasured.is_empty() { + Ok(cells) + } else { + Err(VerificationError::new( + ErrorCode::Io, + format!( + "{} target cells have no committed record: {}", + unmeasured.len(), + unmeasured.join(", ") + ), + )) + } +} + +/// Load the target cells that have a record, each held to the committed rules. +/// A record that exists and fails is an error; only an absent file is unmeasured. +pub fn load_committed_cells(root: &Path) -> Result> { let directory = root.join(TOOLCHAIN_DIR); let mut cells = BTreeMap::new(); for triple in TARGET_TRIPLES { let path = directory.join(format!("{triple}.json")); + if !path.try_exists().map_err(|error| { + VerificationError::new( + ErrorCode::Io, + format!("cannot read {}: {error}", path.display()), + ) + })? { + continue; + } let bytes = read_bytes(&path)?; let record = parse_record(&path, &bytes)?; record.validate_committed_evidence(&path)?; @@ -921,24 +1011,18 @@ pub fn load_target_cells(root: &Path) -> Result = cells - .values() - .filter(|record| matches!(record.cell_class, CellClass::NativeHost)) - .map(TargetCellRecord::triple) - .collect(); - if native.len() != 1 { - return Err(schema_error( - &directory, - format!( - "expected exactly one native-host cell, found {}", - native.len() - ), - )); - } Ok(cells) } +/// Declared cells with no record in `cells`, in catalog target order. +#[must_use] +pub fn unmeasured_cells(cells: &BTreeMap) -> Vec<&'static str> { + TARGET_TRIPLES + .into_iter() + .filter(|triple| !cells.contains_key(*triple)) + .collect() +} + /// Obligations that are not `PASS`, in catalog-identifier order. pub fn blocking_obligations( cells: &BTreeMap, @@ -966,16 +1050,19 @@ fn generate_obligation( ) -> ObligationEvidence { match obligation { "toolchain-manifest" => pass( - observation.rustc.identity(), + observation + .toolchain_manifest + .clone() + .unwrap_or_else(|| observation.rustc.identity()), "Recorded from rustc -vV on this host.", ), - "elf-object" => match observation.elf_objects.get(triple) { + "target-object" => match observation.target_objects.get(triple) { Some(identity) => pass( identity.clone(), "Stored relocatable AOT object for this triple.", ), None => blocking( - "elf-object", + "target-object", "", "No relocatable AOT object is stored for this triple.", "Emit a BamTS AOT object for this triple and record its content hash.", @@ -987,14 +1074,12 @@ fn generate_obligation( identity.clone(), "Stored hardened link artifact for this triple.", ) - } else if matches!(class, CellClass::CrossQemu) - && !observation.linkers.contains_key(triple) - { + } else if class.emulator().is_some() && !observation.linkers.contains_key(triple) { blocked_external( "cross-linker", "", - "No cross GNU linker for this triple is installed.", - "Install the matching *-linux-gnu-gcc linker, perform a hardened link, and record the binary identity.", + "No cross linker for this triple is installed.", + "Install the linker for this triple, perform a hardened link, and record the binary identity.", ) } else { let evidence = observation.linkers.get(triple).cloned().unwrap_or_default(); @@ -1006,34 +1091,34 @@ fn generate_obligation( ) } } - "qemu-functional" => match class { - CellClass::NativeHost => match observation.native_executions.get(triple) { + "target-functional" => match class.emulator() { + None => match observation.native_executions.get(triple) { Some(identity) => pass( identity.clone(), "Native-host functional execution completed without emulation.", ), None => blocking( - "qemu-functional", + "target-functional", "", "No native functional execution is stored for this host triple.", "Execute the linked BamTS image natively and record the functional result.", ), }, - CellClass::CrossQemu => match observation.qemu_user.get(triple) { + Some(emulator) => match observation.emulators.get(triple) { Some(identity) => pass( identity.clone(), - "qemu-user executed a linked image for this triple.", + "An emulator executed a linked image for this triple.", ), None => blocked_external( - "qemu-user", + emulator, "", - "qemu-user for this triple is not installed.", - "Install qemu-user for this triple, execute the linked image, and record qemu version and the run identity.", + "The emulator for this triple is not installed.", + "Install the emulator for this triple, execute the linked image, and record its version and the run identity.", ), }, }, - "native-runtime" => match class { - CellClass::NativeHost => match observation.native_executions.get(triple) { + "native-runtime" => match class.emulator() { + None => match observation.native_executions.get(triple) { Some(identity) => pass( identity.clone(), "This process runs natively on the cell triple.", @@ -1045,16 +1130,16 @@ fn generate_obligation( "Execute a linked BamTS image natively on this host and record the run identity.", ), }, - CellClass::CrossQemu => match observation.qemu_user.get(triple) { + Some(emulator) => match observation.emulators.get(triple) { Some(identity) => pass( identity.clone(), - "qemu-user executed the target runtime image for this triple.", + "An emulator executed the target runtime image for this triple.", ), None => blocked_external( - "qemu-user", + emulator, "", - "qemu-user for this triple is not installed.", - "Install qemu-user for this triple, execute the target runtime image, and record the run identity.", + "The emulator for this triple is not installed.", + "Install the emulator for this triple, execute the target runtime image, and record the run identity.", ), }, }, @@ -1201,10 +1286,10 @@ mod tests { #[test] fn catalog_identifiers_match_the_target_cells_cross_product() { let identifiers = catalog_identifiers(); - assert_eq!(identifiers.len(), 28); - assert_eq!(identifiers[0], "aarch64-unknown-linux-gnu::elf-object"); + assert_eq!(identifiers.len(), 49); + assert_eq!(identifiers[0], "aarch64-apple-darwin::execmem-guard"); assert_eq!( - identifiers[27], + identifiers[48], "x86_64-unknown-linux-gnu::toolchain-manifest" ); let expected: BTreeSet = TARGET_TRIPLES @@ -1233,11 +1318,11 @@ mod tests { TerminalState::Pass ); assert_eq!( - native.obligations()["qemu-functional"].status(), + native.obligations()["target-functional"].status(), TerminalState::Pass ); assert_eq!( - native.obligations()["elf-object"].status(), + native.obligations()["target-object"].status(), TerminalState::BlockingFail ); assert_eq!( @@ -1248,11 +1333,11 @@ mod tests { let aarch64 = cells.get("aarch64-unknown-linux-gnu").expect("cross cell"); assert_eq!(aarch64.cell_class(), CellClass::CrossQemu); assert_eq!( - aarch64.obligations()["qemu-functional"].status(), + aarch64.obligations()["target-functional"].status(), TerminalState::ExternalBlocked ); assert_eq!( - aarch64.obligations()["qemu-functional"].missing_artifact(), + aarch64.obligations()["target-functional"].missing_artifact(), "qemu-user" ); assert_eq!( @@ -1264,19 +1349,19 @@ mod tests { TerminalState::ExternalBlocked ); assert_eq!( - aarch64.obligations()["elf-object"].status(), + aarch64.obligations()["target-object"].status(), TerminalState::BlockingFail ); } #[test] - fn pass_cannot_be_claimed_for_a_missing_elf_object() { + fn pass_cannot_be_claimed_for_a_missing_target_object() { let cell = generate_cell("x86_64-unknown-linux-gnu", &hosted()).expect("native generates"); let mut json = serde_json::to_value(&cell).expect("serialize"); - json["obligations"]["elf-object"]["status"] = serde_json::json!("PASS"); - json["obligations"]["elf-object"]["missing_artifact"] = serde_json::json!(""); - json["obligations"]["elf-object"]["unblock_condition"] = serde_json::json!(""); - json["obligations"]["elf-object"]["evidence"] = serde_json::json!(""); + json["obligations"]["target-object"]["status"] = serde_json::json!("PASS"); + json["obligations"]["target-object"]["missing_artifact"] = serde_json::json!(""); + json["obligations"]["target-object"]["unblock_condition"] = serde_json::json!(""); + json["obligations"]["target-object"]["evidence"] = serde_json::json!(""); let bytes = serde_json::to_vec(&json).expect("bytes"); let error = parse_record(Path::new("cell.json"), &bytes) .expect_err("PASS requires evidence and forbids a gap"); @@ -1287,10 +1372,10 @@ mod tests { fn in_repository_gap_cannot_be_external_blocked() { let cell = generate_cell("x86_64-unknown-linux-gnu", &hosted()).expect("native generates"); let mut json = serde_json::to_value(&cell).expect("serialize"); - json["obligations"]["elf-object"]["status"] = serde_json::json!("EXTERNAL_BLOCKED"); + json["obligations"]["target-object"]["status"] = serde_json::json!("EXTERNAL_BLOCKED"); let bytes = serde_json::to_vec(&json).expect("bytes"); let error = parse_record(Path::new("cell.json"), &bytes) - .expect_err("elf-object is not an environment artifact"); + .expect_err("target-object is not an environment artifact"); assert_eq!(error.code(), ErrorCode::Schema); } @@ -1298,7 +1383,7 @@ mod tests { fn missing_qemu_on_a_cross_cell_must_be_external_blocked() { let cell = generate_cell("s390x-unknown-linux-gnu", &hosted()).expect("cross generates"); let mut json = serde_json::to_value(&cell).expect("serialize"); - json["obligations"]["qemu-functional"]["status"] = serde_json::json!("BLOCKING_FAIL"); + json["obligations"]["target-functional"]["status"] = serde_json::json!("BLOCKING_FAIL"); let bytes = serde_json::to_vec(&json).expect("bytes"); let error = parse_record(Path::new("cell.json"), &bytes) .expect_err("missing qemu-user is external"); @@ -1320,7 +1405,7 @@ mod tests { #[test] fn complete_native_cell_closes_only_with_every_hosted_artifact() { let observation = hosted() - .with_elf_object("x86_64-unknown-linux-gnu", "elf:1") + .with_target_object("x86_64-unknown-linux-gnu", "object:1") .with_hardened_link("x86_64-unknown-linux-gnu", "link:1") .with_execmem_guard("x86_64-unknown-linux-gnu", "guard:1") .with_reproducible_build("x86_64-unknown-linux-gnu", "repro:1"); @@ -1328,7 +1413,7 @@ mod tests { generate_cell("x86_64-unknown-linux-gnu", &observation).expect("complete native"); assert!(cell.is_closed()); assert_eq!( - cell.obligations()["qemu-functional"].status(), + cell.obligations()["target-functional"].status(), TerminalState::Pass ); } @@ -1365,9 +1450,395 @@ mod tests { #[test] fn committed_records_are_content_addressed_and_closed() { let root = Path::new(env!("CARGO_MANIFEST_DIR")).join("../.."); - let cells = load_target_cells(&root).expect("committed cells load"); - assert_eq!(cells.len(), TARGET_TRIPLES.len()); + let cells = load_committed_cells(&root).expect("committed cells load"); + assert_eq!( + cells.len() + unmeasured_cells(&cells).len(), + TARGET_TRIPLES.len() + ); assert!(cells.values().all(TargetCellRecord::is_closed)); assert!(blocking_obligations(&cells).is_empty()); } + + /// Triple, rustc host, machine, and how images run, for every route to a cell. + const ROUTES: [(&str, &str, &str, &str); 9] = [ + ( + "x86_64-unknown-linux-gnu", + "x86_64-unknown-linux-gnu", + "x86_64", + "native", + ), + ( + "aarch64-unknown-linux-gnu", + "aarch64-unknown-linux-gnu", + "aarch64", + "native", + ), + ( + "aarch64-unknown-linux-gnu", + "x86_64-unknown-linux-gnu", + "aarch64", + "qemu-aarch64", + ), + ( + "s390x-unknown-linux-gnu", + "aarch64-unknown-linux-gnu", + "s390x", + "qemu-s390x", + ), + ( + "riscv64gc-unknown-linux-gnu", + "aarch64-unknown-linux-gnu", + "riscv64", + "qemu-riscv64", + ), + ( + "aarch64-apple-darwin", + "aarch64-apple-darwin", + "aarch64", + "native", + ), + ( + "x86_64-apple-darwin", + "aarch64-apple-darwin", + "x86_64", + "rosetta", + ), + ( + "x86_64-apple-darwin", + "x86_64-apple-darwin", + "x86_64", + "native", + ), + ( + "x86_64-pc-windows-msvc", + "x86_64-pc-windows-msvc", + "x86_64", + "native", + ), + ]; + + /// Evidence values no rule fixes, so a changed one is still valid evidence. + const FREE_FORM_FIELDS: [&str; 6] = [ + "command", + "emulator_version", + "entry", + "helpers", + "linker_version", + "rustc_version", + ]; + + fn pin(host: &str) -> RustcPin { + RustcPin::new( + "8bab26f4f68e0e26f0bb7960be334d5b520ea452", + "2026-07-14", + host, + "22.1.6", + "1.97.1", + ) + .expect("test rustc pin") + } + + /// Closed evidence written out per format, not read back from the rules it tests. + fn closed_evidence(route: (&str, &str, &str, &str), obligation: &str) -> String { + let (triple, _, machine, execution) = route; + let hash = "a".repeat(64); + let (format, hardening, guard) = if triple.ends_with("-apple-darwin") { + ("macho", "pie=true; signature=adhoc", "") + } else if triple.ends_with("-pc-windows-msvc") { + ("coff", "aslr=true; dep=true", "") + } else { + ( + "elf", + "relro=true; now=true; noexecstack=true", + "; gnu_stack=RW; dynamic_section=absent", + ) + }; + let emulator = if execution == "native" { + String::new() + } else { + format!("; emulator_version=1.0; emulator_sha256={hash}") + }; + match obligation { + "toolchain-manifest" => { + format!("command=rustc -vV; rustc_version=1.97.1; rustc_sha256={hash}") + } + "target-object" => format!( + "command=emit; object_sha256={hash}; object_format={format}; object_type=relocatable; machine={machine}; entry=bamts_m0_fn_0; helpers=bamts_load_constant" + ), + "hardened-link" => format!( + "command=link; executable_sha256={hash}; object_format={format}; object_type=executable; machine={machine}; linker_version=1.0; linker_sha256={hash}; {hardening}" + ), + "target-functional" | "native-runtime" => format!( + "command=run; executable_sha256={hash}; execution={execution}; exit=0; result=int32:42; fuel=positive{emulator}" + ), + "execmem-guard" => format!( + "command=inspect; executable_sha256={hash}; inspector_sha256={hash}; writable_executable_regions=0{guard}" + ), + "reproducible-build" => format!( + "command=build twice; first_object_sha256={hash}; second_object_sha256={hash}; first_executable_sha256={hash}; second_executable_sha256={hash}; byte_identical=true" + ), + other => panic!("no fixture for {other}"), + } + } + + fn closed_cell(route: (&str, &str, &str, &str)) -> TargetCellRecord { + let (triple, host, _, _) = route; + let obligations = CELL_OBLIGATIONS + .iter() + .map(|obligation| { + let evidence = closed_evidence(route, obligation); + ((*obligation).to_owned(), pass(evidence, "observed")) + }) + .collect(); + TargetCellRecord { + cell_class: cell_class_for(triple, host).expect("route is measurable"), + obligations, + rustc: pin(host), + schema: TargetCellSchema::V2, + triple: triple.to_owned(), + } + } + + fn committed_check( + mut cell: TargetCellRecord, + obligation: &str, + evidence: String, + ) -> Result<()> { + cell.obligations + .get_mut(obligation) + .expect("declared obligation") + .evidence = evidence; + let path = Path::new("cell.json"); + cell.validate(path)?; + cell.validate_committed_evidence(path) + } + + #[test] + fn every_route_closes_on_the_evidence_its_format_states() { + for route in ROUTES { + let cell = closed_cell(route); + cell.validate(Path::new("cell.json")).expect("valid record"); + cell.validate_committed_evidence(Path::new("cell.json")) + .unwrap_or_else(|error| panic!("{route:?}: {error}")); + } + } + + #[test] + fn dropping_any_evidence_field_rejects_the_record() { + for route in ROUTES { + for obligation in CELL_OBLIGATIONS { + let evidence = closed_evidence(route, obligation); + for field in evidence.split("; ") { + let kept: Vec<&str> = evidence + .split("; ") + .filter(|other| *other != field) + .collect(); + committed_check(closed_cell(route), obligation, kept.join("; ")) + .expect_err(&format!("{route:?} {obligation} must require `{field}`")); + } + } + } + } + + #[test] + fn changing_any_fixed_evidence_value_rejects_the_record() { + for route in ROUTES { + for obligation in CELL_OBLIGATIONS { + let evidence = closed_evidence(route, obligation); + for field in evidence.split("; ") { + let (name, _) = field.split_once('=').expect("key=value"); + if FREE_FORM_FIELDS.contains(&name) { + continue; + } + let changed = evidence.replace(field, &format!("{name}=wrong")); + committed_check(closed_cell(route), obligation, changed) + .expect_err(&format!("{route:?} {obligation} must fix `{name}`")); + } + } + } + } + + #[test] + fn two_builds_that_differ_are_not_reproducible() { + let route = ROUTES[5]; + let evidence = closed_evidence(route, "reproducible-build"); + for second in ["second_object_sha256", "second_executable_sha256"] { + let changed = evidence.replace( + &format!("{second}={}", "a".repeat(64)), + &format!("{second}={}", "b".repeat(64)), + ); + assert_ne!(changed, evidence); + committed_check(closed_cell(route), "reproducible-build", changed) + .expect_err("byte identity needs equal hashes"); + } + } + + #[test] + fn a_cell_is_measured_only_from_a_host_that_can_run_its_images() { + let linux_arm = "aarch64-unknown-linux-gnu"; + let linux_x64 = "x86_64-unknown-linux-gnu"; + let mac_arm = "aarch64-apple-darwin"; + let mac_x64 = "x86_64-apple-darwin"; + let windows = "x86_64-pc-windows-msvc"; + for triple in TARGET_TRIPLES { + assert_eq!( + cell_class_for(triple, triple).ok(), + Some(CellClass::NativeHost) + ); + } + assert_eq!( + cell_class_for(mac_x64, mac_arm).ok(), + Some(CellClass::CrossRosetta) + ); + assert_eq!( + cell_class_for(linux_arm, linux_x64).ok(), + Some(CellClass::CrossQemu) + ); + assert_eq!( + cell_class_for("s390x-unknown-linux-gnu", linux_arm).ok(), + Some(CellClass::CrossQemu) + ); + for (triple, host) in [ + (linux_x64, linux_arm), + (linux_arm, mac_arm), + (mac_arm, mac_x64), + (mac_arm, linux_x64), + (windows, mac_arm), + (windows, linux_x64), + ] { + assert_eq!( + cell_class_for(triple, host) + .map_err(|error| error.code()) + .err(), + Some(ErrorCode::Schema), + "{triple} from {host}" + ); + } + } + + #[test] + fn a_record_cannot_claim_a_route_its_host_does_not_have() { + let mut cell = closed_cell(ROUTES[6]); + cell.cell_class = CellClass::CrossQemu; + assert_eq!( + cell.validate(Path::new("cell.json")) + .map_err(|error| error.code()), + Err(ErrorCode::Schema) + ); + + let mut cell = closed_cell(ROUTES[8]); + cell.rustc = pin("x86_64-unknown-linux-gnu"); + cell.validate(Path::new("cell.json")) + .expect_err("Windows has no cross route"); + } + + #[test] + fn missing_rosetta_blocks_an_intel_mac_cell_on_the_environment() { + let observation = HostObservation::new(pin("aarch64-apple-darwin")); + let cell = generate_cell("x86_64-apple-darwin", &observation).expect("rosetta route"); + assert_eq!(cell.cell_class(), CellClass::CrossRosetta); + for obligation in ["target-functional", "native-runtime"] { + assert_eq!( + cell.obligations()[obligation].status(), + TerminalState::ExternalBlocked + ); + assert_eq!(cell.obligations()[obligation].missing_artifact(), "rosetta"); + } + } + + #[test] + fn a_host_generates_only_the_cells_it_can_measure() { + let linux = generate_cells(&hosted()).expect("cells generate"); + assert_eq!( + linux.keys().map(String::as_str).collect::>(), + [ + "aarch64-unknown-linux-gnu", + "riscv64gc-unknown-linux-gnu", + "s390x-unknown-linux-gnu", + "x86_64-unknown-linux-gnu", + ] + ); + let mac = generate_cells(&HostObservation::new(pin("aarch64-apple-darwin"))) + .expect("cells generate"); + assert_eq!( + mac.keys().map(String::as_str).collect::>(), + ["aarch64-apple-darwin", "x86_64-apple-darwin"] + ); + } + + fn scratch_root(name: &str) -> std::path::PathBuf { + let root = + std::env::temp_dir().join(format!("bamts-target-cells-{}-{name}", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + std::fs::create_dir_all(root.join(TOOLCHAIN_DIR)).expect("scratch directory"); + root + } + + fn write_record(root: &Path, file_triple: &str, bytes: &[u8]) { + let path = root.join(TOOLCHAIN_DIR).join(format!("{file_triple}.json")); + std::fs::write(path, bytes).expect("write record"); + } + + #[test] + fn writing_a_closed_record_makes_it_load_and_an_open_one_is_refused() { + let root = + std::env::temp_dir().join(format!("bamts-target-cells-{}-write", std::process::id())); + let _ = std::fs::remove_dir_all(&root); + let closed = closed_cell(ROUTES[5]); + write_committed_record(&root, &closed).expect("the directory is created"); + assert_eq!( + load_committed_cells(&root).expect("loads")[closed.triple()], + closed + ); + + let open = generate_cell("x86_64-unknown-linux-gnu", &hosted()).expect("open cell"); + write_committed_record(&root, &open).expect_err("an open record is not committed"); + assert!( + !root + .join(TOOLCHAIN_DIR) + .join("x86_64-unknown-linux-gnu.json") + .exists() + ); + std::fs::remove_dir_all(root).expect("remove scratch"); + } + + #[test] + fn an_absent_record_is_unmeasured_and_never_closed() { + let root = scratch_root("absent"); + let cell = closed_cell(ROUTES[5]); + write_record(&root, cell.triple(), &encode_record(&cell).expect("encode")); + + let cells = load_committed_cells(&root).expect("one record loads"); + assert_eq!(cells.keys().collect::>(), [cell.triple()]); + assert_eq!(unmeasured_cells(&cells).len(), 6); + assert!(unmeasured_cells(&cells).contains(&"x86_64-pc-windows-msvc")); + + let error = load_target_cells(&root).expect_err("six cells are unmeasured"); + assert_eq!(error.code(), ErrorCode::Io); + assert!(error.to_string().contains("x86_64-pc-windows-msvc")); + std::fs::remove_dir_all(root).expect("remove scratch"); + } + + #[test] + fn a_present_record_that_fails_is_not_read_as_unmeasured() { + let closed = closed_cell(ROUTES[5]); + let encoded = String::from_utf8(encode_record(&closed).expect("encode")).expect("utf8"); + + let stale = scratch_root("stale"); + let v1 = encoded.replace("bamti.target-cell/v2", "bamti.target-cell/v1"); + write_record(&stale, closed.triple(), v1.as_bytes()); + load_committed_cells(&stale).expect_err("an old schema is not an absent record"); + std::fs::remove_dir_all(stale).expect("remove scratch"); + + let open = scratch_root("open"); + let cell = generate_cell("x86_64-unknown-linux-gnu", &hosted()).expect("open cell"); + write_record(&open, cell.triple(), &encode_record(&cell).expect("encode")); + load_committed_cells(&open).expect_err("committed records must be closed"); + std::fs::remove_dir_all(open).expect("remove scratch"); + + let renamed = scratch_root("renamed"); + write_record(&renamed, "x86_64-apple-darwin", encoded.as_bytes()); + load_committed_cells(&renamed).expect_err("a record sits under its own name"); + std::fs::remove_dir_all(renamed).expect("remove scratch"); + } } diff --git a/crates/bamts-verification/src/workspace_guard.rs b/crates/bamts-verification/src/workspace_guard.rs index 16bcc0f7..113d083f 100644 --- a/crates/bamts-verification/src/workspace_guard.rs +++ b/crates/bamts-verification/src/workspace_guard.rs @@ -8,9 +8,10 @@ use std::path::{Path, PathBuf}; use std::process::Command; use toml::{Table, Value}; -const MEMBERS: [(&str, &str); 11] = [ +const MEMBERS: [(&str, &str); 12] = [ ("bamts-compiler", "crates/bamts-compiler"), ("bamts-cancel", "crates/bamts-cancel"), + ("bamts-platform", "crates/bamts-platform"), ("bamts-bytecode", "crates/bamts-bytecode"), ("bamts-runtime", "crates/bamts-runtime"), ("bamts-native", "crates/bamts-native"), @@ -336,7 +337,12 @@ fn validate_member_lints(name: &str, manifest: &Value, context: &str) -> Result< // `deny` policy instead of inheriting the workspace-wide `forbid`. if matches!( name, - "bamts-native" | "bamts-node" | "bamts-codegen" | "bamts-napi" | "bamts-verification" + "bamts-native" + | "bamts-node" + | "bamts-codegen" + | "bamts-napi" + | "bamts-verification" + | "bamts-platform" ) { if lints.contains_key("workspace") { return Err(workspace_error(format!( @@ -363,7 +369,10 @@ fn validate_member_lints(name: &str, manifest: &Value, context: &str) -> Result< "deny", &format!("{context} [lints.rust]"), )?; - if matches!(name, "bamts-codegen" | "bamts-napi" | "bamts-verification") { + if matches!( + name, + "bamts-codegen" | "bamts-napi" | "bamts-verification" | "bamts-platform" + ) { require_exact_string( rust, "unsafe_op_in_unsafe_fn", @@ -993,6 +1002,7 @@ fn expected_internal_graph() -> InternalGraph { let mut graph = BTreeMap::new(); graph.insert("bamts-bytecode".to_owned(), BTreeMap::new()); graph.insert("bamts-cancel".to_owned(), BTreeMap::new()); + graph.insert("bamts-platform".to_owned(), BTreeMap::new()); graph.insert( "bamts-native".to_owned(), BTreeMap::from([("bamts-bytecode".to_owned(), internal_dependency(false, &[]))]), @@ -1013,6 +1023,7 @@ fn expected_internal_graph() -> InternalGraph { "bamts-native".to_owned(), internal_dependency(false, &["gc"]), ), + ("bamts-platform".to_owned(), internal_dependency(false, &[])), ]), ); graph.insert( @@ -1031,6 +1042,7 @@ fn expected_internal_graph() -> InternalGraph { BTreeMap::from([ ("bamts-bytecode".to_owned(), internal_dependency(false, &[])), ("bamts-cancel".to_owned(), internal_dependency(false, &[])), + ("bamts-platform".to_owned(), internal_dependency(false, &[])), ("bamts-runtime".to_owned(), internal_dependency(false, &[])), ("bamts-native".to_owned(), internal_dependency(true, &[])), ]), @@ -1088,6 +1100,7 @@ fn expected_internal_graph() -> InternalGraph { "bamts-node".to_owned(), internal_dependency(false, &["script-compiler"]), ), + ("bamts-platform".to_owned(), internal_dependency(false, &[])), ("bamts-runtime".to_owned(), internal_dependency(false, &[])), ]), ); diff --git a/crates/bamts-verification/tests/class_lowering_order.rs b/crates/bamts-verification/tests/class_lowering_order.rs index fac810da..0fd20113 100644 --- a/crates/bamts-verification/tests/class_lowering_order.rs +++ b/crates/bamts-verification/tests/class_lowering_order.rs @@ -2,7 +2,7 @@ use std::fs; use std::path::{Path, PathBuf}; use std::process::Command; use std::sync::Arc; -use std::time::{SystemTime, UNIX_EPOCH}; +use std::sync::atomic::{AtomicU64, Ordering}; use bamts_compiler::emitter::transforms::ScriptTarget; use bamts_compiler::emitter::transpile::transpile_text; @@ -13,11 +13,14 @@ struct FixtureDir(PathBuf); impl FixtureDir { fn new() -> Self { - let nonce = SystemTime::now() - .duration_since(UNIX_EPOCH) - .expect("system clock is after the Unix epoch") - .as_nanos(); - let path = std::env::temp_dir().join(format!("bamts-class-lowering-{nonce}")); + // The process id and a process-wide counter keep parallel tests from + // sharing a fixture directory name, which a clock reading could not. + static NEXT_FIXTURE: AtomicU64 = AtomicU64::new(0); + let nonce = NEXT_FIXTURE.fetch_add(1, Ordering::Relaxed); + let path = std::env::temp_dir().join(format!( + "bamts-class-lowering-{}-{nonce}", + std::process::id() + )); fs::create_dir(&path).expect("create class lowering fixture directory"); Self(path) } diff --git a/crates/bamts-verification/tests/corpus_differential.rs b/crates/bamts-verification/tests/corpus_differential.rs index 770e0c55..b4876410 100644 --- a/crates/bamts-verification/tests/corpus_differential.rs +++ b/crates/bamts-verification/tests/corpus_differential.rs @@ -7,6 +7,14 @@ use bamts_verification::corpus::{ }; use sha2::{Digest, Sha256}; +/// The repository's pinned `tsc`, launched through `node` because the +/// `node_modules/.bin/tsc` shim is a shell script that Windows cannot run. +fn tsc_command(root: &Path) -> process::Command { + let mut command = process::Command::new("node"); + command.arg(root.join("node_modules/typescript/bin/tsc")); + command +} + fn local_test_provenance(root: &Path, source_dir: &str) -> Provenance { fn collect(dir: &Path, prefix: &str, records: &mut Vec<(String, Vec)>) { for entry in fs::read_dir(dir).expect("read source_dir") { @@ -1357,7 +1365,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -1455,7 +1463,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -1541,7 +1549,7 @@ try { "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -1635,7 +1643,7 @@ try { "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -1861,7 +1869,7 @@ console.log(`fn:${Item(5)};ns:${Item.label};tag:${Item.tag()}`); "--outDir".into(), format!("{source_dir}/oracles"), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2014,7 +2022,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2119,7 +2127,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2225,7 +2233,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2331,7 +2339,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2440,7 +2448,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2559,7 +2567,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2642,7 +2650,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2750,7 +2758,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2843,7 +2851,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -2942,7 +2950,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -3070,7 +3078,7 @@ console.log(trace.join('\n')); "--checkJs".into(), "false".into(), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -3180,7 +3188,7 @@ fn sync_explicit_resource_management_matches_tsc_oracle_in_every_execution_mode( "--outDir".into(), format!("{source_dir}/oracles"), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -3315,7 +3323,7 @@ fn async_explicit_resource_management_matches_tsc_oracle_in_every_execution_mode "--outDir".into(), format!("{source_dir}/oracles"), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -3414,7 +3422,7 @@ fn sync_explicit_resource_management_suppression_matches_tsc_oracle_in_every_exe "--outDir".into(), format!("{source_dir}/oracles"), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -3583,7 +3591,7 @@ fn resource_management_abrupt_completion_matches_tsc_oracle_in_every_execution_m "--outDir".into(), format!("{source_dir}/oracles"), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) @@ -3686,7 +3694,7 @@ fn async_explicit_resource_management_suppression_matches_tsc_oracle_in_every_ex "--outDir".into(), format!("{source_dir}/oracles"), ]; - let status = process::Command::new(root.join("node_modules/.bin/tsc")) + let status = tsc_command(&root) .arg(&source_path) .args(&tsc_args) .current_dir(&root) diff --git a/crates/bamts-verification/tests/lane_serving.rs b/crates/bamts-verification/tests/lane_serving.rs new file mode 100644 index 00000000..e954c504 --- /dev/null +++ b/crates/bamts-verification/tests/lane_serving.rs @@ -0,0 +1,167 @@ +//! A serving compiler lane worker answers many requests from one process, and +//! gives each request the response and the failure of a one-request worker. + +use std::{ + collections::BTreeSet, + fs, + io::{BufRead, BufReader, Write}, + path::{Path, PathBuf}, + process::{Command, Stdio}, +}; + +use bamts_verification::{ + evidence::WorkingDirectoryPolicy, + lane::{ + LANE_WORKER_DONE, LANE_WORKER_READY, LANE_WORKER_REQUEST, LANE_WORKER_SERVE, LaneBinding, + LaneRequest, LaneResponse, ProcessExecutor, ServingExecutor, run_lane, + }, + shard::{ExecutionMode, ObligationKey}, +}; + +const WORKER: &str = env!("CARGO_BIN_EXE_ts_lane_worker"); +const RUN_ID: &str = "cccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccccc"; + +fn scratch(label: &str) -> PathBuf { + let nanos = std::time::SystemTime::now() + .duration_since(std::time::UNIX_EPOCH) + .expect("time") + .as_nanos(); + let dir = std::env::temp_dir().join(format!( + "bamts-lane-serving-{label}-{}-{nanos}", + std::process::id() + )); + fs::create_dir_all(&dir).expect("scratch"); + dir +} + +/// A request whose binding names no snapshot, which a worker answers without one. +fn unbound_request(request_id: u64) -> LaneRequest { + LaneRequest::new( + LaneBinding::unbound(RUN_ID).expect("binding"), + request_id, + ObligationKey::new( + "typescript-7.0.2", + format!("compiler/tests/cases/compiler/case{request_id}.ts"), + "default", + ExecutionMode::Aot, + "x86_64-unknown-linux-gnu", + ) + .expect("key"), + BTreeSet::from(["diagnostics".to_owned()]), + vec!["lane".to_owned()], + WorkingDirectoryPolicy::RepositoryRoot, + 30_000, + ) + .expect("request") +} + +fn write_request(dir: &Path, request: &LaneRequest) -> PathBuf { + let path = dir.join(format!("lane-{RUN_ID}-{}.json", request.request_id())); + fs::write(&path, serde_json::to_vec(request).expect("encode")).expect("write request"); + path +} + +fn response(request_path: &Path) -> LaneResponse { + let body = fs::read(request_path.with_extension("response.json")).expect("response"); + serde_json::from_slice(&body).expect("decode response") +} + +fn one_request_worker(dir: &Path, request_path: &Path) -> std::process::Output { + Command::new(WORKER) + .current_dir(dir) + .env_remove(LANE_WORKER_SERVE) + .env(LANE_WORKER_REQUEST, request_path) + .output() + .expect("one-request worker") +} + +fn serving_worker(dir: &Path) -> std::process::Child { + Command::new(WORKER) + .current_dir(dir) + .env_remove(LANE_WORKER_REQUEST) + .env(LANE_WORKER_SERVE, "1") + .stdin(Stdio::piped()) + .stdout(Stdio::piped()) + .stderr(Stdio::piped()) + .spawn() + .expect("serving worker") +} + +#[test] +fn a_serving_worker_answers_each_request_like_a_one_request_worker() { + let dir = scratch("answers"); + let single = write_request(&dir, &unbound_request(1)); + assert!(one_request_worker(&dir, &single).status.success()); + let expected = response(&single); + + let mut child = serving_worker(&dir); + let mut stdin = child.stdin.take().expect("stdin"); + let mut lines = BufReader::new(child.stdout.take().expect("stdout")) + .lines() + .map(|line| line.expect("line")) + .filter(|line| !line.is_empty()); + assert_eq!(lines.next().as_deref(), Some(LANE_WORKER_READY)); + for request_id in [2, 3] { + let path = write_request(&dir, &unbound_request(request_id)); + writeln!(stdin, "{}", path.display()).expect("send request"); + stdin.flush().expect("flush request"); + assert_eq!( + lines.next(), + Some(format!("{LANE_WORKER_DONE} {request_id}")) + ); + let answered = response(&path); + assert_eq!(answered.request_id(), request_id); + assert_eq!(answered.binding(), expected.binding()); + assert_eq!(answered.outcome(), expected.outcome()); + } + drop(stdin); + let status = child.wait().expect("worker exits"); + let _ = fs::remove_dir_all(&dir); + assert!(status.success(), "{status}"); +} + +#[test] +fn the_serving_executor_records_the_rows_of_one_worker_per_request() { + let dir = scratch("executors"); + let requests: Vec = (1..=3).map(unbound_request).collect(); + let mut one_per_request = ProcessExecutor::new(WORKER, Vec::new(), &dir, dir.join("one")); + let expected = run_lane(&mut one_per_request, &requests).expect("one worker per request"); + let mut serving = ServingExecutor::new(WORKER, Vec::new(), &dir, dir.join("serving")); + let served = run_lane(&mut serving, &requests).expect("one serving worker"); + let started = serving.workers_started(); + drop(serving); + let _ = fs::remove_dir_all(&dir); + assert_eq!(started, 1, "one worker answers every request"); + assert_eq!(served.len(), expected.len()); + for (served, expected) in served.iter().zip(&expected) { + assert_eq!(served.key(), expected.key()); + assert_eq!(served.state(), expected.state()); + assert_eq!(served.artifacts(), expected.artifacts()); + assert_eq!(served.detail(), expected.detail()); + } +} + +#[test] +fn a_serving_worker_ends_on_an_unreadable_request_like_a_one_request_worker() { + let dir = scratch("unreadable"); + let missing = dir.join(format!("lane-{RUN_ID}-9.json")); + let expected = one_request_worker(&dir, &missing); + assert_eq!(expected.status.code(), Some(1)); + + let mut child = serving_worker(&dir); + let mut ready = String::new(); + BufReader::new(child.stdout.take().expect("stdout")) + .read_line(&mut ready) + .expect("ready line"); + assert_eq!(ready.trim_end(), LANE_WORKER_READY); + let mut stdin = child.stdin.take().expect("stdin"); + writeln!(stdin, "{}", missing.display()).expect("send request"); + drop(stdin); + let served = child.wait_with_output().expect("worker exits"); + let _ = fs::remove_dir_all(&dir); + assert_eq!(served.status.code(), Some(1)); + assert_eq!( + String::from_utf8_lossy(&served.stderr), + String::from_utf8_lossy(&expected.stderr) + ); +} diff --git a/crates/bamts/src/lib.rs b/crates/bamts/src/lib.rs index 18cce0bc..9f87aba9 100644 --- a/crates/bamts/src/lib.rs +++ b/crates/bamts/src/lib.rs @@ -15,11 +15,15 @@ use bamts_cancel::{CancellationToken, Cancelled}; use bamts_compiler::diagnostic::{Diagnostic, DiagnosticSeverity}; use bamts_compiler::lint::{LintProfile, LintTable}; use bamts_compiler::lower::LowerOptions; +use bamts_compiler::parser::parse; use bamts_compiler::pipeline::{FrontendMode, compile_program_frontend_with_cancel}; use bamts_compiler::program::{ ProgramLoadError, ProgramLoader, ProgramLowerError, ResolvedProgram, lower_program_with_cancel, }; +use bamts_compiler::project::canonical::canonicalize; use bamts_compiler::project::{ConfigError, ProjectConfig, ProjectRoot}; +use bamts_compiler::scanner::scan; +use bamts_compiler::source::ScriptKind; pub use bamts_bytecode as bytecode; pub use bamts_compiler as compiler; @@ -186,7 +190,37 @@ pub fn compile_source_file_with_cancel( path: impl AsRef, cancel: &CancellationToken, ) -> Result { - let path = canonical_entrypoint_with_cancel(path.as_ref(), cancel)?; + compile_source_file_with_lints(path, &LintTable::new(LintProfile::Default), cancel) +} + +/// Compiles an entrypoint and its complete local module graph under `lints`. +pub fn compile_source_file_with_lints( + path: impl AsRef, + lints: &LintTable, + cancel: &CancellationToken, +) -> Result { + compile_entry(path.as_ref(), lints, false, cancel) +} + +/// Compiles an entrypoint the host runs as an ECMAScript Script. Its top-level +/// `var` and function names live on the global object, and module syntax in it +/// is an error. A file cannot say this of itself, which is why it is a separate +/// entry: Node runs the same bytes as a CommonJS module. +pub fn compile_script_file( + path: impl AsRef, + lints: &LintTable, + cancel: &CancellationToken, +) -> Result { + compile_entry(path.as_ref(), lints, true, cancel) +} + +fn compile_entry( + path: &Path, + lints: &LintTable, + script_entry: bool, + cancel: &CancellationToken, +) -> Result { + let path = canonical_entrypoint_with_cancel(path, cancel)?; let fallback_root = path.parent().map(Path::to_path_buf).ok_or_else(|| { Error::ProgramLoad(ProgramLoadError::InvalidRoot(std::io::Error::new( std::io::ErrorKind::InvalidInput, @@ -197,7 +231,7 @@ pub fn compile_source_file_with_cancel( let frontend = compile_program_frontend_with_cancel( &resolved.program, FrontendMode::Check, - &LintTable::new(LintProfile::Default), + lints, cancel, )?; let diagnostics = frontend @@ -210,17 +244,51 @@ pub fn compile_source_file_with_cancel( if !diagnostics.is_empty() { return Err(Error::Diagnostics { diagnostics }); } + let diagnostics = early_errors(&resolved.program, script_entry); + if !diagnostics.is_empty() { + return Err(Error::Diagnostics { diagnostics }); + } lower_program_with_cancel( &resolved.program, &frontend, LowerOptions { javascript_compatibility: true, + script_entry, }, cancel, ) .map_err(Error::from) } +/// The early errors of every JavaScript module about to run. TypeScript sources +/// answer to the checker instead. +fn early_errors(program: &ResolvedProgram, script_entry: bool) -> Vec { + let entrypoint = program.entrypoint().source_id(); + program + .modules() + .iter() + .filter(|module| { + matches!( + module.script_kind(), + ScriptKind::JavaScript | ScriptKind::JavaScriptReact + ) + }) + .flat_map(|module| { + let scanned = scan( + module.source_id(), + module.script_kind(), + module.source().clone(), + ); + let parsed = parse(scanned); + if script_entry && module.source_id() == entrypoint { + bamts_compiler::early_errors::check_script(parsed.product()) + } else { + bamts_compiler::early_errors::check(parsed.product()) + } + }) + .collect() +} + /// Runs an entrypoint and its complete local module graph with the deterministic /// Node-compatible host. /// @@ -303,7 +371,7 @@ pub fn canonical_entrypoint_with_cancel( .map_err(|error| Error::ProgramLoad(ProgramLoadError::InvalidRoot(error)))? }; cancel.check()?; - let canonical = fs::canonicalize(&absolute).map_err(|source| { + let canonical = canonicalize(&absolute).map_err(|source| { Error::ProgramLoad(ProgramLoadError::Read { path: absolute, source, @@ -409,7 +477,7 @@ pub fn resolve_project_with_cancel( ) -> Result { let project = discover_project_with_cancel(entrypoint, fallback_root, cancel)?; cancel.check()?; - let canonical_root = fs::canonicalize(project.root()) + let canonical_root = canonicalize(project.root()) .map_err(|error| Error::ProgramLoad(ProgramLoadError::InvalidRoot(error)))?; cancel.check()?; let root = ProjectRoot::new(canonical_root).map_err(|error| { @@ -445,9 +513,9 @@ pub fn resolve_project_with_cancel( #[cfg(test)] mod tests { - use super::compile_source_file; #[cfg(feature = "node-host")] use super::run_program; + use super::{canonicalize, compile_source_file}; use std::error::Error; use std::path::{Path, PathBuf}; @@ -529,7 +597,7 @@ mod tests { std::fs::remove_file(directory.join("sub/tsconfig.json"))?; let error = compile_source_file(&entrypoint).expect_err("the root tsconfig.json must be parsed"); - let expected = std::fs::canonicalize(directory.join("tsconfig.json"))?; + let expected = canonicalize(directory.join("tsconfig.json"))?; assert!(matches!( error, super::Error::ProjectConfig { ref path, .. } if *path == expected diff --git a/docs/dev/ast_change_window_protocol.md b/docs/dev/ast_change_window_protocol.md index c39ed9af..ca0c4b02 100644 --- a/docs/dev/ast_change_window_protocol.md +++ b/docs/dev/ast_change_window_protocol.md @@ -54,6 +54,16 @@ Example: } ``` +### Single-worker procedure + +When one worker runs alone, no hub exists. Replace the hub messages with `.outline/change-window.log`: + +1. Before the first edit, append `ACQUIRE_AST_WINDOW: owner= files= pre_hashes= reason=`. Compute hashes with `shasum -a 256 `. +2. Do not acquire if the log's last entry for an overlapping file is an unreleased `ACQUIRE_AST_WINDOW`. Stop and escalate instead. +3. After the last verified edit, append `RELEASE_AST_WINDOW: owner= files= post_hashes=`. + +Fresh reads, append-only discriminants, UTF-16 rules, and diagnostic bands apply unchanged. + ## Content Hashes and Fresh Reads Hash every announced file before the first edit and after the last verified edit. Do not reuse a hash from a previous wave. diff --git a/docs/explanation/architecture.md b/docs/explanation/architecture.md index cc2d99d6..4a464aba 100644 --- a/docs/explanation/architecture.md +++ b/docs/explanation/architecture.md @@ -80,6 +80,7 @@ flowchart TD - If the matching native package is missing or fails verification, artifact loading fails closed with an explicit error (`NativeArtifactNotFoundError` or `NativeArtifactLoadError`). It never falls back to downloading binaries or spawning the CLI. - Cancellation and environment teardown ownership: - `bamts-cancel` owns atomic cancellation handles and execution interrupt triggers. + - `bamts-platform` owns host facts and process observation: host triple, machine facts, process memory regions, and peak memory. OS-specific reads stay in this crate. - `bamts-napi` (private crate) owns N-API native bindings, isolate memory management, and environment teardown. ### 6. Validation diff --git a/docs/release/runbook.md b/docs/release/runbook.md index 7ebddc4f..f3c0f8a5 100644 --- a/docs/release/runbook.md +++ b/docs/release/runbook.md @@ -155,7 +155,7 @@ EVIDENCE: receipt= sha256=<64-hex-digest> ### Global completeness ledger -`proof/completeness-ledger.json` (schema `bamti.completeness-ledger/v1`) is generated from the canonical receipt set. Do not hand-edit it. +`proof/completeness-ledger.json` (schema `bamti.completeness-ledger/v2`) is generated from the canonical receipt set. Do not hand-edit it. ```bash cargo run -p bamts-verification -- ledger rebuild --write diff --git a/npm/test/bamti-native-runner.test.mjs b/npm/test/bamti-native-runner.test.mjs deleted file mode 100644 index 18cd1885..00000000 --- a/npm/test/bamti-native-runner.test.mjs +++ /dev/null @@ -1,135 +0,0 @@ -import assert from "node:assert/strict"; -import { EventEmitter } from "node:events"; -import test from "node:test"; - -import { - deliverOutcome, - encodeRequest, - runNative, -} from "../bamti/native-runner.js"; - -function fakeProcess(platform = "linux") { - return { - platform, - env: { INHERITED: "yes" }, - cwd: () => "/default", - stderr: new RecordingStream("stderr"), - stdout: new RecordingStream("stdout"), - }; -} - -class RecordingStream extends EventEmitter { - constructor(name, events = []) { - super(); - this.name = name; - this.events = events; - } - - write(bytes, callback) { - this.events.push(`${this.name}:${Buffer.from(bytes).toString("utf8")}`); - queueMicrotask(callback); - return false; - } -} - -const addon = { - run: async () => ({ exitCode: 7, stdout: Buffer.from("out"), stderr: Buffer.from("err") }), -}; - -test("run validates only the argument-list shape synchronously", async () => { - let loads = 0; - const loadNativeAddon = () => { - loads += 1; - return addon; - }; - assert.throws( - () => runNative("check", {}, { loadNativeAddon }), - /expects an array/, - ); - assert.equal(loads, 0); - - const promise = runNative([42], {}, { loadNativeAddon, process: fakeProcess() }); - assert.equal(loads, 1); - await assert.rejects(promise, /argument 0 must be a string/); -}); - -test("native package load failures remain synchronous", () => { - const failure = new Error("native unavailable"); - assert.throws( - () => runNative([], {}, { loadNativeAddon: () => { throw failure; } }), - (error) => error === failure, - ); -}); - -test("request encoding snapshots cwd and replaces lone surrogates", () => { - const processObject = fakeProcess(); - const { request, stdio } = encodeRequest(["\uD800"], {}, processObject); - assert.equal(stdio, "inherit"); - assert.equal(request.cwd.toString(), "/default"); - assert.equal(request.args[0].toString("hex"), "efbfbd"); - assert.deepEqual(request.env.map((entry) => entry.toString()), ["INHERITED=yes"]); -}); - -test("environment serialization follows Node ordering and first-equals rules", () => { - const unix = encodeRequest([], { - cwd: "/tmp", - env: { "A=B": "value", OMIT: undefined, NUMBER: 3 }, - }, fakeProcess()).request; - assert.deepEqual( - unix.env.map((entry) => entry.toString()), - ["A=B=value", "NUMBER=3"], - ); - - const windows = encodeRequest([], { - cwd: "C:\\work", - env: { Path: "first", PATH: "second", alpha: "1" }, - }, fakeProcess("win32")).request; - assert.deepEqual( - windows.env.map((entry) => entry.toString()), - ["PATH=second", "alpha=1"], - ); -}); - -test("NUL and invalid stdio failures reject the returned promise", async () => { - const dependencies = { loadNativeAddon: () => addon, process: fakeProcess() }; - await assert.rejects(runNative(["bad\0arg"], {}, dependencies), /NUL/); - await assert.rejects(runNative([], { cwd: "bad\0cwd" }, dependencies), /NUL/); - await assert.rejects(runNative([], { env: { BAD: "x\0y" } }, dependencies), /NUL/); - await assert.rejects(runNative([], { stdio: "wrong" }, dependencies), /stdio/); -}); - -test("inherit writes stderr then stdout and honors completion callbacks", async () => { - const events = []; - const processObject = fakeProcess(); - processObject.stderr = new RecordingStream("stderr", events); - processObject.stdout = new RecordingStream("stdout", events); - const code = await runNative([], {}, { - loadNativeAddon: () => addon, - process: processObject, - }); - assert.equal(code, 7); - assert.deepEqual(events, ["stderr:err", "stdout:out"]); -}); - -test("stream failure maps to exit code one and stops later output", async () => { - const events = []; - const processObject = fakeProcess(); - processObject.stderr = new RecordingStream("stderr", events); - processObject.stderr.write = (_bytes, callback) => { - events.push("stderr:failed"); - queueMicrotask(() => callback(new Error("closed"))); - return false; - }; - processObject.stdout = new RecordingStream("stdout", events); - assert.equal(await deliverOutcome(await addon.run(), "inherit", processObject), 1); - assert.deepEqual(events, ["stderr:failed"]); -}); - -test("ignore and pipe discard native buffers", async () => { - const processObject = fakeProcess(); - processObject.stderr.write = () => { throw new Error("must not write"); }; - processObject.stdout.write = () => { throw new Error("must not write"); }; - const outcome = await addon.run(); - assert.equal(await deliverOutcome(outcome, "ignore", processObject), 7); - assert.equal(await deliverOutcome(outcome, "pipe", processObject), 7); -}); diff --git a/verification/catalog-inputs.json b/verification/catalog-inputs.json index 2c7af6e3..0ab1a9d1 100644 --- a/verification/catalog-inputs.json +++ b/verification/catalog-inputs.json @@ -393,9 +393,9 @@ "kind": "cross-product", "obligations": [ "toolchain-manifest", - "elf-object", + "target-object", "hardened-link", - "qemu-functional", + "target-functional", "native-runtime", "execmem-guard", "reproducible-build" @@ -404,38 +404,62 @@ "x86_64-unknown-linux-gnu", "aarch64-unknown-linux-gnu", "s390x-unknown-linux-gnu", - "riscv64gc-unknown-linux-gnu" + "riscv64gc-unknown-linux-gnu", + "aarch64-apple-darwin", + "x86_64-apple-darwin", + "x86_64-pc-windows-msvc" ] }, "id": "target-cells", "identifiers": [ - "aarch64-unknown-linux-gnu::elf-object", + "aarch64-apple-darwin::execmem-guard", + "aarch64-apple-darwin::hardened-link", + "aarch64-apple-darwin::native-runtime", + "aarch64-apple-darwin::reproducible-build", + "aarch64-apple-darwin::target-functional", + "aarch64-apple-darwin::target-object", + "aarch64-apple-darwin::toolchain-manifest", "aarch64-unknown-linux-gnu::execmem-guard", "aarch64-unknown-linux-gnu::hardened-link", "aarch64-unknown-linux-gnu::native-runtime", - "aarch64-unknown-linux-gnu::qemu-functional", "aarch64-unknown-linux-gnu::reproducible-build", + "aarch64-unknown-linux-gnu::target-functional", + "aarch64-unknown-linux-gnu::target-object", "aarch64-unknown-linux-gnu::toolchain-manifest", - "riscv64gc-unknown-linux-gnu::elf-object", "riscv64gc-unknown-linux-gnu::execmem-guard", "riscv64gc-unknown-linux-gnu::hardened-link", "riscv64gc-unknown-linux-gnu::native-runtime", - "riscv64gc-unknown-linux-gnu::qemu-functional", "riscv64gc-unknown-linux-gnu::reproducible-build", + "riscv64gc-unknown-linux-gnu::target-functional", + "riscv64gc-unknown-linux-gnu::target-object", "riscv64gc-unknown-linux-gnu::toolchain-manifest", - "s390x-unknown-linux-gnu::elf-object", "s390x-unknown-linux-gnu::execmem-guard", "s390x-unknown-linux-gnu::hardened-link", "s390x-unknown-linux-gnu::native-runtime", - "s390x-unknown-linux-gnu::qemu-functional", "s390x-unknown-linux-gnu::reproducible-build", + "s390x-unknown-linux-gnu::target-functional", + "s390x-unknown-linux-gnu::target-object", "s390x-unknown-linux-gnu::toolchain-manifest", - "x86_64-unknown-linux-gnu::elf-object", + "x86_64-apple-darwin::execmem-guard", + "x86_64-apple-darwin::hardened-link", + "x86_64-apple-darwin::native-runtime", + "x86_64-apple-darwin::reproducible-build", + "x86_64-apple-darwin::target-functional", + "x86_64-apple-darwin::target-object", + "x86_64-apple-darwin::toolchain-manifest", + "x86_64-pc-windows-msvc::execmem-guard", + "x86_64-pc-windows-msvc::hardened-link", + "x86_64-pc-windows-msvc::native-runtime", + "x86_64-pc-windows-msvc::reproducible-build", + "x86_64-pc-windows-msvc::target-functional", + "x86_64-pc-windows-msvc::target-object", + "x86_64-pc-windows-msvc::toolchain-manifest", "x86_64-unknown-linux-gnu::execmem-guard", "x86_64-unknown-linux-gnu::hardened-link", "x86_64-unknown-linux-gnu::native-runtime", - "x86_64-unknown-linux-gnu::qemu-functional", "x86_64-unknown-linux-gnu::reproducible-build", + "x86_64-unknown-linux-gnu::target-functional", + "x86_64-unknown-linux-gnu::target-object", "x86_64-unknown-linux-gnu::toolchain-manifest" ] }, diff --git a/verification/completion-program.toml b/verification/completion-program.toml index 1c0756eb..6b4b068d 100644 --- a/verification/completion-program.toml +++ b/verification/completion-program.toml @@ -392,6 +392,16 @@ aspects = ["contract", "evidence", "coverage", "regression", "mutation"] catalogs = ["typescript-7.0.2"] mutation_required = true +[[leaf]] +id = "B7.2" +cluster = "B7" +wave = "W4" +ledger = ".outline/gates/B7.2.md" +owns = ["crates/bamts-verification/src/historical.rs"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = ["typescript-6.0.2", "typescript-5.9.3"] +mutation_required = true + [[leaf]] id = "C0.1" cluster = "C0" @@ -852,6 +862,16 @@ aspects = ["contract", "evidence", "coverage", "regression", "mutation"] catalogs = [] mutation_required = true +[[leaf]] +id = "E2.4" +cluster = "E2" +wave = "W3" +ledger = ".outline/gates/E2.4.md" +owns = ["crates/bamts-codegen/src/jit_memory.rs"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = [] +mutation_required = true + [[leaf]] id = "E3.1" cluster = "E3" @@ -882,6 +902,46 @@ aspects = ["contract", "evidence", "coverage", "regression", "mutation"] catalogs = [] mutation_required = true +[[leaf]] +id = "E3.4" +cluster = "E3" +wave = "W4" +ledger = ".outline/gates/E3.4.md" +owns = ["crates/bamts-cli/src/driver.rs"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = [] +mutation_required = true + +[[leaf]] +id = "E3.5" +cluster = "E3" +wave = "W4" +ledger = ".outline/gates/E3.5.md" +owns = ["crates/bamts-cli/src/windows_link.rs"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = [] +mutation_required = true + +[[leaf]] +id = "E3.6" +cluster = "E3" +wave = "W4" +ledger = ".outline/gates/E3.6.md" +owns = ["crates/bamts-platform/"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = [] +mutation_required = true + +[[leaf]] +id = "E3.7" +cluster = "E3" +wave = "W4" +ledger = ".outline/gates/E3.7.md" +owns = ["crates/bamts-codegen/src/aot/in_process_link.rs"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = [] +mutation_required = true + [[leaf]] id = "E4.1" cluster = "E4" @@ -932,6 +992,36 @@ aspects = ["contract", "evidence", "coverage", "regression", "mutation"] catalogs = ["target-cells"] mutation_required = true +[[leaf]] +id = "E4.6" +cluster = "E4" +wave = "W5" +ledger = ".outline/gates/E4.6.md" +owns = ["verification/toolchains/aarch64-apple-darwin.json"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = ["target-cells"] +mutation_required = true + +[[leaf]] +id = "E4.7" +cluster = "E4" +wave = "W5" +ledger = ".outline/gates/E4.7.md" +owns = ["verification/toolchains/x86_64-apple-darwin.json"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = ["target-cells"] +mutation_required = true + +[[leaf]] +id = "E4.8" +cluster = "E4" +wave = "W5" +ledger = ".outline/gates/E4.8.md" +owns = ["verification/toolchains/x86_64-pc-windows-msvc.json"] +aspects = ["contract", "evidence", "coverage", "regression", "mutation"] +catalogs = ["target-cells"] +mutation_required = true + [[leaf]] id = "E5.1" cluster = "E5" @@ -1125,7 +1215,7 @@ aspects = ["contract", "evidence", "regression"] [[node]] kind = "cluster" id = "B7" -children = ["B7.1"] +children = ["B7.1", "B7.2"] ledger = ".outline/gates/cluster-B7.md" aspects = ["contract", "evidence", "regression"] @@ -1223,21 +1313,21 @@ aspects = ["contract", "evidence", "regression"] [[node]] kind = "cluster" id = "E2" -children = ["E2.1", "E2.2", "E2.3"] +children = ["E2.1", "E2.2", "E2.3", "E2.4"] ledger = ".outline/gates/cluster-E2.md" aspects = ["contract", "evidence", "regression"] [[node]] kind = "cluster" id = "E3" -children = ["E3.1", "E3.2", "E3.3"] +children = ["E3.1", "E3.2", "E3.3", "E3.4", "E3.5", "E3.6", "E3.7"] ledger = ".outline/gates/cluster-E3.md" aspects = ["contract", "evidence", "regression"] [[node]] kind = "cluster" id = "E4" -children = ["E4.1", "E4.2", "E4.3", "E4.4", "E4.5"] +children = ["E4.1", "E4.2", "E4.3", "E4.4", "E4.5", "E4.6", "E4.7", "E4.8"] ledger = ".outline/gates/cluster-E4.md" aspects = ["contract", "evidence", "regression"] @@ -1328,21 +1418,21 @@ aspects = ["contract", "evidence", "regression"] [[node]] kind = "wave" id = "W3" -children = ["B3.5", "B3.6", "B3.7", "B3.8", "B3.9", "B4.1", "B4.2", "B4.3", "B4.4", "B4.5", "C3.1", "C3.2", "C3.3", "C3.4", "C3.5", "C3.6", "C3.7", "C4.1", "C4.2", "C4.3", "C5.1", "C5.2", "C5.3", "C5.4", "E2.1", "E2.2", "E2.3"] +children = ["B3.5", "B3.6", "B3.7", "B3.8", "B3.9", "B4.1", "B4.2", "B4.3", "B4.4", "B4.5", "C3.1", "C3.2", "C3.3", "C3.4", "C3.5", "C3.6", "C3.7", "C4.1", "C4.2", "C4.3", "C5.1", "C5.2", "C5.3", "C5.4", "E2.1", "E2.2", "E2.3", "E2.4"] ledger = ".outline/gates/wave-W3.md" aspects = ["contract", "evidence", "regression"] [[node]] kind = "wave" id = "W4" -children = ["B5.1", "B5.2", "B5.3", "B5.4", "B6.1", "B6.2", "B7.1", "C6.1", "C6.2", "C6.3", "C7.1", "C7.2", "C8.1", "C8.2", "C8.3", "C9", "E3.1", "E3.2", "E3.3"] +children = ["B5.1", "B5.2", "B5.3", "B5.4", "B6.1", "B6.2", "B7.1", "B7.2", "C6.1", "C6.2", "C6.3", "C7.1", "C7.2", "C8.1", "C8.2", "C8.3", "C9", "E3.1", "E3.2", "E3.3", "E3.4", "E3.5", "E3.6", "E3.7"] ledger = ".outline/gates/wave-W4.md" aspects = ["contract", "evidence", "regression"] [[node]] kind = "wave" id = "W5" -children = ["C10.1", "C10.2", "C10.3", "C10.4", "C10.5", "C10.6", "C11.1", "C11.2", "C11.3", "C11.4", "E4.1", "E4.2", "E4.3", "E4.4", "E4.5"] +children = ["C10.1", "C10.2", "C10.3", "C10.4", "C10.5", "C10.6", "C11.1", "C11.2", "C11.3", "C11.4", "E4.1", "E4.2", "E4.3", "E4.4", "E4.5", "E4.6", "E4.7", "E4.8"] ledger = ".outline/gates/wave-W5.md" aspects = ["contract", "evidence", "regression"] diff --git a/verification/manifest.lock.json b/verification/manifest.lock.json index 998a9381..cd686c7b 100644 --- a/verification/manifest.lock.json +++ b/verification/manifest.lock.json @@ -248714,7 +248714,7 @@ "pin": "catalog-inputs/v1", "url": "local://verification/catalog-inputs.json", "digest_algorithm": "sha256", - "digest": "327c86e28f1e613d9b9079a2ba4f6eb9babf55d72508b7415339682f54d27301" + "digest": "32a653282f3f65fb10bea9b096363458856525d8e8908557481910d13e5bdbb0" } }, { @@ -248830,7 +248830,7 @@ "pin": "catalog-inputs/v1", "url": "local://verification/catalog-inputs.json", "digest_algorithm": "sha256", - "digest": "327c86e28f1e613d9b9079a2ba4f6eb9babf55d72508b7415339682f54d27301" + "digest": "32a653282f3f65fb10bea9b096363458856525d8e8908557481910d13e5bdbb0" } }, { @@ -249016,7 +249016,7 @@ "pin": "catalog-inputs/v1", "url": "local://verification/catalog-inputs.json", "digest_algorithm": "sha256", - "digest": "327c86e28f1e613d9b9079a2ba4f6eb9babf55d72508b7415339682f54d27301" + "digest": "32a653282f3f65fb10bea9b096363458856525d8e8908557481910d13e5bdbb0" } }, { @@ -249024,9 +249024,9 @@ "kind": "cross-product", "obligations": [ "toolchain-manifest", - "elf-object", + "target-object", "hardened-link", - "qemu-functional", + "target-functional", "native-runtime", "execmem-guard", "reproducible-build" @@ -249035,47 +249035,71 @@ "x86_64-unknown-linux-gnu", "aarch64-unknown-linux-gnu", "s390x-unknown-linux-gnu", - "riscv64gc-unknown-linux-gnu" + "riscv64gc-unknown-linux-gnu", + "aarch64-apple-darwin", + "x86_64-apple-darwin", + "x86_64-pc-windows-msvc" ] }, "id": "target-cells", - "identifier_count": 28, + "identifier_count": 49, "identifiers": [ - "aarch64-unknown-linux-gnu::elf-object", + "aarch64-apple-darwin::execmem-guard", + "aarch64-apple-darwin::hardened-link", + "aarch64-apple-darwin::native-runtime", + "aarch64-apple-darwin::reproducible-build", + "aarch64-apple-darwin::target-functional", + "aarch64-apple-darwin::target-object", + "aarch64-apple-darwin::toolchain-manifest", "aarch64-unknown-linux-gnu::execmem-guard", "aarch64-unknown-linux-gnu::hardened-link", "aarch64-unknown-linux-gnu::native-runtime", - "aarch64-unknown-linux-gnu::qemu-functional", "aarch64-unknown-linux-gnu::reproducible-build", + "aarch64-unknown-linux-gnu::target-functional", + "aarch64-unknown-linux-gnu::target-object", "aarch64-unknown-linux-gnu::toolchain-manifest", - "riscv64gc-unknown-linux-gnu::elf-object", "riscv64gc-unknown-linux-gnu::execmem-guard", "riscv64gc-unknown-linux-gnu::hardened-link", "riscv64gc-unknown-linux-gnu::native-runtime", - "riscv64gc-unknown-linux-gnu::qemu-functional", "riscv64gc-unknown-linux-gnu::reproducible-build", + "riscv64gc-unknown-linux-gnu::target-functional", + "riscv64gc-unknown-linux-gnu::target-object", "riscv64gc-unknown-linux-gnu::toolchain-manifest", - "s390x-unknown-linux-gnu::elf-object", "s390x-unknown-linux-gnu::execmem-guard", "s390x-unknown-linux-gnu::hardened-link", "s390x-unknown-linux-gnu::native-runtime", - "s390x-unknown-linux-gnu::qemu-functional", "s390x-unknown-linux-gnu::reproducible-build", + "s390x-unknown-linux-gnu::target-functional", + "s390x-unknown-linux-gnu::target-object", "s390x-unknown-linux-gnu::toolchain-manifest", - "x86_64-unknown-linux-gnu::elf-object", + "x86_64-apple-darwin::execmem-guard", + "x86_64-apple-darwin::hardened-link", + "x86_64-apple-darwin::native-runtime", + "x86_64-apple-darwin::reproducible-build", + "x86_64-apple-darwin::target-functional", + "x86_64-apple-darwin::target-object", + "x86_64-apple-darwin::toolchain-manifest", + "x86_64-pc-windows-msvc::execmem-guard", + "x86_64-pc-windows-msvc::hardened-link", + "x86_64-pc-windows-msvc::native-runtime", + "x86_64-pc-windows-msvc::reproducible-build", + "x86_64-pc-windows-msvc::target-functional", + "x86_64-pc-windows-msvc::target-object", + "x86_64-pc-windows-msvc::toolchain-manifest", "x86_64-unknown-linux-gnu::execmem-guard", "x86_64-unknown-linux-gnu::hardened-link", "x86_64-unknown-linux-gnu::native-runtime", - "x86_64-unknown-linux-gnu::qemu-functional", "x86_64-unknown-linux-gnu::reproducible-build", + "x86_64-unknown-linux-gnu::target-functional", + "x86_64-unknown-linux-gnu::target-object", "x86_64-unknown-linux-gnu::toolchain-manifest" ], - "identifiers_sha256": "7fbd77c7414e1b2236482fb16264991ab4a6c59c1e7a80e77d1a10112711e1a9", + "identifiers_sha256": "c3b77261401443fd5bf6f9a3ad0f9d63f8087128259e958b23db8d32d3d7826f", "source": { "pin": "catalog-inputs/v1", "url": "local://verification/catalog-inputs.json", "digest_algorithm": "sha256", - "digest": "327c86e28f1e613d9b9079a2ba4f6eb9babf55d72508b7415339682f54d27301" + "digest": "32a653282f3f65fb10bea9b096363458856525d8e8908557481910d13e5bdbb0" } }, { @@ -249101,7 +249125,7 @@ "pin": "catalog-inputs/v1", "url": "local://verification/catalog-inputs.json", "digest_algorithm": "sha256", - "digest": "327c86e28f1e613d9b9079a2ba4f6eb9babf55d72508b7415339682f54d27301" + "digest": "32a653282f3f65fb10bea9b096363458856525d8e8908557481910d13e5bdbb0" } } ] diff --git a/verification/toolchains/aarch64-apple-darwin.json b/verification/toolchains/aarch64-apple-darwin.json new file mode 100644 index 00000000..561f7dbc --- /dev/null +++ b/verification/toolchains/aarch64-apple-darwin.json @@ -0,0 +1,63 @@ +{ + "cell_class": "native_host", + "obligations": { + "execmem-guard": { + "evidence": "command=bamts-verification toolchain measure --target aarch64-apple-darwin; executable_sha256=335285bc37c9f2e9e485aa48b0e5da26596cb24d054f5907da5352d79d3aceca; inspector_sha256=92b87bfab012271e3fddcddef4c5536022f0873a75ad075f600093e2b6cd40cc; writable_executable_regions=0; pie=true; signature=adhoc", + "missing_artifact": "", + "reason": "Stored execmem-guard observation for this triple.", + "status": "PASS", + "unblock_condition": "" + }, + "hardened-link": { + "evidence": "command=cc -arch arm64 -o probe ../support.o probe.o; executable_sha256=335285bc37c9f2e9e485aa48b0e5da26596cb24d054f5907da5352d79d3aceca; object_format=macho; object_type=executable; machine=aarch64; pie=true; signature=adhoc; linker_version=Apple clang version 21.0.0 (clang-2100.1.1.101); linker_sha256=34129c71a01a74f7f3b2443521519b2e5447553fa187f5fcafaaf8c42cc192e2", + "missing_artifact": "", + "reason": "Stored hardened link artifact for this triple.", + "status": "PASS", + "unblock_condition": "" + }, + "native-runtime": { + "evidence": "command=./probe; executable_sha256=335285bc37c9f2e9e485aa48b0e5da26596cb24d054f5907da5352d79d3aceca; execution=native; exit=0; result=int32:42; fuel=positive", + "missing_artifact": "", + "reason": "This process runs natively on the cell triple.", + "status": "PASS", + "unblock_condition": "" + }, + "reproducible-build": { + "evidence": "command=bamts-verification toolchain measure --target aarch64-apple-darwin; first_object_sha256=6b61d80d002fd1fa97653ab90d54de3ca4d1b541d283b6d1f2d18322573b45d6; second_object_sha256=6b61d80d002fd1fa97653ab90d54de3ca4d1b541d283b6d1f2d18322573b45d6; first_executable_sha256=335285bc37c9f2e9e485aa48b0e5da26596cb24d054f5907da5352d79d3aceca; second_executable_sha256=335285bc37c9f2e9e485aa48b0e5da26596cb24d054f5907da5352d79d3aceca; byte_identical=true", + "missing_artifact": "", + "reason": "Stored reproducible-build identity for this triple.", + "status": "PASS", + "unblock_condition": "" + }, + "target-functional": { + "evidence": "command=./probe; executable_sha256=335285bc37c9f2e9e485aa48b0e5da26596cb24d054f5907da5352d79d3aceca; execution=native; exit=0; result=int32:42; fuel=positive", + "missing_artifact": "", + "reason": "Native-host functional execution completed without emulation.", + "status": "PASS", + "unblock_condition": "" + }, + "target-object": { + "evidence": "command=bamts-verification toolchain object --target aarch64-apple-darwin --out probe.o; object_sha256=6b61d80d002fd1fa97653ab90d54de3ca4d1b541d283b6d1f2d18322573b45d6; object_format=macho; object_type=relocatable; machine=aarch64; bytes=2992; entry=bamts_m0_fn_0; helpers=bamts_load_constant,bamts_consume_fuel", + "missing_artifact": "", + "reason": "Stored relocatable AOT object for this triple.", + "status": "PASS", + "unblock_condition": "" + }, + "toolchain-manifest": { + "evidence": "command=rustc -vV; rustc_version=1.97.1; rustc_host=aarch64-apple-darwin; rustc_sha256=210df6794001b73ec3d453878707fa1e0bdcb63c427024a6e6574bbe5615a4da", + "missing_artifact": "", + "reason": "Recorded from rustc -vV on this host.", + "status": "PASS", + "unblock_condition": "" + } + }, + "rustc": { + "commit": "8bab26f4f68e0e26f0bb7960be334d5b520ea452", + "commit_date": "2026-07-14", + "host": "aarch64-apple-darwin", + "llvm_version": "22.1.6", + "version": "1.97.1" + }, + "schema": "bamti.target-cell/v2", + "triple": "aarch64-apple-darwin" +} diff --git a/verification/toolchains/aarch64-unknown-linux-gnu.json b/verification/toolchains/aarch64-unknown-linux-gnu.json index 71d1c1b6..f1ece48e 100644 --- a/verification/toolchains/aarch64-unknown-linux-gnu.json +++ b/verification/toolchains/aarch64-unknown-linux-gnu.json @@ -1,52 +1,52 @@ { - "cell_class": "cross_qemu", + "cell_class": "native_host", "obligations": { - "elf-object": { - "evidence": "command=cargo run -p bamts-verification -- toolchain object --target aarch64-unknown-linux-gnu --out /tmp/bamts-e4-aarch64-1.o; object_sha256=15964dee552319bf4a1fcf24a39ffb060c4d0452d20827f675cdff2add15497e; elf_type=REL; elf_machine=AArch64; bytes=3960; entry=bamts_m0_fn_0; helpers=bamts_load_constant,bamts_consume_fuel", + "execmem-guard": { + "evidence": "command=bamts-verification toolchain measure --target aarch64-unknown-linux-gnu; executable_sha256=5e5ea995593b0acd9563b6c3a0ee308cd3fedd305925227e2c9d621daa48a850; inspector_sha256=89bd80244c0abd65f7b91b3cb0bee38bba682c23a12646610dfafe4b361ed9e3; writable_executable_regions=0; relro=true; now=true; noexecstack=true; gnu_stack=RW; dynamic_section=absent", "missing_artifact": "", - "reason": "Production bamts-codegen emitted a target-exact ELF64 relocatable object.", + "reason": "Stored execmem-guard observation for this triple.", "status": "PASS", "unblock_condition": "" }, - "execmem-guard": { - "evidence": "command=llvm-readelf -h -l -d /tmp/bamts-e4-aarch64; executable_sha256=45cb270c961d08863d021e597a7385fbad3f8cb7ed57aa48ed9a3d006ac2d116; writable_executable_load_segments=0; gnu_stack=RW; dynamic_section=absent; readelf_version=Homebrew LLVM version 22.1.8; readelf_sha256=ebe528ae4511bcc77f73ab301f8cdda7a1658726b9ce8b8303e7798e4658ab35", + "hardened-link": { + "evidence": "command=ld.lld -m aarch64linux -static -e _start -z relro -z now -z noexecstack -z defs --build-id=none -o probe ../support.o probe.o; executable_sha256=5e5ea995593b0acd9563b6c3a0ee308cd3fedd305925227e2c9d621daa48a850; object_format=elf; object_type=executable; machine=aarch64; relro=true; now=true; noexecstack=true; gnu_stack=RW; dynamic_section=absent; linker_version=Ubuntu LLD 18.1.3 (compatible with GNU linkers); linker_sha256=f8835e48488195c65fb3dcb946053284ddf6b1369922893785209a9073c4e57b", "missing_artifact": "", - "reason": "ELF program headers prove disjoint writable/executable loads and a non-executable stack.", + "reason": "Stored hardened link artifact for this triple.", "status": "PASS", "unblock_condition": "" }, - "hardened-link": { - "evidence": "command=ld.lld -m aarch64linux -static -e _start -z relro -z now -z noexecstack -z defs --build-id=none -o /tmp/bamts-e4-aarch64 /tmp/bamts-e4-start-aarch64.o /tmp/bamts-e4-support-aarch64.o /tmp/bamts-e4-aarch64-1.o; executable_sha256=45cb270c961d08863d021e597a7385fbad3f8cb7ed57aa48ed9a3d006ac2d116; elf_type=EXEC; elf_machine=AArch64; static=true; relro=true; now=true; noexecstack=true; defs=true; build_id=none; linker_version=Homebrew LLD 22.1.8 (compatible with GNU linkers); linker_sha256=88c7a1886ff45d6f8fd98dc719126180fe6dd963896b616d455e7b60326db830", + "native-runtime": { + "evidence": "command=./probe; executable_sha256=5e5ea995593b0acd9563b6c3a0ee308cd3fedd305925227e2c9d621daa48a850; execution=native; exit=0; result=int32:42; fuel=positive", "missing_artifact": "", - "reason": "LLD produced a static hardened executable with all generated helper symbols resolved.", + "reason": "This process runs natively on the cell triple.", "status": "PASS", "unblock_condition": "" }, - "native-runtime": { - "evidence": "command=/tmp/qemu-user-static/usr/bin/qemu-aarch64-static /tmp/bamts-e4-aarch64; executable_sha256=45cb270c961d08863d021e597a7385fbad3f8cb7ed57aa48ed9a3d006ac2d116; execution=qemu-aarch64; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-aarch64 version 7.2.22 (Debian 1:7.2+dfsg-7+deb12u18+b3); emulator_sha256=4f55e2e88dc05dc0f619562d5795b8eb25ed2ad2547504fb4835207a6911c350; package_sha256=c3e3ba2bd87f8c5b9a5da5ef21b5a3b82d7c63b89dd448d9ddaa4eabc5b6e402", + "reproducible-build": { + "evidence": "command=bamts-verification toolchain measure --target aarch64-unknown-linux-gnu; first_object_sha256=7f7d8b3a0dc6a5c7f8dbdd833ae8fb2b13286692320d13a7428920614fe341b5; second_object_sha256=7f7d8b3a0dc6a5c7f8dbdd833ae8fb2b13286692320d13a7428920614fe341b5; first_executable_sha256=5e5ea995593b0acd9563b6c3a0ee308cd3fedd305925227e2c9d621daa48a850; second_executable_sha256=5e5ea995593b0acd9563b6c3a0ee308cd3fedd305925227e2c9d621daa48a850; byte_identical=true", "missing_artifact": "", - "reason": "Static QEMU execution returned generated int32 42 and observed a positive fuel charge.", + "reason": "Stored reproducible-build identity for this triple.", "status": "PASS", "unblock_condition": "" }, - "qemu-functional": { - "evidence": "command=/tmp/qemu-user-static/usr/bin/qemu-aarch64-static /tmp/bamts-e4-aarch64; executable_sha256=45cb270c961d08863d021e597a7385fbad3f8cb7ed57aa48ed9a3d006ac2d116; execution=qemu-aarch64; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-aarch64 version 7.2.22 (Debian 1:7.2+dfsg-7+deb12u18+b3); emulator_sha256=4f55e2e88dc05dc0f619562d5795b8eb25ed2ad2547504fb4835207a6911c350; package_sha256=c3e3ba2bd87f8c5b9a5da5ef21b5a3b82d7c63b89dd448d9ddaa4eabc5b6e402", + "target-functional": { + "evidence": "command=./probe; executable_sha256=5e5ea995593b0acd9563b6c3a0ee308cd3fedd305925227e2c9d621daa48a850; execution=native; exit=0; result=int32:42; fuel=positive", "missing_artifact": "", - "reason": "The target image completed its generated result and fuel contract under QEMU user mode.", + "reason": "Native-host functional execution completed without emulation.", "status": "PASS", "unblock_condition": "" }, - "reproducible-build": { - "evidence": "command=cargo run -p bamts-verification -- toolchain object --target aarch64-unknown-linux-gnu --out /tmp/bamts-e4-aarch64-1.o && cargo run -p bamts-verification -- toolchain object --target aarch64-unknown-linux-gnu --out /tmp/bamts-e4-aarch64-2.o && cmp /tmp/bamts-e4-aarch64-1.o /tmp/bamts-e4-aarch64-2.o; first_object_sha256=15964dee552319bf4a1fcf24a39ffb060c4d0452d20827f675cdff2add15497e; second_object_sha256=15964dee552319bf4a1fcf24a39ffb060c4d0452d20827f675cdff2add15497e; byte_identical=true", + "target-object": { + "evidence": "command=bamts-verification toolchain object --target aarch64-unknown-linux-gnu --out probe.o; object_sha256=7f7d8b3a0dc6a5c7f8dbdd833ae8fb2b13286692320d13a7428920614fe341b5; object_format=elf; object_type=relocatable; machine=aarch64; bytes=4032; entry=bamts_m0_fn_0; helpers=bamts_load_constant,bamts_consume_fuel", "missing_artifact": "", - "reason": "Two independent production codegen emissions were byte-identical.", + "reason": "Stored relocatable AOT object for this triple.", "status": "PASS", "unblock_condition": "" }, "toolchain-manifest": { - "evidence": "command=rustc -vV; rustc_version=1.97.1; rustc_commit=8bab26f4f68e0e26f0bb7960be334d5b520ea452; rustc_host=x86_64-unknown-linux-gnu; rustc_llvm=22.1.6; rustc_sha256=3690cc576ede140504698405d5d8fa3826aaadbe71699c6c4ed0a565d6f493e2", + "evidence": "command=rustc -vV; rustc_version=1.97.1; rustc_host=aarch64-unknown-linux-gnu; rustc_sha256=a3d4dfcd867ddc1e7dca25f13f7236c72229a56f68aa511437c5bb72eb2dfe78", "missing_artifact": "", - "reason": "The compiler toolchain identity and binary content hash were observed on this host.", + "reason": "Recorded from rustc -vV on this host.", "status": "PASS", "unblock_condition": "" } @@ -54,10 +54,10 @@ "rustc": { "commit": "8bab26f4f68e0e26f0bb7960be334d5b520ea452", "commit_date": "2026-07-14", - "host": "x86_64-unknown-linux-gnu", + "host": "aarch64-unknown-linux-gnu", "llvm_version": "22.1.6", "version": "1.97.1" }, - "schema": "bamti.target-cell/v1", + "schema": "bamti.target-cell/v2", "triple": "aarch64-unknown-linux-gnu" } diff --git a/verification/toolchains/riscv64gc-unknown-linux-gnu.json b/verification/toolchains/riscv64gc-unknown-linux-gnu.json index e192998e..03b10124 100644 --- a/verification/toolchains/riscv64gc-unknown-linux-gnu.json +++ b/verification/toolchains/riscv64gc-unknown-linux-gnu.json @@ -1,52 +1,52 @@ { "cell_class": "cross_qemu", "obligations": { - "elf-object": { - "evidence": "command=cargo run -p bamts-verification -- toolchain object --target riscv64gc-unknown-linux-gnu --out /tmp/bamts-e4-riscv64gc-1.o; object_sha256=f0c7378a140d846b47ee6093b3cea773981c009199f1def43177243e63ef4958; elf_type=REL; elf_machine=RISC-V; bytes=4032; entry=bamts_m0_fn_0; helpers=bamts_load_constant,bamts_consume_fuel", + "execmem-guard": { + "evidence": "command=bamts-verification toolchain measure --target riscv64gc-unknown-linux-gnu; executable_sha256=b538691cb0fbe25a3ecb6ba03c3eac587dbb5c42ad665dd834fec6f657a7a460; inspector_sha256=89bd80244c0abd65f7b91b3cb0bee38bba682c23a12646610dfafe4b361ed9e3; writable_executable_regions=0; relro=true; now=true; noexecstack=true; gnu_stack=RW; dynamic_section=absent", "missing_artifact": "", - "reason": "Production bamts-codegen emitted a target-exact RV64GC ELF64 relocatable object.", + "reason": "Stored execmem-guard observation for this triple.", "status": "PASS", "unblock_condition": "" }, - "execmem-guard": { - "evidence": "command=llvm-readelf -h -l -d /tmp/bamts-e4-riscv64gc; executable_sha256=53dfb59e4c12559ee815e5b3039ce4f4a0f576b74d2dfcbdc0c92a91d876792b; writable_executable_load_segments=0; gnu_stack=RW; dynamic_section=absent; readelf_version=Homebrew LLVM version 22.1.8; readelf_sha256=ebe528ae4511bcc77f73ab301f8cdda7a1658726b9ce8b8303e7798e4658ab35", + "hardened-link": { + "evidence": "command=ld.lld -m elf64lriscv -static -e _start -z relro -z now -z noexecstack -z defs --build-id=none -o probe ../support.o probe.o; executable_sha256=b538691cb0fbe25a3ecb6ba03c3eac587dbb5c42ad665dd834fec6f657a7a460; object_format=elf; object_type=executable; machine=riscv64; relro=true; now=true; noexecstack=true; gnu_stack=RW; dynamic_section=absent; linker_version=Ubuntu LLD 18.1.3 (compatible with GNU linkers); linker_sha256=f8835e48488195c65fb3dcb946053284ddf6b1369922893785209a9073c4e57b", "missing_artifact": "", - "reason": "ELF program headers prove disjoint writable/executable loads and a non-executable stack.", + "reason": "Stored hardened link artifact for this triple.", "status": "PASS", "unblock_condition": "" }, - "hardened-link": { - "evidence": "command=ld.lld -m elf64lriscv -static -e _start -z relro -z now -z noexecstack -z defs --build-id=none -o /tmp/bamts-e4-riscv64gc /tmp/bamts-e4-start-riscv64gc.o /tmp/bamts-e4-support-riscv64gc.o /tmp/bamts-e4-riscv64gc-1.o; executable_sha256=53dfb59e4c12559ee815e5b3039ce4f4a0f576b74d2dfcbdc0c92a91d876792b; elf_type=EXEC; elf_machine=RISC-V; static=true; relro=true; now=true; noexecstack=true; defs=true; build_id=none; linker_version=Homebrew LLD 22.1.8 (compatible with GNU linkers); linker_sha256=88c7a1886ff45d6f8fd98dc719126180fe6dd963896b616d455e7b60326db830", + "native-runtime": { + "evidence": "command=/usr/bin/qemu-riscv64-static ./probe; executable_sha256=b538691cb0fbe25a3ecb6ba03c3eac587dbb5c42ad665dd834fec6f657a7a460; execution=qemu-riscv64; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-riscv64 version 8.2.2 (Debian 1:8.2.2+ds-0ubuntu1.18); emulator_sha256=dfa4aa3c7aa272934264a318505c8b7002b78003d78595423a077397e0959849", "missing_artifact": "", - "reason": "LLD produced a static hardened RV64GC executable with all generated helper symbols resolved.", + "reason": "An emulator executed the target runtime image for this triple.", "status": "PASS", "unblock_condition": "" }, - "native-runtime": { - "evidence": "command=/tmp/qemu-user-static/usr/bin/qemu-riscv64-static /tmp/bamts-e4-riscv64gc; executable_sha256=53dfb59e4c12559ee815e5b3039ce4f4a0f576b74d2dfcbdc0c92a91d876792b; execution=qemu-riscv64; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-riscv64 version 7.2.22 (Debian 1:7.2+dfsg-7+deb12u18+b3); emulator_sha256=555165a5f638551572c53cf0844942b781aaa13681bb6a838170d8ef4f1c207c; package_sha256=c3e3ba2bd87f8c5b9a5da5ef21b5a3b82d7c63b89dd448d9ddaa4eabc5b6e402", + "reproducible-build": { + "evidence": "command=bamts-verification toolchain measure --target riscv64gc-unknown-linux-gnu; first_object_sha256=889175cbf5e8a76ab4dd7a785263f66e13268266bb5059ad054638110321da41; second_object_sha256=889175cbf5e8a76ab4dd7a785263f66e13268266bb5059ad054638110321da41; first_executable_sha256=b538691cb0fbe25a3ecb6ba03c3eac587dbb5c42ad665dd834fec6f657a7a460; second_executable_sha256=b538691cb0fbe25a3ecb6ba03c3eac587dbb5c42ad665dd834fec6f657a7a460; byte_identical=true", "missing_artifact": "", - "reason": "Static QEMU execution returned generated int32 42 and observed a positive fuel charge.", + "reason": "Stored reproducible-build identity for this triple.", "status": "PASS", "unblock_condition": "" }, - "qemu-functional": { - "evidence": "command=/tmp/qemu-user-static/usr/bin/qemu-riscv64-static /tmp/bamts-e4-riscv64gc; executable_sha256=53dfb59e4c12559ee815e5b3039ce4f4a0f576b74d2dfcbdc0c92a91d876792b; execution=qemu-riscv64; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-riscv64 version 7.2.22 (Debian 1:7.2+dfsg-7+deb12u18+b3); emulator_sha256=555165a5f638551572c53cf0844942b781aaa13681bb6a838170d8ef4f1c207c; package_sha256=c3e3ba2bd87f8c5b9a5da5ef21b5a3b82d7c63b89dd448d9ddaa4eabc5b6e402", + "target-functional": { + "evidence": "command=/usr/bin/qemu-riscv64-static ./probe; executable_sha256=b538691cb0fbe25a3ecb6ba03c3eac587dbb5c42ad665dd834fec6f657a7a460; execution=qemu-riscv64; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-riscv64 version 8.2.2 (Debian 1:8.2.2+ds-0ubuntu1.18); emulator_sha256=dfa4aa3c7aa272934264a318505c8b7002b78003d78595423a077397e0959849", "missing_artifact": "", - "reason": "The target image completed its generated result and fuel contract under QEMU user mode.", + "reason": "An emulator executed a linked image for this triple.", "status": "PASS", "unblock_condition": "" }, - "reproducible-build": { - "evidence": "command=cargo run -p bamts-verification -- toolchain object --target riscv64gc-unknown-linux-gnu --out /tmp/bamts-e4-riscv64gc-1.o && cargo run -p bamts-verification -- toolchain object --target riscv64gc-unknown-linux-gnu --out /tmp/bamts-e4-riscv64gc-2.o && cmp /tmp/bamts-e4-riscv64gc-1.o /tmp/bamts-e4-riscv64gc-2.o; first_object_sha256=f0c7378a140d846b47ee6093b3cea773981c009199f1def43177243e63ef4958; second_object_sha256=f0c7378a140d846b47ee6093b3cea773981c009199f1def43177243e63ef4958; byte_identical=true", + "target-object": { + "evidence": "command=bamts-verification toolchain object --target riscv64gc-unknown-linux-gnu --out probe.o; object_sha256=889175cbf5e8a76ab4dd7a785263f66e13268266bb5059ad054638110321da41; object_format=elf; object_type=relocatable; machine=riscv64; bytes=4160; entry=bamts_m0_fn_0; helpers=bamts_load_constant,bamts_consume_fuel", "missing_artifact": "", - "reason": "Two independent production codegen emissions were byte-identical.", + "reason": "Stored relocatable AOT object for this triple.", "status": "PASS", "unblock_condition": "" }, "toolchain-manifest": { - "evidence": "command=rustc -vV; rustc_version=1.97.1; rustc_commit=8bab26f4f68e0e26f0bb7960be334d5b520ea452; rustc_host=x86_64-unknown-linux-gnu; rustc_llvm=22.1.6; rustc_sha256=3690cc576ede140504698405d5d8fa3826aaadbe71699c6c4ed0a565d6f493e2", + "evidence": "command=rustc -vV; rustc_version=1.97.1; rustc_host=aarch64-unknown-linux-gnu; rustc_sha256=a3d4dfcd867ddc1e7dca25f13f7236c72229a56f68aa511437c5bb72eb2dfe78", "missing_artifact": "", - "reason": "The compiler toolchain identity and binary content hash were observed on this host.", + "reason": "Recorded from rustc -vV on this host.", "status": "PASS", "unblock_condition": "" } @@ -54,10 +54,10 @@ "rustc": { "commit": "8bab26f4f68e0e26f0bb7960be334d5b520ea452", "commit_date": "2026-07-14", - "host": "x86_64-unknown-linux-gnu", + "host": "aarch64-unknown-linux-gnu", "llvm_version": "22.1.6", "version": "1.97.1" }, - "schema": "bamti.target-cell/v1", + "schema": "bamti.target-cell/v2", "triple": "riscv64gc-unknown-linux-gnu" } diff --git a/verification/toolchains/s390x-unknown-linux-gnu.json b/verification/toolchains/s390x-unknown-linux-gnu.json index 96de1cc2..bbaac175 100644 --- a/verification/toolchains/s390x-unknown-linux-gnu.json +++ b/verification/toolchains/s390x-unknown-linux-gnu.json @@ -1,52 +1,52 @@ { "cell_class": "cross_qemu", "obligations": { - "elf-object": { - "evidence": "command=cargo run -p bamts-verification -- toolchain object --target s390x-unknown-linux-gnu --out /tmp/bamts-e4-s390x-1.o; object_sha256=1bb7eae2d5f5b59b419e256eee58287d2c5ee89b63e9f9f14d82b0c7f534a836; elf_type=REL; elf_machine=IBM S/390; bytes=3944; entry=bamts_m0_fn_0; helpers=bamts_load_constant,bamts_consume_fuel", + "execmem-guard": { + "evidence": "command=bamts-verification toolchain measure --target s390x-unknown-linux-gnu; executable_sha256=6dc2879e849db09b530372aeb64e4dd744b96293fb1299a0d39755b203c26135; inspector_sha256=89bd80244c0abd65f7b91b3cb0bee38bba682c23a12646610dfafe4b361ed9e3; writable_executable_regions=0; relro=true; now=true; noexecstack=true; gnu_stack=RW; dynamic_section=absent", "missing_artifact": "", - "reason": "Production bamts-codegen emitted a target-exact big-endian ELF64 relocatable object.", + "reason": "Stored execmem-guard observation for this triple.", "status": "PASS", "unblock_condition": "" }, - "execmem-guard": { - "evidence": "command=llvm-readelf -h -l -d /tmp/bamts-e4-s390x; executable_sha256=c565254a48635b82fd969425bff8fc99f3c38b18d2e2ecc7953e0494f058253b; writable_executable_load_segments=0; gnu_stack=RW; dynamic_section=absent; readelf_version=Homebrew LLVM version 22.1.8; readelf_sha256=ebe528ae4511bcc77f73ab301f8cdda7a1658726b9ce8b8303e7798e4658ab35", + "hardened-link": { + "evidence": "command=ld.lld -m elf64_s390 -static -e _start -z relro -z now -z noexecstack -z defs --build-id=none -o probe ../support.o probe.o; executable_sha256=6dc2879e849db09b530372aeb64e4dd744b96293fb1299a0d39755b203c26135; object_format=elf; object_type=executable; machine=s390x; relro=true; now=true; noexecstack=true; gnu_stack=RW; dynamic_section=absent; linker_version=Ubuntu LLD 18.1.3 (compatible with GNU linkers); linker_sha256=f8835e48488195c65fb3dcb946053284ddf6b1369922893785209a9073c4e57b", "missing_artifact": "", - "reason": "ELF program headers prove disjoint writable/executable loads and a non-executable stack.", + "reason": "Stored hardened link artifact for this triple.", "status": "PASS", "unblock_condition": "" }, - "hardened-link": { - "evidence": "command=ld.lld -m elf64_s390 -static -e _start -z relro -z now -z noexecstack -z defs --build-id=none -o /tmp/bamts-e4-s390x /tmp/bamts-e4-start-s390x.o /tmp/bamts-e4-support-s390x.o /tmp/bamts-e4-s390x-1.o; executable_sha256=c565254a48635b82fd969425bff8fc99f3c38b18d2e2ecc7953e0494f058253b; elf_type=EXEC; elf_machine=IBM S/390; static=true; relro=true; now=true; noexecstack=true; defs=true; build_id=none; linker_version=Homebrew LLD 22.1.8 (compatible with GNU linkers); linker_sha256=88c7a1886ff45d6f8fd98dc719126180fe6dd963896b616d455e7b60326db830", + "native-runtime": { + "evidence": "command=/usr/bin/qemu-s390x-static ./probe; executable_sha256=6dc2879e849db09b530372aeb64e4dd744b96293fb1299a0d39755b203c26135; execution=qemu-s390x; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-s390x version 8.2.2 (Debian 1:8.2.2+ds-0ubuntu1.18); emulator_sha256=6e12a68a48307e21cfe6ed87555172d4d6aca55f2d71f7c7336ccd3a3e5b018a", "missing_artifact": "", - "reason": "LLD produced a static hardened big-endian executable with all generated helper symbols resolved.", + "reason": "An emulator executed the target runtime image for this triple.", "status": "PASS", "unblock_condition": "" }, - "native-runtime": { - "evidence": "command=/tmp/qemu-user-static/usr/bin/qemu-s390x-static /tmp/bamts-e4-s390x; executable_sha256=c565254a48635b82fd969425bff8fc99f3c38b18d2e2ecc7953e0494f058253b; execution=qemu-s390x; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-s390x version 7.2.22 (Debian 1:7.2+dfsg-7+deb12u18+b3); emulator_sha256=eaae13a250d6f10419de73f3a04ee11f6b27b4f1d05251ac74340a8f5d67dddb; package_sha256=c3e3ba2bd87f8c5b9a5da5ef21b5a3b82d7c63b89dd448d9ddaa4eabc5b6e402", + "reproducible-build": { + "evidence": "command=bamts-verification toolchain measure --target s390x-unknown-linux-gnu; first_object_sha256=10f99c8428a581a4364a718865b2d183f03f5c73bf5ac568333d4896fbc06494; second_object_sha256=10f99c8428a581a4364a718865b2d183f03f5c73bf5ac568333d4896fbc06494; first_executable_sha256=6dc2879e849db09b530372aeb64e4dd744b96293fb1299a0d39755b203c26135; second_executable_sha256=6dc2879e849db09b530372aeb64e4dd744b96293fb1299a0d39755b203c26135; byte_identical=true", "missing_artifact": "", - "reason": "Static QEMU execution returned generated int32 42 and observed a positive fuel charge.", + "reason": "Stored reproducible-build identity for this triple.", "status": "PASS", "unblock_condition": "" }, - "qemu-functional": { - "evidence": "command=/tmp/qemu-user-static/usr/bin/qemu-s390x-static /tmp/bamts-e4-s390x; executable_sha256=c565254a48635b82fd969425bff8fc99f3c38b18d2e2ecc7953e0494f058253b; execution=qemu-s390x; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-s390x version 7.2.22 (Debian 1:7.2+dfsg-7+deb12u18+b3); emulator_sha256=eaae13a250d6f10419de73f3a04ee11f6b27b4f1d05251ac74340a8f5d67dddb; package_sha256=c3e3ba2bd87f8c5b9a5da5ef21b5a3b82d7c63b89dd448d9ddaa4eabc5b6e402", + "target-functional": { + "evidence": "command=/usr/bin/qemu-s390x-static ./probe; executable_sha256=6dc2879e849db09b530372aeb64e4dd744b96293fb1299a0d39755b203c26135; execution=qemu-s390x; exit=0; result=int32:42; fuel=positive; emulator_version=qemu-s390x version 8.2.2 (Debian 1:8.2.2+ds-0ubuntu1.18); emulator_sha256=6e12a68a48307e21cfe6ed87555172d4d6aca55f2d71f7c7336ccd3a3e5b018a", "missing_artifact": "", - "reason": "The target image completed its generated result and fuel contract under QEMU user mode.", + "reason": "An emulator executed a linked image for this triple.", "status": "PASS", "unblock_condition": "" }, - "reproducible-build": { - "evidence": "command=cargo run -p bamts-verification -- toolchain object --target s390x-unknown-linux-gnu --out /tmp/bamts-e4-s390x-1.o && cargo run -p bamts-verification -- toolchain object --target s390x-unknown-linux-gnu --out /tmp/bamts-e4-s390x-2.o && cmp /tmp/bamts-e4-s390x-1.o /tmp/bamts-e4-s390x-2.o; first_object_sha256=1bb7eae2d5f5b59b419e256eee58287d2c5ee89b63e9f9f14d82b0c7f534a836; second_object_sha256=1bb7eae2d5f5b59b419e256eee58287d2c5ee89b63e9f9f14d82b0c7f534a836; byte_identical=true", + "target-object": { + "evidence": "command=bamts-verification toolchain object --target s390x-unknown-linux-gnu --out probe.o; object_sha256=10f99c8428a581a4364a718865b2d183f03f5c73bf5ac568333d4896fbc06494; object_format=elf; object_type=relocatable; machine=s390x; bytes=3984; entry=bamts_m0_fn_0; helpers=bamts_load_constant,bamts_consume_fuel", "missing_artifact": "", - "reason": "Two independent production codegen emissions were byte-identical.", + "reason": "Stored relocatable AOT object for this triple.", "status": "PASS", "unblock_condition": "" }, "toolchain-manifest": { - "evidence": "command=rustc -vV; rustc_version=1.97.1; rustc_commit=8bab26f4f68e0e26f0bb7960be334d5b520ea452; rustc_host=x86_64-unknown-linux-gnu; rustc_llvm=22.1.6; rustc_sha256=3690cc576ede140504698405d5d8fa3826aaadbe71699c6c4ed0a565d6f493e2", + "evidence": "command=rustc -vV; rustc_version=1.97.1; rustc_host=aarch64-unknown-linux-gnu; rustc_sha256=a3d4dfcd867ddc1e7dca25f13f7236c72229a56f68aa511437c5bb72eb2dfe78", "missing_artifact": "", - "reason": "The compiler toolchain identity and binary content hash were observed on this host.", + "reason": "Recorded from rustc -vV on this host.", "status": "PASS", "unblock_condition": "" } @@ -54,10 +54,10 @@ "rustc": { "commit": "8bab26f4f68e0e26f0bb7960be334d5b520ea452", "commit_date": "2026-07-14", - "host": "x86_64-unknown-linux-gnu", + "host": "aarch64-unknown-linux-gnu", "llvm_version": "22.1.6", "version": "1.97.1" }, - "schema": "bamti.target-cell/v1", + "schema": "bamti.target-cell/v2", "triple": "s390x-unknown-linux-gnu" } diff --git a/verification/toolchains/x86_64-unknown-linux-gnu.json b/verification/toolchains/x86_64-unknown-linux-gnu.json deleted file mode 100644 index 1a44e7a4..00000000 --- a/verification/toolchains/x86_64-unknown-linux-gnu.json +++ /dev/null @@ -1,63 +0,0 @@ -{ - "cell_class": "native_host", - "obligations": { - "elf-object": { - "evidence": "command=cargo run -p bamts-verification -- toolchain object --target x86_64-unknown-linux-gnu --out /tmp/bamts-e4-x86_64-1.o; object_sha256=f6ef71297ae66294196bb58d7e2a748d92c06126a8fd2b6c60e28b0f24d2f9a6; elf_type=REL; elf_machine=Advanced Micro Devices X86-64; bytes=3968; entry=bamts_m0_fn_0; helpers=bamts_load_constant,bamts_consume_fuel", - "missing_artifact": "", - "reason": "Production bamts-codegen emitted a target-exact ELF64 relocatable object.", - "status": "PASS", - "unblock_condition": "" - }, - "execmem-guard": { - "evidence": "command=llvm-readelf -h -l -d /tmp/bamts-e4-x86_64; executable_sha256=61f9e4a7acd5a3d1e3fa751ec35d48ade3977e9241e78218ac820a52e3cfb255; writable_executable_load_segments=0; gnu_stack=RW; dynamic_section=absent; readelf_version=Homebrew LLVM version 22.1.8; readelf_sha256=ebe528ae4511bcc77f73ab301f8cdda7a1658726b9ce8b8303e7798e4658ab35", - "missing_artifact": "", - "reason": "ELF program headers prove disjoint writable/executable loads and a non-executable stack.", - "status": "PASS", - "unblock_condition": "" - }, - "hardened-link": { - "evidence": "command=ld.lld -m elf_x86_64 -static -e _start -z relro -z now -z noexecstack -z defs --build-id=none -o /tmp/bamts-e4-x86_64 /tmp/bamts-e4-start-x86_64.o /tmp/bamts-e4-support-x86_64.o /tmp/bamts-e4-x86_64-1.o; executable_sha256=61f9e4a7acd5a3d1e3fa751ec35d48ade3977e9241e78218ac820a52e3cfb255; elf_type=EXEC; elf_machine=Advanced Micro Devices X86-64; static=true; relro=true; now=true; noexecstack=true; defs=true; build_id=none; linker_version=Homebrew LLD 22.1.8 (compatible with GNU linkers); linker_sha256=88c7a1886ff45d6f8fd98dc719126180fe6dd963896b616d455e7b60326db830", - "missing_artifact": "", - "reason": "LLD produced a static hardened executable with all generated helper symbols resolved.", - "status": "PASS", - "unblock_condition": "" - }, - "native-runtime": { - "evidence": "command=/tmp/bamts-e4-x86_64; executable_sha256=61f9e4a7acd5a3d1e3fa751ec35d48ade3977e9241e78218ac820a52e3cfb255; execution=native; exit=0; result=int32:42; fuel=positive", - "missing_artifact": "", - "reason": "Native execution returned generated int32 42 and observed a positive fuel charge.", - "status": "PASS", - "unblock_condition": "" - }, - "qemu-functional": { - "evidence": "command=/tmp/bamts-e4-x86_64; executable_sha256=61f9e4a7acd5a3d1e3fa751ec35d48ade3977e9241e78218ac820a52e3cfb255; execution=native; exit=0; result=int32:42; fuel=positive", - "missing_artifact": "", - "reason": "The native-host functional path executed the same freestanding target image without emulation.", - "status": "PASS", - "unblock_condition": "" - }, - "reproducible-build": { - "evidence": "command=cargo run -p bamts-verification -- toolchain object --target x86_64-unknown-linux-gnu --out /tmp/bamts-e4-x86_64-1.o && cargo run -p bamts-verification -- toolchain object --target x86_64-unknown-linux-gnu --out /tmp/bamts-e4-x86_64-2.o && cmp /tmp/bamts-e4-x86_64-1.o /tmp/bamts-e4-x86_64-2.o; first_object_sha256=f6ef71297ae66294196bb58d7e2a748d92c06126a8fd2b6c60e28b0f24d2f9a6; second_object_sha256=f6ef71297ae66294196bb58d7e2a748d92c06126a8fd2b6c60e28b0f24d2f9a6; byte_identical=true", - "missing_artifact": "", - "reason": "Two independent production codegen emissions were byte-identical.", - "status": "PASS", - "unblock_condition": "" - }, - "toolchain-manifest": { - "evidence": "command=rustc -vV; rustc_version=1.97.1; rustc_commit=8bab26f4f68e0e26f0bb7960be334d5b520ea452; rustc_host=x86_64-unknown-linux-gnu; rustc_llvm=22.1.6; rustc_sha256=3690cc576ede140504698405d5d8fa3826aaadbe71699c6c4ed0a565d6f493e2", - "missing_artifact": "", - "reason": "The compiler toolchain identity and binary content hash were observed on this host.", - "status": "PASS", - "unblock_condition": "" - } - }, - "rustc": { - "commit": "8bab26f4f68e0e26f0bb7960be334d5b520ea452", - "commit_date": "2026-07-14", - "host": "x86_64-unknown-linux-gnu", - "llvm_version": "22.1.6", - "version": "1.97.1" - }, - "schema": "bamti.target-cell/v1", - "triple": "x86_64-unknown-linux-gnu" -}