Skip to content

test(ADR-8): wasmtime-environ as a pinned, dev-only differential oracle - #505

Merged
avrabe merged 8 commits into
mainfrom
spike/wasmtime-environ-oracle
Oct 21, 2026
Merged

avrabe merged 8 commits into
mainfrom
spike/wasmtime-environ-oracle

Conversation

@avrabe

@avrabe avrabe commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Implements the direction decided on 2026-10-20: meld keeps its own component-resolution implementation, and cross-tests it against wasmtime-environ's component translator. The oracle is pinned, dev-only, and upgraded deliberately. ADR-8 records the policy and is open for review.

What is added

  • Dev-dependency wasmtime-environ = "=41.0.4". It is already in the graph through wasmtime, so no new crate and no second copy.
  • Feature oracle, which gates:
    • meld-core/tests/oracle_wasmtime_environ.rs, via required-features;
    • one read-only hook, Fuser::oracle_index_maps(): a snapshot of the merger's index maps that the pipeline writes and never reads.
  • When it runs: the default suite never compiles the oracle, so it is unchanged. Every --all-features run, including CI coverage, runs it, so it cannot silently stop running.
  • What it compares:
    • Output side: meld's --memory multi --component output, translated by the oracle, against the source. For each exported lift and each host lower: the lifted function, memory, realloc, post-return, callback, string encoding and async flag.
    • Resolver side: meld's reading of every source lift against the oracle's.
  • ADR-8, safety/adr/ADR-8-wasmtime-environ-differential-oracle.md, covers:
    • the exact pin, upgraded only together with wasmtime in its own PR;
    • the tie-break when the two disagree: the Component Model spec and wasmtime's behaviour on the unfused component;
    • what the oracle must not do: be a runtime dependency, be fed meld's output, be loosened to get green, or count as proof.

It would have caught the recent bugs

Applied to meld from just before the #481 fix (9926d47), the oracle flags:

Each WAT fixture is fused 10 times, because #484 disagreed only on some hash orders.

An off-by-one injected into meld's resolver turns the resolver-side tests red.

On current main

Every compared fact agrees: the #481, #484 and #490 fixtures, five wit-bindgen fixtures, and two compositions.

It also reached two known open bugs. Those fixtures are marked #[ignore] with the issue number until they are fixed:

Next comparison (not in this PR)

Compare the options of internal component-to-component calls (wasmtime's adapter modules) against meld's adapter-site resolution. That is the #489 and #504 shape, and it is where the corpus's real string, list and record traffic crosses.

The same mechanism measures P3 progress: wasmtime-environ 41 models most async built-ins as trampolines. Three meld built-ins have no counterpart there: ThreadSpawn, ThreadHwConcurrency and ResourceDropAsync.

Test plan

  • cargo fmt --all -- --check: exit 0
  • cargo clippy --workspace --all-targets -- -D warnings: exit 0, with and without --all-features
  • cargo test --workspace: 963 passed, 0 failed (unchanged)
  • cargo test --workspace --all-features: 972 passed, 0 failed
  • cargo test -p meld-core --features oracle --test oracle_wasmtime_environ: 10 passed, 3 ignored (known open issues)

🤖 Generated with Claude Code

https://claude.ai/code/session_01WrmMqf1iuTS1UBEes5TmdC

avrabe and others added 3 commits October 21, 2026 07:42
…cle for component resolution

meld keeps its own component resolution (parser, resolver, component_wrap
canonical options, merger). wasmtime-environ 41.0.4's component Translator,
already in the graph via the wasmtime dev-dependency, becomes a second,
independent implementation that meld is cross-tested against. It is never a
runtime dependency.

- meld-core/Cargo.toml: dev-dep `wasmtime-environ = "=41.0.4"` with
  compile + component-model (the version and features wasmtime already
  enables, so no new crate in the graph); feature `oracle`; the test has
  `required-features = ["oracle"]`, so the default suite never compiles it,
  while --all-features (CI coverage) runs it.
- meld-core/src/lib.rs: `OracleIndexMaps` and `Fuser::oracle_index_maps()`,
  a cfg(feature = "oracle") snapshot of memory_index_map and
  function_index_map that the pipeline writes and never reads.
- meld-core/tests/oracle_wasmtime_environ.rs: two comparisons.
  - Output side: per exported lift and per lowered host import, the source's
    resolution (wasmtime's graph, mapped through meld's maps) is compared
    with the oracle's reading of meld's --memory multi --component output,
    covering the lifted core function, memory, realloc, post-return,
    callback, encoding and async.
  - Resolver side: meld's resolver::core_func_sources and
    core_memory_sources reading of every source lift is compared with the
    oracle's reading of the same source.
- safety/adr/ADR-8: status proposed.

Results on main: every non-ignored fixture agrees. The fixtures are the
#481 and #484 WAT, the PR #490 alias chain, five wit-bindgen runtime
fixtures, compose and compose_record, and the #461 falcon trio. Three cases
are ignored, each with a reason:
- meld#486: on compose_record/provider.wasm, meld exits 0 with a lift that
  has no `memory`, and the oracle rejects it as invalid. The oracle found
  this on its first run.
- meld#485: on the issue's repro, the `run` export re-exports the import,
  and the oracle rejects it.
- falcon --component: meld refuses these inputs. This is the same error
  class as #476; the root cause is not confirmed.

Potency: the committed diff was applied to a fresh export of 9926d47
(7eaeccc^). The oracle flagged:
- #481: lift memory, source 1 against output 0;
- #484: lower memory, encoding and realloc;
- #490: the alias chain's memory and encoding.
An off-by-one injected into core_func_sources turned the resolver-side tests
red.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WrmMqf1iuTS1UBEes5TmdC
…cision date

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WrmMqf1iuTS1UBEes5TmdC
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WrmMqf1iuTS1UBEes5TmdC
@github-actions

Copy link
Copy Markdown

LS-N verification gate

✅ 59/59 approved LS entries verified

count
Passed (≥1 test, all green) 59
Failed (≥1 test failure) 0
Missing (no ls_*_NN_* test found) 0

Approved loss-scenarios.yaml entries are expected to have a
regression test named ls_<letter>_<num>_* (e.g. LS-A-11 →
ls_a_11_*). The gate runs each prefix via cargo test --lib --no-fail-fast and aggregates pass/fail/missing.

Failed LS entries

(none)

Missing regression tests

(none)

Updated automatically by tools/post_verification_comment.py.
Source of truth: safety/stpa/loss-scenarios.yaml.

@cpetig

cpetig commented Oct 21, 2026 •

Copy link
Copy Markdown

wasmtime-environ =41.0.4 sounds significantly outdated to me with 50.0 already released (yesterday).

@avrabe

avrabe commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Agreed, 41 is well behind 50.0.0.

Why this PR uses 41. It deliberately pins the oracle to the wasmtime meld already uses for its runtime tests (wasmtime = "41.0.1"). That way the oracle adds no second copy of wasmtime, and both always agree on one version. ADR-8 makes this the policy: the oracle moves only together with wasmtime, in its own PR, so that any new disagreement can be attributed to the upgrade.

The upgrade to 50.0.0 will follow as that PR, built on this one:

  • wasmtime, wasmtime-wasi and the oracle all go to 50.0.0. All three are dev-only.
  • wasmtime 50 needs Rust 1.97. CI builds on stable, so that is fine, and meld's own minimum Rust version (1.85) is unaffected.
  • About 30 test files use the wasmtime API and will need porting.
  • Any compared fact that changes between 41 and 50 will be reported in that PR, as either a meld bug or a semantic change in wasmtime. Catching those changes is what the oracle is for.

If you would rather land the oracle directly on 50, I can fold the upgrade into this PR instead.

@avrabe

avrabe commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

The upgrade to 50.0.0 is now #508, stacked on this PR. Every oracle fact is identical between 41 and 50, and the shipped meld binary produces byte-identical output with the new lockfile. The details are in #508.

avrabe and others added 2 commits October 21, 2026 11:08
…me-wasi, wasmtime-environ oracle pin) (#508)

Port tests to wasmtime 50 / wat strict component indices; oracle facts unchanged.


Claude-Session: https://claude.ai/code/session_01WrmMqf1iuTS1UBEes5TmdC

Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com>
…s 50.0.0

The direction was decided on 2026-10-20; the policy (exact pin,
upgrade together with wasmtime, tie-break, what the oracle must not do)
was accepted on 2026-10-21 in review of #505. The pin text now matches
the =50.0.0 that #508 moved to, and an upgrade log records that move.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01WrmMqf1iuTS1UBEes5TmdC
@avrabe

avrabe commented Oct 21, 2026

Copy link
Copy Markdown
Contributor Author

Review: approve (the maintainer consents to merging)

What I checked:

Non-blocking notes:

  • The snapshot clones two maps on each fuse, but only under the oracle feature.
  • After a failed fuse, oracle_index_maps() returns the previous snapshot. This is documented on the method.

ADR-8 is now resolved. It records the maintainer's acceptance of the policy on 2026-10-21. The pin text matches =50.0.0, and an upgrade log records #508.

@avrabe
avrabe enabled auto-merge (squash) October 21, 2026 09:09
@temper-pulseengine

Copy link
Copy Markdown
Contributor

Automated review for PR #505

pulseengine/meld:spike/wasmtime-environ-oracle → pulseengine/meld:main

Verdict: 💬 Comment

Summary: The code changes are mostly minor and do not significantly impact the functionality of the Fuser struct. However, there is a new feature introduced that requires additional documentation and testing to ensure its correctness. The addition of the oracle_index_maps field and related methods also adds complexity to the codebase, so it's important to thoroughly test these changes before merging.

Findings: 0 mechanical (rivet) · 1 from local AI model.

Findings (1):

  1. meld-core/src/lib.rs:757
    /// ADR-8: a read-only snapshot of the merger's index maps, taken just before
    
    The oracle_index_maps field is introduced to store the index maps for the differential oracle. However, it is not documented or tested thoroughly.

Generated by a local AI model and post-validated against a strict JSON contract. Each finding includes the verbatim line being criticised — verify by reading the file at the cited location.

Reviewed at b1c0fba

@avrabe
avrabe merged commit 98f8e90 into main Oct 21, 2026
22 checks passed
@avrabe
avrabe deleted the spike/wasmtime-environ-oracle branch October 21, 2026 10:16
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants