[Enrique Saurez (esaurez)]

A sandbox launch checked its distro layer from metadata alone, then gave OpenVMM the layer's path,
which something could replace between the check and OpenVMM's open of it. The scratch image went to
OpenVMM by path with no check at all, and nothing kept two sandboxes from writing the same scratch
image. With this change, a launch hands OpenVMM exactly the objects that its checks found.

What changed

• Scratch binding (nvx::host::sandbox::scratch): ScratchBinding records a scratch image's
  identity (device or volume, and file ID), kind, and length, in a versioned serde form without
  times, and never reads the image. ScratchBinding::check opens the image, compares it with the
  binding from metadata alone, claims it for one sandbox at a time, until that sandbox's VMM exits,
  and compares it again, since the launch whose claim it waited for may have changed it. On Linux,
  the file ID is the inode number, so a replacement of the same length that is created once the
  image is removed can take it again and pass the check:

  • on Linux, the claim is an exclusive flock of the checked descriptor's open file description,
    which the VMM inherits and so keeps;
  • on Windows, it is a guard handle that has no access to the content and denies deleting and
    renaming the image, after a probe that refuses an image that another process, such as another
    sandbox's VMM, has open for writing, and then a claim handle, open for writing and sharing
    every access, which the VMM inherits, so that it holds the claim from its start, even if the
    launcher ends before the VMM has opened the image, which would release the guard.

  A check that finds the image claimed fails with ErrorKind::TimedOut at the caller's deadline
  (CLAIM_RELEASE_WAIT for the CLI).

• Prepared launch (nvx::host::sandbox::PreparedLaunch): built from a SandboxPlan, the
  CheckedDistro of its image, and the CheckedScratch of its upper. It refuses files that are not
  the plan's, renders the plan's OpenVMM arguments and kernel command line, and holds the checked
  files.

  • On Linux, the arguments name /proc/self/fd/N, and PreparedLaunch::inherited feeds
    VmmCommand::inherit, so the VMM inherits exactly those descriptors through the supervisor's
    allowlist. A checked descriptor that took a standard stream's number, as when the caller had
    closed that stream, first moves above the standard streams, with its open file description and
    so its claim.
  • On Windows, the VMM inherits one handle, which no argument names: the scratch image's claim.
    The arguments name each checked image by the final path
    of its handle (GetFinalPathNameByHandleW, in \\?\ form): absolute, through no link, and
    independent of the VMM's working directory. The checked files hold that path in place (the
    distro layer's file denies writing and deleting it, the scratch guard denies deleting and
    renaming it, and NTFS renames no directory while a file below it is open) until the caller drops
    the prepared launch, after authenticated Ready or after tearing down a failed launch. A final
    path that OpenVMM could not take as the path of a raw image, with a comma or semicolon, or
    ending in .vhd, .vhdx, .vmgs, or .iso, which a link, the directory of a relative path,
    or the case of a name on disk can bring, fails the preparation with ErrorKind::Unsupported,
    as such a path in the plan does.
  • Images are attached raw: OpenVMM chooses a container format by file extension, which
    /proc/self/fd/N lacks.

• CLI: provision records the scratch image's binding in config.json under
  scratch_binding; a configuration without one is refused, with no reader for older records.
  start checks the distro and scratch bindings, launches OpenVMM through supervisor::launch with
  the prepared launch's arguments and descriptors, and drops the prepared launch once the guest is
  ready, before the launch commits, or once a failed start's OpenVMM has ended and its launch was
  abandoned.
  run binds and claims its scratch image and runs OpenVMM through a prepared launch, in the
  caller's terminal and process group, passing it no descriptor or handle beyond its standard
  streams and the launch's allowlist: on Linux through the same allowlist as start, and on Windows
  through an explicit handle list that holds copies of the CLI's standard handles and the scratch
  image's claim alone, where
  std::process::Command had passed OpenVMM every inheritable handle of the CLI. --dry-run still
  prints the plan's paths.

• Batch files on Windows: the spawn of run, and the supervisor's spawns, which
  supervisor::launch and so start use, refuse a .bat or .cmd file as OpenVMM with
  ErrorKind::InvalidInput. Windows runs one through cmd.exe, which would read an & in an
  image's path as another command. std::process::Command, which run used before, escapes a batch
  file's arguments; the supervisor's detached spawn never did. A wrapper that adds OpenVMM options
  must be an executable. The CLI reaches a batch file only through NVX_OPENVMM, which start
  refuses altogether.

• Fake OpenVMM: opens the sandbox's block devices as OpenVMM does, holds them until it exits,
  and records what it held, so that the CLI tests check that OpenVMM gets the checked objects.

• Goldens: tests/fixtures/cli.json changes only where config.json records the scratch
  binding and where launches name their checked images: /proc/self/fd/N on Linux, and \\?\
  final paths on Windows.

Staging, the checks, and the preparation of a launch read metadata only: no content reads, mappings,
or hashing.

Known limitation (#780): on Windows, a launcher, such as start or run, that ends after
starting OpenVMM, and before OpenVMM has opened the images, as when it crashes, closes the checked
files that hold the images' final paths, so that a replacement at those paths would reach OpenVMM
unchecked. start never commits such a launch, and the sandbox's next transition ends its VMM;
run keeps no journal, and its OpenVMM runs until it exits. On Linux, OpenVMM opens the
descriptors that it inherited, so it uses the checked images whatever happens to the launcher.

Part of #622. Refs #620, #535.

Validation

Rebased onto dev at 2971c638. The earlier rebase onto d4d00af8, which moved the CLI's
lifecycle onto the sandbox store and supervisor::launch (#772), had these conflicts:

• start_locked (nvx-rs/src/host/cli/sandbox/lifecycle.rs): the prepared launch's arguments,
  kernel command line, and descriptors go into the command that supervisor::launch asks for, and
  the prepared launch is dropped before the launch commits, once the guest is ready, or after
  Starting::abandon, once a failed start's OpenVMM has ended. On an early return, Starting, whose
  drop ends the VMM, drops before the prepared launch.
• tests/fixtures/cli.json: each scenario merged token by token; the goldens change as before.
• tests/cli.rs: the claim test expects the store's sandbox.lock, and the CLI test of a start
  that refuses a batch file as OpenVMM is gone, since start now refuses any NVX_OPENVMM before it
  spawns; the library's tests of the refusal remain.
• nvx-rs/README.md and doc/run.md: dev's store beside the scratch binding.

Onto 9b70ec97, the one conflict was process::spawn, which #721 removed with the other helpers
of the old instance, and to whose docs this PR had added that it fails for a batch file: the
sentence went with the function, and no other patch changed (git range-diff).

Onto 2971c638, which brought #753, whose plan holds the sandbox's network, and #773, the conflicts
were with #753:

• SandboxPlan::render_arguments, through which the prepared launch renders the plan's arguments,
  ends with the network's arguments, as SandboxPlan::openvmm_arguments does, so that a prepared
  launch passes OpenVMM the network that the plan holds.
• start_locked: the network that config.json records is read, and its policy checked again,
  after the distro layer's check and before the scratch image's check, which claims the image.
• config.json records scratch_binding beside network, and the docs of the CLI's state, and the
  README, name both.
• tests/fixtures/cli.json: the scenarios that nvx-rs: lower sandbox networks through one typed policy with a proxy-only mode #753 changed or added take this PR's changes, the
  scratch binding in every config.json that provision writes and the checked files' names in
  launched commands, /proc/self/fd/N on Linux and \\?\ final paths on Windows, while dry runs
  keep the plan's paths. nvx-rs: lower sandbox networks through one typed policy with a proxy-only mode #753 replaced run: OpenVMM refuses its options with
  run: network options without --net, which launches nothing.

After the rebase onto d4d00af8, the final review found that, on Windows, a launcher that ended
between OpenVMM's start and its open of the scratch image released the image's claim, which then
lived in the launcher's guard alone; 51b69fe2 passes the VMM a claim handle, as Linux passes the
claimed descriptor. Copilot then noted that such an early end also releases the guards that hold
the images' final paths on Windows; that is left to #780, which the docs link (33443f58,
60d1839b).

Every commit builds on Windows with cargo check --locked --all-targets --all-features. Unless
noted, the checks ran at 60d1839b, on Windows 11 (WHP) and on Linux in a virtual machine with
MSHV.

• python scripts/nvx.py verify.

• check-nvx-rs, on Windows and Linux, in nvx-rs with NVX_BUNDLE=skip: cargo fmt --all --check,
  cargo clippy --locked --all-targets --all-features -- -D warnings,
  cargo clippy --locked --lib --no-default-features -- -D warnings,
  cargo test --locked --all-features, cargo test --locked,
  cargo doc --locked --no-deps --all-features with RUSTDOCFLAGS=-D warnings,
  cargo +1.89 check --locked --all-targets --all-features, and the artifact-bundling test; on
  Linux also the aarch64-apple-darwin check, nvx.py check-guest-binaries, and
  build-guest-binaries with test-guest-binaries.

• The aci_edge_sandboxes checks, on Windows and Linux: fmt, both Clippy configurations, both test
  configurations, docs, MSRV, and nvx.py test-agent-bundling; the macOS check on Linux.

• tests/runtime-adapter: the protocol-only and host-only configurations on Windows and Linux, and
  the guest-only (musl) configuration on Linux.

• validate-nvx, on Windows and Linux, without its pip install, whose PyYAML both hosts had:
  compileall, the unit-test suites, test_adversarial.py, test_hosts.py, and the --help
  checks.

• New tests:

  • sandbox::scratch: binding without reads, serde round trip, refusals, claims while the VMM
    holds the image, changed identity, kind, and length failing without reads, a check that
    waits for a claim while the holder resizes the image, which fails without the second
    comparison on both Linux and Windows, and, on Windows, a claim that the claim handle keeps once
    the guard is closed, while renames go through as after authenticated Ready.
  • sandbox::prepared: the handoff, refusals of files that are not the plan's, and staging,
    checking, and preparing that read no content, with explicit verification as the positive
    control; on Windows, refusals of final paths that OpenVMM could not take: a semicolon behind a
    junction to the distro layer, a comma behind a junction to the scratch image, and a scratch
    image named .vhd on disk that the plan's path names in capitals. Without the check, that test
    fails, the launch naming distro:file:\\?\…\real;layers\distro.erofs,ro.
  • tests/checked_handoff.rs, the public API end to end with a stand-in VMM: replacing the image
    and, separately, the scratch between the check and the VMM's open; on Linux, the exact
    descriptor set that the VMM gets while another thread spawns processes with inheritable pipes,
    and an orchestrator that closed its standard input, whose scratch check got descriptor 0; on
    Windows, the handle list, which holds the scratch image's claim alone, the guards against
    writing, renaming, and deleting while the VMM reads and writes, released after the prepared
    launch is dropped, and a VMM started in another directory after the junction that the plan's
    relative paths went through was re-pointed to decoys. On both, a launcher whose checked files
    close before its VMM has opened the images, as when it crashes, leaves the scratch image
    claimed by the VMM until the VMM exits; on Windows, without the claim handle, another sandbox's
    check claims the image instead.
  • Six CLI tests: a scratch image changed after provision, a configuration without a valid scratch
    binding, a running sandbox's claim, OpenVMM receiving the checked images, and, on Windows, a run
    whose OpenVMM does not get a handle that the CLI inherited from its caller, which fails without
    the handle list, and a run that refuses a batch file as OpenVMM, which fails without the
    refusal, cmd.exe running the file.
  • platform::windows: a foreground spawn passes its child the allowed handles and no other
    inheritable handle of the caller, and both spawns refuse batch files, including one named only
    .cmd, which an extension check misses, and one named with a trailing dot.
  • seal::windows: final paths shorter and longer than the first buffer match
    fs::canonicalize and hold no NUL.

• Repeated runs, before the rebases: 40 consecutive runs of the Linux library tests
  (--all-features) pass at 08e6fcfa. Before it, 7 of 30 failed: 5 in a scratch test whose
  directory could take the removed image's inode number, and 2 in
  guest::sys::signal::tests::a_thread_that_blocks_sigpipe_keeps_only_a_signal_that_was_pending,
  which this PR does not touch, where a spawn's copy of the pipe's read end can let the write
  succeed. A gate run also failed a check made right after a claim's release, which a process that
  another test spawned held until it executed its program.

• Real VMs, with the OpenVMM built from the pinned 7ff31ed7 and the guest artifacts of dev's
  development release v0.1.0-dev.39510661ffe6, which nvx.py download installed on both:

  • python scripts/nvx.py test-microvm --backend <whp|mshv> --scenario managed-exec-config --scenario sandbox-lifecycle pass on both. On WHP, the one-shot run named
    distro:file:\\?\D:\…\build\ubuntu-distro.erofs,ro and
    scratch:file:\\?\D:\…\run-scratch.ext4; on MSHV, distro:file:/proc/self/fd/3,ro and
    scratch:file:/proc/self/fd/4.
  • The runtime adapter's hypervisor scenarios, cargo test --locked --features host -- --ignored
    in tests/runtime-adapter with its musl guest agent built on Linux, pass on both: the 4
    layered scenarios of tests: launch the runtime adapter's VMs with a layered root #773 and the 2 restart scenarios.
  • python scripts/nvx.py test-aci-edge-sandboxes --backend whp passes, 18 tests.

  The MSHV host's CPU needs --cpu-profile host, and start runs the root's own OpenVMM alone, so
  there the root's OpenVMM was a wrapper that adds the option and keeps argv[0]. On KVM, the same
  test-microvm scenarios passed at 7c392ab4, before the rebases.

Left to CI: KVM after the rebases, test-aci-edge-sandboxes on MSHV and KVM, and the self-hosted
KVM, MSHV, and WHP runners.

[Copilot]

🟡 Changes recommended

Unresolved path-resolution and claim-validation gaps can violate the checked-image handoff guarantee.

4 open issues

• Pin checked image paths across VMM working-directory changes · New
• Revalidate image binding after acquiring the scratch lock · New
• Relocate checked descriptors below standard streams · New
• Stop background spawner when handoff scope unwinds · New

What changed in this PR

This PR adds checked-image handoff to NVX’s shared sandbox launch API and CLI, advancing the cold-launch integration in #622.

Changes:

• Adds metadata-only scratch bindings and exclusive sandbox claims.
• Introduces PreparedLaunch to retain checked images and render platform-specific VMM arguments.
• Integrates the handoff into CLI launches, with fake-VMM coverage and documentation.

File	Description
nvx-rs/​tests/​cli.rs	Tests scratch bindings, claims, and checked-image launches.
nvx-rs/​tests/​checked_handoff.rs	Exercises public handoff APIs and inheritance isolation.
nvx-rs/​src/​testing/​fake_openvmm.rs	Opens, retains, and reports sandbox block files.
nvx-rs/​src/​host/​sys/​process.rs	Shares descriptor restrictions across detached and foreground launches.
nvx-rs/​src/​host/​sys/​mod.rs	Exposes foreground inheritance restrictions internally.
nvx-rs/​src/​host/​seal.rs	Makes file identity reusable by scratch bindings.
nvx-rs/​src/​host/​sandbox/​scratch/​tests.rs	Tests metadata bindings and claim lifetimes.
nvx-rs/​src/​host/​sandbox/​scratch.rs	Implements scratch bindings and platform-specific claims.
nvx-rs/​src/​host/​sandbox/​prepared/​tests.rs	Tests preparation, file matching, and metadata-only operation.
nvx-rs/​src/​host/​sandbox/​prepared.rs	Adds checked-file ownership and launch argument rendering.
nvx-rs/​src/​host/​sandbox/​plan.rs	Allows block arguments to use checked-file references.
nvx-rs/​src/​host/​sandbox/​mod.rs	Exports scratch and prepared-launch APIs.
nvx-rs/​src/​host/​sandbox/​distro.rs	Retains checked-layer path, format, and identity.
nvx-rs/​src/​host/​platform/​other.rs	Adds the fallback foreground inheritance hook.
nvx-rs/​src/​host/​platform/​linux.rs	Reuses allowlist validation for foreground launches.
nvx-rs/​src/​host/​mod.rs	Documents bindings and checked handoff.
nvx-rs/​src/​host/​cli/​sandbox/​state.rs	Persists and reads scratch bindings.
nvx-rs/​src/​host/​cli/​sandbox/​mod.rs	Uses prepared launches for one-shot runs.
nvx-rs/​src/​host/​cli/​sandbox/​lifecycle.rs	Checks both images and supervises prepared starts.
nvx-rs/​README.md	Documents APIs, claim behavior, and compatibility changes.
doc/​run.md	Explains checked-image launches and scratch ownership.

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Windows final-path rendering can introduce unchecked delimiters into OpenVMM block arguments.

1 open issue

• Reject delimiters in resolved Windows disk paths · New

4 resolved since last review

• Revalidate image binding after acquiring the scratch lock
• Pin checked image paths across VMM working-directory changes
• Stop background spawner when handoff scope unwinds
• Relocate checked descriptors below standard streams

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Cross-platform locking, descriptor inheritance, and guard lifetimes need final human review, with MSHV validation still left to CI.

No open issues 🎉

1 resolved since last review

• Reject delimiters in resolved Windows disk paths

_{Review effort: Balanced}

[Copilot]

Warning

Copilot couldn't run its full agentic review because it didn't start before the timeout. Make sure your repository has a runner available, or add a copilot-code-review.yml file specifying one with the runs-on attribute. See the docs for more details.

Copilot review overview

2 open issues

• GetFinalPathNameByHandleW returns the required length including the terminating NUL; the current… · New
• GetStdHandle can return a non-null value that still isn’t a valid duplicable handle in some host… · New

_{Review effort: Lite}

[Copilot]

🔵 Needs a closer look

Security-sensitive image claims and process-inheritance lifetimes across Linux and Windows warrant final human review and CI confirmation on the final revision.

No open issues 🎉

2 resolved since last review

• GetFinalPathNameByHandleW returns the required length including the terminating NUL; the current…
• GetStdHandle can return a non-null value that still isn’t a valid duplicable handle in some host…

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Security-sensitive cross-platform claim lifetimes, file handoff, and process inheritance warrant final human review, with MSHV validation still pending.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Security-sensitive cross-platform image handoff, scratch ownership, and process inheritance guarantees warrant final human review despite extensive regression coverage.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Security-sensitive cross-platform inheritance, Windows sharing semantics, and image-claim lifetimes warrant final human review.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🟡 Changes recommended

Windows launcher termination can release image-path guards before OpenVMM opens the checked images, leaving a replacement race.

1 open issue

• Keep image guards active until OpenVMM opens images · New

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Security-sensitive cross-platform claims, inheritance, and resource lifetimes warrant final human review.

No open issues 🎉

1 resolved since last review

• Keep image guards active until OpenVMM opens images

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Security-sensitive cross-platform inheritance, image locking, and guard lifetimes warrant final human review despite no new actionable findings.

No open issues 🎉

_{Review effort: Balanced}

[Copilot]

🔵 Needs a closer look

Security-sensitive cross-platform image ownership and process inheritance require final human review of their lifetime guarantees.

No open issues 🎉

_{Review effort: Balanced}