Skip to content

nvx-rs: make the nvx-rs CLI the backend of nvx.py run - #704

Open
Pedro Henrique Penna (ppenna) wants to merge 4 commits into
devfrom
agents/fix-issue-638-nvx
Open

Pedro Henrique Penna (ppenna) wants to merge 4 commits into
devfrom
agents/fix-issue-638-nvx

Conversation

@ppenna

@ppenna Pedro Henrique Penna (ppenna) commented Oct 21, 2026 •

Copy link
Copy Markdown
Contributor

Closes #638

Summary

nvx.py run now runs nvx-rs run, a port of command_run, as #359 made nvx.py sandbox run nvx-rs sandbox. nvx-rs checks the options, builds OpenVMM's command line (live shares, aggregate children, nvx_share= tokens and the 1024-byte budget, network options and egress policy files), prints >> COMMAND, and runs OpenVMM in the foreground on the standard streams. Python builds no OpenVMM command line for run anymore, and scripts/nvx_tools/sandbox.py is gone. This is phase 4 of #356.

Commits

  1. nvx-rs: run the microVMs of nvx.py run in the fake OpenVMM: nvx-fake-openvmm accepts the command lines of nvx.py run (no lifecycle, no sandbox block device): cold boots and restores, --processors, --cpu-profile, --memory-capacity, the --restore-* options, shares, and outcome reports. It runs no guest: it prints that it booted or restored, writes a report if asked, and exits with 0 or fake_exit_status; a missing snapshot fails with 1. Like OpenVMM, it decides which share holds each --mount-deny, --mount-allow, and --mount-write path of any launch once it has resolved both the path and the share's root, so a share root that is relative to the working directory, as nvx.py run passes it, holds the absolute policy paths below it. Every existing form and message is otherwise unchanged.
  2. nvx-rs: add nvx-rs run, the port of nvx.py run: nvx-rs run (feature cli), with every option, default, choice, check, message, output, and exit status of nvx.py run, in command_run's order, and its usage and help for --prog nvx.py. It reuses sandbox run's OpenVMM runner, the raw share primitives of nvx::host::sandbox::mount (not SandboxMount), and validate_run_share_targets; sandbox and run share a small NetworkOptions type. Guest::default_memory_mib holds the default memories. A temporary recorder, scripts/nvx_rs_run_fixtures.py, adapted from the retired scripts/nvx_rs_fixtures.py, recorded 170 scenarios per section of nvx-rs/tests/fixtures/cli.json from the Python nvx.py run, on Windows and on Linux with CPython 3.14.7; nvx-rs run replays both with no difference.
  3. nvx.py: run the run command with nvx-rs: the delegation. argparse stays the front end (help byte-identical at 80 and 120 columns, abbreviations, usage errors); _SandboxOption becomes _NvxRsOption for both commands; the Python implementation and the recorder go; tests and docs follow.

CPU-profile guidance

The CPU catalog stays in time_abi.py, which doctor and benchmark also use. For a cold boot that is not a dry run, on a host whose CPU no built-in profile serves, with auto (or host on another vendor's CPU), nvx.py computes the guidance and passes it in the hidden --cold-boot-guidance=TEXT option of nvx-rs run, which neither the usage nor the help shows. nvx.py looks up the OpenVMM path with nvx-rs artifacts only when the guidance applies (time_abi.cpu_profile_guidance_applies).

nvx-rs prints the guidance only after OpenVMM exits with a nonzero status from a cold boot: never after a dry run, a restore, a refused option, a usage error, a spawn failure, or Ctrl-C. nvx-rs prints it, not nvx.py, because a refused option exits with 1 or 2, which OpenVMM can return too.

Default memory

guests.py keeps default_memory_mib, which test-microvm and nvx.py run --help use; Guest::default_memory_mib holds the same 128/512/512. A Python test pins both to the golden fixture: the recorded run help equals nvx.py run --help, and each guest's recorded default dry run uses --memory {default}M. The Rust replay pins nvx-rs to the same fixture.

Behavior change

As for nvx.py sandbox (#359): with two errors on one command line, an invalid choice, an unrecognized argument, or a missing value that argparse reports, or a request for help, now wins over a value that nvx-rs refuses, even when that value comes first. For example, nvx.py run --memory-mib x --guest debian now reports the guest. doc/usage.md and the nvx-rs README document it. Messages, exit statuses, and the help are otherwise unchanged.

Validation

After review (head a94167dcc): the fix for the review thread on the fake's policy paths is folded into commits 1 and 2, with runs_find_absolute_policy_paths_in_relative_share_roots in nvx-rs/tests/fake_openvmm.rs and run_shares_relative_roots_with_absolute_policy_paths in nvx-rs/tests/cli.rs as regression tests; commit 3 is unchanged, with the same patch. On Windows and in WSL, the nvx-rs gate (fmt, clippy for all targets and for the library without default features, tests with all and default features, docs, and the 1.89 MSRV check) and the aci_edge_sandboxes fmt, clippy, and tests pass at the head, and in WSL, each of the first two commits passes clippy and the nvx-rs tests on its own. The results below were recorded on the previous head, 9a3cc30a0, whose tree differs only by these changes.

Windows (Python 3.14.7, Rust 1.95.0 and 1.89):

  • nvx.py verify passes (after git submodule update --init openvmm).
  • check-nvx-rs: fmt --check; clippy for all targets and features and for the library without default features; test --all-features and test; doc with -D warnings; the 1.89 MSRV check; and the runtime adapter's protocol-only and host-only configurations. All pass.
  • check-aci-edge-sandboxes: fmt, clippy, tests, agent bundling, doc, and MSRV all pass.
  • check-quality: ruff check, ruff format --check, pyright for Linux and Windows, Markdown links, and the PowerShell syntax check all pass.
  • validate-nvx, with NVX_RS set to a release build:
    • The 924 tests of test_nvx_tools, test_guest_binaries, test_microvm_tests, test_development_release, test_host_telemetry, test_time_abi, test_tool_versions, and test_markdown_links pass.
    • test_adversarial, test_hosts, and the --help smoke tests pass.
    • test_performance and test_setup_azure fail on this host with or without this change, because bash resolves to WSL's bash.exe there. They pass in WSL.
  • The recorder's parity mode replays the 170 Windows run scenarios byte for byte, and --check confirms that the recording is deterministic. nvx.py run --help is byte-identical before and after at 80 and 120 columns.

WSL Ubuntu (standalone CPython 3.14.7):

  • nvx.py verify passes.
  • check-nvx-rs passes in full: artifact bundling, the macOS check, the guest-binary check, build, and test, and the runtime adapter's protocol, host, and musl guest configurations. The musl guest configuration's 1.89 check was not run, because that toolchain lacks the musl target here.
  • check-aci-edge-sandboxes passes, including the macOS check.
  • check-quality passes: ruff, pyright for both platforms, links, and the Specula tests, which import fcntl and so run only on Linux.
  • validate-nvx passes: the full unittest list, including the NVX_RS-gated end-to-end tests (guidance after a failing OpenVMM, and the pty terminal-restore test through nvx.py run), plus test_adversarial, test_hosts, and the --help smoke tests.
  • Parity passes for the 170 POSIX run scenarios, and the CLI tests pass, including the Ctrl-C tests.
  • Each of the first two commits also builds and passes fmt, clippy, the nvx-rs tests, ruff, pyright, links, and the Python tests on its own.

Trial merge with current dev (919a774c6):

CI:

  • Run 39622963482 on 9a3cc30a0 passed every job, including the KVM, MSHV, and WHP NVX microVM tests, which run the scenarios that call nvx.py run, the KVM debug-kernel microVM tests, the KVM, MSHV, and WHP platform jobs, the OpenVMM vmm-tests and unit tests, both NVX CLI jobs, the quality checks, and the performance gate.
  • Run 39663154475 on a94167dcc, the current head, passed all 27 jobs that a pull request runs, including the KVM, MSHV, and WHP NVX microVM tests and platform jobs and the KVM debug-kernel microVM tests. The 4 skipped jobs, the MSHV and WHP debug-kernel microVM tests, the development release, and the performance baseline, run on dev pushes only.

Notes for reviewers

  • Drafts nvx-rs: prepare the CLI's move onto the store and supervisor #675 and nvx-rs: plan sandbox launches with the image format and upper #676 also edit nvx-rs/src/host/cli/sandbox/mod.rs, nvx-rs/tests/cli.rs, nvx-rs/tests/fixtures/cli.json, the fake OpenVMM, and nvx-rs/README.md; whichever lands second rebases. Here, cli/sandbox/mod.rs only moves its network options to cli/network.rs and choice/integer to cli/args.rs, and widens a few helpers to pub(super).

  • cli.json changes only by:

    • the 2 × 170 new run, scenarios;
    • the 8 recorded top-level usage strings, which now list run and wrap, as argparse wraps them;
    • the comma after each section's former last scenario;
    • its generator line.

    No existing scenario changed otherwise.

Copilot AI balanced review requested due to automatic review settings October 21, 2026 13:41

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Cross-platform launch, terminal, and interrupt behavior still needs human confirmation against the pending real-hypervisor integration results.

1 open issue
What changed in this PR

Moves nvx.py run onto the Rust CLI, completing phase 4 of NVX runtime consolidation.

Changes:

  • Ports run validation and OpenVMM launch construction to Rust.
  • Preserves Python argument handling and CPU-profile guidance.
  • Extends fake OpenVMM coverage and updates documentation.
File Description
scripts/​test_time_abi.py Tests guidance applicability.
scripts/​test_nvx_tools.py Tests delegation and memory parity.
scripts/​nvx.py Delegates run to Rust.
scripts/​nvx_tools/​time_abi.py Exposes guidance applicability.
scripts/​nvx_tools/​sandbox.py Removes Python share helpers.
scripts/​nvx_tools/​nvx_rs.py Updates delegation documentation.
nvx-rs/​tests/​fake_openvmm.rs Tests fake boot and restore.
nvx-rs/​tests/​cli.rs Adds run and interrupt coverage.
nvx-rs/​src/​testing/​fake_openvmm.rs Supports unmanaged microVM runs.
nvx-rs/​src/​host/​cli/​sandbox/​mod.rs Shares network and launch helpers.
nvx-rs/​src/​host/​cli/​run.rs Implements the run command.
nvx-rs/​src/​host/​cli/​network.rs Centralizes CLI network options.
nvx-rs/​src/​host/​cli/​mod.rs Registers run.
nvx-rs/​src/​host/​cli/​args.rs Shares value parsers.
nvx-rs/​src/​host/​artifacts.rs Defines guest memory defaults.
nvx-rs/​README.md Documents run and parity coverage.
doc/​usage.md Explains delegation and error precedence.
doc/​run.md Updates backend documentation.
doc/​project-structure.md Reflects runtime consolidation.
.github/​workflows/​ci.yml Updates backend comments only.
.gitattributes Updates fixture provenance comments only.

Review effort: Balanced

Comment thread nvx-rs/src/testing/fake_openvmm.rs Outdated
@ppenna
Pedro Henrique Penna (ppenna) marked this pull request as ready for review October 21, 2026 15:08
Copilot AI balanced review requested due to automatic review settings October 21, 2026 19:36

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The cross-platform launch migration needs human review and confirmation of the pending KVM, MSHV, and WHP acceptance gates.

No open issues 🎉

1 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The cross-platform launch migration warrants final human review, and an executable-path normalization regression remains unresolved.

1 open issue

Review effort: Balanced

Comment thread nvx-rs/src/host/cli/run.rs Outdated
Copilot AI balanced review requested due to automatic review settings October 22, 2026 20:44

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The cross-platform migration of interactive VM launching and filesystem-policy forwarding warrants final human review.

2 open issues

Review effort: Balanced

Comment on lines +459 to +461
let run = !arguments
.iter()
.any(|argument| argument == "--microvm-lifecycle" || argument == "--microvm-sandbox-block");
Copilot AI balanced review requested due to automatic review settings October 22, 2026 21:05

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The cross-platform launch migration warrants final human review of process, interrupt, and terminal compatibility despite reported passing CI and no unresolved findings.

1 open issue
1 resolved since last review

Review effort: Balanced

Extend nvx::testing::fake_openvmm, and so nvx-fake-openvmm, to the
OpenVMM command lines of nvx.py run, so that the golden scenarios can
record and replay its runs, and nvx-rs run can run them end to end,
without a VM. A command line with neither --microvm-lifecycle nor a
sandbox block device selects them; the managed and one-shot launches
of instances, sandboxes, and aci_edge_sandboxes are unchanged, but for
the policy paths that the fake now resolves, as described below.

- Such a run takes --processors, a positive count, at most one
  --cpu-profile, and either a cold boot, with --memory and an optional
  --memory-capacity in MiB and an existing --kernel and --initrd, or
  the restore of --restore-snapshot, with optional
  --restore-processors, which cannot exceed --processors,
  --restore-memory, and --restore-ready-path, and none of the options
  of a cold boot. It takes no workload identity, control console, or
  capability, and its --cmdline is optional.
- It shares as a sandbox does: one --mount of a directory, or the
  --mount-child children of an aggregate at /run/nvx/shares. A
  relative policy path names a path in the only share, as OpenVMM
  resolves it, and every policy path must exist in a shared directory.
- It runs no guest: after fake_boot_delay_ms, it prints that it booted
  or restored the microVM on standard output, writes an outcome report
  of operation "run", in OpenVMM's form, if --microvm-report asks for
  one, and exits with 0, or with fake_exit_status=STATUS. A restore
  whose snapshot directory is missing fails with status 1 and no
  report. A cold boot records its command line next to the kernel, and
  a restore, which has none, next to its snapshot.

The fake decides whether each --mount-deny, --mount-allow, and
--mount-write path of any launch lies in a shared directory as
OpenVMM's attribute_microvm_filesystem_policy_paths does: once it has
resolved both the path and the root of every directory share. It used
to compare them as given, which refused every valid policy path of an
aggregate whose roots are relative to the working directory: nvx.py
run and nvx-rs run pass each share's host path as given, as in
--mount /a,share --mount /b,other, while the policy paths of several
shares must be absolute. A path that does not exist, or that resolves
outside every shared directory, is still refused with the same
message, and a file share still contains no policy path.

The options of these runs stay unsupported arguments of every other
launch, whose messages and checks keep their order. New tests check
both forms and their refusals, run a boot and restores of the binary,
and run boots whose relative share roots hold absolute policy paths,
with one share and with an aggregate, and the refusals of a path
outside every share and of a missing one.

Part of #638.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port command_run and the run parser of scripts/nvx.py to a run
command of the nvx-rs CLI, so that nvx.py run can run nvx-rs as
nvx.py sandbox does (#638). nvx-rs run takes every option of nvx.py
run, with the same names, defaults, types, choices, usage, and help,
and applies the checks of command_run in its order and with its
messages: the network options that need each other, the options of a
restore, the memory capacity, the guest of a restore, and the egress
policy file, then the OpenVMM executable, kernel, and initramfs, the
shares, their owner and policy paths, and the 1024-byte budget of the
kernel command line. It prints >> COMMAND and, unless --dry-run is
given, runs OpenVMM with the standard streams, as sandbox run does,
and exits with its status.

- A cold boot uses the guest's default memory unless --memory-mib
  gives it, which Guest::default_memory_mib now holds beside the other
  properties of guests. A restore takes no kernel, initramfs, or
  nvx_share= tokens.
- One directory share is a --mount, and several shares, or a file,
  the children of an aggregate, built from the primitives of
  nvx::host::sandbox::mount and checked by validate_run_share_targets,
  which only tests used. The policy paths are host paths, grouped by
  kind and absolute with an aggregate, as nvx.py run passed them.
- The integer options convert as int() does, and the processor counts
  are checked after the conversion, so that an invalid choice shows the
  converted value; paths print as pathlib prints them.
- A hidden --cold-boot-guidance=TEXT option, which the usage and help
  leave out, holds what to print to standard error after OpenVMM fails
  a cold boot, such as the CPU-profile guidance that nvx.py derives
  from the CPU catalog that doctor and benchmark also use. It is never
  printed after a dry run, a restore, an error of nvx-rs, or Ctrl-C.

sandbox and run share their network options through a NetworkOptions
type, which parses them and ports _resolve_network_egress_rules and
_extend_network_arguments, and argparse's choice and int conversions
move to the parser module; run reuses the OpenVMM runner, hypervisor
selection, and path forms of sandbox, whose code is otherwise
unchanged. The top-level parser lists run, so its usage wraps, as
argparse wraps it, in the four usage errors of each section of cli.json
that show it and in tests/cli.rs.

scripts/nvx_rs_run_fixtures.py, adapted from the retired
scripts/nvx_rs_fixtures.py, records nvx.py run from its Python
implementation into 170 scenarios in each section of cli.json, whose
names start with "run, ", with nvx-fake-openvmm as OpenVMM and without
the CPU-profile guidance, which depends on the host: the usage errors
and help of the parser, the refusals of command_run, dry runs of every
guest, cold boots, restores, shares, shared files, policy paths,
owners, network options, egress policy files, CPU profiles, outcome
reports, and the budget, and runs of the fake. It recorded the windows
section with CPython 3.14.7 on Windows and the posix section on Linux,
and its --parity mode plays both with nvx-rs run without a difference.
tests/cli.rs also checks that the guidance follows a failed cold boot
only, the default memory of each guest, the program name of --prog, a
run whose aggregate shares directories by relative roots with absolute
policy paths, as nvx.py run passes them, through to OpenVMM, and on
Unix, a cold boot that Ctrl-C ends without guidance.

Part of #638.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py run now runs `nvx-rs --prog nvx.py --root REPO run` with the
options that the user gave, with nvx.py's standard streams, and exits
with the status of nvx-rs, which ports command_run: its checks, the
OpenVMM command line with its shares, network options, and kernel
command-line budget, and the run of OpenVMM on the terminal (#638).
Python no longer builds an OpenVMM command line for the command.

argparse stays the front end, so that the help, abbreviations, and the
usage errors that it reports (exit 2) work as before, and the help is
unchanged byte for byte at 80 and 120 columns. Every option keeps its
name, metavar, string choices, and help, but none has a type converter
or a default; --processors and --restore-processors show their choices
as metavars, since nvx-rs converts their values before it checks the
choices, as argparse does for an option of type int. _SandboxOption
becomes _NvxRsOption, which records the options of run and sandbox
alike, under their canonical names, in command-line order, and with
their values attached.

nvx.py keeps the CPU-profile guidance, whose CPU catalog in time_abi.py
also serves doctor and benchmark: for a cold boot that is no dry run,
on a host whose CPU no built-in profile serves, with auto, or with host
on another vendor's CPU, it passes the guidance in the hidden
--cold-boot-guidance option of nvx-rs run, which prints it only after
OpenVMM fails the cold boot, so that an option that nvx-rs refuses,
with a status that OpenVMM can also return, gets none. Only then does
nvx.py look up the OpenVMM executable that the guidance names, with
nvx-rs artifacts, as time_abi.cpu_profile_guidance_applies, which
host_cpu_profile_guidance now uses, tells it.

Usage errors keep their status and form, but which of two errors in
one command line is reported changes, as it did for nvx.py sandbox
(#359): an invalid choice, an unrecognized argument, or a missing value
that argparse reports, or a request for help, wins over a value that
nvx-rs refuses, where nvx.py reported whichever came first. doc/usage.md
and the README say so.

scripts/nvx_tools/sandbox.py, _validate_run_share_targets,
_resolve_network_egress_rules, _extend_network_arguments, _run_openvmm,
and the rest of command_run go, and MOUNT_OWNERS moves into nvx.py; the
CI change classifier no longer excludes sandbox.py from the inputs of
the platform jobs, and its test takes openvmm_process.py as a module
that only the microVM tests run. With nvx.py running nvx-rs, the run
scenarios of cli.json are final, so scripts/nvx_rs_run_fixtures.py goes
too, and the fixture's metadata, .gitattributes, tests/cli.rs, the
README, and doc/project-structure.md say so.

The tests of run and sandbox.py whose cases the golden scenarios or the
Rust tests now cover go. New tests check the recorded arguments, with
abbreviations and attached values, the call of nvx-rs and its status,
the usage errors that argparse keeps, a missing nvx-rs, the guidance
only where it applies, with the quoted OpenVMM and the backend, and the
artifact lookup only for it, and that the recorded help of nvx-rs run
and its dry runs of each guest match the help of nvx.py run and the
default memories of guests.py, which test-microvm also uses. With
NVX_RS set, the real nvx-rs runs a dry run, a usage error with
nvx.py's name, and a refused option; on POSIX hosts, a cold boot that
OpenVMM fails gets the guidance, and a failed restore none; and on
Linux, OpenVMM still restores the terminal that its console made raw.
The agent tests compute the child names that they need themselves.

The test-microvm scenarios that run nvx.py run now run nvx-rs run;
doc/usage.md, doc/run.md, the nvx_rs.py docstring, and the comment of
CI's nvx-rs build say so.

Part of #638.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 23, 2026 03:48

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The cross-platform migration of VM launch, terminal, and interruption behavior warrants final human review despite extensive parity coverage and reported passing CI.

1 open issue

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

nvx-rs: make the nvx-rs CLI the backend of nvx.py run

2 participants