Repository navigation
Ship nvx-rs in NVX releases - #598
Open
Pedro Henrique Penna (ppenna) wants to merge 47 commits into
Open
Pedro Henrique Penna (ppenna) wants to merge 47 commits into
Pedro Henrique Penna (ppenna) wants to merge 47 commits into
Conversation
Replace native C helper compilation with the shared Rust guest-binary builder introduced by #528. Build one checked set of all nine executables per initramfs, install it at the established /sbin/nvx-* paths, and record the aggregate source and individual binary digests. Remove the obsolete C compiler flags, helper inventories, and build functions. Keep C test-only flags local until the C-driven suites are deleted later in this stack. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build one checked set of Rust guest executables in the digest-pinned Rust 1.95.0 image and reuse its exact bytes in Alpine, Ubuntu, and Azure Linux through a guest-only base, so the kernel and Linux source targets never compile them. Keep the toolchain available only for Ubuntu's two fresh determinism attempts, and remove the guest C compilers from the base. Bind reusable binary sets to normalized Rust-source, toolchain, target, profile, and executable digests, and refuse the output of an earlier build that a rebuild would upgrade in place. Record all nine helpers in the Azure Linux package manifest. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover the nvx-rs guest inputs in every image identity, package the crate without its target directory, and declare the exact Rust toolchain and musl target in SOURCE-MANIFEST.json. Verify those pins and inputs from nvx.py verify and release packaging. Make device-I/O benchmark provenance validate the shared Rust source digest, and bind the Azure Linux cache input to the newly tracked files. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Hash the complete nvx-rs tree except target, the exact toolchain pin, and the shared build modules into every guest cache, and advance the guest cache schemas. Because hashFiles hashes contents only, also append the path-aware Rust source digest that the guest package manifests record, which the new guest-binary-source-identity command prints, so a module move that keeps every file's bytes still rebuilds the images. Treat nvx-rs changes as OpenVMM VMM-test inputs so they rebuild and boot the affected images. Keep Copilot setup and both Linux setup paths on the pinned musl target, and revalidate Copilot setup for every nvx-rs change. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the nine remaining guest C implementations and the C-compiled Python suites whose behavior the nvx-rs unit, guest-binary, loopback, and real-VM tests now cover. Drop the deleted modules from both CLI validation legs. Add a repository regression that rejects C, header, or assembler sources outside the OpenVMM submodule. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the one pinned build shared by native builds, Docker, and CI, the nine binaries shipped by every guest, their provenance, cache inputs, test ownership, and release source. Now that the Rust definitions are authoritative, replace the obsolete source comparisons with literal wire values and remove C-era wording. Extend the third-party notices for the Rust standard libraries, compiler_builtins and its compiler-rt-derived routines, libc, nix, getrandom, and their linked transitive crates, and musl's complete guest-binary scope. Closes #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port scripts/nvx_tools/egress_policy.py, which compiles the files of
nvx.py's --network-egress-policy-file option, to
nvx::host::network::policy, so that the nvx-rs CLI can replace it.
Python stays authoritative until nvx.py delegates to nvx-rs. The port
accepts the same files, compiles them to the same rules, and fails with
the same messages:
- compile_policy_file reads at most 1 MiB and one byte of a file, and
compile_policy_bytes and compile_policy compile its contents or an
already parsed serde_json value. The CompiledEgressPolicy result has
the serde form {"allow": [...], "deny": [...]}.
- The format keeps Python's root fields, the MXC and legacy flat rule
forms, the tcp, udp, icmp, and any protocols, and TCP and UDP port
ranges, and runs every check in Python's order with its message.
- The lowering keeps Python's steps: address-only rules, rules for every
port of ICMP, TCP, and UDP widened into the address-only cover, the
sweep over TCP and UDP port boundaries, the budget of 256 native rules
that both families of an action share, and the sort by family,
address, prefix length, protocol, and ports.
network.rs becomes network/mod.rs, beside address.rs, the networks and
address intervals, and lowering.rs, the rule model and the lowering.
openvmm_arguments still expands NetworkPolicy rules as before.
The format depends on Python semantics, which a crate-private
host::python module provides for this and later ports:
- python::json reads a document as json.loads does, with hooks in the
place of object_pairs_hook and parse_int: NaN and Infinity are floats,
numbers beyond a float's range are infinities, integers keep their
digits, and a \ud800 escape stays an unpaired surrogate. It keeps
values in an arena and never recurses. The policy reader's hooks
refuse a repeated field when its object closes and an integer of more
than 64 digits when it is read, so their errors surface in Python's
order. serde_json, which nvx-rs already depends on, cannot do this: it
refuses NaN, Infinity, 1e400, and lone surrogates, which Python reads
and the compiler then reports with its own messages. No dependency is
added.
- python::text holds strings that may contain unpaired surrogates,
ordered by code point as Python orders them, and shows a surrogate as
\udXXX, which is how Python prints it.
- python::path gives str(pathlib.Path(path)) for POSIX and Windows
paths, as CPython 3.14 splits them, and the normalized path that
Python opens, so that messages name files as nvx.py does.
Networks are read as ipaddress.ip_interface reads them: host bits are
cleared, IPv4 netmasks and hostmasks, prefix lengths with leading zeros
up to Python's 4300-digit int limit, and embedded IPv4 addresses are
accepted, and IPv6 scopes are found and refused. They print as Python
3.13 and later print them, with IPv4-mapped addresses in dotted form.
One input differs. Python's decoder raises RecursionError for a
document nested more deeply than its stack allows, at a depth that
depends on the Python version and the platform. This reader reads any
depth, so such a document fails as an invalid policy instead.
The tests port each case of scripts/test_egress_policy.py under its
name. The cases that mock ipaddress.summarize_address_range and
_merge_intervals to bound the work read thread-local counters of the
lowering instead. Unit tests cover the JSON reader, strings, paths, and
networks against values that Python 3.14 produced.
Part of #359.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add scripts/generate_egress_policy_fixtures.py, which runs the Python
compiler of scripts/nvx_tools/egress_policy.py over a corpus of policy
files and records each file with its outcome, its native rules or its
error message, in nvx-rs/tests/fixtures/egress-policy.json. The Rust
tests replay every entry through compile_policy_bytes and, from a real
file, compile_policy_file, and need no Python, so the fixture outlives
the Python compiler, which a later change in this stack deletes.
The corpus holds 784 policy files:
- every file that scripts/test_egress_policy.py compiles, which the
script captures by running that suite, except those of the two tests
whose files exceed 64 KiB, which pin their complete outcomes in
Python and in their Rust ports;
- hand-written cases that reach every message and JSON edge: syntax
that json.loads refuses, NaN, Infinity, floats, Booleans, and 64- and
65-digit integers as ports, repeated fields and long integers in
document order, unpaired surrogates in names, networks, scopes, and
protocols, unknown fields in code-point order, each check of both
rule forms in order, network spellings as ip_interface reads them,
the rule limit across protocols and families, and files at and beyond
the 1 MiB limit; and
- 200 random policies from seed 359 that mix both rule forms and both
families, dense and sparse networks written with host bits, masks,
and exceptions, overlapping protocols and port ranges, crowds of
rules near the limit, broken fields, and varied, truncated, and
altered JSON text.
The fixture also records the errors for a missing file and a directory,
and how PurePosixPath and PureWindowsPath print 55 paths, which the
tests compare with python::path. Messages name the policy file
{path}, and show unpaired surrogates as Python prints them. Inputs are
text, or hexadecimal for bytes that are not UTF-8, with runs of 256 or
more characters recorded as repetitions, and each entry is a line of
ASCII JSON, about 1.1 MB in all.
The script needs Python 3.13 or later, whose ipaddress module prints
IPv4-mapped IPv6 addresses in dotted form, as the port does. Its output
depends only on the seed and the corpus size, so --check verifies that
the fixture is current. --random-cases and --output write larger
corpora, and NVX_EGRESS_POLICY_FIXTURE points the tests at one. The
script passes ruff and strict pyright on Linux and Windows.
Part of #359.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx::host::network::openvmm_arguments, which the OpenVMM backend of aci_edge_sandboxes calls, expanded each EgressRule into OpenVMM rules on its own, then sorted and deduplicated the strings. It now lowers each action's rules with the lowering of egress policy files, so that NVX has one lowering. An EgressRule is a policy file's MXC rule whose empty to or ports stands for an absent field, and messages name its fields after EgressRules::name, where a file's messages say allow or deny, as in "network.egress.allow[0].to[1].cidr is not a valid IPv4 or IPv6 CIDR: x". openvmm_arguments keeps the checks that only a NetworkPolicy has, which the Python compiler lacks: no device for a closed posture, the /29 limit of host-loopback access, and the rules of host-loopback forwards. The check that no deny rule covers the guest network of forwards now reads the deny rules as the lowering parsed them, so it follows the same normalization. Before, it parsed them with the stricter Cidr and missed a network written with host bits, which the old expansion also refused. The observable changes, which aci_edge_sandboxes/README.md lists as breaking changes: - The OpenVMM rules follow family, address, prefix length, protocol, and ports, instead of string order. - Merging destinations and port ranges, and subsuming narrower rules, usually needs fewer rules, so some policies beyond the limit of 256 now fit: 256 adjacent /24 destinations become one /16 rule. Port ranges that overlap only partly across networks can also need a rule for their union. - The backend's rule checks report the compiler's messages. Through AciEdgeSandbox, whose request validation runs first, only the limit message, now "network.egress.allow emits at most 256 native rules", changes. The error codes are unchanged. - Without that validation, the backend accepts networks with host bits and IPv4 netmasks, and refuses port 0, which it passed to OpenVMM. Tests: the nvx-rs network tests pin the new order, merging, messages, and coverage check. ipv6_rules_expand_exactly, now ipv6_rules_lower_exactly, expects ::/0 first. inexpressible_rules_are_rejected, now invalid_rules_are_rejected_with_the_policy_compilers_messages, checks each message, and its 300-destination rule, which now lowers to one rule, moved to rules_lower_like_egress_policy_files. exceptions_that_split_a_network_too_finely_are_rejected expects the limit message. network_policies_convert_field_for_field expects the new ICMP message and also checks the scope and limit messages. The other aci_edge_sandboxes tests, including the fake-OpenVMM test that pins the launch arguments, pass unchanged. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Describe nvx::host::network::policy in nvx-rs/README.md: what it ports and that Python stays authoritative until nvx.py delegates to nvx-rs, the policy format, the lowering and the order of its rules, how openvmm_arguments, and so aci_edge_sandboxes, share that lowering, the fidelity of its messages and the one input on which it differs from Python, and the parity fixture with its generator. The module table gains the policy module, and its network row describes NetworkPolicy and the shared lowering instead of the expansion. doc/project-structure.md lists the generator, the parity fixture, and the compiler in its descriptions of scripts/ and nvx-rs/. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
An MXC rule pairs each family of its destinations with each of its port selectors, and `combine`, formerly `mxc_rules`, gave every pair its own copy of the family's address intervals. Python's `_parse_mxc_rule` and `_parse_legacy_rule` share one tuple among the rules of a family instead. A policy file well within the 1 MiB limit, such as 8192 destinations, none adjacent to another, with 8192 port selectors, made the port copy 2 GiB of 32-byte intervals, and the unions of `_compile_family` flattened the shared addresses once for each selector again. - `Rule.addresses` is an `Rc<[Interval]>` that the rules of one rule's selectors share: for MXC rules, legacy rules, and the typed rules of `openvmm_arguments` alike, which all build their rules with `combine`. - The union of the rules for every protocol, and of those for every port of each protocol, merges each distinct address set once. Python merges the same addresses once for each selector; the union is the same. - The port sweep asks once for each address set whether `covered` holds it entirely, rather than once for each rule. - An `AddressSet` compares rule addresses by value, as Python's tuples compare, and compares a shared allocation with itself without reading it, as CPython's dictionaries do. The sweep counts its active rules by it, as `_lower_protocol_rules` counts them in a `Counter`. The output, the errors, and the order of both are unchanged: the parity fixture and the ported Python tests still pass. New tests check that the rules of a rule's selectors share one allocation, that address sets compare by value, that the unions read shared addresses once (64 intervals rather than 4096), and that three 8192-by-8192 policies compile as Python 3.14 compiles them: to `0.0.0.0/0` with a covering rule, and to the 256-rule limit errors without one. Python takes 15 to 27 seconds for each. Before this change, the debug build of that test took 171 seconds and peaked at 4.05 GiB; it now takes 0.5 seconds and 48 MiB. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port SandboxLaunch, SandboxLayer, SandboxMount, and the helpers of scripts/nvx_tools/sandbox.py to a new nvx::host::sandbox module, so that the nvx-rs CLI can launch sandboxes as nvx.py sandbox does. Python stays authoritative and unchanged until nvx.py delegates to nvx-rs. The port keeps the Python contract byte for byte: the limits, option semantics, OpenVMM arguments, kernel command line, and messages, with refused requests as Unsupported, missing images as Unavailable, and failed host queries as Io errors in Python's [Errno N] form. - Layers: one to three EROFS layers, distro, runtime, and custom, ordered by role, with duplicate roles refused in sorted order. A --layer value is ROLE,PATH,EROFS_UUID, and the UUID takes every form that uuid.UUID reads: braces, urn:uuid:, no hyphens, and uppercase. Layer and scratch paths with ',' or ';' are refused. - Settings: an absolute entrypoint and nonempty arguments without whitespace, a lowercase RFC 1123 hostname label, a non-root workload identity, 65534:65534 by default, which parse_workload_identity reads as int(field, 10) does, and positive limits, with nvx_pids_max one above the request. - Shares: --mount GUEST_TARGET,HOST_PATH[,ro|rw], whose target ends at the first comma and whose mode follows the last one; the target rules, at most 4096 UTF-8 bytes, and the reserved targets; at most 128 denied, allowed, and writable paths each; the overlap and single-owner checks; caller ownership refused on Windows; file and directory shares; the aggregate at /run/nvx/shares with children named INDEX-SHA256(TARGET)[:32] and its nvx_share= tokens; and the host-path checks, by Path.resolve() and (st_dev, st_ino) lineage. - Kernel command line: NUL and the nvx_* and virtfs_* tokens refused, the tokens in Python's order, and the 1024 bytes that OpenVMM leaves of the 2048-byte x86 command line, which also hold OpenVMM's bootstrap tokens for the shares and the NUL. The private python module that the egress policy compiler introduced gains what these rules rely on, after CPython 3.14: builtins, with str.isspace, which unlike char::is_whitespace includes U+001C to U+001F, str.split() and str.strip(), repr of a str, and int, with signs, underscores, Unicode digits, surrounding whitespace, and the 4300-digit limit; uuid, for uuid.UUID; unicode, the Unicode 16.0.0 tables of str.isprintable and int; and fs, the queries of os.path and os.stat, with Path.resolve() ported from posixpath.realpath and ntpath.realpath. On Windows, os.stat identifies a file by the 64-bit volume serial number and 128-bit file ID of FILE_ID_INFO, as Python 3.12 and later do, through the windows-sys features that the crate already enables. Operating-system strings that are not valid Unicode, which Python would carry as surrogates, are refused. python::path gains PurePath, a Unicode path of either pathlib flavor with its parts, parents, joins, and case-insensitive comparison on Windows, and Flavor, with the splitroot, join, isabs, and split of posixpath and ntpath, and ntpath.normpath. One parser serves both python_path and PurePath, so the printed and the pure paths cannot disagree. Windows paths follow CPython on Windows, whose splitroot and normpath are C functions that index UTF-16 code units: a character beyond the Basic Multilingual Plane before a colon is no drive there, although Python's own ntpath, which other hosts use, reads one. scripts/generate_egress_policy_fixtures.py becomes scripts/nvx_rs_fixtures.py, one generator for every Python-parity file of nvx-rs, which now also writes the Unicode tables, with one --check for all of them. It needs CPython 3.14, whose Unicode database the tables record, and says so; --output and --random-cases still write a larger egress policy corpus. The egress policy fixture regenerates byte for byte, except its comment, which names the script. Aggregate child names need SHA-256 on the host, so the host feature enables sha2 0.10, which the bundled build script already uses and locks, as does the aci_edge_sandboxes agent, instead of another crate. The unit tests port the launch-profile cases of scripts/test_nvx_tools.py, and check require_file, share kinds, host-path identity, and bind and hard-link identity on temporary directories on Linux and Windows. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port the pure checks that scripts/nvx.py applies to the options of nvx.py sandbox and nvx.py run to nvx::host::sandbox::options, so that the nvx-rs CLI keeps their order, semantics, and messages and stays a thin layer over the library. - SandboxOptions::check: the checks of command_sandbox that precede the launch, in its order: --dry-run outside run, --environment with --environment-file, managed exec options outside exec, the managed exec environment, network policy options outside run and provision, systemd entrypoints, --outcome-report outside run and exec, share options without --mount, --mount outside run and provision, --net without --network-profile or the reverse, and a missing --layer or --scratch. - sandbox_mounts, from _sandbox_mounts: with one --mount, every --mount-deny, --mount-allow, and --mount-write names a path in it; with several, each names a path in the --mount before it. Each share takes the kind that mount_kind finds for its host path. - validate_sandbox_systemd_policy and SYSTEMD_ENTRYPOINTS: a distro layer's manifest, next to it, must be a JSON object of format 1 that names the layer and its SHA-256 digest and lists packages with names, none of them systemd. The format is compared as == compares it, so true and 1.0 match, and the manifest is read with universal newlines, as read_text reads it. - validate_run_share_targets, from _validate_run_share_targets: the guest targets of several nvx.py run shares, or of a shared file. - validate_exec_environment and exec_environment_from_file: at most 256 nonempty KEY=VALUE entries of at most 4096 UTF-8 bytes, without NUL or a repeated key, from --environment or from a JSON array in a file of at most 1 MiB, as encode_exec_environment checks them. These checks fail with the messages of Python's JSON decoder and UTF-8 codec, so the private python module gains them: - python::json keeps its API, its hooks, and its arena, but its reader is now a port of the C scanner of CPython 3.14, scan_once_unicode and the functions that it calls, with the checks of JSONDecoder.decode around them. It accepts what the previous reader accepted, as the egress policy fixture and tests confirm, and a refusal now carries the message and the position of Python's JSONDecodeError, such as Expecting ',' delimiter, instead of the offset at which reading stopped, which is not always Python's position: an unterminated string fails at its opening quote. syntax_error formats it as str() of the error, with its line, column, and character. Loads are the hooks of json.loads without arguments, whose int refuses more than 4300 digits, and Document::get reads a member as the dict that Python builds does. - python::text gains PyText::encode_utf8 and decode_utf8, with the messages of UnicodeEncodeError and UnicodeDecodeError, so PyText is the one string type that holds surrogates. - python::path gains name and with_name, and python::fs exists. Argument parsing and type conversion stay with the CLI, as do the lifecycle operations. The unit tests port the option cases of scripts/test_nvx_tools.py: the attribution of policy paths to shares, file shares, the systemd policy, the run share targets, and the environment rules, with the environment files and distro manifests on temporary files. The JSON and codec errors have tests of their own. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record what the Python code does with fixed and seeded random inputs, and compare the Rust port with those records in golden tests, so that the port stays faithful byte for byte without Python at test time, also after nvx.py delegates to nvx-rs and the Python code goes away. scripts/nvx_rs_fixtures.py now also writes three fixtures to nvx-rs/tests/fixtures, which its --check covers with the others: - python.json: the str.isspace, decimal digit, and str.isprintable tables; repr, str.split(), str.strip(), int, and uuid.UUID of strings; str(), parts, name, parents, and is_absolute of pure paths, joins, and Windows path equality; ntpath.normpath; and the values and errors of json.loads and of the UTF-8 codec. - sandbox.json: parse_workload_identity, SandboxLayer.parse, parse_mount_value, validate_mount_target, aggregate_child_name, SandboxMount with its OpenVMM arguments, command-line fragment, and absolute paths, and SandboxLaunch with its OpenVMM arguments and kernel command line. The launches include, for each kind of share set, command lines that fill the 1024-byte budget exactly and that exceed it by one byte. - sandbox_options.json: _validate_run_share_targets, _sandbox_mounts, encode_exec_environment, the --environment-file rules of command_sandbox, which runs with exec_workload patched, and _validate_sandbox_systemd_policy, on temporary files. The generator patches sandbox.Path with pure POSIX and Windows paths whose current directory is fixed, so that the cases of both flavors come out the same on any host: CPython 3.14 writes identical files on Linux and Windows. It writes ASCII JSON with one case per line. The random corpus is mostly valid input, so that most cases reach the arguments and command lines, and invalid at a fixed rate. Each fixture has a random generator of its own, so the egress policy fixture is unchanged. The Rust golden tests read the fixtures with serde_json, run the same inputs through the port, and report each case that differs. They run the cases of both path flavors on every host, through the internal constructors that take a flavor and a fixed current directory, and the file cases on temporary files. The JSON cases go through python::json with the hooks of json.loads, so they check the reader that the egress policy compiler shares. .gitattributes marks the fixtures, about 1.2 MB, as generated, so that reviews collapse them. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extend nvx::host::artifacts to every artifact of the guests that nvx.py runs, so that nvx-rs can stand in for the runtime lookups of scripts/nvx_tools/common.py: artifact_path, openvmm_binary_path, and the require_file checks of nvx.py run and nvx.py sandbox. - Guest describes the Alpine, Ubuntu, and Azure Linux guests of scripts/nvx_tools/guests.py: their names, distributions, initramfs images and package manifests, and the Ubuntu distro layer, ubuntu-distro.erofs, with its manifest. Guest::from_name fails as guest_descriptor does. - ArtifactDir names them in a repository checkout, as build/NAME and openvmm/target/release/openvmm[.exe], where local builds write them and where nvx.py download installs a release, or in an extracted release package, as guest/NAME and bin/openvmm[.exe], where package_release stages them. ArtifactDir::from_dir tells the layouts apart as Artifacts::from_dir does, and makes the directory absolute without resolving links, so callers pass the resolved REPO_ROOT of nvx.py as it is. - require_openvmm, require_kernel, and require_initramfs check the files with the messages of nvx.py: "OpenVMM release binary not found: PATH", "Linux direct kernel not found: PATH", and "DISTRIBUTION initramfs not found: PATH". sandbox_artifacts checks the OpenVMM executable, the kernel, and the Alpine initramfs in the order of nvx.py sandbox run and start, which call the last "initramfs". - paths lists every artifact as ArtifactPaths, which serializes as JSON, with the guests in nvx.py's order, for nvx-rs artifacts. Like nvx.py, the lookup reads no SOURCE-MANIFEST.json, and checks only the files that a command needs. Artifacts and its discovery, which aci_edge_sandboxes uses, are unchanged: Artifacts::from_dir only shares its layout detection, and the systemd policy of the sandbox options now takes the distro manifest suffix from here. The tests cover both layouts, the layout of this repository, the order and messages of the checks, and the JSON form. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Describe in nvx-rs/README.md the nvx::host::sandbox module, with its mount and options submodules, and the guest artifact lookup that ArtifactDir, Guest, and ArtifactPaths add to nvx::host::artifacts, beside the discovery that aci_edge_sandboxes uses. A new Python parity section explains which Python semantics the ports keep, including the egress policy compiler, the private python module that implements them, and the fixtures that scripts/nvx_rs_fixtures.py records, with the commands that regenerate and check them, the CPython 3.14 that they need, and why CI does not check them; the egress policy section points to it. The features table names sha2, which the host feature now enables. Update the trees and the nvx-rs summary of doc/project-structure.md for the sandbox launch profile and the golden fixtures. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add the cli feature, which implies host, and the nvx-rs binary, with required-features = ["cli"]: the command-line interface to which nvx.py will delegate its sandbox, egress-policy, and artifact steps (#359). This commit adds the command-line framework and three commands; the managed lifecycle of nvx-rs sandbox follows in the next commit, and until then provision, start, exec, stop, and deprovision fail after their option checks. - nvx-rs [--root PATH] artifacts prints ArtifactDir::paths as JSON, as json.dumps(value, indent=2, sort_keys=True) writes it. The root is --root, which nvx.py will pass, or else the release whose bin/ holds the executable, or the checkout whose nvx-rs/target/ holds it, either with its SOURCE-MANIFEST.json. The NVX_OPENVMM variable names an OpenVMM executable to run instead of the root's, such as a test double; a relative path is made absolute. - nvx-rs egress compile PATH prints the rules of compile_policy_file as {"allow": [...], "deny": [...]}, each list in the order in which OpenVMM receives them, and fails with the compiler's message. - nvx-rs sandbox ports the sandbox parser and command_sandbox of scripts/nvx.py: every option with its name, default, type, and choices, and the checks of SandboxOptions::check, the egress policy file, the mounts, the launch, and the systemd policy, in Python's order and with its messages. A one-shot run prints ">> COMMAND", quoted as subprocess.list2cmdline quotes it on Windows and as shlex.join does elsewhere, and unless --dry-run runs OpenVMM with the standard streams and exits with its status. The artifact checks keep the messages and order of nvx.py, and --hypervisor auto resolves to whp on Windows and to kvm elsewhere. The parser reproduces argparse step by step: --opt value and --opt=value; values that start with a dash only after "=", unless they look like negative numbers or hold a space; the alternation of options and positionals; the optional operation positional, which defaults to run; repeated options that accumulate; and the messages and status 2 of usage errors, with unrecognized arguments reported by the top-level parser after the subcommand parsed. It does not expand abbreviations, since nvx.py will pass canonical options; that is an intended deviation. Help and usage are argparse's for 80 columns. Errors print "error: MESSAGE" and exit with status 1, as main() of nvx.py does for a ScriptError. Ctrl-C prints "Interrupted" and exits with status 130. A SIGINT handler on Unix, installed only if SIGINT is not ignored, and a console control handler on Windows record it, and a run then gives OpenVMM 0.25 s to exit before it kills it on POSIX, as subprocess.run does, and waits for it on Windows, where Python cannot interrupt the wait either. Text ends its lines as Python's print ends them, with \r\n on Windows. The private python module of #594 and #595 gains dumps, json.dumps with indent=2, sort_keys=True, and ensure_ascii; shell, subprocess.list2cmdline, shlex.quote, and shlex.join; and float() in builtins for positive_float. The sys and platform modules gain the interrupt primitives, and windows-sys its console bindings. Unit tests compare the parser with argparse, the Python helpers with CPython, and cover the discovery of the artifact root. Golden scenarios recorded from nvx.py follow in a later commit. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port scripts/nvx_tools/sandbox_lifecycle.py, and the ControlSession of
control_session.py that it uses, so that nvx-rs sandbox provision,
start, exec, stop, and deprovision keep the contract of nvx.py
sandbox: the same state directory, files, formats, locks, messages,
outputs, and exit statuses. Either implementation can operate a
sandbox that the other provisioned or started.
State, in sandbox/state.rs:
- config.json in the lowest of formats 1 to 6 that holds the sandbox,
and runtime.json, format 1, with the pid, start_time, and
control_endpoint of OpenVMM. Both are written as json.dumps(value,
indent=2, sort_keys=True) + "\n" writes them, ASCII-escaped and with
\r\n line ends on Windows as write_text produces them, through a
temporary file, chmod 0600, and a rename.
- Reading applies Python's semantics to whatever another writer left:
strict UTF-8, == comparisons of the format, str() and int() of the
fields, and the order and messages of _deserialize_launch.
Locks, in sandbox/lock.rs: lifecycle.lock is locked as Python locks
it, so the implementations exclude each other: flock(LOCK_EX |
LOCK_NB) on POSIX, and on Windows CreateFileW with read, write, and
delete sharing and FILE_FLAG_OPEN_REPARSE_POINT, then LockFile of the
first byte, as msvcrt.locking(LK_NBLCK, 1) takes it. The
standalone-regular-file checks, the retries every 25 ms within
--timeout, the hand-off waits, the "deprovisioned" tombstone, and the
removal of the directory are ported, with Windows errors classed as
CPython maps them to errno.
Process identity, in sandbox/identity.rs, is Python's: on Linux the
start time in /proc/PID/stat, with zombies as exited, and on Windows
the creation time, so that a runtime record that either implementation
wrote is valid for the other.
The lifecycle, in sandbox/lifecycle.rs:
- start generates a 32-byte capability, never all zero, writes
control.capability, passes the capability on OpenVMM's standard
input through a pipe closed before OpenVMM starts, appends its
output to openvmm.log, and starts it detached, in its own session
on POSIX and process group on Windows, with _start's command line:
the managed lifecycle, its network options in _start's order, and
the endpoint STATE/control.sock, or //./pipe/openvmm-microvm-HEX
with a uuid4 on Windows. It records runtime.json, then connects,
attaches, and pings. A failed start ends OpenVMM and removes the
runtime files in _start's order, and reports OpenVMM's exit as
_startup_exit_message does.
- exec buffers the workload's output, writes standard output, then
standard error, and exits with the workload's status. With
--outcome-report, it checks the destination before it runs and
writes {"operation_id", "outcome", "schema_version": 1} as
_write_new_json does.
- stop asks the guest to shut down, waits for OpenVMM, reads its
outcome report, removes the runtime files, and prints nothing, as
nvx.py sandbox stop does. deprovision removes the state directory.
The session, in session.rs, keeps the observable behavior of
control_session.py on nvx::host::session, whose ControlSession gains
send_request and read_response for callers that check the responses
themselves: connection retries every 25 ms until the timeout, then
"managed control endpoint did not become available: PATH"; one
deadline per operation for its request and its responses; "managed
control request timed out" or "managed control response timed out"
by the direction of the transfer; ControlEndpointClosed for a closed
endpoint, but not for a reset; a session closed after a failed
transfer; and exec's checks, in their order and with their messages,
its exit categories, and its refusals with their status and ASCII
category. Waits run in 250 ms slices, so that Ctrl-C ends them.
Intended deviations: an endpoint that a process other than the
launched OpenVMM serves counts as one that has not appeared yet, as
in nvx::host::session; OpenVMM starts without a console window on
Windows, as nvx::host::process starts it; exec refuses more than the
1 MiB of output that the guest agent forwards; and where Python would
end with a traceback, as for a configuration without a field that
_start reads, nvx-rs prints "error: MESSAGE" with the exception's
message and exits with status 1.
The base modules gain what the port needs: process::Launched waits
for, terminates, and kills OpenVMM as Popen does; the Windows platform
module locks the first byte of a file and reads exit codes; and the
python module adds value, str(), int(), and == of JSON values, and
lstat, lexists, and fstat with st_file_attributes on Windows.
Unit tests cover the formats, the JSON writer, identity parsing, the
locks, and the lifecycle helpers. The tests of control_session.py and
golden scenarios recorded from nvx.py follow.
Part of #359.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port scripts/test_control_session.py to the session of nvx-rs sandbox, in host/cli/session/tests.rs, except the cases that the tests of nvx::host::session, whose framing the session reuses, already cover. Session::connect now attaches through Session::attach, which the tests call on their own transports. The ControlSessionTests run on an in-memory stream pair, like the socketpair of the Python tests, with a guest thread that plays OpenVMM's endpoint: exec streams output and returns its status; refuses relative entrypoints, more than 64 arguments, and invalid arguments, timeouts, working directories, environments, and response timeouts before it sends anything; encodes the legacy and extended requests, with their working directory and omitted, empty, replaced, or layered environments, exactly; accepts the whole u32 timeout range; refuses unknown exit categories, including "cancelled", which the CLI never requests; and keeps the status and category of a refusal. A reset is not a closed endpoint, every transfer of an operation gets no more than what is left of its timeout, and a failed transfer closes the session. The stream tests run on the endpoints of each host, which this process serves: on POSIX, Unix-domain sockets whose closure and partial reads report a closed endpoint; on Windows, named pipes whose closure reports a closed endpoint after the data that they held, and whose read timeout closes the session. A stalled endpoint, which reads nothing, times out the request within its timeout on both hosts, and an endpoint that never appears times out with its path. The Windows tests create their pipe servers through windows-sys, so their module allows unsafe code. Not ported: the capability-pipe tests, since the pipe is nvx::host::process::spawn's, which writes the capability and closes the pipe before OpenVMM starts, and which every start of the golden lifecycle scenarios exercises, since the fake OpenVMM requires exactly the capability and the end of the file on its standard input; the mocked transfers that move nothing, which the closed-endpoint tests cover; reads that time out without losing data, which nvx::host::session's timed_out_polls_keep_partial_records covers; and the cancellation of an interrupted pipe write, since the CLI checks Ctrl-C between waits and bounds each write by its timeout. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
As the #359 discussion decided, nvx-rs sandbox start now negotiates the guest's control features, with a required set of its own rather than GuestFeatures::REQUIRED, which describes the direct agent of nvx::host::instance. This is the one intended change from nvx.py sandbox start, which does not ask. GuestFeatures::LAYERED_SANDBOX holds exactly what the CLI's launches use: EXEC_ENVIRONMENT (bit 4), because exec sends a working directory and an environment in an extended EXEC. It holds no CANCEL (bit 0): the CLI never sends APP_CANCEL, since, as nvx.py did, Ctrl-C ends an exec by closing the session, and the guest agent ends the workload when OpenVMM resets the session that it ran for. It holds no WORKLOAD_ACCOUNT (bit 2), which no sandbox launch requests and the layered root does not perform (#566). The NVX guest agents, the Rust agent of #561 and the C agent before it, advertise bits 0, 2, and 4 in sandbox mode. After its PING, start sends FEATURES. An agent that predates the request refuses it as an unsupported operation and so provides none. If the guest lacks a required feature, start ends OpenVMM and removes the runtime files, as a failed start does, and fails with error: the guest image lacks control features that layered sandboxes need (per-execution environments); use an NVX release or build that includes them which names the missing features with GuestFeatures::missing, as the reject_guest of nvx::host::instance does. Tests cover the required set, the features responses of current, older, and misbehaving agents, and, with the golden scenarios that follow, a start against guests that lack the feature or predate the request. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extend nvx::testing::fake_openvmm, and so nvx-fake-openvmm, to the sandbox profile that nvx.py sandbox and nvx-rs sandbox launch, so that both can run end to end without a VM. A --microvm-sandbox-block option selects the profile; direct launches, which nvx::host::instance and aci_edge_sandboxes use, are unchanged. - The fake accepts the sandbox's arguments and checks them as OpenVMM would: read-only distro, runtime, and custom layers and one scratch disk, each an existing file; nvx_sandbox=1, one nvx_entrypoint=, and one nvx_hostname= on the kernel command line; one --mount of a directory, or the --mount-child children of an aggregate at /run/nvx/shares; --mount-owner and --network-proxy; and the network options, which it summarizes as OpenVMM's report does. - --microvm-lifecycle managed serves the control console as before, with the capability on standard input, and its guest advertises GuestFeatures::SANDBOX_AGENT. The guest targets of directory shares, from --mount and the nvx_share= tokens of an aggregate, are directories that a workload can enter. - --microvm-lifecycle one-shot reads no capability and serves no console: it runs the entrypoint and arguments of the kernel command line on its own standard streams, /bin/echo, /bin/sleep, /bin/true, or /bin/false, with any other program not found and status 127, writes an outcome report of operation "run" if asked, and exits with the workload's status, or with fake_exit_status=STATUS. - The outcome reports of sandboxes take the form of OpenVMM's: the schema version, a random instance ID, the backend, the outcome, the network policy, and every teardown step, as JSON with a final newline. The existing knobs keep working for sandboxes, among them fake_exit_on_start, fake_guest_features, and fake_legacy_guest, which the golden scenarios and the guest-feature tests use. The fake OpenVMM tests still pass unchanged. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record what nvx.py sandbox does, as a command, and replay it with nvx-rs in tests that need no Python, so that the CLI stays faithful to nvx.py after PR D deletes the Python code. scripts/nvx_rs_fixtures.py, the generator of #595, now also writes nvx-rs/tests/fixtures/cli.json, which its single --check covers. It builds nvx-rs and nvx-fake-openvmm with cargo and plays scenarios in fresh temporary roots, each with fake guest artifacts and a copy of nvx-fake-openvmm as the root's OpenVMM executable: - 24 usage errors and helps of the sandbox parser, 69 option and launch checks of command_sandbox, 11 dry runs, 11 one-shot runs, the lifecycles of sandboxes in every configuration format, and 8 scenarios of the refusals and failures of the lifecycle operations, 215 steps in all on POSIX and 209 on Windows, where a caller-owned share, and so format 3, is refused; - nvx-rs egress compile and nvx-rs artifacts, from the Python functions that they port, compile_policy_file and the paths of scripts/nvx_tools/common.py and guests.py. Each step records its exit status, standard output and error, and the files that it leaves, such as config.json in formats 1 to 6, runtime.json, the state directory's entries and modes, outcome reports, and the command line that start passed to OpenVMM. The generator runs nvx.py in a subprocess, with artifact_path and openvmm_binary_path patched, unbuffered and with UTF-8 standard streams, no terminal size, and no colors. It replaces the root, as printed and as JSON escapes it, process IDs, start times, and random identifiers with placeholders, and the program names of nvx.py with those of nvx-rs, whose parser names have the same length, so usage wraps alike. The posix and windows sections hold the scenarios of each kind of host, recorded there; the script rewrites or checks the section of the host that runs it and keeps the other. Recording needs CPython 3.14.5 or later, whose argparse quotes the choices of an invalid choice, as nvx-rs does. The script then plays every scenario with nvx-rs, and each lifecycle with nvx.py and nvx-rs taking turns both ways, so that each provisions, starts, executes in, stops, and deprovisions a sandbox of the other, and fails if any step differs from nvx.py. Both sections pass on Windows and Linux. nvx-rs/tests/cli.rs replays the section of its host with nvx-rs, four scenarios at a time, in canonical temporary roots. It also checks what nvx-rs adds: a start against guests that lack per-execution environments or predate the features request, the default root of the executable in a checkout, a relative NVX_OPENVMM, the top-level usage errors, and on Linux, Ctrl-C during a run, which ends OpenVMM, and during an exec, which leaves the sandbox running. .gitattributes marks the fixture, about 370 KB, as generated. This changes the generator of #595 only by adding the CLI section. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Describe in nvx-rs/README.md the nvx-rs CLI that the cli feature builds, which PR D's nvx.py will run: - its commands and how it finds the artifact root: --root, which nvx.py will pass, or else the release or checkout that holds the executable, and the NVX_OPENVMM override; - the JSON forms of artifacts and egress compile, and the contract of sandbox: its options, outputs, exit statuses, outcome reports, and the state directory that it shares with nvx.py; - the guest features that start requires, and why bits 0 and 2 are not among them; - the other intended differences from nvx.py; - the test hooks, --root, NVX_OPENVMM, and the fake OpenVMM's sandbox profile, and the golden scenarios with their interoperability checks. The layout, module, and features tables, the Python parity and testing sections, and the introduction mention the CLI, and doc/project-structure.md lists it in the nvx-rs tree and summary. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
A terminal's Ctrl-C reaches the whole foreground process group, so nvx-rs sandbox run and the OpenVMM that it runs on the terminal receive it together. run_openvmm polled OpenVMM before it checked for the interrupt, so when OpenVMM had already exited from the same Ctrl-C, the run returned OpenVMM's status instead of ending as interrupted: on Unix, a SIGINT-ended OpenVMM made nvx-rs exit with 254 and print nothing, where nvx.py prints "Interrupted" and exits with 130. Python's wait raises KeyboardInterrupt once SIGINT arrives, whether or not the child has exited meanwhile. run_openvmm now checks for the interrupt after each poll and before it reports a status, and only waits for, then kills, an OpenVMM that is still running. main() also lets a recorded Ctrl-C decide how any command ends, as KeyboardInterrupt does wherever it finds a Python command, so that a command that finished while Ctrl-C arrived still prints "Interrupted" and exits with 130. A new Linux test sends SIGINT to the CLI's whole process group, as a terminal does, during a one-shot run whose fake OpenVMM the signal ends at once. It failed with status 254 before this change, in three of three runs, and passes now; the test that interrupts the CLI alone still passes. Reported in the review of #596. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py sandbox will run nvx-rs sandbox with the options that its own argparse front end read, and its type conversions, such as --layer, --workload-user, and the integer options, move to nvx-rs. A value that nvx-rs then refuses would read "usage: nvx-rs sandbox ..." and "nvx-rs sandbox: error: ...", where nvx.py printed the usage and name of its own parser. A new global option, --prog PROG, names the program in usage, help, and error messages, nvx-rs by default; nvx.py will pass its own name. Each parser now describes its syntax as a Syntax value, its command after the program's name, the usage parts of its options and positionals, and the help after its usage, instead of static usage and help text, and Exit::Usage and Exit::Help carry the text that the syntax formats for the program. format_usage ports argparse's HelpFormatter._format_usage for 80 columns: a usage that fits stays on one line; otherwise a name of at most three quarters of the width is followed by the options, and the positionals start a new line, while a longer name takes a line of its own. For the default name, every usage and help is byte for byte what the static text printed, which the golden scenarios check, and for nvx.py, whose name has the length of nvx-rs, the usage wraps as Python's does. The top-level usage lists the new option, so the eight golden steps whose unrecognized arguments the top-level parser reports, and the generator's copy of that usage, change accordingly. The top-level help describes --prog and wraps the help of --root as argparse wraps it; it had one line of 79 columns. Unit tests compare the wrapping of short and long names, and of usages without options, with CPython 3.14's argparse, and a CLI test checks the usage error of nvx.py sandbox for a root --workload-user, recorded from CPython, a long name, the top-level parser, and help. The README documents the option. This changes the CLI of #596; it adds the option and keeps every default output. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py will delegate its sandbox command, and the egress policy and artifact steps of its run command, to the nvx-rs CLI (#359). The new module scripts/nvx_tools/nvx_rs.py finds that executable, builds it when it is missing, and runs it, so that the commands that follow only choose what to delegate. Discovery runs, in order: - the executable that the NVX_RS environment variable names, which must exist; - nvx-rs/target/release/nvx-rs[.exe] in the checkout, binary_path(), where a release build puts it; and - that executable after `cargo build --release --locked --features cli --manifest-path nvx-rs/Cargo.toml --target-dir nvx-rs/target`, which runs from the repository root, so that rust-toolchain.toml selects the pinned Rust release, with its target directory pinned against CARGO_TARGET_DIR. The command line goes to standard error, and so does Cargo's standard output, so that a build never mixes with the output of the command that needs the executable. Without Cargo, nvx.py fails with a ScriptError that names NVX_RS, the build command, rustup, and `nvx.py download`, which PR E makes install a release's bin/nvx-rs[.exe] at binary_path(), as it installs OpenVMM where the OpenVMM build puts it. run() runs `nvx-rs --prog PROGRAM --root ROOT ...` on nvx.py's own standard streams, with nvx.py's repository root, and returns the status that nvx.py then exits with: nvx-rs's own, 128 + N for a signal N that ends nvx-rs, as a shell reports it, and a Windows status above 2^31 as the signed value that SystemExit passes on with every Python. Ctrl-C belongs to nvx-rs, which prints Interrupted and exits with 130. While nvx-rs runs, nvx.py replaces its SIGINT handler, so that it neither raises KeyboardInterrupt, prints Interrupted again, nor kills nvx-rs before it finishes, as subprocess.run would after 0.25 s. On POSIX hosts, it hands nvx-rs the SIGINT, SIGTERM, and SIGHUP that reach nvx.py alone, which a terminal's Ctrl-C to the whole group makes a harmless repeat, including a signal that arrives before nvx-rs has started; if SIGINT ends nvx-rs before it can report it, nvx.py prints Interrupted for it. On Windows, Ctrl-C reaches every process of the console, so nvx.py only ignores it. The handlers are installed before nvx-rs starts and restored when it ends. query() runs nvx-rs for a JSON result, with standard input closed and its output captured, and raises a ScriptError with the message that nvx-rs printed after "error: ", so that nvx.py prints what it printed for the Python code that nvx-rs replaces. compile_egress_policy() and artifacts() read `nvx-rs egress compile` and `nvx-rs artifacts` into typed results and refuse JSON of another shape. NvxRsTests covers discovery and its order, the build command, its working directory, and its output, the missing-Cargo and failed-build errors, the command line, the exit statuses, and the JSON readers. Process tests run nvx_rs.run in a fresh Python with a stand-in for nvx-rs: standard streams and statuses pass through, and on POSIX hosts, SIGINT to the process group or to nvx.py alone ends the command with one Interrupted and status 130, and signals that end nvx-rs give a shell's statuses. Mocks cover an interrupted wait and the restoration of the handlers. With NVX_RS set, a test checks the JSON of the real nvx-rs against what nvx.py reads. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py sandbox now runs `nvx-rs --prog nvx.py --root REPO sandbox` for every operation, with nvx.py's standard streams, and exits with the status of nvx-rs, which ports command_sandbox, the lifecycle of sandbox_lifecycle.py, and the control session of control_session.py (#596). Python no longer checks, converts, or runs anything for the command; the modules that nvx-rs replaced stay for the microVM tests until a later commit moves those onto nvx-rs. argparse stays the front end, so that the help, abbreviations, and the usage errors that it reports (exit 2) work as before, and the help is unchanged byte for byte at 80 and 120 columns. Every option keeps its name, metavar, choices, and help, but none has a type converter or a default: --layer, --workload-user, the integer options, --timeout, and the paths are plain strings that nvx-rs converts and checks, and nvx-rs applies the defaults, so Python duplicates neither. A _SandboxOption action records each option as nvx-rs receives it: - under its canonical name, so that `--dry` reaches nvx-rs as --dry-run, since nvx-rs does not expand abbreviations; - in command-line order, so that each --mount-deny, --mount-allow, and --mount-write follows the --mount whose path it names, which nvx-rs attributes as nvx.py did; and - with its value attached, --option=VALUE, so that a value that starts with "-" stays a value. The operation, run by default, comes first. Only options that the user gave are passed, so that the defaults live in nvx-rs alone. --prog makes the usage errors that nvx-rs reports for the values that it converts read "usage: nvx.py sandbox ..." and "nvx.py sandbox: error: ...", as argparse printed them; the name has the length of nvx-rs, so the usage wraps the same. Usage errors keep their status and form, but which of two errors in one command line is reported changes: argparse reports an invalid choice or an unrecognized argument before nvx-rs sees a value that it refuses, where it used to report whichever came first. Ctrl-C ends the command with nvx-rs's Interrupted and status 130, once. scripts/nvx_rs_fixtures.py recorded the golden fixtures of nvx-rs from the Python implementation of nvx.py sandbox, the egress compiler, and the Python semantics that the ports keep, and imported the helpers that this commit removes from nvx.py. With nvx.py running nvx-rs, the recordings are final, so the generator goes, and the fixtures stay as frozen references that a deliberate change updates by hand. Their metadata, .gitattributes, the README, the module comments that named the generator, and doc/project-structure.md say so. The CLI tests of the Python implementation go, since the golden scenarios of nvx-rs cover each of their cases, as do the systemd, environment, network-option, and policy checks, and the tests of `run` that also ran sandbox keep their `run` half. New tests check the recorded arguments for abbreviations, attached values, order, and the default operation, the call of nvx-rs and its status through main(), the usage errors that argparse keeps, a missing nvx-rs, and, through a fresh nvx.py with a stand-in for nvx-rs, shared standard streams and, on POSIX hosts, one Interrupted and status 130 after Ctrl-C to the process group. With NVX_RS set, the real nvx-rs runs a dry run of options that need an attached value, an abbreviation, and a share policy, a usage error with nvx.py's name, and a lifecycle error. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py run keeps assembling OpenVMM's command line, but the two steps of it that Python duplicated now go through nvx-rs (#359): - --network-egress-policy-file compiles with `nvx-rs egress compile`, whose JSON lists the native rules of each action in the order in which OpenVMM receives them, in place of compile_policy_file of egress_policy.py. The option checks before it stay in nvx.py, as they belong to run's own options, and a refused policy fails with the compiler's message, as before. - The OpenVMM executable, the kernel, and the initramfs of the guest come from `nvx-rs --root REPO artifacts`, in place of openvmm_binary_path and artifact_path of common.py. nvx-rs prints absolute, unchecked paths of the same files; nvx.py still checks each with require_file, in the same order and with the same messages, and names the guest's distribution as nvx-rs prints it. The lookup runs after the egress policy compiles and before the first check, so a run that fails its checks never needs nvx-rs, and the command lines, outputs, and messages of `run` are unchanged. One visible difference: NVX_OPENVMM, which names another OpenVMM for nvx-rs, now also selects the OpenVMM of nvx.py run, as it does for nvx.py sandbox. The microVM scenarios and the build, release, and benchmark code keep using artifact_path and openvmm_binary_path: they build, package, and measure the checkout's artifacts. Tests of run that pass the option checks look up the checkout's artifacts through a mixin that answers nvx_rs.artifacts without nvx-rs. The egress test checks that run passes the compiler's rules, and with NVX_RS set, that the real nvx-rs lowers the policy as the Python compiler did and refuses a scoped IPv6 network with its message before any artifact lookup. New tests check that a refused policy fails before the lookup, and that run takes the paths that nvx-rs prints, checks them in order with nvx.py's messages, and reports a missing executable as before. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Two microVM scenarios still used Python code that nvx-rs replaced (#359), which a later commit deletes: - The L3/L4 policy, port-range, and IPv6 egress scenarios compiled their policy files with compile_policy_file of egress_policy.py to record and check the native rules that `nvx.py run` passes to OpenVMM. They now compile them with `nvx-rs egress compile`, through compile_egress_policy of scripts/nvx_tools/nvx_rs.py, which nvx.py run itself uses, so the scenarios check the rules of the compiler that runs. - The public sandbox lifecycle took the names of the state files and runtime_process_running from sandbox_lifecycle.py. The names are now its own constants, as nvx.py sandbox documents them, and openvmm_running replaces the process check with the scan that the scenario already makes: an OpenVMM process with the recorded ID whose arguments name the scenario's fixture, which no other process names. kill_openvmm takes that name too. The tests of the scenarios compile their cases with nvx-rs, through discovery, as the scenarios do: the check that each port range lowers to one native rule, the IPv6 case table, and the bounded L3/L4 policy. The port-range test, which replaces subprocess.run for the public dry runs, lets nvx-rs's own runs through, and counts only the dry runs. The L3/L4 policy tests, which replace threading.Thread, which subprocess uses on Windows, take their policy's rules from a stub, and a reservation that fails before any policy compiles. The process check of the lifecycle needs no patch of its own, since it follows the scan that the tests already replace, and the kill test ends a real process that a replaced scan reports. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The managed-lifecycle scenario of `nvx.py test-microvm` boots a guest whose init hands the boot to the managed agent, and drove its control session with control_session.py, the Python client of the protocol that #359 removes. Its checks of the protocol move to an ignored integration test, tests/managed_lifecycle.rs, which drives the session with nvx::host::session, as nvx::host::instance does for nvx-rs sandbox: - A first session pings the agent and runs a workload in /tmp with exactly EMPTY= and an entry with spaces, an equals sign, and a snowman, and the workload prints them back. - A second session reaches the same guest: an explicitly empty environment holds no variable, and an exact one holds exactly its entries. - The timeouts 0, 3600001, 86400000, and 4294967295 ms are accepted. - Workloads run as the configured identity, 65534:65534, which the scenario checked only through the refused /root before. - State persists across requests, while working directories and environments do not, and a direct-mode guest has no workload machine ID. - A workload that outlives its 100 ms timeout ends with the timeout category and status 124, and the guest stays usable. - The guest refuses /does-not-exist, /etc/passwd, and /root as working directories with the cwd-failed category, the errors 2, 20, and 13, no output, and a diagnostic that names the directory, and nothing of the refused workload runs. - Stop is acknowledged, and OpenVMM exits with status 0. The test starts OpenVMM itself, with the command line, endpoint, timeout, and output directory that the scenario passes in NVX_MANAGED_LIFECYCLE_* variables, which the next commit sets. OpenVMM reads its capability from a pipe that already holds it and its end, as nvx::host::process::spawn prepares it (#440), writes its output to managed-lifecycle.log, and is killed if the test fails. The guest console of a microVM listens on TCP, and the test copies it, from the address in NVX_MANAGED_LIFECYCLE_CONSOLE, to managed-lifecycle-guest.log until OpenVMM closes it, so that the scenario can scan it. The test records OpenVMM's process ID in NVX_ADVERSARIAL_OPENVMM_PID_JOURNAL, as the scenario did, for the adversarial campaigns. Like the Python client, it bounds each exchange with the scenario's timeout. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The managed-lifecycle scenario was the last user of control_session.py, the Python client of the control protocol, which #359 leaves to nvx-rs alone. Its orchestration stays in Python, and its protocol checks run in nvx-rs's ignored test tests/managed_lifecycle.rs: - run_managed_lifecycle builds OpenVMM's command line as before, with absolute paths, since Cargo runs the test in the nvx-rs directory, and runs `cargo test --manifest-path nvx-rs/Cargo.toml --locked --test managed_lifecycle -- --ignored --nocapture` from the repository root, so that rust-toolchain.toml selects the Rust release, as test-aci-edge-sandboxes runs the aci_edge_sandboxes test. Without Cargo, it fails and says why. - The guest console still listens on TCP, since a microVM console takes no file, and the test, which starts OpenVMM, copies it to managed-lifecycle-guest.log. The time ABI monitor scans that log once the test ends, so that a violation still fails the run, before the test's failure, which it may explain. - The scenario still checks and keeps managed-outcome.json, which the adversarial campaigns collect, and that OpenVMM refuses the managed lifecycle without a control endpoint. The checks of OpenVMM's report that also hold for the layered root now run through nvx.py sandbox too: the sandbox-lifecycle scenario checks that the report of a stopped sandbox names the backend, that no network policy was requested, and that every teardown step completed, besides the managed success that it checked. managed-exec-config already covers the working directories, environments, timeouts, and refused directories for the layered root. The unit tests replace the capability-pipe test of the Python spawn with tests of the Cargo command, its environment, the absolute paths, the missing Cargo, a failed test, a violation on the console, an invalid report, and an unrefused transition. The sandbox stand-in writes a full report, and three new regressions show that the lifecycle acceptance catches another backend, a requested network policy, and an incomplete teardown. The structural test of the TCP consoles counts five, and checks that the managed lifecycle monitors its file console. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py now runs nvx-rs for the sandbox command and for the egress and artifact steps of run, and the microVM tests drive the control protocol through nvx-rs, so the Python implementations that nvx-rs ported have no caller left. #359 asks that Python duplicate none of the control-protocol, lifecycle, or egress logic, so they go: - scripts/nvx_tools/control_session.py, the control-session client, which nvx::host::session and the CLI's session replace, and its tests, scripts/test_control_session.py, which nvx-rs ported to src/host/cli/session/tests.rs; - scripts/nvx_tools/egress_policy.py, the egress policy compiler, which nvx::host::network::policy and `nvx-rs egress compile` replace, and its tests, scripts/test_egress_policy.py, which nvx-rs ported to src/host/network/policy/tests.rs; - scripts/nvx_tools/sandbox_lifecycle.py, the managed lifecycle, which the nvx-rs sandbox command replaces, all of it; - the launch profile of scripts/nvx_tools/sandbox.py: SandboxLaunch, SandboxLayer, SandboxMount, the workload identity, the reserved targets, and the checks of shares, which nvx::host::sandbox replaces. sandbox.py keeps what `nvx.py run` builds its own OpenVMM command line with, as #359 leaves that assembly in Python: the --mount values, the names, arguments, and kernel command-line tokens of the aggregate of several shares, the mount owners and access-policy options, and the kernel command-line budget. Its docstring says so. The unit tests of the deleted code go with it: the managed lifecycle, lock, state, and process-identity tests and the launch-profile tests of SandboxTests, the helpers that only they used, and ControlSessionTests of test_microvm_tests.py. What remains of SandboxTests tests the share helpers of run, as RunShareTests, with the --mount parser and mount kind tested directly rather than through SandboxMount. The Validate NVX action no longer runs the two deleted test modules, and the check that it runs every scripts/test_*.py module looks for one that still exists. nvx-rs comments and README sections that named the Python files now call them former, and keep why the ports match them: the state directories, the lifecycle lock, and the rule output that an older nvx.py shares with nvx-rs. doc/project-structure.md drops the deleted modules. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Open PR #579 (jay23606, fixes #486) reserves the guest paths that the container runtime of a layered sandbox creates or binds over: it creates /run and /run/nvx below the container root, and binds its time ABI state over /run/nvx/time. A live share at one of them would be shadowed, or would expose host data to the runtime's writes. #579 changes scripts/nvx_tools/sandbox.py, which #359 retires, so this ports its behavior to the launch profile that replaced it, as the maintainer asked while #579 is open: - nvx::host::sandbox::mount: RESERVED_MOUNT_TARGETS gains the /run/nvx/time tree, and RESERVED_EXACT_MOUNT_TARGETS /run and /run/nvx, with #579's reasons. validate_mount_target refuses them with the messages that the Python check gave: `sandbox mount target /run is reserved`, and `... overlaps the reserved /run/nvx/time tree` for the tree. Other paths below /run, such as /run/secrets, /run/nvx/other, or /run/nvx/timer, stay valid. - guest/common/nvx-init-agent refuses the same targets, as #579 does, so that a guest refuses them even from a host that would not. - The guest-agent test of aggregate children gains #579's rows for /run, /run/nvx, and /run/nvx/time/state. - doc/usage.md and doc/run.md list the new reserved targets, which #579 left undocumented. tests/fixtures/sandbox.json, which the retired Python launch profile recorded, held nine targets and four shares that #579 now refuses. Its records of those cases are re-evaluated with that Python code, taken from the commit before #359 retired it, with #579's change to sandbox.py applied: 7 targets at /run, one at /run/nvx, and one at /run/nvx/time become refusals, as do 4 shares at /run, and no other record changes. The Rust unit tests check the new refusals, and paths beside them that stay valid. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Open PR #576 (jay23606, refs #556) narrows the sandbox to one EROFS distro layer and the scratch disk: the runtime and custom layers go, and their MMIO windows and interrupts stay unassigned. #576 changes scripts/nvx_tools/sandbox.py, which #359 retires, so this ports its behavior to the launch profile that replaced it, as the maintainer asked while #576 is open, with the messages and checks of #576: - nvx::host::sandbox::layer: LAYER_ROLES, the roles that --layer accepts and a launch holds, is distro alone, so `--layer runtime,...` fails with `unsupported layer role 'runtime'; choose distro`, as does any other role. LayerRole keeps the retired Runtime and Custom roles, which from_name still reads, because the configuration of a sandbox that an older nvx.py provisioned may name them. - SandboxLaunch::new checks the duplicate roles first, over every role, and then requires exactly one layer, the distro layer: `a sandbox requires exactly one read-only distro layer`, which replaces `one to three read-only layers`. As in #576, the check holds for a saved launch as much as for parsed options, so `nvx.py sandbox start` of such a sandbox fails before OpenVMM starts. - guest/common/nvx-init-agent mounts and unmounts only the distro layer, and refuses a second layer: #576's change, verbatim. - The sandbox-blocks scenario attaches the distro layer and scratch, and its guest script checks for only vda and vdb, for no vdc, and that the cmdline assigns neither 0xd0004000 nor 0xd0005000, with #576's codes 30, 31, and 32. Its unit test expects two blocks. - doc/run.md and doc/usage.md describe one distro layer, as in #576. The golden records follow. In tests/fixtures/sandbox.json, the layer and launch cases are re-evaluated with the retired Python launch profile, from the commit before #359 retired it, with #576's and #579's changes to sandbox.py applied. 168 layer values of the retired roles, 146 of which parsed and 22 of which had an invalid UUID, become refusals of their role, which is checked first; the refusals of 28 values of unknown roles list only distro; and 136 launches with retired roles now fail with the duplicate roles or the distro-layer requirement, which come before their other checks. No other record changes. The launch harness builds a retired role's layer as a saved configuration names it, since --layer no longer parses it. In tests/fixtures/cli.json, by hand, for POSIX and Windows alike: - `validation: invalid layer role` lists only distro, and a new `validation: retired layer role` shows `--layer runtime,...` refused with the same usage error; - `dry run: every layer and option` takes the distro layer alone, so its OpenVMM command loses the runtime and custom blocks and their nvx_layer tokens; - `lifecycle: network and launch options` provisions the distro layer alone, so its config.json and its OpenVMM arguments lose the runtime layer. A new CLI test edits a provisioned sandbox's configuration to add a runtime layer, as an older nvx.py wrote it, and checks that start refuses it with status 1 and the message above, leaving no runtime state, and that deprovision still removes it. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py now runs nvx-rs for its sandbox command and for the egress and artifact steps of its run command, and the microVM jobs drive both, in their scenarios and in the steps that call nvx.py sandbox directly. Without an executable, nvx.py would build one with Cargo in every microVM job, so CI builds it once per operating system and hands it over, as it hands over OpenVMM: - The NVX CLI jobs, which every microVM job of their operating system already waits for, build it with `--release --locked --features cli` before the CLI tests, at nvx-rs/target/release/, where nvx.py looks for it, so the tests run with it rather than building it. Linux builds a static x86_64-unknown-linux-musl executable, which runs on every self-hosted Linux runner whatever its C library, and Windows an executable with a static C runtime, which needs no Visual C++ redistributable on the runners. The Windows job already installs the pinned Rust release with the minimal profile. - After the tests pass, each job uploads its executable as a one-day artifact: nvx-rs-linux-musl-executable or nvx-rs-windows-msvc-executable. - run-nvx-microvm-tests.yml takes the artifact's name as a new required input, nvx-rs-artifact, which every microVM job, the debug-kernel ones included, sets for its operating system. It downloads the executable into nvx-rs/target/release/ and restores its executable permission on Linux, before any step runs nvx.py. The platform jobs run neither nvx.py sandbox nor nvx.py run, so they take nothing. The change classification needs no change: nvx-rs/, nvx.py, scripts/nvx_tools/, and these workflows are already test inputs. A new workflow test checks the builds, their targets and their order around the CLI tests, the artifact names and paths, that each microVM job takes the executable of its operating system and the platform jobs none, that the reusable workflow downloads it where nvx_rs.binary_path() looks, and that it does so before it runs nvx.py. The handoff test counts the microVM workflow's fifth download. doc/ci.md describes the handoff, and notes that the Copilot setup does not build nvx-rs, which nvx.py builds on first use. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py now runs nvx-rs for its sandbox command and for the egress and artifact steps of its run command, and test-microvm drives the managed lifecycle through an nvx-rs test, so the documentation says where the executable comes from, what needs Cargo, and what the delegation keeps or changes: - doc/usage.md gains an nvx-rs section: NVX_RS, then nvx-rs/target/release/nvx-rs[.exe], then a Cargo build with the pinned release and its output on standard error, the failure that names these choices without Cargo, and that nvx.py does not rebuild an existing executable after a change. The sandbox section gives the exit statuses and says which usage errors argparse still reports, that abbreviations still work, and that nvx-rs's usage wraps at 80 columns; run says which steps nvx-rs performs; test-microvm describes the managed-lifecycle scenario's nvx-rs test and its need for Cargo. - README.md and doc/setup.md list rustup as a prerequisite of nvx.py run and nvx.py sandbox, whose nvx-rs nvx.py builds on first use until releases include it, and doc/run.md points the sandbox section at the nvx-rs section. - doc/contribute.md says that the control protocol, lifecycle, and egress logic belong to nvx-rs, which the microVM scenarios drive as commands, and that no Python client of them may return. - doc/project-structure.md lists scripts/nvx_tools/nvx_rs.py and the hypervisor test among nvx-rs's tests. - nvx-rs/README.md says how nvx.py finds and invokes the CLI, and describes tests/managed_lifecycle.rs. - .github/copilot-instructions.md tells cloud sessions that nvx.py builds nvx-rs on first use, how to rebuild it after a change, and that the managed-lifecycle scenario needs Cargo. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Three CLI tests run the real nvx-rs only when NVX_RS names it: the egress compilation of nvx.py run, the JSON that nvx.py reads from nvx-rs, and the options that nvx.py sandbox passes. The NVX CLI jobs built nvx-rs where nvx.py finds it, but set no NVX_RS, so those tests skipped and the executables that the jobs hand to the microVM jobs left unexercised by them (review of #597). Each Build nvx-rs step now exports NVX_RS for the rest of its job, naming the executable that it built: with `>> "${GITHUB_ENV}"` on Linux, and on Windows with `Out-File -Encoding utf8 -Append`, as setup-curl writes GITHUB_ENV under Windows PowerShell. The workflow test checks both, and doc/ci.md says so. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The adversarial executor runs each scenario with an allowlist of environment variables, which left out everything that nvx-rs needs now that the control protocol and the egress compilation live there. Of the catalogued scenarios, managed-lifecycle runs the nvx-rs test managed_lifecycle with Cargo, and l3-l4-egress-policy compiles its policy with nvx-rs and runs nvx.py run, which runs nvx-rs too. Under the executor, they ignored a prebuilt nvx-rs that NVX_RS named, and they could not find a Rust toolchain installed outside the default rustup and Cargo homes: without RUSTUP_HOME, rustup starts installing the pinned release into the home directory, within the scenario's timeout. The allowlist now passes NVX_RS, CARGO_HOME, and RUSTUP_HOME, which are paths, not secrets. It still drops RUSTUP_TOOLCHAIN, so that rust-toolchain.toml selects the release, and Cargo's other settings and credentials, such as CARGO_TARGET_DIR, RUSTFLAGS, and registry tokens. Tests check both, and that nvx.py finds NVX_RS in a process that has only the executor's environment. The rest of the executor's contract holds, and is now tested: - managed-lifecycle still preserves OpenVMM's report as managed-outcome.json, with its teardown fields, in the scenario's output directory, where the executor's outcome inventory requires it. A test checks the preserved report against that inventory. - The nvx-rs test records OpenVMM in the journal that NVX_ADVERSARIAL_OPENVMM_PID_JOURNAL names, as the Python scenarios do, but it runs in nvx-rs/, so the scenario now hands it an absolute path: a relative journal resolves where nvx.py runs, as for the Python scenarios. The test reads the variable as an OsString, so a path that is not UTF-8 is no longer skipped, and a new test that needs no hypervisor checks that its entries are exactly what the executor reads. doc/adversarial-testing.md describes what a target needs: the Rust toolchain, with Cargo on the executor's PATH, and nvx-rs and the managed-lifecycle test built at the staged commit before exposure, since a scenario's timeout includes any build, nvx.py never rebuilds an existing nvx-rs, and the target may have no network. doc/ci.md and doc/setup.md point there. The nvx-adversary agent is the strategist's prompt, which names no host prerequisites, so it is unchanged. Verified by driving the local executor on a Linux/MSHV host through the managed-control-lifecycle and l3-l4-network-boundary cases of workload-isolation, with source verification, provenance, and the baseline and post-campaign boots passing: once with NVX_RS naming an nvx-rs outside the checkout, which the scenario ran without building, and once without it, when the scenario built nvx-rs with Cargo. Every journal held the nvx-rs test's OpenVMM, and the inventory found managed-outcome.json with complete teardown in each repetition. On Windows, with HOME and USERPROFILE pointing at an empty directory, the executor's environment built nvx-rs and the managed_lifecycle test through CARGO_HOME and RUSTUP_HOME, with the release that rust-toolchain.toml pins despite a RUSTUP_TOOLCHAIN of 1.89; without those two variables, rustup began installing the release into the empty home. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
converter_input_sha256() hashes an inventory of the files that customize the Ubuntu guest, in the order that sorted() gives their Path objects. That order depends on the OS: PosixPath compares path components with case, and WindowsPath without it. Since #561 added nvx-rs/Cargo.toml and nvx-rs/Cargo.lock to those files, beside nvx-rs/build.rs, Windows sorts build.rs first and Linux last, so Windows computes another digest than the Linux hosts that build the Ubuntu artifacts and record it in their manifest. `nvx.py package` on Windows then fails with "Ubuntu artifact manifest does not match initramfs-ubuntu.cpio.gz", which would fail the windows-whp package of the next development release. The inventory now takes the files in the order of their POSIX paths relative to the repository, component by component and with case. That is the order that Linux has always used, so the digests of existing artifacts and manifests stay valid: on Windows, this tree now gives the digest that the Linux-built manifest records. The callers' own order no longer matters. The Alpine and Azure Linux input digests already sort by POSIX path strings, which compare alike everywhere. A test feeds the files in both orders and checks the inventory's order, Cargo.toml before build.rs and src/cli/args.rs before src/cli.rs, as Linux sorts them; it fails on Windows without the fix. This fixes an issue in code that #561 exposed, found while packaging nvx-rs on Windows for this series. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py runs the nvx-rs that NVX_RS names or else the one in nvx-rs/target/release, and never rebuilds one that exists, so nothing tells whether that executable is a build of the checkout's nvx-rs sources. OpenVMM has the same problem and solves it with build/openvmm.provenance.json, which binds the digest of its binary to its gitlink revision. Releases are about to ship nvx-rs, and packaging and `nvx.py download` need the same binding for it. build/nvx-rs.provenance.json now records, for the nvx-rs that nvx.py runs: - source_sha256: a digest of the files that build nvx-rs, the crate's Cargo.toml, Cargo.lock, build.rs, and src/ tree, and rust-toolchain.toml, by path and with line endings normalized, so that LF and CRLF checkouts of the same sources agree. The tests, README.md, and artifacts.json build nothing, and stay out; - toolchain: the Rust release that rust-toolchain.toml pins; - target: the Rust target of the build; - features: ["cli"], which BUILD_COMMAND builds; - executable_sha256: the digest of the executable. nvx.py records it after each Cargo build that it makes, for the host target that `rustc -vV` reports. The new `nvx.py record-nvx-rs-provenance [--target TARGET]` command records it for a build made otherwise, of the executable that NVX_RS names or else of nvx-rs/target/release/nvx-rs[.exe]. The caller vouches that this checkout built that executable for that target; the record lets later steps check the claim against the sources. RELEASE_TARGETS names the targets that the NVX CLI jobs build for releases, x86_64-unknown-linux-musl and x86_64-pc-windows-msvc, and a record for the musl target requires a static ELF executable, since a build for the host's own Linux target is dynamic. The new constants live in nvx_tools/nvx_rs.py, which no guest artifact digests, so the existing guest provenance stays valid. doc/usage.md documents the command and the record, doc/build.md the file beside the other provenance sidecars, nvx-rs/README.md who records it, and doc/ci.md that the Copilot setup's first nvx-rs build records it too. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
#359 makes nvx-rs the backend of nvx.py sandbox and of the egress and artifact steps of nvx.py run, so a release that nvx.py download installs is of little use without it. Every package now ships the release build of nvx-rs beside OpenVMM: - bin/nvx-rs in the Linux packages, the static x86_64-unknown-linux-musl executable that runs on every x86-64 Linux host, and bin/nvx-rs.exe in the Windows package, the x86_64-pc-windows-msvc executable with a static C runtime; - provenance/nvx-rs.provenance.json, its build provenance; - an nvx_rs section in SOURCE-MANIFEST.json with the executable's digest, the source digest, the Rust release, the target, and the features; - both files in SHA256SUMS, as every packaged file is. `nvx.py package` takes them from nvx-rs/target/release and build/, as it takes OpenVMM, and refuses a provenance that does not describe the executable as a build of the checkout's nvx-rs sources, with its Rust release and features, for the release target of the host's OS, and a Linux executable that is not static. It checks the staged copies against the provenance, as it does for OpenVMM. Source-inclusive packages ship nvx-rs too; their project source archive already holds its sources. CI provides the build. Each NVX CLI job now records the provenance of the nvx-rs that it built and tested, with --target, and uploads it as nvx-rs-linux-musl-provenance or nvx-rs-windows-msvc-provenance beside the executable. The workflows now name the pair by its base name, as they name the OpenVMM artifacts, so run-nvx-microvm-tests.yml appends -executable itself. run-platform.yml takes the base name and passes it to the package-release action, which downloads both into place before it packages. The platform jobs already wait for their CLI job. Before it publishes, publish_development_release.py checks, from each archive's member list, that every platform's package ships bin/openvmm and bin/nvx-rs, or their .exe forms, so that a development release without them never goes out. The bundled feature of nvx-rs extracts and checks only OpenVMM, the Alpine guest, and SOURCE-MANIFEST.json of a release, so a release with bin/nvx-rs stages as the pinned older release does, and the pin stays. The synthetic release of the check-nvx-rs action now has the current layout, with nvx-rs and its provenance in the bundle and in its SHA256SUMS. doc/distribution.md, doc/build.md, doc/usage.md, doc/ci.md, and nvx-rs/README.md describe the package contents and the handoff. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Packages now ship bin/nvx-rs[.exe], which statically links the Rust standard library, the crates of the nvx-rs cli feature, and a C runtime, so THIRD_PARTY_NOTICES.md, which every package ships, must cover them, as it has covered the Rust guest binaries since #561. The crate rows follow `cargo tree --locked --features cli -e normal` in nvx-rs for both release targets, x86_64-unknown-linux-musl and x86_64-pc-windows-msvc: 26 crates, with the licenses that `cargo metadata` reports. - The rows of the crates that the guest binaries already link, the Rust standard library, compiler_builtins, libc, nix, getrandom, bitflags, cfg-if, and memoffset, now name bin/nvx-rs too. - New rows cover serde, serde_core, serde_json and its itoa, memchr, and zmij, sha2 and its RustCrypto dependencies, and windows-sys and windows-link on Windows. serde_derive and the proc-macro2, quote, syn, and unicode-ident crates that it runs on are marked as build time only, not linked. - The musl row now includes the bin/nvx-rs of the linux-kvm and linux-mshv packages, under the musl terms that it already documents. - A new LLVM libunwind row covers the unwinder that the Rust musl target links into the guest binaries and the Linux nvx-rs, under Apache-2.0 WITH LLVM-exception, which lifts the attribution conditions for compiled code. - A new Microsoft C runtime row covers the static CRT of bin/nvx-rs.exe, under the Visual Studio redistribution terms. doc/distribution.md says the same in its licensing section. A test parses nvx-rs/Cargo.lock, requires each of its crates to be classified as shipped or not, and checks that every shipped crate has a row with its exact version that names bin/nvx-rs, and that the runtime rows name it too. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The platform jobs packaged a release only on dev pushes, so a pull request that broke packaging, as one that changes what a package ships can, passed CI and failed the next development release instead. The Windows packaging failure that the first commit of this series fixes had gone unnoticed that way. The Package release step of run-platform.yml now runs whenever the platform packages releases, and tells the package-release action through a new upload input whether to upload the archive: true only on dev pushes. The action stages the package, which checks every input against its provenance, and archives it, which checks the staged files against SHA256SUMS, on every run, and uploads the archive for the development release only when asked. Pull requests still publish nothing; the release job runs only on dev pushes. Packaging adds about a minute and a half to each Linux platform job and twenty seconds to the Windows one. doc/distribution.md describes it, and a test checks the conditions. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx.py download installed OpenVMM and the guest artifacts of a release, so nvx.py sandbox and nvx.py run still needed Cargo to build nvx-rs afterwards. It now installs the package's bin/nvx-rs[.exe] at nvx-rs/target/release/nvx-rs[.exe], where nvx.py finds it, and its provenance at build/nvx-rs.provenance.json, so that both commands run without Cargo after a download. nvx.py runs that executable without ever rebuilding it, and download may fall back to the release of an earlier first-parent commit, so it installs nvx-rs only when the package's provenance describes the build of this checkout's nvx-rs sources that nvx.py runs: the same source digest, Rust release, and features, for the release target of this OS. Otherwise, as for a release from before packages shipped nvx-rs, it installs everything else, removes any earlier nvx-rs and its provenance, which may not match the checkout either, and says that nvx.py builds nvx-rs with Cargo when a command needs it, unless NVX_RS names one. Older releases thus stay installable. A package whose nvx-rs does not match its own provenance is broken, and fails the download before it replaces anything. The installed OpenVMM and nvx-rs are now executable on POSIX whatever mode the archive records. The message that nvx.py prints without Cargo, the discovery notes in nvx_rs.py, doc/usage.md, doc/setup.md, doc/distribution.md, README.md, and nvx-rs/README.md describe the installed nvx-rs. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The adversarial executor validates and hashes OpenVMM, the kernel, the initramfs, and their provenance before the baseline boot, so that a campaign runs what the staged commit built. Scenarios such as l3-l4-egress-policy also run nvx-rs through nvx.py, which never rebuilds an nvx-rs that exists, so one left from before a change of the nvx-rs sources ran unchecked. validate_runtime_artifact_provenance() now also requires the nvx-rs that nvx.py runs, the one that NVX_RS names or else nvx-rs/target/release/nvx-rs[.exe], with a build/nvx-rs.provenance.json that describes it as a build of the checkout's nvx-rs sources, for any target. The executor's artifact metadata hashes both, beside the other artifacts. A stale, unrecorded, or missing nvx-rs now fails the initialization instead of running. Targets already had to build nvx-rs before exposure; the preparation in doc/adversarial-testing.md now also records its provenance, which nvx.py does itself for the builds that it makes, and a target with the release of the staged commit gets both from nvx.py download. doc/ci.md and doc/usage.md mention the check. Closes #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
requested review from
Enrique Saurez (esaurez)
and
a balanced review from Copilot
October 17, 2026 11:36
Contributor
There was a problem hiding this comment.
🟡 Changes recommended
Executable path consistency and thread-aware process liveness need correction before approval.
2 open issues
What changed in this PR
Ships nvx-rs with NVX releases, completing the release-delivery portion of #359 so compatible downloads can run delegated commands without Cargo.
Changes:
- Adds executable provenance, packaging, installation, and publication checks.
- Passes
nvx-rsartifacts through CI and validates them in adversarial execution. - Includes the stacked Rust CLI, guest-build migration, tests, and documentation updates.
| File | Description |
|---|---|
| SOURCE-MANIFEST.json | Records Rust guest toolchain and sources. |
| scripts/test_tool_versions.py | Checks musl target setup. |
| scripts/test_guest_binaries.py | Tests source identities and staged builds. |
| scripts/test_development_release.py | Tests required release executables. |
| scripts/test_adversarial.py | Tests executable discovery and metadata. |
| scripts/setup/setup-linux-runner.sh | Explicitly selects the target toolchain. |
| scripts/setup/setup-linux-mshv.sh | Installs and verifies musl support. |
| scripts/publish_development_release.py | Checks packages before publication. |
| scripts/nvx_tools/ubuntu.py | Normalizes and orders build inputs. |
| scripts/nvx_tools/sandbox_lifecycle_tests.py | Updates lifecycle and teardown checks. |
| scripts/nvx_tools/microvm_test_scripts/sandbox-blocks.sh | Tests distro-only block layout. |
| scripts/nvx_tools/guest_binaries.py | Validates staged Rust guest binaries. |
| scripts/nvx_tools/ci.py | Tracks Rust guest build dependencies. |
| scripts/nvx_tools/build.py | Installs shared Rust guest binaries. |
| scripts/nvx_tools/build_constants.py | Defines Rust build and source packaging inputs. |
| scripts/nvx_tools/benchmark.py | Records Rust guest provenance. |
| scripts/nvx_tools/azurelinux.py | Includes normalized Rust build inputs. |
| scripts/nvx_tools/archive.py | Excludes generated Rust build output. |
| scripts/nvx_tools/adversarial_executor.py | Validates and records nvx-rs. |
| README.md | Explains prebuilt CLI availability. |
| nvx-rs/src/protocol/mod.rs | Updates shared-protocol documentation. |
| nvx-rs/src/protocol/features.rs | Defines layered sandbox requirements. |
| nvx-rs/src/host/sys/mod.rs | Exposes interrupt handling. |
| nvx-rs/src/host/sys/interrupt.rs | Records Unix interrupts. |
| nvx-rs/src/host/session.rs | Exposes CLI request and response methods. |
| nvx-rs/src/host/sandbox/layer.rs | Defines distro-layer parsing and validation. |
| nvx-rs/src/host/python/uuid.rs | Implements Python-compatible UUID parsing. |
| nvx-rs/src/host/python/shell.rs | Implements command display quoting. |
| nvx-rs/src/host/python/mod.rs | Organizes Python compatibility helpers. |
| nvx-rs/src/host/python/dumps.rs | Implements compatible JSON formatting. |
| nvx-rs/src/host/process.rs | Adds CLI process-control methods. |
| nvx-rs/src/host/platform/windows.rs | Adds Windows CLI primitives. |
| nvx-rs/src/host/platform/other.rs | Provides interrupt compatibility hooks. |
| nvx-rs/src/host/platform/linux.rs | Exposes Linux interrupt hooks. |
| nvx-rs/src/host/mod.rs | Exposes sandbox and CLI modules. |
| nvx-rs/src/host/cli/sandbox/identity.rs | Checks sandbox process identity. |
| nvx-rs/src/host/cli/output.rs | Implements CLI stream output. |
| nvx-rs/src/host/cli/egress.rs | Adds egress compilation commands. |
| nvx-rs/src/host/cli/deadline.rs | Adds interruptible deadline handling. |
| nvx-rs/src/host/cli/artifacts.rs | Locates and reports runtime artifacts. |
| nvx-rs/src/guest/time/mod.rs | Updates guest-time documentation. |
| nvx-rs/src/guest/time_probe/mod.rs | Updates probe documentation. |
| nvx-rs/src/guest/reseed.rs | Removes obsolete C references in comments. |
| nvx-rs/src/bin/nvx-rs.rs | Adds the CLI entry point. |
| nvx-rs/Cargo.toml | Declares CLI features and dependencies. |
| guest/common/nvx-reseed.c | Removes the replaced C implementation. |
| guest/common/nvx-mmio-write.c | Removes the replaced C implementation. |
| guest/common/nvx-init-agent | Restricts layers and reserves runtime paths. |
| guest/common/nvx-exit.c | Removes the replaced C implementation. |
| guest/common/nvx-console-pending.c | Removes the replaced C implementation. |
| docker/Dockerfile | Shares Rust binaries across guest builds. |
| doc/setup.md | Documents Rust build prerequisites. |
| doc/run.md | Documents delegated sandbox execution. |
| doc/project-structure.md | Describes the Rust-backed layout. |
| doc/guests/ubuntu-guest.md | Documents Rust guest binaries. |
| doc/guests/azurelinux-guest.md | Documents guest binary provenance. |
| doc/distribution.md | Documents CLI packaging and installation. |
| doc/contribute.md | Clarifies Rust ownership and testing. |
| doc/ci.md | Documents artifact handoff and release checks. |
| doc/build.md | Documents build provenance and package layout. |
| doc/benchmarks.md | Documents Rust benchmark provenance. |
| doc/adversarial-testing.md | Documents target CLI requirements. |
| aci_edge_sandboxes/src/openvmm/mod.rs | Updates shared egress documentation and tests. |
| aci_edge_sandboxes/README.md | Documents egress behavior changes. |
| .github/workflows/run-platform.yml | Packages on PRs without publishing. |
| .github/workflows/run-nvx-microvm-tests.yml | Downloads the CLI for scenarios. |
| .github/workflows/copilot-setup-steps.yml | Tracks all Rust source changes. |
| .github/copilot-instructions.md | Updates build and runtime guidance. |
| .github/actions/validate-nvx/action.yml | Removes replaced Python test suites. |
| .github/actions/package-release/action.yml | Downloads CLI artifacts and gates uploads. |
| .github/actions/check-nvx-rs/action.yml | Updates the synthetic release layout. |
| .github/actions/build-guest-artifacts/action.yml | Keys caches on Rust source identity. |
| .gitattributes | Marks frozen compatibility fixtures generated. |
Review effort: Balanced
nvx-rs sandbox identifies the OpenVMM of a running sandbox by its process ID and start time, which identity.rs reads on Linux from /proc/PID/stat. Ported from the former sandbox_lifecycle.py, it counted every zombie as exited. But /proc reports a process as a zombie as soon as its main thread exits, while its other threads may still run and hold its descriptors, so exec and stop could reject a live VM as stale, and deprovision could remove the state of a VM that still runs. The library's platform::process_start_time, with which the process helpers wait for and kill OpenVMM, already counts such a process as running until its last thread exits, as Windows does, and so does the CLI's own Windows branch; only its Linux branch disagreed. linux_start_time now asks, for a zombie only, whether other threads remain, through the platform's other_threads_remain, which reads /proc/PID/task, and otherwise keeps its Python-compatible parsing and messages. other_threads_remain becomes pub(crate) for that. The stat-line test covers a zombie with and without other threads, and an error while listing them. A new Linux test, like process::tests::a_process_runs_until_its_last_thread_exits, makes the main thread of a Python process exit through pthread_exit while another thread sleeps: process_running holds until the process is killed. It fails without the fix. Copilot's review of #598 flagged this in code that #596 added, the nvx-rs CLI; the platform helper is already on dev. The change stays within the CLI's Linux identity check. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
The executor validates and hashes the nvx-rs that NVX_RS names before the baseline boot, and nvx.py resolves a relative NVX_RS against its working directory: the executor's, at that point. The scenarios inherit NVX_RS unchanged but run in the repository root, so with a relative NVX_RS they ran another executable than the one validated, or found none. sanitized_environment() now makes NVX_RS absolute against the executor's working directory, so that the scenarios run the executable that the initialization checked. An empty NVX_RS, which nvx.py ignores, passes unchanged. A new test validates a relative NVX_RS in one working directory and resolves it in a scenario process in another; it fails without the fix. The environment test now expects the absolute path. Copilot's review of #598 flagged this. #597 added NVX_RS to the scenarios' environment, and the earlier commit of this series that validates the executor's nvx-rs made the mismatch matter. Closes #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #359 (PR 5 of 5).
Summary
Every NVX release package now ships the
nvx-rsCLI to whichnvx.py sandboxand the egress and artifact steps ofnvx.py rundelegate since #597, andnvx.py downloadinstalls it wherenvx.pyfinds it, so a downloaded checkout runs both commands without Cargo. The executable travels with a build provenance, as OpenVMM's does, which packaging,download, and the adversarial executor check.build/nvx-rs.provenance.jsonbinds the executable's SHA-256 to a line-ending-independent digest of the files that buildnvx-rs(Cargo.toml,Cargo.lock,build.rs,src/, andrust-toolchain.toml), the pinned Rust release, the target, and theclifeature.nvx.pyrecords it after its own Cargo builds, and the newnvx.py record-nvx-rs-provenance [--target TARGET]records it for a build made otherwise.nvx.py packageshipsbin/nvx-rs[.exe]andprovenance/nvx-rs.provenance.json, lists both inSHA256SUMS, and records annvx_rssection inSOURCE-MANIFEST.json. It refuses annvx-rswhose provenance does not describe it as the build of the checkout's sources for the release target of the host's OS, and a Linuxnvx-rsthat is not static.nvx-rsthat they build and test, and upload it beside the executable. The platform jobs hand both topackage-release. The platform jobs now also package and archive on pull requests, without uploading, so this PR's platform jobs exercise packaging; before, onlydevpushes did. The development release job refuses packages withoutbin/openvmmandbin/nvx-rs.downloadverifiesSHA256SUMS, then installsbin/nvx-rs[.exe]atnvx-rs/target/release/nvx-rs[.exe], executable, with its provenance inbuild/, when the provenance describes the build of the checkout'snvx-rssources thatnvx.pyruns. Otherwise it installs the rest and leavesnvx-rsto Cargo.nvx-rsthat the scenarios run, so a stalenvx-rs/target/release/nvx-rsfails the initialization instead of running.THIRD_PARTY_NOTICES.mdcovers the crates, the musl, LLVMlibunwind, and Microsoft C runtime code that the shipped executables link.Pathobjects, which Windows compares without case. Since Build the guest binaries from nvx-rs #561 addednvx-rs/Cargo.tomlbesidenvx-rs/build.rs,nvx.py packagefailed on Windows with "Ubuntu artifact manifest does not match initramfs-ubuntu.cpio.gz", which would have failed the nextwindows-whpdevelopment release. The digest now sorts as Linux always has, so existing artifacts stay valid.Package changes
linux-kvm,linux-mshvwindows-whpbin/nvx-rs[.exe](new)bin/nvx-rs: staticx86_64-unknown-linux-muslexecutable, mode 0755bin/nvx-rs.exe:x86_64-pc-windows-msvc, static C runtimeprovenance/nvx-rs.provenance.json(new)format,source_sha256,toolchain(1.95.0),target,features(["cli"]),executable_sha256SOURCE-MANIFEST.jsonnvx_rssection with the executable's SHA-256, the source digest, toolchain, target, and featuresSHA256SUMSChoices:
bin/nvx-rs, or whosenvx-rswas built from othernvx-rssources (a first-parent fallback release), still installs everything else;downloadthen removes any earliernvx-rsand its provenance, says why, andnvx.pybuildsnvx-rswith Cargo when a command needs it, as before this PR. A package whosenvx-rscontradicts its own provenance is broken and fails the download before anything is replaced.bundledpin stays. Thebundledbuild script extracts and checks only OpenVMM, the Alpine guest, and the manifest of a release, so a release withbin/nvx-rsstages as the pinned older one does. The synthetic release of thecheck-nvx-rsaction now has the current layout, and the pinned release still stages and tests.nvx-rstoo; their project source archive already holds its sources.Stack
Based on #597 (
403c7e83d), #596, #595, #594, and #561; it contains their commits until they merge. The merge order after #561 is #594 (A) → #595 (B) → #596 (C) → #597 (D) → this PR (E). This PR's commits:ubuntu: order converter inputs alike on every OS: the Windows packaging fix above, in code that Build the guest binaries from nvx-rs #561 exposedscripts: record the provenance of nvx-rsrelease: package nvx-rs with its provenancenotices: cover the nvx-rs binaryci: package releases on pull requestsrelease: install nvx-rs with nvx.py downloadadversarial: check the nvx-rs that the scenarios runnvx-rs: count a VM as running until its last thread exits(review): the CLI's Linux identity check counted a zombie OpenVMM whose other threads still run as exited, unlike the library and the CLI's Windows branchadversarial: hand the scenarios the nvx-rs that the executor checked(review): the executor now hands the scenarios an absoluteNVX_RS, so they run the executable that it validatedCommits 3, 7, and 9 touch code that #597 owns:
run-nvx-microvm-tests.ymlnow takes the base name of thenvx-rsartifacts and appends-executable, as the workflows name the OpenVMM artifacts; the adversarial executor's validation and metadata covernvx-rs, so targets must now record its provenance, whichnvx.pydoes for its own builds; and the executor hands the scenarios an absoluteNVX_RS. Commit 8 changes the Linux identity check of the CLI, which #596 added. Each commit body explains why.#359 acceptance criteria
nvx.py sandbox …, and the egress and artifact steps ofnvx.py run, go throughnvx-rsnvx-rsransandbox runandrun's artifact and egress steps on KVM, MSHV, and WHP without Cargocontrol_session.py,egress_policy.py,sandbox_lifecycle.py, and the launch profile ofsandbox.py, with their tests ported to Rustnvx-rs, andnvx.py downloadinstalls itSHA256SUMS, manifest, and provenance (commit 3), the publication check (3), the installer (6), their tests intest_nvx_tools.pyandtest_development_release.py, and the hardware runs below. This PR's platform jobs package each platformmanaged-exec-config,sandbox-lifecycle, andmanaged-lifecycleon prometheus32, prometheus30, and prometheus28, and in CI's microVM jobs, which run again on this PRValidation
nvx.py verify; nvx-rs: fmt, clippy (all features, and lib without defaults), tests (all features and default), rustdoc with-D warnings, 1.89 check,aarch64-apple-darwincheck, and the bundled-artifacts steps with the synthetic current-layout release and the pinned release; aci_edge_sandboxes: the commands of check-aci-edge-sandboxes andtest-agent-bundling; Ruff, Ruff format, Pyright (Linux and Windows), the Markdown link check, compileall, the validate-nvx unittest list (996 tests),test_adversarial.py(73),test_hosts.py, and the CLI help commandstest_nvx_tools.py,test_development_release.py,test_markdown_links.py, andtest_adversarial.pywhere it changestest_hosts.py, the Specula wiring tests, and actionlint 1.7.7 (only the pre-existingnvx-adversarial-controllerlabel note).test_adversarial.pypasses on 3.14 and 3.12, and on 3.10 fails onlytest_contained_native_process_preserves_child_signal_status(sigkill), as onorigin/devpackage --binary-onlyandarchive-releasewith #597's CI guest artifacts and OpenVMM, then an install withdownload's installer into a scratch clone:sandbox run --dry-runandrun --dry-runwith an egress policy file, without Cargo and withNVX_RSunsetnvx-rsbuilt forx86_64-unknown-linux-muslas CI does (static-pie), its provenance recorded, alinux-kvmbinary-only package with the host's OpenVMM at5ab8a84and #597's CI guest artifacts, archived (269 MB);sha256sum -c SHA256SUMSpasses; the extracted release'sbin/nvx-rs artifactsfinds the release layout; installed into a scratch checkout withdownload's installer, which verifiesSHA256SUMSand the provenance. Without Cargo and withNVX_RSunset:sandbox run --dry-runand a realsandbox runof/bin/echo(hello-from-a-release, outcome success, complete teardown), the egress policy file throughsandbox runandrun, whose dry run also finds the kernel and initramfs; nothing rebuiltnvx-rslinux-mshvpackage (268 MB): all passnvx-rs.exebuilt with a static C runtime as CI does, and awindows-whppackage (151 MB, 22SHA256SUMSentries, no mismatches): all pass-D warnings, 1.89 andaarch64-apple-darwinchecks; Ruff, Pyright,test_adversarial.py(74). WSL: nvx-rs fmt, clippy, all tests, including the new Linux testa_vm_runs_until_its_last_thread_exits, which fails without the fix, and the 1.89 check;test_adversarial.pyon 3.14 and 3.12 (74), and on 3.10 with only the known sigkill failure. The new executor test fails without its fixdevpushes