Repository navigation
Port the sandbox launch profile and artifact lookup to nvx-rs - #595
Open
Pedro Henrique Penna (ppenna) wants to merge 16 commits into
Open
Pedro Henrique Penna (ppenna) wants to merge 16 commits into
Pedro Henrique Penna (ppenna) wants to merge 16 commits into
Conversation
Replace native C helper compilation with the shared Rust guest-binary builder introduced by #528. Build one checked set of all nine executables per initramfs, install it at the established /sbin/nvx-* paths, and record the aggregate source and individual binary digests. Remove the obsolete C compiler flags, helper inventories, and build functions. Keep C test-only flags local until the C-driven suites are deleted later in this stack. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Build one checked set of Rust guest executables in the digest-pinned Rust 1.95.0 image and reuse its exact bytes in Alpine, Ubuntu, and Azure Linux through a guest-only base, so the kernel and Linux source targets never compile them. Keep the toolchain available only for Ubuntu's two fresh determinism attempts, and remove the guest C compilers from the base. Bind reusable binary sets to normalized Rust-source, toolchain, target, profile, and executable digests, and refuse the output of an earlier build that a rebuild would upgrade in place. Record all nine helpers in the Azure Linux package manifest. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Cover the nvx-rs guest inputs in every image identity, package the crate without its target directory, and declare the exact Rust toolchain and musl target in SOURCE-MANIFEST.json. Verify those pins and inputs from nvx.py verify and release packaging. Make device-I/O benchmark provenance validate the shared Rust source digest, and bind the Azure Linux cache input to the newly tracked files. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Hash the complete nvx-rs tree except target, the exact toolchain pin, and the shared build modules into every guest cache, and advance the guest cache schemas. Because hashFiles hashes contents only, also append the path-aware Rust source digest that the guest package manifests record, which the new guest-binary-source-identity command prints, so a module move that keeps every file's bytes still rebuilds the images. Treat nvx-rs changes as OpenVMM VMM-test inputs so they rebuild and boot the affected images. Keep Copilot setup and both Linux setup paths on the pinned musl target, and revalidate Copilot setup for every nvx-rs change. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Remove the nine remaining guest C implementations and the C-compiled Python suites whose behavior the nvx-rs unit, guest-binary, loopback, and real-VM tests now cover. Drop the deleted modules from both CLI validation legs. Add a repository regression that rejects C, header, or assembler sources outside the OpenVMM submodule. Part of #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Document the one pinned build shared by native builds, Docker, and CI, the nine binaries shipped by every guest, their provenance, cache inputs, test ownership, and release source. Now that the Rust definitions are authoritative, replace the obsolete source comparisons with literal wire values and remove C-era wording. Extend the third-party notices for the Rust standard libraries, compiler_builtins and its compiler-rt-derived routines, libc, nix, getrandom, and their linked transitive crates, and musl's complete guest-binary scope. Closes #358. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port scripts/nvx_tools/egress_policy.py, which compiles the files of
nvx.py's --network-egress-policy-file option, to
nvx::host::network::policy, so that the nvx-rs CLI can replace it.
Python stays authoritative until nvx.py delegates to nvx-rs. The port
accepts the same files, compiles them to the same rules, and fails with
the same messages:
- compile_policy_file reads at most 1 MiB and one byte of a file, and
compile_policy_bytes and compile_policy compile its contents or an
already parsed serde_json value. The CompiledEgressPolicy result has
the serde form {"allow": [...], "deny": [...]}.
- The format keeps Python's root fields, the MXC and legacy flat rule
forms, the tcp, udp, icmp, and any protocols, and TCP and UDP port
ranges, and runs every check in Python's order with its message.
- The lowering keeps Python's steps: address-only rules, rules for every
port of ICMP, TCP, and UDP widened into the address-only cover, the
sweep over TCP and UDP port boundaries, the budget of 256 native rules
that both families of an action share, and the sort by family,
address, prefix length, protocol, and ports.
network.rs becomes network/mod.rs, beside address.rs, the networks and
address intervals, and lowering.rs, the rule model and the lowering.
openvmm_arguments still expands NetworkPolicy rules as before.
The format depends on Python semantics, which a crate-private
host::python module provides for this and later ports:
- python::json reads a document as json.loads does, with hooks in the
place of object_pairs_hook and parse_int: NaN and Infinity are floats,
numbers beyond a float's range are infinities, integers keep their
digits, and a \ud800 escape stays an unpaired surrogate. It keeps
values in an arena and never recurses. The policy reader's hooks
refuse a repeated field when its object closes and an integer of more
than 64 digits when it is read, so their errors surface in Python's
order. serde_json, which nvx-rs already depends on, cannot do this: it
refuses NaN, Infinity, 1e400, and lone surrogates, which Python reads
and the compiler then reports with its own messages. No dependency is
added.
- python::text holds strings that may contain unpaired surrogates,
ordered by code point as Python orders them, and shows a surrogate as
\udXXX, which is how Python prints it.
- python::path gives str(pathlib.Path(path)) for POSIX and Windows
paths, as CPython 3.14 splits them, and the normalized path that
Python opens, so that messages name files as nvx.py does.
Networks are read as ipaddress.ip_interface reads them: host bits are
cleared, IPv4 netmasks and hostmasks, prefix lengths with leading zeros
up to Python's 4300-digit int limit, and embedded IPv4 addresses are
accepted, and IPv6 scopes are found and refused. They print as Python
3.13 and later print them, with IPv4-mapped addresses in dotted form.
One input differs. Python's decoder raises RecursionError for a
document nested more deeply than its stack allows, at a depth that
depends on the Python version and the platform. This reader reads any
depth, so such a document fails as an invalid policy instead.
The tests port each case of scripts/test_egress_policy.py under its
name. The cases that mock ipaddress.summarize_address_range and
_merge_intervals to bound the work read thread-local counters of the
lowering instead. Unit tests cover the JSON reader, strings, paths, and
networks against values that Python 3.14 produced.
Part of #359.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Add scripts/generate_egress_policy_fixtures.py, which runs the Python
compiler of scripts/nvx_tools/egress_policy.py over a corpus of policy
files and records each file with its outcome, its native rules or its
error message, in nvx-rs/tests/fixtures/egress-policy.json. The Rust
tests replay every entry through compile_policy_bytes and, from a real
file, compile_policy_file, and need no Python, so the fixture outlives
the Python compiler, which a later change in this stack deletes.
The corpus holds 784 policy files:
- every file that scripts/test_egress_policy.py compiles, which the
script captures by running that suite, except those of the two tests
whose files exceed 64 KiB, which pin their complete outcomes in
Python and in their Rust ports;
- hand-written cases that reach every message and JSON edge: syntax
that json.loads refuses, NaN, Infinity, floats, Booleans, and 64- and
65-digit integers as ports, repeated fields and long integers in
document order, unpaired surrogates in names, networks, scopes, and
protocols, unknown fields in code-point order, each check of both
rule forms in order, network spellings as ip_interface reads them,
the rule limit across protocols and families, and files at and beyond
the 1 MiB limit; and
- 200 random policies from seed 359 that mix both rule forms and both
families, dense and sparse networks written with host bits, masks,
and exceptions, overlapping protocols and port ranges, crowds of
rules near the limit, broken fields, and varied, truncated, and
altered JSON text.
The fixture also records the errors for a missing file and a directory,
and how PurePosixPath and PureWindowsPath print 55 paths, which the
tests compare with python::path. Messages name the policy file
{path}, and show unpaired surrogates as Python prints them. Inputs are
text, or hexadecimal for bytes that are not UTF-8, with runs of 256 or
more characters recorded as repetitions, and each entry is a line of
ASCII JSON, about 1.1 MB in all.
The script needs Python 3.13 or later, whose ipaddress module prints
IPv4-mapped IPv6 addresses in dotted form, as the port does. Its output
depends only on the seed and the corpus size, so --check verifies that
the fixture is current. --random-cases and --output write larger
corpora, and NVX_EGRESS_POLICY_FIXTURE points the tests at one. The
script passes ruff and strict pyright on Linux and Windows.
Part of #359.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
nvx::host::network::openvmm_arguments, which the OpenVMM backend of aci_edge_sandboxes calls, expanded each EgressRule into OpenVMM rules on its own, then sorted and deduplicated the strings. It now lowers each action's rules with the lowering of egress policy files, so that NVX has one lowering. An EgressRule is a policy file's MXC rule whose empty to or ports stands for an absent field, and messages name its fields after EgressRules::name, where a file's messages say allow or deny, as in "network.egress.allow[0].to[1].cidr is not a valid IPv4 or IPv6 CIDR: x". openvmm_arguments keeps the checks that only a NetworkPolicy has, which the Python compiler lacks: no device for a closed posture, the /29 limit of host-loopback access, and the rules of host-loopback forwards. The check that no deny rule covers the guest network of forwards now reads the deny rules as the lowering parsed them, so it follows the same normalization. Before, it parsed them with the stricter Cidr and missed a network written with host bits, which the old expansion also refused. The observable changes, which aci_edge_sandboxes/README.md lists as breaking changes: - The OpenVMM rules follow family, address, prefix length, protocol, and ports, instead of string order. - Merging destinations and port ranges, and subsuming narrower rules, usually needs fewer rules, so some policies beyond the limit of 256 now fit: 256 adjacent /24 destinations become one /16 rule. Port ranges that overlap only partly across networks can also need a rule for their union. - The backend's rule checks report the compiler's messages. Through AciEdgeSandbox, whose request validation runs first, only the limit message, now "network.egress.allow emits at most 256 native rules", changes. The error codes are unchanged. - Without that validation, the backend accepts networks with host bits and IPv4 netmasks, and refuses port 0, which it passed to OpenVMM. Tests: the nvx-rs network tests pin the new order, merging, messages, and coverage check. ipv6_rules_expand_exactly, now ipv6_rules_lower_exactly, expects ::/0 first. inexpressible_rules_are_rejected, now invalid_rules_are_rejected_with_the_policy_compilers_messages, checks each message, and its 300-destination rule, which now lowers to one rule, moved to rules_lower_like_egress_policy_files. exceptions_that_split_a_network_too_finely_are_rejected expects the limit message. network_policies_convert_field_for_field expects the new ICMP message and also checks the scope and limit messages. The other aci_edge_sandboxes tests, including the fake-OpenVMM test that pins the launch arguments, pass unchanged. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Describe nvx::host::network::policy in nvx-rs/README.md: what it ports and that Python stays authoritative until nvx.py delegates to nvx-rs, the policy format, the lowering and the order of its rules, how openvmm_arguments, and so aci_edge_sandboxes, share that lowering, the fidelity of its messages and the one input on which it differs from Python, and the parity fixture with its generator. The module table gains the policy module, and its network row describes NetworkPolicy and the shared lowering instead of the expansion. doc/project-structure.md lists the generator, the parity fixture, and the compiler in its descriptions of scripts/ and nvx-rs/. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
An MXC rule pairs each family of its destinations with each of its port selectors, and `combine`, formerly `mxc_rules`, gave every pair its own copy of the family's address intervals. Python's `_parse_mxc_rule` and `_parse_legacy_rule` share one tuple among the rules of a family instead. A policy file well within the 1 MiB limit, such as 8192 destinations, none adjacent to another, with 8192 port selectors, made the port copy 2 GiB of 32-byte intervals, and the unions of `_compile_family` flattened the shared addresses once for each selector again. - `Rule.addresses` is an `Rc<[Interval]>` that the rules of one rule's selectors share: for MXC rules, legacy rules, and the typed rules of `openvmm_arguments` alike, which all build their rules with `combine`. - The union of the rules for every protocol, and of those for every port of each protocol, merges each distinct address set once. Python merges the same addresses once for each selector; the union is the same. - The port sweep asks once for each address set whether `covered` holds it entirely, rather than once for each rule. - An `AddressSet` compares rule addresses by value, as Python's tuples compare, and compares a shared allocation with itself without reading it, as CPython's dictionaries do. The sweep counts its active rules by it, as `_lower_protocol_rules` counts them in a `Counter`. The output, the errors, and the order of both are unchanged: the parity fixture and the ported Python tests still pass. New tests check that the rules of a rule's selectors share one allocation, that address sets compare by value, that the unions read shared addresses once (64 intervals rather than 4096), and that three 8192-by-8192 policies compile as Python 3.14 compiles them: to `0.0.0.0/0` with a covering rule, and to the 256-rule limit errors without one. Python takes 15 to 27 seconds for each. Before this change, the debug build of that test took 171 seconds and peaked at 4.05 GiB; it now takes 0.5 seconds and 48 MiB. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port SandboxLaunch, SandboxLayer, SandboxMount, and the helpers of scripts/nvx_tools/sandbox.py to a new nvx::host::sandbox module, so that the nvx-rs CLI can launch sandboxes as nvx.py sandbox does. Python stays authoritative and unchanged until nvx.py delegates to nvx-rs. The port keeps the Python contract byte for byte: the limits, option semantics, OpenVMM arguments, kernel command line, and messages, with refused requests as Unsupported, missing images as Unavailable, and failed host queries as Io errors in Python's [Errno N] form. - Layers: one to three EROFS layers, distro, runtime, and custom, ordered by role, with duplicate roles refused in sorted order. A --layer value is ROLE,PATH,EROFS_UUID, and the UUID takes every form that uuid.UUID reads: braces, urn:uuid:, no hyphens, and uppercase. Layer and scratch paths with ',' or ';' are refused. - Settings: an absolute entrypoint and nonempty arguments without whitespace, a lowercase RFC 1123 hostname label, a non-root workload identity, 65534:65534 by default, which parse_workload_identity reads as int(field, 10) does, and positive limits, with nvx_pids_max one above the request. - Shares: --mount GUEST_TARGET,HOST_PATH[,ro|rw], whose target ends at the first comma and whose mode follows the last one; the target rules, at most 4096 UTF-8 bytes, and the reserved targets; at most 128 denied, allowed, and writable paths each; the overlap and single-owner checks; caller ownership refused on Windows; file and directory shares; the aggregate at /run/nvx/shares with children named INDEX-SHA256(TARGET)[:32] and its nvx_share= tokens; and the host-path checks, by Path.resolve() and (st_dev, st_ino) lineage. - Kernel command line: NUL and the nvx_* and virtfs_* tokens refused, the tokens in Python's order, and the 1024 bytes that OpenVMM leaves of the 2048-byte x86 command line, which also hold OpenVMM's bootstrap tokens for the shares and the NUL. The private python module that the egress policy compiler introduced gains what these rules rely on, after CPython 3.14: builtins, with str.isspace, which unlike char::is_whitespace includes U+001C to U+001F, str.split() and str.strip(), repr of a str, and int, with signs, underscores, Unicode digits, surrounding whitespace, and the 4300-digit limit; uuid, for uuid.UUID; unicode, the Unicode 16.0.0 tables of str.isprintable and int; and fs, the queries of os.path and os.stat, with Path.resolve() ported from posixpath.realpath and ntpath.realpath. On Windows, os.stat identifies a file by the 64-bit volume serial number and 128-bit file ID of FILE_ID_INFO, as Python 3.12 and later do, through the windows-sys features that the crate already enables. Operating-system strings that are not valid Unicode, which Python would carry as surrogates, are refused. python::path gains PurePath, a Unicode path of either pathlib flavor with its parts, parents, joins, and case-insensitive comparison on Windows, and Flavor, with the splitroot, join, isabs, and split of posixpath and ntpath, and ntpath.normpath. One parser serves both python_path and PurePath, so the printed and the pure paths cannot disagree. Windows paths follow CPython on Windows, whose splitroot and normpath are C functions that index UTF-16 code units: a character beyond the Basic Multilingual Plane before a colon is no drive there, although Python's own ntpath, which other hosts use, reads one. scripts/generate_egress_policy_fixtures.py becomes scripts/nvx_rs_fixtures.py, one generator for every Python-parity file of nvx-rs, which now also writes the Unicode tables, with one --check for all of them. It needs CPython 3.14, whose Unicode database the tables record, and says so; --output and --random-cases still write a larger egress policy corpus. The egress policy fixture regenerates byte for byte, except its comment, which names the script. Aggregate child names need SHA-256 on the host, so the host feature enables sha2 0.10, which the bundled build script already uses and locks, as does the aci_edge_sandboxes agent, instead of another crate. The unit tests port the launch-profile cases of scripts/test_nvx_tools.py, and check require_file, share kinds, host-path identity, and bind and hard-link identity on temporary directories on Linux and Windows. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Port the pure checks that scripts/nvx.py applies to the options of nvx.py sandbox and nvx.py run to nvx::host::sandbox::options, so that the nvx-rs CLI keeps their order, semantics, and messages and stays a thin layer over the library. - SandboxOptions::check: the checks of command_sandbox that precede the launch, in its order: --dry-run outside run, --environment with --environment-file, managed exec options outside exec, the managed exec environment, network policy options outside run and provision, systemd entrypoints, --outcome-report outside run and exec, share options without --mount, --mount outside run and provision, --net without --network-profile or the reverse, and a missing --layer or --scratch. - sandbox_mounts, from _sandbox_mounts: with one --mount, every --mount-deny, --mount-allow, and --mount-write names a path in it; with several, each names a path in the --mount before it. Each share takes the kind that mount_kind finds for its host path. - validate_sandbox_systemd_policy and SYSTEMD_ENTRYPOINTS: a distro layer's manifest, next to it, must be a JSON object of format 1 that names the layer and its SHA-256 digest and lists packages with names, none of them systemd. The format is compared as == compares it, so true and 1.0 match, and the manifest is read with universal newlines, as read_text reads it. - validate_run_share_targets, from _validate_run_share_targets: the guest targets of several nvx.py run shares, or of a shared file. - validate_exec_environment and exec_environment_from_file: at most 256 nonempty KEY=VALUE entries of at most 4096 UTF-8 bytes, without NUL or a repeated key, from --environment or from a JSON array in a file of at most 1 MiB, as encode_exec_environment checks them. These checks fail with the messages of Python's JSON decoder and UTF-8 codec, so the private python module gains them: - python::json keeps its API, its hooks, and its arena, but its reader is now a port of the C scanner of CPython 3.14, scan_once_unicode and the functions that it calls, with the checks of JSONDecoder.decode around them. It accepts what the previous reader accepted, as the egress policy fixture and tests confirm, and a refusal now carries the message and the position of Python's JSONDecodeError, such as Expecting ',' delimiter, instead of the offset at which reading stopped, which is not always Python's position: an unterminated string fails at its opening quote. syntax_error formats it as str() of the error, with its line, column, and character. Loads are the hooks of json.loads without arguments, whose int refuses more than 4300 digits, and Document::get reads a member as the dict that Python builds does. - python::text gains PyText::encode_utf8 and decode_utf8, with the messages of UnicodeEncodeError and UnicodeDecodeError, so PyText is the one string type that holds surrogates. - python::path gains name and with_name, and python::fs exists. Argument parsing and type conversion stay with the CLI, as do the lifecycle operations. The unit tests port the option cases of scripts/test_nvx_tools.py: the attribution of policy paths to shares, file shares, the systemd policy, the run share targets, and the environment rules, with the environment files and distro manifests on temporary files. The JSON and codec errors have tests of their own. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Record what the Python code does with fixed and seeded random inputs, and compare the Rust port with those records in golden tests, so that the port stays faithful byte for byte without Python at test time, also after nvx.py delegates to nvx-rs and the Python code goes away. scripts/nvx_rs_fixtures.py now also writes three fixtures to nvx-rs/tests/fixtures, which its --check covers with the others: - python.json: the str.isspace, decimal digit, and str.isprintable tables; repr, str.split(), str.strip(), int, and uuid.UUID of strings; str(), parts, name, parents, and is_absolute of pure paths, joins, and Windows path equality; ntpath.normpath; and the values and errors of json.loads and of the UTF-8 codec. - sandbox.json: parse_workload_identity, SandboxLayer.parse, parse_mount_value, validate_mount_target, aggregate_child_name, SandboxMount with its OpenVMM arguments, command-line fragment, and absolute paths, and SandboxLaunch with its OpenVMM arguments and kernel command line. The launches include, for each kind of share set, command lines that fill the 1024-byte budget exactly and that exceed it by one byte. - sandbox_options.json: _validate_run_share_targets, _sandbox_mounts, encode_exec_environment, the --environment-file rules of command_sandbox, which runs with exec_workload patched, and _validate_sandbox_systemd_policy, on temporary files. The generator patches sandbox.Path with pure POSIX and Windows paths whose current directory is fixed, so that the cases of both flavors come out the same on any host: CPython 3.14 writes identical files on Linux and Windows. It writes ASCII JSON with one case per line. The random corpus is mostly valid input, so that most cases reach the arguments and command lines, and invalid at a fixed rate. Each fixture has a random generator of its own, so the egress policy fixture is unchanged. The Rust golden tests read the fixtures with serde_json, run the same inputs through the port, and report each case that differs. They run the cases of both path flavors on every host, through the internal constructors that take a flavor and a fixed current directory, and the file cases on temporary files. The JSON cases go through python::json with the hooks of json.loads, so they check the reader that the egress policy compiler shares. .gitattributes marks the fixtures, about 1.2 MB, as generated, so that reviews collapse them. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Extend nvx::host::artifacts to every artifact of the guests that nvx.py runs, so that nvx-rs can stand in for the runtime lookups of scripts/nvx_tools/common.py: artifact_path, openvmm_binary_path, and the require_file checks of nvx.py run and nvx.py sandbox. - Guest describes the Alpine, Ubuntu, and Azure Linux guests of scripts/nvx_tools/guests.py: their names, distributions, initramfs images and package manifests, and the Ubuntu distro layer, ubuntu-distro.erofs, with its manifest. Guest::from_name fails as guest_descriptor does. - ArtifactDir names them in a repository checkout, as build/NAME and openvmm/target/release/openvmm[.exe], where local builds write them and where nvx.py download installs a release, or in an extracted release package, as guest/NAME and bin/openvmm[.exe], where package_release stages them. ArtifactDir::from_dir tells the layouts apart as Artifacts::from_dir does, and makes the directory absolute without resolving links, so callers pass the resolved REPO_ROOT of nvx.py as it is. - require_openvmm, require_kernel, and require_initramfs check the files with the messages of nvx.py: "OpenVMM release binary not found: PATH", "Linux direct kernel not found: PATH", and "DISTRIBUTION initramfs not found: PATH". sandbox_artifacts checks the OpenVMM executable, the kernel, and the Alpine initramfs in the order of nvx.py sandbox run and start, which call the last "initramfs". - paths lists every artifact as ArtifactPaths, which serializes as JSON, with the guests in nvx.py's order, for nvx-rs artifacts. Like nvx.py, the lookup reads no SOURCE-MANIFEST.json, and checks only the files that a command needs. Artifacts and its discovery, which aci_edge_sandboxes uses, are unchanged: Artifacts::from_dir only shares its layout detection, and the systemd policy of the sandbox options now takes the distro manifest suffix from here. The tests cover both layouts, the layout of this repository, the order and messages of the checks, and the JSON form. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Describe in nvx-rs/README.md the nvx::host::sandbox module, with its mount and options submodules, and the guest artifact lookup that ArtifactDir, Guest, and ArtifactPaths add to nvx::host::artifacts, beside the discovery that aci_edge_sandboxes uses. A new Python parity section explains which Python semantics the ports keep, including the egress policy compiler, the private python module that implements them, and the fixtures that scripts/nvx_rs_fixtures.py records, with the commands that regenerate and check them, the CPython 3.14 that they need, and why CI does not check them; the egress policy section points to it. The features table names sha2, which the host feature now enables. Update the trees and the nvx-rs summary of doc/project-structure.md for the sandbox launch profile and the golden fixtures. Part of #359. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Pedro Henrique Penna (ppenna)
requested review from
Enrique Saurez (esaurez)
and
a balanced review from Copilot
October 17, 2026 01:48
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Cross-platform behavioral compatibility, unsafe Win32 filesystem calls, and guest-build migration require final human review with CI evidence.
No open issues 🎉
What changed in this PR
Advances #359 by porting Python’s sandbox behavior and artifact lookup into nvx-rs, preparing for later CLI delegation while Python remains authoritative.
Changes:
- Adds sandbox launch types, Python-compatible semantics, and golden parity tests.
- Extends guest artifact discovery and documents the migration.
- Includes stacked changes for shared egress lowering and Rust guest-binary builds, provenance, and CI.
| File | Description |
|---|---|
| THIRD_PARTY_NOTICES.md | Records Rust guest runtime licenses. |
| SOURCE-MANIFEST.json | Adds Rust toolchain and source identities. |
| scripts/test_tool_versions.py | Checks musl target installation. |
| scripts/test_guest_ingress.py | Removes C-based ingress tests. |
| scripts/test_guest_binaries.py | Tests source identity and staged-build validation. |
| scripts/setup/setup-linux-runner.sh | Explicitly selects the target toolchain. |
| scripts/setup/setup-linux-mshv.sh | Installs and verifies the musl target. |
| scripts/nvx.py | Adds the guest-source digest command. |
| scripts/nvx_tools/ubuntu.py | Includes Rust inputs in Ubuntu provenance. |
| scripts/nvx_tools/release.py | Validates Rust source and toolchain metadata. |
| scripts/nvx_tools/guest_binaries.py | Adds staged-build validation and normalized digests. |
| scripts/nvx_tools/ci.py | Tracks Rust inputs for VMM tests. |
| scripts/nvx_tools/build.py | Installs Rust binaries instead of compiling C. |
| scripts/nvx_tools/build_constants.py | Defines Rust build and packaging inputs. |
| scripts/nvx_tools/benchmark.py | Records Rust guest-binary provenance. |
| scripts/nvx_tools/azurelinux.py | Includes Rust inputs in image digests. |
| scripts/nvx_tools/archive.py | Excludes Rust build outputs from archives. |
| README.md | Documents shipped Rust guest binaries. |
| nvx-rs/src/protocol/tests.rs | Pins wire constants with literal assertions. |
| nvx-rs/src/protocol/mod.rs | Updates shared-protocol documentation. |
| nvx-rs/src/protocol/features.rs | Documents shared guest feature definitions. |
| nvx-rs/src/host/sandbox/mod.rs | Adds sandbox launch validation and arguments. |
| nvx-rs/src/host/sandbox/layer.rs | Adds EROFS layer parsing and roles. |
| nvx-rs/src/host/sandbox/golden.rs | Replays Python sandbox parity fixtures. |
| nvx-rs/src/host/python/windows.rs | Implements Python-compatible Win32 filesystem queries. |
| nvx-rs/src/host/python/uuid.rs | Implements Python-compatible UUID parsing. |
| nvx-rs/src/host/python/text.rs | Preserves Python strings and codec errors. |
| nvx-rs/src/host/python/mod.rs | Organizes Python compatibility helpers. |
| nvx-rs/src/host/python/golden.rs | Checks Python semantics against fixtures. |
| nvx-rs/src/host/mod.rs | Exposes sandbox and compatibility modules. |
| nvx-rs/src/guest/time/mod.rs | Updates comments after C retirement. |
| nvx-rs/src/guest/time_probe/mod.rs | Updates compatibility documentation. |
| nvx-rs/src/guest/reseed.rs | Removes obsolete C references from comments. |
| nvx-rs/README.md | Documents sandbox, parity, and guest builds. |
| nvx-rs/Cargo.toml | Enables SHA-256 for sandbox share naming. |
| guest/common/nvx-reseed.c | Removes the replaced C implementation. |
| guest/common/nvx-mmio-write.c | Removes the replaced C implementation. |
| guest/common/nvx-exit.c | Removes the replaced C implementation. |
| guest/common/nvx-device-io.c | Removes the replaced C implementation. |
| guest/common/nvx-console-pending.c | Removes the replaced C implementation. |
| docker/Dockerfile | Shares one Rust build across guest images. |
| doc/usage.md | Documents the guest-source digest command. |
| doc/setup.md | Documents Rust musl prerequisites. |
| doc/project-structure.md | Updates module and source ownership descriptions. |
| doc/guests/ubuntu-guest.md | Documents Rust binaries and provenance. |
| doc/guests/azurelinux-guest.md | Documents shared binaries and image inputs. |
| doc/contribute.md | Updates guest testing guidance. |
| doc/ci.md | Documents Rust-aware caching and setup. |
| doc/build.md | Explains Rust builds and packaging provenance. |
| doc/benchmarks.md | Updates guest-binary provenance descriptions. |
| aci_edge_sandboxes/src/openvmm/mod.rs | Updates lowering documentation and error tests. |
| aci_edge_sandboxes/README.md | Documents shared egress lowering changes. |
| .github/workflows/copilot-setup-steps.yml | Triggers setup validation for Rust changes. |
| .github/copilot-instructions.md | Documents musl target provisioning. |
| .github/actions/validate-nvx/action.yml | Removes retired C test suites. |
| .github/actions/build-guest-artifacts/action.yml | Adds Rust-aware guest cache identities. |
| .gitattributes | Marks parity fixtures and tables as generated. |
Review effort: Balanced
This was referenced Oct 17, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #359 (PR 2 of 5).
Summary
This PR ports the launch profile of
nvx.py sandboxtonvx-rsbyte for byte, and extends artifact discovery to the guests thatnvx.pyruns, so that thenvx-rsCLI (PR C) can take overnvx.py sandbox, and the artifact steps ofnvx.py run, as a thin layer over the library. Python stays authoritative and unchanged until PR D.nvx::host::sandboxportsSandboxLaunch,SandboxLayer,SandboxMount, and the helpers ofscripts/nvx_tools/sandbox.py: one to three EROFS layers and the ext4 scratch disk; the entrypoint, arguments, hostname, workload identity, and resource limits; live shares with their reserved targets, access policies, owner, aggregate, and host-path checks; the--microvm-sandbox-blockand--microvm-workload-identityarguments; and the kernel command line within its 1024-byte budget, all with Python's messages.nvx::host::sandbox::optionsports the pure option checks ofcommand_sandboxand the helpers ofscripts/nvx.py:_sandbox_mounts,_validate_sandbox_systemd_policywithSYSTEMD_ENTRYPOINTS,_validate_run_share_targets,parse_workload_identity, and the managed exec environment rules.nvx::host::artifactsgainsArtifactDir,Guest, andArtifactPaths: the OpenVMM executable, the kernel, the Alpine, Ubuntu, and Azure Linux initramfs images, and the Ubuntu distro layer, in the repository layout (build/,openvmm/target/release/) and the release layout (guest/,bin/), checked in the order and with the messages ofnvx.py.ArtifactPathsserializes as JSON fornvx-rs artifacts. TheArtifactsdiscovery ofaci_edge_sandboxesis unchanged.nvx-rs/tests/fixtures/{python,sandbox,sandbox_options}.json, hold fixed cases and seeded random corpora for both path flavors, and the Rust tests replay them without Python.scripts/generate_egress_policy_fixtures.pyfrom Port egress policies to nvx-rs #594 becomesscripts/nvx_rs_fixtures.py, which writes every Python-parity file ofnvx-rsand checks them all with one--check. The egress policy fixture regenerates byte for byte, except its comment line, which now names the script.Stack
This PR is based on #594 (PR A, the egress policy compiler), which is based on #561. Until they merge, its diff includes their commits. Its own commits are:
10be3490bnvx-rs: port the sandbox launch profile3aa5ca289nvx-rs: port the sandbox option checks of nvx.py6c46965ccnvx-rs: check the sandbox port against golden Python fixturese1b23591envx-rs: look up the guest artifacts that nvx.py runs1fa32b85ddoc: describe the sandbox port and guest artifact lookup of nvx-rsPR C adds the
nvx-rsCLI, PR D makesnvx.pydelegate to it, and PR E ships it in releases and completes #359. #576 and #579, which changesandbox.py, are still open, so this PR ports dev's current behavior; whichever merges needs a port and regenerated fixtures.Faithful-port notes
The
pythonmodulenvx.pychecks its options with Python's own string, integer, UUID, path, and JSON operations, and its messages show values as Python formats them. The ports keep those semantics through the privatenvx::host::pythonmodule that #594 introduced, which this PR extends with one implementation per concern:text(PyText, from Port egress policies to nvx-rs #594) stays the one string type that holds surrogates, and gains the errors of Python's UTF-8 codec:PyText::encode_utf8anddecode_utf8.json(from Port egress policies to nvx-rs #594) keeps its API, hooks, and arena, but its reader is now a port of the C scanner of CPython 3.14, so that a refusal carries the message and position of Python'sJSONDecodeError. The previous reader reported where it stopped, which is not always Python's position: an unterminated string fails at its opening quote.Loadsare the hooks of plainjson.loads, whoseintrefuses more than 4300 digits. The egress policy fixture and Port egress policies to nvx-rs #594's tests pass unchanged.path(from Port egress policies to nvx-rs #594) keepspython_path,posix_str, andwindows_str, and gainsPurePathandFlavor:pathlib's pure paths and thesplitroot,join,isabs, andsplitofos.path, andntpath.normpath, on one parser withpython_path. Windows paths follow CPython on Windows, whose Csplitrootandnormpathindex UTF-16 code units.builtins(str.isspace, which unlike Rust's whitespace includes U+001C to U+001F,str.split(),str.strip(),repr, andint),uuid(uuid.UUID),unicode(the generated Unicode 16.0.0 tables),fs(the queries ofos.path,os.statidentities, andPath.resolve()), andwindows(the Win32 calls of CPython'sos.stat).Deviations from Python
python_pathstill prints such paths as Python does.[Errno N] text: 'path'and[WinError N] text: 'path'forms, but the text comes from Rust'sio::Error, which may word an error differently from CPython.RecursionError, as in Port egress policies to nvx-rs #594.os.statfollows CPython's classic path throughFILE_ID_INFO, not theGetFileInformationByNamefast path of recent Windows builds; the identities agree.PureWindowsPathindexes code points, so a character beyond the Basic Multilingual Plane before:reads as a drive there. The port follows Windows hosts, where it does not.(u32, u32)and the layer roles are an enum, so values that Python could hold but never accepts from options cannot be represented. A--mount-deny,--mount-allow, or--mount-writeindex beyond the--mountoptions, which argparse cannot produce, is an error instead of Python'sIndexError.ArtifactDirmakes its root absolute without resolving links, unlikeREPO_ROOT'sresolve(), so callers pass the resolved root.What PR C still needs from
command_sandboxandparse_argsargparse parsing and its usage errors (exit 2), type conversions and choices, the egress options through #594, building the launch from the options, the
run,provision,start,exec,stop, anddeprovisionoperations with the state-directory layout, configuration formats, and outcome reports,--dry-runprinting, how the CLI learns the artifact root,FEATURESnegotiation, and the control session.Validation
-D warningsfor all targets and features, and for the library without default features; tests with all and with default features (259 and 256 library tests); rustdoc with-D warnings; MSRV 1.89; the synthetic-releasebundledtest; theaarch64-apple-darwinchecknvx.py test-agent-bundlingnvx.py verify; Ruff check and format; Pyright for Linux and Windows;check_markdown_links.py; the validate-nvx unittest list (1110 tests),test_adversarial.py, andtest_hosts.py;nvx_rs_fixtures.py --checknvx_rs_fixtures.py --checkpasses on the files that Windows wroteThe tests need no Python and no hypervisor. Left to CI: the Linux guest-binary steps of check-nvx-rs, the microVM and workload jobs, MSHV, and the benchmarks.
CI's Pythons predate 3.14, so CI does not run
nvx_rs_fixtures.py --check, and nothing in CI notices fixtures that no longer match the Python code. Until PR D retires the generator with the Python code, a change to that code, or to the generator, needs a manual regeneration with CPython 3.14.