You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
nvx-rs: add guest operations for a layered root - #674
Adds nvx::guest::root, the first root operations of the guest library. A guest init or agent, whether or not it is PID 1, bootstraps the guest for a layered root, assembles the root, mounts live shares in it, and tears it down, with typed results and owned handles:
bootstrap(&Layout, &Resources) -> Result<Bootstrap, Error> creates the agent's and the workload's sibling cgroups of a cgroup Layout. It moves every process of their parent into the agent's cgroup, except kernel threads, in passes until none is left, so that a process that a moved one starts meanwhile moves too (at most 16 passes, then EBUSY). It enables the memory controller for the agent's cgroup and writes the agent's memory.low that the caller gives, then prepares the workload cgroup with ExecutionCgroup::prepare of nvx-rs: expose the guest runtime as a library #619, as the managed agent prepares its own, which enables pids and the controllers of the workload's memory.max and pids.max in one write and writes them. It sets user.max_user_namespaces to 0, so that user namespaces stay unavailable whatever the kernel supports; a kernel without sysctls lacks that limit, which the bootstrap accepts only when the kernel has no user namespaces either (/proc/self/ns/user). It mounts the cgroup v2 hierarchy first unless the parent already is a cgroup. A failure names its BootstrapStep. The Bootstrap holds the layout, the workload cgroup's ExecutionCgroup, and the IDs that it moved.
assemble(&Assembly) -> Result<Root, AssemblyError> finds the distro and scratch block devices by the base addresses of their MMIO windows (/proc/iomem, then /sys/devices/virtio-mmio-cmdline), polling every 50 ms until the caller's timeout, checks each one's expected size in sectors, and flushes its buffers (BLKFLSBUF). It checks the EROFS image's expected UUID in its superblock, reading nothing else, before it mounts the image read-only. Then it mounts the ext4 scratch nosuid,nodev, creates separate upper and work directories on it, with mode 0755 whatever the caller's umask, since the overlay's root directory takes the upper's mode, and mounts the overlay, which is the root, nosuid,nodev with metacopy=on,xino=on. When a step fails, it unmounts what it had mounted and reports both that failure and any unmount that failed. Since a root's handle unmounts by path, it holds an exclusive, non-blocking flock of the runtime directory from the start of its assembly until its teardown: before it checks or mounts anything, an assembly refuses a runtime directory that another root holds, in this process or another (Error::Reserved), and one where a file system, or a bind mount of a directory, is mounted already at one of its mount points, such as one that a failed teardown left (Error::Occupied). It compares the IDs of the mounts that a mount point and its parent directory are on, not their devices, since a bind mount of a directory of the parent's own file system shows the parent's device. Those include shares, where an aggregate live share is mounted outside the overlay, so that an aggregate whose unmount alone failed is found too. Since a root mounts and unmounts by path, an assembly also refuses a runtime directory whose path is not canonical, or that a symbolic link leads through (Error::Runtime), and a root mounts nothing where a mount point, or layers, is a symbolic link (Error::MountPoint), so that none of these paths can lead elsewhere once one of the root's mounts hides a directory along it.
Root owns the mounts. mount_share and mount_aggregate port the shell init's live-share checks unchanged: canonical absolute targets, reserved characters and targets, overlaps, no symbolic link along the path in the root, directory or regular-file targets, and modes 0755 and 0644 for the components that they create. They bind each child of an aggregate through guest::mappings::Mapping of nvx-rs: add a guest mapping API with explicit export and target roots #652, the bind and restricting remount with which the managed agent applies its host mapping table, so a child that the aggregate lacks, or holds as another kind, fails to mount before its bind. Every aggregate is mounted at the same place, so a root has one at most: once it has mounted one, even one whose children failed to bind, it refuses another (ShareRefusal::SecondAggregate). teardown(self) unmounts the shares in reverse mount order, then the overlay, the image, and the scratch. It attempts every unmount and returns a TeardownError that lists each failed Unmount. Dropping a Root unmounts the same way and ignores failures.
No second copies of existing components: the root takes Access and the canonical-path check from guest::mappings, the cgroup helpers from guest::cgroup, and the image's UUID as the nvx::uuid::Uuid of nvx-rs: plan sandbox launches with the image format and upper #676. The guest targets that live shares may not take, and the characters that no target may hold, move to the new nvx::protocol::share, which the host's sandbox::mount and the guest's root both check; the host keeps its constants' names and its messages.
Typed errors:Error names the step, role, device, mount or ShareRefusal that failed, and its text keeps the shell's messages where they exist.
The operations collect no child's status, change neither the caller's root nor its credentials, start no async runtime, and never power the VM off.
sys::mount takes the file system's options, and sys gains BLKFLSBUF, an unmount, the ID of the mount that a path is on (from /proc/self/fdinfo), and a non-blocking exclusive lock of a directory.
For code built on an earlier head of this branch: Assembly::distro_uuid is an Option<nvx::uuid::Uuid>, no longer an Option<[u8; 16]>, and the expected and actual of Error::Uuid are nvx::uuid::Uuids. Uuid::from_bytes converts 16 bytes, and the image identity that protocol::root parses already is a Uuid. Nothing else changed in guest::root, protocol::share, or the Mapping binding.
Why
Part of #621: this is the first part of its A3.1, the operations and handles for the bootstrap, the cgroups, and EROFS over an ext4 scratch. The bootstrap uses the cgroup Layout and ExecutionCgroup of #619. Keeping the operations apart from NVX's PID 1 lets guest agents that are not PID 1 call them, with their own transport, reaping, and shutdown. NVX's managed layered root keeps running the shell init: running these operations from NVX's Rust PID 1 is deferred until after A3.4.
Mounts, unmounts, buffer flushes, and control-file writes go through a private seam, so that the unit tests run without privileges, against procfs, sysfs, and cgroup fixtures in a temporary directory. They check the order, flags, and options of every mount, the teardown order, a failed unmount that is reported while the later ones still run, the cleanup of a failed assembly, a second root refused at a runtime directory that another root holds, whether that root is assembled or still waiting for its device, or where a file system is mounted, an aggregate whose unmount alone failed included, or where a directory is bound (with the kernel itself, in a user and mount namespace of the test's own, where the host allows them), runtime directories that are not canonical or that a symbolic link leads through, a symbolic link at each mount point, new upper and work directories that keep mode 0755 under umask 077 (in a process of the test's own) while an existing one keeps its mode, a second aggregate, an aggregate child that the aggregate lacks or holds as another kind, late and missing devices, size and UUID mismatches, every share refusal, the reserved targets that the host and the guest share, and the failure of each bootstrap step, a missing namespace sysctl included. The microVM scenarios run the privileged path once a guest image calls it; until then, see Real VMs below.
Validation
On Linux (WSL 2, Ubuntu 22.04, x86_64), with Rust 1.95.0 and the 1.89 MSRV:
python3 scripts/nvx.py check-guest-binaries, then build-guest-binaries --output <dir> and test-guest-binaries --binaries <dir>
tests/runtime-adapter in its protocol, host and guest (musl) configurations: fmt, clippy, build, test, and the MSRV check
aci_edge_sandboxes: fmt, both clippy configurations, both test configurations, docs, the MSRV check, and the macOS check
python3 scripts/check_markdown_links.py
No guest binary uses the module yet, so the stripped guest binaries keep their sizes and no guest behavior changes; the microVM scenarios are left to CI's backend matrix.
Real VMs: a Rust PID 1, which is deferred and kept without a PR on the branch esaurez/a3-1-pid1-bootstrap-20261020, ran these operations, as of e27915e, before the rebase onto #676 and the switch to nvx::uuid::Uuid, on WHP (Windows), KVM (WSL 2), and MSHV, in test-microvm --scenario managed-lifecycle --scenario managed-exec-config --scenario sandbox-lifecycle and in the workflow's sandbox steps. Among them, a managed sandbox bound an aggregate of two directories and two files, which its workload checked, and its stop left the scratch's ext4 journal clean.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Privileged mount and cgroup behavior needs human review because real-guest validation is deferred and a bootstrap boundary error remains unresolved.
No open issues 🎉
1 previously missed issue
In code that hasn't changed since last review
Bootstrap returns EBUSY after final pass drains all processes
nvx-rs/src/guest/root/bootstrap.rs:280
If the sixteenth pass moves the last remaining userspace process, bootstrap still returns EBUSY: found records the snapshot before migration, not whether processes remain afterward. This aborts bootstrap even though placement succeeded. Check the parent once more before returning EBUSY, without allowing another migration pass, and add a test that drains the parent on the sixteenth pass alongside the persistent-process test.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
The new privileged mount and cgroup lifecycle needs human review because kernel interactions currently have simulated rather than live integration coverage.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
Privileged lifecycle changes need human review, with unresolved aggregate-mount ownership issues and real-kernel integration validation deferred.
No open issues 🎉
2 previously missed issues
In code that hasn't changed since last review
Check aggregate mount for leaks during occupancy and teardown
nvx-rs/src/guest/root.rs:314
The occupancy check omits <runtime>/shares, where an aggregate is mounted outside rootfs. If teardown fails to unmount that aggregate but removes the overlay, image, and scratch, the next assembly succeeds. Its next aggregate mount can then cover the leaked mount, which remains after the new root's teardown. Check the aggregate mount point too, allowing its directory to be absent, and add a regression test where only the aggregate unmount fails.
Prevent duplicate aggregate mounts at the same path
nvx-rs/src/guest/root/share.rs:245
Calling mount_aggregate twice with disjoint child targets stacks both aggregates at this same path. If unmounting the newer aggregate fails, teardown later targets it again under the older aggregate's name, leaving the older mount unattempted and misidentifying the failure. Reject a second aggregate while this root owns one, including after a partially failed call, or use distinct mount points. Add coverage for repeated calls and teardown failures.
The reason will be displayed to describe this comment to others. Learn more.
🔵 Needs a closer look
The new privileged mount and cgroup lifecycle needs human review because successful assembly and bootstrap are not yet exercised by real-guest scenarios.
Add `nvx::guest::root`, the operations through which a guest init or
agent, whether or not it is PID 1, bootstraps the guest for a layered
root, assembles the root, mounts live shares in it, and tears it down.
Each returns typed results and owned handles.
`bootstrap` creates the agent's and the workload's sibling cgroups of a
cgroup `Layout`, and moves every process of their parent into the
agent's, kernel threads excepted, in passes until none is left, so that
a process that a moved one starts meanwhile moves too. It enables the
`pids` controller, and `memory` when a memory value needs it, writes the
agent's `memory.low` and the workload's `memory.max` and `pids.max` that
the caller gives, and sets `user.max_user_namespaces` to 0, so that user
namespaces stay unavailable whatever the kernel supports. It mounts the
cgroup v2 hierarchy first unless the cgroups' parent is a cgroup.
`assemble` finds the distro and scratch block devices by the base
addresses of their MMIO windows in `/proc/iomem`, polling until they
appear, checks each one's expected size, and flushes its buffers. It
checks the EROFS image's expected UUID in its superblock, reading
nothing else, before it mounts the image read-only, then mounts the ext4
scratch `nosuid,nodev`, with separate upper and work directories, and
the overlay with `metacopy=on,xino=on`. It returns a `Root` that owns
the mounts, and unmounts them again when a later step fails.
`Root::mount_share` and `Root::mount_aggregate` mount the VMM's virtio-fs
live shares with the shell init's checks, unchanged: canonical absolute
targets, reserved targets, overlaps, no symbolic link along the path in
the root, and directory or file targets. `Root::teardown` unmounts the
shares in reverse mount order, then the overlay, the image and the
scratch. It attempts every unmount and reports each one that failed;
dropping a root unmounts the same way and ignores failures.
The operations collect no child's status, change neither the caller's
root nor its credentials, start no async runtime, and never power the VM
off. The mounts, unmounts, buffer flushes and control-file writes go
through a private seam, so that unit tests check the order and flags of
each mount and unmount, the typed errors, and the failed unmounts that
teardown reports, against procfs, sysfs and cgroup fixtures in a
temporary directory.
NVX's managed layered root still runs the shell init; this change only
adds the operations.
Part of #621.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
The overlay that `assemble` mounts is the workload's root, so mount it
`nosuid,nodev`, as #621 requires of the merged root: those flags apply to
each mount, and the scratch's do not reach the files of the overlay.
NVX's workloads already run with `no_new_privs` and their own `/dev`, so
nothing that they do changes.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
The bootstrap's placement returned EBUSY whenever its sixteenth pass
found a process, even when that pass moved the last one. Check the
parent cgroup once more, without moving anything, before failing.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
A kernel built without sysctls lacks user.max_user_namespaces, so the
bootstrap failed with ENOENT there, even when the kernel has no user
namespaces to limit. Accept the missing sysctl when /proc/self/ns/user
shows that the kernel has no user namespaces, and keep failing
otherwise, since such a kernel's user namespaces cannot be limited.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
A root unmounts what it mounted by its paths, so a second assembly at
the same runtime directory stacked its mounts on the first root's, and
dropping the first handle then unmounted the second root's mounts.
Refuse an assembly when a file system is mounted already at one of its
mount points, before it mounts anything, with a new Error::Occupied.
A mount that a failed unmount left in place keeps the directory
occupied too.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
Two assemblies for one runtime directory could both pass the check for
mounted file systems before either mounted anything, one waiting for its
device while the other finished. The later one then mounted over the
earlier root, and the two handles, which unmount by path, shared mount
points.
An assembly now takes an exclusive, non-blocking flock of the runtime
directory before it checks the mount points, and the root holds it until
it is torn down or dropped, after its unmounts. A runtime directory that
another root holds, in this process or another, fails the assembly with
Error::Reserved before it checks or mounts anything. The mount-point
check stays for file systems that no root holds, such as one that a
failed teardown left.
A test runs a second assembly while the first waits for its device.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
An aggregate live share is mounted at shares in the runtime directory,
outside the overlay. A teardown that failed to unmount it alone, but
unmounted the overlay, the image, and the scratch, left a runtime
directory that the next assembly accepted, and that root's aggregate
then covered the leaked mount, which outlived its teardown. An assembly
now also refuses a runtime directory where a file system is mounted at
shares, which may be missing.
A second mount_aggregate call stacked another aggregate at the same
path, so a failed unmount of the newer one was retried under the older
one's name, and the older one was never attempted. A root now has one
aggregate at most: once it has mounted one, even one whose children
failed to bind, it refuses another with ShareRefusal::SecondAggregate.
The tests' fake kernel, as the kernel does, now fails to check a
missing directory for a mount.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
An assembly refused a runtime directory where something was mounted
already at one of its mount points by comparing each mount point's
device with its parent directory's. A bind mount of a directory of the
parent's own file system shows the parent's device, so the assembly
missed it and mounted over it.
The kernel's check now compares the IDs of the mounts that the mount
point and its parent are on, which procfs shows in a descriptor's
fdinfo. A test binds a directory at the distro layer's mount point in a
user and mount namespace of its own, where the host allows them, and
checks that the assembly refuses it there.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
A root mounts and unmounts at paths below its runtime directory, as
given, so they must lead to the same directories whatever the root has
mounted. An assembly accepted a runtime directory with a `..` component,
or one that a symbolic link led through, and created its mount points
through symbolic links. Such a path could lead through a directory that
one of the root's own mounts hid later: with a `scratch/probe`
directory, the runtime directory `.../scratch/probe/../..` led nowhere
once the scratch was mounted, so the assembly failed and could unmount
neither the image nor the scratch.
An assembly now refuses a runtime directory whose path is not canonical,
as a live share's target must be, or that a symbolic link leads through,
and a root creates `layers` and the mount points a component at a time,
refusing a symbolic link, as the aggregate's mount point, `shares`, is
too. The share targets' canonical check moves to a helper that both use.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
An assembly created the scratch's upper and work directories with the
caller's umask, and the overlay's root directory takes the upper's
mode. A caller with a restrictive umask, such as 077, so assembled a
root whose directory a workload that is not root could not enter. The
shell init's mkdir -p created them as 0755 under PID 1's umask.
An assembly now gives the upper and work directories that it creates
mode 0755 whatever the umask, and leaves the mode of one that the
scratch holds already, from an earlier boot. A test checks both under
umask 077, in a process of its own, since the umask is the process's.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
… code
The guest root operations repeated components that the guest library and
the host already had. They now use those components instead:
- A root binds each child of an aggregate live share through
guest::mappings::Mapping, the bind and restricting remount with which
NVX's managed agent applies its host mapping table, and takes its
Access. The root keeps its own checks of the targets, which refuse
symbolic links, and creates what is missing of them first. A child
whose entry in the aggregate is missing, or of another kind, now fails
to mount before its bind.
- The root checks its targets and its runtime directory with the
canonical-path check of mappings, which also bounds their length.
- The guest targets that live shares may not take, and the characters
that no target may hold, move to nvx::protocol::share, which the host's
sandbox::mount and the guest's root both check. The host keeps its
constants' names and its messages.
- The bootstrap prepares the workload cgroup with
ExecutionCgroup::prepare, as the managed agent prepares its own, and
creates the agent's cgroup and enables its memory controller through
the same cgroup helpers. BootstrapStep::WorkloadCgroup now covers the
workload cgroup's values, and the Controllers, WorkloadMemoryMax, and
WorkloadPidsMax steps go.
- sys::mount takes the file system's options, so the guest has one mount
wrapper.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
nvx::uuid now holds the canonical UUID that the host and the guest
share. A guest root's assembly took the EROFS image's expected UUID as
16 raw bytes, and displayed a mismatch with a formatter of its own, a
second copy of that type's canonical text. Assembly::distro_uuid and
Error::Uuid now hold an nvx::uuid::Uuid, which the superblock check
builds from the image's bytes, and a mismatch displays through it, with
the same text as before.
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: 59cd1d15-8546-491b-8b77-8f679d03492d
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Part of #621. Refs #619.
What
Adds
nvx::guest::root, the first root operations of the guest library. A guest init or agent, whether or not it is PID 1, bootstraps the guest for a layered root, assembles the root, mounts live shares in it, and tears it down, with typed results and owned handles:bootstrap(&Layout, &Resources) -> Result<Bootstrap, Error>creates the agent's and the workload's sibling cgroups of a cgroupLayout. It moves every process of their parent into the agent's cgroup, except kernel threads, in passes until none is left, so that a process that a moved one starts meanwhile moves too (at most 16 passes, thenEBUSY). It enables thememorycontroller for the agent's cgroup and writes the agent'smemory.lowthat the caller gives, then prepares the workload cgroup withExecutionCgroup::prepareof nvx-rs: expose the guest runtime as a library #619, as the managed agent prepares its own, which enablespidsand the controllers of the workload'smemory.maxandpids.maxin one write and writes them. It setsuser.max_user_namespacesto 0, so that user namespaces stay unavailable whatever the kernel supports; a kernel without sysctls lacks that limit, which the bootstrap accepts only when the kernel has no user namespaces either (/proc/self/ns/user). It mounts the cgroup v2 hierarchy first unless the parent already is a cgroup. A failure names itsBootstrapStep. TheBootstrapholds the layout, the workload cgroup'sExecutionCgroup, and the IDs that it moved.assemble(&Assembly) -> Result<Root, AssemblyError>finds the distro and scratch block devices by the base addresses of their MMIO windows (/proc/iomem, then/sys/devices/virtio-mmio-cmdline), polling every 50 ms until the caller's timeout, checks each one's expected size in sectors, and flushes its buffers (BLKFLSBUF). It checks the EROFS image's expected UUID in its superblock, reading nothing else, before it mounts the image read-only. Then it mounts the ext4 scratchnosuid,nodev, creates separateupperandworkdirectories on it, with mode 0755 whatever the caller's umask, since the overlay's root directory takes the upper's mode, and mounts the overlay, which is the root,nosuid,nodevwithmetacopy=on,xino=on. When a step fails, it unmounts what it had mounted and reports both that failure and any unmount that failed. Since a root's handle unmounts by path, it holds an exclusive, non-blockingflockof the runtime directory from the start of its assembly until its teardown: before it checks or mounts anything, an assembly refuses a runtime directory that another root holds, in this process or another (Error::Reserved), and one where a file system, or a bind mount of a directory, is mounted already at one of its mount points, such as one that a failed teardown left (Error::Occupied). It compares the IDs of the mounts that a mount point and its parent directory are on, not their devices, since a bind mount of a directory of the parent's own file system shows the parent's device. Those includeshares, where an aggregate live share is mounted outside the overlay, so that an aggregate whose unmount alone failed is found too. Since a root mounts and unmounts by path, an assembly also refuses a runtime directory whose path is not canonical, or that a symbolic link leads through (Error::Runtime), and a root mounts nothing where a mount point, orlayers, is a symbolic link (Error::MountPoint), so that none of these paths can lead elsewhere once one of the root's mounts hides a directory along it.Rootowns the mounts.mount_shareandmount_aggregateport the shell init's live-share checks unchanged: canonical absolute targets, reserved characters and targets, overlaps, no symbolic link along the path in the root, directory or regular-file targets, and modes 0755 and 0644 for the components that they create. They bind each child of an aggregate throughguest::mappings::Mappingof nvx-rs: add a guest mapping API with explicit export and target roots #652, the bind and restricting remount with which the managed agent applies its host mapping table, so a child that the aggregate lacks, or holds as another kind, fails to mount before its bind. Every aggregate is mounted at the same place, so a root has one at most: once it has mounted one, even one whose children failed to bind, it refuses another (ShareRefusal::SecondAggregate).teardown(self)unmounts the shares in reverse mount order, then the overlay, the image, and the scratch. It attempts every unmount and returns aTeardownErrorthat lists each failedUnmount. Dropping aRootunmounts the same way and ignores failures.Accessand the canonical-path check fromguest::mappings, the cgroup helpers fromguest::cgroup, and the image's UUID as thenvx::uuid::Uuidof nvx-rs: plan sandbox launches with the image format and upper #676. The guest targets that live shares may not take, and the characters that no target may hold, move to the newnvx::protocol::share, which the host'ssandbox::mountand the guest's root both check; the host keeps its constants' names and its messages.Errornames the step, role, device, mount orShareRefusalthat failed, and its text keeps the shell's messages where they exist.sys::mounttakes the file system's options, andsysgainsBLKFLSBUF, an unmount, the ID of the mount that a path is on (from/proc/self/fdinfo), and a non-blocking exclusive lock of a directory.API change since e27915e
For code built on an earlier head of this branch:
Assembly::distro_uuidis anOption<nvx::uuid::Uuid>, no longer anOption<[u8; 16]>, and theexpectedandactualofError::Uuidarenvx::uuid::Uuids.Uuid::from_bytesconverts 16 bytes, and the image identity thatprotocol::rootparses already is aUuid. Nothing else changed inguest::root,protocol::share, or theMappingbinding.Why
Part of #621: this is the first part of its A3.1, the operations and handles for the bootstrap, the cgroups, and EROFS over an ext4 scratch. The bootstrap uses the cgroup
LayoutandExecutionCgroupof #619. Keeping the operations apart from NVX's PID 1 lets guest agents that are not PID 1 call them, with their own transport, reaping, and shutdown. NVX's managed layered root keeps running the shell init: running these operations from NVX's Rust PID 1 is deferred until after A3.4.Mounts, unmounts, buffer flushes, and control-file writes go through a private seam, so that the unit tests run without privileges, against procfs, sysfs, and cgroup fixtures in a temporary directory. They check the order, flags, and options of every mount, the teardown order, a failed unmount that is reported while the later ones still run, the cleanup of a failed assembly, a second root refused at a runtime directory that another root holds, whether that root is assembled or still waiting for its device, or where a file system is mounted, an aggregate whose unmount alone failed included, or where a directory is bound (with the kernel itself, in a user and mount namespace of the test's own, where the host allows them), runtime directories that are not canonical or that a symbolic link leads through, a symbolic link at each mount point, new
upperandworkdirectories that keep mode 0755 under umask 077 (in a process of the test's own) while an existing one keeps its mode, a second aggregate, an aggregate child that the aggregate lacks or holds as another kind, late and missing devices, size and UUID mismatches, every share refusal, the reserved targets that the host and the guest share, and the failure of each bootstrap step, a missing namespace sysctl included. The microVM scenarios run the privileged path once a guest image calls it; until then, see Real VMs below.Validation
On Linux (WSL 2, Ubuntu 22.04, x86_64), with Rust 1.95.0 and the 1.89 MSRV:
python scripts/nvx.py verify(also on Windows)nvx-rs, withNVX_BUNDLE=skip:cargo fmt --all --check;cargo clippy --locked --all-targets --all-features -- -D warnings;cargo clippy --locked --lib --no-default-features -- -D warnings;cargo test --locked --all-features;cargo test --locked;RUSTDOCFLAGS="-D warnings" cargo doc --locked --no-deps --all-features;cargo +1.89 check --locked --all-targets --all-features;cargo check --locked --all-targets --all-features --target aarch64-apple-darwinpython3 scripts/nvx.py check-guest-binaries, thenbuild-guest-binaries --output <dir>andtest-guest-binaries --binaries <dir>tests/runtime-adapterin its protocol, host and guest (musl) configurations: fmt, clippy, build, test, and the MSRV checkaci_edge_sandboxes: fmt, both clippy configurations, both test configurations, docs, the MSRV check, and the macOS checkpython3 scripts/check_markdown_links.pyNo guest binary uses the module yet, so the stripped guest binaries keep their sizes and no guest behavior changes; the microVM scenarios are left to CI's backend matrix.
Real VMs: a Rust PID 1, which is deferred and kept without a PR on the branch
esaurez/a3-1-pid1-bootstrap-20261020, ran these operations, as of e27915e, before the rebase onto #676 and the switch tonvx::uuid::Uuid, on WHP (Windows), KVM (WSL 2), and MSHV, intest-microvm --scenario managed-lifecycle --scenario managed-exec-config --scenario sandbox-lifecycleand in the workflow's sandbox steps. Among them, a managed sandbox bound an aggregate of two directories and two files, which its workload checked, and its stop left the scratch's ext4 journal clean.