Repository navigation
nvx-rs: assemble a guest root over a tmpfs or a scratch in an overlay mode - #775
Merged
Enrique Saurez (esaurez) merged 3 commits intoOct 23, 2026
Merged
Conversation
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Privileged mount assembly and kernel-dependent validation require human review, with an unresolved stale-mount check when switching upper kinds.
1 open issue
What changed in this PR
Extends NVX’s guest root library with typed image, upper, and overlay-mode inputs, advancing the shared layered sandbox runtime in #621.
Changes:
- Supports EROFS roots over ext4 scratch or tmpfs, with caller-selected journal and overlay policies.
- Validates device roles, mount-option length, and effective overlay options; exposes mounted layers and upper kind.
- Extends the runtime adapter’s configuration, probes, documentation, and tests for the new combinations.
| File | Description |
|---|---|
| tests/runtime-adapter/tests/vm_layered.rs | Adds upper/mode combinations and full-upper scenarios. |
| tests/runtime-adapter/tests/layered_launch.rs | Tests rejection of invalid policy options. |
| tests/runtime-adapter/tests/guest_session.rs | Tests RAM assembly and conflicting configuration. |
| tests/runtime-adapter/src/worker/probes.rs | Adds copy-up, rename, capacity, and disk probes. |
| tests/runtime-adapter/src/worker.rs | Passes typed root settings to guest assembly. |
| tests/runtime-adapter/src/root.rs | Encodes and validates overlay and journal policies. |
| tests/runtime-adapter/src/bin/runtime-adapter-host.rs | Adds overlay-mode and journal-policy arguments. |
| tests/runtime-adapter/README.md | Documents configuration, probes, and scenarios. |
| tests/runtime-adapter/Cargo.toml | Enables filesystem calls for capacity probes. |
| nvx-rs/src/guest/root/tests.rs | Covers upper modes, validation, and teardown failures. |
| nvx-rs/src/guest/root/kernel.rs | Reads effective mount options and extends the fake kernel. |
| nvx-rs/src/guest/root/block.rs | Checks device access and ext4 journal flags. |
| nvx-rs/src/guest/root.rs | Implements typed assembly, tmpfs uppers, and overlay validation. |
| nvx-rs/README.md | Updates the guest root API description. |
Review effort: Balanced
Contributor
There was a problem hiding this comment.
🔵 Needs a closer look
Privileged mount lifecycle changes and security-sensitive, kernel-dependent overlay checks warrant human review alongside the remaining backend CI results.
No open issues 🎉
1 resolved since last review
Review effort: Balanced
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-3-gpt-tmpfs-overlay-20261021
branch
from
October 23, 2026 00:30
27a196b to
24ecfbd
Compare
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-3-gpt-tmpfs-overlay-20261021
branch
from
October 23, 2026 03:16
24ecfbd to
5e8ec70
Compare
Enrique Saurez (esaurez)
marked this pull request as ready for review
October 23, 2026 03:22
… mode A guest root's assembly took an EROFS image and an ext4 scratch, and mounted the overlay with its one option set, metacopy=on,xino=on. The caller now names what it assembles with typed inputs, and nothing is probed: - the image, an EROFS file system with the UUID that the caller expects, if any, as an Image that later formats extend; - the upper: a whole-device ext4 scratch, with a journal policy that the assembly checks in the scratch's superblock feature flags before it mounts it, or a tmpfs at the kernel's default size, mounted nosuid,nodev, for a VM without a scratch device; - the overlay mode, whose one option set the overlay is mounted with: metacopy=on,xino=on by default, or metacopy=off,redirect_dir=nofollow for lowers that may carry untrusted overlay xattrs. The kernel's refusal of the set fails the assembly, with no fallback to another. Nor does the assembly accept the kernel's own fallbacks: without failing the mount, the kernel mounts an overlay read-only when it can't create its work directory, and drops metacopy=on when the upper can't hold the overlay's xattrs. The assembly reads the overlay's options in the kernel's mount table, where an option at its default, which a parameter of the overlay module holds, doesn't show, and refuses an overlay that isn't writable or lacks the mode's options. The assembly also checks each device's access mode against its role's, read-only for the distro and writable for the scratch, so that a device in the wrong role is refused before it is flushed. It checks every device, and the length of the overlay's options, before it mounts anything: mount(2) passes a file system one page of options, and the kernel cuts longer ones short, so options longer than 4,095 bytes are refused rather than mounted without their last options. The root reports what it mounted, its layers, the kind of its upper, and the overlay mode, and its teardown unmounts the live shares and mappings, the overlay, the layers from the top down, and then the upper, reporting every unmount that fails. The runtime adapter's worker names its EROFS image over its scratch with the new inputs, in the default mode. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 30bab0d7-7c3e-4120-9add-6c9cc1dc9e7e
The runtime adapter's worker assembled only an EROFS image over an ext4 scratch, in the default overlay mode. It now also assembles one over a tmpfs in RAM, in the overlay mode that the host names, so that the guest root's uppers and overlay modes get real-kernel coverage: - The root configuration takes two optional fields, overlay-mode and scratch-journal, which the host program's launch sets from --overlay-mode and --scratch-journal. Without them, the guest takes nvx::guest::root's defaults. The worker refuses a configuration that configures a scratch for an upper in RAM, and still refuses a GPT image at the layout stage. - The copy-up probe changes the mode of a lower file that the scenario names, and reports whether the file's upper copy holds its data, which a metadata-only copy-up leaves in the layer. The rename probe renames a lower directory that the scenario names, which the untrusted-lowers mode refuses with EXDEV. - The fill probe fills the upper through the root until a write fails, removes the file, and writes again. The devices probe lists the guest's disks, and the mounts probe also reports a tmpfs upper. The map probe finds the targets that it created on the overlay in the upper of either kind. - New scenarios cover EROFS over a scratch and over RAM in both modes, a full scratch, a full tmpfs, which the kernel sizes at half of the guest's memory, and a scratch with a journal that the launch forbids, which fails the assembly. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 30bab0d7-7c3e-4120-9add-6c9cc1dc9e7e
An assembly checked only the mount point of the upper that it mounts. A root whose teardown failed to unmount its upper, say a scratch that a process held open, left that upper mounted, and the next root, over a tmpfs, passed the check and later reported a clean teardown with the scratch still mounted; and the other way around. The assembly now also checks the other upper's mount point, which may be missing, as it does the aggregate share's, and refuses the runtime directory when a file system, or a bind mount of a directory, is mounted there. Tests cover both transitions after a failed unmount of the upper. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Copilot-Session: 30bab0d7-7c3e-4120-9add-6c9cc1dc9e7e
Enrique Saurez (esaurez)
force-pushed
the
esaurez/a3-3-gpt-tmpfs-overlay-20261021
branch
from
October 23, 2026 04:49
5e8ec70 to
28524ce
Compare
Enrique Saurez (esaurez)
deleted the
esaurez/a3-3-gpt-tmpfs-overlay-20261021
branch
October 23, 2026 05:39
This was referenced Oct 23, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.

A guest root's assembly took an EROFS image and an ext4 scratch, and mounted the overlay with one fixed option set. This PR lets the caller of
nvx::guest::root::assemblename what it assembles, with typed inputs, and the assembly probes nothing:Image::Erofs, with the UUID that the caller expects, if any. The enum is non-exhaustive, for the GPT format that the next PR of A3.3 adds.Upper::Scratch, a whole-device ext4 scratch, mountedrw,nosuid,nodev, with the caller'sJournalpolicy, which allows or forbids a journal, checked in the superblock's feature flags before the scratch is mounted; orUpper::Tmpfs, a tmpfs at the kernel's default size, mountednosuid,nodev, for a VM without a scratch device. The upper and work directories are separate directories of the upper, created 0755 whatever the umask.OverlayMode::TrustedLowers, the default, mounts the overlay withmetacopy=on,xino=on, andOverlayMode::UntrustedLowers, for lowers that may carry untrusted overlay xattrs, withmetacopy=off,redirect_dir=nofollow. Each mode has one option set: when the kernel refuses it, the assembly fails, with no fallback to another. The merged root is mountednosuid,nodev.The kernel has fallbacks of its own, which don't fail the mount: it mounts an overlay read-only when it can't create the work directory, and turns
metacopyoff andredirect_dirtonofollowwhen the upper can't hold the overlay's xattrs. So once the overlay is mounted, the assembly reads its options in the kernel's mount table, where an option at its module default doesn't show, and refuses and unmounts an overlay that isn't writable or lacks its mode's options.The assembly checks everything else before it mounts anything:
mount(2)passes a file system one page of options, and the kernel cuts longer ones short, which would drop the last lower directories or the mode's options. The assembly refuses options longer thanMOUNT_OPTIONS_LIMIT, 4,095 bytes, instead.The
Rootreports what it mounted: itsLowerlayers, itsUpperKind, and itsOverlayMode, whoseoptions()are the overlay's. Its teardown unmounts the live shares and mappings in reverse order, the overlay, the layers from the top down, and then the upper, and reports every unmount that fails.The runtime adapter's layered VMs (
tests/runtime-adapter) now assemble the EROFS image over a scratch or a tmpfs, in the overlay mode and with the journal policy that the host names, and probe copy-up, the rename of a lower directory, a full upper, and the guest's disks. Themapprobe finds the targets that it creates on the overlay in the upper of either kind.Part of #621. Refs #622 and #535.
Validation
On this branch's head, locally:
python scripts/nvx.py verify.nvx-rs, on Linux withNVX_BUNDLE=skip:cargo clippy --locked --all-targets --all-features -- -D warnings,cargo clippy --locked --lib --no-default-features -- -D warnings,cargo test --locked --all-features(1,103 passed),cargo test --locked(508 passed),cargo doc --locked --no-deps --all-featureswithRUSTDOCFLAGS=-D warnings,cargo +1.89 check --locked --all-targets --all-features, andcheck-nvx-rs's macOS check and bundling test;cargo fmt --all --checkon Windows.check-guest-binaries,build-guest-binariesandtest-guest-binaries(34 passed). The stripped guest binaries are byte-identical before and after this change, and so is the Alpine initramfs: 8,222,829 bytes, 19,707,392 uncompressed.tests/runtime-adapter: Clippy, build, tests and the MSRV check in the protocol-only, host and guest configurations on Linux, the guest configuration's MSRV check for the GNU target, and fmt, Clippy, build, tests and the MSRV check in the protocol-only and host configurations on Windows.cargo test --locked --features host --test vm_layered --test vm_restart -- --ignored: 12 layered scenarios, this PR's 10 and the 2 of the root's mappings, and 2 restart tests pass on MSHV and WHP. They show, per mode:TrustedLowersmounts withredirect_dir=on,uuid=on,xino=on,metacopy=onin effect, copies up a file's metadata alone, and renames a lower directory;UntrustedLowersshows onlyuuid=on, copies up the data too, and refuses the rename withEXDEV. The tmpfs upper isrw,nosuid,nodevwithmode=755and nosize=, with no scratch disk in the guest; a full scratch and a full tmpfs, half of the guest's memory, refuse writes withENOSPCuntil the file that filled them is gone.python scripts/nvx.py test-microvm --backend <mshv|whp> --processors 1withmanaged-exec-config,sandbox-lifecycle,sandbox-blocks,scratch-snapshot,snapshot-tiersandmanaged-lifecycle, on MSHV and WHP, with the Alpine initramfs built from this branch.MSHV ran as root and with
--cpu-profile host, since its test host can't present a built-in CPU profile. KVM and the full backend matrix are left to CI.