Skip to content

nvx-rs: enter guest executions with namespaces, a view, and an identity - #777

Merged
Pedro Henrique Penna (ppenna) merged 12 commits into
devfrom
esaurez/a3-4-identity-view-20261021
Oct 23, 2026
Merged

Pedro Henrique Penna (ppenna) merged 12 commits into
devfrom
esaurez/a3-4-identity-view-20261021

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Part of #621 (A3.4).

Adds nvx::guest::entry, through which any guest init or agent has a program that it spawns enter an execution, and runs root profiles of it in the runtime adapter's layered VMs, each with its negative tests.

The entry. An Entry takes typed inputs: the root, which is a mount, an identity, a hostname, a machine ID mode, the publication state, the time status's source, and the device nodes of /dev. Its steps are new launch stages of nvx::guest::spawn::Command, whose LaunchStage stays non-exhaustive, so the child still makes only system calls over storage prepared before it starts, and reports the stage that failed:

  • clone3 into new mount, PID, and UTS namespaces, so that the program is PID 1 of its namespace with no process between it and its caller; recursively private mounts, and the hostname;
  • the root's mounts made nosuid and nodev, then the root moved over /, never entered by chroot alone; a private procfs, a read-only sysfs, and a /dev tmpfs with only the allowed nodes, its own devpts, and an shm, so that every other mount is nodev; for user ID 0, the kernel's risky procfs entries and /sys/firmware read-only or masked;
  • the identity: the supplementary groups, the group and user IDs, and only capabilities of an allowlist of 13, which only user ID 0 may hold, and of which a workload with published listeners holds neither CAP_NET_BIND_SERVICE nor CAP_NET_RAW, applied to every set, with no_new_privs;
  • in a new session without a controlling terminal, every descriptor that the program doesn't inherit closed before it runs, and its working directory, / unless the command names another, entered through no magic link, so that neither a /proc link to a descriptor, a working directory, or a root, nor a symbolic link to one, places the program outside its root.

In existing-account mode, the entry reads the identity's passwd and group entries beneath the root's descriptor, with bounds, and requires one passwd entry with the requested primary group, an existing group, and an absolute home; numeric mode takes the IDs as given. Nothing creates an account. A generated machine ID is one random ID per boot, readable by every execution whatever the umask; image mode leaves the image's own. Entry::bind_runtime binds a generated ID over /etc/machine-id, and the time status at /run/nvx/time, read-only, once, through the mappings module's single bind.

A command can now also lead a new session, and hold its child at a barrier until the caller has placed it. The barrier is a connected socket pair whose drop shuts it down, so that dropping it cancels the launch at once, even while another child that has not run its program yet holds a copy of it. A child that waits at its barrier first closes every descriptor that it doesn't use, so that it holds no copy of another child's pipes or launch report while it waits. The child now joins its cgroup before its other stages, as an entry must while it still holds the privilege to.

Time status. The guest's time tools now keep their locks and private records in /run/nvx rather than in /run/nvx/time, so that the status directory that the entry binds holds only what any process may read: an execution of user ID 0 could otherwise hold the time daemon's locks.

Root profiles. The runtime adapter's guest agent enters its layered root with root profiles and checks, in its VMs: each allowlisted capability, a refused one, and user ID 0 with none; a refused write to /proc/sys as user ID 0; nested chroot and leaked-descriptor escapes, and working directories reached through magic links, which fail the launch; in both publication states, that a root workload can neither lower the unprivileged port threshold nor forge a listener announcement, from a privileged source port or through a raw socket; numeric mode; the image's own machine ID; and that the time status shows only its public files. NVX's own adapter keeps a non-root identity without capabilities.

Stripped sizes, built on Linux, from dev: nvx-managed-agent grows from 496,160 to 512,544 bytes, the Alpine initramfs from 8,223,214 to 8,231,379, the Ubuntu initramfs from 38,780,724 to 38,788,882, and the Ubuntu distro layer from 59,887,616 to 59,899,904; the other eight guest binaries keep their sizes.

Validation

At this branch's head, 9049f18, and at 991eb62, the head of #778, which adds NVX's managed agent switch on top of it:

  • Linux, MSHV host: in nvx-rs, with NVX_BUNDLE=skip, cargo fmt --all --check, cargo clippy --locked --all-targets --all-features -- -D warnings, cargo clippy --locked --lib --no-default-features -- -D warnings, cargo test --locked --all-features, cargo test --locked, the bundled-artifacts test, cargo doc with RUSTDOCFLAGS=-D warnings, cargo +1.89 check, and the macOS check; tests/runtime-adapter in its default, host, and musl guest configurations; the aci_edge_sandboxes checks; nvx.py check-guest-binaries, build-guest-binaries and test-guest-binaries, and test-agent-bundling; the quality gates (ruff, pyright, shellcheck, shfmt, the Python tests, the markdown links, and nvx.py verify). The nvx library tests also ran as root, with the ignored privileged mapping tests and the privileged entry test, which checks the working directories that magic links name too, and 20 times each as root and as a normal user.
  • MSHV, as root: nvx.py test-microvm --backend mshv with every default scenario but cpu-profile-fallback, the public managed sandbox acceptance, the Ubuntu and Azure Linux guest smoke tests, the Ubuntu sandbox steps, the aci_edge_sandboxes lifecycle test, and the runtime adapter's VM scenarios, including root_profiles_enter_the_root. As root, the caller-owned live-share step refuses its root-owned share, and five read-write mapping tests of aci_edge_sandboxes cannot write to root-owned host directories; the five pass as a normal user on WHP.
  • Windows, WHP: the nvx-rs, runtime adapter (default and host), and aci_edge_sandboxes checks, test-agent-bundling, and the Python tests; at 991eb62, the CI microVM job's steps, with the full default suite, and the runtime adapter's VM scenarios.

Left to CI: KVM; the debug kernel; the caller-owned live-share step, which checks a runner without the privilege to change its user ID, on Linux only; cpu-profile-fallback, since the MSHV host's CPU has no built-in profile that OpenVMM can present, so the local runs used --cpu-profile host; the musl MSRV check of the runtime adapter's guest agent; and the Azure Linux image, which the local runs took from the development release.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

Default ACLs can still prevent non-root mapping access, and single-component relative runtime sources fail to bind.

2 open issues
What changed in this PR

Adds reusable guest execution entry to nvx-rs, advancing #621’s layered-root support. Guest agents can launch programs with isolated namespaces, a restricted filesystem view, and a typed identity; the runtime adapter exercises these root profiles.

Changes:

  • Adds execution-entry APIs and launch stages, including session creation and cancellable barriers.
  • Separates private time records from public status and updates mapping creation permissions.
  • Adds root-profile probes, VM assertions, and supporting documentation.
File Description
tests/​runtime-adapter/​tests/​vm_layered.rs Adds root-profile VM assertions.
tests/​runtime-adapter/​src/​worker/​entries.rs Implements execution-entry probes.
tests/​runtime-adapter/​src/​worker.rs Dispatches entry probes.
tests/​runtime-adapter/​src/​lib.rs Defines helper command names.
tests/​runtime-adapter/​src/​bin/​runtime-adapter-guest.rs Handles entry helper commands.
tests/​runtime-adapter/​README.md Documents probes and VM coverage.
tests/​runtime-adapter/​Cargo.toml Enables required syscall features.
nvx-rs/​src/​lib.rs Documents the entry library.
nvx-rs/​src/​guest/​time/​system.rs Moves private records outside public status.
nvx-rs/​src/​guest/​sys/​spawn.rs Integrates entry stages and barriers.
nvx-rs/​src/​guest/​sys/​rooted.rs Adds umask-independent creation helpers.
nvx-rs/​src/​guest/​sys/​mod.rs Exposes new internal syscall helpers.
nvx-rs/​src/​guest/​sys/​fd.rs Adds permission and socket-pair helpers.
nvx-rs/​src/​guest/​sys/​entry.rs Implements namespace, mount, and identity operations.
nvx-rs/​src/​guest/​spawn.rs Exposes sessions, barriers, and execution entry.
nvx-rs/​src/​guest/​root/​tests.rs Checks share-target creation permissions.
nvx-rs/​src/​guest/​root/​kernel.rs Updates fake-kernel creation operations.
nvx-rs/​src/​guest/​root.rs Documents target permission behavior.
nvx-rs/​src/​guest/​mod.rs Exports the entry module.
nvx-rs/​src/​guest/​mappings/​tests.rs Checks mapping permissions under restrictive umasks.
nvx-rs/​src/​guest/​mappings/​privileged.rs Adds non-root mapping-access coverage.
nvx-rs/​src/​guest/​mappings/​kernel.rs Adds a dedicated file-creation operation.
nvx-rs/​src/​guest/​mappings.rs Uses updated creation helpers.
nvx-rs/​src/​guest/​managed_agent/​mappings.rs Tests mapping permissions under a restrictive umask.
nvx-rs/​src/​guest/​managed_agent/​launch.rs Documents revised cgroup placement order.
nvx-rs/​src/​guest/​entry.rs Adds typed entry configuration and runtime binds.
nvx-rs/​README.md Documents entry and expanded spawn APIs.

Review effort: Balanced

Comment thread nvx-rs/src/guest/entry.rs
Comment thread nvx-rs/src/guest/sys/rooted.rs Outdated
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a3-2-rooted-mappings-20261021 branch from 3711ac7 to a48857c Compare October 23, 2026 03:13
Base automatically changed from esaurez/a3-2-rooted-mappings-20261021 to dev October 23, 2026 04:16
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a3-4-identity-view-20261021 branch from db5ceba to 5dccd78 Compare October 23, 2026 05:33
Copilot AI balanced review requested due to automatic review settings October 23, 2026 05:33

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The new unsafe namespace, mount, descriptor, and capability transitions affect the sandbox security boundary and require final human review.

1 open issue
1 resolved since last review

Review effort: Balanced

@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a3-4-identity-view-20261021 branch from 5dccd78 to 298c828 Compare October 23, 2026 06:59
Copilot AI balanced review requested due to automatic review settings October 23, 2026 06:59

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Raw namespace creation, privileged mount operations, descriptor handling, and credential transitions establish a security boundary that warrants final human review.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

@esaurez
Enrique Saurez (esaurez) marked this pull request as ready for review October 23, 2026 07:08
Copilot AI balanced review requested due to automatic review settings October 23, 2026 09:06
@esaurez

Copy link
Copy Markdown
Contributor Author

A final automated review of 298c828 found two defects, which three new commits fix:

  1. An entry's working directory could leave its root (high). The entry's child only made its other descriptors close on exec, and its view's stages left their own descriptors open, such as the O_PATH ones below procfs. So a working directory such as /proc/self/fd/N, which chdir resolved before exec, could place the program outside its root, or in the writable /proc/sys below its read-only bind. 18f6034 closes every descriptor that the program doesn't inherit before the child enters its working directory, which it now opens with openat2 and RESOLVE_NO_MAGICLINKS and enters with fchdir. The privileged entry test, and the runtime adapter's root_profiles_enter_the_root VM scenario in 3b9d9af, check the link of a leaked descriptor, /proc/self/cwd and /proc/self/root, /proc/1/cwd, a symbolic link to a magic link, and the link of each descriptor up to 63.
  2. A child at its barrier held other children's descriptors (medium). Forked with copies of every descriptor of its caller, it kept other children's launch-report writers and output pipes until it was released, which could keep a concurrent execution from ending. d91097d has it close every descriptor that it doesn't use before it waits, and adds a regression test; two barrier tests no longer need processes of their own.

Both heads were validated again on Linux/MSHV and Windows/WHP, as the description lists; #778 is rebased onto 3b9d9af with its commit unchanged.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive namespace, mount, descriptor, and capability boundary warrants final human review despite no confirmed blocking defects.

No open issues 🎉

Review effort: Balanced

Copilot AI balanced review requested due to automatic review settings October 23, 2026 10:21

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive process, mount, and credential transitions warrant human review alongside completion of the remaining platform validation.

No open issues 🎉

Review effort: Balanced

@esaurez

Copy link
Copy Markdown
Contributor Author

A second final automated review, of 3b9d9af, found one more defect (medium), which 9049f18 fixes:

An entry's child could panic on a denied umask. The child of clone3 cleared and restored its umask with nix's umask, which panics when the system call fails, as a seccomp policy can make it. The panic neither reported a launch stage nor ended the child cleanly: with the old code, the launch even reported no failure. The child now makes the system call itself, and a failure to clear the umask is the proc stage's, or to restore it the devices stage's. A new test denies umask with a seccomp filter, in a process of its own, and checks that a simulated entry then fails its proc stage with EPERM; against the old code, the same test fails with nix's [BUG] umask returned invalid Mode panic.

Both heads were validated again on Linux/MSHV and Windows/WHP; #778 is rebased onto 9049f18, with its commit changed only in one re-export's context line.

Enrique Saurez and others added 6 commits October 23, 2026 11:52
A runtime may bind the time-status directory, /run/nvx/time, read-only
into a workload's root. Its lock files and private records are owned by
user ID 0, so a workload of user ID 0 could open them through that bind
even without capabilities, and hold the exclusive lock of state updates
or of the suppression record. That would block the time daemon's
discipline and violation reports, and a restore's repair.

Keep the state and suppression locks, the suppression record, and the
restore record in /run/nvx, beside the port-window lock, so that the
status directory holds only what any process may read: the published
state, the daemon's process identifier, and their temporary files.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
nvx::guest::entry lets any guest init or agent have a program that it
spawns enter an execution, from typed inputs: a root, which is a mount,
an identity, a hostname, a machine ID mode, a publication state, a time
status source, and the device nodes of /dev. The child starts with
clone3 in new mount, PID, and UTS namespaces, so that the program is
PID 1 of its namespace with no process between it and its caller. It
makes its mounts private, sets the hostname, and moves the root over /,
never entering it by chroot alone, after it makes the root's mounts
nosuid and nodev. It mounts a private procfs, a read-only sysfs, and a
/dev tmpfs with only the allowed nodes, its own devpts, and an shm, so
that every other mount is nodev. For user ID 0, the kernel's risky
procfs entries and /sys/firmware are read-only or masked. It then takes
its identity: the supplementary groups, the group and user IDs, and
only the capabilities of an allowlist of 13, which only user ID 0 may
hold, and of which a workload with published listeners holds neither
the binding nor the raw-socket one, applied to every set, with
no_new_privs. Every other descriptor of its caller closes as its
program runs, and it leads a new session without a controlling
terminal.

In existing-account mode, the entry reads the identity's passwd and
group entries below the root's descriptor, with bounds, and requires one
passwd entry with the requested primary group, an existing group, and
an absolute home; numeric mode takes the IDs as given. Nothing creates
an account. A generated machine ID is one random ID that the caller
writes once for the boot, readable by every execution whatever the
umask, and the image's own is left alone. Entry::bind_runtime binds a
generated ID over /etc/machine-id, and the time status at /run/nvx/time,
read-only into the root once, through the mappings module's single bind,
in the caller's mount namespace, of which each execution's starts as a
copy, so that every later execution sees them; a symbolic link at
/etc/machine-id gives way to an empty file first.

Each step is a launch stage of nvx::guest::spawn::Command, whose child
still makes only system calls over storage prepared before it starts,
and reports the stage that failed. A command can now also lead a new
session, and hold its child at a barrier until the caller has placed
it. The barrier is a connected socket pair, so that dropping it cancels
the launch at once by shutting down the caller's end, even while other
children that have not run their programs yet hold copies of it. The
child now joins its cgroup before its other stages, as an entry must
while it still holds the privilege to.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
The adapter's worker gains an entries probe, which enters its assembled
root through nvx::guest::entry and nvx::guest::spawn, as an adapter's
root profiles do. The worker copies its own executable into the root
and runs the executions in a helper that enters a mount namespace of its
own first, as NVX's managed agent does, so that the runtime's binds of a
generated machine ID and of the time status end with it before the
worker tears the root down. Each execution joins the workload cgroup,
receives a descriptor of the guest's own root that its caller leaks
without close-on-exec, and runs the staged executable, which reports
from inside the root what the execution sees and may do.

The root_profiles_enter_the_root scenario checks that user ID 0 with
each allowed capability alone holds only that capability, and that only
its own attempt succeeds, that root with none and a numeric identity
without an account may do none of it, and that the entry refuses the
capabilities outside its allowlist, the binding and raw-socket ones
while listeners are published, capabilities for another user ID, and an
identity without an account in existing-account mode. Every execution
is PID 1 of its namespaces with only its standard streams, cannot write
/proc/sys or lower the unprivileged port start, and stays in its root
after a nested chroot escape. With published listeners, which both
publication modes reach the entry as, a root workload can bind no
privileged source port and open no raw socket, so it cannot forge a
listener announcement. The executions see the root's own machine ID,
then a generated one and the time status that the guest's time daemon
publishes, read-only, in which a root workload finds no lock of the
daemon's, and which leave no mount in the worker's namespace.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
A runtime file that a relative path of one component names, such as a
machine ID that MachineId::generate made at "machine-id", has an empty
parent, which Entry::bind_runtime opened as the source root of its bind,
and failed with ENOENT. The source root of such a file is now the
current directory.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
A child that waits at its barrier holds a copy of every descriptor of
the test process until it runs its program, such as the end of another
test's pipe, which then stays open for as long. Two tests held a child
there for tens of milliseconds while other tests ran. They now run in
processes of their own, as the test of an ended child's barrier does.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
A child that waits at its barrier was forked with copies of every
descriptor of its caller, close-on-exec ones included, and kept them
until it ran its program, which it does only once released. Among them
are the launch report's writer and the output pipes of any child that
the caller started or prepared meanwhile, so an execution that waits
for its pipes to hang up could not end while another child waited.

Before it waits, the child now closes every descriptor that it does not
use, with close_range over the gaps between the sorted numbers that
PreparedChild::new records: its stream sources, root, cgroup file,
barrier reader, report writer, and inherited descriptors. Its copy of
the barrier's writer is among them, which an explicit close dropped
before. So the program of a child with a barrier no longer receives
descriptors that its caller opened without close-on-exec, which an
entry's program never received either.

The barrier tests no longer need processes of their own, and a new test
checks that another child's pipes hang up while a child waits.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
Enrique Saurez and others added 6 commits October 23, 2026 11:52
An entry's child only made its other descriptors close on exec, so the
descriptors that its caller leaked and those that its view's stages
opened, such as the O_PATH ones below procfs, stayed open until its
program ran. Its working directory, which chdir resolved before then,
could name one of them through /proc/self/fd, or a working directory or
root through another magic link, and so place the program outside its
root, or in the writable /proc/sys below its read-only bind, where the
program stayed after exec.

The child now closes every descriptor that it does not use when it
places its descriptors, rather than marking it, and, once its stages
have run, every descriptor but its program's and its launch report's.
It then opens its working directory, / unless its command names
another, with openat2 and RESOLVE_NO_MAGICLINKS, and enters it with
fchdir, so that neither a magic link nor a symbolic link to one
resolves. Either failure is the working-directory stage's.

The privileged entry test checks that / and an ordinary directory are
entered, and that the links of a leaked descriptor, of each descriptor
up to 63, of the child's working directory and root, and of the working
directory of its namespace's first process, and a symbolic link to one
of them, each fail the stage before the program runs.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
The adapter's entries probe now reports each execution's working
directory, the top of the root unless the execution names another, and
runs working-directory cases with the root profile of every allowed
capability: its own case directory, which it enters, and, through magic
links, the directory of the descriptor that its caller leaked, its own
working directory and root, the working directory of its namespace's
first process, a symbolic link to its working directory, and the link
of each descriptor up to 63, whose distinct outcomes it reports.

The layered VM scenario checks that every execution without a working
directory runs at the top of its root, that the case directory is
entered, and that each magic link fails the working-directory stage
before the program runs: with ENOENT for the leaked descriptor, which
the child closed, and ELOOP otherwise.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
An entry's child cleared its umask before it mounted its view, and
restored it after, with nix's umask, which panics when the system call
fails, as a seccomp policy can make it. The child that clone3 starts
must not panic: the panic would neither report the failed stage nor end
the child cleanly, and could unwind into its caller's code.

The child now makes the system call itself. A failure to clear the
umask is the proc stage's, and one to restore it the devices stage's,
so the launch reports it and runs nothing.

A test denies umask with a seccomp filter, in a process of its own,
and checks that a simulated entry then fails its proc stage with EPERM.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot AI balanced review requested due to automatic review settings October 23, 2026 14:58
@ppenna
Pedro Henrique Penna (ppenna) force-pushed the esaurez/a3-4-identity-view-20261021 branch from 9049f18 to d2bbbc4 Compare October 23, 2026 14:58

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive namespace, mount, capability, and post-clone execution boundary needs final human review alongside privileged CI results.

No open issues 🎉

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) merged commit 230d41c into dev Oct 23, 2026
44 checks passed
@ppenna
Pedro Henrique Penna (ppenna) deleted the esaurez/a3-4-identity-view-20261021 branch October 23, 2026 15:31
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants