Skip to content

nvx-rs: enter the layered root's managed executions through the entry - #778

Draft
Enrique Saurez (esaurez) wants to merge 1 commit into
esaurez/a3-4-identity-view-20261021from
esaurez/a3-4-identity-view-20261021-agent
Draft

Enrique Saurez (esaurez) wants to merge 1 commit into
esaurez/a3-4-identity-view-20261021from
esaurez/a3-4-identity-view-20261021-agent

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #777; this PR retargets to dev when it merges.

Part of #621 (A3.4). Fixes #566.

In sandbox mode, NVX's managed agent now has each workload enter the layered root through nvx::guest::entry, as one launch of nvx::guest::spawn, instead of running the container launch and entry helpers, setpriv, and its own execution helper behind their FIFO barrier.

  • Each execution starts as PID 1 of new mount, PID, and UTS namespaces, with the hostname that the agent was given, the entry's private view, and the root moved over /, as the configured non-root identity with no capabilities, whose existing account the entry checks before each launch.
  • When it starts, the agent generates the workload's machine ID, and binds it and the time status read-only into the layered root once, through the entry, in a mount namespace of its own, of which each execution's starts as a copy, so that the binds end with the agent, before the init agent unmounts the root.
  • The child waits at the spawn's barrier until the agent has placed it in the container cgroup, as the FIFO barrier did; a failed placement ends it before its program runs.
  • The agent builds the workload's environment itself, as the execution helper did, and reports a working directory or program that the workload cannot use on its standard error with status 125, as the helper did. Any other stage that fails, the entry's among them, refuses the launch, after the agent names the stage and its error on the workload's standard error.
  • The workload, rather than an unshare parent, now holds the first process of the container cgroup's limit, so the init agent gives that process back for the managed lifecycle, which no longer runs the launcher, and leaves the machine ID to the agent.

The one-shot lifecycle keeps the shell helpers. What only the managed lifecycle used goes: the enter helper's execution-helper hand-off, the execution helper's working directory and its descriptor variable, and the agent's FIFO barrier, SIGPIPE guard, and directory calls.

The layered root creates no accounts, so its agent no longer announces workload account creation (FEATURES bit 2), and the architecture guide says so; the blockless root still announces and performs it. The sandbox probe now also checks the machine ID, that device nodes cannot be created, that the workload has no port I/O, and, for a managed execution, that every mount but /dev and its devpts is nodev.

Stripped sizes in bytes, built on Linux:

Artifact dev #777 This PR
nvx-managed-agent 496,160 512,544 541,216
Alpine initramfs 8,223,214 8,231,379 8,243,865
Ubuntu initramfs 38,780,724 38,788,882 38,801,576
Ubuntu distro layer 59,887,616 59,899,904 59,916,288

The other eight guest binaries keep their sizes. The one-shot lifecycle still uses the shell helpers, so removing what only the managed lifecycle used does not offset the Rust code.

Validation

At this branch's head, 991eb62:

  • Linux, MSHV host: in nvx-rs, with NVX_BUNDLE=skip, cargo fmt --all --check, cargo clippy --locked --all-targets --all-features -- -D warnings, cargo clippy --locked --lib --no-default-features -- -D warnings, cargo test --locked --all-features, cargo test --locked, the bundled-artifacts test, cargo doc with RUSTDOCFLAGS=-D warnings, cargo +1.89 check, and the macOS check; tests/runtime-adapter in its default, host, and musl guest configurations; the aci_edge_sandboxes checks; nvx.py check-guest-binaries, build-guest-binaries and test-guest-binaries, and test-agent-bundling; the quality gates (ruff, pyright, shellcheck, shfmt, the Python tests, the markdown links, and nvx.py verify). The nvx library tests also ran as root.
  • MSHV, as root: nvx.py test-microvm --backend mshv with every default scenario but cpu-profile-fallback, the public managed sandbox acceptance (managed-exec-config, sandbox-lifecycle, and sandbox-proxy-only), the Ubuntu and Azure Linux guest smoke tests, the Ubuntu sandbox layer, live-share, managed live-share, concurrent share, and share-policy steps, the aci_edge_sandboxes lifecycle test, and the runtime adapter's VM scenarios. As root, the caller-owned live-share step refuses its root-owned share, and five read-write mapping tests of aci_edge_sandboxes cannot write to root-owned host directories; the five pass as a normal user on WHP.
  • Windows, WHP: the nvx-rs, runtime adapter (default and host), and aci_edge_sandboxes checks, test-agent-bundling, and the Python tests; every step of the CI microVM job, with the full default suite; and the runtime adapter's VM scenarios.

Left to CI: KVM; the debug kernel; the caller-owned live-share step, which checks a runner without the privilege to change its user ID, on Linux only; cpu-profile-fallback, since the MSHV host's CPU has no built-in profile that OpenVMM can present, so the local runs used --cpu-profile host; the musl MSRV check of the runtime adapter's guest agent; and the Azure Linux image, which the local runs took from the development release.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The privileged namespace, root, and identity migration warrants human security review alongside resolution of the retained findings.

2 open issues
What changed in this PR

This PR moves managed layered-sandbox executions onto nvx::guest::entry and spawn, advancing #621 while retaining shell helpers for one-shot workloads. It also stops advertising layered-root account creation, addressing #566.

Changes:

  • Launch managed workloads directly with isolated namespaces, runtime bindings, and a cgroup placement barrier.
  • Remove managed-only helper machinery and adjust process limits.
  • Expand sandbox probes and tests, and document the lifecycle differences.
File Description
scripts/​test_nvx_tools.py Tests process-limit adjustment and expanded security probes.
scripts/​test_microvm_tests.py Updates managed-entry expectations.
scripts/​nvx_tools/​sandbox_lifecycle_tests.py Requires the managed-entry probe marker.
scripts/​nvx_tools/​managed_exec_tests.py Removes the retired descriptor-variable expectation.
nvx-rs/​tests/​guest_tools.rs Updates direct-helper configuration tests.
nvx-rs/​src/​protocol/​tests.rs Checks corrected sandbox feature bits.
nvx-rs/​src/​protocol/​features.rs Stops advertising layered-root account creation.
nvx-rs/​src/​guest/​sys/​spawn.rs Adds launch-plan test inspection.
nvx-rs/​src/​guest/​sys/​process.rs Removes obsolete SIGPIPE guarding.
nvx-rs/​src/​guest/​sys/​mod.rs Updates syscall exports and test-only helpers.
nvx-rs/​src/​guest/​sys/​filesystem.rs Removes unused directory wrappers.
nvx-rs/​src/​guest/​sys/​fd.rs Restricts FIFO creation to tests.
nvx-rs/​src/​guest/​sys/​entry.rs Extends entry-stage simulation for tests.
nvx-rs/​src/​guest/​managed_agent/​seam.rs Adds entry preparation and runtime-binding operations.
nvx-rs/​src/​guest/​managed_agent/​mod.rs Initializes sandbox entry and machine identity.
nvx-rs/​src/​guest/​managed_agent/​loopback.rs Updates test configuration.
nvx-rs/​src/​guest/​managed_agent/​launch.rs Separates direct and sandbox launch construction.
nvx-rs/​src/​guest/​managed_agent/​helper.rs Limits the helper to direct-mode environment setup.
nvx-rs/​src/​guest/​managed_agent/​exec.rs Integrates entry launches, barriers, and failure handling.
nvx-rs/​src/​guest/​managed_agent/​exec_config.rs Removes working-directory fields from helper configuration.
nvx-rs/​src/​guest/​managed_agent/​environment.rs Removes obsolete environment unsetting.
nvx-rs/​src/​guest/​managed_agent/​dispatch.rs Tests sandbox placement and launch failures.
nvx-rs/​src/​guest/​entry/​tests.rs Adapts stage-failure simulation tests.
nvx-rs/​src/​guest/​entry.rs Adds simulated prepared entries for tests.
nvx-rs/​src/​bin/​nvx-managed-agent.rs Clarifies direct-helper documentation.
nvx-rs/​README.md Documents entry integration and feature changes.
guest/​common/​nvx-sandbox-smoke Adds device, machine-ID, port-I/O, and mount checks.
guest/​common/​nvx-init-agent Adjusts managed process limits and machine-ID ownership.
guest/​alpine/​nvx-container-enter Removes managed-helper injection and handoff.
doc/​run.md Describes managed isolation and expanded probes.
doc/​project-structure.md Updates managed-agent responsibilities.
doc/​guests/​ubuntu-guest.md Distinguishes managed and one-shot launch paths.
doc/​design/​4.4-host-guest-control-interface.md Corrects layered-root feature documentation.
doc/​design/​4.3-isolation-and-resource-model.md Documents entry isolation and account validation.
doc/​design/​4.2-guest-agents-and-workload-model.md Explains helper-free managed sandbox execution.
doc/​design/​1.3-system-components-and-responsibilities.md Limits account creation to the blockless root.

Review effort: Balanced

Comment thread nvx-rs/src/guest/managed_agent/exec.rs
Comment thread nvx-rs/src/guest/managed_agent/exec.rs
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a3-4-identity-view-20261021-agent branch from 9ef1b7e to d18d4fa Compare October 23, 2026 05:33
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a3-4-identity-view-20261021 branch from db5ceba to 5dccd78 Compare October 23, 2026 05:33
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a3-4-identity-view-20261021-agent branch from d18d4fa to 2b2d163 Compare October 23, 2026 06:59
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a3-4-identity-view-20261021 branch from 5dccd78 to 298c828 Compare October 23, 2026 06:59
@esaurez
Enrique Saurez (esaurez) requested a balanced review from Copilot October 23, 2026 07:01

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive changes to namespaces, credentials, mounts, and process supervision warrant final human review alongside the remaining CI validation.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive migration of namespace, credential, mount, and cgroup handling warrants final human review alongside the remaining CI validation.

No open issues 🎉

Review effort: Balanced

In sandbox mode, NVX's managed agent now has each workload enter the
layered root through nvx::guest::entry, as one launch of
nvx::guest::spawn, instead of running the container launch and entry
helpers, setpriv, and its own execution helper behind their FIFO
barrier. Each execution starts as PID 1 of new mount, PID, and UTS
namespaces, with the hostname that the agent was given, the private
view of the entry, and the root moved over /, as the configured
non-root identity with no capabilities, whose existing account the
entry checks before each launch. When it starts, the agent generates
the workload's machine ID, and binds it and the time status read-only
into the layered root once, through the entry, in a mount namespace of
its own, of which each execution's starts as a copy, so that the binds
end with the agent, before the init agent unmounts the root.

The child waits at the spawn's barrier until the agent has placed it in
the container cgroup, as the FIFO barrier did; a failed placement ends
it before its program runs. The agent builds the workload's environment
itself, as the execution helper did: its own with the workload's HOME,
USER, LOGNAME, and PWD, then the request's entries, which replace it
unless they layer over it. A working directory or program that the
workload cannot use is reported on its standard error with status 125,
as the helper reported it. Any other stage that fails, the entry's
among them, refuses the launch, after the agent names the stage and its
error on the workload's standard error.

The workload, rather than an unshare parent, now holds the first
process of the container cgroup's limit, so the init agent gives that
process back for the managed lifecycle, which no longer runs the
launcher, and leaves the machine ID to the agent. The one-shot lifecycle
keeps the shell helpers. What only the managed lifecycle used goes: the
enter helper's execution-helper hand-off, the execution helper's working
directory and its descriptor variable, and the agent's FIFO barrier,
SIGPIPE guard, and directory calls.

The layered root creates no accounts, so its agent no longer announces
workload account creation; the blockless root still does. The sandbox
probe now also checks the machine ID, that device nodes cannot be
created, that the workload has no port I/O, and, for a managed
execution, that every mount but /dev and its devpts is nodev.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e06da46b-cc03-40f6-84e4-199397657333

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

The security-sensitive changes to namespace entry, credentials, and process containment warrant final human review, with KVM validation left to CI.

No open issues 🎉

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants