Skip to content

sandbox: deliver managed live shares as a host mapping table - #794

Merged
Enrique Saurez (esaurez) merged 2 commits into
devfrom
esaurez/a4-pr4-export-composition-20261021-maps
Oct 23, 2026
Merged

Enrique Saurez (esaurez) merged 2 commits into
devfrom
esaurez/a4-pr4-export-composition-20261021-maps

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Oct 23, 2026 •

Copy link
Copy Markdown
Contributor

A managed layered sandbox now receives its live shares as ordered MAPS requests once its guest agent is ready, as the blockless root receives its host directories, instead of as boot tokens. This is the second part of A4's shared export composition: it builds on the export planner of #765 and the descriptor-rooted mapper of #774.

What changed

The managed agent maps host paths in sandbox mode.

  • When the kernel command line announces mappings (nvx_maps=), the sandbox-mode agent applies the ordered MAPS batches through nvx::guest::mappings beneath the container's assembled root, in the mount namespace of its own of which each execution's starts as a copy, from the export mounted at /run/nvx/hostfs/root, with the direct mode's ordering and latching. It refuses the targets that the container runtime keeps (Reserved::container_root) and creates a missing target only on the overlay, never inside another mapping.
  • It admits no workload until the whole table is mounted, and a failed mapping keeps it refusing them.
  • When its session ends, it unmounts the mappings in reverse order before it returns to the init agent, and reports each one that stays mounted (stage=host-unmapping).
  • The sandbox agent advertises the host mapping table (feature bit 6).

The init agent mounts the export.

  • nvx-init-agent's managed branch mounts the virtio-fs export at /run/nvx/hostfs/root, in a directory that only root can enter, with nosuid,nodev, and unmounts it after the agent's mappings and before the overlay.
  • That branch refuses nvx_share= tokens and any other export location. The one-shot lifecycle keeps its tokens, as do /init's plain path (nvx-hostmount) and the virtio restore probe.

start delivers the plan's mapping table.

  • SandboxPlan::openvmm_arguments and kernel_command_line, and so PreparedLaunch::new, render the managed lifecycle: every share as a child of one aggregate at GUEST_EXPORT, and nvx_maps=COUNT. SandboxPlan::mapping_table gives the ordered table. The one-shot renderings, with today's tokens, are crate-private and built only with the cli feature; the CLI's one-shot sandbox run prepares its launch with them.
  • After the guest answers and has the features that start requires, start sends the table in as many MAPS requests as it takes, before the launch commits. A mapping that the guest refuses fails start: the guest first shuts down as stop has it do, unmounting what it mapped, the export, the overlay, and the scratch disk, and OpenVMM is terminated only if the guest does not shut down within --timeout. The sandbox stays provisioned and stopped. A sandbox without shares maps nothing.
  • GuestFeatures::LAYERED_SANDBOX, which start requires, adds the host mapping table to per-execution environments.
  • The fake OpenVMM serves the managed lifecycle's export and mapping table and records the table that it receives, which the CLI goldens check.

Behavior changes

  • The kernel command line no longer bounds the number of a managed sandbox's shares; the aggregate's 256 children do. A single managed share is a child of the aggregate too, rather than a directory export.
  • With --mount-owner caller, the guest agent reaches the shares as guest root, which OpenVMM squashes to the shares' owner, before any workload runs. A managed sandbox whose OpenVMM cannot assume that identity therefore fails to start, rather than start with shares that refuse every operation. The microVM workflow's caller-owned test, whose runner grants OpenVMM neither CAP_SETUID nor CAP_SETGID, now checks that the managed start fails with OpenVMM's EPERM and the agent's host-mappings error; its one-shot part is unchanged.
  • The CLI goldens change only in the managed share lifecycles' start arguments and kernel command lines, and in the mapping table that the fake OpenVMM records.

Part of #622
Refs #621

Validation

At this head, on dev at 00d21385, on Windows 11 (WHP), and on Linux with MSHV on a VM host, where builds and checks ran as a non-root user and VMs as root, which alone may open the hypervisor device there:

  • python scripts/nvx.py verify
  • In nvx-rs, with NVX_BUNDLE=skip: cargo fmt --all --check; cargo clippy --locked --all-targets --all-features -- -D warnings; cargo clippy --locked --lib --no-default-features -- -D warnings; cargo test --locked --all-features; cargo test --locked; cargo build --locked --lib --no-default-features and cargo test --locked --lib --no-default-features; cargo doc --locked --no-deps --all-features with RUSTDOCFLAGS=-D warnings; cargo +1.89 check --locked --all-targets --all-features; on Linux also the aarch64-apple-darwin check, the agent bundling test, and the guest binaries' check and build.
  • aci_edge_sandboxes: the steps of check-aci-edge-sandboxes (fmt, both clippy configurations, both test configurations, docs, MSRV; macOS check on Linux).
  • tests/runtime-adapter: the protocol-only and host-only configurations on both, and the guest-only (musl) configuration on Linux, whose MSRV check used the GNU target there.
  • check-quality's Python and shell checks: ruff check, ruff format --check, pyright --pythonplatform Linux and Windows, shellcheck and shfmt with the pinned images, and the Markdown link check. validate-nvx's compileall, unittest list, and help commands on both; on Windows, the test_setup_azure tests of the Linux setup scripts fail on this host with and without this change.
  • Real VMs on WHP, and on MSHV (OpenVMM with --cpu-profile host on this VM host), with the artifacts of v0.1.0-dev.a37991311d1a, the newest development release in this branch's first-parent history, that of a3799131, whose OpenVMM is 79a501b6, the gitlink of both, and on both the Alpine initramfs built from this branch on Linux (nvx.py build-initramfs --guest alpine):
    • the microVM workflow's Ubuntu sandbox share steps: one-shot and managed shares, concurrent read-write, read-only and file shares, and the access policy;
    • nvx.py test-microvm with filesystem-shares, filesystem-policy, filesystem-comma-paths, filesystem-owner, denied-filesystem-paths, filesystem-snapshot, managed-lifecycle, managed-exec-config, and sandbox-lifecycle;
    • nvx.py test-aci-edge-sandboxes: all pass on WHP; on MSHV, as root, the five tests that write through read-write mappings fail with Permission denied with and without this change, and the other 14 pass;
    • the runtime adapter's VM scenarios (cargo test --features host -- --ignored);
    • the virtio restore probe with a virtio-fs share (nvx.py benchmark --suite device-restore-profile --restore-devices virtiofs --warmups 0 --runs 1), in both activation modes;
    • a managed sandbox whose second mapping crosses a symbolic link of the Ubuntu layer (/bin/...): start fails with the guest's ELOOP mapping error after the guest powers off with status 0, the scratch disk is left clean (no needs_recovery), exec reports the sandbox stopped, and deprovision succeeds.
  • Left to CI: KVM, the caller-owned share step (it needs a non-root user on Linux), the full microVM matrix, and benchmarks.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Privileged mappings, workload admission, and failed-start teardown cross the host–guest boundary and warrant human review with the remaining platform validation.

No open issues 🎉

What changed in this PR

This PR moves managed layered sandboxes from boot-token live shares to ordered host mapping tables, building on the shared export planner and descriptor-rooted mapper. One-shot launches retain their existing boot-token behavior.

Changes:

  • Export managed shares through one aggregate and deliver MAPS requests after guest readiness, before completing start.
  • Apply mappings beneath the container root, block workloads until all mappings succeed, and unmount them during shutdown.
  • Update feature negotiation, failure-path tests, fake OpenVMM behavior, and documentation.
File Description
scripts/​test_nvx_tools.py Tests managed export bootstrap validation and permissions.
nvx-rs/​tests/​cli.rs Tests required features and failed-mapping cleanup.
nvx-rs/​src/​testing/​fake_openvmm.rs Models lifecycle-specific share delivery.
nvx-rs/​src/​protocol/​tests.rs Checks updated mapping-table feature sets.
nvx-rs/​src/​protocol/​features.rs Advertises and requires sandbox mapping-table support.
nvx-rs/​src/​host/​sandbox/​tests.rs Separates managed and one-shot rendering assertions.
nvx-rs/​src/​host/​sandbox/​prepared.rs Prepares launches with lifecycle-specific share delivery.
nvx-rs/​src/​host/​sandbox/​plan/​tests.rs Tests mapping tables, exports, and command-line budgets.
nvx-rs/​src/​host/​sandbox/​plan.rs Renders managed exports and exposes ordered mappings.
nvx-rs/​src/​host/​sandbox/​options/​tests.rs Keeps one-shot option tests on boot-token rendering.
nvx-rs/​src/​host/​sandbox/​mount.rs Generates managed aggregate bootstrap fragments.
nvx-rs/​src/​host/​sandbox/​golden.rs Preserves one-shot Python parity checks.
nvx-rs/​src/​host/​cli/​session.rs Sends mapping batches and handles guest responses.
nvx-rs/​src/​host/​cli/​sandbox/​mod.rs Keeps one-shot runs on boot-token preparation.
nvx-rs/​src/​host/​cli/​sandbox/​lifecycle.rs Delivers mappings before commit and handles failed starts.
nvx-rs/​src/​guest/​managed_agent/​mod.rs Prepares mappings in both modes and tears down sandbox mappings.
nvx-rs/​src/​guest/​managed_agent/​mappings.rs Maps beneath sandbox roots and reports teardown failures.
nvx-rs/​src/​guest/​managed_agent/​dispatch.rs Tests sandbox mapping order, restrictions, and workload admission.
nvx-rs/​src/​guest/​managed_agent/​diagnostics.rs Adds host-unmapping failure diagnostics.
nvx-rs/​README.md Documents managed mapping delivery and feature requirements.
guest/​common/​nvx-init-agent Mounts the managed export separately from one-shot shares.
doc/​usage.md Explains lifecycle-specific share delivery and limits.
doc/​run.md Documents mapping admission, shutdown, and caller ownership behavior.
doc/​design/​5.2-readiness-and-runtime-control.md Updates the managed start sequence.
doc/​design/​4.4-host-guest-control-interface.md Documents sandbox mapping-table support.
doc/​design/​4.1-guest-filesystem-and-image-model.md Describes managed export placement and root mapping.
.github/​workflows/​run-nvx-microvm-tests.yml Expects caller-owned managed starts to fail without required privileges.

Review effort: Balanced

@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a4-pr4-export-composition-20261021 branch from 3fd464f to 1f60357 Compare October 23, 2026 11:08
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a4-pr4-export-composition-20261021-maps branch from 551cea5 to 748a37e Compare October 23, 2026 11:30
@esaurez
Enrique Saurez (esaurez) requested a balanced review from Copilot October 23, 2026 11:30

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive mount admission and failed-start teardown warrant human review, with KVM and full-matrix validation left to CI in the reported results.

No open issues 🎉

Review effort: Balanced

@ppenna
Pedro Henrique Penna (ppenna) force-pushed the esaurez/a4-pr4-export-composition-20261021 branch from 1f60357 to cda9371 Compare October 23, 2026 13:36
Base automatically changed from esaurez/a4-pr4-export-composition-20261021 to dev October 23, 2026 13:55
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a4-pr4-export-composition-20261021-maps branch from 748a37e to 87aa251 Compare October 23, 2026 14:55
Copilot AI balanced review requested due to automatic review settings October 23, 2026 14:55

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive mount isolation and cross-platform failed-start teardown warrant human review, including the remaining CI validation.

No open issues 🎉

Review effort: Balanced

@esaurez
Enrique Saurez (esaurez) marked this pull request as ready for review October 23, 2026 15:03
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a4-pr4-export-composition-20261021-maps branch from 87aa251 to e1ae2ab Compare October 23, 2026 15:56
Copilot AI balanced review requested due to automatic review settings October 23, 2026 15:56

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Privileged mount handling and coordinated host–guest shutdown require human review alongside the backend validation left to CI.

No open issues 🎉

Review effort: Balanced

Enrique Saurez and others added 2 commits October 23, 2026 09:44
The managed agent's sandbox mode gains the host mapping table of its
direct mode. When the kernel command line announces mappings with
nvx_maps=, the agent applies the ordered MAPS batches through the
descriptor-rooted mapper beneath the container's assembled root, from
the export that the guest mounts at /run/nvx/hostfs/root, with the
direct mode's ordering and latching. It refuses the targets that the
container runtime keeps, admits no workload until the whole table is
mounted, and keeps refusing workloads after a failed mapping.

When its session ends, the agent unmounts the mappings in reverse order
before it returns to the init agent, which then unmounts the root, and
reports each mapping that stays mounted. No launch announces mappings
to a sandbox yet, so the agent does not advertise the feature in this
mode until one does.

Part of #622

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e4de0d1f-0989-4804-ab6b-3c2891554406
A managed layered sandbox now receives its live shares as the blockless
root receives its host directories. OpenVMM exports every share as a
child of one aggregate, which the init agent mounts at
/run/nvx/hostfs/root, in a directory that only root can enter, and the
kernel command line announces only the number of mappings. Once the
guest agent answers, start sends the plan's ordered mapping table as
MAPS requests before the launch commits. A mapping that the guest
refuses fails the start, after the guest shuts down as a stop has it do,
unmounting what it mapped, the export, the overlay, and the scratch
disk, and OpenVMM is terminated only if the guest does not shut down.
start requires the host mapping table, which the sandbox agent now
advertises, and the kernel command line no longer bounds the number of
a managed sandbox's shares.

The one-shot lifecycle keeps its nvx_share= boot tokens, as do /init's
plain path and the restore probe, and the init agent's managed branch
refuses them. SandboxPlan and PreparedLaunch render the managed
lifecycle, and their one-shot renderings, with which the CLI's run
prepares its launch, become crate-private.

With --mount-owner caller, the guest agent reaches the shares as guest
root, which OpenVMM squashes to their owner, before any workload runs,
so a managed sandbox whose OpenVMM cannot assume that identity no longer
starts. The microVM workflow's caller-owned test, whose runner grants
OpenVMM neither CAP_SETUID nor CAP_SETGID, checks that now.

Part of #622

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: e4de0d1f-0989-4804-ab6b-3c2891554406
Copilot AI balanced review requested due to automatic review settings October 23, 2026 17:04
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a4-pr4-export-composition-20261021-maps branch from e1ae2ab to a3edbd3 Compare October 23, 2026 17:04

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive mount isolation, workload admission, and failed-start cleanup warrant human review alongside the outstanding cross-backend CI validation.

No open issues 🎉

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants