Skip to content

nvx-rs: hold the guest port permission for one transaction at a time - #662

Merged
Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
esaurez/a1-guest-g5-time-20261020
Oct 21, 2026
Merged

Pedro Henrique Penna (ppenna) merged 2 commits into
devfrom
esaurez/a1-guest-g5-time-20261020

Conversation

@esaurez

@esaurez Enrique Saurez (esaurez) commented Oct 20, 2026 •

Copy link
Copy Markdown
Contributor

nvx-time took the permission for OpenVMM's I/O ports when a command started and held it until the command ended. Linux keeps that permission in the thread's I/O bitmap, which the processes the thread starts inherit and keep across exec, so the time daemon, its restore workers, and anything they started could reach the ports too.

This change scopes the permission to each port transaction, as the time-and-reseed item of #619 asks, and changes nothing else.

  • sys::port: a PortAccess takes the permission on the calling thread for one transaction, refuses a second one on that thread, rolls back a partial grant, and revokes the permission when the transaction ends. A failed revocation is recorded for the thread, and check_released keeps the thread from starting a thread or a process until a later transaction revokes it.
  • nvx-time: the runtime no longer holds the ports. Each violation event writes the debug console in a transaction of its own. A best-of-N sample, the capture request with its restore-packet reads, the generation-ID read, and the restore acknowledgement are one transaction each, and each ends before an event is reported or a file is written. The capture parses the packet's header and body in its transaction, because the header sizes the body's reads, and checks the packet after it.
  • A capture that finishes a restore takes the permission for its acknowledgement first. When it cannot, it leaves the restore unfinished and prints 1, so nvx-snapshot finishes it with restore-finish --ack, which acknowledges it or fails closed.
  • Every process that the time tools start goes through fork_process, which first checks that the thread holds none of the permission. When a revocation failed, the boot checks run without their threads, the daemon stops before it starts anything, and a command fails at its end.
  • A failed revocation that nothing else reports, after an event's write to the debug console or in a daemon sample, goes to the kernel log as an error, such as nvx-time: ioperm revoke: Cannot allocate memory. The daemon's standard streams are on the null device.
  • The commands keep their arguments, output, exit statuses, and golden fixtures.

Tests:

  • Unit tests over a fake port backend cover a refused grant, a partial grant that is rolled back, a failed revocation that keeps the time tools from forking, a daemon that stops before it starts anything when its revocation fails, an acknowledgement that cannot take the permission, which leaves the restore unfinished, and an event whose revocation fails, which logs the failure after its debug-console write.
  • In a VM, snapshot-core runs a new test hook, nvx-time test port-access, after each restore. It takes a sample under the lock that serializes port-B transactions, as the daemon does, then starts a child that reads the port-B status and snapshot ports without taking the permission. The processor must refuse both reads, so the scenario prints NVX-SNAPSHOT-PORT-ACCESS-denied, which the harness requires. A control child that takes the permission itself must read them.

The typed calls of the same item follow in #663, which is stacked on this PR.

Part of #619. Refs #535.

Validation

Local, on a Windows/WHP host, with the Linux steps as root in a Linux VM on it. The Rust and guest checks ran on this head, e079bca, on dev at 3d06d42. The Python checks ran on an earlier revision with the same scripts.

  • python scripts/nvx.py verify.
  • In nvx-rs, with NVX_BUNDLE=skip:
    • cargo fmt --all --check;
    • cargo clippy --locked --all-targets --all-features -- -D warnings and cargo clippy --locked --lib --no-default-features -- -D warnings;
    • cargo test --locked --all-features and cargo test --locked;
    • the bundled artifacts test, and cargo doc --locked --no-deps --all-features with -D warnings;
    • cargo +1.89 check --locked --all-targets --all-features, and cargo check --locked --all-targets --all-features --target aarch64-apple-darwin.
  • python3 scripts/nvx.py check-guest-binaries, then build-guest-binaries and test-guest-binaries.
  • tests/runtime-adapter: cargo fmt, clippy, build, test and the 1.89 check in the default, host, and guest (musl) configurations.
  • The Python tests of validate-nvx on Linux (1062 tests) and Windows; ruff check, ruff format --check, pyright for Linux and Windows, and the Markdown link check; shellcheck for snapshot-core.sh.

Everything passes except three tests that fail the same way on dev 3d06d42 in this environment:

  • tests/cli.rs's nvx_rs_reproduces_the_recorded_scenarios fails as root, which caller-owned shares refuse.
  • test_adversarial.py's SIGKILL status case fails as root.
  • test_setup_azure.py's shell tests fail on Windows, where bash is a Linux VM's. They pass on Linux.

Real VMs, with python scripts/nvx.py test-microvm --backend <backend> and its default scenarios, including guest-boot, snapshot-core, time-abi-conformance, smp-snapshot, snapshot-tiers, restore-processors, restore-memory, and restore-downtime:

  • KVM, in the Linux VM on this host: every default scenario passed (41) on this head, on 4496196, and on nvx-rs: run the guest time and reseed steps through typed calls #663's head, which carries this head. On this head, snapshot-core and the four scenarios after it passed in separate runs. Two full runs had stopped at snapshot-core's check that a restore advances the wall clock and the uptime alike, while NTP stepped the Linux VM's clock forward by about 1.5 s every 34 s; the same check failed on nvx-rs: run the guest time and reseed steps through typed calls #663's head in those conditions.
  • WHP, on this host, with the built-in CPU profile: every default scenario passed (42) on this head, on 4496196, and on nvx-rs: run the guest time and reseed steps through typed calls #663's head.
  • On 4496196, the first KVM and WHP runs each failed one network scenario that uses host ports, while other sessions ran VMs on the same host: the L3/L4 egress policy check saw a connection on a blocked host port, and the private-network ingress check saw a connection to a denied guest port accepted on 127.0.0.1. The guest's own checks passed in both runs, the scenarios don't involve the time tools, and both reruns passed.
  • MSHV, on a Linux/MSHV virtual machine with an AMD EPYC Genoa CPU: every default scenario passed (41) on 419a3b0, an earlier revision of both PRs, before the review fixes.
  • snapshot-core printed NVX-SNAPSHOT-PORT-ACCESS-denied after each restore, in every run.
  • This host's nested KVM doesn't present the built-in profile that --cpu-profile auto selects (intel.alderlake.v1), and OpenVMM fails with E_PROFILE_UNSUPPORTED, as nested MSHV did with amd.genoa.v1. Those runs therefore passed OpenVMM --cpu-profile host, through a local wrapper of its binary.
  • OpenVMM and the kernel came from the development release v0.1.0-dev.0098c837e997, whose OpenVMM is this branch's gitlink, d0bdbed8. The Alpine initramfs and the guest binaries were built from the commit under test.

Stripped static guest binaries (build-guest-binaries) and the Alpine initramfs, in bytes:

Artifact dev 3d06d42 This PR Change
nvx-time 623,184 631,376 +8,192
nvx-reseed 401,888 401,888 0
The other seven binaries 0
initramfs.cpio.gz 8,194,973 8,200,726 +5,753
initramfs.cpio, uncompressed 19,663,360 19,671,552 +8,192

Left to CI, which passed every pull-request check on this head, merged into dev at 7269dbf: nvx-rs on Linux and Windows as a non-root user, the CLI tests, lint and typing, the shared artifacts, the performance gate, and the KVM, MSHV, and WHP microVM and platform tests with the built-in CPU profiles. Merged into dev at 6f0face, which since checks the guest build's inputs (#710), check-guest-binaries, build-guest-binaries, and test-guest-binaries pass locally.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🟡 Changes recommended

The probe lacks sample serialization, and acknowledgement permission failures can incorrectly report restore completion.

2 open issues
What changed in this PR

Scopes guest I/O-port permission to individual transactions, supporting #619’s guest-runtime isolation work.

Changes:

  • Adds thread-bound permission ownership, rollback, and revocation tracking.
  • Updates time commands and daemon startup to respect permission boundaries.
  • Adds unit tests and restored-VM port-access probes.
File Description
scripts/​test_microvm_tests.py Checks snapshot probe integration.
scripts/​nvx_tools/​microvm_tests.py Requires the port-access denial marker.
scripts/​nvx_tools/​microvm_test_scripts/​snapshot-core.sh Runs the probe after restore.
nvx-rs/​src/​guest/​time/​test_mode.rs Adds port-access test hooks.
nvx-rs/​src/​guest/​time/​system.rs Scopes transactions and guards process creation.
nvx-rs/​src/​guest/​time/​daemon.rs Guards startup and separates acknowledgement from signalling.
nvx-rs/​src/​guest/​time/​commands.rs Uses transaction-scoped permissions.
nvx-rs/​src/​guest/​time/​checks.rs Avoids starting threads while permission remains.
nvx-rs/​src/​guest/​sys/​port.rs Implements permission ownership and regression tests.
nvx-rs/​src/​guest/​sys/​mod.rs Exports permission helpers.

Review effort: Balanced

Comment thread nvx-rs/src/guest/time/commands.rs Outdated
Comment thread nvx-rs/src/guest/time/test_mode.rs Outdated
Linux keeps the permission that ioperm grants in each thread, and the
threads and processes that a holder starts inherit it, across exec too.
The time tools took it when a command started and kept it, so the boot
checks' threads, the time daemon, and its restore workers held it for
their whole lives.

A port transaction now takes the permission on the thread that uses the
ports, rolls a partial grant back, and revokes the permission when it
ends. Until a later revocation succeeds, a failed one keeps the thread
from starting a thread or a process, and a command that ends while the
thread may still hold the permission fails. The capture request and the
packet reads, the restore acknowledgement, each time sample, each event's
debug-console write, and the sample and generation-id commands are each
one transaction, and none of them reports an event while it holds the
permission. A capture that finishes a restore and cannot take the
permission for its acknowledgement leaves the restore unfinished, for
nvx-snapshot's restore-finish to acknowledge.

Unit tests fake the permission to cover a refused grant, a rolled-back
partial grant, failed revocations, which stop the time tools before
they fork or start a thread, and an acknowledgement that cannot take the
permission. In a VM, snapshot-core runs the new nvx-time test
port-access hook after each restore: a process that nvx-time starts
after a transaction must not reach OpenVMM's ports.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b57d5617-d565-4d34-a871-0077d2c11184
@esaurez
Enrique Saurez (esaurez) force-pushed the esaurez/a1-guest-g5-time-20261020 branch from 680b261 to 4496196 Compare October 20, 2026 21:49
Copilot AI balanced review requested due to automatic review settings October 20, 2026 21:49

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive permission inheritance and restore sequencing warrant human review alongside backend CI results.

No open issues 🎉

2 resolved since last review

Review effort: Balanced

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need some minor changes here.

Comment thread nvx-rs/src/guest/time/commands.rs Outdated
Comment thread nvx-rs/src/guest/time/commands.rs Outdated
Comment thread nvx-rs/src/guest/time/system.rs Outdated
An event's write to OpenVMM's debug console and the time daemon's
discipline poll ended their port transactions without a word when the
kernel refused to revoke the permission. The thread then starts nothing
until a later transaction revokes it, and the daemon's standard streams
are on the null device. Both now write the failure to the kernel log at
KERN_ERR, such as "nvx-time: ioperm revoke: Cannot allocate memory".

The sample and generation-id commands, and the port-probe test hook,
now end their transactions at the end of a scope rather than with an
explicit drop.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Copilot-Session: b57d5617-d565-4d34-a871-0077d2c11184
Copilot AI balanced review requested due to automatic review settings October 21, 2026 19:09

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🔵 Needs a closer look

Security-sensitive permission inheritance and restore acknowledgement sequencing warrant final human review.

No open issues 🎉

Review effort: Balanced

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants