[olebhansen]

Brings nested virtualization (nested_virt) to Windows x86_64 hosts in the fork, so nestedSandboxes works for digdir-agents on Windows before the next agentctl release instead of waiting for upstream (digdir/digdir-agents#173). Upstream's own nested virtualization work also covers Windows on msb_krun 0.1.42 and adds nested snapshots. When the fork syncs to an upstream release that contains it, these commits should drop out of the queue. This PR keeps to the single nested_virt capability that upstream is likely to keep.

Commits

1. chore(deps): bump libkrun to 0.1.42. msb_krun 0.1.42 sets WHvPartitionPropertyCodeNestedVirtualization on WHP partitions when nesting is enabled. 0.1.41 only left VMX/SVM in the guest CPUID. The bump also brings libkrun's nested-snapshot and timer work, changes Windows x86_64 full snapshots (see Compatibility), and boots Windows guests with tsc_early_khz from the WHP clock.
2. feat(runtime): support nested virtualization on Windows x86_64 hosts. host_support() on Windows x86_64 creates a one-vCPU WHP partition with nesting and xAPIC emulation, sets it up and deletes it. That is the configuration libkrun uses, and WHP refuses nesting at setup without the local APIC emulation. WinHvPlatform.dll is loaded at run time, as Hypervisor.framework is on macOS. The commit also updates the docs and COMPATIBILITY.md, and adds a Windows CI step for the host-check tests.
3. fix(spawn): start detached Windows runtimes in jobs that forbid breakaway. The SDK starts detached runtimes with CREATE_BREAKAWAY_FROM_JOB. Inside a job object without breakaway, such as cargo test or some sandboxed tool shells, that failed with a bare PermissionDenied (os error 5). The SDK now retries without breakaway and logs a warning; the runtime then stays in the caller's job. It is independent of nesting, but without it the Windows integration tests cannot run under cargo test.

The guest kernel needs no change. The released libkrunfw-windows-x86_64.dll is built from the Linux x86_64 config and already has kvm_intel and kvm_amd built in.

Compatibility

• Nesting on Windows: an older SDK refuses before launch. An older Windows runtime advertises nested_virt but refuses nesting when the VM starts. A current SDK that resolves such a runtime therefore fails at start, after replace or a modify restart has changed the sandbox. That is accepted because the runtime ships with the SDK.
• Windows full snapshots: WHP execution state now records the emulated PIT (backend ABI 2). This runtime refuses full snapshots, forks and branches captured by earlier Windows runtimes, and earlier runtimes refuse the new ABI. Recovery is to restart from the snapshot's disks and capture again. Linux, macOS and Windows on Arm execution state are unchanged. digdir-agents uses no snapshots.
• Breakaway fallback: a runtime started inside a job that forbids breakaway ends when that job is terminated or closed with kill-on-close. Before this change it did not start at all. Launchers in no job, or in a job that allows breakaway, behave as before.

QA

Suggested round before the next agentctl release. Run the integration tests with MSB_TEST_EXPECT_NESTED_VIRT=1 on hosts that support nesting:

cargo nextest run -p microsandbox --test nested_virt --run-ignored=only

• All hosts (regressions from the libkrun bump):
  • Create, exec, stop, start and modify().restart().
  • Networking and published ports.
  • A disk-only snapshot and restore.
  • A sandbox without nested_virt has no /dev/kvm.
• Linux x86_64 and macOS (M3 or newer, macOS 15 or newer): nesting behaves as before. A full snapshot of a non-nested sandbox captures and restores.
• Windows x86_64 (Intel, and AMD if available):
  • nested_virt exposes /dev/kvm, and msb inside the guest boots its own sandbox.
  • A full snapshot captured with this runtime restores. One from 0.7.11-digdir.1 is refused with the missing-PIT message.
  • cargo test (not only nextest) runs the integration tests.
• Windows on Arm: nesting is still refused with a clear reason, and ordinary sandboxes are unaffected.
• digdir-agents: with its pin pointed at this branch and runtime digests that match, an Agent with nestedSandboxes: required reports the NestedSandboxes condition as available on Windows. It can then run nested Agents.

Evidence so far

On Windows 11 Enterprise 26200, Intel Core Ultra 7 155H, VBS/HVCI on:

• Unit tests and clippy:
  • MSB_TEST_EXPECT_NESTED_VIRT=1 cargo test -p microsandbox-runtime --lib nested_virt passes.
  • So do the SDK's nested_virt and runtime::spawn::tests::windows_ tests.
  • cargo clippy -p microsandbox-cli -p microsandbox-runtime -p microsandbox -- -D warnings is clean.
• Integration tests: both tests in sdk/rust/tests/nested_virt.rs pass under plain cargo test, which exercises the breakaway fallback. They ran against the msb.exe built from this branch, with the libkrunfw-windows-x86_64.dll from v0.7.11-digdir.1 and MSB_TEST_EXPECT_NESTED_VIRT=1. Before commit 3, nested_virt_can_be_modified_on_an_existing_sandbox failed under cargo test with os error 5.
• Earlier builds of this branch:
  • Inside a nested Debian sandbox, the Linux msb from v0.7.8-digdir.2 ran msb run alpine in about 3 s.
  • A digdir-agents build pinned to this branch ran T3 Code in an Agent with nestedSandboxes: required. T3 Code created nested worker Agents through agentctl in it and connected to one over SSH.

Not covered here: AMD Windows, macOS and Linux, and real Windows full snapshots across versions. Hosted CI runners probably lack nesting, so CI only exercises the probe.

🤖 Generated with Claude Code

[olebhansen]

@martinothamar This is ready for the QA round. Per your comment, nested_virt_windows is gone, so only nested_virt remains. The trade-off is in COMPATIBILITY.md: a current SDK that resolves an older Windows runtime fails when the VM starts, not at admission. The job-breakaway fix is commit 3, so the integration tests also run under plain cargo test on Windows.

The QA checklist per platform is in the description under QA. On Windows 11 with Intel, both nested_virt integration tests pass against an msb.exe built from 17271977. Not covered yet: AMD Windows, macOS, Linux, and Windows full snapshots across versions.

[olebhansen]

Back to draft. Investigating if this can be superseeded by upstream, now that superradcompany#823 is merged.

[olebhansen]

@martinothamar, I'm closing this one. Upstream merged its nested virtualization work into main on 27 October. It covers Windows x86_64 on msb_krun 0.1.42, including nested snapshots, a Windows firmware bundle with Hyper-V enabled, and qualification on both Intel and AMD. It isn't in a release yet (latest is v0.7.11), so Windows nesting will come with the fork sync to the first release that contains it.

What that means here:

• Commits 1 and 2 are dropped. Upstream asks libkrun for host support itself (msb_krun::nested_virt_supported) and reports it as the nested_virt launch capability, so our WHP probe isn't needed.
• The breakaway fix (commit 3) moves to its own PR, fix(spawn): start detached Windows runtimes in jobs that forbid breakaway #12. Upstream still has no fallback.
• The sync will need some API work. The fork's own nested virtualization patch (2b12316f) is replaced by upstream's, which has no nested_virt_host_support(), no modify().nested_virt(), and makes resources.nested_virt an Option that is off by default. digdir-agents uses all three, so the sync needs a thin shim in the fork or changes in digdir-agents.