Skip to content

Discover GPUs through D3DKMT on WSL2 - #32988

Closed
Flora Cui (fcui-amd) wants to merge 1 commit into
microsoft:mainfrom
fcui-amd:fix/wsl-d3dkmt-device-discovery
Closed

Flora Cui (fcui-amd) wants to merge 1 commit into
microsoft:mainfrom
fcui-amd:fix/wsl-d3dkmt-device-discovery

Conversation

@fcui-amd

Copy link
Copy Markdown

Description

Linux GPU discovery scans /sys/class/drm/card*/device and falls back to
/sys/bus/pci/devices. Under WSL2 the GPU is paravirtualized through /dev/dxg:
/sys/class/drm holds no card* entries, and the only device the PCI scan finds is a
Microsoft virtual display adapter — vendor 0x1414, device 0x008e, class 0x030200 —
standing in for the real hardware.

This adds a third discovery tier that enumerates adapters through the D3DKMT entry points
exported by libdxcore.so and reads the vendor/device IDs from
KMTQAITYPE_PHYSICALADAPTERDEVICEIDS. When it yields adapters, the Microsoft placeholders
that those descriptors replace are dropped from the list.

Implementation notes:

  • The D3DKMT structs are declared locally rather than included from the Windows SDK/WDK, so
    a Linux build gains no new header or package dependency.
  • The thunks are loaded through Env::LoadDynamicLibrary, not linked, and the whole path is
    gated on /dev/dxg existing — nothing is loaded off WSL2.
  • D3DKMTEnumAdapters3 is preferred where available because D3DKMTEnumAdapters2 cannot
    report compute-only adapters; it falls back to EnumAdapters2 when the symbol is absent.
  • The adapter LUID is recorded as metadata, giving Linux the device identifier the Windows
    path already exposes.

Motivation and Context

DiscoverDevicesForPlatform() reported a GPU whose vendor and device IDs belong to no
vendor's runtime. Every execution provider that selects its device by matching against the
discovered OrtHardwareDevice list therefore registered zero OrtEpDevices on WSL2, even
though the vendor runtime enumerated the GPU and inference worked through the older
append_execution_provider path. Autoep selection, OrtEpDevice enumeration, and anything
built on top of them were unusable.

Copilot AI balanced review requested due to automatic review settings September 30, 2026 13:33
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot review overview

🟡 Changes recommended

The EnumAdapters3 ABI layout is incorrect, placeholder filtering is overbroad, and the new discovery path lacks tests.

Review effort: Balanced
Findings: 1 High severity · 1 Medium severity · 1 Low severity

Open (3)
What changed in this PR

Adds WSL2 GPU discovery through dynamically loaded D3DKMT entry points.

Changes:

  • Enumerates WSL GPU adapters and records hardware IDs/LUIDs.
  • Replaces WSL virtual GPU placeholders.
  • Adds the implementation to Linux builds.
File Description
wsl_device_discovery.h Defines WSL GPU discovery interface.
wsl_device_discovery.cc Implements D3DKMT enumeration.
device_discovery.cc Integrates WSL-discovered GPUs.
onnxruntime_common.cmake Adds new Linux sources.

💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.

Comment on lines +51 to +55
struct D3DKMT_ENUMADAPTERS3 {
uint64_t Filter;
uint32_t NumAdapters;
D3DKMT_ADAPTERINFO* pAdapters;
};

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

D3DKMT_ENUMADAPTERS_FILTER is 64-bit, not 32-bit. From include/uapi/misc/d3dkmthk.h
in microsoft/WSL2-Linux-Kernel:

union d3dkmt_enumadapters_filter {
    struct {
        __u64	include_compute_only:1;
        __u64	include_display_only:1;
        __u64	reserved:62;
    };
    __u64		value;
};

The kernel struct is filter at 0 (8 bytes), adapter_count at 8, explicit reserved at
12, adapters at 16.


gpu_devices.erase(std::remove_if(gpu_devices.begin(), gpu_devices.end(),
[](const OrtHardwareDevice& gpu_device) {
return gpu_device.vendor_id == kMicrosoftPciVendorId;

Copy link
Copy Markdown
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This block only runs when D3DKMT enumeration succeeded, which requires both /dev/dxg and
libdxcore.so. In that environment every 0x1414 display-class device is by construction
a dxgkrnl stand-in. There is no real Microsoft GPU to preserve. Pinning the predicate to
0x008e would leave other synthetic IDs behind as ghost entries alongside the real adapter,
so the broad match is deliberate. I'll make the comment say so explicitly.


// Runs the two-pass enumeration: the first call reports the adapter count, the second
// fills the caller-provided array.
bool EnumerateAdapters(const DxCoreThunks& thunks, std::vector<D3DKMT_ADAPTERINFO>& adapters_out) {
Linux GPU discovery scans /sys/class/drm/card*/device and falls back to
/sys/bus/pci/devices. Under WSL2 the GPU is paravirtualized through /dev/dxg:
/sys/class/drm is empty, and the only entry the PCI scan finds is a Microsoft
virtual display adapter (vendor 0x1414, class 0x0302) standing in for the real
hardware. DiscoverDevicesForPlatform() therefore reported a GPU whose vendor
and device IDs belong to no vendor's runtime. Every execution provider that
selects its device by matching against the discovered OrtHardwareDevice list
registered zero OrtEpDevices, even though the vendor runtime enumerated the
GPU and worked.

Enumerate adapters through the D3DKMT entry points exported by libdxcore.so
and read vendor/device IDs from KMTQAITYPE_PHYSICALADAPTERDEVICEIDS. When that
yields adapters, drop the Microsoft placeholders the real descriptors replace.

The D3DKMT structs are declared locally rather than included from the Windows
SDK/WDK so that a Linux build gains no new header or package dependency, and
the thunks are loaded through Env::LoadDynamicLibrary rather than linked.
Discovery is gated on /dev/dxg existing, so nothing is loaded off WSL2.
EnumAdapters3 is preferred where available because EnumAdapters2 cannot report
compute-only adapters; it falls back when the symbol is absent.

The adapter LUID is recorded as metadata to give Linux the device identifier
that the Windows path already exposes.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Made-with: GitHub Copilot

Signed-off-by: Flora Cui <flora.cui@amd.com>
@mingyueliuh

Copy link
Copy Markdown
Contributor

Superseded by #34007, which recreates this change on current main.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants