Repository navigation
Discover GPUs through D3DKMT on WSL2 - #32988
Flora Cui (fcui-amd) wants to merge 1 commit into
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The EnumAdapters3 ABI layout is incorrect, placeholder filtering is overbroad, and the new discovery path lacks tests.
Review effort: Balanced
Findings: 1
Open (3)
What changed in this PR
Adds WSL2 GPU discovery through dynamically loaded D3DKMT entry points.
Changes:
- Enumerates WSL GPU adapters and records hardware IDs/LUIDs.
- Replaces WSL virtual GPU placeholders.
- Adds the implementation to Linux builds.
| File | Description |
|---|---|
wsl_device_discovery.h |
Defines WSL GPU discovery interface. |
wsl_device_discovery.cc |
Implements D3DKMT enumeration. |
device_discovery.cc |
Integrates WSL-discovered GPUs. |
onnxruntime_common.cmake |
Adds new Linux sources. |
💡 Configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
| struct D3DKMT_ENUMADAPTERS3 { | ||
| uint64_t Filter; | ||
| uint32_t NumAdapters; | ||
| D3DKMT_ADAPTERINFO* pAdapters; | ||
| }; |
There was a problem hiding this comment.
D3DKMT_ENUMADAPTERS_FILTER is 64-bit, not 32-bit. From include/uapi/misc/d3dkmthk.h
in microsoft/WSL2-Linux-Kernel:
union d3dkmt_enumadapters_filter {
struct {
__u64 include_compute_only:1;
__u64 include_display_only:1;
__u64 reserved:62;
};
__u64 value;
};
The kernel struct is filter at 0 (8 bytes), adapter_count at 8, explicit reserved at
12, adapters at 16.
|
|
||
| gpu_devices.erase(std::remove_if(gpu_devices.begin(), gpu_devices.end(), | ||
| [](const OrtHardwareDevice& gpu_device) { | ||
| return gpu_device.vendor_id == kMicrosoftPciVendorId; |
There was a problem hiding this comment.
This block only runs when D3DKMT enumeration succeeded, which requires both /dev/dxg and
libdxcore.so. In that environment every 0x1414 display-class device is by construction
a dxgkrnl stand-in. There is no real Microsoft GPU to preserve. Pinning the predicate to
0x008e would leave other synthetic IDs behind as ghost entries alongside the real adapter,
so the broad match is deliberate. I'll make the comment say so explicitly.
|
|
||
| // Runs the two-pass enumeration: the first call reports the adapter count, the second | ||
| // fills the caller-provided array. | ||
| bool EnumerateAdapters(const DxCoreThunks& thunks, std::vector<D3DKMT_ADAPTERINFO>& adapters_out) { |
Linux GPU discovery scans /sys/class/drm/card*/device and falls back to /sys/bus/pci/devices. Under WSL2 the GPU is paravirtualized through /dev/dxg: /sys/class/drm is empty, and the only entry the PCI scan finds is a Microsoft virtual display adapter (vendor 0x1414, class 0x0302) standing in for the real hardware. DiscoverDevicesForPlatform() therefore reported a GPU whose vendor and device IDs belong to no vendor's runtime. Every execution provider that selects its device by matching against the discovered OrtHardwareDevice list registered zero OrtEpDevices, even though the vendor runtime enumerated the GPU and worked. Enumerate adapters through the D3DKMT entry points exported by libdxcore.so and read vendor/device IDs from KMTQAITYPE_PHYSICALADAPTERDEVICEIDS. When that yields adapters, drop the Microsoft placeholders the real descriptors replace. The D3DKMT structs are declared locally rather than included from the Windows SDK/WDK so that a Linux build gains no new header or package dependency, and the thunks are loaded through Env::LoadDynamicLibrary rather than linked. Discovery is gated on /dev/dxg existing, so nothing is loaded off WSL2. EnumAdapters3 is preferred where available because EnumAdapters2 cannot report compute-only adapters; it falls back when the symbol is absent. The adapter LUID is recorded as metadata to give Linux the device identifier that the Windows path already exposes. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Made-with: GitHub Copilot Signed-off-by: Flora Cui <flora.cui@amd.com>
0fe2f1c to
d0c6a70
Compare
|
Superseded by #34007, which recreates this change on current main. |



Description
Linux GPU discovery scans
/sys/class/drm/card*/deviceand falls back to/sys/bus/pci/devices. Under WSL2 the GPU is paravirtualized through/dev/dxg:/sys/class/drmholds nocard*entries, and the only device the PCI scan finds is aMicrosoft virtual display adapter — vendor
0x1414, device0x008e, class0x030200—standing in for the real hardware.
This adds a third discovery tier that enumerates adapters through the D3DKMT entry points
exported by
libdxcore.soand reads the vendor/device IDs fromKMTQAITYPE_PHYSICALADAPTERDEVICEIDS. When it yields adapters, the Microsoft placeholdersthat those descriptors replace are dropped from the list.
Implementation notes:
a Linux build gains no new header or package dependency.
Env::LoadDynamicLibrary, not linked, and the whole path isgated on
/dev/dxgexisting — nothing is loaded off WSL2.D3DKMTEnumAdapters3is preferred where available becauseD3DKMTEnumAdapters2cannotreport compute-only adapters; it falls back to
EnumAdapters2when the symbol is absent.path already exposes.
Motivation and Context
DiscoverDevicesForPlatform()reported a GPU whose vendor and device IDs belong to novendor's runtime. Every execution provider that selects its device by matching against the
discovered
OrtHardwareDevicelist therefore registered zeroOrtEpDevices on WSL2, eventhough the vendor runtime enumerated the GPU and inference worked through the older
append_execution_providerpath. Autoep selection,OrtEpDeviceenumeration, and anythingbuilt on top of them were unusable.