FIX intermittent Windows module enumeration failure

[ogrisel]

Fix #217

Summary

• Fix intermittent OSError("GetModuleFileNameEx failed") on Windows when DLLs are loaded or unloaded during library discovery (reported with conda-forge OpenCV).
• Replace the two-phase EnumProcessModulesEx + GetModuleFileNameExW scan with a snapshot-first approach using CreateToolhelp32Snapshot, keeping long-path support via GetModuleFileNameExW and graceful per-module fallbacks.
• Add Windows integration tests for library paths longer than MAX_PATH, inspired by the local repro documented in ENH Support path length > MAX_PATH=260 on Windows #189 (comment).

Closes #217

Test plan

• Windows CI (py311_conda_forge_openblas): new tests test_windows_library_path_longer_than_max_path and test_windows_library_path_exceeds_internal_limit pass
• Windows CI: existing BLAS/OpenMP discovery tests still pass
• Manual repro from issue ThreadpoolController() raises an error intermittently with conda-forge OpenCV on Windows #217 with conda-forge OpenCV:

  import cv2
  from threadpoolctl import ThreadpoolController
  
  for _ in range(1000):
      ThreadpoolController()

AI assistance disclosure

This pull request was prepared with assistance from an AI coding agent (Cursor / Claude).

Made with Cursor

[ogrisel]

Code style review.

[ogrisel]

I think this LGTM. The licensing issue is still there but will be addressed in a follow-up PR to decouple the concerns.

cc @jeremiedbb @tomMoral.

[ogrisel]

It seems that clang conda-forge packaging has changed because it can no longer find its openmp but it's unrelated to this PR.

[ogrisel]

I synced with master and everything is green thanks to @lesteve's recent CI fixes.

Shall we merge?

[lesteve]

Codex /review tells me the following. I am afraid that I haven't tried to understand yet whether it makes sense or not, very sorry about that 😓

[P1] Bound snapshot retries so long paths reach the fallback

File: threadpoolctl.py:1312-1313

When CreateToolhelp32Snapshot persistently returns ERROR_BAD_LENGTH—notably in the long-DLL-path scenario this fallback is intended to handle—this unconditional retry loops forever, so _find_libraries_on_windows never catches an
OSError and never invokes EnumProcessModulesEx.

Limit the retries and raise after exhaustion so discovery can use the long-path-capable fallback.

[lesteve]

I can reproduce the original on my Windows VM so I'll give it a try as a sanity check.

[ogrisel]

Here is the reply from agent to your agent:

The Codex diagnosis is incorrect, but a retry cap would still be a reasonable hardening change.

ERROR_BAD_LENGTH (24) is the documented transient race when the module list changes while CreateToolhelp32Snapshot is taken. MSDN's own wording for TH32CS_SNAPMODULE / TH32CS_SNAPMODULE32 is to retry until it succeeds. The while True loop is that guidance, not a forgotten bound.

Long DLL paths are a different failure mode: MODULEENTRY32W.szExePath is limited to MAX_PATH. If the snapshot succeeds, we get a truncated path and skip that library with a warning. The EnumProcessModulesEx fallback is only used when snapshot creation raises OSError, not when szExePath is truncated. ERROR_BAD_LENGTH is not the long-path error.

test_windows_library_path_longer_than_max_path already exercises that case. If long paths caused a persistent ERROR_BAD_LENGTH spin, that test would hang; it does not.

A bounded retry (say 8-16 attempts, no sleep) then raise OSError(...) would still be good library hygiene, since ThreadpoolController() sits on a hot path and an unbounded loop could hang if GetLastError were ever sticky or mis-attributed. I would treat that as optional hardening, not a merge blocker for #217.

A related but separate gap: if the snapshot succeeds with a truncated szExePath, we currently skip the library instead of resolving that one handle with GetModuleFileNameExW. That is independent of the infinite-loop claim.

[ogrisel]

@copilot resolve the merge conflicts in this pull request

[ogrisel]

I synced with main. The good news is that on newer Python versions, dllist is used instead of the ctypes based win32 codepath (as of the merge of #248) and the non-regression tests still pass \o/

So this means that the ugly extra complexity introduced in that PR to get those non-regression test to also pass on older CPython version will go away in a few years once we stop support CPython < 3.14.

I think this PR deserves to be merged, at least to introduce the non-regression and stress tests. If we don't want to introduce the extra complexity, we could revert it and skip the new tests on older Python versions.

WDYT @lesteve?

[ogrisel]

Actually, my above comment is wrong. The custom win32 code branch is always used, even on newer CPython versions.

Let's check if this is needed or not.

[ogrisel]

Let's see if the new tests fail with dlllist on windows.

[ogrisel]

This is great: the dllist method passes all the non-regression stress tests on Windows with recent versions of Python. This means that once we drop older Python support we will be able to significantly trim the windows specific code path, including the extra complexity introduced in this PR.

[ogrisel]

LGTM. The CI is really stable, even when running the stress test on more cases.