Skip to content

fix: reset the cached compiler when build_ext is reinitialized - #150

Open
DivyamTalwar wants to merge 1 commit into
MagicStack:masterfrom
DivyamTalwar:fix/126-reinitialize-build-ext
Open

fix: reset the cached compiler when build_ext is reinitialized#150
DivyamTalwar wants to merge 1 commit into
MagicStack:masterfrom
DivyamTalwar:fix/126-reinitialize-build-ext

Conversation

@DivyamTalwar

Copy link
Copy Markdown

Fixes #126

The problem

httptools_build_ext.initialize_options() short-circuits after the first pass, on purpose:

def initialize_options(self):
    # initialize_options() may be called multiple times on the
    # same command object, so make sure not to override previously
    # set options.
    if getattr(self, '_initialized', False):
        return
    super().initialize_options()

That early return is load-bearing — it protects the Cython settings and extension mutations applied the first time through. But setuptools calls reinitialize_command('build_ext') between phases, and the base initialize_options() it is bypassing is what clears self.compiler. So a chained invocation reaches the second phase still holding the compiler instance the first phase configured:

python setup.py build bdist_wheel

The wheel phase then runs against a stale compiler rather than selecting one for the phase it is actually in, and the build fails. Anything that drives both phases through one command — which is the normal shape for a build backend — hits it.

The fix

Clear only the transient attribute, inside the early-return path:

if getattr(self, '_initialized', False):
    self.compiler = None
    return

The obvious alternative — dropping the early return and calling the base initializer — was rejected: it discards exactly the cached command options and extension state the early return exists to protect. compiler is the one piece of per-phase state that must not survive reinitialization, so the reset is scoped to it and nothing else.

Test

tests/test_build.py is new. It loads the real setup.py via runpy (with setuptools.setup mocked out) to get the actual build_ext subclass rather than a reimplementation of it, finalizes the command, attaches a compiler, reinitializes, and asserts the run proceeds to build_extensions().

It fails on master and passes with this change.

Verification

check result
focused regression 1 passed
python setup.py build bdist_wheel succeeds
make test 42 passed
make typecheck 0 errors, 0 warnings
git diff --check clean

Not covered locally: other Python versions and non-macOS platforms — left to CI.

`httptools_build_ext.initialize_options()` returns early once
`_initialized` is set, deliberately preserving the Cython and extension
mutations applied on the first pass. Setuptools reinitializes the
`build_ext` command between phases, so a chained invocation such as
`setup.py build bdist_wheel` reaches the wheel phase holding the
compiler instance the earlier phase already configured, and the build
fails instead of selecting a compiler for the current phase.

Clear only the transient `compiler` attribute in that early-return
path. Delegating to the base initializer instead would discard exactly
the cached command options and extension state the early return exists
to protect, so the reset is kept as narrow as the problem.

`tests/test_build.py` loads the real `setup.py`, finalizes the command,
attaches a compiler, reinitializes, and asserts the run proceeds — it
fails without this change.

Verified:
- Focused regression — 1 passed
- `python setup.py build bdist_wheel` — succeeds
- `make test` — 42 passed
- `make typecheck` — 0 errors, 0 warnings
- `git diff --check` — clean

Fixes MagicStack#126
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Custom build_ext breaks parallel build/wheel

1 participant