Skip to content

Adapting to Python 3.15+ IntFlag changes for negative members. #894

Closed

Description

@junkmd

To the community,

In this project, IntFlag subclasses have been generated within friendly modules based on enumeration definitions retrieved from COM type libraries (see #345).
Among these, some enums — such as MsiInstallState defined in the Microsoft Windows Installer Object Library — contain a mix of positive and negative values.

While these definitions worked as expected up to Python 3.14, the internal implementation of IntFlag is changing in Python 3.15.0-alpha, meaning they will no longer be evaluated as they were previously.

Background

The root of this issue lies in the inconsistency of handling negative values within a flag (IntFlag).

As discussed in python/cpython#107538 and python/cpython#132273, Flag is intended to represent a "set of bits". Including negative values as members creates a logical contradiction.

I was the one who originally proposed and implemented the enum definitions in this project, and I should have recognized the irrationality of treating negative values as flags at that time.
I apologize for this oversight.

Problem reproduction (minimal reproducer)

The following is a minimal reproducer using a snippet of the MsiInstallState definition generated by import comtypes.client; comtypes.client.GetModule('msi.dll'), along with a basic value check and test.

In Python 3.15.0-alpha and later, the evaluation of IntFlag is being changed, and negative members will no longer be evaluated as their literal values (e.g., -1).

from enum import IntFlag


class MsiInstallState(IntFlag):
    msiInstallStateNotUsed = -7
    msiInstallStateBadConfig = -6
    msiInstallStateIncomplete = -5
    msiInstallStateSourceAbsent = -4
    msiInstallStateInvalidArg = -2
    msiInstallStateUnknown = -1
    msiInstallStateBroken = 0
    msiInstallStateAdvertised = 1
    msiInstallStateRemoved = 1
    msiInstallStateAbsent = 2
    msiInstallStateLocal = 3
    msiInstallStateSource = 4
    msiInstallStateDefault = 5


if __name__ == "__main__":
    print((repr(MsiInstallState.msiInstallStateUnknown), MsiInstallState.msiInstallStateUnknown.value))
    assert MsiInstallState.msiInstallStateUnknown == -1

Execution result in Python 3.14:

('<MsiInstallState.msiInstallStateUnknown: -1>', -1)

Execution result in Python 3.15:

('<MsiInstallState.msiInstallStateUnknown: 7>', 7)
Traceback (most recent call last):
  File "...\enum_testing.py", line 22, in <module>
    assert MsiInstallState.msiInstallStateUnknown == -1
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError

Why does the value become 7?

In Python 3.15+, IntFlag treats only the bit range covered by the defined positive members as the "valid bit domain".
Negative values are then reinterpreted as "all bits ON (masked)" within that specific range.

  1. Identifying the Valid Bit Range:
    The maximum positive value in MsiInstallState is 5 (101 in binary).
    Consequently, IntFlag determines the valid bit width for this enum to be the lower 3 bits ($2^0, 2^1, 2^2$).

  2. Masking the Negative Value:
    A negative value like -1 has all bits set to 1 in its two's complement representation.
    However, Python 3.15 IntFlag normalizes this to a state where "all bits within the valid 3-bit range are 1."

  3. Calculation Result:
    With the lower 3 bits all set to 1, the value becomes ($2^2 + 2^1 + 2^0 = 7$), losing consistency with the literal -1.

Future direction

To ensure this project continues to function as intended in Python 3.15 and beyond, some form of adaptation is necessary.
At this stage, I want community feedback.

Directions under consideration:

  • Direction 1: modification of code generation logic in comtypes

    • Adjust the generation method within the library to maintain value consistency even in Python 3.15+.
  • Direction 2: proposing changes to CPython regarding the handling of negative values

    • Demonstrate to the CPython community and the core developers that there is a demand for treating negative values as flags and advocate for improvements or maintenance of the Python<=3.14 behavior.

Request to the community and discussion rules

Your feedback is vital.
This issue is something that all Python developers dealing with enums containing negative values may face — regardless of whether they use comtypes, or whether they define enums via dynamic generation or static hard-coding.

To ensure the discussion proceeds smoothly, please adhere to the following:

  1. If you have a specific proposal, please comment on this issue first.
  2. Please do not create a pull request before discussing it here. If discussions happen separately on PRs, it becomes difficult to track the decision-making process and hurts traceability.

If Python 3.15 reaches the release candidate (RC) stage and negative members are still not evaluated as literals, I intend to consider merging the only available proposal or the one deemed most rational at that time.

I look forward to a sincere discussion to find the best possible solution.
Thank you for your input.

Activity

  1. self-assigned this
    on Dec 30, 2025
  2. added
    bugSomething isn't working
    shared_infouse cases, tips and troubleshoots
    help wantedExtra attention is needed
    on Dec 30, 2025
  3. junkmd commented on Dec 31, 2025

    @junkmd
    CollaboratorAuthor

    Proposal: Define enumerations with negative members as IntEnum.

    As a maintainer, I have explored a concrete implementation to address this issue.
    I wanted to see if a viable fix could be achieved without introducing excessive complexity.

    You can find the draft implementation details here: junkmd/comtypes@eea9c0d...52ec418.

    I want to emphasize to the community: Please do not feel pressured by the level of detail in this proposal.
    I encourage everyone to share your ideas, concerns, or alternative suggestions even if they are just quick comments.
    This draft is intended to lower the barrier for discussion, not to set it in stone.

    Technical Rationality

    The core of this proposal is to dynamically switch the base class to IntEnum when an enumeration contains negative values, while maintaining IntFlag for non-negative bitmasks.

    I chose this direction for the following reasons:

    1. Mathematical Consistency:

    Treating negative values as flags (bitmasks) is mathematically inconsistent.
    IntEnum is the appropriate tool for discrete integer states that include negative values.

    2. Maintenance Cost & CPython Alignment:

    comtypes originally adopted the standard enum module with Python 3 to reduce custom maintenance.
    Creating a "custom IntFlag" that mimics the old behavior would fight against CPython's internal implementation, increasing the risk of technical debt as the language evolves.

    3. Accuracy:

    Using IntEnum ensures that literal values (e.g., -1) remain exactly as defined in the type library, preventing the range-masking issues introduced in Python 3.15.

    Note

    Please note that I do not intend to merge this or any proposals until Python 3.15 reaches its Release Candidate (RC) stage at the earliest.

    This is strictly a proposal to serve as a starting point. We have a significant window of time to refine this approach or pivot if the community or the CPython specification suggests a better path.

    I look forward to hearing your thoughts!

  4. pinned this issue on Dec 31, 2025
  5. desaipriyanshu611-art commented on Jul 27, 2026

    @desaipriyanshu611-art

    what should i do now?

  6. junkmd commented on Jul 30, 2026

    @junkmd
    CollaboratorAuthor

    @desaipriyanshu611-art

    Thank you for your interest in this discussion.

    To start with, could you describe how you are utilizing the enums defined in comtypes?

    If you use members with negative values as bitwise flags in the enums generated by comtypes, this change (originating from Python 3.15) could potentially impact your scripts.

    It would also be very helpful if you could share the files generated under comtypes/gen/... when running your script so we can give you a better assessment.

  7. linked a pull request that will close this issueUpdate docs for Python 3.15 support. #945on Oct 4, 2026
  8. PalmtopTiger commented on Oct 11, 2026

    @PalmtopTiger

    @junkmd
    Hi! I came here just to thank you for maintaining such a complex project on your own.

  9. junkmd commented on Oct 11, 2026

    @junkmd
    CollaboratorAuthor

    @PalmtopTiger

    Thank you for your interest in this package and for taking the time to discuss this issue.

    I would also like to share a little about my current situation and what I think can help open source maintenance in the AI era.

    In the past, I worked on a project that depended heavily on this package.
    I have since moved to another team, so I have less direct work-related motivation to spend time on this package than before.
    However, I still work for the same company as my former team, and I believe maintaining this package remains an important priority.

    One reason I have been able to contribute more tests recently is that I have been experimenting with coding agents.
    I use their free plans to see how much I can improve the test coverage of this package.
    The recent increase in my test contributions is largely the result of these experiments.

    I think AI tools can create new ways to support open source projects.
    At the same time, free plans for AI services have been becoming more limited, and the time between usage resets has also been getting longer.
    This makes it harder to depend on free plans for regular maintenance work.

    The amount of AI service usage I can access has a direct effect on how much work I can do for this package.
    In this sense, access to AI tools is becoming an important resource for open source maintenance, just like time and testing environments.

    I have not been accepted into the Codex for Open Source program, so I would be grateful if someone who is able to recommend me could do so.
    I also hope that companies and other groups will offer more ways to help maintainers avoid burnout and keep their projects healthy over the long term.

    For companies that depend on comtypes, a Tidelift subscription or support through GitHub Sponsors would be very helpful.
    Support does not have to be limited to money, either.
    Access to a Windows virtual machine that I can safely use for testing, or an Office subscription for testing COM automation, would make a real difference.
    These resources would help me maintain and test comtypes more reliably over the long term.

    Of course, I understand that not everyone can provide this kind of support. Even a small sign of interest can help.
    When I maintain this package, it can sometimes feel like a lonely job.
    A reaction to an issue, a short comment, or a report about how you use comtypes can remind me that other people care about the project too.

    You do not need to prepare a detailed technical proposal to join a discussion.
    In the future, I would especially welcome even short comments on questions about backward compatibility, such as this one. Hearing how other people use the package helps me make better decisions and understand the real impact of changes.

    I hope we can find ways to make open source maintenance more sustainable together. Thank you again.

  10. PalmtopTiger commented on Oct 12, 2026

    @PalmtopTiger

    @junkmd

    Here are all the snippets from my personal scripts where I use comtypes. None of them use flags, so I’m not sure they’ll help you.

    Retrieving the .exe version

    from comtypes.client import CreateObject
    
    def app_version(path):
    	path = fspath(path) # converting pathlib.Path to str
    	fso = CreateObject("Scripting.FileSystemObject")
    	try:
    		return fso.GetFileVersion(path)
    	except:
    		return ""

    Checking whether the path points to an SSD or an HDD

    This is necessary to determine the number of read/write threads.

    from comtypes.client import CreateObject
    
    def is_ssd(path):
    	letter = path.absolute().drive
    	try:
    		locator = CreateObject("WbemScripting.SWbemLocator")
    		wmi = locator.ConnectServer()
    		item = wmi.AssociatorsOf(f'Win32_LogicalDisk.DeviceID="{letter}"', "Win32_LogicalDiskToPartition").ItemIndex(0)
    		device_id = item.Properties_["DeviceID"].Value
    		item = wmi.AssociatorsOf(f'Win32_DiskPartition.DeviceID="{device_id}"', "Win32_DiskDriveToDiskPartition").ItemIndex(0)
    		device_id = item.Properties_["Index"].Value
    
    		wmi = locator.ConnectServer(".", "root\\Microsoft\\Windows\\Storage")
    		return bool(wmi.ExecQuery(f'SELECT MediaType FROM MSFT_PhysicalDisk WHERE DeviceId = {device_id} AND MediaType = 4').Count)
    	except:
    		return False

    A class for working with SAPI TTS

    I’ve checked—all the constants here are ints.

    from pathlib import PureWindowsPath
    from comtypes.client import CreateObject, Constants
    
    class Voice:
    	def __init__(self, name=None, rate=None):
    		self._engine    = CreateObject("SAPI.SpVoice")
    		self._constants = Constants(self._engine)
    
    		if name:
    			name = name.lower()
    			for voice in self._engine.GetVoices():
    				if name in PureWindowsPath(voice.Id).name.lower():
    					self._engine.Voice = voice
    					break
    
    		if rate is not None:
    			self.rate = rate
    
    		# Init engine
    		self._engine.AudioOutputStream
    
    	@property
    	def rate(self):
    		return self._engine.Rate
    	@rate.setter
    	def rate(self, rate):
    		if not (isinstance(rate, int) and -10 <= rate <= 10):
    			raise ValueError("rate must be an integer in the range from -10 to 10")
    		self._engine.Rate = rate
    
    	def speak(self, text, async=True):
    		flags = self._constants.SVSFIsXML
    		if async:
    			flags |= self._constants.SVSFlagsAsync
    		self._engine.Speak(text, flags)
    
    	def purge(self):
    		if self._engine.Status.RunningState != self._constants.SRSEDone:
    			self._engine.Speak("", self._constants.SVSFPurgeBeforeSpeak)

    Regarding AI, I can only say that there is a noticeable trend of rising costs. If you entrust development to AI, there is a risk of ending up with a heap of unmaintainable spaghetti code. But the decision is, of course, yours. AI-assisted development is probably better than no development at all.

  11. junkmd commented on Oct 12, 2026

    @junkmd
    CollaboratorAuthor

    @PalmtopTiger

    Thank you for sharing these examples!
    They are useful even if most of them do not use flags. The SAPI example is also helpful because it shows how constants are used in real code.

    I agree with your concern about AI-generated code.
    If we accept code without careful review, it can easily become hard to maintain. I do NOT want to use AI to produce more code just for the sake of it.
    However, I think AI assistants can help us solve some long-standing problems in testing and maintenance.

    For example, it is often difficult to use real COM servers as test fixtures because of license or NDA restrictions.
    Also, to test whether COM data is correctly marshalled across language boundaries, we often need a native COM test double. This requires knowledge of the language used to write the test server, in addition to Python and COM itself.
    The large amount of boilerplate code needed to create COM servers is also one reason why COM APIs are not actively developed as much today.

    AI assistants can help reduce these costs.
    Real COM servers often have complex features for their business needs and require complex setup.
    It is much easier to ask an AI assistant to help create a small test server that does just one thing, such as returning a VARIANT with VT_ERROR and a specified error code. This has little business value on its own, but it is very useful for testing comtypes.

    I have also been using AI to test Python features that were previously too costly to test because setting up the required fixtures was difficult.
    Examples include COM calls between threads using threading, and capturing output sent to the debug monitor.

    So, I do NOT see AI as a way to replace careful development or to produce code without review.
    I see it as a way to make tests possible that would otherwise take too much time and effort to build.
    Better tests can help us find bugs and keep the code maintainable.

    I agree that the rising cost of AI services is a real concern.
    This is one reason I think we need better ways to support open source maintainers.

    Thanks again for sharing your code and your thoughts!

  12. junkmd commented on Oct 12, 2026

    @junkmd
    CollaboratorAuthor

    @PalmtopTiger

    I would also welcome contributions of test cases for COM libraries that your projects depend on, if they can be used in GitHub Actions CI on Windows.

    I prefer not to add binary files to this repository. However, the COM library does not have to be built into Windows.
    If we can fetch or install it in Actions without causing problems for its distributor, keep all its source files and other components as plain text in the repository, and build it in Actions, I may be able to add it to our CI pipeline.

    As a maintainer, I know that I may sometimes break something without realizing it.
    Tests and CI are a great help in finding these problems before they reach users.
    The more real-world use cases we can test, the more confidence we can have in changes to comtypes.

    In the past, preparing these tests and CI pipelines often took too much time and effort.
    AI agents have made this work more practical. Recent language models can help create test cases and CI workflows from a description of a real use case, even without using the most advanced models.

    If you have a suitable COM library or use case, I would be happy to discuss how we could add it to our tests.
    Contributions and ideas are very welcome!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

bugSomething isn't workinghelp wantedExtra attention is neededshared_infouse cases, tips and troubleshoots

Type

No type

Projects

No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions