Conversation
Decision record for replacing policyengine.py's microdf dependency with an internal policyengine.weighted module: numpy kernels with a parity test against microcosm.frame.accounting, weights as an ordinary <entity>_weight column, and a closed df.weighted(...) accessor that exposes only weighted operations. The note pins every estimator definition against a named reference, records that the Gini and top-share tie rules change published figures, and stages the migration: Stage 1 (module lands; inequality, decile grouping and labour supply move onto it), Stage 2 (MicroDataFrame leaves the public data model; major version, two releases), Stage 3 (engine return types; a separate decision). Fixes #528 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
CI already tests 3.11 to 3.14, sim-api's runtime images are python:3.13 and axiom-oracles requires 3.13; the country packages allow 3.11 and are unaffected. The kernel import from microcosm-frame then waits only on its PyPI release. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
MaxGhenis
added a commit
to PolicyEngine/microdf
that referenced
this pull request
Sep 28, 2026
…e roadmap (#337) policyengine.py is taking the weighted layer into an internal module (PolicyEngine/policyengine.py#529). microdf stays maintained for the packages and analysis repositories that depend on it; planned work is the fail-closed behaviour in #333 and #264, the poverty definitions in #334 and the documentation fix in #335. Wider coverage of pandas methods that change shape, design-based variance and dataset presets are withdrawn. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #528
Adds
docs/weighted-module-design.md, the decision record for replacing policyengine.py's microdf dependency with an internalpolicyengine.weightedmodule, and lists it in the docs sidebar.What the note decides: numpy estimator kernels with a parity test against
microcosm.frame.accounting; weights as an ordinary<entity>_weightcolumn; a closeddf.weighted(...)accessor that exposes only weighted operations and raises on everything else; the definition of every kernel against a named reference, two of which (Gini, top shares) change published figures by the tie rule; Stage 1 and Stage 2 of the migration. It names Stage 3 (engine return types) and the Python floor as separate decisions.Context: PolicyEngine/microdf#315 and #333 (the September 2026 review of microdf), #527 (inequality.py reimplements Gini and shares with a different tie rule). The counts in the note come from a six-lane code census with each load-bearing fact re-derived by a second reader, and the draft went through a four-lens adversarial review (pandas maintainer, analyst, survey statistician, prose) before this commit.
Docs-only change; no code.
🤖 Generated with Claude Code