Skip to content

UK cut: the personal allowance, from statute to a Manchester family to the whole UK - #2

Open
MaxGhenis wants to merge 8 commits into
mainfrom
uk-version
Open

MaxGhenis wants to merge 8 commits into
mainfrom
uk-version

Conversation

@MaxGhenis

@MaxGhenis MaxGhenis commented Sep 27, 2026 •

Copy link
Copy Markdown
Contributor

A UK cut of the 30-second video, with ?country=uk and tools/render_all.sh uk. It follows one real rule from statute to code to one family to the whole UK: the personal allowance, raised from £12,570 to £15,000 in 2026-27. The US film is unchanged: every poster frame is pixel-identical in both layouts.

What's on screen, and where it comes from

Scene On screen Source
Statute Income Tax Act 2007, Part 3, Chapters 1–3 (4,204 words); the dive lands on s. 35(1), "£12,570" legislation.gov.uk (OGL v3.0), each line checked against the site's HTML
Quote "…made more plaine and short, to th'intent that men might the better understaund them…" — Edward VI · 1551 Nichols (1857), p. 486; verbatim in two OCR scans and the page image
Code policyengine_uk/…/personal_allowance/amount.yaml at 412b25aa Byte-identical to the file the model ran
Family A single parent in Manchester with two kids: +£219 from £15,000 to £48,000 (Universal Credit takes back 55p of each £1), +£486 from £48,700 to £50,270, +£972 at £60,000 A household sweep, asserted at all 80,001 £1 points; identical on the previous pin (policyengine.py 6.1.1, policyengine-uk 2.90.2) and through policyengine.py's household calculator
Nation £22 billion net cost in 2026-27; 75.9% of households gain; 40,000 fewer children in poverty (absolute, before housing costs) tools/uk/national.py: policyengine.py 6.2.0 (policyengine-uk 2.102.3) on the enhanced FRS 2024-25 (policyengine-uk-data 1.56.16), uprated to 2026
Map 12,000 households drawn by weight, each placed at random in its constituency (in Northern Ireland, anywhere in NI) The dataset's constituency_code_oa; ONS July 2024 boundaries
Deciles £168 in the lowest decile to £1,638 in the highest policyengine.py decile impacts, recomputed to the penny

Each chart carries its own small print, as in v1.1.0. The map's tag also carries the ONS/OS licence statements verbatim, since the licence requires them wherever the boundaries are used.

Sanity checks against official figures (2026-09-27)

Sourced reports are in ~/reviews/pe30-sanity-2026-09-27/.

  • Cost holds up. £22.8bn of income tax forgone and a £22.1bn net cost sit within 5–10% of HMRC's ready reckoner scaled to +£2,430 (about £20–22.5bn), with the gap consistent with static vs behavioural costing and liabilities vs receipts.
  • The decile chart is internally consistent. The groups are tenths of people, not households (1.8m households in the top decile, 2.8m in the sixth); households with negative income (about 218k) are then left out, so the bottom decile holds 9.5% of people. Summed with their household counts they come to £22.04bn, plus those 218k households outside the deciles, against £22.13bn in total.
  • The poverty stat did not hold up as labelled (on 6.1.1). policyengine-uk 2.90.2's absolute line was the FYE 2011 one, which DWP replaced for recent years in March 2026. Its absolute child rates (11.8% BHC, 17.0% AHC) sat about 30% below DWP's projections for 2026-27 on the current line (17% and 24%). On relative AHC, the reform raised child poverty by about 120,000 on 6.1.1 (127,143 on 6.2.0). policyengine-uk 2.90.2, which policyengine.py 6.1.2 still pinned, also took the relative median over households rather than people (issue #1865, fixed by PR #1866, first released in 2.99.2). Resolved: the stat uses absolute poverty before housing costs (d509), on DWP's FYE 2025 line (d516), via policyengine.py 6.2.0. On 6.2.0 the model's absolute child rates are 16.3% BHC and 23.7% AHC, against DWP's 17% and 24% projections.
  • Some baselines are off. Households are 31.5m in the model vs 29.0m from ONS. State pension is £127.5bn vs about £153bn from GAD. Child benefit is £17.7bn vs £13.7bn from the OBR. Income tax is £312bn vs £347bn in HMRC's liabilities statistics. None of these is on screen except through "75.9% of households".

The poverty measure (ruled: d509)

The stat uses PolicyEngine's default measure, absolute poverty before housing costs (Max, 2026-09-28). Relative poverty measures inequality: the line is 60% of each run's own median, so this reform raises the line. After-housing-costs measures subtract discretionary housing spending. The default is now documented in PolicyEngine/policyengine.py#534.

Children under 18 Baseline Reform Change
Absolute, BHC (on screen) 16.32% 16.06% 39,996 fewer
Absolute, AHC 23.73% 23.16% 88,179 fewer
Relative, AHC 30.14% 30.97% 127,143 more
Relative, BHC 22.04% 22.77% 113,014 more

The absolute line is DWP's since March 2026 (ruled: d516): 60% of the FYE 2025 median, £431.69 a week before housing costs, uprated by CPI to £456.63 in 2026-27. policyengine-uk 2.102.3 carries it (PolicyEngine/policyengine-uk#1889), and policyengine.py 6.2.0 pins that (PolicyEngine/policyengine.py#536). On the old FYE 2011 line the stat was 59,900. The share gaining, income tax forgone and the family curve don't move at all; net cost moves by £0.36m (Universal Credit withdrawn), so it still reads £22.14bn; decile averages move by up to £8.

Survey data

The FRS is UK Data Service data under the End User Licence. The microdata and the map's record-level draws stay in data/uk/private/ (git-ignored); only aggregates (data/uk/national.json) are committed.

  • video.json names the sample file but doesn't contain it.
  • A clone without the file renders placeholder dots under the MOCK DATA banner. That is what CI renders.
  • A test asserts nothing under data/uk/private/ is tracked, and that national.json holds only aggregates.

Invariants (tested)

  • Accounting.
    • Family: at every plotted point, the change in net income = tax cut + ΔUC; 0 ≤ gain ≤ tax cut; NI unchanged; the gain never falls with earnings.
    • Nation: households' total gain = net Exchequer cost within 0.5% (the gap is £11.7m, £10.1m of it Housing Benefit, which is outside the programme list). policyengine.py's programme totals equal the script's own sums.
  • Reform took effect. The largest allowance the model applied moved from £12,570 to £15,000, and nobody's fell: 111,917 of 113,617 records gain the full £2,430, 120 on the taper gain part of it, and 1,580 with adjusted net income of £130,100 or more have none in either run. A 6 April start date would silently leave 2026-27 on current law; that was checked.
  • Traceability. Every £ figure and stat on screen is read from a computed or sourced number, at the precision shown. The rebuild is a no-op.
  • Differentials.
    • Family curve: policyengine.py 6.2.0 (policyengine-uk 2.102.3) vs 6.1.1 (2.90.2), and vs policyengine.py's household calculator.
    • Nation: decile groups rebuilt from their definition, independently of policyengine.py (all 52,846 households match), and decile averages recomputed from household rows.
    • K-weighting vs pyloudnorm.
  • Property-based.
    • Range rounding lands inside the computed flat stretch.
    • Geography simplification.
    • Sweep levelling: scale invariance, homogeneity, no look-ahead.
  • Determinism. Frames depend only on t, for both countries and both layouts.
  • Citation chip placement. The chip keeps its designed spot wherever no text overlaps it (the US film, both layouts: 602 of 602 frames identical to main) and otherwise moves to the nearest spot clear of text in every frame it shows (property tests in tests/util.test.js; a render fails if no spot exists).

Determinism check

On the Linux runner, one UK frame (landscape, t = 12.4 s) differed between the played-through and the fresh page. The difference was 56 pixels (0.011% of the frame), all in the blurred glow beside the curve's steep segment, at most 4 levels in any channel. Chromium's software rasterizer rounds a blur slightly differently on a partial repaint. The frames match byte for byte locally.

tools/determinism.mjs now reports a frame as "within noise" when no channel of any pixel differs by more than 8 levels and fewer than 0.05% of pixels differ; everything else must still match exactly. An injected one-pixel leftover-state bug still fails, at 4,140 to 7,484 pixels and up to 115 levels. On failure CI uploads both frames as an artifact.

Layout fix: the citation chip (UK only)

The UK statute and parameter file are denser than the US ones, so the "Income Tax Act 2007, s. 35" chip sat on text in both scenes: across "(1) An individual who…" above the boxed £12,570, and over line 19 of the YAML. It now takes the nearest spot clear of text in every frame it shows: beside the "35 Personal allowance" heading, and right of lines 17–18 (75% size in the vertical statute scene, where no full-size spot exists). The US film doesn't change.

Known and recorded

  • 12 records (0.06% of households) lose, through Pension Credit and Housing Benefit falling by more than their tax cut.
  • national.py stamps the uprated 2026 file with the versions that built it and rebuilds it when they change; ensure_datasets would otherwise reuse a file uprated by an older install.
  • The £500 rent is illustrative. tools/uk/rent_sensitivity.py shows how it moves where UC ends.
  • Free school meals and Council Tax Reduction for Manchester aren't modelled.

Details for every string: data/uk/README.md.

🤖 Generated with Claude Code

MaxGhenis and others added 5 commits September 27, 2026 17:55
…mily

?country=uk runs the same timeline on UK law, from data/uk/video.json.
The US film is unchanged pixel for pixel.

Real, and asserted by tools/uk/build_uk_video.py and tests/uk:
- The statute wall is Income Tax Act 2007 Part 3, Chapters 1-3, from
  legislation.gov.uk. The dive lands on s. 35(1), £12,570.
- The quote is Edward VI, 1551 ("…made more plaine and short…"), verbatim
  from Nichols (1857).
- The code panel shows policyengine-uk's personal_allowance/amount.yaml at
  412b25aa, byte-identical to the file the model ran.
- The reform raises the personal allowance to £15,000 in 2026-27.
  - The reform card shows the exact dict the model received, and each
    run asserts that the parameter moved.
- The family is a single parent in Manchester, two kids, housing
  association rent of £500 a month.
  - Gains: +£219 on Universal Credit from £15,000 to £48,000, and +£486
    off UC from £48,700 to £50,270. The pin shows +£972 at £60,000.
  - Every sentence holds at all 80,001 £1 points.
  - The results agree with policyengine-uk 2.102.0 and with
    policyengine.py's household calculator.
  - tools/uk/rent_sensitivity.py reruns the family at £400 and £600 rent.
- The map shows 650 July 2024 Westminster constituencies (ONS, OGL v3.0).
  The ONS and OS statements appear, verbatim, in the map's corner tag, as the boundary licence requires.

MOCK, bannered on every frame: the national figures, dots and deciles.
They stay MOCK until PolicyEngine has permission to process the UK survey
data with AI tools. No survey microdata is loaded:
uk_family.no_microdata() fails any run that tries.

Renderer changes:
- The country, currency and captions now come from the data file.
- An evenly spaced family layout is used for households other than the
  US one.
- Fonts in the code panel, reform card and captions shrink only when
  their text would overflow.
- In portrait, the chart moves down when the family description runs
  long.
- The UK map box ends higher in landscape so its source tag clears the
  south coast. The MOCK decile caption is shortened so it clears the chip.
- statText takes its currency as an option, so stats.map(statText) is
  still "$".

Tests:
- The UK tests run in `bun run test` and in CI.
- UK determinism is part of `bun run determinism`.
- The UK build report no longer carries a timestamp, and the geography
  build no longer records the Python patch version, so test runs leave the
  tree clean under any interpreter. (Recording the version made the
  round-trip test fail once uv moved from 3.14.4 to 3.14.7.)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Max ruled on d404 (9/27): "we can do it; no UKDS permission needed. Proceed with
the national figures."

tools/uk/national.py runs policyengine.py 6.1.1's UK flow on its certified
dataset, the enhanced FRS 2024-25 (policyengine-uk-data 1.56.16; the sha256
matches the release manifest), uprated to 2026. The reform sets a £15,000
personal allowance from 2026-01-01. A 6 April start silently leaves 2026-27 on
current law; that was checked. The run asserts every allowance moved from
£12,570 to £15,000 and none fell.

On screen:
- £22 billion net cost in 2026-27: £22.82bn of income tax forgone, less
  £0.60bn of Universal Credit and £0.08bn of Pension Credit withdrawn.
- 75.9% of households gain.
- 66,800 fewer children in poverty, on the absolute after-housing-costs
  measure. That is a fixed line, like the US SPM thresholds. The relative
  measures rise by 120,367 (AHC) and 136,071 (BHC) children, because
  policyengine-uk draws the relative line at 60% of each run's own median,
  which the reform raises. All four measures are recorded and documented.
- Decile averages: £168 in the lowest decile to £1,635 in the highest.
- Tags: the map's tag names the run, the poverty measure and the ONS
  licence; the decile tag names the run.

Checks (asserted in the run, then tested):
- Households' total gain equals the net Exchequer cost within 0.5%. The gap
  is £11m, which matches Housing Benefit falling outside the programme list.
- policyengine.py's programme totals agree with the script's own weighted
  sums.
- Decile averages recompute from the household rows to the penny.
- 12 records (0.06% of households) lose, through Pension Credit and Housing
  Benefit falling by more than their tax cut. This is recorded, not hidden.

The map's dots:
- 12,000 draws by household weight. Great Britain households carry
  constituency_code_oa in the dataset. Northern Ireland's (2.5%) carry none,
  so their dots fall anywhere in Northern Ireland, and the legend says so.
- The draws are record-level survey derivatives, so they stay in
  data/uk/private/ (git-ignored, with the microdata). video.json only names the
  file.
- A clone without the file renders placeholder dots under the MOCK DATA
  banner, which is what CI sees. A test asserts nothing under data/uk/private/
  is tracked and that national.json holds only aggregates.
- The UK landscape map box ends higher, so the four-line tag clears the south
  coast.

The US film is pixel-identical at every poster frame in both layouts.
Determinism passes for both countries.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
On a mismatch, tools/determinism.mjs writes both the played-through and the fresh
frame to frames/determinism/, and CI uploads them as an artifact. The UK
portrait frame at t=12.4 s differs on the Linux runner but matches locally, with
or without the private sample, so the frames are needed to see what differs.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…ounds

On the Linux runner, the UK landscape frame at t=12.4 s differed between the
played-through and the fresh page. The difference was 56 pixels (0.011% of the
frame), all in the blurred glow beside the curve's steep segment, at most 4
levels in any channel. Chromium's software rasterizer rounds a blur slightly
differently when it repaints only the damaged part of a page than when it paints
the page whole. No state carries between frames in our code, and the frames
match byte for byte locally.

A frame now passes as "within noise" when no channel of any pixel differs by
more than 8 levels and under 0.05% of pixels differ; it is reported as such, so
it stays visible. Everything else is still exact.

Checked both ways:
- The CI frames come out at 56 px and 4 levels, so they pass.
- An injected leftover state, where the chart shifts 1 px after the third
  render, fails at 4,140-7,484 px and up to 115 levels.

The comparison decodes both screenshots in a blank Chromium page, so the check
needs no new dependency.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…est harder

Two independent reviews (Astra, Opus) of PR 2 requested changes.

The US portrait film had changed. The flying "$2,200" landed at the measured
code font (21 px in portrait) instead of its original 22 px, so frames at
4.1-4.8 s differed. The US now keeps 22 px, and other countries follow the
panel.

New: tools/compare_revisions.mjs renders a film from this checkout and from
another every 0.1-0.2 s in both layouts and compares them.
- With the regression reinstated, it flags exactly those 8 portrait frames, at
  up to 204 levels.
- With the fix, all 602 US frames are identical to main.
- CI now runs it against the PR's base on every pull request, unless the PR
  carries the us-film-changes label. The determinism check compares a revision
  with itself and cannot catch this.
- tools/frame_compare.mjs holds the shared comparator (exact, or bounded
  rasterizer noise), and serve() can serve another checkout.

FRS citation (UK Data Service End User Licence, clause 11, which covers any
"printed, electronic or broadcast" publication):
- The film now carries the citation in exactly the form the catalogue entry
  for study 9563 specifies, under the decile chart, with "© Crown copyright".
- The READMEs carry it too, name the other surveys the enhancement imputes
  from, and note clause 12 (report published work to the UK Data Service).
- In landscape the UK decile chart ends 40 px higher so the citation fits.

Tests:
- The child poverty count and its direction now come from the baseline and
  reform headcounts; a flipped stored difference fails.
- Privacy: every tracked file under data/uk/ and audio/uk/ must be on a
  reviewed list, and no tracked JSON may hold survey-like rows (area code plus
  numbers, 100 or more). A companion test shows the scan catches a smuggled
  sample under any key and passes the map geometry.
- CI's placeholder dots now include Northern Ireland rows, so the region
  fallback runs in CI too.

README fixes:
- Allowances rose to at most £15,000 and fell for nobody. The 1,580 records
  with none in either run all have adjusted net income above £130,000 (checked).
- Housing Benefit explains £10.1m of the £11.4m identity gap, not all of it.
- Deciles are ranked by baseline household net income (not equivalised) into
  groups of equal numbers of people, then household-weighted. Read from
  policyengine.py.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
MaxGhenis and others added 3 commits September 28, 2026 08:35
Max's ruling on d509 (2026-09-28): PolicyEngine defaults to absolute poverty
before housing costs. Relative poverty measures inequality rather than
poverty. After-housing-costs measures subtract discretionary spending (a flat
or a mansion), which international poverty measurement does not do.

The UK stat now reads "59,900 fewer children in poverty": children in absolute
poverty before housing costs fall from 11.78% to 11.39% in 2026-27 (59,863,
shown to the hundred). The map tag reads "poverty: absolute, before housing
costs".

The UK README states the default and its reasons, and records all four
measures. It also documents the line itself: policyengine-uk uses HBAI's
FYE 2011 absolute line (£251.40 a week before housing costs, uprated by CPI).
DWP moved its absolute reference year to FYE 2025 in March 2026 (HBAI FYE
2025 report), so rates on the model's line sit below DWP's current figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…lute line

policyengine.py 6.2.0 pins policyengine-uk 2.102.3, which draws the absolute
poverty line as HBAI has since March 2026: 60% of the FYE 2025 median,
£431.69 a week before housing costs, uprated by CPI to £456.63 in 2026-27.
On that line, children in absolute poverty before housing costs are 16.32%
today and 16.06% under the reform: 39,996 fewer, shown as 40,000 (was 59,900
on the old FYE 2011 line). Net cost (£22bn) and the share gaining (75.9%) are
unchanged to the pound; decile averages move by up to £8 with the model's
baseline incomes.

- Every UK compute script now runs on requirements.pe-6.2.0.lock.txt.
- The family curve on 6.1.1 (policyengine-uk 2.90.2) becomes the
  differential, replacing the 2.102.0 run; it matches 6.2.0 to £0.00.
- national.py stamps the uprated 2026 file with the policyengine and
  policyengine-uk versions that built it and rebuilds it on a mismatch:
  ensure_datasets otherwise reuses a file uprated by another install.
- national.json records Housing Benefit's change, which explains most of
  the accounting-identity gap the README cites.
- The decile notes' velocities follow the new bar heights (events.json).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
…d from scratch

An adversarial review of 693efa3 (four lenses, each finding put to a
skeptic) confirmed these; all are fixed here.

- Citation chip (UK only, both layouts): the "Income Tax Act 2007, s. 35"
  chip sat on "(1) An individual who..." in the statute scene and over
  line 19 of the YAML in the code scene. placeChips() now keeps each chip
  at its designed spot unless text overlaps it there, else moves it to the
  nearest spot clear of text in every frame it shows (smaller only if no
  full-size spot exists: 75% in the vertical statute scene). Solved once
  in measure(), so frames stay history-free. The US film is unchanged:
  602 of 602 frames identical to main in both layouts. clearOffset() lives
  in site/util.js with fast-check properties (design spot kept when clear;
  a move clears every obstacle in every frame; the move is the shortest
  the grid allows at the largest scale that allows one; stuck iff nothing
  clears), each shown to catch a planted bug. render.mjs refuses to render
  a chip with no clear spot.
- Deciles: national.py now rebuilds the groups from their definition
  (people-weighted rank of baseline household net income, ties sharing a
  rank, negatives left out) instead of reusing policyengine.py's; all
  52,846 households match. The README now says the 460 negative-income
  records are dropped after ranking, so the bottom decile holds 9.5% of
  people, not "equal numbers".
- Allowance: national.json records who has no allowance (1,580 records,
  lowest adjusted net income £130,100) and who gains the full £2,430
  (111,917) or part of it (120). The docstring no longer claims every
  person's allowance moved from £12,570 to £15,000.
- The uprated-dataset stamp now carries the file's sha256 and goes with
  the file, so a file rewritten by anything else is rebuilt.
- README: policyengine-uk 2.90.2's old line was £251.40 through 2019 and
  £305.70 from 2020, CPI-uprated to £403.42 in 2026-27.

Correction to 693efa3's message: net cost is not "unchanged to the pound".
It rises £360,042.55 (Universal Credit withdrawn) and still reads £22.14bn;
income tax forgone and the share gaining are identical.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant