Skip to content

Extend LA council tax to all eight bands (MHCLG Table 9, 2026-27) - #482

Open
vahid-ahmadi wants to merge 1 commit into
mainfrom
la-council-tax-all-bands
Open

vahid-ahmadi wants to merge 1 commit into
mainfrom
la-council-tax-all-bands

Conversation

@vahid-ahmadi

@vahid-ahmadi vahid-ahmadi commented Sep 15, 2026 •

Copy link
Copy Markdown
Collaborator

What changed

  • storage/la_council_tax.csv: adds eight columns, band_A_amount … band_H_amount, populated for all 296 English billing authorities. All pre-existing columns and all 360 rows are untouched — verified cell-by-cell against HEAD (0 rows of drift in the original 16 columns).
  • storage/build_la_council_tax.py (new): reproducible builder. Downloads the MHCLG workbook, parses Table_9 from the ODS with zipfile + xml.etree (honouring table:number-columns-repeated; no odfpy/pandas ODS engine needed), re-maps the post-2023 South Yorkshire ONS codes, and rewrites only the new columns. Every other cell is round-tripped as its literal text, so a re-run is a no-op (md5-identical on the second run).
  • targets/sources/la_council_tax.py: module docstring only — documents the new columns, their England-only provenance, and the rebuild script. No behaviour change; get_targets() output is unchanged.
  • tests/test_la_council_tax_targets.py: seven new tests (see below).
  • Changelog fragment changelog.d/la-council-tax-all-bands.added.md.

There is no existing builder directory in this repo — la_council_tax.csv was hand-built in #374 with no committed script. I put the builder in policyengine_uk_data/storage/, alongside download_private_prerequisites.py / upload_completed_datasets.py, as the closest existing pattern. Happy to move it if you'd rather it lived elsewhere.

Data source

MHCLG, Council tax levels set by local authorities in England: 2026 to 2027
https://www.gov.uk/government/statistics/council-tax-levels-set-by-local-authorities-in-england-2026-to-2027

Workbook: https://assets.publishing.service.gov.uk/media/69de1fa63e81003ae0422508/Tables_1-9_2026-27.ods, sheet Table_9 — "Area council tax for a dwelling occupied by two adults by band, 2026 to 2027". 296 English authority rows, bands A–H, inclusive of all precepts.

England only. Table 9 has no Welsh or Scottish coverage, so the 22 Welsh, 32 Scottish and 10 NI rows leave all eight new columns blank rather than inheriting an English figure. Wales additionally has a Band I and Scotland has used non-statutory band ratios since the 2017 reform, so neither could be back-filled from Band D anyway. A test enforces the blanks.

Band-ratio verification

Checked every one of the 296 authorities against the statutory ratios (A=6/9, B=7/9, C=8/9, D=1, E=11/9, F=13/9, G=15/9, H=18/9 of Band D). All 296 obey the ratios. Worst absolute deviation by band:

Band Max deviation Authorities over £0.01
A £0.0033 0
B £0.0044 0
C £0.0044 0
D £0.0000 0
E £0.0044 0
F £0.0044 0
G £0.0033 0
H £0.0000 0

That is pure independent-to-the-penny rounding (at most half a penny per band). No authority deviates and nothing in the source was altered. The test tolerance is £0.01, comfortably above the observed £0.0044 worst case.

Year attribution (step 3) — no change was needed

I checked before touching anything: band_d_amount for England already holds the 2026-27 Table 9 Band D figure, exactly, for all 294 English LAs whose ONS code matches directly (max difference £0.00); the remaining two are the South Yorkshire code re-map below, which also match. _YEAR_BAND_D_ENGLAND is already 2026 and _ENGLAND_REF already points at the 2026-27 release.

So _year_for_band_d, _ref_for_band_d and the reference URL are unchanged, and no target's year attribution moved. The builder still writes band_d_amount from Table 9 so the file stays reproducible, but on today's data that write is a no-op. The new band_D_amount column duplicates band_d_amount for England by construction; a test asserts they agree exactly.

South Yorkshire code re-map

MHCLG uses the post-2023 codes E08000038 (Barnsley) and E08000039 (Sheffield); local_authorities_2021.csv uses E08000016 / E08000019. The builder re-maps them, confirmed by authority name. Their Band D values also matched the existing CSV exactly.

Tests

Added to test_la_council_tax_targets.py:

  • row count is still 360;
  • ONS codes are unique;
  • all eight band_*_amount columns exist;
  • all 296 England rows have all eight amounts non-null;
  • all 64 non-England rows have all eight blank;
  • statutory band ratios hold within £0.01;
  • band_D_amount == band_d_amount for England.
$ uv run --python 3.13 pytest -q -m "not slow"
528 passed, 108 skipped, 5 deselected, 1 xfailed, 3 warnings in 97.38s

uv run ruff format . reformatted my two files (now clean). uv run ruff check . reports 61 errors — the identical count with my changes stashed, i.e. entirely pre-existing on origin/main (verified by stashing and re-running, not assumed). None are in code this PR adds.

What I could NOT verify / gaps

  • No target is emitted from the new columns. get_targets() is unchanged, so the band-amount columns are currently data-only, for downstream band-level work. Wiring them into the loss matrix would ripple into test_la_loss_council_tax.py and felt out of scope here — flagging as follow-up rather than doing it silently.
  • The workbook URL is hard-coded to the current assets.publishing.service.gov.uk media path rather than scraped from the release page (as voa_council_tax.py does). It will need updating for 2027-28. Scraping would be more robust; say the word and I'll switch it.
  • I did not cross-check Table 9 against Table 8a (average council tax per authority), so I can't independently confirm Table 9's consistency with the rest of the release beyond the ratio check above.
  • Wales and Scotland remain Band D only. Extending them needs the Welsh Government and Scottish Government publications — separate sources, separate piece of work.

🤖 Generated with Claude Code

Adds band_A_amount..band_H_amount to storage/la_council_tax.csv for the
296 English billing authorities, taken from MHCLG "Council tax levels
set by local authorities in England 2026-27", Tables_1-9_2026-27.ods,
sheet Table_9 ("Area council tax for a dwelling occupied by two adults
by band"). The columns are England only: Table 9 does not cover Wales
or Scotland, so those rows stay blank, and NI has no council tax.

Adds storage/build_la_council_tax.py, which downloads the workbook,
parses the ODS with zipfile plus ElementTree (no odfpy dependency),
re-maps the post-2023 South Yorkshire ONS codes to the pre-2023 codes
used by local_authorities_2021.csv, and rewrites only the new columns.
Every pre-existing cell is round-tripped as its literal text, so the
build is idempotent.

band_d_amount already held the 2026-27 Table 9 Band D figure for all
294 directly-matching English LAs, and _YEAR_BAND_D_ENGLAND / the
England reference URL already point at 2026-27, so no year attribution
changed.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

@juaristi22 juaristi22 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approving as a data-only change. The useful result is the ratio check: all 296 authorities are exact ninths of Band D to within half a penny, so the eight columns carry nothing beyond Band D and the statutory ratios. That settles how Microcosm treats them: no port and no Chronicle ask, since our route is the published Band D fact times the ratio parameter, and, as #484 says, none of these amounts may ever become an additive target.

Three nits, none blocking:

  • band_D_amount duplicates band_d_amount for England by construction. The test guards the pair, but one column is simpler and cannot desync on a refresh.
  • The builder rewrites band_d_amount for England from Table 9 whenever it differs by more than half a penny. That is a no-op today, but on a future refresh it silently moves that column's provenance from Table 10 to Table 9 without a log line. Print the count of rewritten Band D cells.
  • The hard-coded asset URL is fine for now; scraping the release page the way voa_council_tax.py does can wait for the 2027-28 refresh.

Once this and #484 both land, we re-pin the Microcosm inventory sha256s for the loader and the test file.

🤖 Generated with Claude Code

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants