perf(dem): the planner terrain fills its tiles on every core and decodes two rows of GLO-30 squares at a time

[timohueser]

Summary

planner/terrain now fills its tiles on every core and keeps at most two rows of GLO-30 squares decoded. Before, it ran on one thread and decoded all squares at the start.

• One sweep. terrarium::write goes from north to south over the pixel rows of all 13 zooms. The northmost pending pixel row selects the squares to decode, and the sweep drops all other squares. No pending row needs a square north of the squares in memory, so each square decodes once.
• Every core. The tiles of one row fill in parallel (rayon), and a finished row encodes in parallel. The squares of a new row decode in parallel.
• Same bytes. The finished tiles wait in a spool file beside the output. The file is unlinked when it opens, so it goes away with the process. The MBTiles takes the tiles zoom by zoom, in the old row and column order, with the old transactions.
• Disk. The spool holds every tile until the MBTiles takes it, so the step needs about twice the size of terrain.mbtiles on the disk of its output for a short time: about 10.7 GB for DACH.
• Progress. The sweep prints the latitude of each row of squares that it decodes.
• No whole-file reads. Terrain::open hashes the GLO-30 files with the streamed hash_file of the store, in parallel, and decodes nothing. It checks the header of each file with DemTile::check, so a file that is not a DEM tile stops the step before the sweep, and the route build before its OSM import.
• Debug check. Terrain::height asserts in debug builds that the squares of its latitude are decoded.
• API. Terrain::load(lats) decodes the squares of a latitude range and drops the others. Terrain::height reads through &self with a per-caller Recent cache of reference tiles. DemMosaic keeps its lookup memo per thread, so threads can share one mosaic. The memo cannot change an answer. tiles_in and step::glo30 return the TileId of each file.
• Route build. planner-router-build calls load for all squares, so its memory and output do not change.
• rayon. rayon is optional under geotiff, because the parallel work is the GLO-30 decode. The workspace has no rayon entry; obc-bake, obc-draw and obc-pbf also declare rayon = "1".

Why

On the DACH bake of 2026-10-17 (runs/2026-10-17-071235), planner/terrain used 592 s of CPU in 604 s of wall time for 34,271 tiles. Its peak was 6.75 GB: 136 GLO-30 squares decoded as f32 at the same time.

Estimate, not measured on DACH. CPU time stays at about 592 s, so 12 cores give about 60 s. Two rows of about 13 squares are about 1.4 GB, and the open tile rows of all zooms about 0.3 GB.

This is the terrain part of the bake system review.

Tests

Byte check. The develop binary and this binary baked the same boxes from GLO-30 squares of the store. The archives have the same SHA-256. The new binary ran with RAYON_NUM_THREADS=4.

Box	Squares	Tiles	SHA-256	Old: wall, peak	New: wall, peak
8.995,46.995,9.005,47.005 (corner of 4 squares)	4	13	d0d5ea65…	3.3 s, 249 MB	0.9 s, 293 MB
8.9,46.9,9.1,47.1	4	33	eb27ce0b…	1.9 s, 250 MB	1.1 s, 289 MB
8.5,46.5,9.5,47.5	4	321	2ff449d3…	6.6 s, 250 MB	2.7 s, 326 MB
8.9,45.5,9.1,49.5 (5 rows of squares)	10	322	e95b7a28…	9.9 s, 546 MB	6.7 s, 293 MB

With one thread, the 321-tile box used 6.3 s to 6.7 s of CPU in both binaries.

• Band test. tests/planner.rs writes the last posts of N46E008 and the first of N47E008 at the GLO-30 posting. From north to south across 47° N, a terrain that decodes only squares(lat) gives the same height as a terrain that decoded all squares. With no slack in squares, the test fails at 47.000269° N.
• After the review fixes, the corner box and the 5-row box give the same SHA-256 again (d0d5ea65…, e95b7a28…).
• obc test -p obc-dem: 49 passed, 2 skipped (fixture tests). obc suites check: OK.
• planner_terrain fixture test (--features external-fixtures,terrarium), with the assistant-terrain square linked from the store (same SHA-256): passed.
• obc test -p planner-router-build: 65 passed.
• cargo clippy -D warnings on obc-dem (default and terrarium features) and planner-router-build: clean.
• check_dependencies.py and prose.py --check: pass. Not run: gen-third-party.sh --check (no cargo-about here); rayon is already in the lockfile.

Not run: a real terrain bake, because the machine runs other work.

Public docs: no change.

Requirements: none

🤖 Generated with Claude Code

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: e12f7bff-6870-4d62-a2b0-8d19d62ecd5e

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}