perf(bake): one device/map step per leaf ingests the OSM once and cuts every band

[timohueser]

Summary

One step device/map/<i>-<j> replaces the four band steps device/coarse|mid|fine|network/<i>-<j> of a leaf. The step decodes the OSM of the leaf once and cuts every band with obc_bake::cut::cut_ingested, the cutter that is already the byte oracle of the cells. The function is obc-bake/map in host/obc-data-steps/src/functions.rs.

• One ingest per leaf. The four band steps decoded the leaf PBF four times, and each built the node map. Now one decode feeds all bands. The landmarks step still reads the PBF on its own.
• Order for a small peak. The step cuts the network band first and then drops the routable ways. Then it adds land and contours once, cuts coarse and mid with one semantic ladder, drops the ladder, and cuts fine. Before, mid and fine each traced the contours, and coarse and mid each built the semantic ladder.
• Level-outer cutter with a spill. cut_ingested prepares one level set, cuts and encodes it for every cell of the band in parallel, and drops the set before the next level. A merged set stays in the memo until the last level that reads it. Each cell appends its encoded level to a spill file below <out>/.levels/. It reads the file back when the cell is written, and a successful cut removes the directory. Before, a band kept all its level sets until the last cell: that is the fine-band peak that killed device/fine/0037-0033.
• serialize_lods_streaming takes encoded levels (obc_draw::serialize::Region). The monolithic pipeline encodes each tree in its closure, so it still holds one tree at a time.
• Deleted obc-draw/src/step.rs and obc-network/src/step.rs. Job and SourceFiles moved from obc-map-core into the new obc-bake/src/step.rs. obc-network no longer depends on obc-data, and obc-data-steps no longer depends on obc-draw or obc-network.
• Empty lists beside the cells. The step writes cells/<band>/empty.json, as the spec states, and no metadata/. The catalog rewrites metadata/ only for terrain layers. A band with no cells is an error, because the cutter reads an empty selection as every cell of the leaf.

The options of the step are leaf, cells (one list per band, in band-table order) and partial_cells (one list for all bands: partial coverage depends on the square only).

A stop needs no check in the step: since #2743, a stop sends SIGTERM to the process group of the step.

Evidence

Bytes

cells_fixture runs the real step on all four bands of the fixture leaf (Grimsel, real cells.osm.pbf, land, GLO-30 terrain, builder/presets/schema.json). It asserts the SHA-256 of each cell as the four band steps of d0c0650c2 wrote it:

Band	Cell	Size	SHA-256
coarse	0300/0264	4841	102e30645708264e…
mid	0600/0528	124249	eb0668a70925b83d…
fine	1201/1056	425993	eb62f326018cb804…
network	1201/1056	6233	33bc91069aac63b0…

Fine and network are also compared with one cut of the leaf. The cells of the cell_cut fixture (cut_ingested, three test bands) are identical before and after.

Peak memory on the fixture leaf

Each step ran alone in a fresh test process (debug, nice -n 19): the build recorded the request of each map step, and a replay ran it and read VmHWM. The process starts at 15 MB.

Code	Step	Peak RSS	Time
d0c0650c2	device/coarse	303 MB	288 s
d0c0650c2	device/mid	369 MB	302 s
d0c0650c2	device/fine	93 MB	2.4 s
d0c0650c2	device/network	21 MB	0.0 s
this branch	device/map (all four bands)	370 MB	339 s

On this leaf the semantic ladder sets the peak, in old mid and in the new step alike. The leaf has one small cell per band, so it does not show the fine-set peak, and the drawing features that the network phase now holds beside the ways are small here.

Size of the encoded bands in the DACH run

The receipts of run 2026-10-17-071235 give the bytes that the spill now keeps on disk:

Leaf	Band	Cells	Bytes	Largest cell
0037-0032	fine	115	726 MB	14.0 MB
0037-0032	mid	35	34 MB	1.7 MB
0037-0033	mid	111	132 MB	2.4 MB
0037-0033	coarse	30	10.5 MB	0.6 MB

device/fine/0037-0033 has no receipt (it was killed). At the density of 0037-0032 its ~420 fine cells write about 3 GB. In memory, that would sit beside the features, land and contours, so the cutter spills it. With the spill, a worker holds one cell's levels while it writes the cell (the largest cell is 14 MB).

Not measured: the peak of a real leaf. No real bake ran (shared PC).

Tests

• cells_fixture is the byte-equality test (the table above). It now cuts the leaf once: the malformed-network stand-in copies the built cells and truncates one. It takes about 5 min in debug, because coarse and mid build the whole-leaf semantic ladder.
• obc-bake step::tests: a partial empty cell keeps its bytes, and a covered empty cell is only in cells/<band>/empty.json.

Checks run on the branch merged with develop (after #2743 and #2745), against this worktree's own target: cargo nextest run for obc-map-core, obc-draw, obc-network, obc-pack, obc-bake, obc-data and obc-data-steps with external-fixtures (717 of 718 pass), doctests of obc-bake and obc-data-steps, cargo clippy --all-targets -D warnings on those seven crates, cargo fmt --all --check, obc suites check.

Not passing locally: planner_build stops at Cannot find module '@mapbox/vector-tile', because this worktree has no builder/web node modules. This change does not touch the planner build.

Public docs: unchanged. The contract specs/obc-data.md changes with the code: the device/map row and step list, where a map layer lists its empty cells, the metadata/ files of the terrain row (as the code writes them), and the network list that the landmarks read.

Note for the first bake after merge: no device/map step has a receipt of its own name yet. The admission rule of #2743 reads receipts of the same layer, and the old band receipts satisfy it.

Requirements: none

🤖 Generated with Claude Code

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 456a3f59-bdbd-4ee4-b42f-b04f53dd8164

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}