[timohueser]

Summary

tools/planner_climate.py bakes each spatial chunk in a worker process. Before, the chunks ran one after the other on one core.

• Pool. tools/step_pool.py now holds the pool that the snow bake used: one worker per core that the step may use (sched_getaffinity) and at most one per GB of free memory, workers that end with the bake (PR_SET_PDEATHSIG), and threads off Linux, where a fork is not safe. The snow bake uses the same module. The climate pool has at most one worker per spatial chunk.
• No copies. The workers fork from the bake, so they read the region without a copy.
• Same bytes. Each chunk fills its own cells of the region grid, so the order in which the chunks end does not change the archive. The layer revision does not change.
• Chunks are hashed once, in the workers. The parent resolves the path of each chunk once, puts the SHA-256 of its name into the fingerprint, and gives the path to the worker. The worker hashes the bytes it reads against that name, so a damaged cached chunk stops the bake, as before. Develop read and hashed each chunk in one thread.
• Downloads. download keeps its 8 threads and now returns the paths.
• Triggers. tools/step_pool.py triggers the obc-data-steps tests, as tools/step_request.py does.

Why

On the DACH bake of 2026-10-17 (runs/2026-10-17-071235), planner/climate used 281 s of CPU in 259 s of wall time on one core, for about 400 independent spatial chunks.

Estimate, not measured on DACH. On 12 cores with enough free memory, about 259 / 12 s: under 30 s.

This is the climate part of the bake system review.

Memory note

Each worker holds the hourly series of one chunk, at most 32 cells for the ten years. A review measured 277 MB per full-chunk worker. The parent holds the region grids, as before.

Tests

• Byte check. The develop code and this code baked the same synthetic input (FakeSource of the tests) for the box 8.0,48.0,9.0,48.5: 9 spatial chunks. The archives have the same SHA-256, 0963e63f…, with 2 tiles at zoom 8 and 4 at zoom 9.
• Parent killed. A bake with 3 workers: after kill -9 of the parent, no worker was alive 2 s later.
• tools/tests suite, as CI runs it (--group dev --group planner-snow --group planner-sun): 331 tests OK. test_planner_climate and test_planner_snow also pass with -W error::DeprecationWarning.
• obc suites check: OK.
• SourceTest calls path and then read(path); its "damaged" case stays. FakeSource records its digests in path, because read runs in a worker.

Not run: a real climate bake, because the machine runs other work.

Public docs: no change.

Requirements: none

🤖 Generated with Claude Code

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: defaults
• Review profile: CHILL
• Plan: Advanced
• Run ID: 079b18db-0277-4a88-8f88-9d685177d983

• Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands.}