Skip to content

perf(planner): the snow bake skips HR-WSI files that miss a chunk - #2730

Merged
timohueser merged 1 commit into
developfrom
claude/snow-footprint-cache
Oct 17, 2026
Merged

timohueser merged 1 commit into
developfrom
claude/snow-footprint-cache

Conversation

@timohueser

Copy link
Copy Markdown
Owner

Summary

The planner/snow step bakes one zoom-9 chunk at a time. Each chunk opened every HR-WSI file of the region and called transform_bounds before it tested the overlap. transform_bounds sets up a new transform on each call and holds the GIL, so the 4 worker threads did not overlap.

Now laea_footprint gives the EPSG:3035 bounds of each file once per bake. A chunk skips a file that its extent does not touch, before it opens the file. The exact window test for the remaining files does not change.

Measurement

Build: f0d982ee2, on the DACH hr-wsi snapshot 2026-10-16 (2499 files, seasons 2016–2024), one thread, with another bake on the same machine.

Chunk (z9) Before After, first chunk After, cached Planes equal
268/178 73.0 s 52.5 s 11.6 s yes
272/182 73.1 s 9.0 s 8.9 s yes

transform_bounds costs 26.5 ms per call with 1 thread and 26.8 ms with 4 threads. The running DACH bake does about 43 chunks per hour.

Checks

  • python.repository-tools (331 tests): pass.
  • check_data_sources.py, check_fixture_policy.py, prose.py --check, check_dependencies.py: pass.
  • Not run locally: obc-data-steps nextest, obc check fmt and the iOS host check. They do not cover this Python change, and a DACH bake used the shared cargo target. CI runs them.

Public docs: not changed.

Requirements: none

🤖 Generated with Claude Code

…ore it opens them

Each chunk tested every HR-WSI file of the region with `transform_bounds`, which holds the GIL.
The footprint of each file in EPSG:3035 is now computed once per bake.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Oct 17, 2026

Copy link
Copy Markdown

Important

  • 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration
  • Configuration used: defaults
  • Review profile: CHILL
  • Plan: Advanced
  • Run ID: 983db8ac-8ac8-42e3-b67c-8661b9958c84
  • Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@timohueser
timohueser merged commit c801eec into develop Oct 17, 2026
26 checks passed
@timohueser
timohueser deleted the claude/snow-footprint-cache branch October 17, 2026 21:53
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant