Repository navigation
Conversation
_create_spatial_mask built a (n_track_points, nlat, nlon) great-circle distance field per timestep and find_contours_from_point_specified_field contoured the whole (global) field per peak per field. Both results are local by construction: a track point only marks a disc of max_distance_degrees, and a contour is only ever accepted when its furthest point lies within max_gc_distance_*_degrees of the peak. Restrict both searches to a conservative window (Clairaut's relation bounds the longitude sweep of a geodesic of length r to r/cos(phi_max)). Degenerate grids and windows that would cross the antimeridian fall back to the whole-field path, so results are identical. On a synthetic global 0.25 degree field (benchmarks/tc_detection_profile.py): spatial mask 504 ms -> 3 ms, peak search 760 ms -> 24 ms per timestep, i.e. a 20-init x 120-lead forecast eval drops from ~30 min to ~1 min of peak search. tests/test_tropical_cyclone_locality.py pins the equivalence against reference implementations of the previous whole-field code (random point sets, wrapped and ascending grids, dateline and pole cases, contour-level sweeps, and an end-to-end _find_peaks_for_time_slice peak-set comparison). Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
The windows size themselves from the first cell spacing, so a non-uniform grid (or a NaN in a coordinate) could make the latitude window too small or the longitude half-width NaN: the mask silently dropped cells that the whole-field code marked, and the contourer raised on the NaN spacing. Guard both sites with a uniform-spacing check and keep the previous whole-field path for such grids; add regression tests. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Collaborator
Author
|
This PR update is from an optimization that Claude + Qwen3.8 Flash Next landed on while setting up an EWB near-real-time dashboard. I'm running the dashboard and the EWB code on a local machine with 512GB RAM total (256GB dedicated to any EWB runs) and 12 cores/24 threads, so not incredibly large. As far as I can tell, this is a solid improvement. Not going to merge it just yet, want to double check if any edge cases get caught. |
tropical_cyclone.py, not the whole globe
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Two array-side searches in the tropical-cyclone detector scan the whole (global) field to produce results that are local by construction:
_create_spatial_maskbuilds a(n_track_points, nlat, nlon)great-circle distance field — chunked 32 points at a time (thanks to the earlier vectorisation) — only to mark the union of discs of radiusmax_distance_degrees. At 0.25° resolution a 5° disc is ~0.2 % of the globe, so ~99 % of that temporary always compares False; it was the largest temporary in detection (~266 MB float64 per 32-row chunk, plus itshaversineintermediates).find_contours_from_point_specified_fieldsubtracts the point value from the entire field and contours the entire field, once per peak per field (SLP and thickness).find_valid_candidatesthen rejects any contour whose furthest point is more thanmax_gc_distance_slp_contour_degrees/max_gc_distance_dz_contour_degreesfrom the peak — so every contour that can be accepted lies inside a ≤ 6.5° box around the peak.This PR restricts both searches to a conservative window around the point. The window comes from Clairaut's relation: a geodesic of length ≤ r between two points no farther from the equator than φ_max has
|Δlon| ≤ r / cos(φ_max);cosis smallest at the highest |latitude| inside the latitude window, so taking it there cannot clip a point that is in range (and correctly opens to the whole globe at a point near a pole). Latitude uses the exact bound|Δlat| ≤ r. A contour clipped by a window edge extends farther than any contour that can pass the tracker's distance test, so the accept/reject decision cannot change; contours are still returned in full-field index space andfind_contours_from_point_specified_fieldkeeps its old 3-argument behaviour whensearch_radius_degreesis omitted, so no caller changes.The whole-field path is kept as an unconditional fallback: a window that would cross the antimeridian (contour path; the mask wraps instead), a grid with fewer than two coordinates per axis, a non-uniform or non-finite coordinate array, or a degenerate spacing all keep the previous whole-field computation, so results are identical for those grids too.
Type of change
No new dependencies.
Benchmarks
Measured with a profiling script kept outside the repo (gist) on a synthetic-but-realistic global 0.25° field (721 × 1440, 40 localised closed lows, 12 IBTrACS-like rows per valid time,
surface_pressure_threshold = 101 000 Pa); no network or data download.Per-step pricing (8-core machine, 24 steps):
_create_spatial_mask(12 rows)_find_peaks_for_time_sliceReal 2400-step peak-search loop (the array work of a 20-init × 120-lead forecast evaluation, 16-core machine, one run each):
After the change the per-step profile is flat (windowed contouring, the
peak_local_maxmax-filter, the per-peakDataArrayconstruction) — there is no second hot spot worth a PR here.To reproduce the timings, run the gist from a checkout on v1.1.1 and on this branch:
To run the tests:
How Has This Been Tested?
tests/test_tropical_cyclone_locality.py(72 cases, ~25 s, seeded, no network or data files) pins the behaviour against reference implementations of the replaced whole-field code:_create_spatial_maskexactly equals the whole-field mask over random point sets × {0.25°, 1°} grids × {1, 5, 12}° radii × {1, 7, 40} points; explicitly for discs straddling the dateline and sitting on the poles; on 0–360/descending, −180–180/ascending and descending-longitude grids; and on non-uniform and NaN-coordinate grids (fallback path).atol=1e-9); windowed and global contour sets produce identicalfind_valid_candidatesaccept/reject decisions for every minimum of the fixture at several levels._find_peaks_for_time_slicereturns the same peak set with the window as when the window is patched out; the fixture is asserted non-vacuous so the comparison cannot pass trivially.Existing suite:
tests/test_tropical_cyclone.py(43 cases) and the rest oftests/pass unchanged;pre-commit(ruff, ruff-format, mypy) passes.Notes for the reviewer
max_gc_distance_*_degrees+ a 1° marching-squares margin); nothing new is tunable.find_valid_candidatesmeasuring the contour-to-peak distance from the contour's furthest point in index space; every contour that can reach a window edge is farther from the peak than any contour that can pass that test._find_peaks_for_time_slicerebuilds twoxr.DataArrays per time slice per peak (the next-largest line in the profile), andcalc.haversine_distancere-evaluatesnp.radians/np.coson both operands every call; neither can be hoisted as long as the coordinate arrays are rebuilt per slice.🤖 Generated with Claude Code