Skip to content

Feature/fix - #436

Open
PraveenKumar-89 wants to merge 21 commits into
brightbandtech:mainfrom
RAI-SCC:main
Open

PraveenKumar-89 wants to merge 21 commits into
brightbandtech:mainfrom
RAI-SCC:main

Conversation

@PraveenKumar-89

@PraveenKumar-89 PraveenKumar-89 commented Oct 19, 2026 •

Copy link
Copy Markdown

EWB Pull Request: Advanced Tropical Cyclone Tracking, Kalman Filtering & Multi-Metric Evaluation Suite

Overview

  • feat: Add probabilistic evaluation metrics and YAML case loading utilities.
  • feat: Implement IBTrACS local caching and add time conversion utility.

Detailed Summary of Changes

1. Probabilistic Metrics & Utilities

Added a comprehensive suite of ensemble and probabilistic evaluation metrics:

  • SPEDS (Spatial Probabilistic Extreme Dynamics Score — specifically designed to solve the double penalty problem by evaluating spatial tolerance, sharpness, and physical gradient fidelity)
  • NLL (Negative Log-Likelihood)
  • CRPS (Continuous Ranked Probability Score)
  • SpatialFSS (Fractions Skill Score)
  • CoverOneSD (Coverage within 1 standard deviation)
  • HeatWave detection (Hit Rate and False Alarm Rate)

Added utility functions in ewb.cases.py:

  • load_ewb_events_yaml_into_case_list using importlib.resources for clean asset loading.
  • read_incoming_yaml helper function for safe YAML file parsing.

Enhanced ExtremeWeatherBench runner:

  • Added selected_ensemble parameter support in ewb.evaluate.ExtremeWeatherBench.

2. IBTrACS Local Caching & Time Conversion

Previously, IBTrACS data was fetched directly from the internet during every pipeline execution, causing network overhead. This update introduces a local caching mechanism:

  • Checks ~/.cache/extremeweatherbench/ for an existing dataset.
  • Implements file size validation (> 1MB) to prevent loading incomplete or corrupted downloads.
  • Falls back to urllib.request.urlretrieve if the cache is missing or invalid.
  • Switches polars.scan_csv to read from the local disk instead of the URL.

Utility updates (utils.py):

  • Adds convert_init_and_valid_time_to_lead_time to support calculating lead times dynamically from initialization and valid time coordinates.

3. Core & Utility Improvements (src/extremeweatherbench/)

  • utils.py (convert_valid_time_to_init_time):

    • Problem in existing implementation: The current join="outer" approach introduces spurious/ghost dates and dimension alignment issues when shifting valid_time to init_time across different lead times. Also, lead-time dtype conversion and potential init_time coordinate collisions weren't explicitly handled.
    • Solution in this update:
      • Switched from join="outer" to join="inner" to strictly retain only the true forecast initialization times common to all lead times.
      • Added explicit checks to drop existing init_time coordinates and prevent name collisions.
      • Improved robust handling of lead-time timedelta conversions (supporting both hours and nanosecond formats).
  • derived.py & tropical_cyclone.py:

    • Extracted zonal (u_wind) and meridional (v_wind) wind components directly from forecast datasets to provide realistic environmental steering velocities for trajectory tracking.

* feat: add probabilistic evaluation metrics and YAML case loading utilities

Added a comprehensive suite of ensemble and probabilistic evaluation metrics:
- SPEDS (Spatial Probabilistic Extreme Dynamics Score, refactored to NumPy/SciPy)
- NLL (Negative Log-Likelihood)
- CRPS (Continuous Ranked Probability Score)
- SpatialFSS (Fractions Skill Score)
- CoverOneSD (Coverage within 1 standard deviation)
- HeatWave detection (Hit Rate and False Alarm Rate)

Added utility functions in ewb.cases.py:
- load_ewb_events_yaml_into_case_list using importlib.resources for clean asset loading
- read_incoming_yaml helper function for safe YAML file parsing

Enhanced ExtremeWeatherBench runner:
- Added selected_ensemble parameter support in ewb.evaluate.ExtremeWeatherBench

* feat: implement IBTrACS local caching and add time conversion utility

Previously, the IBTrACS data was fetched directly from the internet during every pipeline execution, causing significant network overhead and slowing down repetitive evaluations.

This commit introduces a local caching mechanism in inputs:
- Checks ~/.cache/extremeweatherbench/ for an existing dataset.
- Implements a file size validation (> 1MB) to prevent loading incomplete or corrupted downloads.
- Falls back to urllib.request.urlretrieve to download and save the file locally if the cache is missing or invalid.
- Switches polars.scan_csv to read from the local disk instead of the URL.

Additionally, this commit updates the utils module:
- Adds convert_init_and_valid_time_to_lead_time to support calculating lead times dynamically from initialization and valid time coordinates.

These changes drastically improve data loading speeds, provide resilience against temporary network failures, and enhance temporal alignment capabilities.
@PraveenKumar-89 PraveenKumar-89 changed the title Feature/fix (#1) Feature/fix Oct 19, 2026
PraveenKumar-89 and others added 20 commits October 20, 2026 14:39
* feat: add probabilistic evaluation metrics and YAML case loading utilities

Added a comprehensive suite of ensemble and probabilistic evaluation metrics:
- SPEDS (Spatial Probabilistic Extreme Dynamics Score, refactored to NumPy/SciPy)
- NLL (Negative Log-Likelihood)
- CRPS (Continuous Ranked Probability Score)
- SpatialFSS (Fractions Skill Score)
- CoverOneSD (Coverage within 1 standard deviation)
- HeatWave detection (Hit Rate and False Alarm Rate)

Added utility functions in ewb.cases.py:
- load_ewb_events_yaml_into_case_list using importlib.resources for clean asset loading
- read_incoming_yaml helper function for safe YAML file parsing

Enhanced ExtremeWeatherBench runner:
- Added selected_ensemble parameter support in ewb.evaluate.ExtremeWeatherBench

* feat: implement IBTrACS local caching and add time conversion utility

Previously, the IBTrACS data was fetched directly from the internet during every pipeline execution, causing significant network overhead and slowing down repetitive evaluations.

This commit introduces a local caching mechanism in inputs:
- Checks ~/.cache/extremeweatherbench/ for an existing dataset.
- Implements a file size validation (> 1MB) to prevent loading incomplete or corrupted downloads.
- Falls back to urllib.request.urlretrieve to download and save the file locally if the cache is missing or invalid.
- Switches polars.scan_csv to read from the local disk instead of the URL.

Additionally, this commit updates the utils module:
- Adds convert_init_and_valid_time_to_lead_time to support calculating lead times dynamically from initialization and valid time coordinates.

These changes drastically improve data loading speeds, provide resilience against temporary network failures, and enhance temporal alignment capabilities.

* feat(cyclone): add length-normalized DTW distance matrix and DBA consensus trajectory analysis

* feat(cyclone): add filtered ensemble medoid selection script for tropical cyclones

* feat(cyclone): add DTW agglomerative clustering and dominant-cluster medoid analysis

* feat(cyclone): add grand ensemble mean track extraction and 2D spaghetti visualization

* feat(cyclone): add KMeans clustering with point-wise probabilities, spreadness, and distance-to-mean metrics

* feat(cyclone): add physics-informed Kalman Filter and RTS smoother trajectory tracking script

* feat(evaluation): add EWB heatwave threshold evaluation pipeline and Zarr climatology export

* feat(visualization): add SPEDS score multi-metric evaluation boxplot, line chart, and bar chart generator

* feat(visualization): add generic multi-metric evaluation boxplot script with hierarchical lead-time axes

* feat(cyclone): add native IBTrACS path extraction and Zarr track processing scripts

* feat(core): add convert_valid_time_to_init_time_updated using inner join to eliminate ghost dates and include u_wind/v_wind in derived.py for physics-informed Kalman filtering

* feat(tropical-cyclone): extract u and v wind components from forecast for physics-informed Kalman filter trajectory tracking

* Removed some print statement
* feat: add probabilistic evaluation metrics and YAML case loading utilities

Added a comprehensive suite of ensemble and probabilistic evaluation metrics:
- SPEDS (Spatial Probabilistic Extreme Dynamics Score, refactored to NumPy/SciPy)
- NLL (Negative Log-Likelihood)
- CRPS (Continuous Ranked Probability Score)
- SpatialFSS (Fractions Skill Score)
- CoverOneSD (Coverage within 1 standard deviation)
- HeatWave detection (Hit Rate and False Alarm Rate)

Added utility functions in ewb.cases.py:
- load_ewb_events_yaml_into_case_list using importlib.resources for clean asset loading
- read_incoming_yaml helper function for safe YAML file parsing

Enhanced ExtremeWeatherBench runner:
- Added selected_ensemble parameter support in ewb.evaluate.ExtremeWeatherBench

* feat: implement IBTrACS local caching and add time conversion utility

Previously, the IBTrACS data was fetched directly from the internet during every pipeline execution, causing significant network overhead and slowing down repetitive evaluations.

This commit introduces a local caching mechanism in inputs:
- Checks ~/.cache/extremeweatherbench/ for an existing dataset.
- Implements a file size validation (> 1MB) to prevent loading incomplete or corrupted downloads.
- Falls back to urllib.request.urlretrieve to download and save the file locally if the cache is missing or invalid.
- Switches polars.scan_csv to read from the local disk instead of the URL.

Additionally, this commit updates the utils module:
- Adds convert_init_and_valid_time_to_lead_time to support calculating lead times dynamically from initialization and valid time coordinates.

These changes drastically improve data loading speeds, provide resilience against temporary network failures, and enhance temporal alignment capabilities.

* feat(cyclone): add length-normalized DTW distance matrix and DBA consensus trajectory analysis

* feat(cyclone): add filtered ensemble medoid selection script for tropical cyclones

* feat(cyclone): add DTW agglomerative clustering and dominant-cluster medoid analysis

* feat(cyclone): add grand ensemble mean track extraction and 2D spaghetti visualization

* feat(cyclone): add KMeans clustering with point-wise probabilities, spreadness, and distance-to-mean metrics

* feat(cyclone): add physics-informed Kalman Filter and RTS smoother trajectory tracking script

* feat(evaluation): add EWB heatwave threshold evaluation pipeline and Zarr climatology export

* feat(visualization): add SPEDS score multi-metric evaluation boxplot, line chart, and bar chart generator

* feat(visualization): add generic multi-metric evaluation boxplot script with hierarchical lead-time axes

* feat(cyclone): add native IBTrACS path extraction and Zarr track processing scripts

* feat(core): add convert_valid_time_to_init_time_updated using inner join to eliminate ghost dates and include u_wind/v_wind in derived.py for physics-informed Kalman filtering

* feat(tropical-cyclone): extract u and v wind components from forecast for physics-informed Kalman filter trajectory tracking

* Removed some print statement

* Included the urllib library

* Added the MSESubEnsemble and RMSESubEnsemble mertics to mertrics.py

* Remove example folder from git tracking but keep locally

* Add the select_ensemble parameter to the metric classes. Removed the select_ensemble parameter from the evaluate class

* Fix robust lead-time conversion and eliminate ghost dates in valid_time mapping for convert_valid_time_to_init_time

- Switch from join='outer' to join='inner' in xr.concat to eliminate ghost dates and strictly retain common initialization times across all lead times.
- Add explicit checks to drop existing init_time coordinates and prevent name collisions.
- Improve robust handling of lead-time timedelta conversions supporting both hours and nanosecond formats.

* Add hybrid join logic to auto-detect inner/outer join for single and multiple init times
…dation method to fall back on the less strict method if all candidates for a time slice would be rejected and fix some mypy issues (#4)
* The data directory is actually needed and shouldn't be ignored.

* Reverted gitignore to its original

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants