Repository navigation
Feature/fix - #436
Open
PraveenKumar-89 wants to merge 21 commits into
Open
Feature/fix #436PraveenKumar-89 wants to merge 21 commits into
PraveenKumar-89 wants to merge 21 commits into
Conversation
* feat: add probabilistic evaluation metrics and YAML case loading utilities Added a comprehensive suite of ensemble and probabilistic evaluation metrics: - SPEDS (Spatial Probabilistic Extreme Dynamics Score, refactored to NumPy/SciPy) - NLL (Negative Log-Likelihood) - CRPS (Continuous Ranked Probability Score) - SpatialFSS (Fractions Skill Score) - CoverOneSD (Coverage within 1 standard deviation) - HeatWave detection (Hit Rate and False Alarm Rate) Added utility functions in ewb.cases.py: - load_ewb_events_yaml_into_case_list using importlib.resources for clean asset loading - read_incoming_yaml helper function for safe YAML file parsing Enhanced ExtremeWeatherBench runner: - Added selected_ensemble parameter support in ewb.evaluate.ExtremeWeatherBench * feat: implement IBTrACS local caching and add time conversion utility Previously, the IBTrACS data was fetched directly from the internet during every pipeline execution, causing significant network overhead and slowing down repetitive evaluations. This commit introduces a local caching mechanism in inputs: - Checks ~/.cache/extremeweatherbench/ for an existing dataset. - Implements a file size validation (> 1MB) to prevent loading incomplete or corrupted downloads. - Falls back to urllib.request.urlretrieve to download and save the file locally if the cache is missing or invalid. - Switches polars.scan_csv to read from the local disk instead of the URL. Additionally, this commit updates the utils module: - Adds convert_init_and_valid_time_to_lead_time to support calculating lead times dynamically from initialization and valid time coordinates. These changes drastically improve data loading speeds, provide resilience against temporary network failures, and enhance temporal alignment capabilities.
* feat: add probabilistic evaluation metrics and YAML case loading utilities Added a comprehensive suite of ensemble and probabilistic evaluation metrics: - SPEDS (Spatial Probabilistic Extreme Dynamics Score, refactored to NumPy/SciPy) - NLL (Negative Log-Likelihood) - CRPS (Continuous Ranked Probability Score) - SpatialFSS (Fractions Skill Score) - CoverOneSD (Coverage within 1 standard deviation) - HeatWave detection (Hit Rate and False Alarm Rate) Added utility functions in ewb.cases.py: - load_ewb_events_yaml_into_case_list using importlib.resources for clean asset loading - read_incoming_yaml helper function for safe YAML file parsing Enhanced ExtremeWeatherBench runner: - Added selected_ensemble parameter support in ewb.evaluate.ExtremeWeatherBench * feat: implement IBTrACS local caching and add time conversion utility Previously, the IBTrACS data was fetched directly from the internet during every pipeline execution, causing significant network overhead and slowing down repetitive evaluations. This commit introduces a local caching mechanism in inputs: - Checks ~/.cache/extremeweatherbench/ for an existing dataset. - Implements a file size validation (> 1MB) to prevent loading incomplete or corrupted downloads. - Falls back to urllib.request.urlretrieve to download and save the file locally if the cache is missing or invalid. - Switches polars.scan_csv to read from the local disk instead of the URL. Additionally, this commit updates the utils module: - Adds convert_init_and_valid_time_to_lead_time to support calculating lead times dynamically from initialization and valid time coordinates. These changes drastically improve data loading speeds, provide resilience against temporary network failures, and enhance temporal alignment capabilities. * feat(cyclone): add length-normalized DTW distance matrix and DBA consensus trajectory analysis * feat(cyclone): add filtered ensemble medoid selection script for tropical cyclones * feat(cyclone): add DTW agglomerative clustering and dominant-cluster medoid analysis * feat(cyclone): add grand ensemble mean track extraction and 2D spaghetti visualization * feat(cyclone): add KMeans clustering with point-wise probabilities, spreadness, and distance-to-mean metrics * feat(cyclone): add physics-informed Kalman Filter and RTS smoother trajectory tracking script * feat(evaluation): add EWB heatwave threshold evaluation pipeline and Zarr climatology export * feat(visualization): add SPEDS score multi-metric evaluation boxplot, line chart, and bar chart generator * feat(visualization): add generic multi-metric evaluation boxplot script with hierarchical lead-time axes * feat(cyclone): add native IBTrACS path extraction and Zarr track processing scripts * feat(core): add convert_valid_time_to_init_time_updated using inner join to eliminate ghost dates and include u_wind/v_wind in derived.py for physics-informed Kalman filtering * feat(tropical-cyclone): extract u and v wind components from forecast for physics-informed Kalman filter trajectory tracking * Removed some print statement
* feat: add probabilistic evaluation metrics and YAML case loading utilities Added a comprehensive suite of ensemble and probabilistic evaluation metrics: - SPEDS (Spatial Probabilistic Extreme Dynamics Score, refactored to NumPy/SciPy) - NLL (Negative Log-Likelihood) - CRPS (Continuous Ranked Probability Score) - SpatialFSS (Fractions Skill Score) - CoverOneSD (Coverage within 1 standard deviation) - HeatWave detection (Hit Rate and False Alarm Rate) Added utility functions in ewb.cases.py: - load_ewb_events_yaml_into_case_list using importlib.resources for clean asset loading - read_incoming_yaml helper function for safe YAML file parsing Enhanced ExtremeWeatherBench runner: - Added selected_ensemble parameter support in ewb.evaluate.ExtremeWeatherBench * feat: implement IBTrACS local caching and add time conversion utility Previously, the IBTrACS data was fetched directly from the internet during every pipeline execution, causing significant network overhead and slowing down repetitive evaluations. This commit introduces a local caching mechanism in inputs: - Checks ~/.cache/extremeweatherbench/ for an existing dataset. - Implements a file size validation (> 1MB) to prevent loading incomplete or corrupted downloads. - Falls back to urllib.request.urlretrieve to download and save the file locally if the cache is missing or invalid. - Switches polars.scan_csv to read from the local disk instead of the URL. Additionally, this commit updates the utils module: - Adds convert_init_and_valid_time_to_lead_time to support calculating lead times dynamically from initialization and valid time coordinates. These changes drastically improve data loading speeds, provide resilience against temporary network failures, and enhance temporal alignment capabilities. * feat(cyclone): add length-normalized DTW distance matrix and DBA consensus trajectory analysis * feat(cyclone): add filtered ensemble medoid selection script for tropical cyclones * feat(cyclone): add DTW agglomerative clustering and dominant-cluster medoid analysis * feat(cyclone): add grand ensemble mean track extraction and 2D spaghetti visualization * feat(cyclone): add KMeans clustering with point-wise probabilities, spreadness, and distance-to-mean metrics * feat(cyclone): add physics-informed Kalman Filter and RTS smoother trajectory tracking script * feat(evaluation): add EWB heatwave threshold evaluation pipeline and Zarr climatology export * feat(visualization): add SPEDS score multi-metric evaluation boxplot, line chart, and bar chart generator * feat(visualization): add generic multi-metric evaluation boxplot script with hierarchical lead-time axes * feat(cyclone): add native IBTrACS path extraction and Zarr track processing scripts * feat(core): add convert_valid_time_to_init_time_updated using inner join to eliminate ghost dates and include u_wind/v_wind in derived.py for physics-informed Kalman filtering * feat(tropical-cyclone): extract u and v wind components from forecast for physics-informed Kalman filter trajectory tracking * Removed some print statement * Included the urllib library * Added the MSESubEnsemble and RMSESubEnsemble mertics to mertrics.py * Remove example folder from git tracking but keep locally * Add the select_ensemble parameter to the metric classes. Removed the select_ensemble parameter from the evaluate class * Fix robust lead-time conversion and eliminate ghost dates in valid_time mapping for convert_valid_time_to_init_time - Switch from join='outer' to join='inner' in xr.concat to eliminate ghost dates and strictly retain common initialization times across all lead times. - Add explicit checks to drop existing init_time coordinates and prevent name collisions. - Improve robust handling of lead-time timedelta conversions supporting both hours and nanosecond formats. * Add hybrid join logic to auto-detect inner/outer join for single and multiple init times
…dation method to fall back on the less strict method if all candidates for a time slice would be rejected and fix some mypy issues (#4)
* The data directory is actually needed and shouldn't be ignored. * Reverted gitignore to its original
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
EWB Pull Request: Advanced Tropical Cyclone Tracking, Kalman Filtering & Multi-Metric Evaluation Suite
Overview
Detailed Summary of Changes
1. Probabilistic Metrics & Utilities
Added a comprehensive suite of ensemble and probabilistic evaluation metrics:
Added utility functions in
ewb.cases.py:load_ewb_events_yaml_into_case_listusingimportlib.resourcesfor clean asset loading.read_incoming_yamlhelper function for safe YAML file parsing.Enhanced ExtremeWeatherBench runner:
selected_ensembleparameter support inewb.evaluate.ExtremeWeatherBench.2. IBTrACS Local Caching & Time Conversion
Previously, IBTrACS data was fetched directly from the internet during every pipeline execution, causing network overhead. This update introduces a local caching mechanism:
~/.cache/extremeweatherbench/for an existing dataset.urllib.request.urlretrieveif the cache is missing or invalid.polars.scan_csvto read from the local disk instead of the URL.Utility updates (
utils.py):convert_init_and_valid_time_to_lead_timeto support calculating lead times dynamically from initialization and valid time coordinates.3. Core & Utility Improvements (
src/extremeweatherbench/)utils.py(convert_valid_time_to_init_time):join="outer"approach introduces spurious/ghost dates and dimension alignment issues when shiftingvalid_timetoinit_timeacross different lead times. Also, lead-time dtype conversion and potentialinit_timecoordinate collisions weren't explicitly handled.join="outer"tojoin="inner"to strictly retain only the true forecast initialization times common to all lead times.init_timecoordinates and prevent name collisions.derived.py&tropical_cyclone.py:u_wind) and meridional (v_wind) wind components directly from forecast datasets to provide realistic environmental steering velocities for trajectory tracking.