Repository navigation
Start Rt sampling after infection initialization - #924
cdc-mitzimorris wants to merge 36 commits into
Conversation
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
for more information, see https://pre-commit.ci
|
Thank you for your contribution @cdc-mitzimorris 🚀! Your github-pages is ready for download 👉 here 👈! |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #924 +/- ##
==========================================
+ Coverage 98.89% 98.95% +0.06%
==========================================
Files 59 59
Lines 2344 2396 +52
==========================================
+ Hits 2318 2371 +53
+ Misses 26 25 -1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Harness. 🚀 New features to boost your workflow:
|
There was a problem hiding this comment.
🟡 Changes recommended
Integer initial states cause both AR1 parameterizations to fail for multi-point trajectories because the scan carry changes from integer to floating-point dtype.
2 open issues
What changed in this PR
Implements #923 by starting Rt processes when renewal begins, rather than during infection initialization, while preserving full-length daily infection and Rt records.
Changes:
- Sample Rt only after initialization, shift weekly calendar alignment, and pad daily records with starting values.
- Use exact supplied AR1 starts and standardized innovation draws while preserving stationary ascertainment initialization.
- Add reproducible weekly-Rt data, expand regression and recovery tests, and update documentation.
| File | Description |
|---|---|
test/test_temporal_processes.py |
Tests exact starts, standardized draws, covariance, and forecast replay. |
test/test_subpopulation_infections.py |
Verifies supplied AR1 starting Rt values are preserved. |
test/test_rt_initialization.py |
Adds shared initialization, calendar, and forecast regression tests. |
test/test_pyrenew_builder.py |
Checks distinct Rt and ascertainment timeline contexts. |
test/test_population_infections.py |
Tests exact initial Rt and shifted weekly alignment. |
test/test_datagen_weekly_rt.py |
Checks dataset consistency, reproducibility, and observation means. |
test/test_ascertainment.py |
Verifies stationary AR1 ascertainment starts. |
test/integration/test_population_infections_he_weekly_rt.py |
Uses matching weekly truth for innovation-form recovery tests. |
test/integration/test_population_infections_he_weekly_rt_state_centered.py |
Uses matching weekly truth for state-centered recovery tests. |
test/integration/conftest.py |
Adds weekly-data fixtures and matching model configuration. |
pyrenew/latent/temporal_processes.py |
Updates initialization, innovation scaling, and single-point handling. |
pyrenew/latent/subpopulation_infections.py |
Starts shared subpopulation Rt processes after initialization. |
pyrenew/latent/population_infections.py |
Samples post-initialization Rt and pads daily records. |
pyrenew/latent/mobility_subpopulation_infections.py |
Documents the revised initialization behavior. |
pyrenew/datasets/synthetic_CA_126_weekly_rt/weekly_rt.csv |
Stores the weekly Rt truth. |
pyrenew/datasets/synthetic_CA_126_weekly_rt/weekly_hospital_admissions.csv |
Adds weekly hospital observations and expected counts. |
pyrenew/datasets/synthetic_CA_126_weekly_rt/true_parameters.json |
Records generating parameters and calendar metadata. |
pyrenew/datasets/synthetic_CA_126_weekly_rt/initial_infections.csv |
Stores the exponential initialization history. |
pyrenew/datasets/synthetic_CA_126_weekly_rt/expected_observations.csv |
Stores daily hospital and ED observation means. |
pyrenew/datasets/synthetic_CA_126_weekly_rt/daily_infections.csv |
Stores daily infections and repeated weekly Rt truth. |
pyrenew/datasets/synthetic_CA_126_weekly_rt/daily_hospital_admissions.csv |
Adds synthetic daily hospital observations. |
pyrenew/datasets/synthetic_CA_126_weekly_rt/daily_ed_visits.csv |
Adds synthetic daily ED observations. |
pyrenew/datasets/datagen_he_CA_126.py |
Adds configurable, reproducible weekly-Rt generation. |
pyrenew/ascertainment/base.py |
Leaves temporal initialization unset to preserve stationary AR1 starts. |
pyrenew/arrayutils.py |
Adds a helper for padding records with their first row. |
pyproject.toml |
Includes the weekly-Rt dataset in wheel packaging. |
docs/tutorials/latent_subpopulation_infections.qmd |
Explains exact starts and revised deviation behavior. |
docs/tutorials/latent_mobility_infections.qmd |
Clarifies regional Rt timing and initialization growth. |
docs/tutorials/latent_infections.qmd |
Explains post-initialization Rt sampling and process parameterizations. |
docs/tutorials/ascertainment.qmd |
Clarifies stationary ascertainment starts versus exact latent Rt starts. |
Review effort: Max
|
@dylanhmorris ready for review |
| Daily baseline, deviation, and Rt records retain the full infection axis by prepending copies of their first values. | ||
| Initialization therefore adds no sampled states or innovations, and weekly records cover only the post-initialization period. | ||
| The first recorded Rt week is the calendar week containing the first renewal day; its start date may precede that day, but no initialization-only weeks are sampled. |
There was a problem hiding this comment.
Let me check my understanding:
- Recorded daily
$\mathcal{R}(t)$ prepends (deterministic, not sampled) starting values for the days before$\mathcal{R}(t)$ . The exponential growth initialization at an$r$ derived from$\mathcal{R}(0)$ makes this true in model world, so I think it's okay as long as its documented. That said, it wouldn't be my preference unless it is necessary, since other forms of initialization are possible. - Recorded weekly
$\mathcal{R}(t)$ values do not prepend unsampled$\mathcal{R}(t)$ values.
Is there a key reason to do this? I'd prefer that the two behaviors be more consistent if that's possible. My preference (I think) is no prepending in either case, but I am persuadable.
| Rt determines its exponential initialization growth. Temporal | ||
| processes cover only post-initialization days; daily Rt records | ||
| prepend copies of the starting values for initialization. |
There was a problem hiding this comment.
See question about this above.
dylanhmorris
left a comment
There was a problem hiding this comment.
@cdc-mitzimorris question about prepending behaviors before I review further.
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
Co-authored-by: Dylan H. Morris <dylanhmorris@users.noreply.github.com>
…ov/PyRenew into mem_923_start_Rt_process_post_init
@dylanhmorris - thanks - good catch! We don’t need the prepending. The design included it to preserve existing output shapes: “Array shapes on the shared axis are unchanged.” Preserving those shapes isn’t a requirement here, and it leaves the tutorials and plotting code trimming off values we just added. Instead, we should record daily Rt from the first renewal day, consistent with weekly. This would change the recorded arrays and their indexing, but leave the initialization and renewal calculations unchanged. |


This implements issue #923