Repository navigation
Conversation
|
@coderabbitai review |
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info
📝 Walkthrough
Priority: ⬇️ Low Merge Risk: ⚪ Minimal · up to
|
| Check name | Status | Explanation |
|---|---|---|
| Linked Issues check |  |
Check skipped because no linked issues were found for this pull request. |
| Out of Scope Changes check | |
Check skipped because no linked issues were found for this pull request. |
- Autofix · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts
Comment @coderabbitai help to get the list of available commands.
|
melo-gonzo
left a comment
melo-gonzo
left a comment
There was a problem hiding this comment.
LGTM. The opt-in policy checks consumed HDF5 paths and reopened reads while preserving default linked-input behavior. All 75 HDF5/distributed tests passed in both default and opt-in environments, including saved worker commands.
Preserve normal HDF5 linked-data reads by default. Allow workflows to require ordinary in-file datasets, checking links and storage on each opened handle and retaining the policy in distributed worker commands. Signed-off-by: Trent Nelson <trentn@nvidia.com>
Reuse ordinary h5py lookup when validation is disabled and retain the selected policy across threaded block reads. Validate each newly opened handle when the strict policy is enabled. Signed-off-by: Trent Nelson <trentn@nvidia.com>
4347cd7 to
9a4792f
Compare
This branch has not been deployed
2 participants
HDF5 inputs can depend on other files through links, external raw storage, or virtual datasets. Add
CUPHOTON_XRAY_HDF5_SELF_CONTAINED=1for xRay workflows that require ordinary datasets stored in the selected file. Existing linked-input behavior remains the default.The opt-in mode checks links and dataset storage during probing and reading, including reopened handles and visualization. Distributed worker commands retain the setting in saved local and Slurm scripts. Input handles select their policy once, and threaded block readers reuse that selection while validating each reopened handle. This avoids repeated environment checks for every object or block.
Validation: the xRay suite passes in default and opt-in modes (364 passed, 35 GPU-dependent skips each); repository hooks and mypy pass. Local warm-cache benchmarks show about 4 microseconds of remaining normal-probe overhead, with metadata-heavy probes, block reads using a captured policy, and contiguous/gzip trace processing within measurement noise. These measurements do not cover full concurrent GPU throughput. Documentation explains policy lifetime and the validation boundary.