Fix/ss opm faithful - #69
Merged
Merged
Conversation
…ithful
The wrapper fed the network raw per-cell statistics and the raw batch cell
count (thousands in training, tens for the subsampled test batches), took the
donor of the NeurIPS 2021 batch labels for the day, appended all genes instead
of the original's ~70 HGNC/Reactome-selected genes as raw CITE features, kept
cells without any protein counts (NaN loss on the 2022 CITE data), upcast the
dense multiome targets to float64 (OOM on 2022 ATAC->GEX) and returned the
model's per-cell z-scores as predictions.
- per-cell statistics as in make_additional_files.py (quartiles of the
non-zero values, log1p ratio for multiome), standardized over train+test
- CITE batch singular vectors from per-batch gene medians, standardized over
the training batches, test-only batches projected; cell ratios and cell
count constant (no cell types in the task files, subsampled test set)
- day/donor regexes default to the {day}_{donor} labels of NeurIPS 2022,
constant otherwise
- CITE gene masks rebuilt with the original thresholds from HGNC (Ensembl id
-> symbol) and Reactome, downloaded into the image
- training cells with a constant target vector dropped
- dtype-preserving, zero-row-safe patches of the ss_opm normalizers
- predictions mapped to the target scale by a global affine fit on training
cells (correlations unchanged)
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
The mask construction correlates every candidate gene with all proteins at once via rankdata instead of a full spearmanr matrix per gene (identical values, minutes instead of hours on 14k genes). Bundles written before the metadata rebuild are read with neutral defaults so the pre-trained test resources keep working until they are regenerated. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
benjaminfreyuu
left a comment
benjaminfreyuu
left a comment
Contributor
There was a problem hiding this comment.
Looks good, at least the changes made. I am not so familiar with method, so limited trustworthiness in assessing this
| test_inputs_values=_test_inputs_for_svd, | ||
| test_metadata=_test_metadata_for_svd, | ||
| test_inputs_values=test_inputs if test_inputs is not None else train_inputs, | ||
| test_metadata=test_metadata if test_metadata is not None else train_metadata, |
Contributor
There was a problem hiding this comment.
is this correct? Why is there an else train_metadata/ train_inputs for the test case
…D on train only without a test set reactome.org answers HTTP 403 to Python's default urllib user agent, which failed the docker image build of ss_opm_train in CI (PR #69). Both the image-build step and the runtime fallback now send `User-Agent: Mozilla/5.0`. `fit_preprocess` no longer receives the training set again in place of a missing test set: the original's `use_test_inputs` switch is set from whether `--input_test_mod1` was given (review comment on #69). CHANGELOG entry added. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Update ss_opm model for better scalability and performance. Fixed the following problems of the previous implementation:
Checklist before requesting a review
I have performed a self-review of my code
Check the correct box. Does this PR contain:
Proposed changes are described in the CHANGELOG.md
CI Tests succeed and look good!