Repository navigation
Conversation
Hoeze
added this pull request to stack #16
October 15, 2026 22:32
Hoeze
force-pushed
the
feat/extractor-use-strand
branch
from
October 16, 2026 19:34
78fb5e8 to
32305da
Compare
Hoeze
force-pushed
the
feat/extractor-use-strand
branch
2 times, most recently
from
October 16, 2026 23:51
359eb52 to
e653064
Compare
Hoeze
force-pushed
the
feat/extractor-use-strand
branch
from
October 17, 2026 14:08
e653064 to
9e25d96
Compare
Hoeze
force-pushed
the
feat/extractor-use-strand
branch
from
October 17, 2026 16:22
9e25d96 to
fd2a421
Compare
FastaStringExtractor defaulted to use_strand=False and VariantSeqExtractor to use_strand=True, so a caller who relied on a default could get reverse-complemented sequences or not without noticing. Both extract methods also swallowed unknown keyword arguments, so a typo like fixed_length=False passed silently. BREAKING CHANGE: use_strand is a required keyword-only argument of FastaStringExtractor and VariantSeqExtractor. To keep the 0.1 behavior, pass use_strand=False to FastaStringExtractor and use_strand=True to VariantSeqExtractor. force_upper of FastaStringExtractor is keyword-only too. FastaStringExtractor.extract and VariantSeqExtractor.extract raise a TypeError for unknown arguments.
VariantSeqExtractor wrapped the REF and ALT of each variant in a Subsequence, a str with coordinates, and collected the parts of the result in IntervalSeqBuilder, a list subclass whose restore() swapped reference ranges for slices of the fetched sequence. No code read the coordinates of the ALT parts. extract now splits the variants as (start, end, alt) tuples, collects (start, end) reference ranges and ALT strings, and joins them after the fetch. The results stay the same: 4000 random cases with SNVs, insertions, deletions and MNVs on both strands give equal sequences before and after. Subsequence and IntervalSeqBuilder are gone. They were not in __all__.
pyfaidx reads uncompressed FASTA files, and bgzip files whose name ends in .gz or .bgz if Biopython is installed. For other compressed files it failed with a misleading error: a plain gzip file ending in .gz raised an ImportError about Biopython, and one with another name raised a UnicodeDecodeError while it built the index. FastaStringExtractor now reads the gzip header before pyfaidx opens the file. It raises ValueError for a gzip file that is not a bgzip file, and for a bgzip file whose name does not end in .gz or .bgz. It raises ImportError for a bgzip file if Biopython is missing. The docstrings no longer say that the file can be gzipped.
FastaStringExtractor(fasta_file, *, use_strand, uppercase=True) replaces force_upper=False. It passes sequence_always_upper=uppercase to pyfaidx.Fasta, so pyfaidx uppercases each sequence when it reads it. Encoder and Tokenizer are case-sensitive, so they take the extracted sequences as they are, also in soft-masked repeats. uppercase=False keeps the case of the file, e.g. to find soft-masked bases. The error of the encoders for a lowercase letter now points to FastaStringExtractor, which uppercases by default. BREAKING CHANGE: FastaStringExtractor no longer takes force_upper, and it returns uppercase sequences by default. Pass uppercase=False to keep the lowercase letters of the file.
VariantSeqExtractor took the mutually exclusive pair fasta_file and reference_sequence, and kept the deprecated alias fasta. Its reference had to be a BaseExtractor, whose extract took (interval, *args, **kwargs). So a VariantSeqExtractor counted as a reference too, and failed only when called without variants. BaseExtractor also closed the files in __del__. - SequenceExtractor is a typing.Protocol with extract(interval) -> str. Any object with this method is a reference, without subclassing. - VariantSeqExtractor(reference, *, use_strand) raises TypeError for a reference that is not a SequenceExtractor. The caller owns the reference and closes it. - FastaStringExtractor is a context manager. VariantSeqExtractor passes the reference intervals with the strand ".", so a FastaStringExtractor with use_strand=True as the reference does not reverse complement the sequence twice. A test covers this case. BREAKING CHANGE: BaseExtractor is removed. VariantSeqExtractor takes the reference extractor as its first argument: VariantSeqExtractor( fasta_file=path, use_strand=u) becomes VariantSeqExtractor( FastaStringExtractor(path, use_strand=False), use_strand=u), and VariantSeqExtractor(reference_sequence=ex, use_strand=u) becomes VariantSeqExtractor(ex, use_strand=u). VariantSeqExtractor.close, ref_seq_extractor and the deprecated alias fasta are removed.
FastaStringExtractor.extract returned 5 bases for a 15 bp interval that ran 10 bases past the chromosome end, and raised a pyfaidx FetchError for a negative start. VariantSeqExtractor.extract took the chromosome length as chrom_len: float = math.inf, so without it the reference silently returned fewer bases at the chromosome end. For an anchor outside the chromosome, it put the N at the wrong end. - FastaStringExtractor.chrom_sizes maps each chromosome in the index to its length. The pyfaidx object fasta is private. - FastaStringExtractor.extract raises ValueError for an interval outside [0, chromosome length], and for an unknown chromosome. It raises TypeError if the interval is not an Interval. - SequenceExtractor has chrom_sizes too. - VariantSeqExtractor.extract reads the chromosome length from reference.chrom_sizes. It fetches only bases within the chromosome, marks the others, and turns them into N with is_padding=True. Without is_padding, it raises ValueError if the result needs such a base. 4000 random cases give the same sequences and errors as before with chrom_len set to the chromosome length. BREAKING CHANGE: FastaStringExtractor.extract raises ValueError for an interval outside its chromosome or on an unknown chromosome, and FastaStringExtractor.fasta is private. Use chrom_sizes for the chromosome names and lengths. VariantSeqExtractor.extract no longer takes chrom_len, and a reference needs chrom_sizes.
VariantSeqExtractor.extract applied a variant at its position also if it lay on another chromosome or if its REF did not match the reference. Such a variant comes from a wrong VCF or a wrong genome build, so the sequence would be wrong without a sign. BREAKING CHANGE: VariantSeqExtractor.extract raises a ValueError for a variant on another chromosome than the interval, for a variant outside the chromosome, and for a REF that differs from the reference, ignoring case. It raises a TypeError if the interval is not an Interval or a variant is not a Variant.
VariantSeqExtractor.extract needed an anchor also with fixed_len=False, where the anchor had no effect. One keyword now does both jobs: anchor_pos=None lets the length follow the variants, and an int keeps the width of the interval, anchored at that position. The docstring states the rule for a variant at the anchor, which differs from GenomeKit for an insertion whose REF ends at the anchor. An anchor outside the interval was moved to the nearest end without a sign. The per-call use_strand gave a second way to set the strand handling of an extractor. BREAKING CHANGE: VariantSeqExtractor.extract takes (interval, variants, *, anchor_pos=None, pad_with_n=False). anchor=a becomes anchor_pos=a, anchor=a with fixed_len=False becomes anchor_pos=None, and is_padding becomes pad_with_n. It raises a ValueError unless interval.start <= anchor_pos <= interval.end. FastaStringExtractor.extract and VariantSeqExtractor.extract no longer take use_strand.
The expected values are the outputs of GenomeKit 7.7.0 for the same reference and variants: 11 bp windows anchored at the variant start, windows around the variant without an anchor, and 60 bp haplotypes with the anchors "center", "5p" and "start" on both strands. The two "all" rows with the anchor "center" keep the value of kipoiseq2 on purpose. The REF of the insertion ends at the anchor, so kipoiseq2 puts the insertion left of it. GenomeKit normalizes the insertion and puts it right.
FastaStringExtractor.extract and VariantSeqExtractor.extract repeated the strand rule of Interval. They now call _is_reverse.
The isort guard kept dataclasses first, because the extractors imported Interval from kipoiseq2. They import kipoiseq2.dataclasses now, so the imports of the package follow the isort order.
Hoeze
force-pushed
the
feat/extractor-use-strand
branch
from
October 17, 2026 17:19
fd2a421 to
c7d3b7a
Compare
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
tl;dr:
VariantSeqExtractorreads any reference withextractandchrom_sizes, checks intervals, variants and REFs against it, and takesextract(interval, variants, *, anchor_pos=None, pad_with_n=False).FastaStringExtractorreturns uppercase sequences by default.Strand and arguments:
use_strandis a required keyword-only argument ofFastaStringExtractorandVariantSeqExtractor. Before, the defaults differed (FalseandTrue), so a caller who relied on a default could get reverse-complemented sequences or not without noticing.use_strandof bothextractmethods is gone. The extractor has one strand handling.extractmethods raise a TypeError for unknown arguments. Before, they swallowed them, so a typo likefixed_length=Falsepassed silently.Reference:
kipoiseq2.extractors.base.SequenceExtractoris a protocol withextract(interval)andchrom_sizes. It replacesBaseExtractor, so a reference needs no base class.VariantSeqExtractor(reference, *, use_strand)takes the reference as its first argument and raises a TypeError for other objects. It replacesfasta_fileandreference_sequence. For a FASTA file, passFastaStringExtractor(path, use_strand=False).VariantSeqExtractoropens no file, so it has noclose(). The caller owns the reference and closes it. Before,BaseExtractorclosed the files in__del__.ref_seq_extractorand the deprecated aliasfastaare gone.FastaStringExtractoris a context manager.FastaStringExtractor.chrom_sizesmaps each chromosome to its length, from the index. The pyfaidx objectfastais private.FastaStringExtractoruppercases the sequences by default. The keyword-onlyuppercase=Truereplacesforce_upper=False.uppercase=Falsekeeps the case of the file, e.g. to find soft-masked repeats.EncoderandTokenizerofkipoiseq2.encodersare case-sensitive, so they take the extracted sequences as they are, also in soft-masked repeats. For a lowercase letter, their error now points toFastaStringExtractor, which uppercases by default.FastaStringExtractorraises a ValueError for a gzip file that is not bgzip, or a bgzip file without a.gzor.bgzsuffix. It raises an ImportError for a bgzip file if Biopython is missing. Before, pyfaidx raised a misleading ImportError or a UnicodeDecodeError.Checks:
FastaStringExtractor.extractraises a ValueError for an unknown chromosome and for an interval outside the chromosome. Before, an interval past the chromosome end gave a shorter sequence.VariantSeqExtractor.extractreads the chromosome length from thechrom_sizesof the reference, sochrom_lenis gone. Withpad_with_n=True, it fills the positions outside the chromosome with N. Otherwise it raises a ValueError.VariantSeqExtractor.extractraises a ValueError for a variant on another chromosome, a variant outside the chromosome, and a REF that differs from the reference. Before, it applied such variants without a sign. The REF check ignores case, so it also accepts the lowercase bases of a reference that was read withuppercase=False.anchor_pos:anchorandfixed_len.anchor_pos=Nonelets the length follow the variants, and an int keeps the width of the interval, anchored at that position. Before,fixed_len=Falsestill needed an anchor without effect.anchor_posmust lie frominterval.starttointerval.end. Before, an anchor outside the interval was moved to the nearest end.is_paddingis renamed topad_with_n.Internals: the variant sequences are built from
(start, end, alt)tuples and strings instead of the classesSubsequenceandIntervalSeqBuilder.Verified:
VariantSeqExtractorwith GenomeKit outputs for the same reference and variants. 16 of the 18 cases agree. In the other two, the insertion lies at the anchor, so the tests keep kipoiseq2's value and name GenomeKit's.BREAKING CHANGE:
use_strandis a required keyword-only argument ofFastaStringExtractorandVariantSeqExtractor. To keep the 0.1 behavior, passuse_strand=FalsetoFastaStringExtractoranduse_strand=TruetoVariantSeqExtractor. Theextractmethods no longer takeuse_strand, and raise a TypeError for unknown arguments.BaseExtractoris removed: a reference is any object withextract(interval)andchrom_sizes.VariantSeqExtractor(fasta_file=path, use_strand=u)becomesVariantSeqExtractor(FastaStringExtractor(path, use_strand=False), use_strand=u), andreference_sequence=extractorbecomes the first argument.VariantSeqExtractor.close,ref_seq_extractorandfastaare removed.FastaStringExtractorreads uppercase sequences by default. The keyword-onlyuppercase=Truereplacesforce_upper=False, anduppercase=Falsekeeps the case of the file.FastaStringExtractor.fastais private: usechrom_sizes.FastaStringExtractor.extractraises a ValueError for intervals outside the chromosome.VariantSeqExtractor.extracttakes(interval, variants, *, anchor_pos=None, pad_with_n=False):anchor=abecomesanchor_pos=a,anchor=a, fixed_len=Falsebecomesanchor_pos=None, andis_paddingbecomespad_with_n. It no longer takeschrom_len, and raises a ValueError for ananchor_posoutside the interval, for positions outside the chromosome withoutpad_with_n=True, and for variants on another chromosome, outside the chromosome or with a REF that differs from the reference.feat(extractors)!: require use_strand and reject unknown arguments
fix(extractors): raise a clear error for unreadable FASTA files
feat(extractors)!: uppercase FASTA sequences by default
feat(extractors)!: check intervals against the chromosome sizes
feat(extractors)!: check the chrom and REF of variants
feat(extractors)!: merge anchor and fixed_len into anchor_pos