[Hoeze]

tl;dr: One function, scan_variant_pairs(intervals, variants), replaces SingleVariantMatcher, MultiVariantsMatcher, BaseVariantMatcher and the converters. It takes polars frames and returns the pairs as a LazyFrame. Interval.attrs is gone.

The matchers turned their inputs into polars frames, joined them with polars-bio, and turned the rows back into Interval and Variant objects. SingleVariantMatcher and MultiVariantsMatcher differed only in how they iterated, and MultiVariantsMatcher ignored variant_batch_size. vcf_file duplicated variants=scan_vcf_variants(path), which keeps more columns. BaseVariantMatcher was public, but iterating over it raised NotImplementedError.

scan_variant_pairs(intervals: pl.DataFrame, variants: pl.DataFrame | pl.LazyFrame) -> pl.LazyFrame keeps the polars-bio overlap join. polars-bio holds the intervals in memory and streams the variants through the join. The pairs have the columns of the old pairs(): the interval columns under their own names, interval_idx, the variant columns with the prefix variant_, and variant_idx. They come in no fixed order, and .collect().sort("variant_idx", "interval_idx") gives the old pairs(). sink_parquet writes the pairs to a file, and collect_batches(chunk_size=n) yields batches of n pairs.

The pairs keep all columns of both frames, so interval_attrs is gone: to keep fewer columns, select them first. A null strand in the intervals gives the strand ".", the same as a missing strand column. Before, pairs() kept the null, and the iteration raised a TypeError. The pairs use the names interval_idx and variant_idx and the prefix variant_. So an interval column named interval_idx or starting with variant_ gives a ValueError. A variant column named idx or variant_idx gives a ValueError too, where it gave an unclear polars error before. scan_variant_pairs raises a TypeError for inputs that are not polars frames, and the TypeError for a path as variants points to scan_vcf_variants.

The Interval and Variant inputs, intervals_to_polars, variants_to_polars, the public overlap_variants, iter_pairs, iter_batches and iter_by_interval are gone. Interval and Variant are dataclasses, so pl.DataFrame(intervals) and pl.DataFrame(variants) build the frames from objects. A loop over pairs.iter_rows(named=True) builds Interval and Variant objects, and MIGRATING.md shows it. A later PR adds a batch API that yields the ref and alt sequences of the pairs.

No code returns Interval objects with metadata any more, so Interval.attrs is gone. An Interval has only chrom, start, end and strand.

The README, the API page and MIGRATING.md describe scan_variant_pairs. The kipoiseq section of MIGRATING.md describes the final state for SingleVariantMatcher, MultiVariantsMatcher and vcf_file.

BREAKING CHANGE: SingleVariantMatcher, MultiVariantsMatcher, BaseVariantMatcher, intervals_to_polars, variants_to_polars and overlap_variants are gone. Use scan_variant_pairs(intervals, variants) with variants=scan_vcf_variants(path) instead of vcf_file, and pl.DataFrame(objects) for a list of Interval or Variant objects. The pairs are a LazyFrame in no fixed order: sort them for the order of pairs(). Build the objects from pairs.iter_rows(named=True) instead of iterating over a matcher, and use collect_batches(chunk_size=n) instead of variant_batch_size. Select the columns of intervals instead of passing interval_attrs. A variants column named idx or variant_idx raises a ValueError. Interval.attrs is gone.

feat(dataclasses)!: drop Interval.attrs