Skip to content

fix(python): build Decimal scalars from the unscaled integer and an exponent - #9885

Open
jackylee-ch wants to merge 1 commit into
vortex-data:developfrom
jackylee-ch:fix/py-decimal-scalar-repr
Open

jackylee-ch wants to merge 1 commit into
vortex-data:developfrom
jackylee-ch:fix/py-decimal-scalar-repr

Conversation

@jackylee-ch

Copy link
Copy Markdown
Contributor

decimal_value_to_py (vortex-python/src/scalar/into_py.rs:207) split the stored integer into a
whole and a fractional part and pasted them together as "{whole}.{fraction}". That reimplements
three things Decimal already does, and all three were wrong. Measured against develop:

dtype and stored value produced correct
decimal(10,2) storing 12345 Decimal('123.45') same
decimal(10,2) storing -12345 decimal.InvalidOperation Decimal('-123.45')
decimal(10,2) storing -5 decimal.InvalidOperation Decimal('-0.05')
decimal(10,0) storing -7 Decimal('-7.0') Decimal('-7')
decimal(10,3) storing 5 attempt to exponentiate with overflow Decimal('0.005')
  • A negative value keeps its sign on both parts, so the repr is -123.-45, which Decimal refuses.
    Truncating division makes the smaller case worse: -5 at scale 2 renders as 0.-5, dropping the
    sign entirely even if it had parsed.
  • The pad width is scale as usize, which wraps to ~1.8e19 for a negative scale.
  • The factor is 10.pow(scale.unsigned_abs()) in the storage type, and
    scalar_helper_inner (factory.rs:80) picks the narrowest storage that holds the value, not one
    derived from the dtype. So vx.scalar(5, dtype=vx.decimal(precision=10, scale=3)) — an ordinary
    dtype with a small value — evaluates 10i8.pow(3). This is not an extreme-scale corner.
  • Scale 0 emitted a .0 the dtype does not have, so the returned Decimal's exponent did not match
    the dtype's scale.

On profiles, so the last row is not overstated: pyproject.toml:63 builds wheels with
profile = "release" and Cargo.toml:410 leaves overflow-checks at its default, off. The
overflow panic above is what an editable/dev build does; a release wheel wraps the factor instead
and returns a wrong Decimal. The InvalidOperation and scale-0 rows are string-shape bugs and
hold in both profiles.

Fix

Hand Decimal the unscaled integer and an exponent. It parses that form exactly — the context
precision bounds arithmetic, not construction — and the result carries an exponent of exactly
-scale, so the dtype's scale survives the round trip. The exponent is negated through i16
because -i8::MIN overflows and the scale's lower bound is not held away from i8::MIN.

This deletes the DecimalIntoParts trait, its macro and the i256 impl, for a net -5 lines.

Not in this change

vortex/__init__.py:74 asks whether DecimalScalar is missing from the public exports — it is,
though scalar.pyi:31 declares it. Separately, scalar_helper_inner has no i256 branch, so a
value wider than i128 cannot be built into a decimal scalar at all
(Value can't be represented as decimal). Both are gaps in the other direction and belong in their
own changes.

Tests

pytest vortex-python/test/test_scalar.py: 27 passed, 13 before — the file had no decimal coverage.
Cases span negative values, scale 0, negative scale down to -128, and scale past 38, and assert
the string form rather than numeric equality, since Decimal("-7") == Decimal("-7.0") would let the
scale-0 bug through. pytest vortex-python/test: 354 passed, 2 skipped, 1 xfailed.

For the non-hollowness check the production change was stashed and the extension rebuilt after
uv cache clean vortex-data, since --reinstall-package alone reuses the cached wheel: 5 of the
new cases fail, one of them as a Rust panic through the FFI boundary. The negative-scale path was
probed separately under a 2 GiB RLIMIT_DATA cap so a wrapped pad width could not take the machine
down; it panics at the pow before reaching the formatter.

uvx ruff format --check, uvx ruff check, uvx ty check vortex-python,
cargo +nightly fmt --all --check, cargo clippy --release -p vortex-python --all-features --all-targets --no-deps: all clean.

AI assistance

Written with agentic AI assistance; every row of the table above was reproduced before and after the
change. The decimal(10,3) row was not something I predicted — I found it after noticing the
storage width comes from the value.

…xponent

`decimal_value_to_py` split the stored integer into whole and fractional
parts and pasted them together as `"{whole}.{fraction}"`. That reimplements
three things `Decimal` already does, and got all three wrong:

- a negative value kept its sign on both parts, so `decimal(10,2)` storing
  -12345 rendered as `-123.-45` and `Decimal` raised `InvalidOperation`;
- the pad width came from `scale as usize`, which wraps to ~1.8e19 for a
  negative scale;
- the factor was `10.pow(scale.unsigned_abs())` in the *storage* type, and
  the storage type is the narrowest that holds the value, so an everyday
  `decimal(10,3)` storing 5 evaluated `10i8.pow(3)`.

Scale 0 was wrong too: it emitted a `.0` the dtype does not have.

Pass `Decimal` the unscaled integer and an exponent instead. It parses that
form exactly — the context precision bounds arithmetic, not construction —
and the result carries an exponent of exactly `-scale`, so the dtype's scale
survives the round trip. The exponent is negated through `i16` because
`-i8::MIN` overflows.

This drops the `DecimalIntoParts` trait, its macro and the `i256` impl.

Signed-off-by: jackylee-ch <qcsd2011@gmail.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant