Skip to content

fix(js/ts): round decimals below 0.1 correctly - #5116

Open
sasmithjr wants to merge 1 commit into
fable-compiler:mainfrom
sasmithjr:20261020-decimal-leading-zero
Open

sasmithjr wants to merge 1 commit into
fable-compiler:mainfrom
sasmithjr:20261020-decimal-leading-zero

Conversation

@sasmithjr

Copy link
Copy Markdown
Contributor

A decimal built from a number or a string with a nonzero value whose magnitude is below 0.1 rounds at the wrong place:

Expression .NET Fable before
Math.Floor(decimal -0.0836) -1 -0.1
Math.Floor(Decimal.Parse "-0.0000123") -1 -0.0001
Math.Ceiling(decimal 0.0836) 1 0.1
Math.Round(decimal 0.06) 0 0.1
Math.Round(decimal 0.0836, 2) 0.08 0.084

Decimal literals and arithmetic results aren't affected: Math.Floor(0M - 0.0836M) is -1.

Cause

parse() in lib/big.js strips leading zeros only before the decimal point, so that zero keeps its scale (0.00). For a nonzero value whose magnitude is below 0.1 this also keeps the zeros after the point: -0.0836 is stored as c = [0,8,3,6], e = -1 instead of c = [8,3,6], e = -2. P.round works out where to round from this.e, but rounds a copy, and the copy constructor normalizes. So the precision calculation uses a different exponent from the value being rounded. #4962 fixed the formatting side of the same representation, and 1c02557 worked around it in toFixed.

The same representation made GetHashCode differ between equal decimals, since it hashes c and e directly. As a result, HashSet<decimal> and Dictionary<decimal, _> treated decimal -0.0836 and 0M - 0.0836M as different keys.

Fix

After parse()'s existing rounding step, normalize the value when it has a nonzero digit. Zero is left as is, so 0.00 still keeps its scale. The rounding to Big.DP + 1 digits happens before the normalization, so precision is unchanged.

Testing

  • Two test cases in tests/Js/Main/ArithmeticTests.fs: rounding below 0.1 (for decimals from doubles, strings and literals), and hash consistency (GetHashCode, HashSet, Dictionary). Both fail without the fix and pass with it. They also pass on .NET.
  • The JavaScript and TypeScript suites pass.

🤖 Generated with Claude Code

A decimal built from a number or a string with a nonzero value whose
magnitude is below 0.1, such as `decimal -0.0836` or
`Decimal.Parse "-0.0836"`, kept the zeros after the decimal point at the
start of its digits. Its stored exponent therefore differed from the
normalized exponent of the copy that round() rounds, so Math.Floor,
Math.Ceiling and Math.Round rounded at the wrong decimal place:
Math.Floor(decimal -0.0836) returned -0.1 instead of -1, and
Math.Round(decimal 0.0836, 2) returned 0.084 instead of 0.08.
GetHashCode also differed from that of an equal decimal produced by
arithmetic, which broke HashSet and Dictionary lookups.

Strip those leading zeros in parse() when the value has a nonzero digit.
This gives these values the normalized form that copies and arithmetic
results already have. Zero keeps them, to preserve its scale (0.00).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant