Skip to content

round_float with TiesToAway rounds inputs just below half the smallest subnormal up #67

Closed

Description

@Robertboy18

Hi! I was comparing gfloat's P3109 rounding with FloatLib, a Lean library for these formats, and found a small difference between round_float and round_ndarray.

For K = 8, P = 3, signed, extended, the smallest positive value is 2^-17. An input just below half of that should round to 0 under TiesToAway. round_ndarray returns 0, but round_float returns 2^-17:

import math
import numpy as np
import gfloat
from gfloat.formats import format_info_p3109

fi = format_info_p3109(8, 3, gfloat.Signedness.Signed, gfloat.Domain.Extended)
x = math.nextafter(2.0**-18, 0)  # 0x1.fffffffffffffp-19

print(gfloat.round_float(fi, x, gfloat.RoundMode.TiesToAway, False))
# Actual: 7.62939453125e-06 (2^-17)
# Expected: 0.0
print(gfloat.round_ndarray(fi, np.array([x]), gfloat.RoundMode.TiesToAway, False))
# [0.]

Reproduced on main at caea0b1a, with Python 3.12.13 and NumPy 2.5.4.

Here delta is 0.5 - 2^-54, and delta + 0.5 rounds to 1.0 in binary64. The fix in src/gfloat/round.py appears to be the comparison round_ndarray.py already uses:

             case RoundMode.TiesToAway:
-                should_round_away = delta + 0.5 >= 1.0
+                should_round_away = delta >= 0.5

With this change round_float and round_ndarray agree on inputs below, at and above the midpoint, for both signs. I'm happy to send a PR with a regression test :)

Activity

  1. awf commented on Oct 18, 2026

    @awf
    Collaborator

    Thanks! Taking a look.

  2. awf commented on Oct 18, 2026

    @awf
    Collaborator

    Copilot auto-fix did as predicted, so I've merged it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions