Near DBL_MAX, ROUND returns NaN bit patterns that Excel treats as numbers

FN-ROUND-003: Near DBL_MAX, ROUND returns NaN bit patterns that Excel treats as numbers

Excel differs from the mathematical answer · measured · recorded 2026-11-04

ROUND of a number near 1.8E308 can return a binary64 with an all-ones exponent field. A decimal result of 2^1024 or more keeps its 53-bit significand, and its exponent is carried into the exponent field without a check, so the bits are a NaN. ROUND, ROUNDDOWN and TRUNC return those bits; ROUNDUP gives #NUM!. The recorded rows hold 430 such ROUND rows, 102 ROUNDDOWN and 95 TRUNC, and all 627 follow this. Even ROUND(DBL_MAX,0), whose answer is DBL_MAX, returns 0x7ff000000000000a. The cell shows 2E+308 or 1.8E+308, and a pattern with its sign bit set shows 2E+308 too, with no minus. ISNUMBER gives TRUE, ISERROR FALSE, TYPE 1, ERROR.TYPE #N/A; A1=A1 and A1>0 are TRUE. =A1, N(A1), MAX(A1), ABS(A1) and IFERROR(A1,...) pass the pattern on, as does =-A1 with its sign unchanged. Arithmetic on it (A1+0, A1*1, A1-A1, SUM, ROUND) gives #NUM!, and TEXT(A1,"0") gives #VALUE!. Saved to .xlsx, each such cell is written as the error #NUM!; reopened, the cells read #NUM! until a full recalculation brings the patterns back.

Correct value: DBL_MAX is an integer, so rounding it to 0 places returns it.

Correct value: The rounded value exceeds the largest binary64; Excel reports overflow in its own arithmetic as #NUM!.

Correct value: The rounded value exceeds the largest binary64; Excel reports overflow in its own arithmetic as #NUM!.

Basis: One Excel workbook read cell by cell through Value2 on 2026-11-04, saved, reopened and recalculated; rows from the rounding write-up, 627 of 627.

Inferred: That one decimal-to-binary conversion routine is the source of all these patterns is inferred from the exponent-overflow encoding.

Open: How these patterns sort among themselves and with other numbers, and display width, are not recorded.