Above 0.5, ERFC rounds x*x before the exponential, so the tail drifts by hundreds of units

FN-ERFC-001: Above 0.5, ERFC rounds x*x before the exponential, so the tail drifts by hundreds of units

Excel differs from the mathematical answer · measured · recorded 2026-11-04

With RN(v) meaning v rounded to 64 bits on the x87 unit and then stored to binary64, ERFC for x >= 0.5 behaves as Q = E*F with z = RN(x*x) and E = exp(-z) stored to binary64, then multiplied by a factor F that is not identified. So the rounding error of x*x reaches the answer at full size. At x = 26.543000000000003, x*x exceeds z by 4.64E-14, which at the size of this answer is about 212 units, and Excel's answer is 212 units above the correctly rounded erfc. Near x = 5.93 the same effect is 23 units. For small x, where x*x is exact or nearly so, the answer stays within a few units.

Correct value: mpmath erfc at 800 bits from the binary64 input, correctly rounded.

Correct value: mpmath erfc at 800 bits from the binary64 input, correctly rounded.

Correct value: mpmath erfc at 800 bits from the binary64 input, correctly rounded.

Basis: 220,000 Excel answers from 0.5 to 26.5 (build 20527) compared with E times the true exp(x*x)*erfc(x). With z = RN(x*x) and E stored, the residual stays within about 10 units of 2^-53 at every x. At x = 24 the exact exp(-x*x) spreads with a standard deviation of 287 units, Cody's and fdlibm's fractional splits of x*x spread the same, and plain binary64 x*x leaves outliers of up to 1,024 units.

Inferred: F is not a closed form: y = RN(E*RN(F)) with any offset matches 40% of answers at best. Its noise tracks about a dozen smooth curves, consistent with two polynomials of degree about 8 in x with shapes near Cephes' erfc P and Q, but that is a reading of shape only, and no table has been matched.

Open: F, so the last bits: with a stand-in for F, recorded rows match ERFC 234 of 544 and ERFC.PRECISE 1,062 of 1,863, with no answer more than 6 units away.

Open: The mean of the residual is flat with sharp steps of about 0.4 units, at x = 1.11567925... and 1.1217 among about 80 in [0.5, 27]. No stored polynomial value varies that slowly, and this is unexplained.