Michael Jones 2cb4731902 [libc] adjust strtofloat precision for subnormals ...

Subnormal floating point numbers have a lower effective precision than
normal floating point numbers. This can cause issues for the fuzz test
since the MPFR floats have a constant precision regardless of the
exponent, and the precision must match exactly or else create rounding
errors. To solve this problem, the precision of the MPFR floats is
dynamically calculated.

Reviewed By: lntue

Differential Revision: https://reviews.llvm.org/D154909

2023-07-11 11:27:19 -07:00

5.7 KiB

Raw Blame History

View Raw