Floating Point Precision Calculator

Calculate the nearest values around a floating point value, and get an absolute value for precision.

Floating point representations only have a finite number of possible divisions, depending on their precision. Sometimes it's useful to find out exactly what this precision is for a given value.

This tool takes any floating point value (single precision, double precision or long) and returns a table of information about its nearest available value in each direction, in each precision.

A good example is the distance from Earth to Jupiter: 928,081,020,000m. Entering that shows that single precision gives a step error of 65km by the time you"re at Jupiter"s surface - far too much for an accurate simulation. However, double precision is accurate to a tenth of a millimetre, and long double shows no meaningful step size at all.



You may also find the IEEE 754 analyser useful: http://babbage.cs.qc.cuny.edu/IEEE-754/

float:  value                  = 6.000000e+05 ( 600000.000000)
float:  nearest above          = 6.000001e+05 ( 600000.062500)
float:  nearest below          = 5.999999e+05 ( 599999.937500)
float:  minimum increment      = 6.250000e-02 ( 0.062500)
float:  minimum decrement      = 6.250000e-02 ( 0.062500)
double: value                  = 6.000000e+05 ( 600000.000000)
double: nearest above          = 6.000000e+05 ( 600000.000000)
double: nearest below          = 6.000000e+05 ( 600000.000000)
double: minimum increment      = 1.164153e-10 ( 0.000000)
double: minimum decrement      = 1.164153e-10 ( 0.000000)
long double: value             = 6.000000e+05 ( 600000.000000)
long double: nearest above     = 6.000000e+05 ( 600000.000000)
long double: nearest below     = 6.000000e+05 ( 600000.000000)
long double: minimum increment = 0.000000e+00 ( 0.000000)
long double: minimum decrement = 0.000000e+00 ( 0.000000)