Page 104 - Python Simple
P. 104

B. Floating Point Arithmetic: Issues and Limitations            http://www.vithon.org/tutorial/2.5/node16.html



             So the computer never ``sees'' 1/10: what it sees is the exact fraction given
             above, the best 754 double approximation it can get:

                  >>> .1 * 2**56
                  7205759403792794.0

             If we multiply that fraction by 10**30, we can see the (truncated) value of its
             30 most significant decimal digits:

                  >>> 7205759403792794 * 10**30 / 2**56
                  100000000000000005551115123125L


             meaning that the exact number stored in the computer is approximately
             equal to the decimal value 0.100000000000000005551115123125. Rounding
             that to 17 significant digits gives the 0.10000000000000001 that Python
             displays (well, will display on any 754-conforming platform that does
             best-possible input and output conversions in its C library -- yours may not!).


             Release 2.5, documentation updated on 19th September, 2006.
             See About this document... for information on suggesting changes.
























































     5 of 5                                                                                   08/31/2011 10:52 AM
   99   100   101   102   103   104   105   106   107   108   109