红宝石浮点错误

问题描述

有人可以解释为什么乘以100这里给出的结果不太准确，但乘以10两次会得出更准确的结果吗？

 ±％sc 
加载开发环境（Rails 3.0.1）
>> 129.95 * 100 
 12994.999999999998 
>> 129.95 * 10 
 1299.5 
>> 129.95 * 10 * 10 
 12995.0 
  

解决方案

你用双精度二进制进行计算，这个二进制限制在53个有效位，你会看到发生了什么：

129.95 = 1.0000001111100110011001100110011001100110011001100110 x 2 ^ 7

129.95 * 100 = 1.10010110000101111111111111111111111111111111111111111111111111111111111111111 x 2 ^ 13

这是56位长，所以四舍五入到53它是

1.10010110000101111111111111111111111111111111111111111111111111111111111111 x 2 ^ 13，等于

12994.999999999998181010596454143524169921875

现在129.95 * 10 = 1.01000100110111111111111111111111111111111111111111111 x 2 ^ 10

这是54位有效位长，所以取整为53位1.01000100111 x 2 ^ 10 = 1299.5

现在1299.5 * 10 = 1.1001011000011 x 2 ^ 13 = 12995。

Can somebody explain why multiplying by 100 here gives a less accurate result but multiplying by 10 twice gives a more accurate result?

± % sc
Loading development environment (Rails 3.0.1)
>> 129.95 * 100
12994.999999999998
>> 129.95*10
1299.5
>> 129.95*10*10
12995.0

解决方案

If you do the calculations by hand in double-precision binary, which is limited to 53 significant bits, you'll see what's going on:

129.95 = 1.0000001111100110011001100110011001100110011001100110 x 2^7

129.95*100 = 1.1001011000010111111111111111111111111111111111111111011 x 2^13

This is 56 significant bits long, so rounded to 53 bits it's

1.1001011000010111111111111111111111111111111111111111 x 2^13, which equals

12994.999999999998181010596454143524169921875

Now 129.95*10 = 1.01000100110111111111111111111111111111111111111111111 x 2^10

This is 54 significant bits long, so rounded to 53 bits it's 1.01000100111 x 2^10 = 1299.5

Now 1299.5 * 10 = 1.1001011000011 x 2^13 = 12995.

这篇关于红宝石浮点错误的文章就介绍到这了，希望我们推荐的答案对大家有所帮助，也希望大家多多支持！