0
When I use the windows xp standard calculator (v1.0.0.908) to convert miles to feet it shows 1 mph=5280 feet. But when I convert mph to fph it says 1mph = 5279.98944 feet per hour. And if I do the same thing using the windows power toys calculator it says 1 mph = 1760 feet/hour. Its probably shuwing yards, but I really hesitate to trust either of them for anything other than trivial matters. ps. if you divide 5280 by the error you get exactly 500 000. All other conversions seem to be exact (m/s-mph, kmh-m/s) its just the one involving feet.
Any ideas?
Rbma12
Posted 2016-11-23T19:45:47.717
Reputation: 11
3What's your question? – gronostaj – 2016-11-23T19:53:08.853
Why is there an error? – Rbma12 – 2016-11-23T22:31:03.820
Is it a programming error, rounding error, a symptom of the impending death of my computer? Is the error global? Although I should have mentioned I just installed service pack 4 on a XP pro sp3. – Rbma12 – 2016-11-23T22:37:19.243
Floating point precision problems. All computers do this, but it doesn't really matter in everyday uses. Only starts to play significant role in scientific calculations. – gronostaj – 2016-11-23T22:48:10.583
In school I ran into this problem with a simple temperature conversion program, but only with negative values. This calculator's help menu uses the phrase "extreme precision' about 10 times. It's a simple calculation. – Rbma12 – 2016-11-23T22:56:10.120
Read up about floating point numbers and accuracy errors associated with them. Long story short, computers can't store all numbers with exact precision, they use good approximations. Over time these small deviations add up and you end up with results such as
– gronostaj – 2016-11-24T07:16:42.8200.1 + 0.2 = 0.30000000000000004or what you've discovered.