PDA

View Full Version : IE and floating-point mathematics?



Twey
03-05-2006, 04:22 PM
It just came to my attention that, according to IE's Javascript engine, 0.06 + 0.01 == 0.0699999999999999, but that 0.07 + 0.01 == 0.08.
Also, (element.filters["Alpha"].opacity = 0.58) == 0.57.
Eh?
Any way around this (yet another) apparent bug?

jscheuer1
03-05-2006, 05:31 PM
Alpha opacity is from 0 to 100.

Twey
03-05-2006, 05:39 PM
Ack, I could have sworn I edited this. Sorry. Yes, I did figure that out. Though how 0.06 + 0.01 comes to be 0.06999999999999999, I've still to figure out. This actually appears in every browser; it's only in IE that it breaks the script (due to alpha opacity not working with decimals). I've sorted it with a parseInt in the right place, but still curious as to why it happened.

jscheuer1
03-05-2006, 05:53 PM
Hmm, the easy way around would be to make the script more IEcentric, this also would probably make for simpler math all the way around. Set and compute your numbers as whole numbers, only dividing by 100 for other browsers.

mwinter
03-05-2006, 07:32 PM
It just came to my attention that, according to IE's Javascript engine, 0.06 + 0.01 == 0.0699999999999999, but that 0.07 + 0.01 == 0.08.That's expected.


Also, (element.filters["Alpha"].opacity = 0.58) == 0.57.A little more curious, but there's a golden rule that applies to such a case.


Eh?
Any way around this (yet another) apparent bug?It's not a bug, but the inherent inaccuracy of floating-point number formats, and isn't limited to ECMAScript. Not all numbers can be represented exactly, so they may occasionly need to be approximated. Due to this problem, floating-point values shouldn't be compared for equality under most circumstances. A certain amount of error should be factored in, and a relational comparison should be used instead.

If you want exact arithmetic, use integers, preferably using string manipulation to perform the transformation (you've seen my toCurrency function before, right?)

The comp.lang.javascript FAQ (http://www.jibbering.com/faq/#FAQ4_7) has a brief description of this, with more discussion in the group itself.

Mike

Twey
03-05-2006, 07:57 PM
The comp.lang.javascript FAQ has a brief description of thisAha, now I see. I was vaguely aware of the "precision vs. accuracy" problem, but didn't see how it applied; I hadn't considered the problem of intermediates. That's cleared it up for me, and will indubitably be very useful in the future; thanks.
you've seen my toCurrency function before, right?Indeed. However, the only difficulty here is in deciding in which direction to round; I'm sure no-one's going to notice a 0.00000000001 difference in every opacity step out of fifty. :)

jscheuer1
03-05-2006, 08:18 PM
Switch to integers, you can see examples of this in several DD scripts.

techno_race
04-20-2007, 01:45 AM
Literally, 0.06 + 0.01 = 0.07, NOT 0.06999999999999...

jscheuer1
04-20-2007, 04:44 AM
Since we seem to be resurrecting this old thread, I thought I might just throw in here that, for many years 'Calculator', which has shipped with Widows (misspelling intentional) from the very beginning, was inaccurate with these or some other basic types of math problems. They finally fixed this late in the reign of 3.01 or, actually I think, 98.

Bob90
04-20-2007, 10:04 AM
But isn't the fix to actually calculate the expression to 14 decimal places then round the display to 12 decimal places.
If the 14 decimal places are 0.69999999999999 then to 12 places this is 0.7 ?!?

Twey
04-20-2007, 10:29 AM
Aye, if it can be known to what precision the number is being stored.

mwinter
04-20-2007, 11:07 AM
But isn't the fix to actually calculate the expression to 14 decimal places then round the display to 12 decimal places.

It really depends what the point is. If accuracy is paramount (for instance, when working with currency) then scaled integers are the way to go. Otherwise rounding will do. That said, the Math.round method only rounds to integers, anyway.

Mike

jscheuer1
04-20-2007, 02:08 PM
It really depends what the point is.

Absolutely. And, since in javascript at least, it is better (more efficient, less resources) to deal with whole numbers, if the point is (getting back to style opacity) to come up with a result representative* of the range 0 to 100 we require little or, I think maybe, no floating point math.



*I say representative because, we also need a corresponding value in the range 0 to 1 when doing this.