Friday, October 08, 2010
Saturday, January 30, 2010
Friday, September 11, 2009
Floating Point Fun
Long ago, a very smart man once told me that sometimes one doesn't equal one (1 != 1). I thought he was crazy, but now I believe.
In the world of floating-point arithmetic, equality is tough. Below is a screenshot of an instance when 1.0 is less than 0.001 * 1e3:
I can accept this, but the crazy thing is the little help that Visual Studio is giving me says the evaluation is false and the immediate window tells me it's false. Yet, the code has fallen into the if clause. THAT is why it's crazy!
In the world of floating-point arithmetic, equality is tough. Below is a screenshot of an instance when 1.0 is less than 0.001 * 1e3:
I can accept this, but the crazy thing is the little help that Visual Studio is giving me says the evaluation is false and the immediate window tells me it's false. Yet, the code has fallen into the if clause. THAT is why it's crazy!
Subscribe to:
Posts (Atom)
