Thursday, September 27, 2012

Foating point comparison bug

While testing the new solver I found that in some cases, the solver behaved differently in debug and release mode. it turned out that the cause was the way floating point comparisons were done in some helper classes, using == instead of comparing the difference to a very small value.

Here's the behaviour without the fix, in release mode (the red line shouldn't be moving):



and after the fix:



The changes are commited here.

After some more reading I found Jon Skeet's article about .NET and floating point comparison where he mentions that the cause of the debug and release differences is JIT optimisation: in debug the JIT doesn't make as many optimisations and the == comparison returns true.

No comments: