Floating-point comparisons
ID | STAT-19 |
Title | Do not compare floating-point values using == or ~= . |
Priority | Strongly recommended |
Severity level | 3 |
Description | Do not compare floating-point values using == or ~= . Use a tolerance instead. |
Rationale | Rounding errors due to algorithm design or machine precision can cause unexpected inequalities. |
Avoid:
out = myFcn(in) == sqrt(3);
Instead use:
out = abs(myFcn(in) - sqrt(3)) < 1e-12;