Excel saying my range is incorrect when it's not?

Morning all,

I'm having an issue where excel is telling me that inputted results are out of range when they're not.

For example, i'm creating a workbook for Calibration results where data is inputted and it will specify whether the measurement inputted falls within required tolerance

The tolerance is 0.025 (mm) within the workbook you'll see

2.5 (Tolerance would be 2.475 - 2.525 ) This works fine however I noticed on the next example

7.7 (Tolerance would be 7.675 - 7.725 ) if I enter the actual reading taken as 7.675 it tells me this is out of specification when it's not. basically there are certain numbers that if I enter at the bottom end of spec it tells me it's outside of the tolerance defined. I don't get it, I've also had this very same issue in the past. Sorry I'm unable to explain very well as I'm not great with excel, but hopefully some one can help.

They are out of range if you expand decimal places! Change G11 to this:

=IF(ISBLANK(A11),"",ROUND(C11-A11,3))

Use

Formula:

Please Login or Register  to view this content.

If you increase the number of decimal places, you'll see the values are NOT in tolerance by a tiny amount.

If you are wondering why, research floating point error.

Glad to have helped.

If that takes care of your original question, please choose Thread Tools from the menu link above and mark this thread as SOLVED.

You're welcome. Thanks for the rep.