+ Reply to Thread
Results 1 to 3 of 3

Excel Formula Error

  1. #1
    Forum Contributor
    Join Date
    02-24-2008
    Location
    UK
    MS-Off Ver
    2012 and 2016
    Posts
    123

    Excel Formula Error

    I have a spreadsheet cell that compares 2 values and produces a message depending on whether the values are the same or not. The formula is

    =IF(G196=0,"OKAY","ERROR") and is shown on the attachment.

    As can be seen, the value is 0.00 and so the message should be 'OKAY' but I receive the ERROR message. I have checked the values that are being compared to 20 decimal places and the are the same and so the formula =IF(Summary!C6="","",E196-Summary!D37) should be zero to 20 decimal places.

    However when I expand the value 0.00 to 20 decimal places there is a value that is not exactly zero as shown on the second attachment.

    I do not understand why the subtraction of 2 numbers that are exactly the same (to 20 decimal places) should produce a small difference when subtracted.

    Please help.
    Attached Images Attached Images

  2. #2
    Forum Expert Sam Capricci's Avatar
    Join Date
    06-14-2012
    Location
    Palm Harbor, Florida
    MS-Off Ver
    16.84 for Mac MS 365
    Posts
    8,496

    Re: Excel Formula Error

    That little green triangle in the upper left corner of your cell where the 0.00 is, that means it is text. Your formula is calling for a numeric zero. If you want it to match try putting the zero in your formula in quotes or change the formula that produces the zero so that it’s not text.
    Make contributors happy, click on the "* Add Reputation" as a way to say thank you.
    Sam Capricci

  3. #3
    Valued Forum Contributor
    Join Date
    07-13-2021
    Location
    California, USA
    MS-Off Ver
    2010
    Posts
    513

    Re: Excel Formula Error

    Quote Originally Posted by kencoburn View Post
    As can be seen, the value is 0.00
    [....]
    I do not understand why the subtraction of 2 numbers that are exactly the same (to 20 decimal places) should produce a small difference when subtracted.
    __We__ cannot see much of anything because you did not attach an Excel file, per instructions at the top of the page.

    So we can only answer in generalities.

    First, the solution.... In general, when you expect a calculation to be accurate to some number of decimal places, you should use ROUND to explicitly round to that number of decimal places -- and not to an arbitrary number like 10, as some people suggest.

    For example, change E196-Summary!D37 to ROUND(E196-Summary!D37, 2) .

    The operative word is "expect". Sometimes, it is prudent to delay rounding. Only you can make that decision.

    Yes, it is tedious to explicitly round as needed. But I do not recommend setting the "Precision as displayed" option as an alternative, as someone might suggest. There are oh-so many reasons.

    -----

    Second, the reason.... In general, this is a side-effect of 64-bit binary floating-point, which is how Excel represents numbers internally. It is not a defect. It is a consequence of a design decision that is common to most applications.

    In a nutshell, most decimal fractions cannot be represented exactly in binary. And the binary approximation of a particular decimal fraction might vary depending on the magnitude of the number.

    Consequently, for example, 10.01 - 10 = 0.01 returns FALSE(!). You can see why by looking at =10.01 - 10 formatted to display 17 decimal places, but not by looking at 10.01 itself. The approximation of 0.01 in 10.01 is different from the approximation of 0.01 itself.

    It is possible that there are other factors to consider, as well. For example, additional rounding might be prudent in the formulas(?) in E196 and Summary!D37.

    I can provide more details if you attach an Excel file that demonstrates the problems.

    -----
    PS, possibly TMI....

    I guess we can assume that E196 and Summary!D37 display 3538.29 -- but it would be prudent to format with 11 decimal places to be sure.

    Presuably, one or both are calculated.

    We can see the infinitesimal difference between displayed value (arbitrarily limited to 15 significant digits) and binary value with formulas of the following form, formatted as General or Scientific.

    =SUM(E196, -ROUND(E196, 2))
    =SUM(Summary!D37, -ROUND(Summary!D37, 2))

    We must use SUM instead of =E196-ROUND(E196,2), for example, because for the latter form, Excel sometimes arbitrarily replaces the exact arithmetic difference with exactly zero in order to hide the infinitesimal differences, a dubious "feature" (sigh).
    Last edited by curiouscat408; 10-08-2021 at 11:56 AM. Reason: added detail (PS)

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Excel if formula error
    By zabad in forum Excel Formulas & Functions
    Replies: 3
    Last Post: 07-14-2021, 10:21 PM
  2. Build formula(Like Goal Seek)
    By Masa1989 in forum Excel General
    Replies: 0
    Last Post: 03-20-2020, 04:36 AM
  3. Excel 2003 > excel 2010 formula error (VALUE)
    By PENGUIN88 in forum Excel Formulas & Functions
    Replies: 20
    Last Post: 01-10-2015, 07:59 AM
  4. [SOLVED] Excel does not evaluate a formula or possible formula error ?
    By syed82 in forum Excel Formulas & Functions
    Replies: 4
    Last Post: 12-03-2014, 11:08 PM
  5. Excel Formula gives error
    By gpatel in forum Excel General
    Replies: 8
    Last Post: 12-21-2011, 05:29 AM
  6. Excel Formula Error
    By lozg43 in forum Excel General
    Replies: 5
    Last Post: 08-25-2009, 10:49 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1