+ Reply to Thread
Results 1 to 4 of 4

Strange compare result

  1. #1
    Registered User
    Join Date
    08-12-2008
    Location
    va
    MS-Off Ver
    MS Office Ultimate 2007
    Posts
    12

    Strange compare result

    Cell H520 contains this formula:
    =IF(H507=P522,"Ally Savings totals: match.","Ally Savings totals: NO MATCH!")

    H507 contains a summed formula for a range of rows and correctly shows the value of $991.50.
    P522 contains a summed formula for a different range of different rows and correctly shows the value of $991.50.
    Yet the result of the H520 cell formula returns false instead of true when both H507 and P522 are compared.
    So I evaluated each part of that formula through "Evaluate Formula" button in the formula menu item of Excel 2007:

    H507 evaluates to $991.500000000001 instead of $991.50,
    which triggers a false Boolean result instead of true.
    P522 correctly evaluates to $991.50.
    Both H507 and P522 are formatted in "Accounting" format.

    See the evaluate box:
    formula-error.jpg


    How do I fix this?

  2. #2
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,833

    Re: Strange compare result

    The cause is standard computer round off error inherent in all computer calculations.

    Around here, the first suggestion is to use the ROUND() function somewhere in there. Either put the ROUND() function around the SUM() function in H507 or put the ROUND function around the H507 reference in the IF() function. Considering that this sort of thing is part of all computer calculations, you probably also need to put a ROUND() function around P522 (and any other formulas in the sheet that could possibly be effected by round off error).
    Quote Originally Posted by shg
    Mathematics is the native language of the natural world. Just trying to become literate.

  3. #3
    Forum Contributor
    Join Date
    09-18-2015
    Location
    Republic of Korea
    MS-Off Ver
    2010
    Posts
    314

    Re: Strange compare result

    https://support.microsoft.com/en-au/...int-arithmetic

  4. #4
    Registered User
    Join Date
    08-12-2008
    Location
    va
    MS-Off Ver
    MS Office Ultimate 2007
    Posts
    12

    Re: Strange compare result

    Quote Originally Posted by MrShorty View Post
    The cause is standard computer round off error inherent in all computer calculations.

    Around here, the first suggestion is to use the ROUND() function somewhere in there. Either put the ROUND() function around the SUM() function in H507 or put the ROUND function around the H507 reference in the IF() function. Considering that this sort of thing is part of all computer calculations, you probably also need to put a ROUND() function around P522 (and any other formulas in the sheet that could possibly be effected by round off error).
    Thanks, this worked. I put it around the sum function in H507. Will consider putting it in other places.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] NextRow VBA giving strange result...
    By sintek in forum Excel Programming / VBA / Macros
    Replies: 9
    Last Post: 05-09-2017, 06:05 AM
  2. [SOLVED] Strange xldown result
    By JRidge in forum Excel Programming / VBA / Macros
    Replies: 11
    Last Post: 11-05-2014, 06:56 AM
  3. How to make a pie?...strange result!
    By belfagor in forum Excel Charting & Pivots
    Replies: 17
    Last Post: 11-12-2013, 07:46 AM
  4. [SOLVED] strange result of values comparing
    By blackarrow in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 02-10-2013, 08:55 AM
  5. Vlookup Returns Strange Result
    By ZooTV92 in forum Excel General
    Replies: 5
    Last Post: 09-04-2011, 09:28 AM
  6. Strange MOD() result?
    By rwgrietveld in forum Excel Formulas & Functions
    Replies: 4
    Last Post: 09-11-2008, 11:02 AM
  7. strange result from SpecialCells
    By seron in forum Excel Programming / VBA / Macros
    Replies: 5
    Last Post: 11-26-2006, 02:53 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1