+ Reply to Thread
Results 1 to 14 of 14

Error in calculating differences

  1. #1
    Registered User
    Join Date
    12-22-2022
    Location
    Weymouth
    MS-Off Ver
    Microsoft 365
    Posts
    18

    Error in calculating differences

    Say, in cell A1, you have =180*21850, in cell B1, you have 3933000. Then in cell C1, you have the formula =IF(B1>A1,B1-A1,A1-B1), which Excel computes to be zero

    However, if I do it slightly differently as follows:
    in cell A2, you have =3420000*1.15, in cell B2, you have 3933000. Then in cell C2, you have the formula =IF(B2>A2,B2-A2,A2-B2), which Excel returns the value -4.65661E-10.

    Is this a bug? I would expect Excel to return the value 0 in cell C2 too. Everything is in nice whole numbers.


    Thank you for your help!

  2. #2
    Forum Moderator Glenn Kennedy's Avatar
    Join Date
    07-08-2012
    Location
    Digital Nomad... occasionally based in Ireland.
    MS-Off Ver
    O365 (PC) V 2403
    Posts
    44,036

    Re: Error in calculating differences

    No. It's floating point arithmetic. If you encounter this, you need to round your data to an appropriate level of precision.

    https://learn.microsoft.com/en-GB/of...ccurate-result
    Glenn




    None of us get paid for helping you... we do this for fun. So DON'T FORGET to say "Thank You" to all who have freely given some of their time to help YOU.

    Temporary addition of accented to illustrate ongoing problem to the TT: L? fh?ile P?draig sona dhaoibh

  3. #3
    Banned User!
    Join Date
    02-05-2015
    Location
    San Escobar
    MS-Off Ver
    any on PC except 365
    Posts
    12,168

    Re: Error in calculating differences

    calculated with Power Query
    Column1 Column2 Column3 Result
    3420000
    1.15
    3933000
    0.000000000465661287307739

  4. #4
    Registered User
    Join Date
    12-22-2022
    Location
    Weymouth
    MS-Off Ver
    Microsoft 365
    Posts
    18

    Re: Error in calculating differences

    Thank you! One more thing, why wouldn't Excel at least generate a positive figure, such as 4.65661E-10?

  5. #5
    Forum Expert avk's Avatar
    Join Date
    07-12-2007
    Location
    India
    MS-Off Ver
    Microsoft Office 2013
    Posts
    3,223

    Re: Error in calculating differences

    Its a decimal value difference.


    atul


    If my answer (or that of other members) has helped you, please say "Thanks" by clicking the Add Reputation button at the foot of one of their posts.

    Also: if your problem is solved, please take the time to mark your thread as SOLVED by going to the top of your first post, selecting "Thread Tools" and then "Mark thread as solved".

  6. #6
    Banned User!
    Join Date
    02-05-2015
    Location
    San Escobar
    MS-Off Ver
    any on PC except 365
    Posts
    12,168

    Re: Error in calculating differences


  7. #7
    Registered User
    Join Date
    12-22-2022
    Location
    Weymouth
    MS-Off Ver
    Microsoft 365
    Posts
    18

    Re: Error in calculating differences

    Thanks! Although I can't get over with the fact (?) that there are some decimals lingering out there when 3420000 x 1.15 gives you a nice whole number ...

  8. #8
    Banned User!
    Join Date
    02-05-2015
    Location
    San Escobar
    MS-Off Ver
    any on PC except 365
    Posts
    12,168

    Re: Error in calculating differences

    you just have to love it

  9. #9
    Valued Forum Contributor
    Join Date
    07-13-2021
    Location
    California, USA
    MS-Off Ver
    2010
    Posts
    513

    Re: Error in calculating differences

    Quote Originally Posted by Puiman View Post
    why wouldn't Excel at least generate a positive figure, such as 4.65661E-10?
    Because B2>A2 is false, but B2-A2>0 is true(!).

    It is a quirk of Excel: for comparison operators, Excel rounds each operand to 15 significant digits internally just for the comparison.

    So B2=A2 returns TRUE, even though their binary values are not equal.

    BTW, your original formula could be written more simply as =ABS(B2-A2).

    Then the result would indeed be positive 4.66E-10 (approximately).
    Last edited by curiouscat408; 12-22-2022 at 04:57 AM. Reason: A1,B1 -> A2,B2 to avoid confusion

  10. #10
    Forum Moderator Glenn Kennedy's Avatar
    Join Date
    07-08-2012
    Location
    Digital Nomad... occasionally based in Ireland.
    MS-Off Ver
    O365 (PC) V 2403
    Posts
    44,036

    Re: Error in calculating differences

    Yes, but, yes, but.... 1.15 is not an integer.

    This uses integers throughout and results in a zero:

    =3420000*115/100

  11. #11
    Valued Forum Contributor
    Join Date
    07-13-2021
    Location
    California, USA
    MS-Off Ver
    2010
    Posts
    513

    Re: Error in calculating differences

    Quote Originally Posted by Puiman View Post
    I can't get over with the fact (?) that there are some decimals lingering out there when 3420000 x 1.15 gives you a nice whole number
    It is because most decimal fractions cannot be represented exactly as the sum of 53 consecutive powers of 2, which is how Excel (and most applications) represent numbers internally.

    And the binary approximation of a particular decimal fraction might vary, depending on the magnitude of the number. That is why, for example, 10.01 - 10 = 0.01 returns FALSE(!).

    The binary approximation of 1.15 is exactly 1.149999999999999911182158029987476766109466552734375
    Last edited by curiouscat408; 12-22-2022 at 05:15 AM.

  12. #12
    Forum Moderator Glenn Kennedy's Avatar
    Join Date
    07-08-2012
    Location
    Digital Nomad... occasionally based in Ireland.
    MS-Off Ver
    O365 (PC) V 2403
    Posts
    44,036

    Re: Error in calculating differences

    Quote Originally Posted by curiouscat408 View Post
    The binary approximation of 1.15 is exactly 1.149999999999999911182158029987476766109466552734375
    Did you just happen to know that??

    Happy Christmas!

  13. #13
    Registered User
    Join Date
    12-22-2022
    Location
    Weymouth
    MS-Off Ver
    Microsoft 365
    Posts
    18

    Re: Error in calculating differences

    Definitely a Christmas gift to receive your help on this. Thank you to you all! This morning, I felt my mathematical foundation was shaken but luckily it has something to do with how Excel works inside. Merry Christmas!

  14. #14
    Forum Moderator Glenn Kennedy's Avatar
    Join Date
    07-08-2012
    Location
    Digital Nomad... occasionally based in Ireland.
    MS-Off Ver
    O365 (PC) V 2403
    Posts
    44,036

    Re: Error in calculating differences

    You're welcome.



    It would be very nice if you were to just click the Add Reputation button at the foot of any of the posts of members who helped you reach a solution.

    Finally, if that takes care of your original question, please click on "Thread Tools" from the menu link (just above the first post in the thread) and mark this thread as SOLVED.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. calculating date differences
    By damon9868 in forum Excel Formulas & Functions
    Replies: 4
    Last Post: 03-04-2020, 04:38 PM
  2. Calculating sums and differences
    By Del209 in forum Excel Formulas & Functions
    Replies: 2
    Last Post: 11-14-2019, 02:07 PM
  3. Calculating Time Differences
    By Don Adeks in forum Excel Formulas & Functions
    Replies: 1
    Last Post: 03-13-2019, 06:48 AM
  4. [SOLVED] Calculating differences in time
    By koochandkai in forum Excel General
    Replies: 6
    Last Post: 04-12-2015, 08:55 PM
  5. Calculating differences between two sheets
    By CanadianGSX in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 06-25-2010, 12:01 PM
  6. calculating differences
    By bluesky in forum Excel General
    Replies: 2
    Last Post: 04-16-2009, 11:06 PM
  7. Calculating Date Differences
    By ignax in forum Excel Formulas & Functions
    Replies: 5
    Last Post: 11-03-2007, 12:00 AM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1