+ Reply to Thread
Results 1 to 7 of 7

Excel 2007 : Calculation issue?

  1. #1
    Registered User
    Join Date
    09-27-2011
    Location
    London
    MS-Off Ver
    Excel 365
    Posts
    3

    Calculation issue?

    Hello,

    I have a funny calculation issue using Excel 2007.

    I have the following values in on my column A starting in row 1:
    0.0100
    0.0002
    0.0100
    Now when I add row 1 and 2 and subtract row 3, the result is 0.000200000000000001,
    i.e. there is a 1 on the 18th decimal. Is this a known issue?
    Thanks for helping!

  2. #2
    Forum Moderator zbor's Avatar
    Join Date
    02-10-2009
    Location
    Croatia
    MS-Off Ver
    365 ProPlus
    Posts
    15,602

    Re: Calculation issue?

    Check values in A1, A2 and A3. probably those a bigger numbers but only 4 digits shown.

  3. #3
    Forum Expert Bob Phillips's Avatar
    Join Date
    09-03-2005
    Location
    Wessex
    MS-Off Ver
    Office 2003, 2010, 2013, 2016, 365
    Posts
    3,284

    Re: Calculation issue?

    Yes, it is a common problem in all computers in that use floating point arithmetic, it is difficult to be precise in representing decimal numbers when you only have a binary engine. Take a look at http://support.microsoft.com/kb/42980

  4. #4
    Registered User
    Join Date
    09-27-2011
    Location
    London
    MS-Off Ver
    Excel 365
    Posts
    3

    Re: Calculation issue?

    Interesting. The numbers were typed in as above. And the formula =A1+A2-A3=A2 is giving me FALSE as a result.

  5. #5
    Forum Expert Bob Phillips's Avatar
    Join Date
    09-03-2005
    Location
    Wessex
    MS-Off Ver
    Office 2003, 2010, 2013, 2016, 365
    Posts
    3,284

    Re: Calculation issue?

    See what this returns

    =A1+A2-A3-A2

  6. #6
    Forum Moderator zbor's Avatar
    Join Date
    02-10-2009
    Location
    Croatia
    MS-Off Ver
    365 ProPlus
    Posts
    15,602

    Re: Calculation issue?

    Yes, really interesting... It state TRUE for 1, 0.1, 0.01 but FALSE for 0,001 and lower (in A2)

  7. #7
    Registered User
    Join Date
    09-27-2011
    Location
    London
    MS-Off Ver
    Excel 365
    Posts
    3

    Re: Calculation issue?

    Quote Originally Posted by Bob Phillips View Post
    See what this returns

    =A1+A2-A3-A2
    It's 0.000000000000000001 so the same difference.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1