+ Reply to Thread
Results 1 to 6 of 6

Low decimals incorrect in simple formula

  1. #1
    Registered User
    Join Date
    11-05-2012
    Location
    Sweden
    MS-Off Ver
    Excel 2010
    Posts
    3

    Question Low decimals incorrect in simple formula

    I get a weird result to these formulas when looking at many decimal places, anyone knows why? Especially weird that I dont get it on all of them?

    =94%-1 ==> -6,00000000000001000000%
    =8%+1 ==> 8,00000000000001000000%
    =15%-1 ==> -85,00000000000000000000%

    Best Regards
    Rasmus
    Last edited by kretzen; 11-05-2012 at 08:47 AM.

  2. #2
    Registered User
    Join Date
    10-29-2012
    Location
    hyderabad, India
    MS-Off Ver
    Excel 2007
    Posts
    53

    Re: formula error

    hi kretzen,

    in the above formula, '1' would relate to 100%. hence the results.

    however, "=8%+1", should have given you 108%.

  3. #3
    Registered User
    Join Date
    11-05-2012
    Location
    Sweden
    MS-Off Ver
    Excel 2010
    Posts
    3

    Re: formula error

    Hi.

    My question is not on how % work (although, you are correct, I missed the first two digits on the 1+8%).

    My question is why excel would invent the "1" in the 13:th decimal place?
    Last edited by kretzen; 11-05-2012 at 08:48 AM.

  4. #4
    Forum Moderator zbor's Avatar
    Join Date
    02-10-2009
    Location
    Croatia
    MS-Off Ver
    365 ProPlus
    Posts
    15,645

    Re: Low decimals incorrect in simple formula

    Numbers are stored in a computer in the binary number system.
    Conversion from base 10 to base 16, base 8, or base 2 will almost always introduce error since the computer is using a fixed amount of memory to store this new representation of the data.

    that's why you can have this error.

  5. #5
    Registered User
    Join Date
    11-05-2012
    Location
    Sweden
    MS-Off Ver
    Excel 2010
    Posts
    3

    Re: Low decimals incorrect in simple formula

    Ok!

    Thank you!

  6. #6
    Forum Expert Pepe Le Mokko's Avatar
    Join Date
    05-14-2009
    Location
    Belgium
    MS-Off Ver
    O365 v 2402
    Posts
    13,462

    Re: Low decimals incorrect in simple formula


+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1