+ Reply to Thread
Results 1 to 3 of 3

strange result of values comparing

  1. #1
    Registered User
    Join Date
    07-05-2012
    Location
    Kiev, Ukraine
    MS-Off Ver
    Excel 2010
    Posts
    12

    Angry strange result of values comparing

    Hello, guys!
    Hope there is anybody who can help me. It seems that there is something wrong either with my sanity or with excel…
    I have attached the file. See, we have range B1 with formula =A1*0.65, result that we see in B1 is OK. Then in VBA module we take value from A1 and multiply it by 65/100 (result in variable y). Display result by debug.print. The result is OK, it equals B1, right as we could expect. After that we compare value B1 with y and we get result – they are not equal?
    Data type for calculation in module is double, right as Excel itself uses in worksheets formulas. Mode "PrecisionAsDisplayed" is off.
    What is going on?
    Many thanks in advance for your advice.
    Attached Files Attached Files

  2. #2
    Forum Guru Norie's Avatar
    Join Date
    02-02-2005
    Location
    Stirling, Scotland
    MS-Off Ver
    Microsoft Office 365
    Posts
    19,643

    Re: strange result of values comparing

    It's to do with floating point precision/arithmetic.

    If you try this you'll see the difference between x and y is tiny.
    Please Login or Register  to view this content.
    If you put x into B2 and then enter this formula in a spare cell it will return TRUE.
    Formula: copy to clipboard
    Please Login or Register  to view this content.
    If posting code please use code tags, see here.

  3. #3
    Registered User
    Join Date
    07-05-2012
    Location
    Kiev, Ukraine
    MS-Off Ver
    Excel 2010
    Posts
    12

    Re: strange result of values comparing

    Thank you very much. I spent a few night hours trying to localize and understand the problem. Should learn Excel and math deeper.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1