+ Reply to Thread
Results 1 to 4 of 4

Is it an INTEGER precision problem ?

  1. 03-31-2010, 08:03 AM #1
    ElmerS
    ElmerS is offline
    Forum Contributor
    Join Date
    11-28-2008
    Location
    Reykjavík, Iceland
    MS-Off Ver
    Any of: 2003 & 2007
    Posts
    412

    Is it an INTEGER precision problem ?

    Hi,

    Where can I read some detailed material regarding all those "weird" FALSEs results in the attached WB.

    Thanks, Elm
    Attached Files Attached Files
    Last edited by ElmerS; 03-31-2010 at 08:42 AM.
    Reply With QuoteRegister To Reply
  2. 03-31-2010, 08:08 AM #2
    shg
    shg is offline
    Forum Expert shg's Avatar
    Join Date
    06-20-2007
    Location
    The Great State of Texas
    MS-Off Ver
    2003, 2010
    Posts
    40,678

    Re: Is it an INTEGER precision problem ?

    It's a floating-point precision issue.
    Entia non sunt multiplicanda sine necessitate
    Reply With QuoteRegister To Reply
  3. 03-31-2010, 08:27 AM #3
    DonkeyOte
    DonkeyOte is offline
    Forum Guru DonkeyOte's Avatar
    Join Date
    10-22-2008
    Location
    Northumberland, UK
    MS-Off Ver
    O365
    Posts
    21,531

    Re: Is it an INTEGER precision problem ?

    http://support.microsoft.com/kb/78113
    My Recommended Reading:

    Volatility
    Sumproduct & Arrays
    Pivot Intro
    Email from XL - VBA & Outlook VBA
    Function Dictionary & Function Translations
    Dynamic Named Ranges
    Reply With QuoteRegister To Reply
  4. 03-31-2010, 08:41 AM #4
    ElmerS
    ElmerS is offline
    Forum Contributor
    Join Date
    11-28-2008
    Location
    Reykjavík, Iceland
    MS-Off Ver
    Any of: 2003 & 2007
    Posts
    412

    Re: Is it an INTEGER precision problem ?

    Thank you, both, for replying.

    Thanks, DonkeyOte, for the detailed material as requested.

    Elm
    Reply With QuoteRegister To Reply
+ Reply to Thread
« Previous Thread | Next Thread »

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Forum Rules


Search Engine Friendly URLs by vBSEO 3.6.0 RC 1