+ Reply to Thread
Results 1 to 3 of 3

Dealing with Rounding Errors

  1. #1
    Forum Contributor stockgoblin42's Avatar
    Join Date
    05-26-2011
    Location
    vancouver, canada
    MS-Off Ver
    Excel 2010
    Posts
    222

    Dealing with Rounding Errors

    Hi,

    I've just started experiencing rounding errors in my work and I wanted to get your guys opinion on the general rule of thumb to dealing with them.


    They've made an appearance as I've begun testing:

    if a point is on a line
    if a point is in a triangle

    So, I've been fooling around with different settings of rounding and trying to get reliable results without any false positives but still not missing any positives. Do you just play with the number of decimal places to round to until you're comfortable or what's the standard approach.

    So far, I seem to find rounding my main data calculations to 12 decimal places and setting the point-line tests rounded to 4 decimal places seems to be giving accurate results.

    Am I going about this wrong or is this just a fact of math?

    thanks for any advice
    live logic & long prosper

  2. #2
    Forum Expert NeedForExcel's Avatar
    Join Date
    03-16-2013
    Location
    Pune, India
    MS-Off Ver
    Excel 2016:2019, MS 365
    Posts
    3,873

    Re: Dealing with Rounding Errors

    This is more of what the person wants.. I always use rounding to two decimal places.. I dont think there is any thumb rule of that sort.

    Deep
    Cheers!
    Deep Dave

  3. #3
    Administrator FDibbins's Avatar
    Join Date
    12-29-2011
    Location
    Duncansville, PA USA
    MS-Off Ver
    Excel 7/10/13/16/365 (PC ver 2310)
    Posts
    52,936

    Re: Dealing with Rounding Errors

    Any time you round, you are going to find errors at some point or other, so unless there is a great need to round, why do it?

    I hardly ever round, but just set the number of decimal points to display
    1. Use code tags for VBA. [code] Your Code [/code] (or use the # button)
    2. If your question is resolved, mark it SOLVED using the thread tools
    3. Click on the star if you think someone helped you

    Regards
    Ford

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] dealing with errors in macros
    By wellsw in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 02-06-2013, 03:11 PM
  2. dealing with errors in an array
    By weeble33 in forum Excel General
    Replies: 4
    Last Post: 08-23-2012, 02:49 PM
  3. Dealing with errors in file save macro
    By tcombest in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 06-20-2012, 02:43 PM
  4. Dealing with errors
    By [email protected] in forum Excel General
    Replies: 6
    Last Post: 03-27-2006, 08:35 AM
  5. SUMPRODUCT errors dealing with dates
    By BeeFreedman in forum Excel Formulas & Functions
    Replies: 2
    Last Post: 01-21-2005, 01:08 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1