+ Reply to Thread
Results 1 to 3 of 3

Subtraction of 2 decimal place numbers giving 12 decimal place answer

  1. 01-22-2018, 09:11 PM #1
    njb212
    njb212 is offline
    Registered User
    Join Date
    02-12-2013
    Location
    Perth
    MS-Off Ver
    Excel 2003
    Posts
    3

    Subtraction of 2 decimal place numbers giving 12 decimal place answer

    I was using conditional formatting to highlight any cells that didn't match, and I came across a handful where while looking identical the formula was saying they weren't. One columns was a static list, and the other a simple subtraction formal, and I found the issue was somehow the formula to subtract 2 numbers which only go to 2 decimal places was somehow getting an answer which had a value in the 12th decimal place

    Below is an example of what I'm seeing, any ideas how I'm getting a 1 at 12 decimal places when the numbers used for the calculation only go to 2 decimal places?

    Thanks


    E F static =sum(E1-F1)
    12044.950000000000000000000000000000 11931.650000000000000000000000000000 113.30 113.30000000000100000000
    Last edited by njb212; 01-22-2018 at 09:14 PM.
    Reply With QuoteRegister To Reply
  2. 01-22-2018, 09:16 PM #2
    ben_hensel
    ben_hensel is offline
    Forum Expert ben_hensel's Avatar
    Join Date
    01-26-2012
    Location
    Northeast USA
    MS-Off Ver
    Office 365
    Posts
    2,043

    Re: Subtraction of 2 decimal place numbers giving 12 decimal place answer

    Looks like a floating point arithmetic error.
    Click the [* Add Reputation] Button to thank people for help!
    Post Attachment: Click [Go Advanced] under the reply box; then click [Manage Attachments] to launch the attachments dialogue.
    Mark Thread Solved: Go to the top of the first post; Select [Thread Tools] dropdown; Pick the [Mark thread as Solved] option.
    Reply With QuoteRegister To Reply
  3. 01-22-2018, 09:35 PM #3
    njb212
    njb212 is offline
    Registered User
    Join Date
    02-12-2013
    Location
    Perth
    MS-Off Ver
    Excel 2003
    Posts
    3

    Re: Subtraction of 2 decimal place numbers giving 12 decimal place answer

    Thanks Ben, appreciate the quick reply. I'll just use the round function as it suggests to get me miss-match formatting to work .

    Thanks again
    Reply With QuoteRegister To Reply
+ Reply to Thread
« Previous Thread | Next Thread »

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Extract numbers within text and put decimal place
    By YasserKhalil in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 10-28-2015, 02:13 PM
  2. [SOLVED] Sum of range picking up ^-10 decimal place. Is there a way to sum 2 decimal place only?
    By graym463 in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 08-22-2015, 08:48 AM
  3. Cannot copy numbers with decimal place
    By alan_stephen75@ in forum Excel - New Users/Basics
    Replies: 4
    Last Post: 10-12-2009, 04:32 PM
  4. [SOLVED] Converting 2-place decimal value to floating point decimal number with leading zero
    By Kermit Piper in forum Excel General
    Replies: 3
    Last Post: 03-18-2006, 02:25 PM
  5. How do I get rid of infinite numbers past the decimal place?
    By hd in forum Excel Formulas & Functions
    Replies: 1
    Last Post: 09-06-2005, 03:05 AM
  6. [SOLVED] How do I get rid of infinite numbers past the decimal place?
    By hd in forum Excel Formulas & Functions
    Replies: 1
    Last Post: 09-06-2005, 12:05 AM
  7. How do I get rid of infinite numbers past the decimal place?
    By hd in forum Excel Formulas & Functions
    Replies: 1
    Last Post: 09-05-2005, 10:05 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Forum Rules


Search Engine Friendly URLs by vBSEO 3.6.0 RC 1