Attached is a spreadsheet that shows how Excel is not converting military time (hh:mm) to decimal hours correctly. Any clues as to why? One of the conversions is less than exact and it's throwing off my calculations.
Attached is a spreadsheet that shows how Excel is not converting military time (hh:mm) to decimal hours correctly. Any clues as to why? One of the conversions is less than exact and it's throwing off my calculations.
I got the calculations to work by changing the format of the two cells.
See if that works for you . . . .
What you are seeing, I believe, is the result of a machine that calculates in binary, trying to perform calculations on base 10 values. Sometimes it is not accurate out to the 16th decimal. You simply need to round your results using the ROUND function.
ChemistB
The Chemist is spot on.
In general, binary floating point numbers can't store decimals precisely. If you remember from high school, rational numbers (a/b) are always repeating fractions; for some, the repeating digit is 0, like 1/2 = 0.500000....
For others, it's not zero: 1/9 = 0.1111111 ...
For others, the repeating sequence is more than one digit: 3/7 = 0.428571 428571 428571 ...
In binary, 0.1 (and lots of other decimal) have non-zero repeating binary decimals, and they are stored within the precision that the 64-bit IEEE-754 format allows.
It's not an Excel thing.
That's interesting and complex. At least it's not Excel's fault. Thanks.
There are currently 1 users browsing this thread. (0 members and 1 guests)
Bookmarks