+ Reply to Thread
Results 1 to 7 of 7

variable type changing number of decimal places

  1. #1
    Registered User
    Join Date
    01-06-2014
    Location
    Columbus, IN
    MS-Off Ver
    2013/O365
    Posts
    41

    variable type changing number of decimal places

    Here's what I'm doing: I'm using a macro to assign a cell value to a variable then set another cell value to the variable instead of copy/paste (because even pasting values only was affecting other formulas in the file for some reason)

    The problem: in using the macro, the number being 'copied' is acquiring a few extra decimal places IE 38334.61 is the original number and 38334.609375 is what I end up with. The numbers come from a CSV with only 2 decimal places and I checked by adding decimal places in the format so it's not a formatting/visible digits issue. Since the values are hour meter readings, the extra decimal places end up with very small values outside 0-24 hrs which messes with sorting and usage %. The values are so small all of them together in a month add up to a fraction of a cent but it's one of those things that bugs my OCD by not being right.

    My VBA book explains the min/max capabilities, content type, memory bits but not fiddly details like this.

    So my question is this: right now I'm declaring the clipboard-substitue variable as an single, is there a different one that would work better without adding anything? Criteria are: numeric, 6 digits before the decimal, 2 after, all positive values. (Ie 123456.12)

    EDIT: fixed my senior moment.
    Last edited by russc2541; 05-15-2014 at 07:49 AM.
    -Russell

  2. #2
    Forum Guru romperstomper's Avatar
    Join Date
    11-04-2008
    Location
    A1
    MS-Off Ver
    Most
    Posts
    12,302

    Re: variable type changing number of decimal places

    If your variable is an integer, it won't have any decimal places so I'm a little confused. Could you post your code?
    Remember what the dormouse said
    Feed your head

  3. #3
    Registered User
    Join Date
    01-06-2014
    Location
    Columbus, IN
    MS-Off Ver
    2013/O365
    Posts
    41

    Re: variable type changing number of decimal places

    Quote Originally Posted by romperstomper View Post
    If your variable is an integer, it won't have any decimal places so I'm a little confused. Could you post your code?
    Senior moment. It's a "single" not integer (and no I didn't change it).

    Basically it does two things: increments columns if I get behind (the loop and db variable) and it moves hour meter readings from the top block of data to the lower block. the difference is that if the contents of a room change, a new group gets billed for their time in there thus a new line with their billing info (hence the empty blocks of data to the left, above, below in the example sheet)

    BTW I didn't write this, I just changed it to use the macro variable as a clipboard instead of copy/paste/clearing cells then having to repair the formulas elsewhere. I also removed more commented out obsolete code used for development. Then again, I don't know a better way either.

    Please Login or Register  to view this content.
    hm decimal places.xlsx (problem items highlighted for expediency)
    Last edited by russc2541; 05-15-2014 at 07:47 AM.

  4. #4
    Forum Expert
    Join Date
    05-01-2014
    Location
    California, US
    MS-Off Ver
    Excel 2010
    Posts
    1,795

    Re: variable type changing number of decimal places

    Quote Originally Posted by russc2541 View Post
    The problem: in using the macro, the number being 'copied' is acquiring a few extra decimal places IE 38334.61 is the original number and 38334.609375 is what I end up with. The numbers come from a CSV with only 2 decimal places
    Quote Originally Posted by russc2541 View Post
    Senior moment. It's a "single"
    That's your problem. Excel numbers are stored effectively using type Double. So the variable "hm" should be type Double as well.

    Type Single conforms to the 32-bit binary floating-point standard. Type Double conforms to the 64-bit binary floating-point standard.

    In both cases, most non-integers cannot be represented exactly. Type Single represents numbers as the sum of 24 consecutive powers of 2 ("bits") times an exponential factor. Type Double represents numbers as the sum of 53 consecutive powers of 2 times an exponential factor.

    So when you store a type Double number into a type Single variable, you truncate the sum of powers of 2 by 29 terms. When you store the type Single number back into a type Double variable, there is no way to recover the truncated bits. So the number continues to be accurate to only 24 bits, even though it is represented with 53 bits.

    For example, suppose we estimate 0.1 by 6 consecutive powers of 2 (6 bits):
    1/16 + 1/32 + 0/64 + 0/128 + 1/256 + 1/512 = 0.099609375

    If we truncate that to 3 consecutive powers of 2 (3 bits), the estimate becomes:
    1/16 + 1/32 + 0/64 = 0.09375

    If we store the 3-bit estimate back into a 6-bit variable, the estimate becomes:
    1/16 + 1/32 + 0/64 + 0/128 + 0/256 + 0/512 = 0.09375

    The 64-bit representation of 38334.61 is exactly 38334.6100000000,00582076609134674072265625.

    The 32-bit representation of 38334.61 is exactly 38334.609375.

    (Comma demarcates 15 significant digits, the arbitrary max number of digits that Excel formats.)

  5. #5
    Registered User
    Join Date
    01-06-2014
    Location
    Columbus, IN
    MS-Off Ver
    2013/O365
    Posts
    41

    Re: variable type changing number of decimal places

    ^That is why I asked here I figured it was something along those lines. Thanks!

    Now let me get some water for my brain. It makes sense but jeez, it's not even 9am...

  6. #6
    Forum Contributor
    Join Date
    01-25-2011
    Location
    Virginia
    MS-Off Ver
    Excel 2016
    Posts
    166

    Re: variable type changing number of decimal places

    The results don't make since to me (nothing in the code is changing the number of decimals), but I would try the double data type for hm (instead of single) as that is what excel uses for worksheet calculations, this will fix your issue.

    Edit: Was posting while joeu2004 gave a great answer - note: I was right too but didn't know why!

    Regards,
    Last edited by tom.hogan; 05-15-2014 at 09:19 AM.
    Tom

    If my answer resolves your issue, please mark your Thread as SOLVED.

    If you're satisfied by any members response to your issue please use the star icon at the lower left of their post
    .

  7. #7
    Registered User
    Join Date
    01-06-2014
    Location
    Columbus, IN
    MS-Off Ver
    2013/O365
    Posts
    41

    Re: variable type changing number of decimal places

    Well, you can have a star too Tom, since you still knew the answer even if you couldn't show your work lol

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Adusting decimal places based on greatest number of places in a series
    By anelson87 in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 05-11-2022, 01:05 PM
  2. [SOLVED] copy number with 3 or more decimal places and paste the actual value as 2 decimal number
    By k1dr0ck in forum Excel Programming / VBA / Macros
    Replies: 6
    Last Post: 05-29-2013, 12:57 AM
  3. How to change the number of decimal places in a variable cell reference...
    By kenab1 in forum Excel Programming / VBA / Macros
    Replies: 4
    Last Post: 03-08-2010, 01:32 PM
  4. Paste two decimal number in excel without extra decimal places appearing
    By jeffery_frick in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 08-13-2009, 07:49 PM
  5. How to type in 2 decimal places in 1/4 hour display
    By nic.wood in forum Excel General
    Replies: 9
    Last Post: 03-22-2008, 06:58 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1