+ Reply to Thread
Results 1 to 6 of 6

Array Output to Range Losing Precision Unexpectedly

Hybrid View

  1. #1
    Registered User
    Join Date
    02-24-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    32

    Array Output to Range Losing Precision Unexpectedly

    URL to same post in other forum http://www.ozgrid.com/forum/showthre...d=1#post562139

    The attached macro formats input to output. The final steps shown below transfer the contents of the array 'y' to the sheet 'output'.

    Next: With Sheets("output").[a2].Resize(j, 11): .Value = y: .Offset(, 6).Resize(,5).NumberFormat = "0.00": End With

    A simple message box tells me that array element y(1,10) is 10.4513
    MsgBox y(1,10)

    Excellent! Just what I wanted.

    However, when I look on the output tab in cell J2, I am disappointed to see that it produced a value of 10.45, dropping off two decimal places.I have worked with this for quite some time, but have not found a way to get it to show the correct values, 10.4513 in J2 for example. (impacts entire column j and k)

    What would you guys change in the code above to get it to work?

    Thanks in advance,

    -Michael
    Attached Files Attached Files
    Last edited by msmithdynamicsgp; 06-28-2011 at 08:51 AM.

  2. #2
    Registered User
    Join Date
    10-23-2009
    Location
    Alberta
    MS-Off Ver
    Excel 2007
    Posts
    4

    Re: Array Output to Range Losing Precision Unexpectedly

    In The attached macro which formats input to output, change from .NumberFormat: ="0.00" to .NumberFormat = "0.0000" and make sure your output worksheet column is formated for 4 decimials (cell format)

  3. #3
    Registered User
    Join Date
    02-24-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    32

    Re: Array Output to Range Losing Precision Unexpectedly

    In The attached macro which formats input to output, change from .NumberFormat: ="0.00" to .NumberFormat = "0.0000" and make sure your output worksheet column is formated for 4 decimials (cell format)
    Thank you for the suggested solution.I changed the formatting as indicated above and reran the macro.

    Unfortunately, I tried this before and it did not do anything to resolve the issue.

    It is not a formatting problem, but rather the actual numbers themselves are truncated improperly.

    Any other suggestions?
    Attached Files Attached Files

  4. #4
    Forum Expert romperstomper's Avatar
    Join Date
    08-13-2008
    Location
    England
    MS-Off Ver
    365, varying versions/builds
    Posts
    22,064

    Re: Array Output to Range Losing Precision Unexpectedly

    Your post does not comply with Rule 8 of our Forum RULES. Cross-posting is when you post the same question in other forums on the web. You'll find people are disinclined to respond to cross-posts because they may be wasting their time solving a problem that has been solved elsewhere. We prefer that you not cross-post at all, but if you do (and it's unlikely to go unnoticed), you MUST provide a link (copy the url from the address bar in your browser)to the cross-post. Expect cross-posts without a link to be closed a message will be posted by the moderator explaining why. We are here to help so help us help you!

    Read this to understand why we ask you to do this.

    Please add links in your initial post to any and all crossposts in other forums.
    Everyone who confuses correlation and causation ends up dead.

  5. #5
    Registered User
    Join Date
    02-24-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    32

    Re: Array Output to Range Losing Precision Unexpectedly

    I added the link to my original post. My apologies for cross posting without reference. It is nice to now understand why links are needed.

    After more than a decade of posting on many online forums, this is the first time I thought about it this way (also makes sense since excel solutions take longer to create sometimes).

  6. #6
    Forum Expert romperstomper's Avatar
    Join Date
    08-13-2008
    Location
    England
    MS-Off Ver
    365, varying versions/builds
    Posts
    22,064

    Re: Array Output to Range Losing Precision Unexpectedly

    Thank you.

    And I can see that you have an answer now...

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1