+ Reply to Thread
Results 1 to 4 of 4

Need A Fast Script To Copy 80000 Cells But My Current Script Is Too Slow

  1. #1
    Forum Contributor
    Join Date
    10-29-2019
    Location
    London, England
    MS-Off Ver
    2013
    Posts
    156

    Need A Fast Script To Copy 80000 Cells But My Current Script Is Too Slow

    Im testing excel if it can handle large datasets.

    Basically Im collecting data over 48 hours, & I calculated 800,000 cells are needed, so I'm testing out a smaller sample of around 80000 cells.

    But my current script is ridiculously slow, if I test it on 80,000 random numbered cells.

    1. So my 1st question is, can excel sort 800,000 cells, rapidly, using vba? If not I'll just head over to python.
    2. What is a reasonable number of cells, can vba sort rapidly?

    My current script basically uses for loops to copy all the rows starting from c1 to k1, & all the columns upto c12 , which are copied to sheet2, rows c6 to k6 & columns c11 to c21.

    Its ridiculously slow & pretty simple, & i'm pretty sure there are faster scripts, as my script is just not working fast enough.

    Please provide a faster script, as my current script is ridiculously slow, thanks.



    I tried uploading the excel file, with my original data set, but its over 38mb, so i added a button to populate 800,000 cells instead.

    Also, as its pretty late i'll answer tomorrow morning, hope thats ok.


    Thanks in advance, & much appreciated.

    Please Login or Register  to view this content.
    Attached Files Attached Files
    Last edited by Genus Max; 12-19-2019 at 06:38 PM.

  2. #2
    Forum Expert Roel Jongman's Avatar
    Join Date
    03-28-2015
    Location
    Netherlands
    MS-Off Ver
    Office 365
    Posts
    1,483

    Re: Need A Fast Script To Copy 80000 Cells But My Current Script Is Too Slow

    Loops are for noobs just a joke, but with some truth to it.
    loops are a nice way to learn VBA and can be usefull in many cases but not for copying large ranges. there are far better options for that.


    there is no need to copy cell by cell. if you want to copy and paste 1 area whether it is 8 cells or 8 miljon cell you can do that in one codeline without looping..
    so yes vba can handle large datasets when they are handled in the right way..

    I think this is all you need for the copy.
    Please Login or Register  to view this content.
    to explain..
    .Usedrange is the VBA version of CTRL+A or select all this is usefull when the exact data size is not known but all data is in one range and no other data
    But there are many other ways to control the size of the copy range. depending on your needs..
    sheet1. is the codename of the sheet you called with sheets("sheet1") advantage is that when you change the sheetname on the tab from sheet1 to "rawdata" the codename is still sheet1 and code will run without change when using your version you also need to change the name in the code. That is why I prefer the codename as it is more failsafe

    __
    also for your testdata set you can use a different code that is not that slow as your loop..

    Please Login or Register  to view this content.
    see attachment, generating the testdata is still a few seconds but copying is a splitsecond
    Attached Files Attached Files

  3. #3
    Forum Contributor
    Join Date
    10-29-2019
    Location
    London, England
    MS-Off Ver
    2013
    Posts
    156

    Re: Need A Fast Script To Copy 80000 Cells But My Current Script Is Too Slow

    Thanks, how do i copy that range to a variable, & then copy that variable to a new sheet. I need a loop as I'm continously reading data from the last 800 cells.

    I basically need a fast script to continously loop over the last 800 columns, & rows c17 to x17 & then copy that to sheet2 rows c21 to x21 & columns c21 to c779.


    So if 8000 cells are filled I need to subtract 800 cells, which comes to 7200, & then create a loop from columns c7200 to c8000, & rows c17 to x17

    So for example, I need to subtract the current column c6000 by 800.
    To find the current column, subtract c6000 by 800 which is c5200, which is read by vba as, read rows c17 to x17, & the 800 columns after c5200.

    So basically first find the current column, ie c6000, subtract by 800 & then create a loop to read columns c5200 to c6000 & then copy the data to a new variable, & then on worksheet 2, offset by 21 columns, ie copy the new variable to row c21 to x21 & columns c21 to c779 in sheet2.

    I've done this in my simple loop, but its obviously too slow, so I basically need a faster way of looping & continously reading the last 800 columns, in a sheet thats filled with continous data.

    It has to be a loop of some sort, as I need to adjust how many columns etc the script reads.

    Thanks for taking the time, much appreciated.
    Last edited by Genus Max; 12-20-2019 at 05:44 AM.

  4. #4
    Forum Expert Roel Jongman's Avatar
    Join Date
    03-28-2015
    Location
    Netherlands
    MS-Off Ver
    Office 365
    Posts
    1,483

    Re: Need A Fast Script To Copy 80000 Cells But My Current Script Is Too Slow

    Ok, so usedrange is not suitable for you, no problem..
    But your decription is a bit confusing at times as you seem to mixup columns and rows. So I will translate your needs as I understand them
    (for correct reference the columns have a letter and the rows have a number. So in standard excel reference terms your range is 21 columns wide and 800 rows long..)

    So based on the new description I understand that

    - the source range is always on sheet1 but the copy area changes depending on amount of available date, the copy rangealways starts in column C and then you need the last 800 rows of the available data.
    - the paste or destitnation range is always on sheet2 and in the fixed postion starting in cell C21, so for every copy / paste the old data in sheet two is overwritting / replaced by the new data
    - the ranges is 21 columns (C to X) wide and 800 rows long it has no gaps.
    because there are no gaps in the range, no looping is needed for row to row transfer you can copy 800 rows in 1 go..

    This new macro will find the last filled row of sheet1, column C (lastRow variable)
    then it calculate the start row of based of that value (startRow = lastRow - 800)
    the range-object, helps me to point to topleft cell and bottom right cell to get the total range to copy, all cells between C&startrow and X&lastrows are automaticly included in the copy range. (total of 800 rows with 21 columns each = 16800 cells)
    All those cells will be copied to sheet2 with in topleft cell C21
    Please Login or Register  to view this content.
    let me know if this is closer or hopefully exactly what you need..

    now this should work as a basic setup

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Need to tweak a current script to copy conditional formatting too.
    By peejaygee in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 05-22-2017, 12:06 PM
  2. Need help modifing current script to look for blank cells
    By polska21800 in forum Excel General
    Replies: 1
    Last Post: 07-25-2013, 05:43 PM
  3. Compare and copy cells script
    By fuchetj in forum Excel Programming / VBA / Macros
    Replies: 4
    Last Post: 01-13-2011, 08:52 AM
  4. Very Slow Script... Speed Up Advice?
    By realniceguy5000 in forum Excel Programming / VBA / Macros
    Replies: 7
    Last Post: 04-15-2010, 01:18 PM
  5. Slow Down A Script
    By realniceguy5000 in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 04-28-2008, 03:17 PM
  6. Insert IF-statements by Script, very slow.
    By Parity in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 10-24-2005, 08:05 AM
  7. Excel vba script extremely slow
    By persenena in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 07-05-2005, 08:05 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1