+ Reply to Thread
Results 1 to 13 of 13

Array speed is slow

  1. #1
    Registered User
    Join Date
    01-22-2018
    Location
    England
    MS-Off Ver
    2016
    Posts
    7

    Array speed is slow

    I have 3 tabs in excel
    Tab A - "Pre"
    Tab B - "Post"
    Tab C - "Rec"
    In Tab A & B I have the same column headers (15 dimensions and a value)
    I create a unique lookup id (UID) based on a subset of the columns (and this can change run to run)
    e.g. A|B|D|E|K

    I move the UIDs from Tab A and Tab B to Tab C and filter this combined set of UIDs (and then sort) to get the unique UIDs from both tabs in Tab C.

    I am then doing a sum if on both Tab A and Tab B and put the results in to Tab C in columns B & C (column B would be the Pre Sum and column C would be the Post Sum).

    This gives the sums of the UIDs to allow me to check/compare the 'Pre' and 'Post' values to show the variances.

    I have approx 56,000 lines in Pre and 85,000 lines in Post and 45,000 lines in UID tabs

    Using Sumif, the runtime is about 7 minutes to do all these calculations.

    It has been suggested I switch to using Arrays instead as it should be faster (thanks to the guys on Reddit r/VBA).

    I have tried this (with my limited Array skills) and the runtime is still slower than when doing simple sumif's.

    Can someone please advise what I am doing wrong with the array VBA code (shown below).
    Thanks
    Jon


    Code for the Pre value sum is below.
    Please Login or Register  to view this content.
    Excel attached
    It has the 3 tabs, with limited data in it
    Tab Rec has examples of manual excel formula, VBA with SUMIF and VBA with Array,
    When running across 50000 lines and 30000 UIDs it is slow in all 3 cases
    Attached Files Attached Files
    Last edited by Jonsmile; 01-20-2021 at 12:16 PM.

  2. #2
    Forum Moderator - RIP Richard Buttrey's Avatar
    Join Date
    01-14-2008
    Location
    Stockton Heath, Cheshire, UK
    MS-Off Ver
    Office 365, Excel for Windows 2010 & Excel for Mac
    Posts
    29,464

    Re: Array speed is slow

    Please upload a workbook or a representative cut down copy, anonymised if necessary. It is always easier to advise if we can see your request in its context. We don't need zillions of rows, a representative sample will do.

    Show a before and after situation with manually calculated results, explaining which information is data and which is results, and if it's not blindingly obvious how you have arrived at your results some explanatory notes as well.

    To upload a file click the Go Advanced button at the foot of your post, look underneath the post area for the Manage Attachments section and take it from there.
    Richard Buttrey

    RIP - d. 06/10/2022

    If any of the responses have helped then please consider rating them by clicking the small star icon below the post.

  3. #3
    Forum Guru TMS's Avatar
    Join Date
    07-15-2010
    Location
    The Great City of Manchester, NW England ;-)
    MS-Off Ver
    MSO 2007,2010,365
    Posts
    44,361

    Re: Array speed is slow

    Welcome to the forum

    Please attach a sample workbook (not a picture or pasted copy). Make sure there is just enough data to demonstrate your need. Include a BEFORE sheet and an AFTER sheet in the workbook if needed to show the process you're trying to complete or automate. Make sure your desired results are shown, mock them up manually if necessary.

    Remember to desensitize the data.

    Click on GO ADVANCED and then scroll down to Manage Attachments to open the upload window.
    Trevor Shuttleworth - Retired Excel/VBA Consultant

    I dream of a better world where chickens can cross the road without having their motives questioned

    'Being unapologetic means never having to say you're sorry' John Cooper Clarke


  4. #4
    Registered User
    Join Date
    01-22-2018
    Location
    England
    MS-Off Ver
    2016
    Posts
    7

    Re: Array speed is slow

    Hi

    I have knocked up a small excel with examples of manual, VBA SUMIF and VBA ARRAY
    There is no time difference with 150 rows, but when it gets to 50000 rows and 30000 UID to sum against, both VBA options run for 5+ minutes.

  5. #5
    Forum Moderator - RIP Richard Buttrey's Avatar
    Join Date
    01-14-2008
    Location
    Stockton Heath, Cheshire, UK
    MS-Off Ver
    Office 365, Excel for Windows 2010 & Excel for Mac
    Posts
    29,464

    Re: Array speed is slow

    Before getting too deep unto this slowness issue are you saying this happens
    AFTER you have created the long list of UIDs in column A of the Rec tab
    AND then copy the Formula in C2 and D2 down as far as necessary?

    I note that column R contains a formula to create the text string.
    Have you tried first converting these to hard values before copying them to Rec?

  6. #6
    Registered User
    Join Date
    01-22-2018
    Location
    England
    MS-Off Ver
    2016
    Posts
    7

    Re: Array speed is slow

    Hi
    Sorry. That we my quick and dirty create an excel.

    In real life the uid’s are all hard coded in all tabs (another vba creates these in real life).

    As to when the issue is. It is when the sumif vba runs (or newer array vba).
    I dumped the use of formula directly in excel as it was running like a dog.
    Only added it to show I had constant moved on from this option.

  7. #7
    Forum Moderator - RIP Richard Buttrey's Avatar
    Join Date
    01-14-2008
    Location
    Stockton Heath, Cheshire, UK
    MS-Off Ver
    Office 365, Excel for Windows 2010 & Excel for Mac
    Posts
    29,464

    Re: Array speed is slow

    Are you suggesting that if there were 50000 UIDs on the Rec tab, manually copying the SUMIF formula down all those rows is slow?

    I've just created 100,000 UIDs and manually copied the SUMIF formula down, which then takes about 10 seconds. I wouldn't expect a macro doing the same job would be any different.

  8. #8
    Forum Expert
    Join Date
    07-06-2004
    Location
    Northern California
    MS-Off Ver
    2K, 2003, 2010, O365
    Posts
    1,490

    Re: Array speed is slow

    Since your comments make it appear both Pre_ALL and Rec_range would have their UID columns sorted in ascending order, I'd use

    Please Login or Register  to view this content.

  9. #9
    Forum Guru TMS's Avatar
    Join Date
    07-15-2010
    Location
    The Great City of Manchester, NW England ;-)
    MS-Off Ver
    MSO 2007,2010,365
    Posts
    44,361

    Re: Array speed is slow

    I'd also be interested to see how this performs.

    Please Login or Register  to view this content.

  10. #10
    Registered User
    Join Date
    01-22-2018
    Location
    England
    MS-Off Ver
    2016
    Posts
    7

    Re: Array speed is slow

    All

    Thanks for the replies.

    Once I am back in front of a pc I will try all options against my massive data sets and report back. Hopefully with some good news.

    Jon

  11. #11
    Forum Guru
    Join Date
    08-15-2004
    Location
    Tokyo, Japan
    MS-Off Ver
    2013 O.365
    Posts
    22,516

    Re: Array speed is slow

    Quote Originally Posted by Jonsmile View Post
    I have approx 56,000 lines in Pre and 85,000 lines in Post and 45,000 lines in UID tabs
    See if this runs faster.
    Please Login or Register  to view this content.

  12. #12
    Forum Guru Bo_Ry's Avatar
    Join Date
    09-10-2018
    Location
    Thailand
    MS-Off Ver
    MS 365
    Posts
    7,211

    Re: Array speed is slow

    Maybe try Power Query

    Turn Pre and Post to Table tPre and tPost

    Then at Ribbon data > Get data from Other sources > Blank Query
    Query Ribbon home> advanced editor > paste code > close and load to table

    Please Login or Register  to view this content.
    Attached Files Attached Files
    Last edited by Bo_Ry; 01-21-2021 at 02:52 AM.

  13. #13
    Registered User
    Join Date
    01-22-2018
    Location
    England
    MS-Off Ver
    2016
    Posts
    7

    Re: Array speed is slow

    Thanks all for the help and suggestions

    After much testing and playing . tweaking I am going to move forward based on the code jindon provided.

    It appeared to work the fastest when calculating 100k sums.

    FYI I had to tweak the code because of how it was using the pre and Post UIDs to do the sums but was putting the values in the wrong array field.
    e.g.
    If PRE had
    A 1
    B 2
    D 4
    E 5
    and POST had
    A 1
    B 2
    C 3
    D 4
    E 5
    the code will scan PRE and place
    1
    2
    4
    5
    in the array
    Then in the Post scan it will notice C is new and add the txt as a new txt value
    but will add the 3 to the 3rd line of the array
    so you get
    1 1
    2 2
    4 3
    5 4
    5

    I recoded it to add the entire Rec UID to the txt key first and then looped through the PRE and POST to correctly assign the values to the correct line

    Again thanks for the hints. I expect that there is a better way to get around the above but I have gone from 5-10 mins calc time down to 15 seconds to do 100k sums
    I will keep playing and expanding my array skills as they are both rusty and lacking

    Jon

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] how to speed up my macro, going very slow, row by row
    By dmcgov in forum Excel Programming / VBA / Macros
    Replies: 21
    Last Post: 08-05-2019, 08:56 AM
  2. Need for speed , slow vba code
    By antonio32 in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 08-27-2017, 10:54 AM
  3. Slow VBA Code - Help Speed Up
    By CC268 in forum Excel Programming / VBA / Macros
    Replies: 4
    Last Post: 01-26-2017, 10:45 AM
  4. [SOLVED] How to slow down the animation speed
    By morasrikanth in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 05-28-2015, 10:25 AM
  5. Spreadsheet Really Slow - Can anyone help speed it up?
    By AlexChatz in forum Excel Programming / VBA / Macros
    Replies: 13
    Last Post: 06-18-2014, 01:12 AM
  6. How can I speed up this slow macro?
    By rs2k in forum Excel Programming / VBA / Macros
    Replies: 6
    Last Post: 07-18-2008, 08:34 PM
  7. [SOLVED] slow Macro speed
    By Lam Chop in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 06-04-2006, 11:10 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1