+ Reply to Thread
Results 1 to 8 of 8

Performance of iteration through Words collection degrades as collection gets bigger

  1. #1
    Administrator 6StringJazzer's Avatar
    Join Date
    01-27-2010
    Location
    Tysons Corner, VA, USA
    MS-Off Ver
    MS365 Family 64-bit
    Posts
    24,721

    Performance of iteration through Words collection degrades as collection gets bigger

    I wrote a macro to analyze a document for acronyms, and found that for larger documents the performance seemed to be exponentially worse. I created a simple performance test where I looped through each Word in Words and assigned the text of the word to a variable. Just one line of code inside the loop. I captured the start and end times.

    When I tested this on a document with Words.Count = 200, it took under 1 second to run, so better than 200 words per second.

    When I tested this on a document with Words.Count = 36,121, it took 1:27:26 to run. That averages about 7 words per second.

    Why does the performance degrade so much for large documents? Is there any approach I can take to speed things up?
    Jeff
    | | |會 |會 |會 |會 | |:| | |會 |會
    Read the rules
    Use code tags to [code]enclose your code![/code]

  2. #2
    Forum Guru Izandol's Avatar
    Join Date
    03-29-2012
    Location
    *
    MS-Off Ver
    Excel 20(03|10|13)
    Posts
    2,581

    Re: Performance of iteration through Words collection degrades as collection gets bigger

    What is your code for iteration?
    • Please remember to mark threads Solved with Thread Tools link at top of page.
    • Please use code tags when posting code: [code]Place your code here[/code]
    • Please read Forum Rules

  3. #3
    Administrator 6StringJazzer's Avatar
    Join Date
    01-27-2010
    Location
    Tysons Corner, VA, USA
    MS-Off Ver
    MS365 Family 64-bit
    Posts
    24,721

    Re: Performance of iteration through Words collection degrades as collection gets bigger

    I was using a Do While loop using an index into Words. However, I have since tried other methods and found that a For Each to iterate the collection works incredibly much faster. I am still trying to determine if I can do what I want to do that way.

  4. #4
    Administrator 6StringJazzer's Avatar
    Join Date
    01-27-2010
    Location
    Tysons Corner, VA, USA
    MS-Off Ver
    MS365 Family 64-bit
    Posts
    24,721

    Re: Performance of iteration through Words collection degrades as collection gets bigger

    In the course of the processing I am replacing a single word with a phrase. In this case, using For Each, the cursor continues with the first word I used to replace, rather than continuing with the next word after the one I replaced. I am not sure how to mitigate that. For example:

    Scanning this sentence:

    Here is a BUA. We do not know what it means.

    It scans BUA, then replace it so the line now reads

    Here is a Big Undefined Acronym (BUA). We do not know what it means.

    In this case, I want the next word scanned to be "We". However, it is "Big".

    Here is a simplified version of the code:

    Please Login or Register  to view this content.

  5. #5
    Administrator 6StringJazzer's Avatar
    Join Date
    01-27-2010
    Location
    Tysons Corner, VA, USA
    MS-Off Ver
    MS365 Family 64-bit
    Posts
    24,721

    Re: Performance of iteration through Words collection degrades as collection gets bigger

    The For Each works as a workaround to the performance issue. To deal with the cursor location I added a JumpOver variable to count how many words I inserted so the next JumpOver passes through the loop it skips processing.

    I'm still wondering why performance of For Each is so superior to using an index, and also why the performance using an index gets worse and worse as the document size increases.

    Thanks for you interest!

  6. #6
    Forum Guru Izandol's Avatar
    Join Date
    03-29-2012
    Location
    *
    MS-Off Ver
    Excel 20(03|10|13)
    Posts
    2,581

    Re: Performance of iteration through Words collection degrades as collection gets bigger

    For Each is always the fastest way to iterate a collection but I was very surprised by the difference with speed compared to a For i = 1 to Words.Count loop.

    Which Do loop were you using? Did you use the Range.Next/.Previous methods?

  7. #7
    Administrator 6StringJazzer's Avatar
    Join Date
    01-27-2010
    Location
    Tysons Corner, VA, USA
    MS-Off Ver
    MS365 Family 64-bit
    Posts
    24,721

    Re: Performance of iteration through Words collection degrades as collection gets bigger

    My original loop was something like this

    Please Login or Register  to view this content.
    I started out with this because I occasionally needed to look back or look ahead. I was not aware of the Next/Previous methods. Once I restructured with For Each and started using Next/Previous the thing runs like lightning.

    Any idea why For Each is the fastest way to iterate a collection? I have a computer science/software development background but have no idea how that structure is implemented and why that would be faster.

  8. #8
    Forum Guru Izandol's Avatar
    Join Date
    03-29-2012
    Location
    *
    MS-Off Ver
    Excel 20(03|10|13)
    Posts
    2,581

    Re: Performance of iteration through Words collection degrades as collection gets bigger

    I am not certain but I imagine it is a linked list structure or something very similar - so it is like the Next/Previous methods internally - and therefore there is no looping required to return a specific item.

    Updated: seemingly so: http://stackoverflow.com/questions/2...nted-as-in-vb6
    Last edited by Izandol; 01-14-2014 at 09:59 AM.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 1
    Last Post: 06-21-2012, 03:49 AM
  2. Replies: 2
    Last Post: 12-23-2011, 11:04 AM
  3. Problems storing a Collection within a Collection. Is this possible?
    By Tarball in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 05-17-2007, 10:47 PM
  4. Collection
    By MattShoreson in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 02-16-2007, 10:53 AM
  5. Collection Key
    By gabch in forum Excel Programming / VBA / Macros
    Replies: 4
    Last Post: 03-20-2006, 12:45 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1