+ Reply to Thread
Results 1 to 15 of 15

HELP! My caculation is running SLOW!

  1. #1
    Registered User
    Join Date
    09-06-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    7

    Exclamation HELP! My caculation is running SLOW!

    I hope there is a simple solution to fix my LONG running calculation. It works, but when running against 125k+ rows it takes forever. What I'm basically trying to do is compare the CURRENT row

    =IF(COUNTIFS($A:$A,[FIN], $P:$P, ">"&[Base_Time],$P:$P,"<="&[Target_Time1])>=1,"Excellent",IF(COUNTIFS($A:$A,[FIN], $P:$P, ">"&[Base_Time],$P:$P,">"&[Target_Time1],$P:$P,"<="&[Target_Time2])>=1,"Good",IF(COUNTIFS($A:$A,[FIN], $P:$P, ">"&[Base_Time],$P:$P,">"&[Target_Time2])>=1,"Fair","Poor")))

    What I'm basically trying to do is compare the CURRENT row with other matching [FIN] rows in the table and based on the time range return to the column either "Excellent, Good, Fair, Poor" . Each row is a patient assessment which requires a follow up. If the follow up was within an hour of the current row then return "Excellent"... within 2 hours then "Good"....greater than 2 hours "Fair... no other row older than the current row then "Poor"

    Would it speed things if I just calculated the target times in the formula instead of looking at the column value? (Target_Time1=[Base_Time]+1/24)


    FIN - is the visit ID (Column A)
    Base Time - is the event date/time (Column P)
    Target_Time1 - is the upper time limit for the excellent category (within 1 hour) (Column R)
    Target_Time2 - is the upper time limit for the Good category (within 2 hours) (Column S)
    Target_Time3 - is the upper time limit for the Fair category (Column T)
    POOR is everything else


    I appreciate any help/advice you can provide! and if there is a better way to structure the data, then I'm open to that advice as well.

    Thanks you in advance for your assistance
    Attached Files Attached Files
    Last edited by jrjeffcoat64; 05-29-2019 at 11:23 AM.

  2. #2
    Forum Expert Pepe Le Mokko's Avatar
    Join Date
    05-14-2009
    Location
    Belgium
    MS-Off Ver
    O365 v 2402
    Posts
    13,448

    Re: HELP! My caculation is running SLOW!

    Attach a sample workbook (not a picture or pasted copy). Make sure there is just enough data to demonstrate your need. Include a BEFORE sheet and an AFTER sheet in the workbook if needed to show the process you're trying to complete or automate. Make sure your desired results are shown, mock them up manually if necessary.

    Remember to desensitize the data.

    Click on GO ADVANCED and then scroll down to Manage Attachments to open the upload window.

  3. #3
    Registered User
    Join Date
    09-06-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    7

    Re: HELP! My caculation is running SLOW!

    Thank you! I didn't scroll down far enough to see the attachment feature.

    I have placed a sample file in my message.

    Jim

  4. #4
    Forum Expert Sam Capricci's Avatar
    Join Date
    06-14-2012
    Location
    Palm Harbor, Florida
    MS-Off Ver
    16.84 for Mac MS 365
    Posts
    8,499

    Re: HELP! My caculation is running SLOW!

    Have you considered limiting your ranges from $A:$A and $P:$P etc. to like $A$2:$A$125000 (or something just below the last row)?
    I changed ranges from whole columns to limited ranges on a workbook and the size dropped and processing time improved. Just a thought.
    Make contributors happy, click on the "* Add Reputation" as a way to say thank you.
    Sam Capricci

  5. #5
    Forum Expert ben_hensel's Avatar
    Join Date
    01-26-2012
    Location
    Northeast USA
    MS-Off Ver
    Office 365
    Posts
    2,043

    Re: HELP! My caculation is running SLOW!

    Not sure I fully understand the request.

    [FIN] is basically a key, right? So I think what you're asking is, each row is an event, and what you want to do is check the next row that the key shows up in. And compare the Base_Time of the "current" row to the Base_Time of the "next" row, and deliver a grade based on how many hours apart they were. Or maybe go the other way, and check how long it's been between the "current" row and the "last" row with that key? Not sure which way to go.

    Also what about the fenceposts, how should it assess the "first" or "last" rows that don't have a time to compare against?

    I think this will be a pretty simple CHOOSE, once I understand how the formula should query the other row compared to the current one.
    Click the [* Add Reputation] Button to thank people for help!
    Post Attachment: Click [Go Advanced] under the reply box; then click [Manage Attachments] to launch the attachments dialogue.
    Mark Thread Solved: Go to the top of the first post; Select [Thread Tools] dropdown; Pick the [Mark thread as Solved] option.

  6. #6
    Registered User
    Join Date
    09-06-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    7

    Re: HELP! My caculation is running SLOW!

    Thanks for the response,

    The [FIN] is the unique key for the "customer" visit, there could be multiple rows (events) for each visit. So...

    For the current row go look for all the records that have the same [FIN] AND the [Base_Time] is LATER than the current row's [Base_Time]

    Now count the number of records that have a [Base_Time] LESS THAN EQUAL TO the current row's [Target_Time1] if 1 or more, then "Excellent"

    .... then it goes on to check if there are records in the next time range.....etc... if it does find a "Next EVENT/RECORD" then return "Poor"


    Basically,

    If I talked to a customer... then I need to go back and check on the customer within 60 minutes to be marked as Excellent"..... within 120 minutes for "Good"... greater than 120 minutes "Fair"... if I don't check on them at all then "Poor"

    Does that make sense? My logic may be WAY OFF....lol

    Thanks

    Jim

  7. #7
    Forum Contributor
    Join Date
    03-28-2018
    Location
    Houston, TX
    MS-Off Ver
    Microsoft Office Professional Plus 2016
    Posts
    127

    Re: HELP! My caculation is running SLOW!

    A quick answer to a simple question will give me what I need to help you make this formula more efficient. In your data, does each record have a valid "Base Time"?

    Answering this question will let me know how I should direct you. It appears that a lot of unnecessary calculations are happening in the middle of this formula.

    There are also some efficiencies that could be applied to the "Required" field formula if you're interested.

  8. #8
    Registered User
    Join Date
    09-06-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    7

    Re: HELP! My caculation is running SLOW!

    Thanks for the response,

    The [FIN] is the unique key for the "customer" visit, there could be multiple rows (events) for each visit. So...

    For the current row go look for all the records that have the same [FIN] AND the [Base_Time] is LATER than the current row's [Base_Time]

    Now count the number of records that have a [Base_Time] LESS THAN EQUAL TO the current row's [Target_Time1] if 1 or more, then "Excellent"

    .... then it goes on to check if there are records in the next time range.....etc... if it does find a "Next EVENT/RECORD" then return "Poor"


    Basically,

    If I talked to a customer... then I need to go back and check on the customer within 60 minutes to be marked as Excellent"..... within 120 minutes for "Good"... greater than 120 minutes "Fair"... if I don't check on them at all then "Poor"

    Does that make sense? My logic may be WAY OFF....lol

    Thanks

    Jim

  9. #9
    Forum Contributor
    Join Date
    03-28-2018
    Location
    Houston, TX
    MS-Off Ver
    Microsoft Office Professional Plus 2016
    Posts
    127

    Re: HELP! My caculation is running SLOW!

    This still does not answer the question that is fundamental to providing an efficient formula.

    In your data, does each record have a valid "Base_Time"? (or is it possible for the "Base_Time field to be left blank?)

    [Edit] Let me also ask another question. What would you consider to be a MAXIMUM number of times the same "FIN" would appear in the data?
    Last edited by brant.wheeler; 05-29-2019 at 03:52 PM.

  10. #10
    Registered User
    Join Date
    09-06-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    7

    Re: HELP! My caculation is running SLOW!

    Yes, the Base_Time field is mandatory, there will be "No" blanks

    I would guess the MAX times the [Fin] could be in the dataset would be 500, but that would be rare, probably on average 150

  11. #11
    Forum Contributor
    Join Date
    03-28-2018
    Location
    Houston, TX
    MS-Off Ver
    Microsoft Office Professional Plus 2016
    Posts
    127

    Re: HELP! My caculation is running SLOW!

    Ok. Based on the information you gave me, here is my recommendation:

    The idea behind this method is to considerably reduce the calculations that are happening within each instance of the formula in the "Rating" field.

    Step 1: Use a 2-level sort to sort the table by "FIN" (any order) and then by "Base_Time" (oldest to newest). It appears you were already doing this.
    Step 2: Insert the following formula in the first data row in the "Required" column. (This is just another small efficiency thing I noticed. It should have minimal time impact)
    Formula: copy to clipboard
    Please Login or Register  to view this content.

    Step 3: Insert the following formula in the first data row in the "Rating" column.
    Formula: copy to clipboard
    Please Login or Register  to view this content.


    Explanation... There are 2 basic changes to the formula in the "Rating" column:
    1) I removed the unnecessary conditions from the COUNTIFS for meeting the "Good" and "Fair" ratings. Since "Excellent" is checked first, there is no need to have the criteria for "Base_Time" being greater than "Target_Time1" when identifying a rating of "Good". Similarly, you do not need to check if "Base_Time" is greater than "Target_Time2" when identifying "Fair" ratings.
    2) I limited the number of records that are being checked in the COUNTIFS formula to 500. For a data set of 100,000 records, this means the COUNTIFS criterion in each cell of the "Rating" field will only run 500 times instead of 100,000 times. This should reduce calculation time significantly and allow the table updates to happen in a fraction of the time it was taking. If you need to adjust the maximum number of times the FIN could happen, you can adjust the formula accordingly. Alternately, if you want it to be more dynamic, you could have a column that counts the number of times a FIN occurs, and use an offset formula (that references this count) in lieu of the direct cell references I used here. If you are interested in that, I can change the formula to reflect this.

    Thanks!

  12. #12
    Forum Contributor
    Join Date
    03-28-2018
    Location
    Houston, TX
    MS-Off Ver
    Microsoft Office Professional Plus 2016
    Posts
    127

    Re: HELP! My caculation is running SLOW!

    One last thing I forgot to mention. Do you really need "Target_Time3". Seems like a waste of space and an additional calculation that doesn't need to happen.

  13. #13
    Forum Expert ben_hensel's Avatar
    Join Date
    01-26-2012
    Location
    Northeast USA
    MS-Off Ver
    Office 365
    Posts
    2,043

    Re: HELP! My caculation is running SLOW!

    This goes in V2:
    Please Login or Register  to view this content.
    This is an array formula and must be entered into cell with CTRL+SHIFT+ENTER, not just ENTER key. Then pull down / let that autopopulate the table.

    Then delete Q:U as unnecessary, if you'd like.

    What this will do is find the minimum difference between the time in the "Base_Time" in the current row and all other "Base_Times" that are greater than the current "Base_Time", and that have the same FIN as the current row.

    Then that's fed to the MATCH to generate a simple count of how many hours that was under.

    A zero time difference is assumed to mean that there was no follow up visit after that one, marked "No Return". (That was true in all cases here, but I suppose it's theoretically possible you could have two visits with the exact same time stamp).
    Last edited by ben_hensel; 05-29-2019 at 06:01 PM.

  14. #14
    Registered User
    Join Date
    09-06-2011
    Location
    USA
    MS-Off Ver
    Excel 2010
    Posts
    7

    Re: HELP! My caculation is running SLOW!

    Brant,

    THANKS!!! your solution reduced my calculation times to under a minute!

    I used the advanced sort so I could validate my calculations, but didn't think to take advantage of it for calculations. Thanks again it really helped!

    Jim

  15. #15
    Forum Contributor
    Join Date
    03-28-2018
    Location
    Houston, TX
    MS-Off Ver
    Microsoft Office Professional Plus 2016
    Posts
    127

    Re: HELP! My caculation is running SLOW!

    jrjeffcoat,

    It just dawned on me that I complicated the solution, because I got a little too close to the problem. Please use this solution instead. In the first row of the "Rating" column, paste the following formula:
    Formula: copy to clipboard
    Please Login or Register  to view this content.


    Let me explain why. Since the data is sorted by "FIN" and then by "Base_Time", there is no need to look at the next 500 records and do some COUNTIFS. You simply need to look at the next record ONLY, because we know it will have a "Base_Time" that is closest to the record above it. This formula looks only at the record below (which is all that is needed) and determines the "Rating" based on that. This should further reduce the calculation time and will probably take the time down to less than a second. Sorry I got a little too much in the weeds and didn't provide this solution the first time around.

    Thanks!

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] Workbook running very slow
    By katiedee1625 in forum Excel Formulas & Functions
    Replies: 7
    Last Post: 09-01-2017, 04:46 PM
  2. VBA running slow
    By Toonies in forum Excel Programming / VBA / Macros
    Replies: 11
    Last Post: 11-25-2013, 04:13 PM
  3. Macro is running real slow and makes navigating the worksheet really slow after execution.
    By MichWolverines in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 10-03-2013, 04:29 PM
  4. Macro to hide/unhide cells not running or running slow
    By mbp727 in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 05-17-2013, 04:22 PM
  5. VBA Code running very slow. Need help
    By krjoshi in forum Excel Programming / VBA / Macros
    Replies: 9
    Last Post: 03-21-2013, 02:13 PM
  6. Macro running very slow
    By papaexcel in forum Excel Programming / VBA / Macros
    Replies: 7
    Last Post: 06-11-2009, 02:47 PM
  7. VBA running VERY slow
    By [email protected] in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 07-01-2005, 12:05 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1