+ Reply to Thread
Results 1 to 7 of 7

Alternative to AVERAGEIFS to calculate 2 columns with about 1 Million rows taking days

  1. #1
    Registered User
    Join Date
    03-22-2019
    Location
    Los Angeles
    MS-Off Ver
    2013
    Posts
    34

    Question Alternative to AVERAGEIFS to calculate 2 columns with about 1 Million rows taking days

    Hello,

    I have a spreadsheet that is growing up to nearly 1 Million rows. I am calculating the average and the max along with another calculation that depends on both, but the calculation of the entire sheet causes Excel to freeze, CPU reaches 100% and because I left it overnight it was able to update all correct values (11 hrs later). Right now I am not able to save as it's not responding (9+ hrs of trying to save) and calculating threads has been at 0% (not sure if it's working the background to save?). Whether I add the formulas manually or through a macro it's the same issue.

    The following formulas are what I am using (no other formulas exist in the sheet):
    Please Login or Register  to view this content.
    Please Login or Register  to view this content.
    Please Login or Register  to view this content.
    Attached is a simplified sheet of what I am trying to accomplish. My laptop has an i7, 32GBs of RAM, so hardware is not an issue. Is there an alternate method that I can employ?
    Attached Files Attached Files
    Last edited by mrbarba; 04-25-2020 at 11:10 PM.

  2. #2
    Forum Expert
    Join Date
    11-26-2013
    Location
    Colac, Victoria, Australia
    MS-Off Ver
    Excel 2016
    Posts
    1,309

    Re: Alternative to AVERAGEIFS to calculate 2 columns with about 1 Million rows taking days

    This is outside my experience, but I think you are probably asking Excel to do more than it's designed for. Especially with AVERAGEIFS and MAXIFS formulas.

    I think you probably need a database program, or maybe the POWERQUERY add-in for Excel.

  3. #3
    Forum Guru
    Join Date
    03-02-2006
    Location
    Los Angeles, Ca
    MS-Off Ver
    WinXP/MSO2007;Win10/MSO2016
    Posts
    12,612

    Re: Alternative to AVERAGEIFS to calculate 2 columns with about 1 Million rows taking days

    Functions like Average and max are 'usually' used to summarize/compare data. You have "millions" of rows of data so, in columns E & F you have millions of cells each one calculating avgs and max's on millions of cells. How many actual unique dates to you really need to analyze at one time?
    Ben Van Johnson

  4. #4
    Registered User
    Join Date
    03-22-2019
    Location
    Los Angeles
    MS-Off Ver
    2013
    Posts
    34

    Re: Alternative to AVERAGEIFS to calculate 2 columns with about 1 Million rows taking days

    Fortunately, after 44+ hours the document saved and I was able to convert the formulas to values to avoid further headaches. Very happy about this. I guess I can set this as solved now, but would like some pointers if possible for the code below.

    Quote Originally Posted by David A Coop View Post
    This is outside my experience, but I think you are probably asking Excel to do more than it's designed for. Especially with AVERAGEIFS and MAXIFS formulas.

    I think you probably need a database program, or maybe the POWERQUERY add-in for Excel.
    I've only used POWERQUERY to merge a large number of files into a table but didn't consider using it for my other needs. Based on my searches I would have to create my own formulas to get the data that I am seeking since Excel functions are not available through POWERQUERY. I have some homework to do regarding databases and POWERQUERY. Thanks for the suggestions.

    Quote Originally Posted by protonLeah View Post
    Functions like Average and max are 'usually' used to summarize/compare data. You have "millions" of rows of data so, in columns E & F you have millions of cells each one calculating avgs and max's on millions of cells. How many actual unique dates to you really need to analyze at one time?
    I realized it wasn't efficient to do it this way once each cell was looking at millions of rows. Each type (A, B, C, etc) has nearly a year and half of data. I need all these values available on the fly, so I don't have an alternative. I am thinking a VBA macro could work with For and IFs loops to look at and compare if B2 and C2 remain the same one row below and if true this would develop a range for values I need to calculate in column D. Once the range is created it would calculate the average and max for this small range, and continue through the workbook. This isn't my strong suit, so it'll take me a bit to develop.

    So far this test code works to select my target column. I am just not sure how to create the range I need using another For or If loop.
    Please Login or Register  to view this content.

  5. #5
    Forum Guru Bo_Ry's Avatar
    Join Date
    09-10-2018
    Location
    Thailand
    MS-Off Ver
    MS 365
    Posts
    7,211

    Re: Alternative to AVERAGEIFS to calculate 2 columns with about 1 Million rows taking days

    I would use Pivot Table, with a few click
    But LF need to add formula
    =K2/L2/24

    or Power Pivot by Insert Pivot and check add to data model
    Power Pivot ribbon New Measure

    Average
    =AVERAGE(Range[Value])

    Max
    =MAX([Value])

    LF
    =DIVIDE([Average],[Max])/24
    Attached Files Attached Files

  6. #6
    Forum Expert BMV's Avatar
    Join Date
    01-13-2020
    Location
    St-Petersburg, Russia
    MS-Off Ver
    2013-2016
    Posts
    1,329

    Re: Alternative to AVERAGEIFS to calculate 2 columns with about 1 Million rows taking days

    mrbarba for your source you can use
    Please Login or Register  to view this content.
    that decreased calculation
    For maxif also
    But i would like to use PivotTable and get data by GETPIVOTDATA
    Attached Files Attached Files

  7. #7
    Registered User
    Join Date
    03-22-2019
    Location
    Los Angeles
    MS-Off Ver
    2013
    Posts
    34

    Re: Alternative to AVERAGEIFS to calculate 2 columns with about 1 Million rows taking days

    Quote Originally Posted by Bo_Ry View Post
    I would use Pivot Table, with a few click
    But LF need to add formula
    =K2/L2/24

    or Power Pivot by Insert Pivot and check add to data model
    Power Pivot ribbon New Measure

    Average
    =AVERAGE(Range[Value])

    Max
    =MAX([Value])

    LF
    =DIVIDE([Average],[Max])/24
    I was trying to do this with by adding a PivotTable using the Data Ribbon tab, but wasn’t able to. I didn’t know about the Power Pivot COM Add-in. Thank you for providing this information Bo_Ry! This would have saved me so much time. I have much to research now.

    Quote Originally Posted by BMV View Post
    mrbarba for your source you can use
    Please Login or Register  to view this content.
    that decreased calculation
    For maxif also
    But i would like to use PivotTable and get data by GETPIVOTDATA
    This is good because it can save excel from looking at millions of cells (based on my data) which will reduce calculation time. Thanks BMV.
    Based on your guys feedback the PivotTable is the best solution, specially now that I now of the Power Pivot.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 4
    Last Post: 03-31-2016, 05:50 AM
  2. Replies: 7
    Last Post: 02-19-2016, 03:50 PM
  3. Faster alternative to sumifs and averageifs needed
    By BlueDan2015 in forum Excel Formulas & Functions
    Replies: 2
    Last Post: 01-30-2015, 06:44 AM
  4. Replies: 7
    Last Post: 08-05-2014, 05:34 AM
  5. Replies: 9
    Last Post: 12-18-2012, 04:33 PM
  6. Delete duplicate rows with same data in certain columns for million rows
    By amlan009 in forum Excel Programming / VBA / Macros
    Replies: 9
    Last Post: 02-08-2012, 12:53 AM
  7. Excel 2007 : AVERAGEIFS alternative
    By shaunk in forum Excel General
    Replies: 5
    Last Post: 04-09-2009, 09:20 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1