+ Reply to Thread
Results 1 to 9 of 9

Power Query: Extract data from table after other rows are filtered

  1. #1
    Forum Contributor
    Join Date
    11-17-2004
    MS-Off Ver
    Office 2016
    Posts
    527

    Power Query: Extract data from table after other rows are filtered

    I have a table that has over 130k rows of data (the example attachment was cut down to meet forum standards of 1m). The original source is about 390+ rows multiplied by the number of years since 1990 times the 12 months of the year. I'm looking for the data from only the 12th month of each year, except for the latest year, which is only available for the first 6 months of the year.

    In the attachment the resultant table shows the data for the all of the years from 2010 to 2016 but I need a temporary row 2017 that I can populate with the 6th month data alongside the other data. I'd be able to do this with an index/match in a regular table, but I'm trying to use PQ to accomplish this due to the amount of data in the original source and the time it takes to do it in Excel. I'm not M expert so trying to perform this task is hurting my brain...

    The attachment was reduced to only June and December data for 2010 - 2017 only. Any assistance is greatly appreciated...
    Attached Files Attached Files

  2. #2
    Forum Expert avk's Avatar
    Join Date
    07-12-2007
    Location
    India
    MS-Off Ver
    Microsoft Office 2013
    Posts
    3,223

    Re: Power Query: Extract data from table after other rows are filtered

    Firstly remove both sheet table.
    In "Report Data" sheet in "F4" [shift+ctl+enter]
    Formula: copy to clipboard
    Please Login or Register  to view this content.

    copy paste across.


    atul


    If my answer (or that of other members) has helped you, please say "Thanks" by clicking the Add Reputation button at the foot of one of their posts.

    Also: if your problem is solved, please take the time to mark your thread as SOLVED by going to the top of your first post, selecting "Thread Tools" and then "Mark thread as solved".

  3. #3
    Forum Contributor
    Join Date
    11-17-2004
    MS-Off Ver
    Office 2016
    Posts
    527

    Re: Power Query: Extract data from table after other rows are filtered

    Thanks, but I still need a power query result due to the amount of source data I'm working with. This is a fine solution for the size of the data set I provided in the example but I cut out 8mb of data that the index/match would have to cut through and that can cause the PC to practically shut down any time it recalculates. It's over 130,000 rows of data. Also, there's a ton of calculations in 15 or so additional columns that I will be creating that perform comparisons between one year's data to the next, etc. These additional calculations along with the Index/Match's was killing the performance and was the reason to move this initial step to power query. Otherwise this is a great way to get the source data together.

    In my PQ example I used a filtered column to filter out everything but the December data but I need a temporary column for any current year since it will not have December data yet. In the example data, that is the 6th month.

    The complete dataset can be found at: https://www.bls.gov/web/metro/ssamatab1.zip if anyone is interested.
    Last edited by JimDandy; 08-21-2017 at 08:01 PM.

  4. #4
    Forum Expert avk's Avatar
    Join Date
    07-12-2007
    Location
    India
    MS-Off Ver
    Microsoft Office 2013
    Posts
    3,223

    Re: Power Query: Extract data from table after other rows are filtered

    Why not use Pivot. Refer attachment.
    In attachment i have cut down data record.
    You can add your data in sheet "ssamatab1"
    & in Pivot sheet go to "Option" tab & click on refresh all.
    Attached Files Attached Files

  5. #5
    Forum Guru xlnitwit's Avatar
    Join Date
    06-27-2016
    Location
    London
    MS-Off Ver
    Windows: 2010; Mac: 16.13 (O365)
    Posts
    7,085

    Re: Power Query: Extract data from table after other rows are filtered

    Hi,

    You might use a separate query to return the maximum month for each year and then join that query to your main query to restrict the data to only the relevant month and year combinations.
    Don
    Please remember to mark your thread 'Solved' when appropriate.

  6. #6
    Forum Contributor
    Join Date
    11-17-2004
    MS-Off Ver
    Office 2016
    Posts
    527

    Re: Power Query: Extract data from table after other rows are filtered

    I don't see how the Pivot solution helps. The requirement is that I find the Employment value for each year and the 12th month except for the current year (2017), which only has the 6th month of data available. In the Pivot I would get blank values for the current year.

    My Power Query and M skills are very limited so I'm trying to work through the suggestion of a separate query.

  7. #7
    Forum Contributor
    Join Date
    11-17-2004
    MS-Off Ver
    Office 2016
    Posts
    527

    Re: Power Query: Extract data from table after other rows are filtered

    Is there any way to replace the Filtered Rows step in the query to one that selects, for each individual year, the maximum value in the Month column? This would result in selecting the 12th month for all years except 2017, where it would select the 6th month?

  8. #8
    Forum Guru xlnitwit's Avatar
    Join Date
    06-27-2016
    Location
    London
    MS-Off Ver
    Windows: 2010; Mac: 16.13 (O365)
    Posts
    7,085

    Re: Power Query: Extract data from table after other rows are filtered

    Yes- here is your file revised to show my suggested approach. I removed the pivot in order to get replace the filtering steps and link on the new key column, so you will need to reinstate the pivot for the layout you want but I hope that the approach is fairly self-explanatory.
    Attached Files Attached Files

  9. #9
    Forum Contributor
    Join Date
    11-17-2004
    MS-Off Ver
    Office 2016
    Posts
    527

    Re: Power Query: Extract data from table after other rows are filtered

    xlnitwit, this is a perfect approach, and I very much appreciate your help. Thank you...

    In my original source, I had nearly a dozen additional calculated columns and I needed to insert your solution into the middle of the mix and, after renaming all the year columns, everything came out great.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Replies: 11
    Last Post: 08-01-2017, 06:08 AM
  2. [SOLVED] Pivot table from power query
    By pccamara in forum Excel Charting & Pivots
    Replies: 12
    Last Post: 05-17-2017, 10:09 AM
  3. Power Query - add column with Table name
    By zico8 in forum Excel Charting & Pivots
    Replies: 2
    Last Post: 05-11-2017, 12:58 AM
  4. [SOLVED] Is there a way to add a normal excel formula to a Power Query Table
    By beaglesBuddy in forum Excel General
    Replies: 1
    Last Post: 07-09-2016, 04:30 PM
  5. Adding data to the end of a table with Power Query
    By afila in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 06-14-2016, 08:29 AM
  6. Remove Unique rows in Power Query
    By danielmorrison in forum Excel General
    Replies: 2
    Last Post: 03-11-2016, 09:06 AM
  7. Replies: 4
    Last Post: 04-25-2013, 09:13 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1