+ Reply to Thread
Results 1 to 6 of 6

Merging 2 large CSV files for 1 Power Pivot analysis

  1. #1
    Forum Contributor Portuga's Avatar
    Join Date
    02-20-2004
    Location
    Portugal
    MS-Off Ver
    365
    Posts
    852

    Merging 2 large CSV files for 1 Power Pivot analysis

    Hi, I have 2 large CSV Files (600 MB+ each with over 2 million rows each) that I want to merge in excel for a single Power Pivot analysis.

    Both files have the same structure and column titles. (each file has data for a different year)

    I managed to create a power pivot for each file without issues.
    I then tried to "create relationship" on the "design" menu for both datasets but got a:
    "relationship between (...) is a many-to-many relationship which is not supported" error message.

    Is there another way I can "merge" these 2 datasets this in excel?
    (Excel 2016)

    Thanks

  2. #2
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,779

    Re: Merging 2 large CSV files for 1 Power Pivot analysis

    Have you tried merging them using Power Query?
    Ali


    Enthusiastic self-taught user of MS Excel who's always learning!
    Don't forget to say "thank you" in your thread to anyone who has offered you help.
    You can reward them by clicking on * Add Reputation below their user name on the left, if you wish.

    Forum Rules (updated August 2023): please read them here.

  3. #3
    Forum Contributor Portuga's Avatar
    Join Date
    02-20-2004
    Location
    Portugal
    MS-Off Ver
    365
    Posts
    852

    Re: Merging 2 large CSV files for 1 Power Pivot analysis

    I tried to see the query option but abandoned it when I saw that I had to load the data into excel.
    Wouldnīt that make the query too big (over the supported row limit) ?

  4. #4
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,779

    Re: Merging 2 large CSV files for 1 Power Pivot analysis

    Sorry - I understood that you wanted to do this in Excel from the final line of your opening post. Now I'm confused! Are you saying you do not want to get the data into Excel? Have you looked at creating a data model (lots of help on doing this online)?

  5. #5
    Forum Contributor Portuga's Avatar
    Join Date
    02-20-2004
    Location
    Portugal
    MS-Off Ver
    365
    Posts
    852

    Re: Merging 2 large CSV files for 1 Power Pivot analysis

    I will have a look into trhis. (Not my specialty).

    Thanks
    If you found the solution to your question. Mark the thread as "Solved"
    Thank everyone that helped you with a valid solution by clicking on their

    There is no such thing as a problem, only a temporary lack of a solution

  6. #6
    Forum Contributor Portuga's Avatar
    Join Date
    02-20-2004
    Location
    Portugal
    MS-Off Ver
    365
    Posts
    852

    Re: Merging 2 large CSV files for 1 Power Pivot analysis

    Using the query/data model combo option was indeed the answer:

    After choosing the csv file on the "New Query" tab:
    "Load to"
    "Only Create Connection"
    "Add this data to the Data Model"

    After both files are loaded, on the "New Query" tab:
    "Combine Queries" + "Ammend"
    This creates a 3rd Query with the merged data.

    The merged query can the be accessed with Power Pivot.

    I donīt like the fact that it created a 160MB sheet, but it does what it says on the tin.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Sub-Forum for Excel Power Tools (Power Query, Power Pivot & Power BI)
    By chullan88 in forum Suggestions for Improvement
    Replies: 10
    Last Post: 06-28-2018, 02:25 PM
  2. Power Query, merge 2 or more large files in one
    By Remphan in forum Excel General
    Replies: 0
    Last Post: 03-02-2016, 04:28 AM
  3. Power Query, merge 2 or more large files in one
    By Remphan in forum Excel General
    Replies: 0
    Last Post: 03-02-2016, 03:30 AM
  4. pivot tables vs. power pivot for large data set
    By stephme55 in forum Excel General
    Replies: 3
    Last Post: 02-08-2016, 05:40 PM
  5. Replies: 0
    Last Post: 07-28-2015, 09:58 AM
  6. Replies: 2
    Last Post: 03-06-2013, 06:43 AM
  7. Importing multiple large .csv files for analysis (Run Time Error - 7)
    By xxtoddyxx in forum Excel Programming / VBA / Macros
    Replies: 6
    Last Post: 12-11-2012, 11:36 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1