+ Reply to Thread
Results 1 to 4 of 4

Is it possible to save my query criteria to apply to new data source / connection

  1. #1
    Registered User
    Join Date
    03-06-2015
    Location
    London
    MS-Off Ver
    Mac Office
    Posts
    45

    Is it possible to save my query criteria to apply to new data source / connection

    Hi,

    I have a huge csv database split across about 8 files (due to file size). To extract the data I need I run queries in excel and edit the query (involves filtering, splitting rows, changing number formats etc.). So this means doing this 8 times for each csv file (the criteria are the same for each one).

    Currently I've saved the query editor formulas I need in a word doc and copy and paste them in but I was wondering if it's possible to save the query critieria so that I can load them up when I load the next csv file?

    E.g. I'd do one sheet, then load a new connection to the next sheet and then somehow just apply / load my saved query to apply to the newly loaded sheet?

  2. #2
    Valued Forum Contributor Neil_'s Avatar
    Join Date
    04-19-2013
    Location
    Yorkshire
    MS-Off Ver
    Office 365 Enterprise E3 2013 / 2016
    Posts
    479

    Re: Is it possible to save my query criteria to apply to new data source / connection

    huge csv database split across about 8 files (due to file size)
    It may be time to get this into Access. You can link Access to the single .csv file and create connections in Excel to the database whilst levering Access to do the queries for you.
    Last edited by Neil_; 10-18-2016 at 03:05 AM.
    Frob first, tweak later

  3. #3
    Registered User
    Join Date
    03-06-2015
    Location
    London
    MS-Off Ver
    Mac Office
    Posts
    45

    Re: Is it possible to save my query criteria to apply to new data source / connection

    Hi Neil, thanks for the quick response. I've had a little exposure to Access but have a vague recollection that there's some problem about 2GB limit for Access (I've got 8x 8GB files).

    So are you suggesting I could combine into one master CSV file (that would be say 64GB) and then run query in Access to extract the data (the data I need probably would be under 2gb when filtered so that could work if I understand correctly?

    The problem then is that I then need to use a pivot table to filter that data into the format I need (which ultimately does boil it down to around 40,000 lines of data) and have also split the csv files because trying to run query and import to excel goes well over the rows Excel can handle (even using 64 bit version).

    Sorry a bit jumbled there maybe so hope that makes sense?

  4. #4
    Valued Forum Contributor Neil_'s Avatar
    Join Date
    04-19-2013
    Location
    Yorkshire
    MS-Off Ver
    Office 365 Enterprise E3 2013 / 2016
    Posts
    479

    Re: Is it possible to save my query criteria to apply to new data source / connection

    sorry I thought you were splitting it first yourself. You could link to each of the files.
    Create a new access database. Go to the External data tab and select text file. Tick 'Link' rather than import. repeat for each of your sheets. As long as the sheets have the same file names every time, the links will stay good. You can create queries in Access at this point to filter and combine the data. The final query can be the data source for your pivot table.

    I'd try it before writing it off. The limit is on the size of the Access file, not it's source. However it is likely to be slow manipulating the data if they're that huge. A local machine running SQL Server would be more appropriate (you could virtualize this on your machine), but that has it's own learning curve.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Auto save Excel ODBC Connection to data source without opening file
    By newbie4 in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 04-01-2015, 01:37 PM
  2. Pivot table - break connection to data source
    By TheRobsterUK in forum Excel General
    Replies: 8
    Last Post: 04-09-2014, 07:21 AM
  3. Replies: 2
    Last Post: 10-08-2013, 10:42 AM
  4. Replies: 3
    Last Post: 06-14-2012, 02:57 PM
  5. Replies: 3
    Last Post: 05-24-2011, 05:33 AM
  6. Extracting of data from a website using VBA or XML or data source connection
    By joelhuang in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 12-19-2010, 09:57 PM
  7. [SOLVED] Save DB connection and query within spreadsheet
    By Doug in forum Excel General
    Replies: 2
    Last Post: 01-30-2006, 01:50 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1