+ Reply to Thread
Results 1 to 2 of 2

Log External Data Over Time

  1. #1
    Registered User
    Join Date
    07-25-2017
    Location
    Boise, ID
    MS-Off Ver
    2016
    Posts
    1

    Post Log External Data Over Time

    Hello,

    I am currently working on a project that requires me to import data from other workbooks. I am proficient in external data sources and setting up macros to pull the data into my main workbook. The issue is I need to write part of a sub that somehow charts the data over time. Ideally it would check today's date, if the date does not match (ie in the future) it would paste the data on the next fully blank row. This way I could assign a sheet to each employee and keep a running 30 day record of their data in order to build daily, weekly, and monthly dashboard sheets within my workbook.

    Ex:

    07/24/17:
    -------------------------------------
    Employee Data
    -------------------------------------
    07/25/17:
    -------------------------------------
    Employee Data for this date
    -------------------------------------
    .
    .
    .

    Etc for each time the "Import Data" Button is clicked. My goal is to have 3 main sheets, a daily, weekly, and monthly dashboard that only displays data from the correct time frame. I can figure this bit out, but getting the data to import in a way that I can reference it properly is where I am getting hung up. Is this something that can be accomplished with external data connections instead of VBA?


    Any input or ideas would be much appreciated.

  2. #2
    Forum Guru
    Join Date
    04-13-2005
    Location
    North America
    MS-Off Ver
    2002/XP and 2007
    Posts
    15,814

    Re: Log External Data Over Time

    After half a day, I will offer this to help discussion. I will say that I am not an expert on this kind of data processing/database management, so someone who is more skilled at this may have better suggestions to offer.

    I might suggest a more detailed example. More detail about the text file. More detail about how you are currently processing these files. More details about what you know and don't know how to do. For example, are the two "records" given very accurate as far as the arrangement of the text file? Is there really a row of hyphens in between date and data like indicated? How many fields in the employee data? This is the kind of question that probably benefits greatly from a good sample file or files (maybe a sample of your text file and a sample of the current spreadsheet you are using to process the files that shows what is working, what you are having trouble with, and an indication of what you expect at the end of the process).

    Here's a brief overview/flowchart of how I think I would process these files (note that I probably would not use VBA for this, but one should be able to program the steps in VBA as well. It depends on which programming language is preferred).

    0) I am assuming that the text file is comma delimited or otherwise easily parsed into fields. I'm not sure how much difference there is if I assume rows of hypens or not.
    1) Import raw data file into an empty tab. Import External Data -> From Text -> text import settings as needed -> Save query definition so that the file can easily be refreshed as it changes.
    2) In an adjacent tab, use lookup functions (MATCH(), INDEX(), etc.) to rearrange the data into a more "database" format where I get one record/row per date.
    Ex: date -- data field 1 -- data field 2 -- data field 3 --...
    2a) I would be concerned that the lookups in this could be slow, as they will likely need to be exact match linear searches. I would try to pay attention to duplicated efforts in these lookups to try to make them as streamlined as possible.
    2b) If I had any control over the program creating these text files, I would see if there were options in that program to arrange the data better.
    3) With the data arranged in a "one row per record" kind of format, I would have a very nice "database" source for pivot tables (can we assume you are familiar with pivot tables?).
    3a) one pivot table to do daily summary
    3b) a second pivot table to do weekly summary
    3c) a third pivot table to do the monthly summary

    Does that look anything like the procedure you are trying to use? I get the impression that you know how to do step 1 and step 3 already, so maybe step 2 is the step that you are having trouble with?
    Quote Originally Posted by shg
    Mathematics is the native language of the natural world. Just trying to become literate.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] Extracting data from external source time out
    By excelhelpguy123 in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 06-08-2017, 09:25 AM
  2. External Data from Microsoft Query - Excel File - Causes external file to open.
    By lee1000d in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 06-25-2015, 02:25 PM
  3. Looking up to external data.....but......time critical
    By jimmy05 in forum Excel General
    Replies: 4
    Last Post: 03-12-2014, 04:56 AM
  4. Creating a macro to auto-update external data and time macro was excecuted
    By UNWATCHABLE in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 01-28-2010, 05:21 AM
  5. External Data Query | Time Out
    By John Bates in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 04-29-2009, 01:08 PM
  6. Replies: 3
    Last Post: 06-27-2007, 03:16 AM
  7. [SOLVED] “Import External Data:Import External Data
    By Jeff in forum Excel Formulas & Functions
    Replies: 0
    Last Post: 05-03-2005, 06:06 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1