+ Reply to Thread
Results 1 to 8 of 8

Import TXT and Speed up processing Time

  1. #1
    Forum Contributor Webbers's Avatar
    Join Date
    09-06-2005
    Location
    Poinciana, FL
    MS-Off Ver
    Microsoft Office 365 ProPlus
    Posts
    337

    Import TXT and Speed up processing Time

    I found a code that works. But I need to accomplish a bit more.

    1) I need to be able to add the headers into this. I am dealing with end users with basic Excel skills so I need to anticipate problems. Headers being deleted has been an issue in other files used by this group. There are a total of 37 headers, if you will simply provide that script and the syntax so I can see how it is done, I can overwrite the first few, then add the additional ones. OR would the creation of headers from another worksheet be easier? I have the list there for another purpose. The named range is: codes_Headers and is located on the sheet named "codes" (sheet1) H2:H38. I don't know the best way to do this, so I want to offer all the information.

    2) At almost the bottom of the code is an array for the data type in the columns. In this example, there are 8 numbers listed in the array. I am assuming that the data had 8 columns. The 1 is listed as 1. I am guessing as 1, is the for General? Now I have a few columns that are dates and a few that are text. I also have one that is currently showing as custom, as it is a date/time stamp (for example: 1/28/2019 15:27). Now because I have a total of 37 columns, do I need 37 numbers within my array, or is there an easier way to do that part?

    3) My actual data file has over 37K records. This is the first file I have received, at the very least this is an average, I do not know how much more this file could potentially have. The run time of this macro was 40 seconds. Is there anything within the code that can be made more efficient so it is faster when it is run?

    I am including a sample file. I have converted the confidential data to garbage. There are about 10 records or so in the sample, but the format is represented.


    Please Login or Register  to view this content.
    Attached Images Attached Images
    Attached Files Attached Files
    ~*~ Sherry ~*~
    Poinciana, FL

  2. #2
    Valued Forum Contributor PaulSP8's Avatar
    Join Date
    11-21-2017
    Location
    Chorley, England
    MS-Off Ver
    2013 & 2021
    Posts
    437

    Re: Import TXT and Speed up processing Time

    If you want to check for a header row then something like;

    Please Login or Register  to view this content.
    For the import part then record a Macro of you Importing the data you want and then replace (copy / paste) the enclosed With statements from the recorded Macro to this existing code.

  3. #3
    Forum Contributor Webbers's Avatar
    Join Date
    09-06-2005
    Location
    Poinciana, FL
    MS-Off Ver
    Microsoft Office 365 ProPlus
    Posts
    337

    Re: Import TXT and Speed up processing Time

    PaulSP8,

    Okay, That solves item #1 on my list. Thanks. Now to figure out answers to the other 2 issues.

    Please Login or Register  to view this content.

  4. #4
    Valued Forum Contributor PaulSP8's Avatar
    Join Date
    11-21-2017
    Location
    Chorley, England
    MS-Off Ver
    2013 & 2021
    Posts
    437

    Re: Import TXT and Speed up processing Time

    Did you record yourself Importing the data?

    Yes - You'll need to change the Array to make it fit your data.

    3 is the number you're looking for with date fields (M/D/Y) and not 1 (https://docs.microsoft.com/en-us/off...columndatatype)

    Please Login or Register  to view this content.
    But when you're dealing with Excel and dates then I've found that anything can happen with regards to the way it formats them when importing, no matter what you do. You can tell it MM/DD/YYYY but it'll still do DD/MM/YYYY.

    (I very often end up with a mixed bag of DD/MM/YYYY, MM/DD/YYYY type dates.)

    Your custom date / time stamp field will probably sort it self out under an "General" import on the field - I don't know of a way of telling Excel that it's a custom date / time stamp field specifically.

    With regards to the speed of it all - Then it looks like everything is already done that you can do that way - Sometimes Excel is just slow at doing these things and it'll very much depend on the PC's capabilities with Processing Speed and Memory coming into play. You can try closing everything else down while it's Importing and just leaving the computer to it.

  5. #5
    Forum Contributor Webbers's Avatar
    Join Date
    09-06-2005
    Location
    Poinciana, FL
    MS-Off Ver
    Microsoft Office 365 ProPlus
    Posts
    337

    Re: Import TXT and Speed up processing Time

    PaulSP8,

    I completely missed that... this week has been a blur so far. I just did the recording of the import. I thought so on the speed... like most companies, we are working on cheap off the shelf "Playskool" lapstops with no power. They want us to muti-task, but these machines are barely capable of having Outlook and Excel open at the same time let alone actually doing something! LOL

    Please Login or Register  to view this content.
    Last edited by Webbers; 10-09-2019 at 10:25 AM.

  6. #6
    Valued Forum Contributor PaulSP8's Avatar
    Join Date
    11-21-2017
    Location
    Chorley, England
    MS-Off Ver
    2013 & 2021
    Posts
    437

    Re: Import TXT and Speed up processing Time

    Yup, heard that - We're the same here.

    On the plus side, it drives you to streamline your processes as much as you can so you actually become more efficient because of it, I think.

    Anywaaaaaaaaaaaaay.

    Are you good with your code now?

    Should look a little something like this?

    Please Login or Register  to view this content.

  7. #7
    Forum Contributor Webbers's Avatar
    Join Date
    09-06-2005
    Location
    Poinciana, FL
    MS-Off Ver
    Microsoft Office 365 ProPlus
    Posts
    337

    Re: Import TXT and Speed up processing Time

    Paul---

    Question, can we do something about this part of the code? The path and file name cannot be hard coded into the file like this as this is something being created for an end user. And there cannot be any hard coding of paths and file names as they change. Can ya help?

    Please Login or Register  to view this content.

  8. #8
    Valued Forum Contributor PaulSP8's Avatar
    Join Date
    11-21-2017
    Location
    Chorley, England
    MS-Off Ver
    2013 & 2021
    Posts
    437

    Re: Import TXT and Speed up processing Time

    If they change then you should probably use the File Open dialog, try this;

    Please Login or Register  to view this content.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Excel Processing Speed
    By wilfrid147 in forum Excel General
    Replies: 10
    Last Post: 08-03-2015, 08:19 AM
  2. Speed up the processing
    By robtuby in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 10-31-2014, 11:41 AM
  3. Reduce processing time for processing multiple excel sheets
    By rexer231 in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 01-06-2014, 09:25 AM
  4. Excel processing speed
    By tony.nz in forum Excel General
    Replies: 2
    Last Post: 11-13-2009, 03:42 PM
  5. Increasing Processing Speed
    By boylejob in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 06-11-2008, 02:19 AM
  6. Processing speed
    By maacmaac in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 03-07-2008, 02:21 PM
  7. [SOLVED] speed of processing
    By Chas in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 06-08-2006, 09:20 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1