+ Reply to Thread
Results 1 to 2 of 2

How to import big txt file into excel fast!!!

  1. #1
    Registered User
    Join Date
    02-03-2015
    Location
    Toronto
    MS-Off Ver
    2007
    Posts
    1

    How to import big txt file into excel fast!!!

    Hello I'm trying to import big txt file (800 000 lines and 10 columns) into excel sheet as fast as possible. I'm not sure if the code below is doing it efficiently but there is another problem with this code. After importing data into sheet1 all other
    data in sheet1 are cleared. Do you have any suggestion how to change the code so all data from column K on are saved and not cleared? Also do you think there is more efficient way how to import big txt file into excel? Thank you for help.


    Option Explicit

    Public Sub test()
    Columns("A:M").Select
    Selection.ClearContents
    Range("A1").Select
    copyDataFromCsvFileToSheet "E:\Robota\quotes.txt", ",", "Sheet1"
    Columns("A:A").EntireColumn.AutoFit
    End Sub

    Private Sub copyDataFromCsvFileToSheet(parFileName As String, parDelimiter As String, parSheetName As String)

    Dim data As Variant

    data = getDataFromFile(parFileName, parDelimiter)
    If Not isArrayEmpty(data) Then
    With Sheets(parSheetName)
    .Cells.ClearContents
    .Cells(1, 1).Resize(UBound(data, 1), UBound(data, 2)) = data
    End With
    End If

    End Sub

    Public Function isArrayEmpty(parArray As Variant) As Boolean
    'Returns false if not an array or dynamic array that has not been initialised (ReDim) or has been erased (Erase)

    If IsArray(parArray) = False Then isArrayEmpty = True
    On Error Resume Next
    If UBound(parArray) < LBound(parArray) Then isArrayEmpty = True: Exit Function Else: isArrayEmpty = False

    End Function

    Private Function getDataFromFile(parFileName As String, parDelimiter As String, Optional parExcludeCharacter As String = "") As Variant
    'parFileName is supposed to be a delimited file (csv...)
    'parDelimiter is the delimiter, "," for example in a comma delimited file
    'Returns an empty array if file is empty or can't be opened
    'number of columns based on the line with the largest number of columns, not on the first line
    'parExcludeCharacter: sometimes csv files have quotes around strings: "XXX" - if parExcludeCharacter = """" then removes the quotes


    Dim locLinesList() As Variant
    Dim locData As Variant
    Dim i As Long
    Dim j As Long
    Dim locNumRows As Long
    Dim locNumCols As Long
    Dim fso As Variant
    Dim ts As Variant
    Const REDIM_STEP = 10000

    Set fso = CreateObject("Scripting.FileSystemObject")

    On Error GoTo error_open_file
    Set ts = fso.OpenTextFile(parFileName)
    On Error GoTo unhandled_error

    'Counts the number of lines and the largest number of columns
    ReDim locLinesList(1 To 1) As Variant
    i = 0
    Do While Not ts.AtEndOfStream
    If i Mod REDIM_STEP = 0 Then
    ReDim Preserve locLinesList(1 To UBound(locLinesList, 1) + REDIM_STEP) As Variant
    End If
    locLinesList(i + 1) = Split(ts.ReadLine, parDelimiter)
    j = 10 'number of columns
    If locNumCols < j Then locNumCols = j
    i = i + 1
    Loop

    ts.Close

    locNumRows = i

    If locNumRows = 0 Then Exit Function 'Empty file

    ReDim locData(1 To locNumRows, 1 To locNumCols + 1) As Variant

    'Copies the file into an array
    If parExcludeCharacter <> "" Then

    For i = 1 To locNumRows
    For j = 0 To UBound(locLinesList(i), 1)
    If Left(locLinesList(i)(j), 1) = parExcludeCharacter Then
    If Right(locLinesList(i)(j), 1) = parExcludeCharacter Then
    locLinesList(i)(j) = Mid(locLinesList(i)(j), 2, Len(locLinesList(i)(j)) - 2) 'If locTempArray = "", Mid returns ""
    Else
    locLinesList(i)(j) = Right(locLinesList(i)(j), Len(locLinesList(i)(j)) - 1)
    End If
    ElseIf Right(locLinesList(i)(j), 1) = parExcludeCharacter Then
    locLinesList(i)(j) = Left(locLinesList(i)(j), Len(locLinesList(i)(j)) - 1)
    End If
    locData(i, j + 1) = locLinesList(i)(j)
    Next j
    Next i

    Else

    For i = 1 To locNumRows
    For j = 0 To UBound(locLinesList(i), 1)
    locData(i, j + 1) = locLinesList(i)(j)
    Next j
    Next i

    End If

    getDataFromFile = locData

    Exit Function

    error_open_file: 'returns empty variant
    unhandled_error: 'returns empty variant

    End Function

  2. #2
    Forum Contributor
    Join Date
    12-29-2014
    Location
    Indonesia
    MS-Off Ver
    Office 2010, 2013
    Posts
    125

    Re: How to import big txt file into excel fast!!!

    hi,
    perhaps this can help you, download this add in for excel and please try.

    http://www.rondebruin.nl/win/addins/rdbmerge.htm

    i'm alrady try it and it works...

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Import data from Excel file into filtered cells in another Excel file
    By Rebecca12 in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 12-14-2012, 04:05 PM
  2. VBA: Import CSV file into an Excel Workbook with user defined file name and directory
    By LarryLegend in forum Excel Programming / VBA / Macros
    Replies: 4
    Last Post: 09-18-2012, 04:47 AM
  3. File import - select file and import sheet(1) from closed workbook
    By Armitage2k in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 08-22-2011, 03:15 AM
  4. VBA code to import text file to the fixed excel file
    By wangdian in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 08-30-2010, 10:13 AM
  5. Replies: 0
    Last Post: 08-10-2010, 07:00 AM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1