+ Reply to Thread
Results 1 to 5 of 5

VBA web scrape include the header and html table

  1. #1
    Registered User
    Join Date
    07-18-2011
    Location
    Hong Kong
    MS-Off Ver
    Excel 2003, 2007, 2010
    Posts
    31

    VBA web scrape include the header and html table

    hi all,

    I have VBA code that helps me to scrape a web page. The code below can only scrape the first table and it is without header.
    I want to:
    1. Search for all tables and copy the data into different worksheets separately starting at A1. (table 1 pastes to sheet1; table 2 pastes to sheet2, etc)
    2. Copy all the data including the header.
    Please Login or Register  to view this content.

  2. #2
    Forum Expert
    Join Date
    03-28-2012
    Location
    TBA
    MS-Off Ver
    Office 365
    Posts
    12,454

    Re: VBA web scrape include the header and html table

    Instead of looping over thousands of lines on a table, the fastest and easiest way is to use 'Get from web' on Get and Transform.
    I know you do not have excel 2013 and beyond. You can also excel pre-2013 get from a web function and download all tables at once. The benefits of these functions are auto refrsh and no need to loop again.
    Last edited by AB33; 12-20-2018 at 11:19 AM.

  3. #3
    Forum Guru Norie's Avatar
    Join Date
    02-02-2005
    Location
    Stirling, Scotland
    MS-Off Ver
    Microsoft Office 365
    Posts
    19,643

    Re: VBA web scrape include the header and html table

    As far as I can see that code loops through all the tables on the page but you prematurely exit the loop after the first table using Exit For.
    If posting code please use code tags, see here.

  4. #4
    Registered User
    Join Date
    07-18-2011
    Location
    Hong Kong
    MS-Off Ver
    Excel 2003, 2007, 2010
    Posts
    31

    Re: VBA web scrape include the header and html table

    @Norie i have changed code a bit, to include header and and take away the exit for. Still there is some improvement need. I am still working to improve my code.

    Sub WebScrape()

    Dim ie As Object, i As Long, strText As String

    Dim doc As Object, hTable As Object, hBody As Object, hTR As Object, hTD As Object, hHead As Object
    Dim tb As Object, bb As Object, tr As Object, td As Object, hth As Object, hh As Object

    Dim y As Long, z As Long, wb As Excel.Workbook, ws As Excel.Worksheet

    Set wb = Excel.ActiveWorkbook
    Set ws = wb.ActiveSheet

    Set ie = CreateObject("InternetExplorer.Application")
    ie.Visible = False

    y = 1 'Column A in Excel
    z = 1 'Row 1 in Excel

    ie.navigate "https://www.investing.com/rates-bonds/financial-futures", , , , "Content-Type: application/x-www-form-urlencoded" & vbCrLf

    Do While ie.Busy: DoEvents: Loop
    Do While ie.READYSTATE <> 4: DoEvents: Loop

    Set doc = ie.document
    Set hTable = doc.getElementsByTagName("table")
    For Each tb In hTable
    'tTable > thead > tr > th
    Set hHead = tb.getElementsByTagName("thead")
    For Each hh In hHead
    Set hTR = hh.getElementsByTagName("tr")
    For Each tr In hTR

    Set hth = tr.getElementsByTagName("th")
    y = 1 ' Resets back to column A
    For Each th In hth
    Debug.Print th.innerText
    ws.Cells(z, y).Value = th.innerText
    y = y + 1
    Next th
    DoEvents

    Next tr
    Exit For

    Next hh
    'th.innerText


    Set hBody = tb.getElementsByTagName("tbody")
    For Each bb In hBody

    Set hTR = bb.getElementsByTagName("tr")
    For Each tr In hTR

    Set hTD = tr.getElementsByTagName("td")
    y = 1 ' Resets back to column A
    For Each td In hTD
    Debug.Print td.innerText
    z = 2
    ws.Cells(z, y).Value = td.innerText
    y = y + 1
    Next td
    DoEvents
    z = z + 1
    Next tr
    Exit For
    Next bb
    'Exit For
    Next tb

    End Sub
    @AB33, i tried method you mentioned, however, experience comes to me that script error. Therefore, i do not keep on using method you suggested.

  5. #5
    Forum Expert
    Join Date
    03-28-2012
    Location
    TBA
    MS-Off Ver
    Office 365
    Posts
    12,454

    Re: VBA web scrape include the header and html table

    No, it is not a method. You do not have to write any code. Just type the web address and choose the number of tables you want to download. There are 10 tables and I downloaded all in 20 seconds.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. VBA Scrape data from a table in flash
    By JasperD in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 07-07-2017, 06:39 AM
  2. Scrape HTML meta data
    By ck248 in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 02-18-2017, 05:36 PM
  3. [SOLVED] VBA scrape data from website HTML
    By Strutland in forum Excel Programming / VBA / Macros
    Replies: 8
    Last Post: 02-11-2017, 06:18 PM
  4. How to add HTML header/footer to export as HTML?
    By JGCA in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 05-08-2009, 04:29 PM
  5. Convert Field to Include HTML Tags...
    By ctroyp in forum Excel Formulas & Functions
    Replies: 0
    Last Post: 05-22-2007, 01:33 PM
  6. [SOLVED] Include headers/footers when saving as HTML?
    By quartz in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 05-06-2005, 11:06 AM
  7. Include headers/footers when saving as HTML
    By quartz in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 05-05-2005, 02:06 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1