Hello everyone,
I'm trying to do the following:
1. Visit a website
2. Fill out an input box
3. Click Submit
4. Scrape text from the resulting webpage
I've pasted my current code at the end of this post. For some reason, that last htmlDoc.body.innerHTML shows the body of the first webpage, not the new webpage after the submit button was clicked. I assumed that by saying Set htmlDoc = objIE.Document after clicking submit, it sets htmlDoc as the new active webpage, so when I grab the body text, it should grab the text from the second page, no?
I'm really new to all this, so if anyone has a link to a primer on scraping data from web pages with Excel, it would be greatly appreciated as well.
Dim objIE As InternetExplorer
Dim htmlDoc As HTMLDocument 'Microsoft HTML Object Library
Dim htmlInput As HTMLInputElement
Dim htmlColl As IHTMLElementCollection
Set objIE = New InternetExplorer
With objIE
.Navigate "https://website.com" ' Main page
.Visible = 1
Do While .READYSTATE <> 4: DoEvents: Loop
Application.Wait (Now + TimeValue("0:00:02"))
'set user name and password
Set htmlDoc = .Document
Set htmlColl = htmlDoc.getElementsByTagName("INPUT")
Do While htmlDoc.READYSTATE <> "complete": DoEvents: Loop
For Each htmlInput In htmlColl
If htmlInput.Name = "email" Then
htmlInput.Value = "[email protected]"
End If
Next htmlInput
'click login
Set htmlDoc = .Document
Set htmlColl = htmlDoc.getElementsByTagName("input")
Do While htmlDoc.READYSTATE <> "complete": DoEvents: Loop
For Each htmlInput In htmlColl
If Trim(htmlInput.Type) = "submit" Then
htmlInput.Click
Exit For
End If
Next htmlInput
End With
Set htmlDoc= objIE.Document
Sheets("Sheet1").Range("A1").Value = htmlDoc.body.innerHTML
Bookmarks