Looking for Assistance with Parsing HTML for Strings

Hi everyone

Im looking for someone that would be willing to collaborate and help me build a Sub Function, that would search some HTML of a website and return very particular results from the page. Specifically I need to search a company website, and "collect" the specs for computers that are found on that website.

The results then need to be placed into an excel spreadsheet.

I have no previous experience in Parsing HTML text and getting a string/result from it, thats why im asking for such help.

If you think you up for the challenge, or maybe this kind of thing is easy, then please message back with your interest.

example website: http://www.harveynorman.com.au/homep...ptop-blue.html

would like to gather the specs that appear on the specs tab of the page, and if possible the Price of the item that appears there.

Would preferably like it to be a module function that i could use for various other projects.

Any help would be greatly appreciated, and i would go into more detail with someone who would be willing to help / give up the time to assist and teach.

Cheers.

Andrew.

Just to be clear, are you wanting to parse information fust from http://www.harveynorman.com.au/ or different websites also?

Just Harvey Norman, that I assume would make it a lot easier.

It would be impossible otherwise

How's this as a starter for 10:

Please Login or Register  to view this content.

Hi Kyle 123

i get a error at

With oRequest
.Open "GET", URL, False
.send <<THE ERROR HAPPENS ON THIS LINE
ohtml.body.innerhtml = .responsetext
End With

The error message states that the operation has timed out.

Im not sure it makes any difference but im working with Office 2007.

Also i know i gave you the link to a specific web address, ideally what we would like to happen is, a string variable is loaded with a "Product Code" for a computer from the website.

The address that then searches the Harvey Norman website http://www.harveynorman.com.au/catalogsearch/result/?q= [INSERT PRODUCT CODE STRING HERE]

for example: if the string variable was loaded with: "F550DP-XX008H"

and the address was: http://www.harveynorman.com.au/catal...=F550DP-XX008H

Then the landing page result would be: http://www.harveynorman.com.au/compu...8h-laptop.html

Can we preform the Search/Query of the website first, to see if that item is on the website, and if it is, then parse the page for the specs for that computer.

And if that product code / computer is not found on the website, a msgbox would state that its not available.

I know this is probably challenging but it would be of a lot of help.


Cheers

Andrew

It may time out because it doesn't exist anymore? I can't access it either now - have you tried with other urls?

i just tried this one

http://www.harveynorman.com.au/compu...8h-laptop.html

and that didn't seem to work, even tho i can navigate to it with a browser.

Is there any specific References that i would need to add into the project?

Originally Posted by mc84excel

Yes I could help out provided it isn't urgent (there may be 2-3 days delay between replies).

Also be warned that I usually use an inefficient method to return web data. I automate the IE browser - it's slower so this would be a drawback if you were intending to process hundreds of pages but it is the easiest to learn.

See attached demo.

so far your method works, and its rather fast.

any way you can work on what i mentioned in Post #5 - that would make you truly a champ!

cheers.

I modified the URL to be the one that searches the website first and then gets a result. And it worked perfectly actually.

I would just like to be able to pull the Price from the webpage (If you could comment where this occurs) that would be handy.

And i noticed that it would display a message if the element we were looking for could not be found.

Would it be possible to return focus to excel, as the IE page is remaining on top. and you don't notice that the msgbox has been displayed, or when a result is found, you don't notice that it has finished as IE remains in focus.

When i get a chance i will use the debug mode to see if i can insert code to return focus to excel / my userform. i assume i will just need to use a Application.activate request to do this. but if you know of a easy method this would be great.

Cheers

Andrew

All good buddy. Iv been working on your code all day.

Iv modified it to do exactly what i want. managed to work out how it was working

Is it possible to HIDE Internet Explorer while its being parsed?

I noticed in one of the modules there is:
With rIE
.visible = Not blnInvisible

would this allow me to hide it if it was changed? and if so what would make it hidden/invisible?

Cheers

Andrew.

Im going to see if i can implement getting the price from the website today.

will let you know how i go.

Change this line

Please Login or Register  to view this content.

INTO

Please Login or Register  to view this content.

Hi mc84Excel

i just wanted to give you a big THANK YOU, today i have been working again on your code for getting the ONLINE price from the website and i have managed to get it to work.

this is what i went with in the end.

Please Login or Register  to view this content.

i know my solution is probably a little crude but it does seem to work.

many thanks for your code / help

regards

Andrew

Originally Posted by mc84excel

Heh. I told you that automating IE is the easiest method to learn when trying to scrape data from the net.



You can hide IE while automating it but if you were going to go down that route, you may as well go the whole way IMO - i.e. avoid using the IE automation method and switch to using the xmlHTTP method

Generally whenever I use IE automation method I prefer to leave it visible. I haven't found it to save much in speed by hiding it and leaving it visible has the advantage that if something goes wrong (and it will eventually - webpages don't remain the same forever), you can see where it has failed.
(Also visible has the somewhat dubious benefit of impressing non-VBA users. "Look! No hands!!" )



Did you try passing the boolean as True?

I did managed to work out hiding IE, that was also easy.

I need it hidden, because it would open IE every time you selected an item on a list.

The program works by having a list of product codes, you select one to see if its on sale. i just wanted to pull the online price, with that "look no hands" effect, and also to keep the computer screen less cluttered. i set the mouse to fmMousePointerAppStarting, so that the user knows something is happen in the background.

I continually surprise myself at how easy visual basic is to learn and interpret. but then thats probably why its called " Basic" :P



i'm assuing that if i went down the xmlHTTP path that would be a lot harder to get the result i want / learn?

regards