+ Reply to Thread
Results 1 to 24 of 24

Web scraping using VBA

  1. #1
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Web scraping using VBA

    Hi Guys,

    i need to get product description from this url:

    xxx

    Attachment 687253

    I need to get whole paragraph after description atribute:

    Attachment 687254

    Please help how to do this
    Jack
    Last edited by jaryszek; 07-24-2020 at 06:29 AM.

  2. #2
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    Try this;

    Please Login or Register  to view this content.
    Last edited by Haluk; 07-18-2020 at 07:43 AM.

  3. #3
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    Hi Haluk,

    o wow thank you!

    It is possible to use something like here to not open all the time IE (if i will have array loop across urls).::

    source:

    https://stackoverflow.com/questions/...sing-excel-vba

    Please Login or Register  to view this content.

  4. #4
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    I tried to use the XMLHTTP library before posting my message but the code returned an error code 403 which means "Forbidden".

    So, I had to use the IE object.

  5. #5
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    thank you Haluk,

    so they have some kind of restrictions there?

    How to do a loop for more urls in order to not open IE constantly ?

    And second questions, how can yo get class using XMLHTTP library?

    Jacek

  6. #6
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    As I've mentioned, you cannot use XMLHTTP library for this URL.

    Please Login or Register  to view this content.

    Or, using WINHTTP class;


    Please Login or Register  to view this content.
    If it was not restricted; you could get the "response text" and write to a HTML file in memory and then get the desired data from that response.
    Last edited by Haluk; 07-18-2020 at 08:21 AM.

  7. #7
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    thank you very much,

    so in this case it is just getting responseText so this is not by class but it is taking all and you have to loop and find specific class or json or wahtever yes ?

    ok,
    how to loop through a lot of urls and not constantly close and open IE?

    Best,
    Jacek

  8. #8
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    Your question is very general, so I can only say that; this depends on what you want to fetch from the URLs and the HTML design of the URLs.

    Good luck,
    Last edited by Haluk; 07-18-2020 at 03:53 PM.

  9. #9
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    let say iwant to het specific html class. How can i do this using XMLHTTP class?

    and second questions --> using IE how can i loop thrugh a lot of urls in order to open only one time IE. not many times to not have bad performance.

    Jacek

  10. #10
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    See this for web scraping using XMLHTTP object;

    https://www.excelforum.com/excel-pro...ml#post5364077

  11. #11
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    Thank you very much!!!!

    Ok,

    i trued code and works wonderful!

    Ok asume i have 2 urls to scratch:

    https://www.g2a.com/pl-pl/minecraft-...10000032198001

    https://www.g2a.com/human-fall-flat-...10000022075006

    how can i do this in the loop?
    (i do not want to close all the time Ie and wondering if i can just add loop and this will be working without any issues?

    Jacek

  12. #12
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    Assuming that the URLs are in cells starting from A2 to down ..... the fetched data will be placed in cells from B2 to down;

    Please Login or Register  to view this content.

  13. #13
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    You are the boss !!

    Thanks!

    one more question, it is possible to take not clear text but text formatted in HTML including signs like <p> <br> and so on?

    JAcek

  14. #14
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    Change this:

    Please Login or Register  to view this content.

    to:

    Please Login or Register  to view this content.

  15. #15
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    thank you so much!!!!!


    Jacek

  16. #16
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    You're welcome...

  17. #17
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    Sorry i can not get more repo points for you

    Jacek

  18. #18
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    No problem Jacek, you're welcome ...

  19. #19
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    Haluk,

    one more thing,

    i have about 100 hundreds links and first IE class macro is goigng very slowly.
    It is possible to fasten it somehow or rather not becasue of opening internet explorer?

    this method seems to be faster but it is showing only empty msgbox in the end:

    Please Login or Register  to view this content.
    Best Wishes,
    Jacek
    Last edited by jaryszek; 07-20-2020 at 03:07 AM.

  20. #20
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: Web scraping using VBA

    Jacek, if the server housing the URL does not support fetching data via XMLHHTP, I am afraid that you've to continue using the IE object.

    But, the data may be fetched using Selenium drivers and in this case you don't have to use IE and Chrome can be used instead.

    You will have to search for using Selenium in VBA since I've not used before.

  21. #21
    Valued Forum Contributor Naveed Raza's Avatar
    Join Date
    11-04-2012
    Location
    India, Hyderabad
    MS-Off Ver
    Excel, Access 2007/2010
    Posts
    1,338

    Re: Web scraping using VBA

    You can download and install selenium basic from GITHUB and then you can write Selenium Code in Excel VBA
    Thanks - Naveed
    -----------------------------
    If the suggestion helps you, then Click * to Add Reputation
    To Attach File: Go Advanced>>Manage Attachments>>Add Files (In Top Right Corner)>>SelectFiles>>.........Locate Your File(s)>>Upload Files>>Done (In Bottom Right)
    1. Use [code] code tags [\code]. It keeps posts clean, easy-to-read, and maintains VBA formatting.
    2. If you are happy with a solution to your problem, mark the thread as [SOLVED] using the tools at the top.

  22. #22
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    Hi Guys,

    thanks!

    did you test selenium? Why selenium is faster than VBA IE class ?

    Jacek
    Last edited by jaryszek; 07-20-2020 at 04:08 AM.

  23. #23
    Valued Forum Contributor Naveed Raza's Avatar
    Join Date
    11-04-2012
    Location
    India, Hyderabad
    MS-Off Ver
    Excel, Access 2007/2010
    Posts
    1,338

    Re: Web scraping using VBA

    Hi Always use Excel VBA Selenium for any Web automation and scrapping

  24. #24
    Forum Contributor
    Join Date
    01-16-2014
    Location
    Poland
    MS-Off Ver
    Excel 2016-365
    Posts
    2,940

    Re: Web scraping using VBA

    Hi Naveed Raza,

    thanks!

    i created topics because i have issues:

    Please Login or Register  to view this content.
    Are you usingg VBA selenium add - please connect to the topic above.

    Best,
    Jacek

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] VBA Web Scraping
    By dddddmex in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 07-13-2020, 11:36 AM
  2. Web Scraping
    By CheekyMonkeh in forum Excel General
    Replies: 1
    Last Post: 10-15-2018, 02:43 PM
  3. Web Scraping using VBA
    By perpectuals in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 02-05-2018, 03:08 PM
  4. Web Scraping Help
    By rbuchei in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 03-01-2016, 11:39 PM
  5. help from web scraping
    By kirby21 in forum Excel Programming / VBA / Macros
    Replies: 7
    Last Post: 08-31-2015, 08:59 PM
  6. New to VBA - Web Scraping
    By cringle in forum Hello..Introduce yourself
    Replies: 2
    Last Post: 12-17-2013, 08:26 AM
  7. [SOLVED] Web scraping with xml
    By abousetta in forum Excel Programming / VBA / Macros
    Replies: 4
    Last Post: 09-24-2012, 05:25 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1