+ Reply to Thread
Results 1 to 2 of 2

HTML scraping with a help of Selenium

Hybrid View

  1. #1
    Forum Contributor
    Join Date
    09-21-2016
    Location
    UK
    MS-Off Ver
    2016
    Posts
    131

    HTML scraping with a help of Selenium

    Hello all,

    I currently got a VBA code with elements of Selenium which helps me to return web data into the sheet where I can use it to generate pivot table. Currently this code to return data has to open web browser and click individual list object. I'm new with selenium so I am wondering if it is possible to optimise this code so the web browser doesn't need to be open or any other suggestions improve run time are appreciated.

    Sub optigura_scraper_v2()
        Dim driver As New ChromeDriver
        Dim elems As Object, post As Object
    
        driver.Get "https://www.optigura.com/uk/product/gold-standard-100-whey/"
    
        Set elems = driver.FindElementsByXPath("//span[@class='img']/img")
        i = 2
    
        For n = 1 To elems.Count
            driver.FindElementsByXPath("//span[@class='img']/img")(n).Click
            driver.Wait 1000
            For Each post In driver.FindElementsByXPath("//div[@class='colright']//ul[@class='opt2']//label")
                Cells(i, 1) = driver.FindElementByXPath("//h1[@itemprop='name']").Text & " " & post.Text
                Cells(i, 2) = Split(driver.FindElementByXPath("//li[@class='active']//span[@class='img']/img").Attribute("alt"), "-")(1)
                Cells(i, 3) = driver.FindElementByXPath("//span[@class='price']").Text
                i = i + 1
            Next post
        Next n
    End Sub
    Capture.PNG

  2. #2
    Forum Contributor
    Join Date
    11-19-2015
    Location
    India
    MS-Off Ver
    excel 2013
    Posts
    124

    Re: HTML scraping with a help of Selenium

    Ok if you want web browser invisible you can use PhantomJs.

    Sub optigura_scraper_v2()
        Dim driver As New PhantomJSDriver
        Dim elems As Object, post As Object
    
        driver.Get "https://www.optigura.com/uk/product/gold-standard-100-whey/"
    
        Set elems = driver.FindElementsByXPath("//span[@class='img']/img")
        i = 2
    
        For n = 1 To elems.Count
            driver.FindElementsByXPath("//span[@class='img']/img")(n).Click
            driver.Wait 1000
            For Each post In driver.FindElementsByXPath("//div[@class='colright']//ul[@class='opt2']//label")
                Cells(i, 1) = driver.FindElementByXPath("//h1[@itemprop='name']").Text & " " & post.Text
                Cells(i, 2) = Split(driver.FindElementByXPath("//li[@class='active']//span[@class='img']/img").Attribute("alt"), "-")(1)
                Cells(i, 3) = driver.FindElementByXPath("//span[@class='price']").Text
                i = i + 1
            Next post
        Next n
    End Sub

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. VBA scraping of HTML
    By ChipsSlave in forum Excel Programming / VBA / Macros
    Replies: 6
    Last Post: 07-28-2017, 01:06 PM
  2. Selenium Basic for Excel VBA References
    By Naveed Raza in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 04-06-2017, 08:12 AM
  3. [SOLVED] VBA scraping specific value from website HTML
    By Strutland in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 03-25-2017, 10:06 AM
  4. Replies: 2
    Last Post: 01-07-2016, 12:25 PM
  5. VBA and Selenium
    By cabroncito29 in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 02-05-2015, 10:46 AM
  6. Need Help with Nested Errors and Selenium
    By BYizz in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 03-21-2014, 10:02 AM
  7. HTML scraping using VBA
    By MacroLide in forum Excel Programming / VBA / Macros
    Replies: 4
    Last Post: 05-26-2012, 11:12 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1