+ Reply to Thread
Results 1 to 22 of 22

Loop dropdown selections on web page

  1. #1
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Loop dropdown selections on web page

    Hi All...

    I have no experience at web scraping

    What snippet of code am I to make use of to access each drop down selection on this site...
    I am wanting to extract each email...

    I know the basic code but require the element to access..
    https://www.dros.co.za/stores.html

    Elements Inspection reveals...

    Untitled.png

    Any ideas...thanks
    Good Luck
    I don't presume to know what I am doing, however, just like you, I too started somewhere...
    One-day, One-problem at a time!!!
    If you feel I have helped, please click on the star to left of post [Add Reputation]
    Also....add a comment if you like!!!!
    And remember...Mark Thread as Solved.
    Excel Forum Rocks!!!

  2. #2
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    if I'm not mistaken, as you have already noticed, the select element in HTML is the dropdown feature. and the option attribute inside of that is what sets the values inside the dropdown. however, you should be aware that anti-hacking and anit-ANYTHING tactics are constantly being developed, strictly for the purpose of allowing corporate websites to remain as private as possible and keeping the money flowing into *them* only, so other people don't steal anything and try to compete with them. and it's been that way for at least 10 years now.

    you also need to know, if you don't, as I have said in another thread already, that there *are* actual policies against scraping sites that are indicated by companies. so doing it on some sites is actually illegal. however, if you don't care as most don't, you would probably never get caught because they are too busy making money to care, even if they do see your activity in the transaction logs.

    in the source of the webpage you have shown, click the down arrow next to the select tag and see if the word option appears in there multiple times.

  3. #3
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Re: Loop dropdown selections on web page

    as you have already noticed, the select element in HTML is the dropdown feature.
    Yep...Now need the code syntax...hence my post...

  4. #4
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    didn't I already give you that? what do you mean the code syntax?

  5. #5
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Re: Loop dropdown selections on web page

    Code to access the elements to extract the email addresses...

    Anyway...This works...Although I know it can be accessed directly...
    Please Login or Register  to view this content.

  6. #6
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,830

    Re: Loop dropdown selections on web page

    @vba_php

    You have not provided any VBA code as far as I can see. Sintek is asking for some code that will help him extract the multiple E-mail addresses. Telling him where to look in the HTML is not what he was hoping for.

    As there do not appear to be any T&Cs on that website forbidding the scraping of data (correct me if I'm wrong), I don't believe there is a problem. The website owner would need to explicitly state that to be the case if they wanted to 'prevent' someone from thinking about doing it.
    Ali


    Enthusiastic self-taught user of MS Excel who's always learning!
    Don't forget to say "thank you" in your thread to anyone who has offered you help.
    You can reward them by clicking on * Add Reputation below their user name on the left, if you wish.

    Forum Rules (updated August 2023): please read them here.

  7. #7
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    well then I must have been confused. I wasn't trying to be unhelpful or a jerk.

    hey sintek,

    your code does work just fine. however, innerText is an attribute that is not always available. and it doesn't apply to all elements. just FYI I guess. not to mention the fact that you are looping through literally everything. you could probably greatly shorten that by using the javascript statement: getElementsByTagName(), or even getElementById(). do you know javascript and DOM by chance?
    Last edited by AliGW; 08-28-2020 at 06:27 AM. Reason: Please don't quote unnecessarily!

  8. #8
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,830

    Re: Loop dropdown selections on web page

    I wasn't trying to be unhelpful or a jerk.
    I didn't say you were.

  9. #9
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Re: Loop dropdown selections on web page

    Adam...

    you could probably greatly shorten that by using the javascript statement: getElementsByTagName(), or even getElementById()
    There is specific VBA syntax for this...That is why I posted...To get a solution...
    There are members who are experts at web scraping...I am not...

    Gonna leave this thread unsolved so as to get the required code to extract the emails correctly...

  10. #10
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    Quote Originally Posted by sintek View Post
    Adam..Gonna leave this thread unsolved so as to get the required code to extract the emails correctly...
    well I'm not expert either! I don't do much of it. but I CAN get you in touch with a seriously smart professor in South America who does this all day long, and teaches it too!

    so....are you asking for the code, properly, to do this? you said you got the solution already. are you asking for a better one?

  11. #11
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Re: Loop dropdown selections on web page

    are you asking for a better one?
    Yep, I am asking for the code that loops the drop down selections and extracts the emails from each store...
    By using the ElementById or ElementsByName or ElementsBytagName Features...

  12. #12
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    Quote Originally Posted by sintek View Post
    Yep, I am asking for the code that loops the drop down selections and extracts the emails from each store...
    By using the ElementById or ElementsByName or ElementsBytagName Features...
    it doesn't look like it is traditionally written code, my friend. the email addresses are being manipulated by a javascript function. see below images. they are not technically written in the dropdown as attributes, as it seems. your way might be the only one to get what you need....
    Attached Images Attached Images

  13. #13
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    on a second go-through, it seems that the function is being called when a selection is made. do you want me to run that experiment and see what happens? I didn't do that. did you?

  14. #14
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,830

    Re: Loop dropdown selections on web page

    I think that this is something that you could probably do with PowerQuery. If you are interested, I'll put out a call to our resident PQ wizard, Olly - I have seen solutions offered by him to web scraping queries in the past, and he may well know of a quick way to get the data you want.

  15. #15
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Re: Loop dropdown selections on web page

    Power Query not a option... Kyle123 is our Forum Web scraping Guru with VBA...Have sent a msg...Tx AliGw

    This code extracts from Spur Steak Ranches...
    Please Login or Register  to view this content.
    There must be a general code scenario for websites...Any body have any ideas or links where one could learn...
    Last edited by sintek; 08-28-2020 at 12:59 PM.

  16. #16
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    Quote Originally Posted by sintek View Post
    There must be a general code scenario for websites...Any body have any ideas or links where one could learn...
    what are you looking for, sintek? what do you mean by a general scenario? are you talking about programming standards for building websites?

  17. #17
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Re: Loop dropdown selections on web page

    Nope...Knowing by inspecting the HTML elements what elements to make use of in ones VBA code...

    https://www.wiseowl.co.uk/blog/s393/...bsite-html.htm

  18. #18
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    if you're talking about not needing to go through 55 of these:
    Please Login or Register  to view this content.
    I don't think there's a way to shorten the process, because the query giving the results can't be seen by a browser. furthermore, it looks like pagination code is being either echoed out after the search of is static code on the page anyway. if you wanted to get really crazy, maybe you could try injecting an extra query string argument into the URL and see what it gets you.

    and there's always sql injection too, as a way to dump all the DB's information, but I'm sure you're not a criminal, so best to refrain from that! =)

  19. #19
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Re: Loop dropdown selections on web page

    Greek to me Adam...I skipped the basics lol...

  20. #20
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    Quote Originally Posted by sintek View Post
    Greek to me Adam...I skipped the basics lol...
    hmmmm.....well sorry to make you feel bad. so then....have you got this solved or are you waiting on a response from the VBA expert you mentioned a while back?

  21. #21
    Forum Expert sintek's Avatar
    Join Date
    12-04-2015
    Location
    Cape Town
    MS-Off Ver
    2013 | 2016 | 2019
    Posts
    13,236

    Re: Loop dropdown selections on web page

    Gonna chalk this one up as solved...And carry on tinkering...Different sites for email extractions...

  22. #22
    Banned User!
    Join Date
    02-06-2020
    Location
    Iowa City, IA, USA
    MS-Off Ver
    2016 - 365 / 2007
    Posts
    2,014

    Re: Loop dropdown selections on web page

    Quote Originally Posted by sintek View Post
    Greek to me Adam...I skipped the basics lol...
    I know this is solved, but if I were you I would learn Russian instead of greek. That way, you would understand their level of technical competence. LOL.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] Dropdown box cascading selections
    By Roadrun030 in forum Excel General
    Replies: 2
    Last Post: 10-01-2019, 09:58 AM
  2. Merging dropdown selections into 1 cell
    By paul_lowry in forum Excel Formulas & Functions
    Replies: 4
    Last Post: 05-20-2015, 10:48 AM
  3. Dropdown selections based on adjacent dropdown selected
    By mtvufg8r in forum Excel General
    Replies: 2
    Last Post: 11-07-2014, 12:25 PM
  4. Zooming Dropdown List Selections
    By enagli in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 02-03-2014, 05:37 PM
  5. Dropdown list selections
    By Hustad in forum Excel - New Users/Basics
    Replies: 5
    Last Post: 10-23-2009, 03:18 PM
  6. Tallying dropdown selections
    By curtjer in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 03-27-2009, 01:56 PM
  7. Help with dropdown boxes and preset selections
    By Keving in forum Excel Programming / VBA / Macros
    Replies: 2
    Last Post: 07-07-2006, 01:15 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1