+ Reply to Thread
Results 1 to 27 of 27

Extract Information From Different Websites

  1. #1
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Extract Information From Different Websites

    Im trying to extract different elements from different websites using their own individual VBA macros. I'm using XMLHTTP.

    However I can't seem to make it work. I have looked into the codes as shown below and they all look so different that I dont know what to apply to extract what I want based on what I have researched so far.

    Site #1 using "View page source"
    Extract 5,061

    Please Login or Register  to view this content.
    Site #2 using "View page source"
    Extract $76,8692
    Please Login or Register  to view this content.
    This is how a part of my codes looks like

    Please Login or Register  to view this content.

  2. #2
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    Extract 5,061:
    HTML.getElementsByClassName("col-md-3 px-1")(0).getElementsByTagName("p")(1).innerText

    Extract $76,8692:
    HTML.getElementsByClassName("Text__TextBase-sc-1cait9d-0-div Text__TextContainerBase-sc-1cait9d-1 hlvKRM")(0).innerText

  3. #3
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    Hi efvba, thats a neat approach. I really like it and its simplicity. One of the codes really worked! Thank you!
    However, the other one is not working. I am getting a "Object variable or With block variable not set".

    This is not working:
    Please Login or Register  to view this content.
    Last edited by richardking; 04-15-2019 at 04:15 AM.

  4. #4
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    This is my general code for both websites. I just change the column for each macro depending on which list of websites I want it to take from.
    I have applied one of your codes here.

    Please Login or Register  to view this content.
    Last edited by richardking; 04-15-2019 at 04:16 AM.

  5. #5
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    Maybe class "Text__TextBase-sc-1cait9d-0-div Text__TextContainerBase-sc-1cait9d-1 hlvKRM" duplicate. You must know their index. First is 0, second is 1, and so on. I suppose its index is 0.

    Post your HTML or URL for a test.
    Last edited by efvba; 04-15-2019 at 09:18 PM.

  6. #6
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    Hi efvba, thank you so much. I have a followup question though.

    I want to extract $74,000 from the "View page source" code below:
    Please Login or Register  to view this content.
    Im using
    Please Login or Register  to view this content.
    and it does not seem to work. Why is that? The website requires a login but I have already placed the credentials on that website even before opening any excel file.

  7. #7
    Forum Guru Kyle123's Avatar
    Join Date
    03-10-2010
    Location
    Leeds
    MS-Off Ver
    365 Win 11
    Posts
    7,238

    Re: Extract Information From Different Websites

    You need to login through VBA if doing this method. Your web browser and VBA are unrelated

  8. #8
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    But there is also a hyperlink on the same workbook that will allow me to access the specific page on that website, and it takes me there with no problem at all.

  9. #9
    Forum Guru Kyle123's Avatar
    Join Date
    03-10-2010
    Location
    Leeds
    MS-Off Ver
    365 Win 11
    Posts
    7,238

    Re: Extract Information From Different Websites

    Hyperlinks open in your browser, they're nothing to do with the XMLHTTP60 object

  10. #10
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    I see. Good point. Can you help me with the code?

  11. #11
    Forum Guru Kyle123's Avatar
    Join Date
    03-10-2010
    Location
    Leeds
    MS-Off Ver
    365 Win 11
    Posts
    7,238

    Re: Extract Information From Different Websites

    Not without the website

  12. #12
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites


  13. #13
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    I think you didn't succeed in getting HTML you want.

    After getting the reponseText, you can check if the Class exists in HTML.

  14. #14
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    How to do that? Is it through your code?

  15. #15
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    Open the Immediate Window (Ctrl + G) in VBA. "debug.print" will output responseText to there.

    Please Login or Register  to view this content.

  16. #16
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    Hi efvba, that solution already worked. Sorry I forgot the emphasize it, but I did mention it in post#6.
    I had a follow up question though, which is to extract $74,000 from the code below.
    Please Login or Register  to view this content.
    I already tried
    Please Login or Register  to view this content.
    but it does not seem to work. The website is www.NARRPR.com and it requires a login, but I have already entered my login credentials.
    Last edited by richardking; 04-17-2019 at 05:40 AM.

  17. #17
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    Could you post the responeText output from the VBA Immediate Window?

  18. #18
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    There is none because I'm getting an "Object variable or With block variable not set" error message at
    Please Login or Register  to view this content.
    I dont know if this will help, but this is from the website:

    Please Login or Register  to view this content.
    And this is my full code:

    Please Login or Register  to view this content.
    Last edited by richardking; 04-18-2019 at 05:37 AM.

  19. #19
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    Your code is ok, so I need to view your HTML.

    Please Login or Register  to view this content.
    You just need to run the above code. "url" is that you cannot extract the data, then copy the output from the VBA Immediate Window and post it.

  20. #20
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    Hi efvba, thank you so much for spending your time in helping me out with this problem.
    The result from the responsetext from the VBA Immediate Window is on the attached notepad file. Excelforum wont let me put the result as a comment.
    Its quite lengthy too.
    Attached Files Attached Files
    Last edited by richardking; 04-19-2019 at 12:31 AM.

  21. #21
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    Your responseText is login HTML only, so you cannot get your data.

    You may try to find the class "price priceGray" that is not found in Notepad. You must get the right HTML to extract the data you want.

  22. #22
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    I actually put the URL on that VBA where it would lead me to the website that contains the information that I want, which is under the class "price priceGray". I cant put the exact URL here because i think it would lead you to the login page first before you can actually go to the desired part of that website.
    Do you think the reason why it returned the login HMTL instead of the intended HMTL is because it is asking for the login credentials first? Maybe its something that we need to get around first?

  23. #23
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    For the meantime, how to extract "$52,000" from the code in this image?

    Attachment 620915
    Last edited by richardking; 04-21-2019 at 02:46 AM.

  24. #24
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    Your attachment is not valid.

    I cannot test your URL because it needs credentials. You can try the following code to automate IE:

    Please Login or Register  to view this content.

  25. #25
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    Let me try to upload a workbook containing the image instead. I'm trying to extract "$52,000".

    The code you provided did an excellent job at logging in into the website.
    The problem is it does not go to the desired sub URL after. Also, the second time I run it, it would look for the login page again, to which the browser would normally skip it because you are already logged in.
    Attached Files Attached Files
    Last edited by richardking; 04-21-2019 at 03:03 AM.

  26. #26
    Registered User
    Join Date
    04-01-2019
    Location
    HK
    MS-Off Ver
    2010
    Posts
    32

    Re: Extract Information From Different Websites

    Try:

    Please Login or Register  to view this content.

  27. #27
    Forum Contributor
    Join Date
    01-15-2015
    Location
    philippines
    MS-Off Ver
    Excel 365
    Posts
    116

    Re: Extract Information From Different Websites

    Hi efvba, its my fault. The www.narrpr.com is a different concern from the one on the attachment. They are two different websites of which i want to extract two different information.
    i want to extract "price priceGray" from the www.narrpr.com and "$52,000" from the one on the attachment.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. VBA to extract tables from multiple websites using cells in url
    By irukae in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 10-01-2017, 07:10 AM
  2. Replies: 5
    Last Post: 06-26-2016, 12:52 AM
  3. Excel: Pulling in information from websites automatically
    By brian123321 in forum Excel General
    Replies: 3
    Last Post: 09-21-2013, 07:08 PM
  4. Information Extract
    By mfjanoo in forum Excel General
    Replies: 5
    Last Post: 04-10-2013, 04:47 AM
  5. Can excel retrieve information from websites?
    By SimonsGE in forum Excel General
    Replies: 2
    Last Post: 04-07-2013, 10:08 PM
  6. Connecting to a Websites Information
    By e4excel in forum Excel Programming / VBA / Macros
    Replies: 11
    Last Post: 01-09-2011, 07:15 AM
  7. Replies: 2
    Last Post: 10-23-2008, 11:19 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1