+ Reply to Thread
Results 1 to 18 of 18

Get HTTP MetaData into Excel

  1. #1
    Registered User
    Join Date
    08-26-2010
    Location
    London, UK
    MS-Off Ver
    Excel 2010
    Posts
    8

    Cool Get HTTP MetaData into Excel

    Hi Everyone

    Can anyone offer a way to do the following:-

    Get HTTP MetaData into Excel
    1. Take list of URLS
    2. Enter to XLS Sheet
    3. Have XLS (VBA Function) look up each URL
    4. Pull down Meta Title & Meta Description of each URL
    5. Display Result in table
    I have found many resources which do some, but not all of these reqs. Also "Get External Data" is not sufficient as it only gathers on page data sources, and I am looking to import the Meta Data from a list web pages into Excel.

    As a newbie to XLS - This is my first post to the forum so thanks in advance for your help.
    • There must be a XLS Champion out there who knows how to do this?


    Cheers

    XLS_GUNDA
    Last edited by XLS_Gunda; 08-26-2010 at 10:02 AM.

  2. #2
    Registered User
    Join Date
    08-26-2010
    Location
    London, UK
    MS-Off Ver
    Excel 2010
    Posts
    8

    Re: How to Get HTTP MetaData into Excel ?

    Just to say, I know there are lots of programmes that can solve this question, such as http://www.link-assistant.com but what i am looking for is the soltuion in XLS format....

    Any gurus out there?

  3. #3
    Forum Contributor
    Join Date
    07-13-2007
    Posts
    151

    Re: How to Get HTTP MetaData into Excel ?

    I threw together a quick UDF that you could use to insert the Meta Description into a Excel Spreadsheet cell. For exmple if the URL is in A1, put this formula in B1 after you've inserted the code below into a standard module in the workbook, and created a reference to the HTML library for the project.

    Please Login or Register  to view this content.
    Please Login or Register  to view this content.

  4. #4
    Registered User
    Join Date
    08-26-2010
    Location
    London, UK
    MS-Off Ver
    Excel 2010
    Posts
    8

    Angry Re: How to Get HTTP MetaData into Excel ?

    Hi Shred Dude

    Have done this below
    Quote Originally Posted by ShredDude View Post
    put this formula in B1 after you've inserted the code below into a standard module in the workbook,
    What is this? "and created a reference to the HTML library for the project."


    After I state the getMetaDescription() there is an error in debug at
    Please Login or Register  to view this content.

    Thanks for offering this up, I'm new to Excel/VBA, and have kinda caught the bug!

    BTW, have Office 2010

    Cheers

    XLS_Gunda

  5. #5
    Forum Contributor
    Join Date
    07-13-2007
    Posts
    151

    Re: How to Get HTTP MetaData into Excel ?

    In the Visual Basic Editor (VBE), go tot he Tools Menu, then References. Scroll down in the list that appears looking for Microsoft HTMLObject Library. Check that box and then click OK. That establishes a Reference to the HTML object library.

    Your profile shows you're using Excel 2003, you might want to update that if you're really using 2010.

  6. #6
    Registered User
    Join Date
    08-26-2010
    Location
    London, UK
    MS-Off Ver
    Excel 2010
    Posts
    8

    Re: How to Get HTTP MetaData into Excel ?

    Hi

    Have done as you suggest, and attached the XLS here....

    Have i got it all set up correctly, as you say, because it is not returning the Meta as thought to do...

    Feels like "we" are almost there....


    thoughts?
    Attached Files Attached Files

  7. #7
    Forum Contributor
    Join Date
    07-13-2007
    Posts
    151

    Re: How to Get HTTP MetaData into Excel ?

    The file you posted contains no macro code. If it did it would have the .xlsm file extension.

    Attached is a version containing the code and the appropriate reference.

    There is no error checking in the UDF so if a page has no Meta Description tag within it's source, you'll get a #VALUE returned in the spreadsheet. Add whatever error checking you feel is appropriate for you needs.
    Attached Files Attached Files

  8. #8
    Forum Contributor
    Join Date
    07-13-2007
    Posts
    151

    Re: How to Get HTTP MetaData into Excel ?

    Looks like the createDocumentFromUrl likes to have "http://" at the front of the URL.

    Added a line of code to handle that...

    Please Login or Register  to view this content.

  9. #9
    Registered User
    Join Date
    08-26-2010
    Location
    London, UK
    MS-Off Ver
    Excel 2010
    Posts
    8

    Thumbs up SOLVED Re: How to Get HTTP MetaData into Excel ?

    ShredDude, thanks a tonne for the interaction and solution. This is a tremendous help in my day to day work. Thank you very much.
    As a rider question, will this code (which is based on your code) work for the "title" - note i have dropped the "Item" because the title looks like this

    Please Login or Register  to view this content.

    Please Login or Register  to view this content.
    Last edited by XLS_Gunda; 08-31-2010 at 11:56 AM.

  10. #10
    Forum Contributor
    Join Date
    07-13-2007
    Posts
    151

    Re: How to Get HTTP MetaData into Excel ?

    No, I don't think that'll do it. getelementsbytagname returns an IHTMLElementCollection. You'd need to then reference a particular element within that Collection.

    If you're certain there'd only be one tag with the name "Title", you might try something like:

    Please Login or Register  to view this content.

    Or, abandon the HTML Document method and do an xmlhttp request then string parse the responsetext for the Title Tag's contents.

    Please Login or Register  to view this content.
    I didn't use this method on the Description because of the way the Description Tags were formatted. I found it easier to use the DOM to extract the "Content" of a node.

  11. #11
    Registered User
    Join Date
    08-26-2010
    Location
    London, UK
    MS-Off Ver
    Excel 2010
    Posts
    8

    Re: How to Get HTTP MetaData into Excel ?

    Cheers Sir - you are a true gent....... - both those methods worked. I went with the
    Please Login or Register  to view this content.
    method.

    When I started out thinking about this task I did think about the msxml2.xmlhttp example as i had been shown an example of it a few years back.

    The final part of the jigsaw for me is to try to
    1. Go to the site, open up source code using your method
    2. Check for a string in the markup (a tracking reference <a ONCLICK="tc_log('/Downloads/biog');" href="http://www.mysite.com">And St Johnstone FC rule!</a>')
    3. Output the string, or a relative 1 or 0 to show if it is on the page or not?

    Please Login or Register  to view this content.
    The script above is what i was trying to hack up - sadly to my shame no avail!!!!!


    If either of the scripts up above this could you gimme a heads up on how to reuse them for that purpose?

    Notwithstanding, thanks for all your help over the past week. Most pleased.
    Last edited by XLS_Gunda; 09-01-2010 at 12:35 PM.

  12. #12
    Forum Contributor
    Join Date
    07-13-2007
    Posts
    151

    Re: Get HTTP MetaData into Excel

    The Item property is looking for the NAME attribute of the element. That's why it worked on the Meta Tag because within the Meta Tags each one was named, and there was one named "Description".

    OnClick is not a Name, rather a property of an Anchor element in this case.

    So, if it's the OnClick property that is your unique identifier you want to serach on, I'd suggest grabbing all the anchor elements into a collection and then looping through them until you find the one you want.

    Try something like this:

    Please Login or Register  to view this content.

    That is if you want to use the HTML method. Depending on the site, this could be slow if there are many Anchor Elements to search through. You may want to consider using the XMLHTTP approach, and making yourself a little parser routine to grab the href that follows the OnClick you're looking for.

  13. #13
    Registered User
    Join Date
    08-26-2010
    Location
    London, UK
    MS-Off Ver
    Excel 2010
    Posts
    8

    Smile Re: Get HTTP MetaData into Excel

    Shred....Promise this is the last pass!

    So, have tried the function getTracking() method, sadly to no avail.

    Also had a look at entering other anchors/properties to check that too....

    Tomorrow (Friday) is my last day on this blasted two-week contract (recession = ), this help you give will help me leave a fellow Scot with a bit of code that makes his impossible job infinitely easier....as for the nazi boss...she be none the wiser either way!

    Your tutorials have been brilliant, i have learned alot about XLS. V Many thanks.

    I've attached the XLS....... in for a penny and all that!
    Attached Files Attached Files

  14. #14
    Forum Contributor
    Join Date
    07-13-2007
    Posts
    151

    Re: Get HTTP MetaData into Excel

    VBA doesn't like it when you name a module the same as a function. That was the cause of the #NAME error.

    After that, I fixed the code to find the instance where you have just the part you're interested in in the OnClick property.

    So, how much money am I helping you make?
    Attached Files Attached Files

  15. #15
    Registered User
    Join Date
    08-26-2010
    Location
    London, UK
    MS-Off Ver
    Excel 2010
    Posts
    8

    Smile Re: Get HTTP MetaData into Excel

    Quote Originally Posted by ShredDude View Post
    So, how much money am I helping you make?
    The contract is paying well, but it is not about the Excel functions.

    It was more for my own interest that I started trying to work out how to do these functions (and failing miserably!).

    To be honest with you mate, this contract has been so so so sh*t that had it not been for these excel functions and trying to solve a problem that no one asked to be solved, while learning and interacting with the likes of yourself.....well it would have been even more sh*t.

    PM your PayPal details and I'll buy you a few beers....as a personal thank you.

  16. #16
    Registered User
    Join Date
    07-31-2012
    Location
    Missouri, USA
    MS-Off Ver
    Excel 2010
    Posts
    2

    Re: Get HTTP MetaData into Excel

    I know that this thread is ancient but I can't get this to work for me. I've built a Browser History processor for my job (which seems to be mostly checking up on employee browing habits this week) and going through history logs for IE, Firefox, and Chrome is a pain.

    My spreadsheet takes data from 4 sheets of data (IE, Chrome, Firefox, and Safari - added it because I have a history viewer for it) then weeds out any allowed sites (work or research related) then spits out the data on a final sheet that also is color highlighted for breaks and lunch browsing. Leaving the manager the end data to look over for possible discipline.

    I was using some code that used IE to get the title but if the page pops a dialog warning (or a download dialog) it causes me to have to use interaction.

    I plugged this code into my macro (not using it as a formula based approach as I have thousands of URLs to process) and I get a method "open" of the object "IXMLHTTP" failed.

    Any ideas?

  17. #17
    Registered User
    Join Date
    10-30-2012
    Location
    India
    MS-Off Ver
    Excel 2003
    Posts
    92

    Re: Get HTTP MetaData into Excel

    When i run the code i am getting an error
    "Object Variable or With Block Variable not set"

    Can you please resolve this!!!

    Please Login or Register  to view this content.

  18. #18
    Forum Guru
    Join Date
    03-02-2006
    Location
    Los Angeles, Ca
    MS-Off Ver
    WinXP/MSO2007;Win10/MSO2016
    Posts
    12,612

    Re: Get HTTP MetaData into Excel

    kiran527

    Unfortunately your post does not comply with Rule 2 of our Forum RULES. Do not post a question in the thread of another member -- start your own thread.

    If you feel an existing thread is particularly relevant to your need, provide a link to the other thread in your new thread.

    Old threads are often only monitored by the original participants. New threads not only open you up to all possible participants again, they typically get faster response, too.
    Ben Van Johnson

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1