+ Reply to Thread
Results 1 to 9 of 9

Multiple web queries and reducing the time it takes

  1. #1
    Registered User
    Join Date
    02-02-2010
    Location
    playa del carmen, mexico
    MS-Off Ver
    ms office 2007
    Posts
    52

    Multiple web queries and reducing the time it takes

    Hi

    I have been coding an excel sheet which will simplify the management and data entry for a wiki site.
    The front page contains various sections of 8 items which i plan to change on a semi-regular basis.
    Things like a random list of 8 pages, easily coded.
    Newest entries, same again.
    And such forth.

    I have also got a list which is for the 8 most visited pages, of which the only way I have been able to find the information is on the bottom of each page itself, which states:-

    This page has been accessed **** times.

    Now.. I have set up a sheet which calculates the URL based on the name of the worksheet it is looking for, runs a web query for that URL, imports the data to the front page management sheet, searches for the line which states "this page....." etc (which is in a different position on each page, depending on what information is on the page), extracts the number, and inserts it into the appropriate cell.

    In other words, it works.

    However, I only have maybe 50 pages in my wiki right now, and with time this will stretch into the 1000's and up, so obviously, this method is a little slow right now, with having to do 50 separate web queries just to find one piece of information (takes maybe 5 mins - which right now as I only plan on updating once a week), but once I am into the 1000's of web queries this will take a long time, so I was wondering if anyone could help with any suggestions, tips, or useful tidbits of information to pre-empt my future hassles.

    The coding I use thus far is as follows:

    Please Login or Register  to view this content.
    Thanks

  2. #2
    Forum Moderator Leith Ross's Avatar
    Join Date
    01-15-2005
    Location
    San Francisco, Ca
    MS-Off Ver
    2000, 2003, & 2010
    Posts
    23,258

    Re: Multiple web queries and reducing the time it takes

    Hello globalpontoon,

    This data could be retrieved using Internet Explorer. To write the code, I would need to know the URL you are using.
    Sincerely,
    Leith Ross

    Remember To Do the Following....

    1. Use code tags. Place [CODE] before the first line of code and [/CODE] after the last line of code.
    2. Thank those who have helped you by clicking the Star below the post.
    3. Please mark your post [SOLVED] if it has been answered satisfactorily.


    Old Scottish Proverb...
    Luathaid gu deanamh maille! (Rushing causes delays!)

  3. #3
    Registered User
    Join Date
    02-02-2010
    Location
    playa del carmen, mexico
    MS-Off Ver
    ms office 2007
    Posts
    52

    Re: Multiple web queries and reducing the time it takes

    And therein lies the reason for the previous vagueness.

    Whilst the content of my wiki is not illegal, immoral or contains any form of nudity, it is an adult content guide and therefore I would assume it is not wise for me to include the specific links on this site.

    If there is a way in which you would be able to help me without the link, or at least point me in the direction of something that would be able to.

    As I previously mentioned, I have been able to retrieve the information, but obviously there is a practically unlimited scope as to how many pages I may have, and running a web query for each page (even if it is only necessary once every two weeks to update the 'most viewed pages' list) will potentially take a long time

  4. #4
    Registered User
    Join Date
    02-02-2010
    Location
    playa del carmen, mexico
    MS-Off Ver
    ms office 2007
    Posts
    52

    Re: Multiple web queries and reducing the time it takes

    As an addition I am becoming reasonably competent with most other forms of VBA, so if i knew the path to look down, I could code something myself that would work. It's just my first dabble into web queries, and have not found any decent information pages on the processes and syntax involved either.

  5. #5
    Registered User
    Join Date
    02-02-2010
    Location
    playa del carmen, mexico
    MS-Off Ver
    ms office 2007
    Posts
    52

    Re: Multiple web queries and reducing the time it takes

    And as one additional point that may help.

    I use the same wiki-creation software as wikipedia, so if you can get the information off of a wikipedia page (and have it so that the only needed to change would be the URL to be able to get the same information from a different wikipedia page) then that would work too.

  6. #6
    Registered User
    Join Date
    02-02-2010
    Location
    playa del carmen, mexico
    MS-Off Ver
    ms office 2007
    Posts
    52

    Re: Multiple web queries and reducing the time it takes

    And sorry for adding another post to this myself - i should really have considered the whole response before typing.

    But on wikipedia, they have removed the "This page has been accessed *** times" part of the footer, but in the the wiki-software it is a standard below the "This page was last modified on **/**/****" so if you could get that information from any (and potentially all) pages on wikipedia I am sure it would be a direct alteration to get the desired information i need.

  7. #7
    Forum Moderator Leith Ross's Avatar
    Join Date
    01-15-2005
    Location
    San Francisco, Ca
    MS-Off Ver
    2000, 2003, & 2010
    Posts
    23,258

    Re: Multiple web queries and reducing the time it takes

    Hello globalpontoon,

    You will need to program Internet Explorer to do this for you. A Web Query is not sufficient for your needs. To do so you will need to be familiar with HTML and the DOM (Document Object Model) used to control web browsers. This is really outside the realm of classic VBA. The documentation available on this subject is scarce and what you will find is not very detailed. Sealbh math dhuit! (Good luck!)

  8. #8
    Registered User
    Join Date
    02-02-2010
    Location
    playa del carmen, mexico
    MS-Off Ver
    ms office 2007
    Posts
    52

    Re: Multiple web queries and reducing the time it takes

    In that sense, is it possible to do the same thing with firefox instead - it is my browser of choice.

  9. #9
    Forum Moderator Leith Ross's Avatar
    Join Date
    01-15-2005
    Location
    San Francisco, Ca
    MS-Off Ver
    2000, 2003, & 2010
    Posts
    23,258

    Re: Multiple web queries and reducing the time it takes

    Hello globalpontoon,

    You could use FireFox. You would then have to learn XML and the FireFox API calls.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1