Web Scraping

So what exactly is happening? Are you able to capture any element value on that web page? Because you are using the get element by ID method from the document object model as a sub method of get elements by class name. I've never seen that done in my 20 years of doing this type of work.

What do you mean that you need help getting the first line from the web page? And are you sure the element that you are targeting or that you are trying to capture actually has an ID attribute and that it's actually classified as the type of class you are pointing to?

The code I included in my post captures a 10 day forecast for Miami, FL, even though the forecast has 14 days on the site. The code no longer gets the first day (today's forecast) but captures all other 13 days in the forecast. Maybe the ID attribute changed for today's forecast?!

Well based on the code that you wrote and the fact that you're looking at every element in a loop, I would seriously doubt that the first element that's supposed to give you what you're looking for should not work. However, have you looked at the web pages source code to make sure that the code you're looking at is actually accurate?if the ID attribute of the element that you were missing did indeed change, you would certainly see it by looking at the source code of the web page.

The other thing you should know is that a lot of sophisticated websites or websites that are run by technology companies that have been around forever, a lot of that crap is spit out by servers on an automated basis and people aren't even writing code anymore. So given that reality, who knows what goes on inside those companies.

Furthermore, a little off topic here but,there's also a boatload of things that you can do on a web page in terms of using HTML elements to display content in a legitimate manner and an illegitimate manner. you can actually write an entire web pages source code and hide it behind the page that's actually visible. But doing so will get you blacklisted by search engines.

This alternative fetches day & night temperatures and the summary;

Please Login or Register  to view this content.

Haluk,

I might be wrong about this but the URL that you included in that code of yours looks like you ran a search and the web app produced a hash output assigned to that search temporarily. and if that's the case that solution is not going to work for the question ask her here because that hash is going to be outdated very soon. And therefore it'll result in a page server error.

As far as I know almost every web application in the world, when you run a search by clicking a search button produces some sort of identification hash like the one you just posted. And it's never good forever it expires at some point.

I just used the URL which was given in the original post, but you may probably correct.

If that's the case, the related hash should be fetched and then the request should be sent.

There's no way to fetch a hash because it's generated dynamically by the server when the search button is pressed

Haluk, thanks for the help. I can definitely work with this. Take care!

Originally Posted by CK76

Nah, this one is place ID held on backend side from the looks of it. Not dynamically generated hash. You can trace it using Developer tool in the browser.
Attachment 697293

wow that's actually pretty useful. thanks much. a lot of what I've seen is generally dynamically and assigned timeouts. but apparently not in this case. nice.

Sorry, but is there a way of getting a code that fetches the data faster? I made the code work, but it takes a long time to run the code for multiple cities in my sheet.

Gomez,

If you have to run separate URLs and separate hashes for each City then obviously the code cannot run faster because every hash and every URL has to go through its own rendering process so essentially you're doing it as fast as you possibly can. That's my view on it anyway. The only chance you've got to make it faster is to use a more sophisticated language that has been a resources and VBA and that's pretty much any language in the world. Just pick one. Python, c sharp, c++,, anything.