Results 1 to 5 of 5

Scraping data from website

Threaded View

  1. #1
    Registered User
    Join Date
    12-30-2015
    Location
    florida
    MS-Off Ver
    2010
    Posts
    3

    Scraping data from website

    I am very new to vba programming and am still learning the rules about the language. I am trying to write a macro that scrapes data from a website. The problem that i am having right now is that not all of the data is being scraped. This macro will be run on a daily basis, so the amount of data to be scraped will be different. Basically, for the data that is to be scraped today, there are 7 pages of data, but the macro that i have right now will only scrape until page 6. I cannot seem to figure out why it will only scrape until page 6 and not page 7. When i run this macro for the data from 2 days, it pulls in and scrapes all of the data and there were about 8 pages of data to scrape from 2 days ago. This is the macro that I have so far. Any help is greatly appreciated.

    
    'Macro to query Delinquency Status Search for DFB Counties
    'Run Monday to pull data from Friday
    
    Sub queryActivityDailyMforFWorking()
    
    Dim nextrow As Integer, i As Long
    Dim dates
    dates = Date - 1
    
    i = 1
    
    Application.ScreenUpdating = False
    Application.DisplayStatusBar = True
    
    Do
      
        'i = i + 1
        Application.StatusBar = "Processing Page " & i
        nextrow = ActiveSheet.Cells(Rows.Count, "A").End(xlUp).Row + 1
        'lastRow = ActiveSheet.Cells(Rows.Count, "A").End(xlToLeft).Column + 1
    
            'With ActiveSheet.QueryTables.Add(Connection:= _
            '"URL;https://www.myfloridalicense.com/delinquency_results.asp?SID=&page=" & i + county + x & "&status=NS&send_date=" & dates & "&search_1.x=1", _
            'Destination:=Range("A" & nextrow))
            
            With ActiveSheet.QueryTables.Add(Connection:= _
            "URL;https://www.myfloridalicense.com/delinquency_results.asp?SID=&page=" & i & "&county_1=11,%2012,%2013,%2014,%2015,%2016,%2017,%2018,%2019,%2020,%2021,%2022,%2023,%2024,%2025,%2026,%2027,%2028,%2080,%2029,%2030,%2031,%2032,%2033,%2034,%2035,%2036,%2037,%2038,%2039,%2040,%2041,%2042,%2043,%2044,%2045,%2046,%2047,%2048,%2049,%2050,%2051,%2052,%2053,%2054,%2055,%2056,%2057,%2058,%2059,%2079,%2060,%2061,%2062,%2063,%2064,%2067,%2068,%2069,%2065,%2066,%2070,%2071,%2072,%2073,%2078,%2074,%2075,%2076,%2077&status=NS&send_date=" & dates & "&search_1.x=1", _
            Destination:=Range("A" & nextrow))
            
      
    
            
            
            '.Name = _
            "2015&search_1.x=40&search_1.y=11&date=on&county_1=AL&lic_num_del=&lic_num_rep=&status=NS&biz_name=&owner_name="
            .FieldNames = False
            .RowNumbers = False
            .FillAdjacentFormulas = False
            .PreserveFormatting = True
            .RefreshOnFileOpen = False
            .BackgroundQuery = True
            .RefreshStyle = xlInsertDeleteCells
            .SavePassword = False
            .SaveData = True
            .AdjustColumnWidth = True
            .RefreshPeriod = 0
            .WebSelectionType = xlSpecifiedTables
            .WebFormatting = xlWebFormattingNone
            .WebTables = "10"
            .WebPreFormattedTextToColumns = True
            .WebConsecutiveDelimitersAsOne = True
            .WebSingleBlockTextImport = False
            .WebDisableDateRecognition = False
            .WebDisableRedirections = False
            .Refresh BackgroundQuery:=False
    
      
        
    
        'autofit columns
        Columns("A:G").Select
        Selection.EntireColumn.AutoFit
    
       'check for filter, if not then turn on filter
       ActiveSheet.AutoFilterMode = False
        If Not ActiveSheet.AutoFilterMode Then
        ActiveSheet.Range("A:G").AutoFilter
        End If
    i = i + 1
    End With
    
    ActiveCell.value = ActiveCell.Value * 2
    ActiveCell.Offset(1,0).Select
    
    Loop Until IsEmpty(ActiveCell.Value)
    
    Application.StatusBar = False
    
    'Align text left
    Cells.Select
    With Selection
        .HorizontalAlignment = xlLeft
        .VerticalAlignment = xlBottom
        .WrapText = False
        .Orientation = 0
        .AddIndent = False
        .IndentLevel = 0
        .ShrinkToFit = False
        .ReadingOrder = xlContext
        .MergeCells = False
    End With
    
    
    
    
    
    End Sub
    Last edited by novicevba; 12-30-2015 at 10:58 AM.

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Web Scraping Data From A Website Where Hyperlink Does Not Change
    By NeedForExcel in forum Excel Programming / VBA / Macros
    Replies: 4
    Last Post: 10-01-2015, 02:27 AM
  2. Data Scraping from the WebSite
    By sathis in forum Excel Programming / VBA / Macros
    Replies: 11
    Last Post: 07-26-2015, 10:46 AM
  3. problem while scraping data from website
    By dps700 in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 12-09-2014, 09:46 AM
  4. Automated data scraping from website into excel
    By kavin in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 09-10-2014, 01:42 PM
  5. Replies: 1
    Last Post: 02-17-2014, 11:21 PM
  6. scraping data from a website
    By redpanda in forum Excel Programming / VBA / Macros
    Replies: 9
    Last Post: 07-05-2012, 01:20 PM
  7. Scraping website data into Excel
    By oliver79 in forum Excel General
    Replies: 0
    Last Post: 06-13-2010, 02:15 PM

Tags for this Thread

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1