+ Reply to Thread
Results 1 to 21 of 21

What if SQL query return more records than excel sheet row limit?

  1. #1
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    What if SQL query return more records than excel sheet row limit?

    Below code works fine, but my question is: how should I update my code for the case of more than 1048570? It has never happened with more than 1048570, but theoretically the maximum number of record could be almost 4 millions (very very low possibility).

    Question: I only download 5 columns(Group_number, ID, FirstName, LastName, Score) of data, if the number of record > 1048570 and <=2,000,000, then download first million record in columns 1-5, and download remaining record in columns 6-11; if more than 2 millions, .... then download first millions to columns 1-5, download 2nd million to columns 6-11, and so on. .....

    Is there a way to update the code to download all records if it is indeed more than 1048570? In whatever way.

    The website prevents SQL code, so I add some # in the code.
    Admin's note: Our system does screen for SQL injection attacks, and is overly aggressive in doing so. A workaround is to pick a letter in the word and set the color to black. It will still look the same, but the extra color tags will change the string so it will not be screened. I've done that for you here for readability. --6StringJazzer

    Post on another site: https://www.reddit.com/r/vba/comment...xcel_sheet_if/

    Please Login or Register  to view this content.
    Last edited by VAer; 03-21-2024 at 09:14 PM.

  2. #2
    Forum Expert
    Join Date
    10-06-2017
    Location
    drevni ruchadlo
    MS-Off Ver
    old
    Posts
    2,151

    Re: What if SQL query return more records than excel sheet row limit?

    You can use "GetRows" instead of "CopyFromRecordset", and then for example you can divide the acquired data into individual sheets in the workbook.

  3. #3
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by mjr veverka View Post
    You can use "GetRows" instead of "CopyFromRecordset", and then for example you can divide the acquired data into individual sheets in the workbook.
    How to write the code? For example, GetRows from row 3 to 7.

  4. #4
    Forum Expert
    Join Date
    10-06-2017
    Location
    drevni ruchadlo
    MS-Off Ver
    old
    Posts
    2,151

    Re: What if SQL query return more records than excel sheet row limit?

    What is about with this "Where 3 to 7" ?
    Is it about the array of "GetRows" or about database ?

  5. #5
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by mjr veverka View Post
    What is about with this "Where 3 to 7" ?
    Is it about the array of "GetRows" or about database ?
    MS SQL database, or Sybase database.

    My code is just about SQL query, not array.

    We have both MS SQL and Sybase, but the code should be same, I guess.
    Last edited by VAer; 03-21-2024 at 08:44 PM.

  6. #6
    Forum Expert rorya's Avatar
    Join Date
    08-13-2008
    Location
    East Sussex, UK
    MS-Off Ver
    365 Ent Monthly Channel / Insiders Beta
    Posts
    8,913

    Re: What if SQL query return more records than excel sheet row limit?

    You can just loop:

    Please Login or Register  to view this content.
    Rory

  7. #7
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by rorya View Post
    You can just loop:

    Please Login or Register  to view this content.
    Thank you very much!

  8. #8
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by rorya View Post
    You can just loop:

    Please Login or Register  to view this content.
    Thank you, your code works. I just tested your code with a smaller number than 1000000 (since I have never encountered more than one million records for the query yet).

    Question 1: Does the 5 after 1000000 mean I download 5 columns/fields of data? If I want a blank column in between output data, then I should use outputcell = outputcell.offset(, 6) , correct?

    Question 2: Could you please further modify your code to include field name? Currently, WB.Sheets("Download Sheet").Cells(1, 1).Offset(, j) = RecorddownloadField.Name only applies to first set of output data, how can I get this part of code into loop?

    Thanks.
    Last edited by VAer; 03-22-2024 at 11:28 AM.

  9. #9
    Forum Expert rorya's Avatar
    Join Date
    08-13-2008
    Location
    East Sussex, UK
    MS-Off Ver
    365 Ent Monthly Channel / Insiders Beta
    Posts
    8,913

    Re: What if SQL query return more records than excel sheet row limit?

    #1: Yes
    #2: Just copy the header range from the first 5 columns each time.

  10. #10
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by rorya View Post
    #2: Just copy the header range from the first 5 columns each time.
    You are right, my brain is short circuit.

    Thank you.

  11. #11
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by rorya View Post
    #1: Yes
    #2: Just copy the header range from the first 5 columns each time.
    Please Login or Register  to view this content.
    Sorry for bothering again, not familiar with Offset, what is wrong with the code? It only copies first field name, not the other four field names.

  12. #12
    Forum Expert rorya's Avatar
    Join Date
    08-13-2008
    Location
    East Sussex, UK
    MS-Off Ver
    365 Ent Monthly Channel / Insiders Beta
    Posts
    8,913

    Re: What if SQL query return more records than excel sheet row limit?

    You copied A1:A5 not A1:E1

  13. #13
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by rorya View Post
    You copied A1:A5 not A1:E1
    Oh.... my brain is really short circuit yesterday.

  14. #14
    Forum Expert
    Join Date
    10-06-2017
    Location
    drevni ruchadlo
    MS-Off Ver
    old
    Posts
    2,151

    Re: What if SQL query return more records than excel sheet row limit?

    If I remember correctly, MySQL has "LIMIT" and "OFFSET" clauses that you could use to retrieve data.
    The general query syntax would be as follows:

    Please Login or Register  to view this content.
    Such an instruction can be used in a loop by substituting appropriate values for "x" and "y".

  15. #15
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by rorya View Post
    You copied A1:A5 not A1:E1
    Please Login or Register  to view this content.
    The code works, but still minor error. Header has been copied one extra time. I mean if there 2.5 millions of record, then it breaks up to 3 sets of output, but header has shown up four times.

  16. #16
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: What if SQL query return more records than excel sheet row limit?

    You can check the attached file where the recordset is listed on Sheet2 with 5 rows of data groups
    Attached Files Attached Files

  17. #17
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by Haluk View Post
    You can check the attached file where the recordset is listed on Sheet2 with 5 rows of data groups
    Thank you very much. By the way, what does $ mean in From [Sheet1$] ?
    Last edited by VAer; 03-24-2024 at 11:21 AM.

  18. #18
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: What if SQL query return more records than excel sheet row limit?

    That's a requirement in use of ADO/SQL in VBA...... Sheets are considered as tables and they must end with a "$" sign.

    .

  19. #19
    Forum Contributor
    Join Date
    09-10-2016
    Location
    USA
    MS-Off Ver
    Office 365
    Posts
    680

    Re: What if SQL query return more records than excel sheet row limit?

    Quote Originally Posted by Haluk View Post
    That's a requirement in use of ADO/SQL in VBA...... Sheets are considered as tables and they must end with a "$" sign.

    .
    Thanks for the information. I have not used SQL on excel sheet source data (this is not something I knew before), when I use SQL, I deal with MS SQL or Sybase database.

  20. #20
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: What if SQL query return more records than excel sheet row limit?

    You're welcome and thx for the rep.

  21. #21
    Valued Forum Contributor Haluk's Avatar
    Join Date
    02-14-2019
    Location
    Turkiye
    MS-Off Ver
    2010 - 64 Bit on Windows-11 (22 H2) 64 Bit
    Posts
    1,150

    Re: What if SQL query return more records than excel sheet row limit?

    I've just noticed a small bug in the code. Please use the following where you can specify the maximum number of rows in the variable "MaxRows"

    Please Login or Register  to view this content.
    Last edited by Haluk; 03-24-2024 at 12:04 PM.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] Writing a SELECT query to return only records with duplicate values in one column
    By mc84excel in forum Access Tables & Databases
    Replies: 3
    Last Post: 04-24-2015, 12:16 AM
  2. Excel query/macro , delete records where the aggregate value is not over 100
    By guitarplyr in forum Excel Formulas & Functions
    Replies: 7
    Last Post: 03-17-2015, 02:59 PM
  3. Query Access database and return all records to Excel
    By bfs3 in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 02-18-2014, 11:01 PM
  4. Replies: 2
    Last Post: 12-20-2013, 01:09 PM
  5. [SOLVED] Query Oracle using range in Excel to return individual records for each cell in range
    By bigwillydier in forum Excel Programming / VBA / Macros
    Replies: 7
    Last Post: 03-18-2013, 06:37 PM
  6. VBA Read, write records to one sheet. Query
    By dan007004 in forum Excel Programming / VBA / Macros
    Replies: 1
    Last Post: 10-21-2011, 03:03 AM
  7. [SOLVED] How do I get beyond the upper limit of 65,500 records on Excel?
    By Nicholas Korsakov in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 10-01-2005, 12:05 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1