+ Reply to Thread
Results 1 to 21 of 21

Adding manual data to a power query?

  1. #1
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69

    Adding manual data to a power query?

    I'll set the scene..

    We have a big ident list at work with around 1000 items on it. Each entry has a unique ID number. The list is split into 6 sheets because different people are responsible for looking after each one.

    I need to do two things..
    1. I need a combined list of all 6 sheets so that I can easily search and reference data with INDEX & MATCH etc.
    2. I need to be able to add my own data to the end few columns.

    I've managed to work out how to get power query to append all the sheets together onto one master list (I've also removed some of the columns I don't need). I was hoping that I could then simply add the columns onto the appended output for manual data entry from there but it doesn't seem to like it for some reason. I tried putting in some dates and it keeps removing them at random when I refresh the query.

    Is there a way I can tell it to leave those few columns alone or something along those lines?

  2. #2
    Forum Expert CK76's Avatar
    Join Date
    06-16-2015
    Location
    ONT, Canada
    MS-Off Ver
    MS365 Apps for enterprise
    Posts
    5,887

    Re: Adding manual data to a power query?

    PQ is light ETL tool and not meant as data entry tool.

    If you need to add data column wise.
    You can do so using some unique ID column along with data columns to be added in a workbook, load it to PQ.
    Then join the data onto master table using Left Outer (or full outer as needed) using unique Id column as key.

    However, I'd recommend loading data to some data base if you are able (MS Access, MS SQL Express etc) and then querying and loading data to the model through native query.
    ?Progress isn't made by early risers. It's made by lazy men trying to find easier ways to do something.?
    ― Robert A. Heinlein

  3. #3
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69

    Re: Adding manual data to a power query?

    Thanks for the reply.

    So there's no simple way of adding data to an appended table in excel? By the sound of it it's best to leave those tables alone?

    I made a similar file before, not usuing query but using about 10'000 VLOOKUPS. It wasn't ideal because it required both files to be open, couldn't tell if a new row was added, and it was pretty heavy on the memory usage. So I was hoping for an easier solution but I guess it's just not as simple as it sounds in my head :/

  4. #4
    Forum Expert CK76's Avatar
    Join Date
    06-16-2015
    Location
    ONT, Canada
    MS-Off Ver
    MS365 Apps for enterprise
    Posts
    5,887

    Re: Adding manual data to a power query?

    It really depends on your set up. Without sample workbooks and how you are transferring data from one to another. Bit hard to help you with specifics.

    Though in general yes, you wouldn't manually append/merge data to existing table in PowerQuery. There are ways to do it, but it will require unique ID field to manipulate specific row of data.

  5. #5
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69

    Re: Adding manual data to a power query?

    I could send you a very rough file if you'd like to take a look? I'll have to rename and delete a few things and I'd rather not paste the link directly on here as it's confidential but I can send you a version of it?

  6. #6
    Forum Expert CK76's Avatar
    Join Date
    06-16-2015
    Location
    ONT, Canada
    MS-Off Ver
    MS365 Apps for enterprise
    Posts
    5,887

    Re: Adding manual data to a power query?

    Sure, you can desensitize data as much as you can, and you can PM me. I'm bit busy over the weekend, but I can take a look when I find time.

  7. #7
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69

    Re: Adding manual data to a power query?

    Thanks

    I sent a PM. Not sure if its actually gone or not because its not showing in my Sent folder.

  8. #8
    Forum Expert CK76's Avatar
    Join Date
    06-16-2015
    Location
    ONT, Canada
    MS-Off Ver
    MS365 Apps for enterprise
    Posts
    5,887

    Re: Adding manual data to a power query?

    I've received it.

    Few suggestions.

    1. I'd recommend adding another sheet for set up.
    It can house CELL() function to dynamically change file location and query path (to pass on to PQ). You can find tutorial on this in thread below (post #6)
    https://www.excelforum.com/excel-gen...mic-range.html

    2. Since you have Trial no. as unique. I'd have separate sheet for data entry inside Master workbook, using Trial no. to add columns. Then merge (not append) that info to the query using left join.

    What would be the process flow?

  9. #9
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69

    Re: Adding manual data to a power query?

    1. Thanks ill look into that.

    2. Ok so this would allow me to input data into the 2nd sheet and then it'd be displayed on the master list? I guess that could be a solution. The only problem is, as you can see, some of the trial numbers haven't been assigned yet. So if the 2nd sheet is independant of the source file then it won't update when a new trial no. gets added.

    I feel like this is probabaly as close as we're going to get though.. Maybe it's best to just talk to the trial managers and see if they'll let me add the columns to the tables on the source file?

  10. #10
    Forum Expert CK76's Avatar
    Join Date
    06-16-2015
    Location
    ONT, Canada
    MS-Off Ver
    MS365 Apps for enterprise
    Posts
    5,887

    Re: Adding manual data to a power query?

    That would be the simplest method. Yes.

    As well, another thing you could implement, is to have it mandated that before you take any action, trial no. must be assigned. If that's possible.
    Then you filter out null values on Trial no. column in your query.

    Oh, here's sample query, which is done in single query instead of using separate query to each sheet. I'll send the file in PM.
    Please Login or Register  to view this content.

  11. #11
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,916

    Re: Adding manual data to a power query?

    Please attach the desensitised file here so that anyone learning PQ can benefit from the help given here. Nothing should go via PM. This is a public forum for the benefit of all members. Thanks.
    Ali


    Enthusiastic self-taught user of MS Excel who's always learning!
    Don't forget to say "thank you" in your thread to anyone who has offered you help.
    You can reward them by clicking on * Add Reputation below their user name on the left, if you wish.

    Forum Rules (updated August 2023): please read them here.

  12. #12
    Forum Expert CK76's Avatar
    Join Date
    06-16-2015
    Location
    ONT, Canada
    MS-Off Ver
    MS365 Apps for enterprise
    Posts
    5,887

    Re: Adding manual data to a power query?

    My bad.

    @S7ewi
    If you are ok with it. I'll upload the file here.

  13. #13
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69

    Re: Adding manual data to a power query?

    Thanks for your help. Ill give that a try :D

    Ermmm.. I'll make another copy with less info in it. I know I'm being paranoid but I could literally be fired if I was found to have let that out.

  14. #14
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,916

    Re: Adding manual data to a power query?

    With the greatest of respect to both of you, that sensitive data has already been shared with a complete stranger. CK76 probably is completely trustworthy, but has still been party to confidential data. You really should not do this again as you are pretty certain to already be in breach of company policy. Once again, this is a public forum, and nobody using it, including myself, has been vetted in any way.

  15. #15
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69

    Re: Adding manual data to a power query?

    Ok well if it's that much of an issue then feel free to delete the thread. I was after some help with a personal issue, CK76 was kind enough to help out. The file is safe and doesn't share any confidential information but there's still enough on there to trace it back to me if the right person saw it. I wasn't aware every detail had to completely public. I've not been on that kind of forum before.

    I apologise, and if it's that much of a problem please feel free to delete the topic.

  16. #16
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,916

    Re: Adding manual data to a power query?

    There is no problem for us, but I am trying to explain that you shouldn’t be sharing company data with anyone on any public forum, either via private message or otherwise. I would strongly suggest you take greater care in future. My comments are meant to serve as advice to you. I hope you understand.

  17. #17
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69
    I appreciate the advice but I know my company policy. There's nothing on there that breaks any kind of confidentiality agreement. I'm just paranoid that it could still be traced back to me becuase I had to leave some basic info in, otherwise the table would have been empty.

    It seems as though, what I was originally asking, can't be done anyway so that's pretty much it :/

  18. #18
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,916

    Re: Adding manual data to a power query?

    Paranoia in this case is a good thing.

    I hope you understand why sample files should be attached to the thread, though. If I didn’t make that clear, please feel free to ask for clarification.

  19. #19
    Forum Expert CK76's Avatar
    Join Date
    06-16-2015
    Location
    ONT, Canada
    MS-Off Ver
    MS365 Apps for enterprise
    Posts
    5,887

    Re: Adding manual data to a power query?

    Here's completely desensitized sample.
    Attached Files Attached Files
    Last edited by AliGW; 01-19-2019 at 01:27 AM. Reason: .zip file replaced with .xlsx files

  20. #20
    Registered User
    Join Date
    01-18-2019
    Location
    UK
    MS-Off Ver
    365
    Posts
    69

    Re: Adding manual data to a power query?

    I understand

    Thanks guys, I think I'm going to rewrite the thread as more of a project because I think I need to look at other solutions now.

    Thanks

  21. #21
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,916

    Re: Adding manual data to a power query?

    If that takes care of your original question, please select Thread Tools from the menu link above and mark this thread as SOLVED. Thanks.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. Sub-Forum for Excel Power Tools (Power Query, Power Pivot & Power BI)
    By chullan88 in forum Suggestions for Improvement
    Replies: 10
    Last Post: 06-28-2018, 02:25 PM
  2. Adding Column in Power Query
    By fuerte in forum Excel Formulas & Functions
    Replies: 9
    Last Post: 06-28-2018, 12:39 PM
  3. Adding Column in Power Query
    By fuerte in forum Excel Programming / VBA / Macros
    Replies: 3
    Last Post: 06-28-2018, 11:15 AM
  4. Replies: 0
    Last Post: 04-05-2018, 01:16 AM
  5. Power Query and adding a column changing dates
    By ammartino44 in forum Excel General
    Replies: 0
    Last Post: 01-09-2018, 05:24 AM
  6. [SOLVED] Adding Missing Dates To A .CSV File Via Power Query?..
    By Ourkid123uk in forum Excel Formulas & Functions
    Replies: 5
    Last Post: 05-04-2017, 03:54 PM
  7. Adding data to the end of a table with Power Query
    By afila in forum Excel Programming / VBA / Macros
    Replies: 0
    Last Post: 06-14-2016, 08:29 AM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1