+ Reply to Thread
Results 1 to 16 of 16

Remove Duplicates giving trouble

  1. #1
    Forum Contributor
    Join Date
    07-26-2012
    Location
    Netherlands
    MS-Off Ver
    Excel 2016 Office 365 ProPlus
    Posts
    822

    Remove Duplicates giving trouble

    Attached a list with material numbers which contain duplicates.

    Now I sorted the list (not yet in the example to have my beginning situation) as one would do before using remove duplicates.
    But after remove duplicates still a few materials are double.

    For example 1234250-12

    I checked the length of both cells but both length 10 so no blanks.

    Does anybody know what the problem is?
    Thanks.
    Attached Files Attached Files

  2. #2
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,980

    Re: Remove Duplicates giving trouble

    I am not seeing the problem here in Excel 2017 365.
    Ali


    Enthusiastic self-taught user of MS Excel who's always learning!
    Don't forget to say "thank you" in your thread to anyone who has offered you help.
    You can reward them by clicking on * Add Reputation below their user name on the left, if you wish.

    Forum Rules (updated August 2023): please read them here.

  3. #3
    Forum Contributor
    Join Date
    07-26-2012
    Location
    Netherlands
    MS-Off Ver
    Excel 2016 Office 365 ProPlus
    Posts
    822

    Re: Remove Duplicates giving trouble

    Well unfortunately I do see it in office 365 excel.....

    How many lines do you have left after remove?

    Should be 1894

  4. #4
    Forum Expert 63falcondude's Avatar
    Join Date
    08-22-2016
    Location
    USA
    MS-Off Ver
    365
    Posts
    6,266

    Re: Remove Duplicates giving trouble

    Odd... I don't see the problem if I remove duplicates before sorting.

    But if I sort and then remove duplicates, there are a few dups left... Looking into it further.


    Edit: Disregard the first line of this post.
    Last edited by 63falcondude; 10-18-2017 at 11:14 AM.

  5. #5
    Forum Contributor
    Join Date
    07-26-2012
    Location
    Netherlands
    MS-Off Ver
    Excel 2016 Office 365 ProPlus
    Posts
    822

    Re: Remove Duplicates giving trouble

    Also removing before sorting leaves you with 1907 values so still some duplicates (1894 is unique)

  6. #6
    Forum Expert 63falcondude's Avatar
    Join Date
    08-22-2016
    Location
    USA
    MS-Off Ver
    365
    Posts
    6,266

    Re: Remove Duplicates giving trouble

    You're right, I stand corrected.

    Looking into a solution as to why there are dups left.

  7. #7
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,980

    Re: Remove Duplicates giving trouble

    Your example is not duplicated (1234250-12). I haven't completely scrutinised the list, but can't see any duplication at first glance. Will double check.

    EDIT:

    Excel 2016 (Windows) 32 bit
    F
    G
    2
    2001
    Y
    4
    22215
    Y
    5
    22260
    Y
    6
    22290
    Y
    7
    22295
    Y
    8
    23242
    Y
    9
    23245
    Y
    10
    28000
    Y
    11
    28001
    Y
    186
    1000185-115 Y
    1482
    22225M Y
    1483
    22225MJ Y
    1484
    22235M Y
    1485
    22235MJ Y
    1837
    2001
    Y
    1838
    22215
    Y
    1839
    22260
    Y
    1840
    22290
    Y
    1841
    22295
    Y
    1842
    23242
    Y
    1843
    23245
    Y
    1844
    28000
    Y
    1845
    28001
    Y
    1849
    1000185-115 Y
    1875
    22225M Y
    1876
    22225MJ Y
    1877
    22235M Y
    1878
    22235MJ Y
    Sheet: Sheet1
    Last edited by AliGW; 10-18-2017 at 10:58 AM.

  8. #8
    Forum Expert 63falcondude's Avatar
    Join Date
    08-22-2016
    Location
    USA
    MS-Off Ver
    365
    Posts
    6,266

    Re: Remove Duplicates giving trouble

    I have to admit, I have never seen this before.

    If I remove duplicates, there are some duplicates left.

    I have to sort the list and then remove duplicates again for all of the duplicates to be removed.

  9. #9
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,980

    Re: Remove Duplicates giving trouble

    By the way, your profile says you are using 2010 - please update it.

  10. #10
    Forum Expert 63falcondude's Avatar
    Join Date
    08-22-2016
    Location
    USA
    MS-Off Ver
    365
    Posts
    6,266

    Re: Remove Duplicates giving trouble

    Ali, can you confirm whether or not you are experiencing the same behavior?

    1) Open the workbook from post #1
    2) Highlight all of column A and Remove Duplicates (shows 1907 unique values remaining)
    3) Sort column A (you can see that the first 2 rows are dups)
    4) Remove duplicates again (shows 1894 unique values remaining)

    After step 2, you can run a COUNTIF formula in column B and see that there are duplicates left.
    Last edited by 63falcondude; 10-18-2017 at 11:20 AM.

  11. #11
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,980

    Re: Remove Duplicates giving trouble

    Yes, if I follow your steps. The list I posted above was what was left when I copied the unsorted list from A to F and removed duplicates, then did a count check to see which were still duplicated - I filtered the list to copy it here.

  12. #12
    Forum Expert 63falcondude's Avatar
    Join Date
    08-22-2016
    Location
    USA
    MS-Off Ver
    365
    Posts
    6,266

    Re: Remove Duplicates giving trouble

    So the question would then be why are those duplicates left after running Remove Duplicates?

    After a little bit of research, I have found that the built-in Remove Duplicates function can be unreliable, especially when the data is unsorted and has a mix of numbers and text.

    The weird thing is that even if I sort it before removing dups, it doesn't find them all in the first go.

  13. #13
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,980

    Re: Remove Duplicates giving trouble

    Interesting. I have never had a problem with it, and I use it a lot, although not on mixed lists - mostly on pure text.

  14. #14
    Forum Expert 63falcondude's Avatar
    Join Date
    08-22-2016
    Location
    USA
    MS-Off Ver
    365
    Posts
    6,266

    Re: Remove Duplicates giving trouble

    Same. Which leads me to wonder if I have made any errors in the past due to this.

    It seems that the most reliable ways of removing duplicates is either through code or formulas.

    rpinxt, I do not have an answer as to why the built-in Remove Duplicates feature is not reliable. That being said, here is a formulaic approach:

    B2 =COUNTIF(A$2:A2,A2) copied down column B

    Filter column B to show values > 1

    Delete those rows.

  15. #15
    Forum Contributor
    Join Date
    07-26-2012
    Location
    Netherlands
    MS-Off Ver
    Excel 2016 Office 365 ProPlus
    Posts
    822

    Re: Remove Duplicates giving trouble

    Hmmm ok.
    So the conclusion is that build in remove duplicates is unreliable on mixed lists.
    And that using a formula for it is the workaround?

  16. #16
    Forum Moderator AliGW's Avatar
    Join Date
    08-10-2013
    Location
    Retired in Ipswich, Suffolk, but grew up in Sawley, Derbyshire (England)
    MS-Off Ver
    MS 365 Subscription Insider Beta Channel v. 2404 (Windows 11 22H2 64-bit)
    Posts
    80,980

    Re: Remove Duplicates giving trouble

    Yup - so it would seem.

    You might be able to record a macro to do it.

+ Reply to Thread

Thread Information

Users Browsing this Thread

There are currently 1 users browsing this thread. (0 members and 1 guests)

Similar Threads

  1. [SOLVED] SumIF formula giving me a bit of trouble.
    By AlieKat in forum Excel General
    Replies: 10
    Last Post: 09-26-2017, 03:05 PM
  2. Replies: 2
    Last Post: 05-30-2017, 12:22 AM
  3. INDEX/MATCH with =LARGE is giving duplicates
    By keith740 in forum Excel General
    Replies: 6
    Last Post: 10-07-2015, 03:35 AM
  4. Dates are giving me trouble!
    By Daniel86 in forum Excel Formulas & Functions
    Replies: 3
    Last Post: 07-08-2013, 12:29 PM
  5. Replies: 1
    Last Post: 11-01-2012, 05:34 AM
  6. Replies: 5
    Last Post: 02-28-2012, 02:52 PM
  7. IF statements giving me trouble...
    By Twisty20 in forum Excel Formulas & Functions
    Replies: 7
    Last Post: 12-11-2006, 05:18 PM

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts

Search Engine Friendly URLs by vBSEO 3.6.0 RC 1