How do I delete multiple double entries?

Hi,
I have to regularly import a series of excel data sheets from different sources and compare them. The total entries add up to around 10,000+ each time - but there are actually only about 4000 to 8000 unique entries since many are exact replications.

Currently, I sort the data according to column which causes the replica entries to run concurrently and are, therefore, obvious and simple to delete - but as you can imagine, this takes days.

I was thinking there must be a program or script or method of some kind, either within Excel itself or as an add-on or separate script, which would allow me to request the deletion of multiple identical entries - is there?

Any help much appreciated.
Thanks for reading.

Have you looked at Advanced Filter's "Unique only" feature combined with its Copy to Different Location?

try asap utiliies
http://www.asap-utilities.com/
has an excellent duplicate remover
mind u it can take some time to run on large spread sheets select column
then use column/rows conditional row-column hide select or delete option
from this select duplicate values
then check the rows and keep first value options
**** warning this is not reversable*****
so copy workbook and practice first