How to Find (and Remove) Duplicate Files in Windows 10 Open CCleaner. Select Tools from the left sidebar. Choose Duplicate Finder. For most users, running the scan with the default selections is fine. Choose the drive or folder you want to scan. Click the Search button to start the scan. Select the files you’d like to remove (carefully). Compare Folders for Duplicate Files Windows 10 November 30, 2016 November 27, 2018 Raza Ali Kazmi Duplicates, Solutions, Windows 10 Comparing folders involves viewing differences between the contents of two or more folders.
Even though the size of hard disc drives these days stretches into the Terabytes, it still doesn’t take long to start filling them up with all kinds of files which don’t necessarily belong or need to be there. These can come in various forms such as junk files, temporary files or even those which you just don’t need anymore. Another type of data you probably don’t really want is duplicate files.
It’s easily done that you can copy several files onto your drive and either give them different names that help recognize them easier, or forget they are there and then copy them over again and place everything into a different folder. This means there are now multiple copies of the files on the drive where a single copy would be enough, the rest are now taking up extra space and making organization more complicated. Start doing this with 1,000’s of files or large files and Gigabytes of storage could be needlessly lost.
Simply searching for a certain file name usually isn’t enough on its own which is why many tools employ advanced methods to try and find identical files, even though they may not look the same on the face of it. Here’s a selection of 10 free utilities that can help you search for and identify these duplicate files and help reclaim some of your hard drive space.
Note: As with any tool that allows you to delete files from the system, a degree of care is needed because removing files that belong to Windows or software could in the worst case cause Windows to become unusable. Some files are duplicated in Windows by design for backup and protection purposes. In general it’s recommend to restrict these actions to areas such as data drives and My Document folders etc, places that only affect user data and not critical areas of your computer.1. Heatsoft Clone Cleaner Lite
Clone Cleaner Lite isn’t blessed with loads of options, but that does make it relatively easy to use. Sadly the Lite version cannot search more than one location at once but does contain include / exclude, size, date and attribute filters and once you have pressed the Search and Content Comparison buttons, all the results will show in the lower window.
Clone Cleaner Lite uses a series of tabs across the middle of the window to show you the scan result for; same name, same size, same name and size, same content, full clones and zero size. Usage of Smart Mark will allow you to automatically mark the files according to your criteria and also setup a To Keep area to stop deletions from certain folders. Works on Windows 2000 and above.
Download Heatsoft Clone Cleaner Lite
2. SearchMyFiles
SearchMyFiles is another smart little portable utility from NirSoft and is more of a general file searching tool, but has a duplicate finder function built in. Simply select Duplicates Search in the Search Mode drop down at the top. In this mode the program will first search using the settings you configure in the main window such as attributes, files sizes, time and date, file contains and exclusions etc, then in a second pass will take the first pass files that match and compare them byte for byte.
The finished matches will show in the results window with different color blocks to group the duplicates together. Selecting the duplicates for removal has to be done manually and do be aware that some color blocks look very similar when next to each other, you don’t want to delete files that look like duplicates when they aren’t. Works on Windows 2000 and above
Download SearchMyFiles
3. Anti-Twin
Anti-Twin is free for personal use and is able to search for duplicate files based on a number of different conditions. These include a file filter to exclude file sizes and extensions, search for certain types of file, compare by extension, filename, byte by byte, and also to compare images by pixels (a preview window is included). You can also define a match percentage for files which you would consider close enough to be included.
After a scan, there are some useful options to exclude specified folders from the delete selection (such as a master folder) and to delete the files first with an older creation date or deeper down inside the directory structure. A second window will offer tips for the best results. Anti-Twin works on Windows 2000 and above and has setup and portable versions.
Download Anti-Twin
4. Auslogics Duplicate File Finder
Auslogics may be better known for their Boostspeed and Disk Defrag tools, but Duplicate File Finder is another program that’s easy to use and might have enough options for most of the average user’s needs. It can look for certain file types and choose to ignore above and below certain file sizes, also names and dates can be ignored meaning the program will search and find matches based on MD5 checksums, network drive searching is not supported.
After scanning you can easily select one or all duplicates of each file by the click of the drop down menu. A handy fallback feature is the option to send files to the recycle bin or even make a backup of deleted files to the Auslogics Rescue Center, accidental deletes can then be restored later. Works on Windows XP and above. Watch out on install for the Auslogics (Ask) toolbar adware being offered.
Download Auslogics Duplicate File Finder
5. Easy Duplicate Finder
Easy Duplicate Finder is quite an easy to use portable utility which is able to auto exclude operating system files such as DLL or SYS and also the Windows and Program Files folders preventing accidental deletion. You can also configure multiple include or exclude masks if needed and also the same with scan locations, along with min and max file size restrictions. In Settings is the option to perform a byte by byte comparison or using CRC32 checksums and file sizes.
The resulting duplicates can be sorted for deletion by newest, oldest or folder / subfolder. A useful feature is you can rename all the duplicates or move them to a folder if you don’t want to delete anything yet. Do note this is an older version of the program, the latest version is a trial and only allows deleting / moving 10 files.
Download Easy Duplicate Finder
6. Fast Duplicate File Finder
The free version of this program has a few features missing such as similar files, filtering, quick checking, export to XML and command line options, but still has plenty of functions to get all but the most advanced tasks done easily. System files and folders are disabled by default to help prevent mistakes and scanning is done by checking the contents of each file or by file names. A useful Auto check button can select files by newest / oldest dates and sizes, or more advanced selections can be made to the names, dates or file name wildcards and extensions.
A handy preview window can show the content of all files and the duplicate files can be moved to another structured folder or deleted to the Recycle Bin. Fast Duplicate File Finder works from Windows 8 right back to Windows 98.
Download Fast Duplicate File Finder
7. Bytessence DuplicateFinder
DuplicateFinder is a small and optionally portable tool that has several functions crammed into its small wizard like interface. By default it will exclude the Windows folder and several system file types like .COM, .DLL and .SYS from any scan, and those along with inclusion file types can be edited in step 2. File attributes, the file dates and sizes can be configured for matching the files and there are a total of 6 duplicate file finding methods; contents (CRC32), name, size and accessed / created / modified dates.
The results also have several options for what to do with the duplicates such as move, copy, delete (with or without the Recycle Bin), run an external program and there are another 6 options to select the specified duplicate files from the list. Bytessence DuplicateFinder works on Windows NT and above.
Download Bytessence DuplicateFinder
8. Duplicate Cleaner Free
Duplicate Cleaner Free is missing a few features from its shareware big brother such as finding similar files and images, searching inside zip files, and the ability to hard link files. The free version is still full of features including an audio mode which can find by artist, title or album, the option to ignore mount points and hardlinks, and several different content type comparisons of MD5, SHA and byte to byte. All the standard include and exclude filters are available while system areas and files are disabled by default.
When you have a list of duplicates, the selection assistant button can help pick out the files that are best matched to be removed, and once chosen they can be deleted with or without the recycle bin, moved or copied. Works on Windows XP and above, .NET Framework v2 is required.
Download Duplicate Cleaner Free
9. AllDup
AllDup is a duplicate file finding tool that is absolutely packed with functions, in fact it’s got so many that average users have to pay special attention that they’re using the correct options for their specific purposes. There are comprehensive option windows for searching, filtering, which folders to search and what to preview. It has the standard name, date, attributes, size and extension search modes, but also can search hard links and in byte to byte mode can choose to search just the back end of the file first.
Another useful feature not found on many other free tools is the ability to search for duplicates inside ZIP and RAR archives. When all the duplicates are found, AllDup has another ton of different options to select the files to delete, move, copy or rename. A very powerful program in the right hands. Works on Windows 2000 to Windows 8, a portable version is also available.
Download AllDup
10. CloneSpy
CloneSpy is quite an interesting tool because it can compare 2 different sets of multiple folders or drives against each other, which is done inside the program by the use of Pool 1 and Pool 2 and selecting the Both Pools Mode. The program has 4 main scan modes which are searching for files with the same content, same content + name, same name and also same name and size. It can also save the checksums to a file which can then be loaded at a later date to compare against the selections in the current Pool.
A function to be used with care is the Actions which can be configured to delete files automatically, although for protection most system folders and file types are disabled by default, and other include or exclude filters are accessible from the Options. A portable version can be made by running the setup installer. Works on Windows 2000 and above.
Download CloneSpy
Editor’s Note: There are so many of these duplicate file finding tools around that appear to do at least a reasonable job, it almost seems unfair to only list 10, but obviously we have to stop somewhere. Some of the other tools you might like to look at that didn’t make the list were; DupKiller, DoubleKiller, VisiPics (very good for checking just images), Search My Files and Duplicate Filter.
You might also like:
7 Ways To Remove Duplicate Lines in Text Files6 Ways to Safely Remove Windows XP Update Backup Files to Free Up Hard Disk Space8 Free Tools to Find What Files and Folders are Taking Up Hard Drive Space8 Free Tools to Securely Wipe Free Disk Space Preventing Recovery2 Tools to Search Any Files on Local Area Network Shared Folders 16 Comments - Write a Comment
I’ve depended on Doublekiller for over fifteen years!? There may be more sophisticated tools now, but the essence of a good tool, like a knife, is it does a clear function extremely well. I am conditioned to Tab-advance rapidly to the boxes/entry windows, and I can cut-and-paste from other search tools to narrow the range of folders/subfolders searched. One can enter advanced filters, of any suffix, and select which duplicates to move/delete. The Results window is uncluttered, and groups files with a slight shading change between each set, which I learned to use for quick visual skimming before deleting. If any group of files needs to be “deselected,” after the preliminary choice has been made, mouse drag select can include a range, or with Ctrl-click, add or subtract specific files, which one can then “Uncheck selected files.” Results doesn’t confuse with too much information, but you can open any file there, and click it to open its parent folder in Explorer.
Old as it is, it is tiny, one click, and will search USB and other external drives, even mounted virtual drive images in case you want to see if data is already safely backed up elsewhere.
I like this software because it works very great.
ReplyThat is a pretty good collection of tools for finding and removing the duplicate files and freeing up the disc’s space .
ReplyI was all frustrated because of this duplicate files issue but this sounds like an interesting work. I will surely do it.
ReplyThat’s a pretty good list! I use Easy Duplicate Finder on both my Windows and Mac laptops and it works very well. But you have an ancient screenshot here – the program looks very different now.
ReplyAs noted in the article this is for the last free version that was available, it’s now shareware and costs a small fortune.
ReplyHey, would you mind telling me the equivalent of these for a mac laptop?
much appretiated…. :)
To the rescue again, Hal9000! Much appreciated. Great list of freebies, with Auslogics being my favourite duplicate file finder.
ReplyThanks – as always, your articles are very useful :-)
Replythanks bro – was looking for somebody to “cut to the chase” and you did.
appreciated
“Duplicate File Finder” is absolutely roaring through duplicate photo archives here –
much appreciated
ReplyThis is the best free hassle free software i have ever used! Dont waste paying money for deleting files. This should be your one stop solution!
Thanks for sharing the information, i could get back almost 5gb of disk space from my mp3 collection!
Thanks Raymond a good start.
I think the speed in the first 3 is because they don’t do a full binary compare first up. Certainly HCCLite makes its lists based on name, size etc. Then it will do a Binary compare on any of the lists it found with the fast methods (using the toolbar button at the bottom). It takes much longer to do this. FileDup may be doing this on the first pass which would explain why it is so slow.
Ciao
ReplyRaymond,
for a couple of years I believed ‘Black Viper’ was the most respected Guru online …. I have to say …. you have snatched that crown, in my humble opinion.
Thank you for a very credible and useful site. My compliments – you are required reading in my arsenal now.
Merlin
Thanks … I was getting fed up myself with all the duplicates on my poor overused computer ;-)
ReplyLol, same situation here. A lot of drives, small space left and no idea of what is inside. Very nice tools, will try it soon…
Replythanks Raymond
Reply