AllDup Manual

About AllDup

  • Product description
  • Requirements
  • Important note
  • Other software products

Main Window

  • Source folder
  • Search Method
  • Search options
  • Archive Files
  • Database
  • Filter
    • File filter
    • Folder filter
    • Environment variables
  • File preview
  • Profiles
  • Search Results
  • Options
  • Internet update

Search Result

  • Search result
  • Menu bar / Tool bar
  • Context Menu
  • File manager
  • Shortcuts

Miscellaneous

  • Command-line
  • Wildcards for text
  • Placeholder for date and time
  • Speed Comparison Test


AllDup

Find and remove duplicate files

AllDup is a freeware tool for searching and removing file duplicates on your computer, network shares or external storage media. The fast search algorithm find duplicates of any file type, e.g., text, pictures, music or movies.

The powerful search engine enables you to find duplicates with a combination of the following criteria: File Name, File Extension, File Size, File Content, File Dates and File Attributes.

Additionally you can also search for similar file names, similar or almost identical pictures and similar or almost identical music files. Furthermore, you can find video & audio files with the same or almost same audio length or search your hard disk for Hard Links.

AllDup Features

  • The file duplicates can be copied or moved to a folder of your choice
  • The built-in file viewer allows you to preview many different file formats and analyze the content of the file before deciding what to do with it
  • Entire folders or individual files can be excluded from the search by masks or size conditions
  • Search inside archives with the following files extensions: 7Z, ARJ, CAB, CB7, CBR, CBT, CBZ, EPUB, GZ, ISO, LZH, LHA, NRG, RAR, TAR, ZIP
  • Find duplicate files with a combination of the following criteria: file content, file name, file extension, file dates or file attributes!
  • Search for hard links
  • Ignore the ID3 tags of MP3 files
  • Search for duplicates of digital photo files
  • Search is performed in multiple specified folders, drives, media storages, CD/DVDs...
  • Search for duplicates of executable and any other files
  • Many flexible commands helps you to select unnecessary duplicates automatically
  • Delete duplicates files and create hard links
  • Export the search result to a text file, CSV file or an Excel file
  • Ignore the EXIF data of JPEG and CR2 files
  • Save and restore the search result to continue working later
  • The unnecessary duplicates can be deleted permanently
  • Find duplicate pictures
  • Search for similar pictures
  • Find similar files
  • List non-duplicate files
  • For your safety all files that have to be deleted can be moved in backup folder
  • Search through an unlimited number of files and folders
  • Create shortcuts or hard links to the last original file
  • Search for duplicates of music and video files
  • Fast search algorithm
  • Search for files with similar names
  • Find video & music files with the same or almost same audio length
  • For your safety all files that have to be deleted can be moved to the Recycle Bin
  • Detailed log file about all actions
  • Ignore the meta data of FLAC files
  • Convenient search result list with all duplicate files

What AllDup can do for you?

  • No more file clones or file duplicates!
  • Clear your computer, music or photo collection from duplicate files!
  • Remove duplicate files - Easily and Anywhere!
  • AllDup helps you to find, view and remove files that have duplicate content, regardless of name!
  • Find duplicate photos
  • Remove duplicate files faster and safer than with any other duplicate file finder!
  • Find duplicate MP3 files with different ID3 Tags!
  • It can be especially useful if you have large Music, Pictures, Downloads or Documents folders!
  • Find similar music files
  • Find and remove duplicate files!
  • Find and hardlink duplicate files!
  • Free up used hard disk space on your PC!
  • Find, remove, delete, copy or move duplicate files!
  • AllDup is a powerful tool to search for file duplicates on your computer!
  • Remove duplicate photos and images!
  • Find and delete duplicate files!
  • Find and remove duplicate files of ANY type!
  • Find similar pictures
  • Remove duplicate files - as you want!
  • Delete duplicate music files!
  • Find and remove duplicate files from your computer, notebook, network drives, flash drives, memory drives!
  • Find duplicate pictures
  • Find files with similar names
  • Finding out whether a file has hard links!
  • Find and delete hard links


AllDup Requirements

AllDup is compatible with all versions of the following operating systems (32/64-bit):

Workstation:
  • Microsoft Windows 11
  • Microsoft Windows 10
  • Microsoft Windows 8
  • Microsoft Windows 7
  • Microsoft Windows Vista*
  • Microsoft Windows XP*

Server:
  • Microsoft Windows Server 2022
  • Microsoft Windows Server 2019
  • Microsoft Windows Server 2016
  • Microsoft Windows Server 2012
  • Microsoft Windows Server 2008*
  • Microsoft Windows Server 2003*

* Please use the AllDup Portable Edition if you want to use AllDup with these older Windows operating systems. The AllDup setup file can only be installed from Windows 7 / Server 2012.


Important Note

Deleting, moving or renaming files can prevent Windows or programs from starting. You should only delete, move or rename files if you are absolutely sure that they are not required for the operation of Windows or other programs.

Please understand that we cannot give you any individual assistance with questions about which files may be deleted.


Other Software Products

AllSync - File Sync and Backup Software

AllSync

Data Backup & Folder Synchronization

AllSync is the professional solution for your data synchronization, imaging, backup, update, replication and archiving needs under Microsoft Windows.


Source Folders

In the source folder list, you can specify which drives and folders to scan for duplicate files. You enable a source folder for searching by checking the box in front of the folder path.

Exclude Subfolders
If you clear the check box in the Search all subfolders column for a source folder, then all subfolders in this source folder will be excluded from the search.

File Protection
If you activate the check box in the File protection column for a source folder, then AllDup cannot delete, move or rename any files within this source folder. File protection can be changed later if necessary in the search result using the File protection menu item in the menu bar.

Add new source folder
You can add a new folder to the source folder list by clicking the Source Folder button in the toolbar, by right-clicking in the source folder list, by clicking the + button in the top right corner of the source folder list or drag and drop a folder from Windows Explorer into the source folder list.

Edit source folder
If you right-click on a source folder, you will see a menu with various commands, such as Edit, Delete, Open...

Comparison Method

  • Compare files from all source folders
  • Compare only files within the same source folder
  • Compare only files between different source folders
  • Compare only files within the same source and sub folder

Options

Don't search folders (NTFS junction points) that point to the contents of another folder
NTFS junction points are empty folders that point to the contents of another folder. This option enables you to prevent these folders from being searched and processed. This option is activated by default. When deactivating this option you should check first that no loops are created by the junction points and no data items are processed twice.

Disable windows 64bit file system redirection
This option enables AllDup to determine the correct count of all system files on a 64bit Windows system.


Search Method

The following search methods are available to find duplicate files:
  • Find duplicate files on the basis of their properties
  • Find similar file names
  • Find similar pictures
  • Find similar audio files
  • Find video & audio files on the basis of the audio length
  • Find hard links

Find duplicate files on the basis of their properties

You can search for duplicate files by a combination of the following file properties:
  • File Name
    Finds files with the same name (without extension)
  • File Extension
    Finds files with the same extension
  • File Size
    Finds files with the same size
  • File Content
    Finds files with the same or similar content
  • Last Modified Date
    Finds files with the same last modified date
  • Create Date
    Finds files with the same create date
  • File Attributes
    Finds files with the same attributes
Ignore timestamp
This option enables you to ignore the timestamp when using the search method create date or last modified date.

Ignore seconds
This option enables you to ignore the seconds from the timestamp when using the search method create date or last modified date.

Options

Don't compare hard linked files
This option prevents the content comparison and the checksum creation of files which are hard links of themselves.

Find video & audio files on the basis of the audio length

This search method enables you to find video and audio files with the same or almost same audio length. The first audio track is always used to determine the audio length. Video and audio files with the following file extensions are supported:

Audio: AAC, AC3, AIFF, APE, AU, DTS, FLAC, M4A, MP2, MP3, OGG, WAV, WMA
Video: ASF, AVI, OGV, M2TS, M4V, MKV, MOV, MP4, MPG, WEBM, WMV

Ignore time difference
This option enables you to find video and audio files with almost the same audio length by specifying a maximum time difference between the duration of the audio tracks.

Compare only files with the same properties
  • File Name
    This option enables you to compare only files with the same file name.
  • File Extension
    This option enables you to compare only files with the same file extension.


Search Method 'File Name'

Compare method

  • Compare all characters of a file name
  • Compare only characters at the beginning of a file name
    This option enables you to compare only a certain amount of characters at the beginning of a file name and ignore the rest of the name.
  • Compare only characters at the end of a file name
    This option enables you to compare only a certain amount of characters at the end of a file name and ignore the rest of the name.
  • Ignore characters at the end of a file name
    This option enables you to ignore a certain amount of characters at the end of a file name.

Ignore specific characters from the file name

Here you can specify multiple characters that will be ignored when comparing two file names.
Example: Specifying the minus character (-) will treat the file names "data-01" and "data01" as a duplicate file name.

Ignore text patterns from the file name

Here you can specify multiple text patterns to be ignored when comparing two file names. Multiple text patterns must be separated with the slash char (/).
Example: The text patterns "001/002" will treat the file names "data001" and "data002" as a duplicate file name.

Find similar file names

Algorithm

The following algorithms are available to search for similar file names:
  • SmartMatch
  • FuzzyMatch
  • Levenshtein
  • Ratcliff-Obershelp
  • MatchDiff
  • WordMatch
  • FuzzyPercent
  • Simil

All algorithms except WordMatch performing different text comparisons and calculations between the two file names to calculate the perceptual match between them. The algorithm WordMatch splits the file names into words and calculates the perceptual match by comparing the words. All algorithms are not case-sensitive, meaning no distinction is made between uppercasing and lowercasing when performing comparisons.

Match

This is the degree of match that must exist between the file names in order to be offered as duplicates.

Delimiter

Here you can specify the delimiter characters for the algorithm WordMatch to split the file names into words. For example the file name "aa_bb_cc" will be splitted into the word collection "aa", "bb" and "cc" by using the delimiter character space (" "). The standard delimiter characters are: space, comma, underscore, dot, semicolon and minus. You can add or remove any number of delimiter characters.

Examples

The following file name comparison examples give you an overview which algorithm could be suitable for your search for similar file names.
The duration at the right table column shows the necessary time to perform 10.000 file name comparisons.

1. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Wolfgang_Amadeus_Mozart"
Compare Method | Match | Duration
FuzzyMatch | 88,34% | 0,63s
SmartMatch | 91,30% | 0,75s
Levenshtein | 91,30% | 1,16s
Ratcliff-Obershelp | 82,61% | 1,53s
MatchDiff | 91,30% | 0,09s
WordMatch | 100,00% | 0,33s
FuzzyPercent | 56,58% | 0,33s
Simil | 83,33% | 0,01s

2. Example

File name A: "Wolfgang Amadeus Mozart - 001"
File name B: "Wolfgang Amadeus Mozart - 002"
Compare Method | Match | Duration
FuzzyMatch | 96,43% | 1,41s
SmartMatch | 96,55% | 1,09s
Levenshtein | 96,55% | 1,83s
Ratcliff-Obershelp | 96,55% | 0,22s
MatchDiff | 96,55% | 0,07s
WordMatch | 75,00% | 0,50s
FuzzyPercent | 98,06% | 0,54s
Simil | 93,33% | 0,01s

3. Example

File name A: "001 - Wolfgang Amadeus Mozart"
File name B: "002 - Wolfgang Amadeus Mozart"
Compare Method | Match | Duration
FuzzyMatch | 96,31% | 1,38s
SmartMatch | 96,55% | 1,00s
Levenshtein | 96,55% | 1,81s
Ratcliff-Obershelp | 96,55% | 0,60s
MatchDiff | 96,55% | 0,07s
WordMatch | 75,00% | 0,42s
FuzzyPercent | 71,84% | 0,49s
Simil | 93,33% | 0,01s

4. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Wolfgang Amadeus Mozart - BACKUP"
Compare Method | Match | Duration
FuzzyMatch | 70,97% | 1,93s
SmartMatch | 83,64% | 0,83s
Levenshtein | 71,88% | 1,59s
Ratcliff-Obershelp | 83,64% | 0,15s
MatchDiff | 71,88% | 0,05s
WordMatch | 75,00% | 0,39s
FuzzyPercent | 77,95% | 0,91s
Simil | 78,57% | 0,01s

5. Example

File name A: "Wolfgang - Amadeus , Mozart"
File name B: "Mozart_Amadeus;Wolfgang"
Compare Method | Match | Duration
FuzzyMatch | 49,29% | 0,42s
SmartMatch | 32,00% | 0,53s
Levenshtein | 37,04% | 1,43s
Ratcliff-Obershelp | 32,00% | 1,88s
MatchDiff | 29,63% | 0,30s
WordMatch | 100,00% | 0,36s
FuzzyPercent | 26,32% | 0,38s
Simil | 23,08% | 0,02s

6. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Mozart Wolfgang Amadeus"
Compare Method | Match | Duration
FuzzyMatch | 70,55% | 0,65s
SmartMatch | 69,57% | 0,56s
Levenshtein | 39,13% | 1,20s
Ratcliff-Obershelp | 69,57% | 0,40s
MatchDiff | 69,57% | 0,17s
WordMatch | 100,00% | 0,34s
FuzzyPercent | 57,89% | 0,37s
Simil | 0,00% | 0,01s

7. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Wulfgang Amadues Mazort"
Compare Method | Match | Duration
FuzzyMatch | 88,54% | 0,86s
SmartMatch | 82,61% | 0,66s
Levenshtein | 78,26% | 1,23s
Ratcliff-Obershelp | 73,91% | 1,30s
MatchDiff | 82,61% | 0,26s
WordMatch | 0,00% | 0,33s
FuzzyPercent | 36,84% | 0,33s
Simil | 58,33% | 0,02s

8. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Ludwig van Beethoven"
Compare Method | Match | Duration
FuzzyMatch | 37,15% | 1,07s
SmartMatch | 32,56% | 0,75s
Levenshtein | 21,74% | 1,13s
Ratcliff-Obershelp | 27,91% | 3,68s
MatchDiff | 21,74% | 0,61s
WordMatch | 0,00% | 0,30s
FuzzyPercent | 5,26% | 0,29s
Simil | 0,00% | 0,01s

9. Example

File name A: "Beethoven"
File name B: "nevohteeb"
Compare Method | Match | Duration
FuzzyMatch | 72,22% | 0,21s
SmartMatch | 22,22% | 0,14s
Levenshtein | 33,33% | 0,25s
Ratcliff | 22,22% | 0,34s
MatchDiff | 22,22% | 0,14s
WordMatch | 0,00% | 0,15s
FuzzyPercent | 8,70% | 0,12s
Simil | 0,00% | 0,01s


Search Method 'File Content'

This search method enables you to compare the content of files with the same file size.

Comparison Method

  • Byte by byte
  • Checksum MD5 (128-Bit)
  • Checksum SHA-1 (160-Bit)
  • Checksum SHA-2 (256-Bit)
  • Checksum SHA-2 (384-Bit)
  • Checksum SHA-2 (512-Bit)

Byte by byte

This comparison method reads the content of both files chunk by chunk and compares the chunks byte by byte. With a specified percentage match of 100% the comparison of the two files will be stopped if the chunks are not identical. With a percentage match below 100% the complete content of both files will be analyzed to calculate the percentage match between them.
Example: With a specified percentage match of 90% all files with a percentage match of 90% or higher will be classified as duplicates.

Checksum

The checksum method reads the complete file content and creates a checksum of the content for further comparisons. Using a checksum could be much faster than a byte-by-byte comparison when you compare a lot of files. Comparing only the stored checksum will avoid to read the complete content of these files again and again. See Speed Comparison Test.

NOTE: Files with a size of 0 bytes will be automatically excluded from the search.

Ignore ID3 Tags of MP3 files

This option enables you to ignore the following tags inside MP3 files:
  • ID3v1
  • ID3v2
  • Lyrics3 v2
  • APE v2
Info: AllDup also ignores ID3v2 tags at the beginning of the MP3 file that exists twice and ID3v1 tags at the end of the file that exists twice.

Ignore metadata of JPEG/CR2 files

With this option, only the compressed image data inside files with the extension "JPEG", "JPG" and "CR2" will be compared.

Ignore metadata of FLAC files

This option enables you to ignore all the metadata inside FLAC files.

First compare a data block at the end of the files

With this option a data block at the end of both files will be compared first. The file comparison will be stopped if the data blocks are not identical. Otherwise AllDup continues the file comparison at the start of the file.

Block size of the read buffer

This option enables you to change the block size of the read buffer for the content compare.
  1. The file comparison starts with a block size specified by the start value.
  2. AllDup reads the block size from the two files into the memory and compares them.
  3. The file comparison will be stopped if the data blocks are not identical.
  4. Otherwise the block size will be increased by the increment value until it reaches the maximum value and continues comparing the files (2).


Find Similar File Names

Comparison Method

The following comparison methods are available:
  • SmartMatch
  • FuzzyMatch
  • Levenshtein
  • Ratcliff-Obershelp
  • MatchDiff
  • WordMatch
  • FuzzyPercent
  • Simil
All comparison methods except WordMatch performing different text comparisons and calculations between the two file names to calculate the perceptual match between them. The comparison method WordMatch splits the file names into separate words and calculates the perceptual match by comparing the words. All comparison methods are case-insensitive, meaning no distinction is made between uppercasing and lowercasing when performing the comparison.

Match

This is the degree of match that must exist between the file names in order to be offered as duplicates.

Delimiter

Here you can specify the delimiter characters for the comparison method WordMatch to split the file names into words. For example the file name "aa_bb_cc" will be splitted into the word collection "aa", "bb" and "cc" by using the delimiter character underscore ("_"). The standard delimiter characters are space (" "), comma (","), underscore ("_"), dot ("."), semicolon (";") and minus ("-"). You can specify any number of delimiter characters.

Examples

The following file name comparison examples give you an overview which comparison method could be suitable for your search for similar file names. The duration at the right table column shows the necessary time to perform 10.000 file name comparisons.

1. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Wolfgang_Amadeus_Mozart"
Comparison Method | Match | Duration
FuzzyMatch | 88,34% | 0,63s
SmartMatch | 91,30% | 0,75s
Levenshtein | 91,30% | 1,16s
Ratcliff-Obershelp | 82,61% | 1,53s
MatchDiff | 91,30% | 0,09s
WordMatch | 100,00% | 0,33s
FuzzyPercent | 56,58% | 0,33s
Simil | 83,33% | 0,01s

2. Example

File name A: "Wolfgang Amadeus Mozart - 001"
File name B: "Wolfgang Amadeus Mozart - 002"
Comparison Method | Match | Duration
FuzzyMatch | 96,43% | 1,41s
SmartMatch | 96,55% | 1,09s
Levenshtein | 96,55% | 1,83s
Ratcliff-Obershelp | 96,55% | 0,22s
MatchDiff | 96,55% | 0,07s
WordMatch | 75,00% | 0,50s
FuzzyPercent | 98,06% | 0,54s
Simil | 93,33% | 0,01s

3. Example

File name A: "001 - Wolfgang Amadeus Mozart"
File name B: "002 - Wolfgang Amadeus Mozart"
Comparison Method | Match | Duration
FuzzyMatch | 96,31% | 1,38s
SmartMatch | 96,55% | 1,00s
Levenshtein | 96,55% | 1,81s
Ratcliff-Obershelp | 96,55% | 0,60s
MatchDiff | 96,55% | 0,07s
WordMatch | 75,00% | 0,42s
FuzzyPercent | 71,84% | 0,49s
Simil | 93,33% | 0,01s

4. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Wolfgang Amadeus Mozart - BACKUP"
Comparison Method | Match | Duration
FuzzyMatch | 70,97% | 1,93s
SmartMatch | 83,64% | 0,83s
Levenshtein | 71,88% | 1,59s
Ratcliff-Obershelp | 83,64% | 0,15s
MatchDiff | 71,88% | 0,05s
WordMatch | 75,00% | 0,39s
FuzzyPercent | 77,95% | 0,91s
Simil | 78,57% | 0,01s

5. Example

File name A: "Wolfgang - Amadeus , Mozart"
File name B: "Mozart_Amadeus;Wolfgang"
Comparison Method | Match | Duration
FuzzyMatch | 49,29% | 0,42s
SmartMatch | 32,00% | 0,53s
Levenshtein | 37,04% | 1,43s
Ratcliff-Obershelp | 32,00% | 1,88s
MatchDiff | 29,63% | 0,30s
WordMatch | 100,00% | 0,36s
FuzzyPercent | 26,32% | 0,38s
Simil | 23,08% | 0,02s

6. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Mozart Wolfgang Amadeus"
Comparison Method | Match | Duration
FuzzyMatch | 70,55% | 0,65s
SmartMatch | 69,57% | 0,56s
Levenshtein | 39,13% | 1,20s
Ratcliff-Obershelp | 69,57% | 0,40s
MatchDiff | 69,57% | 0,17s
WordMatch | 100,00% | 0,34s
FuzzyPercent | 57,89% | 0,37s
Simil | 0,00% | 0,01s

7. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Wulfgang Amadues Mazort"
Comparison Method | Match | Duration
FuzzyMatch | 88,54% | 0,86s
SmartMatch | 82,61% | 0,66s
Levenshtein | 78,26% | 1,23s
Ratcliff-Obershelp | 73,91% | 1,30s
MatchDiff | 82,61% | 0,26s
WordMatch | 0,00% | 0,33s
FuzzyPercent | 36,84% | 0,33s
Simil | 58,33% | 0,02s

8. Example

File name A: "Wolfgang Amadeus Mozart"
File name B: "Ludwig van Beethoven"
Comparison Method | Match | Duration
FuzzyMatch | 37,15% | 1,07s
SmartMatch | 32,56% | 0,75s
Levenshtein | 21,74% | 1,13s
Ratcliff-Obershelp | 27,91% | 3,68s
MatchDiff | 21,74% | 0,61s
WordMatch | 0,00% | 0,30s
FuzzyPercent | 5,26% | 0,29s
Simil | 0,00% | 0,01s

9. Example

File name A: "Beethoven"
File name B: "nevohteeb"
Comparison Method | Match | Duration
FuzzyMatch | 72,22% | 0,21s
SmartMatch | 22,22% | 0,14s
Levenshtein | 33,33% | 0,25s
Ratcliff | 22,22% | 0,34s
MatchDiff | 22,22% | 0,14s
WordMatch | 0,00% | 0,15s
FuzzyPercent | 8,70% | 0,12s
Simil | 0,00% | 0,01s


Find Similar Pictures

This search method enables you to find similar or almost identical pictures. For each image will be a checksum calculated for further comparison with all other images.
  • Comparison Method
  • Hints
  • Image Formats
  • Percentage Match
  • Picture Area
  • Compare Size
  • Checksum
  • Compare only pictures with the same properties
  • Detect picture modifications (slower)
  • Recognition Rate Test
  • Examples

Comparison Method

The comparison methods aHash, bHash, dHash, mHash and pHash enables you to find similar or almost identical pictures by using a percent match lower than 100%. If you want find exactly the same pictures you have to use a percent match of 100% or the comparison methods MD5/SHA. An overview of the recognition rate of the comparison methods aHash, bHash, dHash, mHash and pHash can be found here.
  • aHash
    The comparison method aHash (Average Hash) resizes the image to 8x8 or 16x16 pixel. The image is then converted to grayscale and the average color value of all image pixels is calculated. Then all image pixels are compared with the average color value and the checksum is calculated.
  • bHash
    The comparison method bHash (Blockhash) resizes the image to 128x128, 256x256 or 512x512 pixel. The image will be divided into a block matrix and the median value of all blocks will be calculated to create the checksum. The options "Fast" and "Precise" enables you to influence the accuracy of the checksum calculation.
  • dHash
    The comparison method dHash (Difference Hash) resizes the image to 8x8 or 16x16 pixel. After that, the image is converted to grayscale and the checksum is created by comparing the difference in brightness values ??between all neighboring pixels.
  • mHash
    The comparison method mHash (Median Hash) resizes the image to 8x8 or 16x16 pixel. The image is then converted to grayscale and the mean color value of all image pixels is determined. Then all image pixels are compared with the mean color value and the checksum is calculated.
  • pHash
    The comparison method pHash (Perceptual Hash) resizes the image to 32x32 pixel. The image is then converted to grayscale and transformed by a discrete cosine transform (DCT). Next, the mean color value of all pixels in the image area (8x8) in the top left of the image is determined. Then the checksum is calculated by comparing the color value of all pixels from the top left image area with the mean color value.
  • MD5, SHA
    These comparison methods can only be used to find exactly the same images. The following comparison methods are available:
    • MD5 (128-Bit)
    • SHA-1 (160-Bit)
    • SHA-2 (256-Bit)
    • SHA-2 (384-Bit)
    • SHA-2 (512-Bit)
For more information on the aHash, bHash, dHash, mHash and pHash comparison methods, see articles Testing different image hash functions and Detection of Duplicate Images Using Image Hash Functions.

Calculation time of the checksums
Using the comparison methods aHash, bHash, dHash, mHash and pHash, we have calculated the time required to create a checksum and listed it in the following table:
Comparison Method | Picture Area | Checksum | Expenditure of Time
aHash | 8x8 | 64-bit | 0,0450 ms
aHash | 16x16 | 256-bit | 0,1425 ms
bHash (fast) | 256x256 | 256-bit | 9,6030 ms
bHash (precise) | 256x256 | 256-bit | 28,4792 ms
dHash | 8x8 | 64-bit | 0,0458 ms
dHash | 16x16 | 256-bit | 0,0988 ms
mHash | 8x8 | 64-bit | 0,1435 ms
mHash | 16x16 | 256-bit | 1,1012 ms
pHash | 32x32 | 64-bit | 8,6922 ms

Hints

The following files will be automatically excluded from the search:
  • Files with a size of 0 bytes
  • Pictures with a width or height smaller than the specified compare size
  • Corrupted, invalid or incomplete pictures (*)
  • Files with a blocked read access (*)
(*) An error message will be shown at the log pane.

Image Formats

Here you can specify which image formats should be checked during a search. Image files with the following file extensions are supported: 3FR, ARW, BMP, CR2, CRW, CUT, DCR, DIB, DNG, EMF, ERF, GIF, HDP, ICO, IFF, J2C, J2K, JP2, JPE, JPG, JPEG, JPX, JFIF, KDC, MDC, MEF, MOS, MRW, NEF, ORF, PEF, PBM, PCX, PGM, PNG, PPM, PSD, RAF, RAS, RAW, RW2, SRW, TGA, TIF, TIFF, RAS, RLE, WBMP, WEBP, WMF and X3F.

Percentage Match

Here you can specify the minimum percentage matching of two pictures. The calculated percentage matching between two pictures will be shown at the column Match. The percentage always refers to the reference picture of a group which will be shown in a different text color.

Picture Area

This option enables you to specify the picture area to be used to create the checksum. The following options are available:
  1. entire picture
  2. area at the upper left corner
  3. area at the upper right corner
  4. area at the lower left corner
  5. area at the lower right corner
When using the options 2, 3, 4 or 5, the size of the picture area can be specified in pixel. For example, a value of 16 pixel will be using a picture area with a width and height of 16 pixel at the selected corner.

Compare Size

Here you can specify the maximum width and height of the pictures to be compared. A lower compare size finds more similar pictures and speeds up the comparison time. A higher compare size finds more identical pictures and less similar pictures and of course needs more time to compare them.

Checksum

The size of the checksum in bits is displayed here. The checksum can only be changed when using the bHash comparison method.

Compare only pictures with the same properties

This option will be performed before the option Detect picture modifications.
  • File Name
    This option enables you to compare only pictures with the same file name.
  • File Extension
    This option enables you to compare only pictures with the same file extension.
  • Width and Height
    This option enables you to compare only pictures with the same width and height.
  • Orientation
    This option enables you to compare only pictures with the same orientation (portrait or landscape).
  • Aspect Ratio
    This option enables you to compare only pictures with the same aspect ratio. The calculation of the aspect ratio is done by the formula "width divided by height". The result of the calculation is truncated to one decimal place. A picture with 1920x1080 pixel has an aspect ratio of "1.7".

Detect picture modifications (slower)

This option enables you to detect different picture modifications when comparing two pictures. For this purpose, each picture modification will be performed with the picture to be compared and in each case an additional checksum will be created. The following picture modifications can be detected:
  • Rotated 90° to the right
  • Rotated 180° to the right
  • Rotated 90° to the left
  • Flipped horizontally
  • Rotated 90° to the right and flipped horizontally
  • Flipped vertically
  • Rotated 90° to the left and flipped horizontally

Recognition Rate Test

We have carried out tests with various comparison methods to determine the recognition rate for different image changes. For this purpose, 29 copies of a JPEG image (1600x1200px, 606KB) were created and their color, size and format changed. The tests always compared the original JPEG image and a modified copy of the image.

The following comparison methods were used in the test:
Comparison Method | Compare Size | Checksum
aHash¹ | 8x8 | 64-bit
aHash² | 16x16 | 256-bit
bHash¹ (fast) | 256x256 | 256-bit
bHash² (precise) | 256x256 | 256-bit
dHash | 16x16 | 256-bit
mHash | 16x16 | 256-bit
pHash | 32x32 | 64-bit

The percentage values in the table below show how well a comparison method recognizes the change in the image copy:
Image Modification | aHash¹ | aHash² | bHash¹ | bHash² | dHash | mHash | pHash
Image reduction to 75% | 100% | 100% | 100% | 100% | 100% | 100% | 97%
Image reduction to 50% | 100% | 100% | 100% | 100% | 99% | 100% | 100%
Image reduction to 25% | 100% | 100% | 100% | 100% | 99% | 100% | 97%
Image enlargement to 150% | 100% | 100% | 100% | 100% | 100% | 100% | 100%
Image enlargement to 200% | 100% | 100% | 100% | 100% | 100% | 100% | 100%
Conversion to grayscale | 97% | 98% | 97% | 93% | 97% | 96% | 97%
Brightness increased by 30% | 95% | 99% | 100% | 99% | 98% | 97% | 91%
Brightness decreased by 30% | 95% | 95% | 89% | 93% | 92% | 94% | 91%
Contrast increased by 30% | 100% | 99% | 95% | 96% | 96% | 96% | 94%
Contrast decreased by 30% | 100% | 99% | 100% | 100% | 98% | 100% | 94%
JPEG quality reduced by 25% | 100% | 100% | 100% | 100% | 100% | 100% | 100%
JPEG quality reduced by 50% | 100% | 100% | 100% | 100% | 99% | 100% | 100%
JPEG quality reduced by 75% | 100% | 100% | 100% | 100% | 99% | 100% | 94%
Image rotated 90° to the left | 47% | 52% | 68% | 54% | 46% | 50% | 47%
Image rotated 90° to the right | 47% | 52% | 54% | 55% | 48% | 50% | 44%
Image flipped vertically | 81% | 81% | 77% | 73% | 71% | 80% | 50%
Image flipped horizontally | 69% | 59% | 62% | 64% | 54% | 61% | 47%
Added white frame (30px) | 84% | 82% | 68% | 61% | 86% | 82% | 88%
Added black frame (30px) | 94% | 94% | 92% | 89% | 92% | 92% | 88%
Image height reduced to 80% | 100% | 100% | 100% | 100% | 100% | 100% | 100%
Image width reduced to 80% | 100% | 100% | 100% | 100% | 99% | 100% | 100%
Conversion to PNG 24bit | 100% | 100% | 100% | 100% | 100% | 100% | 100%
Conversion to BMP 24bit | 100% | 100% | 100% | 100% | 100% | 100% | 100%
Conversion to GIF 256 colors | 100% | 100% | 100% | 99% | 99% | 100% | 97%
Cropped 100 pixel from the left edge | 92% | 91% | 88% | 71% | 82% | 92% | 78%
Cropped 100 pixel from the right edge | 97% | 92% | 86% | 86% | 85% | 90% | 91%
Cropped 100 pixel from the top edge | 94% | 88% | 86% | 83% | 86% | 89% | 84%
Cropped 100 pixel from the bottom edge | 98% | 91% | 87% | 86% | 86% | 91% | 84%
Cropped 100 pixel from all edges | 89% | 80% | 77% | 75% | 70% | 80% | 62%

The following table gives you an overview of how many duplicates were found with the various comparison methods in the test. The evaluation was carried out with a Percentage Match of at least 70%, 80% and 90%.
Comparison Method | 70% | 80% | 90%
aHash¹ (64-bit) | 26 | 26 | 23
aHash² (256-bit) | 26 | 26 | 22
bHash¹ (fast) | 25 | 23 | 18
bHash² (precise) | 25 | 22 | 18
dHash (256-bit) | 26 | 24 | 19
mHash (256-bit) | 26 | 26 | 22
pHash (64-bit) | 26 | 24 | 19

Examples

The displayed search results were created with the following settings:
  • Comparison method: aHash
  • Compare size: 16x16
  • Checksum: 256-bit
All images used are from the Caltech101 image collection.

Different shots of the same person
similar pictures of a woman similar pictures of a man

Shots of different motorcycles
similar pictures of motorcycles

The same object in a different pose
similar pictures of an moving object

Different partial shots of the same subject
similar pictures from partial shots

False positive duplicates
false-positive duplicate pictures


Find Similar Audio Files

This search method enables you to find similar or almost identical audio files between the file formats MP3, FLAC, WMA, WAV, OGG, APE, AAC, MKA, M4A, M4R and MP2. The audio stream of each audio file will be analyzed and a checksum is calculated for further comparison with other audio files.

The following files will be automatically excluded from the search:
  • Files without the extension MP3, FLAC, WMA, WAV, OGG, APE, AAC, AIF, AIFF, MKA, M4A, M4R and MP2
  • Files with a size of 0 bytes
  • Corrupted, invalid or incomplete audio files (*)
  • Files with a blocked read access (*)
(*) An error message will be shown at the log pane.

Scan length

Here you can specify how many seconds at the beginning of the audio file should be used to calculate the checksum. A value between 15 and 999 seconds can be set for the scan length. The standard value for the scan length is 30 seconds.

Percentage Match

Here you can specify the minimum percentage matching of two audio files. The calculated percentage matching between two audio files will be shown at the column Match. The percentage always refers to the reference audio file of a group which will be shown in a different text color.

Comparison Method

Here you can specify the comparison method to be used to generate the checksum. The standard value for this option is 4.
  • 1 - no information available
  • 2 - no information available
  • 4 - Ignores leading silence
  • 5 - no information available

Compare only files with the same file extension

This option enables you to compare only audio files with the same file extension.

Recognition Rate

We have done several tests to determine the recognition rate for similar or almost identical audio files using different options. For the tests we have converted a WAV file (1411 Kbits/s 16bit 44100kHz Stereo) into different audio formats. All new audio files have been converted with stereo sound unless explicitly described otherwise. The percent match values at the following tables will give you an overview which options are the best to detect duplicates between the different audio formats. The title of the column header with highest percent matches in a column was set into square brackets.

Test 'Comparison Methods'

At this test all audio files were compared 4 times with one base file using all available comparison methods (1, 2, 4 and 5). A scan length of 30 seconds was used by this test.

Base file: WAV 16bit 44100 kHz
 | File size | 1 | 2 | [4] | 5
AAC 128kbps 44100kHz | 2,42 MB | 79% | 79% | 77% | 75%
AAC 192kbps 48000kHz | 3,12 MB | 82% | 81% | 78% | 74%
AAC 256kbps 48000kHz | 3,12 MB | 82% | 81% | 78% | 74%
AAC 48000kHz 32bit | 2,46 MB | 74% | 77% | 83% | 58%
AAC 64kbps 22050kHz mono | 0,72 MB | 73% | 72% | 73% | 68%
AAC 96kbps 22050kHz | 1,43 MB | 76% | 74% | 74% | 70%
APE 48000kHz 16bit | 15,84 MB | 87% | 86% | 92% | 71%
FLAC 22050kHz 16bit mono | 5,14 MB | 100% | 100% | 100% | 100%
FLAC 44100kHz 16bit | 16,93 MB | 100% | 100% | 100% | 100%
FLAC 48000kHz 16bit | 17,38 MB | 100% | 100% | 100% | 100%
M4A 128kbps 44100kHz | 2,42 MB | 79% | 79% | 77% | 75%
M4A 192kbps 48000kHz | 3,12 MB | 82% | 81% | 78% | 74%
M4A 256kbps 48000kHz | 3,12 MB | 82% | 81% | 78% | 74%
M4A 64kbps 22050kHz mono | 0,72 MB | 73% | 72% | 73% | 68%
M4A 96kbps 22050kHz | 1,43 MB | 76% | 74% | 74% | 70%
M4R 160kbps 44100kHz | 0,76 MB | 94% | 95% | 90% | 95%
M4R 256kbps 44100kHz | 1,21 MB | 95% | 96% | 96% | 97%
MKA 48000kHz 32bit | 4,72 MB | 82% | 80% | 83% | 68%
MP2 128kbps 44100kHz | 2,40 MB | 84% | 86% | 94% | 69%
MP2 160kbps 44100kHz | 3,00 MB | 84% | 86% | 96% | 69%
MP2 256kbps 44100kHz | 4,79 MB | 84% | 86% | 99% | 70%
MP2 64kbps 22050kHz mono | 1,20 MB | 76% | 76% | 99% | 58%
MP3 128kbps 44100kHz | 2,40 MB | 91% | 91% | 92% | 87%
MP3 192kbps 48000kHz | 3,59 MB | 96% | 95% | 96% | 96%
MP3 256kbps 48000kHz | 4,79 MB | 98% | 99% | 98% | 97%
MP3 320kbps 48000kHz | 5,99 MB | 99% | 99% | 100% | 98%
MP3 96kbps 22050kHz mono | 1,80 MB | 97% | 95% | 97% | 95%
MP3 VBR0 44100kHz | 4,24 MB | 66% | 67% | 96% | 54%
MP3 VBR1 44100kHz | 3,88 MB | 67% | 67% | 97% | 54%
MP3 VBR5 44100kHz | 2,68 MB | 65% | 66% | 90% | 53%
MP3 VBR9 44100kHz | 2,40 MB | 66% | 67% | 96% | 54%
OGG 128kbps 44100kHz | 2,17 MB | 93% | 90% | 90% | 87%
OGG 160kbps 44100kHz | 2,17 MB | 93% | 90% | 90% | 87%
OGG 240kbps 48000kHz | 3,31 MB | 96% | 94% | 95% | 94%
OGG 320kbps 48000kHz | 4,48 MB | 97% | 96% | 98% | 96%
OGG 64kbps 22050kHz mono | 1,19 MB | 94% | 93% | 93% | 93%
WAV 8bit 22050kHz mono | 3,30 MB | 99% | 100% | 93% | 98%
WMA 128kbps 44100kHz | 2,43 MB | 53% | 54% | 93% | 49%
WMA 192kbps 48000kHz | 3,64 MB | 54% | 55% | 93% | 50%
WMA 256kbps 48000kHz | 4,85 MB | 98% | 97% | 97% | 97%
WMA 384kbps 48000kHz | 4,85 MB | 98% | 97% | 97% | 97%
WMA 96kbps 22050kHz mono | 0,39 MB | 54% | 54% | 73% | 49%

Base file: MP3 192kbps 48000 kHz
 | File size | [1] | 2 | 4 | 5
AAC 128kbps 44100kHz | 2,42 MB | 79% | 77% | 76% | 75%
AAC 192kbps 48000kHz | 3,12 MB | 81% | 80% | 78% | 75%
AAC 256kbps 48000kHz | 3,12 MB | 81% | 80% | 78% | 75%
AAC 48000kHz 32bit | 2,46 MB | 74% | 76% | 82% | 57%
AAC 64kbps 22050kHz mono | 0,72 MB | 74% | 72% | 73% | 67%
AAC 96kbps 22050kHz | 1,43 MB | 75% | 75% | 74% | 69%
APE 48000kHz 16bit | 15,84 MB | 86% | 85% | 91% | 71%
FLAC 22050kHz 16bit mono | 5,14 MB | 96% | 95% | 96% | 96%
FLAC 44100kHz 16bit | 16,93 MB | 96% | 95% | 96% | 96%
FLAC 48000kHz 16bit | 17,38 MB | 96% | 95% | 96% | 96%
M4A 128kbps 44100kHz | 2,42 MB | 79% | 77% | 76% | 75%
M4A 192kbps 48000kHz | 3,12 MB | 81% | 80% | 78% | 75%
M4A 256kbps 48000kHz | 3,12 MB | 81% | 80% | 78% | 75%
M4A 64kbps 22050kHz mono | 0,72 MB | 74% | 72% | 73% | 67%
M4A 96kbps 22050kHz | 1,43 MB | 75% | 75% | 74% | 69%
M4R 160kbps 44100kHz | 0,76 MB | 93% | 94% | 89% | 94%
M4R 256kbps 44100kHz | 1,21 MB | 94% | 93% | 93% | 94%
MKA 48000kHz 32bit | 4,72 MB | 80% | 81% | 82% | 68%
MP2 128kbps 44100kHz | 2,40 MB | 83% | 86% | 93% | 69%
MP2 160kbps 44100kHz | 3,00 MB | 84% | 85% | 94% | 69%
MP2 256kbps 44100kHz | 4,79 MB | 84% | 85% | 96% | 69%
MP2 64kbps 22050kHz mono | 1,20 MB | 76% | 76% | 96% | 57%
MP3 128kbps 44100kHz | 2,40 MB | 91% | 90% | 89% | 86%
MP3 256kbps 48000kHz | 4,79 MB | 97% | 96% | 96% | 95%
MP3 320kbps 48000kHz | 5,99 MB | 96% | 96% | 96% | 95%
MP3 96kbps 22050kHz mono | 1,80 MB | 95% | 94% | 95% | 93%
MP3 VBR0 44100kHz | 4,24 MB | 66% | 67% | 93% | 54%
MP3 VBR1 44100kHz | 3,88 MB | 67% | 66% | 94% | 54%
MP3 VBR5 44100kHz | 2,68 MB | 65% | 66% | 90% | 53%
MP3 VBR9 44100kHz | 2,40 MB | 66% | 67% | 94% | 54%
OGG 128kbps 44100kHz | 2,17 MB | 91% | 90% | 89% | 87%
OGG 160kbps 44100kHz | 2,17 MB | 91% | 90% | 89% | 87%
OGG 240kbps 48000kHz | 3,31 MB | 94% | 92% | 93% | 91%
OGG 320kbps 48000kHz | 4,48 MB | 95% | 94% | 94% | 94%
OGG 64kbps 22050kHz mono | 1,19 MB | 93% | 92% | 91% | 93%
WAV 16bit 44100kHz | 26,41 MB | 96% | 95% | 96% | 96%
WAV 8bit 22050kHz mono | 3,30 MB | 96% | 96% | 90% | 96%
WMA 128kbps 44100kHz | 2,43 MB | 54% | 54% | 92% | 49%
WMA 192kbps 48000kHz | 3,64 MB | 54% | 55% | 92% | 50%
WMA 256kbps 48000kHz | 4,85 MB | 95% | 96% | 96% | 94%
WMA 384kbps 48000kHz | 4,85 MB | 95% | 96% | 96% | 94%
WMA 96kbps 22050kHz mono | 0,39 MB | 54% | 54% | 74% | 49%

Base file: MP3 VBR0 44100 kHz
 | File size | 1 | 2 | [4] | 5
AAC 128kbps 44100kHz | 2,42 MB | 66% | 64% | 77% | 52%
AAC 192kbps 48000kHz | 3,12 MB | 65% | 66% | 77% | 52%
AAC 256kbps 48000kHz | 3,12 MB | 65% | 66% | 77% | 52%
AAC 48000kHz 32bit | 2,46 MB | 78% | 77% | 84% | 63%
AAC 64kbps 22050kHz mono | 0,72 MB | 62% | 62% | 73% | 51%
AAC 96kbps 22050kHz | 1,43 MB | 62% | 62% | 75% | 52%
APE 48000kHz 16bit | 15,84 MB | 63% | 62% | 91% | 52%
FLAC 22050kHz 16bit mono | 5,14 MB | 66% | 67% | 96% | 54%
FLAC 44100kHz 16bit | 16,93 MB | 66% | 67% | 96% | 54%
FLAC 48000kHz 16bit | 17,38 MB | 66% | 67% | 96% | 54%
M4A 128kbps 44100kHz | 2,42 MB | 66% | 64% | 77% | 52%
M4A 192kbps 48000kHz | 3,12 MB | 65% | 66% | 77% | 52%
M4A 256kbps 48000kHz | 3,12 MB | 65% | 66% | 77% | 52%
M4A 64kbps 22050kHz mono | 0,72 MB | 62% | 62% | 73% | 51%
M4A 96kbps 22050kHz | 1,43 MB | 62% | 62% | 75% | 52%
M4R 160kbps 44100kHz | 0,76 MB | 66% | 68% | 91% | 54%
M4R 256kbps 44100kHz | 1,21 MB | 66% | 68% | 95% | 54%
MKA 48000kHz 32bit | 4,72 MB | 62% | 63% | 82% | 51%
MP2 128kbps 44100kHz | 2,40 MB | 72% | 73% | 92% | 57%
MP2 160kbps 44100kHz | 3,00 MB | 72% | 73% | 94% | 57%
MP2 256kbps 44100kHz | 4,79 MB | 73% | 74% | 96% | 57%
MP2 64kbps 22050kHz mono | 1,20 MB | 79% | 81% | 96% | 63%
MP3 128kbps 44100kHz | 2,40 MB | 66% | 66% | 90% | 54%
MP3 192kbps 48000kHz | 3,59 MB | 66% | 67% | 93% | 54%
MP3 256kbps 48000kHz | 4,79 MB | 66% | 68% | 95% | 54%
MP3 320kbps 48000kHz | 5,99 MB | 66% | 68% | 96% | 54%
MP3 96kbps 22050kHz mono | 1,80 MB | 66% | 67% | 94% | 54%
MP3 VBR1 44100kHz | 3,88 MB | 95% | 96% | 96% | 93%
MP3 VBR5 44100kHz | 2,68 MB | 91% | 92% | 92% | 86%
MP3 VBR9 44100kHz | 2,40 MB | 94% | 95% | 95% | 94%
OGG 128kbps 44100kHz | 2,17 MB | 65% | 67% | 90% | 53%
OGG 160kbps 44100kHz | 2,17 MB | 65% | 67% | 90% | 53%
OGG 240kbps 48000kHz | 3,31 MB | 66% | 67% | 93% | 53%
OGG 320kbps 48000kHz | 4,48 MB | 67% | 68% | 96% | 54%
OGG 64kbps 22050kHz mono | 1,19 MB | 66% | 67% | 92% | 53%
WAV 16bit 44100kHz | 26,41 MB | 66% | 67% | 96% | 54%
WAV 8bit 22050kHz mono | 3,30 MB | 66% | 68% | 93% | 54%
WMA 128kbps 44100kHz | 2,43 MB | 59% | 64% | 91% | 51%
WMA 192kbps 48000kHz | 3,64 MB | 62% | 66% | 90% | 52%
WMA 256kbps 48000kHz | 4,85 MB | 66% | 67% | 95% | 54%
WMA 384kbps 48000kHz | 4,85 MB | 66% | 67% | 95% | 54%
WMA 96kbps 22050kHz mono | 0,39 MB | 58% | 61% | 75% | 51%

Test 'Scan length'

At this test all audio files were compared 4 times with one base file using the different scan lengths 10, 30, 60 and 90 seconds and the comparison method number 4.

Base file: MP3 192kbps 48000 kHz
 | File size | [10s] | 30s | 60s | 90s
AAC 128kbps 44100kHz | 2,42 MB | 78% | 76% | 76% | 76%
AAC 192kbps 48000kHz | 3,12 MB | 77% | 78% | 76% | 76%
AAC 256kbps 48000kHz | 3,12 MB | 77% | 78% | 76% | 76%
AAC 48000kHz 32bit | 2,46 MB | 82% | 82% | 82% | 82%
AAC 64kbps 22050kHz mono | 0,72 MB | 76% | 73% | 71% | 71%
AAC 96kbps 22050kHz | 1,43 MB | 74% | 74% | 72% | 72%
APE 48000kHz 16bit | 15,84 MB | 92% | 91% | 90% | 90%
FLAC 22050kHz 16bit mono | 5,14 MB | 96% | 96% | 96% | 96%
FLAC 44100kHz 16bit | 16,93 MB | 96% | 96% | 96% | 96%
FLAC 48000kHz 16bit | 17,38 MB | 96% | 96% | 96% | 96%
M4A 128kbps 44100kHz | 2,42 MB | 78% | 76% | 76% | 76%
M4A 192kbps 48000kHz | 3,12 MB | 77% | 78% | 76% | 76%
M4A 256kbps 48000kHz | 3,12 MB | 77% | 78% | 76% | 76%
M4A 64kbps 22050kHz mono | 0,72 MB | 76% | 73% | 71% | 71%
M4A 96kbps 22050kHz | 1,43 MB | 74% | 74% | 72% | 72%
M4R 160kbps 44100kHz | 0,76 MB | 89% | 89% | 88% | 88%
M4R 256kbps 44100kHz | 1,21 MB | 95% | 93% | 94% | 94%
MKA 48000kHz 32bit | 4,72 MB | 86% | 82% | 81% | 81%
MP2 128kbps 44100kHz | 2,40 MB | 94% | 93% | 93% | 93%
MP2 160kbps 44100kHz | 3,00 MB | 94% | 94% | 94% | 94%
MP2 256kbps 44100kHz | 4,79 MB | 95% | 96% | 96% | 96%
MP2 64kbps 22050kHz mono | 1,20 MB | 96% | 96% | 96% | 96%
MP3 128kbps 44100kHz | 2,40 MB | 91% | 89% | 89% | 89%
MP3 256kbps 48000kHz | 4,79 MB | 96% | 96% | 95% | 95%
MP3 320kbps 48000kHz | 5,99 MB | 97% | 96% | 96% | 96%
MP3 96kbps 22050kHz mono | 1,80 MB | 96% | 95% | 95% | 95%
MP3 VBR0 44100kHz | 4,24 MB | 92% | 93% | 94% | 94%
MP3 VBR1 44100kHz | 3,88 MB | 93% | 94% | 93% | 93%
MP3 VBR5 44100kHz | 2,68 MB | 87% | 90% | 91% | 91%
MP3 VBR9 44100kHz | 2,40 MB | 93% | 94% | 93% | 93%
OGG 128kbps 44100kHz | 2,17 MB | 83% | 89% | 89% | 89%
OGG 160kbps 44100kHz | 2,17 MB | 83% | 89% | 89% | 89%
OGG 240kbps 48000kHz | 3,31 MB | 94% | 93% | 94% | 94%
OGG 320kbps 48000kHz | 4,48 MB | 91% | 94% | 94% | 94%
OGG 64kbps 22050kHz mono | 1,19 MB | 87% | 91% | 92% | 92%
WAV 16bit 44100kHz | 26,41 MB | 96% | 96% | 96% | 96%
WAV 8bit 22050kHz mono | 3,30 MB | 91% | 90% | 91% | 91%
WMA 128kbps 44100kHz | 2,43 MB | 93% | 92% | 92% | 92%
WMA 192kbps 48000kHz | 3,64 MB | 93% | 92% | 92% | 92%
WMA 256kbps 48000kHz | 4,85 MB | 96% | 96% | 96% | 96%
WMA 384kbps 48000kHz | 4,85 MB | 96% | 96% | 96% | 96%
WMA 96kbps 22050kHz mono | 0,39 MB | 78% | 74% | 72% | 72%


Search Options

Before the search starts

Confirm the use of the file and folder filter
This option prompts you to use the file and folder filters before the search starts.

Set the CPU priority for AllDup to 'low'
Activate this option if your CPU is overburdened while executing a search and other applications take too long to respond to user input. AllDup's CPU priority is set back to 'normal' when completing the search.

During the search process

Determine the file owner (slower)
This option causes to determine the file owner of each file and show it at the search result in a separate column. With this option deactivated you also will see the file owner at the search result when clicking on a file or just moving the cursor over the file owner column.

Determine the hard link count for each file (slower)
This option causes to determine the hard link count for each file. The hard link count will be shown at column 'hard links'. You can manually determine the hard link count by clicking on a file or moving the cursor over the column 'hard links' at the search result.

Determine picture dimension (slower)
This option causes to determine the picture dimension of each image file and show it at the search result in the columns 'Width' and 'Height'. With this option deactivated you also will see the picture dimension at the search result when clicking on a file or just moving the cursor over the columns 'Width' or 'Height'.

Determine duration, bit rate and sample rate of files with the following extensions (slower)'
This option causes to determine the duration, bit rate and sample rate of audio files using the PlugIn 'MediaInfo'. With this option deactivated you also will see the duration, bit rate and sample rate at the search result when clicking on a file or just moving the cursor over the columns 'Duration', 'Bit Rate' or 'Sample Rate'.

Log every x minutes the search statistic of the scan progress info
This option enables you to log the search statistic shown up at the progress info for analytical purposes.

Abort the search operation after a certain amount of duplicates was found
This option enables you to abort the search operation automatically after a certain amount of duplicates was found.

After the search is complete

Expand all groups
This option expand all groups after the search is complete.

Remove all groups where the files not exist at all source folders
This option removes all groups where the files not exist at all source folders.

Show only files without duplicates at the search result
This option causes to show only files without duplicates at the search result.

Log all files without duplicates (slower)
This option causes to log all files without duplicates.

Scan Archive Files

This option enables to scan the content of archive files for duplicates. Single archive files with the following file extensions are supported:
  • .7z
  • .arj
  • .cab
  • .cb7
  • .cbr
  • .cbt
  • .cbz
  • .epub
  • .gz
  • .iso
  • .lzh
  • .lha
  • .nrg
  • .rar
  • .tar
  • .zip

Splitted archive files with the following file extensions are supported:
  • .zip (subsequent file: .z01)
  • .zip.001
  • .rar (subsequent file: .r00)
  • .part1.rar
  • .7z.001
  • .tar.001

Notes:
  • Password-protected files from inside an archive and password-protected archive files will be automatically filtered when using the search method 'File content', 'Similar pictures' or 'Similar music'
  • AllDup's file manager can only delete files from inside archives with the extension .zip or .7z.

Delete temporary files after their checksum is created
This option enables you to delete temporary files directly after their checksum was created. This has the advantage of a lower disk space usage while scanning many archive files.
This option will only be used in the following circumstances:
  • The search method Similar Pictures is used and the option 'Detect picture modifications' is deactivated.
  • The search method File Content is used and a checksum method is selected at the compare method.

Log password-protected files
This option enables you to log password-protected files from inside an archive and password-protected archive files.

Don't extract files with a size equal or more than...
This option enables to ignore files inside an archive with a size equal or larger than specified.


Archive Files

Scan archive files with the following file extensions

This option enables you to scan the content of archive files for duplicate files. Single archive files with the following file extensions are supported:
  • .7z
  • .arj
  • .cab
  • .cb7
  • .cbr
  • .cbt
  • .cbz
  • .epub
  • .gz
  • .iso
  • .lzh
  • .lha
  • .nrg
  • .rar
  • .tar
  • .zip

Splitted archive files with the following file extensions are supported:
  • .zip (subsequent file: .z01)
  • .zip.001
  • .rar (subsequent file: .r00)
  • .part1.rar
  • .7z.001
  • .tar.001

Notes:
  • Password-protected files from inside an archive and password-protected archive files will be automatically filtered when using the search method File Content, Similar Pictures or Similar music.
  • AllDup's file manager can only delete files from inside archives with the extension .zip or .7z.

Delete temporary files after their checksum is created

This option enables you to delete temporary files directly after their checksum was created. This has the advantage of a lower disk space usage while scanning many archive files.
This option will only be used in the following circumstances:
  • The search method Similar Pictures is used and the option 'Detect picture modifications' is deactivated.
  • The search method File Content is used and a checksum method is selected at the compare method.

Log password-protected files

This option enables you to log password-protected files from inside an archive and password-protected archive files.

Don't extract files with a size equal or more than...

This option enables to ignore files inside an archive with a size equal or larger than specified.


Database

This feature enables you to store the calculated checksums during a scan for duplicate files into a database. That has the advantage that further file scans requires considerably less time because of the reuse of the stored checksums.

The storage of the checksums can be activated for the following search methods:
  • File content
    A separate checksum is stored in the database for each comparison method. With 5 comparison methods, up to 5 different checksums can be stored in the database per file.
  • Similar pictures
    A separate checksum is stored in the database for each comparison method and checked picture modification. With 9 comparison methods and 7 additional checks for possible picture modifications, up to 72 different checksums can be stored in the database for each picture.
  • Similar audio files
    A separate checksum is stored in the database for each scan length and comparison method used.
Note: The database is not available on Windows XP.

Options

Recalculate checksum if the file size was changed
The checksum of a file will be recalculated if the current file size not match with the stored file size in the database.

Recalculate checksum if the file modified date was changed
The checksum of a file will be recalculated if the current modified date of the file not match with the stored modified date in the database.

Database Tools

Cleanup
Checks the existence of all stored files in the database and removes the files not exist anymore from the database. The process can be canceled with the F10 key.

Reset
Deletes all checksums in the database.

New
Deletes the existing database file and creates a new database file.


File Filters

File filters enable you to filter specific files during a search operation, i.e. exclude them from the search operation. The filtering methods inclusive and exclusive enable you to determine how filter texts are to be applied during a search operation.

Inclusive

Only files matching an activated filter text are included in the search operation. Example: If you specify the filter text "*.ini" only files with the ".ini" extension are processed during a search operation. All other files are filtered (= excluded) from the search operation.

Exclusive

All the files matching a filter text are excluded from the search operation. Example: If you specify the filter text "*.ini", all files with the ".ini" extension are filtered, i.e. excluded from the search operation.

Filter List

The filter list contains all of the filter texts set up for files. Each filter text can be enabled or disabled via its checkbox. Click on the button File Filters at the toolbar to open a menu to add new filters. Right-click an item at the filter list to edit or remove a filter.

Filter Texts

Comparisons using the filter text are always done using the file name. When a filter text contains a backslash (\) comparison is made with the entire path of the current file. Filters are not case-sensitive, meaning no distinction is made between uppercasing and lowercasing when performing comparisons. The filter text may contain various wildcards and Windows environment variables. You can specify multiple file filters using the special character '|' as a separator.

Examples:
  • *.txt
  • temp.*
  • *temp.txt
  • temp??.*
  • temp.00?
  • %USERNAME%.txt
  • *\Temp\*.tmp
  • C:\Temp\*.ini
  • C:\Temp\%Username%.*
  • *.bmp|*.jpg|*.jpeg|*.png

Options

Ignore wildcards in filter text (treat like normal text)

This causes the characters of wildcards in a filter text to be treated as normal text.

Make filtering case-sensitive

This makes filtering distinguish between uppercasing and lowercasing when comparing file names.

File Properties

Ignore 0-Byte files
This option enables to ignore files with a size of 0 bytes.

Exclude files by size
This option enables you to exclude files on the basis of the size.

Exclude files by date
This option enables you to exclude files on the basis of the created/modified date.


Folder Filters

Folder filters enable you to filter specific folders during a search operation, i.e. exclude them from the search operation. The filtering methods inclusive and exclusive enable you to determine how filter texts are to be applied during a search operation.

Inclusive

Only folders matching an activated filter text are included during a search operation. Example: If you specify the filter text "Test" only folders named 'Test' are searched during a search operation. All other folders are filtered (= excluded) from the copy operation.

Exclusive

All the folders matching a filter text are excluded from the search operation. Example: If you specify the filter text "Test", all folders named 'Test' and their subfolders are filtered, i.e. excluded from the search operation.

Filter List

The filter list contains all of the filter texts set up for folders. Each filter text can be enabled or disabled for a copy operation via its checkbox. Click on the button Folder Filters at the toolbar to open a menu to add new filters. Right-click an item at the filter list to edit or remove a filter.

Filter Texts

Comparisons using a filter text are always done using the folder name. When a filter text contains a backslash (\) comparison is made with the entire path of the current folder. Filters are not case-sensitive, meaning no distinction is made between uppercasing and lowercasing when during comparisons. The filter text may contain various wildcards and Windows environment variables. You can specify multiple folder filters using the special character '|' as a separator.

Examples:
  • C:\Windows
  • C:\Prog*
  • temp??
  • C:\users\%USERNAME%
  • C:\Windows|C:\Users|C:\Temp

Options

Ignore wildcards in filter text (treat like normal text)
This causes the characters of wildcards in a filter text to be treated as normal text.

Make filtering case-sensitive
This makes filtering distinguish between uppercasing and lowercasing when comparing folder names.

Don't filter subfolders
This option doesn't have any effect unless you are using the inclusive filter method and filter the entire path. Then all subfolders are taken into consideration during a search operation and aren't excluded.


Windows Environment Variables

Windows environment variables can be used at the file and folder filters.

Syntax:

%Environment Variable%

Example:

The environment variable %USERNAME% is replaced with the user logged on. Example: The folder "C:\%USERNAME%" becomes "C:\Admin".


File Preview

The Plugins of the file preview enables you to view many file types directly at the search result without open them with their associated application. The file extensions at a PlugIn list can be selected or unselected by clicking on the checkbox in front of them. Using the right mouse button at a PlugIn list will show up a context menu with the following commands:
  • Add
    Adds a new file extension to the PlugIn list.
  • Remove
    Removes a file extension from the PlugIn list.
  • Select all
    Selects all file extensions at the PlugIn list.
  • Deselect all
    Deselects all file extensions at the PlugIn list.
  • Default values
    Clears the list and adds the default file extensions for this PlugIn to the list.

PlugIn's

  • Windows WordPad
  • Windows Internet Explorer
  • Windows Media Player
  • Image Viewer
  • VLC Media Player
  • EXIF Viewer
  • Media Info Viewer

PlugIn: Windows WordPad

With this PlugIn you can view all files at the file preview pane that are supported by Microsoft WordPad or any file with text content inside. For example, files with the following extensions can be viewed with this PlugIn:

TXT, BAT, INI, LOG, CFG, DIZ, NFO, CSS, ASP, PHP, RTF, WRI, XML, INF, URL, REG, LST, EML, ERR, CSV, 1ST, VBS...

PlugIn: Windows Internet Explorer

With this PlugIn you can view all files at the file preview pane that are supported by the Windows Internet Explorer. This PlugIn requires an existing installation of the Windows Internet Explorer. For example, files with the following extensions can be viewed with this PlugIn at the file preview pane:

BMP, CAB, DIB, EMF, GIF, ICO, PNG, BMP, JPEG, JPG, HTM, HTML, PDF¹, DOC², DOCX², XLS², XLSX², PPT², PPS², SVG, SWF, TXT, WMF and ZIP

¹ requires the installation of a PDF Reader
² requires the installation of Microsoft Office

PlugIn: Windows Media Player

With this PlugIn you can view all audio and video files at the file preview pane that are supported by the Windows Media Player. This PlugIn requires the installation of the Windows Media Player. For example, files with the following extensions can be viewed with this PlugIn at the file preview pane:

AU, AIF, AIFC, AIFF, ASF, AVI, M1V, MID, MIDI, MOV, MP4, MP3, MPEG, MPG, MP2, MPA, MPE, RMI, SND, WAV, WM, WMA, WMV ...

PlugIn: Image Viewer

With this PlugIn you can view all image files with the following extensions at the file preview pane:

3FR, ARW, BMP, CR2, CRW, CUT, DCR, DIB, DNG, EMF, ERF, GIF, HDP, HEIC¹, HEIF¹, ICO, IFF, J2C, J2K, JP2, JPE, JPG, JPEG, JPX, JFIF, KDC, MDC, MEF, MOS, MRW, NEF, ORF, PEF, PBM, PCX, PGM, PNG, PPM, PSD, RAF, RAS, RAW, RW2, SRW, TGA, TIF, TIFF, RAS, RLE, WBMP, WEBP, WMF and X3F.

¹ An HEIC/HEIF image extension (e.g. CopyTrans HEIC) must be installed for these file types if the files cannot be displayed as a preview or thumbnail using Windows Explorer.

PlugIn: VLC Media Player

With this PlugIn you can view all audio and video files that are supported by the VLC Media Player at the file preview pane. For example, files with the following extensions can be viewed with this PlugIn at the file preview pane:

3G2, 3GA, 3GP, 3GP2, 3GPP, 669, A52, AAC, AC3, ADT, ADTS, AU, AIF, AIFC, AIFF, AMR, AMV, AOB, APE, ASF, ASX, AU, AVI, BIK, CAF, DIVX, DRC, DTS, DV, DVR-MS, EVO, F4V, FLAC, FLV, GVI, GXF, IT, M1V, M2T, M2TS, M2V, M4A, M4P, M4V, MID, MKA, MKV, MLP, MOD, MOV, MP1, MP2, MP2V, MP3, MP4, MP4V, MPA, MPC, MPE, MPEG, MPEG1, MPEG2, MPEG4, MPG, MPGA, MPV2, MTS, MTV, MXF, NSV, NUV, OGA, OGG, OGM, OGV, OGX, OMA, OPUS, QCP, RA, REC, RM, RMI, RMVB, RPL, S3M, SDP, SND, SPX, THP, TOD, TP, TS, TTA, TTS, VOB, VOC, VQF, VRO, W64, WAV, WEBM, WMA, WMV, WTV, WV, XA, XESC and XM.

This PlugIn requires the 32-bit version of the freeware VLC Media Player version 3 or higher. The 32-bit version of the VLC Media Player must be either installed at your windows system, available as a portable edition on your windows system or manually integrated into AllDup's installation folder.

Using the portable edition of the VLC Media Player:
Register the file axvlc.dll from inside the folder of the VLC Media Player at your Windows system using the following command: regsvr32.exe axvlc.dll

Manual integration of the VLC Media Player:
  1. Open the FTP archive at the official website of the VLC Media Player.
  2. Scroll down to the end of the FTP archive, click on the folder with the latest version number (e.g. "3.0.16") and then click on the folder "32bit".
  3. Download the file with the "zip" extension (e.g. "vlc-3.0.16-win32.zip").
  4. Extract the following files and folders from the ZIP file into the folder of AllDup:
    • files: axvlc.dll, libvlc.dll, libvlccore.dll
    • folder: plugins
  5. Run AllDup with admin rights once to register the file axvlc.dll at your Windows system automatically or register the file axvlc.dll at your Windows system manually using the command: regsvr32.exe axvlc.dll

PlugIn: EXIF Viewer

With this PlugIn you can view the EXIF data of files with the following extensions at the file preview pane: JPEG, JPG, TIF, TIFF, PNG, GIF und WAV. You can activate the PlugIn at the toolbar 'File Preview' using the button 'EXIF Viewer'.

PlugIn: Media Info Viewer

With the PlugIn MediaInfo you can view technical information about media files, as well as tag information for many audio and video files. You can activate the PlugIn at the toolbar 'File Preview' using the button 'Media Info Viewer'.


Profiles

Profiles enables you to save your search settings (e.g. source folders, search method, etc.) into a file and load these settings when needed. The profile list show you all available profile files. A right-click on a profile enables you to load, save, rename or delete a profile.
The standard profile is automatically loaded when starting AllDup. All settings at the main window will be saved to the standard profile when quitting AllDup.

Create a new profile

When creating a new profile all settings such as source folders and search method will be saved to the new profile.

Load a profile

Double-click a profile at the list to load it or use the right-click context menu action Load profile. The name of the loaded profile will be shown at the window title.

Save a profile

Use the right-click context menu action Save profile to save the current search settings into the selected profile. You have to load a profile and using the file manager to save the settings of the file manager into the profile.

Rename profile

Use the right-click context menu action Rename profile to rename a profile.

Delete profile

Use the right-click context menu action Delete profile to delete a profile.

Options

Don't import any source folders when loading a profile

This option enables you to prevent the loading of the stored source folders from a profile.


Stored Search Results

Here you will get a list of all stored search results. The list will be empty if no search results are available. Right-click on a search result inside the list will open a menu to load or remove a search result.


Options

General

Add AllDup to the context menu of the Windows Explorer
Activate this option so that you can start a search process with a folder via the context menu of Windows Explorer. Under Windows 11, AllDup must be started with explicit admin rights so that this option can be changed.

Unicode-Text Support
This option enables AllDup to display Unicode characters from languages such as Chinese, Japanese, Korean or Russian.

File Date Display Format
Here you can specify the display format of the file date at the search result.

Application data folder
Here you can view and open the folder were AllDup stores the application data.

Folder for temporary files
At this folder AllDup will extract files from archives or create files necessary for the file preview. All temporary files that have been created after the search for duplicate files was started will be automatically deleted after closing the search result.

User Interface

Here you can set the font and background color of the AllDup user interface.

Log

Delete entries from log file when they are more than ...
This option causes entries to be automatically removed from the log file after a specified number of days.

Search Result

External program for comparing the contents of two files:
This option enables you to specify an external file comparison utility to be used with two highlighted files at the search result. Right-click two highlighted files at the search result and use the command 'Compare highlighted files'. The two highlighted files from the search result have to be specified via the wildcards %1 and %2. Example: C:\Windiff.exe %1 %2

%1 = "C:\Data\db.txt"
%2 = "X:\Backup\abc.txt"
%3 = C:\Data
%4 = X:\Backup
%5 = db.txt
%6 = abc.txt

Compare these file types using Microsoft Word:
This option enables you to specify various file extensions that will be opened with the compare function of Microsoft Word. Each of the file extensions has to be separated by a comma. Example: doc,docx,rtf

External Program to open a folder
This option enables you to specify an external program to be used to open a folder.
Example: explorer.exe "%1"
The placeholder %1 will be replaced with file path & file name and the placeholder %2 with the file path only:
%1 = "C:\Data\db.txt"
%2 = "C:\Data\

External Program to open with file(s)
This option enables you to specify an external program to be used to open with all highlighted files.
Example: C:\texteditor.exe %1
The placeholder %1 with be replaced with all highlighted files: "file1" "file2" ...


AllDup Internet Update

AllDup's Update Wizard enables you to check whether a new version of AllDup is available for download and to install it, if necessary.

Version

Clicking on the Check button causes a check to see whether a new version is available. Clicking on Download causes the new version to be downloaded and installed. Information about a new version are displayed at the log window.

Options

Automatically check for new version
This option causes a check to automatically be conducted for a new version of AllDup on the AllDup website every time you open the Internet Update dialog.

Show Setup dialog when installing an update
Not activating this option cause updating to take place in the background, without any dialog being popped up.

Download Folder
Here you can specify the folder in which the update file for the new version should be saved

Proxy

If necessary, you can specify the access data for your proxy server here.

Data Privacy

When checking for a new version, only a text file containing information is downloaded from our website and evaluated. No private data on your computer is transmitted via the Internet.


Search Result

Groups

All duplicate files will be sorted into groups and displayed at the search result upon completion of the search. Each group contain files that all match together by the search method you have chosen. Each group has one reference file displayed with different text color. All other files inside the group are duplicates of the reference file.

Sorting groups and files

The content of the search result can be sorted by clicking with the left or right mouse button on a column header. Clicking with the left mouse button on a column header will sort only the group names but not the content of the groups. Clicking with the right mouse button on a column header will sort the files inside every group.

Select files

All files at the search result can be selected or unselected by clicking the checkbox next to the file name. You also can use the automatic selection commands at the Menu bar or Toolbar.

Highlight files or groups

You can highlight multiple files or groups by holding down the CTRL key and clicking on the files you would like to highlight. Clicking with the right mouse button on the highlighted files or groups will open a context menu with several commands.

Delete, copy, move or rename files

Open the File Manager via the toolbar or menu bar to process all selected files from the search result. You also can delete, copy or move files without using the file manager by holding the CTRL key and clicking with the right mouse button on a file.

Mouse commands

  • Clicking with the right mouse button on a group or file will open context menu with several commands.
  • Holding the CTRL key and clicking with the right mouse button on a file will open a context menu with several file commands.
  • Clicking with the middle mouse button on a file will show the Windows explorer context menu for this file.
  • A double click on a file name opens the file with the associated application.
  • A double click on a file path opens the file path with the Windows explorer.

Keyboard Shortcuts

The following keyboard shortcuts are available:
  • With the space bar you can select or unselect a highlighted file or group.
  • The F2 key enables you to rename a highlighted file.
  • The DEL key enables you to remove all highlighted groups and files from the search result.
  • Holding the CTRL key and pressing the DEL key will delete a highlighted file.
You can create your own keyboard shortcuts for all the commands at the menu bar, toolbar or context menus by using the option Hot Keys at the menu bar.


Menu Bar / Toolbar

  • File
  • Columns
  • Filter
  • Groups
  • Mode
  • Select
  • Deselect
  • Remove
  • File Protection
  • Toolbars
  • Options
Hint: The toolbar context menu (right-click) enables you to hide or show buttons.

File

  • File Manager - Delete, copy, move or rename selected files.
  • Save search result
  • Save as...
  • Export search result
  • Print search result
  • Log
  • Save and close search result
  • Save search result and exit AllDup
  • Close search result
  • Exit AllDup

Columns

Here you can show, hide or resize the columns at the search result.
  • Show column 'Path'
  • Show column 'Archive Path'
  • Show column 'Size'
  • Show column 'Last Modified Date'
  • Show column 'Create Date'
  • Show column 'Attributes'
  • Show column 'Owner'
  • Show column 'Hard Links'
  • Show column 'Hard Link ID'
  • Show column 'Match'
  • Show column 'Width'
  • Show column 'Height'
  • Show column 'Duration'
  • Show column 'Bit rate'
  • Show column 'Sample rate'
  • Optimize column widths
    Changes the column widths to automatically fit the content using only visible rows for calculation.
  • Optimize column widths automatically
    This option enables you to change the column widths automatically when expanding or collapsing groups.
  • Consider all rows when optimizing column widths (slower)
    This option enables you to use all rows of the search result for calculation to optimize the column widths.
  • Ignore column header text when optimizing column width

Filter

The Display Filter enables you to specify which groups should be displayed at the search result. A combination of the following options is available:
  • Show groups without selected files
  • Show groups with all files selected
  • Show groups with some files selected

Groups

The following actions enables you to expand or collapse certain groups:
  • Collapse all groups
  • Expand all groups
  • Expand only groups with selected files
  • Expand only groups without selected files
  • Expand only groups with all files selected
  • Expand only groups with at least 2 files not selected
  • Expand only groups with more than a certain number of files
  • Expand only groups that contain files with different names
  • Expand only groups that contain files with different names (without file extension)

Mode

The Process Mode enables you to specify which groups are affected from the 'Select', 'Deselect' and 'Remove' actions. You can choose one of the following options:
  • All groups
  • Expanded groups
  • Collapsed groups
  • Groups without selected files
  • Groups with all files selected
  • Groups with some selected files
  • Highlighted groups
  • Non-highlighted groups

Select

The following actions enables you to select certain files at the search result:
  • Select all files
  • Highlight all files
  • Select all files recursively in a specific folder
  • Select all files which are duplicates of files from the folder
  • Select all files except the first file in each group
  • Select all files except the last file in each group
  • Select all files except the newest file in each group
  • Select all files except the oldest file in each group
  • Select only the first file in each group
  • Select only the last file in each group
  • Select all files except the smallest file in each group
  • Select all files except the largest file in each group
  • Select all files except the file with the shortest name in each group
  • Select all files except the file with the longest name in each group
  • Select all files except the file with the shortest path in each group
  • Select all files except the file with the longest path in each group
  • Select files by name
  • Select files by path
  • Select files by size
  • Select all files with a Percent Match greater or equal to a specific percent value in each group
  • Select all files with a Percent Match lower or equal to a specific percent value in each group
  • Select all files except the picture with the most megapixels in each group
  • Select only files without hard links
  • Reverse file selection
  • Undo
    Enables you to reverse the last select or deselect file action. Please note that the complete undo list will be reset after a file is removed from the search result!

      Deselect

      The following actions enables you to deselect certain files at the search result:
      • Deselect all files
      • Deselect all files recursively in a specific folder
      • Deselect all files which are duplicates of files from the folder
      • Deselect files by name
      • Deselect files by path

      Remove

      The following actions enables you to remove certain groups or files from the search result:
      • Remove all groups with a certain number of files
      • Remove all groups where the files not exist at all source folders
      • Remove all groups with all files selected
      • Remove all groups with some files selected
      • Remove all groups that contain only files stored in archives
      • Remove all hard links from the search result
      • Remove all groups that contain only hard links
      • Remove all groups that contain only hard links with the same ID
      • Recursively remove all files in a specific folder from the search result
      • Remove all non-existing files from the search result
      • Remove all selected files from the search result

      File protection

      Here you can set the file protection for each source folder. Active file protection prevents files from being deleted, moved or renamed within this source folder.

      Toolbars

      Here you can show or hide the toolbars.
      • Toolbar 'Files'
      • Toolbar 'Filter'
      • Toolbar 'Groups'
      • Toolbar 'Search'
      • Toolbar 'Select'
      • Hide all toolbars
      • Use large icons

      Options

      • Hot Keys
      • Confirm file delete
      • Show prompt to save search result on exit
      • Show file info icons
        This option will display different icons for hard linked files and files from inside an archive file at the left edge of the search result. The icons will give you a quick overview of whether a file is a hard link or inside what kind of archive type the file is stored.
      • Decrease font size
      • Increase font size
      • Choose font
        This command enables you to change the font of the search result.
      • Max. thumbnail size in pixel
        Here you can specify the maximum width of a thumbnail displayed at the file preview pane after selecting a group
      • Show all file properties in a single column


Context Menu

Various actions can be accessed during context menus at the search result:

File

This context menu will be shown when you right click a file at a group.
Hint: Doing a right click on a file while pressing the CTRL key will show only the submenu with the file actions.
  • Select/Deselect file
  • File actions:
    • Open
      Open the file with the associated application.
    • Open file with...
    • Open file location
      Opens the folder of the file with the windows explorer or a user-specified external program
    • Open file with the external program
    • Copy file to ...
    • Move file to ...
    • Rename file
    • Delete file permanently
      Deletes the file permanently from the data storage.
    • Move file to the Windows Recycle Bin
    • Show file properties
  • Remove file from the search result
  • Move to the top of the group
    This command enables you to specify the source file to create hard links if needed.
  • Select all other files at this group
  • Select all files with the same path
  • Select all files inside the folder structure of this file
  • Deselect all files with the same path
  • Deselect all files inside the folder structure of this file
  • Delete all other files from this group permanently
  • Move all other files of this group to the Windows Recycle Bin
  • Filter lists:
    • Add file name to the file filter
    • Add file path to the folder filter
  • Add file extension to the a PlugIn list
    Enables you to add the file extension to the Plugin list "Microsoft WordPad", "Windows Internet Explorer", "Windows Media Player" or "VLC Media Player"
  • Remove file extension from the Plugin lists
  • Copy to the clipboard
  • Windows Explorer System Menu
    Opens the windows explorer context menu for this file.

Highlighted Files

You can highlight multiple files by pressing the CTRL key and clicking on each file. Click the highlighted files with the right mouse button to show up this context menu:
  • Select highlighted files
  • Deselect highlighted files
  • Compare highlighted files
  • Open highlighted files with external program
  • Switch file names
  • Copy highlighted files
  • Move highlighted files
  • Delete highlighted files permanently
  • Remove highlighted files from the search result
  • Copy to the clipboard

Groups

  • Select/Deselect Group
  • Reverse file selection
  • Remove selected group(s)
    Removes the selected group(s) from the search result.
  • Open all files from inside the group with external program


File Manager

The file manager enables you to delete, copy, move or rename the selected files from the search result.

  • Which groups should be processed?
  • What action is to be performed?
  • Folder
  • Options

Which groups should be processed?

  • All groups (ignore filter)
  • Only groups displayed by the filter

Don't process any groups with all files selected
This option will ensure that you not accidentally delete all similar files from inside a group.

What action is to be performed?

  1. Delete files
    This action enables you to delete your selected files. If you want to delete a file from inside an archive and the file is the only file inside the archive, the archive file will be deleted instead. Deleting files from an archive with multiple files inside is only possible with Zip- or 7z-archives.
    The 'Secure delete' option replaces the file content with zeros before deleting it and sets the file size to 0.

  2. Move files to the Windows Recycle Bin
    This action enables you to move your selected files into the Windows Recycle Bin.
    Important notes:
    • Make sure that the Windows Recycle Bin is activated on the partition with the selected files.
    • Set the size of the Windows Recycle Bin to 100%.
    • Selected files located on another computer will be deleted and not moved to the Windows Recycle Bin!
    • Files inside an archive cannot be moved into the Windows Recycle Bin

  3. Copy files to a folder
    This action enables you to copy your selected files to a folder of your choice.

  4. Move files to a folder
    This action enables you to move your selected files to a folder of your choice.

  5. Rename files
    This action enables you to rename all selected files. You can use the following place holder to rename your files:
    Placeholder | Description
    %1 | file name without file extension
    %2 | file extension with leading dot
    %YYYY | Year (4-digit)
    %YY | Year (2-digit)
    %MM | Month
    %DD | Day
    %HH | Hour
    %NN | Minute
    %SS | Second
NOTE: The actions 2, 3, 4 and 5 have no effect on selected files inside an archive file (ZIP, RAR...).

Show file protection note
This option shows a warning message before the selected files will be deleted, moved or renamed.

Folder

Here you can specify the folder to copy or move the selected files.

Create Subfolder(s)
This option causes a new subfolder to be created into which the files are copied. You can use placeholders for Date and Time and Windows variables.

Examples:
Text: "Backup of" %USERNAME% "on" yyyy-mm-dd "at" hh_nn_ss
Folder: Backup of USER NAME on 2011-02-26 at 10_15_00

Text: "%USERNAME%\"YYYY-MM-DD HH-NN-SS
Folders: USER NAME\2011-02-26 10-15-00

Keep folder structure of the files
This option causes the files to be copied/moved with the complete folder structure into the specified folder. Without this option the file will be directly copied/moved to the folder. Existing files will always will be renamed: e.g. Data.txt -> Data-001.txt

Options

Create Shortcuts
This option enables you to create a shortcut (.lnk) at the place of all files that will be deleted or removed. The shortcut will be point to the first file that is not selected at the same group; see Example for Shortcuts.

Create Hard Links
This option enables you to link all selected files of the same group; see Example for Hard Links. This has the advantage to save hard disk space because all linked files share the same data on the hard disk: for example, if you link 1000 files with a size of 10 MB each file you will save more than 9 GB hard disk space!

Please note the following hints:
  • Only files stored on the same volume can be linked together.
  • The files must be stored on a local NTFS partition.
  • Hard links cannot be created on a network drive.
  • Hard links cannot be created in a virtual folder that points to the data from an external source, such as local delivery of files from a cloud.
  • At least 2 files have to be selected in a group to create hard links.
  • The Windows file system has a limit of 1024 hard links on a file.
  • AllDup automatically determine a source file with a hard link count lower than 1024 of all selected files in a group. All other selected files in this group will be linked with this source file.
  • A new source file will be determined as soon as the current source file reaches 1024 hard links. This allows you to create an unlimited amount of hard links for each group.
  • The file context menu command 'Move to the top of the group' enables you to specify the source file to create hard links if needed.
  • Only files with a hard link count of zero can be linked to the source file. The hard link count will be checked by the value displayed at the column 'Hard links'. You need to activate the search option Determine the hard link count for each file so that the hard link count for each file will be determined during the search.
  • Files stored inside an archive will be skipped.

Remove files from the search result
This option causes to remove every processed file from the search result.

Remove groups with only one file
This option causes to remove all groups with only one file.

Check for groups with all files selected
This option show a warning if you try to delete all files at a group.

Don't show cancel dialog if an error occurred
This option causes to continue the selected file action after an error occurred while processing a file.

Delete folders which are empty after Move/Delete
This option causes to delete all empty folders after all files have been deleted inside the folder.

Don't log any file actions
This option disables the logging of the file actions.

After action is completed
  • Close file manager
  • Exit AllDup

Example for Hard Links

A group contain the following files:
  1. C:\Work\notes.txt
  2. C:\User\readme.txt
  3. C:\Backup\notes.txt
  4. C:\Mail\new-user-readme.txt
All files inside the group are selected (?). The first file will be automatically used for the source to create the hard links. All other files (2-4) will be deleted and a hard link to the first file will be created with the name of the deleted file.

Example for Shortcuts

A group contain the following files:
  1. C:\Work\notes.txt
  2. C:\User\readme.txt
  3. C:\Backup\notes.txt
  4. C:\Mail\new-user-readme.txt
The first file is not selected and will be used as the target for the shortcuts to be created. All other files in the group are selected (?). The selected files will be deleted and for each deleted file a shortcut to the first file will be created with the name of the deleted file.


Hot Keys

Here you can create keyboard shortcuts for all items of the menu bar and tool bar.

Click on a row at the list and press a key to set the shortcut for this item.
Press the DEL key to remove the shortcut of a selected item.
Press CTRL + DEL to delete the shortcuts of all items.

Keys

You can use the following keys to create a shortcut:
  • CTRL
  • ALT
  • SHIFT
  • A-Z
  • 0-9
  • F2-F12
  • Num0-Num9
  • Backspace
  • Enter
  • Insert

Shortcut Examples

  • CTRL + A
  • ALT + 1
  • SHIFT + L
  • CTRL + ALT + M
  • F2
  • Enter


Command Line

Syntax

AllDup.exe -s[1-9,a,b,c] -c[0-5] -h[0-9] -f[0-7] -m[1-100] -l[10-999] -a[1,2,4,5] -n -p:"[profile name]" "[Folder1]" "[Folder2]" ....

Parameters

-s[1-9,a]

This parameter enables you to specify the search method:
  • 1 = File Name
  • 2 = File Extension
  • 3 = File Size
  • 4 = File Content
  • 5 = File Last Modified date
  • 6 = File Create date
  • 7 = File Attributes
  • 8 = Hard Links
  • 9 = Similar Pictures
  • a = Similar Music
  • b = Audio Track
  • c = Similar File Names
The file properties 1 to 7 can be combined together such as "-s124". The search method 8, 9, a, b and c can only be used alone such as "-s8" or "-sa".

-c[0-5]

This parameter is optionally and can be used in combination with the parameter -s4 to specify the comparison method for the comparison criteria File Content.
  1. Byte by byte
  2. MD5 (128-Bit)
  3. SHA-1 (160-Bit)
  4. SHA-2 (256-Bit)
  5. SHA-2 (384-Bit)
  6. SHA-2 (512-Bit)

-h[0-9]

This parameter is optionally and can be used in combination with the parameter -s9 to specify the comparison method of the search method Similar Pictures.
  1. aHash
  2. bHash
  3. dHash
  4. mHash
  5. pHash
  6. MD5 (128-Bit)
  7. SHA-1 (160-Bit)
  8. SHA-2 (256-Bit)
  9. SHA-2 (384-Bit)
  10. SHA-2 (512-Bit)

-f[0-7]

This parameter is optionally and can be used in combination with the parameter -sc to specify the comparison method of the search method Similar File Names.
  1. SmartMatch
  2. FuzzyMatch
  3. Levenshtein
  4. Ratcliff
  5. MatchDiff
  6. WordMatch
  7. FuzzyPercent
  8. Simil

-m[1-100]

This parameter is optionally and can be used in combination with the parameter -s9 and -sa to specify the percentage match of the search method Similar Pictures and Similar Music.

-d[0-4]

This parameter is optionally and can be used in combination with the parameter -s9 to specify the compare size of the search method Similar Pictures.
  1. 8x8 (aHash, dHash, mHash)
  2. 16x16 (aHash, dHash, mHash)
  3. 128x128 (bHash)
  4. 256x256 (bHash)
  5. 512x512 (bHash)

-l[10-999]

This parameter is optionally and can be used in combination with the parameter -sa to specify the scan length in seconds of the search method Similar Music.

-a[1,2,4,5]

This parameter is optionally and can be used in combination with the parameter -sa to specify the comparison method of the search method Similar Music.

-p:"[profile name]"

This parameter is optionally and enables you to load a profile.

-n

With this parameter the specified folders will not added to the source folder list.
This parameter is optionally.

"[Folder]"

This parameter is necessary and enables you to specify the folder(s) to be searched for duplicates. Folder paths that include space(s) have to be enclosed in double-quotes.

Examples

AllDup.exe -s13 "C:\Work"

Search for duplicates with the same file name and file size inside the folder "C:\Work".

AllDup.exe -s9 -h3 -m90 "C:\Pics"

Search for similar pictures inside the folder "C:\Pics" using the compare method pHash and a percentage match of 90%.

AllDup.exe -sa -l25 -a4 -m80 "C:\Audio"

Search for similar music files inside the folder "C:\Audio" using a scan length of 25 seconds with the compare method no.4 and a percentage match of 80%.

AllDup.exe -s4 -c0 "C:\Work" "C:\Data"

Search for duplicates with the same content inside the folders "C:\Work" and "C:\Data".

AllDup.exe -s14 -c0 "C:\Work"

Search for files with the same name and content inside the folder "C:\Work".

AllDup.exe "C:\Work"

Search for duplicates inside the folder "C:\Work" with the current program settings.

AllDup.exe -p:"test" "C:\Work"

Search for duplicates inside the folder "C:\Work" with the settings of the profile "test".

AllDup.exe C:\Work "D:\Data Backup"

Search for duplicates inside the folders "C:\Work" and "D:\Data Backup" with the current program settings.


Wildcards for Text

Wildcard | Description
* | No or several characters.
? | Any single character.
# | Any single numeral (0 - 9).
[charlist] | Any single character in charlist.
[!charlist] | Any single character that isn't contained in charlist.

The wildcards can be combined in any sequence.

The following characters have to be inserted in square brackets in order to be used for a compare operation: left square bracket ([), question mark (?), pound sign (#) and asterisk (*). The right square bracket (]) can't be used in a string of characters being compared. However, it can be indicated outside of a string as an individual character.

You can also specify a range of characters in charlist by indicating the largest and smallest value of the range, separated by a hyphen (-). For example, [A-Z] results in a match if the respective character position in charlist is a letter between A and Z. You can specify a sequence of several ranges in square brackets without using separators.

Other key rules for using wildcards:

  • An exclamation point (!) at the beginning of charlist means that a match results when any character except for the characters in charlist is found. If the exclamation point is used outside of the square brackets it serves as a wildcard for itself.
  • A hyphen (-) can appear at the beginning (after an exclamation point, if present) or at the end of charlist in order to serve as a wildcard for itself. In any other position a hyphen serves to designate a character range.
  • When a range of characters is specified, the characters have to be listed in ascending order (from the lowest to the highest). This means that [A-Z] is a permissible pattern whereas [Z-A] isn't.

Examples:

Type of correspondence | Text | Correspondence | No correspondence
Several characters | a*a | aa, aBa, aBBBa | aBC
 | *ab* | abc, AABB, Xab | aZb, bac
[ ? # * | a[*]a | a*a | aaa
Several characters | ab* | abcdefg, abc | cab, aab
Individual characters | a?a | aaa, a3a, aBa | aBBBa
Individual numerals | a#a | a0a, a1a, a2a | aaa, a10a
Character range | [a-z] | f, p, j | 2, &
Outside of range | [!a-z] | 9, &, % | b, a
No numerals | [!0-9] | A, a, &, ~ | 0, 1, 9
Combination | a[!b-m]# | An9, az0, a99 | abc, aj0


Placeholders for the Date and Time

Characters | Description
: | Time delimiter
/ | Date delimiter
d, dd | Day (1 – 31, 01 – 31)
Ddd | Abbreviated day of the week (Sun - Sat)
dddd | Day of the week (Sunday - Saturday)
ddddd | Short date format
dddddd | Long date format
w | Calendar week day (1 for Sunday to 7 for Saturday)
ww | Calendar week (1 — 54)
m, mm | Month (1 - 12, 01 – 12)
mmm | Abbreviated names of the months (Jan - Dec)
mmmm | Names of the months (January - December)
q | Quarter (1 – 4)
y | Calendar day (1 – 366)
yy, yyyy | Year (00 – 99, 100 - 9999)
h, hh | Time/hour (0 - 23, 00 - 23)
m, mm, n, nn | Time/minute (0 - 59, 00 - 59)
s, ss | Time/second (0 - 59, 00 - 59)
AM/PM, A/P, AMPM | 12-hour format


Speed Comparison Test

Here you will get information about a speed comparison between the comparison method Byte by Byte and SHA-1. The additional options "Ignore Exif metadata of JPEG files" and "Log every x minutes the search statistic of the scan progress info" where activated for each scan.

The files of the scanned folder where stored on an external hard disk connected via USB2 and encrypted with TrueCrypt. AllDup was installed at a computer running Windows 7 x64 with 8 GB RAM and an i3 CPU.

The log of the first scan using the comparison method Byte by Byte:

The Log of the second scan using the comparison method SHA-1:

As expected, the comparison method SHA-1 wins the rally. Comparing only the checksums stored inside the RAM reduced the file read access and saved a lot of time.