Duplicate Finder
Find duplicates in any list of values. Paste your list — one item per line — and the tool shows how many duplicates exist, which items repeat, and how many times. Useful for cleaning email lists, customer names, product codes, and any list with potential repeats.
Duplicates Found
Unique List (No Duplicates)
What Is a Duplicate Finder?
A duplicate finder scans a list of values and identifies items that appear more than once. It shows which values are repeated and how many times each appears — essential for cleaning contact lists, product catalogs, invoice logs, or any data where duplicates cause problems.
Duplicates waste money (duplicate mailers, duplicate email sends), confuse reports (double-counted revenue), and cause errors (duplicate customer records in a CRM). Finding and removing them is one of the most common data-cleaning tasks.
How Duplicate Detection Works
The tool processes your list in three steps:
- Normalize — if case-insensitive, convert all items to lowercase; if trim is on, remove leading/trailing spaces.
- Count — use a dictionary to count the frequency of each normalized item.
- Identify — any item with a count greater than 1 is a duplicate. The tool reports each duplicate and its frequency.
The unique list is generated by taking the first occurrence of each item (preserving original case and spacing) and removing subsequent duplicates.
Worked Example
Input list: apple, banana, apple, orange, grape, banana, kiwi, apple
- Total items: 8
- Unique items: 5 (apple, banana, orange, grape, kiwi)
- Duplicate items: 2 (apple, banana)
- Duplicate instances: 3 (2 extra apples + 1 extra banana)
- Duplicates found: apple (3×), banana (2×)
- Unique list: apple, banana, orange, grape, kiwi
Common Uses
- Email list cleanup — remove duplicate email addresses before sending a campaign
- Customer database deduplication — check for repeated customer names or IDs
- Product catalog cleanup — remove duplicate SKUs or product codes
- Invoice audit — check for duplicate invoice numbers
- Survey/data analysis — spot repeated responses or values
- Inventory reconciliation — remove duplicate item entries after data import
- Contact form entries — filter out duplicate form submissions
Frequently Asked Questions
Is my data private?
Yes — everything runs in your browser. Your data is not sent to any server, not stored, and not shared. When you close the tab, the data is gone. This is especially important for email lists and customer data, which should never be uploaded to an unknown server. Copy or download the results before closing the tab if you need to keep them.
What is the difference between "duplicate items" and "duplicate instances"?
"Duplicate items" is the number of distinct values that appear more than once. In the example, that is 2 (apple and banana). "Duplicate instances" is the number of extra copies beyond the first occurrence — apple appears 3 times (2 extra), banana 2 times (1 extra), so the total instances is 3. If you removed all duplicates and kept only one of each, you would remove 3 rows.
Does the tool handle case sensitivity?
Yes — you can choose whether "Apple" and "apple" are treated as duplicates. If case-insensitive (the default), they are considered the same and the tool reports one duplicate entry. If case-sensitive, they are treated as distinct values. Most email and customer list deduplication uses case-insensitive matching because email addresses are technically case-insensitive.
What about near-duplicates like "John Smith" and "Jon Smith"?
This tool finds exact duplicates (after case and whitespace normalization). It does not find near-duplicates (fuzzy matches, typos, abbreviations). Near-duplicate detection requires fuzzy matching algorithms (Levenshtein distance, Jaro-Winkler similarity) which are more complex. For near-duplicate detection in customer data, use a dedicated data-cleaning tool like OpenRefine or a CRM's built-in deduplication feature.
How many items can I process?
This tool handles lists up to tens of thousands of items comfortably in the browser. Performance depends on your device — a modern browser on a laptop can process 10,000–50,000 items almost instantly. For millions of items, use a command-line tool like `sort | uniq -c` on Linux/Mac, or `sort` and deduplicate in Excel/Sheets. For most realistic small-to-medium lists (emails, customer names), the browser tool is fast enough.
Can I download the results?
Yes — click "Download Unique List" to save the deduplicated list as a `.txt` file. Each item appears once, in original order. You can then paste it into a spreadsheet, CRM, or email tool. The duplicate report itself is not downloadable as a file, but you can copy the numbers from the results box if you need to record how many duplicates were found.