Open Source Libs

Open Source Libraries 👉Duplicates

35 Open Source Duplicates Software Projects

Free and open source duplicates code projects including engines, APIs, generators, and tools.

Copy/paste detector for programming source code.

Rmlint 1177 ⭐

Extremely fast tool to remove duplicates and other lint from your filesystem

Yarn Deduplicate 1107 ⭐

Deduplication tool for yarn.lock files

Python Hashes 226 ⭐

Interesting (non-cryptographic) hashes implemented in pure Python.

F483 Dejavu 154 ⭐

Quickly detect already witnessed data.

Redux Dataloader 141 ⭐

Loads async data for Redux apps focusing on preventing duplicated requests and dealing with async dependencies.

Mongoose Beautiful Unique Validation 119 ⭐

Plugin for Mongoose that turns duplicate errors into regular Mongoose validation errors

CLI utility to find duplicate files

Duplicut 515 ⭐

Remove duplicates from MASSIVE wordlist, without sorting it (for dictionary-based password cracking)

Advanced similarity and duplicate source code proof of concept for our research efforts.

Src D Gemini 46 ⭐

Advanced similarity and duplicate source code at scale.

Near Duplicate Code Detector 56 ⭐

A simple tool for detecting near-duplicate source code

Deplicate 54 ⭐

Advanced Duplicate File Finder for Python

Es Dedupe 46 ⭐

Tool for removing duplicate documents from Elasticsearch

Potential Duplicates Bot 31 ⭐

A configurable GitHub App which checks for potential issue duplicates using Damerau–Levenshtein distance algorithm.

Vidupe is a program that can find duplicate and similar video files. V1.211 released on 2019-09-18, Windows exe here:

Gduplicate Finder 26 ⭐

GDuplicate Finder - A Groovy way to find duplicates among your computer and network shares!

Finddups 22 ⭐

Find duplicate files on your computer

⌚️ A faster unique() function

Duplicate file finder written in Nim

A CLI tool to find/remove duplicate files supporting multi-core and different algorithms (MD5, SHA256, and XXHash).

Url Rewrites 19 ⭐

Magento 1.x module to target the URL Rewrite issue

No dups, no doubts

Indexed String Variation 16 ⭐

Experimental JavaScript module to generate all possible variations of strings over an alphabet using an n-ary virtual tree

Duplicatecontactsremover 17 ⭐

📒A simple app to optimize your address book and remove duplicate contacts.

Danielpclark Dfm 10 ⭐

Duplicate File Manager

Fs Inspect 11 ⭐

FS-Inspect is an easy to use tool designed to give you an overview about your files and directories (Disk Usage).

Lastfm Smart Deduper 31 ⭐

JS script that allows you to remove duplicates from your Last.fm scrobbles library.

Czkawka 6063 ⭐

Multi functional app to find duplicates, empty folders, similar images etc.

Stackoverflownotes 9 ⭐

SO [r] tag list of dupes, help, links, etc.

Removedupes 30 ⭐

Remove Duplicate Messages

Backdown 46 ⭐

Dupe_eraser 10 ⭐

A command-line tool which automate the deletion of duplicate files based on their hash or perceptual-hash.

Duplikaatti 11 ⭐

Remove duplicate files.

Nextcloud Media Duplicate Collector application