Open Source Libs
Find Open Source Packages
Open Source Libraries
👉
Data Processing
👉
Dedupe
18 Open Source Dedupe Software Projects
Free and open source dedupe code projects including engines, APIs, generators, and tools.
Restic
11057 ⭐
Fast, secure, efficient backup program
Borg
6613 ⭐
Deduplicating archiver with compression and authenticated encryption.
Dedupe
2784 ⭐
:id: A python library for accurate and scalable fuzzy matching, record deduplication and entity-resolution.
Yarn Deduplicate
725 ⭐
Deduplication tool for yarn.lock files
Free Style
689 ⭐
Make CSS easier and more maintainable by using JavaScript
Jdupes
664 ⭐
A powerful duplicate file finder and an enhanced fork of 'fdupes'.
Recordlinkage
473 ⭐
A toolkit for record linkage and duplicate detection in Python
Csvdedupe
298 ⭐
:id: Command line tool for deduplicating CSV files
Dedupe Examples
255 ⭐
:id: Examples for using the dedupe library
Dupe Krill
98 ⭐
A fast file deduplicator
Maildir Deduplicate
91 ⭐
:e-mail: CLI to deduplicate mails from maildir folders
Imgdupes
99 ⭐
Finding and deleting near-duplicate images based on perceptual hash.
Pgdedupe
33 ⭐
A simple command line interface to the datamade/dedupe library.
Watsondedupe
27 ⭐
Self-contained C# library for data deduplication using Sqlite
Django Super Deduper
23 ⭐
Utilities for de-duping Django model instances
Duplex
19 ⭐
Duplicate code finder for Elixir
Dduper
24 ⭐
Fast block-level out-of-band BTRFS deduplication tool.
Duplicut
260 ⭐
Remove duplicates from MASSIVE wordlist, without sorting it (for dictionnary-based password cracking)