howto:duplicate
Find Duplicates
There are various tools and commands to find duplicate files (i.e. find command), but those usually find exact matches which does not help with media files (music,video,images) that have been resized,reencoded,contain watermarks, etc). The following tools help find those duplicates.
Video Duplicate Finder (Win/Linux) finds Videos and Images based on similarity:
https://github.com/0x90d/videoduplicatefinder/releases
wget "https://github.com/0x90d/videoduplicatefinder/releases/download/2.0.7/VDF.Linux-x64.tar.gz" tar -xvzf VDF.Linux-x64.tar.gz rm VDF.Linux-x64.tar.gz cd VDF.Linux-x64 ./VideoDuplicateFinderLinux
Dupeguru (Win/Linux/Mac) finds Music, Images and general files:
https://dupeguru.voltaicideas.net/
sudo apt-add-repository ppa:dupeguru/ppa sudo apt-get update sudo apt-get install dupeguru
For ModuleNotFoundError: No module named 'core.pe._block
' error, link to the proper libraries:
python3 --version #note the version number and add links. Replace '38' with major and minor version on the system. replace 36 if necessary. sudo ln /usr/share/dupeguru/core/pe/_cache.cpython-36m-x86_64-linux-gnu.so /usr/share/dupeguru/core/pe/_cache.cpython-38-x86_64-linux-gnu.so sudo ln /usr/share/dupeguru/core/pe/_block.cpython-36m-x86_64-linux-gnu.so /usr/share/dupeguru/core/pe/_block.cpython-38-x86_64-linux-gnu.so sudo ln /usr/share/dupeguru/qt/pe/_block_qt.cpython-36m-x86_64-linux-gnu.so /usr/share/dupeguru/qt/pe/_block_qt.cpython-38-x86_64-linux-gnu.so
FSlint is a find frontend and finds exact matches, temp files, bad symlinks etc on linux
sudo apt-get install fslint
howto/duplicate.txt · Last modified: 2023/05/29 11:55 by 127.0.0.1