Today's article has you removing duplicate files (from the terminal, of course)...

[KGIII]

Today's article is similar to another article, but that was about removing duplicates with rdfind. This time around, we're doing the same thing - but we're doing it with fdupes. You can use one or the other, assuming you know about both (which you now do, if you didn't already).

Of course, one should exercise care when removing duplicates. I once tested this by blindly running rdfind on the root directory. Sure enough, that VM never booted properly again. So, don't do that. Pay attention to what's being deleted - if you even bother. After all, today you have plenty of disk space for not a whole lot of money. Saving a few bits and bytes isn't going to make too much of a difference.

Find And Remove Duplicate Files With fdupes • Linux Tips

Today's article has you cleaning up your storage space as you learn to find and remove duplicate files with fdupes.

linux-tips.us

Enjoy!

(And yes, WordPress happily ignored the publication schedule today. It was on vacation, or something like that.)

 

[BigBadBeef]

Would you characterize Linux as being unclean enough to warrant the use of such a command? Or would you label the "clean" and "autoremove" commands to be sufficient?

 

[KGIII]

Would you characterize Linux as being unclean enough to warrant the use of such a command?

Nah, I wouldn't use it unless you need it. Say you work with a bunch of files as a part of your job and some of them are duplicates, that sort of environment.

Or would you label the "clean" and "autoremove" commands to be sufficient?

On distros that support that, by all means. Really, few people are going to want to remove duplicates. Disk space is absurdly cheap now.

 

[BigBadBeef]

I would also urge caution. Multiple files may have the same name, be located in different folders and have revised content. You wouldn't want use fdupes to "pop" one of those, people have been known to die of shock for less than that!

 

[KGIII]

I would also urge caution.

I agree 100%.

Multiple files may have the same name, be located in different folders and have revised content.

If you check the man page, that should not happen. It compares things like the MD5 checksum and then a byte-by-byte comparison takes place.

So, in theory, it should only find true duplicates.

I think I mentioned in the article that I once tried this on the root directory of an Ubuntu virtual machine and that it must have removed something important because it would no longer boot. I didn't dig any deeper than that, I just wanted to see if it would break the system.

 

[bob466]

Thanks for that...very handy to know.

In Mint Cinnamon if I want to remove duplicate entries...maybe every 6 mths...I'll go to Software Sources, Maintenance, Remove duplicate entries...then update cache.

 

[KGIII]

Thanks for that...very handy to know.

In Mint Cinnamon if I want to remove duplicate entries...maybe every 6 mths...I'll go to Software Sources, Maintenance, Remove duplicate entries...then update cache.

Yeah, this is used to do a lot more of a forensic operation, able to scan all the files and then compare them with one another. Used improperly, it will trash your OS - or at least it did when I went against all advice and ran it across the whole system.

I think your GUI is probably sudo apt clean and sudo apt autoremove. I think... I'd have to dig into it further, but I think that's what it does. Both are good practices to keep a tidy operating system. It will also save you some space but, as I've mentioned before, disk space is just so cheap these days.

We truly are living through a golden era of computation. There have been other such eras and there will be more, but I think we should appreciate today. Today, without any investment beyond the purchase, you can buy a computer and monitor that is a decade old and still comfortably run Linux on that hardware.

It's absolutely amazing and awesome. I remember paying a buck a MB for RAM and paying hundreds of dollars for an HDD that stored like 40 MB. Then, it was eventually hundreds of dollars for an HDD that had 40 GB. Today, I'd be hard-pressed to spend even $100 on storage for a PC. (I store a lot over the network.)

 

[bob466]

People coming from windowz think Linux fills up with crap too...but as we all know it doesn't and never slows down either.

I wrote this for beginners...don't know if they'll take any notice...
https://www.linux.org/threads/do-you-really-need-to-clean-mint.43752/

 

[KGIII]

but as we all know it doesn't and never slows down either.

That kinda depends on the user. It's quite possible, almost easily possible, for a user to break their system. So, as they add and remove stuff, they may find some slowdown - but it's nothing like winrot.

I think the two that amuse me is the people who feel they absolutely must have antivirus software and the people who are convinced that you have to defrag your disks. You can do those things, if you insist on it, but you don't really need either - especially in modern times with SSDs.