Delete millions of files slowly without loading the server

There a situations when we need to delete a great deal of files from our filesystem and if we just execute

rm -Rf

the server will surely get loaded and the service it provides will degrade! What if you cannot reformat the filesystem, because the server use it extensively, but you need to delete let’s say a couple of millions file from it? We can use find and usleep (in most linux distro this program is installed by an additional package). The idea is to delete files one by one tuning the pause between every delete. Here you can execute this command in the background or a screen:

find /mnt/storage/old/ -type f -exec echo {} \; -exec rm {} \; -exec usleep 200000 \;

usleep accepts microseconds, so 200000 microseconds are 0.2 seconds. You can tune it precisely with a step of just a microsecond. In the real world under the bash console we probably will use values of max 1/10 of a second around above 100000 microseconds. Execute the command and then watch your server load and tune.

  • usleep in CentOS 7 is installed with package “initscripts”, which is installed by default
  • usleep in Ubuntu is missing and probably won’t find any safe place to download a package to install, but it can be sort of replace with “sleep <floating_point_number>s”, GNU sleep could accept floating point number for the delay and when added “s” at the end it could sleep for a fractions of a seconds. So the command for the Ubuntu is slightly changed:
    find /mnt/storage/old/ -type f -exec echo {} \; -exec rm {} \; -exec sleep 0.2s \;
    
  • not GNU version of sleep require NUMBER, so the smallest sleep is only 1 second, which is too big for the purpose. Check your man manual to see if your system has GNU sleep command.

Leave a Reply

Your email address will not be published.