Hi We've got a directory with approx 1 million files. Mac user ... grr Is there a quick and dirty way to remove them? I was going with: ls | xargs rm but maybe rsync -a --delete empty/ full/ would be less expensive? This is on a distributed replicated volume. Any pointers appreciated! -- Pete Smith DevOp/System Administrator Realise Studio 12/13 Poland Street, London W1F 8QB T. +44 (0)20 7165 9644 realisestudio.com
Stephan von Krawczynski
2013-Jun-12 14:38 UTC
[Gluster-users] Quickest way to delete many small files
On Wed, 12 Jun 2013 15:26:46 +0100 Pete Smith <pete at realisestudio.com> wrote:> Hi > > We've got a directory with approx 1 million files. Mac user ... grr > > Is there a quick and dirty way to remove them? > > I was going with: > > ls | xargs rm > > but maybe rsync -a --delete empty/ full/ > > would be less expensive? > > This is on a distributed replicated volume. > > Any pointers appreciated!Admin question ;-) mv <dir> <tempdirname> mkdir <dir> rm -rf <tempdirname> -- Regards, Stephan
On 06/12/2013 10:26 AM, Pete Smith wrote:> Hi > > We've got a directory with approx 1 million files. Mac user ... grr > > Is there a quick and dirty way to remove them? > > I was going with: > > ls | xargs rm > > but maybe rsync -a --delete empty/ full/ > > would be less expensive? > > This is on a distributed replicated volume.http://linuxnote.net/jianingy/en/linux/a-fast-way-to-remove-huge-number-of-files.html tl;dr The rsync method is likely to be faster because of readdir batching. In fact, the results from that article are likely to be even more applicable to any distributed filesystem where readdir is even more expensive. This difference is completely orthogonal to the difference between deleting files through a client mountpoint or directly on the brick, with or without renaming first.