Greetings, I'm looking for clues about speeding up unlink performance. I have an ext3 file system (well, across several machines) that have a temp directory. The application needs to clean up the temp directory before the next run of the application begins and the engineers want to clean out this temp directory "quickly". I have no hard numbers yet on what they are seeing but the contents of the directory involves "many files" and "many directories" of various sizes. The file system in question is mounted with noatime,nodiratime with the following filesystem features: has_journal, resize_inode, dir_index, filetype, needs_recovery, sparse_super and large_file The operating system is Fedora Core 6 with fedora's 2.6.20-1.2933 kernel. The mounted file system is about 1.2TB in size and is a software raid-5 over four, 7200 rpm SATA disks. The disks were formatted with all the defaults. Pre-loading the file system cache (a la "find /path/to/temp -type f -print >/dev/null") followed by an "rm -rf /path/to/temp" seems to be pretty speedy to me. Any other suggestions of things I can experiment with to build performance numbers? Cheers, Ryan
On Tue, Aug 14, 2007 at 11:08:28AM -0700, Ryan Dooley wrote:> I'm looking for clues about speeding up unlink performance. I have an > ext3 file system (well, across several machines) that have a temp > directory. The application needs to clean up the temp directory before > the next run of the application begins and the engineers want to clean out > this temp directory "quickly".Depending on your usage (and not dependant of specifics of filesystem), you might be able to get away with: mv /path/to/temp /path/to/temp.old mkdir /path/to/temp rm -rf /path/to/temp.old & as a workaround. in that way, the new run which fills the '/path/to/temp' can commence practically without any delay even while old data is still being removed, without any clash.> The file system in question is mounted with noatime,nodiratime with the > following filesystem features: > > has_journal, resize_inode, dir_index, filetype, needs_recovery, sparse_super and large_fileWhat is the journal size ? (echo 'stat <8>' | debugfs /dev/md0) You can try increasing it. How much RAM is in the machine ?> The operating system is Fedora Core 6 with fedora's 2.6.20-1.2933 kernel. > The mounted file system is about 1.2TB in size and is a software raid-5 > over four, 7200 rpm SATA disks. The disks were formatted with all the > defaults.RAID5 would NOT be fastest choice for writes (including deletes), especially depending on the stripe size, and is dependant on "mke2fs -E stride" option.> Pre-loading the file system cache (a la "find /path/to/temp -type f -print > >/dev/null") followed by an "rm -rf /path/to/temp" seems to be pretty > speedy to me.Then there is no problem, yes ? :) -- Opinions above are GNU-copylefted.
On 14 Aug 2007, at 19:08, Ryan Dooley wrote:> Pre-loading the file system cache (a la "find /path/to/temp -type f > -print >/dev/null") followed by an "rm -rf /path/to/temp" seems to > be pretty speedy to me.Do you mean that:- find /path/to/temp -type f -print >/dev/null rm -rf /path/to/temp is faster than just rm -rf /path/to/temp or do you mean that you have arranged to do the find before the point where you want to delete? If the former, that surprises me somewhat. Nigel. -- [ Nigel Metheringham Nigel.Metheringham at InTechnology.co.uk ] [ - Comments in this message are my own and not ITO opinion/policy - ]