I have an ext3 filesystem that has several directories and each directory gets a large number of files inserted and then deleted over time. The filesystem is basically used as a temp store before files are processed. The issue is over time the directory scans get extremely slow even if the directories are empty. I have noticed the directories can range in size from 4k - 100M even when they are empty. Is there a way to fix this without recreating the directories or bringing the filesystem offline? File system Info: tune2fs 1.35 (28-Feb-2004) Filesystem volume name: <none> Last mounted on: <not available> Filesystem UUID: 7cbda7aa-e8e7-4da1-9c7c-de45668e98f3 Filesystem magic number: 0xEF53 Filesystem revision #: 1 (dynamic) Filesystem features: has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file Default mount options: (none) Filesystem state: clean Errors behavior: Continue Filesystem OS type: Linux Inode count: 98304000 Block count: 196608000 Reserved block count: 9830400 Free blocks: 31795332 Free inodes: 83024519 First block: 0 Block size: 4096 Fragment size: 4096 Reserved GDT blocks: 1024 Blocks per group: 32768 Fragments per group: 32768 Inodes per group: 16384 Inode blocks per group: 512 Filesystem created: Thu Aug 10 11:10:59 2006 Last mount time: Tue Oct 3 00:10:48 2006 Last write time: Tue Oct 3 00:10:48 2006 Mount count: 4 Maximum mount count: 21 Last checked: Thu Aug 10 11:10:59 2006 Check interval: 15552000 (6 months) Next check after: Tue Feb 6 10:10:59 2007 Reserved blocks uid: 0 (user root) Reserved blocks gid: 0 (group root) First inode: 11 Inode size: 128 Journal inode: 8 Default directory hash: tea Directory Hash Seed: 59fd108a-7ec7-45f9-8967-b9f3aaec3edf Journal backup: inode blocks Matt D. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://listman.redhat.com/archives/ext3-users/attachments/20061004/c2ccd283/attachment.htm>
(please refrain from sendin HTML mails) On Wed, 4 Oct 2006, Matt Dodson wrote:> I have an ext3 filesystem that has several directories and each > directory gets a large number of files inserted and then deleted over > time.Can you specify "large number"? What do "ls large-directory | wc -l" say?> The filesystem is basically used as a temp store before files are > processed. The issue is over time the directory scans get extremely slow > even if the directories are empty. I have noticed the directories can > range in size from 4k - 100M even when they are empty.proably deleted-but-still-open files. When lsof(8) is installed, you can find out with: "lsof -ln | grep large-but-empty-directory" Can you specify "slow" as well? You also might want strace(1) an "ls" on your large directory to see what is taking so long.> Is there a way to fix this without recreating the directories or > bringing the filesystem offline?You have enabled htree (dir_index) already:> Filesystem features: has_journal resize_inode dir_index filetype > needs_recovery sparse_super large_fileIf you've enabled dir_index after the directories have been created, you might want to "e2fsck -D" (see the manpage for details) the filesystem. For partitions with temprary files you could play with "noatime","async" and "data" mount-options (please read the manpage, really!). Which kernel do you use? Which arch? C. -- BOFH excuse #83: Support staff hung over, send aspirin and come back LATER.
On Oct 04, 2006 16:33 -0500, Matt Dodson wrote:> I have an ext3 filesystem that has several directories and each > directory gets a large number of files inserted and then deleted over > time. The filesystem is basically used as a temp store before files are > processed. The issue is over time the directory scans get extremely slow > even if the directories are empty. I have noticed the directories can > range in size from 4k - 100M even when they are empty. Is there a way > to fix this without recreating the directories or bringing the > filesystem offline?No way to fix this w/o offline e2fsck -fD. ext3 doesn't shrink directories when deleting files. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.