Ext3 users - May 2003 - ext2/ext3 directory handling

I have just been thru rebuilding a server that had been suffering 
severe performance problems.  I transferred the partitions to another 

hard drive, copying into freshly formatted filesystems.  This made 
a HUGE difference in performance.  One key directory structure 
stores email messages in an structure where there is a directory 
for each mail account, and the messages are stored in individual 
files.	Some of these had grown to contain thousands of messages. 
A couple of weeks ago we went thru a purge, and deleted message 
files for inactive accounts, and were able to dump 75% of the 
files.	Just doing that made a big performance improvement. Then 
a couple of days ago things got bad again, so I did the rebuild. 

My theory: these directories had obviously expanded to contain 
thousands of file entries.  From the fact that just copying to 
clean filesystems made such a difference, I assume that deleting 
the files probably does nothing to remove the vacant entries and 
shrink the directories themselves.  

All this  raises a few questions. . .

(1) Is the assumption that directories don't compress when deleting 
files correct?	How is this handled (in general terms)?

(2) Is there any difference between ext2 and ext3?  

(3) Does the htree code change the picture any (even 
though I don't use it, and won't until it is production) ?

(4) Is it possible that the directories themselves 
were fragmented?

(5) After doing a "mkdir" to create a new directory, how many 
file entries can it hold before it would be expanded to accept 
another file?  When a directory is expanded, how many additional 
file entries can be stored before needing another expansion?

(6) Say I have a directory containing some files, then I delete 
some files, and finally I start adding files.  Will new file 
entries use empty or vacated directory slots before expanding 
the directory?

(7) I am aware of e2defrag (latest version I have found is 0.73). 
Does this program (or any other any tool) perform any 
directory optimization that would affect this problem?

(8) If e2defrag would be helpful, has it/is it being brought 
forward to operate correctly with current (RH 8/9) systems?
I see some warnings about blocksise restrictions, etc.

(9) In designing new systems, are there some useful guidelines 
about the maximum number of files that can exist in a single 
directory without significant performance loss?  
I am interested in ext2, ext3, and htree.

TIA, 
A. Becker

"Alan R.Becker" <beckera@mail-now.com>
wrote:
>
> (1) Is the assumption that directories don't compress when deleting 
> files correct?	How is this handled (in general terms)?
That is correct.  A deleted file leaves a "hole" in the directory
which a new addition can fill (if it fits).
> (2) Is there any difference between ext2 and ext3?  
No.
> (3) Does the htree code change the picture any (even 
> though I don't use it, and won't until it is production) ?
No, htree will not release directory blocks.
> (4) Is it possible that the directories themselves 
> were fragmented?
Yes, very probable.

However to understand why things slowed down a bit more info is needed.

It is probable that the many little files in one typical directory are
splattered all over the disk.  Does your workload regularly touch all the
file in these directories?  If so then it maybe suffering from this lack of
inter-file locality.

If not then yes, perhaps the problem is due to large, fragmented
directories.

How many bytes does a typical directory consume?  If you have the disk
space, and are confident that (say) 64k is "enough" then perhaps you
could
grow each user's mail directory to (say) 64k when that user is created. 
This way they will have a nice unfragmented directory for all time.

> (5) After doing a "mkdir" to create a new directory, how many 
> file entries can it hold before it would be expanded to accept 
> another file?
4 kilobytes.  Each directory entry consumes eight bytes, plus the length of
the name rounded up to a multiple of 4 bytes.
> When a directory is expanded, how many additional 
> file entries can be stored before needing another expansion?
Another 4 kilobytes.
> (6) Say I have a directory containing some files, then I delete 
> some files, and finally I start adding files.  Will new file 
> entries use empty or vacated directory slots before expanding 
> the directory?
Deletion causes holes.  Holes are coalesced within a 4k block.  Holes are
allocated from on a first-fit basis.
> (7) I am aware of e2defrag (latest version I have found is 0.73). 
> Does this program (or any other any tool) perform any 
> directory optimization that would affect this problem?
It's obsolete.

For your purposes, all you'd need to do to defrag a directory is

	mkdir new
	ln old/* new/
	rm -rf old
	mv old new

If you use `cp' instead of `ln' then you'll defrag the files
themselves,
and lay them out close to each other.  Which is only important if you app
regularly touches lots of files in a single directory.  It probably does
not..
> (8) If e2defrag would be helpful, has it/is it being brought 
> forward to operate correctly with current (RH 8/9) systems?
> I see some warnings about blocksise restrictions, etc.
I haven't heard of anyone using it in ages.
> (9) In designing new systems, are there some useful guidelines 
> about the maximum number of files that can exist in a single 
> directory without significant performance loss?  
> I am interested in ext2, ext3, and htree.
Non-htree gets awkward at a few thousand.  htree appears to be OK up to
hundreds of thousands.  Its practical scalability is unknown, really.