Ext3 users - Oct 2002 - Periodic lockup problem with ext3

On several machines with ext3 we have a periodic "unresponsiveness"
problem. 

Take for example our mailserver: When it handles a lot of
email (lots of deliveries to Maildirs), it shovels the data into the
Maildirs. 

But every now and then (the interval being >> 5s, the commit
interval) the machine becomes unresponsive, your hear a lot of disk
activity, and after about 12-18s the machine is back to normal.
Nobody knows what's happening during that phase.

The same with my laptop, but not as dire, since the machine doesn't do
that much disk i/o at all (except when untarring a kernel or patching
a kernel tree).

This happened with various kernels (2.4.20-pre9, 2.4.19-ac4) on
various machines.

Is this maybe an IDE problem with 2.4.x kernels?

Sorry for being so vague, but this has been baffling us.

-- 
Ralf Hildebrandt (Im Auftrag des Referat V a)   Ralf.Hildebrandt@charite.de
Charite Campus Mitte                            Tel.  +49 (0)30-450 570-155
Referat V a - Kommunikationsnetze -             Fax.  +49 (0)30-450 570-916
| <- You must be smarter than this stick to ride the Internet

On Oct 09, 2002  17:17 +0200, Ralf Hildebrandt wrote:
> On several machines with ext3 we have a periodic
"unresponsiveness"
> problem. 
> 
> Take for example our mailserver: When it handles a lot of
> email (lots of deliveries to Maildirs), it shovels the data into the
> Maildirs. 
> 
> But every now and then (the interval being >> 5s, the commit
> interval) the machine becomes unresponsive, your hear a lot of disk
> activity, and after about 12-18s the machine is back to normal.
> Nobody knows what's happening during that phase.
This happens when the journal becomes full (and probably also has lots
of pending data buffers to flush, for data=ordered) and it must flush
the journal before any more filesystem activity can occur.  The way to
solve this is to have a flush interval which is shorter than the time
it takes to fill the journal.

Either decrease the flush interval so that less data is outstanding at
any time, or increase the size of the journal so that you always have
enough journal space for at least 30 seconds of changes, to allow the
data buffers to be flushed in the background.

Since the latter is not very practical (it might involve huge journals,
and correspondingly more data loss on a crash, unless you are running
with data=journal and your application is syncing all I/O), the
preferred method is simply to reduce the flush interval.  There was a
patch from AKPM which allows setting this on a per-mount basis, please
check the list archives.

Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/