I'd like to hear the thoughts of list members on which type of storage
method seems "best" for inboxes and for folders.
The filesystem is GFS2 and for various reasons I can't change it.
Inboxes - currently Mbox format. Some users have upwards of 5000
messages in there (the largest is about 18k entries) and thanks to
attachments some inboxes are 40-200Mb with a few sitting at 2-3Gb.
(Suicidal, I know but enforcing rules is politik-laden and getting
academics to behave sensibly is like herding cats)
Folders: Mdir format. Thanks to GFS2 filesystem limitations(*) I'm
currently enforcing a limit of 4000 messages/folder. It'd be nice to
have a "better" way of storing these which doesn't drive up backup
loads
tremendously.
The mail spool has around 6 million files in the folders, covering about
400Gb.
GFS2 filesystem limitations are painful.
While there are no hard limits, It really doesn't cope well with a lot
of files in any given directory thanks to limitations in the way that
cluster locking is propagated, There are marked slowdowns about 100
files, this becomes awful above 1000 and effectively unusable above
about 8000 entries as the entire IO system gets halted momentarily when
such directories are opened and the directory may take up to 5 minutes
to return a "ls" result.
Some users had upwards of 80k files in their mail folders (~40k
messages) and accessing these brought the entire system to a halt.
If I migrate to other formats (eg mdbox), then it needs to be able to be
done on the fly. (Taking the mailservers down for a day won't go down
well, even an hour raises howls).
Any suggestions?