Daniel Watts wrote:> Could I possibly have some feedback on what the recommended filesystems > are? I've heard of ReiserFS but was wondering what other options there > are and how they compare.HTH http://www.debian-administration.org/articles/388 http://fsbench.netnation.com/ http://linuxgazette.net/122/TWDT.html#piszcz Kind Regards Brent Clark
Hi, I've heard that for Dovecot/Mailir systems there are filesystems that are optimised for the situation of many small files in one folder. Could I possibly have some feedback on what the recommended filesystems are? I've heard of ReiserFS but was wondering what other options there are and how they compare. If I get a good comprehensive response I'll build a wiki summary page out of the data gathered. Best wishes, Daniel
On Fri, 2006-05-05 at 14:29 +0100, Daniel Watts wrote:> Hi, > > I've heard that for Dovecot/Mailir systems there are filesystems that > are optimised for the situation of many small files in one folder. > > Could I possibly have some feedback on what the recommended filesystems > are? I've heard of ReiserFS but was wondering what other options there > are and how they compare. > > If I get a good comprehensive response I'll build a wiki summary page > out of the data gathered. > > Best wishes, > DanielFrom my personal experiences, I'd heartily recommend xfs. I've been using reiserfs since around the time it was merged into the stock kernel and was the only journalling fs in the main kernel tree. I still use reiserfs in a few places where it hasn't been practical to convert to xfs. I started using xfs on my workstation shortly before it became part of the main kernel tree, because I was quite interested in POSIX ACLs and it also performed better than reiserfs in my testing. Since that time, usage has fanned out to most of the boxes I administer, and I've found it performs quite a bit better than reiserfs for me -- especially when dealing with lots of small files (e.g. Maildir.) I'm echoing some of the more recent conversation now, but perhaps just as important or moreso than raw performance is failure recovery: 4-5 years of experience with each FS is ample time to see some hardware failures, and reiserfs has dealt rather poorly with filesystem corruption in my experience. Most recently, I had a handful of sectors go bad on a drive full of Maildirs, and this was brought to my attention not by kernel errors being logged, but by the system spontaneously and repeatedly rebooting. xfs, on the other hand, has been extremely graceful when it runs into fs corruption -- something especially important when physical access to the system isn't readily available (a few of the boxes I admin are ~900mi away.) My other complaint with reiserfs is that reiserfsck is painstakingly slow -- especially when you need to resort to --rebuild-tree (as I did in the above scenario) -- which means more downtime when something Really Bad(tm) happens. I don't remember how long it took to repair that filesystem once I'd moved it to another drive, but I'm sure it was at least a couple of hours. Unfortunately, between xfs and reiserfs, I haven't extensively used any other filesystems recently enough to have a good idea of Maildir performance or how well they deal with hardware failures. I would recommend xfs over reiserfs in a heartbeat, though, after having dealt with both on failing drives. YMMV, of course -- these are just my experiences. HTH, -- Ben Winslow <rain at bluecherry.net> -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 827 bytes Desc: This is a digitally signed message part URL: <http://dovecot.org/pipermail/dovecot/attachments/20060508/79c166bf/attachment.bin>
> I've heard that for Dovecot/Mailir systems there are filesystems that > are optimised for the situation of many small files in one folder. > > Could I possibly have some feedback on what the recommended > filesystems are? I've heard of ReiserFS but was wondering what other > options there are and how they compare.I happen to also be investigating this, and I too keep hearing the Reiser is the way to go for Maildirs. However, when I went looking for benchmarks to support this, the information out there is sparse, confusing and contradicting. For instance, here's a benchmark showing Reiser being among the slowest filesystems in many areas... http://linuxgazette.net/122/piszcz.html Here's a guy who claims that Maildir + ReiserFS is a win but then when you look at his numbers it appears that Reiser was slower than ext3... http://www.decisionsoft.com/pdw/mailbench.html This guy has Reiser as the slowest... http://www.thesmbexchange.com/eng/qmail_fs_benchmark.html And so on. You can find benchmarks that show Reiser as the best, ext3 as the best, xfs as the best, jfs as the best. All for maildirs. So, I'm not sure if I can trust any of the benchmarks. One thing that seemed consistent is that everyone had xfs performing pretty well. I am configuring a new RAID array for my Maildirs and so I am in a position where I can test various filesystems now. I tried xfs on Fedora Core 5 and it crashed the kernel, so that was a bummer. As soon as I did the mkfs -t xfs and rebooted, system hangs during startup right when it is looking at the disk array to find the filesystems. I am probably going to go to suse enterprise linux or redhat enterprise linux for this server and see if stability is there, but I'm really starting to lean towards the "ext3 because it works good enough" camp, and optimize elsewhere. More disks in the array, more memory in the servers, etc... -Fran -- Fran Fabrizio Senior Systems Analyst Department of Computer and Information Sciences University of Alabama at Birmingham http://www.cis.uab.edu/ 205.934.0653
> Date: Wed, 10 May 2006 00:24:23 +0100 > From: Daniel Watts <d at nielwatts.com> > Subject: Re: [Dovecot] Recommended FS for Dovecot Maildir > To: Les Mikesell <lesmikesell at gmail.com> > > Can you remember any general findings? > > Thanks to all for responding in this thread - I am reading them all > and > making notes. All very useful information. > > Can I phrase the question in a different way? Of those of you who are > running 10,000+ user system - what FS do you use? > Or do you all use special file servers (netapps, emc etc?) with > proprietary filesystems? > > DanielWe're probably right at your 10000 user threshold, but we're an ISP so we get a LOT of mail traffic. Our mail system consists of dual exim servers processing incoming mail, writing to maildirs on a home- built NFS system. The home-built system is a dual-xeon machine with lots of SATA disks in a RAID5. We then have two client-facing machines running dovecot and squirrelmail. These are all SuperMicro servers running RedHat ES 4. They all use SATA disks. We are using EXT3 for the RAID5 filesystem, and I have had no performance issues to speak of. I had considered Reiser or xfs, but as I have limited experience with either filesystem I didn't feel comfortable using them in production. -- Roger J. Weeks Systems & Network Administrator Mendocino Community Network