Mark Haney wrote:> On 09/07/2017 01:57 PM, hw wrote: >> >> Hi, >> >> is there anything that speaks against putting a cyrus mail spool onto a >> btrfs subvolume? >> > I might be the lone voice on this, but I refuse to use btrfs for anything, much less a mail spool. I used it in production on DB and Web servers and fought corruption issues and scrubs hanging the system more times than I can count. (This was within the last 24 months.) I was told by certain mailing lists, that btrfs isn't considered production level. So, I scrapped the lot, went to xfs and haven't had a problem since. > > I'm not sure why you'd want your mail spool on a filesystem and seems to hate being hammered with reads/writes. Personally, on all my mail spools, I use XFS or EXT4. OUr servers here handle 600million messages a month without trouble on those filesystems. > > Just my $0.02.Btrfs appears rather useful because the disks are SSDs, because it allows me to create subvolumes and because it handles SSDs nicely. Unfortunately, the SSDs are not suited for hardware RAID. The only alternative I know is xfs or ext4 on mdadm and no subvolumes, and md RAID has severe performance penalties which I?m not willing to afford. Part of the data I plan to store on these SSDs greatly benefits from the low latency, making things about 20--30 times faster for an important application. So what should I do?
PS: What kind of storage solutions do people use for cyrus mail spools? Apparently you can not use remote storage, at least not NFS. That even makes it difficult to use a VM due to limitations of available disk space. I?m reluctant to use btrfs, but there doesn?t seem to be any reasonable alternative. hw wrote:> Mark Haney wrote: >> On 09/07/2017 01:57 PM, hw wrote: >>> >>> Hi, >>> >>> is there anything that speaks against putting a cyrus mail spool onto a >>> btrfs subvolume? >>> >> I might be the lone voice on this, but I refuse to use btrfs for anything, much less a mail spool. I used it in production on DB and Web servers and fought corruption issues and scrubs hanging the system more times than I can count. (This was within the last 24 months.) I was told by certain mailing lists, that btrfs isn't considered production level. So, I scrapped the lot, went to xfs and haven't had a problem since. >> >> I'm not sure why you'd want your mail spool on a filesystem and seems to hate being hammered with reads/writes. Personally, on all my mail spools, I use XFS or EXT4. OUr servers here handle 600million messages a month without trouble on those filesystems. >> >> Just my $0.02. > > Btrfs appears rather useful because the disks are SSDs, because it > allows me to create subvolumes and because it handles SSDs nicely. > Unfortunately, the SSDs are not suited for hardware RAID. > > The only alternative I know is xfs or ext4 on mdadm and no subvolumes, > and md RAID has severe performance penalties which I?m not willing to > afford. > > Part of the data I plan to store on these SSDs greatly benefits from > the low latency, making things about 20--30 times faster for an important > application. > > So what should I do? > _______________________________________________ > CentOS mailing list > CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos
I hate top posting, but since you've got two items I want to comment on, I'll suck it up for now. Having SSDs alone will give you great performance regardless of filesystem.? BTRFS isn't going to impact I/O any more significantly than, say, XFS.? It does have serious stability/data integrity issues that XFS doesn't have.? There's no reason not to use SSDs for storage of immediate data and mechanical drives for archival data storage. As for VMs we run a huge Zimbra cluster in VMs on VPC with large primary SSD volumes and even larger (and slower) secondary volumes for archived mail.? It's all CentOS 6 and works very well.? We process 600 million emails a month on that virtual cluster.? All EXT4 inside LVM. I can't tell you what to do, but it seems to me you're viewing your setup from a narrow SSD/BTRFS standpoint.? Lots of ways to skin that cat. On 09/08/2017 08:07 AM, hw wrote:> > PS: > > What kind of storage solutions do people use for cyrus mail spools?? > Apparently > you can not use remote storage, at least not NFS.? That even makes it > difficult > to use a VM due to limitations of available disk space. > > I?m reluctant to use btrfs, but there doesn?t seem to be any > reasonable alternative. > > > hw wrote: >> Mark Haney wrote: >>> On 09/07/2017 01:57 PM, hw wrote: >>>> >>>> Hi, >>>> >>>> is there anything that speaks against putting a cyrus mail spool >>>> onto a >>>> btrfs subvolume? >>>> >>> I might be the lone voice on this, but I refuse to use btrfs for >>> anything, much less a mail spool. I used it in production on DB and >>> Web servers and fought corruption issues and scrubs hanging the >>> system more times than I can count.? (This was within the last 24 >>> months.)? I was told by certain mailing lists, that btrfs isn't >>> considered production level.? So, I scrapped the lot, went to xfs >>> and haven't had a problem since. >>> >>> I'm not sure why you'd want your mail spool on a filesystem and >>> seems to hate being hammered with reads/writes. Personally, on all >>> my mail spools, I use XFS or EXT4.? OUr servers here handle >>> 600million messages a month without trouble on those filesystems. >>> >>> Just my $0.02. >> >> Btrfs appears rather useful because the disks are SSDs, because it >> allows me to create subvolumes and because it handles SSDs nicely. >> Unfortunately, the SSDs are not suited for hardware RAID. >> >> The only alternative I know is xfs or ext4 on mdadm and no subvolumes, >> and md RAID has severe performance penalties which I?m not willing to >> afford. >> >> Part of the data I plan to store on these SSDs greatly benefits from >> the low latency, making things about 20--30 times faster for an >> important >> application. >> >> So what should I do?-- Mark Haney Network Engineer at NeoNova 919-460-3330 option 1 mark.haney at neonova.net www.neonova.net