On Mon, Feb 16, 2015 at 12:07 AM, Michael Schumacher <michael.schumacher at pamas.de> wrote:> Btw., are you sure you want to use XFS for a mail server? I made some > tests about a year ago and found that EXT4 is by the factor 10 faster > compared to XFS. The tests I performed were using the "maildir" style > postfix installation that results in many thousands files in the user > directories.This is a recent benchmarking using Postmark which supposedly simulates mail servers. XFS stacks up a bit better than ext4. http://www.phoronix.com/scan.php?page=article&item=linux-3.19-ssd-fs&num=3 A neat trick for big busy mail servers that comes up on linux-raid@ and the XFS list from time to time, is using md linear/concat to put together the physical drives into a single logical block device, and then format it XFS. XFS will create multiple AG's across all of those devices, and do parallel writes across all of them. It's often quite a bit better performing than raid0 specifically because of the many thousands of small files in many directories workload. -- Chris Murphy
On 16/02/2015 10:04, Chris Murphy wrote:> This is a recent benchmarking using Postmark which supposedly > simulates mail servers. XFS stacks up a bit better than ext4. > http://www.phoronix.com/scan.php?page=article&item=linux-3.19-ssd-fs&num=3 > > A neat trick for big busy mail servers that comes up on linux-raid@ > and the XFS list from time to time, is using md linear/concat to put > together the physical drives into a single logical block device, and > then format it XFS. XFS will create multiple AG's across all of those > devices, and do parallel writes across all of them. It's often quite a > bit better performing than raid0 specifically because of the many > thousands of small files in many directories workload.Hey Chris, I am unsure I understand what you wrote. "XFS will create multiple AG's across all of those devices," Are you comparing md linear/concat to md raid0? and that the upper level XFS will run on top them? (Just to make sure I understood what you have written.) Eliezer
On Mon, Feb 16, 2015 at 6:47 AM, Eliezer Croitoru <eliezer at ngtech.co.il> wrote:> I am unsure I understand what you wrote. > "XFS will create multiple AG's across all of those > devices," > Are you comparing md linear/concat to md raid0? and that the upper level XFS > will run on top them?Yes to the first question, I'm not understanding the second question. Allocation groups are created at mkfs time. When the workload IO involves a lot of concurrency, XFS over linear will beat XFS or ext4 over raid0. Whereas for streaming performance workloads, striped raid will work better. If redundancy is needed, mdadm permits creation of 1+linear, as compared to 10. http://xfs.org/docs/xfsdocs-xml-dev/XFS_Filesystem_Structure/tmp/en-US/html/Allocation_Groups.html You can think of XFS on linear as being something like raid0 at a file level, rather than at a block level. On a completely empty file system if you start copying a pile of dozens or more (typically hundreds or thousands) of files in mail directories, XFS distributes those across AG's and hence across all drives, in parallel. ext4 would for the most part focus all writes to the first device until mostly full, then the 2nd device, then the 3rd. And on raid0 you'll get a bunch of disk contention that isn't really necessary because everyones files are striped across all drives. So contrary to popular opinion on XFS being mainly useful for large files, it's actually quite useful for concurrent read write workflows of small files on a many disk linear/concat arrangement. This extends to using raid1 + linear instead of raid10 if some redundancy is desired. -- Chris Murphy