Hi,
I am using lustre version: 2.5.0 and corresponding kernel 2.6.32-358.
Apart from default patches which comes with lustre 2.5.0, I applied
below patches in kernel 2.6.32-358.
raid5-configurable-cachesize-rhel6.patch
raid5-large-io-rhel5.patch
raid5-stats-rhel6.patch
raid5-zerocopy-rhel6.patch
raid5-mmp-unplug-dev-rhel6.patch
raid5-mmp-unplug-dev.patch
raid5-maxsectors-rhel5.patch
raid5-stripe-by-stripe-handling-rhel6.patch
I have taken all above patches from below link:
https://github.com/Xyratex/lustre-stable/tree/b_neo_1.4.0/lustre/kernel_patches/patches
My question is: If I am writing entire stripe then whether RAID6 md
driver need to read any of the blocks from underlying device?
I am asking this question on lustre mailing list because I have seen
that lustre community has changed RAID driver a lot.
I have created RAID6 device with default (512K) chunk size with total 6
RAID devices.
cat /sys/block/md127/queue/optimal_io_size =>2097152
I believe this is full stripe (512K * 4 data disks).
If I write 2MB data, I am expected to dirty entire stripe hence what I
believe I need not require to read either any of the data block or
parity blocks. Thus avoiding RAID6 penalties.
Whether md/raid driver supports full stripe writes by avoiding RAID 6
penalties?
I also expected 6 disks will receive 512K writes each. (4 data disk+ 2
parity disks).
If I do IO directly on block device /dev/md127, I do observe reads
happening on md device and underlying raid devices as well.
#mdstat o/p:
md127 : active raid6 sdah1[5] sdai1[4] sdaj1[3] sdcg1[2] sdch1[1]
sdci1[0]
41926656 blocks super 1.2 level 6, 512k chunk, algorithm 2
[6/6] [UUUUUU]
# raw -qa
/dev/raw/raw1: bound to major 9, minor 127
#time (dd if=/dev/zero of=/dev/raw/raw1 bs=2M count=1 && sync)
(also tried with of=/dev/md127 oflag=direct but the same results.)
# iostat shows:
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sdaj1 7.00 0.00 205.20 0 1026
sdai1 6.20 0.00 205.20 0 1026
sdah1 9.80 0.00 246.80 0 1234
sdcg1 6.80 0.00 205.20 0 1026
sdci1 9.60 0.00 246.80 0 1234
sdch1 6.80 0.00 205.20 0 1026
md127 0.80 0.00 819.20 0 4096
I assume if I perform writes in multiples of "optimal_io_size" I would
be doing full stripe writes thus avoiding reads.
But unfortunately with two 2M writes, I do see reads happening for some
these drives.
Same case for count=4 or 6 (equal to data disks or total disks).
# time (dd if=/dev/zero of=/dev/raw/raw1 bs=2M count=2 && sync)
Device: tps Blk_read/s Blk_wrtn/s Blk_read Blk_wrtn
sdaj1 13.40 204.80 410.00 1024 2050
sdai1 11.20 0.00 410.00 0 2050
sdah1 15.80 0.00 464.40 0 2322
sdcg1 13.20 204.80 410.00 1024 2050
sdci1 16.60 0.00 464.40 0 2322
sdch1 12.40 192.00 410.00 960 2050
md127 1.60 0.00 1638.40 0 8192
I believe RAID6 penalties will exist if it's a random write, but in case
of seq. write, whether they will still exist in some other form in Linux
md/raid driver?
My aim is to maximize RAID6 Write IO rate with sequential Writes
withoutRAID6 penalties.
Rectify me wherever my assumptions are wrong. Let me know if any other
configuration param (for block device or md device) is required to
achieve the same.
Thanks,
Aayush
_______________________________________________
Lustre-devel mailing list
Lustre-devel-aLEFhgZF4x6X6Mz3xDxJMA@public.gmane.org
http://lists.lustre.org/mailman/listinfo/lustre-devel