-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 hi, i have a system connected to a StorageTek 2530 SAS array (12 disks), on which i want to run ZFS. in the past, when using ZFS on an external array, i would simply create one LUN on the array and create the ZFS pool on this; but for various well-documented reasons (e.g. checksums, disk failure) this is usually frowned on. so, my question is: what is the best way to use this storage with ZFS? is it feasible to export 12 LUNs from the array, one for each disk, or will this impact performance? (the array has no JBOD mode, and anyway i want to keep using the array''s nvram cache.) should i create, say, 4 RAID-0 LUNs (of 3 disks each) and use RAID-Z for redundancy in the host? this seems like a reasonable compromise, as it allows ZFS to recover from errors without needing a JBOD-style array configuration. on the other hand, it would prevent ZFS from identifying exactly which disk has failed. i would like to use the entire capacity of the storage, so creating two RAID10 volumes and mirroring them in ZFS is not acceptable. the array<>host connection is using MPxIO, in case this makes a difference. thanks, river. -----BEGIN PGP SIGNATURE----- iD8DBQFJe1QLIXd7fCuc5vIRAloJAJ4lekMNbAGDS5zprtjfcYGUtr6vfQCffwOM dS3oZ6qF2taKZH5twyY3qYE=uWdg -----END PGP SIGNATURE-----
On Sat, 24 Jan 2009, River Tarnell wrote:> > i have a system connected to a StorageTek 2530 SAS array (12 disks), on which i > want to run ZFS. in the past, when using ZFS on an external array, i would > simply create one LUN on the array and create the ZFS pool on this; but for > various well-documented reasons (e.g. checksums, disk failure) this is usually > frowned on. > > so, my question is: what is the best way to use this storage with ZFS? is it > feasible to export 12 LUNs from the array, one for each disk, or will this > impact performance? (the array has no JBOD mode, and anyway i want to keep > using the array''s nvram cache.)Please make sure to read my write-up of how I configured a StorageTek 2540 array (FC version of same hardware) for use with ZFS. It can be found at "http://www.simplesystems.org/users/bfriesen/zfs-discuss/2540-zfs-performance.pdf". The 2530/2540 do not have "JBOD mode" but it is still possible to configure the hardware to behave as a JBOD. The CAM interface is a bit tedious and slow to configure but once it is done, the rest of the configuration is in ZFS''s hands and is quite speedy. Creating and destroying ZFS pools is extremely fast so you can easily test many "what if" scenarios before you make your final decision. It has been a year now, but I am still not regretting anything about my ZFS setup. Then again, I have not experienced any disk failure or even a hickup which needs to be diagnosed. It seems that the primary maintenance issue will be with identifying the physical disk which failed if ZFS detects a problem, but the 2540 array does not. Based on what I have heard, performance with the 2530 likely exceeds performance with the 2540 a bit when using ZFS. Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Bob Friesenhahn:> Please make sure to read my write-up of how I configured a StorageTek > 2540 array (FC version of same hardware) for use with ZFS. It can be > found at > "http://www.simplesystems.org/users/bfriesen/zfs-discuss/2540-zfs-performance.pdf".thanks - that''s very helpful. i was concerned that doing it this way might affect performance, but it seems that''s not the case. as for associating host disks with disks in the array, the WWN visible to the host seems to match the LUN WWN visible in the array, so it should be possible to identify which host LUN maps to each physical disk... thanks, river. -----BEGIN PGP SIGNATURE----- iD8DBQFJe3WMIXd7fCuc5vIRAhAfAKC9BvwRvFcjB6pHKPgv7+lVHZ5V1wCgrcpU v68sAjuD+nRONutYRdQmlcE=ehKo -----END PGP SIGNATURE-----
On Sat, 24 Jan 2009, River Tarnell wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > Bob Friesenhahn: >> Please make sure to read my write-up of how I configured a StorageTek >> 2540 array (FC version of same hardware) for use with ZFS. It can be >> found at >> "http://www.simplesystems.org/users/bfriesen/zfs-discuss/2540-zfs-performance.pdf". > > thanks - that''s very helpful. i was concerned that doing it this way might > affect performance, but it seems that''s not the case.It seems that the performance bottleneck is the array controller hardware (particularly for writes), followed by the connection between the host and the array. ZFS does increase the amount of I/O traffic from the host to the array (double for mirroring) but there seems to be minimal performance impact from that as long as the load is reasonably balanced. Bob =====================================Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/