Hi Everyone,
I thought I''d share some benchmarking and playing around that we had
done with making zpools from "disks" that were iscsi volumes. The
numbers are representative of 6 benchmarking rounds per.
The interesting finding at least for us was the filebench varmail
(50:50 reads-writes) results where we had a RAIDZ pool containing 9
volumes vs other combinations (we also did 8,7,6,5,4 volumes. Groups
of 3 and 9 seemed to form the boundary cases).
Regards, Jason
More detailed at http://svn.joyent.com/public/experiments/equal-sun-
iscsi-zfs-fun.txt
## Hardware Setup
- A T1000 server, 8 core, 8GB of RAM
- Equallogic PS300E storage arrays
- Standard 4 dedicated switch arrangements (http://joyeur.com/
2006/05/04/what-sysadmins-start-doing-when-hanging-around-designers)
## Software benchmarks
- FileBench (http://www.opensolaris.org/os/community/performance/
filebench/) with varmail and webserver workloads
- Bonnie++
## Questions
1) In a zpool of 3x RAIDZ groups of 3 volumes each, can we offline a
total of 3 "drives" (that could come from 3 different physical
arrays)? What are the performance differences between 9 online and 6
online drives?
2) What are the differences between zpools containing
- 3x RAIDZ groups of 3 volumes each,
- a single zpool of 9 volumes with no mirroring or RAIDZ,
- a single RAIDZ group with 9 volumes, and
- two RAIDZ groups with 9 volumes each?
3) Can we saturate a single gigabit connection between the server and
iSCSI storage?
## Findings
1) Tolerated offlining 3 of 9 drives in a 3x RAIDZ of 3 drives each.
DEGRADED (6 of 9 online) versus ONLINE (9 of 9)
a) Filebench varmail (50:50 reads-writes):
- 2045.2 ops/s in "state: DEGRADED"
- 2473.0 ops/s in "state: ONLINE"
b) Filebench webserver (90:10 reads-writes)
- 54530.5 ops/s "in state: DEGRADED"
- 54328.1 ops/s "in state: ONLINE".
2) Filebench RAIDZ of 3x3 vs "RAID0" vs RAIDZ of 1x9 vs RAIDZ of 2x9
a) Varmail (50:50 reads-writes):
- 2473.0 ops/s (RAIDZ of 3x3)
- 4316.8 ops/s (RAID0),
- 13144.8 ops/s (RAIDZ of 1x9),
- 11363.7 ops/s (RAIDZ of 2x9)
b) Webserver (90:10 reads-writes):
- 54328.1 ops/s (RAIDZ of 3x3),
- 54386.9 ops/s (RAID0),
- 53960.1 ops/s (RAIDZ of 1x9),
- 56897.2 ops/s (RAIDZ of 2x9)
3) We could saturate a single gigabit connection out to the storage.