I recently put together a new system to act as a media server for home. As
such, the most important thing is that it has lots of storage, but as long
as it can stream data fast enough for clients other general computing
performance doesn''t matter much.
I put together a system containing:
1x Intel BOXD945GCLF2
http://www.newegg.com/Product/Product.aspx?Item=N82E16813121359
1x Crucial 2GB DIMM
http://www.newegg.com/Product/Product.aspx?Item=N82E16820148097
1x Supermicro AOC-SAT2-MV8
http://www.newegg.com/Product/Product.aspx?Item=N82E16815121009
1x Western Digital WD1600AAJS 160GB
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136075
8x Western Digital WD10EADS 1TB
http://www.newegg.com/Product/Product.aspx?Item=N82E16822136317
1x Supermicro CSE-743TQ-865B-SQ
http://www.newegg.com/Product/Product.aspx?Item=N82E16811152117
Some notes on the build:
The 160GB drive is one that I had laying around. It''s the
system/swap/dump
drive and is attached to the motherboard''s ICH7.
The 1TB drives are connected to the AOC-SAT2-MV8 via the case''s
backplane.
The SAS/SATA backplane taps the SATA channel for drive activity, so the disk
light just work. It''s very cool to watch. I''m easily amused.
The case is very nice, with one rear 92mm PWM fan and two 80mm PWM fans in
the midplane. They''re all hot-plug fans. There are carriers for two
more
midplane fans and another rear fan. Unfortunately, the Intel board only has
a single 3-pin header. I will need to buy a molex to fan adapter, but in the
short term I''ve left the case open and a small desk fan blowing air in.
The
case is very quiet for something so massive, but that might change because I
will probably add two more 80mm fans soon. I''d absolutely buy another
Supermicro 743 case in the future, but probably one with a smaller power
supply that already had the extra fans.
The front panel cable requires a split-out adapter to attach to headers on
the board, and doesn''t fit snuggly on the pins. The lowest SATA plug on
the
AOC-SAT2-MV8 interferes with the front display header on the motherboard. As
such, I bent the on/off pins about 20 degrees and only attached that pair.
The Intel board was a bit picky about booting off my USB DVD drive until I
flashed the BIOS, and I wasn''t able to get it to boot off of a thumb
drive
to install.
I''ve done some benchmarking of different pool sizes with Bonnie++. The
zpool
is created, 10 second sleep, the test is run, then the pool is destroyed and
a new zpool configuration is tried.
I used the following test to try to cover configurations for folks with 2, 4
or 8 drive bays.
No redundancy:
1, 2, 4 and 8 disk (RAID 0)
Mirrors:
2-disk (RAID1)
2 x 2-disk (RAID 10)
zpool:
3, 4 and 8 disk (RAID 5)
2 x 4 disk (RAID 50)
zpool2:
4 and 8 disk
Even with a very weak CPU the system is close to saturating the PCI bus for
reads with most configurations.
And now for the data:
Filesystem size used avail capacity Mounted on
tank 914G 19K 914G 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 22223 67 49756 26 35130 26 30559 98 83006 23
292.0 2
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8321 99 +++++ +++ 12321 100 6260 99 +++++ +++ 12837
99
Filesystem size used avail capacity Mounted on
tank 1.8T 19K 1.8T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 22670 68 52327 27 38701 29 30490 98 98079 27
552.2 5
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8309 100 +++++ +++ 12413 100 6246 100 +++++ +++ 12797
100
Filesystem size used avail capacity Mounted on
tank 3.6T 19K 3.6T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
c1t5d0 ONLINE 0 0 0
c1t6d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 22969 69 53778 28 39717 30 30610 98 99164 27
829.9 7
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8248 99 +++++ +++ 12363 100 6210 100 +++++ +++ 12656
100
Filesystem size used avail capacity Mounted on
tank 7.1T 19K 7.1T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
c1t5d0 ONLINE 0 0 0
c1t6d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 23037 69 54069 29 39652 30 30480 98 99930 27 1304
13
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8515 99 +++++ +++ 12470 100 6425 100 +++++ +++ 12747
100
Filesystem size used avail capacity Mounted on
tank 914G 19K 914G 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror ONLINE 0 0 0
c1t6d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 27977 85 32954 18 25611 19 29790 96 98799 28
507.0 4
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8325 98 +++++ +++ 12194 100 8495 100 +++++ +++ 13069
99
Filesystem size used avail capacity Mounted on
tank 1.8T 19K 1.8T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror ONLINE 0 0 0
c1t6d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
mirror ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 28410 87 32987 17 26155 20 30003 96 99355 28
762.3 7
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8158 99 +++++ +++ 12022 99 6030 99 +++++ +++ 12337
100
Filesystem size used avail capacity Mounted on
tank 1.8T 25K 1.8T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 20347 62 39846 21 28824 22 30290 98 98587 30
472.1 5
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8133 99 +++++ +++ 11997 100 7695 99 +++++ +++ 11165
94
Filesystem size used avail capacity Mounted on
tank 2.7T 28K 2.7T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 21289 65 41549 22 33384 26 29623 96 98639 31
463.1 6
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8247 97 +++++ +++ 12206 100 8231 100 +++++ +++ 13038
99
Filesystem size used avail capacity Mounted on
tank 6.2T 33K 6.2T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
c1t5d0 ONLINE 0 0 0
c1t6d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 22551 69 46825 25 33098 26 28163 91 90284 28
506.5 8
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8486 99 +++++ +++ 12433 100 6000 100 +++++ +++ 10731
99
Filesystem size used avail capacity Mounted on
tank 5.3T 28K 5.3T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
raidz1 ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
c1t5d0 ONLINE 0 0 0
c1t6d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 21079 64 42316 22 31645 24 28227 91 97882 29
758.8 9
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8512 99 +++++ +++ 7936 93 7854 99 +++++ +++ 12560
99
Filesystem size used avail capacity Mounted on
tank 1.8T 28K 1.8T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 21046 65 28221 15 23178 18 27971 90 93931 27
441.1 5
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8455 99 +++++ +++ 12361 100 6369 99 +++++ +++ 12728
99
Filesystem size used avail capacity Mounted on
tank 5.3T 43K 5.3T 1% /tank
pool: tank
state: ONLINE
scrub: none requested
config:
NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
raidz2 ONLINE 0 0 0
c1t0d0 ONLINE 0 0 0
c1t1d0 ONLINE 0 0 0
c1t2d0 ONLINE 0 0 0
c1t3d0 ONLINE 0 0 0
c1t4d0 ONLINE 0 0 0
c1t5d0 ONLINE 0 0 0
c1t6d0 ONLINE 0 0 0
c1t7d0 ONLINE 0 0 0
errors: No known data errors
Version 1.03c ------Sequential Output------ --Sequential Input-
--Random-
-Per Chr- --Block-- -Rewrite- -Per Chr- --Block--
--Seeks--
Machine Size K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec
%CP
zfs 4G 21404 65 42320 22 29985 23 26391 85 80496 24 807.2
10
------Sequential Create------ --------Random
Create--------
-Create-- --Read--- -Delete-- -Create-- --Read---
-Delete--
files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec
%CP
16 8472 99 +++++ +++ 12350 100 6319 100 +++++ +++ 12677
100
--
Brandon High : bhigh at freaks.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090331/682e41de/attachment.html>