zfs discuss - Sep 2009 - Comments on home OpenSolaris/ZFS server

Hello,

I have been researching building a home storage server based on  
OpenSolaris and ZFS, and I would appreciate any time people could take  
to comment on my current leanings.

I''ve tried to gather old information from this list as well as the  
HCL, but I would welcome anyone''s experience on both compatibility and
appropriateness for my goals.  I''d love if that white box server wiki  
page were set up now, but for now I''ll have to just ask here.

My priorities:

1)  Data security.  I''m hoping I can get this via ECC RAM and  
enterprise drives that hopefully don''t lie to ZFS about flushing to  
disk?  I''ll run mirrored pools for redundancy (which leads me to want  
a case w/a lot of bays).
2)  Compatibility.  For me this translates into low upkeep cost  
(time).  I''m not looking to be the first person to get OpenSolaris  
running on some particular piece of hardware.
3)  Scaleable.  I''d like to not have to upgrade every year.  I can  
always use something like an external JBOD array, but there''s some  
appeal to having enough space in the case for reasonable growth.  I''d  
also like to have enough performance to keep up with scaling data  
volume and ZFS features.
4)  Ability to run some other (lightweight) services on the box.  I''ll
be using NFS (iTunes libraries for OS X clients) and iSCSI (Time  
Machine backups) primarily, but my current home server also runs a few  
small services (MySQL etc...) that are very lightweight but  
nevertheless might be difficult to do on a ZFS (or "ZFS like")
appliance
5)  Cost.  All things being equal cheaper is better, but I''m willing  
to pay more to accomplish particularly 1-3 above.

My current thinking:

SuperMicro 7046A-3 Workstation
http://supermicro.com/products/system/4U/7046/SYS-7046A-3.cfm
8 hot swappable drive bays (SAS or SATA, I''d use SATA)
Network/Main board/SAS/SATA controllers seem well supported by  
OpenSolaris
Will take IPMI card for remote admin (with video and iso redirection)
12 RAM slots so I can buy less dense chips
2x 5.25" drive bays.  I''d use a SuperMicro Mobile Rack M14T
(http://www.supermicro.com/products/accessories/mobilerack/CSE-M14.cfm
) to get 4 2.5" SAS drives in one of these.  2 would be used for a  
mirrored boot pool leaving 2 for potential future use (like a ZIL on  
SSD).

Nehalem E5520 CPU
These are clearly more than enough now, but I''m hoping to have decent  
CPU performance for say 5 years (and I''m willing to pay for it up  
front vs. upgrading every 2 years...I don''t want this to be too time  
consuming of a hobby).  I''d like to have processor capacity for  
compression and (hopefully reasonably soon) de-duplication as well as  
obviously support ECC RAM.

Crucial RAM in 4 GB density (price scales linearly up through this  
point and I''ve had good support from Crucial)

Seagate Barracuda ES.2 1TB SATA (Model ST31000340NS) for storage  
pool.  I would like to use a larger drive, but I can''t find anything  
rated to run 24x7 larger than 1TB from Seagate.  I''d like to have  
drives rated for 24x7 use, and I''ve had good experience w/Seagate.   
Again, a larger case gives me some flexibility here.

Misc (mainly interested in compatibility b/c it will hardly be used):
Sun XVR-100 video card from eBay
Syba SY-PCI45004
(http://www.newegg.com/Product/Product.aspx?Item=N82E16816124025
) IDE card for CD-ROM
Sony DDU1678A (http://www.newegg.com/Product/Product.aspx?Item=N82E16827131061 
) CD-ROM

Thanks a lot for any thoughts you might have.

--Ware

This seems like you''re doing an awful lot of planning for only 8 SATA
+ 4 SAS bays?

I agree - SOHO usage of ZFS is still a scary "will this work?" deal. I
found a working setup and I cloned it. It gives me 16x SATA + 2x SATA
for mirrored boot, 4GB ECC RAM and a quad core processor - total cost
without disks was ~ $1k I believe. Not too shabby. Emphasis was also
for acoustics - rack dense would be great but my current living
situation doesn''t warrant that. The noisiest components are the 5-in-3
chassis used in the front of the case. I have to keep the fans on high
(I tried to swap out for larger, quieter fans, but could not get the
fan alarm to shut up) or they go over Seagate''s recommended <= 50
degrees.

I really should post my parts list up on my blog. I had to choose
everything to the best of my research online and hope for the best.


On Mon, Sep 28, 2009 at 1:12 PM, Ware Adams <rwalists at washdcmail.com>
wrote:
> Hello,
>
> I have been researching building a home storage server based on OpenSolaris
> and ZFS, and I would appreciate any time people could take to comment on my
> current leanings.
>
> I''ve tried to gather old information from this list as well as the
HCL, but
> I would welcome anyone''s experience on both compatibility and
> appropriateness for my goals. ?I''d love if that white box server
wiki page
> were set up now, but for now I''ll have to just ask here.
>
> My priorities:
>
> 1) ?Data security. ?I''m hoping I can get this via ECC RAM and
enterprise
> drives that hopefully don''t lie to ZFS about flushing to disk?
?I''ll run
> mirrored pools for redundancy (which leads me to want a case w/a lot of
> bays).
> 2) ?Compatibility. ?For me this translates into low upkeep cost (time).
?I''m
> not looking to be the first person to get OpenSolaris running on some
> particular piece of hardware.
> 3) ?Scaleable. ?I''d like to not have to upgrade every year. ?I can
always
> use something like an external JBOD array, but there''s some appeal
to having
> enough space in the case for reasonable growth. ?I''d also like to
have
> enough performance to keep up with scaling data volume and ZFS features.
> 4) ?Ability to run some other (lightweight) services on the box.
?I''ll be
> using NFS (iTunes libraries for OS X clients) and iSCSI (Time Machine
> backups) primarily, but my current home server also runs a few small
> services (MySQL etc...) that are very lightweight but nevertheless might be
> difficult to do on a ZFS (or "ZFS like") appliance
> 5) ?Cost. ?All things being equal cheaper is better, but I''m
willing to pay
> more to accomplish particularly 1-3 above.
>
> My current thinking:
>
> SuperMicro 7046A-3 Workstation
> http://supermicro.com/products/system/4U/7046/SYS-7046A-3.cfm
> 8 hot swappable drive bays (SAS or SATA, I''d use SATA)
> Network/Main board/SAS/SATA controllers seem well supported by OpenSolaris
> Will take IPMI card for remote admin (with video and iso redirection)
> 12 RAM slots so I can buy less dense chips
> 2x 5.25" drive bays. ?I''d use a SuperMicro Mobile Rack M14T
> (http://www.supermicro.com/products/accessories/mobilerack/CSE-M14.cfm) to
> get 4 2.5" SAS drives in one of these. ?2 would be used for a mirrored
boot
> pool leaving 2 for potential future use (like a ZIL on SSD).
>
> Nehalem E5520 CPU
> These are clearly more than enough now, but I''m hoping to have
decent CPU
> performance for say 5 years (and I''m willing to pay for it up
front vs.
> upgrading every 2 years...I don''t want this to be too time
consuming of a
> hobby). ?I''d like to have processor capacity for compression and
(hopefully
> reasonably soon) de-duplication as well as obviously support ECC RAM.
>
> Crucial RAM in 4 GB density (price scales linearly up through this point
and
> I''ve had good support from Crucial)
>
> Seagate Barracuda ES.2 1TB SATA (Model ST31000340NS) for storage pool. ?I
> would like to use a larger drive, but I can''t find anything rated
to run
> 24x7 larger than 1TB from Seagate. ?I''d like to have drives rated
for 24x7
> use, and I''ve had good experience w/Seagate. ?Again, a larger case
gives me
> some flexibility here.
>
> Misc (mainly interested in compatibility b/c it will hardly be used):
> Sun XVR-100 video card from eBay
> Syba SY-PCI45004
> (http://www.newegg.com/Product/Product.aspx?Item=N82E16816124025) IDE card
> for CD-ROM
> Sony DDU1678A
> (http://www.newegg.com/Product/Product.aspx?Item=N82E16827131061) CD-ROM
>
> Thanks a lot for any thoughts you might have.
>
> --Ware
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

On Sep 28, 2009, at 4:20 PM, Michael Shadle wrote:
> I agree - SOHO usage of ZFS is still a scary "will this work?"
deal. I
> found a working setup and I cloned it. It gives me 16x SATA + 2x SATA
> for mirrored boot, 4GB ECC RAM and a quad core processor - total cost
> without disks was ~ $1k I believe. Not too shabby. Emphasis was also
> for acoustics - rack dense would be great but my current living
> situation doesn''t warrant that
This sounds interesting.  Do you have any info on it (case you started  
with, etc...).

I''m concerned about noise too as this will be in a closet close to the
room where our television is.  Currently there is a MacPro in there  
which isn''t terribly quiet, but the SuperMicro case is reported to be  
fairly quiet.

Thanks,
Ware

Yeah - give me a bit to rope together the parts list and double check
it, and I will post it on my blog.


On Mon, Sep 28, 2009 at 2:34 PM, Ware Adams <rwalists at washdcmail.com>
wrote:
> On Sep 28, 2009, at 4:20 PM, Michael Shadle wrote:
>
>> I agree - SOHO usage of ZFS is still a scary "will this
work?" deal. I
>> found a working setup and I cloned it. It gives me 16x SATA + 2x SATA
>> for mirrored boot, 4GB ECC RAM and a quad core processor - total cost
>> without disks was ~ $1k I believe. Not too shabby. Emphasis was also
>> for acoustics - rack dense would be great but my current living
>> situation doesn''t warrant that
>
> This sounds interesting. ?Do you have any info on it (case you started
with,
> etc...).
>
> I''m concerned about noise too as this will be in a closet close to
the room
> where our television is. ?Currently there is a MacPro in there which
isn''t
> terribly quiet, but the SuperMicro case is reported to be fairly quiet.
>
> Thanks,
> Ware
>

personally i like this case:


http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021

it''s got 20 hot swap bays, and it''s surprisingly well built. 
For the money,
it''s an amazing deal.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090928/ec0ed360/attachment.html>

rackmount chassis aren''t usually designed with acoustics in mind :)

however i might be getting my closet fitted so i can put half a rack
in. might switch up my configuration to rack stuff soon.

On Mon, Sep 28, 2009 at 3:04 PM, Thomas Burgess <wonslung at gmail.com>
wrote:
> personally i like this case:
>
>
> http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
>
> it''s got 20 hot swap bays, and it''s surprisingly well
built.? For the money,
> it''s an amazing deal.
>
>
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
>

i own this case, it''s really not that bad.  It''s got 4 fans
but they are
really big and don''t make nearly as much noise as you''d think.
honestly,
it''s not bad at all.  I know someone who sits it vertically as well,
honestly, it''s a good case for the money


On Mon, Sep 28, 2009 at 6:06 PM, Michael Shadle <mike503 at gmail.com>
wrote:
> rackmount chassis aren''t usually designed with acoustics in mind
:)
>
> however i might be getting my closet fitted so i can put half a rack
> in. might switch up my configuration to rack stuff soon.
>
> On Mon, Sep 28, 2009 at 3:04 PM, Thomas Burgess <wonslung at
gmail.com>
> wrote:
> > personally i like this case:
> >
> >
> > http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
> >
> > it''s got 20 hot swap bays, and it''s surprisingly
well built.  For the
> money,
> > it''s an amazing deal.
> >
> >
> >
> > _______________________________________________
> > zfs-discuss mailing list
> > zfs-discuss at opensolaris.org
> > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
> >
> >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090928/22afbdcd/attachment.html>

well when i start looking into rack configurations i will consider it. :)

here''s my configuration - enjoy!
http://michaelshadle.com/2009/09/28/my-recipe-for-zfs-at-home/

On Mon, Sep 28, 2009 at 3:10 PM, Thomas Burgess <wonslung at gmail.com>
wrote:
> ?i own this case, it''s really not that bad.? It''s got 4
fans but they are
> really big and don''t make nearly as much noise as you''d
think.? honestly,
> it''s not bad at all.? I know someone who sits it vertically as
well,
> honestly, it''s a good case for the money
>
>
> On Mon, Sep 28, 2009 at 6:06 PM, Michael Shadle <mike503 at
gmail.com> wrote:
>>
>> rackmount chassis aren''t usually designed with acoustics in
mind :)
>>
>> however i might be getting my closet fitted so i can put half a rack
>> in. might switch up my configuration to rack stuff soon.
>>
>> On Mon, Sep 28, 2009 at 3:04 PM, Thomas Burgess <wonslung at
gmail.com>
>> wrote:
>> > personally i like this case:
>> >
>> >
>> > http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
>> >
>> > it''s got 20 hot swap bays, and it''s surprisingly
well built.? For the
>> > money,
>> > it''s an amazing deal.
>> >
>> >
>> >
>> > _______________________________________________
>> > zfs-discuss mailing list
>> > zfs-discuss at opensolaris.org
>> > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>> >
>> >
>
>

On Mon, Sep 28, 2009 at 06:04:01PM -0400, Thomas Burgess
wrote:
> personally i like this case:
> 
> 
> http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
> 
> it''s got 20 hot swap bays, and it''s surprisingly well
built.  For the money,
> it''s an amazing deal.
You don''t like
http://www.supermicro.com/products/nfo/chassis_storage.cfm ?
I must admit I don''t have a price list of these.

When running that many hard drives I would insist on redundant
power supplies, and server motherboards with ECC memory. Unless
it''s for home use, where a downtime of days or weeks is not critical.

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a>
http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE

I think it *IS* for home use.  I like the supermicro stuff, i just
personally find it to be a little pricy for a home NAS.   I personally find
the norco 4020''s to be the best deal for a home nas.  I LOVE mine. 
I''m
about to build a second one.

On Tue, Sep 29, 2009 at 2:41 AM, Eugen Leitl <eugen at leitl.org> wrote:
> On Mon, Sep 28, 2009 at 06:04:01PM -0400, Thomas Burgess wrote:
> > personally i like this case:
> >
> >
> > http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
> >
> > it''s got 20 hot swap bays, and it''s surprisingly
well built.  For the
> money,
> > it''s an amazing deal.
>
> You don''t like
http://www.supermicro.com/products/nfo/chassis_storage.cfm?
> I must admit I don''t have a price list of these.
>
> When running that many hard drives I would insist on redundant
> power supplies, and server motherboards with ECC memory. Unless
> it''s for home use, where a downtime of days or weeks is not
critical.
>
> --
> Eugen* Leitl <a href="http://leitl.org">leitl</a>
http://leitl.org
> ______________________________________________________________
> ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
> 8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090929/b1f8f474/attachment.html>

On Sep 29, 2009, at 2:41 AM, Eugen Leitl wrote:
> On Mon, Sep 28, 2009 at 06:04:01PM -0400, Thomas Burgess wrote:
>> personally i like this case:
>>
>>
>> http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
>>
>> it''s got 20 hot swap bays, and it''s surprisingly well
built.  For
>> the money,
>> it''s an amazing deal.
>
> You don''t like
http://www.supermicro.com/products/nfo/chassis_storage.cfm
>  ?
> I must admit I don''t have a price list of these.
>
> When running that many hard drives I would insist on redundant
> power supplies, and server motherboards with ECC memory. Unless
> it''s for home use, where a downtime of days or weeks is not
critical.
I hadn''t thought of going that way because I was looking for at least  
a somewhat pre-packaged system, but another posted pointed out how  
many more drives I could get by choosing case/motherboard separately.   
I agree, with this much trouble it doesn''t make sense to settle for  
fewer drive slots than I can get.

I agree completely with the ECC.  It''s for home use, so the power  
supply issue isn''t huge (though if it''s possible
that''s a plus).  My
concern with this particular option is noise.  It will be in a closet,  
but one with louvered doors right off a room where people watch TV.   
Anything particularly loud would be an issue.  The comments on Newegg  
make this sound pretty loud.  Have you tried one outside of a server  
room environment?

Thanks,
Ware

> On Mon, Sep 28, 2009 at 06:04:01PM -0400, Thomas Burgess wrote:
>> personally i like this case:
>>
>>
>> http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
>>
>> it''s got 20 hot swap bays, and it''s surprisingly well
built.  For the
>> money,
>> it''s an amazing deal.
>
> You don''t like
http://www.supermicro.com/products/nfo/chassis_storage.cfm
> ?
> I must admit I don''t have a price list of these.
>
I am using an SC846xxx for a project here at work.
The hardware consists of an ASUS server-level motherboard with 2 quad-core
Xeons, 8GB of RAM, an LSI PCI-e SAS/SATA card, and 24 1.5TB HD, all in one
of these cases.
The drives are in one pool with 3x 7+1 raid-z sets. Raw is 32TB, usable is
about 24TB. Total price was about $6000. (It''d be about $800 less now
that
1.5TB drives have dropped in price.)

I built it for disk to disk backups. Right now, I''m using backuppc for
backing up the OS''es of our DB servers and such, and rsync and
snapshots
for the databases themselves.
I get about 50MB/sec read and write speeds, but I think that''s because
the
version of the SC846 I got has a single backplane for the SAS/SATA drives,
and one connector to the LSI card. Of course, for what I''m doing,
that''s
fine.

Paul

Oh, I think the SC846 I got was about $1100.
http://www.cdw.com/shop/search/results.aspx?key=sc846&searchscope=All&sr=1&Find+it.x=0&Find+it.y=0

9:51am, Ware Adams wrote:
> On Sep 29, 2009, at 9:32 AM, paul at paularcher.org wrote:
>
>> I am using an SC846xxx for a project here at work.
>> The hardware consists of an ASUS server-level motherboard with 2
quad-core
>> Xeons, 8GB of RAM, an LSI PCI-e SAS/SATA card, and 24 1.5TB HD, all in
one
>> of these cases.
>> The drives are in one pool with 3x 7+1 raid-z sets. Raw is 32TB, usable
is
>> about 24TB. Total price was about $6000. (It''d be about $800
less now that
>> 1.5TB drives have dropped in price.)
>
> If I can go with something like this it''s going to be the easiest
way to get
> lots of drives.  Do you have this outside of a server room?  Would the
noise
> be manageable if say it were mounted in an enclosed rack with sound 
> deadening?
>
It''s in a server room, but I had it here in the office while I was
putting
it together. The case really isn''t too loud. 24 hard drives make a fair
bit of noise--but I think if you had it in a closet with some 
soundproofing, it wouldn''t be bad. And if you went with a smaller 
enclosure (12 drives, for instance) that would help.

Paul

>> You don''t like
http://www.supermicro.com/products/nfo/chassis_storage.cfm
>> ?
>> I must admit I don''t have a price list of these.
>>
>
> I am using an SC846xxx for a project here at work.
> The hardware consists of an ASUS server-level motherboard with 2 quad-core
> Xeons, 8GB of RAM, an LSI PCI-e SAS/SATA card, and 24 1.5TB HD, all in one
> of these cases.
> The drives are in one pool with 3x 7+1 raid-z sets. Raw is 32TB, usable is
> about 24TB. Total price was about $6000. (It''d be about $800 less
now that
> 1.5TB drives have dropped in price.)
>
> I built it for disk to disk backups. Right now, I''m using backuppc
for
> backing up the OS''es of our DB servers and such, and rsync and
snapshots
> for the databases themselves.
> I get about 50MB/sec read and write speeds, but I think that''s
because the
> version of the SC846 I got has a single backplane for the SAS/SATA drives,
> and one connector to the LSI card. Of course, for what I''m doing,
that''s
> fine.
>
> Paul
>
> Oh, I think the SC846 I got was about $1100.
>
http://www.cdw.com/shop/search/results.aspx?key=sc846&searchscope=All&sr=1&Find+it.x=0&Find+it.y=0
>
One thing I forgot to mention: there is a wart with this case. The 
connectors for the low-profile CDROM drive are too short, and the power 
connector for the internal drive hits the lid of the case. I actually had 
to find a low-profile molex power connector for the hard drive, and I can 
only use the CDROM drive if I open the case up and loosen the internal 
hard drive so I can plug the CDROM in. Otherwise, though, the case is very 
well built.

Paul

On Tue, 29 Sep 2009, rwalists at washdcmail.com wrote:
>
> I agree completely with the ECC.  It''s for home use, so the power
supply
> issue isn''t huge (though if it''s possible that''s
a plus).  My concern with
> this particular option is noise.  It will be in a closet, but one with 
> louvered doors right off a room where people watch TV.  Anything
particularly
> loud would be an issue.  The comments on Newegg make this sound pretty
loud.
> Have you tried one outside of a server room environment?
Definitely lose the louvered doors and install solid-core doors.  I 
have most of my stuff in a large closet.  There is 8" ducting and 
duplex ventilation fans (Broan BRO-L250L, 
http://www.airshack.com/store/index.cfm?fuseaction=getproductpg&manuf=broan&series=losone%20i-in-line)
to assure that there is always 250 CFM of air running through it. 
The fans are hung with nylon straps from the rafters in the attic so 
they are totally silent.

With the doors open, the gigabit ethernet switch is the noisiest but 
the doors block that high pitched noise.  If the temperature hits 82.5 
degrees, then the StorageTek 2540 fans go on high and then there is 
audible sound outside the room.  The key to silence is to keep the 
equipment cool enough that their fans run on low speed.  If the doors 
were replaced with solid core doors, then there would be a lot more 
silence.

Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

On Tue, Sep 29, 2009 at 07:28:13AM -0400, rwalists at washdcmail.com wrote:
> I agree completely with the ECC.  It''s for home use, so the power
> supply issue isn''t huge (though if it''s possible
that''s a plus).  My
> concern with this particular option is noise.  It will be in a closet,  
> but one with louvered doors right off a room where people watch TV.   
> Anything particularly loud would be an issue.  The comments on Newegg  
> make this sound pretty loud.  Have you tried one outside of a server  
> room environment?
No, basically all rackmount gear (especially 1-2 height units) which 
dissipates nontrivial power is loud, since it has to maintain air flow, 
which at small geometries means high-rpm and high-pitched noise. I''ve
just hauled a 3U Supermicro chassis from my office, where it has
been running for a couple weeks into a server room, where such
systems belongs.

The only way to deal with that is watercooling, or operating the thing
out of your earshot (cellar, etc). 

There are large enclosures with large, slow-moving fans which are
suitable for the living room, but I doubt you can miss 16-24 drives
in action. 

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a>
http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE

On Tue, September 29, 2009 01:41, Eugen Leitl wrote:
> Unless
> it''s for home use, where a downtime of days or weeks is not
critical.
I hate to think what would happen if I were to tell my housemates that
critical services would be down for a WEEK!
-- 
David Dyer-Bennet, dd-b at dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

On Tue, 29 Sep 2009, Eugen Leitl wrote:
>
> No, basically all rackmount gear (especially 1-2 height units) which
> dissipates nontrivial power is loud, since it has to maintain air flow,
> which at small geometries means high-rpm and high-pitched noise.
I''ve
The good news is that high-pitched noise is the easiest to block out 
via a server-closet.  It is mostly blocked by even a panel door.  Any 
low-pitched noise is hardest to eliminate since it is transmitted as 
vibration.  The key here is to isolate the vibration.  Using an 
equipment rack helps but there could be transmission via the floor. 
Hanging everything via the nylon straps that you will find in the 
plumbing/AC section of the hardware store is by far the best way to 
eliminate transmission of vibration.

Bob
--
Bob Friesenhahn
bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer,    http://www.GraphicsMagick.org/

On Tue, Sep 29, 2009 at 7:28 AM, <rwalists at washdcmail.com> wrote:
> On Sep 29, 2009, at 2:41 AM, Eugen Leitl wrote:
>
>  On Mon, Sep 28, 2009 at 06:04:01PM -0400, Thomas Burgess wrote:
>>
>>> personally i like this case:
>>>
>>>
>>> http://www.newegg.com/Product/Product.aspx?Item=N82E16811219021
>>>
>>> it''s got 20 hot swap bays, and it''s surprisingly
well built.  For the
>>> money,
>>> it''s an amazing deal.
>>>
>>
>> You don''t like
http://www.supermicro.com/products/nfo/chassis_storage.cfm
>>  ?
>> I must admit I don''t have a price list of these.
>>
>> When running that many hard drives I would insist on redundant
>> power supplies, and server motherboards with ECC memory. Unless
>> it''s for home use, where a downtime of days or weeks is not
critical.
>>
>
> I hadn''t thought of going that way because I was looking for at
least a
> somewhat pre-packaged system, but another posted pointed out how many more
> drives I could get by choosing case/motherboard separately.  I agree, with
> this much trouble it doesn''t make sense to settle for fewer drive
slots than
> I can get.
>
> For the money, it''s a much better option. you''ll be able
to afford many
more drives.  In my opinion, for a home system, the more you can save on the
case and power supply, the more hard drives you can buy.  Right now 1 TB and
1.5 TB drives seem to be the best.  I used 1 TB drives and 2 compact flash
cards for the os (with sata to compact flash adapters)  They are really
small and easy to find a place to mount, which allows you to use the hotswap
bays for even more storage.

> I agree completely with the ECC.  It''s for home use, so the power
supply
> issue isn''t huge (though if it''s possible that''s
a plus).  My concern with
> this particular option is noise.  It will be in a closet, but one with
> louvered doors right off a room where people watch TV.  Anything
> particularly loud would be an issue.  The comments on Newegg make this
sound
> pretty loud.  Have you tried one outside of a server room environment?
>
> Yes, ecc is nice.  They also sell dual powersupplies that FIT in a single
atx slot.  Just look around.  The noise isn''t THAT bad.  If you have it
in a
closet i''ll be very surprised if it''s a problem,
that''s exactly what i do
and it''s in the SAME room as the tv and i don''t notice it.  I
have 2 norco
4020''s in there and 2 more 2u servers, one is running my router
software
(openbsd with pf) and the other is an older hp proliant box  I don''t
have a
problem at all with the noise.  I highly recommend that case.  It''s not
designed to be quiet, but if you replace the stock fans with low noise fans,
it''s much much much more quiet than you''d think. and
it''s designed well
enough to keep the drives cool.  It''s perfect for home and small office
use,
and will allow you to put more money into buying storage, which is the POINT
of what you are doing ANYWAYS.


> Thanks,
> Ware
>
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090930/d3bafbd9/attachment.html>

On Wed, September 30, 2009 07:14, Thomas Burgess wrote:
> For the money, it''s a much better option. you''ll be able
to afford many
> more drives.  In my opinion, for a home system, the more you can save on
> the
> case and power supply, the more hard drives you can buy.  Right now 1 TB
> and
> 1.5 TB drives seem to be the best.  I used 1 TB drives and 2 compact flash
> cards for the os (with sata to compact flash adapters)  They are really
> small and easy to find a place to mount, which allows you to use the
> hotswap bays for even more storage.
I''ve been running a home ZFS server for a while now; mine currently has
two two-way mirrors of 400GB disks, i.e. 800GB usable data space.  I''ve
got a couple hundred GB free currently.  This server holds my music
collection, plus my digital photography, plus what''s scanned of my film
photography, plus my ebook collection, plus the usual random personal
files &c.  And it serves as a backup pool for several laptops.

I can see that people heavily active in live audio or (especially) video
recording would fill disks considerably faster than my still photography
does (about 12MB per image, before I start editing it and storing extra
copies).  But I have to say that I''m finding the size NAS boxes people
are
building for what they call "home use" to be rather startling. 
I''m using
4 400GB disks with 100% redundancy; lots of people are talking about using
8 or more 1TB or bigger disks with 25% redundancy.  That''s a hugely
bigger
pool!  Do you actually fill up that space?  With what?

I''ve got 8 hot-swap bays but only 6 controller channels on the
motherboard.  And I''m using two of those for the boot disks. 
I''ve thought
about going away from rotating disks for boot, to something like CF cards,
or USB.  USB is slow, but will that hurt me any when the system is being a
file server?

What going to USB does for me is free up two SATA controllers, so I can
expand my pool without buying another controller and messing about inside
the box.  Also, I don''t need a mirrored pool for boot if it''s
a cheap USB
drive and I can keep a spare copy or two, and just swap them if there''s
any problem with the first one.

-- 
David Dyer-Bennet, dd-b at dd-b.net; http://dd-b.net/
Snapshots: http://dd-b.net/dd-b/SnapshotAlbum/data/
Photos: http://dd-b.net/photography/gallery/
Dragaera: http://dragaera.info

On Wed, Sep 30, 2009 at 10:48 AM, David Dyer-Bennet <dd-b at dd-b.net>
wrote:
>
> On Wed, September 30, 2009 07:14, Thomas Burgess wrote:
> > For the money, it''s a much better option. you''ll be
able to afford many
> > more drives.  In my opinion, for a home system, the more you can save
on
> > the
> > case and power supply, the more hard drives you can buy.  Right now 1
TB
> > and
> > 1.5 TB drives seem to be the best.  I used 1 TB drives and 2 compact
> flash
> > cards for the os (with sata to compact flash adapters)  They are
really
> > small and easy to find a place to mount, which allows you to use the
> > hotswap bays for even more storage.
>
> I''ve been running a home ZFS server for a while now; mine
currently has
> two two-way mirrors of 400GB disks, i.e. 800GB usable data space. 
I''ve
> got a couple hundred GB free currently.  This server holds my music
> collection, plus my digital photography, plus what''s scanned of my
film
> photography, plus my ebook collection, plus the usual random personal
> files &c.  And it serves as a backup pool for several laptops.
>
> I can see that people heavily active in live audio or (especially) video
> recording would fill disks considerably faster than my still photography
> does (about 12MB per image, before I start editing it and storing extra
> copies).  But I have to say that I''m finding the size NAS boxes
people are
> building for what they call "home use" to be rather startling. 
I''m using
> 4 400GB disks with 100% redundancy; lots of people are talking about using
> 8 or more 1TB or bigger disks with 25% redundancy.  That''s a
hugely bigger
> pool!  Do you actually fill up that space?  With what?
>
> I''ve got 8 hot-swap bays but only 6 controller channels on the
> motherboard.  And I''m using two of those for the boot disks. 
I''ve thought
> about going away from rotating disks for boot, to something like CF cards,
> or USB.  USB is slow, but will that hurt me any when the system is being a
> file server?
>
> What going to USB does for me is free up two SATA controllers, so I can
> expand my pool without buying another controller and messing about inside
> the box.  Also, I don''t need a mirrored pool for boot if
it''s a cheap USB
> drive and I can keep a spare copy or two, and just swap them if
there''s
> any problem with the first one.
>
> i fill mine up with TV shows and Movies.  I have a LOT of hd stuff, 1080p
and 720p

A 1080p Movie can take up from 8 gb to 20 gb depending on encoding. 
I''m a
digital packrack.  I''ve replaced cable with a ZFS backed network of
htpcs
running xbmc on the ionitx boards.  Each htpc uses about 30 watts of power
peak and does 1080p without a problem.  each box also has a dvd player in it
if we want to watch an old dvd.  I also have rtorrent running using rss to
grab all the new shows, which normally show up a few minutes to an hour
after they air.  I''ve got them set to automatically sort and placed in
the
correct spot.   It''s easy to fill up many tb''s of space with
whole seasons
of 720p and 1080p TV, and hundreds of movies.  Using xbmc and some of the
wonderful skins you can make some amazing alternatives to cable.  I just got
tired of channel surfing.  Also, i use my multi TB system to run rsync
backups on all the computers i care about.  Snapshots allow me to return to
any day....

I''m also saving right now to build a backup system to have a second
copy of
the stuff i don''t want to lose.  I also try not to go over 50-70% full.
 when i get that full i start looking at ways to upgrade.  I started with
linux and an xfs based system on a single tb drive and just kept expanding
it...when i found out about ZFS i knew that was the way to go.  I can''t
stand to delete the stuff i have unless i find better copies...so as long as
there is new stuff coming out, i''ll probably keep expanding my system.
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090930/6d37f812/attachment.html>

Heh :-)  Disk usage is directly related to available space.

At home I have a 4x1Tb raidz filled to overflowing with music, photos,  
movies, archives, and backups for 4 other machines in the house. I''ll  
be adding another 4 and an SSD shortly.

It starts with importing CDs into iTunes or WMP, then comes the TV  
recordings, then comes ripping your DVD collection... Hey disk is  
cheap, right?

Once you have gotten out  of the habit of using shiny discs for music,  
video is a logical progression. You also stop being finicky about  
minimizing file space - I''ve gone from high quality mp3 to lossless  
formats.

I also have some colleagues that have Flip Mimos and equivalents that  
capture 720p video and that just chews through disk space. Those 12Mb  
shots of baby taking his/her first steps are now multi gigabyte raw  
video files.

Trust me, it''s easy.

Erik

On 30 sept. 2009, at 16:48, David Dyer-Bennet wrote:
> I can see that people heavily active in live audio or (especially)  
> video
> recording would fill disks considerably faster than my still  
> photography
> does (about 12MB per image, before I start editing it and storing  
> extra
> copies).  But I have to say that I''m finding the size NAS boxes  
> people are
> building for what they call "home use" to be rather startling. 
I''m
> using
> 4 400GB disks with 100% redundancy; lots of people are talking about  
> using
> 8 or more 1TB or bigger disks with 25% redundancy.  That''s a
hugely
> bigger
> pool!  Do you actually fill up that space?  With what?

On Mon, Sep 28, 2009 at 1:12 PM, Ware Adams <rwalists at washdcmail.com>
wrote:
> SuperMicro 7046A-3 Workstation
> http://supermicro.com/products/system/4U/7046/SYS-7046A-3.cfm
I''m using a SuperChassis 743TQ-865B-SQ for my home NAS, which is what
that workstation uses. It''s very LARGE and very quiet. Did I mention
it''s HUGE? I bought two more 2800 rpm fans for it. The case is
designed for four but only comes with two for noise, I didn''t notice
an increase in sound. You can find the fans (part # FAN-0104L4)
online.

I think the dual socket board you chose is a bit overkill for just a
NAS box. I used an ASUS motherboard because I wanted to use AMD, and
went with a 4850e and 8GB ECC memory. It got me a board that supports
ECC and PCI-X slots (so I could use the AOC-SAT-MV8 board). I also
host some (mostly idle) VMs on the machine and they run fine.

Supermicro has a 3 x 5.25" bay rack that holds 5 x 3.5" drives. This
doesn''t leave space for a optical drive, but I used a USB drive to
install the OS and don''t need it anymore.

-B

-- 
Brandon High : bhigh at freaks.com
If it wasn''t for pacifists, we could achieve peace.

I too went with a 5in3 case for HDDs, in a nice portable Mini-ITX case, with 
Intel Atom. More of a SOHO NAS for home use, rather than a beast. Still, I can 
get about 10TB in it.

http://lundman.net/wiki/index.php/ZFS_RAID

I can also recommend the embeddedSolaris project for making a small bootable 
Solaris. Very flexible and can put on the Admin GUIs, and so on.

https://sourceforge.net/projects/embeddedsolaris/

Lund

-- 
Jorgen Lundman       | <lundman at lundman.net>
Unix Administrator   | +81 (0)3 -5456-2687 ext 1017 (work)
Shibuya-ku, Tokyo    | +81 (0)90-5578-8500          (cell)
Japan                | +81 (0)3 -3375-1767          (home)

i looked at possibly doing one of those too - but only 5 disks was too
small for me. and i was too nervous about compatibility with mini-itx
stuff.

On Wed, Sep 30, 2009 at 6:22 PM, Jorgen Lundman <lundman at gmo.jp>
wrote:
>
> I too went with a 5in3 case for HDDs, in a nice portable Mini-ITX case,
with
> Intel Atom. More of a SOHO NAS for home use, rather than a beast. Still, I
> can get about 10TB in it.
>
> http://lundman.net/wiki/index.php/ZFS_RAID
>
> I can also recommend the embeddedSolaris project for making a small
bootable
> Solaris. Very flexible and can put on the Admin GUIs, and so on.
>
> https://sourceforge.net/projects/embeddedsolaris/
>
> Lund
>
> --
> Jorgen Lundman ? ? ? | <lundman at lundman.net>
> Unix Administrator ? | +81 (0)3 -5456-2687 ext 1017 (work)
> Shibuya-ku, Tokyo ? ?| +81 (0)90-5578-8500 ? ? ? ? ?(cell)
> Japan ? ? ? ? ? ? ? ?| +81 (0)3 -3375-1767 ? ? ? ? ?(home)
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

On Wed, Sep 30, 2009 at 05:03:21PM -0700, Brandon High wrote:
> Supermicro has a 3 x 5.25" bay rack that holds 5 x 3.5" drives.
This
> doesn''t leave space for a optical drive, but I used a USB drive to
> install the OS and don''t need it anymore.
I''ve had such a bay rack for years, and it survived one big tower,
and is now dwelling in a cheap Sharkoon case. The fan is a bit noisy,
but then, the server is behind a couple of doors, and serves the
house LAN. It''s currently running Linux, but has already a FreeNAS
on an IDE DOM preinstalled.

-- 
Eugen* Leitl <a href="http://leitl.org">leitl</a>
http://leitl.org
______________________________________________________________
ICBM: 48.07100, 11.36820 http://www.ativel.com http://postbiota.org
8B29F6BE: 099D 78BA 2FD3 B014 B08A  7779 75B0 2443 8B29 F6BE