Hi guys, Previously, I had three 1TB drives in my desktop using the Intel''s southbridge RAID for storage. The only problem with that is every time Windows Vista took a dump, I would be in jeopardy of corrupting the storage space; thus, I decided to have a dedicated machine just for serving up files. I now have a 3ware 9650SE 16 port host controller and currently four 2TB Western Digital WD2002FYPS drives (I also have a leftover drive from an old machine that''s currently the boot drive), Supermicro server MB and a Celeron 440 2GHz chip (primarly because it''s 35watts and I wanted to minimize power usage when the machine sat idle). I was weary about using a Microsoft OS as I wanted it to be reliable and also didn''t want to have to pay for another license, etc. I tried FreeNAS and it was ok, but somewhat limited in what you can do with it. I have a squeezebox and would want to install their SqueezeCenter server to server up lossless compressed audio files. I was also intrigued by ZFS after reading so much about it. I did try around with creating a raidz1 zpool, but then learned that the current implementation is somewhat limited in that you can''t expand the raidset. Using a raidz storage pool would also not take advantage of the 3ware''s dedicated hardware for computing parity, etc. Anyway, I successfully created a RAID5 set using three drives and created a zpool. I then migrated the RAID5 set to add an additional 2TB drive (took 3 days!). The server is currently down because I needed to use the RAM elsewhere, but after expanding the storage area, the filesystem still stayed the original size using the three 2TB drives. I tried finding a way to get the full space allocation, but it seems that many use simple SATA ports and the raidz solution. I''m also a n00b, so any advice would be greatly appreciated. If you think I''m going down the wrong path, I would like to hear it. TIA, Chester -- This message posted from opensolaris.org
While it is true that you can not add a single device to a raidz vdev, you can easily add more vdevs, and this is by far the best way to grow because each vdev adds much speed to the array. Raidz is more advanced than raid5, the fact that your card doesn''t do the raid calculations is not a limitation, it''s a gift. Modern CPU''s are much more equipt for handling raid calculation than the "cpu" on your raidcard. As i see it you can do a couple different things. my system has 3 raidz vdevs each with 4 hard drives and i get AMAZING speeds. Instead of buying a super expensive raid card, i go a cheap card and put the money into more ram/bigger cpu. But even with a raidcard like yours you can take advantage of all ZFS has to offer...just set it up in jbod mode. You don''t have to do raidz, you can do mirrored vdevs if you want...these are much faster. did you read the best practices guide? http://www.solarisinternals.com/wiki/index.php/ZFS_Best_Practices_Guide On Mon, Aug 10, 2009 at 12:17 AM, Chester <no-reply at opensolaris.org> wrote:> Hi guys, > > Previously, I had three 1TB drives in my desktop using the Intel''s > southbridge RAID for storage. The only problem with that is every time > Windows Vista took a dump, I would be in jeopardy of corrupting the storage > space; thus, I decided to have a dedicated machine just for serving up > files. > > I now have a 3ware 9650SE 16 port host controller and currently four 2TB > Western Digital WD2002FYPS drives (I also have a leftover drive from an old > machine that''s currently the boot drive), Supermicro server MB and a Celeron > 440 2GHz chip (primarly because it''s 35watts and I wanted to minimize power > usage when the machine sat idle). > > I was weary about using a Microsoft OS as I wanted it to be reliable and > also didn''t want to have to pay for another license, etc. I tried FreeNAS > and it was ok, but somewhat limited in what you can do with it. I have a > squeezebox and would want to install their SqueezeCenter server to server up > lossless compressed audio files. I was also intrigued by ZFS after reading > so much about it. > > I did try around with creating a raidz1 zpool, but then learned that the > current implementation is somewhat limited in that you can''t expand the > raidset. Using a raidz storage pool would also not take advantage of the > 3ware''s dedicated hardware for computing parity, etc. Anyway, I > successfully created a RAID5 set using three drives and created a zpool. I > then migrated the RAID5 set to add an additional 2TB drive (took 3 days!). > The server is currently down because I needed to use the RAM elsewhere, but > after expanding the storage area, the filesystem still stayed the original > size using the three 2TB drives. I tried finding a way to get the full > space allocation, but it seems that many use simple SATA ports and the raidz > solution. I''m also a n00b, so any advice would be greatly appreciated. If > you think I''m going down the wrong path, I would like to hear it. > > TIA, > Chester > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090810/1324591a/attachment.html>
Yes, vdevs allow you to expand your zpool. One zpool consists of groups of hard drives. Your zpool consists of one group of drives. That group contains 4 drives that are 2TB. You can easily add another group of drives to your zpool. You can not change the number of discs of a group, but you can add several groups. Each group is called a vdev. -- This message posted from opensolaris.org
Thanks for the info so far. Yes, I understand that you can add more vdevs, but at what cost? With the 2TB drives costing $300 each, I wanted to get more or less the bare minimum and then add more drives once I filled the capacity. I understand that raidz1 is similar to RAID5 (it can recover from a single drive failure) and raidz2 is similar to RAID6 (recovery from up to two drive failures). Since I have four drives now, I would leave that with single parity and probably the next time I added a drive, I would migrate over to double parity. In your scenario, once I fill up my storage capacity, I would need to add another three drives; therefore dedicating two drives for parity (one for the four disk set and one for the three disk set), which would be similar to my plan of moving to double parity. However, what about after that? Three drives dedicated to single parity for three different sets? Certainly, I would get to a point where I wouldn''t want 16 drives constantly spinning and I would hope by then either solid state disks have moved up in storage size and down in terms of price so I could start cutting over to those. Is there a way to expand the zpool to take advantage of the increased size of the hardware once I add a disk on the 3ware controller? I looked zfsadmin document and see an autoexpand property, but that feature doesn''t appear to be support by OpenSolaris. -- This message posted from opensolaris.org
Why not just do simple mirrored vdevs? or use cheaper 1tb drives for the second vdev? I don''t know....it''s up to you...To me the benefits of ZFS far outweigh the limitations. Also, in my opinion, when you are expanding your storage, it''s a good idea to add it in chunks like this...adding a 4 drive vdev is the way *I* do it right now....though i use 1tb drives because the 2tb drives aren''t worth it atm. 1tb drives are around 80 bucks and 7200 rpm, 2tb drives are 250-300 and 5400 rpm...for the cost of 2 2tb drives you could EASILY add vdevs of 1tb drives... On Mon, Aug 10, 2009 at 7:03 AM, Chester <no-reply at opensolaris.org> wrote:> Thanks for the info so far. Yes, I understand that you can add more vdevs, > but at what cost? With the 2TB drives costing $300 each, I wanted to get > more or less the bare minimum and then add more drives once I filled the > capacity. I understand that raidz1 is similar to RAID5 (it can recover from > a single drive failure) and raidz2 is similar to RAID6 (recovery from up to > two drive failures). Since I have four drives now, I would leave that with > single parity and probably the next time I added a drive, I would migrate > over to double parity. > > In your scenario, once I fill up my storage capacity, I would need to add > another three drives; therefore dedicating two drives for parity (one for > the four disk set and one for the three disk set), which would be similar to > my plan of moving to double parity. However, what about after that? Three > drives dedicated to single parity for three different sets? Certainly, I > would get to a point where I wouldn''t want 16 drives constantly spinning and > I would hope by then either solid state disks have moved up in storage size > and down in terms of price so I could start cutting over to those. > > Is there a way to expand the zpool to take advantage of the increased size > of the hardware once I add a disk on the 3ware controller? I looked > zfsadmin document and see an autoexpand property, but that feature doesn''t > appear to be support by OpenSolaris. > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090810/d6d3921e/attachment.html>
Lets not forget despite the fact us lunatic fringe use OpenSolaris on
anything we can get our hands on. Sun Microsystems use Solaris to run
mission critical environments and adding disk in "chunks" like you
have
to do in ZFS to a commercial organisation is no big deal. The data is
worth far more to most organisations than the disks. To get the
functionality Sun''s customers now have with ZFS for zero dollars (with
the exception of shrinking lets not go down that rat hole), they use to
have to pay many many dollars to Veritas. Or they pay lots of money to
Network Appliance .
For Sun''s paying customers to quoute Thomas "the benefits of
ZFS far outweigh the limitations"
Lets not forget: UFS/xVFS SVM/xVM and the whole RAID industry, have
many more years of development and use. ZFS is still the new kid on the
block, he might not be as good as some of the old boys in the
playground, but he is creating a stir and gowning up fast!
Thomas Burgess wrote:
Why not just do simple mirrored vdevs? or use cheaper 1tb
drives for the second vdev?
I don''t know....it''s up to you...To me the benefits of ZFS far
outweigh
the limitations. Also, in my opinion, when you are expanding your
storage, it''s a good idea to add it in chunks like this...adding a 4
drive vdev is the way *I* do it right now....though i use 1tb drives
because the 2tb drives aren''t worth it atm.
1tb drives are around 80 bucks and 7200 rpm, 2tb drives are 250-300 and
5400 rpm...for the cost of 2 2tb drives you could EASILY add vdevs of
1tb drives...
On Mon, Aug 10, 2009 at 7:03 AM, Chester
<no-reply@opensolaris.org>
wrote:
Thanks
for the info so far. Yes, I understand that you can add more vdevs,
but at what cost? With the 2TB drives costing $300 each, I wanted to
get more or less the bare minimum and then add more drives once I
filled the capacity. I understand that raidz1 is similar to RAID5 (it
can recover from a single drive failure) and raidz2 is similar to RAID6
(recovery from up to two drive failures). Since I have four drives
now, I would leave that with single parity and probably the next time I
added a drive, I would migrate over to double parity.
In your scenario, once I fill up my storage capacity, I would need to
add another three drives; therefore dedicating two drives for parity
(one for the four disk set and one for the three disk set), which would
be similar to my plan of moving to double parity. However, what about
after that? Three drives dedicated to single parity for three
different sets? Certainly, I would get to a point where I wouldn''t
want 16 drives constantly spinning and I would hope by then either
solid state disks have moved up in storage size and down in terms of
price so I could start cutting over to those.
Is there a way to expand the zpool to take advantage of the increased
size of the hardware once I add a disk on the 3ware controller? I
looked zfsadmin document and see an autoexpand property, but that
feature doesn''t appear to be support by OpenSolaris.
--
This message posted from opensolaris.org
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
www.eagle.co.nz
This email is confidential and may be legally
privileged. If received in error please destroy and immediately notify
us.
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
I guess it''s all in what you want. At the end of the day, you have to weigh the pros and cons and decide what is right for you. To me, adding drives in mirrored or raidz groups is a small price to pay for the speed, ease of administration, and data integrity that zfs offers. Snapshots and clones allow you to do things in an entirely new way and make life much more interesting. To me, i''d rather put my money into cheap, large disks and ram than expensive raid controllers. I''d rather have a self healing filesystem than....any other filesystem that pops into my head. There ARE some limitations in how you set up your pool, but if you have a halfway decent plan you can make them work FOR you instead of against you. If you don''t wait until your out of space to upgrade you can plan your upgrade much better. Currently my pool is made up of 3 raidz vdevs with 4 1tb drives each. When i first set the system up it wasn''t with thee same vdev size but by making good backups and planning upgrades i''ve been able to cope..now i just add a new vdev when i get i feel i need the space...i''m currently planning my next upgrade...which is another 4 drive vdev and perhaps a 3 drive vdev and a spare....theres nothing that says i have to get it all at once... I''m also considering upgrading my 1tb drives with 2 tb drives...or using 2 tb drives for the next vdev....i haven''t completely decided yet but the point is if you make some kind of plan you can build your pool accordingly. If i was only able to add a couple drives at a time i think i''d go with mirrored groups. On Mon, Aug 10, 2009 at 5:33 PM, Trevor Pretty <trevor_pretty at eagle.co.nz>wrote:> > Lets not forget despite the fact us lunatic fringe use OpenSolaris on > anything we can get our hands on. Sun Microsystems use Solaris to run > mission critical environments and adding disk in "chunks" like you have to > do in ZFS to a commercial organisation is no big deal. The data is worth far > more to most organisations than the disks. To get the functionality Sun''s > customers now have with ZFS for zero dollars (with the exception of > shrinking lets not go down that rat hole), they use to have to pay many many > dollars to Veritas. Or they pay lots of money to Network Appliance . > > For Sun''s *paying *customers to quoute Thomas "the benefits of ZFS far > outweigh the limitations" > > Lets not forget: UFS/xVFS SVM/xVM and the whole RAID industry, have many > more years of development and use. ZFS is still the new kid on the block, he > might not be as good as some of the old boys in the playground, but he is > creating a stir and gowning up fast! > > > Thomas Burgess wrote: > > Why not just do simple mirrored vdevs? or use cheaper 1tb drives for the > second vdev? > I don''t know....it''s up to you...To me the benefits of ZFS far outweigh the > limitations. Also, in my opinion, when you are expanding your storage, it''s > a good idea to add it in chunks like this...adding a 4 drive vdev is the way > *I* do it right now....though i use 1tb drives because the 2tb drives aren''t > worth it atm. > > 1tb drives are around 80 bucks and 7200 rpm, 2tb drives are 250-300 and > 5400 rpm...for the cost of 2 2tb drives you could EASILY add vdevs of 1tb > drives... > > On Mon, Aug 10, 2009 at 7:03 AM, Chester <no-reply at opensolaris.org> wrote: > >> Thanks for the info so far. Yes, I understand that you can add more >> vdevs, but at what cost? With the 2TB drives costing $300 each, I wanted to >> get more or less the bare minimum and then add more drives once I filled the >> capacity. I understand that raidz1 is similar to RAID5 (it can recover from >> a single drive failure) and raidz2 is similar to RAID6 (recovery from up to >> two drive failures). Since I have four drives now, I would leave that with >> single parity and probably the next time I added a drive, I would migrate >> over to double parity. >> >> In your scenario, once I fill up my storage capacity, I would need to add >> another three drives; therefore dedicating two drives for parity (one for >> the four disk set and one for the three disk set), which would be similar to >> my plan of moving to double parity. However, what about after that? Three >> drives dedicated to single parity for three different sets? Certainly, I >> would get to a point where I wouldn''t want 16 drives constantly spinning and >> I would hope by then either solid state disks have moved up in storage size >> and down in terms of price so I could start cutting over to those. >> >> Is there a way to expand the zpool to take advantage of the increased size >> of the hardware once I add a disk on the 3ware controller? I looked >> zfsadmin document and see an autoexpand property, but that feature doesn''t >> appear to be support by OpenSolaris. >> -- >> This message posted from opensolaris.org >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> > > > > > > * > > * > > www.eagle.co.nz > > This email is confidential and may be legally privileged. If received in > error please destroy and immediately notify us. > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090810/46e0347c/attachment.html>
I think the point, Chester, which everyone seems to be dancing around or missing, is that your planning may need to go back to the drawing board on this one. Absorb the resources out there for how to best configure your pools and vdevs, *then* implement. That''s the most efficient way to go about doing what you want to do. You can''t add a single disk to an existing vdev, as near as I can tell. There are other discussions on this list about expanding or transforming vdevs of a certain type to another type, but this functionality appears to be low on the ZFS development teams list of priorities (probably with good reason) and somewhat high on complexity. Now, I''ll include my list of things you might want to go back and read (and possibly re-read) to do planning for your migration from the current RAIDZ to a new implementation of ZFS that gives you your desired outcome. Please excuse me if you have spent time with these, but I figure it''s worth saying just in case: ZFS Admin Guide: http://docs.sun.com/app/docs/doc/819-5461?l=en ZFS @ Solaris Internals: http://www.solarisinternals.com//wiki/index.php?title=Category:ZFS Of course, the ZFS community on OpenSolaris.org: http://opensolaris.org/os/community/zfs/ No need to mention zfs-discuss. Anyway, the answer is that what you say you want to do, you cannot do. There is probably another way to accomplish your ultimate goal, but you want to get clear about that goal in DETAIL then see how, or if, you can use ZFS to achieve that goal. Since ZFS hails from a world of people who are fairly accustomed to thinking in detailed, enterprise terms, you''ll either want to start looking at things in this way or seek out solutions which fit how you want to use your software and hardware. HTH. Cheers! On Mon, Aug 10, 2009 at 00:17, Chester <no-reply at opensolaris.org> wrote:> Hi guys, > > Previously, I had three 1TB drives in my desktop using the Intel''s > southbridge RAID for storage. The only problem with that is every time > Windows Vista took a dump, I would be in jeopardy of corrupting the storage > space; thus, I decided to have a dedicated machine just for serving up > files. > > I now have a 3ware 9650SE 16 port host controller and currently four 2TB > Western Digital WD2002FYPS drives (I also have a leftover drive from an old > machine that''s currently the boot drive), Supermicro server MB and a Celeron > 440 2GHz chip (primarly because it''s 35watts and I wanted to minimize power > usage when the machine sat idle). > > I was weary about using a Microsoft OS as I wanted it to be reliable and > also didn''t want to have to pay for another license, etc. I tried FreeNAS > and it was ok, but somewhat limited in what you can do with it. I have a > squeezebox and would want to install their SqueezeCenter server to server up > lossless compressed audio files. I was also intrigued by ZFS after reading > so much about it. > > I did try around with creating a raidz1 zpool, but then learned that the > current implementation is somewhat limited in that you can''t expand the > raidset. Using a raidz storage pool would also not take advantage of the > 3ware''s dedicated hardware for computing parity, etc. Anyway, I > successfully created a RAID5 set using three drives and created a zpool. I > then migrated the RAID5 set to add an additional 2TB drive (took 3 days!). > The server is currently down because I needed to use the RAM elsewhere, but > after expanding the storage area, the filesystem still stayed the original > size using the three 2TB drives. I tried finding a way to get the full > space allocation, but it seems that many use simple SATA ports and the raidz > solution. I''m also a n00b, so any advice would be greatly appreciated. If > you think I''m going down the wrong path, I would like to hear it. > > TIA, > Chester > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- "You can choose your friends, you can choose the deals." - Equity Private AlphaGuy - http://alphaguy.blogspot.com On Twitter - @khyron4eva -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090811/baae0f78/attachment.html>