Two conflicting answers to the same question? I guess we need someone to break the tie :)> Hello, > > I have been reading alot of good things about Raid-z, > but before I jump into it I have one unanswered > question i can''t find a clear answer for. > > Is it possible to enlarge the initial RAID size by > adding single drives later on? > > If i start off with 4*500gb (should give me 1.5tb), > can I add one 500gb to the raid later, and will the > total size then grow 500gb and still have the same > protection?This message posted from opensolaris.org
I was considering the same thing (but with 250GB drives) and instead decided to start with 5*250GB so that I could do raid-z2 instead thinking that once you add one more drive you''d rather have raidz2 than raidz On 4/7/07, MC <rac at eastlink.ca> wrote:> > Two conflicting answers to the same question? I guess we need someone to > break the tie :) > > > Hello, > > > > I have been reading alot of good things about Raid-z, > > but before I jump into it I have one unanswered > > question i can''t find a clear answer for. > > > > Is it possible to enlarge the initial RAID size by > > adding single drives later on? > > > > If i start off with 4*500gb (should give me 1.5tb), > > can I add one 500gb to the raid later, and will the > > total size then grow 500gb and still have the same > > protection? > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070407/544bff17/attachment.html>
You cannot add 1 drive at a time to a raid-z or raid-2z. You need to add the same number of disks that were used per stripe.. So, if you start with 5 disks, you would have to add 5 more in the future to add disk space. There is also a method of swapping each disk one at a time with a larger disk and performing a scrub inbetween each replacement to increase the pool size. Eric This message posted from opensolaris.org
My current understanding is that RAIDZ and RAIDZ2 setups CANNOT be expanded, nor can they converted between. There has been considerable discussion on this point, and I think the consensus is that expansion is a highly desirable feature, but the exact timetable for adding expansion capability isn''t set. Currently, the only things which can be done to a zpool are attachment of additional drives to an existing mirror, and additions of whole stripes. Shrinking isn''t possible, nor is conversion between RAID types, nor is expansion of RAIDZ/RAIDZ2 vdevs. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
One option is you can replace all the existing devices in a raidz vdev with larger devices, and then export/import the pool and the vdev will grow in size. I agree that you simply can''t add a single device to grow a raidz vdev. This message posted from opensolaris.org
On April 7, 2007 11:11:37 PM -0700 Eric Haycraft <earl507 at yahoo.com> wrote:> You cannot add 1 drive at a time to a raid-z or raid-2z. You need to add > the same number of disks that were used per stripe.You cannot add drives to a raid-z or raid-z2, period. -frank
Hmmm... I definitely missed this one... I thought the documentation said that using zpool attach would add a new drive to the existing raidz(2) and then it would start resilvering... Malachi On 4/7/07, Eric Haycraft <earl507 at yahoo.com> wrote:> > You cannot add 1 drive at a time to a raid-z or raid-2z. You need to add > the same number of disks that were used per stripe.. So, if you start with 5 > disks, you would have to add 5 more in the future to add disk space. There > is also a method of swapping each disk one at a time with a larger disk and > performing a scrub inbetween each replacement to increase the pool size. > > Eric > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070408/04d41ac5/attachment.html>
[top-posting corrected] On April 8, 2007 1:43:48 PM -0700 Malachi de ?lfweald <malachid at gmail.com> wrote:> On 4/7/07, Eric Haycraft <earl507 at yahoo.com> wrote: >> >> You cannot add 1 drive at a time to a raid-z or raid-2z. You need to add >> the same number of disks that were used per stripe.. So, if you start >> with 5 disks, you would have to add 5 more in the future to add disk >> space. There is also a method of swapping each disk one at a time with a >> larger disk and performing a scrub inbetween each replacement to >> increase the pool size. > > Hmmm... I definitely missed this one... I thought the documentation said > that using zpool attach would add a new drive to the existing raidz(2) and > then it would start resilvering...zpool(1M): zpool attach [-f] pool device new_device Attaches new_device to an existing zpool device. The existing device cannot be part of a raidz configuration. -frank
Malachi de ?lfweald wrote:> Hmmm... I definitely missed this one... I thought the documentation > said that using zpool attach would add a new drive to the existing > raidz(2) and then it would start resilvering... > > Malachi >The man page is a little unclear. zpool attach is for adding drives to a mirror vdev (or, creating a mirrored vdev from a single drive). It is NOT for use with RAIDZ or RAIDZ2 vdevs (hopefully, in the future...) Also, to be even more clear, the only way to add devices to a zpool with RAIDZ/RAIDZ2 is to add another stripe. Which really isn''t adding anything to the RAIDZ vdev. That is, if you have a zpool consisting of a 4-drive RAIDZ, as in: zpool create tank raidz c1t0d0 c1t1d0 c1t2d0 c1t3d0 you can only expand it by adding sufficient disk to create a whole new stripe - so, effectively, your data will be striped between the existing RAIDZ vdev, and the one you add. i.e. zpool add tank raidz c2t0d0 c2t1d0 c2t2d0 c2t3d0 So, if you started out with 4 100GB drives in the above config, you''d have 300GB usable. You''d then have to add 4 more 100GB drives, and the final zpool size would be 600GB, with data in striped across 2 4-drive RAIDZ vdevs. What most people here would prefer is that you could instead (if desired) end up with a single RAIDZ2 vdev of 6 data drives and 2 parity drives, but that is NOT currently possible. -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
Yeah, I am not sure what docs I was originally looking at... Although we may want to ensure that the ZFS Admin Guide is a bit more clear on the matter: Additional disks can be added similarly to a RAID-Z configuration. Malachi On 4/8/07, Frank Cusack <fcusack at fcusack.com> wrote:> > [top-posting corrected] > > On April 8, 2007 1:43:48 PM -0700 Malachi de ?lfweald <malachid at gmail.com> > wrote: > > On 4/7/07, Eric Haycraft <earl507 at yahoo.com> wrote: > >> > >> You cannot add 1 drive at a time to a raid-z or raid-2z. You need to > add > >> the same number of disks that were used per stripe.. So, if you start > >> with 5 disks, you would have to add 5 more in the future to add disk > >> space. There is also a method of swapping each disk one at a time with > a > >> larger disk and performing a scrub inbetween each replacement to > >> increase the pool size. > > > > Hmmm... I definitely missed this one... I thought the documentation said > > that using zpool attach would add a new drive to the existing raidz(2) > and > > then it would start resilvering... > > zpool(1M): > > zpool attach [-f] pool device new_device > > Attaches new_device to an existing zpool device. The > existing device cannot be part of a raidz configuration. > > -frank >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070408/2e90af87/attachment.html>
Yeah, what I had been hoping (and to be honest kinda counting on) was that I could do a raidz2 across the 7x250GB drives (which is not actually going to happen since I am going to mirror 2 of them for boot) and then just add another 250GB drive when more space is needed. Realistically, I am not going to need that functionality right away, but have to admit it was one of the reasons I have been touting ZFS to everyone. Malachi On 4/8/07, Erik Trimble <Erik.Trimble at sun.com> wrote:> > Malachi de ?lfweald wrote: > > Hmmm... I definitely missed this one... I thought the documentation > > said that using zpool attach would add a new drive to the existing > > raidz(2) and then it would start resilvering... > > > > Malachi > > > The man page is a little unclear. zpool attach is for adding drives to > a mirror vdev (or, creating a mirrored vdev from a single drive). It is > NOT for use with RAIDZ or RAIDZ2 vdevs (hopefully, in the future...) > > Also, to be even more clear, the only way to add devices to a zpool with > RAIDZ/RAIDZ2 is to add another stripe. Which really isn''t adding > anything to the RAIDZ vdev. That is, if you have a zpool consisting of a > 4-drive RAIDZ, as in: > > zpool create tank raidz c1t0d0 c1t1d0 c1t2d0 c1t3d0 > > you can only expand it by adding sufficient disk to create a whole new > stripe - so, effectively, your data will be striped between the existing > RAIDZ vdev, and the one you add. > > i.e. > > zpool add tank raidz c2t0d0 c2t1d0 c2t2d0 c2t3d0 > > So, if you started out with 4 100GB drives in the above config, you''d > have 300GB usable. You''d then have to add 4 more 100GB drives, and the > final zpool size would be 600GB, with data in striped across 2 4-drive > RAIDZ vdevs. What most people here would prefer is that you could > instead (if desired) end up with a single RAIDZ2 vdev of 6 data drives > and 2 parity drives, but that is NOT currently possible. > > -- > Erik Trimble > Java System Support > Mailstop: usca22-123 > Phone: x17195 > Santa Clara, CA > Timezone: US/Pacific (GMT-0800) > >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070408/f0354f95/attachment.html>
Since no one seems to believe that you can expand a raidz pool, I have attached the following output from solaris 11/06 showing me doing just that. The first expanision is with like sized disks, and the second expansion is with larger disks. I realize that the documentation only has examples using mirrors, but raidz and raid2z are fully supported for adding disk space. # mkfile 64m 1 2 3 4 5 6 # mkfile 128m 01 02 03 # zpool create testpool raidz `pwd`/1 `pwd`/2 `pwd`/3 # zpool list testpool NAME SIZE USED AVAIL CAP HEALTH ALTROOT testpool 178M 100K 178M 0% ONLINE - # zpool add testpool raidz `pwd`/4 `pwd`/5 `pwd`/6 # zpool list testpool NAME SIZE USED AVAIL CAP HEALTH ALTROOT testpool 356M 305K 356M 0% ONLINE - # zpool add testpool raidz `pwd`/01 `pwd`/02 `pwd`/03 # zpool list testpool NAME SIZE USED AVAIL CAP HEALTH ALTROOT testpool 726M 475K 726M 0% ONLINE - # This message posted from opensolaris.org
Eric Haycraft wrote:> Since no one seems to believe that you can expand a raidz pool, I have attached the following output from solaris 11/06 showing me doing just that. The first expanision is with like sized disks, and the second expansion is with larger disks. I realize that the documentation only has examples using mirrors, but raidz and raid2z are fully supported for adding disk space. > > > # mkfile 64m 1 2 3 4 5 6 > # mkfile 128m 01 02 03 > > # zpool create testpool raidz `pwd`/1 `pwd`/2 `pwd`/3 > # zpool list testpool > NAME SIZE USED AVAIL CAP HEALTH ALTROOT > testpool 178M 100K 178M 0% ONLINE - > > # zpool add testpool raidz `pwd`/4 `pwd`/5 `pwd`/6 > # zpool list testpool > NAME SIZE USED AVAIL CAP HEALTH ALTROOT > testpool 356M 305K 356M 0% ONLINE - > > # zpool add testpool raidz `pwd`/01 `pwd`/02 `pwd`/03 > # zpool list testpool > NAME SIZE USED AVAIL CAP HEALTH ALTROOT > testpool 726M 475K 726M 0% ONLINE - > # > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >No, you are NOT adding to a RAIDZ vdev. You are adding to a ZPOOL. Your new pool has 2 stripes of RAIDZ vdevs. The final "testpool" zpool above consists of 3 RAIDZ vdevs, each with 3 members. While expanding a zpool in the way you''ve show is useful, it has nowhere near the flexibility of simply adding single disks to existing RAIDZ vdevs, which was the original desire expressed. This conversation has been had several times now (take a look in the archives around Jan for the last time it came up). To make myself clear, what we''re talking about is some way to move from: zpool create tank raidz d1 d2 d3 to the equivalent of zpool create tank raidz d1 d2 d3 d4 d5 d6 in increments. Right now, that''s not possible. Also, I believe there is a bug in zfs list for Solaris 10 U3''s zfs - I have to check it out. Because: # cd /tmp # mkfile 64m 1 2 3 4 5 6 7 8 # zpool create testpool1 raidz /tmp/1 /tmp/2 /tmp/3 /tmp/4 raidz /tmp/5 /tmp/6 /tmp/7 /tmp/8 # zfs list testpool1 NAME USED AVAIL REFER MOUNTPOINT testpool1 131K 444M 49K /testpool1 # zpool destroy testpool1 # zpool create testpool2 raidz /tmp/1 /tmp/2 /tmp/3 /tmp/4 /tmp/5 /tmp/6 /tmp/7 /tmp/8 # zfs list testpool2 NAME USED AVAIL REFER MOUNTPOINT testpool2 77K 444M 3K /testpool2 Which is _clearly_ incorrect. "testpool1" should have a size of approximately 6*64m = 384, not 7*64m = 448m as in "testpool2". -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA Timezone: US/Pacific (GMT-0800)
Eric Haycraft wrote:> Since no one seems to believe that you can expand a raidz pool, I have attached the following output from solaris 11/06 showing me doing just that. The first expanision is with like sized disks, and the second expansion is with larger disks. I realize that the documentation only has examples using mirrors, but raidz and raid2z are fully supported for adding disk space.No one has said that you can''t increase the size of a zpool. What can''t be increased is the size of a RAID-Z vdev (except by increasing the size of all of the components of the RAID-Z). You have created additional RAID-Z vdevs and added them to the pool. -- --Ed
MC, If you originally had 4 * 500 GB disks configured in RAID-Z, you cannot add 1 single disk and grow the capacity of the pool (with same protection). This is not allowed. Regards, Sanjeev. MC wrote:> Two conflicting answers to the same question? I guess we need someone to break the tie :) > > >> Hello, >> >> I have been reading alot of good things about Raid-z, >> but before I jump into it I have one unanswered >> question i can''t find a clear answer for. >> >> Is it possible to enlarge the initial RAID size by >> adding single drives later on? >> >> If i start off with 4*500gb (should give me 1.5tb), >> can I add one 500gb to the raid later, and will the >> total size then grow 500gb and still have the same >> protection? >> > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-- Solaris Revenue Products Engineering, India Engineering Center, Sun Microsystems India Pvt Ltd. Tel: x27521 +91 80 669 27521
Erik Trimble wrote:> While expanding a zpool in the way you''ve show is useful, it has nowhere > near the flexibility of simply adding single disks to existing RAIDZ > vdevs, which was the original desire expressed. This conversation has > been had several times now (take a look in the archives around Jan for > the last time it came up).Perhaps, this should be added to the FAQ? Cheers Manoj
Cindy.Swearingen at Sun.COM
2007-Apr-09 16:41 UTC
[zfs-discuss] Re: Re: simple Raid-Z question
Malachi, The section on adding devices to a ZFS storage pool in the ZFS Admin guide, here, provides an example of adding to a raidz configuration: http://docsview.sfbay/app/docs/doc/817-2271/6mhupg6ft?a=view I think I need to provide a summary of what you can do with both raidz and mirrored configs since you all had trouble finding it. Thanks for the feedback, Cindy Malachi de ?lfweald wrote:> Yeah, I am not sure what docs I was originally looking at... > > Although we may want to ensure that the ZFS Admin Guide is a bit more > clear on the matter: > Additional disks can be added similarly to a RAID-Z configuration. > > Malachi > > > On 4/8/07, *Frank Cusack* <fcusack at fcusack.com > <mailto:fcusack at fcusack.com>> wrote: > > [top-posting corrected] > > On April 8, 2007 1:43:48 PM -0700 Malachi de ?lfweald > <malachid at gmail.com <mailto:malachid at gmail.com>> > wrote: > > On 4/7/07, Eric Haycraft < earl507 at yahoo.com > <mailto:earl507 at yahoo.com>> wrote: > >> > >> You cannot add 1 drive at a time to a raid-z or raid-2z. You > need to add > >> the same number of disks that were used per stripe.. So, if you > start > >> with 5 disks, you would have to add 5 more in the future to add > disk > >> space. There is also a method of swapping each disk one at a > time with a > >> larger disk and performing a scrub inbetween each replacement to > >> increase the pool size. > > > > Hmmm... I definitely missed this one... I thought the > documentation said > > that using zpool attach would add a new drive to the existing > raidz(2) and > > then it would start resilvering... > > zpool(1M): > > zpool attach [-f] pool device new_device > > Attaches new_device to an existing zpool device. The > existing device cannot be part of a raidz configuration. > > -frank > > > > ------------------------------------------------------------------------ > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Cindy.Swearingen at Sun.COM
2007-Apr-09 17:07 UTC
[zfs-discuss] Re: Re: simple Raid-Z question
Here''s the correct link: http://docs.sun.com/app/docs/doc/817-2271/6mhupg6ft?a=view The same example exists on page 52 of the 817-2271 PDF posted on the opensolaris.../zfs/documentation page. Cindy Malachi de ?lfweald wrote:> FYI That page is not publicly viewable. It was the 817-2271 pdf I was > looking at though. > > Malachi > > On 4/9/07, * Cindy.Swearingen at sun.com <mailto:Cindy.Swearingen at sun.com>* > <Cindy.Swearingen at sun.com <mailto:Cindy.Swearingen at sun.com>> wrote: > > Malachi, > > The section on adding devices to a ZFS storage pool in the ZFS Admin > guide, here, provides an example of adding to a raidz configuration: > > http://docsview.sfbay/app/docs/doc/817-2271/6mhupg6ft?a=view > > I think I need to provide a summary of what you can do with > both raidz and mirrored configs since you all had trouble finding > it. > > Thanks for the feedback, > > Cindy > > Malachi de ?lfweald wrote: > > Yeah, I am not sure what docs I was originally looking at... > > > > Although we may want to ensure that the ZFS Admin Guide is a bit more > > clear on the matter: > > Additional disks can be added similarly to a RAID-Z configuration. > > > > Malachi > > > > > > On 4/8/07, *Frank Cusack* <fcusack at fcusack.com > <mailto:fcusack at fcusack.com> > > <mailto: fcusack at fcusack.com <mailto:fcusack at fcusack.com>>> wrote: > > > > [top-posting corrected] > > > > On April 8, 2007 1:43:48 PM -0700 Malachi de ?lfweald > > < malachid at gmail.com <mailto:malachid at gmail.com> > <mailto:malachid at gmail.com <mailto:malachid at gmail.com>>> > > wrote: > > > On 4/7/07, Eric Haycraft < earl507 at yahoo.com > <mailto:earl507 at yahoo.com> > > <mailto:earl507 at yahoo.com <mailto:earl507 at yahoo.com>>> wrote: > > >> > > >> You cannot add 1 drive at a time to a raid-z or raid-2z. You > > need to add > > >> the same number of disks that were used per stripe.. So, > if you > > start > > >> with 5 disks, you would have to add 5 more in the future > to add > > disk > > >> space. There is also a method of swapping each disk one at a > > time with a > > >> larger disk and performing a scrub inbetween each > replacement to > > >> increase the pool size. > > > > > > Hmmm... I definitely missed this one... I thought the > > documentation said > > > that using zpool attach would add a new drive to the existing > > raidz(2) and > > > then it would start resilvering... > > > > zpool(1M): > > > > zpool attach [-f] pool device new_device > > > > Attaches new_device to an existing zpool device. The > > existing device cannot be part of a raidz > configuration. > > > > -frank > > > > > > > > > ------------------------------------------------------------------------ > > > > _______________________________________________ > > zfs-discuss mailing list > > zfs-discuss at opensolaris.org <mailto:zfs-discuss at opensolaris.org> > > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org <mailto:zfs-discuss at opensolaris.org> > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > >
>No one has said that you can''t increase the size of a zpool. What can''t >be increased is the size of a RAID-Z vdev (except by increasing the size >of all of the components of the RAID-Z). You have created additional >RAID-Z vdevs and added them to the pool.If following is nonsense, please bear with me being a newbie... Suppose you partition the each of the n disk into m equally sized partitions, make m raidz vdevs and finally create one pool from them all. When you then wish to add another disk, you first partition it as the previous ones, then for i in 1:m { remove raidz vdev number i from the pool destroy raidz vdev number i create a raidz vdev from partition i of each of the n+1 disks add the new raidz vdev to the pool } This would demand that 1/m:th of the available disk space would be free, but would make it possible to add additional disks without backup/restore the entire data. Also, it would make it possible to add disks of different sizes, since some raidz:s could include partitions from more disks than others. The interesting question is, what would the performance hit be for pooling m raidz:s from partitions instead of using one regular raidz? Does the driver in some way collect requests for the same disk, which would be destroyed when multiple vdevs in the same pool come from the same disk? Cheers Anders This message posted from opensolaris.org