Hello list, This has probably been discussed, however I would like to bring it up again, so that the powers that be, know someone else is looking for this feature. I would like to be able to shrink a pool and remove a non-redundant disk. Is this something that is in the works? It would be fantastic if I had this capability. Thanks! - Cassandra Unix Administrator "From a little spark may burst a mighty flame." -Dante Alighieri -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100706/06714f53/attachment.html>
----- Original Message -----> Hello list, > > This has probably been discussed, however I would like to bring it up > again, so that the powers that be, know someone else is looking for > this feature. > > I would like to be able to shrink a pool and remove a non-redundant > disk. > > Is this something that is in the works? > > It would be fantastic if I had this capability.You''re a little unclear on what you want, but it seems to me you want to change a raidz2 to a raidz1 or something like that. That is, AFAIK, in the works, with the block rewrite functionality. As with most other parts of progress in OpenSolaris, nothing is clear about when or if this will get integrated into the system. For now, you can''t change a raidz(n) VDEV and you can''t detach a VDEV from a pool. The only way is to build a new pool and move the data with things like zfs send/receive. You can also remove a drive from a raidz(n), and reducing its redundancy, but you can''t change the value of n. Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
The pool is not redundant, so I would suppose, yes, is is Raid-1 on the software level. I have a few drives, which are on a specific array, which I would like to remove from this pool. I have discovered the "replace" command, and I am going to try and replace, 1 for 1, the drives I would like to remove. However, it would be nice if there were a way to simply "remove" the disks, if space allowed. - Cassandra (609) 243-2413 Unix Administrator "From a little spark may burst a mighty flame." -Dante Alighieri On Tue, Jul 6, 2010 at 11:55 AM, Roy Sigurd Karlsbakk <roy at karlsbakk.net>wrote:> ----- Original Message ----- > > Hello list, > > > > This has probably been discussed, however I would like to bring it up > > again, so that the powers that be, know someone else is looking for > > this feature. > > > > I would like to be able to shrink a pool and remove a non-redundant > > disk. > > > > Is this something that is in the works? > > > > It would be fantastic if I had this capability. > > You''re a little unclear on what you want, but it seems to me you want to > change a raidz2 to a raidz1 or something like that. That is, AFAIK, in the > works, with the block rewrite functionality. As with most other parts of > progress in OpenSolaris, nothing is clear about when or if this will get > integrated into the system. > > For now, you can''t change a raidz(n) VDEV and you can''t detach a VDEV from > a pool. The only way is to build a new pool and move the data with things > like zfs send/receive. You can also remove a drive from a raidz(n), and > reducing its redundancy, but you can''t change the value of n. > > Vennlige hilsener / Best regards > > roy > -- > Roy Sigurd Karlsbakk > (+47) 97542685 > roy at karlsbakk.net > http://blogg.karlsbakk.net/ > -- > I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det > er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av > idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate > og relevante synonymer p? norsk. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100706/5e537cbf/attachment.html>
----- Original Message -----> The pool is not redundant, so I would suppose, yes, is is Raid-1 on > the software level. > > I have a few drives, which are on a specific array, which I would like > to remove from this pool. > > I have discovered the "replace" command, and I am going to try and > replace, 1 for 1, the drives I would like to remove. > > However, it would be nice if there were a way to simply "remove" the > disks, if space allowed.zfs attach new drive, zfs detach old drive, that will replace the drive without much hazzle Vennlige hilsener / Best regards roy -- Roy Sigurd Karlsbakk (+47) 97542685 roy at karlsbakk.net http://blogg.karlsbakk.net/ -- I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate og relevante synonymer p? norsk.
I tried zfs replace, however the new drive is slightly smaller, and even with a -f, it refuses to replace the drive. I guess i will have to export the pool and destroy this one to get my drives back. Still would like the ability to shrink a pool. - Cassandra (609) 243-2413 Unix Administrator "From a little spark may burst a mighty flame." -Dante Alighieri On Tue, Jul 6, 2010 at 1:02 PM, Roy Sigurd Karlsbakk <roy at karlsbakk.net>wrote:> ----- Original Message ----- > > The pool is not redundant, so I would suppose, yes, is is Raid-1 on > > the software level. > > > > I have a few drives, which are on a specific array, which I would like > > to remove from this pool. > > > > I have discovered the "replace" command, and I am going to try and > > replace, 1 for 1, the drives I would like to remove. > > > > However, it would be nice if there were a way to simply "remove" the > > disks, if space allowed. > > zfs attach new drive, zfs detach old drive, that will replace the drive > without much hazzle > > Vennlige hilsener / Best regards > > roy > -- > Roy Sigurd Karlsbakk > (+47) 97542685 > roy at karlsbakk.net > http://blogg.karlsbakk.net/ > -- > I all pedagogikk er det essensielt at pensum presenteres intelligibelt. Det > er et element?rt imperativ for alle pedagoger ? unng? eksessiv anvendelse av > idiomer med fremmed opprinnelse. I de fleste tilfeller eksisterer adekvate > og relevante synonymer p? norsk. >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100706/4c03e317/attachment.html>
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Cassandra Pugh > > I would like to be able to shrink a pool and remove a non-redundant > disk. > > Is this something that is in the works?I think the request is to remove vdev''s from a pool. Not currently possible. Is this in the works?
> I think the request is to remove vdev''s from a pool. > Not currently possible. Is this in the works?Actually, I think this is two requests, hashed over hundreds of times in this forum: 1. Remove a vdev from a pool 2. Nondisruptively change vdev geometry #1 above has a stunningly obvious use case. Suppose, despite your best efforts, QA, planning and walkthroughs, you accidentally fat finger a "zpool attach" and unintentionally "zpool add" a disk to a pool. There is no way to reverse that operation without *significant* downtime. I have discussed #2 above multiple times and has at least one obvious use case. Suppose, just for a minute, that over the years since you deployed a zfs pool with nearly constant uptime, that your business needs change and you need to add a disk to a RAIDZ vdev, or move from RAIDZ1 to RAIDZ2, or disks have grown so big that you wish to remove a disk from a vdev. The responses from the community on the two requests seem to be: 1. Don''t ever make this mistake and if you do, then tough luck 2. No business ever changes, or technology never changes, or zfs deployments have short lives, or businesses are perfectly ok with large downtimes to effect geometry changes. Both responses seem antithetical to the zfs ethos of survivability in the face of errors and nondisruptive flexibility. Honestly, I still don''t understand the resistance to adding those features. -- This message posted from opensolaris.org
I believe that long term folks are working on solving this problem. I believe bp_rewrite is needed for this work. Mid/short term, the solution to me at least seems to be to migrate your data to a new zpool on the newly configured array, etc. Most enterprises don''t incrementally upgrade an array (except perhaps to add more drives, etc.) Disks are cheap enough that its usually not that hard to justify a full upgrade every few years. (Frankly, spinning rust MTBFs are still low enough that I think most sites wind up assuming that they are going to have to replace their storage on a 3-5 year cycle anyway. We''ve not yet seen what SSDs do that trend, I think.) - Garrett On Wed, 2010-07-07 at 10:54 -0700, Marty Scholes wrote:> > I think the request is to remove vdev''s from a pool. > > Not currently possible. Is this in the works? > > Actually, I think this is two requests, hashed over hundreds of times in this forum: > 1. Remove a vdev from a pool > 2. Nondisruptively change vdev geometry > > #1 above has a stunningly obvious use case. Suppose, despite your best efforts, QA, planning and walkthroughs, you accidentally fat finger a "zpool attach" and unintentionally "zpool add" a disk to a pool. There is no way to reverse that operation without *significant* downtime. > > I have discussed #2 above multiple times and has at least one obvious use case. Suppose, just for a minute, that over the years since you deployed a zfs pool with nearly constant uptime, that your business needs change and you need to add a disk to a RAIDZ vdev, or move from RAIDZ1 to RAIDZ2, or disks have grown so big that you wish to remove a disk from a vdev. > > The responses from the community on the two requests seem to be: > 1. Don''t ever make this mistake and if you do, then tough luck > 2. No business ever changes, or technology never changes, or zfs deployments have short lives, or businesses are perfectly ok with large downtimes to effect geometry changes. > > Both responses seem antithetical to the zfs ethos of survivability in the face of errors and nondisruptive flexibility. > > Honestly, I still don''t understand the resistance to adding those features.
On 2010-Jul-08 02:39:05 +0800, Garrett D''Amore <garrett at nexenta.com> wrote:>I believe that long term folks are working on solving this problem. I >believe bp_rewrite is needed for this work.Accepted.>Mid/short term, the solution to me at least seems to be to migrate your >data to a new zpool on the newly configured array, etc.IMHO, this isn''t an acceptable solution. Note that (eg) DEC/Compaq/HP AdvFS has supported vdev removal from day 1 and (until a couple of years ago), I had an AdvFS pool that had, over a decade, grown from a mirrored pair of 4.3GB disks to six pairs of mirrored 36GB disks - without needing any downtime for disk expansion. [Adding disks was done with mirror pairs because AdvFS didn''t support any RAID5/6 style redundancy, the big win was being able to remove older vdevs so those disk slots could be reused].> Most >enterprises don''t incrementally upgrade an array (except perhaps to add >more drives, etc.)This isn''t true for me. It is not uncommon for me to replace an xGB disk with a (2x)GB disk to expand an existing filesystem - in many cases, it is not possible to add more drives because there are no physical slots available. And, one of the problems with ZFS is that, unless you don''t bother with any data redundancy, it''s not possible to add single drives - you can only add vdevs that are pre-configured with the desired level of redundancy.> Disks are cheap enough that its usually not that >hard to justify a full upgrade every few years. (Frankly, spinning rust >MTBFs are still low enough that I think most sites wind up assuming that >they are going to have to replace their storage on a 3-5 year cycle >anyway. We''ve not yet seen what SSDs do that trend, I think.)Maybe in some environments. We tend to run equipment into the ground and I know other companies with similar policies. And getting approval for a couple of thousand dollars of new disks is very much easier than getting approval for a complete new SAN with (eg) twice the capacity of the existing one. -- Peter Jeremy
On Wed, Jul 7, 2010 at 3:13 PM, Peter Jeremy <peter.jeremy at alcatel-lucent.com> wrote:> On 2010-Jul-08 02:39:05 +0800, Garrett D''Amore <garrett at nexenta.com> wrote: >>I believe that long term folks are working on solving this problem. ?I >>believe bp_rewrite is needed for this work. > > Accepted. > >>Mid/short term, the solution to me at least seems to be to migrate your >>data to a new zpool on the newly configured array, etc. > > IMHO, this isn''t an acceptable solution. > > Note that (eg) DEC/Compaq/HP AdvFS has supported vdev removal from day > 1 and (until a couple of years ago), I had an AdvFS pool that had, > over a decade, grown from a mirrored pair of 4.3GB disks to six pairs > of mirrored 36GB disks - without needing any downtime for disk > expansion. ?[Adding disks was done with mirror pairs because AdvFS > didn''t support any RAID5/6 style redundancy, the big win was being > able to remove older vdevs so those disk slots could be reused].None of that requires removing top-level vdevs, and is entirely possible with ZFS today: Create the initial pool: zpool create poolname mirror disk01 disk02 Add another mirror: zpool add poolname mirror disk03 disk04 Replace one of the drives with a larger on (this may not be perfectly correct, going from memory): zpool attach poolname disk05 disk01 zpool detach poolname disk01 Carry on with the add and replace methods as needed until you have your 6-mirror pool. No vdev removals required. -- Freddie Cash fjwcash at gmail.com
On 7/7/2010 3:13 PM, Peter Jeremy wrote:> On 2010-Jul-08 02:39:05 +0800, Garrett D''Amore<garrett at nexenta.com> wrote: > >> I believe that long term folks are working on solving this problem. I >> believe bp_rewrite is needed for this work. >> > Accepted. > > >> Mid/short term, the solution to me at least seems to be to migrate your >> data to a new zpool on the newly configured array, etc. >> > IMHO, this isn''t an acceptable solution. >Problem is, it''s the long-term solution, or nothing. There''s no "partial solution". Myself and several other people have looked at (and I know I''ve tried) implementing something like a "grow a RAIDZ vdev" operation. As well as the "evacuate a vdev to shrink a zpool" operation. Conceptually, it''s not that difficult. However, as the saying goes, The Devil Is In The Details. Until there''s a reasonable way to do block pointer changes (which is generally what is encompassed in the "BP Rewrite" project/concept), you can''t implement any of these proposed methods. The edge cases will kill you. Or at least your data. All too predictably and permanently. Unfortunately, I''m not hooked into the development priority schedule, so I don''t know when bp rewrite is due, or even how it''s coming. I wish I did. A whole bunch of interesting stuff depends on getting bp rewrite implemented. Myself, I''m most interested in working on a layout optimizer (aka defrager), which would help with several current issues: resilver times, maximum disk space utilization, and performance bottlenecks.> Note that (eg) DEC/Compaq/HP AdvFS has supported vdev removal from day > 1 and (until a couple of years ago), I had an AdvFS pool that had, > over a decade, grown from a mirrored pair of 4.3GB disks to six pairs > of mirrored 36GB disks - without needing any downtime for disk > expansion. [Adding disks was done with mirror pairs because AdvFS > didn''t support any RAID5/6 style redundancy, the big win was being > able to remove older vdevs so those disk slots could be reused]. > >Yes, but none of those system were Copy On Write, which adds a layer of complexity. And, of course, what you describe above is currently possible in ZFS. That said, it''s simple to grow ZFS pool in several ways: (1) add another vdev to the pool (which doesn''t have to be redundant) (2) attach a disk/file/etc. to an existing vdev, to create a mirror (3) replace a disk/file/etc. with a larger one. (4) breaking a mirror, and using one of the former mirror disks to create another mirror All are possible with no downtime. What isn''t really possible right now is: (1) permanently removing a vdev from a pool (2) reconfiguring a raidz[123] vdev in any way>> Most >> enterprises don''t incrementally upgrade an array (except perhaps to add >> more drives, etc.) >> > This isn''t true for me. It is not uncommon for me to replace an xGB > disk with a (2x)GB disk to expand an existing filesystem - in many > cases, it is not possible to add more drives because there are no > physical slots available. And, one of the problems with ZFS is that, > unless you don''t bother with any data redundancy, it''s not possible to > add single drives - you can only add vdevs that are pre-configured with > the desired level of redundancy. >The first item you want is currently possible. Simply swap in the new drive. Now, the extra space may not be available until the ENTIRE vdev you''ve "upgraded" has the same size drives, but it''s still possible. The second falls under the case of "reconfiguring raidz[123] vdevs" and is dependent on the bp rewrite functionality.>> Disks are cheap enough that its usually not that >> hard to justify a full upgrade every few years. (Frankly, spinning rust >> MTBFs are still low enough that I think most sites wind up assuming that >> they are going to have to replace their storage on a 3-5 year cycle >> anyway. We''ve not yet seen what SSDs do that trend, I think.) >> > Maybe in some environments. We tend to run equipment into the ground > and I know other companies with similar policies. And getting approval > for a couple of thousand dollars of new disks is very much easier than > getting approval for a complete new SAN with (eg) twice the capacity > of the existing one. >For the most part, this is solved, with the caveaut that you need to buy enough replacement disks to upgrade a full vdev (i.e every disk in the vdev), but you don''t otherwise have to get a new enclosure. I''d love to get any status update on the BP Rewrite code, but, given our rather tight-lipped Oracle policies these days, I''m not hopeful. <sigh> -- Erik Trimble Java System Support Mailstop: usca22-123 Phone: x17195 Santa Clara, CA