Recently, I was in a position where I was aiding someone in configuring five disks in RAID-Z1, and we were discussing whether or not it would be possible to add (not replace) disks to the pool without destroying and recreating the filesystem. As far as I know, this is not currently possible (as of nv b46) - is it a planned feature to be able to dynamically add and remove disks without destroying and recreating the RAID-Z pool? - Rich
On October 14, 2006 11:58:16 PM -0400 Rince <rincebrain at gmail.com> wrote:> Recently, I was in a position where I was aiding someone in > configuring five disks in RAID-Z1, and we were discussing whether or > not it would be possible to add (not replace) disks to the pool > without destroying and recreating the filesystem. > > As far as I know, this is not currently possible (as of nv b46) - is > it a planned feature to be able to dynamically add and remove disks > without destroying and recreating the RAID-Z pool?You don''t have a raid-z pool, you have a raid-z vdev in a zfs pool. It is not possible to add devices to a raid-z vdev, but you may not want to anyway. You may prefer to add another raid-z vdev to the zfs pool for better performance. This does limit the granularity of disks that you can add though, and increases the amount of "wasted" space. Someone said being able to add devices to a raidz vdev is on the todo list. -frank
Frank Cusack wrote:> Someone said being able to add devices to a raidz vdev is on the todo list.Don''t hold your breath. This isn''t being worked on, or on anyone''s todo list at this point, and implementing it would be pretty nontrivial. --matt
On Sun, 2006-10-15 at 20:54 -0700, Matthew Ahrens wrote:> Frank Cusack wrote: > > Someone said being able to add devices to a raidz vdev is on the todo list. > > Don''t hold your breath. This isn''t being worked on, or on anyone''s todo > list at this point, and implementing it would be pretty nontrivial.on the other hand, the source code is out there, so anyone sufficiently motivated can can start working on it now... - Bill
Hi guys, On 16 Oct 2006, at 14:30, Bill Sommerfeld wrote:> On Sun, 2006-10-15 at 20:54 -0700, Matthew Ahrens wrote: >> Frank Cusack wrote: >>> Someone said being able to add devices to a raidz vdev is on the >>> todo list. >> >> Don''t hold your breath. This isn''t being worked on, or on >> anyone''s todo >> list at this point, and implementing it would be pretty nontrivial. > > on the other hand, the source code is out there, so anyone > sufficiently > motivated can can start working on it now...Funny you should say that... I''ve been following the OpenSolaris/ZFS story with some interest as a basis for a final year computer science project. ZFS seems unique in its integration of RAID at the fs layer (and it''s open source :) !). My proposal is in its final stages now, and this is one of the items I hope to be addressing. In particular the project centres around creating redundant arrays out of mismatched disks. The ability to to dynamically grow/shrink the array vdev is the second focus (again with disks of not necessarily the same size). Anyway, I suspect you''ll be hearing a lot more from me in the near future! Cheers, James
> On 16 Oct 2006, at 14:30, Bill Sommerfeld wrote: > > On Sun, 2006-10-15 at 20:54 -0700, Matthew Ahrens wrote: > >> Frank Cusack wrote: > >>> Someone said being able to add devices to a raidz vdev is on the > >>> todo list. > >> > >> Don''t hold your breath. This isn''t being worked on, or on > >> anyone''s todo > >> list at this point, and implementing it would be pretty nontrivial. > > > > on the other hand, the source code is out there, so anyone > > sufficiently > > motivated can can start working on it now... > > Funny you should say that... I''ve been following the OpenSolaris/ZFS > story with some interest as a basis for a final year computer science > project. ZFS seems unique in its integration of RAID at the fs layer > (and it''s open source :) !). My proposal is in its final stages now, > and this is one of the items I hope to be addressing.The hard part seems to be how to deal with fragmentation of available space. Adding one disk to a full vdev wouldn''t give you much space.> In particular the project centres around creating redundant arrays > out of mismatched disks. The ability to to dynamically grow/shrink > the array vdev is the second focus (again with disks of not > necessarily the same size).Shrinking the vdevs requires moving data. Once you move data, you''ve got to either invalidate the snapshots or update them. I think that will be one of the more difficult parts. -- Darren Dunham ddunham at taos.com Senior Technical Consultant TAOS http://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. >
Hello,> Shrinking the vdevs requires moving data. Once you move data, you''ve > got to either invalidate the snapshots or update them. I think that > will be one of the more difficult parts.Updating snapshots would be non-trivial, but doable. Perhaps some sort of reverse mapping or brute force search to relate snapshots to blocks. -- Regards, Jeremy
> > Shrinking the vdevs requires moving data. Once you move data, you''ve > > got to either invalidate the snapshots or update them. I think that > > will be one of the more difficult parts. > > Updating snapshots would be non-trivial, but doable. Perhaps some sort > of reverse mapping or brute force search to relate snapshots to > blocks.Today, accounting for blocks used by snapshots is done at least partially by the internal generation number. If you modify a snapshot, you''re going to have to modify a lot of if (because of the checksums, you''ll end up having to write all the way up to the root). You''ll then have to either lie about the generation number or the existing algorithm will break. Faking the old generation number seems hard, especially since rewriting the snapshot won''t be atomic unless you have both the old and new points on disk at the same time. -- Darren Dunham ddunham at taos.com Senior Technical Consultant TAOS http://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. >
Robert Milkowski
2006-Oct-23 21:48 UTC
[zfs-discuss] Changing number of disks in a RAID-Z?
Hello Jeremy, Monday, October 23, 2006, 5:04:09 PM, you wrote: JT> Hello,>> Shrinking the vdevs requires moving data. Once you move data, you''ve >> got to either invalidate the snapshots or update them. I think that >> will be one of the more difficult parts.JT> Updating snapshots would be non-trivial, but doable. Perhaps some sort JT> of reverse mapping or brute force search to relate snapshots to JT> blocks. IMHO ability to shrink/grow pools even if restricted so no snapshots and clones can be present in a pool during shrinking/growing would still be a great feature. -- Best regards, Robert mailto:rmilkowski at task.gda.pl http://milek.blogspot.com
Robert Milkowski wrote:> Hello Jeremy, > > Monday, October 23, 2006, 5:04:09 PM, you wrote: > > JT> Hello, > >>> Shrinking the vdevs requires moving data. Once you move data, you''ve >>> got to either invalidate the snapshots or update them. I think that >>> will be one of the more difficult parts. > > > JT> Updating snapshots would be non-trivial, but doable. Perhaps some sort > JT> of reverse mapping or brute force search to relate snapshots to > JT> blocks. > > IMHO ability to shrink/grow pools even if restricted so no snapshots > and clones can be present in a pool during shrinking/growing would > still be a great feature.FYI, we''re working on being able to shrink pools with no restrictions. Unfortunately I don''t have an ETA for you on this, though. And as I''m sure you know, you can always grow pools :-) --matt
The ability to expand (and, to a less extent, shrink) a RAIDZ or RAIDZ2 device is actually one of the more critical missing features from ZFS, IMHO. It is very common for folks to add additional shelf or shelves into an existing array setup, and if you have created a pool which uses RAIDZ across the shelves (a good idea), then you want to add the new shelves into the existing RAIDZ setup. Honestly, you really don''t want to lose any snapshots of this either. I''ve talked to quite a few people who manage a good amount of storage (and I do too), and the scenario of being able to easily integrate and migrate ZFS pools around comes up often. Honestly, I''d be interested in seeing how other people feel, but I''m really waiting for more flexibility to add and remove disks from a pool. The current ZFS limitations are quite exacerbating when you deal with large amounts of disks whose arrangement changes frequently. -Erik
Erik Trimble wrote:> The ability to expand (and, to a less extent, shrink) a RAIDZ or RAIDZ2 > device is actually one of the more critical missing features from ZFS, > IMHO. It is very common for folks to add additional shelf or shelves > into an existing array setup, and if you have created a pool which uses > RAIDZ across the shelves (a good idea), then you want to add the new > shelves into the existing RAIDZ setup.Out of curiosity, what software (filesystem and/or volume manager) and configuration are you using today to achieve this? --matt
Matthew Ahrens wrote:> Erik Trimble wrote: >> The ability to expand (and, to a less extent, shrink) a RAIDZ or >> RAIDZ2 device is actually one of the more critical missing features >> from ZFS, IMHO. It is very common for folks to add additional shelf >> or shelves into an existing array setup, and if you have created a >> pool which uses RAIDZ across the shelves (a good idea), then you want >> to add the new shelves into the existing RAIDZ setup. > > Out of curiosity, what software (filesystem and/or volume manager) and > configuration are you using today to achieve this? > > --mattI can''t speak for VxVM, since I can''t remember if it has the capability, but most hardware RAID controllers and SAN controllers have had this ability for ages (which, combines with VxVM or other FS''s that can grow/shrink a FS when the underlying partition size changes). See: IBM''s ServeRAID controllers, HP''s MSA-series array heads, etc. -Erik
Robert Milkowski
2006-Oct-24 19:54 UTC
[zfs-discuss] Changing number of disks in a RAID-Z?
Hello Matthew,
Tuesday, October 24, 2006, 3:36:13 AM, you wrote:
MA> FYI, we''re working on being able to shrink pools with no
restrictions.
MA> Unfortunately I don''t have an ETA for you on this, though.
That''s great!
MA> And as I''m sure you know, you can always grow pools :-)
Well, partially - I mean that I can grow pool but old data won''t be
re-aranged (which can be good or bad - depends on situation). Also I
can''t grow raidz[12].
:)
--
Best regards,
Robert mailto:rmilkowski at task.gda.pl
http://milek.blogspot.com
Erik Trimble wrote:> Matthew Ahrens wrote: >> Erik Trimble wrote: >>> The ability to expand (and, to a less extent, shrink) a RAIDZ or >>> RAIDZ2 device is actually one of the more critical missing features >>> from ZFS, IMHO. It is very common for folks to add additional shelf >>> or shelves into an existing array setup, and if you have created a >>> pool which uses RAIDZ across the shelves (a good idea), then you want >>> to add the new shelves into the existing RAIDZ setup. >> >> Out of curiosity, what software (filesystem and/or volume manager) and >> configuration are you using today to achieve this? >> >> --matt > I can''t speak for VxVM, since I can''t remember if it has the capability, > but most hardware RAID controllers and SAN controllers have had this > ability for ages (which, combines with VxVM or other FS''s that can > grow/shrink a FS when the underlying partition size changes). > > See: IBM''s ServeRAID controllers, HP''s MSA-series array heads, etc.Right, but those are volume managers or hardware devices that export LUNs. They can shrink the LUN by simply throwing away the end of it. ZFS''s zvols can do this too. Shrinking the *filesystem* that sits on top of that LUN is a much more difficult problem! (But, as I''ve mentioned, it''s one we''re going to solve.) --matt