Hi, I''ve read a few articles about the lack of ''simple'' raidz pool expansion capability in ZFS. I am interested in having a go at developing this functionality. Is anyone working on this at the moment? I''ll explain what I am proposing. As mentioned in many forums, the concept is really simple: allow a raidz pool to grow by adding one or more disks to an existing pool. My intended user group is the consumer market, as opposed to the enterprise, so I expect I''ll put some rather strict limitations on how/when this functionality will operate: to make the first implementation more achievable. The use case I will try and solve first is, what I see as, the simplest. I have a raidz pool configured with 1 file system on top; no snapshots. I want to add an additional disk (must be at least the same size as the rest of the disks in the pool). I don''t mind if there is some downtime. I want all my data to take advantage of the additional disk. What is the benefit to the consumer? The answer is simple: - more flexibility in growing storage i.e. can have an odd number of disks. - more disk space available for use e.g. 2 pools of 3 disks gives less available space than 1 pool of 6 disks. - consistent with many RAID-5 implementations - opens up the consumer market for raidz: growable small backup/SAN/Home Theatre appliances I''m no expert on any of this stuff, but I do have many years experience as a software engineer. Is there a mentoring program that Sun offers so I can get some assistance when necessary? My expectation is that this isn''t impossible to do but it isn''t simple to do either. Are there any procedural hoops I need to jump through to take on this piece of work? Regards, Ty -- This message posted from opensolaris.org
I''m just a casual at ZFS but you want something that now don''t exists. The most of the consumers want this but Sun is not interested in that market. To grow a existing RAIDZ just adding more disk to the RAIDZ would be great but at this moment there isn''t anything like that. -- This message posted from opensolaris.org
On Sep 2, 2009, at 7:14 PM, rarok wrote:> I''m just a casual at ZFS but you want something that now don''t > exists. The most of the consumers want this but Sun is not > interested in that market. To grow a existing RAIDZ just adding more > disk to the RAIDZ would be great but at this moment there isn''t > anything like that.I would change "customers" to "users", many people who use ZFS for their home server would like this, but they are often not customers. Henrik http://sparcv9.blogspot.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090902/9dc8fa8c/attachment.html>
Sorry, I''m not english native and sometimes I don''t find the correct words for explaining some things. But yes I''m a costumer, I''m running a NAS with a intel atom and opensolaris with a RAIDZ, but it''s a pain that I can''t expand it just adding another disk. -- This message posted from opensolaris.org
On Fri, Aug 28, 2009 at 11:22 PM, Ty Newton<ty.newton at copperchipgames.com> wrote:> Hi, > I''ve read a few articles about the lack of ''simple'' raidz pool expansion capability in ZFS. ?I am interested in having a go at developing this functionality. ?Is anyone working on this at the moment? > > I''ll explain what I am proposing. ?As mentioned in many forums, the concept is really simple: allow a raidz pool to grow by adding one or more disks to an existing pool. ?My intended user group is the consumer market, as opposed to the enterprise, so I expect I''ll put some rather strict limitations on how/when this functionality will operate: to make the first implementation more achievable. > > The use case I will try and solve first is, what I see as, the simplest. ?I have a raidz pool configured with 1 file system on top; no snapshots. ?I want to add an additional disk (must be at least the same size as the rest of the disks in the pool). ?I don''t mind if there is some downtime. ?I want all my data to take advantage of the additional disk.Have you looked at the ''add'' section of the zpool manpage? You can add another vdev, provided it provides similar parity, something like: zpool add data raidz2 c4t14d0 c4t15d0 c5t12d0 c5t13d0 c5t14d0 which I did a few weeks ago. Here, I had a raidz2 pool called ''data'' made up of 5 disks. I added another 5 disks also configured as raidz2 with this command.
> On Sep 2, 2009, at 7:14 PM, rarok wrote: > >> I''m just a casual at ZFS but you want something that now don''t exists. >> The most of the consumers want this but Sun is not interested in that >> market. To grow a existing RAIDZ just adding more disk to the RAIDZ >> would be great but at this moment there isn''t anything like that.Out of curiosity, what do the folks who want to grow their raidzs do for backups? Is restoring a backup to a newly created enlarged raidz any more dangerous than the rewriting involved in doing it on the fly? Hardware is so cheap these days, why not make a backup raidz server (power it up only to do backups, or better yet, switch to it periodically to make sure it works), and when the time comes to make the raidz bigger, just do it, one server at a time? You can run off the backup whilst the new, larger server is resilvering and have negligable downtime that way. If you are really cheap, get a couple of huge USB drives and do the backups there. Either way, they are important, and zfs send/recv is such a great way of making verifiable backups. Cheers -- Frank
With that you add another raidz to the pool but you don''t enlarge the zpool. I think that the word is that you can''t change the geometry of a created raidz -- This message posted from opensolaris.org
At least for me, I don''t use backups If I have 4x 1 TB disks in RAIDZ it means that I would need to spent at least 300? prior to add just a 100? disk. And about USB drives... they are even expensive, because they have the hard disk and also the adapter for USB. I''m just a home user with many data to store. -- This message posted from opensolaris.org
rarok wrote:> I''m just a casual at ZFS but you want something that now don''t exists. The most of the consumers want this but Sun is not interested in that market. To grow a existing RAIDZ just adding more disk to the RAIDZ would be great but at this moment there isn''t anything like that.Can we stop the FUD please. It isn''t that Sun is not interested is that it is hard code to implement and get correct and there are higher priority tasks ahead of getting this done. There are foundations that are needed for this and other things being worked on just now. -- Darren J Moffat
On Wed, 02 Sep 2009 10:44:40 PDT, rarok <rarokillo at gmail.com> wrote:> Blake <blake.irvin at gmail.com> wrote:> > Have you looked at the ''add'' section of the zpool > > manpage? You can add another vdev, provided it > > provides similar parity, something like: > > > > zpool add data raidz2 c4t14d0 c4t15d0 c5t12d0 c5t13d0 c5t14d0 > > > > which I did a few weeks ago. Here, I had a raidz2 pool > > called ''data'' made up of 5 disks. I added another 5 > > disks also configured as raidz2 with this command.> With that you add another raidz to the pool but > you don''t enlarge the zpool.Yes, he enlarged the pool. The existing vdev (the first set of disks) is still there and he added another vdev (a second set of disks, in raidz2 configuration) to the pool. So, the pool is certainly bigger.> I think that the word is that you can''t > change the geometry of a created raidz.That''s not fully true. You can replace every disk in vdev (mirror, raidz, raidz2) by a bigger one, one by one, resilvering after every replacement, and grow the existing vdev, thus growing the pool while keeping hte same configuration. -- ( Kees Nuyt ) c[_]
If anyone is interested in tackling this project, I found a blog spelling out how to go about it at http://blogs.sun.com/ahl/entry/expand_o_matic_raid_z. That also spells out their position on the priorities related to this project. I''m not in an enterprise situation, nor in a home situation, but rather an edu department. We cannot afford to just go out and buy another 5 drives or so whenever we want to expand a zpool (by adding another raidz to it). Typically, a faculty member with a research grant will want some more space, and we ask them to buy a drive. I would love to be able to just add that drive to an existing raidz. -- This message posted from opensolaris.org
ZFS is worth putting a little thought into your system when you START it. If you want to be able easily add a couple disks at a time, just use mirrors, I user raidz vdevs of 4 and when i need to expand i have 2 options. I add a new raidz vdev of 4 disks OR i replace all 4 disks in one vdev with larger disks. ZFS is amazing, and well worth this method of upgrades. Hard drives are cheap enough these days where you can buy a few at a time, also, i can tell you from experience that when you DO grow a nas you almost ALWAYS want to add a lot more than the last time. My first nas was 3 hard drives, when i added just a single disk i found this to not be nearly enough, i''ve ended up always adding more each time. I switched to ZFS around 8 disks. I''m very happy with it. The snapshots and clones make your daily usage so much nicer. I think the ability to grow by a single disk will eventually be added to ZFS but even without it, it is an amazing filesystem and by far the best for my needs. Like i said, if you put a little pre planning into your system, it won''t be a major issue, and there is NOTHING wrong with using mirrored groups. On Fri, Aug 28, 2009 at 11:22 PM, Ty Newton <ty.newton at copperchipgames.com>wrote:> Hi, > I''ve read a few articles about the lack of ''simple'' raidz pool expansion > capability in ZFS. I am interested in having a go at developing this > functionality. Is anyone working on this at the moment? > > I''ll explain what I am proposing. As mentioned in many forums, the concept > is really simple: allow a raidz pool to grow by adding one or more disks to > an existing pool. My intended user group is the consumer market, as opposed > to the enterprise, so I expect I''ll put some rather strict limitations on > how/when this functionality will operate: to make the first implementation > more achievable. > > The use case I will try and solve first is, what I see as, the simplest. I > have a raidz pool configured with 1 file system on top; no snapshots. I > want to add an additional disk (must be at least the same size as the rest > of the disks in the pool). I don''t mind if there is some downtime. I want > all my data to take advantage of the additional disk. > > What is the benefit to the consumer? The answer is simple: > - more flexibility in growing storage i.e. can have an odd number of disks. > - more disk space available for use e.g. 2 pools of 3 disks gives less > available space than 1 pool of 6 disks. > - consistent with many RAID-5 implementations > - opens up the consumer market for raidz: growable small backup/SAN/Home > Theatre appliances > > > I''m no expert on any of this stuff, but I do have many years experience as > a software engineer. Is there a mentoring program that Sun offers so I can > get some assistance when necessary? My expectation is that this isn''t > impossible to do but it isn''t simple to do either. > > Are there any procedural hoops I need to jump through to take on this piece > of work? > > > Regards, > Ty > -- > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090916/ee178fe3/attachment.html>