zfs discuss - Dec 2009 - Planed ZFS-Features - Is there a List or something else

Hi There,

does anybody know, if there''s a Roadmap oder simply a List of the
future features of ZFS?

Would be interesting to see, what will happen in the future.

THX
Henri
-- 
This message posted from opensolaris.org

Hi Henri,

The slides from the SNIA conference this past fall provide a description
of upcoming features, here:

http://www.snia.org/events/storage-developer2009/presentations/monday/JeffBonwick_zfs-What_Next-SDC09.pdf

Cindy



On 12/09/09 05:25, Henri Maddox wrote:
> Hi There,
> 
> does anybody know, if there''s a Roadmap oder simply a List of the
future features of ZFS?
> 
> Would be interesting to see, what will happen in the future.
> 
> THX
> Henri

I didn''t see "remove a simple device" anywhere in there.

Is it:
too hard to even contemplate doing, 
or
too silly a thing to do to even consider letting that happen
or 
too stupid a question to even consider
or
too easy and straightforward to do the procedure I see recommended (export the
whole pool, destroy the pool, remove the device, remake the pool, then reimport
the pool) to even bother with?
-- 
This message posted from opensolaris.org

What you''re talking about is a side-benefit of the BP rewrite section
of the linked slides.

I believe that once BP rewrite is fully baked, we''ll soon afterwards
see a device removal feature arrive.

/dale

On Dec 9, 2009, at 3:46 PM, R.G. Keen wrote:
> I didn''t see "remove a simple device" anywhere in there.
> 
> Is it:
> too hard to even contemplate doing, 
> or
> too silly a thing to do to even consider letting that happen
> or 
> too stupid a question to even consider
> or
> too easy and straightforward to do the procedure I see recommended (export
the whole pool, destroy the pool, remove the device, remake the pool, then
reimport the pool) to even bother with?
> -- 
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

* R.G. Keen (keen at geofex.com) wrote:
> I didn''t see "remove a simple device" anywhere in there.
> 
> Is it:
> too hard to even contemplate doing, 
> or
> too silly a thing to do to even consider letting that happen
> or 
> too stupid a question to even consider
> or
> too easy and straightforward to do the procedure I see recommended (export
the whole pool, destroy the pool, remove the device, remake the pool, then
reimport the pool) to even bother with?
You missed:

Too hard to do correctly with current resource levels and other higher
priority work.

As always, volunteers I''m sure are welcome. :-)

Cheers,

-- 
Glenn

R.G. Keen wrote:
> I didn''t see "remove a simple device" anywhere in there.
>
> Is it:
> too hard to even contemplate doing, 
> or
> too silly a thing to do to even consider letting that happen
> or 
> too stupid a question to even consider
> or
> too easy and straightforward to do the procedure I see recommended (export
the whole pool, destroy the pool, remove the device, remake the pool, then
reimport the pool) to even bother with?
>   
It is too complicated to implement directly.

As with lvm2 and comparable technologies, one would have to first have a
feature that moves all extents from one physical volume to the other
available phys.-vols. Then, when allocating the replacement blocks, the
algorithm could quickly become _very_ unwieldy, because the pool will
still have to keep it''s redundancy guarantees [1].

As you can imagine this can be very complex with ZFS mixture of raid,
parity, _dynamic_ striping (simply realllocating the blocks could cause
massively fragmented disks if the pool/vdev previoiusly used dynamic
striping). Using ''copies=n'' and extra parity (raidz2,raidz3)
further
complicates the matter. In all circumstances about the only algorithm to
specify for the transformation _with_ the guarantee that all invariants
are (logically[2]) checked is to use the wellknown send/recv kludge. In
that case you''ll simply need double the storage and a lot of processing
resources to make the transform.

There are a number of situations in which the logic can safely be
simplified (using only dynamic striping and using only full disks and
when there is a ''third'' (recent disk) not involved in any of
the
existing stripes to receive the relocated stripes.... etc. In effect, I
doubt that these situations are ever going to  cover more than what
''detach'' and ''replace'' offer at this moment
in time.

So, in a word, yes this is (very) complicated.  The complicating thing
is that ZFS does dynamic striping and RAID redundancy properties
_automagically_. This dynamicity make it very hard to define what needs
to happen when a disk is removed (likewise for replacing with a smaller
disk). ''Static'' RAID tools have the advantage here, because
they can
guarantee how stripes are layout across a ''pool'', and also
because the
admin can limit the options used for a pool precisely to enable
''special
operations'' like ''remove physdev''. However, even if
so, removal off a
disk (as opposed to replacement) is a very uncommon use case for any
RAID solution that I know of.




[1] of course, you could replace that complexity by a burden on the
user: let removal of a drive have the same effect as physically failing
that device, degrading the pool. Then you would have to either replace
the vdev or re-add a vdev to restore the redundancy.

[2] by which I mean, barring bugs in, say, send/recv

Dale Ghent wrote:
> What you''re talking about is a side-benefit of the BP rewrite
section of the linked slides.
>
> I believe that once BP rewrite is fully baked, we''ll soon
afterwards see a device removal feature arrive.
>
> /dale
>
> On Dec 9, 2009, at 3:46 PM, R.G. Keen wrote:
>
>   
>> I didn''t see "remove a simple device" anywhere in
there.
>>
>> Is it:
>> too hard to even contemplate doing, 
>> or
>> too silly a thing to do to even consider letting that happen
>> or 
>> too stupid a question to even consider
>> or
>> too easy and straightforward to do the procedure I see recommended
(export the whole pool, destroy the pool, remove the device, remake the pool,
then reimport the pool) to even bother with?
>> --
BP rewrite is key to several oft-asked features:  vdev removal, defrag, 
raidz expansion, among others.

-- 
Erik Trimble
Java System Support
Mailstop:  usca22-123
Phone:  x17195
Santa Clara, CA
Timezone: US/Pacific (GMT-0800)

On 12/09/09 13:52, Glenn Lagasse wrote:
> * R.G. Keen (keen at geofex.com) wrote:
>> I didn''t see "remove a simple device" anywhere in
there.
>>
>> Is it:
>> too hard to even contemplate doing, 
>> or
>> too silly a thing to do to even consider letting that happen
>> or 
>> too stupid a question to even consider
>> or
>> too easy and straightforward to do the procedure I see recommended
(export the whole pool, destroy the pool, remove the device, remake the pool,
then reimport the pool) to even bother with?
> 
> You missed:
> 
> Too hard to do correctly with current resource levels and other higher
> priority work.
> 
> As always, volunteers I''m sure are welcome. :-)
> 
This gives the impression that development is not actively working
on it. This is not true. As has been said often it is a difficult problem
and has been actively worked on for a few months now. I don''t think
we are prepared to give a date as to when it will be delivered though.

Neil.

* Neil Perrin (Neil.Perrin at Sun.COM) wrote:
> 
> 
> On 12/09/09 13:52, Glenn Lagasse wrote:
> >* R.G. Keen (keen at geofex.com) wrote:
> >>I didn''t see "remove a simple device" anywhere
in there.
> >>
> >>Is it:
> >>too hard to even contemplate doing, or
> >>too silly a thing to do to even consider letting that happen
> >>or too stupid a question to even consider
> >>or
> >>too easy and straightforward to do the procedure I see recommended
(export the whole pool, destroy the pool, remove the device, remake the pool,
then reimport the pool) to even bother with?
> >
> >You missed:
> >
> >Too hard to do correctly with current resource levels and other higher
> >priority work.
> >
> >As always, volunteers I''m sure are welcome. :-)
> >
> 
> This gives the impression that development is not actively working
> on it. This is not true. As has been said often it is a difficult problem
True.  I apologize for the misleading nature of my comment.  I should
have pointed out that I don''t work on the ZFS project but was relating
what I believed the possible answer could be based upon past list
postings of the subject.
> and has been actively worked on for a few months now. I don''t
think
> we are prepared to give a date as to when it will be delivered though.
Cool!

-- 
Glenn

Neil Perrin wrote:
>
>
> On 12/09/09 13:52, Glenn Lagasse wrote:
>> * R.G. Keen (keen at geofex.com) wrote:
>>> I didn''t see "remove a simple device" anywhere
in there.
>>>
>>> Is it:
>>> too hard to even contemplate doing, or
>>> too silly a thing to do to even consider letting that happen
>>> or too stupid a question to even consider
>>> or
>>> too easy and straightforward to do the procedure I see recommended 
>>> (export the whole pool, destroy the pool, remove the device, remake
>>> the pool, then reimport the pool) to even bother with?
>>
>> You missed:
>>
>> Too hard to do correctly with current resource levels and other higher
>> priority work.
>>
>> As always, volunteers I''m sure are welcome. :-)
>>
>
> This gives the impression that development is not actively working
> on it. This is not true. As has been said often it is a difficult problem
> and has been actively worked on for a few months now. I don''t
think
> we are prepared to give a date as to when it will be delivered though.

This should go on a "collection of things people ask about a lot," if 
such a thing were to exist.  Oh, wait...

http://hub.opensolaris.org/bin/view/Community+Group+zfs/faq#HCandevicesberemovedfromaZFSpool