zfs discuss - Feb 2007 - suggestion: directory promotion to filesystem

Not sure how technically feasible it is, but something I thought of while
shuffling some files around my home server.  My poor understanding of ZFS
internals is that the entire pool is effectivly a tree structure, with nodes
either being data or metadata.  Given that, couldnt ZFS just change a directory
node to a filesystem with little effort, allowing me do everything ZFS does with
filesystems on a subset of my filesystem :)

Say you have some filesystems you created early on before you had a good idea of
usage.  Say for example I made a large share filesystem and started filling it
up with photos and movies and some assorted downloads.  A few months later I
realise it would be so much nicer to be able to snapshot my movies and photos
seperatly for backups, instead of doing the whole share.

Not hard to work around - zfs create and a mv/tar command and it is done... some
time later.  If there was say  a "zfs graft <directory>
<newfs>" command, you could just break of the directory as a new
filesystem and away you go - no copying, no risking cleaning up the wrong files
etc.

Corollary - zfs "merge" - take a filesystem and merge it into an
existing filesystem.

Just a thought - any comments welcome.
 
 
This message posted from opensolaris.org

Adrian,

Seems like a cool idea to me :-) Not sure if there is anything of this 
kind being thought about...
Would be a good idea to file an RFE.

Regards,
Sanjeev

Adrian Saul wrote:
> Not sure how technically feasible it is, but something I thought of while
shuffling some files around my home server.  My poor understanding of ZFS
internals is that the entire pool is effectivly a tree structure, with nodes
either being data or metadata.  Given that, couldnt ZFS just change a directory
node to a filesystem with little effort, allowing me do everything ZFS does with
filesystems on a subset of my filesystem :)
>
> Say you have some filesystems you created early on before you had a good
idea of usage.  Say for example I made a large share filesystem and started
filling it up with photos and movies and some assorted downloads.  A few months
later I realise it would be so much nicer to be able to snapshot my movies and
photos seperatly for backups, instead of doing the whole share.
>
> Not hard to work around - zfs create and a mv/tar command and it is done...
some time later.  If there was say  a "zfs graft <directory>
<newfs>" command, you could just break of the directory as a new
filesystem and away you go - no copying, no risking cleaning up the wrong files
etc.
>
> Corollary - zfs "merge" - take a filesystem and merge it into an
existing filesystem.
>
> Just a thought - any comments welcome.
>  
>  
> This message posted from opensolaris.org
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>

> Not sure how technically feasible it is, but something I thought of
> while shuffling some files around my home server.  My poor
> understanding of ZFS internals is that the entire pool is effectivly a
> tree structure, with nodes either being data or metadata.  Given that,
> couldnt ZFS just change a directory node to a filesystem with little
> effort, allowing me do everything ZFS does with filesystems on a
> subset of my filesystem :)
> Not hard to work around - zfs create and a mv/tar command and it is
> done... some time later.  If there was say a "zfs graft
<directory>
> <newfs>" command, you could just break of the directory as a new
> filesystem and away you go - no copying, no risking cleaning up the
> wrong files etc.
I think there are some details in the tree that keep you from simply
splitting them off immediately.  

Some of the issues were discussed a while back.  I don''t know if anyone
has tried to work on it or talked about alternative solutions.

See also:
http://www.opensolaris.org/jive/thread.jspa?messageID=28262
http://bugs.opensolaris.org/view_bug.do?bug_id=6400399

-- 
Darren Dunham                                           ddunham at taos.com
Senior Technical Consultant         TAOS            http://www.taos.com/
Got some Dr Pepper?                           San Francisco, CA bay area
         < This line left intentionally blank to confuse you. >

Adrian Saul wrote:
> Not hard to work around - zfs create and a mv/tar command and it is
> done... some time later.  If there was say  a "zfs graft
<directory>
> <newfs>" command, you could just break of the directory as a new
> filesystem and away you go - no copying, no risking cleaning up the
> wrong files etc.
Yep, this idea was previously discussed on this list -- search for "zfs 
split" and see the following RFE:

6400399 want "zfs split"

"zfs join" was also discussed but I don''t think it''s
especially feasible or
useful.

--matt

On Wed, Feb 21, 2007 at 10:11:43AM -0800, Matthew Ahrens
wrote:
> Adrian Saul wrote:
> >Not hard to work around - zfs create and a mv/tar command and it is
> >done... some time later.  If there was say  a "zfs graft
<directory>
> ><newfs>" command, you could just break of the directory as a
new
> >filesystem and away you go - no copying, no risking cleaning up the
> >wrong files etc.
> 
> Yep, this idea was previously discussed on this list -- search for
"zfs split" and see the following RFE:
> 
> 6400399 want "zfs split"
> 
> "zfs join" was also discussed but I don''t think
it''s especially feasible or useful.
''zfs join'' can be hard because of inode number collisions, but
may be
useful. Imagine a situation that you have the following file systems:

	/tank
	/tank/foo
	/tank/bar

and you want to move huge amount of data from /tank/foo to /tank/bar.
If you use mv/tar/dump it will copy entire data. Much faster will be to
''zfs join tank tank/foo && zfs join tank tank/bar''
then just mv the data
and ''zfs split'' them back:)

-- 
Pawel Jakub Dawidek                       http://www.wheel.pl
pjd at FreeBSD.org                           http://www.FreeBSD.org
FreeBSD committer                         Am I Evil? Yes, I Am!
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070221/d321e266/attachment.bin>

On 2/22/07, Pawel Jakub Dawidek <pjd at freebsd.org>
wrote:
>
> and you want to move huge amount of data from /tank/foo to /tank/bar.
> If you use mv/tar/dump it will copy entire data. Much faster will be to
> ''zfs join tank tank/foo && zfs join tank
tank/bar'' then just mv the data
> and ''zfs split'' them back:)


Wouldn''t something like ''zfs move'' solve both these
problems?

Since the storage pool stays the same, you could:
zfs create newpool
zfs move /data/ /tank/newpool

then remount newpool in the appropiate place.

Nicholas
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070222/f9fb2138/attachment.html>

Hello Nicholas,

Thursday, February 22, 2007, 12:28:32 AM, you wrote:

>

On 2/22/07, Pawel Jakub Dawidek <pjd@freebsd.org> wrote:

and you want to move huge amount of data from /tank/foo to /tank/bar.

If you use mv/tar/dump it will copy entire data. Much faster will be to

''zfs join tank tank/foo && zfs join tank
tank/bar'' then just mv the data 

and ''zfs split'' them back:)

Wouldn''t something like ''zfs move'' solve both these
problems?

Since the storage pool stays the same, you could:

zfs create newpool

zfs move /data/ /tank/newpool

Or even mv distributed with Solaris in case of moving a file form file system A
to file system B

in the same pool could "just" change some metadata via libzfs (not
that there''s something like this right now).

It was also discussed here IIRC but I do not remember the conclusions. 

-- 

Best regards,

 Robert                            mailto:rmilkowski@task.gda.pl

                                       http://milek.blogspot.com


_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss

On Feb 21, 2007, at 12:11 PM, Matthew Ahrens wrote:
> Adrian Saul wrote:
>> Not hard to work around - zfs create and a mv/tar command and it is
>> done... some time later.  If there was say  a "zfs graft
<directory>
>> <newfs>" command, you could just break of the directory as a
new
>> filesystem and away you go - no copying, no risking cleaning up the
>> wrong files etc.
>
> Yep, this idea was previously discussed on this list -- search for  
> "zfs split" and see the following RFE:
>
> 6400399 want "zfs split"
>
Note that current draft specification for NFSv4.1 has the capability
to split a filesystem such that the NFSv4.1 client will recognize it.
Then the new filesystem can be migrated to another server is needed.

Spencer

thanks for the replies - I imagined it would have been discussed but must have
been searching the wrong terms :)

Any idea on the timeline or future of "zfs split" ?

Cheers,
 Adrian
 
 
This message posted from opensolaris.org

Adrian Saul wrote:
> Any idea on the timeline or future of "zfs split" ?
It isn''t a priority for now.

--matt

Matthew Ahrens wrote:
> Adrian Saul wrote:
>> Any idea on the timeline or future of "zfs split" ?
> 
> It isn''t a priority for now.
For Sun, for the ZFS community, or both?

Hello Jeff,

Friday, February 23, 2007, 4:18:53 PM, you wrote:

JV> Matthew Ahrens wrote:
>> Adrian Saul wrote:
>>> Any idea on the timeline or future of "zfs split" ?
>> 
>> It isn''t a priority for now.
JV> For Sun, for the ZFS community, or both?

For developers currently working on ZFS?
If you want to implement it I guess no one will object :)


-- 
Best regards,
 Robert                            mailto:rmilkowski at task.gda.pl
                                       http://milek.blogspot.com

Robert Milkowski wrote:
> Hello Jeff,
> 
> Friday, February 23, 2007, 4:18:53 PM, you wrote:
> 
> JV> Matthew Ahrens wrote:
>>> Adrian Saul wrote:
>>>> Any idea on the timeline or future of "zfs split" ?
>>> It isn''t a priority for now.
> 
> JV> For Sun, for the ZFS community, or both?
> 
> For developers currently working on ZFS?
> If you want to implement it I guess no one will object :)
That''s right, I mean for ZFS developers at Sun.  Hopefully this
reflects
the goals of the community at large as well (unfortunately, we have 
limited resources).  Of course if you are interested in contributing an 
implementation, we would be thrilled to work with you on it!

--matt