Adrian Saul
2007-Feb-21 14:11 UTC
[zfs-discuss] suggestion: directory promotion to filesystem
Not sure how technically feasible it is, but something I thought of while shuffling some files around my home server. My poor understanding of ZFS internals is that the entire pool is effectivly a tree structure, with nodes either being data or metadata. Given that, couldnt ZFS just change a directory node to a filesystem with little effort, allowing me do everything ZFS does with filesystems on a subset of my filesystem :) Say you have some filesystems you created early on before you had a good idea of usage. Say for example I made a large share filesystem and started filling it up with photos and movies and some assorted downloads. A few months later I realise it would be so much nicer to be able to snapshot my movies and photos seperatly for backups, instead of doing the whole share. Not hard to work around - zfs create and a mv/tar command and it is done... some time later. If there was say a "zfs graft <directory> <newfs>" command, you could just break of the directory as a new filesystem and away you go - no copying, no risking cleaning up the wrong files etc. Corollary - zfs "merge" - take a filesystem and merge it into an existing filesystem. Just a thought - any comments welcome. This message posted from opensolaris.org
Sanjeev Bagewadi
2007-Feb-21 14:16 UTC
[zfs-discuss] suggestion: directory promotion to filesystem
Adrian, Seems like a cool idea to me :-) Not sure if there is anything of this kind being thought about... Would be a good idea to file an RFE. Regards, Sanjeev Adrian Saul wrote:> Not sure how technically feasible it is, but something I thought of while shuffling some files around my home server. My poor understanding of ZFS internals is that the entire pool is effectivly a tree structure, with nodes either being data or metadata. Given that, couldnt ZFS just change a directory node to a filesystem with little effort, allowing me do everything ZFS does with filesystems on a subset of my filesystem :) > > Say you have some filesystems you created early on before you had a good idea of usage. Say for example I made a large share filesystem and started filling it up with photos and movies and some assorted downloads. A few months later I realise it would be so much nicer to be able to snapshot my movies and photos seperatly for backups, instead of doing the whole share. > > Not hard to work around - zfs create and a mv/tar command and it is done... some time later. If there was say a "zfs graft <directory> <newfs>" command, you could just break of the directory as a new filesystem and away you go - no copying, no risking cleaning up the wrong files etc. > > Corollary - zfs "merge" - take a filesystem and merge it into an existing filesystem. > > Just a thought - any comments welcome. > > > This message posted from opensolaris.org > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Darren Dunham
2007-Feb-21 16:12 UTC
[zfs-discuss] suggestion: directory promotion to filesystem
> Not sure how technically feasible it is, but something I thought of > while shuffling some files around my home server. My poor > understanding of ZFS internals is that the entire pool is effectivly a > tree structure, with nodes either being data or metadata. Given that, > couldnt ZFS just change a directory node to a filesystem with little > effort, allowing me do everything ZFS does with filesystems on a > subset of my filesystem :)> Not hard to work around - zfs create and a mv/tar command and it is > done... some time later. If there was say a "zfs graft <directory> > <newfs>" command, you could just break of the directory as a new > filesystem and away you go - no copying, no risking cleaning up the > wrong files etc.I think there are some details in the tree that keep you from simply splitting them off immediately. Some of the issues were discussed a while back. I don''t know if anyone has tried to work on it or talked about alternative solutions. See also: http://www.opensolaris.org/jive/thread.jspa?messageID=28262 http://bugs.opensolaris.org/view_bug.do?bug_id=6400399 -- Darren Dunham ddunham at taos.com Senior Technical Consultant TAOS http://www.taos.com/ Got some Dr Pepper? San Francisco, CA bay area < This line left intentionally blank to confuse you. >
Matthew Ahrens
2007-Feb-21 18:11 UTC
[zfs-discuss] suggestion: directory promotion to filesystem
Adrian Saul wrote:> Not hard to work around - zfs create and a mv/tar command and it is > done... some time later. If there was say a "zfs graft <directory> > <newfs>" command, you could just break of the directory as a new > filesystem and away you go - no copying, no risking cleaning up the > wrong files etc.Yep, this idea was previously discussed on this list -- search for "zfs split" and see the following RFE: 6400399 want "zfs split" "zfs join" was also discussed but I don''t think it''s especially feasible or useful. --matt
Pawel Jakub Dawidek
2007-Feb-21 20:21 UTC
[zfs-discuss] suggestion: directory promotion to filesystem
On Wed, Feb 21, 2007 at 10:11:43AM -0800, Matthew Ahrens wrote:> Adrian Saul wrote: > >Not hard to work around - zfs create and a mv/tar command and it is > >done... some time later. If there was say a "zfs graft <directory> > ><newfs>" command, you could just break of the directory as a new > >filesystem and away you go - no copying, no risking cleaning up the > >wrong files etc. > > Yep, this idea was previously discussed on this list -- search for "zfs split" and see the following RFE: > > 6400399 want "zfs split" > > "zfs join" was also discussed but I don''t think it''s especially feasible or useful.''zfs join'' can be hard because of inode number collisions, but may be useful. Imagine a situation that you have the following file systems: /tank /tank/foo /tank/bar and you want to move huge amount of data from /tank/foo to /tank/bar. If you use mv/tar/dump it will copy entire data. Much faster will be to ''zfs join tank tank/foo && zfs join tank tank/bar'' then just mv the data and ''zfs split'' them back:) -- Pawel Jakub Dawidek http://www.wheel.pl pjd at FreeBSD.org http://www.FreeBSD.org FreeBSD committer Am I Evil? Yes, I Am! -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 187 bytes Desc: not available URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070221/d321e266/attachment.bin>
Nicholas Lee
2007-Feb-21 23:28 UTC
[zfs-discuss] suggestion: directory promotion to filesystem
On 2/22/07, Pawel Jakub Dawidek <pjd at freebsd.org> wrote:> > and you want to move huge amount of data from /tank/foo to /tank/bar. > If you use mv/tar/dump it will copy entire data. Much faster will be to > ''zfs join tank tank/foo && zfs join tank tank/bar'' then just mv the data > and ''zfs split'' them back:)Wouldn''t something like ''zfs move'' solve both these problems? Since the storage pool stays the same, you could: zfs create newpool zfs move /data/ /tank/newpool then remount newpool in the appropiate place. Nicholas -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20070222/f9fb2138/attachment.html>
Robert Milkowski
2007-Feb-22 00:03 UTC
Re[2]: suggestion: directory promotion to filesystem
Hello Nicholas, Thursday, February 22, 2007, 12:28:32 AM, you wrote: > On 2/22/07, Pawel Jakub Dawidek <pjd@freebsd.org> wrote: and you want to move huge amount of data from /tank/foo to /tank/bar. If you use mv/tar/dump it will copy entire data. Much faster will be to ''zfs join tank tank/foo && zfs join tank tank/bar'' then just mv the data and ''zfs split'' them back:) Wouldn''t something like ''zfs move'' solve both these problems? Since the storage pool stays the same, you could: zfs create newpool zfs move /data/ /tank/newpool Or even mv distributed with Solaris in case of moving a file form file system A to file system B in the same pool could "just" change some metadata via libzfs (not that there''s something like this right now). It was also discussed here IIRC but I do not remember the conclusions. -- Best regards, Robert mailto:rmilkowski@task.gda.pl http://milek.blogspot.com _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
Spencer Shepler
2007-Feb-22 00:14 UTC
[zfs-discuss] suggestion: directory promotion to filesystem
On Feb 21, 2007, at 12:11 PM, Matthew Ahrens wrote:> Adrian Saul wrote: >> Not hard to work around - zfs create and a mv/tar command and it is >> done... some time later. If there was say a "zfs graft <directory> >> <newfs>" command, you could just break of the directory as a new >> filesystem and away you go - no copying, no risking cleaning up the >> wrong files etc. > > Yep, this idea was previously discussed on this list -- search for > "zfs split" and see the following RFE: > > 6400399 want "zfs split" >Note that current draft specification for NFSv4.1 has the capability to split a filesystem such that the NFSv4.1 client will recognize it. Then the new filesystem can be migrated to another server is needed. Spencer
Adrian Saul
2007-Feb-22 11:09 UTC
[zfs-discuss] Re: suggestion: directory promotion to filesystem
thanks for the replies - I imagined it would have been discussed but must have been searching the wrong terms :) Any idea on the timeline or future of "zfs split" ? Cheers, Adrian This message posted from opensolaris.org
Matthew Ahrens
2007-Feb-23 04:01 UTC
[zfs-discuss] Re: suggestion: directory promotion to filesystem
Adrian Saul wrote:> Any idea on the timeline or future of "zfs split" ?It isn''t a priority for now. --matt
Jeff Victor
2007-Feb-23 15:18 UTC
[zfs-discuss] Re: suggestion: directory promotion to filesystem
Matthew Ahrens wrote:> Adrian Saul wrote: >> Any idea on the timeline or future of "zfs split" ? > > It isn''t a priority for now.For Sun, for the ZFS community, or both?
Robert Milkowski
2007-Feb-23 16:03 UTC
[zfs-discuss] Re: suggestion: directory promotion to filesystem
Hello Jeff, Friday, February 23, 2007, 4:18:53 PM, you wrote: JV> Matthew Ahrens wrote:>> Adrian Saul wrote: >>> Any idea on the timeline or future of "zfs split" ? >> >> It isn''t a priority for now.JV> For Sun, for the ZFS community, or both? For developers currently working on ZFS? If you want to implement it I guess no one will object :) -- Best regards, Robert mailto:rmilkowski at task.gda.pl http://milek.blogspot.com
Matthew Ahrens
2007-Feb-23 18:38 UTC
[zfs-discuss] Re: suggestion: directory promotion to filesystem
Robert Milkowski wrote:> Hello Jeff, > > Friday, February 23, 2007, 4:18:53 PM, you wrote: > > JV> Matthew Ahrens wrote: >>> Adrian Saul wrote: >>>> Any idea on the timeline or future of "zfs split" ? >>> It isn''t a priority for now. > > JV> For Sun, for the ZFS community, or both? > > For developers currently working on ZFS? > If you want to implement it I guess no one will object :)That''s right, I mean for ZFS developers at Sun. Hopefully this reflects the goals of the community at large as well (unfortunately, we have limited resources). Of course if you are interested in contributing an implementation, we would be thrilled to work with you on it! --matt