If you''ve got nested zfs filesystems, and you''re in some
subdirectory where
there''s a file or something you want to rollback, it''s
presently difficult
to know how far back up the tree you need to go, to find the correct
".zfs"
subdirectory, and then you need to figure out the name of the snapshots
available, and then you need to perform the restore, even after you figure
all that out.
This is pretty good, but it could be better. The solution should be
cross-platform compatible, because the user who wishes to perform such
operations may be working across an NFS or CIFS mountpoint.
If something like this already exists, please let me know. Otherwise, I
plan to:
Create "zfshistory" command, written in python. (open source, public,
free)
zfshistory would have the following subcommands:
ls list snapshots that contain the specified file or directory
cp copies a file or directory from a past snapshot to a new name
in the current version of the filesystem
rollback delete the present version of a named file or directory, and
replace it with the specified past snapshot version
locate list complete paths to all the snapshot versions of
the specified file or directory
Example usage:
rm somefile
(Whoops!)
zfshistory ls somefile
somefile at frequent-2010-04-16-12-45-00
somefile at frequent-2010-04-16-12-30-00
somefile at frequent-2010-04-16-12-15-00
somefile at frequent-2010-04-16-12-00-00
somefile at hourly-2010-04-16-12-00-00
zfshistory cp somefile at frequent-2010-04-16-12-45-00 ./mynewfilename
zfshistory rollback somefile
(restores somefile to the latest snapshot available)
zfshistory rollback somefile at frequent-2010-04-16-12-00-00
(restores somefile to the specified snapshot)
zfshistory locate somefile
/tank/.zfs/snapshot/frequent-2010-04-16-12-45-00/home/username/somefile
/tank/.zfs/snapshot/frequent-2010-04-16-12-30-00/home/username/somefile
/tank/.zfs/snapshot/frequent-2010-04-16-12-15-00/home/username/somefile
/tank/.zfs/snapshot/frequent-2010-04-16-12-00-00/home/username/somefile
/tank/.zfs/snapshot/hourly-2010-04-16-12-00-00/home/username/somefile
On Fri, Apr 16, 2010 at 01:54:45PM -0400, Edward Ned Harvey wrote:> If you''ve got nested zfs filesystems, and you''re in some subdirectory where > there''s a file or something you want to rollback, it''s presently difficult > to know how far back up the tree you need to go, to find the correct ".zfs" > subdirectory, and then you need to figure out the name of the snapshots > available, and then you need to perform the restore, even after you figure > all that out.I''ve a ksh93 script that lists all the snapshotted versions of a file... Works over NFS too. % zfshist /usr/bin/ls History for /usr/bin/ls (/.zfs/snapshot/*/usr/bin/ls): -r-xr-xr-x 1 root bin 33416 Jul 9 2008 /.zfs/snapshot/install/usr/bin/ls -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-12-07-20:47:58/usr/bin/ls -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-12-01-00:42:30/usr/bin/ls -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-07-17-21:08:45/usr/bin/ls -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-06-03-03:44:34/usr/bin/ls % It''s not perfect (e.g., it doesn''t properly canonicalize its arguments, so it doesn''t handle symlinks and ..s in paths), but it''s a start. Nico --
On Apr 16, 2010, at 1:37 PM, Nicolas Williams wrote:> On Fri, Apr 16, 2010 at 01:54:45PM -0400, Edward Ned Harvey wrote: >> If you''ve got nested zfs filesystems, and you''re in some subdirectory where >> there''s a file or something you want to rollback, it''s presently difficult >> to know how far back up the tree you need to go, to find the correct ".zfs" >> subdirectory, and then you need to figure out the name of the snapshots >> available, and then you need to perform the restore, even after you figure >> all that out. > > I''ve a ksh93 script that lists all the snapshotted versions of a file... > Works over NFS too. > > % zfshist /usr/bin/ls > History for /usr/bin/ls (/.zfs/snapshot/*/usr/bin/ls): > -r-xr-xr-x 1 root bin 33416 Jul 9 2008 /.zfs/snapshot/install/usr/bin/ls > -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-12-07-20:47:58/usr/bin/ls > -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-12-01-00:42:30/usr/bin/ls > -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-07-17-21:08:45/usr/bin/ls > -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-06-03-03:44:34/usr/bin/ls > % > > It''s not perfect (e.g., it doesn''t properly canonicalize its arguments, > so it doesn''t handle symlinks and ..s in paths), but it''s a start.There are some interesting design challenges here. For the general case, you can''t rely on the snapshot name to be in time order, so you need to sort by the mtime of the destination. It would be cool to only list files which are different. If you mv a file to another directory, you might want to search by filename or a partial directory+filename. Regexp :-) Or maybe you just setup your tracker.cfg and be happy? -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
On Fri, Apr 16, 2010 at 02:19:47PM -0700, Richard Elling wrote:> On Apr 16, 2010, at 1:37 PM, Nicolas Williams wrote: > > I''ve a ksh93 script that lists all the snapshotted versions of a file... > > Works over NFS too. > > > > % zfshist /usr/bin/ls > > History for /usr/bin/ls (/.zfs/snapshot/*/usr/bin/ls): > > -r-xr-xr-x 1 root bin 33416 Jul 9 2008 /.zfs/snapshot/install/usr/bin/ls > > -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-12-07-20:47:58/usr/bin/ls > > -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-12-01-00:42:30/usr/bin/ls > > -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-07-17-21:08:45/usr/bin/ls > > -r-xr-xr-x 1 root bin 37612 Nov 21 2008 /.zfs/snapshot/2009-06-03-03:44:34/usr/bin/ls > > % > > > > It''s not perfect (e.g., it doesn''t properly canonicalize its arguments, > > so it doesn''t handle symlinks and ..s in paths), but it''s a start. > > There are some interesting design challenges here. For the general case, you > can''t rely on the snapshot name to be in time order, so you need to sort by the > mtime of the destination.I''m using ls -ltr.> It would be cool to only list files which are different.True. That''d not be hard.> If you mv a file to another directory, you might want to search by filename > or a partial directory+filename.Or even inode number.> Or maybe you just setup your tracker.cfg and be happy?Exactly. Nico --
On Fri, Apr 16, 2010 at 2:19 PM, Richard Elling <richard.elling at gmail.com>wrote:> Or maybe you just setup your tracker.cfg and be happy? >What''s a "tracker.cfg", and how would it help ZFS users on non-Solaris systems? ;) -- Freddie Cash fjwcash at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100416/0a083b41/attachment.html>
On Apr 16, 2010, at 2:58 PM, Freddie Cash wrote:> On Fri, Apr 16, 2010 at 2:19 PM, Richard Elling <richard.elling at gmail.com> wrote: > Or maybe you just setup your tracker.cfg and be happy? > > What''s a "tracker.cfg", and how would it help ZFS users on non-Solaris systems? ;)tracker is the gnome answer to spotlight. http://projects.gnome.org/tracker/ -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
> From: Richard Elling [mailto:richard.elling at gmail.com] > > There are some interesting design challenges here. For the general > case, you > can''t rely on the snapshot name to be in time order, so you need to > sort by the > mtime of the destination.Actually ... drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-01-00-00-00/ drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-02-00-00-00/ drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-03-00-00-00/ drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-04-00-00-00/ drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-05-00-00-00/ drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-06-00-00-00/ Um ... All the same time. Even if I "stat" those directories ... Access: Modify: and Change: are all useless... How in the heck can you identify when a snapshot was taken, if you''re not relying on the name of the snapshot?> It would be cool to only list files which are different.Know of any way to do that?
On Apr 16, 2010, at 5:33 PM, Edward Ned Harvey wrote:>> From: Richard Elling [mailto:richard.elling at gmail.com] >> >> There are some interesting design challenges here. For the general >> case, you >> can''t rely on the snapshot name to be in time order, so you need to >> sort by the >> mtime of the destination. > > Actually ... > > drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-01-00-00-00/ > drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-02-00-00-00/ > drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-03-00-00-00/ > drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-04-00-00-00/ > drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-05-00-00-00/ > drwxr-xr-x 16 root root 20 Mar 29 12:07 daily-2010-04-06-00-00-00/ > > Um ... All the same time. > Even if I "stat" those directories ... > Access: Modify: and Change: are all useless...which is why you need to stat the destination :-)> > How in the heck can you identify when a snapshot was taken, if you''re not > relying on the name of the snapshot?"zfs list -t snapshot" lists in time order.> > >> It would be cool to only list files which are different. > > Know of any way to do that?cmp But since the world has moved onto time machine, time slider, and whatever Windows users use, is this tool relegated to the CLI dinosaurs? ;-) -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
> From: Richard Elling [mailto:richard.elling at gmail.com] > > > > Um ... All the same time. > > Even if I "stat" those directories ... > > Access: Modify: and Change: are all useless... > > which is why you need to stat the destination :-)Ahh. I see it now. By stat''ing the destination instead of the snapshot directory, you get the added bonus of being able to identify which snapshot versions actually have a changed file in it. I like it. The only negative I see about that is: Some tools such as TrueCrypt intentionally keep the mtime on their files constant. It would be nice to be able to identify authoritatively which files have changed or not, without relying on the mtime. But I think that''s a corner case, and nothing that can be improved for now. So, this is probably the best strategy, I agree with you. ;-) Stat the file itself, not the snapshot directory.> > How in the heck can you identify when a snapshot was taken, if you''re > not > > relying on the name of the snapshot? > > "zfs list -t snapshot" lists in time order.Good to know. I''ll keep that in mind for my "zfs send" scripts but it''s not relevant for the case at hand. Because "zfs list" isn''t available on the NFS client, where the users are trying to do this sort of stuff.> >> It would be cool to only list files which are different. > > > > Know of any way to do that? > > cmpOh, no. Because cmp and diff require reading both files, it could take forever, especially if you have a lot of snapshots to check, with a large file or set of files... Well, what the heck. Might as well make it optional. Sometimes people will just want to check a single small file.> But since the world has moved onto time machine, time slider, and > whatever Windows users use, is this tool relegated to the CLI > dinosaurs? ;-)It''s not hard to add an IE context extension and/or create some other gui frontend in whatever OS you''re using. But command line comes first.
On 4/17/2010 9:03 AM, Edward Ned Harvey wrote:> >>>> It would be cool to only list files which are different. >>>> >>> Know of any way to do that? >>> >> cmp >> > Oh, no. Because cmp and diff require reading both files, it could take > forever, especially if you have a lot of snapshots to check, with a large > file or set of files... Well, what the heck. Might as well make it > optional. Sometimes people will just want to check a single small file. > >I think I saw an ARC case go by recently for anew ''zfs diff'' command. I think it allows you compare 2 snapshots, or maybe the live filesystem and a snapshot and see what''s changed. It sounds really useful, Hopefully it will integrate soon.
> From: Kyle McDonald [mailto:kmcdonald at egenera.com] > > I think I saw an ARC case go by recently for anew ''zfs diff'' command. I > think it allows you compare 2 snapshots, or maybe the live filesystem > and a snapshot and see what''s changed. > > It sounds really useful, Hopefully it will integrate soon.Yes, I''m hopeful for that too. But it won''t help any NFS or CIFS client.
On Sat, Apr 17, 2010 at 09:03:33AM -0400, Edward Ned Harvey wrote:> > "zfs list -t snapshot" lists in time order. > > Good to know. I''ll keep that in mind for my "zfs send" scripts but it''s not > relevant for the case at hand. Because "zfs list" isn''t available on the > NFS client, where the users are trying to do this sort of stuff.I''ll note for comparison that the Netapp shapshots do expose this in one way. The actual snapshot directory access time is set to the time of the snapshot. That makes it visible over NFS. Would be handy to do something similar in ZFS. # ls -lut total 72 drwxr-xr-x 8 root root 4096 Apr 20 09:24 manual_snap drwxr-xr-x 8 root root 4096 Apr 20 08:00 hourly.0 drwxr-xr-x 8 root root 4096 Apr 20 00:00 nightly.0 drwxr-xr-x 8 root root 4096 Apr 19 20:00 hourly.1 [...] -- Darren
On Tue, Apr 20, 2010 at 04:28:02PM +0000, A Darren Dunham wrote:> On Sat, Apr 17, 2010 at 09:03:33AM -0400, Edward Ned Harvey wrote: > > > "zfs list -t snapshot" lists in time order. > > > > Good to know. I''ll keep that in mind for my "zfs send" scripts but it''s not > > relevant for the case at hand. Because "zfs list" isn''t available on the > > NFS client, where the users are trying to do this sort of stuff. > > I''ll note for comparison that the Netapp shapshots do expose this in one > way. > > The actual snapshot directory access time is set to the time of the > snapshot. That makes it visible over NFS. Would be handy to do > something similar in ZFS.The .zfs/snapshot directory is most certainly available over NFS. But note that .zfs does not appear in directory listings of dataset roots -- you have to actually refer to it: % ls -f|fgrep .zfs % ls -f .zfs . .. snapshot % ls .zfs/snapshot <snapshots> % nfsstat -m $PWD /net/.../pool/nico from ...:/pool/nico Flags: vers=4,proto=tcp,sec=sys,hard,intr,link,symlink,acl,mirrormount,rsize=1048576,wsize=1048576,retrans=5,timeo=600 Attr cache: acregmin=3,acregmax=60,acdirmin=30,acdirmax=60 % And you can even create, rename and destroy snapshots by creating, renaming and removing directories in .zfs/snapshot: % mkdir .zfs/snapshot/foo % mv .zfs/snapshot/foo .zfs/snapshot/bar % rmdir .zfs/snapshot/bar (All this also works locally, not just over ZFS.) Nico --
Nicolas Williams wrote:> On Tue, Apr 20, 2010 at 04:28:02PM +0000, A Darren Dunham wrote: >> On Sat, Apr 17, 2010 at 09:03:33AM -0400, Edward Ned Harvey wrote: >>>> "zfs list -t snapshot" lists in time order. >>> Good to know. I''ll keep that in mind for my "zfs send" scripts but it''s not >>> relevant for the case at hand. Because "zfs list" isn''t available on the >>> NFS client, where the users are trying to do this sort of stuff. >> I''ll note for comparison that the Netapp shapshots do expose this in one >> way. >> >> The actual snapshot directory access time is set to the time of the >> snapshot. That makes it visible over NFS. Would be handy to do >> something similar in ZFS. > > The .zfs/snapshot directory is most certainly available over NFS.[ irrelevant stuff removed ] Yes, it is. With a useless timestamp. Please re-read the thread. -- Carson
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Nicolas Williams > > The .zfs/snapshot directory is most certainly available over NFS.I''m not sure you''ve been following this thread. Nobody said .zfs/snapshot wasn''t available over NFS. It was said that all the snapshot subdirectories ".zfs/snapshot/frequent-blah" and ".zfs/snapshot/hourly-foo" and so on ... Over NFS there''s no way to know the time those snapshots were taken. There is a convention of writing the time of the snapshot into the name of the snapshot, but if you can''t rely on that, then the NFS client doesn''t know the order of snapshots.> And you can even create, rename and destroy snapshots by creating, > renaming and removing directories in .zfs/snapshot: > > % mkdir .zfs/snapshot/foo > % mv .zfs/snapshot/foo .zfs/snapshot/bar > % rmdir .zfs/snapshot/bar > > (All this also works locally, not just over ZFS.)Holy crap, for real? I''ll have to try that. ;-)
On Fri, Apr 16, 2010 at 10:54 AM, Edward Ned Harvey <solaris2 at nedharvey.com> wrote:> there''s a file or something you want to rollback, it''s presently difficult > to know how far back up the tree you need to go, to find the correct ".zfs" > subdirectory, and then you need to figure out the name of the snapshotsThere is one feature that OnTap has which I miss in zfs. Every directory has a hidden .snapshot directory, so you never need to look in the parents. -B -- Brandon High : bhigh at freaks.com
On 21/04/2010 05:09, Edward Ned Harvey wrote:>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- >> bounces at opensolaris.org] On Behalf Of Nicolas Williams >> >> The .zfs/snapshot directory is most certainly available over NFS. > > I''m not sure you''ve been following this thread. Nobody said .zfs/snapshot > wasn''t available over NFS. It was said that all the snapshot subdirectories > ".zfs/snapshot/frequent-blah" and ".zfs/snapshot/hourly-foo" and so on ... > > Over NFS there''s no way to know the time those snapshots were taken. There > is a convention of writing the time of the snapshot into the name of the > snapshot, but if you can''t rely on that, then the NFS client doesn''t know > the order of snapshots.Correct and think this is because POSIX has not such thing as a file creation date (ctime,mtime,atime is all it has) - ZFS actually does have a creation date but there isn''t a way to expose that over NFS that I''m aware of and ls/stat can''t see it because stat(2) doesn''t have a way to expose it. The snapshot dir is just another directory and over NFS you are looking at it with NFS so it shows the ctime,mtime,atime of the top level directory of the ZFS dataset at the time the snapshot was created. This RFE if it were to be implemented could give a possible way to get this information from a remote filesystem client: 6527390 want to read zfs properties over nfs (eg via .zfs/props) -- Darren J Moffat
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Edward Ned Harvey > > > From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > > bounces at opensolaris.org] On Behalf Of Nicolas Williams > > > > And you can even create, rename and destroy snapshots by creating, > > renaming and removing directories in .zfs/snapshot: > > > > % mkdir .zfs/snapshot/foo > > % mv .zfs/snapshot/foo .zfs/snapshot/bar > > % rmdir .zfs/snapshot/bar > > > > (All this also works locally, not just over ZFS.) > > Holy crap, for real? I''ll have to try that. ;-)I just tested that, and it works (cool, thanks for the tip Nicolas.) However, I think you phrased it slightly incorrect. At least on my systems (solaris 10), let me correct this: You can create/destroy/rename snapshots via mkdir, rmdir, mv inside the .zfs/snapshot directory, however, it will only work if you''re running the command locally. It will not work from a NFS client.
On 4/21/10 6:49 AM, Edward Ned Harvey wrote:>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- >> bounces at opensolaris.org] On Behalf Of Edward Ned Harvey >> >>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- >>> bounces at opensolaris.org] On Behalf Of Nicolas Williams >>> >>> And you can even create, rename and destroy snapshots by creating, >>> renaming and removing directories in .zfs/snapshot: >>> >>> % mkdir .zfs/snapshot/foo >>> % mv .zfs/snapshot/foo .zfs/snapshot/bar >>> % rmdir .zfs/snapshot/bar >>> >>> (All this also works locally, not just over ZFS.) >> >> Holy crap, for real? I''ll have to try that. ;-) > > I just tested that, and it works (cool, thanks for the tip Nicolas.) > However, I think you phrased it slightly incorrect. At least on my systems > (solaris 10), let me correct this: > > You can create/destroy/rename snapshots via mkdir, rmdir, mv inside the > .zfs/snapshot directory, however, it will only work if you''re running the > command locally. It will not work from a NFS client. >It will work over NFS or SMB, but you will need to allow it via the necessary delegated administration permissions.
On 04/21/10 03:24 AM, Darren J Moffat wrote:> On 21/04/2010 05:09, Edward Ned Harvey wrote: >>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- >>> bounces at opensolaris.org] On Behalf Of Nicolas Williams >>> >>> The .zfs/snapshot directory is most certainly available over NFS. >> >> I''m not sure you''ve been following this thread. Nobody said .zfs/snapshot >> wasn''t available over NFS. It was said that all the snapshot >> subdirectories >> ".zfs/snapshot/frequent-blah" and ".zfs/snapshot/hourly-foo" and so on >> ... >> >> Over NFS there''s no way to know the time those snapshots were taken. >> There >> is a convention of writing the time of the snapshot into the name of the >> snapshot, but if you can''t rely on that, then the NFS client doesn''t know >> the order of snapshots. > > Correct and think this is because POSIX has not such thing as a file > creation date (ctime,mtime,atime is all it has) - ZFS actually does have > a creation date but there isn''t a way to expose that over NFS that I''m > aware of and ls/stat can''t see it because stat(2) doesn''t have a way to > expose it. >You can see it with ls: # ls -ld -% all /net/server/export/ws/timh/nvc drwxr-xr-x 9 timh staff 13 Apr 21 01:25 /net/server/export/ws/timh/nvc/ timestamp: atime Apr 21 07:54:05 2010 timestamp: ctime Apr 21 01:25:48 2010 timestamp: mtime Apr 21 01:25:48 2010 timestamp: crtime Jun 22 08:27:23 2009 and I believe you can retrieve it with fgetattr() from the read/write view. -tim> The snapshot dir is just another directory and over NFS you are looking > at it with NFS so it shows the ctime,mtime,atime of the top level > directory of the ZFS dataset at the time the snapshot was created. > > This RFE if it were to be implemented could give a possible way to get > this information from a remote filesystem client: > > 6527390 want to read zfs properties over nfs (eg via .zfs/props) >
> From: Mark Shellenbaum [mailto:mark.shellenbaum at oracle.com] > > > > You can create/destroy/rename snapshots via mkdir, rmdir, mv inside > the > > .zfs/snapshot directory, however, it will only work if you''re running > the > > command locally. It will not work from a NFS client. > > > > It will work over NFS or SMB, but you will need to allow it via the > necessary delegated administration permissions.Go on? I tried it over NFS and it didn''t work. So ... what are the "necessary permissions?" I did it from a NFS client as root, where root maps to root.
On Wed, Apr 21, 2010 at 10:45:24AM -0400, Edward Ned Harvey wrote:> > From: Mark Shellenbaum [mailto:mark.shellenbaum at oracle.com] > > > > > > You can create/destroy/rename snapshots via mkdir, rmdir, mv inside > > the > > > .zfs/snapshot directory, however, it will only work if you''re running > > the > > > command locally. It will not work from a NFS client. > > > > > > > It will work over NFS or SMB, but you will need to allow it via the > > necessary delegated administration permissions. > > Go on? > I tried it over NFS and it didn''t work. So ... what are the "necessary > permissions?"See zfs(1M), search for "delegate".> I did it from a NFS client as root, where root maps to root.Huh; dunno why that didn''t work. Nico --
On 04/21/10 08:45 AM, Edward Ned Harvey wrote:>> From: Mark Shellenbaum [mailto:mark.shellenbaum at oracle.com] >>> >>> You can create/destroy/rename snapshots via mkdir, rmdir, mv inside >> the >>> .zfs/snapshot directory, however, it will only work if you''re running >> the >>> command locally. It will not work from a NFS client. >>> >> >> It will work over NFS or SMB, but you will need to allow it via the >> necessary delegated administration permissions. > > Go on? > I tried it over NFS and it didn''t work. So ... what are the "necessary > permissions?" >Depends on what you want to do If all you want is to allow a certain NFS/SMB user the ability to create snapshots then all you would need is something like this # zfs allow <user> snapshot <dataset> But if you want to let them delete snapshot then you would also need to give out destroy,mount. # zfs allow <user> destroy,mount <dataset> renaming will also require "create" permission.> I did it from a NFS client as root, where root maps to root. >If you aren''t using delegation then the code will require PRIV_SYS_MOUNT and an NFS won''t have that privilege.
On Apr 20, 2010, at 10:21 PM, Brandon High wrote:> On Fri, Apr 16, 2010 at 10:54 AM, Edward Ned Harvey > <solaris2 at nedharvey.com> wrote: >> there''s a file or something you want to rollback, it''s presently difficult >> to know how far back up the tree you need to go, to find the correct ".zfs" >> subdirectory, and then you need to figure out the name of the snapshots > > There is one feature that OnTap has which I miss in zfs. Every > directory has a hidden .snapshot directory, so you never need to look > in the parents.What happens when you remove the directory? -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Tim Haley > > You can see it with ls: > # ls -ld -% all /net/server/export/ws/timh/nvc > drwxr-xr-x 9 timh staff 13 Apr 21 01:25 > /net/server/export/ws/timh/nvc/ > timestamp: atime Apr 21 07:54:05 2010 > timestamp: ctime Apr 21 01:25:48 2010 > timestamp: mtime Apr 21 01:25:48 2010 > timestamp: crtime Jun 22 08:27:23 2009This is precisely the point of the discussion. That doesn''t work. Observe: (First of all, what system do you have -% on? I don''t have it on either solaris 10, or rhel 4) You want to get the atime, ctime, mtime, crtime. I am now repeating what''s already been discussed in this thread. Observe: The atime, ctime, mtime are all useless. Because here are two different snapshots, taken on different days, and the mtime etc are identical. You can''t use that information to figure out which is newer, if you distrust the name of the snap to give you the time of the snap. It''s been mentioned that you can "zfs list -t snapshot" to get them sorted correctly, but that''s the best we''ve got, and it''s only available on the host itself. Not available on NFS client. (From the ZFS file server itself) I haven''t found any way to even display the mtime. Neither ls or stat. This was all I found: [root at nas snapshot]# python>>> import os >>> os.stat(''/share/.zfs/snapshot/daily-2010-04-21-00-00-00'')(16877, 3L, 47514090L, 16, 0, 0, 20L, 1269878840, 1269878847, 1269878847)>>> >>> os.stat(''/share/.zfs/snapshot/daily-2010-04-20-00-00-00'')(16877, 3L, 47514101L, 16, 0, 0, 20L, 1269878840, 1269878847, 1269878847) The last 3 are atime, mtime, ctime. All identical. (From a NFS client, RHEL 4) [eharvey at air ~]$ stat /share/.zfs/snapshot/daily-2010-04-21-00-00-00 /share/.zfs/snapshot/daily-2010-04-20-00-00-00 File: `/share/.zfs/snapshot/daily-2010-04-21-00-00-00'' Size: 20 Blocks: 3 IO Block: 32768 directory Device: 10h/16d Inode: 3 Links: 16 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2010-03-29 12:07:20.478083718 -0400 Modify: 2010-03-29 12:07:27.110569183 -0400 Change: 2010-03-29 12:07:27.110569183 -0400 File: `/share/.zfs/snapshot/daily-2010-04-20-00-00-00'' Size: 20 Blocks: 3 IO Block: 32768 directory Device: 10h/16d Inode: 3 Links: 16 Access: (0755/drwxr-xr-x) Uid: ( 0/ root) Gid: ( 0/ root) Access: 2010-03-29 12:07:20.478083718 -0400 Modify: 2010-03-29 12:07:27.110569183 -0400 Change: 2010-03-29 12:07:27.110569183 -0400
> From: Brandon High [mailto:bhigh at freaks.com] > > On Fri, Apr 16, 2010 at 10:54 AM, Edward Ned Harvey > <solaris2 at nedharvey.com> wrote: > > there''s a file or something you want to rollback, it''s presently > difficult > > to know how far back up the tree you need to go, to find the correct > ".zfs" > > subdirectory, and then you need to figure out the name of the > snapshots > > There is one feature that OnTap has which I miss in zfs. Every > directory has a hidden .snapshot directory, so you never need to look > in the parents.Agreed. But I don''t know why it''s not implemented here. Some patent or something? Maybe.
> From: Richard Elling [mailto:richard.elling at gmail.com] > > On Apr 20, 2010, at 10:21 PM, Brandon High wrote: > > > On Fri, Apr 16, 2010 at 10:54 AM, Edward Ned Harvey > > <solaris2 at nedharvey.com> wrote: > >> there''s a file or something you want to rollback, it''s presently > difficult > >> to know how far back up the tree you need to go, to find the correct > ".zfs" > >> subdirectory, and then you need to figure out the name of the > snapshots > > > > There is one feature that OnTap has which I miss in zfs. Every > > directory has a hidden .snapshot directory, so you never need to look > > in the parents. > > What happens when you remove the directory?Same thing that happens when you remove the .zfs directory. You can''t. Also, the .snapshot directory isn''t visible via regular ls or whatever, unless you explicitly name it, just like .zfs directory. Long story short, the .snapshot directory, and the .zfs directory are pretty much identical concepts, except that the .snapshot directory is accessible in every subdirectory, and it has a list of the snapshots inside it, already down at that directory level. Whereas the .zfs directory you can only find if you go back up to the head of the filesystem, and then you have to select a snapshot and tunnel down again to the level you''re at. Make sense? The .snapshot idea is better. But presumably not allowable in ZFS for either technical or legal reasons.
On Apr 21, 2010, at 8:35 AM, Edward Ned Harvey wrote:>> From: Richard Elling [mailto:richard.elling at gmail.com] >> >> On Apr 20, 2010, at 10:21 PM, Brandon High wrote: >> >>> On Fri, Apr 16, 2010 at 10:54 AM, Edward Ned Harvey >>> <solaris2 at nedharvey.com> wrote: >>>> there''s a file or something you want to rollback, it''s presently >> difficult >>>> to know how far back up the tree you need to go, to find the correct >> ".zfs" >>>> subdirectory, and then you need to figure out the name of the >> snapshots >>> >>> There is one feature that OnTap has which I miss in zfs. Every >>> directory has a hidden .snapshot directory, so you never need to look >>> in the parents. >> >> What happens when you remove the directory? > > Same thing that happens when you remove the .zfs directory. You can''t.Are you sure I cannot rmdir on a NetApp? That seems like basic functionality to me. Or are you thinking "rmdir dirname/.snapshot" when I''m thinking "rmdir dirname; mkdir dirname" which is a common operation in a developer environment. Or "mv dirname dirname-old; mv dirname-new dirname" which is common when managing software upgrades that are not clone-aware.> Also, the .snapshot directory isn''t visible via regular ls or whatever, > unless you explicitly name it, just like .zfs directory. > > Long story short, the .snapshot directory, and the .zfs directory are pretty > much identical concepts, except that the .snapshot directory is accessible > in every subdirectory, and it has a list of the snapshots inside it, already > down at that directory level. Whereas the .zfs directory you can only find > if you go back up to the head of the filesystem, and then you have to select > a snapshot and tunnel down again to the level you''re at. > > Make sense? The .snapshot idea is better. But presumably not allowable in > ZFS for either technical or legal reasons.This works only if you do not change the directory structure (DOS 1.0? :-) -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
On 21/04/2010 16:35, Edward Ned Harvey wrote:>> From: Richard Elling [mailto:richard.elling at gmail.com] >> >> On Apr 20, 2010, at 10:21 PM, Brandon High wrote: >> >>> On Fri, Apr 16, 2010 at 10:54 AM, Edward Ned Harvey >>> <solaris2 at nedharvey.com> wrote: >>>> there''s a file or something you want to rollback, it''s presently >> difficult >>>> to know how far back up the tree you need to go, to find the correct >> ".zfs" >>>> subdirectory, and then you need to figure out the name of the >> snapshots >>> >>> There is one feature that OnTap has which I miss in zfs. Every >>> directory has a hidden .snapshot directory, so you never need to look >>> in the parents. >> >> What happens when you remove the directory? > > Same thing that happens when you remove the .zfs directory. You can''t.I don''t think it was the ".zfs" directory but the normal POSIX directory. For example: /foo is the filesystem /foo/bar is a directory in the filesystem cd /foo/bar/ touch stuff [ you wait, time passes; a snapshot is taken ] At this point /foo/bar/.snapshot/.../stuff exists Now do this: rm -rf /foo/bar There is a snapshot of /foo/bar/stuff in the ZFS model to get to it you go to /foo/.zfs/snapshot/<name>/bar and in there you will find the file called stuff. How do you find what was /foo/bar/stuff in the model where the .snapshot directory exists at every subdir rather than just at the filesystem root when the subdirs have been removed ? What does it look like when the directory hierarchy is really deep ? -- Darren J Moffat
> From: Richard Elling [mailto:richard.elling at gmail.com] > > >> What happens when you remove the directory? > > > > Same thing that happens when you remove the .zfs directory. You > can''t. > > Are you sure I cannot rmdir on a NetApp? That seems like basic > functionality to me. > > Or are you thinking "rmdir dirname/.snapshot" when I''m thinking > "rmdir dirname; mkdir dirname" which is a common operation > in a developer environment. Or "mv dirname dirname-old; > mv dirname-new dirname" which is common when managing > software upgrades that are not clone-aware.I don''t see what the confusion is. In ZFS: cd /tank/home/eharvey mkdir foo touch foo/bar sudo zfs snapshot tank at bestsnapshotever rm -rf foo ls /tank/.zfs/snapshot/bestsnapshotever/home/eharvey/foo bar In NetApp: cd /netapp-nfs-mountpoint/home/eharvey mkdir foo touch foo/bar (as root on the netapp) snap create vol0 somesnapshot (back on my nfs client) rm foo/bar ls -l foo/.snapshot/somesnapshot bar ls -l .snapshot/somesnapshot/foo bar rmdir foo ls -l foo/.snapshot foo: No such file or directory ls -l .snapshot/somesnapshot/foo bar Point is, in Data Ontap, *every* directory has a .snapshot subdirectory. If you rmdir a directory, then you can find that directory in the .snapshot of its parent. Does that answer it? Point is, this way you never have to guess how many filesystems are nested within higher level zfs filesystems, you never have to guess how far up the tree you need to go, in order to find the correct .zfs subdirectory which contains the snapshots for your PWD. It''s simply more convenient to use netapp style .snapshot subdirectories instead. Plus, all my users can remember ".snapshot" but they don''t remember ".zfs" or even worse: "Use the .zfs, but only at the 3rd level of parent directories, to access snaps for your home, but go one level higher for the snaps of anything outside your home directory..." This is a tiny little point, where netapp is simply better. And I''m sure there are some other points, but I don''t personally care about them as much as the advantage zfs has over netapp. Namely, installable on normal hardware, no license fees necessary to snap replicate (zfs send) filesystem copies all over my world. Snap onto removable disks. Restore using a free black box, mount it in my laptop, etc etc. At present, the workaround I have for zfs is: ln -s .zfs/snapshot snapshot This makes the snapshot directory plainly visible to all NFS and CIFS users. Easy to find every time, easy to remember. Especially important for Mac cifs clients, because there''s no addressbar to type in ".zfs" even if you knew that''s what you want to do.
POSIX doesn''t allow us to have special dot files/directories outside filesystem root directories. Nico --
ISTR POSIX also doesn''t allow a number of features that can be turned
on with zfs (even ignoring the current issues that prevent ZFS from
being fully POSIX compliant today). I think an additional option for
the snapdir property (''directory'' ?) that provides this
behavior (with
suitable warnings about posix compliance) would be reasonable.
I believe it''s sufficient that zfs provide the necessary options to
act in a posix compliant manner (much like you have to set $PATH
correctly to get POSIX conforming behavior, even though that might not
be the default), though I''m happy to be corrected about this.
On Wed, Apr 21, 2010 at 12:45 PM, Nicolas Williams
<Nicolas.Williams at oracle.com> wrote:> POSIX doesn''t allow us to have special dot files/directories
outside
> filesystem root directories.
>
> Nico
> --
> _______________________________________________
> zfs-discuss mailing list
> zfs-discuss at opensolaris.org
> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
>
On Apr 21, 2010, at 10:38 AM, Edward Ned Harvey wrote:>> From: Richard Elling [mailto:richard.elling at gmail.com] >> >>>> What happens when you remove the directory? >>> >>> Same thing that happens when you remove the .zfs directory. You >> can''t. >> >> Are you sure I cannot rmdir on a NetApp? That seems like basic >> functionality to me. >> >> Or are you thinking "rmdir dirname/.snapshot" when I''m thinking >> "rmdir dirname; mkdir dirname" which is a common operation >> in a developer environment. Or "mv dirname dirname-old; >> mv dirname-new dirname" which is common when managing >> software upgrades that are not clone-aware. > > I don''t see what the confusion is. > > In ZFS: > cd /tank/home/eharvey > mkdir foo > touch foo/bar > sudo zfs snapshot tank at bestsnapshotever > rm -rf foo > ls /tank/.zfs/snapshot/bestsnapshotever/home/eharvey/foo > bar > > In NetApp: > cd /netapp-nfs-mountpoint/home/eharvey > mkdir foo > touch foo/bar > (as root on the netapp) snap create vol0 somesnapshot > (back on my nfs client) > rm foo/bar > ls -l foo/.snapshot/somesnapshot > bar > ls -l .snapshot/somesnapshot/foo > bar > rmdir foo > ls -l foo/.snapshot > foo: No such file or directory > ls -l .snapshot/somesnapshot/foo > bar > > Point is, in Data Ontap, *every* directory has a .snapshot subdirectory. If > you rmdir a directory, then you can find that directory in the .snapshot of > its parent. > > Does that answer it?So you are saying that the OnTap .snapshot directory is equivalent to a symlink to $FSROOT/.zfs/snapshot? That would "solve" the directory shuffle problem.> Point is, this way you never have to guess how many filesystems are nested > within higher level zfs filesystems, you never have to guess how far up the > tree you need to go, in order to find the correct .zfs subdirectory which > contains the snapshots for your PWD. It''s simply more convenient to use > netapp style .snapshot subdirectories instead. Plus, all my users can > remember ".snapshot" but they don''t remember ".zfs" or even worse: "Use the > .zfs, but only at the 3rd level of parent directories, to access snaps for > your home, but go one level higher for the snaps of anything outside your > home directory..." > > This is a tiny little point, where netapp is simply better. And I''m sure > there are some other points, but I don''t personally care about them as much > as the advantage zfs has over netapp. Namely, installable on normal > hardware, no license fees necessary to snap replicate (zfs send) filesystem > copies all over my world. Snap onto removable disks. Restore using a free > black box, mount it in my laptop, etc etc. > > At present, the workaround I have for zfs is: > ln -s .zfs/snapshot snapshot > This makes the snapshot directory plainly visible to all NFS and CIFS users. > Easy to find every time, easy to remember. Especially important for Mac > cifs clients, because there''s no addressbar to type in ".zfs" even if you > knew that''s what you want to do.So a cron job with a find will solve the problem, no? -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
On Wed, Apr 21, 2010 at 01:03:39PM -0500, Jason King wrote:> ISTR POSIX also doesn''t allow a number of features that can be turned > on with zfs (even ignoring the current issues that prevent ZFS from > being fully POSIX compliant today). I think an additional option for > the snapdir property (''directory'' ?) that provides this behavior (with > suitable warnings about posix compliance) would be reasonable. > > I believe it''s sufficient that zfs provide the necessary options to > act in a posix compliant manner (much like you have to set $PATH > correctly to get POSIX conforming behavior, even though that might not > be the default), though I''m happy to be corrected about this.Yes, that''s true. But you couldn''t rely on this behavior, whereas you can rely on dataset roots having .zfs. If you''re going to script this, then you''ll want to rely on the current (POSIX-compliant) behavior.
Richard Elling wrote:> So you are saying that the OnTap .snapshot directory is equivalent to a symlink > to $FSROOT/.zfs/snapshot? That would "solve" the directory shuffle problem.Not quite. It''s equivalent(ish) to: cd "$MYDIR" && mkdir .snapshot && cd .snapshot for s in "$FSROOT"/.zfs/snapshot/*; do test -d "$FSROOT"/.zfs/snapshot/"$s"/"${MYDIR##$FSROOT}" && \ ln -s "$FSROOT"/.zfs/snapshot/"$s"/"${MYDIR##$FSROOT}" "$s" done I''d have to test to see what happens when a directory is renamed. I _suspect_ the .snapshot/$snapname dirs will either not exist or be empty. -- Carson
Nicolas Williams wrote:> On Wed, Apr 21, 2010 at 01:03:39PM -0500, Jason King wrote: >> ISTR POSIX also doesn''t allow a number of features that can be turned >> on with zfs (even ignoring the current issues that prevent ZFS from >> being fully POSIX compliant today). I think an additional option for >> the snapdir property (''directory'' ?) that provides this behavior (with >> suitable warnings about posix compliance) would be reasonable. >> >> I believe it''s sufficient that zfs provide the necessary options to >> act in a posix compliant manner (much like you have to set $PATH >> correctly to get POSIX conforming behavior, even though that might not >> be the default), though I''m happy to be corrected about this. > > Yes, that''s true. But you couldn''t rely on this behavior, whereas you > can rely on dataset roots having .zfs. If you''re going to script this, > then you''ll want to rely on the current (POSIX-compliant) behavior.This isn''t about scripting, it''s about users. For a user to self-restore a "whoops" deletion, she needs to: sd="$PWD" while ! test -d "$sd"/.zfs/snapshot; do sd="${sd%/*}; done if test -n "$sd"; then echo "Snapshots are in $sd/.zfs/snapshot" else echo "No snapshot dir found" fi Which is so user hostile the SA gets a call/email/ticket to do it for them. And what happens if the root of the ZFS filesystem isn''t even NFS mounted? User home dirs, for example, where we don''t make every home dir its own zfs? There is no .zfs dir, so... -- Carson
On Wed, Apr 21, 2010 at 10:38 AM, Edward Ned Harvey <solaris2 at nedharvey.com> wrote:> At present, the workaround I have for zfs is: > ? ? ? ?ln -s .zfs/snapshot snapshot > This makes the snapshot directory plainly visible to all NFS and CIFS users. > Easy to find every time, easy to remember. ?Especially important for Mac > cifs clients, because there''s no addressbar to type in ".zfs" even if you > knew that''s what you want to do.You can also set snapdir=visible for the datasets that you care about, which will make them show up for all users. -B -- Brandon High : bhigh at freaks.com
It still has the issue that the end user has to know where the root of the filesystem is in the tree (assuming it''s even accessible on the system -- might not be for an NFS mount). On Wed, Apr 21, 2010 at 6:01 PM, Brandon High <bhigh at freaks.com> wrote:> On Wed, Apr 21, 2010 at 10:38 AM, Edward Ned Harvey > <solaris2 at nedharvey.com> wrote: >> At present, the workaround I have for zfs is: >> ? ? ? ?ln -s .zfs/snapshot snapshot >> This makes the snapshot directory plainly visible to all NFS and CIFS users. >> Easy to find every time, easy to remember. ?Especially important for Mac >> cifs clients, because there''s no addressbar to type in ".zfs" even if you >> knew that''s what you want to do. > > You can also set ?snapdir=visible for the datasets that you care > about, which will make them show up for all users. > > -B > > -- > Brandon High : bhigh at freaks.com > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
> From: Nicolas Williams [mailto:Nicolas.Williams at oracle.com] > > POSIX doesn''t allow us to have special dot files/directories outside > filesystem root directories.So? Tell it to Netapp. They don''t seem to have any problem with it.
> From: Richard Elling [mailto:richard.elling at gmail.com] > > So you are saying that the OnTap .snapshot directory is equivalent to a > symlink > to $FSROOT/.zfs/snapshot? That would "solve" the directory shuffle > problem.Not quite. In Ontap, all you do is go into .snapshot, and select which snap you want, and then you see the contents of PWD, from a time gone by. The process is exactly the same, no matter how deep into a tree your PWD happens to be, and no matter how far up is the root of the filesystem. All of the following would be identical in Data Ontap: /share/home/joeuser/foo/.snapshot/bestsnapever/bar /share/home/joeuser/.snapshot/bestsnapever/foo/bar /share/home/.snapshot/bestsnapever/joeuser/foo/bar /share/.snapshot/bestsnapever/home/joeuser/foo/bar This means, if you''ve got a bunch of snapshots, snap1, snap2, snap3, etc... You would see those listed inside of *any* of the .snapshot directories, and if you went into it, you would be already at the same path depth that you were before. If you did the symlink .snapshot --> $FSROOT/.zfs/snapshot, and somehow made that magically appear in every directory all the time, you would have this: /share/home/joeuser/foo/.snapshot/bestsnapever/home/joeuser/foo/bar /share/home/joeuser/.snapshot/bestsnapever/home/joeuser/foo/bar /share/home/.snapshot/bestsnapever/home/joeuser/foo/bar /share/.snapshot/bestsnapever/home/joeuser/foo/bar Point is, after you choose which snap you want to look at (bestsnapever, snap1, snap2, etc) you have to tunnel down through all the parent directories again, to reach the equivalent of your PWD. Ontap does that automatically for you.> So a cron job with a find will solve the problem, no?I hope I don''t need to list all the ways that''s wrong, given what I wrote above. But I''ll list two. #1 No, it doesn''t work. #2 On my system, which is a single 4U server, it would take a full day for the cron job to walk the tree once. Well, maybe not. But you get the point.
> If you did the symlink .snapshot --> $FSROOT/.zfs/snapshot, and somehow > made > that magically appear in every directory all the time, you would have > this: > /share/home/joeuser/foo/.snapshot/bestsnapever/home/joeuser/foo/bar > /share/home/joeuser/.snapshot/bestsnapever/home/joeuser/foo/bar > /share/home/.snapshot/bestsnapever/home/joeuser/foo/bar > /share/.snapshot/bestsnapever/home/joeuser/foo/barHehehe, not to mention, you''d have this: /share/home/.snapshot/bestsnapever/.snapshot/bestsnapever/.snapshot/bestsnap ever/.snapshot/bestsnapever/.snapshot/bestsnapever/
On Apr 21, 2010, at 7:27 PM, Edward Ned Harvey wrote:>> From: Richard Elling [mailto:richard.elling at gmail.com] >> >> So you are saying that the OnTap .snapshot directory is equivalent to a >> symlink >> to $FSROOT/.zfs/snapshot? That would "solve" the directory shuffle >> problem. > > Not quite. > > In Ontap, all you do is go into .snapshot, and select which snap you want, > and then you see the contents of PWD, from a time gone by. The process is > exactly the same, no matter how deep into a tree your PWD happens to be, and > no matter how far up is the root of the filesystem. > > All of the following would be identical in Data Ontap: > /share/home/joeuser/foo/.snapshot/bestsnapever/bar > /share/home/joeuser/.snapshot/bestsnapever/foo/bar > /share/home/.snapshot/bestsnapever/joeuser/foo/bar > /share/.snapshot/bestsnapever/home/joeuser/foo/barRepeating my previous question in another way... So how do they handle "mv home/joeuser home/moeuser" ? Does that mv delete all snapshots below home/joeuser? To make this work in ZFS, does this require that the mv(1) command only work when the user has snapshot delete privilege? I fear that nothing in this thread is moving the problem closer to RFE status :-( -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
On Wed, Apr 21, 2010 at 10:13 PM, Richard Elling <richard.elling at gmail.com> wrote:> Repeating my previous question in another way... > So how do they handle "mv home/joeuser home/moeuser" ? > Does that mv delete all snapshots below home/joeuser?If you wanted to go into home/joeuser/.snapshot , I think you''d have to look at home/.snapshot/joeuser. I think the way the .snapshot dir works is similar to if the user looks at $VOL_ROOT/home/user1/files/.snapshot, the directory is "magically" directed to $VOL_ROOT/.snapshot/home/user1/files> To make this work in ZFS, does this require that the mv(1) > command only work when the user has snapshot delete privilege?No, because the snapshots don''t exist for each directory. The path is internally redirected to the volume''s snapshot list, but starting at the current directory.> I fear that nothing in this thread is moving the problem closer to > RFE status :-(That''s a shame, the "secret" .snapshot directories are really nice to have. If FUSE was ready, it''d be trivial to work up a POC on top of zfs. -B -- Brandon High : bhigh at freaks.com
On 22/04/2010 00:14, Jason King wrote:> It still has the issue that the end user has to know where the root of > the filesystem is in the tree (assuming it''s even accessible on the > system -- might not be for an NFS mount).For CIFS ZFS provides the Volume Shadow Service (Previous Versions in Windows Explorer). See this blog entry for pictures of how this looks to a Windows user: http://blogs.sun.com/amw/entry/using_the_previous_versions_tab For (local) OpenSolaris clients the Nautilus file browser works "as if" the snapshots were visible in each directory - click the little clock icon that is between refresh and home. This only works locally though not over NFS. -- Darren J Moffat
> From: Richard Elling [mailto:richard.elling at gmail.com] > > Repeating my previous question in another way... > So how do they handle "mv home/joeuser home/moeuser" ? > Does that mv delete all snapshots below home/joeuser? > To make this work in ZFS, does this require that the mv(1) > command only work when the user has snapshot delete privilege? > > I fear that nothing in this thread is moving the problem closer to > RFE status :-(It''s not a real directory. Just like the .zfs directory, it is magically accessible in every directory or subdirectory, without any need to mkdir or anything. Whenever you mv some directory to a new name, there''s still a magical .snapshot directory inside of it, but all the contents are magically generated upon access, so the new .snapshot will reference the new directory name. It''s all just a "frontend" in the filesystem. You do something like "cd .zfs" or "cd .snapshot" (or ls, or cp, or whatever) and the filesystem responds as if that were a real directory. But there are no *actual* contents of any actual directory of that name. The filesystem just generates a response for you which looks like subdirectories, but are really links to the appropriate stuff, and makes it simply look as if it were normal directories. To move closer to RFE status ... I think the description would have to be written in verbage pertaining to zfs which is more than I know. I can describe how they each work, but I can''t make it technical enough to be an RFE for zfs.
On Thu, 22 Apr 2010, Edward Ned Harvey wrote:> > To move closer to RFE status ... I think the description would have to be > written in verbage pertaining to zfs which is more than I know. I can > describe how they each work, but I can''t make it technical enough to be an > RFE for zfs.Someone would also need to verify that this feature is not protected by a patent. Bob -- Bob Friesenhahn bfriesen at simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/ GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
On Apr 22, 2010, at 4:50 AM, Edward Ned Harvey wrote:>> From: Richard Elling [mailto:richard.elling at gmail.com] >> >> Repeating my previous question in another way... >> So how do they handle "mv home/joeuser home/moeuser" ? >> Does that mv delete all snapshots below home/joeuser? >> To make this work in ZFS, does this require that the mv(1) >> command only work when the user has snapshot delete privilege? >> >> I fear that nothing in this thread is moving the problem closer to >> RFE status :-( > > It''s not a real directory. Just like the .zfs directory, it is magically > accessible in every directory or subdirectory, without any need to mkdir or > anything. Whenever you mv some directory to a new name, there''s still a > magical .snapshot directory inside of it, but all the contents are magically > generated upon access, so the new .snapshot will reference the new directory > name. > > It''s all just a "frontend" in the filesystem. You do something like "cd > .zfs" or "cd .snapshot" (or ls, or cp, or whatever) and the filesystem > responds as if that were a real directory. But there are no *actual* > contents of any actual directory of that name. The filesystem just > generates a response for you which looks like subdirectories, but are really > links to the appropriate stuff, and makes it simply look as if it were > normal directories.One last try. If you change the "real" directory structure, how are those changes reflected in the "snapshot" directory structure? Consider: echo "whee" > /a/b/c/d.txt [snapshot] mv /a/b /a/B What does /a/B/c/.snapshot point to? If the answer is "nothing," then I see significantly less value in the feature. IIRC, POSIX does not permit hard links to directories. Moving or renaming the directory structure gets disconnected from the original because these are relative relationships. Clearly, NetApp achieves this in some manner which is not constrained by POSIX -- a manner which appears to be beyond your ability to describe.> To move closer to RFE status ... I think the description would have to be > written in verbage pertaining to zfs which is more than I know. I can > describe how they each work, but I can''t make it technical enough to be an > RFE for zfs.I''m not disputing the value here, but the RFE may need something other than ZPL. Today, NetApp might enjoy temporary advantage because their file system does not have to be POSIX compliant and they limit the total number of snapshots. OTOH, the only barrier for someone writing a non-POSIX file system layer on ZFS is resources and enthusiasm (unlimited snapshots are already available on ZFS). -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
Richard Elling <richard.elling at gmail.com> wrote:> IIRC, POSIX does not permit hard links to directories. Moving or renaming > the directory structure gets disconnected from the original because these > are relative relationships. Clearly, NetApp achieves this in some manner > which is not constrained by POSIX -- a manner which appears to be beyond > your ability to describe.I have recently frequently seen the name "POSIX" and I am not sure whether the people who used the name know what they are talking about. POSIX e.g. of course permits harlinks to directories. If you like a fact based discussion, I in general would like to a verification of the related claim as this allows to prove or disprove a claim. There e.g. never has been an explanation on why a special directory like .zfs or whatever should not be allowed by POSIX. Let me make an example..... In Sommer 2001, a person from Microsoft asked the Microsoft notation of named streams (filename:streamname) to be introduced into POSIX. It was easy to explain him that this would not be POSIX compliant as it would introduce another forbidden character ('':'') for filenames. Currently only ''/'' and ''\0'' are discallowed in filenames. He later asked to introduce a special directory "..." that could hold the named streams for files. This is also non POSIX compliant as it would require to modify all implementationd for find(1), ls(1), tar(1) and similar to know about the secific meaning of "...". As a result, Sun did introduce the "attibute directory", runat(1), openat(2) and similar around August 2001 in order to preove that there is a POSIX compliant way to implement named streams in files. If we could have a discussion at a similar level, I would be happy to help with the discussion. J?rg -- EMail:joerg at schily.isdn.cs.tu-berlin.de (home) J?rg Schilling D-13353 Berlin js at cs.tu-berlin.de (uni) joerg.schilling at fokus.fraunhofer.de (work) Blog: http://schily.blogspot.com/ URL: http://cdrecord.berlios.de/private/ ftp://ftp.berlios.de/pub/schily
On Wed, Apr 21, 2010 at 10:10:09PM -0400, Edward Ned Harvey wrote:> > From: Nicolas Williams [mailto:Nicolas.Williams at oracle.com] > > > > POSIX doesn''t allow us to have special dot files/directories outside > > filesystem root directories. > > So? Tell it to Netapp. They don''t seem to have any problem with it.And while it''s on by default, there is certainly an option to remove it. -- Darren
On Wed, Apr 21, 2010 at 04:49:30PM +0100, Darren J Moffat wrote:> /foo is the filesystem > /foo/bar is a directory in the filesystem > > cd /foo/bar/ > touch stuff > > [ you wait, time passes; a snapshot is taken ] > > At this point /foo/bar/.snapshot/.../stuff exists > > Now do this: > > rm -rf /foo/bar > > There is a snapshot of /foo/bar/stuff in the ZFS model to get to it > you go to /foo/.zfs/snapshot/<name>/bar and in there you will find > the file called stuff.Same thing on a netapp except for the name of the virtual directory.> How do you find what was /foo/bar/stuff in the model where the > .snapshot directory exists at every subdir rather than just at the > filesystem root when the subdirs have been removed ?The .snapshot directory still exists at the filesystem root. It''s not a replacement for that. Asking for the contents of /a/b/c/d/e/.snapshot gives you a view as if you had asked for /a/.snapshot/b/c/d/e (assuming /a is the filesystem). The benefits arise when the filesystem root is not mounted, or you don''t have access to it, or you don''t know where it is, and the hierarchy isn''t under constant change (which is true any place that my users care about).> What does it look like when the directory hierarchy is really deep ?Same thing that .zfs/<snapshot>/a/b/c/d/e/f/g/h looks like, but you can enter the snapshot tree at any directory that exists in the live filesystem as well as from the top. -- Darren
> From: Richard Elling [mailto:richard.elling at gmail.com] > > One last try. If you change the "real" directory structure, how are > those > changes reflected in the "snapshot" directory structure? > > Consider: > echo "whee" > /a/b/c/d.txt > [snapshot] > mv /a/b /a/B > > What does /a/B/c/.snapshot point to? If the answer is "nothing," then > I see > significantly less value in the feature.Actually, I find this very surprising: Question posted: http://lopsa.org/pipermail/tech/2010-April/004356.html Answered: http://lopsa.org/pipermail/tech/2010-April/004358.html My comments about it: http://lopsa.org/pipermail/tech/2010-April/004361.html and http://lopsa.org/pipermail/tech/2010-April/004362.html So apparently, the netapp snapshot is more directory based, and not so much filesystem based. Very surprising, to me. But this is good, because if implemented in ZFS, it leaves room for improvement: There''s no reason why the snapshots under "a" couldn''t be identical to the snapshots under "a/e" The only difficulty is for the filesystem to recognize a "mv" command as such, and to link things appropriately behind the scenes. In this case, "mv" is different from "cp ; rm" which was a long-time problem of svn, analogous to this one in the filesystem. But eventually, svn got it right, and now recognizes "mv" should preserve the history for items after they''re renamed. IMHO, the (ideal) desired behavior is *neither* what ZFS currently does, or what netapp currently does. IMHO, the ideal desired behavior is as described in that last email above. Namely: * Let the snap of the parent directory remain the same as it was at the time of the snapshot, And * Let the snapshots of the renamed subdirectory go along with the new name of the subdirectory. Anyway, it''s all pointless at this moment. Because obviously that''s all nontrivial, and very diminishing returns. So I maintain very near zero hope that it''ll ever happen.
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Edward Ned Harvey > > Actually, I find this very surprising: > Question posted: > http://lopsa.org/pipermail/tech/2010-April/004356.htmlAs the thread unfolds, it appears, although netapp may sometimes have some problems with "mv" directories ... This is evidence that appears to be weakening ... Sometimes they do precisely what you would want them to do. Which is, to have the .snapshot directory available under every subdirectory, and the contents are the best you could hope for that location. Meaning, the parent directory .snapshot contains an image of the way the filesystem looked at the time of the snapshot, but the snapshots of the children, even after the children were renamed, preserve the history of where the children came from (before they were renamed via "mv"). Still waiting to know more; maybe the bad behavior was a bug that was fixed in some release. But to anyone who may have read that thread already: there are conflicting results coming from different people now. Meaning, 3 data points, with 2 good and 1 bad.
On Fri, Apr 23, 2010 at 7:17 PM, Edward Ned Harvey <solaris2 at nedharvey.com> wrote:> As the thread unfolds, it appears, although netapp may sometimes have some > problems with "mv" directories ... This is evidence that appears to be > weakening ... Sometimes they do precisely what you would want them to do.Richard and I conversed off list and I gave him the output of shapshot listings after some renames. To answer the question you linked to: .shapshot/snapname.0/a/b/c/d.txt from the top of the filesystem a/.snapshot/snapname.0/b/c/d.txt a/e/.shapshot/snapname.0/c/d.txt a/e/c/.snapshot/snapname.0/d.txt -B -- Brandon High : bhigh at freaks.com
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Edward Ned Harvey > > > From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > > bounces at opensolaris.org] On Behalf Of Edward Ned Harvey > > > > Actually, I find this very surprising: > > Question posted: > > http://lopsa.org/pipermail/tech/2010-April/004356.html > > As the thread unfolds, it appears, although netapp may sometimes have > some > problems with "mv" directories ... This is evidence that appears to be > weakening ...Nope. That discussion seems to be concluded now. And the netapp does not have the problem that was suspected. The .snapshot directories do precisely what you would want them to do. Which is: The .snapshot directory belonging to a parent contains a copy of the filesystem as it looked at the time of the snapshot. But when you mv or rename a subdirectory, then the .snapshot subdir of the subdirectory correctly maps, to preserve the snapshots inside that directory. Make sense?
On Apr 24, 2010, at 5:27 AM, Edward Ned Harvey wrote:>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- >> bounces at opensolaris.org] On Behalf Of Edward Ned Harvey >> >>> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- >>> bounces at opensolaris.org] On Behalf Of Edward Ned Harvey >>> >>> Actually, I find this very surprising: >>> Question posted: >>> http://lopsa.org/pipermail/tech/2010-April/004356.html >> >> As the thread unfolds, it appears, although netapp may sometimes have >> some >> problems with "mv" directories ... This is evidence that appears to be >> weakening ... > > Nope. That discussion seems to be concluded now. And the netapp does not > have the problem that was suspected.I do not recall reaching that conclusion. I think the definition of the problem is what you continue to miss.> The .snapshot directories do precisely what you would want them to do. > Which is: The .snapshot directory belonging to a parent contains a copy of > the filesystem as it looked at the time of the snapshot. But when you mv or > rename a subdirectory, then the .snapshot subdir of the subdirectory > correctly maps, to preserve the snapshots inside that directory.Agree, but the path is lost.> Make sense?Yes. The contents of the directory are there, but the full pathname does not follow because you changed a name in the path. Whether this is useful or not depends on whether you expect the path to be followed. It seems, the NetApp snapshot is a directory-level snapshot rather than a file system snapshot. I cannot see how to merge the two, so perhaps adding such a feature to ZFS could not leverage the file system snapshot? -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
On 24 apr 2010, at 16.43, Richard Elling wrote:> I do not recall reaching that conclusion. I think the definition of the problem > is what you continue to miss.Me too then, I think. Can you please enlighten us about the definition of the problem?>> The .snapshot directories do precisely what you would want them to do. >> Which is: The .snapshot directory belonging to a parent contains a copy of >> the filesystem as it looked at the time of the snapshot. But when you mv or >> rename a subdirectory, then the .snapshot subdir of the subdirectory >> correctly maps, to preserve the snapshots inside that directory. > > Agree, but the path is lost.In what way? On 24 apr 2010, at 09.18, Brandon High wrote:> To answer the question you linked to: > .shapshot/snapname.0/a/b/c/d.txt from the top of the filesystem > a/.snapshot/snapname.0/b/c/d.txt > a/e/.shapshot/snapname.0/c/d.txt > a/e/c/.snapshot/snapname.0/d.txtI really don''t understand what you mean, I think the path is there just fine, and IMHO pretty much in the way you would expect.>> Make sense? > > Yes. The contents of the directory are there, but the full pathname does > not follow because you changed a name in the path. Whether this is > useful or not depends on whether you expect the path to be followed. > > It seems, the NetApp snapshot is a directory-level snapshot rather than a > file system snapshot. I cannot see how to merge the two, so perhaps > adding such a feature to ZFS could not leverage the file system snapshot?I believe NetApp snapshots are volume based in a fashion very much like zfs volume snapshots. /ragge
On Sat, Apr 24, 2010 at 2:17 PM, Ragnar Sundblad <ragge at csc.kth.se> wrote:> > On 24 apr 2010, at 16.43, Richard Elling wrote: > > On 24 apr 2010, at 09.18, Brandon High wrote: > > To answer the question you linked to: > > .shapshot/snapname.0/a/b/c/d.txt from the top of the filesystem > > a/.snapshot/snapname.0/b/c/d.txt > > a/e/.shapshot/snapname.0/c/d.txt > > a/e/c/.snapshot/snapname.0/d.txt > > I really don''t understand what you mean, I think the path is there > just fine, and IMHO pretty much in the way you would expect. > > >> Make sense? > > > > Yes. The contents of the directory are there, but the full pathname does > > not follow because you changed a name in the path. Whether this is > > useful or not depends on whether you expect the path to be followed. > > > > It seems, the NetApp snapshot is a directory-level snapshot rather than a > > file system snapshot. I cannot see how to merge the two, so perhaps > > adding such a feature to ZFS could not leverage the file system snapshot? > > I believe NetApp snapshots are volume based in a fashion very much > like zfs volume snapshots. >>From the sounds of things, it would appear that NetApp snapshots happen atthe volume/filesystem level. However, they also provide a "shortcut" .snapshot/ directory in every directory, allowing you to access that part of the tree directly. For example, with a filesystem root of /root: /root/branch/leaf/file.txt Snapshots are taken. You can either file.txt via any of the following: /root/.snapshot/branch/leaf/file.txt /root/branch/.snapshot/leaf/file.txt /root/branch/leaf/.snapshot/file.txt Wheras with ZFS you can only access file.txt via: /root/.zfs/snapshot/snapname/branch/leaf/file.txt>From the sounds of it, the .snapshot directory is just a pointer to thecorresponding directory in the actual snapshot tree. The snapshots are not actually saved per-directory. They just give you a handy shortcut into that same level in the snapshot directory tree. Something like this would be very useful in ZFS, especially if you have deep directory trees in a single ZFS filesystem, and you want to compare files/directories in multiple snapshots. For example, our backups server has /storage/backup/.zfs/snapshot/snap-date/sitename/servername/home/u/username/yadda-yadda Which leads to very long cd commands to move between snapshots. Being able to just "cd .snapshot/snapname/" at arbitrary points in the tree, to access that part of the tree in another snapshot, would be handy. However, I''ve never actually used a NetApp system, so the above could be totally out in left field. It''s just what I gathered from this discussion. -- Freddie Cash fjwcash at gmail.com -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20100424/ab15d7d1/attachment.html>
On Apr 24, 2010, at 2:17 PM, Ragnar Sundblad wrote:> On 24 apr 2010, at 16.43, Richard Elling wrote: > >> I do not recall reaching that conclusion. I think the definition of the problem >> is what you continue to miss. > > Me too then, I think. Can you please enlighten us about the > definition of the problem?Last chance.>>> The .snapshot directories do precisely what you would want them to do. >>> Which is: The .snapshot directory belonging to a parent contains a copy of >>> the filesystem as it looked at the time of the snapshot. But when you mv or >>> rename a subdirectory, then the .snapshot subdir of the subdirectory >>> correctly maps, to preserve the snapshots inside that directory. >> >> Agree, but the path is lost. > > In what way? > > On 24 apr 2010, at 09.18, Brandon High wrote: >> To answer the question you linked to: >> .shapshot/snapname.0/a/b/c/d.txt from the top of the filesystem >> a/.snapshot/snapname.0/b/c/d.txt >> a/e/.shapshot/snapname.0/c/d.txt >> a/e/c/.snapshot/snapname.0/d.txt > > I really don''t understand what you mean, I think the path is there > just fine, and IMHO pretty much in the way you would expect.Next, mv /a/e /a/E ls -l a/e/.snapshot/snaptime ENOENT? ls -l a/E/.snapshot/snapname/d.txt this should be ENOENT because d.txt did not exist in a/E at snaptime. -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
> From: Richard Elling [mailto:richard.elling at gmail.com] > Sent: Saturday, April 24, 2010 10:43 AM > > > Nope. That discussion seems to be concluded now. And the netapp > does not > > have the problem that was suspected. > > I do not recall reaching that conclusion. I think the definition of > the problem > is what you continue to miss. > > > The .snapshot directories do precisely what you would want them to > do. > > Which is: The .snapshot directory belonging to a parent contains a > copy of > > the filesystem as it looked at the time of the snapshot. But when > you mv or > > rename a subdirectory, then the .snapshot subdir of the subdirectory > > correctly maps, to preserve the snapshots inside that directory. > > Agree, but the path is lost.In the original test, done by Jonathan, it seemed that way. But after two other people responded with their own repeats of that test, which concluded with no problems, Jonathan retried his, and found that he just needs to repeat the same "ls" command twice to get the proper result on the 2nd try. He is running a very old version of Ontap, so most likely he''s just seeing the result of some bug. But others don''t have that issue. The path is not lost. The following all work just fine: a/.snapshot/vol0_deleteme/b/c/d.txt or a/e/.snapshot/vol0_deleteme/c/d.txt a/e/c/.snapshot/vol0_deleteme/d.txt> It seems, the NetApp snapshot is a directory-level snapshot rather than > a > file system snapshot. I cannot see how to merge the two, so perhaps > adding such a feature to ZFS could not leverage the file system > snapshot?Again, in Jonathan''s original post, that''s what it seemed like. But after Adam and Tom replied showing they didn''t have the same behavior ... Jonathan reran his test, and got a different result. Which concluded: The .snapshot directory does indeed maintain a filesystem-level snapshot of the whole filesystem, but it also provides an easy-to-access interface within any subdirectory, even after renaming a directory. It works. Although Jonathan apparently saw a problem in an old version of the OS.
> From: Ragnar Sundblad [mailto:ragge at csc.kth.se] > Sent: Saturday, April 24, 2010 5:18 PM > > > To answer the question you linked to: > > .shapshot/snapname.0/a/b/c/d.txt from the top of the filesystem > > a/.snapshot/snapname.0/b/c/d.txt > > a/e/.shapshot/snapname.0/c/d.txt > > a/e/c/.snapshot/snapname.0/d.txt > > I really don''t understand what you mean, I think the path is there > just fine, and IMHO pretty much in the way you would expect.Precisely. The snapshot exists, with the original name, from the higher level directory. But even after renaming, you go into the subdirectory with the new name, and you can still find the snapshots there. Very convenient.
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Freddie Cash > > From the sounds of it, the .snapshot directory is just a pointer to the > corresponding directory in the actual snapshot tree. The snapshots are > not actually saved per-directory. They just give you a handy shortcut > into that same level in the snapshot directory tree.Precisely correct.> Something like this would be very useful in ZFS, especially if you have > deep directory trees in a single ZFS filesystem, and you want to > compare files/directories in multiple snapshots. For example, our > backups server hasEspecially what you said. But especially *especially* the following: If you have a zfs filesystem /exports, and a nested filesystem /exports/home, and another one /exports/home/jbond ... If you sometimes do your work in /exports/home/jbond/important/files/and/documents ... And you sometimes do your work in /exports/mi6/007/secret/directory ... Then you sometimes need to access snapshots under /exports/.zfs and sometimes under /exports/home/jbond/.zfs It''s really nice if you don''t have to figure out how far up the tree to go, in order to find the correct .zfs directory for what you want to find right now.
> From: Richard Elling [mailto:richard.elling at gmail.com] > Sent: Saturday, April 24, 2010 7:42 PM > > Next, > mv /a/e /a/E > ls -l a/e/.snapshot/snaptime > > ENOENT? > > ls -l a/E/.snapshot/snapname/d.txt > > this should be ENOENT because d.txt did not exist in a/E at snaptime.Incorrect. E did exist. Inode 12345 existed, but it had a different name at the time of snapshot. Therefore, a/e/.snapshot/snapname/c/d.txt is the file at the time of snapshot. But these are also the same thing: a/E/.snapshot/snapname/c/d.txt a/E/c/.snapshot/snapname/d.txt It would be very annoying if you could have a directory named "foo" which contains all the snapshots for its own history, and then "mv foo bar" and suddenly the snapshots all disappear. This is not the behavior. The behavior is: If you "mv foo bar" then the snapshots which were previously accessible under foo are now accessible under bar. However, if you look in the snapshot of foo''s parent, then you will see "foo" and not "bar". Just the way it would have looked, at the time of the snapshot.
On Apr 25, 2010, at 5:45 AM, Edward Ned Harvey wrote:>> From: Richard Elling [mailto:richard.elling at gmail.com] >> Sent: Saturday, April 24, 2010 7:42 PM >> >> Next, >> mv /a/e /a/E >> ls -l a/e/.snapshot/snaptime >> >> ENOENT? >> >> ls -l a/E/.snapshot/snapname/d.txt >> >> this should be ENOENT because d.txt did not exist in a/E at snaptime. > > Incorrect. > > E did exist. Inode 12345 existed, but it had a different name at the time > of snapshot. Therefore, > a/e/.snapshot/snapname/c/d.txt is the file at the time of snapshot. > But these are also the same thing: > a/E/.snapshot/snapname/c/d.txt > a/E/c/.snapshot/snapname/d.txtOK, I''ll believe you. How about this? mv a/E/c a/c mv a/E a/c mv a/c a/E now a/E/.snapshot/snapname/c/d.txt is ENOENT, correct?> It would be very annoying if you could have a directory named "foo" which > contains all the snapshots for its own history, and then "mv foo bar" and > suddenly the snapshots all disappear. This is not the behavior. > > The behavior is: If you "mv foo bar" then the snapshots which were > previously accessible under foo are now accessible under bar. However, if > you look in the snapshot of foo''s parent, then you will see "foo" and not > "bar". Just the way it would have looked, at the time of the snapshot. >The only way I know to describe this is that the path is lost. In other words, you cannot say ../.snapshot/snapname/self is the same as self/.snapshot/snapname, thus the relationship previously described as: Snapshots are taken. You can either file.txt via any of the following: /root/.snapshot/branch/leaf/file.txt /root/branch/.snapshot/leaf/file.txt /root/branch/leaf/.snapshot/file.txt is not guaranteed to be correct. -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
> From: Richard Elling [mailto:richard.elling at gmail.com] > Sent: Sunday, April 25, 2010 2:12 PM > > > E did exist. Inode 12345 existed, but it had a different name at the > time > > OK, I''ll believe you. > > How about this? > > mv a/E/c a/c > mv a/E a/c > mv a/c a/EThe thing that''s still confusing you is the idea that directory names or locations matter. They don''t. Remember that a directory is just an inode, with text and data inside it, which stores an association of child names and child inode numbers. Suppose "somedir" is inode 12345. Then if you "ls somedir/.snapshot/somesnap" then the system is reading a version of inode 12345 in a time gone by. At that time, inode 12345 may have been referenced by its parent using the name "foo" instead of "somedir" but that won''t even matter in this case because we''ve only instructed the system to read the contents of a past version of inode 12345. In this case, we haven''t told the system to do anything even slightly related to any parent of that inode. We''re not even going to know what name was associated with inode 12345 at that time. At the time of somesnap, inode 12345 had contents which indicate "a.txt" is inode 1000 and "b.txt" is inode 1050 and so on. So "a.txt" and "b.txt" will appear in the directory listing, and if you cat a.txt or b.txt, the system will fetch inode 1000 or 1050 as it appeared at the time of the snapshot. Does that help? There is no actual entity called ".snapshot" It''s a magical thing, just like there is no actual entity called ".zfs" If you "ls somedir" or "ls somezfsfilesystem" you will see, that the parent inode does not contain any reference to anything called ".snapshot" or ".zfs" (Unless you turned it on for some reason.) However, if you "cd .snapshot" or "cd .zfs" then there''s some magic behind the scenes that''s able to handle that differently. I don''t know how they do that. But I do know it''s not listed in the inode like any other normal child subdirectory or file.
On Apr 26, 2010, at 5:02 AM, Edward Ned Harvey wrote:>> From: Richard Elling [mailto:richard.elling at gmail.com] >> Sent: Sunday, April 25, 2010 2:12 PM >> >>> E did exist. Inode 12345 existed, but it had a different name at the >> time >> >> OK, I''ll believe you. >> >> How about this? >> >> mv a/E/c a/c >> mv a/E a/c >> mv a/c a/E > > The thing that''s still confusing you is the idea that directory names or > locations matter. They don''t.Maybe directory consistency doesn''t matter for MS-DOS 1.0, but I''m pretty sure that directory consistency is useful in UNIX.> Remember that a directory is just an inode, with text and data inside it, > which stores an association of child names and child inode numbers. Suppose > "somedir" is inode 12345. Then if you "ls somedir/.snapshot/somesnap" then > the system is reading a version of inode 12345 in a time gone by. At that > time, inode 12345 may have been referenced by its parent using the name > "foo" instead of "somedir" but that won''t even matter in this case because > we''ve only instructed the system to read the contents of a past version of > inode 12345. In this case, we haven''t told the system to do anything even > slightly related to any parent of that inode. We''re not even going to know > what name was associated with inode 12345 at that time. > > At the time of somesnap, inode 12345 had contents which indicate "a.txt" is > inode 1000 and "b.txt" is inode 1050 and so on. So "a.txt" and "b.txt" will > appear in the directory listing, and if you cat a.txt or b.txt, the system > will fetch inode 1000 or 1050 as it appeared at the time of the snapshot. > > Does that help?I completely understand this. No magic here.> There is no actual entity called ".snapshot" It''s a magical thing, just > like there is no actual entity called ".zfs" If you "ls somedir" or "ls > somezfsfilesystem" you will see, that the parent inode does not contain any > reference to anything called ".snapshot" or ".zfs" (Unless you turned it > on for some reason.)Yes. And you agree that the relationship to parent directories does not matter, correct? In other words, a tool that looks at either the parent or child snapshot directories is useless. Put another way, you cannot implement something like time machine using directory-level snapshot subdirectories.> However, if you "cd .snapshot" or "cd .zfs" then there''s some magic behind > the scenes that''s able to handle that differently. I don''t know how they do > that. But I do know it''s not listed in the inode like any other normal > child subdirectory or file.''nuff said -- richard ZFS storage and performance consulting at http://www.RichardElling.com ZFS training on deduplication, NexentaStor, and NAS performance Las Vegas, April 29-30, 2010 http://nexenta-vegas.eventbrite.com
On 25 apr 2010, at 20.12, Richard Elling wrote:> On Apr 25, 2010, at 5:45 AM, Edward Ned Harvey wrote: > >>> From: Richard Elling [mailto:richard.elling at gmail.com] >>> Sent: Saturday, April 24, 2010 7:42 PM >>> >>> Next, >>> mv /a/e /a/E >>> ls -l a/e/.snapshot/snaptime >>> >>> ENOENT? >>> >>> ls -l a/E/.snapshot/snapname/d.txt >>> >>> this should be ENOENT because d.txt did not exist in a/E at snaptime. >> >> Incorrect. >> >> E did exist. Inode 12345 existed, but it had a different name at the time >> of snapshot. Therefore, >> a/e/.snapshot/snapname/c/d.txt is the file at the time of snapshot. >> But these are also the same thing: >> a/E/.snapshot/snapname/c/d.txt >> a/E/c/.snapshot/snapname/d.txt > > OK, I''ll believe you. > > How about this? > > mv a/E/c a/c > mv a/E a/c > mv a/c a/E > > now a/E/.snapshot/snapname/c/d.txt is ENOENT, correct?Sadly I can''t test it myself right now, maybe someone else can, but I''d except: [start: we have a file: a/E/c/d.txt] [snap1] mv a/E/c a/c [snap2] mv a/E a/c mv a/c a/E would result in: a/.snapshot/snap1/E/c/d.txt a/.snapshot/snap2/E/ (empty) a/.snapshot/snap2/c/d.txt a/E/.snapshot/snap1/c/d.txt a/E/.snapshot/snap2/ (empty) a/E/ (empty) Wouldn''t that be logical, and what would be the problem?>> It would be very annoying if you could have a directory named "foo" which >> contains all the snapshots for its own history, and then "mv foo bar" and >> suddenly the snapshots all disappear. This is not the behavior. >> >> The behavior is: If you "mv foo bar" then the snapshots which were >> previously accessible under foo are now accessible under bar. However, if >> you look in the snapshot of foo''s parent, then you will see "foo" and not >> "bar". Just the way it would have looked, at the time of the snapshot. >> > > The only way I know to describe this is that the path is lost. > In other words, you cannot say ../.snapshot/snapname/self is the same as > self/.snapshot/snapname, thus the relationship previously described as: > > Snapshots are taken. You can either file.txt via any of the following: > /root/.snapshot/branch/leaf/file.txt > /root/branch/.snapshot/leaf/file.txt > /root/branch/leaf/.snapshot/file.txt > > is not guaranteed to be correct.No, not if the hierarchy is changed between the snapshots, I think it was just a way to illustrate how the .snapshot directories work. It isn''t in zfs either, if the example above would be a zfs, we would have: a/.zfs/snapshot/snap1/E/c/d.txt a/.zfs/snapshot/snap2/c/d.txt a/E/ (empty) I still don''t understand why the OnTap model is losing more paths than zfs. I''d be happy if you could take one more shot at explaining. /ragge
> From: zfs-discuss-bounces at opensolaris.org [mailto:zfs-discuss- > bounces at opensolaris.org] On Behalf Of Edward Ned Harvey > > If something like this already exists, please let me know. Otherwise, > I > plan to: > > Create "zfshistory" command, written in python. (open source, public, > free)So, I decided to rename this "zhist" and started a project on google code. I''m not very far along yet, except, based on all the discussion in this thread, have a very good idea how it should all be implemented. Particular thanks to Richard Elling, whose in-depth discussion of path renames and moves made me think a lot about implementation, and settle on inode tracking. If anyone would like to contribute, please let me know off-list.