Hi! I am new to zfs / opensolaris but getting the hang of it. :) In my setup i have a large zpool called ''backup'' which has zfs''es for servers that i backup with rsync. I export those zfs''es via NFS to the remote servers so i have easy access to the most recent backup and the snapshots which i make just before starting the rsync. Recently i switched on ''snapdir=visible'' on one of the zfs volumes to easily expose the available snapshots and then i noticed rsync -removes- snapshots even though i am not able to do so myself, even as root, with plain /bin/rm. Example: | root at host:~# zfs snapshot backup/remotehost at 2009-10-16 | root at host:~# ls /backup/remotehost/.zfs/snapshot/ | 2009-10-16 | | root at host:~# /usr/bin/rsync --archive --delete user at remotehost:/ /backup/remotehost | | [ .. lots of read-only file system errors skipped .. ] | | rsync: delete_file: unlink(.zfs/snapshot/2009-10-16/.history) failed: Read-only file system (30) | cannot delete non-empty directory: .zfs/snapshot/2009-10-16 | deleting .zfs/snapshot/2009-10-16/ | cannot delete non-empty directory: .zfs/snapshot | rsync: delete_file: rmdir(.zfs/snapshot) failed: Operation not applicable (89) | rsync: delete_file: rmdir(.zfs/shares) failed: Operation not applicable (89) | cannot delete non-empty directory: .zfs | | root at host:~# ls -la /backup/remotehost/.zfs/snapshot/ | total 0 | dr-xr-xr-x 2 root root 2 2009-10-05 09:32 . | dr-xr-xr-x 4 root root 4 2009-10-05 09:32 .. | | root at host:~# zfs snapshot backup/remotehost at 2009-10-16 | root at host:~# ls /backup/remotehost/.zfs/snapshot/ | 2009-10-16 What''s the deal here. :) I was under the impression that rsync uses unlink() like ''rm'' does? How come rsync -can- remove the snapshot directory when exposed with ''snapdir=visible'' and i can''t do it with /bin/rm? Anyone else seeing this? | root at host:~# rsync --version | rsync version 3.0.6 protocol version 30 | Copyright (C) 1996-2009 by Andrew Tridgell, Wayne Davison, and others. | Web site: http://rsync.samba.org/ | Capabilities: | 64-bit files, 64-bit inums, 32-bit timestamps, 64-bit long ints, | socketpairs, hardlinks, symlinks, no IPv6, batchfiles, inplace, | append, ACLs, no xattrs, iconv, no symtimes | | root at host:~# uname -a | SunOS host 5.11 snv_111b i86pc i386 i86pc Solaris With regards, -Sander. -- | Daylight savings time - why are they saving it and where do they keep it? | 4096R/6D40 - 1A20 B9AA 87D4 84C7 FBD6 F3A9 9442 20CC 6CD2
On Fri, Oct 16, 2009 at 01:42:49PM +0200, Sander Smeenk wrote:> Recently i switched on ''snapdir=visible'' on one of the zfs volumes to > easily expose the available snapshots and then i noticed rsync -removes- > snapshots even though i am not able to do so myself, even as root, with > plain /bin/rm.I never liked this interface. I want snapshots to be immutable to operations within the filesystem itself.> What''s the deal here. :) > > I was under the impression that rsync uses unlink() like ''rm'' does? > How come rsync -can- remove the snapshot directory when exposed with > ''snapdir=visible'' and i can''t do it with /bin/rm?I think you mean ''rmdir'' rather than unlink. My assumption is that ''rm'' is being smart and won''t attempt to remove a directory that it thinks has data (which would normally fail). In looking at a ''truss'' of /bin/rm <snap>, it never issued an unlink or rmdir, just complained that the directory wasn''t empty. /bin/rmdir on the other hand did do a rmdir(2), and that does destroy the snapshot. Although I see sometimes the rmdir fails. If I''ve been doing "something" in a snapshot (sometimes some stats are enough), then it seems to "lock" the snapshot and the rmdir fails (with EBUSY). -- Darren
Quoting A Darren Dunham (ddunham at taos.com):> > i noticed rsync -removes- snapshots even though i am not able to do > > so myself, even as root, with plain /bin/rm. > I never liked this interface. I want snapshots to be immutable to > operations within the filesystem itself.Well, thats what i would expect too. It seems strange that you can''t edit or remove singular files from snapshots though you can rmdir acomplete snapshot ''directory'' at once. Is this by design? I call it a bug. ;)> /bin/rmdir on the other hand did do a rmdir(2), and that does destroy > the snapshot.It seems to me that the ''base directory'' or ''mountpoint'' of the snapshot, e.g. .zfs/snapshot/<name> is not immutable, as all files and directories inside the snapshot are: root at host:/backup/host/.zfs/snapshot/2009-10-14# rmdir bin rmdir: bin: Read-only file system> Although I see sometimes the rmdir fails. If I''ve been doing > "something" in a snapshot (sometimes some stats are enough), then it > seems to "lock" the snapshot and the rmdir fails (with EBUSY).Hmm. I haven''t noticed such behaviour myself. I guess i''m gonna switch off snapdir visibility while rsync runs, or something. Regards, -Sndr. -- | So this magician is walking down the street and turns into a grocery store. | 4096R/6D40 - 1A20 B9AA 87D4 84C7 FBD6 F3A9 9442 20CC 6CD2
On Sun, 2009-10-18 at 18:12 +0200, Sander Smeenk wrote:> Well, thats what i would expect too. It seems strange that you can''t > edit or remove singular files from snapshots [...]That would make the snapshot not a snapshot anymore. There would be differences..
Quoting dick hoogendijk (dick at nagual.nl):> > Well, thats what i would expect too. It seems strange that you can''t > > edit or remove singular files from snapshots [...] > That would make the snapshot not a snapshot anymore. There would be > differences..I''m well aware of that. You''re replying to just a part of my sentence. I tried to indicate that it''s strange that rmdir works on the snapshot directory while files inside snapshots are immutable. This, to me, is a bug. -Sndr. -- | Did you hear about the cat that ate a ball of wool? -- It got mittens. | 4096R/6D40 - 1A20 B9AA 87D4 84C7 FBD6 F3A9 9442 20CC 6CD2
On Oct 18, 2009, at 11:37 AM, Sander Smeenk wrote:> I tried to indicate that it''s strange that rmdir works on the snapshot > directory while files inside snapshots are immutable. > > This, to me, is a bug.If you have a snapshot named "pool at snap", this: # rmdir /pool/.zfs/snapshot/snap is equivalent to this: # zfs destroy pool at snap Similarly, this: # mkdir /pool/.zfs/snapshot/snap is equivalent to this: # zfs snapshot pool at snap This can be very handy if you want to create or destroy a snapshot from an NFS client, for example. -Chris
Quoting Chris Kirby (chris.kirby at sun.com):> If you have a snapshot named "pool at snap", this: > # rmdir /pool/.zfs/snapshot/snap > is equivalent to this: > # zfs destroy pool at snap > > Similarly, this: > # mkdir /pool/.zfs/snapshot/snap > is equivalent to this: > # zfs snapshot pool at snap > > This can be very handy if you want to create or destroy > a snapshot from an NFS client, for example.Oh, right. This is where my newbieness kicks in. I didn''t know just doing a mkdir in the snapshot directory actually -creates- a snapshot. Thanks for this clarification. I think i will toggle snapdir=visible/hidden to overcome this problem, or add it to --excludes perhaps. Thanks! -Sndr. -- | If you jog backwards, will you gain weight? | 4096R/6D40 - 1A20 B9AA 87D4 84C7 FBD6 F3A9 9442 20CC 6CD2
On Oct 18, 2009, at 11:23 AM, Sander Smeenk wrote:> Quoting Chris Kirby (chris.kirby at sun.com): > >> If you have a snapshot named "pool at snap", this: >> # rmdir /pool/.zfs/snapshot/snap >> is equivalent to this: >> # zfs destroy pool at snap >> >> Similarly, this: >> # mkdir /pool/.zfs/snapshot/snap >> is equivalent to this: >> # zfs snapshot pool at snap >> >> This can be very handy if you want to create or destroy >> a snapshot from an NFS client, for example. > > Oh, right. This is where my newbieness kicks in. I didn''t know just > doing a mkdir in the snapshot directory actually -creates- a snapshot. > Thanks for this clarification. > > I think i will toggle snapdir=visible/hidden to overcome this problem, > or add it to --excludes perhaps.Yes. This is why the snapshots are not visible by default... legacy backup software (eg tar) would want to back them up, which would be redundant, redundant. http://bugs.opensolaris.org/bugdatabase/view_bug.do?bug_id=6399899 -- richard