ZFS claims to provide transactional filesystems, filesystem snapshots, and nested filesystems. I supposed that this means that it provides nested transactional snapshots of nested filesystems. I was wrong: # mkfile 64m /storage # zpool create tank /storage # zfs create tank/level2 # touch /tank/level2/test # zfs snapshot tank/level2 at first # zfs snapshot tank at first # touch /tank/level2/test # ls -l /tank/level2 total 1 -rw------- 1 root root 0 Dec 13 15:37 test # ls -l /tank/level2/.zfs/snapshot/first total 1 -rw------- 1 root root 0 Dec 13 15:36 test # cp -r tank tank2 # ls -l tank2/level2 total 0 -rw------- 1 root root 0 Dec 13 15:37 test ***Ok, so far, so good (other than that odd "total 0" there near the end; is that a bug?) ***But now: # ls -l /tank/.zfs/snapshot/first/level2/test /tank/.zfs/snapshot/first/level2/test: No such file or directory # ls -l /tank/.zfs/snapshot/first/level2 total 0 ***Since a recursive copy (cp -r) of a filesystem includes all subfilesystems, and presumably tar does too, I expected that a snapshot of a filesystem would include all subfilesystems. But in fact, ZFS snapshots exclude the subfilesystems. Why? ***Anyway, continuing onward: # zfs rollback tank at first cannot unmount ''/tank'': Device busy # fuser tank tank: ***What''s up with that? It''s busy, yet nobody''s using it? ***Anyway, continuing onward: # zfs rollback -f tank at first # ls /tank/level2/test /tank/level2/test: No such file or directory # ls /tank level2 # ls -l /tank/level2 total 0 ***So rolling back the parent filesystem leaves the subfilesystems in place, but deletes their contents? WTF? ***Continuing onward: # ls /tank/level2/.zfs/snapshot/first /tank/level2/.zfs/snapshot/first: No such file or directory # ls /tank/level2/.zfs /tank/level2/.zfs: No such file or directory zfs list -o name,mountpoint NAME MOUNTPOINT tank /tank tank at first - tank/level2 /tank/level2 tank/level2 at first - # rollback tank/level2 at first rollback: not found ***So the filesystem tank/level2 still exists, and it''s still mounted, but not only did tank''s rollback delete tank/level2''s contents, it even deleted its .zfs directory! And now I can''t rollback to tank/level2 at first, even though ZFS says that that snapshot does still exist. (BTW I''m using SXCR b28.) The standard semantics of nested transactions is that rolling back any transaction automatically rolls back all nested transactions. I was assuming that snapshots work like transaction checkpoints, and rolling back would undo all changes made after the snapshot, _including_ all changes _and all snapshots_ made in all subfilesystems. It appears that ZFS in fact did this when it rolled back to tank at first (other than the odd bug of deleting the /tank/level2/.zfs directory), including deleting the tank/level2 at first snapshot (though it forgot to remove it from the list shown by "zfs list"), but it failed to roll tank back to the original state which tank was in when the tank at first snapshot was made because when that snapshot was made it failed to include the state of the tank/level2 subfilesystem. So, I have two questions for anybody who knows how ZFS is supposed to work: Is everything which ZFS did above exactly what it was supposed to do? And if so, then if snapshots of a filesystem are not supposed to include the state of subfilesystems, then why does rolling back those snapshots roll back the subfilesystems'' changes? It should either both include them and roll them back, or neither include them nor roll them back; doing one without doing the other is inconsistent. And since doing neither would defeat the ability to use snapshots to manipulate filesystems as complete units in the same way as cp -r and tar (thus leading to such annoyances as the need for scripts to do recursive snapshots (as other people have mentioned), with which it''s impossible to take atomic snapshots of a filesystem along with its subfilesystems), it seems that doing both is the only right thing to do. This message posted from opensolaris.org