Apologies if this has been addressed, but looking at some of the sun blogs and google searches I have not been able to find an answer. Does ZFS support on write automatic snapshots? For example, according to defined policy, every time a file is written a snapshot is created with the diff stored. I can see this being useful in high security environments and companies that have extreme regulatory requirements. If not, would there be a way besides scripts/programs to emulate this feature? Thank You, Alex
Alex Barclay wrote:> Apologies if this has been addressed, but looking at some of the sun > blogs and google searches I have not been able to find an answer. > > Does ZFS support on write automatic snapshots? > > For example, according to defined policy, every time a file is written > a snapshot is created with the diff stored. I can see this being > useful in high security environments and companies that have extreme > regulatory requirements.I don''t think you really want every write, do you ? Do you really want a snapshot taken very time some application does this: buf_size = 1 byte. while (some_condition) write(fd, buf, buf_size); What if you have 2 such apps or 10 or 100 ? Those snapshots wouldn''t really be that useful plus it would be very difficult to know which of them are actually useful - if any. What if the app is making the disk run at full speed 1 byte at time, thats a lot of snapshots in a second! I claim to be a security person and I don''t see how that amount of snapshots is actually good from a security view point. If you really do want to know about all the writes we can already do that using Solaris BSM Audit. I believe this is actually better info than taking snapshots since the audit trail will tell you exactly who did what to what where they came from and down to a nanosecond (IIRC) when it happened. Now if what you really mean is snapshot on file closure I think you might well be on to something useful. Whats more NTFS has some cool stuff in this area for consolidating identical files. The hooks that would need to be put into ZFS to do snapshot on file close could be used for other things like single instance storage (though isn''t that the opposite of ditto blocks on user data hmn whats the opposite of ditto :-)). > If not, would there be a way besides scripts/programs to emulate this feature? Why not scripts and programs how else would we do it ? You could do this with a very simple dtrace script that uses system() to do the zfs snapshot. You can also use dtrace to simulate the every single write case and see for yourself the massive explosion of snapshots that would occur as a result. -- Darren J Moffat
Darren, thank you for your reply. While it didn''t come out correctly (need to brush up on nomenclature), I did mean snapshot on closure.> Now if what you really mean is snapshot on file closure I think you > might well be on to something useful. Whats more NTFS has some cool > stuff in this area for consolidating identical files. The hooks that > would need to be put into ZFS to do snapshot on file close could be used > for other things like single instance storage (though isn''t that the > opposite of ditto blocks on user data hmn whats the opposite of ditto :-)).Hmmm, I will try and research what NTFS and others do.> You can also use dtrace to simulate the every single write case and see > for yourself the massive explosion of snapshots that would occur as a > result.Yea, this is would be bad. Thank you, will try and see if other filesystems do anything with a closure hook. -- Alex Barclay University of Tulsa Center for Information Security Enterprise Research Group
On Mon, May 22, 2006 at 04:47:07PM +0100, Darren J Moffat wrote:> Now if what you really mean is snapshot on file closure I think you > might well be on to something useful. Whats more NTFS has some cool > stuff in this area for consolidating identical files. The hooks that > would need to be put into ZFS to do snapshot on file close could be used > for other things like single instance storage (though isn''t that the > opposite of ditto blocks on user data hmn whats the opposite of ditto :-)).Hmmm, maybe not on close, but on open(2) with O_W*. But anyways, you''re not going to capture *only the* event that you will *later* be interested in when that event is indistinguishable from millions of others like it a priori -- you either have to capture all of them and related attributes (a huge mountain of hay) or capture summary event information (a smaller mountain of hay). BSM auditing does the latter. If you *really* wanted the former I''d suggest not so much that every write(2), open(2) for write, or close(2), or ZFS tranzation should lead to a snapshot, but that every transaction be streamed onto backup media. Then you could reconstruct the state of your file systems as it would have been on-disk, at any given time. ZFS doesn''t provide this functionality now, and you''d need backup media proportional to the rate of change of your data (potentially a DoS by an attacker who''s trying to hide their tracks). Is this desirable functionality? In any case, if your goal is to make recovery from damage from insider attacks quick, that''s a worthy goal, and I commend you for noticing [implicitly] that you can''t stop insiders from trying. Unfortunately rolling back your filesystems wouldn''t be enough as you''d be potentially losing legitimate data. Recovery can be made easier, but you can''t guarantee trivial recovery from attack. The best you can do is to make the system auditable, so that you can track down insiders gone bad, and so deter them. Nico --
On 5/23/06, Nicolas Williams <Nicolas.Williams at sun.com> wrote:> On Mon, May 22, 2006 at 04:47:07PM +0100, Darren J Moffat wrote: > > Now if what you really mean is snapshot on file closure I think you > > might well be on to something useful. Whats more NTFS has some cool > > stuff in this area for consolidating identical files. The hooks that > > would need to be put into ZFS to do snapshot on file close could be used > > for other things like single instance storage (though isn''t that the > > opposite of ditto blocks on user data hmn whats the opposite of ditto :-)). > > Hmmm, maybe not on close, but on open(2) with O_W*. > > But anyways, you''re not going to capture *only the* event that you will > *later* be interested in when that event is indistinguishable from > millions of others like it a priori -- you either have to capture all of > them and related attributes (a huge mountain of hay) or capture summary > event information (a smaller mountain of hay). > > BSM auditing does the latter. > > If you *really* wanted the former I''d suggest not so much that every > write(2), open(2) for write, or close(2), or ZFS tranzation should lead > to a snapshot, but that every transaction be streamed onto backup media. > Then you could reconstruct the state of your file systems as it would > have been on-disk, at any given time. ZFS doesn''t provide this > functionality now, and you''d need backup media proportional to the rate > of change of your data (potentially a DoS by an attacker who''s trying to > hide their tracks). Is this desirable functionality? > > In any case, if your goal is to make recovery from damage from insider > attacks quick, that''s a worthy goal, and I commend you for noticing > [implicitly] that you can''t stop insiders from trying. Unfortunately > rolling back your filesystems wouldn''t be enough as you''d be potentially > losing legitimate data. Recovery can be made easier, but you can''t > guarantee trivial recovery from attack. The best you can do is to make > the system auditable, so that you can track down insiders gone bad, and > so deter them.Sorry for coming late to this conversation. It seems to me that this may be the kind of place that AUDIT and/or ALARM ACEs would be useful. Only snapshot on a subset of files. Are there any plans for anything to make use of them, not for this specifically, but for anything at all? Boyd Melbourne, Australia