I do a daily snapshot of two filesystems, and over the past few months it''s obviously grown to a bunch. "zfs list" shows me all of those. I can change it to use the "-t" flag to not show them, so that''s good. However, I''m worried about boot times and other things. Will it get to a point with 1000''s of snapshots that it takes a long time to boot, or do any sort of sync or scrub activities? Thanks :)
I think it''s filesystems, not snapshots, that take a long time to enumerate. (If I''m wrong, somebody correct me :) On Sun, Mar 8, 2009 at 10:10 PM, mike <mike503 at gmail.com> wrote:> I do a daily snapshot of two filesystems, and over the past few months > it''s obviously grown to a bunch. > > "zfs list" shows me all of those. > > I can change it to use the "-t" flag to not show them, so that''s good. > However, I''m worried about boot times and other things. > > Will it get to a point with 1000''s of snapshots that it takes a long > time to boot, or do any sort of sync or scrub activities? > > Thanks :) > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >
Richard Morris - Sun Microsystems - Burlington United States
2009-Mar-09 18:49 UTC
[zfs-discuss] Is there a limit to snapshotting?
On 03/08/09 23:16, Blake wrote:> I think it''s filesystems, not snapshots, that take a long time to > enumerate. (If I''m wrong, somebody correct me :)The time needed to iterate through the same number of snapshots and filesystems should be about the same. However, whenever any of the ZFS filesystems or snapshots to be listed are not already cached in memory, then it does take time for zfs list to load them from disk. However, some prefetching has been added to help speed this up. No extra time is needed to boot a system with thousands of filesystems or snapshots. However, ZFS mounts filesystems by default and it does take time for thousands of filesystems to be mounted. Changes have been made to speed this up by reducing the number of mnttab lookups. And zfs list has been changed to no longer show snapshots by default. But it still might make sense to limit the number of snapshots saved: http://blogs.sun.com/timf/entry/zfs_automatic_snapshots_0_10 -- Rich> On Sun, Mar 8, 2009 at 10:10 PM, mike <mike503 at gmail.com> wrote: > >> I do a daily snapshot of two filesystems, and over the past few months >> it''s obviously grown to a bunch. >> >> "zfs list" shows me all of those. >> >> I can change it to use the "-t" flag to not show them, so that''s good. >> However, I''m worried about boot times and other things. >> >> Will it get to a point with 1000''s of snapshots that it takes a long >> time to boot, or do any sort of sync or scrub activities? >> >> Thanks :) >> _______________________________________________ >> zfs-discuss mailing list >> zfs-discuss at opensolaris.org >> http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >> >> > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20090309/b0a8e92e/attachment.html>
Well, I could just use the same script to create my daily snapshot to remove a snapshot with the same prefix, just different date (say, keep 30 days only or something) My hope was to just keep a running archive indefinitely. But I guess snapshots are only as good as needed, and I doubt I will realize I need a file I lost 6+ months ago... On Mon, Mar 9, 2009 at 11:49 AM, Richard Morris - Sun Microsystems - Burlington United States <Richard.Morris at sun.com> wrote:> On 03/08/09 23:16, Blake wrote: > > I think it''s filesystems, not snapshots, that take a long time to > enumerate. (If I''m wrong, somebody correct me :) > > The time needed to iterate through the same number of snapshots and > filesystems should be about the same.? However, whenever any of the > ZFS filesystems or snapshots to be listed are not already cached in > memory, then it does take time for zfs list to load them from disk. > However, some prefetching has been added to help speed this up. > > No extra time is needed to boot a system with thousands of filesystems > or snapshots.? However, ZFS mounts filesystems by default and it does > take time for thousands of filesystems to be mounted.? Changes have > been made to speed this up by reducing the number of mnttab lookups. > > And zfs list has been changed to no longer show snapshots by default. > But it still might make sense to limit the number of snapshots saved: > http://blogs.sun.com/timf/entry/zfs_automatic_snapshots_0_10 > > -- Rich > > On Sun, Mar 8, 2009 at 10:10 PM, mike <mike503 at gmail.com> wrote: > > > I do a daily snapshot of two filesystems, and over the past few months > it''s obviously grown to a bunch. > > "zfs list" shows me all of those. > > I can change it to use the "-t" flag to not show them, so that''s good. > However, I''m worried about boot times and other things. > > Will it get to a point with 1000''s of snapshots that it takes a long > time to boot, or do any sort of sync or scrub activities? > > Thanks :) > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss > >
mike wrote:> Well, I could just use the same script to create my daily snapshot to > remove a snapshot with the same prefix, just different date (say, keep > 30 days only or something) >NB the autosnapshot feature (aka Time Slider Manager) already has this capability.> My hope was to just keep a running archive indefinitely. But I guess > snapshots are only as good as needed, and I doubt I will realize I > need a file I lost 6+ months ago... >famous last words... :-) -- richard
Bear in mind I am not that comfortable with smf and manifests and Solaris userland and such... I wouldn''t want to be messing up anything or not setting it up correctly. My little PHP script (yes, I use PHP for shell scripting) does a perfect job of what I need and I got a neat idea from Brad Stone about rolling up daily snapshots into monthly snapshots, which would roll up into yearly snapshots... On Mon, Mar 9, 2009 at 1:29 PM, Richard Elling <richard.elling at gmail.com> wrote:> mike wrote: >> >> Well, I could just use the same script to create my daily snapshot to >> remove a snapshot with the same prefix, just different date (say, keep >> 30 days only or something) >> > > NB the autosnapshot feature (aka Time Slider Manager) already > has this capability. > >> My hope was to just keep a running archive indefinitely. But I guess >> snapshots are only as good as needed, and I doubt I will realize I >> need a file I lost 6+ months ago... >> > > famous last words... :-) > -- richard > >
mike wrote:> Bear in mind I am not that comfortable with smf and manifests and > Solaris userland and such... I wouldn''t want to be messing up anything > or not setting it up correctly. >For the benefit of others who may be lurking, the policies are: svc interval keep --------------------------------------------- auto-snapshot:frequent 15 mins 4 auto-snapshot:hourly 1 hours 24 auto-snapshot:daily 1 days 31 auto-snapshot:weekly 7 days 4 auto-snapshot:monthly 1 months 12> My little PHP script (yes, I use PHP for shell scripting) does a > perfect job of what I need and I got a neat idea from Brad Stone about > rolling up daily snapshots into monthly snapshots, which would roll up > into yearly snapshots... >For scripting wizards, the same table is available from: $ for i in frequent hourly daily weekly monthly; do echo $i $(svcprop -p zfs/period auto-snapshot:$i) $(svcprop -p zfs/interval auto-snapshot:$i) keep $(svcprop -p zfs/keep auto-snapshot:$i) done frequent 15 minutes keep 4 hourly 1 hours keep 24 daily 1 days keep 31 weekly 7 days keep 4 monthly 1 months keep 12 -- richard> On Mon, Mar 9, 2009 at 1:29 PM, Richard Elling <richard.elling at gmail.com> wrote: > >> mike wrote: >> >>> Well, I could just use the same script to create my daily snapshot to >>> remove a snapshot with the same prefix, just different date (say, keep >>> 30 days only or something) >>> >>> >> NB the autosnapshot feature (aka Time Slider Manager) already >> has this capability. >> >> >>> My hope was to just keep a running archive indefinitely. But I guess >>> snapshots are only as good as needed, and I doubt I will realize I >>> need a file I lost 6+ months ago... >>> >>> >> famous last words... :-) >> -- richard >> >> >>