On 11/10/2015 12:16 PM, Warren Young wrote:> > Well, be fair, rsync can also miss files if files are changing while the backup occurs. Once rsync has passed through a given section of the tree, it will not see any subsequent changes.I think you miss my meaning. Consider this sequence of events: * "find" begins and processes dirA and then dirB * another application writes files in dirA * "find" completes * a new timestamp file is written Now, the new file in dirA wasn't seen by find during this run, and it won't be seen on the next run either. That's what I mean by missed. Not temporarily missed, but permanently. That file won't ever be backed up in this very na?ve process. There's no benefit to the process, either. rsync can efficiently examine and synchronize filesystems without using find. And while it may miss files that are written while it's running, it *will* get them on the next run, unlike using "find".> If you need guaranteed-complete filesystem-level snapshots, you need to be using something at the kernel level that can atomically collect the set of modified blocks/files, rather than something that crawls the tree in user space.Generally, I agree with you. In fact: https://bitbucket.org/gordonmessmer/dragonsdawn-snapshot https://github.com/rsnapshot/rsnapshot/pull/44 Doing block-level differentials is nice, if you're using ZFS. But not everyone wants to run ZFS on Linux. I do think that backing up snapshots is important, though.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 On 10/11/15 21:05, Gordon Messmer wrote:> On 11/10/2015 12:16 PM, Warren Young wrote: >> >> Well, be fair, rsync can also miss files if files are changing >> while the backup occurs. Once rsync has passed through a given >> section of the tree, it will not see any subsequent changes. > > I think you miss my meaning. Consider this sequence of events: > > * "find" begins and processes dirA and then dirB * another > application writes files in dirA * "find" completes * a new > timestamp file is written > > Now, the new file in dirA wasn't seen by find during this run, and > it won't be seen on the next run either. That's what I mean by > missed. Not temporarily missed, but permanently. That file won't > ever be backed up in this very na?ve process.That's plain bad system analysis. Read the start date, record the current date and THEN start processing. You will get the odd extra file but will not loose any.> > There's no benefit to the process, either. rsync can efficiently > examine and synchronize filesystems without using find. And while > it may miss files that are written while it's running, it *will* > get them on the next run, unlike using "find". > >> If you need guaranteed-complete filesystem-level snapshots, you >> need to be using something at the kernel level that can >> atomically collect the set of modified blocks/files, rather than >> something that crawls the tree in user space. > > Generally, I agree with you. In fact: > https://bitbucket.org/gordonmessmer/dragonsdawn-snapshot > https://github.com/rsnapshot/rsnapshot/pull/44 > > Doing block-level differentials is nice, if you're using ZFS. But > not everyone wants to run ZFS on Linux. I do think that backing > up snapshots is important, though. > > _______________________________________________ CentOS mailing > list CentOS at centos.org > https://lists.centos.org/mailman/listinfo/centos-----BEGIN PGP SIGNATURE----- Version: GnuPG v2.0.22 (GNU/Linux) iQIcBAEBAgAGBQJWQn/cAAoJEAF3yXsqtyBl6YwQAKRneHW0APPxY9znjXFcSiuC M3r5a5cYDOIdXNkFRg3Psoa38vaiHnmV3BFWfWWZzo4qAYfUP9fNP5i9McXWslK+ Jv2S0s4WoUcvyguRStpw++HmOBJ4Hywr2qXQnnPNuzuJXLD/EgxGgA5hv0lbYAN1 fzci8UuGpZI/WsCMIib3f8u6tQ4CTsaZm7AUjAZ/EdxyhNl17bjAC/U9tWYpCKtr DF5jEyU18ZiHcWDym45SI1fjaiRf8jR0OrennViFyThYVLsaYgRiHn0gZAl9H2jx 5oL0fONkNY0f4hNN0XFyJkbDfdF4QCNu6WBHByv5I2MlOBIZBOI5zWagRTZ1uLCG mMvpnj2Mmbugq+8p+XAhyB/xvMLaIA5suUSmIDej+rylH5FgrSUa+aHflUNBWZcK P7odBqEDIc27CCSckdrAxY16+Drs5gjfw0EFmKXVaZ2SIyeJ4gTKGororEdOH9bI ry6ItSuKD732OLx33/j15YuEUBhzwGHYiuE4xCZCegKK64/zFvVtLVO6z6kBN17T BtOMX6VOyfJl3zxmlMfiZx6A01TxSqOw+QS9Tepcz0tAauFGN3UOow/Kv2RnrwhZ LwbjRqPto/0599owBromt6lzZSoI9/4JX6DS+XQeLOPsuq8z6myxZc6AXFxE20NR G9NoAxU24PbTq4VtLiKV =+3+X -----END PGP SIGNATURE-----
On 11/10/2015 03:38 PM, J Martin Rushton wrote:> That's plain bad system analysis. Read the start date, record the > current date and THEN start processing. You will get the odd extra > file but will not loose any.That's my point. "find" doesn't do that and na?ve implementations of the original suggestion are likely to do work poorly. For no reason. Just don't use "find" to feed rsync a list of files to sync. It's not more efficient, it might miss files, it won't sync deleted files, etc etc. rsync is designed to synchronize two directory trees. It doesn't need external helpers (except for a pipe, like ssh).
On Wed, Nov 11, 2015 at 5:08 AM, J Martin Rushton <martinrushton56 at btinternet.com> wrote:> -----BEGIN PGP SIGNED MESSAGE----- > Hash: SHA1 > > On 10/11/15 21:05, Gordon Messmer wrote: >> On 11/10/2015 12:16 PM, Warren Young wrote: >>> >>> Well, be fair, rsync can also miss files if files are changing >>> while the backup occurs. Once rsync has passed through a given >>> section of the tree, it will not see any subsequent changes. >> >> I think you miss my meaning. Consider this sequence of events: >> >> * "find" begins and processes dirA and then dirB * another >> application writes files in dirA * "find" completes * a new >> timestamp file is written >> >> Now, the new file in dirA wasn't seen by find during this run, and >> it won't be seen on the next run either. That's what I mean by >> missed. Not temporarily missed, but permanently. That file won't >> ever be backed up in this very na?ve process. > > That's plain bad system analysis. Read the start date, record the > current date and THEN start processing. You will get the odd extra > file but will not loose any. >Heartily agree. I was about to post my response but saw yours. Cheers, -- Arun Khan