Low, Alex
2009-Jun-22 01:37 UTC
rsync is fantastic except for one usual feature we want from it .. please help ?
Hi, Rsync is a fantastic program, and it does everything we need in terms of file transfer and syncing source and destination file directories. We do have a situation now where we need rsync to transfer the files once only from the source directory. Please let me briefly explain using a simple generic exmple: 1. At t0, we transfer f0 location A to location B 2. At t1, 10 new files are added to Location A Therefore, rysnc faithfully transfers all 10 new files to location B At location B, we now have f0 to f10. 3. At t2, f0 to f3 are deleted from location B, and we don't ever want the deleted files to be copied again from location A 4. At t3, rysnc waits up, transfers files f0 and f10 again plus any new files that may have been added. So, at location B, we end with the deleted files f0 to f3 again. Can you please think of a way we can soft-switch rsync NOT to do step 4 above ? Our help much appreciated. Regards, Alex low Asgard Capital Management Limited Level 39, 152-158 St Georges Terrace, PERTH WA 6000 * (08) 9415 5758 * 0423 820 728 7 (08) 9481 8481 alex.low@asgardwealthsolutions.com.au www.asgard.com.au <http://www.asgard.com.au/> Asgard Wealth Solutions Advice. Solutions. Confidence. ====================================================== IMPORTANT INFORMATION This message and any files transmitted with it are confidential and should be read only by those persons to whom it is addressed. If you have received this message in error, please notify us immediately by way of reply. Please also destroy and delete the message from your computer. Any unauthorised form of reproduction of this message is strictly prohibited. It is the duty of the recipient to virus scan and otherwise test the information provided before loading on to any computer system. No warranty is given or made by Asgard Wealth Solutions Limited, Asgard Capital Management Limited or SECURITOR Financial Group Limited that the information is free of a virus or any other defect or error and they will not be liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. Any views expressed in this message are those of the individual sender, except where the sender specifically states they are the views of Asgard Wealth Solutions Limited, Asgard Capital Management Limited or SECURITOR Financial Group Limited, as the case may be. ======================================================-------------- next part -------------- HTML attachment scrubbed and removed
On Mon, 2009-06-22 at 09:24 +0800, Low, Alex wrote:> We do have a situation now where we need rsync to transfer the files > once only from the source directory. Please let me briefly explain > > using a simple generic exmple: > > 1. At t0, we transfer f0 location A to location B > > 2. At t1, 10 new files are added to Location A > > Therefore, rysnc faithfully transfers all 10 new files to location B > > At location B, we now have f0 to f10. > > 3. At t2, f0 to f3 are deleted from location B, and we don?t ever want > the deleted files to be copied again from location A > > 4. At t3, rysnc waits up, transfers files f0 and f10 again plus any > new files that may have been added. > > So, at location B, we end with the deleted files f0 to f3 again. > > Can you please think of a way we can soft-switch rsync NOT to do step > 4 above ?As a stateless one-way mirroring tool, rsync doesn't support that (unless you count --remove-source-files). If you could provide some more insight into your workflow, I might be able to come up with a suitable alternative approach. -- Matt
Julian Pace Ross
2009-Jun-22 07:32 UTC
rsync is fantastic except for one usual feature we want from it .. please help ?
You're probably looking for the option " --existing". You can also play around with --update, --ignore-existing, and --remove-source-files off the top of my head... not sure which combination will work for you. 2009/6/22 Low, Alex <Alex.Low@asgardwealthsolutions.com.au>> Hi, > > > > Rsync is a fantastic program, and it does everything we need in terms of > file transfer and syncing source and destination file directories. > > > > We do have a situation now where we need rsync to transfer the files once > only from the source directory. Please let me briefly explain > > using a simple generic exmple: > > > > 1. At t0, we transfer f0 location A to location B > > > > 2. At t1, 10 new files are added to Location A > > Therefore, rysnc faithfully transfers all 10 new files to location B > > At location B, we now have f0 to f10. > > > > 3. At t2, *f0 to f3* are deleted from location B, and we don?t ever want > the deleted files to be copied again from location A > > > > 4. At t3, rysnc waits up, transfers files f0 and f10 again plus any new > files that may have been added. > > > > So, at location B, we end with the deleted files *f0 to f3* again. > > > > Can you please think of a way we can soft-switch rsync NOT to do step 4 > above ? > > > > Our help much appreciated. > > > > Regards, > > > > *Alex low* > > > > Asgard Capital Management Limited > > Level 39, 152-158 St Georges Terrace, PERTH WA 6000 > ( (08) 9415 5758 ( 0423 820 728 > 7 (08) 9481 8481 > alex.low@asgardwealthsolutions.com.au* > www.asgard.com.au * > > *Asgard Wealth Solutions** > *Advice. Solutions. Confidence. > > > > ======================================================> IMPORTANT INFORMATION > > This message and any files transmitted with it are confidential and should > be read only by those persons to whom it is addressed. If you have received > this message in error, please notify us immediately by way of reply. Please > also destroy and delete the message from your computer. Any unauthorised > form of reproduction of this message is strictly prohibited. > > It is the duty of the recipient to virus scan and otherwise test the > information provided before loading on to any computer system. No warranty > is given or made by Asgard Wealth Solutions Limited, Asgard Capital > Management Limited or SECURITOR Financial Group Limited that the information > is free of a virus or any other defect or error and they will not be liable > for the proper and complete transmission of the information contained in > this communication, nor for any delay in its receipt. > > Any views expressed in this message are those of the individual sender, > except where the sender specifically states they are the views of Asgard > Wealth Solutions Limited, Asgard Capital Management Limited or SECURITOR > Financial Group Limited, as the case may be. > > ======================================================> > -- > Please use reply-all for most replies to avoid omitting the mailing list. > To unsubscribe or change options: > https://lists.samba.org/mailman/listinfo/rsync > Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html >-------------- next part -------------- HTML attachment scrubbed and removed
Conway, Timothy
2009-Jun-22 15:06 UTC
rsync is fantastic except for one usual feature we want from it ..please help ?
That's not a usual feature. That's a very unusual feature, as it would never be included in a syncronization tool. What you want is a stateful file transfer tool. Since you wish to transfer only new files, rsync has no advantage over scp in your application. find src -type d -print|while read dir do if grep "^$dir$" /file/thathasthelistofwhatyouvesent>/dev/null then : else scp -rp $dir locationb:`dirname $dir` && echo "$dir" >> /file/thathasthelistofwhatyouvesent fi done find src -type f -print|while read file do if grep "^$file$" /file/thathasthelistofwhatyouvesent>/dev/null then : else scp -p $file locationb:`dirname $file` && echo "$file" >> /file/thathasthelistofwhatyouvesent fi done Just a skeleton to work from. If you're doing a flat directory, the second stanza is all you need. And by all means, you can use rsync instead of scp - I was just looking for things that are universally available. Heck, you could scripted ftp, but it's harder to determine success there. ________________________________ From: rsync-bounces+timothy.conway=jbssa.com@lists.samba.org [mailto:rsync-bounces+timothy.conway=jbssa.com@lists.samba.org] On Behalf Of Low, Alex Sent: Sunday, June 21, 2009 7:24 PM To: rsync@lists.samba.org Subject: rsync is fantastic except for one usual feature we want from it ..please help ? Hi, Rsync is a fantastic program, and it does everything we need in terms of file transfer and syncing source and destination file directories. We do have a situation now where we need rsync to transfer the files once only from the source directory. Please let me briefly explain using a simple generic exmple: 1. At t0, we transfer f0 location A to location B 2. At t1, 10 new files are added to Location A Therefore, rysnc faithfully transfers all 10 new files to location B At location B, we now have f0 to f10. 3. At t2, f0 to f3 are deleted from location B, and we don't ever want the deleted files to be copied again from location A 4. At t3, rysnc waits up, transfers files f0 and f10 again plus any new files that may have been added. So, at location B, we end with the deleted files f0 to f3 again. Can you please think of a way we can soft-switch rsync NOT to do step 4 above ? Our help much appreciated. Regards, Alex low Asgard Capital Management Limited Level 39, 152-158 St Georges Terrace, PERTH WA 6000 * (08) 9415 5758 * 0423 820 728 7 (08) 9481 8481 alex.low@asgardwealthsolutions.com.au www.asgard.com.au <http://www.asgard.com.au/> Asgard Wealth Solutions Advice. Solutions. Confidence. ======================================================IMPORTANT INFORMATION This message and any files transmitted with it are confidential and should be read only by those persons to whom it is addressed. If you have received this message in error, please notify us immediately by way of reply. Please also destroy and delete the message from your computer. Any unauthorised form of reproduction of this message is strictly prohibited. It is the duty of the recipient to virus scan and otherwise test the information provided before loading on to any computer system. No warranty is given or made by Asgard Wealth Solutions Limited, Asgard Capital Management Limited or SECURITOR Financial Group Limited that the information is free of a virus or any other defect or error and they will not be liable for the proper and complete transmission of the information contained in this communication, nor for any delay in its receipt. Any views expressed in this message are those of the individual sender, except where the sender specifically states they are the views of Asgard Wealth Solutions Limited, Asgard Capital Management Limited or SECURITOR Financial Group Limited, as the case may be. ====================================================== -------------- next part -------------- HTML attachment scrubbed and removed