HI all, efficiency question for VERY low bandwith networks Suppose I know the list of files that are changed What is the most efficient way to make rsync sync this list. Currently I use --include-from --exclude to generate a 'filelist' but I suspect that client and/or server exchange the list of files in the module to be synced. this traffic can be avoided since the include-from points to FILE names and not FILE patterns Suggestions ? W
You did not specifically mention it: compression (-z) would probably help more than anything. Otherwise, you could do something like: Have a file (ie filelist.txt) that contains the filename (with relative paths), one file per line. rsync -rRWz `cat filelist.txt` user@hostname::module The only limitation to this is if you have a large file list you may get a shell error. It has already been suggested in this list, as well as by myself in the rsync wishlist for a new option to specify a file that has a list of files to be transferred. wim delvaux wrote:> HI all, > > efficiency question for VERY low bandwith networks > > Suppose I know the list of files that are changed > What is the most efficient way to make rsync sync this list. > > Currently I use --include-from --exclude to generate a 'filelist' but I > suspect that client and/or server exchange the list of files in the module to > be synced. this traffic can be avoided since the include-from points to FILE > names and not FILE patterns > > Suggestions ? > > W-- Aaron W Morris decep PGP Key ID: 259978D1
On Saturday 04 January 2003 19:49, Aaron Morris wrote:> You did not specifically mention it: compression (-z) would probably > help more than anything. Otherwise, you could do something like:I thought it was on by default ?> > Have a file (ie filelist.txt) that contains the filename (with relative > paths), one file per line.That is what I have> > rsync -rRWz `cat filelist.txt` user@hostname::module > > The only limitation to this is if you have a large file list you may get > a shell error.Yes that MIGHT be a problem for me too> > It has already been suggested in this list, as well as by myself in the > rsync wishlist for a new option to specify a file that has a list of > files to be transferred.Perhaps i need to have a look a the sources and see what it takes to put that option in (say --files) Thanx W
I usually use rsync 2.5.4 on AIX and compression is not enabled by default. 2.5.5 may be different. Another important option would be to use "-u" since it would only transfer a file if it has changed (even if it is in the file list). wim delvaux wrote:> On Saturday 04 January 2003 19:49, Aaron Morris wrote: > >>You did not specifically mention it: compression (-z) would probably >>help more than anything. Otherwise, you could do something like: > > > I thought it was on by default ? > > >>Have a file (ie filelist.txt) that contains the filename (with relative >>paths), one file per line. > > > That is what I have > > >>rsync -rRWz `cat filelist.txt` user@hostname::module >> >>The only limitation to this is if you have a large file list you may get >> a shell error. > > > Yes that MIGHT be a problem for me too > > >>It has already been suggested in this list, as well as by myself in the >>rsync wishlist for a new option to specify a file that has a list of >>files to be transferred. > > > Perhaps i need to have a look a the sources and see what it takes > to put that option in (say --files) > > Thanx > > W-- Aaron W Morris decep PGP Key ID: 259978D1
Aaron Morris wrote:> You did not specifically mention it: compression (-z) would probably > help more than anything. Otherwise, you could do something like: > > Have a file (ie filelist.txt) that contains the filename (with relative > paths), one file per line. > > rsync -rRWz `cat filelist.txt` user@hostname::module >as I understand this works only when connecting to a rsync daemon... what if I wanted to use ssh as a shell? Lorenzo -- +-----------------------------------------------------+ | Lorenzo Bettini ICQ# lbetto, 16080134 | | PhD student in Computer Science | | Dip. Sistemi e Informatica, Univ. di Firenze | | Tel +39 055 4796741, Fax +39 055 4796730 | | Florence - Italy (Linux User # 158233) | | Home Page : http://www.lorenzobettini.it | | E-Mail : bettini@dsi.unifi.it | | http://music.dsi.unifi.it XKlaim language | | http://www.lorenzobettini.it/purple Cover Band | | http://www.gnu.org/software/src-highlite | | http://www.gnu.org/software/gengetopt | +-----------------------------------------------------+