I've done quite a bit of looking, but I haven't found an answer that answers this question. Environment: cygwin on Windows rsync 3.0.4 I know that rsync isn't optimzed for speed on local copies - that's clear in my testing. I'm attemting to sync a large volume of files. (In this case, I'm syncing a rdiff-backup set...) An initial sync will be about one fourth as fast a a cp of the same files. "Ok, that's not a huge deal. Subsequent syncs should be a lot faster" I though. No. Using a native windows sync tool, a sync of the source to the destination takes perhaps 5 minutes. (~24G, 80K files) However using rsync takes a very long time. (I've never let it finish, but it's still running an hour later or more...) Here's how I run it: rsync --recursive --delete --verbose --stats --perms --super /cygdrive/e/somedir/ /cygdrive/f/somedir/ Thoughts? I'm mainly using rsync 'cause I know it, and use it. If there's some alternative I'd be glad for a pointer. I don't know of any other good sync tools like rsync. (Unison I guess...though I only need one way sync.) [Whole file copy is fine if needed, though some of the files are large 1 GB+ and a delta copy would speed these up...] TIA! -Greg
listserv.traffic@sloop.net wrote:> > I've done quite a bit of looking, but I haven't found an answer that > answers this question. > > Environment: > cygwin on Windows > rsync 3.0.4 > > I know that rsync isn't optimzed for speed on local copies - that's > clear in my testing. I'm attemting to sync a large volume of files. > > (In this case, I'm syncing a rdiff-backup set...) > > An initial sync will be about one fourth as fast a a cp of the same > files. "Ok, that's not a huge deal. Subsequent syncs should be a lot > faster" I though. > > No. > > Using a native windows sync tool, a sync of the source to the > destination takes perhaps 5 minutes. (~24G, 80K files) However using > rsync takes a very long time. (I've never let it finish, but it's > still running an hour later or more...) > > Here's how I run it: > rsync --recursive --delete --verbose --stats --perms --super > /cygdrive/e/somedir/ /cygdrive/f/somedir/ > > Thoughts?Any copying, local, from and to the same spindle looks like; read some stuff seek to the other place write some stuff seek to the other place Timing should be dominated by how much stuff in each place before it seeks to the new place. For Windows, ROBOCOPY may do what you want (Resource kit XP and below, native on Vista) You will probably want the switches /R:1 and /W:1 or similare.> > I'm mainly using rsync 'cause I know it, and use it. If there's some > alternative I'd be glad for a pointer. I don't know of any other good > sync tools like rsync. (Unison I guess...though I only need one way > sync.) [Whole file copy is fine if needed, though some of the files > are large 1 GB+ and a delta copy would speed these up...]Why do you think I use Linux ;=) Cygwin is an incredible program. It almost makes Windows useable. Kinda like a very bad unix. Opinions are my own and not necessarily representative of anyone sane.
listserv.traffic@sloop.net wrote:> I've done quite a bit of looking, but I haven't found an answer that > answers this question. > > Environment: > cygwin on Windows > rsync 3.0.4 > > > Using a native windows sync tool, a sync of the source to the > destination takes perhaps 5 minutes. (~24G, 80K files) However using > rsync takes a very long time. (I've never let it finish, but it's > still running an hour later or more...)That's weird. I routinely use rsync to backup all my harddrives under win64 (using cwrsync) and I haven't noticed any excessive times (at least not beyond common sense). Although my config is much more complicated, the "core" approach is the same. I never felt any real reason to compare to cp or other regular programs (drive example: 0.5TB, 137k files, although only ~100GB is copied due to hide/show/etc. rules). I'll do some comparisons later, when I have a bit of time.
> I've done quite a bit of looking, but I haven't found an answer that > answers this question.> Environment: > cygwin on Windows > rsync 3.0.4> I know that rsync isn't optimzed for speed on local copies - that's > clear in my testing. I'm attemting to sync a large volume of files.> (In this case, I'm syncing a rdiff-backup set...)> An initial sync will be about one fourth as fast a a cp of the same > files. "Ok, that's not a huge deal. Subsequent syncs should be a lot > faster" I though.> No.> Using a native windows sync tool, a sync of the source to the > destination takes perhaps 5 minutes. (~24G, 80K files) However using > rsync takes a very long time. (I've never let it finish, but it's > still running an hour later or more...)> Here's how I run it: > rsync --recursive --delete --verbose --stats --perms --super > /cygdrive/e/somedir/ /cygdrive/f/somedir/> Thoughts?> I'm mainly using rsync 'cause I know it, and use it. If there's some > alternative I'd be glad for a pointer. I don't know of any other good > sync tools like rsync. (Unison I guess...though I only need one way > sync.) [Whole file copy is fine if needed, though some of the files > are large 1 GB+ and a delta copy would speed these up...]> TIA! > -GregOk, belay all that... It seems that I was misreading the verbose output - (An "example" log file as part of the documentation might be helpful...) Once I realized it was updating all the files, I started trying to figure out why. (I thought it was simply telling me it was checking each file to determine if it should update them...rather than actually finding them as "changed.") Somehow I missed a --times directive. Thus, it viewed every file as "changed." Problem solved. --- Sometimes I wonder if I need my head checked. :) -Greg