Hi All, I made a Mac UI application and am running rsync 3 from "do shell script" (applescript) and am trying to parse the output so I can get a progress bar to give a fairly true reading of size , or # of files, copied. I notice that the output reads for example, ..... 1650 of 8345 files to copy and then later ....30 of 11345 files to copy and eventually down to ....1 of 18000 files to copy So I get the idea that rsync 3 is reading in "batches" or at least doesn't give the total # of files up front (like apple rsync or rsyncx does with a true % ) so you get what looks like an exaggerated fast beginning on the progress bar which then slows to a crawl as it nears the finish line. Is there any way to get progress to give a more overall picture. I tried --stats which didn't work at all and -v and of course -- progress. Just wondering if I have missed something. Thanks, Rob D
On Tue, 2008-01-29 at 14:25 -0500, Robert DuToit wrote:> I made a Mac UI application and am running rsync 3 from "do shell > script" (applescript) and am trying to parse the output so I can get a > progress bar to give a fairly true reading of size , or # of files, > copied. I notice that the output reads for example, > > ..... 1650 of 8345 files to copy > > and then later > > ....30 of 11345 files to copy > > and eventually down to > > ....1 of 18000 files to copy > > So I get the idea that rsync 3 is reading in "batches" or at least > doesn't give the total # of files up front (like apple rsync or rsyncx > does with a true % ) so you get what looks like an exaggerated fast > beginning on the progress bar which then slows to a crawl as it nears > the finish line. > > Is there any way to get progress to give a more overall picture. I > tried --stats which didn't work at all and -v and of course -- > progress. Just wondering if I have missed something.Well, you can disable incremental recursion and force rsync to scan all the files up front by passing --no-i-r. If you pass -v and --progress, the initial scan will print " N files..." messages that you can show to give the user an idea of the scan progress. Once the scan is finished, the total won't change. With this approach, your rsync GUI will act like the Finder copy operation that (IIRC) scans the files up front, saying something like "Preparing to copy... N files" as it does so. Matt
On Jan 30, 2008, at 3:46 AM, Paul Slootman wrote:> > Do note that disabling the incremental recursion will impact the > performance, esp. with a large number of files. >I did some tests (I am becoming the OSX rsync_3 benchmark guy!) and for 15GB Home folder, there was only a slight difference in performance but see that could grow considerably with larger numbers. Without incremental recursion, the initial file count seems to take forever, though it catches up a bit after that. The incremental backups are almost the same in speed at this size. The one option, I mentioned before, that does seem to affect performance is the osx- creation-dates patch. Oddly, it is much slower for the incremental backup scans (2+ times) than for the initial full copy, proportionally. I think I leave the --no-i-r- option as an option since I now include a text field for roll-your-own options in the UI. Half of the people want these things decided for them in the code but a lot have asked if rsync could do this or that, so now everyone can be happy. Me too. As for the UI progress bar with the recursion, I will represent the # of files copied per increment count and leave out any mention of size copied (since that was always just an extrapolation for # of files.) That will give a true idea of what rsync is really doing. And maybe I can fool the progress bar into slowing down the initial race forward. Thanks, Rob Dutoit