Hi, While researching on where rsync-3.0.6 spending more time when we have huge number of files in directory(~1000K files). I got below statistics: 2009/05/28 19:05:38 [9660] building file list . . . 2009/05/28 21:40:09 [9660] Number of files: 724591 2009/05/28 21:40:09 [9660] Number of files transferred: 5674 2009/05/28 21:40:09 [9660] Total file size: 63492447862 bytes 2009/05/28 21:40:09 [9660] Total transferred file size: 1472465959 bytes 2009/05/28 21:40:09 [9660] Literal data: 94119888 bytes 2009/05/28 21:40:09 [9660] Matched data: 1378346071 bytes 2009/05/28 21:40:09 [9660] File list size: 17239106 2009/05/28 21:40:09 [9660] File list generation time: 0.671 seconds 2009/05/28 21:40:09 [9660] File list transfer time: 0.000 seconds 2009/05/28 21:40:09 [9660] Total bytes sent: 115513595 2009/05/28 21:40:09 [9660] Total bytes received: 4873849 2009/05/28 21:40:09 [9660] sent 115513595 bytes received 4873849 bytes 12983.28 bytes/sec 2009/05/28 21:40:09 [9660] total size is 63492447862 speedup is 527.40 Total time for synching is around 2.5 hrs ( sent_bytes / XferRate ). Here I am surprised when I see file list generation time = 0.671 seconds !!! Does that mean rsync traversed complete 1000k files in 0.671 seconds? Jignesh. -------------- next part -------------- HTML attachment scrubbed and removed
Wayne Davison
2009-May-30 17:12 UTC
rsync-3.0.6 prints file list generation time wrong ???
On Fri, May 29, 2009 at 06:29:22PM +0530, Jignesh Shah wrote:> Does that mean rsync traversed complete 1000k files in 0.671 seconds?Rsync uses an incremental recursion scan by default, so when that is active the time mentioned is just for the getting-started part of the scan. I've been thinking aobut making the sender time each extra scan that it does as the incremental scan progresses and add the times up, but that is not what rsync currently does. ..wayne..