Hi everyone I'm still getting strange sizes on the daemon logs. After transferring around 1.66GB over 14 hours (256kbit link) to an absolutely empty directory, I get: 2007/01/26 04:13:43 [26441] sent 57014 bytes received 44708 bytes total size 1947222268 du reports the final size of the directory on the daemon side as 1.6 G, and the "received" bytes can't be 44708?! I know about the daemon side sometimes doubling the total size when --delete is used, and this was fixed in 3.0.0. But the size isnt double in this case, and the received bytes is nowhere near 1.6 GB, which was what was actually received in this run. am i missing something here? This is 2.6.9 btw. Julian -------------- next part -------------- HTML attachment scrubbed and removed
On 1/26/07, Julian Pace Ross <linux@prisma.com.mt> wrote:> I'm still getting strange sizes on the daemon logs. > After transferring around 1.66GB over 14 hours (256kbit link) to an > absolutely empty directory, I get: > > 2007/01/26 04:13:43 [26441] sent 57014 bytes received 44708 bytes total > size 1947222268 > > du reports the final size of the directory on the daemon side as 1.6 G, and > the "received" bytes can't be 44708?!I can reproduce this problem! I also figured out what is causing it. Any daemon process that calls exit_cleanup(0) prints a statistics line in log_exit, but the receiver doesn't exit: it waits for the generator to kill it. Thus, only the generator prints statistics. However, the generator's value of "bytes received" is incorrect because it does not include bytes received by the receiver since the generator forked. Furthermore, the statistic printing can't simply be moved to the receiver because the receiver's "bytes sent" does not include bytes sent by the generator after the fork. I think the best fix is for the receiver to tell the generator how many bytes it has received in a message just before the second MSG_DONE; then the generator can correctly print the statistics. Matt
On Sat, Jan 27, 2007 at 09:21:57AM -0500, Matt McCutchen wrote:> I think the best fix is for the receiver to tell the generator how > many bytes it has received in a message just before the second > MSG_DONE; then the generator can correctly print the statistics.I came to the same conclusion and checked in a fix for this a little bit ago. ..wayne..
Thanks wayne/matt Would be simple to make a patch for this for 2.6.9? I would love this feature to work on a server without shifting to 3.0.0 cvs on this particular machine as yet... Cheers Julian ----- Original Message ----- From: "Wayne Davison" <wayned@samba.org> To: "Matt McCutchen" <hashproduct+rsync@gmail.com> Cc: "Julian Pace Ross" <linux@prisma.com.mt>; <rsync@lists.samba.org> Sent: Saturday, January 27, 2007 4:02 PM Subject: Re: daemon sizes> On Sat, Jan 27, 2007 at 09:21:57AM -0500, Matt McCutchen wrote: >> I think the best fix is for the receiver to tell the generator how >> many bytes it has received in a message just before the second >> MSG_DONE; then the generator can correctly print the statistics. > > I came to the same conclusion and checked in a fix for this a little > bit ago. > > ..wayne.. >
On Sat, Jan 27, 2007 at 08:16:27PM +0100, Julian Pace Ross wrote:> Would [it] be simple to make a patch for this for 2.6.9?I have worked up a patch for 2.6.9 that fixes the file-size total and the received byte count when the daemon is the receiver. This is similar to what I did for the CVS version. ..wayne.. -------------- next part -------------- A non-text attachment was scrubbed... Name: daemon_stats.patch Type: text/x-diff Size: 2079 bytes Desc: not available Url : http://lists.samba.org/archive/rsync/attachments/20070202/45edf706/daemon_stats.bin
Patched it and seems to be working fine now. Thanks Wayne..much appreciated.> On Sat, Jan 27, 2007 at 08:16:27PM +0100, Julian Pace Ross wrote: >> Would [it] be simple to make a patch for this for 2.6.9? > > I have worked up a patch for 2.6.9 that fixes the file-size total > and the received byte count when the daemon is the receiver. This > is similar to what I did for the CVS version. > > ..wayne.. >
On Wed, Sep 19, 2007 at 10:17:52PM +0200, Julian Pace Ross wrote:> I may be wrong, but it seems that when "total size" is more than > FFFFFFFF bytes (int32), the total size is still displayed wrongly in > the daemon logs when the daemon is the receiver.What version are you talking about? The CVS version has had this fixed for quite some time now (using write_varlong(), which handles 64-bit numbers). ..wayne..