rsync - Jul 2014 - [Bug 10738] New: report --stats output when termination signal arrives

https://bugzilla.samba.org/show_bug.cgi?id=10738

           Summary: report --stats output when termination signal arrives
           Product: rsync
           Version: 3.0.9
          Platform: x64
        OS/Version: Linux
            Status: NEW
          Severity: enhancement
          Priority: P5
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: mark.bugzilla at winksmith.com
         QAContact: rsync-qa at samba.org


i am using Ubuntu 12.04.4LTS.  The use case I have is to run an rsync during
the hours of 1am to 7am.  I use the timeout(1) program for this purpose:

    timeout -s ALRM 6h rsync --stats -av /source /destination

With the above usage, rsync receives a SIGALRM signal after 6h of running. 
when this happens, it just terminates with no further output.

I would find it useful to display the current --stats output when the signal
arrives.  This signal may SIGINT, SIGTERM, or SIGALRM, but in my case, I would
really like SIGALRM to be handled at the very least.  I can definitely see a
user wanting to see stats even when they type ^C on the terminal though
(SIGINT).

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10738

--- Comment #1 from roland <devzero at web.de> 2014-07-24 21:44:38 UTC ---
this sounds really useful !

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10738

--- Comment #2 from Kevin Korb <rsync at sanitarium.net> 2014-07-24
21:48:12 UTC ---
This sounds like a good idea to me.  It also sounds a lot like what dd does if
you send it sigusr1.  Therefore I will suggest that if this is added to rsync
we should also have the ability to get current --stats output with a sigusr1
without actually ending the rsync run.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10738

--- Comment #3 from Mark Smith <mark.bugzilla at winksmith.com> 2014-07-24
22:50:06 UTC ---
markie likes the SIGUSR1 idea too and it has precedents with dd as well.

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10738

Ben RUBSON <ben.rubson at gmail.com> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
                 CC|                            |ben.rubson at gmail.com

--- Comment #4 from Ben RUBSON <ben.rubson at gmail.com> ---
Hello,

I'm facing this "issue" too, I really would like to have stats
displayed when
rsync receives a termination signal.
Sometimes rsync runs all night long and is killed in the morning to save
bandwidth, would then really be interesting to have stats to help with
reporting.

Any progress in this then ?

Many thx !

Ben

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10738

--- Comment #5 from Ben RUBSON <ben.rubson at gmail.com> ---
So I just went through the code and found that this feature request is already
implemented.
We just have to use SIGUSR2 and rsync will stop, displaying stats.
kill -USR2 <rsync_pid>
Perfect !

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10738

--- Comment #6 from Ben RUBSON <ben.rubson at gmail.com> ---
Mmmh however some of the statistics seem to be wrongly calculated.

### Example with tsync 3.1.2 :

Number of files: 8,732 (reg: 7,568, dir: 1,163, link: 1)
Number of created files: 1,232 (reg: 1,187, dir: 45)
Number of deleted files: 0
Number of regular files transferred: 1,187
Total file size: 17.92G bytes
Total transferred file size: 1.48G bytes
Literal data: 914.70M bytes
Matched data: 0 bytes
File list size: 27.07K
File list generation time: 0.001 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 0
Total bytes received: 0

sent 0 bytes  received 0 bytes  -0.00 bytes/sec
total size is 17.92G  speedup is -9,223,372,036,854,775,808nf

### Example with an old rsync 2.6.9 :

Number of files: 24626 
Number of files transferred: 827 
Total file size: 69­.44G bytes 
Total transferred file size: 1­.20G bytes 
Literal data: 633­.50M bytes 
Matched data: 441­.70K bytes 
File list size: 526135 
File list generation time: 0­.588 seconds 
File list transfer time: 0­.000 seconds 
Total bytes sent: 0 
Total bytes received: 0 

sent 0 bytes  received 0 bytes  -0­.0 bytes/sec 
total size is 69­.44G  speedup is inf

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10738

--- Comment #7 from Ben RUBSON <ben.rubson at gmx.com> ---
Don't know whether or not this could be closed, thanks to "kill -USR2
<rsync_pid>".
For the stats bug, let's refer to the whole report
https://github.com/WayneD/rsync/issues/7.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.

https://bugzilla.samba.org/show_bug.cgi?id=10738

Wayne Davison <wayne at opencoder.net> changed:

           What    |Removed                     |Added
----------------------------------------------------------------------------
             Status|NEW                         |ASSIGNED

--- Comment #8 from Wayne Davison <wayne at opencoder.net> ---
We don't have all the stats info to output until the end of the run
exchanges
data between the procs (there are 3 processes in the transfer, and some of the
processes don't have access to the data until the end). For instance, since
the
receiver is the only process on the receiving side to have access to the
incoming socket data, the generator can't output received-data stats until
the
receiver reports the count at the end. And if the sender is local, it
doesn't
know the deleted-file counts from the remote generator until the end.

I wouldn't recommend sending either of the SIGUSR signals to the rsync
processes since they are used internally to indicate that the transfer has
ended successfully. Any data output by the processes would thus be inaccurate.

In the future adding something like this may be possible, but only after a big
redesign in how the receiving side works.

-- 
You are receiving this mail because:
You are the QA Contact for the bug.