rsync - Apr 2010 - DO NOT REPLY [Bug 7349] New: Add --limit-time=<seconds> and --fit-time=[seconds] options

https://bugzilla.samba.org/show_bug.cgi?id=7349

           Summary: Add --limit-time=<seconds> and --fit-time=[seconds]
                    options
           Product: rsync
           Version: 3.1.0
          Platform: All
        OS/Version: All
            Status: NEW
          Severity: enhancement
          Priority: P3
         Component: core
        AssignedTo: wayned at samba.org
        ReportedBy: bugzilla.samba.simon at arlott.org
         QAContact: rsync-qa at samba.org


More (cheaper) bandwidth is available between 2am and 6am, so it is the ideal
time to run rsync for backup purposes. However, in order to allow the maximum
possible time for rsync to complete it must be started at 2am even if it would
only take a shorter period of time (e.g. 2 hours). If there are a lot of users
(of an ISP) doing this then there will be a large undesireable spike at 2am.

Ideally they would all schedule times so they don't overlap but this is
impractical. Instead, rsync could automatically reduce the bandwidth limit so
that the transfer takes the full time. This would require determining the size
of all files to be transferred in advance and then adjusting the bandwidth
limit based on remaining time.

--limit-time would limit the maximum duration of the process
--fit-time would lower the --bwlimit (which may not be set; this will
dynamically change if it's not going fast enough) so that the transfer takes
the intended time

The default for --fit-time should be 95% of the --limit-time, allowing some
time for the inexact nature of adjusting bandwidth dynamically so that the
transfer can complete successfully. Transfers should never exceed --bwlimit
even if the time would not be met. Compression is likely to complicate the
situation so that transfers get progressively slower, but the assumption is
that these files are already compressed so this will not be used.

Any checksum-based options would need to be disabled unless they could avoid
reading files twice (once at the start and once at the transfer stage). Partial
transfers and appending are probably an implicit requirement of limiting the
transfer time. Currently I have a cron job kill rsync at 6am.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7349





------- Comment #1 from bugzilla.samba.simon at arlott.org  2010-04-11 05:32 CST
-------
An additional option, which requires --limit-time (and optionally --fit-time)
as well as --bwlimit to be set accurately to the expected minimum available
bandwidth:

--delay-start=<seconds> which would delay starting of the actual transfer
by up
to the specified amount of time if it is expected to then complete before
--limit-time

If --fit-time is specified, the transfer should also be slowed down if it will
complete within the given time.

Example:
Bandwidth between 18:00 and 09:00 is "off peak", and between 02:00 and
06:00 is
"night". Ideally transfers should complete during "night"
but may use all of
the "off peak" period. The actual size of the transfer varies from day
to day.

Run rsync with --limit-time=54000 --fit-time=14400 --bwlimit=200
--delay-start=28800. The limit is 15 hours (the "off peak" period),
but the
transfer may be delayed by up to 8 hours (bringing it to 02:00) if it would
still complete by 09:00. Based on the bandwidth limit set, if the transfer
could be completed by 06:00 at a slower rate, the bandwidth would be
dynamically reduced as in the original proposal.

If the transfer would not finish by 09:00 when starting at 02:00 it would
adjust the start delay backwards, e.g. to 01:00 or 18:00 if necessary or there
is too much data.


-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.

https://bugzilla.samba.org/show_bug.cgi?id=7349

--- Comment #2 from roland <devzero at web.de> 2013-03-09 12:08:37 UTC ---
have a look at https://bugzilla.samba.org/show_bug.cgi?id=7120 , as it
describes a method of externally adjusting bandwidth at runtime without
changing rsync at all

-- 
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.