https://bugzilla.samba.org/show_bug.cgi?id=9624
Summary: rsync p2p extensions
Product: rsync
Version: 3.1.0
Platform: All
OS/Version: All
Status: NEW
Severity: enhancement
Priority: P5
Component: core
AssignedTo: wayned at samba.org
ReportedBy: mark at ibiblio.org
QAContact: rsync-qa at samba.org
Summary
======Add p2p enhancements to rsync to allow allow the rsync client to source
the
file from multiple endpoints. The rsync server would track and nominate a list
of valid mirrors.
Background
=========(Incredibly) large internet archive sites are moving towards phasing
out
anonymous rsync and FTP in favour of BitTorrent P2P systems which offload the
majority of the transfer load to other sites. With the use of torrents then P2P
clients can also help with the traffic offloading. P2P also allows continued
functionality in the cases where network traffic isn't available to the
original server, due to network outages or attacks.
The torrent approach is being investigated now (see
http://www.terasaur.org/browse/category/software ) and the issues being ironed
out (
https://docs.google.com/a/ibiblio.org/document/d/1d2gJYsesalyXkYvrnY9AOornHYeGlwcLX1gjR8VMFlw/edit
)
The problem is when torrents are working adequately the intent is to phase out
rsync and FTP. This causes issues in the unix world with auto-mirroring from
cron, access via a cli, fetching files using regexps, accessing torrents from
behind corporate etc firewalls and reinventing the agility of rsync into the
bittorrent clients.
Enhancement
==========rsync needs to extend its protocol to support more P2P functions.
- mirrors
- mirrors could nominate themselves when fetching content from the
central site, possibly with timeouts for caching their existance
- mirrors could download the mirror list using the P2P system and
re-share it to P2P clients
- clients
- clients attach to the server and negotiate P2P fetching. They
would then switch from classic rsync behaviour to P2P behaviour and
spread the traffic load across a number of mirrors, possibly client
peers in the future.
Searching google for "rsync p2p" returns a number of older projects
which have
attempted this. It's possible a lot of those efforts could be leveraged to
reduce the development burden.
Justification
============Adding these and other P2P enhancements will future proof rsync so
that archive
sites can utilize rsync more without worrying that excessive loads will
continue on the central site. The main traffic transfer cost can be shared with
sites around the world. rsync's value will remain high and those of us who
rely
on it will continue to be offered it.
--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug.
