Source is local to rsync, copying from a zfs file system, destination is remote over a dsl connection. Takes forever to just go through the unchanged files. Going the other way is not a problem, it takes a fraction of the time. Anybody seen that? Suggestions? -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081201/2eee6932/attachment.html>
Upstream when using DSL is much slower than downstream? Blake On Dec 1, 2008, at 7:42 PM, "Francois Dion" <fdion at atriumwindows.com> wrote:> Source is local to rsync, copying from a zfs file system, > destination is remote over a dsl connection. Takes forever to just > go through the unchanged files. Going the other way is not a > problem, it takes a fraction of the time. Anybody seen that? > Suggestions? > > > _______________________________________________ > zfs-discuss mailing list > zfs-discuss at opensolaris.org > http://mail.opensolaris.org/mailman/listinfo/zfs-discuss-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081201/97df8fed/attachment.html>
>>"Francois Dion" wrote: >> Source is local to rsync, copying from a zfs file system, >> destination is remote over a dsl connection. Takes forever to just >> go through the unchanged files. Going the other way is not a >> problem, it takes a fraction of the time. Anybody seen that? >> Suggestions? >De: Blake Irvin [mailto:blake.irvin at gmail.com] >Upstream when using DSL is much slower than downstream?No, that''s not the problem. I know ADSL is assymetrical. When there is an actual data transfer going on, the cpu drops to 0.2%. It''s only when rsync is doing its thing (reading, not writing) locally that it pegs the cpu. We are talking 15 minutes in one direction while in the other it looks like I''ll pass the 24 hours mark before the rsync is complete. And there were less than 100MB added on each side. BTW, the only other process I''ve seen that pegs the cpu solid for as long as it runs on my v480 is when I downloaded Belenix through a python script (btdownloadheadless). -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://mail.opensolaris.org/pipermail/zfs-discuss/attachments/20081202/21ab5526/attachment.html>
How are the two sides different? If you run something like ''openssl md5sum'' on both sides is it much faster on one side? Does one machine have a lot more memory/ARC and allow it to skip the physical reads? Is the dataset compressed on one side? -- This message posted from opensolaris.org
Francois Dion wrote:> >>"Francois Dion" wrote: > >> Source is local to rsync, copying from a zfs file system, > >> destination is remote over a dsl connection. Takes forever to just > >> go through the unchanged files. Going the other way is not a > >> problem, it takes a fraction of the time. Anybody seen that? > >> Suggestions? > >De: Blake Irvin [mailto:blake.irvin at gmail.com] > >Upstream when using DSL is much slower than downstream? > > No, that''s not the problem. I know ADSL is assymetrical. When there is > an actual data transfer going on, the cpu drops to 0.2%. It''s only when > rsync is doing its thing (reading, not writing) locally that it pegs the > cpu. We are talking 15 minutes in one direction while in the other it > looks like I''ll pass the 24 hours mark before the rsync is complete. And > there were less than 100MB added on each side. > > BTW, the only other process I''ve seen that pegs the cpu solid for as > long as it runs on my v480 is when I downloaded Belenix through a python > script (btdownloadheadless).Is the list of files long? rsync 3.0.X does not use a monolithic file list pull and uses less memory... Are you using a -c option or other option that causes rsync to checksum every block of all the files? Is the zfs file system compressed, so it has to decompress each block so that rsync can checksum it?