samba-bugs@samba.org
2005-Sep-24 14:22 UTC
[Bug 3116] New: large tar files: 1 gig size: retransmitted: rsync_rsh
https://bugzilla.samba.org/show_bug.cgi?id=3116
Summary: large tar files: 1 gig size: retransmitted: rsync_rsh
Product: rsync
Version: 2.6.6
Platform: x86
OS/Version: FreeBSD
Status: NEW
Severity: normal
Priority: P3
Component: core
AssignedTo: wayned@samba.org
ReportedBy: drinking.coffee@gmail.com
QAContact: rsync-qa@samba.org
Once a week, I'm performing an offsite backup. These files are each 1 gig in
size, average speedup is 1.18.
Rsync does the initial run, and then send some of the tar files again. I
initially thought the archives were too large, so I reduced them to 512M, but
had the same symtoms. My next step, is to install an rsync daemon on the remote
end.
When I do a second batch of mostly static content, I don't have any issues.
Below are my options:
#!/bin/sh
TERM="VT100"
RSYNC_RSH="ssh -c blowfish -2 -i /usr/username/.ssh/id_dsa"
export RSYNC_RSH
echo ' '
/bin/date
/bin/rsync -lrtv --stats --timeout=300 --delete-after \
/home/backup/psa username@hostname.net:/home/username/backup
source ssh version: OpenSSH_3.9p1
destination ssh version: OpenSSH_3.6.1p2
Output of rsync log:
Sat Sep 24 03:01:01 EDT 2005
building file list ... done
psa/
psa/psa.dump.full.monthly.aa
psa/psa.dump.full.monthly.ab
psa/psa.dump.full.monthly.ac
psa/psa.dump.full.monthly.ad
psa/psa.dump.full.monthly.ae
psa/psa.dump.full.monthly.af
psa/psa.dump.full.monthly.ag
psa/psa.dump.full.monthly.ah
psa/psa.dump.full.monthly.ai
psa/psa.dump.full.monthly.aj
psa/psa.dump.full.monthly.ak
psa/psa.dump.full.monthly.al
psa/psa.dump.full.monthly.am
psa/psa.dump.full.monthly.an
psa/psa.dump.full.monthly.ao
psa/psa.dump.full.monthly.aa
psa/psa.dump.full.monthly.ad
psa/psa.dump.full.monthly.ae
psa/psa.dump.full.monthly.ag
psa/psa.dump.full.monthly.ah
psa/psa.dump.full.monthly.ai
psa/psa.dump.full.monthly.aj
psa/psa.dump.full.monthly.ak
psa/psa.dump.full.monthly.al
psa/psa.dump.full.monthly.am
Number of files: 16
Number of files transferred: 25
Total file size: 15400460674 bytes
Total transferred file size: 26137878914 bytes
Literal data: 13011870082 bytes
Matched data: 13126008832 bytes
File list size: 230
File list generation time: 0.306 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 13016666532
Total bytes received: 18628384
sent 13016666532 bytes received 18628384 bytes 892125.72 bytes/sec
total size is 15400460674 speedup is 1.18
Source Files:
-rwx------ 1 root wheel 1G Sep 22 02:19 psa.dump.full.monthly.aa
-rwx------ 1 root wheel 1G Sep 22 02:33 psa.dump.full.monthly.ab
-rwx------ 1 root wheel 1G Sep 22 02:47 psa.dump.full.monthly.ac
-rwx------ 1 root wheel 1G Sep 22 03:00 psa.dump.full.monthly.ad
-rwx------ 1 root wheel 1G Sep 22 03:16 psa.dump.full.monthly.ae
-rwx------ 1 root wheel 1G Sep 22 03:29 psa.dump.full.monthly.af
-rwx------ 1 root wheel 1G Sep 22 03:44 psa.dump.full.monthly.ag
-rwx------ 1 root wheel 1G Sep 22 03:54 psa.dump.full.monthly.ah
-rwx------ 1 root wheel 1G Sep 22 04:04 psa.dump.full.monthly.ai
-rwx------ 1 root wheel 1G Sep 22 04:15 psa.dump.full.monthly.aj
-rwx------ 1 root wheel 1G Sep 22 04:24 psa.dump.full.monthly.ak
-rwx------ 1 root wheel 1G Sep 22 04:35 psa.dump.full.monthly.al
-rwx------ 1 root wheel 1G Sep 22 04:45 psa.dump.full.monthly.am
-rwx------ 1 root wheel 1G Sep 22 05:02 psa.dump.full.monthly.an
-rwx------ 1 root wheel 351M Sep 22 05:11 psa.dump.full.monthly.ao
Second Batch Output:
-rwx------ 1 backup backup 849K Sep 23 01:01 host1-backup-etc.zip
-rwx------ 1 backup backup 581M Sep 23 01:06 host1-backup-home-a2e.zip
-rwx------ 1 backup backup 155M Sep 23 01:07 host1-backup-home-f2j.zip
-rwx------ 1 backup backup 423M Sep 23 01:09 host1-backup-home-k2o.zip
-rwx------ 1 backup backup 341M Sep 23 01:10 host1-backup-home-p2t.zip
-rwx------ 1 backup backup 374M Sep 23 01:12 host1-backup-home-u2z.zip
-rwx------ 1 backup backup 13M Sep 23 01:01 host1-backup-mysql.zip
-rwx------ 1 backup backup 264M Sep 23 01:02 host1-backup-staff.zip
-rwx------ 1 backup backup 80M Sep 23 01:01 host1-backup-var.zip
-rwx------ 1 backup backup 29B Sep 23 01:12 host1-zip-end.txt
-rwx------ 1 backup backup 29B Sep 23 01:01 host1-zip-start.txt
-rwx------ 1 backup backup 836K Sep 23 01:01 host2-backup-etc.zip
-rwx------ 1 backup backup 513M Sep 23 01:04 host2-backup-home-a2e.zip
-rwx------ 1 backup backup 393M Sep 23 01:06 host2-backup-home-f2j.zip
-rwx------ 1 backup backup 406M Sep 23 01:08 host2-backup-home-k2o.zip
-rwx------ 1 backup backup 375M Sep 23 01:10 host2-backup-home-p2t.zip
-rwx------ 1 backup backup 130M Sep 23 01:10 host2-backup-home-u2z.zip
-rwx------ 1 backup backup 11M Sep 23 01:01 host2-backup-mysql.zip
-rwx------ 1 backup backup 133M Sep 23 01:01 host2-backup-staff.zip
-rwx------ 1 backup backup 43M Sep 23 01:01 host2-backup-var.zip
-rwx------ 1 backup backup 29B Sep 23 01:10 host2-zip-end.txt
-rwx------ 1 backup backup 29B Sep 23 01:01 host2-zip-start.txt
Sat Sep 24 07:06:26 EDT 2005
building file list ... done
archive/
archive/host1-backup-etc.zip
archive/host1-backup-home-a2e.zip
archive/host1-backup-home-f2j.zip
archive/host1-backup-home-k2o.zip
archive/host1-backup-home-p2t.zip
archive/host1-backup-home-u2z.zip
archive/host1-backup-mysql.zip
archive/host1-backup-staff.zip
archive/host1-backup-var.zip
archive/host1-zip-end.txt
archive/host1-zip-start.txt
archive/host2-backup-etc.zip
archive/host2-backup-home-a2e.zip
archive/host2-backup-home-f2j.zip
archive/host2-backup-home-k2o.zip
archive/host2-backup-home-p2t.zip
archive/host2-backup-home-u2z.zip
archive/host2-backup-mysql.zip
archive/host2-backup-staff.zip
archive/host2-backup-var.zip
archive/host2-zip-end.txt
archive/host2-zip-start.txt
Number of files: 23
Number of files transferred: 22
Total file size: 4449287013 bytes
Total transferred file size: 4449287013 bytes
Literal data: 390371025 bytes
Matched data: 4058915988 bytes
File list size: 528
File list generation time: 0.016 seconds
File list transfer time: 0.000 seconds
Total bytes sent: 391530333
Total bytes received: 1792478
sent 391530333 bytes received 1792478 bytes 456555.79 bytes/sec
total size is 4449287013 speedup is 11.31
--
Configure bugmail: https://bugzilla.samba.org/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are the QA contact for the bug, or are watching the QA contact.
Possibly Parallel Threads
Wisdom of the Ancients
