I'm not really what you would call a linux newbie, but I managed to run
into a problem with rsync that has me stumped.
I want to conduct un-attended backups of various files and directories
over about 3000 miles of Internet, so I chose sshd and rsync for the job.
I wrote a pair of scripts to do the job, the first to run incremental tars
on the localhost, the second to run the rsync client.
The first script is owned by root and the other owned by me ($USER). The
root file tars a bunch of directories and changes their ownerships and
permissions to $USER:$USER 644, then invokes the second script. The
second script makes the ssh connection and does an rsync backup to the
fileserver. Everything seems to be in order, but it doesn't work.
Here's the command that runs rsync on the client:
/usr/bin/rsync --verbose --progress --stats --compress --rsh=/usr/bin/ssh \
--recursive --times --perms --links --delete --include "*" \
/web/tars/* kmastin@216.138.230.185:/chroot/mcleod/
Here's a sample of the output:
kmastin.tar
635361280 100% 3.76MB/s 0:00:00
layer1.tar
24145920 100% 3.95MB/s 0:00:00
logs.tar
229990400 100% 1.59MB/s 0:00:00
mkstemp .kmastin.tar.yK1Qmu failed: Permission denied
mcleodlake.tar
85278720 100% 3.91MB/s 0:00:00
mlib.tar
528435200 100% 3.15MB/s 0:00:00
paulf.tar
25323520 100% 3.90MB/s 0:00:00
peace.tar
583680 100% 4.71MB/s 0:00:00
pguinane.tar
63293440 100% 1.94MB/s 0:00:00
mkstemp .layer1.tar.v3E0Mk failed: Permission denied
rdgroot.tar
70676480 100% 4.47MB/s 0:00:00
mkstemp .logs.tar.GvS2jd failed: Permission denied
mkstemp .mcleodlake.tar.mbt6rh failed: Permission denied
mkstemp .mlib.tar.E9pqVH failed: Permission denied
mkstemp .paulf.tar.lkvoJY failed: Permission denied
mkstemp .peace.tar.ldSPcm failed: Permission denied
mkstemp .pguinane.tar.UvpoSJ failed: Permission denied
mkstemp .rdgroot.tar.4XbXBG failed: Permission denied
rsync[19081] (sender) heap statistics:
arena: 348792 (bytes from sbrk)
ordblks: 3 (chunks not in use)
smblks: 0
hblks: 1 (chunks from mmap)
hblkhd: 266240 (bytes from mmap)
usmblks: 0
fsmblks: 0
uordblks: 341344 (bytes used)
fordblks: 7448 (bytes free)
keepcost: 3352 (bytes in releasable chunk)
Number of files: 22
Number of files transferred: 22
Total file size: 2483555627 bytes
Total transferred file size: 2483555627 bytes
Literal data: 814454539 bytes
Matched data: 1669101088 bytes
File list size: 446
Total bytes written: 407556250
Total bytes read: 10169052
wrote 407556250 bytes read 10169052 bytes 97839.40 bytes/sec
total size is 2483555627 speedup is 5.95
rsync error: partial transfer (code 23) at main.c(578)
The files on the server are unchanged, and a few files that I added as
controls did not transfer (timestamps and sizes are unchanged), although
they all took up bandwidth for transfers/updates.
I looked in the logs and did a google for this, but found nothing
relevant. There was one reference to the fact that rsyncd.conf spec's
rsync to run as uid/gid nobody:nobody, so I tried changing them to
kmastin:kmastin. No joy, same errors.
The second suggestion on the same thread was to make read only = false in
the rsyncd.conf file, which I did but also get the same errors.
Here's my rsync.conf file from the server:
log file = /var/log/rsyncd.log
pid file = /var/run/rsyncd.pid
lock file = /var/run/rsync.lock
[mcleod]
path = /chroot/mcleod
comment = Rsync Backup Server for mail.McLeodLake.com
uid = nobody
gid = nobody
read only = false
list = yes
auth users = kmastin
secrets file = /etc/rsyncd.scrt
hosts allow = mail.macleodlake.com
hosts deny = *
The rsyncd.scrt file:
kmastin:<passwd>
As you can see from the above outpuyt, I can connect and it looks like
things are working for the first minute or so, and then the errors start.
The filenames are about 15 characters each max (ie. /web/tars/home.tar ->
/chroot/mcleod/home.tar), so I think long filenames are not the issue
here.
The system is redhat-7.3, all up2date'd. I got so PO'd that I did rpm -e
rsync and installed from sources, thinking the latest version (2.5.6-2)
might be a fix, but again no joy.
Any help/indications would be appreciated.
On another note, I also noted that there's no entries into rsyncd.log. Any
ideas on that?
TIA
--
Keith Mastin
BeechTree Information Technology Services Inc.
Toronto, Canada
(416)696 6070
_______________________________________________
Valhalla-list mailing list
Valhalla-list@redhat.com
https://listman.redhat.com/mailman/listinfo/valhalla-list
--
Keith Mastin
BeechTree Information Technology Services Inc.
Toronto, Canada
(416)696 6070
On Fri, May 02, 2003 at 09:48:42PM -0400, Keith Mastin wrote:> I'm not really what you would call a linux newbie, but I managed to run > into a problem with rsync that has me stumped.I was going to let someone else respond but none have.> I want to conduct un-attended backups of various files and directories > over about 3000 miles of Internet, so I chose sshd and rsync for the job. > I wrote a pair of scripts to do the job, the first to run incremental tars > on the localhost, the second to run the rsync client.What do you mean by incremental tars? Using rsync, whole tars would make more sense. Then again, unless you are doing cross-platform why bother tarring at all.> The first script is owned by root and the other owned by me ($USER). The > root file tars a bunch of directories and changes their ownerships and > permissions to $USER:$USER 644, then invokes the second script. The > second script makes the ssh connection and does an rsync backup to the > fileserver. Everything seems to be in order, but it doesn't work. > > Here's the command that runs rsync on the client: > /usr/bin/rsync --verbose --progress --stats --compress --rsh=/usr/bin/ssh \ > --recursive --times --perms --links --delete --include "*" \ > /web/tars/* kmastin@216.138.230.185:/chroot/mcleod/You seem to have gone hog-wild with discrete options so i'll start there. --include "*" is the default unless you have excluded something. Are there any hardlinks to preserve? is /web/tars/* going to match any directories to warrant the --recursive option? Please turn off progress if you are going to post log output. -vv would be better for debugging.> Here's a sample of the output: > kmastin.tar > 635361280 100% 3.76MB/s 0:00:00 > layer1.tar > 24145920 100% 3.95MB/s 0:00:00 > logs.tar > 229990400 100% 1.59MB/s 0:00:00 > mkstemp .kmastin.tar.yK1Qmu failed: Permission denied > mcleodlake.tar > 85278720 100% 3.91MB/s 0:00:00 >[snip] The permission denied can be a result of not having permission to create the file or lacking permission to chmod the file. It doesn't look like it from the logs but are there any subdirs that you did a chmod 644 to?> The files on the server are unchanged, and a few files that I added as > controls did not transfer (timestamps and sizes are unchanged), although > they all took up bandwidth for transfers/updates. > > I looked in the logs and did a google for this, but found nothing > relevant. There was one reference to the fact that rsyncd.conf spec's > rsync to run as uid/gid nobody:nobody, so I tried changing them to > kmastin:kmastin. No joy, same errors. > > The second suggestion on the same thread was to make read only = false in > the rsyncd.conf file, which I did but also get the same errors.You aren't connecting to rsyncd so the rsyncd.conf is irrelevant and there will be log rsyncd.log entries. [snip]> As you can see from the above outpuyt, I can connect and it looks like > things are working for the first minute or so, and then the errors start. > > The filenames are about 15 characters each max (ie. /web/tars/home.tar -> > /chroot/mcleod/home.tar), so I think long filenames are not the issue > here. > > The system is redhat-7.3, all up2date'd. I got so PO'd that I did rpm -e > rsync and installed from sources, thinking the latest version (2.5.6-2) > might be a fix, but again no joy.Is that both ends or just the client? What pray tell is 2.5.6-2? There is no such official version. -- ________________________________________________________________ J.W. Schultz Pegasystems Technologies email address: jw@pegasys.ws Remember Cernan and Schmitt