Rob Furr
2002-Mar-01 06:08 UTC
rsync: dumping core, breaking pipes, not understanding modules - why?
I'm running rsync 2.5.2_1 on two machines - a production server (athlon 1500+, 512mb, 160gb raid array, running FreeBSD 4.4-release) and a machine I'd like to use as a backup system (a p2/350, 192mb, 90gb, running FreeBSD 4.5-release.) I've got rsync running as a daemon on the production server, and I'd like to use rsync to back up the key directories on that machine to the backup system. However, it seems clear that I'm an idiot, and I'd massively appreciate any illumination as to how big an idiot I'm being: I'm using the following in my rsyncd.sh file to initialize rsync as a daemon on startup on the production server: --- !#/bin/sh /usr/local/bin/rsync --daemon --config=/etc/rsyncd.conf --- This gets rsync going just fine (I assume.) /etc/rsyncd.conf on the production server contains these lines: --- log file = /var/log/rsyncd.log pid file = /var/run/rsyncd.pid lock file = /var/run/rsync.lock uid = rsfurr # my own account - this will change once I get this working. gid = web # the group that the files I want to back up are owned by hosts allow = ___.___.___.___ # the ip of the backup server [upeg_web] path=/usr/local/share/doc/apache comment = the web directories read only = yes --- Here's the first problem: On the production server, typing rsync localhost:: results in this: upeg# rsync localhost:: upeg_web the upeg web directories! So I'm reasonably sure that rsyncd.conf is being read and understood. However, when I do this (on the backup server): rsync -a rsfurr@upeg.uncg.edu::upeg_web/ /bak/upeg_web/ I get this: @ERROR: access denied to upeg_web from unknown () rsync: connection unexpectedly closed (62 bytes read so far) rsync error: error in rsync protocol data stream (code 12) at io.c(140) and when I do *this*: rsync -a rsfurr@upeg.uncg.edu:upeg_web/ /bak/upeg_web/ I get *this*: link_stat upeg_web/. : No such file or directory client: nothing to do: perhaps you need to specify some filenames or the --recursive option? rsync error: partial transfer (code 23) at main.c(23) So, clearly, I'm either not issuing the right command (which I could have sworn I am), or I'm misconfiguring the module badly (which I don't *think* I am.) Additionally, when I do *this: rsync -a rsfurr@upeg.uncg.edu:/usr/local/share/doc/apache/ /bak/upeg_web/ it works - it transfers all the files that rsfurr is allowed to read to the backup server ... but when it's done, it does this: write failed flushing stdout buffer. write stdout: Broken pipe rsync error: partial transfer (code 23) at main.c(883) There's two problems with this: while it does seem to get all the files, I was under the impression that setting up a module (e.g., [upeg_web] ) would restrict rsync access to just that module. If this isn't the case, that's okay, but I am a bit puzzled. The other problem is that I'd really like to not get error messages cluttering up my inbox when I've got this going on a regular basis. Also - the partial transfer worries me. Does this mean that at some point, the program might only partially transfer the last file? On a related issue, I couldn't get ssh authentication working, for some reason (ssh -v revealed that no matter what key format I had ssh-keygen make, it still fell through all the way down to password authentication, which made me testy, and I'm under a bit of a time crunch here, so I didn't want to pursue *that* problem right now...) so I installed expect and wrote/swiped-from-a-web-page a short little expect script to handle the issue: --- #!/usr/local/bin/expect -f log_user 0 spawn /usr/local/bin/rsync -a rsfurr@upeg.uncg.edu:/usr/local/share/doc/apache/ /bak/upeg_web/ expect "password: " send "my_password_that_i'm_not_sharing_here/n" log_user 1 interact --- (and yes, I've got this in /root with 700 permissions, and the backup server in question has no services at all running. Well, it's got ssh running, but once it goes into production, that's going off, and then it'll be a closed box except for the occasional rsync run.) When I put this script in /etc/crontab in this way: 1 1 * * * root /foo/expect_script rsync not only gives me the broken pipe error, but it also dumps core and leaves big hornkin' core dump files around. Why? Can I do anything about it? I've checked the faq-o-matic and the list archives on this issue (well, I've checked them for *all* these issues, and either they're not addressed, or I am, as I said before, an idiot.) The reason I'm using rsync instead of a web crawler of some kind to do this backup is simple - there are other files on this server that need backup besides the web files. I'm just only worrying about getting the site backed up before I start worrying about the database directories and etc and var and like that. Any suggestions as to where I went wrong will be, as I said at the top of this mess, definitely appreciated. ---- Rob Furr LAAPhysics rsfurr@curie.uncg.edu
Martin Pool
2002-Mar-01 07:23 UTC
rsync: dumping core, breaking pipes, not understanding modules - why?
> link_stat upeg_web/. : No such file or directory > client: nothing to do: perhaps you need to specify some filenames or the > rsync error: partial transfer (code 23) at main.c(23)I think you're hitting a bug in 2.5.2. Would you mind please trying 2.5.3pre1 and letting us know if that fixes the crash? There should be a 2.5.3 release shortly. -- Martin
Rob Furr
2002-Mar-01 07:41 UTC
rsync: dumping core, breaking pipes, not understanding modules - why?
At 12:23 PM 2/28/2002 -0800, you wrote:> > link_stat upeg_web/. : No such file or directory > > client: nothing to do: perhaps you need to specify some filenames or the > > rsync error: partial transfer (code 23) at main.c(23) > >I think you're hitting a bug in 2.5.2. Would you mind please trying >2.5.3pre1 and letting us know if that fixes the crash? There should >be a 2.5.3 release shortly.Well, it was doing it with 2.5.0 and 2.5.1_1 as well - when I first encountered the problems, I poked until I was thoroughly confused, and then I went out and got what looked like the latest and greatest stable version in the hopes that would help. I'll give 2.5.3pre1 a shot, though. ---- Rob Furr LAAPhysics rsfurr@curie.uncg.edu