rsync - Jun 2002 - Rsync ssh script execution fails under cron?

I've spent more than a day trying to write a script to backup my remote
server to a local machine using ssh. The script works perfectly when I execute
it directly (and I've gotten a number of variations to work okay), but I
can't get cron to successfully execute it. There is no result output, so it
seems cron is hanging on something, but I don't know what.

I'm running it as root, so it doesn't seem like it would be a file
permissions issue. The only time I've been able to get the script to execute
was when I used a dummy remote server name (of course it reported that it
failed, but cron did at least ran it).

I'm no expert, but I've written a few scripts for cron before without
any problem. I appreciate that this could be considered an issue with cron and
not rsync, but I thought users of rsync would probably have experience writing
such cron scripts.

------------------------------------------------------------------------------------
#!/bin/sh
# \
exec expect "$0" ${1+"$@"}
#
# -n dry run // --progress -v  for testing
# -z for file compression
log_user 0 
spawn time /usr/bin/rsync -anrultvPz --delete --exclude="vu/" -e
/usr/bin/ssh  --timeout 30 root@apache:/home/jhill/ /usr/local/bkup/home/jhill
expect "password: " 
send "secret\n" 
log_user 1 
interact 
-------------------------------------------------------------------------------------

Thanks for any suggestions,

Jeff Hill


----
Jeff Hill

After that warm response, I decided to try for another day to get a script 
working, to no avail.

Yes, a blank line and a "set -x" might be required (but certainly not 
always required for all cron scripts). It made no difference here, so far. 
As for the commands you didn't recognize (log_user, spawn, expect, send), 
they were part of the "expect" program the script executed, something
that
I thought was fairly common.

My intent was to avoid the security issues of logging into a remote server 
as root without a password or passphrase. At this stage, I'm willing to 
fall back to trying ssh keys without a passphrase. Again, easy enough to 
implement and launch from a script directly, but it isn't going anywhere 
under cron.

#! /bin/sh
#
/usr/bin/rsync -anrultvPz --delete -e 
/usr/bin/ssh  root@apache:"/home/jhill/"
"/usr/local/bkup/home/jhill"

Setting up the keys with a blank passphrase, the above works from the 
command line and requests neither password nor passphrase. But it still 
doesn't work from cron. I've tried variations using ssh-add and/or 
ssh-agent. I've tried running it under the Debian "crontab" that
allows for
setting the user and under the standard vixie cron.  I've tried running it 
as a straight command from cron, and of course as a bash script.

I'm using rsync 2.5.6cvs, Open_SSH 3.0.2p1 on Debian sid and woody servers, 
and I'm using RSA keys. I've looked at dozens of rsa-ssh scripts, but
most
of them are pushing data to a remote and I need to pull it down from the 
server. I've read how rsync encountered freezes with OpenSSH, in the past, 
but that appears to be resolved now.

  From what I've now read, it seems like I need both ssh-add and ssh-agent. 
I've created a script for cron to call and execute ssh-agent which then 
calls another script to execute ssh-add and rsync:

----------------------
rsync1:
#! /bin/sh
#
/usr/bin/ssh-agent /bin/bash /etc/rsync2

----------------------
rsync2:
#! /bin/sh
#
/usr/bin/ssh-add /root/.ssh/id_rsa
/usr/bin/rsync -anrultvPz --delete -e "/usr/bin/ssh  -A" 
root@apache:"/home/jhill/" "/usr/local/bkup/home/jhill"
----------------------

Of course, once again, it works fine from the command line but fails when I 
try it in the crontab.

At 12:03 AM 24/06/2002 +0200, Peter T. Breuer wrote:
>"A month of sundays ago Jeff Hill wrote:"
>(in immensely long lines that I have cut to 72 chars)
I didn't wrap lines to avoid problems in understanding the line breaks in 
the script.

So . . ., I'd still appreciate some assistance, but if you're just going
to
'bash' me ;), I can do without it.

Regards,

Jeff

> > I'm no expert, but I've written a few scripts for cron before
without
>
>You don't have an empty line at the end. You don't have a "set
-x".
>These mark you out as definitely not an expert. There is no need
>to guess .. just do these two things and you will have all the info you
>need.
>
> > any problem.  I appreciate that this could be considered an issue with
> > cron and not rsync, but I thought users of rsync would probably have
> > experience writing such cron scripts.
>
>Well, you know that the PATH is not the same as yours in cron, yes?
>So you should START by setting the PATH  in the script.
>
> > #!/bin/sh
> > # \
> > exec expect "$0" ${1+"$@"}
>
>Error. That's an exec. We never get beyond there. And surely tehre's
no
>attached console at this point, so how can it possibly work? What's
>this for? Did you mean to comment it out?
>
> > # -n dry run // --progress -v  for testing
> > # -z for file compression
> > log_user 0
>
>log_user is nothing I've ever heard of! What?
>
> > spawn time /usr/bin/rsync -anrultvPz --delete
--exclude="vu/" -e
> /usr/bin/ssh  --timeout 30 root@apache:/home/jhill/
/usr/local/bkup/home/jhill
>
>spawn? spawn? What? What's that? Are you trying to launch this in
>background? Then you want a "&". There's nothing to detach
from, so
>don't waorry about it!
>
>In any case, you want to set up ssh to let you log in as root without a
>passwd. That's all.
>
>
>
> > expect "password: "
> > send "secret\n"
>
>Eh? This can't possibly do anything. This is part of a script that you
>think is taklking to something, and it is not.
>
>
>
> > log_user 1
> > interact
>
>Eh?
>
>
>It looks incredibly confused. I think you are thinking of launching
>something in the background while you deal with passwd stuff in the
>foreground. That is completely mistaken. It makes no sense.
>
>
>Set ssh to permit login without passwd for this user. That's all. It
>only needs RSA authenticatuon to be sufficient, and to have each
>others kerys in authorized_keys in .ssh both sides (well, one side,
>but you may as well have both).
>
>
>Peter

Jeff:  Sorry you got flamed.  I can't imagine taking the trouble to 
respond to a strangers question with only the purpose of belittling him 
(though I do sometimes include manpage headers with the quotes from the 
man pages, as a non-so-subtle "RTFM").  I haven't been very
diligent about
keeping up with the list.  I can't find your original post, so i've got
to
go with what's left here.

I've used expect only incidentally, so I'm no expert there, but I often 
find that programs behave differently when STDIO is a socket, fifo, pipe 
... whatever your systems cron provides.  I usually completely wrap things 
I run from cron, as in putting them inside
++++++++++++++++++++++++++
{
script code
} </dev/null >&0 2>&0
++++++++++++++++++++++++++
Inside that, I may be redirecting STDIN, STDOUT, STDERR, as necessary.  It 
just sets to a consistent environment.  I hope that helps.

I've got to learn expect myself.  I've got a couple of applications 
wherein it just might excel.

Good luck.

Tim Conway
tim.conway@philips.com
303.682.4917 office, 3039210301 cell
Philips Semiconductor - Longmont TC
1880 Industrial Circle, Suite D
Longmont, CO 80501
Available via SameTime Connect within Philips, n9hmg on AIM
perl -e 'print pack(nnnnnnnnnnnn, 
19061,29556,8289,28271,29800,25970,8304,25970,27680,26721,25451,25970), 
".\n" '
"There are some who call me.... Tim?"




Jeff Hill <jhill@hrpost.com>
Sent by: rsync-admin@lists.samba.org
06/24/2002 01:31 PM

 
        To:     ptb@it.uc3m.es
        cc:     rsync@lists.samba.org
(bcc: Tim Conway/LMT/SC/PHILIPS)
        Subject:        Re: Rsync ssh script execution fails under cron?
        Classification: 



After that warm response, I decided to try for another day to get a script 

working, to no avail.

Yes, a blank line and a "set -x" might be required (but certainly not 
always required for all cron scripts). It made no difference here, so far. 

As for the commands you didn't recognize (log_user, spawn, expect, send), 
they were part of the "expect" program the script executed, something
that

I thought was fairly common.

My intent was to avoid the security issues of logging into a remote server 

as root without a password or passphrase. At this stage, I'm willing to 
fall back to trying ssh keys without a passphrase. Again, easy enough to 
implement and launch from a script directly, but it isn't going anywhere 
under cron.

#! /bin/sh
#
/usr/bin/rsync -anrultvPz --delete -e 
/usr/bin/ssh  root@apache:"/home/jhill/"
"/usr/local/bkup/home/jhill"

Setting up the keys with a blank passphrase, the above works from the 
command line and requests neither password nor passphrase. But it still 
doesn't work from cron. I've tried variations using ssh-add and/or 
ssh-agent. I've tried running it under the Debian "crontab" that
allows
for 
setting the user and under the standard vixie cron.  I've tried running it 

as a straight command from cron, and of course as a bash script.

I'm using rsync 2.5.6cvs, Open_SSH 3.0.2p1 on Debian sid and woody 
servers, 
and I'm using RSA keys. I've looked at dozens of rsa-ssh scripts, but
most

of them are pushing data to a remote and I need to pull it down from the 
server. I've read how rsync encountered freezes with OpenSSH, in the past, 

but that appears to be resolved now.

  From what I've now read, it seems like I need both ssh-add and 
ssh-agent. 
I've created a script for cron to call and execute ssh-agent which then 
calls another script to execute ssh-add and rsync:

----------------------
rsync1:
#! /bin/sh
#
/usr/bin/ssh-agent /bin/bash /etc/rsync2

----------------------
rsync2:
#! /bin/sh
#
/usr/bin/ssh-add /root/.ssh/id_rsa
/usr/bin/rsync -anrultvPz --delete -e "/usr/bin/ssh  -A" 
root@apache:"/home/jhill/" "/usr/local/bkup/home/jhill"
----------------------

Of course, once again, it works fine from the command line but fails when 
I 
try it in the crontab.

At 12:03 AM 24/06/2002 +0200, Peter T. Breuer wrote:
>"A month of sundays ago Jeff Hill wrote:"
>(in immensely long lines that I have cut to 72 chars)
I didn't wrap lines to avoid problems in understanding the line breaks in 
the script.

So . . ., I'd still appreciate some assistance, but if you're just going
to 
'bash' me ;), I can do without it.

Regards,

Jeff

> > I'm no expert, but I've written a few scripts for cron before
without
>
>You don't have an empty line at the end. You don't have a "set
-x".
>These mark you out as definitely not an expert. There is no need
>to guess .. just do these two things and you will have all the info you
>need.
>
> > any problem.  I appreciate that this could be considered an issue with
> > cron and not rsync, but I thought users of rsync would probably have
> > experience writing such cron scripts.
>
>Well, you know that the PATH is not the same as yours in cron, yes?
>So you should START by setting the PATH  in the script.
>
> > #!/bin/sh
> > # \
> > exec expect "$0" ${1+"$@"}
>
>Error. That's an exec. We never get beyond there. And surely tehre's
no
>attached console at this point, so how can it possibly work? What's
>this for? Did you mean to comment it out?
>
> > # -n dry run // --progress -v  for testing
> > # -z for file compression
> > log_user 0
>
>log_user is nothing I've ever heard of! What?
>
> > spawn time /usr/bin/rsync -anrultvPz --delete
--exclude="vu/" -e
> /usr/bin/ssh  --timeout 30 root@apache:/home/jhill/ 
/usr/local/bkup/home/jhill
>
>spawn? spawn? What? What's that? Are you trying to launch this in
>background? Then you want a "&". There's nothing to detach
from, so
>don't waorry about it!
>
>In any case, you want to set up ssh to let you log in as root without a
>passwd. That's all.
>
>
>
> > expect "password: "
> > send "secret\n"
>
>Eh? This can't possibly do anything. This is part of a script that you
>think is taklking to something, and it is not.
>
>
>
> > log_user 1
> > interact
>
>Eh?
>
>
>It looks incredibly confused. I think you are thinking of launching
>something in the background while you deal with passwd stuff in the
>foreground. That is completely mistaken. It makes no sense.
>
>
>Set ssh to permit login without passwd for this user. That's all. It
>only needs RSA authenticatuon to be sufficient, and to have each
>others kerys in authorized_keys in .ssh both sides (well, one side,
>but you may as well have both).
>
>
>Peter

-- 
To unsubscribe or change options: 
http://lists.samba.org/mailman/listinfo/rsync
Before posting, read: http://www.tuxedo.org/~esr/faqs/smart-questions.html