Hi All, I need to write a script that I will manually start (or a cron job in future) but I need it to do a number of things in order one after another. How do i do that so everything gets dont as the steps depend on each other. Example: cd /system_backups/ tar cvf apache-conf.tar /etc/httpd/conf/* gzip -v9 apache-conf.tar tar cvf apache-data.tar /var/www/* gzip -v9 apache-data.tar then last step... tar cvf <current_date>-system_backup.tar <all> the gzip files above gzip -v9 <current_date>-system_backup.tar scp <current_date>-system_backup.tar.gz user at 10.0.0.1:/. etc...etc.... My questions: 1. How do I execute each statement and make sure subsequent statements are not executed until the previous is done? 2. How do I error check so if a step fails the script stops? 3. Since I run an SMTP Server on this box can I e-mail myself from bash the nightly results? 4. when I want to run the scp to send over the file to another machine for safety, how can I have it know the password to the machine I am scp'ing to? My mind is going crazy sort of with the things that I could do to protect myself in case of a system failure and making restoring easier. Can anyone provide insight for me? -Jason
Jason Todd Slack-Moehrle <mailinglists at ...> writes:> > Hi All, > > I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. How do i do that so everything gets dont as the steps depend > on each other. > > Example: > > cd /system_backups/ > > tar cvf apache-conf.tar /etc/httpd/conf/* > gzip -v9 apache-conf.tar > > tar cvf apache-data.tar /var/www/* > gzip -v9 apache-data.tar > > then last step... > tar cvf <current_date>-system_backup.tar <all> the gzip files above > gzip -v9 <current_date>-system_backup.tar > > scp <current_date>-system_backup.tar.gz user at ...:/. > etc...etc.... > > My questions: > 1. How do I execute each statement and make sure subsequent statements > are not executed until the previous is done?by using && example cd /system_backups/ && \ tar cvf apache-conf.tar /etc/httpd/conf/* && \ gzip -v9 apache-conf.tar> 2. How do I error check so if a step fails the script stops?I am afraid I do not have enough knowledge to help you in the above question> > 3. Since I run an SMTP Server on this box can I e-mail myself from > bash the nightly results?Yes> > 4. when I want to run the scp to send over the file to another machine > for safety, how can I have it know the password to the machine I am > scp'ing to?You cant You will have to ssh-keygen Thanks and Regards Rajagopal
On Thu, May 07, 2009 at 10:12:59PM -0700, Jason Todd Slack-Moehrle wrote:> Hi All, > > I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. How do i do that so everything gets dont as the steps depend > on each other. > > Example: > > cd /system_backups/ > > tar cvf apache-conf.tar /etc/httpd/conf/* > gzip -v9 apache-conf.tar > > tar cvf apache-data.tar /var/www/* > gzip -v9 apache-data.tar > > then last step... > tar cvf <current_date>-system_backup.tar <all> the gzip files above > gzip -v9 <current_date>-system_backup.tar > > scp <current_date>-system_backup.tar.gz user at 10.0.0.1:/. > etc...etc.... > > My questions: > 1. How do I execute each statement and make sure subsequent statements > are not executed until the previous is done?This will happen on its own unless you intentionally force a command into the background with &.> > 2. How do I error check so if a step fails the script stops?You can use && or || or check the status of $?. For example: tar cvfz apache-conf.tar.gz /etc/httpd/conf/* if [ $? != 0 ]; then echo "Error creating tar file." exit 255 fi $? contains the exit code of the previously executed command. Or something like... tar cvfz apache-conf.tar.gz /etc/httpd/conf/* || \ { echo "Problem creating tar file."; exit 255; } The first method is probably a little more clear.> > 3. Since I run an SMTP Server on this box can I e-mail myself from > bash the nightly results?Sure. If you call it from cron, anything on stdout will be emailed to the user running the job. Alternately you can call the "mail" command. You can also surround several commands and pipe their output to a command: ( echo "Test" echo "Test 2" ) | mail email at email.com> 4. when I want to run the scp to send over the file to another machine > for safety, how can I have it know the password to the machine I am > scp'ing to?You'll want to set up ssh keys for this.> > My mind is going crazy sort of with the things that I could do to > protect myself in case of a system failure and making restoring easier. > > Can anyone provide insight for me? > > -JasonRay
On Thu, May 07, 2009 at 10:12:59PM -0700, Jason Todd Slack-Moehrle wrote:> > I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. How do i do that so everything gets dont as the steps depend > on each other. > > My questions: > 1. How do I execute each statement and make sure subsequent statements > are not executed until the previous is done?Use the '&&' (logical "and") operator to pipeline the commands: cd /foo/bar && tar cvf /path/to/tarball This will only execute the "tar" command if the previous "cd" succeeded.> 2. How do I error check so if a step fails the script stops?Most commands set a return status upon completion or error; this value is returned to the shell in the "$?" variable. You could write the above as something like: cd /foo/bar if [ $? -ne 0 ] then echo "cd failed ($?) - bailing out" exit 1 fi tar cvf /path/to/tarball if [ $? -ne 0 ] then echo "tar failed ($?) - bailing out" exit 2 fi This checks the return status of each command, checks to see if it's not 0 (0 is "success") and if it is not alerts the user and exits with it's own error status.> 3. Since I run an SMTP Server on this box can I e-mail myself from > bash the nightly results?Yep. "echo error message" | mail -s "script failure" user at dom.ain Will mail the specified user a message containing "error message" with a subject of "script failure".> 4. when I want to run the scp to send over the file to another machine > for safety, how can I have it know the password to the machine I am > scp'ing to?You do this with public key authentication. "man ssh-keygen" for information on how to create a keypair.> My mind is going crazy sort of with the things that I could do to > protect myself in case of a system failure and making restoring easier.We've all been there.> Can anyone provide insight for me?Test everything. On regular schedules. If you have the hardware available simulate a disaster condition and restore your backups to identical hardware to prove (to yourself and to management) that your disaster recovery steps actually do work. John -- "I'm sorry but our engineers do not have phones." As stated by a Network Solutions Customer Service representative when asked to be put through to an engineer. "My other computer is your windows box." Ralf Hildebrandt <sxem> trying to play sturgeon while it's under attack is apparently not fun. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: not available URL: <http://lists.centos.org/pipermail/centos/attachments/20090508/34d64bdc/attachment-0004.sig>
Jason Todd Slack-Moehrle wrote:> Hi All, > > I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. How do i do that so everything gets dont as the steps depend > on each other.A few commands can be stuck together in the cron command. Separating them with ';'' makes them execute sequentially regardless of status; separating with '&&' will continue only if the previous command succeeds. For more than a few, put the commands in a script file> > Example: > > cd /system_backups/ > > tar cvf apache-conf.tar /etc/httpd/conf/* > gzip -v9 apache-conf.tarI'd: cd /etc/httpd/conf && tar -czvf /system_backups/apache-conf.tar.gz instead.> tar cvf apache-data.tar /var/www/* > gzip -v9 apache-data.tar > > then last step... > tar cvf <current_date>-system_backup.tar <all> the gzip files above > gzip -v9 <current_date>-system_backup.tar > > scp <current_date>-system_backup.tar.gz user at 10.0.0.1:/. > etc...etc.... > > My questions: > 1. How do I execute each statement and make sure subsequent statements > are not executed until the previous is done?Commands in a script execute sequentially unless you explicitly use an & to put them in the background.> 2. How do I error check so if a step fails the script stops?Each command will return a status in the $? variable that you can check. The shell philosophy is that there are lots of ways to fail but usually only one way to succeed, so the value 0 represents success, anything else is failure. You can use the 'test' operator (usually written as [, but read the test man page to see what it does), or just: command1 && command2 (runs command2 only if command1 succeeds) command1 || command2 (runs command2 only if command1 fails - you can exit this way)> 3. Since I run an SMTP Server on this box can I e-mail myself from > bash the nightly results?Yes, cron will automatically mail any output to the user that owns the cron job. Just add an alias for this local user (probably root) to make it go where you want.> 4. when I want to run the scp to send over the file to another machine > for safety, how can I have it know the password to the machine I am > scp'ing to?Use ssh keys instead.> My mind is going crazy sort of with the things that I could do to > protect myself in case of a system failure and making restoring easier. > > Can anyone provide insight for me?You'd probably be better off running something like backuppc (http://backuppc.sourceforge.net/) instead of rolling your own. And you might want some kind of version control system like subversion for your content and most important configuration files. -- Les Mikesell lesmikesell at gmail.com
Jason Todd Slack-Moehrle wrote:> Hi All, > > I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. How do i do that so everything gets dont as the steps depend > on each other. > > Example: > > cd /system_backups/CD = $(date +%Y-%m-%d) if ! tar cvf apache-conf.tar /etc/httpd/conf/* && gzip -v9 apache-conf.tar; then echo "FATAL ERROR 1" exit elif if ! tar cvf apache-data.tar /var/www/* && gzip -v9 apache-data.tar; then echo "FATAL ERROR 2" exit elif if ! tar cvf $CD-system_backup.tar apache-*.tar.gz && gzip -v9 $CD-system_backup.tar; then echo "FATAL ERR 3" exit elif scp $CD-system_backup.tar.gz user at 10.0.0.1:/. if this is run in a cronjob, any output will be mailed to the user who owns the cronjob. this user's email can be aliased via /etc/aliases to another account if needed.
> I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. ?How do i do that so everything gets dont as the steps depend > on each other. > > Example: > > cd /system_backups/# timestamp in YYYY-MM-DD format TIME_STAMP="$(date +%Y-%m-%d)" # ip address of remote host to scp files to REMOTE_IP="10.0.0.X" # comma separated list of users to receive notification alerts $AUDIENCE="user1 at example.com,user2 at example.com" cd /system_backups> tar cvf apache-conf.tar /etc/httpd/conf/* > gzip -v9 apache-conf.tartar cvfz apache-conf.tar.gz /etc/httpd/conf if [[ $? -ne 0 ]] then echo "Failed" | /bin/mail -s "Apache configuration directory backup failed" # no more execution since previous step did not succeed exit -1 fi> tar cvf apache-data.tar /var/www/* > gzip -v9 apache-data.tartar cvfz apache-data.tar.gz /var/www if [[ $? -ne 0 ]] then echo "Failed" | /bin/mail -s "Apache data directory backup failed" exit -1 fi> then last step... > tar cvf <current_date>-system_backup.tar <all> the gzip files above > gzip -v9 <current_date>-system_backup.tartar cvfz ${TIME_STAMP}-system-backup.tar.gz apache-conf.tar.gz apache-data.tar.gz if [[ $? -ne 0 ]] then echo "Failed" | /bin/mail -s "Apache system backup failed" exit -1 fi> > scp <current_date>-system_backup.tar.gz user at 10.0.0.1:/.scp ${TIME_STAMP}-system-backup.tar.gz user at 10.0.0.1:/ if [[ $? -ne 0 ]] then echo "Failed" | /bin/mail -s "Apache scp transfer failed" exit -1 else echo "Success" | /bin/mail -s "Apache system backup success" $AUDIENCE exit 0 fi> My questions: > 1. How do I execute each statement and make sure subsequent statements > are not executed until the previous is done?if [[ $? -ne 0 ]] statement does that for you> 2. How do I error check so if a step fails the script stops?exit X> 4. when I want to run the scp to send over the file to another machine > for safety, how can I have it know the password to the machine I am > scp'ing to?You need to setup password-less login first Try http://blogs.translucentcode.org/mick/archives/000230.html> Can anyone provide insight for me?This is a very basic script. You can use functions for error checking and make the code smaller etc.> -JasonHope this helps.
On Thu, May 07, 2009 at 10:12:59PM -0700, Jason Todd Slack-Moehrle wrote:> Hi All, > > I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. How do i do that so everything gets dont as the steps depend > on each other.In the script, before any command, add the line: set -e That'll cause the shell to abort if a command fails. -- lfr 0/0 -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 197 bytes Desc: not available URL: <http://lists.centos.org/pipermail/centos/attachments/20090508/ade24481/attachment-0004.sig>
On Thu, May 7, 2009 at 10:12 PM, Jason Todd Slack-Moehrle <mailinglists at mailnewsrss.com> wrote:> Hi All, > > I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. ?How do i do that so everything gets dont as the steps depend > on each other. >If you're asking these kinds of questions, a basic primer on how to use bash, and shell scripting in general, seems to be in order. It never hurts to have good manuals around if you are not thoroughly familiar with a particular paradigm, and it makes for handy reference in the future. You will (and did) get lots of help here, but you'll run into more and mroe of this as you go, and these kinds of issues are frequently covered in detail in a good manual. FTR, no, I don't use a shell manual any more. I've been writing shell and make scripts for over 20 years, and I still occasionally use the man page for details that elude me from time to time. Man has been a great tool for me, for this. HTH mhr
On Fri, May 8, 2009 at 1:12 AM, Jason Todd Slack-Moehrle <mailinglists at mailnewsrss.com> wrote:> > I need to write a script that I will manually start (or a cron job in > future) but I need it to do a number of things in order one after > another. ?How do i do that so everything gets dont as the steps depend > on each other. > > Example: > > cd /system_backups/ > > tar cvf apache-conf.tar /etc/httpd/conf/* > gzip -v9 apache-conf.tar > > tar cvf apache-data.tar /var/www/* > gzip -v9 apache-data.tar > > then last step... > tar cvf <current_date>-system_backup.tar <all> the gzip files above > gzip -v9 <current_date>-system_backup.tar > > scp <current_date>-system_backup.tar.gz user at 10.0.0.1:/. > etc...etc.... > > My questions: > 1. How do I execute each statement and make sure subsequent statements > are not executed until the previous is done? > > 2. How do I error check so if a step fails the script stops?After each command you can check the return code. For example: touch /tmp/foo RESULT_CODE=$? A non-zero error code will signify that some sort of error has occurred, so your script can check for these. You can then use the error code in a case statement and either print an error message then halt, or do something to resolve.> 3. Since I run an SMTP Server on this box can I e-mail myself from > bash the nightly results?Yes, you can use the mail/mailx commands to send yourself an email. Here's somethign that I use: #!/bin/sh # # $ID: $ LOG_RECIPIENTS="klowe at rccl.com" LOG_PATH=/usr/local/logs TODAY=`${DATE} -u +%Y%m%d` SCRIPT_NAME=`basename $0|sed -e 's/\.sh//'` SCRIPT_DESCRIPTION="Replace Me With Description" LOG_FILE=${LOG_PATH}/${SCRIPT_NAME}_${TODAY}.log MACHINE_NAME=`hostname` log_header(){ echo ----------------------------------------------------- echo ${SCRIPT_DESCRIPTION} on ${MACHINE_NAME} echo SCRIPT: $0 echo ----------------------------------------------------- echo echo Script started on `date` echo } log_footer(){ echo echo Script completed on `date` echo echo ----------------------------------------------------- echo ${SCRIPT_DESCRIPTION} on ${MACHINE_NAME} echo ----------------------------------------------------- } mail_logs(){ for recipient in ${LOG_RECIPIENTS}; do cat ${LOG_FILE} | ${MAILX} -s "${SCRIPT_DESCRIPTION} for ${TODAY}" ${recipient} done } exec 1>${LOG_FILE} 2>&1 log_header log_footer mail_logs> 4. when I want to run the scp to send over the file to another machine > for safety, how can I have it know the password to the machine I am > scp'ing to?Look up "ssh passwordless authentication" on how to generate a public/private key and install.