Xen users - Aug 2011 - OT: Staging startups for dependent machines

It''s a bit OT for this list, but I''m sure people here will
have dealt with it.

Does anyone have any suggestions for simple methods of staging 
machine (Linux in my case) startups when there are interdependencies 
? Eg, most of my servers rely on the DNS, so it would be nice if they 
could wait till the DNS is running before starting - otherwise I end 
up having to log in and either restart them or restart affected 
services.

Or would it be easier to have a site-wide hosts file with key hosts 
in it, and automatically distribute that ?

-- 
Simon Hobson

Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
author Gladys Hobson. Novels - poetry - short stories - ideal as
Christmas stocking fillers. Some available as e-books.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thursday, August 25, 2011 11:46:48 AM Simon Hobson
wrote:
> It''s a bit OT for this list, but I''m sure people here
will have dealt with
> it.
Dealt, sort of... Still working on a proper solution actually.
> Does anyone have any suggestions for simple methods of staging
> machine (Linux in my case) startups when there are interdependencies
> ? Eg, most of my servers rely on the DNS, so it would be nice if they
> could wait till the DNS is running before starting - otherwise I end
> up having to log in and either restart them or restart affected
> services.
> 
> Or would it be easier to have a site-wide hosts file with key hosts
> in it, and automatically distribute that ?
If it''s just DNS, then yes. But I have a few more dependencies which
I''d like
to resolve, like NFS-shares.

One way I''ve been thinking of is to delay startup of dependent
domU''s till the
"login" text appears in the /var/log/xen-consoles/* logfile for the
domU.

I configured different stage-init-scripts per domain to make it easier and 
have the reverse action (shutting down) correct.

For now, boot-up still requires manual work.

--
Joost

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> ------------------------------------------------------
> Date: Thu, 25 Aug 2011 11:46:48 +0100
> From: Simon Hobson <linux@thehobsons.co.uk>
> Subject: [Xen-users] OT: Staging startups for dependent machines
> To: Xen-users@lists.xensource.com
> Message-ID: <p0624083dca7bd6c91fab@simon.thehobsons.co.uk>
> Content-Type: text/plain; charset="us-ascii" ;
format="flowed"
>
> It''s a bit OT for this list, but I''m sure people here
will have dealt with it.
IMO, this shouldn''t be OT since its xen management related.
>
> Does anyone have any suggestions for simple methods of staging
> machine (Linux in my case) startups when there are interdependencies
> ? Eg, most of my servers rely on the DNS, so it would be nice if they
> could wait till the DNS is running before starting - otherwise I end
> up having to log in and either restart them or restart affected
> services.
I had this exact problem a few months ago deploying a xen install.
Tried looking for a solution online but found nothing much
interesting.

Then I came across this:
http://forums.novell.com/suse/suse-product-discussion-forums/suse-linux-enterprise/suse-linux-enterprise-server-sles/sles-virtualization/343455-how-change-order-automatic-start-virtual-machines-when-power-post1639925.html#post1639925

But it was written for SLES and uses a fixed delay between each VM. So
using my novice knowledge of bash, I came up with the attached init.d
script based off the one linked to above. Basically instead of just
waiting for a fixed delay, a time limit is used instead. The script
will continuously ping an address specified for that VM during this
time limit. If the address comes up, it moves on to the next VM. If it
doesn''t, you can configure it to fail or continue with the next VM
anyway. Also, adapted for Debian use.

I''m pretty sure there are some nasty bugs in there (pretty much
hammered it randomly till it worked, and stopped there), so the usual
disclaimer applies.

!!ALSO, do check through the stop part of the script since I wrote
that as a theoretical method to individually stop/save the VMs during
system reboot/shutdown. Didn''t really test it much though since I
decided that manually stopping the VMs would be much easier and safer.
It is particularly dangerous since it falls back to just killing the
VMs if all other soft methods fail. So if you''re unsure, just
remove/disable the stop part. Or use the older script that I''ve
attached as well that only does the staggered VM starting.

Hope this will be of help
>
> Or would it be easier to have a site-wide hosts file with key hosts
> in it, and automatically distribute that ?
>
> --
> Simon Hobson
>
> Visit http://www.magpiesnestpublishing.co.uk/ for books by acclaimed
> author Gladys Hobson. Novels - poetry - short stories - ideal as
> Christmas stocking fillers. Some available as e-books.


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users