Xen users - Feb 2007 - Problem with automatic VM (domU) creation

Hello,

I worte a little perl script, which asks me how many xen guests I want to
create, e.g. 3 , and the
it creates 3 xen-guests automatically. The basis is an installed
debian-guest-vm, which would be
just renamed and copied, and its config file at /etc/xen.

My Question:
I successfully create the new config file, with a new hostname, and new
ipaddress, but when I want
to automatically start the new vm with:

xm xreate <newconfigfile>

it boots up, but its hostname and ip address is not the (new) one i configured
in the
/etc/xen/<newconfigfile>, but the one of the basis image!

Wheres the mistake?

Thank you,

Frank







_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On 2/22/07, Frank D <mailing@fdoelitzscher.de>
wrote:
> I worte a little perl script, which asks me how many xen guests I want to
create, e.g. 3 , and the
> it creates 3 xen-guests automatically. The basis is an installed
debian-guest-vm, which would be
> just renamed and copied, and its config file at /etc/xen.
check out xen-tools. It does not have a feature to copy a vm 3 times
with one call, but it can use image templates and adjust them,
probably it will also handle setting the hostname right.

It is made by a debian developer, mainly for debian, but it runs quite
nice on fedora and suse, too.
> it boots up, but its hostname and ip address is not the (new) one i
configured in the
> /etc/xen/<newconfigfile>, but the one of the basis image!
>
> Wheres the mistake?
Not enough doc reading about Xen and it''s architecture, and the config
parameters :)

xm doesn''t change the settings inside of a guest.
The "name" setting in a VM config is NOT the dns hostname, and not the
/etc/hostname.
Often, one wants to have it like that, but this is not Xen''s job.

Henning

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Feb 22, 2007 at 08:30:56PM +0530, Ligesh wrote:
> > The "name" setting in a VM config is NOT the dns hostname,
and not the
> > /etc/hostname.
> > Often, one wants to have it like that, but this is not Xen''s
job.
> 
> > but this is not Xen''s job.
> 
>  And how''s that?
  The simple solution here would be to specify your xen guests use
 DHCP, then hardwire a MAC address in the Xen configuration file.

  This would allow your DHCP server to set the name of the domU
 as part of its job of giving out a lease.

Steve
-- 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Feb 22, 2007 at 02:54:46PM +0100, Henning Sprang
wrote:
> Not enough doc reading about Xen and it''s architecture, and the
config
> parameters :)
> 
> xm doesn''t change the settings inside of a guest.
> The "name" setting in a VM config is NOT the dns hostname, and
not the
> /etc/hostname.
> Often, one wants to have it like that, but this is not Xen''s job.
> but this is not Xen''s job.
 And how''s that? It is not the job of xen:the hyperVisor of course, but
it should be part of the xen, the platform. Does XenEnterprise do this at least?


 Thanks.





_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Feb 22, 2007 at 08:52:15PM +0530, Ligesh wrote:
>   For linux, the easier way is to mount the domU and create
> the network configuration files from the dom0 at every boot time. 
 I think we probably view "easier" in a different way.

  Really though I don''t think it is xen''s job to setup the
hostname
 of managed instances and there isn''t a really portable way of doing
 it when you consider Windows, etc.

  Personally I''d setup static details at creation time and not worry
 about it - but if you did want to be slightly dynamic then using
 DHCP is probably a more standard approach with less updates to make
 for different distributions than poking the filesystem.
 (i.e. /etc/network/interfaces on Debian, vs. /etc/conf.d/ on Gentoo,
  vs. /etc/sysconfig/network-scripts/ on RedHat.)

Steve
-- 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Feb 22, 2007 at 03:00:40PM +0000, Steve Kemp
wrote:
> 
>   The simple solution here would be to specify your xen guests use
>  DHCP, then hardwire a MAC address in the Xen configuration file.
> 
> Steve
  For linux, the easier way is to mount the domU and create the network
configuration files from the dom0 at every boot time. You can do

-------
 mount domU
 cp network-management  domU
 chroot domU sh network-managment <ipaddresses>
 sync
 umount domU
 xm create domU
 -------

  This way you get full static ipaddresses, indistinguishable from a normal
setup, but fully controlled from the dom0.


>   This would allow your DHCP server to set the name of the domU
>  as part of its job of giving out a lease.
 OK, How do you configure multiple ipaddresses for the same ethernet card using
dhcp? As far as redhat is concerned, the eth0:0 will not support dhcp. I
don''t need it for linux, but I think there''s no other way for
windows.

 Thanks.



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, 2007-02-22 at 20:30 +0530, Ligesh wrote:
> On Thu, Feb 22, 2007 at 02:54:46PM +0100, Henning Sprang wrote:
> > Not enough doc reading about Xen and it''s architecture, and
the config
> > parameters :)
> > 
> > xm doesn''t change the settings inside of a guest.
> > The "name" setting in a VM config is NOT the dns hostname,
and not the
> > /etc/hostname.
> > Often, one wants to have it like that, but this is not Xen''s
job.
> 
> > but this is not Xen''s job.
> 
>  And how''s that? It is not the job of xen:the hyperVisor of
course, but it should be part of the xen, the platform.
> Does XenEnterprise do this at least?
> 
> 
>  Thanks
cat << EOF > /pathto/mounted/vm/fs/etc/hosts

127.0.0.1  localhost.localdomain localhost
1.2.3.4    ${VMLONGNAME} ${VMSHORTNAME}
EOF

cat << EOF > /pathto/mounted/vm/fs/etc/network/interfaces

auto eth0
iface eth0 inet static
 address ${VM_IPADDR}
  ... and so on
EOF

Unmount, and boot it.

It is very, very easy to make scripts to setup guests. Parse a .csv file
that you obtain centrally if you wish, or spend the 3 - 5 hours needed
to do something with php/mysql centrally and use lynx or whatever else
you want to fetch the values to feed the scripts you make. 

There is almost an infinite amount of ways to have your guests ready to
roll in just a few minutes with a little scripting.

Xen is like Leggos. Snap together how you like and incorporate it how
you like, or use one of any number of tools to do this. I think the
reason so many system integrators really love Xen is because its so easy
to work into any scenario. 

In 10 seconds, I can create a paravirtualized guest with networking
setup, filesystems formatted and ready, root password, hostname,
everything set that needs to be.

I fill in about a dozen values in a csv file , run one script and I''m
done. That script took all of .. 2 hours to throw together. Elegant,
no .. effective predictable and reliable, yes :)

Its really not Xen''s job to do this. I use Xen mostly in distributed
grids where my domname means something completely different than the
system''s hostname. So many uses for xen way beyond the simple creating
of PV guests, if they started doing too much for me, I''d just have
quite
a bit of stuff to un-do.

Xen (open source) is a set of very powerful tools that need somewhat
skilled hands to use them. I think if the project shifted focus such as
you suggest, it would lose some of its support and community.

Best,
--Tim


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Feb 22, 2007 at 03:17:04PM +0000, Steve Kemp
wrote:
> On Thu, Feb 22, 2007 at 08:52:15PM +0530, Ligesh wrote:
> 
> >   For linux, the easier way is to mount the domU and create
> > the network configuration files from the dom0 at every boot time. 
> 
>  I think we probably view "easier" in a different way.
 :-) To me ''easy'' is something that might be hard to setup
initially, but will not have any run time issues. DHCP is actually the generic
solution, but it still needs the network to function properly to work. With
mount and hard network setup, you have 100% guarantee the network is setup,
whatever else might fail.


[quote] 
> 
>   Really though I don''t think it is xen''s job to setup
the hostname
>  of managed instances and there isn''t a really portable way of
doing
>  it when you consider Windows, etc.
[/quote] 

 Not Xen the hypervisor''s. It _can''t_, actually. But it is the
duty of any good management tool (xen, the platform). DHCP is a good idea, and
considering that you can''t still mount ntfs partitions in linux, DHCP
is the only way to go for windows guests.


[quote] 
> 
>   Personally I''d setup static details at creation time and not
worry
>  about it - but if you did want to be slightly dynamic then using
>  DHCP is probably a more standard approach with less updates to make
>  for different distributions than poking the filesystem.
>  (i.e. /etc/network/interfaces on Debian, vs. /etc/conf.d/ on Gentoo,
>   vs. /etc/sysconfig/network-scripts/ on RedHat.)
[/quote] 
 
  Actually I am doing network management of linux guests this way. But for
windows I need DHCP, and I couldn''t find a way to assign multiple
ipaddresses to the same ethernet card using DHCP. Is this actually possible? A
net search and cursory read through the manual (searched for
''multiple''), didn''t yield any results. On redhat I
think it may not be possible at all, since redhat very specifically says that
eth0:0 will not support dhcp, but I don''t know about windows. You have
any idea whether windows will configure itself with multiple ipaddresses (for
the same ethernet) via dhcp?

  Thanks.




_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Feb 22, 2007 at 11:29:13PM +0800, Tim Post
wrote:
> On Thu, 2007-02-22 at 20:30 +0530, Ligesh wrote:
> > On Thu, Feb 22, 2007 at 02:54:46PM +0100, Henning Sprang wrote:
> > > Not enough doc reading about Xen and it''s architecture,
and the config
> > > parameters :)
> > > 
> Xen (open source) is a set of very powerful tools that need somewhat
> skilled hands to use them. I think if the project shifted focus such as
> you suggest, it would lose some of its support and community.
Hey, I had said: "It is not part of xen the hypervisor, but xen as a
platform". For instance, xenEnterprise. Or any good management tool built
on Xen should provide the means to do a little more than mere create/delete.
Your solution is actually nice.  The problem with your templated osimages is
that the scripts can be run only once. ONce you have setup the domU, then the
placeholder variables all disappear and then a _reconfiguration_ is not
possible. Other than that, the idea is quite innovative.

 A better idea would be create template files separately:

 /etc/hosts.xentemplate

 So a generic script can scan ''/etc'' for all files ending with
xentemplate then configure the values and save them without the suffix.

 for i in `find -name ''*.xentemplate'' /mount/domU/etc/` ; do
	 sed ''s/$IPADDRESS/1.2.3.4/g'' < $i > `basename $i
.xentempate`
	 sed ''s/$HOSTNAME/host.domain.com/g'' < $i > `basename
$i .xentempate`
 done

 This is actually a wonderful idea. That way, the script remains generic, and
each distro can prepare itself Xen with the proper template files.
 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, 2007-02-22 at 21:25 +0530, Ligesh wrote:
> On Thu, Feb 22, 2007 at 11:29:13PM +0800, Tim Post wrote:
> > On Thu, 2007-02-22 at 20:30 +0530, Ligesh wrote:
> > > On Thu, Feb 22, 2007 at 02:54:46PM +0100, Henning Sprang wrote:
> > > > Not enough doc reading about Xen and it''s
architecture, and the config
> > > > parameters :)
> > > > 
> > Xen (open source) is a set of very powerful tools that need somewhat
> > skilled hands to use them. I think if the project shifted focus such
as
> > you suggest, it would lose some of its support and community.
> 
> Hey, I had said: "It is not part of xen the hypervisor, but xen as a
platform".
> For instance, xenEnterprise. Or any good management tool built on Xen
should provide
> the means to do a little more than mere create/delete. Your solution is
actually nice.
> The problem with your templated osimages is that the scripts can be run
only once.
> ONce you have setup the domU, then the placeholder variables all disappear
and then
> a _reconfiguration_ is not possible. Other than that, the idea is quite
innovative.
I use quite a bit more ''helpers'', reconfiguration is very
easy. I was
just giving a rough idea of how I initially "pave" guests.
> 
>  A better idea would be create template files separately:
> 
>  /etc/hosts.xentemplate
> 
>  So a generic script can scan ''/etc'' for all files ending
with xentemplate then configure the values
> and save them without the suffix.
> 
>  for i in `find -name ''*.xentemplate'' /mount/domU/etc/` ;
do
> 	 sed ''s/$IPADDRESS/1.2.3.4/g'' < $i > `basename $i
.xentempate`
> 	 sed ''s/$HOSTNAME/host.domain.com/g'' < $i >
`basename $i .xentempate`
>  done
> 
>  This is actually a wonderful idea. That way, the script remains generic,
and each distro can prepare
> itself Xen with the proper template files.
I agree with everything but sed, but I''m ultra paranoid about line
editors that malloc() way more than they need to because they have no
idea whats in your file :) On dom-0 contiguous cache is prime
real-estate, I treat it like I would any embedded / small memory model
system and code accordingly.

In fact, I helped write grawk, our utility that combines some of the
most frequently grep + awk pipes together into something quite a bit
smaller : http://dev1.netkinetics.net/grawk/ for log sawing and easy
string extraction from delimited files / output scraping. Feel free to
use it instead of sed for such a simple task. Patches for additional
regex are of course welcome, provided they do not significantly increase
the size of the executable and malloc() wisely. 

That being said, I do in fact realize that I am *over paranoid* about
it :) I freely admit that I suffer from many odd ''ticks'' that
cause me
to obsess about every single byte of RAM in every single system I
maintain. Its a wonder I get anything done.

Best,
--Tim



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, Feb 23, 2007 at 12:25:55AM +0800, Tim Post
wrote:
> In fact, I helped write grawk, our utility that combines some of the
> most frequently grep + awk pipes together into something quite a bit
> smaller : http://dev1.netkinetics.net/grawk/ for log sawing and easy
> string extraction from delimited files / output scraping. Feel free to
> use it instead of sed for such a simple task. Patches for additional
> regex are of course welcome, provided they do not significantly increase
> the size of the executable and malloc() wisely. 
> 
> That being said, I do in fact realize that I am *over paranoid* about
> it :) I freely admit that I suffer from many odd ''ticks''
that cause me
> to obsess about every single byte of RAM in every single system I
> maintain. Its a wonder I get anything done.
> 
 After becoming almost irrelevant, RAM is again becoming a very critical
commodity. In the case of single machines, saving a couple of bytes ram was
pretty much absurd, especially with enough swap, the affect would only be
temporary, and there wasn''t any real price difference between a 52MB
system and a 512MB system. But with virtualization, 32MB means you can have 10
times the number of virtual machines, and the price actually goes down 1/10th.

 ONce virtualization picks up, memory optimized systems are going to make a come
back. And it seems to be an entirely new market, since all mainstream
applications seems to have given up on memory optimization. For instance, the
three of the most popular applicatins: bind, spamassassin, apache are all memory
hogs, using memory unncessarily. I mean, why the heck do you need 200MB for
running a web server? That''s quite insane.

 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, 2007-02-22 at 22:18 +0530, Ligesh wrote:
> On Fri, Feb 23, 2007 at 12:25:55AM +0800, Tim Post wrote:
> > In fact, I helped write grawk, our utility that combines some of the
> > most frequently grep + awk pipes together into something quite a bit
> > smaller : http://dev1.netkinetics.net/grawk/ for log sawing and easy
> > string extraction from delimited files / output scraping. Feel free to
> > use it instead of sed for such a simple task. Patches for additional
> > regex are of course welcome, provided they do not significantly
increase
> > the size of the executable and malloc() wisely. 
> > 
> > That being said, I do in fact realize that I am *over paranoid* about
> > it :) I freely admit that I suffer from many odd
''ticks'' that cause me
> > to obsess about every single byte of RAM in every single system I
> > maintain. Its a wonder I get anything done.
> > 
>  After becoming almost irrelevant, RAM is again becoming a very critical
commodity.
> In the case of single machines, saving a couple of bytes ram was pretty
much absurd,
> especially with enough swap, the affect would only be temporary, and there
wasn''t any
> real price difference between a 52MB system and a 512MB system.
I think its very much like SUVs (which I call stupid useless vehicles).
They don''t seem so attractive anymore when the price of fueling them
continues to climb. Some applications and services are SUVs.

My first ''real'' machines were 286''s with 2 MB ( 2 x
1MB SIPPs), so I
really do see your point. Those machines housed my dial up BBS really
well. 

When I saw a 64 MB chip, I barked "Who the heck could even take
advantage of that for home use?", like many other people did. 
> But with virtualization, 32MB means you can have 10 times the number of
virtual machines,
> and the price actually goes down 1/10th.
And to achieve true virtualization, you want isolated ram. You hit it
right on the head.
>  ONce virtualization picks up, memory optimized systems are going to make a
come back.
> And it seems to be an entirely new market, since all mainstream
applications seems to have
> given up on memory optimization. For instance, the three of the most
popular applicatins:
> bind, spamassassin, apache are all memory hogs, using memory unncessarily.
I mean, why
> the heck do you need 200MB for running a web server? That''s quite
insane.
Thank PHP, Perl, Java, mod_sec, mod_rewrite, etc. All very useful and
needed things that add up. One of the reasons I love lighttpd so much :)

You shouldn''t *need* more than 64MB for a web server, but you do. The
fact remains that predicting normal usage and application deployment is
increasingly difficult. All it takes is one good ''digg'' to
make you
re-evaluate how much you need. You never know what one of your users is
going to decide to upload.

So I just try to be paranoid where I can, where things *can* be
predicted such as dom-0. The smaller I can make dom-0, the more ram I
can give to the guests. So I force myself to constantly ask "Could this
be done with fewer forks? Why are we grabbing this much ram?" The less
we take up on the back end, the more public ''business'' can get
done.

The other problem is web based control panels which everyone loves, like
C-Panel. Its one of the most inefficient piece of junk I''ve ever laid
eyes on, but my users demand it. Many people who simply just do not know
any better insist on running all services on one machine, not realizing
the cut throat competition that goes on within the process tree to grab
contiguous memory blocks. All I can do is be sure they have ample, plus
a little more. <shrug>.

Anyway, we better shut up before we''re flogged tarred and feathered for
going horribly off topic :) Hey, I got dom-0 in there twice! Put down
that carton of eggs!


Best,
--Tim



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On 2/22/07, Ligesh <myself@ligesh.com> wrote:
> > but this is not Xen''s job.
>
>  And how''s that?
> It is not the job of xen:the hyperVisor of course, but it should be part of
the xen, the platform.
What is "xen, the platform" supposed to be? For now, I know Xen as a
virtualization software, not systems management software. It operates
on a totally different level.

And the Xen developers have enough to do to get this one right and
nice. You might have realized, it works well in many scenarios, but is
lacking in quite some areas of features, maturity, and documentation
and usability. They need to concentrate on the core virtualization
features.

And, why reinvent the wheel? As said, xen-tools do a good job with the
things the original poster wants.
>From the "non-virtualized" world, there are many good management
tools
that can do such a job, depending on your taste. Look at cfengine,
FAI, OpenQRM, puppet (I didn''t look at the latter two myself yet).
If at all, virtualization management tools and system management tools
should be made compatible. But i don''t see any incompatibility of Xen
with FAI, cfengine and others, and openQRM seems to target at
integrating virtualization ,management into systems management.

When each virtualization solution brings it''s own proprietary systems
management stack, I''d need to manage my configurations in vmware, xen,
qemu, and kvm syntax, that would not be much fun.
> Does XenEnterprise do this at least?
I don''t care if XenEnterprise offers such a feature, I don''t
want
closed source/non-free software to tinker with my system config files.
I avoid it wherever I can.

Henning

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Feb 22, 2007 at 08:12:01PM +0100, Henning Sprang
wrote:
> On 2/22/07, Ligesh <myself@ligesh.com> wrote:
> >> but this is not Xen''s job.
> >
> > And how''s that?
> >It is not the job of xen:the hyperVisor of course, but it should be
part
> >of the xen, the platform.
> 
> What is "xen, the platform" supposed to be? For now, I know Xen
as a
> virtualization software, not systems management software. It operates
> on a totally different level.
 Something similar to vmware''s virtual center. Maybe for geeks it is
fun to have a lot of tiny tools to play around with, but I think the end user
needs a system that will do everything out of the box. And clearly the person
who posed the original question was expecting this. And it is also clear that
xen currently doesn''t have such a tool, not open source at least.

 Again, I am not saying that the xen should provide it, but that there do exist
a gap that needs to be filled--that is a tool that will do everything, including
configure the hostname, network of the virtual machine from the dom0
automatically.
 
> 
> And the Xen developers have enough to do to get this one right and
> nice. You might have realized, it works well in many scenarios, but is
> lacking in quite some areas of features, maturity, and documentation
> and usability. They need to concentrate on the core virtualization
> features.
> 
> And, why reinvent the wheel? As said, xen-tools do a good job with the
> things the original poster wants.
 Of course, one has to always the keep the priorities in perspective at all
times, and not get sidetracked on unnecessary features. But even if are
considering core virtualization, I think xen should provide some mechanism to
directly communicate with the domU using the domU''s name as the
identifier.

> If at all, virtualization management tools and system management tools
> should be made compatible. But i don''t see any incompatibility of
Xen
> with FAI, cfengine and others, and openQRM seems to target at
> integrating virtualization ,management into systems management.
 Does any of them provide the capabilities of virtual center? Or at least the
ability to configure the virtual machines from dom0?
> >Does XenEnterprise do this at least?
> 
> I don''t care if XenEnterprise offers such a feature, I
don''t want
> closed source/non-free software to tinker with my system config files.
> I avoid it wherever I can.
:-) Despite your personal antagonism, you can''t ignore that it does
exist, and it does serve some purpose (otherwise people won''t pay for
it). So rather than dismissing it, maybe you should mimic its features in open
source.



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users