Xen users - Oct 2005 - Installing Xen on Debian Sarge

I have sent this mail to debian-user@lists.debian.org, but I have gotten 
any answers. Maybe I should have sent it here first.

I have the following machine:

System:
  - Debian Sarge, stock debian kernel, fully updated pkgs.
  - uname -a:
	Linux debian 2.6.8-2-386 #1 Thu May 19 17:40:50 JST 2005 i686 GNU/Linux
  - Nvidia drivers built with module-assistant: 		
	nvidia-kernel-2.6.8-2-386_1.0.7174-3+2.6.8-16_i386.deb

  - root filesystem on LVM
  - /etc/fstab (extract)

/dev/mapper/debianvg-rootlv /     reiserfs defaults 0 1
/dev/hda3                   /boot ext3     defaults 0 2
/dev/mapper/debianvg-homelv /home reiserfs defaults 0 2
/dev/hda5                   none  swap     sw       0 0

- Pentium IV 2.0GHz, 384 MB RAM
- 2 IDE''s:
	hda: for current debian sarge, dual booted with W2K
	hdb: data plus free space for lvm for the xen domU''s
- 1 ethernic 10/100 Mb  with tulip driver


I have read a lot of sites regarding the various ways of setting up
debian sarge to run Xen.

- http://mark.foster.cc/wiki/index.php/Xen_Tips
- http://mark.foster.cc/wiki/index.php/Debian_Sarge_on_Xen
- http://jclement.ca/xen/host_setup.html
- http://wiki.xensource.com/xenwiki/DebianDomU
- http://dhassler.com/snipsnap/space/UnixResources/InstallingXen
- http://cosi.clarkson.edu/knowledge/workshops/sp05/installingxen/
xen-tutorial.html
- http://wiki.blitzed.org/User:grifferz/Xen
- http://www.option-c.com/xwiki/Xen_Debian_Quick_Start
- http://julien.danjou.info/xen.html
Too much information to swallow.

A couple of remarks,

1. Almost all of these docs are about installing Xen. Just one site has
scenarios where you would like to use Xen as the optimal solution.
2. They are all about installing on plain devices, not lvm volumes.

I will be testing a distributed system I built for work. For this I need
a central node and 10 remote nodes. I thought that Xen will be the
perfect solution. Just create 11 DomUs with minimal debian in each one,
run the test of the system and collect statistics.

I have put a lot of time in my current desktop, so I would like to
continue using my desktop, but running as Dom0. I would like to use the
second hard drive to install the virtual machines.

After a lot of problems and 3 days of search and testing I finally
booted my machine as Dom0. These are the steps I followed:

1. I used the unstable pkgs for xen
2. I downloaded the src from:
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-2.0.7-src.tgz
and compiled using my current kernel config in /boot.
3. I installed the new XenLinux kernel in /boot and configured menu.lst
according.
3. I had to build a initrd image so that the boot process could mount
the root partition on lvm. I use the script from
/usr/share/doc/lvm2/examples/lvm2create_initrd.gz
to build the initrd, as mkinitrd has a bug with lvm root partitions.
4. I updated grub to use the initrd and all worked ok.

After that, I had my system running Xen but without graphical setup.
This is  not a problem, as I probably could configured correctly with
patience.
I am now ready for creating domains for the real work and I can say that
I have managed to install Xen successfully.
In spite of that, I have the feeling that the solution isn''t optimal or
the best.

So, the questions:

- I''m doing this the proper way?

- Is this the intended way of installing Xen?
I mean, should I use my configured and personalised original machine
(postgresql, apache, firefox, thunderbird, /home, etc) on top of Xen as
Dom0 and from time to time to create DomUs for specific task? Or should
I use a minimal debian for Dom0, just for hosting the DomUs and use one
of the DomU as workstation, with all my configuration and selection of
packages inside and the other DomUs for working as needed?

- If I use Dom0 for my workstation, I think I will have full control of
all the others DomUs, and I could use them as I wanted. I could access
it with xm console without problems. But if I use one DomU for my
workstation how could I access the other DomUs? With ssh or vnc only?
What happend if I wanted to access the DomO from DomU? I think that I
couldn''t do that.

- What will pass with the updates of my kernel in Dom0?  Will I need to
redo all the steps for configuring my new xen-enabled kernel, creating
an initrd image, compiling nvidia drivers and, when my Dom0 was updated,
then update all of my DomUs?

A don''t want to create a flame war, but I would like to hear the
scenarios you used and the decisions you made when configuring Xen.
Also, I would like to read your toughts about your use of Xen.

Thank you all for the answers, in advance

Miguel Cobá






_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

I installed debian 3.1 on a lvm2 root, and it worked without problems. Even with
the standard mkinitrd.

Perhaps the problem is in your mkinitrd config ?

Johnny

Am Mittwoch, 19. Oktober 2005 17:11 schrieb Miguel Enrique Cobá
Martínez:
> [...]
> 3. I had to build a initrd image so that the boot process could mount 
> the root partition on lvm. I use the script from
> /usr/share/doc/lvm2/examples/lvm2create_initrd.gz
> to build the initrd, as mkinitrd has a bug with lvm root partitions.
> 4. I updated grub to use the initrd and all worked ok.
> [...]
-- 
jabber:johnny@wh-netz.de
icq:236971856

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Miguel Enrique Cobá Martínez wrote:
> 2. They are all about installing on plain devices, not lvm volumes.
>
If you are serious about using Xen for production systems you will want 
use LVM or something other than simple file-based disks. Sounds like 
some others are using true SANs or gnbd or drbd? My personal preference 
is to assign an 8GB logical volume for "/" and another volume for
swap,
the size depending on how much RAM will be assigned to the DomU. That 
fits my bill, but in the physical world I much prefer giving separate 
partitions for /boot, /usr, /var and so on. It''s just too much 
(maintenance) overhead with virtual systems. I also prefer to use RAID 
0+1 over SCSI.
> I will be testing a distributed system I built for work. For this I need
> a central node and 10 remote nodes. I thought that Xen will be the
> perfect solution. Just create 11 DomUs with minimal debian in each one,
> run the test of the system and collect statistics.
>
> I have put a lot of time in my current desktop, so I would like to
> continue using my desktop, but running as Dom0. I would like to use the
> second hard drive to install the virtual machines.
I would recommend a dedicated system for Xen Dom0, as running it on your 
desktop is asking for trouble. Running your desktop in Xen as a DomU is 
also silly.
Keep your Dom0 as small as possible to minimize your risk when patching 
(you are patching right?)
>
> After a lot of problems and 3 days of search and testing I finally
> booted my machine as Dom0. These are the steps I followed:
>
> 1. I used the unstable pkgs for xen
OK.
> 2. I downloaded the src from:
> http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-2.0.7-src.tgz 
>
> and compiled using my current kernel config in /boot.
Huh? You used the pkgs AND you installed from src?
> 3. I installed the new XenLinux kernel in /boot and configured menu.lst
> according.
> 3. I had to build a initrd image so that the boot process could mount
> the root partition on lvm. I use the script from
> /usr/share/doc/lvm2/examples/lvm2create_initrd.gz
> to build the initrd, as mkinitrd has a bug with lvm root partitions.
Ah, good info!
>
> 4. I updated grub to use the initrd and all worked ok.
>
> After that, I had my system running Xen but without graphical setup.
> This is  not a problem, as I probably could configured correctly with
> patience.
> I am now ready for creating domains for the real work and I can say that
> I have managed to install Xen successfully.
> In spite of that, I have the feeling that the solution isn''t
optimal or
> the best.
>
> So, the questions:
>
> - I''m doing this the proper way? 
>
> - Is this the intended way of installing Xen?
> I mean, should I use my configured and personalised original machine
> (postgresql, apache, firefox, thunderbird, /home, etc) on top of Xen as
> Dom0 and from time to time to create DomUs for specific task? Or should
> I use a minimal debian for Dom0, just for hosting the DomUs and use one
> of the DomU as workstation, with all my configuration and selection of
> packages inside and the other DomUs for working as needed?
I wouldn''t do what you are doing, but to each his own.
>
> - If I use Dom0 for my workstation, I think I will have full control of
> all the others DomUs, and I could use them as I wanted. I could access
> it with xm console without problems. But if I use one DomU for my
> workstation how could I access the other DomUs? With ssh or vnc only?
> What happend if I wanted to access the DomO from DomU? I think that I
> couldn''t do that.
ssh or vnc or whatever! The only thing you lose is console access from 
your desktop to the DomUs, but that can be done by ssh-ing to DomO and 
''xm console'' from there.
>
> - What will pass with the updates of my kernel in Dom0?  Will I need to
> redo all the steps for configuring my new xen-enabled kernel, creating
> an initrd image, compiling nvidia drivers and, when my Dom0 was updated,
> then update all of my DomUs?
You can specify a default kernel in /boot/grub/menu.lst you will 
sidestep any kernels installed via apt-get (or aptitude)
>
> A don''t want to create a flame war, but I would like to hear the
> scenarios you used and the decisions you made when configuring Xen.
> Also, I would like to read your toughts about your use of Xen.
Using LVM was a good idea. LVM snapshots are also interesting way to 
save disk space and have multiple systems based on a common denominator, 
but I have yet to get that going. If anyone else has, I''d sure like to 
know how to set it up "right".
Good luck.

-- 
Some days it''s just not worth chewing through the restraints...
Mark D. Foster, CISSP <mark@foster.cc>  http://mark.foster.cc/



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hans-Christian Armingeon wrote:
> I installed debian 3.1 on a lvm2 root, and it worked without problems. Even
with the standard mkinitrd.
> 
> Perhaps the problem is in your mkinitrd config ?
> 
> Johnny
Yes, I have my current debian sarge root on a lvm partition and I have 
no problems. But when I wanted to boot my custom compiled xen-kernel, it 
didn''t boot, because the root was on a lvm, so, searching on google and
the lists, I found that it was a problem when usint lvm roots. The 
solution given there was to add a initrd so that the lvm root could be 
mounted correctly.
But when I tried to:

debian:~# mkinitrd -o initrd-2.6.11debianxen0 2.6.11debianxen0
File descriptor 3 left open
File descriptor 4 left open
File descriptor 5 left open
File descriptor 6 left open
File descriptor 7 left open
     Finding all volume groups
     Finding volume group "xenvg"
     Finding volume group "debianvg"
/usr/sbin/mkinitrd: /dev/mapper/debianvg-rootlv: Kernel does not support LVM

So in further research, I found in

http://pigeon.dyndns.org/stuff/lvm-root/lvm-root.html

that /usr/share/doc/lvm2/examples/lvm2create_initrd.gz
was made to address this problem
after that, I added to menu.lst

title Debian GNU/Linux, Xen 2.6.11xen0, testing 25/5/2005
root (hd0,2)
kernel /xen.gz dom0_mem=195584
module /xen-linux-2.6.11debianxen0 root=/dev/ram0 
lvm2root=/dev/mapper/debianvg-rootlv ro
module /initrd-lvm2-2.6.11debianxen0.gz

and it booted ok.

Auf wiederlesen!
> 
> Am Mittwoch, 19. Oktober 2005 17:11 schrieb Miguel Enrique Cobá Martínez:
> 
>>[...]
>>3. I had to build a initrd image so that the boot process could mount 
>>the root partition on lvm. I use the script from
>>/usr/share/doc/lvm2/examples/lvm2create_initrd.gz
>>to build the initrd, as mkinitrd has a bug with lvm root partitions.
>>4. I updated grub to use the initrd and all worked ok.
>>[...]
> 
> 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Mark Foster wrote:
> Miguel Enrique Cobá Martínez wrote:
> 
>> 2. They are all about installing on plain devices, not lvm volumes.
>>
> If you are serious about using Xen for production systems you will want 
> use LVM or something other than simple file-based disks. Sounds like 
> some others are using true SANs or gnbd or drbd? My personal preference 
> is to assign an 8GB logical volume for "/" and another volume for
swap,
> the size depending on how much RAM will be assigned to the DomU. That 
> fits my bill, but in the physical world I much prefer giving separate 
> partitions for /boot, /usr, /var and so on. It''s just too much 
> (maintenance) overhead with virtual systems. I also prefer to use RAID 
> 0+1 over SCSI.
> 
Yes I agree, in my future servers I will be using something more robust 
that simple IDE disk. For the moment, as I am starting with Xen, I have 
to use my desktop for all the tests. Unfortunately, until I can show a 
working system and the advantages for use it, they aren''t investing
much
money for this :D
>> I will be testing a distributed system I built for work. For this I
need
>> a central node and 10 remote nodes. I thought that Xen will be the
>> perfect solution. Just create 11 DomUs with minimal debian in each one,
>> run the test of the system and collect statistics.
>>
>> I have put a lot of time in my current desktop, so I would like to
>> continue using my desktop, but running as Dom0. I would like to use the
>> second hard drive to install the virtual machines.
> 
> 
> I would recommend a dedicated system for Xen Dom0, as running it on your 
> desktop is asking for trouble. Running your desktop in Xen as a DomU is 
> also silly.
> Keep your Dom0 as small as possible to minimize your risk when patching 
> (you are patching right?)
Yes they are a lot of trouble. I finally have a Dom0 and three DomU, all 
with sarge, but the dom0 nvidia card is quite hard to configure.
> 
>>
>> After a lot of problems and 3 days of search and testing I finally
>> booted my machine as Dom0. These are the steps I followed:
>>
>> 1. I used the unstable pkgs for xen
> 
> 
> OK.
> 
>> 2. I downloaded the src from:
>>
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-2.0.7-src.tgz
>>
>> and compiled using my current kernel config in /boot.
> 
> 
> Huh? You used the pkgs AND you installed from src?
Yes, because the packages only install /boot/xen.gz, xend and xendomains 
  and symlinks in /etc/init.d/. You are supposed to build or download 
your dom0 and domU images.
> 
>> 3. I installed the new XenLinux kernel in /boot and configured menu.lst
>> according.
>> 3. I had to build a initrd image so that the boot process could mount
>> the root partition on lvm. I use the script from
>> /usr/share/doc/lvm2/examples/lvm2create_initrd.gz
>> to build the initrd, as mkinitrd has a bug with lvm root partitions.
> 
> 
> Ah, good info!
> 
>>
>> 4. I updated grub to use the initrd and all worked ok.
>>
>> After that, I had my system running Xen but without graphical setup.
>> This is  not a problem, as I probably could configured correctly with
>> patience.
>> I am now ready for creating domains for the real work and I can say
that
>> I have managed to install Xen successfully.
>> In spite of that, I have the feeling that the solution isn''t
optimal or
>> the best.
>>
>> So, the questions:
>>
>> - I''m doing this the proper way? 
> 
> 
>>
>> - Is this the intended way of installing Xen?
>> I mean, should I use my configured and personalised original machine
>> (postgresql, apache, firefox, thunderbird, /home, etc) on top of Xen as
>> Dom0 and from time to time to create DomUs for specific task? Or should
>> I use a minimal debian for Dom0, just for hosting the DomUs and use one
>> of the DomU as workstation, with all my configuration and selection of
>> packages inside and the other DomUs for working as needed?
> 
> 
> I wouldn''t do what you are doing, but to each his own.
> 
>>
>> - If I use Dom0 for my workstation, I think I will have full control of
>> all the others DomUs, and I could use them as I wanted. I could access
>> it with xm console without problems. But if I use one DomU for my
>> workstation how could I access the other DomUs? With ssh or vnc only?
>> What happend if I wanted to access the DomO from DomU? I think that I
>> couldn''t do that.
> 
> 
> ssh or vnc or whatever! The only thing you lose is console access from 
> your desktop to the DomUs, but that can be done by ssh-ing to DomO and 
> ''xm console'' from there.
> 
>>
>> - What will pass with the updates of my kernel in Dom0?  Will I need to
>> redo all the steps for configuring my new xen-enabled kernel, creating
>> an initrd image, compiling nvidia drivers and, when my Dom0 was
updated,
>> then update all of my DomUs?
> 
> 
> You can specify a default kernel in /boot/grub/menu.lst you will 
> sidestep any kernels installed via apt-get (or aptitude)
> 
>>
>> A don''t want to create a flame war, but I would like to hear
the
>> scenarios you used and the decisions you made when configuring Xen.
>> Also, I would like to read your toughts about your use of Xen.
> 
> 
> Using LVM was a good idea. LVM snapshots are also interesting way to 
> save disk space and have multiple systems based on a common denominator, 
> but I have yet to get that going. If anyone else has, I''d sure
like to
> know how to set it up "right".
> Good luck.
> 
I think I''ll try them soon.

Thanks

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users