I want to accomplish the following: I want to have a xen guest that I can also boot physically. Some indepth explanation: I want to virtualise my mediacenter with pci passthrough of my 2 dvb-s cards. In case of problems with my motherboard I want to boot the guest physically on other hardware without xen. Current hardware: Asrock Z77 pro 4 intel core i7 3370 1x 2,5" sata xen disk on as media controller 4x 2 TB WD disks in lvm raid10 on intel sata controller. I have tried the following scenario''s without success. Experiment 1 Ubuntu 13.04 with xen 4.2.2 create hvm guest with passthrough of intel sata controller: can see the disks, but cannot boot from it. I tried to complie xen myself but failed. raid and lvm managed by guest Experiment 2 Ubuntu 12.04 with xen 4.1.0 create hvm guest with passthrough of intel sata controller: can see the disks, but cannot boot from it. raid and lvm managed by guest Experiment 3 Ubuntu 12.04 with compiled xen 4.2.2 and qemu-upstream and seabios 1.7.2 create hvm guest with passthrough of intel sata controller: can boot from the disks if device_model_version = ''qemu-xen'' and device_model_override = ''/usr/lib/xen/bin/qemu-system-i386'' is used in config. But raid is not stable. raid managed by guest Experiment 4 Ubuntu 12.04 with compiled xen 4.2.2 and qemu-upstream and seabios 1.7.2 create hvm guest with passthrough physical disks seperately : the raid is detected in dom0 so this also poses an issue. I tried booting raid=noautodetect as grub parameter but without success, raid still starts. raid and lvm managed by guest Experiment 5: This is where I''m lost Ubuntu 12.04 with compiled xen 4.2.2 and no qemu-upstream as I cannot boot from cdrom with qemu-upstream I was thinking of adding a small file based disk for /boot and grub and using the previously installed lvm volume as root. When I boot the guest from a rescuecd no diskpartitions are detected which is logical as the lvm volume is used as a filesystem and not a raw lvm disk. Is there a way to use the filebased lvm as root? If I use a lvm volume as a raw disk, would it be possible to boot from it with grub without virtualisation.
On Wed, Jun 05, 2013 at 11:55:44PM +0200, Daniël Boone wrote:> I want to accomplish the following: > > I want to have a xen guest that I can also boot physically. > Some indepth explanation: > > I want to virtualise my mediacenter with pci passthrough of my 2 > dvb-s cards. > In case of problems with my motherboard I want to boot the guest > physically on other hardware without xen. > > Current hardware: > Asrock Z77 pro 4 > intel core i7 3370 > 1x 2,5" sata xen disk on as media controller > 4x 2 TB WD disks in lvm raid10 on intel sata controller. > > > I have tried the following scenario''s without success. > > Experiment 1 > Ubuntu 13.04 with xen 4.2.2 > create hvm guest with passthrough of intel sata controller: can see > the disks, but cannot boot from it. > I tried to complie xen myself but failed. > raid and lvm managed by guest > > Experiment 2 > Ubuntu 12.04 with xen 4.1.0 > create hvm guest with passthrough of intel sata controller: can see > the disks, but cannot boot from it. > raid and lvm managed by guest >What are the errors? If you can see (and presumably mount) the disks, what''s the issue? Also, is the root partition on the RAID? I would suggest _not_ to do that, as assembling RAIDs from the ramdisk is just throwing more variables into the equation.> Experiment 3 > Ubuntu 12.04 with compiled xen 4.2.2 and qemu-upstream and seabios 1.7.2 > create hvm guest with passthrough of intel sata controller: can boot > from the disks if device_model_version = ''qemu-xen'' and > device_model_override = ''/usr/lib/xen/bin/qemu-system-i386'' is used > in config. But raid is not stable. > raid managed by guest >What do you mean by "RAID is not stable"?> Experiment 4 > > Ubuntu 12.04 with compiled xen 4.2.2 and qemu-upstream and seabios 1.7.2 > create hvm guest with passthrough physical disks seperately : the > raid is detected in dom0 so this also poses an issue. I tried > booting raid=noautodetect as grub parameter but without success, > raid still starts. > raid and lvm managed by guest >This seems more like an Ubuntu issue. There is most definitely a way to prevent the dom0 from assembling the array. I would suggest asking an Ubuntu mailing list about that, as I have no experience with Ubuntu.> Experiment 5: > > This is where I''m lost > Ubuntu 12.04 with compiled xen 4.2.2 and no qemu-upstream as I > cannot boot from cdrom with qemu-upstream > I was thinking of adding a small file based disk for /boot and grub > and using the previously installed lvm volume as root. > When I boot the guest from a rescuecd no diskpartitions are detected > which is logical as the lvm volume is used as a filesystem and not a > raw lvm disk. > > Is there a way to use the filebased lvm as root? > If I use a lvm volume as a raw disk, would it be possible to boot > from it with grub without virtualisation. >LVM is always a tricky beast. To get the terminology straight, a Volume Group is made up of a bunch of real partitions called Physical Volumes. Logical Volumes are virtual partitions that belong to a Volume Group and physically reside on the Physical Volumes. Since the hard drives that the Physical Volumes reside on can only be "given" to either dom0 or domU, the Volume Group can only be managed by one or the other, not both. The domain that owns the Volume Group is the only domain that can directly access its Logical Volumes. If (and only if) you have the PV disk IO drivers running in the domU (either because it is PV or if you load PVHVM drivers), you can pass a partition straight from dom0 to a domU. It will show up to the domain as e.g. xvda1, as long as you have PV drivers. This is very dependent on your domU OS. Also, if you''re creating a special extra "file" disk to boot the machine in Xen, there is no need to make it LVM. Make that a simple hard disk with Grub installed on it, and it should be able to see the other hard drives on the virtual system and boot them. Grub2 can boot to LVM root partitions if I recall correctly. Now, I''m still uncertain as to whether I''ve answered your questions. Feel free to clarify if I''ve misunderstood. --Sean
On Thu, Jun 6, 2013 at 3:41 AM, Sean Greenslade <sean@seangreenslade.com>wrote:> On Wed, Jun 05, 2013 at 11:55:44PM +0200, Daniël Boone wrote: > > I want to accomplish the following: > > > > I want to have a xen guest that I can also boot physically. > > Some indepth explanation: > > > > I want to virtualise my mediacenter with pci passthrough of my 2 > > dvb-s cards. > > In case of problems with my motherboard I want to boot the guest > > physically on other hardware without xen. > > > > Current hardware: > > Asrock Z77 pro 4 > > intel core i7 3370 > > 1x 2,5" sata xen disk on as media controller > > 4x 2 TB WD disks in lvm raid10 on intel sata controller. > > > > >To clarify: 4x 2 TB in raid10 with each disk that has 3 partitions: p1: bios_grub partition p2: swap p3: raid partition. The raid10 spans all p3 partitions from the disks. On top of this I have LVM with 1 pv , 1 vg and 3 lv''s(root, data, vm_data) > I have tried the following scenario''s without success.> > > > Experiment 1 > > Ubuntu 13.04 with xen 4.2.2 > > create hvm guest with passthrough of intel sata controller: can see > > the disks, but cannot boot from it. > > I tried to complie xen myself but failed. > > raid and lvm managed by guest > > > > Experiment 2 > > Ubuntu 12.04 with xen 4.1.0 > > create hvm guest with passthrough of intel sata controller: can see > > the disks, but cannot boot from it. > > raid and lvm managed by guest > > > > What are the errors? If you can see (and presumably mount) the disks, > what''s the issue? Also, is the root partition on the RAID? I would > suggest _not_ to do that, as assembling RAIDs from the ramdisk > is just throwing more variables into the equation. > > The disks are not enumerated in the bios, hence you cannot boot from it.If you boot a rescuecd it can see the disks and raid. I this this has to do with the bios and qemu version that comes with mainstream xen.> > Experiment 3 > > Ubuntu 12.04 with compiled xen 4.2.2 and qemu-upstream and seabios 1.7.2 > > create hvm guest with passthrough of intel sata controller: can boot > > from the disks if device_model_version = ''qemu-xen'' and > > device_model_override = ''/usr/lib/xen/bin/qemu-system-i386'' is used > > in config. But raid is not stable. > > raid managed by guest > > > > What do you mean by "RAID is not stable"? > > When booting, disks are dropping out, getting messages that the responseof the disk is to slow.> > Experiment 4 > > > > Ubuntu 12.04 with compiled xen 4.2.2 and qemu-upstream and seabios 1.7.2 > > create hvm guest with passthrough physical disks seperately : the > > raid is detected in dom0 so this also poses an issue. I tried > > booting raid=noautodetect as grub parameter but without success, > > raid still starts. > > raid and lvm managed by guest > > > > This seems more like an Ubuntu issue. There is most definitely a way to > prevent the dom0 from assembling the array. I would suggest asking an > Ubuntu mailing list about that, as I have no experience with Ubuntu. > > I think it''s more a kernel/initrd issue. I know the parameter israid=noautodetect is the correct one. I can''t remove the modules from the initrd as I can''t find a way to unpack the xen initrd.> > Experiment 5: > > > > This is where I''m lost > > Ubuntu 12.04 with compiled xen 4.2.2 and no qemu-upstream as I > > cannot boot from cdrom with qemu-upstream > > I was thinking of adding a small file based disk for /boot and grub > > and using the previously installed lvm volume as root. > > When I boot the guest from a rescuecd no diskpartitions are detected > > which is logical as the lvm volume is used as a filesystem and not a > > raw lvm disk. > > > > Is there a way to use the filebased lvm as root? > > If I use a lvm volume as a raw disk, would it be possible to boot > > from it with grub without virtualisation. > > > > LVM is always a tricky beast. To get the terminology straight, a Volume > Group is made up of a bunch of real partitions called Physical Volumes. > Logical Volumes are virtual partitions that belong to a Volume Group and > physically reside on the Physical Volumes. Since the hard drives that > the Physical Volumes reside on can only be "given" to either dom0 or > domU, the Volume Group can only be managed by one or the other, not > both. The domain that owns the Volume Group is the only domain that can > directly access its Logical Volumes. If (and only if) you have the PV > disk IO drivers running in the domU (either because it is PV or if you > load PVHVM drivers), you can pass a partition straight from dom0 to a > domU. It will show up to the domain as e.g. xvda1, as long as you have > PV drivers. This is very dependent on your domU OS. > > Also, if you''re creating a special extra "file" disk to boot the machine > in Xen, there is no need to make it LVM. Make that a simple hard disk > with Grub installed on it, and it should be able to see the other hard > drives on the virtual system and boot them. Grub2 can boot to LVM root > partitions if I recall correctly. > > Now, I''m still uncertain as to whether I''ve answered your questions. > Feel free to clarify if I''ve misunderstood. > > --Sean >The filebased boot disk would be a normal disk without any lvm on it. I will try if a pv host and see if that will work. -- Groeten Daniël Boone _______________________________________________ Xen-users mailing list Xen-users@lists.xen.org http://lists.xen.org/xen-users
Going to try my best to trim this down while keeping it readable. On Fri, Jun 07, 2013 at 12:24:14AM +0200, daniel boone wrote:> On Thu, Jun 6, 2013 at 3:41 AM, Sean Greenslade <sean@seangreenslade.com>wrote: > > > On Wed, Jun 05, 2013 at 11:55:44PM +0200, Daniël Boone wrote: > > > I want to accomplish the following: > > > > > > I want to have a xen guest that I can also boot physically. > > > Some indepth explanation: > > ><snip>> > > To clarify: > 4x 2 TB in raid10 with each disk that has 3 partitions: > p1: bios_grub partition > p2: swap > p3: raid partition. > > The raid10 spans all p3 partitions from the disks. > On top of this I have LVM with 1 pv , 1 vg and 3 lv''s(root, data, vm_data) >Perhaps I am still misunderstanding. Are all four disks identically partitioned? Because from what I understand of this, you have four bios_grub partitions and four swap partitons, and then four RAID partitions that get RAID10 software assembled into the single PV for an LVM VG.> > I have tried the following scenario''s without success. > > > > > > Experiment 1 > > > Ubuntu 13.04 with xen 4.2.2 > > > create hvm guest with passthrough of intel sata controller: can see > > > the disks, but cannot boot from it. > > > I tried to complie xen myself but failed. > > > raid and lvm managed by guest > > > > > > Experiment 2 > > > Ubuntu 12.04 with xen 4.1.0 > > > create hvm guest with passthrough of intel sata controller: can see > > > the disks, but cannot boot from it. > > > raid and lvm managed by guest > > > > > > > What are the errors?<snip>> If you boot a rescuecd it can see the disks and raid. I this this has to do > with the bios and qemu version that comes with mainstream xen.So again, what''s the problem? If it can see the disks and assemble the RAID, there''s no reason it shouldn''t be able to boot. What exactly is preventing it from booting? Specific error messages well help.> > > > Experiment 3 > > > Ubuntu 12.04 with compiled xen 4.2.2 and qemu-upstream and seabios 1.7.2 > > > create hvm guest with passthrough of intel sata controller: can boot > > > from the disks if device_model_version = ''qemu-xen'' and > > > device_model_override = ''/usr/lib/xen/bin/qemu-system-i386'' is used > > > in config. But raid is not stable. > > > raid managed by guest > > > > > > > What do you mean by "RAID is not stable"? > > > > When booting, disks are dropping out, getting messages that the response > of the disk is to slow. >Once again, specific error messages would be helpful, though I suspect I will have little to offer in that regard. It seems like that would be from a QEMU issue.> > > > Experiment 4 > > > > > > Ubuntu 12.04 with compiled xen 4.2.2 and qemu-upstream and seabios 1.7.2 > > > create hvm guest with passthrough physical disks seperately : the > > > raid is detected in dom0 so this also poses an issue. I tried > > > booting raid=noautodetect as grub parameter but without success, > > > raid still starts. > > > raid and lvm managed by guest > > > > > > > This seems more like an Ubuntu issue. There is most definitely a way to > > prevent the dom0 from assembling the array. I would suggest asking an > > Ubuntu mailing list about that, as I have no experience with Ubuntu. > > > I think it''s more a kernel/initrd issue. I know the parameter is > raid=noautodetect is the correct one. > I can''t remove the modules from the initrd as I can''t find a way to unpack > the xen initrd.If you are truly setting that kernel parameter correctly (and I would suggest confirming that by editing the selected boot line in Grub as it boots), then you have more than likely discovered a kernel bug. That may warrant further investigation. Also, there is no Xen initrd. The initrd is the dom0''s initrd, which is why I said this is more of an Ubuntu issue than a Xen issue. You want Ubuntu to not assemble the RAID on boot, something that Xen should not be affecting in any way.> > > > Experiment 5: > > > > > > This is where I''m lost > > > Ubuntu 12.04 with compiled xen 4.2.2 and no qemu-upstream as I > > > cannot boot from cdrom with qemu-upstream > > > I was thinking of adding a small file based disk for /boot and grub > > > and using the previously installed lvm volume as root. > > > When I boot the guest from a rescuecd no diskpartitions are detected > > > which is logical as the lvm volume is used as a filesystem and not a > > > raw lvm disk. > > > > > > Is there a way to use the filebased lvm as root? > > > If I use a lvm volume as a raw disk, would it be possible to boot > > > from it with grub without virtualisation. > > > > > > > LVM is always a tricky beast.< LVM explanation snipped >> The filebased boot disk would be a normal disk without any lvm on it. > I will try if a pv host and see if that will work.Yes, this is the most sensible route. No sense in making that complicated. And yes, PV is the preferable way to run domUs. As long as your domU kenel has the PV drivers compiled in, running it in PV mode should work fine. You may have to make some strage entries in your fstab, but it should be doable. --Sean