Xen users - Nov 2011 - Fully-Virtualized XEN domU not Booting over iSCSI

Hello,

I am currently trying to move my VMs from running on local host storage to
a shared storage (trying out iSCSI) but I am
facing a bit of a booting dilemma.

The domUs are a mix of paravirtualized and fully-virtualized VMs. They all
boot and run like clockwork when on local
storage.

The paravirtualized domUs appear not to have a problem when I relocate and
boot them from the shared storage
though the fully-virtualized ones give me an error on bootup.

************************************************************
Boot from Hard Disk failed: could not read the hard disk
FATAL: No bootable device
************************************************************

The iSCSI is offering the domUs as file images (.img) to the initiator i.e
.Type=fileio.

My host''s processor (as you may have deduced) supports Intel-VTx. Does
my
shared storage need to have this capability to be able to
boot my fully-virtualized domU?

Any pointers/suggestions would be appreciated.

Phillip

BASIC SYSTEM INFO:
------------------------------------------------------------
== BEGIN uname -rmi =2.6.18-238.19.1.el5xen i686 i386
== END   uname -rmi =
== BEGIN rpm -qa \*-release\* =centos-release-5-6.el5.centos.1
rpmforge-release-0.5.2-2.el5.rf
centos-release-notes-5.6-0
== END   rpm -qa \*-release\* =
== BEGIN cat /etc/redhat-release =CentOS release 5.6 (Final)
== END   cat /etc/redhat-release =
== BEGIN getenforce =Disabled
== END   getenforce =
== BEGIN free -m =             total       used       free     shared    buffers
cached
Mem:          1102       1072         29          0        208        356
-/+ buffers/cache:        507        594
Swap:         4031          0       4031
== END   free -m =
_----------------------------------------------------------------



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,

Just some more details:

 - Host is currently running Xen 3.1.2 (shipped with Centos 5.6)
 - The HVM were installed using isos (playing around with turnkey
appliances).

Any assistance/pointers will be highly appreciated.

Phill


On Thu, November 3, 2011 4:54 pm, Phillip Nkubito wrote:
> Hello,
>
> I am currently trying to move my VMs from running on local host storage to
> a shared storage (trying out iSCSI) but I am
> facing a bit of a booting dilemma.
>
> The domUs are a mix of paravirtualized and fully-virtualized VMs. They all
> boot and run like clockwork when on local
> storage.
>
> The paravirtualized domUs appear not to have a problem when I relocate and
> boot them from the shared storage
> though the fully-virtualized ones give me an error on bootup.
>
> ************************************************************
> Boot from Hard Disk failed: could not read the hard disk
> FATAL: No bootable device
> ************************************************************
>
> The iSCSI is offering the domUs as file images (.img) to the initiator i.e
> .Type=fileio.
>
> My host''s processor (as you may have deduced) supports Intel-VTx.
Does my
> shared storage need to have this capability to be able to
> boot my fully-virtualized domU?
>
> Any pointers/suggestions would be appreciated.
>
> Phillip
>
> BASIC SYSTEM INFO:
> ------------------------------------------------------------
> == BEGIN uname -rmi => 2.6.18-238.19.1.el5xen i686 i386
> == END   uname -rmi =>
> == BEGIN rpm -qa \*-release\* => centos-release-5-6.el5.centos.1
> rpmforge-release-0.5.2-2.el5.rf
> centos-release-notes-5.6-0
> == END   rpm -qa \*-release\* =>
> == BEGIN cat /etc/redhat-release => CentOS release 5.6 (Final)
> == END   cat /etc/redhat-release =>
> == BEGIN getenforce => Disabled
> == END   getenforce =>
> == BEGIN free -m =>              total       used       free     shared 
buffers     cached
> Mem:          1102       1072         29          0        208        356
> -/+ buffers/cache:        507        594
> Swap:         4031          0       4031
> == END   free -m =>
> _----------------------------------------------------------------
>
>

Kind Regards,

Phillip Nkubito



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Nov 10, 2011 at 8:01 PM, Phillip Nkubito <phil@crg.ee.uct.ac.za>
wrote:
> Hi,
>
> Just some more details:
>
>  - Host is currently running Xen 3.1.2 (shipped with Centos 5.6)
>  - The HVM were installed using isos (playing around with turnkey
> appliances).
>
> Any assistance/pointers will be highly appreciated.

I doubt anyone will be able to help you. You didn''t give out the most
important details. For example:
- how were the domU disks originally created? as LVM on dom0? as file images?
- how did you move it to the shared storage? dd? cp?
- how does the domU access the shared storage now? using phy:/ ?

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Thanks for the pointer Fajar...I got engrossed in the problem and
neglected important details.

1.Im guessing LVM on domO.(Newbie alert!) I installed the PV domU disks
through a network tree install, on hosts'' local storage first and then
moved the image to the shared iscsi storage.(probably a hard way to do it
but made sense at the time).

2. I used a cp command to move from local to shared storage.

3. The domU images are exported as type=fileio and I notice in my domU
config files the disk parameter is indeed a phy:/

All the PV domUs are CentOS 5.6/7 and are working fine on shared storage
but the HVM guests (turnkey appliances and Windows) cannot boot with the
below mentioned error.

Thanks.
Phillip.

On Thu, November 10, 2011 3:11 pm, Fajar A. Nugraha
wrote:
> On Thu, Nov 10, 2011 at 8:01 PM, Phillip Nkubito
<phil@crg.ee.uct.ac.za>
> wrote:
>> Hi,
>>
>> Just some more details:
>>
>>  - Host is currently running Xen 3.1.2 (shipped with Centos 5.6)
>>  - The HVM were installed using isos (playing around with turnkey
>> appliances).
>>
>> Any assistance/pointers will be highly appreciated.
>
>
> I doubt anyone will be able to help you. You didn''t give out the
most
> important details. For example:
> - how were the domU disks originally created? as LVM on dom0? as file
> images?
> - how did you move it to the shared storage? dd? cp?
> - how does the domU access the shared storage now? using phy:/ ?
>
> --
> Fajar
>

Kind Regards,

Phillip Nkubito



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, Nov 10, 2011 at 11:27 PM, Phillip Nkubito <phil@crg.ee.uct.ac.za>
wrote:
> Thanks for the pointer Fajar...I got engrossed in the problem and
> neglected important details.
>
> 1.Im guessing LVM on domO.(Newbie alert!) I installed the PV domU disks
> through a network tree install, on hosts'' local storage first and
then
> moved the image to the shared iscsi storage.(probably a hard way to do it
> but made sense at the time).
>
> 2. I used a cp command to move from local to shared storage.
Your answers conflict with each other. If the domU originally uses an
LV on local storage, then you''d usually copy it using dd (or similar),
not cp. Or did you perhaps mount both the domU storage and remote
iscsi disk on dom0, and copy the CONTENTS (not the disk itself)
manually? That won''t work.

I''d take your word for it though and believe that (somehow) you manage
to copy an LV and turn it into a file (.img) on remote server.
>
> 3. The domU images are exported as type=fileio
xen doesn''t really care about this bit of information. It''s
specific
to the iscsi server you''re using.
> and I notice in my domU
> config files the disk parameter is indeed a phy:/
If you''ve already imported the iscsi disks on dom0, and can see it as
"normal" disks on dom0 (e.g. /dev/sdc), then it should work.
>
> All the PV domUs are CentOS 5.6/7 and are working fine on shared storage
> but the HVM guests (turnkey appliances and Windows) cannot boot with the
> below mentioned error.
The iscsi server might change the disk geometry. Try booting the domU
with systemrescued and use ntfsreloc (or is it relocntfs?) on the
partition containing windows. Not sure what happened to your
appliances though. One possible cause: your copy process is simply
wrong.

-- 
Fajar

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

Hi,

if you used "cp" to copy the files on the disk instead of
"dd" to copy the
blocks themselves you have probably not transferred the MBR.

I would recommend installing the HVM guest directly onto the iSCSI lun
instead, this will save you the pain of migration, which can be more
difficult to troubleshoot than you might think at first.

Joseph.

On 11 November 2011 06:40, Fajar A. Nugraha <list@fajar.net> wrote:
> On Thu, Nov 10, 2011 at 11:27 PM, Phillip Nkubito
<phil@crg.ee.uct.ac.za>
> wrote:
> > Thanks for the pointer Fajar...I got engrossed in the problem and
> > neglected important details.
> >
> > 1.Im guessing LVM on domO.(Newbie alert!) I installed the PV domU
disks
> > through a network tree install, on hosts'' local storage first
and then
> > moved the image to the shared iscsi storage.(probably a hard way to do
it
> > but made sense at the time).
> >
> > 2. I used a cp command to move from local to shared storage.
>
> Your answers conflict with each other. If the domU originally uses an
> LV on local storage, then you''d usually copy it using dd (or
similar),
> not cp. Or did you perhaps mount both the domU storage and remote
> iscsi disk on dom0, and copy the CONTENTS (not the disk itself)
> manually? That won''t work.
>
> I''d take your word for it though and believe that (somehow) you
manage
> to copy an LV and turn it into a file (.img) on remote server.
>
> >
> > 3. The domU images are exported as type=fileio
>
> xen doesn''t really care about this bit of information.
It''s specific
> to the iscsi server you''re using.
>
> > and I notice in my domU
> > config files the disk parameter is indeed a phy:/
>
> If you''ve already imported the iscsi disks on dom0, and can see it
as
> "normal" disks on dom0 (e.g. /dev/sdc), then it should work.
>
> >
> > All the PV domUs are CentOS 5.6/7 and are working fine on shared
storage
> > but the HVM guests (turnkey appliances and Windows) cannot boot with
the
> > below mentioned error.
>
> The iscsi server might change the disk geometry. Try booting the domU
> with systemrescued and use ntfsreloc (or is it relocntfs?) on the
> partition containing windows. Not sure what happened to your
> appliances though. One possible cause: your copy process is simply
> wrong.
>
> --
> Fajar
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>


-- 
*
Founder | Director | VP Research
Orion Virtualisation Solutions* | www.orionvm.com.au | Phone: 1300 56 99 52
| Mobile: 0428 754 846


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users