Xen users - Apr 2010 - how can you boot an existing linux domU off the cdrom on sles11 dom0?

I have a sles10 domU running on a sles11 dom0. xen is version 3.4.
We''re
trying to pass through a scsi card to the domU. We assign the domU the
scsi card through virt manager. When we try to boot the domU it won''t
boot. We need to update fstab and menu.lst, but we need to boot with the
hardware attached to see how it''s being seen by the domU. 

On a physical machine we usually boot off the sles boot cd in rescue
mode and modify the file accordingly. In this case, I can''t get the
domU
to boot from the cd. 

Any suggestions on how to do this? I''ve done it with Citrix XenServer,
but have never been able to do it on a sles dom0. 

I''ve tried adding a cdrom and using xm commands to save config, delete
the domU, modify the config enabling the cdrom as bootable, then
restoring the domU with xm new -F command, but then it won''t boot at
all. 

Help is appreciated. 

Thanks,
James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

What type of file system does the domU have, it may be possilbe
to stop the domU and mount the file system on dom0 and make
the change that way.

Steve


On Thu, 22 Apr 2010, James Pifer wrote:
> I have a sles10 domU running on a sles11 dom0. xen is version 3.4.
We''re
> trying to pass through a scsi card to the domU. We assign the domU the
> scsi card through virt manager. When we try to boot the domU it
won''t
> boot. We need to update fstab and menu.lst, but we need to boot with the
> hardware attached to see how it''s being seen by the domU.
>
> On a physical machine we usually boot off the sles boot cd in rescue
> mode and modify the file accordingly. In this case, I can''t get
the domU
> to boot from the cd.
>
> Any suggestions on how to do this? I''ve done it with Citrix
XenServer,
> but have never been able to do it on a sles dom0.
>
> I''ve tried adding a cdrom and using xm commands to save config,
delete
> the domU, modify the config enabling the cdrom as bootable, then
> restoring the domU with xm new -F command, but then it won''t boot
at
> all.
>
> Help is appreciated.
>
> Thanks,
> James
>
>
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
>
-- 
------------------------------------------------------------------
Steven C. Timm, Ph.D  (630) 840-8525
timm@fnal.gov  http://home.fnal.gov/~timm/
Fermilab Computing Division, Scientific Computing Facilities,
Grid Facilities Department, FermiGrid Services Group, Assistant Group Leader.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, 2010-04-22 at 15:02 -0500, Steven Timm wrote:
> What type of file system does the domU have, it may be possilbe
> to stop the domU and mount the file system on dom0 and make
> the change that way.
> 
> Steve
I was thinking that too and I can mount the filesystem, but the domU
needs to be booted with the new devices attached to see how to change
fstab/menu.lst. 

James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thursday, April 22, 2010 03:06 PM CDT, James Pifer
<jep@obrien-pifer.com> wrote:
 
> On Thu, 2010-04-22 at 15:02 -0500, Steven Timm wrote:
> > What type of file system does the domU have, it may be possilbe
> > to stop the domU and mount the file system on dom0 and make
> > the change that way.
> > 
> > Steve
> 
> I was thinking that too and I can mount the filesystem, but the domU
> needs to be booted with the new devices attached to see how to change
> fstab/menu.lst. 
> 
> James
> 
> 
> _______________________________________________
> Xen-users mailing list
> Xen-users@lists.xensource.com
> http://lists.xensource.com/xen-users
 
 
Booting a domu from "CD" should be easy. I do it all the time but I
use iso files instead. Basically, just edit your config file for the specific
domu to include the /dev/cdrom (in the case of a physical drive) or the iso
location and change the boot order. These vary depending on if you use libvirt,
xen-tools, or other on exactly how you edit it though.
 
 
-- 
 
Donny B. 

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, 2010-04-22 at 15:14 -0500, Donny Brooks wrote:
> 
> On Thursday, April 22, 2010 03:06 PM CDT, James Pifer
<jep@obrien-pifer.com> wrote:
>  
> > On Thu, 2010-04-22 at 15:02 -0500, Steven Timm wrote:
> > > What type of file system does the domU have, it may be possilbe
> > > to stop the domU and mount the file system on dom0 and make
> > > the change that way.
> > > 
> > > Steve
> > 
> > I was thinking that too and I can mount the filesystem, but the domU
> > needs to be booted with the new devices attached to see how to change
> > fstab/menu.lst. 
> > 
> > James
> > 
> > 
> > _______________________________________________
> > Xen-users mailing list
> > Xen-users@lists.xensource.com
> > http://lists.xensource.com/xen-users
>  
> 
> Booting a domu from "CD" should be easy. I do it all the time but
I
> use iso files instead. Basically, just edit your config file for the
> specific domu to include the /dev/cdrom (in the case of a physical
> drive) or the iso location and change the boot order. These vary
> depending on if you use libvirt, xen-tools, or other on exactly how
> you edit it though.
>  
> 
It should be, but in practice hasn''t been for me. Let me ask you this.
There seems to be multiple ways to modify the environment. On my servers
I currently use virt-manager for starting and stopping vms. You can also
make some changes to configs, which is where I added the pass through
for the scsi card. I assume this updates the xen store?

You can also do the following:
xm list -l vmname > vmname.conf
xm delete vmname
edit the conf file as needed
xm new -F vmname.conf

Then there''s also some type of configuration file under /etc/xen/vm
that
looks like:
name="vmtest03"
uuid="d5bfecad-e163-3661-1732-448c61a6a213"
memory=512
maxmem=512
vcpus=4
on_poweroff="destroy"
on_reboot="restart"
on_crash="destroy"
localtime=0
keymap="en-us"
builder="linux"
bootloader="/usr/lib/xen/boot/domUloader.py"
bootargs="--entry=xvdb1:/boot/vmlinuz-xenpae,/boot/initrd-xenpae"
extra=" "
disk=[ ''phy:/dev/sr0,xvda:cdrom,r'',
''file:/var/lib/xen/images/vmtest03/disk1,xvdb,w'', ]
vif=[ ''mac=00:16:3e:29:83:e6,bridge=br0'', ]
vfb=[''type=vnc,vncunused=1'']

So which way are you saying to edit the config? I have no problem using
an iso, but my experience has been, even after adding a cdrom, that it
sill boots from the hard disk, not the specified cdrom. 

Can you outline a few steps on how it should work?

Thanks,
James







_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Thu, 2010-04-22 at 15:51 -0400, James Pifer wrote:
> I have a sles10 domU running on a sles11 dom0. xen is version 3.4.
We''re
> trying to pass through a scsi card to the domU. We assign the domU the
> scsi card through virt manager. When we try to boot the domU it
won''t
> boot. We need to update fstab and menu.lst, but we need to boot with the
> hardware attached to see how it''s being seen by the domU. 
> 
> On a physical machine we usually boot off the sles boot cd in rescue
> mode and modify the file accordingly. In this case, I can''t get
the domU
> to boot from the cd. 
> 
> Any suggestions on how to do this? I''ve done it with Citrix
XenServer,
> but have never been able to do it on a sles dom0. 
> 
> I''ve tried adding a cdrom and using xm commands to save config,
delete
> the domU, modify the config enabling the cdrom as bootable, then
> restoring the domU with xm new -F command, but then it won''t boot
at
> all. 
Can anyone else provide some assistance here? Really need to be able to
boot off a CD. 

Thanks,
James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

>>> On 2010/04/23 at 05:06, James Pifer <jep@obrien-pifer.com>
wrote:
> On Thu, 2010-04-22 at 15:51 -0400, James Pifer wrote:
>> I have a sles10 domU running on a sles11 dom0. xen is version 3.4.
We''re
>> trying to pass through a scsi card to the domU. We assign the domU the
>> scsi card through virt manager. When we try to boot the domU it
won''t
>> boot. We need to update fstab and menu.lst, but we need to boot with
the
>> hardware attached to see how it''s being seen by the domU. 
>> 
>> On a physical machine we usually boot off the sles boot cd in rescue
>> mode and modify the file accordingly. In this case, I can''t
get the domU
>> to boot from the cd. 
>> 
>> Any suggestions on how to do this? I''ve done it with Citrix
XenServer,
>> but have never been able to do it on a sles dom0. 
>> 
>> I''ve tried adding a cdrom and using xm commands to save
config, delete
>> the domU, modify the config enabling the cdrom as bootable, then
>> restoring the domU with xm new -F command, but then it won''t
boot at
>> all. 
> 
> Can anyone else provide some assistance here? Really need to be able to
> boot off a CD. 
> 
> Thanks,
> James
PV domUs don''t really boot off of a device, per se.  PV domUs must
first load a kernel (and possibly initrd), then the software inside the domU
deals with finishing off the boot process.  In SLES, this happens using either
domUloader or pygrub to find and grab the kernel out of the domU disk and pass
the correct argument.  vm-install searches for a kernel file or kernel package
on the ISO or CD, and unpacks and/or loads that.

All that said, here''s what I''d suggest:
- Use virt-manager and/or vm-install to kick off an installation.
- Specify your existing domU disk, and the SLES11 CD-ROM
- Configure hardware as needed
- On the final screen, where you choose the installation media, make sure to
specify the CD-ROM as the install source (it usually defaults to hard disk if it
finds something on it)
- On that same screen is a place for extra arguments.  Add "rescue=1"
as an extra argument and boot.  This should take you into the rescue system for
the CD-ROM, and you can modify the files as needed.

Another alternative is to do the following:
- Edit your VM config file and add the following for the extra= line:
"init=/bin/bash"
- Reload the config file and boot.
- The system should boot and take you directly to a bash prompt, where you can
remount the root device as R/W (mount -o remount,rw /) and then edit the files.
- Once you''re done, remount as R/O, shut down (or destroy) the domain,
and undo the change to the config file.

-Nick



--------
This e-mail may contain confidential and privileged material for the sole use of
the intended recipient.  If this email is not intended for you, or you are not
responsible for the delivery of this message to the intended recipient, please
note that this message may contain SEAKR Engineering (SEAKR)
Privileged/Proprietary Information.  In such a case, you are strictly prohibited
from downloading, photocopying, distributing or otherwise using this message,
its contents or attachments in any way.  If you have received this message in
error, please notify us immediately by replying to this e-mail and delete the
message from your mailbox.  Information contained in this message that does not
relate to the business of SEAKR is neither endorsed by nor attributable to
SEAKR.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> PV domUs don''t really boot off of a device, per se.  PV domUs must
first load a kernel (and possibly initrd), then the software inside the domU
deals with finishing off the boot process.  In SLES, this happens using either
domUloader or pygrub to find and grab the kernel out of the domU disk and pass
the correct argument.  vm-install searches for a kernel file or kernel package
on the ISO or CD, and unpacks and/or loads that.
> 
> All that said, here''s what I''d suggest:
> - Use virt-manager and/or vm-install to kick off an installation.
> - Specify your existing domU disk, and the SLES11 CD-ROM
> - Configure hardware as needed
> - On the final screen, where you choose the installation media, make sure
to specify the CD-ROM as the install source (it usually defaults to hard disk if
it finds something on it)
> - On that same screen is a place for extra arguments.  Add
"rescue=1" as an extra argument and boot.  This should take you into
the rescue system for the CD-ROM, and you can modify the files as needed.
> 
> Another alternative is to do the following:
> - Edit your VM config file and add the following for the extra= line:
"init=/bin/bash"
> - Reload the config file and boot.
> - The system should boot and take you directly to a bash prompt, where you
can remount the root device as R/W (mount -o remount,rw /) and then edit the
files.
> - Once you''re done, remount as R/O, shut down (or destroy) the
domain, and undo the change to the config file.
> 
> -Nick
> 
Nick, 

Thanks for the response. Always appreciate your help. 

I don''t think the first part will work because I need the scsi pci card
attached for the domU when it boots. So doing it as a new installation
won''t work, unless I can somehow add that PCI device. 

The second part might work if I add the init=/bin/bash and also add
pci=[''0e:04.0'',''0e:04.1''] to the config.
Then xm create it. So it would
be:

xm delete domU
modify config file for extra parameter and pci line
xm create domU
hopefully get to a bash prompt and have enough info to do what is needed
xm destroy domU
xm delete domU
modify config again and remove "extra" parameter
xm create domU

That cover it?

Side note question. How come Citrix XenServer can do this so easily? I
tested XenServer a couple months again, and through their XenCenter
management package it was as easy as selecting a cd iso for the cdrom
device, and the domU would boot. Now that I think about it, maybe that
was only an Windows HVM domU. Maybe a linux PV would have the same
issue...

Thanks,
James


_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

> 
> Nick, 
> 
> Thanks for the response. Always appreciate your help. 
> 
> I don''t think the first part will work because I need the scsi pci
card
> attached for the domU when it boots. So doing it as a new installation
> won''t work, unless I can somehow add that PCI device. 
Ah, I see.
> 
> The second part might work if I add the init=/bin/bash and also add
> pci=[''0e:04.0'',''0e:04.1''] to the
config. Then xm create it. So it would
> be:
> 
> xm delete domU
> modify config file for extra parameter and pci line
> xm create domU
> hopefully get to a bash prompt and have enough info to do what is needed
> xm destroy domU
> xm delete domU
> modify config again and remove "extra" parameter
> xm create domU
> 
> That cover it?
Yes, that should cover it.
> 
> Side note question. How come Citrix XenServer can do this so easily? I
> tested XenServer a couple months again, and through their XenCenter
> management package it was as easy as selecting a cd iso for the cdrom
> device, and the domU would boot. Now that I think about it, maybe that
> was only an Windows HVM domU. Maybe a linux PV would have the same
> issue...
> 
I wouldn''t be surprised if XenServer makes this easier for PV domUs -
vm-install makes it relatively easy, but, for whatever reason, virt-manager
doesn''t really make it easy to tell an existing domU to boot from CD. 
The code is there to make it happen, just not in the correct place.

-Nick



--------
This e-mail may contain confidential and privileged material for the sole use of
the intended recipient.  If this email is not intended for you, or you are not
responsible for the delivery of this message to the intended recipient, please
note that this message may contain SEAKR Engineering (SEAKR)
Privileged/Proprietary Information.  In such a case, you are strictly prohibited
from downloading, photocopying, distributing or otherwise using this message,
its contents or attachments in any way.  If you have received this message in
error, please notify us immediately by replying to this e-mail and delete the
message from your mailbox.  Information contained in this message that does not
relate to the business of SEAKR is neither endorsed by nor attributable to
SEAKR.

_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users

On Fri, 2010-04-23 at 13:25 -0600, Nick Couchman wrote:
> > 
> > Nick, 
> > 
> > Thanks for the response. Always appreciate your help. 
> > 
> > I don''t think the first part will work because I need the
scsi pci
> card
> > attached for the domU when it boots. So doing it as a new
> installation
> > won''t work, unless I can somehow add that PCI device. 
> 
> Ah, I see.
> 
> > 
> > The second part might work if I add the init=/bin/bash and also add
> > pci=[''0e:04.0'',''0e:04.1''] to the
config. Then xm create it. So it
> would
> > be:
> > 
> > xm delete domU
> > modify config file for extra parameter and pci line
> > xm create domU
> > hopefully get to a bash prompt and have enough info to do what is
> needed
> > xm destroy domU
> > xm delete domU
> > modify config again and remove "extra" parameter
> > xm create domU
> > 
> > That cover it?
> 
> Yes, that should cover it.
Not having much luck with this. I was able to boot to bash using the
init=/bin/bash, but I don''t think the pci subsystem gets loaded, so it
doesn''t help a lot. 

Then I found this article, which says if you want to boot an existing
machine off a CD, you have to temporarily change it to an HVM:
http://www.novell.com/communities/node/3046/how-boot-paravirtual-xen-domu-cd-rom-or-iso-sles-10-sp1-or-opensuse-103

I''m going to start another thread though. Further investigation it
seams
that the issue is more scsi related than anything, because the system
boots, just dies during boot. 

Thanks,
James



_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users