xen discuss - Mar 2009 - pygrub or pvgrub for linux pv domU on raw device ext3

I have tried pygrub first but it stats nothing found.
Then I downloaded pvgrub it looks good but I think it depends on xen
version (current sxce seems to be too old, would be really nice to have
a 3.3 as fast as possible)

Do someone have more positive experience?

regards,
Florian

2009/3/14 Florian Manschwetus
<florianmanschwetus@gmx.de>:
> I have tried pygrub first but it stats nothing found.
> Do someone have more positive experience?
I''m using pygrub with no problems at all. I use zvols directly as
domU''s storage (without partition).
Does your domU has /boot/grub/menu.lst? AFAIK as long as that file is
present and what it points to (kernel and initrd) exists pygrub can
work correctly (you don''t need grub on domU).

Regards,

Fajar

Fajar A. Nugraha schrieb:
> 2009/3/14 Florian Manschwetus <florianmanschwetus@gmx.de>:
>> I have tried pygrub first but it stats nothing found.
> 
>> Do someone have more positive experience?
> 
> I''m using pygrub with no problems at all. I use zvols directly as
> domU''s storage (without partition).
would be like:
/dev/dsk/c0t600A0B800049E902000007FD49B54652d0p1
(what works well for the domU if I map it as xvda1)

normally I map
/dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0 directly as xvda to
domU so the domU has a full disk to partition on its own.
> Does your domU has /boot/grub/menu.lst? AFAIK as long as that file is
I have a /boot/grub/grub.conf
> present and what it points to (kernel and initrd) exists pygrub can
I have no initrd (isn''t needed normally)
> work correctly (you don''t need grub on domU).
> 
> Regards,
> 
> Fajar
> 
Regards,
Florian

On Sat, Mar 14, 2009 at 9:40 PM, Florian Manschwetus
<florianmanschwetus@gmx.de> wrote:
>> I''m using pygrub with no problems at all. I use zvols directly
as
>> domU''s storage (without partition).
> would be like:
> /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p1
> (what works well for the domU if I map it as xvda1)
>
> normally I map
> /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0 directly as xvda to
> domU so the domU has a full disk to partition on its own.
It depends more on your preferences, actually.
You could map a file (.img, .vmdk, whatever), a zvol, a disk, or a
partition as domU''s disk or partition. So mapping
/dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0 as xvda1 should work
as well. But that would somewhat limit scalability since xvda1 will be
limited to that size (i.e. you can''t grow it), which is why I prefer
to use zvol (on opensolaris) or LVM (on Linux).

pygrub can work with block devices mapped as disk (xvda) or partition
(xvda1), so I''m curious why you''re experiencing problem.
>
>> Does your domU has /boot/grub/menu.lst? AFAIK as long as that file is
> I have a /boot/grub/grub.conf
Try symlink it to menu.lst. Should work.
> I have no initrd (isn''t needed normally)
So I take it you''re not using RHEL/Centos or Debian/Ubuntu domU, or
you''re building your own kernel? By default those systems need initrd
to boot correctly.
>From pygrub''s perspective it doesn''t really matter though,
it can work
with or without initrd.

Regards,

Fajar

Fajar A. Nugraha schrieb:
> On Sat, Mar 14, 2009 at 9:40 PM, Florian Manschwetus
> <florianmanschwetus@gmx.de> wrote:
>>> I''m using pygrub with no problems at all. I use zvols
directly as
>>> domU''s storage (without partition).
>> would be like:
>> /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p1
>> (what works well for the domU if I map it as xvda1)
>>
>> normally I map
>> /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0 directly as xvda to
>> domU so the domU has a full disk to partition on its own.
The ...p0 I use as full disk (xvda)
> 
> It depends more on your preferences, actually.
> You could map a file (.img, .vmdk, whatever), a zvol, a disk, or a
> partition as domU''s disk or partition. So mapping
> /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0 as xvda1 should work
> as well. But that would somewhat limit scalability since xvda1 will be
> limited to that size (i.e. you can''t grow it), which is why I
prefer
> to use zvol (on opensolaris) or LVM (on Linux).
Those device are mapped from FC SAN.
> 
> pygrub can work with block devices mapped as disk (xvda) or partition
> (xvda1), so I''m curious why you''re experiencing problem.
> 
so why he always say that he is unable to find anything (what is about
symlinks for the kernel?)
>> I have a /boot/grub/grub.conf
> 
> Try symlink it to menu.lst. Should work.
Those symlink is already in place, but I''ll alter it to use a base
path,
not relative one.
> 
>> I have no initrd (isn''t needed normally)
> 
> So I take it you''re not using RHEL/Centos or Debian/Ubuntu domU,
or
> you''re building your own kernel? By default those systems need
initrd
> to boot correctly.
> 
Yes I build my kernel on my own it is normal for gentoo linux
>>From pygrub''s perspective it doesn''t really matter
though, it can work
> with or without initrd.
> 
So where is the problem?
I pygrub unable to read the ext3 partition when the dom0 is solaris
nevada (SXCE b109)?
> Regards,
> 
> Fajar
> 
Regards,
Florian

Ok, I think I got it, the pygrub delivered with SXCE b109, supports only
Solaris (or similar), if I read the source right.
There is nothing in it about ext or so.

Regards,
Florian

Florian Manschwetus schrieb:
> Fajar A. Nugraha schrieb:
>> On Sat, Mar 14, 2009 at 9:40 PM, Florian Manschwetus
>> <florianmanschwetus@gmx.de> wrote:
>>>> I''m using pygrub with no problems at all. I use zvols
directly as
>>>> domU''s storage (without partition).
>>> would be like:
>>> /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p1
>>> (what works well for the domU if I map it as xvda1)
>>>
>>> normally I map
>>> /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0 directly as xvda
to
>>> domU so the domU has a full disk to partition on its own.
> The ...p0 I use as full disk (xvda)
>> It depends more on your preferences, actually.
>> You could map a file (.img, .vmdk, whatever), a zvol, a disk, or a
>> partition as domU''s disk or partition. So mapping
>> /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0 as xvda1 should work
>> as well. But that would somewhat limit scalability since xvda1 will be
>> limited to that size (i.e. you can''t grow it), which is why I
prefer
>> to use zvol (on opensolaris) or LVM (on Linux).
> Those device are mapped from FC SAN.
>> pygrub can work with block devices mapped as disk (xvda) or partition
>> (xvda1), so I''m curious why you''re experiencing
problem.
>>
> so why he always say that he is unable to find anything (what is about
> symlinks for the kernel?)
> 
>>> I have a /boot/grub/grub.conf
>> Try symlink it to menu.lst. Should work.
> Those symlink is already in place, but I''ll alter it to use a base
path,
> not relative one.
>>> I have no initrd (isn''t needed normally)
>> So I take it you''re not using RHEL/Centos or Debian/Ubuntu
domU, or
>> you''re building your own kernel? By default those systems need
initrd
>> to boot correctly.
>>
> Yes I build my kernel on my own it is normal for gentoo linux
> 
>>> >From pygrub''s perspective it doesn''t really
matter though, it can work
>> with or without initrd.
>>
> So where is the problem?
> I pygrub unable to read the ext3 partition when the dom0 is solaris
> nevada (SXCE b109)?
> 
>> Regards,
>>
>> Fajar
>>
> 
> Regards,
> Florian
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> xen-discuss mailing list
> xen-discuss@opensolaris.org

2009/3/15 Florian Manschwetus
<florianmanschwetus@gmx.de>:
> Ok, I think I got it, the pygrub delivered with SXCE b109, supports only
> Solaris (or similar), if I read the source right.
> There is nothing in it about ext or so.
That can''t be right, as I''m using it.
Try this :
- make sure the directory /var/run/xend/boot exists. If it''s not,
create it with mode 700, user/group xvm/xvm (creating a domU should
also create this directory, if it doesn''t exist)
- run this

/usr/lib/xen/bin/pygrub /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0

(or whatever your block device is). It should bring up a pygrub menu
and, afterwards print something like this

linux (kernel /var/run/xend/boot/boot_kernel.-iUVQK)(ramdisk
/var/run/xend/boot/boot_ramdisk.M44YrM)(args "ro root=/dev/hda1")

If you get that, then pygrub is working correctly. Delete the files
/var/run/xend/boot/boot_kernel.-iUVQK and
/var/run/xend/boot/boot_ramdisk.M44YrM (or whatever file pygrub print
out).

If it doesn''t work, try this :

- map the FC SAN LUN to a Linux dom0, and try it there
- post the content of your /boot/grub/menu.lst. Perhaps there''s an
error in it. It will also be helpful to post your domU''s partition
table.

Regards,

Fajar

Fajar A. Nugraha schrieb:
> That can''t be right, as I''m using it.
> Try this :
> - make sure the directory /var/run/xend/boot exists. If it''s not,
No was not, created it.
> create it with mode 700, user/group xvm/xvm (creating a domU should
> also create this directory, if it doesn''t exist)
~

saturn 10:41:35 # ls /var/run/xend/boot
/var/run/xend/boot: Datei oder Verzeichnis nicht gefunden
~
saturn 12:38:28 # mkdir /var/run/xend/boot
~
saturn 12:38:51 # chown xvm:xvm /var/run/xend/boot
~
saturn 12:38:54 # chmod 700 /var/run/xend/boot
> - run this
> 
> /usr/lib/xen/bin/pygrub /dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0
> 
> (or whatever your block device is). It should bring up a pygrub menu
> and, afterwards print something like this
Sorry,
~
saturn 12:41:57 # /usr/lib/xen/bin/pygrub
/dev/dsk/c0t600A0B800049E902000007FD49B54652d0p1Traceback (most recent
call last):
  File "/usr/lib/xen/bin/pygrub", line 658, in ?
    chosencfg = run_grub(file, entry, fs)
  File "/usr/lib/xen/bin/pygrub", line 509, in run_grub
    g = Grub(file, fs)
  File "/usr/lib/xen/bin/pygrub", line 198, in __init__
    self.read_config(file, fs)
  File "/usr/lib/xen/bin/pygrub", line 388, in read_config
    raise RuntimeError, "couldn''t find bootloader config file in
the
image provided."
RuntimeError: couldn''t find bootloader config file in the image
provided.
~
saturn 12:41:59 # /usr/lib/xen/bin/pygrub
/dev/dsk/c0t600A0B800049E902000007FD49B54652d0p0
Traceback (most recent call last):
  File "/usr/lib/xen/bin/pygrub", line 658, in ?
    chosencfg = run_grub(file, entry, fs)
  File "/usr/lib/xen/bin/pygrub", line 509, in run_grub
    g = Grub(file, fs)
  File "/usr/lib/xen/bin/pygrub", line 198, in __init__
    self.read_config(file, fs)
  File "/usr/lib/xen/bin/pygrub", line 388, in read_config
    raise RuntimeError, "couldn''t find bootloader config file in
the
image provided."
RuntimeError: couldn''t find bootloader config file in the image
provided.
> If it doesn''t work, try this :
> 
> - map the FC SAN LUN to a Linux dom0, and try it there
No linux next there.
> - post the content of your /boot/grub/menu.lst. Perhaps there''s an
(attached and mounted in an other udom)
~
compute1 12:41:53 # mount /dev/xvdc1 /mnt/cdrom/
~
compute1 12:43:54 # cat /mnt/cdrom/boot/grub/menu.lst
# This is a sample grub.conf for use with Genkernel, per the Gentoo handbook
#
http://www.gentoo.org/doc/en/handbook/handbook-x86.xml?part=1&chap=10#doc_chap2
# If you are not using Genkernel and you need help creating this file, you
# should consult the handbook. Alternatively, consult the
grub.conf.sample that
# is included with the Grub documentation.

default 0
fallback 1
timeout 30
splashimage=(hd0,0)/boot/grub/splash.xpm.gz

title  Gentoo Linux
root (hd0,0)
kernel (hd0,0)/boot/current root=/dev/xvda1 pcie_aspm.policy=powersave

title  Gentoo Linux Backup
root (hd0,0)
kernel /boot/backup root=/dev/xvda1

# vim:ft=conf:
> error in it. It will also be helpful to post your domU''s partition
> table.
fdisk -l /dev/xvdc

Disk /dev/xvdc: 12.8 GB, 12884901888 bytes
255 heads, 63 sectors/track, 1566 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

    Device Boot      Start         End      Blocks   Id  System
/dev/xvdc1   *           1        1305    10482381   83  Linux
/dev/xvdc2            1306        1566     2096482+  83  Linux

Regards,
Florian

On Sun, Mar 15, 2009 at 6:50 PM, Florian Manschwetus
<florianmanschwetus@gmx.de> wrote:
> RuntimeError: couldn''t find bootloader config file in the image
provided.
>> - post the content of your /boot/grub/menu.lst. Perhaps
there''s an
> (attached and mounted in an other udom)
> ~
At this point I''m out of ideas. Sorry. I did some experiments though,
and I know that :
- pygrub (both on opensolaris and linux) should be able to read
/boot/grub/grub.conf or menu.lst, whicever one exists
- it will work whether the block device has partitions or directly
formatted as ext3

One last test : try installing pygrub (on RHEL this is done with "yum
install xen") on your other domU (the one that can mount that block
device), and run

pygrub /dev/xvdc

and

pygrub /dev/xvdc1

both should be able to return a kernel and initrd.
If it does, there''s probably something weird in your opensolaris.
If it doesn''t, there''s something wrong in the way your domU is
setup.

On Sat, Mar 14, 2009 at 12:40:43PM +0100, Florian Manschwetus wrote:
> I have tried pygrub first but it stats nothing found.
> Then I downloaded pvgrub it looks good but I think it depends on xen
> version (current sxce seems to be too old, would be really nice to have
> a 3.3 as fast as possible)
They made an incompatible change to ext3 fairly recently. Most likely,
this is your problem. You''ll have to wait for 3.3 which has changes for
the ext3 format change.

regards
john