Pkg xen devel - Jan 2019 - Bug#776450: Xen PVH support for grub-xen in Buster

On Sun, Jan 06, 2019 at 01:45:53AM +0100, Hans van Kranenburg
wrote:
> Hm, so I have a PV (or PVH) domU in a test environment here, and I tried
> to use it in the Debian way of booting it.
> 
> If I install grub-xen-bin inside the domU then I get some stuff, but not
> update-grub. When I just also install grub2 it drags in more stuff, and
> then I end up with update-grub and a /boot/grub/grub.cfg file as result.
For the record, you wanted grub-xen rather than grub-xen-bin.
> I just copied *.mod that I got from my grub build (with the make blah
> shown before) into /usr/lib/grub/i386-xen_pvh of the domU,
I'd expect them to need to be in /boot/grub/i386-xen_pvh/ in order for
the boot loader to read them; grub-install copies modules there.
(However, a standalone image normally shouldn't need to read modules
from the guest anyway.)
> When trying to start that, I get:
> 
> Reading (memdisk)/boot/grub/grub.cfg
> 
> and then it exits back to the dom0 prompt and the domU is gone.
Seeing (memdisk) there suggests that $root is wrong.  I think that
putting your bootstrap configuration file in /boot/grub/grub.cfg in the
memdisk is a recipe for confusion; grub-xen-host puts it in /grub.cfg
instead, and generally hooks things up a bit differently.  You can
inspect debian/rules to see what it does.

Anyway, this sounds close enough that we can probably go for it and see
where it lands :-)

-- 
Colin Watson                                       [cjwatson at debian.org]

On 1/6/19 11:28 AM, Colin Watson wrote:
> On Sun, Jan 06, 2019 at 01:45:53AM +0100, Hans van Kranenburg wrote:
>> Hm, so I have a PV (or PVH) domU in a test environment here, and I
tried
>> to use it in the Debian way of booting it.
>>
>> If I install grub-xen-bin inside the domU then I get some stuff, but
not
>> update-grub. When I just also install grub2 it drags in more stuff, and
>> then I end up with update-grub and a /boot/grub/grub.cfg file as
result.
> 
> For the record, you wanted grub-xen rather than grub-xen-bin.
Yes, indeed, you're right.
>> I just copied *.mod that I got from my grub build (with the make blah
>> shown before) into /usr/lib/grub/i386-xen_pvh of the domU,
> 
> I'd expect them to need to be in /boot/grub/i386-xen_pvh/ in order for
> the boot loader to read them; grub-install copies modules there.
> (However, a standalone image normally shouldn't need to read modules
> from the guest anyway.)
> 
>> When trying to start that, I get:
>>
>> Reading (memdisk)/boot/grub/grub.cfg
>>
>> and then it exits back to the dom0 prompt and the domU is gone.
> 
> Seeing (memdisk) there suggests that $root is wrong.  I think that
> putting your bootstrap configuration file in /boot/grub/grub.cfg in the
> memdisk is a recipe for confusion; grub-xen-host puts it in /grub.cfg
> instead, and generally hooks things up a bit differently.  You can
> inspect debian/rules to see what it does.
Ok, I've read your explanation in the other email, I reread the xen wiki
page, and now is probably the moment to go back to where I started,
reading debian/rules, and see if I understand it a bit better now.

So, if I go to my upstream build directory and follow the same recipe:

-$ mkdir -p grub_dir/boot/grub
-$ sed -e "s/@@PVBOOT_ARCH@@/i386-xen_pvh/" <
~/path/to/debian/grub/packaging/debian/grub-xen-host_grub.cfg >
grub_dir/grub.cfg
-$ tar -cf - -C grub_dir grub.cfg > grub_memdisk
-$ ./grub-mkimage -O i386-xen_pvh -c
~/build/grub/grub/debian/grub-xen-host_grub-bootstrap.cfg -d
./grub-core/ ./grub-core/*.mod -m grub_memdisk -o grub-i386-xen_pvh.bin

Now I scp this grub-i386-xen_pvh.bin to my Xen dom0 and use it in the
guest config:

---- >8 ----
kernel = "/root/grub-i386-xen_pvh.bin"
type = "pvh"
---- >8 ----

Now I start with xl create -c and voila, a blue grub menu, a countdown
of 5 seconds and the whole thing starts correctly!

To confirm:

-# dmesg |grep PVH
[    0.192293] Booting paravirtualized kernel on Xen PVH

I've been looking at the modules in the domU and even had copied
everything into /boot/grub/i386-xen_pvh manually, but it seems this is
not used at all. It also works if I remove all of that again. So I'm
still wondering what that's for. Maybe the amount of information and
different scenarios and routes that can be chosen is a little bit too
much for me to all grasp at once.
> Anyway, this sounds close enough that we can probably go for it and see
> where it lands :-)
Well, proof of concept (tm) is a great success already.

Hans

On Mon, Jan 07, 2019 at 12:43:12AM +0100, Hans van Kranenburg
wrote:
> So, if I go to my upstream build directory and follow the same recipe:
> 
> -$ mkdir -p grub_dir/boot/grub
> -$ sed -e "s/@@PVBOOT_ARCH@@/i386-xen_pvh/" <
> ~/path/to/debian/grub/packaging/debian/grub-xen-host_grub.cfg >
> grub_dir/grub.cfg
> -$ tar -cf - -C grub_dir grub.cfg > grub_memdisk
> -$ ./grub-mkimage -O i386-xen_pvh -c
> ~/build/grub/grub/debian/grub-xen-host_grub-bootstrap.cfg -d
> ./grub-core/ ./grub-core/*.mod -m grub_memdisk -o grub-i386-xen_pvh.bin
> 
> Now I scp this grub-i386-xen_pvh.bin to my Xen dom0 and use it in the
> guest config:
> 
> ---- >8 ----
> kernel = "/root/grub-i386-xen_pvh.bin"
> type = "pvh"
> ---- >8 ----
> 
> Now I start with xl create -c and voila, a blue grub menu, a countdown
> of 5 seconds and the whole thing starts correctly!
Excellent!  Thanks for the test.
> I've been looking at the modules in the domU and even had copied
> everything into /boot/grub/i386-xen_pvh manually, but it seems this is
> not used at all. It also works if I remove all of that again. So I'm
> still wondering what that's for.
The modules in the domU are used if you use the two-stage system, but
you aren't doing that at this point.  It's also possible for them to be
used in a less reliable way in the one-stage system if your grub.cfg
involves (explicitly or implicitly) loading modules that aren't in the
core image that was created by grub-mkimage.  There are certainly
possible setups that wouldn't use modules from the domU.

-- 
Colin Watson                                       [cjwatson at debian.org]