Xen devel - Apr 2006 - Deficiencies in guest bootloader design

I have tried to use a dynamically activated block device with pygrub 
(actually drbd, but could be enbd, nbd, or simply any vbd other than phy: or 
file:). That does not work.

The problem is that the bootloader is not called from Xend, after virtual 
block devices are setup, but instead it is called directly from xm, i.e. in 
xm/create.py.

If I have looked correctly, (1) a bootloader cannot be used for devices such 
as nbd: or enbd: devices, and (2) a bootloader cannot be used when accessing 
Xend via XML-RPC (or libvirt or similar), because the whole notion of a 
"bootloader" at domain creation time is only available in
"xm", but not in
Xend.

It should be fixable, since at least on guest *restart*, 
xend/XendDomainInfo.py already uses the bootloader to re-extract the kernel 
image from the vbd.

Is there anyone working on bootloader improvements? If not, is a change to 
move the bootloader stuff from xm to Xend acceptable? In that case I could 
try to come up with a patch.

There are some issues though. In XendDomainInfo.initDomain, image.create is 
done before self.createDevices. Can this be changed easily?

Thanks for any help or guidance.

Best Regards,
Michael Paesold 



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Michael Paesold wrote:
> I have tried to use a dynamically activated block device with pygrub 
> (actually drbd, but could be enbd, nbd, or simply any vbd other than 
> phy: or file:). That does not work.
> 
Have you looked at domUloader?  I inquired about a recent version and 
was told there was no project page, yet.  The patches I have will not 
patch against 3.0-testing, way too many rejects.

Thank you,
Matt Ayres

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Apr 13, 2006 at 04:29:23PM +0200, Michael Paesold wrote:
> I have tried to use a dynamically activated block device with pygrub 
> (actually drbd, but could be enbd, nbd, or simply any vbd other than phy: 
> or file:). That does not work.
> 
> The problem is that the bootloader is not called from Xend, after virtual 
> block devices are setup, but instead it is called directly from xm, i.e. in
> xm/create.py.
> 
> If I have looked correctly, (1) a bootloader cannot be used for devices 
> such as nbd: or enbd: devices, and (2) a bootloader cannot be used when 
> accessing Xend via XML-RPC (or libvirt or similar), because the whole 
> notion of a "bootloader" at domain creation time is only
available in "xm",
> but not in Xend.
> 
> It should be fixable, since at least on guest *restart*, 
> xend/XendDomainInfo.py already uses the bootloader to re-extract the kernel
> image from the vbd.
> 
> Is there anyone working on bootloader improvements? If not, is a change to 
> move the bootloader stuff from xm to Xend acceptable? In that case I could 
> try to come up with a patch.
Certainly such a patch would be accepted.  There was a long thread discussing
bootloaders recently -- you should dig it out of the archive and see what the
conclusions were -- there seem to be a few competing ideas for bootloader
support in Xen (pygrub and domUloader being the two prominent ones).  If you
were going to put some effort into this, it would be appreciated by a number
of the Xen users, certainly.
> There are some issues though. In XendDomainInfo.initDomain, image.create is
> done before self.createDevices. Can this be changed easily?
Off the top of my head, I can''t think why this would be too much of a
problem.
Why do you ask though?  If the bootloader is running inside the guest, then
that is happening after both of those things, because at the point at which we
are doing initDomain, the guest hasn''t even started running yet.  Why
does
ordering here concern you?

Don''t forget that VMX domains handle their devices differently to
para-virt
domains, and the VMX folks won''t appreciate it if you break their code
;-)

Cheers,

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ewan Mellor wrote:
> On Thu, Apr 13, 2006 at 04:29:23PM +0200, Michael Paesold wrote:
>
>   
>> I have tried to use a dynamically activated block device with pygrub 
>> (actually drbd, but could be enbd, nbd, or simply any vbd other than
phy:
>> or file:). That does not work.
>>
>> The problem is that the bootloader is not called from Xend, after
virtual
>> block devices are setup, but instead it is called directly from xm,
i.e. in
>> xm/create.py.
>>
>> If I have looked correctly, (1) a bootloader cannot be used for devices
>> such as nbd: or enbd: devices, and (2) a bootloader cannot be used when
>> accessing Xend via XML-RPC (or libvirt or similar), because the whole 
>> notion of a "bootloader" at domain creation time is only
available in "xm",
>> but not in Xend.
>>
>> It should be fixable, since at least on guest *restart*, 
>> xend/XendDomainInfo.py already uses the bootloader to re-extract the
kernel
>> image from the vbd.
>>
>> Is there anyone working on bootloader improvements? If not, is a change
to
>> move the bootloader stuff from xm to Xend acceptable? In that case I
could
>> try to come up with a patch.
>>     
>
> Certainly such a patch would be accepted.
>   
Just as a warning, it''s not going to be simple to run something like 
pygrub from Xend.  The reason it runs from xm now is that it needs to 
interact with the user.  If it were launched from Xend, it need to find 
a way to attach to a PTY that xenconsoled would use so that it can 
display the text over the normal console before actually launching the 
domain.

It''s certainly not impossible but it touches quite a few things.

Regards,

Anthony Liguori
>> There are some issues though. In XendDomainInfo.initDomain,
image.create is
>> done before self.createDevices. Can this be changed easily?
>>     
>
> Off the top of my head, I can''t think why this would be too much
of a problem.
> Why do you ask though?  If the bootloader is running inside the guest, then
> that is happening after both of those things, because at the point at which
we
> are doing initDomain, the guest hasn''t even started running yet. 
Why does
> ordering here concern you?
>
> Don''t forget that VMX domains handle their devices differently to
para-virt
> domains, and the VMX folks won''t appreciate it if you break their
code ;-)
>
> Cheers,
>
> Ewan.
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, Apr 13, 2006 at 10:24:44AM -0500, Anthony Liguori wrote:
> Ewan Mellor wrote:
> >On Thu, Apr 13, 2006 at 04:29:23PM +0200, Michael Paesold wrote:
> >
> >  
> >>I have tried to use a dynamically activated block device with
pygrub
> >>(actually drbd, but could be enbd, nbd, or simply any vbd other
than phy:
> >>or file:). That does not work.
> >>
> >>The problem is that the bootloader is not called from Xend, after
virtual
> >>block devices are setup, but instead it is called directly from xm,
i.e.
> >>in xm/create.py.
> >>
> >>If I have looked correctly, (1) a bootloader cannot be used for
devices
> >>such as nbd: or enbd: devices, and (2) a bootloader cannot be used
when
> >>accessing Xend via XML-RPC (or libvirt or similar), because the
whole
> >>notion of a "bootloader" at domain creation time is only
available in
> >>"xm", but not in Xend.
> >>
> >>It should be fixable, since at least on guest *restart*, 
> >>xend/XendDomainInfo.py already uses the bootloader to re-extract
the
> >>kernel image from the vbd.
> >>
> >>Is there anyone working on bootloader improvements? If not, is a
change
> >>to move the bootloader stuff from xm to Xend acceptable? In that
case I
> >>could try to come up with a patch.
> >>    
> >
> >Certainly such a patch would be accepted.
> >  
> 
> Just as a warning, it''s not going to be simple to run something
like
> pygrub from Xend.  The reason it runs from xm now is that it needs to 
> interact with the user.  If it were launched from Xend, it need to find 
> a way to attach to a PTY that xenconsoled would use so that it can 
> display the text over the normal console before actually launching the 
> domain.
> 
> It''s certainly not impossible but it touches quite a few things.
So how _is_ the virtual frame buffer coming along?  ;-)

Ewan.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ewan Mellor wrote:
> On Thu, Apr 13, 2006 at 10:24:44AM -0500, Anthony Liguori wrote: 
>   
>> Just as a warning, it''s not going to be simple to run
something like
>> pygrub from Xend.  The reason it runs from xm now is that it needs to 
>> interact with the user.  If it were launched from Xend, it need to find
>> a way to attach to a PTY that xenconsoled would use so that it can 
>> display the text over the normal console before actually launching the 
>> domain.
>>
>> It''s certainly not impossible but it touches quite a few
things.
>>     
>
> So how _is_ the virtual frame buffer coming along?  ;-)
>   
Yeah, I''ve talked to Jeremy about the fact that the virtual frame
buffer
just makes the situation 100 times worse :-)

Regards,

Anthony Liguori
> Ewan.
>   

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ewan Mellor wrote:
>> Is there anyone working on bootloader improvements? If not, is a change
>> to
>> move the bootloader stuff from xm to Xend acceptable? In that case I 
>> could
>> try to come up with a patch.
>
> Certainly such a patch would be accepted.
I already read Anthony''s reply -- so I see some obstacles now :-)
> There was a long thread discussing
> bootloaders recently -- you should dig it out of the archive and see what 
> the
> conclusions were -- there seem to be a few competing ideas for bootloader
> support in Xen (pygrub and domUloader being the two prominent ones).  If 
> you
> were going to put some effort into this, it would be appreciated by a 
> number
> of the Xen users, certainly.
I am going to do that now.
>> There are some issues though. In XendDomainInfo.initDomain,
image.create
>> is
>> done before self.createDevices. Can this be changed easily?
>
> Off the top of my head, I can''t think why this would be too much
of a
> problem.
> Why do you ask though?  If the bootloader is running inside the guest, 
> then
> that is happening after both of those things, because at the point at 
> which we
> are doing initDomain, the guest hasn''t even started running yet. 
Why does
> ordering here concern you?
Well, with pygrub at least, the bootloader is not really executed in the 
guest, but in dom0. It extracts kernel and initrd into temporary files which 
are than booted as if they were directly specified in the guest config. So 
to extract the kernel, we would have to first setup the vbds.

I am going to check the archives now.

Best Regards,
Michael Paesold 



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Matt Ayres wrote:
> Michael Paesold wrote:
>> I have tried to use a dynamically activated block device with pygrub 
>> (actually drbd, but could be enbd, nbd, or simply any vbd other than 
>> phy: or file:). That does not work.
>>
> 
> Have you looked at domUloader?  I inquired about a recent version and 
> was told there was no project page, yet.  The patches I have will not 
> patch against 3.0-testing, way too many rejects.
No, not yet. Where did you get the patches from, this mailing list? -- in 
that case I should find them in the archives.

I am going to take a look. Thanks.

Best Regards,
Michael Paesold

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Anthony Liguori wrote:
> Ewan Mellor wrote:
>> On Thu, Apr 13, 2006 at 04:29:23PM +0200, Michael Paesold wrote:
 >>
>>> I have tried to use a dynamically activated block device with
pygrub
>>> (actually drbd, but could be enbd, nbd, or simply any vbd other
than
>>> phy: or file:). That does not work.
...
>>> Is there anyone working on bootloader improvements? If not, is a 
>>> change to move the bootloader stuff from xm to Xend acceptable? In 
>>> that case I could try to come up with a patch.
>>>     
>>
>> Certainly such a patch would be accepted.
> 
> Just as a warning, it''s not going to be simple to run something
like
> pygrub from Xend.  The reason it runs from xm now is that it needs to 
> interact with the user.  If it were launched from Xend, it need to find 
> a way to attach to a PTY that xenconsoled would use so that it can 
> display the text over the normal console before actually launching the 
> domain.
> 
> It''s certainly not impossible but it touches quite a few things.
Thanks for that important hint. Hmm, output of pygrub is only really 
important in the case when xm create is called with -c. In that case the 
console must be setup anyway, but I guess that this only happens after at 
least part of the guest initialization.

<digression>
After reading through past discussion, I have a feeling that porting real 
boot loaders to xen (e.g. grub) would be a better long term solution.

Just for booting linux, pygrub is really ok, and I prefer it over 
domUloader because it relies on existing infrastructure in domU (menu.lst) 
to get the correct kernel to boot ("yum update" inside domU will do
the
right thing once grub configuration is setup correctly).

On the other hand pygrub only handles ext2/3 and reiserfs, so booting a bsd 
guest won''t work easily.
</digression>

Best Regards,
Michael Paesold

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ewan Mellor wrote:
>> Is there anyone working on bootloader improvements? If not, is a change
to
>> move the bootloader stuff from xm to Xend acceptable? In that case I
could
>> try to come up with a patch.
> 
> Certainly such a patch would be accepted.  There was a long thread
discussing
> bootloaders recently -- you should dig it out of the archive and see what
the
> conclusions were -- there seem to be a few competing ideas for bootloader
> support in Xen (pygrub and domUloader being the two prominent ones).  If
you
> were going to put some effort into this, it would be appreciated by a
number
> of the Xen users, certainly.
I have read through the threads about pygrub/domUloader in the archives, 
although I have not found discussion about the initial inclusion of pygrub. 
I mostly see discussions about the actual features of the boot loaders 
themselves, not really about integration relevant to my question.

I see the most important question is that of interaction with the user, 
since that seems to be the main reason for bootloader being invoked from xm 
instead of Xend. I will study the code to see what can be done.

I appreciate any input or help!

Best Regards,
Michael

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel