Greetings.
I''ve been asked to work on a PXE solution for booting Xen
domU''s from a
network, and would like some feedback and any help anyone is willing to
provide.
I''ve been limiting my work to HVM domU''s at the moment,
because I don''t
think the NBI''s (as far as I know, these are bootloaders) that I have
available work in paravirtualized domU''s. Something build around the
Linux kernel and kexec would be really cool for paravirtualized domU''s.
So, I''d like to know if something like the PXE ROM''s that come
on most
servers nowadays would be more useful than doing something like booting
from PXE client ISO image. Ideally, I think the configuration may look
something like this:
(device
(vif
(bridge xenbr0) (mac 00:16:3e:00:00:11) (type ioemu)
(pxe true)
)
)
Then, I''m thinking the behavior would be to boot PXE on that NIC by
default, then try other boot devices. Of course, the ''boot''
option that
currently accepts ''a'',''b'', or
''c'' could include pxe as an option, but
it''s unclear to me which order the NICs would try to boot in. Maybe
all
of the NICs with PXE ROMs in configuration order.
It looks like Etherboot 5.4.2 has at least the beginnings of full PXE
support for some drivers. AMD pcnet32 looked okay, but I don''t have a
really demanding PXE environment for testing.
Many thanks for your help!
Andrew
====Andrew D. Ball <aball@us dot ibm dot com>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
> It looks like Etherboot 5.4.2 has at least the beginnings of > full PXE support for some drivers. AMD pcnet32 looked okay, > but I don''t have a really demanding PXE environment for testing.We''ll have to see whether vmxassist is up to the job of running the 16b etherboot code on VT hosts. Also, focusing on etherboot rlt8139 support rather than ne2000 or pcnet32 would be better. Best, Ian _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Andrew, For what it''s worth, we''ve been using a PXE solution based upon etherboot. It''s effectively a floppy-like boot image that works quite well. My memory is that there were some instruction decode/emulation issues on VMX, but that we''ve submitted all of those patches to this list. I believe that they were incorporated into xen-unstable. This approach has been working quite well for us for a while now, but may not completely address all of your needs. fwiw, -b On 7/31/06, Andrew D. Ball <aball@us.ibm.com> wrote:> > Greetings. > > I''ve been asked to work on a PXE solution for booting Xen domU''s from a > network, and would like some feedback and any help anyone is willing to > provide. > > I''ve been limiting my work to HVM domU''s at the moment, because I don''t > think the NBI''s (as far as I know, these are bootloaders) that I have > available work in paravirtualized domU''s. Something build around the > Linux kernel and kexec would be really cool for paravirtualized domU''s. > > So, I''d like to know if something like the PXE ROM''s that come on most > servers nowadays would be more useful than doing something like booting > from PXE client ISO image. Ideally, I think the configuration may look > something like this: > > (device > (vif > (bridge xenbr0) (mac 00:16:3e:00:00:11) (type ioemu) > (pxe true) > ) > ) > > Then, I''m thinking the behavior would be to boot PXE on that NIC by > default, then try other boot devices. Of course, the ''boot'' option that > currently accepts ''a'',''b'', or ''c'' could include pxe as an option, but > it''s unclear to me which order the NICs would try to boot in. Maybe all > of the NICs with PXE ROMs in configuration order. > > It looks like Etherboot 5.4.2 has at least the beginnings of full PXE > support for some drivers. AMD pcnet32 looked okay, but I don''t have a > really demanding PXE environment for testing. > > Many thanks for your help! > Andrew > ====> Andrew D. Ball <aball@us dot ibm dot com> > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, 2006-07-31 at 10:57 -0400, Andrew D. Ball wrote:> I''ve been asked to work on a PXE solution for booting Xen domU''s from a > network, and would like some feedback and any help anyone is willing to > provide.Cool> I''ve been limiting my work to HVM domU''s at the moment, because I don''t > think the NBI''s (as far as I know, these are bootloaders) that I have > available work in paravirtualized domU''s. Something build around the > Linux kernel and kexec would be really cool for paravirtualized domU''s.HVM is definitely the first step here.> So, I''d like to know if something like the PXE ROM''s that come on most > servers nowadays would be more useful than doing something like booting > from PXE client ISO image. Ideally, I think the configuration may look > something like this:*nod* So are you looking at integrating into the rombios?> (device > (vif > (bridge xenbr0) (mac 00:16:3e:00:00:11) (type ioemu) > (pxe true) > ) > )Quite possibly PXE would even default to true.> Then, I''m thinking the behavior would be to boot PXE on that NIC by > default, then try other boot devices. Of course, the ''boot'' option that > currently accepts ''a'',''b'', or ''c'' could include pxe as an option, but > it''s unclear to me which order the NICs would try to boot in. Maybe all > of the NICs with PXE ROMs in configuration order.I think it''s definitely better to go with pxe as an option to boot and then doing the NICs in config order. This matches up with how real hardware works[1] Jeremy [1] Also -- does anyone know if the boot option currently take multiple arguments? I''ll have to try when I get to the office. It should be a list and fall back through them as things aren''t available so that a default order of say PXE, CD, HD can be set up and be nice and useful :) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
As far as I know, the boot order option only accepts one of
{''a'',''b'',''c''}. I more
descriptive list like you suggested would be
much, much better.
Peace.
Andrew
On Wed, 2006-08-02 at 08:42 -0400, Jeremy Katz wrote:> On Mon, 2006-07-31 at 10:57 -0400, Andrew D. Ball wrote:
> > I''ve been asked to work on a PXE solution for booting Xen
domU''s from a
> > network, and would like some feedback and any help anyone is willing
to
> > provide.
>
> Cool
>
> > I''ve been limiting my work to HVM domU''s at the
moment, because I don''t
> > think the NBI''s (as far as I know, these are bootloaders)
that I have
> > available work in paravirtualized domU''s. Something build
around the
> > Linux kernel and kexec would be really cool for paravirtualized
domU''s.
>
> HVM is definitely the first step here.
>
> > So, I''d like to know if something like the PXE ROM''s
that come on most
> > servers nowadays would be more useful than doing something like
booting
> > from PXE client ISO image. Ideally, I think the configuration may
look
> > something like this:
>
> *nod* So are you looking at integrating into the rombios?
>
> > (device
> > (vif
> > (bridge xenbr0) (mac 00:16:3e:00:00:11) (type ioemu)
> > (pxe true)
> > )
> > )
>
> Quite possibly PXE would even default to true.
>
> > Then, I''m thinking the behavior would be to boot PXE on that
NIC by
> > default, then try other boot devices. Of course, the
''boot'' option that
> > currently accepts ''a'',''b'', or
''c'' could include pxe as an option, but
> > it''s unclear to me which order the NICs would try to boot in.
Maybe all
> > of the NICs with PXE ROMs in configuration order.
>
> I think it''s definitely better to go with pxe as an option to boot
and
> then doing the NICs in config order. This matches up with how real
> hardware works[1]
>
> Jeremy
>
> [1] Also -- does anyone know if the boot option currently take multiple
> arguments? I''ll have to try when I get to the office. It should
be a
> list and fall back through them as things aren''t available so that
a
> default order of say PXE, CD, HD can be set up and be nice and useful :)
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Hi Ben, On 8/1/06, Ben Thomas <ben@virtualiron.com> wrote:> [...] > For what it''s worth, we''ve been using a PXE solution based upon etherboot. > It''s effectively a floppy-like boot image that works quite well. My memory > is that there were some instruction decode/emulation issues on VMX, but that > we''ve submitted all of those patches to this list. I believe that they were > incorporated into xen-unstable.This is probably more for the xen-users list, but here is the only post I found about somebody doing etherboot with xen hvm, which I am trying currently without luck. Does your mail mean that with xen-testing at the moment my efforts to try booting xen hvm with an etherboot image and pxe cannot succeed? When booting from the etherboot iso, I get "Loading pcnet32.zli....Ready" but no dhcp requests or anything happens. xm dmesg''s last lines are: (XEN) (GUEST: 60) Booting from CD-Rom... (XEN) (GUEST: 60) Unknown opcode at 0F60:0E8A=0x1048A (XEN) (GUEST: 60) Halt called from %eip 0xD39D1 No interesting info xend-debug.log I''d be a candidate for testing xen pxe solutions, because I really want to use it. Henning _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> This is probably more for the xen-users list, but here is the only > post I found about somebody doing etherboot with xen hvm, which I am > trying currently without luck. > Does your mail mean that with xen-testing at the moment my efforts to > try booting xen hvm with an etherboot image and pxe cannot succeed?Tim Deegan this working with the pcnet32 driver some time back, but right now I''d be inclined to try it with an etherboot with an rtl8139 driver (now the default). Ian> When booting from the etherboot iso, I get > "Loading pcnet32.zli....Ready" > > but no dhcp requests or anything happens. xm dmesg''s last lines are: > > (XEN) (GUEST: 60) Booting from CD-Rom... > (XEN) (GUEST: 60) Unknown opcode at 0F60:0E8A=0x1048A > (XEN) (GUEST: 60) Halt called from %eip 0xD39D1 > > No interesting info xend-debug.log > > I''d be a candidate for testing xen pxe solutions, because I really > want to use it. > > Henning > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Henning, It''s been a little while, and my recollection is rusty. I do know that we ended up fixing some opcode emulation when getting the PXE booting to work. I was pretty sure that we had posted that to the xen-devel list, but again I plead a faulty memory on the specifics. I do know that we are constantly (hundreds to thousands of times per day) booting many, many xen-based systems via PXE for both VMX and SVM systems. Currently, we''re using the pcnet32 support. As Ian will (or probably already did) point out, it''s probably worthwhile to move forward. I''d be happy to give you more specifics on what we''re doing to get PXE booted. I doubt that the entire list cares, so please send me an email if you''re interested. -b Henning Sprang wrote:> Hi Ben, > > On 8/1/06, Ben Thomas <ben@virtualiron.com> wrote: > >> [...] >> For what it''s worth, we''ve been using a PXE solution based upon >> etherboot. >> It''s effectively a floppy-like boot image that works quite well. My >> memory >> is that there were some instruction decode/emulation issues on VMX, >> but that >> we''ve submitted all of those patches to this list. I believe that they >> were >> incorporated into xen-unstable. > > > This is probably more for the xen-users list, but here is the only > post I found about somebody doing etherboot with xen hvm, which I am > trying currently without luck. > Does your mail mean that with xen-testing at the moment my efforts to > try booting xen hvm with an etherboot image and pxe cannot succeed? > > When booting from the etherboot iso, I get > "Loading pcnet32.zli....Ready" > > but no dhcp requests or anything happens. xm dmesg''s last lines are: > > (XEN) (GUEST: 60) Booting from CD-Rom... > (XEN) (GUEST: 60) Unknown opcode at 0F60:0E8A=0x1048A > (XEN) (GUEST: 60) Halt called from %eip 0xD39D1 > > No interesting info xend-debug.log > > I''d be a candidate for testing xen pxe solutions, because I really > want to use it. > > Henning-- ------------------------------------------------------------------------ Ben Thomas Virtual Iron Software bthomas@virtualiron.com Tower 1, Floor 2 978-849-1214 900 Chelmsford Street Lowell, MA 01851 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Good morning. Here''s how far I got before I was swamped by other work: -- generated a ROM image for the pcnet32 driver from etherboot. (was before the realtek device) -- loaded the image in different parts of memory that is scanned by the ROMBIOS for expansion ROMS -- Noticed what was probably memory corruption errors preventing the PXE ROM from loading and running This dropped down to the bottom of my priorities, as it''s possible to get the same functionality with etherboot as a virtual ISO image instead of an expansion ROM. This would still be useful as an expansion ROM though, IMHO. The most important work to be done that I can think of before this can work is to clearly document which parts of memory are occupied by what firmware for HVM domU''s and proceed with troubleshooting the memory corruption. If you''d like, we can set up a wiki page to work on this. Hope this helps. Andrew On Mon, 2006-10-09 at 11:57 -0400, Gregory Haskins wrote:> >>> On Mon, Jul 31, 2006 at 10:57 AM, in message > <1154357859.9876.14.camel@localhost.localdomain>, "Andrew D. Ball" > <aball@us.ibm.com> wrote: > > Greetings. > > > > I''ve been asked to work on a PXE solution for booting Xen domU''s from a > > network, and would like some feedback and any help anyone is willing to > > provide. > > Hi Andrew, > I am just wondering what the status of this project is? I have the need for this type of functionality as well. We could possibly collaborate on this effort if work still remains to be done. > > -Greg Haskins > Novell >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Mon, 2006-10-16 at 13:46 -0400, Ben Thomas wrote:> > The most important work to be done that I can think of before > this can > work is to clearly document which parts of memory are occupied > by > what firmware for HVM domU''s and proceed with troubleshooting > the > memory corruption. > > > > > Is this along the lines of what you are looking for ? >Sure is! Didn''t know this was logged -- it''s been awhile since I worked on this part. I think it would be useful to document this layout somewhere, probably on the wiki. I saw an SMBIOS fix that scared me, because it used 0xc0000 as a scrap area. Shouldn''t hurt anything, since the VGA BIOS is loaded after the SMBIOS tables, but scares me nonetheless because I don''t know of any documentation for the HVM virtual firmware other than the code and the logs. Peace. Andrew> (XEN) (GUEST: 6) HVM Loader > (XEN) (GUEST: 6) Detected Xen v3.0-unstable > (XEN) (GUEST: 6) Writing SMBIOS tables [0009f000 - 0009f140] ... > (XEN) (GUEST: 6) Loading ROMBIOS [000f0000 - 00100000] ... > (XEN) (GUEST: 6) Creating MP tables ... > (XEN) (GUEST: 6) Loading Cirrus VGABIOS [000c0000 - 000c8000] ... > (XEN) (GUEST: 6) Loading ACPI [000ea000 - 000eb000] ... > (XEN) (GUEST: 6) Loading VMXAssist [000d0000 - 000d5200] ... > (XEN) (GUEST: 6) VMX go ... > > I have some questions/concerns about the layout of memory, which is > why I have this info. > > -b_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Andrew, I added the address logging to check out a few questions I had. The use of 0C0000 is interesting, but as you note, it''s safe as it''s used as a scratch area before it''s filled in with the vga bios. My main concern is with the use of memory below 0A0000. I''m chasing a problem in an OS, and it sure looks like a problem I submitted a fix for (to the OS, not this project) about a year back. It was related to too little free memory in the first 1MB. Hence, my interest in the layout, especially the use of memory below 0A0000. It''s likely that my suspicions are incorrect, but curiousity usually wins. Anyhow, I saw your note and thought I''d pass on what little I had about it. -b -----Original Message----- From: Andrew D. Ball [mailto:aball@us.ibm.com] Sent: Mon 10/16/2006 3:33 PM To: Ben Thomas; xen-devel@lists.xensource.com Subject: Re: [Xen-devel] Xen PXE solution On Mon, 2006-10-16 at 13:46 -0400, Ben Thomas wrote:> > The most important work to be done that I can think of before > this can > work is to clearly document which parts of memory are occupied > by > what firmware for HVM domU''s and proceed with troubleshooting > the > memory corruption. > > > > > Is this along the lines of what you are looking for ? >Sure is! Didn''t know this was logged -- it''s been awhile since I worked on this part. I think it would be useful to document this layout somewhere, probably on the wiki. I saw an SMBIOS fix that scared me, because it used 0xc0000 as a scrap area. Shouldn''t hurt anything, since the VGA BIOS is loaded after the SMBIOS tables, but scares me nonetheless because I don''t know of any documentation for the HVM virtual firmware other than the code and the logs. Peace. Andrew> (XEN) (GUEST: 6) HVM Loader > (XEN) (GUEST: 6) Detected Xen v3.0-unstable > (XEN) (GUEST: 6) Writing SMBIOS tables [0009f000 - 0009f140] ... > (XEN) (GUEST: 6) Loading ROMBIOS [000f0000 - 00100000] ... > (XEN) (GUEST: 6) Creating MP tables ... > (XEN) (GUEST: 6) Loading Cirrus VGABIOS [000c0000 - 000c8000] ... > (XEN) (GUEST: 6) Loading ACPI [000ea000 - 000eb000] ... > (XEN) (GUEST: 6) Loading VMXAssist [000d0000 - 000d5200] ... > (XEN) (GUEST: 6) VMX go ... > > I have some questions/concerns about the layout of memory, which is > why I have this info. > > -b_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel