Syslinux - Mar 2016 - Updated status on UEFI compliant version of the pxechn-module

Hi,
________________________________________
From: Gene Cumm <gene.cumm at gmail.com>
Sent: Wednesday, March 16, 2016 00:56
To: Jan Frode J?ger
Cc: syslinux at zytor.com
Subject: Re: [syslinux] Updated status on UEFI compliant version of the
pxechn-module

On Tue, Mar 15, 2016 at 9:32 AM, Jan Frode J?ger
<jan.frode.jaeger at ntnu.no> wrote:

Without trying to figure out where data is stored or the possibility
of calls to manipulate the data, I can't speak for the possibility of
implementing anything that manipulates the options like pxechn.c32 for
UEFI.  I'd like to get UEFI chaining working first.

The shear length comparison speaks a lot for the differences.  Yes,
I've read through most of the PXE spec as it relates to the boot
process.  As bad as you think PXE is, consider how UEFI has been a
moving target in some respects.

What do you expect to PXE chainload?  If you could provide details on
what you hope to achieve, someone might be able to offer a suggestion
or workaround.

I understand. Just let me say that I appreciate all the work that has been done
on syslinux and that it is a great tool. I was only wanting to hear what your
plans were.

As goes the PXE chainloading - I am trying to redirect from a PXELINUX menu to a
WDS boot server with SCCM. So the option in my menu looks like this:

***SNIP***
label sccmserver
   menu label SCCM-server
   kernel /mods/pxechn.c32
   append 1.2.3.4::boot\x86\wdsnbp.com
***SNIP***

The IP here is of course just changed for this example. I have also tried to add
the "-W" parameter to pxechn, but without any change in result. It
always says "Error: Unable to retrieve first packet" and then
"Could not unpack packet".
> I just compiled up the 6.04-prerelease and can see that the pxechn-module
has been removed from the two efi-trees. As a sidenote: the NEWS-file in the
6.04-prerelease is missing information about changes introduced in it (seems to
be the file from 6.03).
NEWS has NOT been updated yet.
> As another sidenote and possible separate thread: since I couldn't get
UEFI to boot pxelinux 6.03 properly before I saw your dhcp options fix (vendor
encapsulated options), I implemented a simple PXE boot server ack-packet UDP
server in perl (the PXE client basically tries to perform a PXE boot server
discovery procedure). Works like a charm without the dhcp options under all the
Intel Boot Agent versions I tried - only need to set the mtftp-server address as
per recommendations in the PXE 2.1 spec. Is there any interest in me publishing
this boot server ack-packet script for others to use? Microsoft and others want
us to drop DHCP options and instead implement iphelpers in the router (if
clients and servers are on separate subnets), but shouldn't be necessary as
far as I can see. The origin of all this mess is again the PXE specification in
my opinion...
Intel Boot Agent?  Really?  I've NEVER seen mention of "Intel Boot
Agent" on UEFI.

I'd certainly be interested in this Perl script.

Sorry, misspeak - I have been working on this issue for an intensive period....
Of course the Intel Boot Agent (IBA) is under BIOS. Under UEFI PXE-boot I
developed the aforementioned boot server script to deal with the boot server
discovery being attempted by the UEFI PXE agent. It is based on some tcpdumps of
the network traffic between the tftp-server and an ack-packet from a WDS boot
server. Stripped away the proprietary dhcp options from the WDS boot server
though and used the definiton of an ack-packet in the PXE 2.1 specification as
soon as I understood what was going on. It is a very rudimentary script and was
only made to see if I could get the UEFI client to accept my Linux tftp-server
and its image-file (which was the syslinux.efi file for efi64 from version
6.03). How do you want the script - send it to your private email address?

Best,

    Jan-Frode

--
-Gene
> ________________________________________
> From: Gene Cumm <gene.cumm at gmail.com>
> Sent: Tuesday, March 15, 2016 13:42
> To: Jan Frode J?ger
> Cc: syslinux at zytor.com
> Subject: Re: [syslinux] Updated status on UEFI compliant version of the
pxechn-module
>
> On Tue, Mar 15, 2016 at 3:51 AM, Jan Frode J?ger <syslinux at
zytor.com> wrote:
>> Hi,
>>
>>
>> I have browsed through your syslinux archives and see that the issue of
an UEFI compliant version of the pxechn-module has come up several times:
>>
>>
>> http://www.syslinux.org/archives/2014-August/022611.html
>>
>> http://www.syslinux.org/archives/2015-March/023320.html
>>
>> http://www.syslinux.org/archives/2015-September/024231.html
>>
>>
>> Now the issue as I understand it is that the original pxechn-module has
been written to interface with legacy BIOS and not UEFI. It also seems that
Patricks update as of March 2015 gave positive indications that a general fix
for com32-modules were possible.
>>
>>
>> It would have been nice to receive some update on both com32-module
compliance for the UEFI architecture as well as for the pxechn-module in
specific. We have a setup at my workplace that are dependant upon this working
and right now the only solution is to use legacy BIOS PXE-boot.
>>
>>
>> Is there any fix for this in the recently mentioned 6.04 pre-release
(soon to become release?)?
>
> No, I removed all modules with BIOSisms from UEFI builds like pxechn.c32.
>
> More importantly, Syslinux can't load arbitrary UEFI images yet.
>
> --
> -Gene

On Wed, Mar 16, 2016 at 2:58 AM, Jan Frode J?ger
<jan.frode.jaeger at ntnu.no> wrote:
> Hi,
> ________________________________________
> From: Gene Cumm <gene.cumm at gmail.com>
> Sent: Wednesday, March 16, 2016 00:56
> To: Jan Frode J?ger
> Cc: syslinux at zytor.com
> Subject: Re: [syslinux] Updated status on UEFI compliant version of the
pxechn-module
>
> On Tue, Mar 15, 2016 at 9:32 AM, Jan Frode J?ger
> <jan.frode.jaeger at ntnu.no> wrote:
>
> Without trying to figure out where data is stored or the possibility
> of calls to manipulate the data, I can't speak for the possibility of
> implementing anything that manipulates the options like pxechn.c32 for
> UEFI.  I'd like to get UEFI chaining working first.
>
> The shear length comparison speaks a lot for the differences.  Yes,
> I've read through most of the PXE spec as it relates to the boot
> process.  As bad as you think PXE is, consider how UEFI has been a
> moving target in some respects.
>
> What do you expect to PXE chainload?  If you could provide details on
> what you hope to achieve, someone might be able to offer a suggestion
> or workaround.
>
> I understand. Just let me say that I appreciate all the work that has been
done on syslinux and that it is a great tool. I was only wanting to hear what
your plans were.
>
> As goes the PXE chainloading - I am trying to redirect from a PXELINUX menu
to a WDS boot server with SCCM. So the option in my menu looks like this:
>
> ***SNIP***
> label sccmserver
>    menu label SCCM-server
>    kernel /mods/pxechn.c32
>    append 1.2.3.4::boot\x86\wdsnbp.com
> ***SNIP***
I had suspicions.

That will be impossible as-is.  You can not chainload to a BIOS NBP in
a UEFI environment.  If your WDS isn't ready for UEFI, you need to
correct that first.  Many UEFI systems include a CSM (Compatibility
Support Module) but you can't boot across like that on any system I've
ever seen.  Your boot payload will be one of two, depending on the
CPU, a x86 (32-bit) or x64 (64-bit; aka x86-64) UEFI binary.  Although
less common, x86 UEFI does exist on numerous embedded systems and some
low-end systems.
> The IP here is of course just changed for this example. I have also tried
to add the "-W" parameter to pxechn, but without any change in result.
It always says "Error: Unable to retrieve first packet" and then
"Could not unpack packet".
Use designated documentation IPs or private IPs.  "192.0.2.0/24
(TEST-NET-1), 198.51.100.0/24 (TEST-NET-2), and 203.0.113.0/24
(TEST-NET-3) are provided for use in documentation." RFC5737.  RFC
1918 specifies 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16 as
private.  RFC6598 allocated 100.64.0.0/10 for CGN, also a private
space.  APNIC had to declare several subnets as forbidden when it was
issued 1.0.0.0/8 for real allocation in 2010, including for example
1.1.1.0/24 and 1.2.3.0/24, and waste some of its allocation.
>> I just compiled up the 6.04-prerelease and can see that the
pxechn-module has been removed from the two efi-trees. As a sidenote: the
NEWS-file in the 6.04-prerelease is missing information about changes introduced
in it (seems to be the file from 6.03).
>
> NEWS has NOT been updated yet.
>
>> As another sidenote and possible separate thread: since I couldn't
get UEFI to boot pxelinux 6.03 properly before I saw your dhcp options fix
(vendor encapsulated options), I implemented a simple PXE boot server ack-packet
UDP server in perl (the PXE client basically tries to perform a PXE boot server
discovery procedure). Works like a charm without the dhcp options under all the
Intel Boot Agent versions I tried - only need to set the mtftp-server address as
per recommendations in the PXE 2.1 spec. Is there any interest in me publishing
this boot server ack-packet script for others to use? Microsoft and others want
us to drop DHCP options and instead implement iphelpers in the router (if
clients and servers are on separate subnets), but shouldn't be necessary as
far as I can see. The origin of all this mess is again the PXE specification in
my opinion...
>
> Intel Boot Agent?  Really?  I've NEVER seen mention of "Intel Boot
> Agent" on UEFI.
>
> I'd certainly be interested in this Perl script.
>
> Sorry, misspeak - I have been working on this issue for an intensive
period.... Of course the Intel Boot Agent (IBA) is under BIOS. Under UEFI
PXE-boot I developed the aforementioned boot server script to deal with the boot
server discovery being attempted by the UEFI PXE agent. It is based on some
tcpdumps of the network traffic between the tftp-server and an ack-packet from a
WDS boot server. Stripped away the proprietary dhcp options from the WDS boot
server though and used the definiton of an ack-packet in the PXE 2.1
specification as soon as I understood what was going on. It is a very
rudimentary script and was only made to see if I could get the UEFI client to
accept my Linux tftp-server and its image-file (which was the syslinux.efi file
for efi64 from version 6.03). How do you want the script - send it to your
private email address?
Either is sufficient.

-- 
-Gene
> Best,
>
>     Jan-Frode
>
> --
> -Gene
>
>> ________________________________________
>> From: Gene Cumm <gene.cumm at gmail.com>
>> Sent: Tuesday, March 15, 2016 13:42
>> To: Jan Frode J?ger
>> Cc: syslinux at zytor.com
>> Subject: Re: [syslinux] Updated status on UEFI compliant version of the
pxechn-module
>>
>> On Tue, Mar 15, 2016 at 3:51 AM, Jan Frode J?ger <syslinux at
zytor.com> wrote:
>>> Hi,
>>>
>>>
>>> I have browsed through your syslinux archives and see that the
issue of an UEFI compliant version of the pxechn-module has come up several
times:
>>>
>>>
>>> http://www.syslinux.org/archives/2014-August/022611.html
>>>
>>> http://www.syslinux.org/archives/2015-March/023320.html
>>>
>>> http://www.syslinux.org/archives/2015-September/024231.html
>>>
>>>
>>> Now the issue as I understand it is that the original pxechn-module
has been written to interface with legacy BIOS and not UEFI. It also seems that
Patricks update as of March 2015 gave positive indications that a general fix
for com32-modules were possible.
>>>
>>>
>>> It would have been nice to receive some update on both com32-module
compliance for the UEFI architecture as well as for the pxechn-module in
specific. We have a setup at my workplace that are dependant upon this working
and right now the only solution is to use legacy BIOS PXE-boot.
>>>
>>>
>>> Is there any fix for this in the recently mentioned 6.04
pre-release (soon to become release?)?
>>
>> No, I removed all modules with BIOSisms from UEFI builds like
pxechn.c32.
>>
>> More importantly, Syslinux can't load arbitrary UEFI images yet.
>>
>> --
>> -Gene

Hi,
________________________________________
From: Gene Cumm <gene.cumm at gmail.com>
Sent: Wednesday, March 16, 2016 11:49
To: Jan Frode J?ger
Cc: syslinux at zytor.com
Subject: Re: [syslinux] Updated status on UEFI compliant version of the
pxechn-module

On Wed, Mar 16, 2016 at 2:58 AM, Jan Frode J?ger
<jan.frode.jaeger at ntnu.no> wrote:
> Hi,
> ________________________________________
> From: Gene Cumm <gene.cumm at gmail.com>
> Sent: Wednesday, March 16, 2016 00:56
> To: Jan Frode J?ger
> Cc: syslinux at zytor.com
> Subject: Re: [syslinux] Updated status on UEFI compliant version of the
pxechn-module
>
> On Tue, Mar 15, 2016 at 9:32 AM, Jan Frode J?ger
> <jan.frode.jaeger at ntnu.no> wrote:
>
> Without trying to figure out where data is stored or the possibility
> of calls to manipulate the data, I can't speak for the possibility of
> implementing anything that manipulates the options like pxechn.c32 for
> UEFI.  I'd like to get UEFI chaining working first.
>
> The shear length comparison speaks a lot for the differences.  Yes,
> I've read through most of the PXE spec as it relates to the boot
> process.  As bad as you think PXE is, consider how UEFI has been a
> moving target in some respects.
>
> What do you expect to PXE chainload?  If you could provide details on
> what you hope to achieve, someone might be able to offer a suggestion
> or workaround.
>
> I understand. Just let me say that I appreciate all the work that has been
done on syslinux and that it is a great tool. I was only wanting to hear what
your plans were.
>
> As goes the PXE chainloading - I am trying to redirect from a PXELINUX menu
to a WDS boot server with SCCM. So the option in my menu looks like this:
>
> ***SNIP***
> label sccmserver
>    menu label SCCM-server
>    kernel /mods/pxechn.c32
>    append 1.2.3.4::boot\x86\wdsnbp.com
> ***SNIP***
I had suspicions.

That will be impossible as-is.  You can not chainload to a BIOS NBP in
a UEFI environment.  If your WDS isn't ready for UEFI, you need to
correct that first.  Many UEFI systems include a CSM (Compatibility
Support Module) but you can't boot across like that on any system I've
ever seen.  Your boot payload will be one of two, depending on the
CPU, a x86 (32-bit) or x64 (64-bit; aka x86-64) UEFI binary.  Although
less common, x86 UEFI does exist on numerous embedded systems and some
low-end systems.

Yes, I must have given you the wrong config from the bios-folder of my
tftp-server (I have three folders: bios, uefi32 and uefi64). It should read:

***SNIP***
label sccmserver
   menu label SCCM-server
   kernel /mods/pxechn.c32
   append a.b.c.d::smsboot\x64\wdsmgfw.efi
***SNIP***

In any case it does not work and the reason is of course that pxechn is made
with BIOSisms (as it was called). It does however work to have the DHCP-server
route all UEFI-booting to the WDS-server with the above image and leaving our
server-environment with only legacy BIOS PXE-boot. That is our temporary
workaround, but I would like to get UEFI to work properly with our own syslinux
boot-menu.
> The IP here is of course just changed for this example. I have also tried
to add the "-W" parameter to pxechn, but without any change in result.
It always says "Error: Unable to retrieve first packet" and then
"Could not unpack packet".
Use designated documentation IPs or private IPs.  "192.0.2.0/24
(TEST-NET-1), 198.51.100.0/24 (TEST-NET-2), and 203.0.113.0/24
(TEST-NET-3) are provided for use in documentation." RFC5737.  RFC
1918 specifies 10.0.0.0/8, 172.16.0.0/12 and 192.168.0.0/16 as
private.  RFC6598 allocated 100.64.0.0/10 for CGN, also a private
space.  APNIC had to declare several subnets as forbidden when it was
issued 1.0.0.0/8 for real allocation in 2010, including for example
1.1.1.0/24 and 1.2.3.0/24, and waste some of its allocation.

I do not think IP-address is an issue here. We have a B-type ipv4 network here
at the university with valid addresses.
>> I just compiled up the 6.04-prerelease and can see that the
pxechn-module has been removed from the two efi-trees. As a sidenote: the
NEWS-file in the 6.04-prerelease is missing information about changes introduced
in it (seems to be the file from 6.03).
>
> NEWS has NOT been updated yet.
>
>> As another sidenote and possible separate thread: since I couldn't
get UEFI to boot pxelinux 6.03 properly before I saw your dhcp options fix
(vendor encapsulated options), I implemented a simple PXE boot server ack-packet
UDP server in perl (the PXE client basically tries to perform a PXE boot server
discovery procedure). Works like a charm without the dhcp options under all the
Intel Boot Agent versions I tried - only need to set the mtftp-server address as
per recommendations in the PXE 2.1 spec. Is there any interest in me publishing
this boot server ack-packet script for others to use? Microsoft and others want
us to drop DHCP options and instead implement iphelpers in the router (if
clients and servers are on separate subnets), but shouldn't be necessary as
far as I can see. The origin of all this mess is again the PXE specification in
my opinion...
>
> Intel Boot Agent?  Really?  I've NEVER seen mention of "Intel Boot
> Agent" on UEFI.
>
> I'd certainly be interested in this Perl script.
>
> Sorry, misspeak - I have been working on this issue for an intensive
period.... Of course the Intel Boot Agent (IBA) is under BIOS. Under UEFI
PXE-boot I developed the aforementioned boot server script to deal with the boot
server discovery being attempted by the UEFI PXE agent. It is based on some
tcpdumps of the network traffic between the tftp-server and an ack-packet from a
WDS boot server. Stripped away the proprietary dhcp options from the WDS boot
server though and used the definiton of an ack-packet in the PXE 2.1
specification as soon as I understood what was going on. It is a very
rudimentary script and was only made to see if I could get the UEFI client to
accept my Linux tftp-server and its image-file (which was the syslinux.efi file
for efi64 from version 6.03). How do you want the script - send it to your
private email address?
Either is sufficient.

Ok, it is attached to this email. I have cleaned the code up a little bit, but
it is still fairly rudimentary. If I were to make more out of it, I would
rewrite a bit and in addition add the capability to send more DHCP options as
the PXE client may ask for. In any case...

It is pretty self explainatory and you can run the script without any parameters
and get a list of options with its explanation:

./bsack.pl

For a basic number of paramters to properly start the boot server ack-server,
you must do the following:

./bsack.pl -t IP_OF_TFTP_SERVER -i NBP_TO_LOAD 

In addition there are some parameters to specify debugging, which interface to
bind to and port to listen to (default is 4011, but one may get a client that
asks for 67 dependent I guess upon the setting of PXE_DISCOVERY_CONTROL). The
debugging writes the in-packet and out-packet to /tmp/bsack-[in|out].raw for
inspection. The script defaults to bind to the interface that has the IP address
of the tftp-server given with the -t option.

A full set of options might be:

./bsack.pl -d -t 1.2.3.4 -i uefi64/syslinux.efi -b 4.3.2.1 -p 4011

The options do not have any preferred sequence. The only settings needed in the
DHCP-server is (as an example):

***SNIP***
filename "uefi64/syslinux.efi";
next-server 1.2.3.4;
option vendor-class-identifier "PXEClient";
# set PXE option 1 (mftp-server-ip)
option vendor-encapsulated-options 01:04:00:00:00:00:ff;
***SNIP***

In other words the most normal PXE options in the dhcpd.conf-file, including the
setting of the mtftp-server to 0-values.

As a sidenote and cocktail entertainment for PXE-geeks: it seems the boot-server
ack-packet that the Microsoft WDS-server creates on Server 2012 R2 has a bug. It
basically switches the ciaddr og yiaddr fields in the top of the packet.
According to the PXE 2.1 specification the ciaddr field is supposed to be zeroed
out, but is in reality filled with the PXE-clients ip-address, while the
yiaddr-field which is supposed to be the PXE-clients address is zeroed out. The
PXE client on the other hand is happy as a hamster....

Hope my script is of some use for someone. We are not using it right now here,
but it is nice to have for testing purposes.

Best regards,

     Jan-Frode

--
-Gene
> Best,
>
>     Jan-Frode
>
> --
> -Gene
>
>> ________________________________________
>> From: Gene Cumm <gene.cumm at gmail.com>
>> Sent: Tuesday, March 15, 2016 13:42
>> To: Jan Frode J?ger
>> Cc: syslinux at zytor.com
>> Subject: Re: [syslinux] Updated status on UEFI compliant version of the
pxechn-module
>>
>> On Tue, Mar 15, 2016 at 3:51 AM, Jan Frode J?ger <syslinux at
zytor.com> wrote:
>>> Hi,
>>>
>>>
>>> I have browsed through your syslinux archives and see that the
issue of an UEFI compliant version of the pxechn-module has come up several
times:
>>>
>>>
>>> http://www.syslinux.org/archives/2014-August/022611.html
>>>
>>> http://www.syslinux.org/archives/2015-March/023320.html
>>>
>>> http://www.syslinux.org/archives/2015-September/024231.html
>>>
>>>
>>> Now the issue as I understand it is that the original pxechn-module
has been written to interface with legacy BIOS and not UEFI. It also seems that
Patricks update as of March 2015 gave positive indications that a general fix
for com32-modules were possible.
>>>
>>>
>>> It would have been nice to receive some update on both com32-module
compliance for the UEFI architecture as well as for the pxechn-module in
specific. We have a setup at my workplace that are dependant upon this working
and right now the only solution is to use legacy BIOS PXE-boot.
>>>
>>>
>>> Is there any fix for this in the recently mentioned 6.04
pre-release (soon to become release?)?
>>
>> No, I removed all modules with BIOSisms from UEFI builds like
pxechn.c32.
>>
>> More importantly, Syslinux can't load arbitrary UEFI images yet.
>>
>> --
>> -Gene
-------------- next part --------------
A non-text attachment was scrubbed...
Name: bsack.pl
Type: application/x-perl
Size: 7235 bytes
Desc: bsack.pl
URL:
<http://www.zytor.com/pipermail/syslinux/attachments/20160316/79b0d8ac/attachment.pl>

>>> I'd like to get UEFI chaining working first.
<<<
I have sent in the past two patches; one allows SL to chainload to efi
executives and the other one
allows to link the gnu-efi layer to a com32 module in order to access the EFI
firmware from the com32 code.
Despite minor changes I might've made during all this time that code
works...
Best,Patrick

On Wed, Mar 16, 2016 at 7:21 PM, Patrick Masotta <masottaus at yahoo.com>
wrote:
>>>>
> I'd like to get UEFI chaining working first.
>
>
> <<<
>
> I have sent in the past two patches; one allows SL to chainload to efi
> executives and the other one
> allows to link the gnu-efi layer to a com32 module in order to access the
> EFI firmware from the com32 code.
> Despite minor changes I might've made during all this time that code
> works...
Do you happen to have these accessible in a public repository by chance?

-- 
-Gene

On Wed, Mar 16, 2016 at 7:21 PM, Patrick Masotta <masottaus at yahoo.com>
wrote:
>>>>
> I'd like to get UEFI chaining working first.
>
>
> <<<
>
> I have sent in the past two patches; one allows SL to chainload to efi
> executives and the other one
> allows to link the gnu-efi layer to a com32 module in order to access the
> EFI firmware from the com32 code.
> Despite minor changes I might've made during all this time that code
> works...
>
> Best,
> Patrick
>
Patrick, this is the last message I see on my Yahoo account.

Geert/HPA/Shao, I see nothing indicating the issue.

-- 
-Gene