Hi, Didier Spaier wrote:> The issue is the inability of booting of a *physical* (not virtual) optical > media, like for instance a DVD or a CD, using syslinux.efi.Actually firmware cannot find any UEFI specified booting starting points in an ISO 9660 filesystem that was created with mkisofs -b and -hard-disk-boot, but without a boot image which was marked as EFI System Partition. The boot success with UEFI and -cdrom must have other reasons. I suspect the BIOS emulation of the firmware. About SYSLINUX EFI: Do we really have confirmed examples where SYSLINUX EFI software works from qemu DVD-ROM ? Decisive test would be to make an ISO which offers no El Torito boot image for BIOS, but only an EFI System Partition image as El Torito boot image for Platform Id 0xef. In that partiton image there would have to be SYSLINUX stuff - not GRUB or a directly bootable kernel. Why i doubt: When it comes to reading data, there should be no perceivable difference between a real optical drive at a real computer and the virtual DVD-ROM of qemu. Both get operated in the end by SCSI commands. Of course there could be some driver shortage in SYSLINUX which does not show up with qemu DVD-ROM but prevents use of modern hardware. But i understand that EFI itself is a little operating system, which is supposed to operate hardware, and that SYSLINUX EFI software uses the API of EFI to perform its hardware operations. Have a nice day :) Thomas
> Hi, > > Didier Spaier wrote: > > The issue is the inability of booting of a *physical* (not virtual) optical > > media, like for instance a DVD or a CD, using syslinux.efi. > > Actually firmware cannot find any UEFI specified booting starting points > in an ISO 9660 filesystem that was created with mkisofs -b and > -hard-disk-boot, but without a boot image which was marked as EFI System > Partition. > > The boot success with UEFI and -cdrom must have other reasons. I suspect > the BIOS emulation of the firmware. > > > About SYSLINUX EFI: > > Do we really have confirmed examples where SYSLINUX EFI software works > from qemu DVD-ROM ? Decisive test would be to make an ISO which offers > no El Torito boot image for BIOS, but only an EFI System Partition image as > El Torito boot image for Platform Id 0xef. In that partiton image there > would have to be SYSLINUX stuff - not GRUB or a directly bootable kernel. > > Why i doubt: > > When it comes to reading data, there should be no perceivable difference > between a real optical drive at a real computer and the virtual DVD-ROM > of qemu. Both get operated in the end by SCSI commands. > > Of course there could be some driver shortage in SYSLINUX which does not > show up with qemu DVD-ROM but prevents use of modern hardware. But i > understand that EFI itself is a little operating system, which is supposed > to operate hardware, and that SYSLINUX EFI software uses the API of EFI > to perform its hardware operations. > > > Have a nice day :) > > Thomas >I almost fell into the "trap" (yet again) of replying with several comments to every single email in this email thread that was posted after my last one. Considering that the topic of syslinux.efi and optical media (and/or dd'ing such iso images to a USB device) has been mostly ignored by relevant Syslinux developer(s) in the past, I decided to avoid such waste of my time. (Additionally, my tolerance for top-posting is getting smaller and smaller; sadly, I am getting older and grumpier). @Thomas, yes, I performed a similar test "successfully". I created a 30MiB HDD image, MBR plus one partition, flagged as both "boot" and "esp", FAT16. The FAT16 partition includes "/EFI/BOOT/" and within it the relevant syslinux.efi-related files (with the EFI bootloader renamed to the default "BOOT<arch_type>.EFI" naming). Based on this HDD image, I created an ISO image, iso9660 only (no fs extensions, no UDF). The iso9660 filesystem is actually empty (I added no files). I set the ISO image to be bootable with El Torito *HDD emulation* - please keep reading, and then see [1] below - and for the UEFI platform only (no BIOS part). I attached the resulting ISO image to a VirtualBox VM set to boot in (U)EFI mode. No other (virtual) storage was attached to the VM. The VM successfully boots this virtual optical media, arriving to the syslinux.efi boot prompt. From the boot prompt, I did not attempt to boot any kernel, but I successfully executed a couple of c32 modules, then returning back to the boot prompt. Additional comments about this test: _ This was a relatively quick dirty test. For several reasons, the whole procedure was performed under a Windows OS with GUI tools. I am hoping that whoever would want to replicate it under a different OS using command-line tools, will be able to figure it out. _ The size of the HDD image was purposely smaller than 32 MiB, so to reduce potential conflicts with size limitations in El Torito / UEFI specs. Someone else might be willing to test with bigger images. _ Someone else might be willing to perform a similar test while also adding a BIOS part to the boot catalog (instead of only using the UEFI platform). _ Syslinux 6.04-pre1 can only access the boot volume alone, therefore only the content of the HDD image (i.e. El Torito HDD emulation image) can be seen from syslinux.efi. In other words, syslinux.efi can't access the content of the iso9660 data area. [1]: _ Please note that the UEFI specs state that the EFI System Partition stored in the boot catalog is supposed to be stored as "no emulation" mode. Although I explicitly set the emulation mode to HDD and the platform to UEFI when creating the ISO image, the (U)EFI firmware in VirtualBox had no complains and used the HDD image to boot in UEFI mode. This is one of the reasons I am of the opinion that some restrictions "imposed" by ISO-building tools should be relaxed (as in, "advice correctness but allow deviations when the user chooses to do so"). Now, is there any OS (e.g. Linux) distribution that is willing to use this method for booting in UEFI mode? Is there anyone that is willing to chose this method to boot optical media in UEFI mode? Is there anyone that is willing to add more options and alternatives to isohybrid (because we all know that this is going to be the first immediate request)? I think I have read more than once that "patches are welcome". Hmm... Regards, Ady.> _______________________________________________ > Syslinux mailing list > Submissions to Syslinux at zytor.com > Unsubscribe or set options at: > http://www.zytor.com/mailman/listinfo/syslinux
Hi, Ady wrote:> [hard disk emulation boot image works with VirtualBox]For a reader of the UEFI specs this comes very unexpected. Together with the report of Pascal that a hard-disk-emulation works even with mkisofs option -b (i.e. Platform Id x86 BIOS) it appears that the UEFI implementation treats the El Torito boot image like the whole device, regardless of Platform Id. On (virtual) CD-ROM this makes some sense. But on virtual hard disk (as reported by Pascal) it deviates from all tradition: El Torito is not to be interpreted on hard disk. Not by BIOS, not by UEFI. I am in doubt whether other implementations of UEFI do this too. Have a nice day :) Thomas
Le jeudi 22 d?cembre 2016, Ady Ady via Syslinux <syslinux at zytor.com> a ?crit :> > > Hi, > > > > Didier Spaier wrote: > > > The issue is the inability of booting of a *physical* (not virtual) > optical > > > media, like for instance a DVD or a CD, using syslinux.efi. > > > > Actually firmware cannot find any UEFI specified booting starting points > > in an ISO 9660 filesystem that was created with mkisofs -b and > > -hard-disk-boot, but without a boot image which was marked as EFI System > > Partition. > > > > The boot success with UEFI and -cdrom must have other reasons. I suspect > > the BIOS emulation of the firmware. > > > > > > About SYSLINUX EFI: > > > > Do we really have confirmed examples where SYSLINUX EFI software works > > from qemu DVD-ROM ? Decisive test would be to make an ISO which offers > > no El Torito boot image for BIOS, but only an EFI System Partition image > as > > El Torito boot image for Platform Id 0xef. In that partiton image there > > would have to be SYSLINUX stuff - not GRUB or a directly bootable kernel. > > > > Why i doubt: > > > > When it comes to reading data, there should be no perceivable difference > > between a real optical drive at a real computer and the virtual DVD-ROM > > of qemu. Both get operated in the end by SCSI commands. > > > > Of course there could be some driver shortage in SYSLINUX which does not > > show up with qemu DVD-ROM but prevents use of modern hardware. But i > > understand that EFI itself is a little operating system, which is > supposed > > to operate hardware, and that SYSLINUX EFI software uses the API of EFI > > to perform its hardware operations. > > > > > > Have a nice day :) > > > > Thomas > > > > > I almost fell into the "trap" (yet again) of replying with several > comments to every single email in this email thread that was posted > after my last one. Considering that the topic of syslinux.efi and > optical media (and/or dd'ing such iso images to a USB device) has been > mostly ignored by relevant Syslinux developer(s) in the past, I decided > to avoid such waste of my time. (Additionally, my tolerance for > top-posting is getting smaller and smaller; sadly, I am getting older > and grumpier). > > @Thomas, yes, I performed a similar test "successfully". > > I created a 30MiB HDD image, MBR plus one partition, flagged as both > "boot" and "esp", FAT16. > > The FAT16 partition includes "/EFI/BOOT/" and within it the relevant > syslinux.efi-related files (with the EFI bootloader renamed to the > default "BOOT<arch_type>.EFI" naming). > > Based on this HDD image, I created an ISO image, iso9660 only (no fs > extensions, no UDF). The iso9660 filesystem is actually empty (I added > no files). I set the ISO image to be bootable with El Torito *HDD > emulation* - please keep reading, and then see [1] below - and for the > UEFI platform only (no BIOS part).I've the same hdd image structure : 11Mb with vmlinuz and core.gz (from TinyCoreLinux) in /boot/ folder and syslinux.cfg well configured.> > I attached the resulting ISO image to a VirtualBox VM set to boot in > (U)EFI mode. No other (virtual) storage was attached to the VM. The VM > successfully boots this virtual optical media, arriving to the > syslinux.efi boot prompt. From the boot prompt, I did not attempt to > boot any kernel, but I successfully executed a couple of c32 modules, > then returning back to the boot prompt.my iso image boots in cdrom/bios+uefi and in hda/uefi and TCL runs perfectly.> > Additional comments about this test: > > _ This was a relatively quick dirty test. For several reasons, the > whole procedure was performed under a Windows OS with GUI tools. I am > hoping that whoever would want to replicate it under a different OS > using command-line tools, will be able to figure it out.for me, all was done under Linux (archlinux up to date).> > _ The size of the HDD image was purposely smaller than 32 MiB, so to > reduce potential conflicts with size limitations in El Torito / UEFI > specs. Someone else might be willing to test with bigger images. > > _ Someone else might be willing to perform a similar test while also > adding a BIOS part to the boot catalog (instead of only using the UEFI > platform). > > _ Syslinux 6.04-pre1 can only access the boot volume alone, therefore > only the content of the HDD image (i.e. El Torito HDD emulation image) > can be seen from syslinux.efi. In other words, syslinux.efi can't > access the content of the iso9660 data area. > > [1]: > _ Please note that the UEFI specs state that the EFI System Partition > stored in the boot catalog is supposed to be stored as "no emulation" > mode. Although I explicitly set the emulation mode to HDD and the > platform to UEFI when creating the ISO image, the (U)EFI firmware in > VirtualBox had no complains and used the HDD image to boot in UEFI > mode. This is one of the reasons I am of the opinion that some > restrictions "imposed" by ISO-building tools should be relaxed (as in, > "advice correctness but allow deviations when the user chooses to do > so"). > > Now, is there any OS (e.g. Linux) distribution that is willing to use > this method for booting in UEFI mode? Is there anyone that is willing > to chose this method to boot optical media in UEFI mode? Is there > anyone that is willing to add more options and alternatives to > isohybrid (because we all know that this is going to be the first > immediate request)? I think I have read more than once that "patches > are welcome". Hmm... > > Regards, > Ady. > > > _______________________________________________ > > Syslinux mailing list > > Submissions to Syslinux at zytor.com <javascript:;> > > Unsubscribe or set options at: > > http://www.zytor.com/mailman/listinfo/syslinux > > > > _______________________________________________ > Syslinux mailing list > Submissions to Syslinux at zytor.com <javascript:;> > Unsubscribe or set options at: > http://www.zytor.com/mailman/listinfo/syslinux >
Hi, Ady wrote:> the UEFI specs do state that the ESP is supposed > to be set/used as no-emulation mode > ... > the UEFI specs should tolerate / accept this situation, rather than< restricting the ESP to no-emulation mode. My criticism towards this UEFI implementation is that it probably cannot serve as realistic test bed for the first stage of UEFI booting. It seems far too tolerant towards standards deviations. Not only does Pascal's working ISO bear a hard-disk-emulation boot image, which constitutes a whole disk and not a disk partition, but also it is not marked as Platform Id 0xEF. Still it is used by the EFI implementation which he uses with his VM. Especially weird is that El Torito has an effect when the ISO is presented on hard disk. In this situation the UEFI specs prescribe that the ESP is to be marked by MBR partition or GPT partition. But no partition table was written to the start of the ISO image. (That would be the job of isohybrid or a partition editor or mkisofs options like -G.) Even if Unified EFI Inc. (or Intel) would be willing to widen the specs, there is still the problem of existing implementations which took the existing UEFI specs for serious. -------------------------------------------------------------------------- Practically there is Pascal's initial obstacle of his ISO not booting from hard disk via BIOS firmware. To solve this problem, one could use MBR x86 code which chainloads the MBR x86 code of the hard-disk-emulation image. mkisofs option -G would put such code to the start of the ISO. ... or an independent boot path via isolinux.bin and isohybrid. Like: -b .../isolinux.bin -no-emul-boot -boot-load-size 4 -boot-info-table \ -eltorito-alt-boot \ -e boot/core.img -hard-disk-boot \ and then applying isohybrid _without_ its UEFI options. isohybrid will add MBR x86 code which will hop on isolinux.bin. Of course isolinux.bin then needs a configuration file in the ISO (not inside the hard-disk-emulation image) and early operating system stages which enable it to boot completely or to peek into the hard-disk-emulation image. Well, at latest when somebody has to document the nature and reason of such a boot equipment, it will be hard to justify it in the light of published specs. -------------------------------------------------------------------------- in previous post, Ady wrote:> Is there anyone that is willing to add more options and alternatives > to isohybrid [...] ?That's not easy. The original BIOS oriented isohybrid feature puts MBR x86 code to the start of the ISO, which knows the block address of the isolinux.bin file in the ISO. The MBR code just has to load that program and execute it. Then it is on the same execution path as booting via El Torito from optical media. But this cannot be done with a hard-disk-emulation image, because it is not an executable program. One would have to invent a whole new isohybridization mechanism, like hpa invented isohybrid for isolinux.bin. Like the chainloading idea ... maybe. Have a nice day :) Thomas
2016-12-22 10:19 GMT+01:00 Thomas Schmitt via Syslinux <syslinux at zytor.com>:> Hi, > > Didier Spaier wrote: > > The issue is the inability of booting of a *physical* (not virtual) > optical > > media, like for instance a DVD or a CD, using syslinux.efi. > > Actually firmware cannot find any UEFI specified booting starting points > in an ISO 9660 filesystem that was created with mkisofs -b and > -hard-disk-boot, but without a boot image which was marked as EFI System > Partition. > > The boot success with UEFI and -cdrom must have other reasons. I suspect > the BIOS emulation of the firmware. >I used "OVMF-pure-efi.fd" and not "OVMF-with-csm.fd" (from https://www.kraxel.org/repos/jenkins/edk2/)> > > About SYSLINUX EFI: > > Do we really have confirmed examples where SYSLINUX EFI software works > from qemu DVD-ROM ? Decisive test would be to make an ISO which offers > no El Torito boot image for BIOS, but only an EFI System Partition image as > El Torito boot image for Platform Id 0xef. In that partiton image there > would have to be SYSLINUX stuff - not GRUB or a directly bootable kernel. > > Why i doubt: > > When it comes to reading data, there should be no perceivable difference > between a real optical drive at a real computer and the virtual DVD-ROM > of qemu. Both get operated in the end by SCSI commands. > > Of course there could be some driver shortage in SYSLINUX which does not > show up with qemu DVD-ROM but prevents use of modern hardware. But i > understand that EFI itself is a little operating system, which is supposed > to operate hardware, and that SYSLINUX EFI software uses the API of EFI > to perform its hardware operations. > > > Have a nice day :) > > Thomas > > _______________________________________________ > Syslinux mailing list > Submissions to Syslinux at zytor.com > Unsubscribe or set options at: > http://www.zytor.com/mailman/listinfo/syslinux >
Hi, i wrote:> > The boot success with UEFI and -cdrom must have other reasons. I suspect > > the BIOS emulation of the firmware.Pascal wrote:> I used "OVMF-pure-efi.fd" and not "OVMF-with-csm.fd" (from > https://www.kraxel.org/repos/jenkins/edk2/ [www.kraxel.org])The not UEFI specs compliant path of finding file bootx64.efi seems to use El Torito device emulation, even on hard disk. Reading http://www.linux-kvm.org/downloads/lersek/ovmf-whitepaper-c770f8c.txt i would expect this to happen by the DXE step. I see in https://github.com/tianocore/edk2/blob/18b00c8c8773e62b1e2c4a36f490996325570b67/MdeModulePkg/Universal/Disk/PartitionDxe/ElTorito.c a case distinction where ELTORITO_HARD_DISK is _not_ handled by the code part which says: "unsupported El Torito boot media type" (This overextents UEFI specs which say that the EFI System partition shall be announced as "no-emulation" El Torito boot image.) A similar case distinction leads to registration as "partition" in function FatFindEltoritoPartitions() of https://github.com/tianocore/edk2/blob/18b00c8c8773e62b1e2c4a36f490996325570b67/FatPkg/FatPei/Part.c where function FatFindPartitions() first looks for an MBR and then for El Torito without distinguishing the medium type. (This overextents UEFI specs which do not mention El Torito outside optical media.) The function FatFindEltoritoPartitions() does not distinguish between Platform Id 0x00 (x86 BIOS) and 0xEF (UEFI). (This overextents UEFI specs which say that El Torito boot images shall have Platform Id 0xEF.) I did not yet identify the code part, which then looks into the registered El Torito emulated hard disk "partition" for a partiton table leading to the FAT filesystem with bootx64.efi. This is similar to the EBR mechanism of extended partitions. So possibly function PartitionInstallMbrChildHandles() in https://github.com/tianocore/edk2/blob/18b00c8c8773e62b1e2c4a36f490996325570b67/MdeModulePkg/Universal/Disk/PartitionDxe/Mbr.c is a valid suspect for that last step. Now the firmware would see an MBR partition of type 0xEF and have reason to look into it for /efi/boot/bootx64.efi. This would need three instances of ignoring written UEFI specs. If not your test results would support this theory, then i would believe this to be dead code in the edk2 tree. But together with your results my frightening suspicion seems realistic. It is frightening because it might execute code which was not intended for EFI instead of a real EFI boot program which is marked in an UEFI compliant way. Have a nice day :) Thomas