Harry Schmalzbauer
2017-May-16 16:26 UTC
EFI loader doesn't handle md_preload (md_image) correct?
Bez?glich Toomas Soome's Nachricht vom 16.05.2017 18:20 (localtime):> >> On 16. mai 2017, at 19:13, Harry Schmalzbauer <freebsd at omnilan.de> wrote: >> >> Bez?glich Toomas Soome's Nachricht vom 16.05.2017 18:00 (localtime): >>> >>>> On 16. mai 2017, at 18:45, Harry Schmalzbauer <freebsd at omnilan.de >>>> <mailto:freebsd at omnilan.de>> wrote: >>>> >>>> Bez?glich Harry Schmalzbauer's Nachricht vom 16.05.2017 17:28 (localtime): >>>>> Bez?glich Toomas Soome's Nachricht vom 16.05.2017 16:57 (localtime): >>>>>>> On 16. mai 2017, at 17:55, Harry Schmalzbauer <freebsd at omnilan.de >>>>>>> <mailto:freebsd at omnilan.de>> wrote: >>>>>>> >>>>>>> Hello, >>>>>>> >>>>>>> unfortunately I had some trouble with my preferred MFS-root setups. >>>>>>> It seems EFI loader doesn't handle type md_image correctly. >>>>>>> >>>>>>> If I load any md_image with loader invoked by gptboot or gptzfsboot, >>>>>>> 'lsmod' >>>>>>> shows "elf kernel", "elf obj module(s)" and "md_image". >>>>>>> >>>>>>> Using the same loader.conf, but EFI loader, the md_image-file is >>>>>>> prompted and sems to be loaded, but not registered. There's no >>>>>>> md_image >>>>>>> with 'lsmod', hence it's not astonsihing that kernel doesn't attach md0 >>>>>>> so booting fails since there's no rootfs. >>>>>>> >>>>>>> Any help highly appreciated, hope Toomas doesn't mind beeing >>>>>>> initially CC'd. >>>>>>> >>>>>>> Thanks, >>>>>>> >>>>>>> -harry >>>>>> >>>>>> The first question is, how large is the md_image and what other >>>>>> modules are loaded? >>>>> Thanks for your quick response. >>>>> >>>>> The images are 50-500MB uncompressed (provided by gzip compressed file). >>>>> Small ammount of elf modules, 5, each ~50kB. >>>> >>>> On the real HW, there's vmm and some more: >>>> Id Refs Address Size Name >>>> 1 46 0xffffffff80200000 16M kernel >>>> 2 1 0xffffffff8121d000 86K unionfs.ko >>>> 3 1 0xffffffff81233000 3.1M zfs.ko >>>> 4 2 0xffffffff81545000 51K opensolaris.ko >>>> 5 7 0xffffffff81552000 279K usb.ko >>>> 6 1 0xffffffff81598000 67K ukbd.ko >>>> 7 1 0xffffffff815a9000 51K umass.ko >>>> 8 1 0xffffffff815b6000 46K aesni.ko >>>> 9 1 0xffffffff815c3000 54K uhci.ko >>>> 10 1 0xffffffff815d1000 65K ehci.ko >>>> 11 1 0xffffffff815e2000 15K cc_htcp.ko >>>> 12 1 0xffffffff815e6000 3.4M vmm.ko >>>> 13 1 0xffffffffa3a21000 12K ums.ko >>>> 14 1 0xffffffffa3a24000 9.1K uhid.ko >>>> >>>> Providing md_image uncompressed doesn't change anything. >>>> >>>> Will deploy a /usr separated rootfs, which is only ~100MB uncompressed >>>> and see if that changes anything. >>>> That's all I can provide, code is far beyond my knowledge... >>>> >>>> -harry >>> >>> >>> The issue is, that current UEFI implementation is using 64MB staging >>> memory for loading the kernel and modules and files. When the boot is >>> called, the relocation code will put the bits from staging area into the >>> final places. The BIOS version does not need such staging area, and that >>> will explain the difference. >>> >>> I actually have different implementation to address the same problem, >>> but thats for illumos case, and will need some work to make it usable >>> for freebsd; the idea is actually simple - allocate staging area per >>> loaded file and relocate the bits into the place by component, not as >>> continuous large chunk (this would also allow to avoid the mines like >>> planted by hyperv;), but right now there is no very quick real solution >>> other than just build efi loader with larger staging size. >> >> Ic, thanks for the explanation. >> While not aware about the purpose of the staging area nor the >> consequences of enlarging it, do you think it's feasable increasing it >> to 768Mib? >> >> At least now I have an idea baout the issue and an explanation why >> reducing md_imgae to 100MB hasn't helped ? still more than 64... >> >> Any quick hint where to define the staging area size highly appreciated, >> fi there are no hard objections against a 768MB size. >> >> -harry > > The problem is that before UEFI Boot Services are not switched off, the memory is managed (and owned) by the firmware,Hmm, I've been expecting something like that (owend by firmware) ;-) So I'll stay with CSM for now, and will happily be an early adopter if you need someone to try anything (-stable mergable). Thanks, -harry
Harry Schmalzbauer
2017-Jun-29 08:24 UTC
EFI loader doesn't handle md_preload (md_image) correct?
Bez?glich Harry Schmalzbauer's Nachricht vom 16.05.2017 18:26 (localtime):> B?>>>> The issue is, that current UEFI implementation is using 64MB staging >>>> memory for loading the kernel and modules and files. When the boot is >>>> called, the relocation code will put the bits from staging area into the >>>> final places. The BIOS version does not need such staging area, and that >>>> will explain the difference. >>>> >>>> I actually have different implementation to address the same problem, >>>> but thats for illumos case, and will need some work to make it usable >>>> for freebsd; the idea is actually simple - allocate staging area per >>>> loaded file and relocate the bits into the place by component, not as >>>> continuous large chunk (this would also allow to avoid the mines like >>>> planted by hyperv;), but right now there is no very quick real solution >>>> other than just build efi loader with larger staging size. >>> Ic, thanks for the explanation. >>> While not aware about the purpose of the staging area nor the >>> consequences of enlarging it, do you think it's feasable increasing it >>> to 768Mib? >>> >>> At least now I have an idea baout the issue and an explanation why >>> reducing md_imgae to 100MB hasn't helped ? still more than 64... >>> >>> Any quick hint where to define the staging area size highly appreciated, >>> fi there are no hard objections against a 768MB size. >>> >>> -harry >> The problem is that before UEFI Boot Services are not switched off, the memory is managed (and owned) by the firmware, > Hmm, I've been expecting something like that (owend by firmware) ;-) > > So I'll stay with CSM for now, and will happily be an early adopter if > you need someone to try anything (-stable mergable).Toomas, thanks for your help so far! I'm just curious if there's news on this. Was there a decision made whether kernel should be utilized to relocate the MD image modules or the loader should be extended to handle (x-)large staging areas? I'd like to switch back to UEFI booting for various reasons (most priority has consistency), but can't since it breaks md-rootfs with that machine (the other run ESXi still). If there's anything to test, please let me know. Thanks, -harry
Harry Schmalzbauer
2019-Apr-08 06:44 UTC
EFI loader doesn't handle md_preload (md_image) correct?
Am 16.05.2017 um 18:26 schrieb Harry Schmalzbauer:> Bez?glich Toomas Soome's Nachricht vom 16.05.2017 18:20 (localtime): >>> On 16. mai 2017, at 19:13, Harry Schmalzbauer <freebsd at omnilan.de> wrote: >>> >>> Bez?glich Toomas Soome's Nachricht vom 16.05.2017 18:00 (localtime): >>>>> On 16. mai 2017, at 18:45, Harry Schmalzbauer <freebsd at omnilan.de >>>>> <mailto:freebsd at omnilan.de>> wrote: >>>>> >>>>> Bez?glich Harry Schmalzbauer's Nachricht vom 16.05.2017 17:28 (localtime): >>>>>> Bez?glich Toomas Soome's Nachricht vom 16.05.2017 16:57 (localtime): >>>>>>>> On 16. mai 2017, at 17:55, Harry Schmalzbauer <freebsd at omnilan.de >>>>>>>> <mailto:freebsd at omnilan.de>> wrote: >>>>>>>> >>>>>>>> Hello, >>>>>>>> >>>>>>>> unfortunately I had some trouble with my preferred MFS-root setups. >>>>>>>> It seems EFI loader doesn't handle type md_image correctly. >>>>>>>> >>>>>>>> If I load any md_image with loader invoked by gptboot or gptzfsboot, >>>>>>>> 'lsmod' >>>>>>>> shows "elf kernel", "elf obj module(s)" and "md_image". >>>>>>>> >>>>>>>> Using the same loader.conf, but EFI loader, the md_image-file is >>>>>>>> prompted and sems to be loaded, but not registered. There's no >>>>>>>> md_image >>>>>>>> with 'lsmod', hence it's not astonsihing that kernel doesn't attach md0 >>>>>>>> so booting fails since there's no rootfs. >>>>>>>> >>>>>>>> Any help highly appreciated, hope Toomas doesn't mind beeing >>>>>>>> initially CC'd. >>>>>>>> >>>>>>>> Thanks, >>>>>>>> >>>>>>>> -harry >>>>>>> The first question is, how large is the md_image and what other >>>>>>> modules are loaded? >>>>>> Thanks for your quick response. >>>>>> >>>>>> The images are 50-500MB uncompressed (provided by gzip compressed file). >>>>>> Small ammount of elf modules, 5, each ~50kB. >>>>> On the real HW, there's vmm and some more: >>>>> Id Refs Address Size Name >>>>> 1 46 0xffffffff80200000 16M kernel >>>>> 2 1 0xffffffff8121d000 86K unionfs.ko >>>>> 3 1 0xffffffff81233000 3.1M zfs.ko >>>>> 4 2 0xffffffff81545000 51K opensolaris.ko >>>>> 5 7 0xffffffff81552000 279K usb.ko >>>>> 6 1 0xffffffff81598000 67K ukbd.ko >>>>> 7 1 0xffffffff815a9000 51K umass.ko >>>>> 8 1 0xffffffff815b6000 46K aesni.ko >>>>> 9 1 0xffffffff815c3000 54K uhci.ko >>>>> 10 1 0xffffffff815d1000 65K ehci.ko >>>>> 11 1 0xffffffff815e2000 15K cc_htcp.ko >>>>> 12 1 0xffffffff815e6000 3.4M vmm.ko >>>>> 13 1 0xffffffffa3a21000 12K ums.ko >>>>> 14 1 0xffffffffa3a24000 9.1K uhid.ko >>>>> >>>>> Providing md_image uncompressed doesn't change anything. >>>>> >>>>> Will deploy a /usr separated rootfs, which is only ~100MB uncompressed >>>>> and see if that changes anything. >>>>> That's all I can provide, code is far beyond my knowledge... >>>>> >>>>> -harry >>>> >>>> The issue is, that current UEFI implementation is using 64MB staging >>>> memory for loading the kernel and modules and files. When the boot is >>>> called, the relocation code will put the bits from staging area into the >>>> final places. The BIOS version does not need such staging area, and that >>>> will explain the difference. >>>> >>>> I actually have different implementation to address the same problem, >>>> but thats for illumos case, and will need some work to make it usable >>>> for freebsd; the idea is actually simple - allocate staging area per >>>> loaded file and relocate the bits into the place by component, not as >>>> continuous large chunk (this would also allow to avoid the mines like >>>> planted by hyperv;), but right now there is no very quick real solution >>>> other than just build efi loader with larger staging size. >>> Ic, thanks for the explanation. >>> While not aware about the purpose of the staging area nor the >>> consequences of enlarging it, do you think it's feasable increasing it >>> to 768Mib? >>> >>> At least now I have an idea baout the issue and an explanation why >>> reducing md_imgae to 100MB hasn't helped ? still more than 64... >>> >>> Any quick hint where to define the staging area size highly appreciated, >>> fi there are no hard objections against a 768MB size. >>> >>> -harry >> The problem is that before UEFI Boot Services are not switched off, the memory is managed (and owned) by the firmware, > Hmm, I've been expecting something like that (owend by firmware) ;-) > > So I'll stay with CSM for now, and will happily be an early adopter if > you need someone to try anything (-stable mergable).Hello Toomas, thanks for your ongoing FreeBSD commits, saw your recent libstand improvements and the efiloader commit. Which remembers me nagging the skilled ones for my unmet needs ;-) I guess nobody had time to look at the MFS-root limitation with EFI vs. BIOS. If you have any news/plans, please share. The ability to boot via EFI gives a much better console experience/usability for admins, but on MFS-root system, I'm still forced to use the old loader path, because of the 64MB size limit. Do you think there's a chance that this will be resolved for FreeBSD? Thanks, -harry