freebsd stable - May 2017 - EFI loader doesn't handle md_preload (md_image) correct?

Bez?glich Harry Schmalzbauer's Nachricht vom 16.05.2017 17:28
(localtime):
> Bez?glich Toomas Soome's Nachricht vom 16.05.2017 16:57 (localtime):
>>> On 16. mai 2017, at 17:55, Harry Schmalzbauer <freebsd at
omnilan.de> wrote:
>>>
>>> Hello,
>>>
>>> unfortunately I had some trouble with my preferred MFS-root setups.
>>> It seems EFI loader doesn't handle type md_image correctly.
>>>
>>> If I load any md_image with loader invoked by gptboot or
gptzfsboot,
>>> 'lsmod'
>>> shows "elf kernel", "elf obj module(s)" and
"md_image".
>>>
>>> Using the same loader.conf, but EFI loader, the md_image-file is
>>> prompted and sems to be loaded, but not registered.  There's no
md_image
>>> with 'lsmod', hence it's not astonsihing that kernel
doesn't attach md0
>>> so booting fails since there's no rootfs.
>>>
>>> Any help highly appreciated, hope Toomas doesn't mind beeing
initially CC'd.
>>>
>>> Thanks,
>>>
>>> -harry
>>
>> The first question is, how large is the md_image and what other modules
are loaded?
> Thanks for your quick response.
>
> The images are 50-500MB uncompressed (provided by gzip compressed file).
> Small ammount of elf modules, 5, each ~50kB.
On the real HW, there's vmm and some more:
Id Refs Address             Size Name
 1   46 0xffffffff80200000   16M kernel
 2    1 0xffffffff8121d000   86K unionfs.ko
 3    1 0xffffffff81233000  3.1M zfs.ko
 4    2 0xffffffff81545000   51K opensolaris.ko
 5    7 0xffffffff81552000  279K usb.ko
 6    1 0xffffffff81598000   67K ukbd.ko
 7    1 0xffffffff815a9000   51K umass.ko
 8    1 0xffffffff815b6000   46K aesni.ko
 9    1 0xffffffff815c3000   54K uhci.ko
10    1 0xffffffff815d1000   65K ehci.ko
11    1 0xffffffff815e2000   15K cc_htcp.ko
12    1 0xffffffff815e6000  3.4M vmm.ko
13    1 0xffffffffa3a21000   12K ums.ko
14    1 0xffffffffa3a24000  9.1K uhid.ko

Providing md_image uncompressed doesn't change anything.

Will deploy a /usr separated rootfs, which is only ~100MB uncompressed
and see if that changes anything.
That's all I can provide, code is far beyond my knowledge...

-harry

> On 16. mai 2017, at 18:45, Harry Schmalzbauer <freebsd at omnilan.de>
wrote:
> 
> Bez?glich Harry Schmalzbauer's Nachricht vom 16.05.2017 17:28
(localtime):
>> Bez?glich Toomas Soome's Nachricht vom 16.05.2017 16:57
(localtime):
>>>> On 16. mai 2017, at 17:55, Harry Schmalzbauer <freebsd at
omnilan.de> wrote:
>>>> 
>>>> Hello,
>>>> 
>>>> unfortunately I had some trouble with my preferred MFS-root
setups.
>>>> It seems EFI loader doesn't handle type md_image correctly.
>>>> 
>>>> If I load any md_image with loader invoked by gptboot or
gptzfsboot,
>>>> 'lsmod'
>>>> shows "elf kernel", "elf obj module(s)" and
"md_image".
>>>> 
>>>> Using the same loader.conf, but EFI loader, the md_image-file
is
>>>> prompted and sems to be loaded, but not registered. 
There's no md_image
>>>> with 'lsmod', hence it's not astonsihing that
kernel doesn't attach md0
>>>> so booting fails since there's no rootfs.
>>>> 
>>>> Any help highly appreciated, hope Toomas doesn't mind
beeing initially CC'd.
>>>> 
>>>> Thanks,
>>>> 
>>>> -harry
>>> 
>>> The first question is, how large is the md_image and what other
modules are loaded?
>> Thanks for your quick response.
>> 
>> The images are 50-500MB uncompressed (provided by gzip compressed
file).
>> Small ammount of elf modules, 5, each ~50kB.
> 
> On the real HW, there's vmm and some more:
> Id Refs Address             Size Name
> 1   46 0xffffffff80200000   16M kernel
> 2    1 0xffffffff8121d000   86K unionfs.ko
> 3    1 0xffffffff81233000  3.1M zfs.ko
> 4    2 0xffffffff81545000   51K opensolaris.ko
> 5    7 0xffffffff81552000  279K usb.ko
> 6    1 0xffffffff81598000   67K ukbd.ko
> 7    1 0xffffffff815a9000   51K umass.ko
> 8    1 0xffffffff815b6000   46K aesni.ko
> 9    1 0xffffffff815c3000   54K uhci.ko
> 10    1 0xffffffff815d1000   65K ehci.ko
> 11    1 0xffffffff815e2000   15K cc_htcp.ko
> 12    1 0xffffffff815e6000  3.4M vmm.ko
> 13    1 0xffffffffa3a21000   12K ums.ko
> 14    1 0xffffffffa3a24000  9.1K uhid.ko
> 
> Providing md_image uncompressed doesn't change anything.
> 
> Will deploy a /usr separated rootfs, which is only ~100MB uncompressed
> and see if that changes anything.
> That's all I can provide, code is far beyond my knowledge...
> 
> -harry

The issue is, that current UEFI implementation is using 64MB staging memory for
loading the kernel and modules and files. When the boot is called, the
relocation code will put the bits from staging area into the final places. The
BIOS version does not need such staging area, and that will explain the
difference.

I actually have different implementation to address the same problem, but thats
for illumos case, and will need some work to make it usable for freebsd; the
idea is actually simple - allocate staging area per loaded file and relocate the
bits into the place by component, not as continuous large chunk (this would also
allow to avoid the mines like planted by hyperv;), but right now there is no
very quick real solution other than just build efi loader with larger staging
size.

rgds,
toomas