libvirt users - Oct 2020 - Encrypting boot partition Libvirt not showing the OS booting up

Hi,

I have installed Debian Buster with encrypted LVM so apon installation
my root partition is encrypted.
So far so good but as soon as I encrypt the boot partition, after reboot
the OS won't start.
If I start the drive directly with qemu, it works but it looks like
Libvirt is somehow not able to deel with it.

What am I missing?

--
John Doe

On Mon, Oct 12, 2020 at 10:03:15 +0200, john doe wrote:
> Hi,
> 
> I have installed Debian Buster with encrypted LVM so apon installation
> my root partition is encrypted.
> So far so good but as soon as I encrypt the boot partition, after reboot
> the OS won't start.
> If I start the drive directly with qemu, it works but it looks like
> Libvirt is somehow not able to deel with it.
This is not enough information to diagnose the problem.

We'll need the following:

1) Did you encrypt the partition using the debian installer

2) what vm XML you used:
    a) during installation
    b) when trying to finally boot the vm

3) what qemu command line you've used at the point you claim it worked

4) what is the error/final state when the VM fails to boot with libvirt

On 10/12/2020 11:53 AM, john doe wrote:
> On 10/12/2020 11:37 AM, Peter Krempa wrote:
>> On Mon, Oct 12, 2020 at 11:27:20 +0200, john doe wrote:
>>> Hi, thank you for your answer, I'm sending this privatly as you
asked
>>> for private information.
>>> Can I ask you to keep those information private?
>>>
>>> On 10/12/2020 10:29 AM, Peter Krempa wrote:
>>>> On Mon, Oct 12, 2020 at 10:03:15 +0200, john doe wrote:
>>>>> Hi,
>>>>>
>>>>> I have installed Debian Buster with encrypted LVM so apon
installation
>>>>> my root partition is encrypted.
>>>>> So far so good but as soon as I encrypt the boot partition,
after
>>>>> reboot
>>>>> the OS won't start.
>>>>> If I start the drive directly with qemu, it works but it
looks like
>>>>> Libvirt is somehow not able to deel with it.
>>>>
>>>> This is not enough information to diagnose the problem.
>>>>
>>>> We'll need the following:
>>>>
>>>> 1) Did you encrypt the partition using the debian installer
>>>>
>>>
>>> No, I did it after installation following the instructions at (1).
>>>
>>>> 2) what vm XML you used:
>>>>       a) during installation
>>>
>>> The domain xml file was created by virt-install with the following
>>> command:
>>> $ virsh destroy try01; virsh undefine try01; time virt-install
>>> --name=try01 --ram=1024 --noreboot --cpuset=auto --cpu host
>>> --vcpus=1,maxvcpus=4
--disk=path=/mnt/usbkey01/machines/try/try01,size=6
>>> --graphic none --pxe --os-variant=debian10 --network
>>> bridge=br0,mac=0e:35:32:84:c3:f3 --filesystem
>>>
type=mount,mode=mapped,source=/mnt/usbkey01/public,target=public_dir
>>>
>>>>       b) when trying to finally boot the vm
>>>>
>>>
>>> Attached as 'try01.xml' obtained by doing 'virsh
dumpxml try01 >
>>> try01.xml'.
>>>
>>>> 3) what qemu command line you've used at the point you
claim it worked
>>>>
>>>
>>> qemu-system-x86_64 -drive file=/mnt/usbkey01/machines/try/try01 -m
1024
>>> -boot c -accel kvm   -machine q35 -nographic
>>>
>>>> 4) what is the error/final state when the VM fails to boot with
libvirt
>>>>
>>>
>>> After having encrypted the boot partition:
>>>
>>> $virsh console try01
>>> root@0e-35-32-84-c3-f3:# [  208.513259] watchdog: watchdog0:
watchdog
>>> did not stop!
>>> [  208.855971] reboot: Restarting system
>>>
>>>
>>> $ qemu-system-x86_64 -drive file=/mnt/usbkey01/machines/try/try01
-m
>>> 1024 -boot c -accel kvm   -machine q35 -nographic
>>> SeaBIOS (version 1.12.0-1)
>>>
>>>
>>> iPXE (http://ipxe.org) 00:02.0 C980 PCI2.10 PnP
PMM+3FF8FE80+3FECFE80
>>> C980
>>>
>>>
>>>
>>> Booting from Hard Disk...
>>> Attempting to decrypt master key...
>>> Enter passphrase for hd0,msdos1 (43a322dfc8ba4628b80afc66d49642a7):
>>>
>>>
>>> As you can see above, if I invoked qemu directly, I get prompted
for the
>>> boot passthrase but I'm not getting it when using libvirt.
>>
>> Okay, so the root cause ... or "problem" here is that you
don't see the
>> console via 'virsh console' after the guest OS rebooted.
>>
>> My suspicion according to the VM XML is that the VM restart triggered a
>> restart of the qemu process and thus our internal handler of the
console
>> passthrough disconnected.
>>
>> Please try a "virsh destroy $VM" if the VM is running/stuck
waiting for
>> the password without actually showing it and then start it using
>>
>> virsh start --console $VM
>>
>> This will start it and connect to the console immediately.
>>
>> Please report your findings, we might want to discuss what happens when
>> a console is connected and the guest uses a setting of:
>>
>> <on_reboot>restart</on_reboot>
>>
>> when the VM is rebooted.
>>
>
> Still no luck:
>
> $ virsh destroy try01; virsh start --console try01
> error: Failed to destroy domain try01
> error: Requested operation is not valid: domain is not running
>
> Domain try01 started
> Connected to domain try01
> Escape character is ^]
>
>
>
>
>
> I did not modify anything during the time I sent my answer and seeing
> yours.
>
I sent privately the requested xml file to 'Peter Krempa
<pkrempa@redhat.com>'.
Peter Krempa 's privately answered me back suggesting to add the
following in the domain xml file:


<bios useserial='yes'/> under <os>

such as ...

    <os>
      <type arch='x86_64'
machine='pc-q35-3.1'>hvm</type>
      <boot dev='hd'/>
      <bios useserial='yes'/>
    </os>


This does not help at all and still give the output sent previously.

--
John Doe

On 10/12/20 2:14 PM, john doe wrote:
>> <snip/>
> 
> I sent privately the requested xml file to 'Peter Krempa
> <pkrempa@redhat.com>'.
> Peter Krempa 's privately answered me back suggesting to add the
> following in the domain xml file:
Solving things privately doesn't help the community.
> 
> 
> <bios useserial='yes'/> under <os>
> 
> such as ...
> 
>     <os>
>       <type arch='x86_64'
machine='pc-q35-3.1'>hvm</type>
>       <boot dev='hd'/>
>       <bios useserial='yes'/>
>     </os>
> 
Try adding:

     <loader type='rom'>/usr/share/seabios/bios.bin</loader>

to <os/> so that qemu actually uses SGA bios.

Michal