libvirt users - Jan 2020 - Re: aarch64 vm doesn't boots

Greetings Andrea,
> Sent: Monday, January 06, 2020 at 11:37 AM
> From: "Andrea Bolognani" <abologna@redhat.com>
> To: daggs <daggs@gmx.com>, libvirt-users@redhat.com
> Subject: Re: [libvirt-users] aarch64 vm doesn't boots
>
> On Thu, 2019-12-26 at 09:00 +0100, daggs wrote:
> > Greetings,
> >
> > I'm trying to bring up a alpine rpi aarch64 image within kvm but
I'm ended up with a stuck system, here is the xml:
> [...]
> > generated using this cmd:
> > virt-install --cpu cortex-a53 --name alpine_rpi4_dev_machine --cdrom
./alpine-virt-3.11.2-aarch64.iso --disk path=alpine-rpi4.qcow2,size=8 --vcpus 4
--memory 2048 --os-type linux --arch aarch64
> >
> > I've tried adding a vnc server and vga device but the screen stays
black, qxl doesn't work.
> > I'm using ubuntu 16.04 with libvirt 1.3.1, if this is a version
issue, I can upgrade to latest version.
> > what I'm I missing?
>
> Your version of libvirt (as well as QEMU and virt-manager, I assume)
> is fairly old, and aarch64 support specifically has seen quite a few
> improvements in the meantime. On my Fedora 31 laptop, which has
>
>   qemu-system-aarch64-4.2.0-2.fc31.x86_64
>   libvirt-5.10.0-2.fc31.x86_64
>   virt-manager-2.2.1-2.fc31.noarch
>
> I can run the above and, after some time, get a login prompt on the
> serial console.
>
I see, I'll try to bump the versions, can you share you're vm's xml
file?

Thanks,

Dagg

On Mon, 2020-01-06 at 20:31 +0100, daggs wrote:
> > > I'm using ubuntu 16.04 with libvirt 1.3.1, if this is a
version issue, I can upgrade to latest version.
> > > what I'm I missing?
> > 
> > Your version of libvirt (as well as QEMU and virt-manager, I assume)
> > is fairly old, and aarch64 support specifically has seen quite a few
> > improvements in the meantime. On my Fedora 31 laptop, which has
> > 
> >   qemu-system-aarch64-4.2.0-2.fc31.x86_64
> >   libvirt-5.10.0-2.fc31.x86_64
> >   virt-manager-2.2.1-2.fc31.noarch
> > 
> > I can run the above and, after some time, get a login prompt on the
> > serial console.
> 
> I see, I'll try to bump the versions, can you share you're vm's
xml file?
This is the exact virt-install command I used:

  $ virt-install \
    --cpu cortex-a53 \
    --name alpine-aarch64 \
    --cdrom ~/Downloads/alpine-virt-3.11.2-aarch64.iso \
    --disk size=8 \
    --vcpus 4 \
    --memory 2048 \
    --os-variant alpinelinux3.8 \
    --arch aarch64 \
    --boot
loader=/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw,loader.readonly=yes,loader.type=pflash,nvram.template=/usr/share/edk2/aarch64/vars-template-pflash.raw,loader_secure=no

The '--boot ...' part is only necessary because of an issue in
current libvirt that's already being worked on.

Here's the full XML:

  <domain type='qemu'>
    <name>alpine-aarch64</name>
    <uuid>e4b687d8-b535-436a-bee7-fac688d3d1b4</uuid>
    <metadata>
      <libosinfo:libosinfo
xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
        <libosinfo:os
id="http://alpinelinux.org/alpinelinux/3.8"/>
      </libosinfo:libosinfo>
    </metadata>
    <memory unit='KiB'>2097152</memory>
    <currentMemory unit='KiB'>2097152</currentMemory>
    <vcpu placement='static'>4</vcpu>
    <os>
      <type arch='aarch64'
machine='virt-4.2'>hvm</type>
      <loader readonly='yes' secure='no'
type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader>
      <nvram
template='/usr/share/edk2/aarch64/vars-template-pflash.raw'>/var/lib/libvirt/qemu/nvram/alpine-aarch64_VARS.fd</nvram>
      <boot dev='hd'/>
    </os>
    <features>
      <acpi/>
      <gic version='2'/>
    </features>
    <cpu mode='custom' match='exact' check='none'>
      <model fallback='allow'>cortex-a53</model>
    </cpu>
    <clock offset='utc'/>
    <on_poweroff>destroy</on_poweroff>
    <on_reboot>restart</on_reboot>
    <on_crash>destroy</on_crash>
    <devices>
      <emulator>/usr/bin/qemu-system-aarch64</emulator>
      <disk type='file' device='disk'>
        <driver name='qemu' type='qcow2'/>
        <source
file='/var/lib/libvirt/images/alpine-aarch64.qcow2'/>
        <target dev='vda' bus='virtio'/>
        <address type='pci' domain='0x0000'
bus='0x05' slot='0x00' function='0x0'/>
      </disk>
      <disk type='file' device='cdrom'>
        <driver name='qemu' type='raw'/>
        <target dev='sda' bus='scsi'/>
        <readonly/>
        <address type='drive' controller='0' bus='0'
target='0' unit='0'/>
      </disk>
      <controller type='usb' index='0'
model='qemu-xhci' ports='15'>
        <address type='pci' domain='0x0000'
bus='0x02' slot='0x00' function='0x0'/>
      </controller>
      <controller type='scsi' index='0'
model='virtio-scsi'>
        <address type='pci' domain='0x0000'
bus='0x03' slot='0x00' function='0x0'/>
      </controller>
      <controller type='pci' index='0'
model='pcie-root'/>
      <controller type='pci' index='1'
model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='1' port='0x8'/>
        <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x0'
multifunction='on'/>
      </controller>
      <controller type='pci' index='2'
model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='2' port='0x9'/>
        <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x1'/>
      </controller>
      <controller type='pci' index='3'
model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='3' port='0xa'/>
        <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x2'/>
      </controller>
      <controller type='pci' index='4'
model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='4' port='0xb'/>
        <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x3'/>
      </controller>
      <controller type='pci' index='5'
model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='5' port='0xc'/>
        <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x4'/>
      </controller>
      <controller type='pci' index='6'
model='pcie-root-port'>
        <model name='pcie-root-port'/>
        <target chassis='6' port='0xd'/>
        <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x5'/>
      </controller>
      <controller type='virtio-serial' index='0'>
        <address type='pci' domain='0x0000'
bus='0x04' slot='0x00' function='0x0'/>
      </controller>
      <interface type='network'>
        <mac address='52:54:00:5a:9c:da'/>
        <source network='default'/>
        <model type='virtio'/>
        <address type='pci' domain='0x0000'
bus='0x01' slot='0x00' function='0x0'/>
      </interface>
      <serial type='pty'>
        <target type='system-serial' port='0'>
          <model name='pl011'/>
        </target>
      </serial>
      <console type='pty'>
        <target type='serial' port='0'/>
      </console>
      <channel type='unix'>
        <target type='virtio'
name='org.qemu.guest_agent.0'/>
        <address type='virtio-serial' controller='0'
bus='0' port='1'/>
      </channel>
    </devices>
  </domain>

-- 
Andrea Bolognani / Red Hat / Virtualization

Greetings Andrea,
> Sent: Tuesday, January 07, 2020 at 1:55 PM
> From: "Andrea Bolognani" <abologna@redhat.com>
> To: daggs <daggs@gmx.com>
> Cc: libvirt-users@redhat.com
> Subject: Re: [libvirt-users] aarch64 vm doesn't boots
>
> On Mon, 2020-01-06 at 20:31 +0100, daggs wrote:
> > > > I'm using ubuntu 16.04 with libvirt 1.3.1, if this is a
version issue, I can upgrade to latest version.
> > > > what I'm I missing?
> > >
> > > Your version of libvirt (as well as QEMU and virt-manager, I
assume)
> > > is fairly old, and aarch64 support specifically has seen quite a
few
> > > improvements in the meantime. On my Fedora 31 laptop, which has
> > >
> > >   qemu-system-aarch64-4.2.0-2.fc31.x86_64
> > >   libvirt-5.10.0-2.fc31.x86_64
> > >   virt-manager-2.2.1-2.fc31.noarch
> > >
> > > I can run the above and, after some time, get a login prompt on
the
> > > serial console.
> >
> > I see, I'll try to bump the versions, can you share you're
vm's xml file?
>
> This is the exact virt-install command I used:
>
>   $ virt-install \
>     --cpu cortex-a53 \
>     --name alpine-aarch64 \
>     --cdrom ~/Downloads/alpine-virt-3.11.2-aarch64.iso \
>     --disk size=8 \
>     --vcpus 4 \
>     --memory 2048 \
>     --os-variant alpinelinux3.8 \
>     --arch aarch64 \
>     --boot
loader=/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw,loader.readonly=yes,loader.type=pflash,nvram.template=/usr/share/edk2/aarch64/vars-template-pflash.raw,loader_secure=no
>
> The '--boot ...' part is only necessary because of an issue in
> current libvirt that's already being worked on.
>
> Here's the full XML:
>
>   <domain type='qemu'>
>     <name>alpine-aarch64</name>
>     <uuid>e4b687d8-b535-436a-bee7-fac688d3d1b4</uuid>
>     <metadata>
>       <libosinfo:libosinfo
xmlns:libosinfo="http://libosinfo.org/xmlns/libvirt/domain/1.0">
>         <libosinfo:os
id="http://alpinelinux.org/alpinelinux/3.8"/>
>       </libosinfo:libosinfo>
>     </metadata>
>     <memory unit='KiB'>2097152</memory>
>     <currentMemory unit='KiB'>2097152</currentMemory>
>     <vcpu placement='static'>4</vcpu>
>     <os>
>       <type arch='aarch64'
machine='virt-4.2'>hvm</type>
>       <loader readonly='yes' secure='no'
type='pflash'>/usr/share/edk2/aarch64/QEMU_EFI-pflash.raw</loader>
>       <nvram
template='/usr/share/edk2/aarch64/vars-template-pflash.raw'>/var/lib/libvirt/qemu/nvram/alpine-aarch64_VARS.fd</nvram>
>       <boot dev='hd'/>
>     </os>
>     <features>
>       <acpi/>
>       <gic version='2'/>
>     </features>
>     <cpu mode='custom' match='exact'
check='none'>
>       <model fallback='allow'>cortex-a53</model>
>     </cpu>
>     <clock offset='utc'/>
>     <on_poweroff>destroy</on_poweroff>
>     <on_reboot>restart</on_reboot>
>     <on_crash>destroy</on_crash>
>     <devices>
>       <emulator>/usr/bin/qemu-system-aarch64</emulator>
>       <disk type='file' device='disk'>
>         <driver name='qemu' type='qcow2'/>
>         <source
file='/var/lib/libvirt/images/alpine-aarch64.qcow2'/>
>         <target dev='vda' bus='virtio'/>
>         <address type='pci' domain='0x0000'
bus='0x05' slot='0x00' function='0x0'/>
>       </disk>
>       <disk type='file' device='cdrom'>
>         <driver name='qemu' type='raw'/>
>         <target dev='sda' bus='scsi'/>
>         <readonly/>
>         <address type='drive' controller='0'
bus='0' target='0' unit='0'/>
>       </disk>
>       <controller type='usb' index='0'
model='qemu-xhci' ports='15'>
>         <address type='pci' domain='0x0000'
bus='0x02' slot='0x00' function='0x0'/>
>       </controller>
>       <controller type='scsi' index='0'
model='virtio-scsi'>
>         <address type='pci' domain='0x0000'
bus='0x03' slot='0x00' function='0x0'/>
>       </controller>
>       <controller type='pci' index='0'
model='pcie-root'/>
>       <controller type='pci' index='1'
model='pcie-root-port'>
>         <model name='pcie-root-port'/>
>         <target chassis='1' port='0x8'/>
>         <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x0'
multifunction='on'/>
>       </controller>
>       <controller type='pci' index='2'
model='pcie-root-port'>
>         <model name='pcie-root-port'/>
>         <target chassis='2' port='0x9'/>
>         <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x1'/>
>       </controller>
>       <controller type='pci' index='3'
model='pcie-root-port'>
>         <model name='pcie-root-port'/>
>         <target chassis='3' port='0xa'/>
>         <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x2'/>
>       </controller>
>       <controller type='pci' index='4'
model='pcie-root-port'>
>         <model name='pcie-root-port'/>
>         <target chassis='4' port='0xb'/>
>         <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x3'/>
>       </controller>
>       <controller type='pci' index='5'
model='pcie-root-port'>
>         <model name='pcie-root-port'/>
>         <target chassis='5' port='0xc'/>
>         <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x4'/>
>       </controller>
>       <controller type='pci' index='6'
model='pcie-root-port'>
>         <model name='pcie-root-port'/>
>         <target chassis='6' port='0xd'/>
>         <address type='pci' domain='0x0000'
bus='0x00' slot='0x01' function='0x5'/>
>       </controller>
>       <controller type='virtio-serial' index='0'>
>         <address type='pci' domain='0x0000'
bus='0x04' slot='0x00' function='0x0'/>
>       </controller>
>       <interface type='network'>
>         <mac address='52:54:00:5a:9c:da'/>
>         <source network='default'/>
>         <model type='virtio'/>
>         <address type='pci' domain='0x0000'
bus='0x01' slot='0x00' function='0x0'/>
>       </interface>
>       <serial type='pty'>
>         <target type='system-serial' port='0'>
>           <model name='pl011'/>
>         </target>
>       </serial>
>       <console type='pty'>
>         <target type='serial' port='0'/>
>       </console>
>       <channel type='unix'>
>         <target type='virtio'
name='org.qemu.guest_agent.0'/>
>         <address type='virtio-serial' controller='0'
bus='0' port='1'/>
>       </channel>
>     </devices>
>   </domain>
thanks. I'll test it out.

Dagg.