libvirt users - Feb 2018 - Re: "scripts are not supported on interfaces of type vhostuser" error

Hi,

I'm having trouble starting a VM with vhostuser interfaces.

I have a simple configuration where a VM running Debian has 1 vhostuser
interface plugged into an OVS switch where a DPDK interface is already
plugged in.
$ ovs-vsctl show:
Bridge "switch1"
    Port "switch1"
        Interface "switch1"
            type: internal
    Port "1.switch1"
        Interface "1.switch1"
            type: dpdk
            options: {dpdk-devargs="0000:0b:00.0"}
    Port "0.switch1"
        Interface "0.vm"
            type: dpdkvhostuserclient
            options: {vhost-server-path="/opt/oa/vhost/0.vm.sock"}


The relevant excerpt from the XML of my VM is:

<?xml version="1.0" ?>
<domain type="kvm"
xmlns:ns0="http://libvirt.org/schemas/domain/qemu/1.0">
 <!-- [...]  -->
 <devices>
    <emulator>/opt/oa/bin/qemu-system-x86_64</emulator>
    <!-- [...]  -->
    <interface type="vhostuser">
      <script path="/opt/oa/etc/qemu-ifup"/>
      <source mode="server"
path="/opt/oa/vhost/0.vm.sock" type="unix"/>
      <mac address="52:54:00:41:37:08"/>
      <model type="virtio"/>
      <driver queues="1">
        <host csum="off"/>
      </driver>
    </interface>
    <!-- [...]  -->
  </devices>
  <ns0:commandline>
    <ns0:arg value="-object"/>
    <ns0:arg
value="memory-backend-file,id=mem,size=2G,mem-path=/dev/hugepages,share=on"/>
    <ns0:arg value="-numa"/>
    <ns0:arg value="node,memdev=mem"/>
    <ns0:arg value="-mem-prealloc"/>
  </ns0:commandline>
</domain>


Now, if I try to start my VM, I get the following error and the VM is not
started at all:
$ virsh create vm.xml
error: Failed to create domain from vm.xml
error: unsupported configuration: scripts are not supported on interfaces
of type vhostuser


The logs from libvirtd.log say:
2018-02-22 09:18:24.982+0000: 2033: warning :
qemuProcessStartWarnShmem:4539 : Detected vhost-user interface without any
shared memory, the interface might not be operational
2018-02-22 09:18:24.982+0000: 2033: error : qemuBuildHostNetStr:3894 :
unsupported configuration: scripts are not supported on interfaces of type
vhostuser

The logs from qemu simply say:
2018-02-22 09:26:15.857+0000: shutting down, reason=failed

And finally, ovs-vswitchd.log:
2018-02-22T09:18:24.715Z|00328|dpdk|INFO|VHOST_CONFIG: vhost-user client:
socket created, fd: 51
2018-02-22T09:18:24.716Z|00329|netdev_dpdk|INFO|vHost User device '0.vm'
created in 'client' mode, using client socket
'/opt/oa/vhost/0.vm.sock'
2018-02-22T09:18:24.718Z|00330|dpdk|WARN|VHOST_CONFIG: failed to connect to
/opt/oa/vhost/0.vm.sock: No such file or directory
2018-02-22T09:18:24.718Z|00331|dpdk|INFO|VHOST_CONFIG:
/opt/oa/vhost/0.vm.sock: reconnecting...
2018-02-22T09:18:24.718Z|00332|bridge|INFO|bridge switch1: added interface
0.vm on port 5


Am I missing something on the openvswitch or on the libvirt side?

It looks like openvswitch can't find /opt/oa/vhost/0.vm.sock, but isn't
either openvswitch or libvirt in charge of creating it?
Then, I'm not too sure about the error messages in libvirtd.log...

My software versions are: libvirt 3.10.0, qemu 2.10.2, openvswitch 2.8.1
and DPDK 17.11.

Thanks a lot!

Riccardo

On 02/22/2018 11:41 AM, Riccardo Ravaioli wrote:
> Hi,
> 
> I'm having trouble starting a VM with vhostuser interfaces.
> 
> I have a simple configuration where a VM running Debian has 1 vhostuser
> interface plugged into an OVS switch where a DPDK interface is already
> plugged in.
> $ ovs-vsctl show:
> Bridge "switch1"
>     Port "switch1"
>         Interface "switch1"
>             type: internal
>     Port "1.switch1"
>         Interface "1.switch1"
>             type: dpdk
>             options: {dpdk-devargs="0000:0b:00.0"}
>     Port "0.switch1"
>         Interface "0.vm"
>             type: dpdkvhostuserclient
>             options:
{vhost-server-path="/opt/oa/vhost/0.vm.sock"}
> 
> 
> The relevant excerpt from the XML of my VM is:
> 
> <?xml version="1.0" ?>
> <domain type="kvm"
xmlns:ns0="http://libvirt.org/schemas/domain/qemu/1.0">
>  <!-- [...]  -->
>  <devices>
>     <emulator>/opt/oa/bin/qemu-system-x86_64</emulator>
>     <!-- [...]  -->
>     <interface type="vhostuser">
>       <script path="/opt/oa/etc/qemu-ifup"/>
^^^^^^^^^^^^^^^^ Why do you have this line?
>       <source mode="server"
path="/opt/oa/vhost/0.vm.sock" type="unix"/>
>       <mac address="52:54:00:41:37:08"/>
>       <model type="virtio"/>
>       <driver queues="1">
>         <host csum="off"/>
>       </driver>
>     </interface>
>     <!-- [...]  -->
>   </devices>
>   <ns0:commandline>
>     <ns0:arg value="-object"/>
>     <ns0:arg
>
value="memory-backend-file,id=mem,size=2G,mem-path=/dev/hugepages,share=on"/>
>     <ns0:arg value="-numa"/>
>     <ns0:arg value="node,memdev=mem"/>
>     <ns0:arg value="-mem-prealloc"/>
>   </ns0:commandline>
> </domain>
> 
> 
> Now, if I try to start my VM, I get the following error and the VM is
> not started at all:
> $ virsh create vm.xml
> error: Failed to create domain from vm.xml
> error: unsupported configuration: scripts are not supported on
> interfaces of type vhostuser
This error message tells you exactly what is wrong. The <script> element
is only supported for interfaces of type='ethernet' - these are
interfaces for which a tap device is created, but it's not connected to
anything; the purpose of the script is to connect the tap to device to
[something user-defined]. Other types of interface (e.g. vhost-user)
should have enough intelligence built into the code to perform all
necessary setup.

Do you actually have something in this qemu-ifup file?

> 
> 
> The logs from libvirtd.log say:
> 2018-02-22 09:18:24.982+0000: 2033: warning :
> qemuProcessStartWarnShmem:4539 : Detected vhost-user interface without
> any shared memory, the interface might not be operational
> 2018-02-22 09:18:24.982+0000: 2033: error : qemuBuildHostNetStr:3894 :
> unsupported configuration: scripts are not supported on interfaces of
> type vhostuser
> 
> The logs from qemu simply say:
> 2018-02-22 09:26:15.857+0000: shutting down, reason=failed
> 
> And finally, ovs-vswitchd.log:
> 2018-02-22T09:18:24.715Z|00328|dpdk|INFO|VHOST_CONFIG: vhost-user
> client: socket created, fd: 51
> 2018-02-22T09:18:24.716Z|00329|netdev_dpdk|INFO|vHost User device
'0.vm'
> created in 'client' mode, using client socket
'/opt/oa/vhost/0.vm.sock'
> 2018-02-22T09:18:24.718Z|00330|dpdk|WARN|VHOST_CONFIG: failed to connect
> to /opt/oa/vhost/0.vm.sock: No such file or directory
> 2018-02-22T09:18:24.718Z|00331|dpdk|INFO|VHOST_CONFIG:
> /opt/oa/vhost/0.vm.sock: reconnecting...
> 2018-02-22T09:18:24.718Z|00332|bridge|INFO|bridge switch1: added
> interface 0.vm on port 5
> 
> 
> Am I missing something on the openvswitch or on the libvirt side?
> 
> It looks like openvswitch can't find /opt/oa/vhost/0.vm.sock, but
isn't
> either openvswitch or libvirt in charge of creating it?
> Then, I'm not too sure about the error messages in libvirtd.log...
> 
> My software versions are: libvirt 3.10.0, qemu 2.10.2, openvswitch 2.8.1
> and DPDK 17.11.
> 
> Thanks a lot!
> 
> Riccardo
> 
> 
> _______________________________________________
> libvirt-users mailing list
> libvirt-users@redhat.com
> https://www.redhat.com/mailman/listinfo/libvirt-users
>

On 02/22/2018 05:41 PM, Riccardo Ravaioli wrote:
> Hi,
> 
> I'm having trouble starting a VM with vhostuser interfaces.
> 
> I have a simple configuration where a VM running Debian has 1 vhostuser
> interface plugged into an OVS switch where a DPDK interface is already
> plugged in.
> $ ovs-vsctl show:
> Bridge "switch1"
>     Port "switch1"
>         Interface "switch1"
>             type: internal
>     Port "1.switch1"
>         Interface "1.switch1"
>             type: dpdk
>             options: {dpdk-devargs="0000:0b:00.0"}
>     Port "0.switch1"
>         Interface "0.vm"
>             type: dpdkvhostuserclient
>             options:
{vhost-server-path="/opt/oa/vhost/0.vm.sock"}
> 
> 
> The relevant excerpt from the XML of my VM is:
> 
> <?xml version="1.0" ?>
> <domain type="kvm"
xmlns:ns0="http://libvirt.org/schemas/domain/qemu/1.0">
>  <!-- [...]  -->
>  <devices>
>     <emulator>/opt/oa/bin/qemu-system-x86_64</emulator>
>     <!-- [...]  -->
>     <interface type="vhostuser">
>       <script path="/opt/oa/etc/qemu-ifup"/>
>       <source mode="server"
path="/opt/oa/vhost/0.vm.sock" type="unix"/>
>       <mac address="52:54:00:41:37:08"/>
>       <model type="virtio"/>
>       <driver queues="1">
>         <host csum="off"/>
>       </driver>
>     </interface>
>     <!-- [...]  -->
>   </devices>
>   <ns0:commandline>
>     <ns0:arg value="-object"/>
>     <ns0:arg
>
value="memory-backend-file,id=mem,size=2G,mem-path=/dev/hugepages,share=on"/>
>     <ns0:arg value="-numa"/>
>     <ns0:arg value="node,memdev=mem"/>
>     <ns0:arg value="-mem-prealloc"/>
>   </ns0:commandline>
Unrelated, to avoid command line passthrough you can do:

  <domain>
    <memoryBacking>
      <hugepages/>
    </memoryBacking>
    ..
    <cpu>
      <numa>
        <cell id='0' cpus='$CPUS' memory='$MEM'
memAccess='shared'/>
      </numa>
    </cpu>
    ..
  </domain>

Michal

Spot on! The script qemu-ifup was indeed the cause of the problem. It's a
dummy script that only does "exit 0" and it was apparently needed in
the
past with old versions of libvirt. I see examples on the net  where it is
used to configure interfaces with shell commands. I don't need it.

I see that everything works fine without the <script
path="/opt/oa/etc/qemu-ifup"/> line, for vhostuser and ethernet
interfaces.
Are there any side effects or can I safely remove that line?

Thanks again!

Riccardo

On 22 February 2018 at 20:55, Laine Stump <laine@redhat.com> wrote:
> [...]
> >     <interface type="vhostuser">
> >       <script path="/opt/oa/etc/qemu-ifup"/>
>
> ^^^^^^^^^^^^^^^^ Why do you have this line?
>
> Spo> $ virsh create vm.xml
> > error: Failed to create domain from vm.xml
> > error: unsupported configuration: scripts are not supported on
> > interfaces of type vhostuser
>
> This error message tells you exactly what is wrong. The <script>
element
> is only supported for interfaces of type='ethernet' - these are
> interfaces for which a tap device is created, but it's not connected to
> anything; the purpose of the script is to connect the tap to device to
> [something user-defined]. Other types of interface (e.g. vhost-user)
> should have enough intelligence built into the code to perform all
> necessary setup.
>
> Do you actually have something in this qemu-ifup file?
>
>
> >
> >
> > The logs from libvirtd.log say:
> > 2018-02-22 09:18:24.982+0000: 2033: warning :
> > qemuProcessStartWarnShmem:4539 : Detected vhost-user interface without
> > any shared memory, the interface might not be operational
> > 2018-02-22 09:18:24.982+0000: 2033: error : qemuBuildHostNetStr:3894 :
> > unsupported configuration: scripts are not supported on interfaces of
> > type vhostuser
> >
> > The logs from qemu simply say:
> > 2018-02-22 09:26:15.857+0000: shutting down, reason=failed
> >
> > And finally, ovs-vswitchd.log:
> > 2018-02-22T09:18:24.715Z|00328|dpdk|INFO|VHOST_CONFIG: vhost-user
> > client: socket created, fd: 51
> > 2018-02-22T09:18:24.716Z|00329|netdev_dpdk|INFO|vHost User device
'0.vm'
> > created in 'client' mode, using client socket
'/opt/oa/vhost/0.vm.sock'
> > 2018-02-22T09:18:24.718Z|00330|dpdk|WARN|VHOST_CONFIG: failed to
connect
> > to /opt/oa/vhost/0.vm.sock: No such file or directory
> > 2018-02-22T09:18:24.718Z|00331|dpdk|INFO|VHOST_CONFIG:
> > /opt/oa/vhost/0.vm.sock: reconnecting...
> > 2018-02-22T09:18:24.718Z|00332|bridge|INFO|bridge switch1: added
> > interface 0.vm on port 5
> >
> >
> > Am I missing something on the openvswitch or on the libvirt side?
> >
> > It looks like openvswitch can't find /opt/oa/vhost/0.vm.sock, but
isn't
> > either openvswitch or libvirt in charge of creating it?
> > Then, I'm not too sure about the error messages in libvirtd.log...
> >
> > My software versions are: libvirt 3.10.0, qemu 2.10.2, openvswitch
2.8.1
> > and DPDK 17.11.
> >
> > Thanks a lot!
> >
> > Riccardo
> >
> >
> > _______________________________________________
> > libvirt-users mailing list
> > libvirt-users@redhat.com
> > https://www.redhat.com/mailman/listinfo/libvirt-users
> >
>
>