Radhika Niranjan
2012-Oct-15 22:29 UTC
[libvirt-users] cannot start domain with SR-IOV interface
Hi,
I've been using libvirt (versions 0.9.11.5, and recently upgraded to
0.10.2), and have been attaching an SR-IOV VF to my VM with the following
device configuration:
<hostdev mode='subsystem' type='pci'
managed='yes'>
<source>
<address domain='0x0000' bus='0x07'
slot='0x10' function='0x0'/>
</source>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x05'
function='0x0'/>
</hostdev>
I had no problems with bringing up the VM and using the VF with this
configuration.
Recently I noticed that I would be able to assign a mac address to the
SR-IOV VF if I used the interface configuration and changed over to using:
<interface type='hostdev'>
<mac address='52:54:00:d3:23:4f'/>
<source>
<address type='pci' domain='0x0000'
bus='0x07' slot='0x10'
function='0x0'/>
</source>
<model type='virtio'/>
<address type='pci' domain='0x0000' bus='0x00'
slot='0x03'
function='0x0'/>
</interface>
My VM fails to start up with this interface configuration. The error
reported is:
error: Failed to start domain exp85-1
error: An error occurred, but the cause is unknown
When I look into /var/log/messages I see the following messages:
Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15 22:25:48.145+0000:
17713: error : virFileReadAll:457 : Failed to open file
'/var/run/libvirt/qemu/p1p1_vf0': No such file or directory
Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15 22:25:48.146+0000:
17713: error : qemuRemoveCgroup:740 : internal error Unable to find cgroup
for exp85-1
Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15 22:25:48.146+0000:
17713: warning : qemuProcessStop:4136 : Failed to remove cgroup for exp85-1
I was wondering if anyone else has faced this problem or has a solution to
this.
I'm using Fedora17, KVM, and an 82599EB Intel NIC.
Thanks much in advance.
With best regards
Radhika
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20121015/6812ba0a/attachment.htm>
Laine Stump
2012-Oct-24 17:38 UTC
[libvirt-users] cannot start domain with SR-IOV interface
On 10/15/2012 06:29 PM, Radhika Niranjan wrote:> Hi, > > I've been using libvirt (versions 0.9.11.5, and recently upgraded to > 0.10.2), and have been attaching an SR-IOV VF to my VM with the > following device configuration: > <hostdev mode='subsystem' type='pci' managed='yes'> > <source> > <address domain='0x0000' bus='0x07' slot='0x10' function='0x0'/> > </source> > <address type='pci' domain='0x0000' bus='0x00' slot='0x05' > function='0x0'/> > </hostdev> > > I had no problems with bringing up the VM and using the VF with this > configuration. > > Recently I noticed that I would be able to assign a mac address to the > SR-IOV VF if I used the interface configuration and changed over to using: > <interface type='hostdev'>Since your <hostdev> used managed='yes', you should use it here too: <interface type='hostdev' managed='yes'>> <mac address='52:54:00:d3:23:4f'/> > <source> > <address type='pci' domain='0x0000' bus='0x07' slot='0x10' > function='0x0'/> > </source> > <model type='virtio'/><model> is unused for interface type='hostdev', so that line is harmless, but also meaningless. (actually it should probably log an error when it sees that, but currently just ignores it).> <address type='pci' domain='0x0000' bus='0x00' slot='0x03' > function='0x0'/> > </interface> > > My VM fails to start up with this interface configuration. The error > reported is: > error: Failed to start domain exp85-1 > error: An error occurred, but the cause is unknownThis is a bit odd. It means that some code path has returned an error without logging an error message. I tried removing "managed='yes'" and adding <model type='virtio'/> to a similar interface definition on the one machine where I have an SR-IOV card, and it failed to start, but gave a slightly less meaningless message: error: Unable to read from monitor: Connection reset by peer (This is less meaningless because that generally indicates that the qemu-kvm process encountered an error and exited, so I know I can look in /var/log/libvirt/qemu/$guest.log for an error. And indeed, when I look there I see this: Failed to assign device "hostdev0" : Device or resource busy qemu-kvm: -device pci-assign,host=03:10.1,id=hostdev0,configfd=32,bus=pci.0,addr=0x7: Device 'pci-assign' could not be initialized which makes sense.> > When I look into /var/log/messages I see the following messages: > > Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15 > 22:25:48.145+0000: 17713: error : virFileReadAll:457 : Failed to open > file '/var/run/libvirt/qemu/p1p1_vf0': No such file or directoryThat file is used to save the previous MAC address and vlan tag of the VF for restoration when the guest is finished with it. I'm not sure why that error wasn't propogated back to virsh. (btw, this error isn't the root error anyway - it would only be issued if a failure happened while attempting to save the old MAC address/vlan of the VF. Perhaps there's a failure in that bit of code that fails without logging an error.) How handy are you at running gdb? If you could install the debuginfo for libvirt, then attach gdb and set a breakpoint on the function qemuDomainHostdevNetConfigReplace then single step, I'm pretty sure whatever failure you're encountering is somewhere within one of the functions called by that function. Otherwise we can probably find it by manual examination, but that will take *much* more effort/time.> Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15 > 22:25:48.146+0000: 17713: error : qemuRemoveCgroup:740 : internal > error Unable to find cgroup for exp85-1 > Oct 15 15:25:48 dcswitch85 libvirtd[17709]: 2012-10-15 > 22:25:48.146+0000: 17713: warning : qemuProcessStop:4136 : Failed to > remove cgroup for exp85-1 > > I was wondering if anyone else has faced this problem or has a > solution to this. > > I'm using Fedora17, KVM, and an 82599EB Intel NIC. > > Thanks much in advance. > > With best regards > Radhika > > > > > > > _______________________________________________ > libvirt-users mailing list > libvirt-users at redhat.com > https://www.redhat.com/mailman/listinfo/libvirt-users-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://listman.redhat.com/archives/libvirt-users/attachments/20121024/780de40b/attachment.htm>