On Mon, Nov 17, 2014 at 12:38:16PM +0200, Michael S. Tsirkin
wrote:> On Mon, Nov 17, 2014 at 09:44:23AM +0200, Gleb Natapov wrote:
> > On Sun, Nov 16, 2014 at 08:56:04PM +0200, Michael S. Tsirkin wrote:
> > > On Sun, Nov 16, 2014 at 06:18:18PM +0200, Gleb Natapov wrote:
> > > > Hi Michael,
> > > >
> > > > I am playing with vhost multiqueue capability and have a
question about
> > > > vhost multiqueue and RSS (receive side steering). My setup
has Mellanox
> > > > ConnectX-3 NIC which supports multiqueue and RSS. Network
related
> > > > parameters for qemu are:
> > > >
> > > > -netdev tap,id=hn0,script=qemu-ifup.sh,vhost=on,queues=4
> > > > -device
virtio-net-pci,netdev=hn0,id=nic1,mq=on,vectors=10
> > > >
> > > > In a guest I ran "ethtool -L eth0 combined 4" to
enable multiqueue.
> > > >
> > > > I am running one tcp stream into the guest using iperf.
Since there is
> > > > only one tcp stream I expect it to be handled by one queue
only but
> > > > this seams to be not the case. ethtool -S on a host shows
that the
> > > > stream is handled by one queue in the NIC, just like I would
expect,
> > > > but in a guest all 4 virtio-input interrupt are incremented.
Am I
> > > > missing any configuration?
> > >
> > > I don't see anything obviously wrong with what you describe.
> > > Maybe, somehow, same irqfd got bound to multiple MSI vectors?
> > It does not look like this is what is happening judging by the way
> > interrupts are distributed between queues. They are not distributed
> > uniformly and often I see one queue gets most interrupt and others get
> > much less and then it changes.
>
> Weird. It would happen if you transmitted from multiple CPUs.
> You did pin iperf to a single CPU within guest, did you not?
>
No, I didn't because I didn't expect it to matter for input interrupts.
When I run iperf on a host rx queue that receives all packets depends
only on a connection itself, not on a cpu iperf is running on (I tested
that). When I pin iperf in a guest I do indeed see that all interrupts
are arriving to the same irq vector. Is a number after virtio-input
in /proc/interrupt any indication of a queue a packet arrived to (on
a host I can use ethtool -S to check what queue receives packets, but
unfortunately this does not work for virtio nic in a guest)? Because if
it is the way RSS works in virtio is not how it works on a host and not
what I would expect after reading about RSS. The queue a packets arrives
to should be calculated by hashing fields from a packet header only.
--
Gleb.