Linux Virtualization - Nov 2014 - [RFC PATCH] virtio-mmio: support for multiple irqs

Hi Shannon, 
>Type of backend         bandwith(GBytes/sec)
>virtio-net              0.66
>vhost-net               1.49
>vhost-net with irqfd    2.01
>
>Test cmd: ./iperf -c 192.168.0.2 -P 1 -i 10 -p 5001 -f G -t 60
Impressive results !
Could you please detail your setup ? which platform are you using and which GbE
controller ?
As a reference, it would be good also to have result with an iperf to the HOST
to see how far we are from a native configuration...

Also, I assume a pending Qemu patch is necessary to assign multiple irqs ?
I'm correct ?

Thanks a lot, 
Best regards
R?my

-----Message d'origine-----
De?: Shannon Zhao [mailto:zhaoshenglong at huawei.com] 
Envoy??: mercredi 5 novembre 2014 09:00
??: linux-kernel at vger.kernel.org
Cc?: mst at redhat.com; peter.maydell at linaro.org; john.liuli at huawei.com;
joel.schopp at amd.com; GAUGUEY R?my 228890; qemu-devel at nongnu.org;
n.nikolaev at virtualopensystems.com; virtualization at
lists.linux-foundation.org; peter.huangpeng at huawei.com; hangaohuai at
huawei.com
Objet?: Re: [RFC PATCH] virtio-mmio: support for multiple irqs


On 2014/11/4 17:35, Shannon Zhao wrote:
> As the current virtio-mmio only support single irq, so some advanced 
> features such as vhost-net with irqfd are not supported. And the net 
> performance is not the best without vhost-net and irqfd supporting.
> 
Hi Joel, Peter, Mst,

Some virtio-net with virtio-mmio performance data on ARM added as followed:

Type of backend         bandwith(GBytes/sec)
virtio-net              0.66
vhost-net               1.49
vhost-net with irqfd    2.01

Test cmd: ./iperf -c 192.168.0.2 -P 1 -i 10 -p 5001 -f G -t 60
>From this test data, irqfd has great improvement (about 30%) on performance.
So maybe it's necessary to enable multiple irq support to make vhost-net
with virtio-mmio on ARM be able to use irqfd.

How do you guys think? Look forward for your feedback.

Thanks,
Shannon
> This patch support virtio-mmio to request multiple irqs like 
> virtio-pci. With this patch and qemu assigning multiple irqs for 
> virtio-mmio device, it's ok to use vhost-net with irqfd on arm/arm64.
>

Hi R?my,

On 2014/11/5 16:26, GAUGUEY R?my 228890 wrote:
> Hi Shannon, 
> 
>> Type of backend         bandwith(GBytes/sec)
>> virtio-net              0.66
>> vhost-net               1.49
>> vhost-net with irqfd    2.01
>>
>> Test cmd: ./iperf -c 192.168.0.2 -P 1 -i 10 -p 5001 -f G -t 60
> 
> Impressive results !
> Could you please detail your setup ? which platform are you using and which
GbE controller ?
Sorry for not telling the test scenario. This test scenario is from Host to
Guest. It just
compare the performance of different backends. I did this test on ARM64
platform.

The setup was based on:
1)on host kvm-arm should support ioeventfd and irqfd
	The irqfd patch is from Eric "ARM: KVM: add irqfd support".
	http://www.spinics.net/lists/kvm-arm/msg11014.html

	The ioeventfd patch is reworked by me from Antonios.
	http://www.spinics.net/lists/kvm-arm/msg08413.html

2)qemu should enable ioeventfd support for virtio-mmio
	This patch is refer to Ying-Shiuan Pan and reworked for new qemu branch.
	https://lists.gnu.org/archive/html/qemu-devel/2014-11/msg00594.html

3)qemu should enable multiple irqs for virtio-mmio
	This patch isn't sent to qemu maillist as we want to check whether this is
the right direction.
	If you want to test, I'll send it to you.

4)in guest should enable support virtio-mmio to request multiple irqs
	This is what this patch do.
> As a reference, it would be good also to have result with an iperf to the
HOST to see how far we are from a native configuration...
Agree!
> 
> Also, I assume a pending Qemu patch is necessary to assign multiple irqs ?
I'm correct ?
Yes, the patch is on it's way :)
> 
> Thanks a lot, 
> Best regards
> R?my
> 
> -----Message d'origine-----
> De : Shannon Zhao [mailto:zhaoshenglong at huawei.com] 
> Envoy? : mercredi 5 novembre 2014 09:00
> ? : linux-kernel at vger.kernel.org
> Cc : mst at redhat.com; peter.maydell at linaro.org; john.liuli at
huawei.com; joel.schopp at amd.com; GAUGUEY R?my 228890; qemu-devel at
nongnu.org; n.nikolaev at virtualopensystems.com; virtualization at
lists.linux-foundation.org; peter.huangpeng at huawei.com; hangaohuai at
huawei.com
> Objet : Re: [RFC PATCH] virtio-mmio: support for multiple irqs
> 
> 
> On 2014/11/4 17:35, Shannon Zhao wrote:
>> As the current virtio-mmio only support single irq, so some advanced 
>> features such as vhost-net with irqfd are not supported. And the net 
>> performance is not the best without vhost-net and irqfd supporting.
>>
> Hi Joel, Peter, Mst,
> 
> Some virtio-net with virtio-mmio performance data on ARM added as followed:
> 
> Type of backend         bandwith(GBytes/sec)
> virtio-net              0.66
> vhost-net               1.49
> vhost-net with irqfd    2.01
> 
> Test cmd: ./iperf -c 192.168.0.2 -P 1 -i 10 -p 5001 -f G -t 60
> 
>>From this test data, irqfd has great improvement (about 30%) on
performance.
> So maybe it's necessary to enable multiple irq support to make
vhost-net with virtio-mmio on ARM be able to use irqfd.
> 
> How do you guys think? Look forward for your feedback.
> 
> Thanks,
> Shannon
> 
>> This patch support virtio-mmio to request multiple irqs like 
>> virtio-pci. With this patch and qemu assigning multiple irqs for 
>> virtio-mmio device, it's ok to use vhost-net with irqfd on
arm/arm64.
>>
> .
> 
-- 
Shannon

On 11/05/2014 03:12 AM, Shannon Zhao wrote:
> Hi R?my,
>
> On 2014/11/5 16:26, GAUGUEY R?my 228890 wrote:
>> Hi Shannon, 
>>
>>> Type of backend         bandwith(GBytes/sec)
>>> virtio-net              0.66
>>> vhost-net               1.49
>>> vhost-net with irqfd    2.01
>>>
>>> Test cmd: ./iperf -c 192.168.0.2 -P 1 -i 10 -p 5001 -f G -t 60
>> Impressive results !
>> Could you please detail your setup ? which platform are you using and
which GbE controller ?
> Sorry for not telling the test scenario. This test scenario is from Host to
Guest. It just
> compare the performance of different backends. I did this test on ARM64
platform.
>
> The setup was based on:
> 1)on host kvm-arm should support ioeventfd and irqfd
> 	The irqfd patch is from Eric "ARM: KVM: add irqfd support".
> 	http://www.spinics.net/lists/kvm-arm/msg11014.html
>
> 	The ioeventfd patch is reworked by me from Antonios.
> 	http://www.spinics.net/lists/kvm-arm/msg08413.html
>
> 2)qemu should enable ioeventfd support for virtio-mmio
> 	This patch is refer to Ying-Shiuan Pan and reworked for new qemu branch.
> 	https://lists.gnu.org/archive/html/qemu-devel/2014-11/msg00594.html
>
> 3)qemu should enable multiple irqs for virtio-mmio
> 	This patch isn't sent to qemu maillist as we want to check whether
this is the right direction.
> 	If you want to test, I'll send it to you.
I'm not a maintainer so my opinion isn't worth a lot here, but this
seems like the right direction to me.  I'd like to see the qemu patch
(do mention the dependency on the kernel patch) on the qemu-devel
mailing list.  I think these numbers also support some of the prereqs
listed above that have gone through several iterations getting queued up
for 3.19.