Wei Liu
2013-Aug-26 11:59 UTC
[PATCH net-next 0/3] xen-netback: switch to NAPI + kthread 1:1 model
This series implements NAPI + kthread 1:1 model for Xen netback. This model - provides better scheduling fairness among vifs - is prerequisite for implementing multiqueue for Xen network driver The second patch has the real meat: - make use of NAPI to mitigate interrupt - kthreads are not bound to CPUs any more, so that we can take advantage of backend scheduler and trust it to do the right thing Benchmark is done on a Dell T3400 workstation with 4 cores, running 4 DomUs. Netserver runs in Dom0. DomUs do netperf to Dom0 with following command: /root/netperf -H Dom0 -fm -l120 IRQs are distributed to 4 cores by hand in the new model, while in the old model vifs are automatically distributed to 4 kthreads. * New model %Cpu0 : 0.5 us, 20.3 sy, 0.0 ni, 28.9 id, 0.0 wa, 0.0 hi, 24.4 si, 25.9 st %Cpu1 : 0.5 us, 17.8 sy, 0.0 ni, 28.8 id, 0.0 wa, 0.0 hi, 27.7 si, 25.1 st %Cpu2 : 0.5 us, 18.8 sy, 0.0 ni, 30.7 id, 0.0 wa, 0.0 hi, 22.9 si, 27.1 st %Cpu3 : 0.0 us, 20.1 sy, 0.0 ni, 30.4 id, 0.0 wa, 0.0 hi, 22.7 si, 26.8 st Throughputs: 2027.89 2025.95 2018.57 2016.23 aggregated: 8088.64 * Old model %Cpu0 : 0.5 us, 68.8 sy, 0.0 ni, 16.1 id, 0.5 wa, 0.0 hi, 2.8 si, 11.5 st %Cpu1 : 0.4 us, 45.1 sy, 0.0 ni, 31.1 id, 0.4 wa, 0.0 hi, 2.1 si, 20.9 st %Cpu2 : 0.9 us, 44.8 sy, 0.0 ni, 30.9 id, 0.0 wa, 0.0 hi, 1.3 si, 22.2 st %Cpu3 : 0.8 us, 46.4 sy, 0.0 ni, 28.3 id, 1.3 wa, 0.0 hi, 2.1 si, 21.1 st Throughputs: 1899.14 2280.43 1963.33 1893.47 aggregated: 8036.37 We can see that the impact is mainly on CPU usage. The new model moves processing from kthread to NAPI (software interrupt). Wei Liu (3): xen-netback: remove page tracking facility xen-netback: switch to NAPI + kthread 1:1 model xen-netback: rename functions drivers/net/xen-netback/common.h | 150 +++++-- drivers/net/xen-netback/interface.c | 135 ++++-- drivers/net/xen-netback/netback.c | 833 +++++++++++------------------------ 3 files changed, 445 insertions(+), 673 deletions(-) -- 1.7.10.4
Wei Liu
2013-Aug-28 09:01 UTC
Re: [PATCH net-next 0/3] xen-netback: switch to NAPI + kthread 1:1 model
Dave, ping. This series has been acked by maintainer. If there is no objection we would like to see it in. Thanks Wei. On Mon, Aug 26, 2013 at 12:59:36PM +0100, Wei Liu wrote:> This series implements NAPI + kthread 1:1 model for Xen netback. > > This model > - provides better scheduling fairness among vifs > - is prerequisite for implementing multiqueue for Xen network driver > > The second patch has the real meat: > - make use of NAPI to mitigate interrupt > - kthreads are not bound to CPUs any more, so that we can take > advantage of backend scheduler and trust it to do the right thing > > Benchmark is done on a Dell T3400 workstation with 4 cores, running 4 > DomUs. Netserver runs in Dom0. DomUs do netperf to Dom0 with > following command: /root/netperf -H Dom0 -fm -l120 > > IRQs are distributed to 4 cores by hand in the new model, while in the > old model vifs are automatically distributed to 4 kthreads. > > * New model > %Cpu0 : 0.5 us, 20.3 sy, 0.0 ni, 28.9 id, 0.0 wa, 0.0 hi, 24.4 si, 25.9 st > %Cpu1 : 0.5 us, 17.8 sy, 0.0 ni, 28.8 id, 0.0 wa, 0.0 hi, 27.7 si, 25.1 st > %Cpu2 : 0.5 us, 18.8 sy, 0.0 ni, 30.7 id, 0.0 wa, 0.0 hi, 22.9 si, 27.1 st > %Cpu3 : 0.0 us, 20.1 sy, 0.0 ni, 30.4 id, 0.0 wa, 0.0 hi, 22.7 si, 26.8 st > Throughputs: 2027.89 2025.95 2018.57 2016.23 aggregated: 8088.64 > > * Old model > %Cpu0 : 0.5 us, 68.8 sy, 0.0 ni, 16.1 id, 0.5 wa, 0.0 hi, 2.8 si, 11.5 st > %Cpu1 : 0.4 us, 45.1 sy, 0.0 ni, 31.1 id, 0.4 wa, 0.0 hi, 2.1 si, 20.9 st > %Cpu2 : 0.9 us, 44.8 sy, 0.0 ni, 30.9 id, 0.0 wa, 0.0 hi, 1.3 si, 22.2 st > %Cpu3 : 0.8 us, 46.4 sy, 0.0 ni, 28.3 id, 1.3 wa, 0.0 hi, 2.1 si, 21.1 st > Throughputs: 1899.14 2280.43 1963.33 1893.47 aggregated: 8036.37 > > We can see that the impact is mainly on CPU usage. The new model moves > processing from kthread to NAPI (software interrupt). > > Wei Liu (3): > xen-netback: remove page tracking facility > xen-netback: switch to NAPI + kthread 1:1 model > xen-netback: rename functions > > drivers/net/xen-netback/common.h | 150 +++++-- > drivers/net/xen-netback/interface.c | 135 ++++-- > drivers/net/xen-netback/netback.c | 833 +++++++++++------------------------ > 3 files changed, 445 insertions(+), 673 deletions(-) > > -- > 1.7.10.4
David Miller
2013-Aug-28 14:28 UTC
Re: [PATCH net-next 0/3] xen-netback: switch to NAPI + kthread 1:1 model
From: Wei Liu <wei.liu2@citrix.com> Date: Wed, 28 Aug 2013 10:01:35 +0100> This series has been acked by maintainer. If there is no objection we > would like to see it in.It''s in the patchwork queue and will be applied when I get to it. Sending queriest like this to me makes no sense at all when you can always look at the state of your patch at patchwork. http://patchwork.ozlabs.org/project/netdev/list/
Wei Liu
2013-Aug-28 14:41 UTC
Re: [PATCH net-next 0/3] xen-netback: switch to NAPI + kthread 1:1 model
On Wed, Aug 28, 2013 at 10:28:21AM -0400, David Miller wrote:> From: Wei Liu <wei.liu2@citrix.com> > Date: Wed, 28 Aug 2013 10:01:35 +0100 > > > This series has been acked by maintainer. If there is no objection we > > would like to see it in. > > It''s in the patchwork queue and will be applied when I get to it. > > Sending queriest like this to me makes no sense at all when you can > always look at the state of your patch at patchwork. > > http://patchwork.ozlabs.org/project/netdev/list/Yes, I checked their state before querying. Just that you applied some later posted patches and I thought that you didn''t notice those acks. Sorry for the noise. Wei.
David Miller
2013-Aug-28 16:08 UTC
Re: [PATCH net-next 0/3] xen-netback: switch to NAPI + kthread 1:1 model
From: Wei Liu <wei.liu2@citrix.com> Date: Wed, 28 Aug 2013 15:41:49 +0100> Just that you applied some later posted patches and I thought that > you didn''t notice those acks.I apply patches in the order that are convenient for me and match my cognitive quota at the given point in time. This means I''ll review a lot of easy to review patches before those that require more thought and consideration. Therefore never read anything into the order in which I apply patches.