Jason Wang
2021-Jan-06 03:54 UTC
[PATCH netdev 0/5] virtio-net support xdp socket zero copy xmit
On 2021/1/6 ??10:55, Xuan Zhuo wrote:> On Wed, 6 Jan 2021 10:46:43 +0800, Jason Wang <jasowang at redhat.com> wrote: >> On 2021/1/5 ??8:42, Xuan Zhuo wrote: >>> On Tue, 5 Jan 2021 17:32:19 +0800, Jason Wang <jasowang at redhat.com> wrote: >>>> On 2021/1/5 ??5:11, Xuan Zhuo wrote: >>>>> The first patch made some adjustments to xsk. >>>> Thanks a lot for the work. It's rather interesting. >>>> >>>> >>>>> The second patch itself can be used as an independent patch to solve the problem >>>>> that XDP may fail to load when the number of queues is insufficient. >>>> It would be better to send this as a separated patch. Several people >>>> asked for this before. >>>> >>>> >>>>> The third to last patch implements support for xsk in virtio-net. >>>>> >>>>> A practical problem with virtio is that tx interrupts are not very reliable. >>>>> There will always be some missing or delayed tx interrupts. So I specially added >>>>> a point timer to solve this problem. Of course, considering performance issues, >>>>> The timer only triggers when the ring of the network card is full. >>>> This is sub-optimal. We need figure out the root cause. We don't meet >>>> such issue before. >>>> >>>> Several questions: >>>> >>>> - is tx interrupt enabled? >>>> - can you still see the issue if you disable event index? >>>> - what's backend did you use? qemu or vhost(user)? >>> Sorry, it may just be a problem with the backend I used here. I just tested the >>> latest qemu and it did not have this problem. I think I should delete the >>> timer-related code? >> >> Yes, please. >> >> >>>>> Regarding the issue of virtio-net supporting xsk's zero copy rx, I am also >>>>> developing it, but I found that the modification may be relatively large, so I >>>>> consider this patch set to be separated from the code related to xsk zero copy >>>>> rx. >>>> That's fine, but a question here. >>>> >>>> How is the multieuque being handled here. I'm asking since there's no >>>> programmable filters/directors support in virtio spec now. >>>> >>>> Thanks >>> I don't really understand what you mean. In the case of multiple queues, >>> there is no problem. >> >> So consider we bind xsk to queue 4, how can you make sure the traffic to >> be directed queue 4? One possible solution is to use filters as what >> suggested in af_xdp.rst: >> >> ????? ethtool -N p3p2 rx-flow-hash udp4 fn >> ????? ethtool -N p3p2 flow-type udp4 src-port 4242 dst-port 4242 \ >> ????????? action 16 >> ... >> >> But virtio-net doesn't have any filters that could be programmed from >> the driver. >> >> Anything I missed here? >> >> Thanks > I understand what you mean, this problem does exist, and I encountered it when I > tested qemu. > > First of all, this is that the problem only affects recv. This patch is for > xmit. Of course, our normal business must also have recv scenarios. > > My solution in developing the upper-level application is to bond all the queues > to ensure that we can receive the packets we want.I'm not sure I get you here. Note that. one advantage of AF_XDP is that is allows XSK to be bound to a specific queue and the rest could still be used by kernel.> And I think in the > implementation of the use, even if the network card supports filters, we should > also bond all the queues, because we don't know which queue the traffic we care > about will arrive from.With the help of filters the card can select a specific queue based on hash or n-tuple so it should work?> > Regarding the problem of virtio-net, I think our core question is whether we > need to deal with this problem in the driver of virtio-net, I personally think > that we should add the virtio specification to define this scenario.Yes, so do you want to do that? It would make virtio-net more user friendly to AF_XDP. (Or if you wish I can post patch to extend the spec).> > When I tested it, I found that some cloud vendors' implementations guarantee > this queue selection algorithm.Right, though spec suggest a automatic steering algorithm but it's not mandatory. Vendor can implement their own. But hash or ntuple filter should be still useful. Thanks> > Thanks!! > >> >>>>> Xuan Zhuo (5): >>>>> xsk: support get page for drv >>>>> virtio-net: support XDP_TX when not more queues >>>>> virtio-net, xsk: distinguish XDP_TX and XSK XMIT ctx >>>>> xsk, virtio-net: prepare for support xsk >>>>> virtio-net, xsk: virtio-net support xsk zero copy tx >>>>> >>>>> drivers/net/virtio_net.c | 643 +++++++++++++++++++++++++++++++++++++++----- >>>>> include/linux/netdevice.h | 1 + >>>>> include/net/xdp_sock_drv.h | 10 + >>>>> include/net/xsk_buff_pool.h | 1 + >>>>> net/xdp/xsk_buff_pool.c | 10 +- >>>>> 5 files changed, 597 insertions(+), 68 deletions(-) >>>>> >>>>> -- >>>>> 1.8.3.1 >>>>>