search for: af_xdp

On Mon, 31 Jul 2023 08:46:51 -0700, Jakub Kicinski <kuba at kernel.org> wrote: > On Mon, 31 Jul 2023 09:23:29 +0800 Jason Wang wrote: > > > I'd step back and ask you why do you want to use AF_XDP with virtio. > > > Instead of bifurcating one virtio instance into different queues why > > > not create a separate virtio instance? > > > > I'm not sure I get this, but do you mean a separate virtio device that > > owns AF_XDP queues only? If I understand it...

...cost of > > > creating an extra instance for DPDK or whatever else is very low. > > > > The extra instance is virtio-net? > > > > I think there is a gap. So let me give you a brief introduction of our case. > > > > Firstly, we donot use dpdk. We use the AF_XDP, because of that the AF_XDP is > > more simpler and easy to deploy for the nginx. > > > > We use the AF_XDP to speedup the UDP of the quic. By the library, the APP just > > needs some simple change. > > > > On the AliYun, the net driver is virtio-net. So we want...

...rg> wrote: > On Tue, 1 Aug 2023 10:57:30 +0800 Xuan Zhuo wrote: > > > You have this working and benchmarked or this is just and idea? > > > > This is not just an idea. I said that has been used on large scale. > > > > This is the library for the APP to use the AF_XDP. We has open it. > > https://gitee.com/anolis/libxudp > > > > This is the Alibaba version of the nginx. That has been opened, that supported > > to work with the libray to use AF_XDP. > > http://tengine.taobao.org/ > > > > I supported this on our kernel rel...

...l.org/all/20230214072704.126660-9-xuanzhuo at linux.alibaba.com/ > > > > > > Then the driver must do dma operation(map and sync) by these virtio_dma_* APIs. > > > No care the device is non-dma device or dma device. > > > > yes > > > > > Then the AF_XDP must use these virtio_dma_* APIs for virtio device. > > > > We'll worry about AF_XDP when the patch is posted. > > YES. > > We discussed it. They voted 'no'. > > http://lore.kernel.org/all/20230424082856.15c1e593 at kernel.org Hi guys, this topic is stuck...

...ki <kuba at kernel.org> wrote: > On Fri, 28 Jul 2023 14:02:33 +0800 Xuan Zhuo wrote: > > Hi guys, this topic is stuck again. How should I proceed with this work? > > > > Let me briefly summarize: > > 1. The problem with adding virtio_dma_{map, sync} api is that, for AF_XDP and > > the driver layer, we need to support these APIs. The current conclusion of > > AF_XDP is no. > > > > 2. Set dma_set_mask_and_coherent, then we can use DMA API uniformly inside > > driver. This idea seems to be inconsistent with the framework design of DMA. The...

...uanzhuo at linux.alibaba.com/ > > > > > > > > Then the driver must do dma operation(map and sync) by these virtio_dma_* APIs. > > > > No care the device is non-dma device or dma device. > > > > > > yes > > > > > > > Then the AF_XDP must use these virtio_dma_* APIs for virtio device. > > > > > > We'll worry about AF_XDP when the patch is posted. > > > > YES. > > > > We discussed it. They voted 'no'. > > > > http://lore.kernel.org/all/20230424082856.15c1e593 at k...

...her > >>> cases as well. > >>> > >>> Please review > >> I think the idea of speeding up userspace access is a good one. > >> However I think that moving all checks to start is way too aggressive. > > > > > > So did packet and AF_XDP. Anyway, sharing address space and access > > them directly is the fastest way. Performance is the major > > consideration for people to choose backend. Compare to userspace > > implementation, vhost does not have security advantages at any > > level. If vhost is still slow,...

...her > >>> cases as well. > >>> > >>> Please review > >> I think the idea of speeding up userspace access is a good one. > >> However I think that moving all checks to start is way too aggressive. > > > > > > So did packet and AF_XDP. Anyway, sharing address space and access > > them directly is the fastest way. Performance is the major > > consideration for people to choose backend. Compare to userspace > > implementation, vhost does not have security advantages at any > > level. If vhost is still slow,...

...t;kuba at kernel.org> wrote: > > On Fri, 28 Jul 2023 14:02:33 +0800 Xuan Zhuo wrote: > > Hi guys, this topic is stuck again. How should I proceed with this work? > > > > Let me briefly summarize: > > 1. The problem with adding virtio_dma_{map, sync} api is that, for AF_XDP and > > the driver layer, we need to support these APIs. The current conclusion of > > AF_XDP is no. > > > > 2. Set dma_set_mask_and_coherent, then we can use DMA API uniformly inside > > driver. This idea seems to be inconsistent with the framework design of DMA. The...

...; >> Test shows about 24% improvement on TX PPS. It should benefit other >> cases as well. >> >> Please review > I think the idea of speeding up userspace access is a good one. > However I think that moving all checks to start is way too aggressive. So did packet and AF_XDP. Anyway, sharing address space and access them directly is the fastest way. Performance is the major consideration for people to choose backend. Compare to userspace implementation, vhost does not have security advantages at any level. If vhost is still slow, people will start to develop backen...

...; >> Test shows about 24% improvement on TX PPS. It should benefit other >> cases as well. >> >> Please review > I think the idea of speeding up userspace access is a good one. > However I think that moving all checks to start is way too aggressive. So did packet and AF_XDP. Anyway, sharing address space and access them directly is the fastest way. Performance is the major consideration for people to choose backend. Compare to userspace implementation, vhost does not have security advantages at any level. If vhost is still slow, people will start to develop backen...

...ill drop these > packets. Well, then we'd better fix it, otherwise it would be even worse than not having it. For cpumap, it can be done through queuing skb in the pointer ring with some encoding/decoding. > It might not be possible to implement some features, think > of (AF_XDP) zero-copy. Yes, but can AF_XDP support header adjustment? Consider the assumption of zerocopy, I was considering maybe we need a way to tell AF_XDP of whether or not zerocopy is supported by the driver. > > 2. This can easily cause out-of-order packets. I may miss something, but it look...

...handle this case. I don't like where this is going. I don't like intermixing the native XDP and generic XDP in this way, for several reasons: 1. XDP generic is not feature complete, e.g. cpumap will drop these packets. It might not be possible to implement some features, think of (AF_XDP) zero-copy. 2. This can easily cause out-of-order packets. 3. It makes it harder to troubleshoot, when diagnosing issues around #1, we have a hard time determining what path an XDP packet took (the xdp tracepoints doesn't know). [...] > @@ -590,25 +526,14 @@ static struct sk_buff *...

...handle this case. I don't like where this is going. I don't like intermixing the native XDP and generic XDP in this way, for several reasons: 1. XDP generic is not feature complete, e.g. cpumap will drop these packets. It might not be possible to implement some features, think of (AF_XDP) zero-copy. 2. This can easily cause out-of-order packets. 3. It makes it harder to troubleshoot, when diagnosing issues around #1, we have a hard time determining what path an XDP packet took (the xdp tracepoints doesn't know). [...] > @@ -590,25 +526,14 @@ static struct sk_buff *...

...> The purpose of this patch set is to refactor these code. Reduce the difficulty > > > of subsequent maintenance. Subsequent developers will not introduce new bugs > > > because of some complex logical relationships. > > > > > > In addition, the supporting to AF_XDP that I want to submit later will also need > > > to reuse the logic of XDP, such as the processing of actions, I don't want to > > > introduce a new similar code. In this way, I can reuse these codes in the > > > future. > > > > > > Please review. &g...

...gt;>>> cases as well. >>>>> >>>>> Please review >>>> I think the idea of speeding up userspace access is a good one. >>>> However I think that moving all checks to start is way too aggressive. >>> >>> So did packet and AF_XDP. Anyway, sharing address space and access >>> them directly is the fastest way. Performance is the major >>> consideration for people to choose backend. Compare to userspace >>> implementation, vhost does not have security advantages at any >>> level. If vhost is...

...for kthread like vhost in the future, we will manually pick the one we really need (but it should have little possibility). Please notice we only access metdata through remapping not the data itself. This idea has been used for high speed userspace backend for years, e.g packet socket or recent AF_XDP. The only difference is the page was remap to from kernel to userspace. > I don't > like the idea I have to say. As a first step, why don't we switch to > unsafe_put_user/unsafe_get_user etc? Several reasons: - They only have x86 variant, it won't have any difference...

...for kthread like vhost in the future, we will manually pick the one we really need (but it should have little possibility). Please notice we only access metdata through remapping not the data itself. This idea has been used for high speed userspace backend for years, e.g packet socket or recent AF_XDP. The only difference is the page was remap to from kernel to userspace. > I don't > like the idea I have to say. As a first step, why don't we switch to > unsafe_put_user/unsafe_get_user etc? Several reasons: - They only have x86 variant, it won't have any difference...

...s page, xdp_page processing, etc. > > The purpose of this patch set is to refactor these code. Reduce the difficulty > of subsequent maintenance. Subsequent developers will not introduce new bugs > because of some complex logical relationships. > > In addition, the supporting to AF_XDP that I want to submit later will also need > to reuse the logic of XDP, such as the processing of actions, I don't want to > introduce a new similar code. In this way, I can reuse these codes in the > future. > > Please review. > > Thanks. I really want to see that code...

...rever. > > So no THP, no NUMA rebalancing, > > > This is the price of all GUP users not only vhost itself. Yes. GUP is just not a great interface for vhost to use. > What's more > important, the goal is not to be left too much behind for other backends > like DPDK or AF_XDP (all of which are using GUP). So these guys assume userspace knows what it's doing. We can't assume that. > > > userspace-controlled > > amount of memory locked up and not accounted for. > > > It's pretty easy to add this since the slow path was still kept....