On Tue, Jan 19, 2021 at 05:45:09PM +0800, Xuan Zhuo
wrote:> v2:
> 1. add priv_flags IFF_TX_SKB_NO_LINEAR instead of netdev_feature
> 2. split the patch to three:
> a. add priv_flags IFF_TX_SKB_NO_LINEAR
> b. virtio net add priv_flags IFF_TX_SKB_NO_LINEAR
> c. When there is support this flag, construct skb without linear
space
> 3. use ERR_PTR() and PTR_ERR() to handle the err
>
>
> v1 message log:
> ---------------
>
> This patch is used to construct skb based on page to save memory copy
> overhead.
>
> This has one problem:
>
> We construct the skb by fill the data page as a frag into the skb. In
> this way, the linear space is empty, and the header information is also
> in the frag, not in the linear space, which is not allowed for some
> network cards. For example, Mellanox Technologies MT27710 Family
> [ConnectX-4 Lx] will get the following error message:
>
> mlx5_core 0000:3b:00.1 eth1: Error cqe on cqn 0x817, ci 0x8, qn 0x1dbb,
opcode 0xd, syndrome 0x1, vendor syndrome 0x68
> 00000000: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00000020: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00000030: 00 00 00 00 60 10 68 01 0a 00 1d bb 00 0f 9f d2
> WQE DUMP: WQ size 1024 WQ cur size 0, WQE index 0xf, len: 64
> 00000000: 00 00 0f 0a 00 1d bb 03 00 00 00 08 00 00 00 00
> 00000010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> 00000020: 00 00 00 2b 00 08 00 00 00 00 00 05 9e e3 08 00
> 00000030: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
> mlx5_core 0000:3b:00.1 eth1: ERR CQE on SQ: 0x1dbb
>
> I also tried to use build_skb to construct skb, but because of the
> existence of skb_shinfo, it must be behind the linear space, so this
> method is not working. We can't put skb_shinfo on desc->addr, it
will be
> exposed to users, this is not safe.
>
> Finally, I added a feature NETIF_F_SKB_NO_LINEAR to identify whether the
> network card supports the header information of the packet in the frag
> and not in the linear space.
>
> ---------------- Performance Testing ------------
>
> The test environment is Aliyun ECS server.
> Test cmd:
> ```
> xdpsock -i eth0 -t -S -s <msg size>
> ```
>
> Test result data:
>
> size 64 512 1024 1500
> copy 1916747 1775988 1600203 1440054
> page 1974058 1953655 1945463 1904478
> percent 3.0% 10.0% 21.58% 32.3%
Just making sure, are these test results with v2?
>
> Xuan Zhuo (3):
> net: add priv_flags for allow tx skb without linear
> virtio-net: support IFF_TX_SKB_NO_LINEAR
> xsk: build skb by page
>
> drivers/net/virtio_net.c | 3 +-
> include/linux/netdevice.h | 3 ++
> net/xdp/xsk.c | 112
++++++++++++++++++++++++++++++++++++++--------
> 3 files changed, 99 insertions(+), 19 deletions(-)
>
> --
> 1.8.3.1