search for: build_skb

For small packets data copy was observed to take up about 15% CPU time. Switch to build_skb and avoid the copy when using mergeable rx buffers. As a bonus, medium-size skbs that fit in a page will be completely linear. Of course, we now need to lower the lower bound on packet size, to make sure a sane number of skbs fits in rx socket buffer. By how much? I don't know yet. It might...

For small packets data copy was observed to take up about 15% CPU time. Switch to build_skb and avoid the copy when using mergeable rx buffers. As a bonus, medium-size skbs that fit in a page will be completely linear. Of course, we now need to lower the lower bound on packet size, to make sure a sane number of skbs fits in rx socket buffer. By how much? I don't know yet. It might...

...1 +0800 Jason wrote: > This patch implement build XDP buffers in vhost_net. The idea is do > userspace copy in vhost_net and build XDP buff based on the > page. Vhost_net can then submit one or an array of XDP buffs to > underlayer socket (e.g TUN). TUN can choose to do XDP or call > build_skb() to build skb. To support build skb, vnet header were also > stored into the header of the XDP buff. > > This userspace copy and XDP buffs building is key to achieve XDP > batching in TUN, since TUN does not need to care about userspace copy > and then can disable premmption for se...

...? __fib_lookup+0x6a/0xb0 > > > [280155.348626] ip_route_output_key_hash_rcu+0x650/0xa30 > > > [280155.348631] ? __alloc_skb+0x9b/0x1d0 > > > [280155.348634] inet_rtm_getroute+0x3f7/0xb80 > > > > inet_rtm_getroute builds a new packet with inet_rtm_getroute_build_skb > > here without dev or sk. > > Ack > > > > > > Problem is the synthesized skb for output route resolution does not have > > > skb->dev or skb->sk set. When a multipath route is hit and > > > net.ipv4.fib_multipath_hash_policy is set the flow di...

.../net/tun.c | 9 ++++++--- > 1 file changed, 6 insertions(+), 3 deletions(-) > > diff --git a/drivers/net/tun.c b/drivers/net/tun.c > index f8cdcfa392c3..389aa0727cc6 100644 > --- a/drivers/net/tun.c > +++ b/drivers/net/tun.c > @@ -1675,10 +1675,12 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun, > * of xdp_prog above, this should be rare and for simplicity > * we do XDP on skb in case the headroom is not enough. > */ > - if (hdr->gso_type || !xdp_prog) > + if (hdr->gso_type || !xdp_prog) { > *skb_xdp = 1; > - else > - *skb_...

This patch implement build XDP buffers in vhost_net. The idea is do userspace copy in vhost_net and build XDP buff based on the page. Vhost_net can then submit one or an array of XDP buffs to underlayer socket (e.g TUN). TUN can choose to do XDP or call build_skb() to build skb. To support build skb, vnet header were also stored into the header of the XDP buff. This userspace copy and XDP buffs building is key to achieve XDP batching in TUN, since TUN does not need to care about userspace copy and then can disable premmption for several XDP buffs to achiev...

...ative redirect, and push the responsibility to > cpumap for detecting this too-small frame "missing tailroom" (and avoid > crashing...). (If we really want to support this, cpumap could fallback > to dev_alloc_skb, and handle it gracefully). > Right but it will be slower than build_skb(). Thanks

...ative redirect, and push the responsibility to > cpumap for detecting this too-small frame "missing tailroom" (and avoid > crashing...). (If we really want to support this, cpumap could fallback > to dev_alloc_skb, and handle it gracefully). > Right but it will be slower than build_skb(). Thanks

...65535/ 2/ 0%/ +8% 65535/ 4/ 0%/ +1% 65535/ 8/ 0%/ +3% Please review. Thanks Jason Wang (11): net: sock: introduce SOCK_XDP tuntap: switch to use XDP_PACKET_HEADROOM tuntap: enable bh early during processing XDP tuntap: simplify error handling in tun_build_skb() tuntap: tweak on the path of non-xdp case in tun_build_skb() tuntap: split out XDP logic tuntap: move XDP flushing out of tun_do_xdp() tun: switch to new type of msg_control tuntap: accept an array of XDP buffs through sendmsg() tap: accept an array of XDP buffs through sendmsg() vh...

...introduce helper to initialize tx iov iter vhost_net: introduce vhost_exceeds_weight() vhost_net: introduce vhost_has_more_pkts() vhost_net: split out datacopy logic vhost_net: batch update used ring for datacopy TX tuntap: enable premmption early tuntap: simplify error handling in tun_build_skb() tuntap: tweak on the path of non-xdp case in tun_build_skb() tuntap: split out XDP logic vhost_net: build xdp buff vhost_net: passing raw xdp buff to tun vhost_net: batch submitting XDP buffers to underlayer sockets drivers/net/tun.c | 226 +++++++++++++++++++++++++++---------- d...

...0%/ +2% 16384/ 4/ 0%/ -3% 16384/ 8/ 0%/ +4% 65535/ 1/ 0%/ +10% 65535/ 2/ 0%/ +8% 65535/ 4/ 0%/ +1% 65535/ 8/ 0%/ +3% Please review. Thanks Changes from V1: - don't hold page refcnt for XDP_DROP - release page if build_skb() fails - introduce a helper to build skb for the path of no batching mode - rename tun_do_xdp() as tun_xdp_act() and use negative for reporting errors instead of a pointer to integer - introduce a num field in tun_msg_ctl - introduce a new structure for storing metadata in the head of XDP pack...

...0%/ +2% 16384/ 4/ 0%/ -3% 16384/ 8/ 0%/ +4% 65535/ 1/ 0%/ +10% 65535/ 2/ 0%/ +8% 65535/ 4/ 0%/ +1% 65535/ 8/ 0%/ +3% Please review. Thanks Changes from V1: - don't hold page refcnt for XDP_DROP - release page if build_skb() fails - introduce a helper to build skb for the path of no batching mode - rename tun_do_xdp() as tun_xdp_act() and use negative for reporting errors instead of a pointer to integer - introduce a num field in tun_msg_ctl - introduce a new structure for storing metadata in the head of XDP pack...

...+ sizeof(int); > + int buflen = *(int *)xdp->data_hard_start; > + int vnet_hdr_len = 0; > + struct tap_dev *tap; > + struct sk_buff *skb; > + int err, depth; > + > + if (q->flags & IFF_VNET_HDR) > + vnet_hdr_len = READ_ONCE(q->vnet_hdr_sz); > + > + skb = build_skb(xdp->data_hard_start, buflen); > + if (!skb) { > + err = -ENOMEM; > + goto err; > + } So fundamentally why is it called XDP? We just build and skb, don't we? > + > + skb_reserve(skb, xdp->data - xdp->data_hard_start); > + skb_put(skb, xdp->data_end - xdp-&gt...

...ith a "data_hard_end" pointer. Right, and then cpumap can warn and drop packets with insufficient tailroom. But it should be a patch on top of this I think. Thanks > > Now cpumap is more safe... instead of crashing, it can now know/see when > it is safe to create an SKB using build_skb (could fallback to > dev_alloc_skb). >

...ith a "data_hard_end" pointer. Right, and then cpumap can warn and drop packets with insufficient tailroom. But it should be a patch on top of this I think. Thanks > > Now cpumap is more safe... instead of crashing, it can now know/see when > it is safe to create an SKB using build_skb (could fallback to > dev_alloc_skb). >

...+++--------------------- > 1 file changed, 16 insertions(+), 21 deletions(-) > > diff --git a/drivers/net/tun.c b/drivers/net/tun.c > index 372caf7d67d9..f8cdcfa392c3 100644 > --- a/drivers/net/tun.c > +++ b/drivers/net/tun.c > @@ -1642,7 +1642,7 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun, > int len, int *skb_xdp) > { > struct page_frag *alloc_frag = &current->task_frag; > - struct sk_buff *skb; > + struct sk_buff *skb = NULL; > struct bpf_prog *xdp_prog; > int buflen = SKB_DATA_ALIGN(sizeof(struct skb_shared_info))...

...+++--------------------- > 1 file changed, 16 insertions(+), 21 deletions(-) > > diff --git a/drivers/net/tun.c b/drivers/net/tun.c > index 372caf7d67d9..f8cdcfa392c3 100644 > --- a/drivers/net/tun.c > +++ b/drivers/net/tun.c > @@ -1642,7 +1642,7 @@ static struct sk_buff *tun_build_skb(struct tun_struct *tun, > int len, int *skb_xdp) > { > struct page_frag *alloc_frag = &current->task_frag; > - struct sk_buff *skb; > + struct sk_buff *skb = NULL; > struct bpf_prog *xdp_prog; > int buflen = SKB_DATA_ALIGN(sizeof(struct skb_shared_info))...

...+++++++++++------------------- > 1 file changed, 51 insertions(+), 33 deletions(-) > > diff --git a/drivers/net/tun.c b/drivers/net/tun.c > index 389aa0727cc6..21b125020b3b 100644 > --- a/drivers/net/tun.c > +++ b/drivers/net/tun.c > @@ -1635,6 +1635,44 @@ static bool tun_can_build_skb(struct tun_struct *tun, struct tun_file *tfile, > return true; > } > > +static u32 tun_do_xdp(struct tun_struct *tun, > + struct tun_file *tfile, > + struct bpf_prog *xdp_prog, > + struct xdp_buff *xdp, > + int *err) > +{ > + u32 act =...

...+++++++++++------------------- > 1 file changed, 51 insertions(+), 33 deletions(-) > > diff --git a/drivers/net/tun.c b/drivers/net/tun.c > index 389aa0727cc6..21b125020b3b 100644 > --- a/drivers/net/tun.c > +++ b/drivers/net/tun.c > @@ -1635,6 +1635,44 @@ static bool tun_can_build_skb(struct tun_struct *tun, struct tun_file *tfile, > return true; > } > > +static u32 tun_do_xdp(struct tun_struct *tun, > + struct tun_file *tfile, > + struct bpf_prog *xdp_prog, > + struct xdp_buff *xdp, > + int *err) > +{ > + u32 act =...

...;data_hard_start + PAGE_SIZE, but ixgbe+virtio_net broke that assumption). It should actually be fairly easy to implement: * Simply extend xdp_buff with a "data_hard_end" pointer. Now cpumap is more safe... instead of crashing, it can now know/see when it is safe to create an SKB using build_skb (could fallback to dev_alloc_skb). -- Best regards, Jesper Dangaard Brouer MSc.CS, Principal Kernel Engineer at Red Hat LinkedIn: http://www.linkedin.com/in/brouer