search for: truesize

...--- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -270,6 +270,23 @@ static void skb_xmit_done(struct virtqueue *vq) > netif_wake_subqueue(vi->dev, vq2txq(vq)); > } > > +#define MRG_CTX_HEADER_SHIFT 22 > +static void *mergeable_len_to_ctx(unsigned int truesize, > + unsigned int headroom) > +{ > + return (void *)(unsigned long)((headroom << MRG_CTX_HEADER_SHIFT) | truesize); > +} > + > +static unsigned int mergeable_ctx_to_headroom(void *mrg_ctx) > +{ > + return (unsigned long)mrg_ctx >> MRG_CTX_HEADER_SHIFT; >...

...--- a/drivers/net/virtio_net.c > +++ b/drivers/net/virtio_net.c > @@ -270,6 +270,23 @@ static void skb_xmit_done(struct virtqueue *vq) > netif_wake_subqueue(vi->dev, vq2txq(vq)); > } > > +#define MRG_CTX_HEADER_SHIFT 22 > +static void *mergeable_len_to_ctx(unsigned int truesize, > + unsigned int headroom) > +{ > + return (void *)(unsigned long)((headroom << MRG_CTX_HEADER_SHIFT) | truesize); > +} > + > +static unsigned int mergeable_ctx_to_headroom(void *mrg_ctx) > +{ > + return (unsigned long)mrg_ctx >> MRG_CTX_HEADER_SHIFT; >...

...ingle-stream regression for benchmarks with large > average packet size. There is no single optimal buffer size for all workloads. > For workloads with packet size <= MTU bytes, MTU + virtio-net header-sized > buffers are preferred as larger buffers reduce the TCP window due to SKB > truesize. However, single-stream workloads with large average packet sizes > have higher throughput if larger (e.g., PAGE_SIZE) buffers are used. > > This commit auto-tunes the mergeable receiver buffer packet size by choosing > the packet buffer size based on an EWMA of the recent packet sizes...

...ingle-stream regression for benchmarks with large > average packet size. There is no single optimal buffer size for all workloads. > For workloads with packet size <= MTU bytes, MTU + virtio-net header-sized > buffers are preferred as larger buffers reduce the TCP window due to SKB > truesize. However, single-stream workloads with large average packet sizes > have higher throughput if larger (e.g., PAGE_SIZE) buffers are used. > > This commit auto-tunes the mergeable receiver buffer packet size by choosing > the packet buffer size based on an EWMA of the recent packet sizes...

skb_page_frag_refill currently permits only order-0 page allocs unless GFP_WAIT is used. Change skb_page_frag_refill to attempt higher-order page allocations whether or not GFP_WAIT is used. If memory cannot be allocated, the allocator will fall back to successively smaller page allocs (down to order-0 page allocs). This change brings skb_page_frag_refill in line with the existing page allocation

skb_page_frag_refill currently permits only order-0 page allocs unless GFP_WAIT is used. Change skb_page_frag_refill to attempt higher-order page allocations whether or not GFP_WAIT is used. If memory cannot be allocated, the allocator will fall back to successively smaller page allocs (down to order-0 page allocs). This change brings skb_page_frag_refill in line with the existing page allocation

...ngle-stream regression for benchmarks with large > average packet size. There is no single optimal buffer size for all > workloads. For workloads with packet size <= MTU bytes, MTU + virtio-net > header-sized buffers are preferred as larger buffers reduce the TCP window > due to SKB truesize. However, single-stream workloads with large average > packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers > are used. > > This commit auto-tunes the mergeable receiver buffer packet size by > choosing the packet buffer size based on an EWMA of the recent packet...

...ngle-stream regression for benchmarks with large > average packet size. There is no single optimal buffer size for all > workloads. For workloads with packet size <= MTU bytes, MTU + virtio-net > header-sized buffers are preferred as larger buffers reduce the TCP window > due to SKB truesize. However, single-stream workloads with large average > packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers > are used. > > This commit auto-tunes the mergeable receiver buffer packet size by > choosing the packet buffer size based on an EWMA of the recent packet...

...to MTU-size. However, the merge buffer size does not take into account the size of the virtio-net header. Consequently, packets that are MTU-size will take two buffers intead of one (to store the virtio-net header), substantially decreasing the throughput of MTU-size traffic due to TCP window / SKB truesize effects. This commit changes the mergeable buffer size to include the virtio-net header. The buffer size is cacheline-aligned because skb_page_frag_refill will not automatically align the requested size. Benchmarks taken from an average of 5 netperf 30-second TCP_STREAM runs between two QEMU VMs...

...to MTU-size. However, the merge buffer size does not take into account the size of the virtio-net header. Consequently, packets that are MTU-size will take two buffers intead of one (to store the virtio-net header), substantially decreasing the throughput of MTU-size traffic due to TCP window / SKB truesize effects. This commit changes the mergeable buffer size to include the virtio-net header. The buffer size is cacheline-aligned because skb_page_frag_refill will not automatically align the requested size. Benchmarks taken from an average of 5 netperf 30-second TCP_STREAM runs between two QEMU VMs...

...ize, introducing a single-stream regression for benchmarks with large average packet size. There is no single optimal buffer size for all workloads. For workloads with packet size <= MTU bytes, MTU + virtio-net header-sized buffers are preferred as larger buffers reduce the TCP window due to SKB truesize. However, single-stream workloads with large average packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers are used. This commit auto-tunes the mergeable receiver buffer packet size by choosing the packet buffer size based on an EWMA of the recent packet sizes for the receive que...

...et.c index 1f8c15c..8fae9a8 100644 --- a/drivers/net/virtio_net.c +++ b/drivers/net/virtio_net.c @@ -270,6 +270,23 @@ static void skb_xmit_done(struct virtqueue *vq) netif_wake_subqueue(vi->dev, vq2txq(vq)); } +#define MRG_CTX_HEADER_SHIFT 22 +static void *mergeable_len_to_ctx(unsigned int truesize, + unsigned int headroom) +{ + return (void *)(unsigned long)((headroom << MRG_CTX_HEADER_SHIFT) | truesize); +} + +static unsigned int mergeable_ctx_to_headroom(void *mrg_ctx) +{ + return (unsigned long)mrg_ctx >> MRG_CTX_HEADER_SHIFT; +} + +static unsigned int mergeable_ctx_to_tr...

...ze, introducing a single-stream regression for benchmarks with large average packet size. There is no single optimal buffer size for all workloads. For workloads with packet size <= MTU bytes, MTU + virtio-net header-sized buffers are preferred as larger buffers reduce the TCP window due to SKB truesize. However, single-stream workloads with large average packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers are used. This commit auto-tunes the mergeable receiver buffer packet size by choosing the packet buffer size based on an EWMA of the recent packet sizes for the receive que...

...stats *stats) { - struct virtio_net_hdr_mrg_rxbuf *hdr = buf; - int num_buf = virtio16_to_cpu(vi->vdev, hdr->num_buffers); - struct page *page = virt_to_head_page(buf); - int offset = buf - page_address(page); - struct sk_buff *head_skb, *curr_skb; - struct bpf_prog *xdp_prog; unsigned int truesize = mergeable_ctx_to_truesize(ctx); unsigned int headroom = mergeable_ctx_to_headroom(ctx); unsigned int tailroom = headroom ? sizeof(struct skb_shared_info) : 0; unsigned int room = SKB_DATA_ALIGN(headroom + tailroom); - unsigned int frame_sz; - int err; + struct virtio_net_hdr_mrg_rxbuf *hdr;...

...linux/skbuff.h index 2c15497..e34652b 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1372,6 +1372,9 @@ static inline void skb_fill_page_desc(struct sk_buff *skb, int i, void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off, int size, unsigned int truesize); +void skb_coalesce_rx_frag(struct sk_buff *skb, int i, int off, int size, + unsigned int truesize); + #define SKB_PAGE_ASSERT(skb) BUG_ON(skb_shinfo(skb)->nr_frags) #define SKB_FRAG_ASSERT(skb) BUG_ON(skb_has_frag_list(skb)) #define SKB_LINEAR_ASSERT(skb) BUG_ON(skb_is_nonlinear(sk...

...linux/skbuff.h index 2c15497..e34652b 100644 --- a/include/linux/skbuff.h +++ b/include/linux/skbuff.h @@ -1372,6 +1372,9 @@ static inline void skb_fill_page_desc(struct sk_buff *skb, int i, void skb_add_rx_frag(struct sk_buff *skb, int i, struct page *page, int off, int size, unsigned int truesize); +void skb_coalesce_rx_frag(struct sk_buff *skb, int i, int off, int size, + unsigned int truesize); + #define SKB_PAGE_ASSERT(skb) BUG_ON(skb_shinfo(skb)->nr_frags) #define SKB_FRAG_ASSERT(skb) BUG_ON(skb_has_frag_list(skb)) #define SKB_LINEAR_ASSERT(skb) BUG_ON(skb_is_nonlinear(sk...

...ion for benchmarks with large > > average packet size. There is no single optimal buffer size for all workloads. > > For workloads with packet size <= MTU bytes, MTU + virtio-net header-sized > > buffers are preferred as larger buffers reduce the TCP window due to SKB > > truesize. However, single-stream workloads with large average packet sizes > > have higher throughput if larger (e.g., PAGE_SIZE) buffers are used. > > > > This commit auto-tunes the mergeable receiver buffer packet size by choosing > > the packet buffer size based on an EWMA of the...

...ngle-stream regression for benchmarks with large > average packet size. There is no single optimal buffer size for all > workloads. For workloads with packet size <= MTU bytes, MTU + virtio-net > header-sized buffers are preferred as larger buffers reduce the TCP window > due to SKB truesize. However, single-stream workloads with large average > packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers > are used. > > This commit auto-tunes the mergeable receiver buffer packet size by > choosing the packet buffer size based on an EWMA of the recent packet &...

...ngle-stream regression for benchmarks with large > average packet size. There is no single optimal buffer size for all > workloads. For workloads with packet size <= MTU bytes, MTU + virtio-net > header-sized buffers are preferred as larger buffers reduce the TCP window > due to SKB truesize. However, single-stream workloads with large average > packet sizes have higher throughput if larger (e.g., PAGE_SIZE) buffers > are used. > > This commit auto-tunes the mergeable receiver buffer packet size by > choosing the packet buffer size based on an EWMA of the recent packet &...

On Wed, May 06, 2020 at 02:16:33PM +0800, Jason Wang wrote: > We should not exclude headroom and tailroom when XDP is set. So this > patch fixes this by initializing the truesize from PAGE_SIZE when XDP > is set. > > Cc: Jesper Dangaard Brouer <brouer at redhat.com> > Signed-off-by: Jason Wang <jasowang at redhat.com> Seems too aggressive, we do not use up the whole page for the size. > --- > drivers/net/virtio_net.c | 6 ++++-- > 1 fi...