Linux Virtualization - Jan 2014 - [PATCH net-next v2 1/4] net: allow > 0 order atomic page alloc in skb_page_frag_refill

On Wed, 2014-01-08 at 21:18 +0200, Michael S. Tsirkin
wrote:
> On Wed, Jan 08, 2014 at 10:26:03AM -0800, Eric Dumazet wrote:
> > On Wed, 2014-01-08 at 20:08 +0200, Michael S. Tsirkin wrote:
> > 
> > > Eric said we also need a patch to add __GFP_NORETRY, right?
> > > Probably before this one in series.
> > 
> > Nope, this __GFP_NORETRY has nothing to do with this.
> > 
> > I am not yet convinced we want it.
> > 
> > This needs mm guys advice, as its a tradeoff for mm layer more than
> > networking...
> 
> Well maybe Cc linux-mm then?
Well, I do not care of people mlocking the memory and complaining that
compaction does not work.

If these people care, they should contact mm guys, eventually.

Really this is an issue that has nothing to do with this patch set.

On Wed, Jan 8, 2014 at 2:46 PM, Eric Dumazet <eric.dumazet at gmail.com>
wrote:
> On Wed, 2014-01-08 at 21:18 +0200, Michael S. Tsirkin wrote:
>> On Wed, Jan 08, 2014 at 10:26:03AM -0800, Eric Dumazet wrote:
>> > On Wed, 2014-01-08 at 20:08 +0200, Michael S. Tsirkin wrote:
>> >
>> > > Eric said we also need a patch to add __GFP_NORETRY, right?
>> > > Probably before this one in series.
>> >
>> > Nope, this __GFP_NORETRY has nothing to do with this.
>> >
>> > I am not yet convinced we want it.
>> >
>> > This needs mm guys advice, as its a tradeoff for mm layer more
than
>> > networking...
>>
>> Well maybe Cc linux-mm then?
>
> Well, I do not care of people mlocking the memory and complaining that
> compaction does not work.
>
> If these people care, they should contact mm guys, eventually.
>
> Really this is an issue that has nothing to do with this patch set.
>
Actually I have more data on this:

1. __GFP_NORETRY really does help and should go into stable tree.

2. You may want to consider GFP_NOKSWAPD, because even in the
GFP_ATOMIC case you are waking up kswapd to do reclaims on a
continuous basis even when you don't enter direct reclaim.

3. mlocking memory had very little to do with it, that was a
red-herring. I tested out the problem scenario with no mlocks. You
simply need memory pressure from page_cache, and mm ends up constantly
reclaiming and trying to keep another 1-2GB free on our systems (8GB
phys ~4GB left for kernel, ~3GB optimally used for page_cache).

4. I think perhaps using a kmem_cache allocation for this buffer is
the right way to make this work. I am experimenting with a patch to do
this.

-Debabrata

-Debabrata

On Wed, 2014-01-08 at 16:54 -0500, Debabrata Banerjee wrote:
> Actually I have more data on this:
> 
Could you please stop polluting this thread ?
> 1. __GFP_NORETRY really does help and should go into stable tree.
> 
Not at all. You are free to patch your kernel if you want.

It helps you workload, and breaks all others.

If compaction is never triggered, we'll never be able to get high order
pages, and performance goes back to what we had 2 years ago.

That discussion does not belong to this thread, again.
> 2. You may want to consider GFP_NOKSWAPD, because even in the
> GFP_ATOMIC case you are waking up kswapd to do reclaims on a
> continuous basis even when you don't enter direct reclaim.
> 
> 3. mlocking memory had very little to do with it, that was a
> red-herring. I tested out the problem scenario with no mlocks. You
> simply need memory pressure from page_cache, and mm ends up constantly
> reclaiming and trying to keep another 1-2GB free on our systems (8GB
> phys ~4GB left for kernel, ~3GB optimally used for page_cache).
> 
> 4. I think perhaps using a kmem_cache allocation for this buffer is
> the right way to make this work. I am experimenting with a patch to do
> this.
Seriously... I think you missed whole point of having frag allocation on
pages, not kmem_cache.