search for: deflate_on_oom

...> There were apparently good reasons for moving away from OOM notifier > > callback: > > https://lkml.org/lkml/2018/7/12/314 > > https://lkml.org/lkml/2018/8/2/322 > > > > In particular the OOM notifier is worse than the shrinker because: > > The issue is that DEFLATE_ON_OOM is under-specified. > > > > > 1. It is last-resort, which means the system has already gone through > > heroics to prevent OOM. Those heroic reclaim efforts are expensive > > and impact application performance. > > That's *exactly* what "deflate on...

...> There were apparently good reasons for moving away from OOM notifier > > callback: > > https://lkml.org/lkml/2018/7/12/314 > > https://lkml.org/lkml/2018/8/2/322 > > > > In particular the OOM notifier is worse than the shrinker because: > > The issue is that DEFLATE_ON_OOM is under-specified. > > > > > 1. It is last-resort, which means the system has already gone through > > heroics to prevent OOM. Those heroic reclaim efforts are expensive > > and impact application performance. > > That's *exactly* what "deflate on...

...> notifier > > > callback: > > > https://lkml.org/lkml/2018/7/12/314 > > > https://lkml.org/lkml/2018/8/2/322 > > > > > > In particular the OOM notifier is worse than the shrinker because: > > > > The issue is that DEFLATE_ON_OOM is under-specified. > > > > > > > > 1. It is last-resort, which means the system has already gone > through > > > heroics to prevent OOM. Those heroic reclaim efforts are > expensive > > > and impact application performance. &...

...> notifier > > > callback: > > > https://lkml.org/lkml/2018/7/12/314 > > > https://lkml.org/lkml/2018/8/2/322 > > > > > > In particular the OOM notifier is worse than the shrinker because: > > > > The issue is that DEFLATE_ON_OOM is under-specified. > > > > > > > > 1. It is last-resort, which means the system has already gone > through > > > heroics to prevent OOM. Those heroic reclaim efforts are > expensive > > > and impact application performance. &...

...are some use cases where "deflate on OOM" is > desired and > > > others where "deflate on pressure" is desired. > > > This suggests adding a new feature bit "DEFLATE_ON_PRESSURE" that > > > registers the shrinker, and reverting DEFLATE_ON_OOM to use the OOM > > > notifier callback. > > > > > > This lets users configure the balloon for their use case. > > > > You want the old behavior back, so why should we introduce a new > one? Or > > am I missing something? (you did w...

...> It seems there are some use cases where "deflate on OOM" is desired and > > others where "deflate on pressure" is desired. > > This suggests adding a new feature bit "DEFLATE_ON_PRESSURE" that > > registers the shrinker, and reverting DEFLATE_ON_OOM to use the OOM > > notifier callback. > > > > This lets users configure the balloon for their use case. > > You want the old behavior back, so why should we introduce a new one? Or > am I missing something? (you did want us to revert to old handling,...

...y good reasons for moving away from OOM notifier > > callback: > > https://lkml.org/lkml/2018/7/12/314 > > https://lkml.org/lkml/2018/8/2/322 > > > > In particular the OOM notifier is worse than the shrinker because: > > The issue is that DEFLATE_ON_OOM is under-specified. > > > > >? 1. It is last-resort, which means the system has already gone through > >? ? ?heroics to prevent OOM. Those heroic reclaim efforts are expensive > >? ? ?and impact application performance. > > That's *exactly*...

...> 4k, but no way to enforce it. > >> and will deflate in certain situations. > > What does this refer to? A Linux guest will deflate the balloon (all or some pages) in the following scenarios: a) page migration b) unload virtio-balloon kernel module c) hibernate/suspension d) (DEFLATE_ON_OOM) A Linux guest will touch memory without deflating: a) During a kexec() dump d) On reboots (regular, after kexec(), system_reset) > >> Any change we >> introduce will break backwards compatibility. > > Why does this have to be the case If we suddenly enforce the existing...

...> 4k, but no way to enforce it. > >> and will deflate in certain situations. > > What does this refer to? A Linux guest will deflate the balloon (all or some pages) in the following scenarios: a) page migration b) unload virtio-balloon kernel module c) hibernate/suspension d) (DEFLATE_ON_OOM) A Linux guest will touch memory without deflating: a) During a kexec() dump d) On reboots (regular, after kexec(), system_reset) > >> Any change we >> introduce will break backwards compatibility. > > Why does this have to be the case If we suddenly enforce the existing...

There were apparently good reasons for moving away from OOM notifier callback: https://lkml.org/lkml/2018/7/12/314 https://lkml.org/lkml/2018/8/2/322 In particular the OOM notifier is worse than the shrinker because: 1. It is last-resort, which means the system has already gone through heroics to prevent OOM. Those heroic reclaim efforts are expensive and impact application performance.

There were apparently good reasons for moving away from OOM notifier callback: https://lkml.org/lkml/2018/7/12/314 https://lkml.org/lkml/2018/8/2/322 In particular the OOM notifier is worse than the shrinker because: 1. It is last-resort, which means the system has already gone through heroics to prevent OOM. Those heroic reclaim efforts are expensive and impact application performance.

...20 21:32, Tyler Sanderson wrote: > There were apparently good reasons for moving away from OOM notifier > callback: > https://lkml.org/lkml/2018/7/12/314 > https://lkml.org/lkml/2018/8/2/322 > > In particular the OOM notifier is worse than the shrinker because: The issue is that DEFLATE_ON_OOM is under-specified. > > 1. It is last-resort, which means the system has already gone through > heroics to prevent OOM. Those heroic reclaim efforts are expensive > and impact application performance. That's *exactly* what "deflate on OOM" suggests. Assume you...

...ions. > > > > What does this refer to? > > A Linux guest will deflate the balloon (all or some pages) in the > following scenarios: > a) page migration It inflates it first, doesn't it? > b) unload virtio-balloon kernel module > c) hibernate/suspension > d) (DEFLATE_ON_OOM) You need to set a flag in the balloon to allow this, right? > A Linux guest will touch memory without deflating: > a) During a kexec() dump > d) On reboots (regular, after kexec(), system_reset) > > > >> Any change we > >> introduce will break backwards compa...

...leak_balloon() > tries to take that balloon_lock and deadlocks > > Signed-off-by: Tetsuo Handa <penguin-kernel at I-love.SAKURA.ne.jp> This doesn't deflate on oom if lock is contended, and we acked DEFLATE_ON_OOM so host actually expects us to. The proper fix isn't that hard - just avoid allocations under lock. Patch posted, pls take a look. > --- > drivers/virtio/virtio_balloon.c | 16 +++++++++++----- > 1 file changed, 11 insertions(+), 5 deletions(-) > > diff --git a/drivers/virt...

...leak_balloon() > tries to take that balloon_lock and deadlocks > > Signed-off-by: Tetsuo Handa <penguin-kernel at I-love.SAKURA.ne.jp> This doesn't deflate on oom if lock is contended, and we acked DEFLATE_ON_OOM so host actually expects us to. The proper fix isn't that hard - just avoid allocations under lock. Patch posted, pls take a look. > --- > drivers/virtio/virtio_balloon.c | 16 +++++++++++----- > 1 file changed, 11 insertions(+), 5 deletions(-) > > diff --git a/drivers/virt...

On Monday, February 10, 2020 11:57 AM, Tetsuo Handa wrote: > Then, "node-A's NR_FILE_PAGES is already 0 and node-B's NR_FILE_PAGES is > not 0, but allocation request which triggered this shrinker wants to allocate > from only node-A" > would be confused by this change, for the pagecache pages for allocating > thread's interested node are already depleted but

On Monday, February 10, 2020 11:57 AM, Tetsuo Handa wrote: > Then, "node-A's NR_FILE_PAGES is already 0 and node-B's NR_FILE_PAGES is > not 0, but allocation request which triggered this shrinker wants to allocate > from only node-A" > would be confused by this change, for the pagecache pages for allocating > thread's interested node are already depleted but

...locations are stopped even if they could succeed > > by deflating the balloon. > > Please note that some people use the balloon for actual memory unplug - > so initiating to deflate the balloon under any circumstances is > undesired. It's different with "VIRTIO_BALLOON_F_DEFLATE_ON_OOM" being > set - however that is barely the case (at least in the setups I know :) ). > > So yes, free page reporting is a different thing, because it really is > used to "hint" and not to "agree to unplug" in any scenario. > > -- > > Thanks, >...

...locations are stopped even if they could succeed > > by deflating the balloon. > > Please note that some people use the balloon for actual memory unplug - > so initiating to deflate the balloon under any circumstances is > undesired. It's different with "VIRTIO_BALLOON_F_DEFLATE_ON_OOM" being > set - however that is barely the case (at least in the setups I know :) ). > > So yes, free page reporting is a different thing, because it really is > used to "hint" and not to "agree to unplug" in any scenario. > > -- > > Thanks, >...

...rantees. Nobody can hinder the guest from > deflating/reusing inflated memory. Reusing without deflate is forbidden with TELL_HOST, right? > In addition, it might make perfect > sense to have both, virtio-balloon and virtio-mem at the same time, > especially looking at the DEFLATE_ON_OOM or STATS features of > virtio-balloon. While virtio-mem is all about guarantees, virtio- > balloon is about cooperation. Thanks, and I intend to look more into this next week. -- MST