search for: drop_slab

On Thursday, February 6, 2020 5:32 PM, David Hildenbrand wrote: > > If the page cache is empty, a drop_slab() will deflate the whole balloon if I > am not wrong. > > Especially, a echo 3 > /proc/sys/vm/drop_caches > > will first drop the page cache and then drop_slab() Then that's the problem of "echo 3 > /proc/sys/vm/drop_cache" itself. It invokes other shrinkers as...

On Thursday, February 6, 2020 5:10 PM, David Hildenbrand wrote: > so dropping caches (echo 3 > /proc/sys/vm/drop_caches) will no longer > deflate the balloon when conservative_shrinker=true? > Should be. Need Tyler's help to test it. Best, Wei

On Thursday, February 6, 2020 5:10 PM, David Hildenbrand wrote: > so dropping caches (echo 3 > /proc/sys/vm/drop_caches) will no longer > deflate the balloon when conservative_shrinker=true? > Should be. Need Tyler's help to test it. Best, Wei

...eflate on OOM" makes it pretty clear when deflation should > happen - after other approaches to reclaim memory failed, not while > reclaiming. This allows to minimize the footprint of a guest - memory > will only be taken out of the balloon when really needed. > > Especially, a drop_slab() will result in the whole balloon getting > deflated - undesired. Could you explain why some more? drop_caches shouldn't be really used in any production workloads and if somebody really wants all the cache to be dropped then why is balloon any different? -- Michal Hocko SUSE Labs

...eflate on OOM" makes it pretty clear when deflation should > happen - after other approaches to reclaim memory failed, not while > reclaiming. This allows to minimize the footprint of a guest - memory > will only be taken out of the balloon when really needed. > > Especially, a drop_slab() will result in the whole balloon getting > deflated - undesired. Could you explain why some more? drop_caches shouldn't be really used in any production workloads and if somebody really wants all the cache to be dropped then why is balloon any different? -- Michal Hocko SUSE Labs

...y, February 6, 2020 5:10 PM, David Hildenbrand wrote: >> so dropping caches (echo 3 > /proc/sys/vm/drop_caches) will no longer >> deflate the balloon when conservative_shrinker=true? >> > > Should be. Need Tyler's help to test it. > If the page cache is empty, a drop_slab() will deflate the whole balloon if I am not wrong. Especially, a echo 3 > /proc/sys/vm/drop_caches will first drop the page cache and then drop_slab() While I like the general idea, it looks more like a hack to me, to try to teach the shrinker something it was not built for/does not support...

...io > balloon for the VMware balloon. If the verdict is that this is damaging and > the OOM notifier should be used instead, I will submit patches to move to > OOM notifier as well. Will do. It all sounds sub-optimal to me at this point ... but I prefer the old variant where a simple "drop_slab()" won't deflate the balloon. That looks broken to me. -- Thanks, David / dhildenb

...io > balloon for the VMware balloon. If the verdict is that this is damaging and > the OOM notifier should be used instead, I will submit patches to move to > OOM notifier as well. Will do. It all sounds sub-optimal to me at this point ... but I prefer the old variant where a simple "drop_slab()" won't deflate the balloon. That looks broken to me. -- Thanks, David / dhildenb

...eflate on OOM" makes it pretty clear when deflation should > happen - after other approaches to reclaim memory failed, not while > reclaiming. This allows to minimize the footprint of a guest - memory > will only be taken out of the balloon when really needed. > > Especially, a drop_slab() will result in the whole balloon getting > deflated - undesired. While handling it via the OOM handler might not be > perfect, it keeps existing behavior. If we want a different behavior, then > we need a new feature bit and document it properly (although, there should > be a clear us...

...eflate on OOM" makes it pretty clear when deflation should > happen - after other approaches to reclaim memory failed, not while > reclaiming. This allows to minimize the footprint of a guest - memory > will only be taken out of the balloon when really needed. > > Especially, a drop_slab() will result in the whole balloon getting > deflated - undesired. While handling it via the OOM handler might not be > perfect, it keeps existing behavior. If we want a different behavior, then > we need a new feature bit and document it properly (although, there should > be a clear us...

...en deflation should > >> happen - after other approaches to reclaim memory failed, not while > >> reclaiming. This allows to minimize the footprint of a guest - memory > >> will only be taken out of the balloon when really needed. > >> > >> Especially, a drop_slab() will result in the whole balloon getting > >> deflated - undesired. > > > > Could you explain why some more? drop_caches shouldn't be really used in > > any production workloads and if somebody really wants all the cache to > > be dropped then why is balloon a...

...lined/migrated). - "virtio-mem: Allow to offline partially unplugged memory blocks" -- Adapt to Michals suggestion on core-mm part. - "virtio-mem: Better retry handling" -- Optimize retry intervals - "virtio-mem: Drop slab objects when unplug continues to fail" -- Call drop_slab()/drop_slab_node() when unplug keeps failing for a longer time. - Multiple cleanups and fixes. -------------------------------------------------------------------------- 5. Future work -------------------------------------------------------------------------- The separate patches contain a lot...

...lined/migrated). - "virtio-mem: Allow to offline partially unplugged memory blocks" -- Adapt to Michals suggestion on core-mm part. - "virtio-mem: Better retry handling" -- Optimize retry intervals - "virtio-mem: Drop slab objects when unplug continues to fail" -- Call drop_slab()/drop_slab_node() when unplug keeps failing for a longer time. - Multiple cleanups and fixes. -------------------------------------------------------------------------- 5. Future work -------------------------------------------------------------------------- The separate patches contain a lot...

Two fixes for issues I stumbled over while working on patch #3. Switch back to the good ol' OOM handler for VIRTIO_BALLOON_F_DEFLATE_ON_OOM as the switch to the shrinker introduce some undesired side effects. Keep the shrinker in place to handle VIRTIO_BALLOON_F_FREE_PAGE_HINT. Lengthy discussion under [1]. I tested with QEMU and "deflate-on-oom=on". Works as expected. Did not test

...using emulated NVDIMMs or virtio-pmem. > Would it be possible?to add a mechanism that explicitly causes page > cache to shrink without requiring the system to be under memory pressure? > We do have a sysctl "drop_caches" which calls iterate_supers(drop_pagecache_sb, NULL) and drop_slab(). doc/Documentation/sysctl/vm.txt: ============================================================== drop_caches Writing to this will cause the kernel to drop clean caches, as well as reclaimable slab objects like dentries and inodes. Once dropped, their memory becomes free. To free pagecache:...

...Mware balloon. If the verdict is that this is damaging and > > the OOM notifier should be used instead, I will submit patches to move to > > OOM notifier as well. > > Will do. It all sounds sub-optimal to me at this point ... but I prefer > the old variant where a simple "drop_slab()" won't deflate the balloon. > That looks broken to me. Okay. Could you post a patch? > -- > Thanks, > > David / dhildenb

...makes it pretty clear when deflation should >> happen - after other approaches to reclaim memory failed, not while >> reclaiming. This allows to minimize the footprint of a guest - memory >> will only be taken out of the balloon when really needed. >> >> Especially, a drop_slab() will result in the whole balloon getting >> deflated - undesired. > > Could you explain why some more? drop_caches shouldn't be really used in > any production workloads and if somebody really wants all the cache to > be dropped then why is balloon any different? > Def...

...eflate on OOM" makes it pretty clear when deflation should > happen - after other approaches to reclaim memory failed, not while > reclaiming. This allows to minimize the footprint of a guest - memory > will only be taken out of the balloon when really needed. > > Especially, a drop_slab() will result in the whole balloon getting > deflated - undesired. While handling it via the OOM handler might not be > perfect, it keeps existing behavior. If we want a different behavior, then > we need a new feature bit and document it properly (although, there should > be a clear us...

...eflate on OOM" makes it pretty clear when deflation should > happen - after other approaches to reclaim memory failed, not while > reclaiming. This allows to minimize the footprint of a guest - memory > will only be taken out of the balloon when really needed. > > Especially, a drop_slab() will result in the whole balloon getting > deflated - undesired. While handling it via the OOM handler might not be > perfect, it keeps existing behavior. If we want a different behavior, then > we need a new feature bit and document it properly (although, there should > be a clear us...

...eflate on OOM" makes it pretty clear when deflation should > happen - after other approaches to reclaim memory failed, not while > reclaiming. This allows to minimize the footprint of a guest - memory > will only be taken out of the balloon when really needed. > > Especially, a drop_slab() will result in the whole balloon getting > deflated - undesired. While handling it via the OOM handler might not be > perfect, it keeps existing behavior. If we want a different behavior, then > we need a new feature bit and document it properly (although, there should > be a clear us...