Linux Virtualization - Feb 2020 - [PATCH RFC] virtio_balloon: conservative balloon page shrinking

On Monday, February 10, 2020 11:57 AM, Tetsuo Handa
wrote:
> Then, "node-A's NR_FILE_PAGES is already 0 and node-B's
NR_FILE_PAGES is
> not 0, but allocation request which triggered this shrinker wants to
allocate
> from only node-A"
> would be confused by this change, for the pagecache pages for allocating
> thread's interested node are already depleted but the balloon cannot
shrink
> when it should because the pagecache pages for allocating thread's
> uninterested nodes are not yet depleted.
The existing balloon isn't numa aware. "but the balloon cannot shrink
" - even we
let balloon to shrink, it could shrink pages from the uninterested node.

When we have a numa aware balloon, we could further update the shrinker
to check with the per node counter , node_page_state(NR_FILE_PAGES).
> 
> >
> Well, my comment is rather: "Do not try to reserve guest's memory.
In other
> words, do not try to maintain balloons on the guest side. Since host would
> be able to cache file data on the host's cache, guests would be able to
> quickly fetch file data from host's cache via normal I/O
requests." ;-)
Didn't this one. The discussion was about guest pagecache pages v.s. guest
balloon pages.
Why is host's pagecache here?

Best,
Wei

On 2020/02/10 16:27, Wang, Wei W wrote:
>> Well, my comment is rather: "Do not try to reserve guest's
memory. In other
>> words, do not try to maintain balloons on the guest side. Since host
would
>> be able to cache file data on the host's cache, guests would be
able to
>> quickly fetch file data from host's cache via normal I/O
requests." ;-)
> 
> Didn't this one. The discussion was about guest pagecache pages v.s.
guest balloon pages.
> Why is host's pagecache here?
I'm expecting a mode: "Guests should try to minimize pagecache pages
(and teach
host to treat reclaimed pages as if POSIX_FADV_DONTNEED) instead of managing
guest balloon pages". In other words, as if

  while :; sleep 5; echo 1 > /proc/sys/vm/drop_caches; done

is running in the guest's kernel. And as if

  echo 2 > /proc/sys/vm/drop_caches

is triggered in the guest's kernel when host requested guests to reclaim
memory. No long-life balloons. Guest balloons do not need to care about
NUMA. Just leave the management of pagecache pages to the host.

Sorry for the slow reply.

Re: Module parameters: I prefer not to have module parameters since they
are controlled by the guest. In general, in virtualized environments the
admins controlling the hypervisor are more knowledgeable about these things
than the users. A feature bit seems useful so that the host knows what the
guest behavior will be, and can change the host side implementation to make
the experience good for the guest.

I worry that requiring global_node_page_state(NR_FILE_PAGES) == 0 before
allowing deflation is too strict. One of the benefits of the shrinker API
is that it is invoked before vmscan.c has gone through heroic efforts to
reclaim the world. I'm not familiar enough with the code to judge how this
patch impacts this, but would it be beneficial to allow deflation when
vmscan.c is trying "too hard" to reclaim pages? Is there some softer
condition than "global_node_page_state(NR_FILE_PAGES) == 0"?

For my own understanding, does this patch work by returning 0 pages when
asked for pages? Are there cases where that results in an unnecessary OOM?
For example, if global_node_page_state(NR_FILE_PAGES) == 1, and the guest
needs 2?

Regarding other shrinkers (like KVM MMU cache): Reclaiming other shrinkers
first would match the behavior of DEFLATE_ON_OOM when it was using the OOM
notifier callback. On the other hand (awkwardly), the memory stats reported
on the stats queue for "available memory" do not count shrinker memory
as
"available". So a balloon implementation that aims to reclaim some
amount
of available memory would not be able to tell how much memory was in the
shrinkers and probably doesn't expect to reclaim them. For this reason, I
think only looking at page cache size is the right choice. There should be
a 1:1 relationship between stats reported and when DEFLATE_ON_OOM is
invoked. Maybe in the future we add another stat that reports shrinker
sizes, in which case we should also add a feature bit that allows other
shrinkers to be pressured.

Regarding NUMA awareness: I agree it's out of scope for this patch since
all implementations so far are not NUMA aware.

Would it be possible to back port this patch to 4.19 when the change to
shrinker API was made?

On Tue, Feb 11, 2020 at 6:20 AM Tetsuo Handa <
penguin-kernel at i-love.sakura.ne.jp> wrote:
> On 2020/02/10 16:27, Wang, Wei W wrote:
> >> Well, my comment is rather: "Do not try to reserve
guest's memory. In
> other
> >> words, do not try to maintain balloons on the guest side. Since
host
> would
> >> be able to cache file data on the host's cache, guests would
be able to
> >> quickly fetch file data from host's cache via normal I/O
requests." ;-)
> >
> > Didn't this one. The discussion was about guest pagecache pages
v.s.
> guest balloon pages.
> > Why is host's pagecache here?
>
> I'm expecting a mode: "Guests should try to minimize pagecache
pages (and
> teach
> host to treat reclaimed pages as if POSIX_FADV_DONTNEED) instead of
> managing
> guest balloon pages". In other words, as if
>
>   while :; sleep 5; echo 1 > /proc/sys/vm/drop_caches; done
>
> is running in the guest's kernel. And as if
>
>   echo 2 > /proc/sys/vm/drop_caches
>
> is triggered in the guest's kernel when host requested guests to
reclaim
> memory. No long-life balloons. Guest balloons do not need to care about
> NUMA. Just leave the management of pagecache pages to the host.
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.linuxfoundation.org/pipermail/virtualization/attachments/20200214/401639d5/attachment.html>