libvirt users - Feb 2020 - problems with understanding of the memory parameters in the xml file

Hi guys,

despite reading hours and hours in the internet i'm still struggling with
"memory", "currentmemory" and "maxMemory".

Maybe you can help me to sort it out.

My idea is that a guest has an initial value of memory (which "memory"
seems to be) when booting.
We have some Windows 10 guests which calculate some stuff and i would like to
increase memory during runtime
until it reaches a fixed maximum value.
My hope was that a higher "maxMemory" could solve this, that the guest
claims more memory and gets it.
I didn't get it. Is my idea wrong ? Do i need a balloon driver for that ?
What i find concerning ballooning is that it doesn't work automatically but
has to be adjusted
manually. Is that right ?
Balloon drivers for windows are available.

Is my idea right, does that work basically ? If yes how do i have to set the
parameters ?
Is the memory released after the guest has e.g. finished his calculation ?
Does that work automatically or do i have to adjust that manually ?

Thanks.


Bernd

-- 

Bernd Lentes 
Systemadministration 
Institute for Metabolism and Cell Death (MCD) 
Building 35.34 - Raum 208 
HelmholtzZentrum münchen 
bernd.lentes@helmholtz-muenchen.de 
phone: +49 89 3187 1241 
phone: +49 89 3187 3827 
fax: +49 89 3187 2294 
http://www.helmholtz-muenchen.de/mcd 

Perfekt ist wer keine Fehler macht 
Also sind Tote perfekt
Helmholtz Zentrum München

Helmholtz Zentrum Muenchen
Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH)
Ingolstaedter Landstr. 1
85764 Neuherberg
www.helmholtz-muenchen.de
Aufsichtsratsvorsitzende: MinDir.in Prof. Dr. Veronika von Messling
Geschaeftsfuehrung: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Kerstin Guenther
Registergericht: Amtsgericht Muenchen HRB 6466
USt-IdNr: DE 129521671

On Tue, Feb 11, 2020 at 18:48:43 +0100, Lentes, Bernd
wrote:
> Hi guys,
Hi Bernd,
> 
> despite reading hours and hours in the internet i'm still struggling
with
> "memory", "currentmemory" and "maxMemory".
to briefly summarize what those three knobs do:

1) memory - this is the initial memory size for a VM. qemu grants this
amount of memory to the VM on start. This is also the memory the guest
is able to use if the balloon driver is not loaded as the balloon driver
voluntarily gives up memory from the guest OS to the host.

2) currentmemory - in case the guest is using the balloon driver this is
the actual memory size it's using. This field is dynamically updated to
reflect the size reported by the balloon driver

3) maxMemory - This knob controls the maximum size of the memory when
memory hotplug is used. This basically sets the amount of address space
and memory slots the VM has so that new memory can be plugged in later.
> 
> Maybe you can help me to sort it out.
> 
> My idea is that a guest has an initial value of memory (which
"memory" seems to be) when booting.
Yes, until the memballoon driver is loaded the guest can use 'memory' +
any hotpluggable memory modules.
> We have some Windows 10 guests which calculate some stuff and i would like
to increase memory during runtime
> until it reaches a fixed maximum value.
> My hope was that a higher "maxMemory" could solve this, that the
guest claims more memory and gets it.
> I didn't get it. Is my idea wrong ? Do i need a balloon driver for that
?
No this is for memory hotplug and doesn't work automatically. That
configuration knob just sets the size and number of slots.

You then have to add a entry into the <devices> section:

<memory model='dimm'>
  <target>
    <size unit='MiB'>2048</size>
    <node>0</node>
  </target>
</memory>

The above can also be added during runtime e.g. using virsh
attach-device. Hence hotplug. It can also be unplugged during runtime
but that requires guest cooperation and there are a few caveats of this.
Namely to successfully unplug the memory the guest must not write any
non-movable pages into it so that it can give up the memory later. On
linux that means that no memory-mapped I/O regions can be created there
which may lead to weird guest behaviour if the memory is onlined as
movable. I'm not sure how windows behaves in this regard though, but
AFAIK it supports memory hotplug just fine.
> What i find concerning ballooning is that it doesn't work automatically
but has to be adjusted
> manually. Is that right ?
No, unfortunately none of this works automatically.
> Balloon drivers for windows are available.
> 
> Is my idea right, does that work basically ? If yes how do i have to set
the parameters ?
> Is the memory released after the guest has e.g. finished his calculation ?
> Does that work automatically or do i have to adjust that manually ?
When using the balloon driver you can set the 'currentMemory' size down
to some reasonable value and the balloon driver will then return the
memory to the host. There were some attempts to make this automatic, but
I don't remember how they went. One other caveat is that any memory
returned by the balloon driver to the host may be available to the guest
again e.g. on reboot when the balloon driver is removed.

For a 1 NUMA node guest the memory hotplug an balloon can theoretically
be combined but unplugging of the memory might not work while the ballon
is inflated.

I hope this clarified it somehwat.

> Thanks.
> 
> 
> Bernd
> 
> -- 
> 
> Bernd Lentes 
> Systemadministration 
> Institute for Metabolism and Cell Death (MCD) 
> Building 35.34 - Raum 208 
> HelmholtzZentrum münchen 
> bernd.lentes@helmholtz-muenchen.de 
> phone: +49 89 3187 1241 
> phone: +49 89 3187 3827 
> fax: +49 89 3187 2294 
> http://www.helmholtz-muenchen.de/mcd 
> 
> Perfekt ist wer keine Fehler macht 
> Also sind Tote perfekt
> Helmholtz Zentrum München
> 
> Helmholtz Zentrum Muenchen
> Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH)
> Ingolstaedter Landstr. 1
> 85764 Neuherberg
> www.helmholtz-muenchen.de
> Aufsichtsratsvorsitzende: MinDir.in Prof. Dr. Veronika von Messling
> Geschaeftsfuehrung: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Kerstin
Guenther
> Registergericht: Amtsgericht Muenchen HRB 6466
> USt-IdNr: DE 129521671
> 
> 
> 
>

----- On Feb 12, 2020, at 8:34 AM, Peter Krempa pkrempa@redhat.com wrote:
> to briefly summarize what those three knobs do:
> 
> 1) memory - this is the initial memory size for a VM. qemu grants this
> amount of memory to the VM on start. This is also the memory the guest
> is able to use if the balloon driver is not loaded as the balloon driver
> voluntarily gives up memory from the guest OS to the host.
> 
> 2) currentmemory - in case the guest is using the balloon driver this is
> the actual memory size it's using. This field is dynamically updated to
> reflect the size reported by the balloon driver
> 
> 3) maxMemory - This knob controls the maximum size of the memory when
> memory hotplug is used. This basically sets the amount of address space
> and memory slots the VM has so that new memory can be plugged in later.
Aaaah.
 
> The above can also be added during runtime e.g. using virsh
> attach-device. Hence hotplug. It can also be unplugged during runtime
> but that requires guest cooperation and there are a few caveats of this.
> Namely to successfully unplug the memory the guest must not write any
> non-movable pages into it so that it can give up the memory later. On
> linux that means that no memory-mapped I/O regions can be created there
> which may lead to weird guest behaviour if the memory is onlined as
> movable. I'm not sure how windows behaves in this regard though, but
> AFAIK it supports memory hotplug just fine.
> 
>> What i find concerning ballooning is that it doesn't work
automatically but has
>> to be adjusted
>> manually. Is that right ?
> 
> No, unfortunately none of this works automatically.
> 
>> Is my idea right, does that work basically ? If yes how do i have to
set the
>> parameters ?
>> Is the memory released after the guest has e.g. finished his
calculation ?
>> Does that work automatically or do i have to adjust that manually ?
> 
> When using the balloon driver you can set the 'currentMemory' size
down
> to some reasonable value and the balloon driver will then return the
> memory to the host. There were some attempts to make this automatic, but
> I don't remember how they went. One other caveat is that any memory
> returned by the balloon driver to the host may be available to the guest
> again e.g. on reboot when the balloon driver is removed.
> 
> For a 1 NUMA node guest the memory hotplug an balloon can theoretically
> be combined but unplugging of the memory might not work while the ballon
> is inflated.
> 
> I hope this clarified it somehwat.
Yes it did.Thanks.
Ballon and Memeory hotplug are two different things, right ?
Which is better, where are the advantages and disadvantages ? 
I played a bit around with ballooning and it went like a charm.
If i try to use Hotplugging and inserts "maxMemory" and "memory
model='dimm'" in the config,
libvirt complains i have to add a "numa" entry.
I don't know much about Numa, so maybe it's better not to use
hotplugging.

While reading in the internet i stumbled across KSM for Linux, which is
recommended for the host if you have
several guests of the same OS.
What do you think about it ? 

Btw: is it also possible to add cpu's to guests during runtime ?

Thanks.


Bernd
Helmholtz Zentrum München

Helmholtz Zentrum Muenchen
Deutsches Forschungszentrum fuer Gesundheit und Umwelt (GmbH)
Ingolstaedter Landstr. 1
85764 Neuherberg
www.helmholtz-muenchen.de
Aufsichtsratsvorsitzende: MinDir.in Prof. Dr. Veronika von Messling
Geschaeftsfuehrung: Prof. Dr. med. Dr. h.c. Matthias Tschoep, Kerstin Guenther
Registergericht: Amtsgericht Muenchen HRB 6466
USt-IdNr: DE 129521671