libvirt users - Apr 2018 - start a domain, minimal memory request.

Hi:
   I use libvirt manage virtual guest, When I start many virtual domain(my host
128G physical memory）, example, I start 100 guest, 1G per guest。sometime I will
encounter "cannot allocate memory", "cannot set up guest memory
\'pc.ram\'： cannot allocate memory". sometime encounter qemu
process killed by system(oom),
  I want to know what minimal memory need when start a domain ? and how many
host free memory reserve can ensure vm running steady.
 Thanks

On 04/09/2018 09:00 AM, llilulu wrote:
> Hi:
>    I use libvirt manage virtual guest, When I start many virtual domain(my
host 128G physical memory）, example, I start 100 guest, 1G per guest。sometime I
will encounter "cannot allocate memory", "cannot set up guest
memory \'pc.ram\'： cannot allocate memory". sometime encounter qemu
process killed by system(oom),
>   I want to know what minimal memory need when start a domain ? and how
many host free memory reserve can ensure vm running steady.
There's no answer to this question I'm afraid. Not only it's
undecidable
problem but we also have history with it. I mean, at some point in the
past somebody came up with 'great' idea to have libvirt computing some
max value and setting it in cgroups to prevent qemu with a memleak
eating all the resources. It's better to be safe than sorry, right?
We've came up with several equations trying to figure that out. But then
we had to revert the patches because no matter how hard we tried some
users were still hitting the limit with valid use cases.

IOW, qemu memory consumption is not static through domain lifetime. It
changes. IIRC, hotplug operation was consuming surprisingly lot of
memory (on the other hand, for a fraction of a second). Also, memory
consumption is going to be dependant on domain configuration. Then, qemu
is linking with some libraries too - they will add their own overhead.

Long story short, I don't think there's an answer to your question.

Michal

On 2018-04-09 10:07, Michal Privoznik wrote:
> On 04/09/2018 09:00 AM, llilulu wrote:
>> Hi:
>>     I use libvirt manage virtual guest, When I start many virtual
domain(my host 128G physical memory）, example, I start 100 guest, 1G per
guest。sometime I will encounter "cannot allocate memory", "cannot
set up guest memory \'pc.ram\'： cannot allocate memory". sometime
encounter qemu process killed by system(oom),
>>    I want to know what minimal memory need when start a domain ? and
how many host free memory reserve can ensure vm running steady.
> There's no answer to this question I'm afraid. Not only it's
undecidable
> problem but we also have history with it. I mean, at some point in the
> past somebody came up with 'great' idea to have libvirt computing
some
> max value and setting it in cgroups to prevent qemu with a memleak
> eating all the resources. It's better to be safe than sorry, right?
> We've came up with several equations trying to figure that out. But
then
> we had to revert the patches because no matter how hard we tried some
> users were still hitting the limit with valid use cases.
>
> IOW, qemu memory consumption is not static through domain lifetime. It
> changes. IIRC, hotplug operation was consuming surprisingly lot of
> memory (on the other hand, for a fraction of a second). Also, memory
> consumption is going to be dependant on domain configuration. Then, qemu
> is linking with some libraries too - they will add their own overhead.
>
> Long story short, I don't think there's an answer to your question.
There may not be an answer that is provable by fancy maths but you may 
be able to estimate the needed physical memory. You could also spawn a 
few guests and observe the memory you have left, eventually adding some 
if you need more. Or you could, if your guests are all the clones of 
each other, observe one and multiply that estimate with the number of 
guests you are about to spawn. Add some percent to the outcome plus 2gb 
for safety plus a little bit of salt and you should have a very good 
approximation. With that you should come to a close match of the memory 
you are going to need. Note also that your host should have configured 
some - in your case I'd calculate at least 16gb - swap, so you may well 
be able to handle memory bursts that go beyond the physically available 
memory. Surely guests will suffer performance penalties unless you are 
willing to mount more ram.