libvirt users - Apr 2018 - qemu: qemu_thread_create: Resource temporarily unavailable

Hi:
  My libvirt is 3.4.0 and qemu is 2.9.0, My host has a big memory(128G), I run
many guest on my host(80 guests), when I batch stop and start guests, some guest
will crash, the qemu log is "qemu: qemu_thread_create: Resource temporarily
unavailable". I search qemu code, I find qemu_thread_create just like
pthread_create function.  I search on internet, someone say that is a bug of
qemu.Because my host run many guest ,so I cann't ensure whether qemu bug or
my host load heavy cause my guest crash. Can someone tell me what reason cause
the promble? and what can I do avoid the promble?
  Thanks

On Wed, Apr 18, 2018 at 06:27:10PM +0800, llilulu wrote:
> Hi:
>   My libvirt is 3.4.0 and qemu is 2.9.0, My host has a big memory(128G),
> I run many guest on my host(80 guests), when I batch stop and start guests,
> some guest will crash, the qemu log is "qemu: qemu_thread_create:
Resource
> temporarily unavailable". I search qemu code, I find
qemu_thread_create
> just like pthread_create function.  I search on internet, someone say
> that is a bug of qemu.Because my host run many guest ,so I cann't
ensure
> whether qemu bug or my host load heavy cause my guest crash. Can someone
> tell me what reason cause the promble? and what can I do avoid the promble?
It isn't a QEMU bug - it is the kernel telling you that you have hit the max
process limit.

80 guests is honestly not a very large value though, so I'm surprised it
would hit a max process limit.  Unless perhaps you're using RBD storage
as that creates an insanely huge number (100+) of threads  sometimes.

Check /proc/$PID/limits for one of the guests that it successfully
running and it will tell you current "Max processes" limit. Then 
in /etc/libvirt/qemu.conf try setting  max_processes to a larger value
and restart libvirtd.


Regards,
Daniel
-- 
|: https://berrange.com      -o-    https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org         -o-            https://fstop138.berrange.com :|
|: https://entangle-photo.org    -o-    https://www.instagram.com/dberrange :|