libvirt users - Oct 2022 - failed to load seccomp syscall filter in kernel: Operation canceled

Hello community,

   I am operating an openstack cluster where applications (libvirt/nova
etc) are running using containers. The compute node's arch is aarch64
(phytium 2500), when there are virtual machines around 60 or 70, I failed
to boot new virtual machines and faced with following error message,

error: internal error: qemu unexpectedly closed the monitor:
2022-10-24T06:23:54.545685Z qemu-system-aarch64: -sandbox
on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny:
failed to load seccomp syscall filter in kernel: Operation canceled


Interestingly, if I virsh stop one virtual machine, I am able to boot
another. Besides, I managed to manually boot a virtual machine without any
issue. So my question is what could be the potential cause of this behavior
and how can I deal with it? Thank you very much in advance for the help.

-- 

Best Regards,

Jiatong Shen
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20221025/179bac9c/attachment.htm>

On 10/25/22 02:38, Jiatong Shen wrote:
> Hello community,
> 
> ? ?I am operating an openstack cluster where applications (libvirt/nova
> etc) are running using containers. The compute node's arch is aarch64
> (phytium 2500), when there are virtual machines around 60 or 70, I
> failed to boot new virtual machines and faced with following error message,
> 
> error: internal error: qemu unexpectedly closed the monitor:
> 2022-10-24T06:23:54.545685Z qemu-system-aarch64: -sandbox
> on,obsolete=deny,elevateprivileges=deny,spawn=deny,resourcecontrol=deny:
> failed to load seccomp syscall filter in kernel: Operation canceled
> 
This error message comes from qemu, which sees seccomp_load() fail.
Looking further [1], seccomp_load() returns -ECANCELED. This is because
qemu does not set SCMP_FLTATR_API_SYSRAWRC attribute and thus public
APIs fail with this meaningless error. I've posted patch here [1].

But the symptoms suggest that you are hitting a limit (for eBPF perhaps?).

1: https://lists.gnu.org/archive/html/qemu-devel/2022-10/msg04509.html
> 
> Interestingly, if I virsh stop one virtual machine, I am able to boot
> another. Besides, I managed to manually boot a virtual machine without
> any issue. So my question is what could be the potential cause of this
> behavior and how can I deal with it? Thank you very much in advance for
> the help.
> 
I wonder whether the fact that openstack runs VMs in container has
something to do with this. Perhaps it touches the limit/sets different
accounting for it?

Michal