libvirt users - Nov 2012 - libvirt with lxc: internal error The 'cpuacct', 'devices' & 'memory' cgroups controllers must be mounted

Hi,

I'm running OpenStack to manage LXC instance through libvirt. The same
setting runs perfectly well on Ubuntu 12.04, while on CentOS 6u3, libvirt
dumped following message when starting lxc instance.

virsh -c lxc:/// start instance-00000032
error: internal error The 'cpuacct', 'devices' &
'memory' cgroups
controllers must be mounted


Really appreciated if someone could shed some lights. Seems to some libvirt
/ cgroup settings related.

$ cat /proc/mounts
..
none /cgroup cgroup
rw,relatime,net_prio,perf_event,blkio,net_cls,freezer,devices,memory,cpuacct,cpu,ns,cpuset
0 0
..


$ chkconfig --list | grep cg
cgconfig        0:off   1:off   2:off   3:off   4:off   5:off   6:off
cgred           0:off   1:off   2:off   3:off   4:off   5:off   6:off



xml definition
--
<domain type="lxc">
  <uuid>252a2dc5-452c-4e9c-a593-8c386c9f47e6</uuid>
  <name>instance-00000032</name>
  <memory>524288</memory>
  <vcpu>2</vcpu>
  <os>
    <type>exe</type>
    <cmdline>console=ttyS0</cmdline>
    <init>/sbin/init</init>
  </os>
  <clock offset="utc"/>
  <devices>
    <filesystem type="mount">
      <source
dir="/home/stack/nova_state/instances/instance-00000032/rootfs"/>
      <target dir="/"/>
    </filesystem>
    <interface type="bridge">
      <mac address="fa:16:3e:73:67:54"/>
      <source bridge="br100"/>
      <filterref
filter="nova-instance-instance-00000032-fa163e736754">
        <parameter name="IP" value="10.0.0.8"/>
        <parameter name="DHCPSERVER"
value="10.0.0.1"/>
        <parameter name="PROJNET" value="10.0.0.0"/>
        <parameter name="PROJMASK"
value="255.255.255.0"/>
      </filterref>
    </interface>
    <console type="pty"/>
  </devices>
</domain>


Log message with LIBVIRT_DEBUG=1 if that is helpful.
http://pastebin.com/LjZ6Euma

--
Qiu Yu
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20121107/8a890dc6/attachment.htm>

On 11/07/2012 07:21 AM, unicell wrote:
> Hi,
> 
> I'm running OpenStack to manage LXC instance through libvirt. The same
> setting runs perfectly well on Ubuntu 12.04, while on CentOS 6u3, libvirt
> dumped following message when starting lxc instance.
> 
> virsh -c lxc:/// start instance-00000032
> error: internal error The 'cpuacct', 'devices' &
'memory' cgroups
> controllers must be mounted
> 
> 
> Really appreciated if someone could shed some lights. Seems to some libvirt
> / cgroup settings related.
> 
> $ cat /proc/mounts
> ..
> none /cgroup cgroup
>
rw,relatime,net_prio,perf_event,blkio,net_cls,freezer,devices,memory,cpuacct,cpu,ns,cpuset
> 0 0
That leading 'none' in the first field of your mount output looks fishy;
on my systems, I see that field as 'cgroup'.  Perhaps that means you
haven't really mounted cgroups after all, in which case, that explains
why libvirt is complaining that it can't run LXC domains (where cgroups
is a prerequisite).
> $ chkconfig --list | grep cg
> cgconfig        0:off   1:off   2:off   3:off   4:off   5:off   6:off
That also looks fishy; on my RHEL 6.3 system, I see:
cgconfig       	0:off	1:off	2:on	3:on	4:on	5:on	6:off

because libvirt enables the cgconfig service as a prerequisite.  Since
CentOS derives from RHEL, it makes me wonder what you have done to get
your system configured differently.
> Log message with LIBVIRT_DEBUG=1 if that is helpful.
> http://pastebin.com/LjZ6Euma
That looks like you traced the client, not the actual libvirtd process
that was actually trying to use cgroups.  But enabling the cgconfig
service then restarting libvirtd may be all the more you need to do.

-- 
Eric Blake   eblake at redhat.com    +1-919-301-3266
Libvirt virtualization library http://libvirt.org

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 617 bytes
Desc: OpenPGP digital signature
URL:
<http://listman.redhat.com/archives/libvirt-users/attachments/20121107/a68af702/attachment.sig>