thr3ads.net - libvirt users - Re: [libvirt-users] cgroup error starting domains [Jun 2013]

If this information is useful, please help other people find it:
Share via:

Daniel P. Berrange

2013-Jun-07 10:24 UTC

Re: [libvirt-users] cgroup error starting domains

On Thu, Jun 06, 2013 at 06:26:23PM +0200, Matteo Bernardini
wrote:> On 05/22/2013 11:01 AM, Matteo Bernardini wrote:
> > Hi,
> > 
> > I've got a small problem using libvirt-1.0.5.1 (with the latest
patch in
> > the v1.0.5-maint branch on git added).
> > I'm using slackware64-14.0 but the situation is exactly the same
described
> > on a debian bug
> > 
> > http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=707201
> > 
> > When trying to start a domain I got
> > 
> > error: Failed to start domain debian-wheezy-xfce
> > error: Unable to initialize /machine cgroup: Invalid argument
> > 
> > with this in libvirtd.log
> > 
> > virCgroupDetect:373 : Failed to detect mounts for /machine
> > qemuInitCgroup:425 : Unable to initialize /machine cgroup: Invalid
argument
> > 
> > and the error goes away if I unmount /sys/fs/cgroup.
> > 
> > Reverting to 1.0.4 fixes this,
> 
> sorry, just to report the same effect with 1.0.6: this is my
> /sys/fs/cgroup layout
> 
> http://pastebin.com/Gxsgb8KG
> 
> and this is the corresponding stuff in /var/lib/libvirt/libvirtd.log
> (with log level 1) when I issue a
> 
> # virsh start s64cont
> 
> http://pastebin.com/Rhg3tSzt
> 
> 
> as I've had a look at
> 
> http://libvirt.org/cgroups.html
> 
> I'm just wondering if systemd is necessary (we don't have it in
> Slackware) or I'm simply missing something...
No, systemd is definitely /not/ mandatory.  Can you provide your
/proc/mounts file contents

Libvirt seems to be failing to detect the mount points

2013-06-06 15:49:37.688+0000: 29305: debug : virCgroupNewPartition:1248 :
path=/machine create=1 controllers=5f
2013-06-06 15:49:37.688+0000: 29305: debug : virCgroupNew:808 : parent=(nil)
path=/machine controllers=95
2013-06-06 15:49:37.688+0000: 29305: debug : virCgroupDetect:359 :
group=0x7f601007f6e0 controllers=95 path=/machine parent=(nil)
2013-06-06 15:49:37.688+0000: 29305: error : virCgroupDetect:366 : Failed to
detect mounts for /machine
2013-06-06 15:49:37.688+0000: 29305: error : qemuInitCgroup:733 : Unable to
initialize /machine cgroup: Invalid argument


also do you have anything in /etc/libvirt/qemu.conf set for the
cgroups controllers setting ?

Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Matteo Bernardini

2013-Jun-07 11:07 UTC

head link

Re: [libvirt-users] cgroup error starting domains

Thanks Daniel for helping with this :)

# cat /proc/mounts
rootfs / rootfs rw 0 0
/dev/root / ext4 rw,noatime,data=ordered 0 0
devtmpfs /dev devtmpfs rw,relatime,size=1970508k,nr_inodes=492627,mode=755
0 0
proc /proc proc rw,relatime 0 0
sysfs /sys sysfs rw,relatime 0 0
tmpfs /run tmpfs rw,relatime,mode=755 0 0
devpts /dev/pts devpts rw,relatime,gid=5,mode=620,ptmxmode=000 0 0
fusectl /sys/fs/fuse/connections fusectl rw,relatime 0 0
/dev/sda1 /media/oldslack ext4 rw,noatime,data=ordered 0 0
/dev/sdb1 /home ext4 rw,noatime,data=ordered 0 0
/dev/md0p1 /data ext4 rw,noatime,data=ordered 0 0
tmpfs /dev/shm tmpfs rw,relatime 0 0
gvfsd-fuse /home/poncez/.gvfs fuse.gvfsd-fuse
rw,nosuid,nodev,relatime,user_id=1000,group_id=100 0 0
cgroup /sys/fs/cgroup cgroup
rw,relatime,bfqio,hugetlb,perf_event,blkio,net_cls,freezer,devices,memory,cpuacct,cpu,cpuset,clone_children
0 0

if useful, also

# cat /etc/mtab
/dev/sda2 / ext4 rw,noatime 0 0
proc /proc proc rw 0 0
sysfs /sys sysfs rw 0 0
/dev/sda1 /media/oldslack ext4 rw,noatime 0 0
/dev/sdb1 /home ext4 rw,noatime 0 0
/dev/md0p1 /data ext4 rw,noatime 0 0
tmpfs /dev/shm tmpfs rw 0 0
gvfsd-fuse /home/poncez/.gvfs fuse.gvfsd-fuse rw,nosuid,nodev,user=poncez 0
0
cgroup /sys/fs/cgroup cgroup rw 0 0

in /etc/libvirt/qemu.conf I uncommented this line

cgroup_controllers = [ "cpu", "devices", "memory",
"blkio", "cpuset",
"cpuacct" ]

Matteo

Daniel P. Berrange

2013-Jun-07 11:15 UTC

head link

Re: [libvirt-users] cgroup error starting domains

On Fri, Jun 07, 2013 at 01:07:01PM +0200, Matteo Bernardini
wrote:> Thanks Daniel for helping with this :)
> 
> # cat /proc/mounts
[snip]
> cgroup /sys/fs/cgroup cgroup 
rw,relatime,bfqio,hugetlb,perf_event,blkio,net_cls,freezer,devices,memory,cpuacct,cpu,cpuset,clone_children
Ok so here you've mounted all the cgroups controllers at the same
place. This is really strongly recommended against. It is better
to mount each controller separately at

   /sys/fs/cgroup/<controller name>

and have /sys/fs/cgroup as a tmpfs.

http://www.freedesktop.org/wiki/Software/systemd/PaxControlGroups

eg to get something along the lines of:

$ ls -l /sys/fs/cgroup/
total 0
drwxr-xr-x. 3 root root  0 Jun  3 09:41 blkio
lrwxrwxrwx. 1 root root 11 Jun  3 09:41 cpu -> cpu,cpuacct
lrwxrwxrwx. 1 root root 11 Jun  3 09:41 cpuacct -> cpu,cpuacct
drwxr-xr-x. 3 root root  0 Jun  3 09:41 cpu,cpuacct
drwxr-xr-x. 3 root root  0 Jun  3 09:41 cpuset
drwxr-xr-x. 3 root root  0 Jun  3 09:41 devices
drwxr-xr-x. 3 root root  0 Jun  3 09:41 freezer
drwxr-xr-x. 3 root root  0 Jun  3 09:41 memory
drwxr-xr-x. 2 root root  0 Jun  3 09:41 net_cls
drwxr-xr-x. 2 root root  0 Jun  3 09:41 perf_event
drwxr-xr-x. 4 root root  0 Jun  3 09:42 systemd
$ grep cgroup /proc/mounts 
tmpfs /sys/fs/cgroup tmpfs rw,seclabel,nosuid,nodev,noexec,mode=755 0 0
cgroup /sys/fs/cgroup/systemd cgroup
rw,nosuid,nodev,noexec,relatime,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd
0 0
cgroup /sys/fs/cgroup/cpuset cgroup rw,nosuid,nodev,noexec,relatime,cpuset 0 0
cgroup /sys/fs/cgroup/cpu,cpuacct cgroup
rw,nosuid,nodev,noexec,relatime,cpuacct,cpu 0 0
cgroup /sys/fs/cgroup/memory cgroup rw,nosuid,nodev,noexec,relatime,memory 0 0
cgroup /sys/fs/cgroup/devices cgroup rw,nosuid,nodev,noexec,relatime,devices 0 0
cgroup /sys/fs/cgroup/freezer cgroup rw,nosuid,nodev,noexec,relatime,freezer 0 0
cgroup /sys/fs/cgroup/net_cls cgroup rw,nosuid,nodev,noexec,relatime,net_cls 0 0
cgroup /sys/fs/cgroup/blkio cgroup rw,nosuid,nodev,noexec,relatime,blkio 0 0
cgroup /sys/fs/cgroup/perf_event cgroup
rw,nosuid,nodev,noexec,relatime,perf_event 0 0


(obviously the 'system' mount point is not required if not using
systemd).
> in /etc/libvirt/qemu.conf I uncommented this line
> 
> cgroup_controllers = [ "cpu", "devices",
"memory", "blkio", "cpuset",
> "cpuacct" ]
And this, together with your mount setup is what causes the problem.

Here you have told libvirt to *only* use those 6 listed cgroups controllers,
 but because you've mounted all 9 controllers at the same mount point,
it is impossible for libvirt to satisfy  the config you requested. It can't
use the 6 controllers you asked for, without also using the other 3 you have
excluded in the config.

Either comment out that cgroup_controllers setting (to let libvirt do
auto-detection of mounts), or switch to using a separate mount per
controller, or both.

Regards,
Daniel
-- 
|: http://berrange.com      -o-    http://www.flickr.com/photos/dberrange/ :|
|: http://libvirt.org              -o-             http://virt-manager.org :|
|: http://autobuild.org       -o-         http://search.cpan.org/~danberr/ :|
|: http://entangle-photo.org       -o-       http://live.gnome.org/gtk-vnc :|

Apparently Analagous Threads

Search for more seemingly similar threads

libvirt users - Jun 2013 - Re: cgroup error starting domains

Re: [libvirt-users] cgroup error starting domains

Re: [libvirt-users] cgroup error starting domains

Re: [libvirt-users] cgroup error starting domains

Apparently Analagous Threads