Jonatan Schlag
2016-Aug-04 11:59 UTC
Re: [libvirt-users] Libvirt: dynamic ownership did not work
Am Do, 4. Aug, 2016 um 1:38 schrieb Michal Privoznik <mprivozn@redhat.com>:> On 04.08.2016 12:12, Jonatan Schlag wrote: >> >> >> Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik >> <mprivozn@redhat.com>: >>> On 03.08.2016 21:17, Jonatan Schlag wrote: >>>> Hi, >>>> I have a very strange problem with libvirt. I work on some >>>> machines >>>> with >>>> libvirt (Debian/ Arch Linux) and libvirt set the ownership of >>>> images >>>> file automatically to the qemu user / group for example on Arch >>>> Linux to >>>> nobody:kvm. >>>> So when I copy an image file with root and use I then with qemu, >>>> libvirt >>>> change the owner/ group to nobody:kvm. >>>> >>>> But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) >>>> and on >>>> this machine libvirt did not change the ownership of the image >>>> files >>>> which results in this error: >>>> >>>> libvirtError: internal error: process exited while connecting to >>>> monitor: able-ticketing,seamless-migration=on -device >>>> >>>> >>>> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 >>>> >>>> -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device >>>> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev >>>> spicevmc,id=charredir0,name=usbredir -device >>>> usb-redir,chardev=charredir0,id=redir0 -device >>>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg >>>> timestamp=on >>>> 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive >>>> >>>> >>>> file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0: >>>> >>>> Could not open '/data/hdd1/libvirt/images/test.img': Permission >>>> denied >>> >>> Can you please share the debug logs? >>> >>> http://wiki.libvirt.org/page/DebugLogs >>> >>> Also, my initial suspect, before diving any deeper is that >>> usually, when >>> users compile libvirt on their own, they forget to set the correct >>> prefix, therefore libvirt is looking for its config files NOT under >>> /etc/libvirt but /usr/local/etc/ or whatever. >>> >>> BTW: is the daemon running under root? >>> >>> Michal >> >> Hi, >> >> The daemon runs under root. >> >> I uploaded the debug logs to: >> >> http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log >> >> The UID of the user nobody is 99, the GID of the group kvm is 1011. >> >> I added my configure options to the bug report. >> >> Following the log the ownership is changed but why is the file still >> owned by root:root? > > Right. the file is set ownership.But the file ist still owned by root:root and so it is not accessable by qemu as nobody:kvm. In the moment the only possible way is that the change of the ownership fail, but then there should be an error message, but there is no error message in the log.> One thing though - libvirt does not > label the whole path, just the file. So maybe you should check whether > nobody:kvm has access into the /data/hdd1/libvirt/images dir (and also > each one in the path).When is set the ownership manually to nobody:kcm qemu is able to acces the fail so i think nobody:kvm has access to each folder in the path> > > What about apparmor? In the bug report you say that selinux is > disabled, > but what about apparmor?apparmor is also disabled or better it is like se linux not installed.> > > Michal
Laine Stump
2016-Aug-04 15:02 UTC
Re: [libvirt-users] Libvirt: dynamic ownership did not work
On 08/04/2016 07:59 AM, Jonatan Schlag wrote:> > > Am Do, 4. Aug, 2016 um 1:38 schrieb Michal Privoznik > <mprivozn@redhat.com>: >> On 04.08.2016 12:12, Jonatan Schlag wrote: >> >> Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik >> <mprivozn@redhat.com <mailto:mprivozn@redhat.com>>: >> >> On 03.08.2016 21:17, Jonatan Schlag wrote: >> >> Hi, I have a very strange problem with libvirt. I work on >> some machines with libvirt (Debian/ Arch Linux) and >> libvirt set the ownership of images file automatically to >> the qemu user / group for example on Arch Linux to >> nobody:kvm. So when I copy an image file with root and >> use I then with qemu, libvirt change the owner/ group to >> nobody:kvm. But I also compiled libvirt for a machine >> (gcc 4.9.4 glibc 2.12) and on this machine libvirt did >> not change the ownership of the image files which results >> in this error: libvirtError: internal error: process >> exited while connecting to monitor: >> able-ticketing,seamless-migration=on -device >> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 >> -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device >> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev >> spicevmc,id=charredir0,name=usbredir -device >> usb-redir,chardev=charredir0,id=redir0 -device >> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg >> timestamp=on 2016-08-03T18:19:47.494512Z >> qemu-system-x86_64: -drive >> file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0: >> Could not open '/data/hdd1/libvirt/images/test.img': >> Permission denied >> >> Can you please share the debug logs? >> http://wiki.libvirt.org/page/DebugLogs Also, my initial >> suspect, before diving any deeper is that usually, when users >> compile libvirt on their own, they forget to set the correct >> prefix, therefore libvirt is looking for its config files NOT >> under /etc/libvirt but /usr/local/etc/ or whatever. BTW: is >> the daemon running under root? Michal >> >> Hi, The daemon runs under root. I uploaded the debug logs to: >> http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log >> <http://people.ipfire.org/%7Ejschlag/1363864/1_libvirtd.log> The >> UID of the user nobody is 99, the GID of the group kvm is 1011. I >> added my configure options to the bug report. Following the log >> the ownership is changed but why is the file still owned by >> root:root? >> >> Right. the file is set ownership. > But the file ist still owned by root:root and so it is not accessable > by qemu as nobody:kvm. In the moment the only possible way is that the > change of the ownership fail, but then there should be an error > message, but there is no error message in the log.Are you checking it after the operation has failed and libvirt has continued? If libvirt fails to access the file, it will then set everything back to its original state, so looking at the file ownership after the fact tells you nothing. If you really want to see if the file has the proper ownership, you need to attach gdb to libvirtd and set an appropriate breakpoint.
Michal Privoznik
2016-Aug-04 15:07 UTC
Re: [libvirt-users] Libvirt: dynamic ownership did not work
On 04.08.2016 13:59, Jonatan Schlag wrote:> > > Am Do, 4. Aug, 2016 um 1:38 schrieb Michal Privoznik <mprivozn@redhat.com>: >> On 04.08.2016 12:12, Jonatan Schlag wrote: >>> >>> >>> Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik >>> <mprivozn@redhat.com>: >>>> On 03.08.2016 21:17, Jonatan Schlag wrote: >>>>> Hi, >>>>> I have a very strange problem with libvirt. I work on some machines >>>>> with >>>>> libvirt (Debian/ Arch Linux) and libvirt set the ownership of images >>>>> file automatically to the qemu user / group for example on Arch >>>>> Linux to >>>>> nobody:kvm. >>>>> So when I copy an image file with root and use I then with qemu, >>>>> libvirt >>>>> change the owner/ group to nobody:kvm. >>>>> >>>>> But I also compiled libvirt for a machine (gcc 4.9.4 glibc 2.12) >>>>> and on >>>>> this machine libvirt did not change the ownership of the image files >>>>> which results in this error: >>>>> >>>>> libvirtError: internal error: process exited while connecting to >>>>> monitor: able-ticketing,seamless-migration=on -device >>>>> >>>>> >>>>> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 >>>>> >>>>> >>>>> -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device >>>>> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev >>>>> spicevmc,id=charredir0,name=usbredir -device >>>>> usb-redir,chardev=charredir0,id=redir0 -device >>>>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg timestamp=on >>>>> 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive >>>>> >>>>> >>>>> file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0: >>>>> >>>>> >>>>> Could not open '/data/hdd1/libvirt/images/test.img': Permission >>>>> denied >>>> >>>> Can you please share the debug logs? >>>> >>>> http://wiki.libvirt.org/page/DebugLogs >>>> >>>> Also, my initial suspect, before diving any deeper is that usually, >>>> when >>>> users compile libvirt on their own, they forget to set the correct >>>> prefix, therefore libvirt is looking for its config files NOT under >>>> /etc/libvirt but /usr/local/etc/ or whatever. >>>> >>>> BTW: is the daemon running under root? >>>> >>>> Michal >>> >>> Hi, >>> >>> The daemon runs under root. >>> >>> I uploaded the debug logs to: >>> >>> http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log >>> >>> The UID of the user nobody is 99, the GID of the group kvm is 1011. >>> >>> I added my configure options to the bug report. >>> >>> Following the log the ownership is changed but why is the file still >>> owned by root:root? >> >> Right. the file is set ownership. > But the file ist still owned by root:root and so it is not accessable by > qemu as nobody:kvm. In the moment the only possible way is that the > change of the ownership fail, but then there should be an error message, > but there is no error message in the log.Then the other option that comes to my mind is a race with somebody else on the system. You can attach gdb to the daemon and set breakpoint to virSecurityDACSetOwnershipInternal(). In the arguments you should see the path eventually among with uid:gid. BTW: what's the domain XML? Michal
Jonatan Schlag
2016-Aug-04 18:28 UTC
Re: [libvirt-users] Libvirt: dynamic ownership did not work
Am Do, 4. Aug, 2016 um 5:07 schrieb Michal Privoznik <mprivozn@redhat.com>:> On 04.08.2016 13:59, Jonatan Schlag wrote: >> >> >> Am Do, 4. Aug, 2016 um 1:38 schrieb Michal Privoznik >> <mprivozn@redhat.com>: >>> On 04.08.2016 12:12, Jonatan Schlag wrote: >>>> >>>> >>>> Am Do, 4. Aug, 2016 um 11:32 schrieb Michal Privoznik >>>> <mprivozn@redhat.com>: >>>>> On 03.08.2016 21:17, Jonatan Schlag wrote: >>>>>> Hi, >>>>>> I have a very strange problem with libvirt. I work on some >>>>>> machines >>>>>> with >>>>>> libvirt (Debian/ Arch Linux) and libvirt set the ownership of >>>>>> images >>>>>> file automatically to the qemu user / group for example on >>>>>> Arch >>>>>> Linux to >>>>>> nobody:kvm. >>>>>> So when I copy an image file with root and use I then with >>>>>> qemu, >>>>>> libvirt >>>>>> change the owner/ group to nobody:kvm. >>>>>> >>>>>> But I also compiled libvirt for a machine (gcc 4.9.4 glibc >>>>>> 2.12) >>>>>> and on >>>>>> this machine libvirt did not change the ownership of the >>>>>> image files >>>>>> which results in this error: >>>>>> >>>>>> libvirtError: internal error: process exited while connecting >>>>>> to >>>>>> monitor: able-ticketing,seamless-migration=on -device >>>>>> >>>>>> >>>>>> >>>>>> qxl-vga,id=video0,ram_size=67108864,vram_size=67108864,vram64_size_mb=0,vgamem_mb=16,bus=pci.0,addr=0x2 >>>>>> >>>>>> >>>>>> -device intel-hda,id=sound0,bus=pci.0,addr=0x4 -device >>>>>> hda-duplex,id=sound0-codec0,bus=sound0.0,cad=0 -chardev >>>>>> spicevmc,id=charredir0,name=usbredir -device >>>>>> usb-redir,chardev=charredir0,id=redir0 -device >>>>>> virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x7 -msg >>>>>> timestamp=on >>>>>> 2016-08-03T18:19:47.494512Z qemu-system-x86_64: -drive >>>>>> >>>>>> >>>>>> >>>>>> file=/data/hdd1/libvirt/images/test.img,format=raw,if=none,id=drive-virtio-disk0: >>>>>> >>>>>> >>>>>> Could not open '/data/hdd1/libvirt/images/test.img': >>>>>> Permission >>>>>> denied >>>>> >>>>> Can you please share the debug logs? >>>>> >>>>> http://wiki.libvirt.org/page/DebugLogs >>>>> >>>>> Also, my initial suspect, before diving any deeper is that >>>>> usually, >>>>> when >>>>> users compile libvirt on their own, they forget to set the >>>>> correct >>>>> prefix, therefore libvirt is looking for its config files NOT >>>>> under >>>>> /etc/libvirt but /usr/local/etc/ or whatever. >>>>> >>>>> BTW: is the daemon running under root? >>>>> >>>>> Michal >>>> >>>> Hi, >>>> >>>> The daemon runs under root. >>>> >>>> I uploaded the debug logs to: >>>> >>>> http://people.ipfire.org/~jschlag/1363864/1_libvirtd.log >>>> >>>> The UID of the user nobody is 99, the GID of the group kvm is >>>> 1011. >>>> >>>> I added my configure options to the bug report. >>>> >>>> Following the log the ownership is changed but why is the file >>>> still >>>> owned by root:root? >>> >>> Right. the file is set ownership. >> But the file ist still owned by root:root and so it is not >> accessable by >> qemu as nobody:kvm. In the moment the only possible way is that the >> change of the ownership fail, but then there should be an error >> message, >> but there is no error message in the log. > > Then the other option that comes to my mind is a race with somebody > else > on the system. You can attach gdb to the daemon and set breakpoint to > virSecurityDACSetOwnershipInternal(). In the arguments you should see > the path eventually among with uid:gid. > > BTW: what's the domain XML?What did you need the xml file the domain is called test and the image file is /data/hdd1/libvirt/images/test.img> > > MichalI did together with Michael Tremer some debugging and Michael posted our results in the bugtracker. So it seems that the chown function is not executed, because a other function return a wrong value. Maybe the describtion in the bug report hepls to go furhter, when not say what you need (logs) to debug the problem. Regards Jonatan