Manuel Ullmann
2016-Dec-13 22:05 UTC
[libvirt-users] virsh not detecting hugepage mount; disabled by config?
Hi, I’m struggling with virsh not detecting my hugepage mount. I have the following kernel command line: BOOT_IMAGE=/vmlinuz-4.8.13-gentoo root=/dev/mapper/gensd-gentoo ro quiet splash intel_iommu=on video=efifb:off,vesafb:off,simplefb:off splash=verbose,theme:livedvd-aurora kvm.ignore_msrs=1 transparent_hugepage=never hugepages=3072 softlevel=qemuvm My startup script outputs the following: hugetlbfs /var/lib/hugetlbfs hugetlbfs rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 0 0 [2016.12.13 22:22:50 virsh 2808] ERROR Failed to start domain win10 [2016.12.13 22:22:50 virsh 2808] ERROR internal error: hugetlbfs filesystem is not mounted or disabled by administrator config virsh was unsuccessful So hugetlbfs is definitively mounted, but virsh does either not detect it or some magical option, that I’ve not found in a day searching for it, disables it (the documentation could explain that better). Since the Ubuntu guys refer to the KVM_HUGEPAGES environment, I tried adding it to start-stop-daemon environment, but it seems non-relevant. 77 is the qemu user id, but I’m quite sure the permission issues were excluded (tried root permissions as well). Besides I had permission errors occasionally occasionally and don’t have them anymore. If you could provide a hint to the magical administrator configuration option, that would be helpful. Tried hugeadm as well, pointing the config to the correct destination. Thanks in advance, Manuel PS: The virsh script reads like this: #!/bin/sh cmdline="$(cat /proc/cmdline)" if [[ "${cmdline##* }" == "softlevel=qemuvm" ]]; then mount -o rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 -t hugetlbfs hugetlbfs /var/lib/hugetlbfs sysctl kernel.shmmax=6442450944 cat /proc/mounts | grep hugetlb >> /var/log/virsh.log sed -i -e '/^hugetlb/{s/^/\#/}' /etc/libvirt/qemu.conf counter=0 while [ "${counter}" -lt 10 && ! -S /var/run/libvirt/libvirt-sock ]; do sleep 1 counter=$(( ++counter )) done if [ $counter -gt 9 ]; then echo "libvirtd socket generation timed out" >> /var/log/virsh.log fi if pidof libvirtd &>/dev/null; then LC_ALL=C /usr/bin/virsh -l /var/log/virsh.log start win10 if [ $? -gt 0 ]; then echo "virsh was unsuccessful" >> /var/log/virsh.log reboot fi else echo "libvirtd is not started" >> /var/log/virsh.log reboot fi fi reboot is more convenient, since this is a vfio-igd passthrough vm.
Martin Kletzander
2016-Dec-14 09:32 UTC
Re: [libvirt-users] virsh not detecting hugepage mount; disabled by config?
On Tue, Dec 13, 2016 at 11:05:11PM +0100, Manuel Ullmann wrote:>Hi, > >I’m struggling with virsh not detecting my hugepage mount. > >I have the following kernel command line: >BOOT_IMAGE=/vmlinuz-4.8.13-gentoo root=/dev/mapper/gensd-gentoo ro quiet >splash intel_iommu=on video=efifb:off,vesafb:off,simplefb:off >splash=verbose,theme:livedvd-aurora kvm.ignore_msrs=1 >transparent_hugepage=never hugepages=3072 softlevel=qemuvm > >My startup script outputs the following: >hugetlbfs /var/lib/hugetlbfs hugetlbfs >rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 0 0 >[2016.12.13 22:22:50 virsh 2808] ERROR Failed to start domain win10 >[2016.12.13 22:22:50 virsh 2808] ERROR internal error: hugetlbfs >filesystem is not mounted or disabled by administrator config >virsh was unsuccessful > >So hugetlbfs is definitively mounted, but virsh does either not detect >it or some magical option, that I’ve not found in a day searching for >it, disables it (the documentation could explain that better). Since the >Ubuntu guys refer to the KVM_HUGEPAGES environment, I tried adding it to >start-stop-daemon environment, but it seems non-relevant. 77 is the qemu >user id, but I’m quite sure the permission issues were excluded (tried >root permissions as well). Besides I had permission errors occasionally >occasionally and don’t have them anymore. If you could provide a hint to >the magical administrator configuration option, that would be helpful.Nothing magical about that. If you have no hugepages configured in qemu.conf, libvirtd will *try* to find all the mounts if running as root. The only thing is that in the error message "or" should be changed to "and", I guess.>Tried hugeadm as well, pointing the config to the correct destination. > >Thanks in advance, >Manuel > >PS: >The virsh script reads like this: >#!/bin/sh >cmdline="$(cat /proc/cmdline)" >if [[ "${cmdline##* }" == "softlevel=qemuvm" ]]; then > mount -o rw,relatime,pagesize=2097152,uid=77,gid=77,mode=0770 -t >hugetlbfs hugetlbfs /var/lib/hugetlbfsIf libvirtd starts before this ^^> sysctl kernel.shmmax=6442450944 > cat /proc/mounts | grep hugetlb >> /var/log/virsh.log > sed -i -e '/^hugetlb/{s/^/\#/}' /etc/libvirt/qemu.confor this ^^ is done, then the settings are not reflected. That's the only thing I could glean from the quick look. I bet adding 'service libvirtd restart' here helps.> counter=0 > while [ "${counter}" -lt 10 && ! -S /var/run/libvirt/libvirt-sock ]; do > sleep 1 > counter=$(( ++counter )) > done > if [ $counter -gt 9 ]; then > echo "libvirtd socket generation timed out" >> /var/log/virsh.log > fi > if pidof libvirtd &>/dev/null; then > LC_ALL=C /usr/bin/virsh -l /var/log/virsh.log start win10 > if [ $? -gt 0 ]; then > echo "virsh was unsuccessful" >> /var/log/virsh.log > reboot > fi > else > echo "libvirtd is not started" >> /var/log/virsh.log > reboot > fi >fi > >reboot is more convenient, since this is a vfio-igd passthrough vm. > > >_______________________________________________ >libvirt-users mailing list >libvirt-users@redhat.com >https://www.redhat.com/mailman/listinfo/libvirt-users
Seemingly Similar Threads
- Re: virsh not detecting hugepage mount; disabled by config?
- about sharing the hugepage memory segment between the host and the container
- HugePages - can't start guest that requires them
- Re: HugePages - can't start guest that requires them
- PATCH: Hugepage support for Domains booting with 4KB pages