Hi experts, The libvirtd can’t start on my server after the server interruption of power supply, the status is below: [root@openstack3 libvirt]# service libvirtd status Redirecting to /bin/systemctl status libvirtd.service libvirtd.service - Virtualization daemon Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled) Active: failed (Result: signal) since Tue 2015-03-24 16:10:31 CST; 7min ago Process: 27372 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=killed, signal=SEGV) Main PID: 27372 (code=killed, signal=SEGV) Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: 27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: 27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: 27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: 27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: 27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: 27383: debug : virStateInitialize:847 : Running global init for netcf state driver Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: 27383: debug : virObjectNew:199 : OBJECT_NEW: obj=0x7f73f0080d00 classname=virNetcfDriverState Mar 24 16:10:31 openstack3 libvirtd[27372]: ====== end of log ====Mar 24 16:10:31 openstack3 systemd[1]: libvirtd.service: main process exited, code=killed, status=11/SEGV Mar 24 16:10:31 openstack3 systemd[1]: Unit libvirtd.service entered failed state. And the attachment is the log after restart the libvirtd. After long time search on google, I didn’t get any useful information, so anyone can give me some hint on this issue? -BR, Andy
On Tue, Mar 24, 2015 at 04:29:07PM +0800, 梅磊 wrote:>Hi experts, > The libvirtd can’t start on my server after the server interruption >of power supply, the status is below: > >[root@openstack3 libvirt]# service libvirtd status >Redirecting to /bin/systemctl status libvirtd.service >libvirtd.service - Virtualization daemon > Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled) > Active: failed (Result: signal) since Tue 2015-03-24 16:10:31 CST; 7min >ago > Process: 27372 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS (code=killed, >signal=SEGV) > Main PID: 27372 (code=killed, signal=SEGV) > >Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >27383: debug : virFileClose:90 : Closed fd 22 >Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >27383: debug : virFileClose:90 : Closed fd 22 >Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >27383: debug : virFileClose:90 : Closed fd 22 >Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >27383: debug : virFileClose:90 : Closed fd 22 >Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: >27383: debug : virFileClose:90 : Closed fd 22 >Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: >27383: debug : virStateInitialize:847 : Running global init for netcf state >driver >Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: >27383: debug : virObjectNew:199 : OBJECT_NEW: obj=0x7f73f0080d00 >classname=virNetcfDriverState >Mar 24 16:10:31 openstack3 libvirtd[27372]: ====== end of log ====>Mar 24 16:10:31 openstack3 systemd[1]: libvirtd.service: main process >exited, code=killed, status=11/SEGV >Mar 24 16:10:31 openstack3 systemd[1]: Unit libvirtd.service entered failed >state. > >And the attachment is the log after restart the libvirtd. > >After long time search on google, I didn’t get any useful information, so >anyone can give me some hint on this issue? >What did you search for? According to the log: abrt-hook-ccpp: Saved core dump of pid 28928 (/usr/sbin/libvirtd) to /var/tmp/abrt/ccpp-2015-03-24-16:19:22-28928 (97218560 bytes) libvirtd has segfaulted and you have the coredump. Use gdb to open it: # gdb -c /var/tmp/abrt/ccpp-2015-03-24-16:19:22-28928 and then look at what happened, if you can't make anything of it, post it as an attachment here or create a bug from it: (gdb) bt (gdb) t a a bt full>-BR, >Andy >>_______________________________________________ >libvirt-users mailing list >libvirt-users@redhat.com >https://www.redhat.com/mailman/listinfo/libvirt-users
On 3/24/2015 3:29 AM, 梅磊 wrote:> Hi experts, > The libvirtd can’t start on my server after the server interruption > of power supply, the status is below:May not be related at all but a couple of weeks ago I couldn't get libvirtd started after it had been running fine for a while and it was because /etc/locale.gen had been changed at some point. Unfortunately I didn't keep any logs of it, fixed it and moved on. So a quick check here would be to make sure the following doesn't return an empty value assuming you have a /etc/locale.gen of course. grep -v "^ *\(#.*\)*$" /etc/locale.gen
My environment is Centos 7 , no locale.gen exist on my server. Thanks for your reply anyway. Thanks, Andy -----邮件原件----- 发件人: libvirt-users-bounces@redhat.com [mailto:libvirt-users-bounces@redhat.com] 代表 Mr Queue 发送时间: 26 March 2015 09:01 收件人: libvirt-users@redhat.com 主题: Re: [libvirt-users] libvirtd can't start On 3/24/2015 3:29 AM, 梅磊 wrote:> Hi experts, > The libvirtd can’t start on my server after the server interruption > of power supply, the status is below:May not be related at all but a couple of weeks ago I couldn't get libvirtd started after it had been running fine for a while and it was because /etc/locale.gen had been changed at some point. Unfortunately I didn't keep any logs of it, fixed it and moved on. So a quick check here would be to make sure the following doesn't return an empty value assuming you have a /etc/locale.gen of course. grep -v "^ *\(#.*\)*$" /etc/locale.gen _______________________________________________ libvirt-users mailing list libvirt-users@redhat.com https://www.redhat.com/mailman/listinfo/libvirt-users
On Thu, Mar 26, 2015 at 02:55:04PM +0800, 梅磊 wrote:>Thanks for your reply, Martin, I have read the coredump, but I am not familiar with libvirt, so could you help me to figure it out why it crash? >Attachment is the coredump gdb output and all the generate files when libvirtd stop. >I forgot to ask you to install debuginfo packages to further investigate. But looking at the log only netcf-debuginfo is needed. Could you install it and then run the "t a a bt f" again, please? From the current logs I can only guess that there is some network XML mangled and netcf can't read it. When you're power went out, as you said, is it possible you were updating something in the meantime? Try running "rpm -Va" to see if there's something broken in the system.>And Do you think this issue is the same issue with the below root cause? >https://bugzilla.redhat.com/show_bug.cgi?id=1179981 >It is not, that is caused by something else.> > > >-Best Regards, >Andy > > >-----邮件原件----- >发件人: Martin Kletzander [mailto:mkletzan@redhat.com] >发送时间: 24 March 2015 21:20 >收件人: 梅磊 >抄送: libvirt-users@redhat.com >主题: Re: [libvirt-users] libvirtd can't start > >On Tue, Mar 24, 2015 at 04:29:07PM +0800, 梅磊 wrote: >>Hi experts, >> The libvirtd can’t start on my server after the server interruption of >>power supply, the status is below: >> >>[root@openstack3 libvirt]# service libvirtd status Redirecting to >>/bin/systemctl status libvirtd.service libvirtd.service - >>Virtualization daemon >> Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled) >> Active: failed (Result: signal) since Tue 2015-03-24 16:10:31 CST; >>7min ago >> Process: 27372 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS >>(code=killed, >>signal=SEGV) >> Main PID: 27372 (code=killed, signal=SEGV) >> >>Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: >>27383: debug : virStateInitialize:847 : Running global init for netcf >>state driver Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 >>08:10:31.413+0000: >>27383: debug : virObjectNew:199 : OBJECT_NEW: obj=0x7f73f0080d00 >>classname=virNetcfDriverState Mar 24 16:10:31 openstack3 >>libvirtd[27372]: ====== end of log ===== Mar 24 16:10:31 openstack3 >>systemd[1]: libvirtd.service: main process exited, code=killed, >>status=11/SEGV Mar 24 16:10:31 openstack3 systemd[1]: Unit >>libvirtd.service entered failed state. >> >>And the attachment is the log after restart the libvirtd. >> >>After long time search on google, I didn’t get any useful information, >>so anyone can give me some hint on this issue? >> > >What did you search for? > >According to the log: >abrt-hook-ccpp: Saved core dump of pid 28928 (/usr/sbin/libvirtd) to >/var/tmp/abrt/ccpp-2015-03-24-16:19:22-28928 (97218560 bytes) > >libvirtd has segfaulted and you have the coredump. Use gdb to open >it: ># gdb -c /var/tmp/abrt/ccpp-2015-03-24-16:19:22-28928 > >and then look at what happened, if you can't make anything of it, post it as an attachment here or create a bug from it: > >(gdb) bt >(gdb) t a a bt full > >>-BR, >>Andy >> > > >>_______________________________________________ >>libvirt-users mailing list >>libvirt-users@redhat.com >>https://www.redhat.com/mailman/listinfo/libvirt-users
Hi Martin: Actually, I have installed all the debuginfo for this coredump file. But the netcf debuginfo report "mismatch" as below: warning: the debug information found in "/usr/lib/debug/usr/lib64/libnetcf.so.1.4.0.debug" does not match "/lib64/libnetcf.so.1" (CRC mismatch). So I ignore this warning before. You can check what I did on the attachment "netcf_debuginfo_install.log". Another attachment is the result of "rpm -Va" to check all the system config files status, I didn't find anything wrong. Thanks for your help! -Best Regards, Andy -----邮件原件----- 发件人: Martin Kletzander [mailto:mkletzan@redhat.com] 发送时间: 26 March 2015 15:54 收件人: 梅磊 抄送: libvirt-users@redhat.com 主题: Re: 答复: [libvirt-users] libvirtd can't start On Thu, Mar 26, 2015 at 02:55:04PM +0800, 梅磊 wrote:>Thanks for your reply, Martin, I have read the coredump, but I am not familiar with libvirt, so could you help me to figure it out why it crash? >Attachment is the coredump gdb output and all the generate files when libvirtd stop. >I forgot to ask you to install debuginfo packages to further investigate. But looking at the log only netcf-debuginfo is needed. Could you install it and then run the "t a a bt f" again, please?>From the current logs I can only guess that there is some network XML mangled and netcf can't read it. When you're power went out, as you said, is it possible you were updating something in the meantime? Try running "rpm -Va" to see if there's something broken in the system.>And Do you think this issue is the same issue with the below root cause? >https://bugzilla.redhat.com/show_bug.cgi?id=1179981 >It is not, that is caused by something else.> > > >-Best Regards, >Andy > > >-----邮件原件----- >发件人: Martin Kletzander [mailto:mkletzan@redhat.com] >发送时间: 24 March 2015 21:20 >收件人: 梅磊 >抄送: libvirt-users@redhat.com >主题: Re: [libvirt-users] libvirtd can't start > >On Tue, Mar 24, 2015 at 04:29:07PM +0800, 梅磊 wrote: >>Hi experts, >> The libvirtd can’t start on my server after the server interruption >>of power supply, the status is below: >> >>[root@openstack3 libvirt]# service libvirtd status Redirecting to >>/bin/systemctl status libvirtd.service libvirtd.service - >>Virtualization daemon >> Loaded: loaded (/usr/lib/systemd/system/libvirtd.service; enabled) >> Active: failed (Result: signal) since Tue 2015-03-24 16:10:31 CST; >>7min ago >> Process: 27372 ExecStart=/usr/sbin/libvirtd $LIBVIRTD_ARGS >>(code=killed, >>signal=SEGV) >> Main PID: 27372 (code=killed, signal=SEGV) >> >>Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.412+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: >>27383: debug : virFileClose:90 : Closed fd 22 Mar 24 16:10:31 >>openstack3 libvirtd[27372]: 2015-03-24 08:10:31.413+0000: >>27383: debug : virStateInitialize:847 : Running global init for netcf >>state driver Mar 24 16:10:31 openstack3 libvirtd[27372]: 2015-03-24 >>08:10:31.413+0000: >>27383: debug : virObjectNew:199 : OBJECT_NEW: obj=0x7f73f0080d00 >>classname=virNetcfDriverState Mar 24 16:10:31 openstack3 >>libvirtd[27372]: ====== end of log ===== Mar 24 16:10:31 openstack3 >>systemd[1]: libvirtd.service: main process exited, code=killed, >>status=11/SEGV Mar 24 16:10:31 openstack3 systemd[1]: Unit >>libvirtd.service entered failed state. >> >>And the attachment is the log after restart the libvirtd. >> >>After long time search on google, I didn’t get any useful information, >>so anyone can give me some hint on this issue? >> > >What did you search for? > >According to the log: >abrt-hook-ccpp: Saved core dump of pid 28928 (/usr/sbin/libvirtd) to >/var/tmp/abrt/ccpp-2015-03-24-16:19:22-28928 (97218560 bytes) > >libvirtd has segfaulted and you have the coredump. Use gdb to open >it: ># gdb -c /var/tmp/abrt/ccpp-2015-03-24-16:19:22-28928 > >and then look at what happened, if you can't make anything of it, post it as an attachment here or create a bug from it: > >(gdb) bt >(gdb) t a a bt full > >>-BR, >>Andy >> > > >>_______________________________________________ >>libvirt-users mailing list >>libvirt-users@redhat.com >>https://www.redhat.com/mailman/listinfo/libvirt-users