Peter Peltonen
2008-Jan-05 16:46 UTC
[Xen-users] after centos 5 yum update xen guest won''t boot
After a yum update my two guests that had been shutdown do not start any more. I have two other guests running that seem to work okay but I am now afraid that they won''t come up again if I restart them. I am running CentOS5 with kernel 2.6.18-8.1.8.el5xen. Previously I had following xen versions installed: Oct 14 23:45:10 Updated: xen-libs.i386 3.0.3-25.0.4.el5 Oct 14 23:46:05 Updated: xen.i386 3.0.3-25.0.4.el5 After a yum update I now have: Jan 03 22:22:59 Installed: kernel-xen.i686 2.6.18-53.1.4.el5 Jan 03 22:23:55 Updated: xen.i386 3.0.3-41.el5 I have now two guests that won''t boot. This is what I get after issuing `xm create web04`: Using config file "/etc/xen/web04". Error: destroyDevice() takes exactly 3 arguments (2 given) My config file looks likes this: name = "web04" memory = "512" disk = [ ''phy:/dev/VolGroup00/LogVol03-web04,xvda,w'', ] vif = [ ''mac=e9:00:36:fe:dc:e0, bridge=xenbr0'', ] vfb = ["type=vnc,vncunused=1"] #uuid = "cc64dbee-64f6-4f6b-9d3f-fc3e185206bc" bootloader="/usr/bin/pygrub" cpus = "^0,1" # run only on cpu 1 vcpus=1 on_reboot = ''restart'' on_crash = ''restart'' This is what comes to /var/log/xen/xend.log: [2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:190) XendDo mainInfo.create([''vm'', [''name'', ''web04''], [''memory'', ''512''], [''on_reboot'', ''rest art''], [''on_crash'', ''restart''], [''vcpus'', 1], [''cpus'', ''^0,1''], [''bootloader'', '' /usr/bin/pygrub''], [''image'', [''linux'', [''ramdisk'', ''/var/lib/xen/boot_ramdisk.2C 2kAy''], [''kernel'', ''/var/lib/xen/boot_kernel.Tg4J_f''], [''args'', ''ro root=/dev/Vo lGroup00/LogVol00'']]], [''device'', [''vbd'', [''uname'', ''phy:/dev/VolGroup00/LogVol0 3-web04''], [''dev'', ''xvda''], [''mode'', ''w'']]], [''device'', [''vif'', [''bridge'', ''xenb r0''], [''mac'', ''e9:00:36:fe:dc:e0'']]], [''device'', [''vkbd'']], [''device'', [''vfb'', [ ''vncunused'', ''1''], [''type'', ''vnc''], [''xauthority'', ''/root/.Xauthority'']]]]) [2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:296) parseC onfig: config is [''vm'', [''name'', ''web04''], [''memory'', ''512''], [''on_reboot'', ''res tart''], [''on_crash'', ''restart''], [''vcpus'', 1], [''cpus'', ''^0,1''], [''bootloader'', ''/usr/bin/pygrub''], [''image'', [''linux'', [''ramdisk'', ''/var/lib/xen/boot_ramdisk.2 C2kAy''], [''kernel'', ''/var/lib/xen/boot_kernel.Tg4J_f''], [''args'', ''ro root=/dev/V olGroup00/LogVol00'']]], [''device'', [''vbd'', [''uname'', ''phy:/dev/VolGroup00/LogVol 03-web04''], [''dev'', ''xvda''], [''mode'', ''w'']]], [''device'', [''vif'', [''bridge'', ''xen br0''], [''mac'', ''e9:00:36:fe:dc:e0'']]], [''device'', [''vkbd'']], [''device'', [''vfb'', [''vncunused'', ''1''], [''type'', ''vnc''], [''xauthority'', ''/root/.Xauthority'']]]] [2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:397) parseC onfig: result is {''shadow_memory'': None, ''start_time'': None, ''uuid'': None, ''on_c rash'': ''restart'', ''on_reboot'': ''restart'', ''localtime'': None, ''image'': [''linux'', [''ramdisk'', ''/var/lib/xen/boot_ramdisk.2C2kAy''], [''kernel'', ''/var/lib/xen/boot_k ernel.Tg4J_f''], [''args'', ''ro root=/dev/VolGroup00/LogVol00'']], ''on_poweroff'': No ne, ''bootloader_args'': None, ''cpus'': [1], ''name'': ''web04'', ''backend'': [], ''vcpus '': 1, ''cpu_weight'': None, ''features'': None, ''vcpu_avail'': None, ''memory'': 512, '' device'': [(''vbd'', [''vbd'', [''uname'', ''phy:/dev/VolGroup00/LogVol03-web04''], [''dev '', ''xvda''], [''mode'', ''w'']]), (''vif'', [''vif'', [''bridge'', ''xenbr0''], [''mac'', ''e9:0 0:36:fe:dc:e0'']]), (''vkbd'', [''vkbd'']), (''vfb'', [''vfb'', [''vncunused'', ''1''], [''typ e'', ''vnc''], [''xauthority'', ''/root/.Xauthority'']])], ''bootloader'': ''/usr/bin/pygr ub'', ''cpu'': None, ''maxmem'': None} [2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:1264) XendD omainInfo.construct: None [2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:1296) XendD omainInfo.initDomain: 8 1.0 [2008-01-05 17:59:56 xend 2632] DEBUG (balloon:127) Balloon: 4412080 KiB free; n eed 524288; done. [2008-01-05 17:59:56 xend 2632] INFO (image:136) buildDomain os=linux dom=8 vcpu s=1 [2008-01-05 17:59:56 xend 2632] DEBUG (image:199) dom = 8 [2008-01-05 17:59:56 xend 2632] DEBUG (image:200) image = /var/lib/xen/ boot_kernel.Tg4J_f [2008-01-05 17:59:56 xend 2632] DEBUG (image:201) store_evtchn = 1 [2008-01-05 17:59:56 xend 2632] DEBUG (image:202) console_evtchn = 2 [2008-01-05 17:59:56 xend 2632] DEBUG (image:203) cmdline = ro root=/dev /VolGroup00/LogVol00 [2008-01-05 17:59:56 xend 2632] DEBUG (image:204) ramdisk = /var/lib/xen/ boot_ramdisk.2C2kAy [2008-01-05 17:59:56 xend 2632] DEBUG (image:205) vcpus = 1 [2008-01-05 17:59:56 xend 2632] DEBUG (image:206) features [2008-01-05 17:59:56 xend 2632] DEBUG (blkif:24) exception looking up device num ber for xvda: [Errno 2] No such file or directory: ''/dev/xvda'' [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:110) DevController: writing {''backend-id'': ''0'', ''virtual-device'': ''51712'', ''device-type'': ''disk'', ''state'': ''1'', ''backend'': ''/local/domain/0/backend/vbd/8/51712''} to /local/domain/8/device /vbd/51712. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:112) DevController: writing {''domain'': ''web04'', ''frontend'': ''/local/domain/8/device/vbd/51712'', ''dev'': ''xvd a'', ''state'': ''1'', ''params'': ''/dev/VolGroup00/LogVol03-web04'', ''mode'': ''w'', ''onli ne'': ''1'', ''frontend-id'': ''8'', ''type'': ''phy''} to /local/domain/0/backend/vbd/8/51 712. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:110) DevController: writing {''backend-id'': ''0'', ''mac'': ''e9:00:36:fe:dc:e0'', ''handle'': ''0'', ''state'': ''1'', ''b ackend'': ''/local/domain/0/backend/vif/8/0''} to /local/domain/8/device/vif/0. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:112) DevController: writing {''bridge'': ''xenbr0'', ''domain'': ''web04'', ''handle'': ''0'', ''script'': ''/etc/xen/scri pts/vif-bridge'', ''state'': ''1'', ''frontend'': ''/local/domain/8/device/vif/0'', ''mac'' : ''e9:00:36:fe:dc:e0'', ''online'': ''1'', ''frontend-id'': ''8''} to /local/domain/0/bac kend/vif/8/0. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:110) DevController: writing {''state'': ''1'', ''backend-id'': ''0'', ''backend'': ''/local/domain/0/backend/vkbd/8/0'' } to /local/domain/8/device/vkbd/0. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:112) DevController: writing {''frontend-id'': ''8'', ''domain'': ''web04'', ''frontend'': ''/local/domain/8/device/vkb d/0'', ''state'': ''1'', ''online'': ''1''} to /local/domain/0/backend/vkbd/8/0. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:110) DevController: writing {''state'': ''1'', ''backend-id'': ''0'', ''backend'': ''/local/domain/0/backend/vfb/8/0''} to /local/domain/8/device/vfb/0. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:112) DevController: writing {''vncunused'': ''1'', ''domain'': ''web04'', ''frontend'': ''/local/domain/8/device/vfb/0 '', ''xauthority'': ''/root/.Xauthority'', ''state'': ''1'', ''online'': ''1'', ''frontend-id'' : ''8'', ''type'': ''vnc''} to /local/domain/0/backend/vfb/8/0. [2008-01-05 17:59:56 xend 2632] DEBUG (vfbif:92) No VNC passwd configured for vf b access [2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:715) Storin g VM details: {''shadow_memory'': ''0'', ''uuid'': ''546a262f-1696-181f-6785-8355e6a78e 37'', ''on_reboot'': ''restart'', ''start_time'': ''1199548796.41'', ''on_poweroff'': ''dest roy'', ''name'': ''web04'', ''xend/restart_count'': ''0'', ''vcpus'': ''1'', ''vcpu_avail'': ''1 '', ''memory'': ''512'', ''on_crash'': ''restart'', ''image'': "(linux (ramdisk /var/lib/xe n/boot_ramdisk.2C2kAy) (kernel /var/lib/xen/boot_kernel.Tg4J_f) (args ''ro root=/ dev/VolGroup00/LogVol00''))", ''maxmem'': ''512''} [2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:750) Storin g domain details: {''console/ring-ref'': ''630052'', ''console/port'': ''2'', ''name'': ''w eb04'', ''console/limit'': ''1048576'', ''vm'': ''/vm/546a262f-1696-181f-6785-8355e6a78e 37'', ''domid'': ''8'', ''cpu/0/availability'': ''online'', ''memory/target'': ''524288'', ''s tore/ring-ref'': ''630053'', ''store/port'': ''1''} [2008-01-05 17:59:56 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:940) XendDo mainInfo.handleShutdownWatch [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:143) Waiting for devices vi f. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:149) Waiting for 0. [2008-01-05 17:59:56 xend 2632] DEBUG (DevController:471) hotplugStatusCallback /local/domain/0/backend/vif/8/0/hotplug-status. [2008-01-05 18:01:36 xend 2632] ERROR (xmlrpclib2:184) destroyDevice() takes exa ctly 3 arguments (2 given) Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line 162, in _ marshaled_dispatch response = self._dispatch(method, params) File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 406, in _dispatch return func(*params) File "<string>", line 1, in <lambda> File "/usr/lib/python2.4/site-packages/xen/xend/server/XMLRPCServer.py", line 38, in dispatch return getattr(info, fn)(*args) File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1610, in waitForDevices self.testvifsComplete() File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 1101, in waitForDevices_ self.info.update({''security'': [[''ssidref'', str(info[''ssidref''])]]}) File "/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py", line 145, in waitForDevices return map(self.waitForDevice, self.deviceIDs()) File "/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py", line 154, in waitForDevice self.destroyDevice(devid) TypeError: destroyDevice() takes exactly 3 arguments (2 given) [2008-01-05 18:01:36 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:1463) XendD omainInfo.destroy: domid=8 [2008-01-05 18:01:36 xend.XendDomainInfo 2632] DEBUG (XendDomainInfo:1471) XendD omainInfo.destroyDomain(8) Is anytone else with CentOS 5 experiencing this kind of problems? Should I try downgrading? Regards, Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Nico Kadel-Garcia
2008-Jan-05 17:39 UTC
Re: [Xen-users] after centos 5 yum update xen guest won''t boot
Peter Peltonen wrote:> After a yum update my two guests that had been shutdown do not start > any more. I have two other guests running that seem to work okay but I > am now afraid that they won''t come up again if I restart them. > > I am running CentOS5 with kernel 2.6.18-8.1.8.el5xen. Previously I had > following xen versions installed: > > Oct 14 23:45:10 Updated: xen-libs.i386 3.0.3-25.0.4.el5 > Oct 14 23:46:05 Updated: xen.i386 3.0.3-25.0.4.el5 >Did you reboot with the new kernel and the new Xen? New kernels aren''t running until *AFTER* the reboot. Switching kernels on a live system is an.... old problem that''s never been solved well. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
jim burns
2008-Jan-05 21:21 UTC
Re: [Xen-users] after centos 5 yum update xen guest won''t boot
On Saturday 05 January 2008 11:46:51 am Peter Peltonen wrote:> After a yum update my two guests that had been shutdown do not start > any more. I have two other guests running that seem to work okay but I > am now afraid that they won''t come up again if I restart them. > > I am running CentOS5 with kernel 2.6.18-8.1.8.el5xen. Previously I had > following xen versions installed: > > Oct 14 23:45:10 Updated: xen-libs.i386 3.0.3-25.0.4.el5 > Oct 14 23:46:05 Updated: xen.i386 3.0.3-25.0.4.el5 > > After a yum update I now have: > > Jan 03 22:22:59 Installed: kernel-xen.i686 2.6.18-53.1.4.el5 > Jan 03 22:23:55 Updated: xen.i386 3.0.3-41.el5 > > I have now two guests that won''t boot. This is what I get after > issuing `xm create web04`: > > Using config file "/etc/xen/web04". > Error: destroyDevice() takes exactly 3 arguments (2 given)This error looks like a classic mismatch between xen and some other xen component. I find it curious that your xen.i386 was updated, but your xen-libs was not. (BTW, I''m assuming the yum update was on dom0?) Try poking around the Centos update repo to find an update for xen-libs. (If this was a dom0 update, then updating the kernel-xen shouldn''t affect your domu, since you are loading a kernel on the domu image with pygrub.) _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
jim burns
2008-Jan-06 19:46 UTC
Re: UNS: Re: [Xen-users] after centos 5 yum update xen guest won''t boot
On Sunday 06 January 2008 02:12:30 pm you wrote:> Any other ideas?Short of rebooting (recommended), you could try an ''/etc/init.d/xend restart''. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Peter Peltonen
2008-Jan-14 10:44 UTC
Re: [Xen-users] after centos 5 yum update xen guest won''t boot
On Jan 5, 2008 7:39 PM, Nico Kadel-Garcia <nkadel@gmail.com> wrote:> Did you reboot with the new kernel and the new Xen? New kernels aren''t > running until *AFTER* the reboot. Switching kernels on a live system is > an.... old problem that''s never been solved well.Rebooting the server solved all my problems, thanks. Restarting the xend services did not help. Peter _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users