Hi,
I''m running two dom0s between which I (live) migrate domUs.
Sometimes. Other times the migration seems to succeed, but the
migrated domU is never unpaused, as the vif devices does not appear on
the receiving host:
[2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Memory reloaded (1523
pages)
[2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Domain ready to be built.
[2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Restore exit with rc=0
[2008-10-13 16:56:21 19638] DEBUG (XendCheckpoint:344) store-mfn 719436
[2008-10-13 16:56:21 19638] DEBUG (XendCheckpoint:344) console-mfn 778540
[2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1838)
XendDomainInfo.completeRestore
[2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1008) Storing domain details:
{''console/ring-ref'': ''778540'',
''image/entry'': ''3224596480'',
''console/port'': ''2'',
''store/ring-ref'': ''719436'',
''image/loader'': ''gene
ric'', ''vm'':
''/vm/15bd1442-affa-1f26-e205-026998b2fddd'',
''control/platform-feature-multiprocessor-suspend'':
''1'', ''image/hv-start-low'':
''4118806528'', ''image/guest-os'':
''linux'', ''image/virt-base'':
''3221225472'', ''m
emory/target'': ''131072'',
''image/guest-version'': ''2.6'',
''image/pae-mode'': ''yes'',
''console/limit'': ''1048576'',
''image/hypercall-page'': ''3222278144'',
''image/suspend-cancel'': ''1'',
''cpu/0/availability'': ''online'', ''im
age/features/pae-pgdir-above-4gb'': ''1'',
''image/features/writable-page-tables'': ''0'',
''console/type'': ''xenconsoled'',
''name'': ''service.grid'',
''domid'': ''5'',
''image/xen-version'': ''xen-3.0'',
''store/port'': ''1''}
[2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1851)
XendDomainInfo.completeRestore done
[2008-10-13 16:56:21 19638] DEBUG (DevController:151) Waiting for devices vif.
[2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1092)
XendDomainInfo.handleShutdownWatch
[2008-10-13 16:56:21 19638] DEBUG (DevController:156) Waiting for 0.
[2008-10-13 16:56:21 19638] DEBUG (DevController:595) hotplugStatusCallback
/local/domain/0/backend/vif/5/0/hotplug-status.
[2008-10-13 16:58:01 19638] ERROR (XendCheckpoint:297) Device 0 (vif) could not
be connected. Hotplug scripts not working.
Traceback (most recent call last):
File
"/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendCheckpoint.py",
line 295, in restore
dominfo.waitForDevices() # Wait for backends to set up
File
"/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendDomainInfo.py",
line 586, in waitForDevices
self.getDeviceController(devclass).waitForDevices()
File
"/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/server/DevController.py",
line 152, in waitForDevices
return map(self.waitForDevice, self.deviceIDs())
File
"/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/server/DevController.py",
line 167, in waitForDevice
(devid, self.deviceClass))
VmError: Device 0 (vif) could not be connected. Hotplug scripts not working.
Bummer. If I reboot the receiving dom0, the same migration succeeds
without any problem. How could I further debug this problem? It used
to work, but since then lots of software have been updated on the dom0
(like udev for example). The hypervisors are 3.2.1, the dom0 kernels
are 2.6.18. The same problems exists with straight domain creation,
too: when the migration fails, I can''t even create domains on that
dom0, only after reboot...
--
I''d be grateful for any idea.
Thanks,
Feri.
_______________________________________________
Xen-users mailing list
Xen-users@lists.xensource.com
http://lists.xensource.com/xen-users
On Di, Okt 14, 2008 at 12:18:54 +0200, Ferenc Wagner wrote:> Hi, > > I''m running two dom0s between which I (live) migrate domUs. > Sometimes. Other times the migration seems to succeed, but the > migrated domU is never unpaused, as the vif devices does not appear on > the receiving host: > > [2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Memory reloaded (1523 pages) > [2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Domain ready to be built. > [2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Restore exit with rc=0 > [2008-10-13 16:56:21 19638] DEBUG (XendCheckpoint:344) store-mfn 719436 > [2008-10-13 16:56:21 19638] DEBUG (XendCheckpoint:344) console-mfn 778540 > [2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1838) XendDomainInfo.completeRestore > [2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1008) Storing domain details: {''console/ring-ref'': ''778540'', ''image/entry'': ''3224596480'', ''console/port'': ''2'', ''store/ring-ref'': ''719436'', ''image/loader'': ''gene > ric'', ''vm'': ''/vm/15bd1442-affa-1f26-e205-026998b2fddd'', ''control/platform-feature-multiprocessor-suspend'': ''1'', ''image/hv-start-low'': ''4118806528'', ''image/guest-os'': ''linux'', ''image/virt-base'': ''3221225472'', ''m > emory/target'': ''131072'', ''image/guest-version'': ''2.6'', ''image/pae-mode'': ''yes'', ''console/limit'': ''1048576'', ''image/hypercall-page'': ''3222278144'', ''image/suspend-cancel'': ''1'', ''cpu/0/availability'': ''online'', ''im > age/features/pae-pgdir-above-4gb'': ''1'', ''image/features/writable-page-tables'': ''0'', ''console/type'': ''xenconsoled'', ''name'': ''service.grid'', ''domid'': ''5'', ''image/xen-version'': ''xen-3.0'', ''store/port'': ''1''} > [2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1851) XendDomainInfo.completeRestore done > [2008-10-13 16:56:21 19638] DEBUG (DevController:151) Waiting for devices vif. > [2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1092) XendDomainInfo.handleShutdownWatch > [2008-10-13 16:56:21 19638] DEBUG (DevController:156) Waiting for 0. > [2008-10-13 16:56:21 19638] DEBUG (DevController:595) hotplugStatusCallback /local/domain/0/backend/vif/5/0/hotplug-status. > [2008-10-13 16:58:01 19638] ERROR (XendCheckpoint:297) Device 0 (vif) could not be connected. Hotplug scripts not working. > Traceback (most recent call last): > File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendCheckpoint.py", line 295, in restore > dominfo.waitForDevices() # Wait for backends to set up > File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 586, in waitForDevices > self.getDeviceController(devclass).waitForDevices() > File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/server/DevController.py", line 152, in waitForDevices > return map(self.waitForDevice, self.deviceIDs()) > File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/server/DevController.py", line 167, in waitForDevice > (devid, self.deviceClass)) > VmError: Device 0 (vif) could not be connected. Hotplug scripts not working. > > Bummer. If I reboot the receiving dom0, the same migration succeeds > without any problem. How could I further debug this problem? It used > to work, but since then lots of software have been updated on the dom0 > (like udev for example). The hypervisors are 3.2.1, the dom0 kernels > are 2.6.18. The same problems exists with straight domain creation, > too: when the migration fails, I can''t even create domains on that > dom0, only after reboot...What does show the command brctl show on the host in that moment?> -- > I''d be grateful for any idea. > Thanks, > Feri. > > _______________________________________________ > Xen-users mailing list > Xen-users@lists.xensource.com > http://lists.xensource.com/xen-users-- WBR, i.m.chubin _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users
Igor Chubin <igor@chub.in> writes:> On Di, Okt 14, 2008 at 12:18:54 +0200, Ferenc Wagner wrote: > >> I''m running two dom0s between which I (live) migrate domUs. >> Sometimes. Other times the migration seems to succeed, but the >> migrated domU is never unpaused, as the vif devices does not appear on >> the receiving host: >> >> [2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Memory reloaded (1523 pages) >> [2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Domain ready to be built. >> [2008-10-13 16:56:21 19638] INFO (XendCheckpoint:373) Restore exit with rc=0 >> [2008-10-13 16:56:21 19638] DEBUG (XendCheckpoint:344) store-mfn 719436 >> [2008-10-13 16:56:21 19638] DEBUG (XendCheckpoint:344) console-mfn 778540 >> [2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1838) XendDomainInfo.completeRestore >> [2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1008) Storing domain details: {''console/ring-ref'': ''778540'', ''image/entry'': ''3224596480'', ''console/port'': ''2'', ''store/ring-ref'': ''719436'', ''image/loader'': ''gene >> ric'', ''vm'': ''/vm/15bd1442-affa-1f26-e205-026998b2fddd'', ''control/platform-feature-multiprocessor-suspend'': ''1'', ''image/hv-start-low'': ''4118806528'', ''image/guest-os'': ''linux'', ''image/virt-base'': ''3221225472'', ''m >> emory/target'': ''131072'', ''image/guest-version'': ''2.6'', ''image/pae-mode'': ''yes'', ''console/limit'': ''1048576'', ''image/hypercall-page'': ''3222278144'', ''image/suspend-cancel'': ''1'', ''cpu/0/availability'': ''online'', ''im >> age/features/pae-pgdir-above-4gb'': ''1'', ''image/features/writable-page-tables'': ''0'', ''console/type'': ''xenconsoled'', ''name'': ''service.grid'', ''domid'': ''5'', ''image/xen-version'': ''xen-3.0'', ''store/port'': ''1''} >> [2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1851) XendDomainInfo.completeRestore done >> [2008-10-13 16:56:21 19638] DEBUG (DevController:151) Waiting for devices vif. >> [2008-10-13 16:56:21 19638] DEBUG (XendDomainInfo:1092) XendDomainInfo.handleShutdownWatch >> [2008-10-13 16:56:21 19638] DEBUG (DevController:156) Waiting for 0. >> [2008-10-13 16:56:21 19638] DEBUG (DevController:595) hotplugStatusCallback /local/domain/0/backend/vif/5/0/hotplug-status. >> [2008-10-13 16:58:01 19638] ERROR (XendCheckpoint:297) Device 0 (vif) could not be connected. Hotplug scripts not working. >> Traceback (most recent call last): >> File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendCheckpoint.py", line 295, in restore >> dominfo.waitForDevices() # Wait for backends to set up >> File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/XendDomainInfo.py", line 586, in waitForDevices >> self.getDeviceController(devclass).waitForDevices() >> File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/server/DevController.py", line 152, in waitForDevices >> return map(self.waitForDevice, self.deviceIDs()) >> File "/usr/lib/xen-3.2-1/bin/../lib/python/xen/xend/server/DevController.py", line 167, in waitForDevice >> (devid, self.deviceClass)) >> VmError: Device 0 (vif) could not be connected. Hotplug scripts not working. >> >> Bummer. If I reboot the receiving dom0, the same migration succeeds >> without any problem. How could I further debug this problem? It used >> to work, but since then lots of software have been updated on the dom0 >> (like udev for example). The hypervisors are 3.2.1, the dom0 kernels >> are 2.6.18. The same problems exists with straight domain creation, >> too: when the migration fails, I can''t even create domains on that >> dom0, only after reboot... > > What does show the command > brctl show > on the host in that moment?I''ll check that next time. However, I''m using a static bridge setup, so the bridges are always up: bridge name bridge id STP enabled interfaces br891 8000.00065b8e7272 no vlan891 br894 8000.00065b8e7272 no vlan894 br897 8000.00065b8e7272 no vlan897 Xen doesn''t touch them except for adding the vifs. I forgot to state that in xend-config.sxp (network-script network-dummy) (vif-script vif-bridge) What seems more relevant is that udevmonitor doesn''t say a word in the failing case, though usually it reports lots of events (amongst them vif additions) on domU creation. This suggests some hotplug problem. -- Thanks, Feri. _______________________________________________ Xen-users mailing list Xen-users@lists.xensource.com http://lists.xensource.com/xen-users