Hi All,
I tried to migrate VM''s (--hvm as well as --paravirt) from one host to
another host with following settings
1. Both hosts are running same xVM (snv_79)
2. TCP port is open and ready to accept connection
bash-3.2# netstat -a | grep 8002
*.8002 *.* 0 0 49152 0 LISTEN
3. Both hosts are in same subnet
4. NFS shared storage.
Still I am getting errors. Please see /var/log/xen/xend.log output
a. For --paravirt and non-live
********************************************************************************
[2007-12-21 23:28:13 xend 3339] ERROR (XendCheckpoint:111) Save failed
on domain solaris79 (37).
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/xen/xend/XendCheckpoint.py",
line 69, in save
File "/usr/lib/python2.4/site-packages/xen/xend/XendCheckpoint.py",
line 37, in write_exact
OSError: [Errno 32] Broken pipe
[2007-12-21 23:28:13 xend.XendDomainInfo 3339] DEBUG
(XendDomainInfo:757) Storing domain details:
{''console/ring-ref'':
''1850346'', ''console/port'':
''2'', ''name'':
''migrating-solaris79'',
''console/limit'': ''1048576'',
''vm'':
''/vm/43a65d79-bb0e-2d90-627d-561bf1d6385b'',
''domid'': ''37'',
''cpu/0/availability'': ''online'',
''memory/target'': ''1048576'',
''store/ring-ref'': ''1850347'',
''store/port'': ''1''}
[2007-12-21 23:28:13 xend 3339] ERROR (xmlrpclib2:218) [Errno 32] Broken
pipe
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line
197, in _marshaled_dispatch
File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 406, in
_dispatch
return func(*params)
File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line
1180, in domain_migrate
File "/usr/lib/python2.4/site-packages/xen/xend/XendCheckpoint.py",
line 116, in save
OSError: [Errno 32] Broken pipe
[2007-12-21 23:28:13 xend.XendDomainInfo 3339] DEBUG
(XendDomainInfo:757) Storing domain details:
{''console/ring-ref'':
''1850346'', ''console/port'':
''2'', ''name'': ''solaris79'',
''console/limit'':
''1048576'', ''vm'':
''/vm/43a65d79-bb0e-2d90-627d-561bf1d6385b'',
''domid'':
''37'', ''cpu/0/availability'':
''online'', ''memory/target'':
''1048576'',
''store/ring-ref'': ''1850347'',
''store/port'': ''1''}
********************************************************************************
b. for --paravirt and --live
********************************************************************************
[2007-12-21 23:26:18 xend 3339] DEBUG (balloon:116) Balloon: 1086412 KiB
free; need 5120; done.
[2007-12-21 23:26:18 xend 3339] ERROR (XendCheckpoint:111) Save failed
on domain solaris79 (37).
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/xen/xend/XendCheckpoint.py",
line 69, in save
File "/usr/lib/python2.4/site-packages/xen/xend/XendCheckpoint.py",
line 37, in write_exact
OSError: [Errno 32] Broken pipe
[2007-12-21 23:26:18 xend 3339] ERROR (xmlrpclib2:218) [Errno 32] Broken
pipe
Traceback (most recent call last):
File "/usr/lib/python2.4/site-packages/xen/util/xmlrpclib2.py", line
197, in _marshaled_dispatch
File "/usr/lib/python2.4/SimpleXMLRPCServer.py", line 406, in
_dispatch
return func(*params)
File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line
1180, in domain_migrate
File "/usr/lib/python2.4/site-packages/xen/xend/XendCheckpoint.py",
line 116, in save
OSError: [Errno 32] Broken pipe
[2007-12-21 23:26:18 xend.XendDomainInfo 3339] DEBUG
(XendDomainInfo:757) Storing domain details:
{''console/ring-ref'':
''1850346'', ''console/port'':
''2'', ''name'':
''migrating-solaris79'',
''console/limit'': ''1048576'',
''vm'':
''/vm/43a65d79-bb0e-2d90-627d-561bf1d6385b'',
''domid'': ''37'',
''cpu/0/availability'': ''online'',
''memory/target'': ''1048576'',
''store/ring-ref'': ''1850347'',
''store/port'': ''1''}
[2007-12-21 23:26:18 xend.XendDomainInfo 3339] DEBUG
(XendDomainInfo:757) Storing domain details:
{''console/ring-ref'':
''1850346'', ''console/port'':
''2'', ''name'': ''solaris79'',
''console/limit'':
''1048576'', ''vm'':
''/vm/43a65d79-bb0e-2d90-627d-561bf1d6385b'',
''domid'':
''37'', ''cpu/0/availability'':
''online'', ''memory/target'':
''1048576'',
''store/ring-ref'': ''1850347'',
''store/port'': ''1''}
********************************************************************************
Are any additional settings required ?
I am able to migrate same domain on localhost.
Thanks,
--Navin
Check the "xend-relocation-*" properties of the xvm/xend service. They limit the set of people from whom we accept incoming migration requests. dme.
Hi David, Thanks for the response. After editing "xend-relocation-*" properties, migration started (at least log files are saying this).However it stopped after some time and migration was not successful. xm list outputs before and after migration in source machine bash-3.2# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 4947 4 r----- 151429.5 *solaris794* 45 1023 1 -b---- 121.3 xvm-x2200-01-vm1 33 1032 1 -b---- 1393.9 bash-3.2# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 4947 4 r----- 151465.8 solaris794 1024 1 0.0 xvm-x2200-01-vm1 33 1032 1 -b---- 1394.6 Please see following /var/log/xen/xend.log on source and target machines. Source Machine: ************************************************************************************************************** [2007-12-24 07:23:33 xend 3339] DEBUG (balloon:116) Balloon: 1086412 KiB free; need 5120; done. [2007-12-24 07:23:33 xend 3339] DEBUG (XendCheckpoint:80) [xc_save]: /usr/lib/xen/bin/xc_save 5 45 0 0 1 [2007-12-24 07:23:33 xend 3339] INFO (XendCheckpoint:247) Had 0 unexplained entries in p2m table [2007-12-24 07:23:38 xend 3339] INFO (XendCheckpoint:247) Saving memory pages: iter 1 20%PT Race: [10000000,508] pte=100000002a8467, mfn=000002a8 [2007-12-24 07:23:38 xend 3339] INFO (XendCheckpoint:247) PT Race: [10000000,509] pte=100000002a9467, mfn=000002a9 [2007-12-24 07:23:38 xend 3339] INFO (XendCheckpoint:247) PT Race: [10000000,510] pte=100000002aa427, mfn=000002aa [2007-12-24 07:23:38 xend 3339] INFO (XendCheckpoint:247) PT Race: [10000000,511] pte=100000002ab427, mfn=000002ab [2007-12-24 07:23:48 xend 3339] INFO (XendCheckpoint:2 85%PT Race: [10000000,140] pte=10000000fcd467, mfn=00000fcd 1: sent 261326, skipped 562, delta 17055ms, dom0 33%, target 4%, sent 502Mb/s, dirtied 4Mb/s 2582 pages 2: sent 2577, skipped 5, delta 156ms, dom0 0%, target 0%, sent 541Mb/s, dirtied 8Mb/s 41 pages 3: sent 41, skipped 0, Start last iterationeckpoint:247) Saving memory pages: iter 3 0% [2007-12-24 07:23:50 xend 3339] DEBUG (XendCheckpoint:218) suspend [2007-12-24 07:23:50 xend 3339] DEBUG (XendCheckpoint:83) In saveInputHandler suspend [2007-12-24 07:23:50 xend 3339] DEBUG (XendCheckpoint:85) Suspending 45 ... [2007-12-24 07:23:50 xend.XendDomainInfo 3339] DEBUG (XendDomainInfo:421) XendDomainInfo.shutdown [2007-12-24 07:23:50 xend.XendDomainInfo 3339] DEBUG (XendDomainInfo:824) XendDomainInfo.handleShutdownWatch [2007-12-24 07:23:50 xend.XendDomainInfo 3339] DEBUG (XendDomainInfo:824) XendDomainInfo.handleShutdownWatch [2007-12-24 07:23:51 xend 3339] INFO (XendCheckpoint:90) Domain 45 suspended. [2007-12-24 07:23:51 xend 3339] DEBUG (XendCheckpoint:95) Written done [2007-12-24 07:23:51 xend.XendDomainInfo 3339] INFO (XendDomainInfo:1000) Domain has shutdown: name=migrating-solaris794 id=45 reason=suspend. [2007-12-24 07:23:51 xend 3339] INFO (XendCheckpoint:247) SUSPEND shinfo 00000fcd eip fffffffffb83d5d4 edx 0015baee [2007-12-24 07:23:51 xend 3339] INFO (XendCheckpoint:247) delta 323ms, dom0 30%, target 88%, sent 4Mb/s, dirtied 25Mb/s 250 pages 4: sent 250, skipped 0, delta 11ms, dom0 81%, target 0%, sent 744Mb/s, dirtied 744Mb/s 250 pages [2007-12-24 07:23:51 xend 3339] INFO (XendCheckpoint:247) Total pages sent= 264194 (1.01x) [2007-12-24 07:23:51 xend 3339] INFO (XendCheckpoint:247) (of which 0 were fixups) [2007-12-24 07:23:51 xend 3339] INFO (XendCheckpoint:247) All memory is saved [2007-12-24 07:23:51 xend 3339] INFO (XendCheckpoint:247) Save exit rc=0 [2007-12-24 07:23:51 xend.XendDomainInfo 3339] DEBUG (XendDomainInfo:1527) XendDomainInfo.destroyDomain(45) ************************************************************************************************************** Target Machine: ************************************************************************************************************** [2007-12-24 07:08:06 xend 2342] DEBUG (XendDomain:431) Adding Domain: 8 [2007-12-24 07:08:06 xend 2342] DEBUG (balloon:116) Balloon: 1049448 KiB free; need 1048576; done. [2007-12-24 07:08:06 xend 2342] DEBUG (XendCheckpoint:163) [xc_restore]: /usr/lib/xen/bin/xc_restore 25 8 262144 1 2 [2007-12-24 07:08:06 xend 2342] INFO (XendCheckpoint:247) xc_linux_restore start: max_pfn = 40000 [2007-12-24 07:08:07 xend 2342] INFO (XendCheckpoint:247) Increased domain reservation by 100000 KB [2007-12-24 07:08:07 xend 2342] INFO (XendCheckpoint:247) Reloading memory pages: 0% [2007-12-24 07:08:24 xend 2342] INFO (XendCheckpoint:247) Received all pages (0 races) [2007-12-24 07:08:24 xend 2342] INFO (XendCheckpoint:2100% [2007-12-24 07:08:24 xend 2342] INFO (XendCheckpoint:247) Memory reloaded. [2007-12-24 07:08:24 xend 2342] INFO (XendCheckpoint:247) Decreased reservation by 256 pages [2007-12-24 07:08:24 xend 2342] INFO (XendCheckpoint:247) Domain ready to be built. [2007-12-24 07:08:24 xend 2342] INFO (XendCheckpoint:247) Restore exit with rc=0 [2007-12-24 07:08:24 xend 2342] DEBUG (XendCheckpoint:218) store-mfn 2201068 [2007-12-24 07:08:24 xend 2342] DEBUG (XendCheckpoint:218) console-mfn 2201067 [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:149) Waiting for devices vif. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:154) Waiting for 0. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:561) hotplugStatusCallback /local/domain/0/backend/vif/8/0/hotplug-status. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:575) hotplugStatusCallback 1. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:149) Waiting for devices usb. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:149) Waiting for devices vbd. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:154) Waiting for 0. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:561) hotplugStatusCallback /local/domain/0/backend/vbd/8/0/hotplug-status. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:575) hotplugStatusCallback 2. [2007-12-24 07:08:24 xend 2342] DEBUG (DevController:549) destroyCallback /local/domain/8/device/vbd/0. [2007-12-24 07:08:34 xend.XendDomainInfo 2342] DEBUG (XendDomainInfo:1519) XendDomainInfo.destroy: domid=8 [2007-12-24 07:08:34 xend.XendDomainInfo 2342] DEBUG (XendDomainInfo:1527) XendDomainInfo.destroyDomain(8) [2007-12-24 07:08:34 xend 2342] DEBUG (DevController:549) destroyCallback /local/domain/8/device/vbd/0. [2007-12-24 07:08:34 xend 2342] DEBUG (DevController:557) destroyCallback /local/domain/8/device/vbd/0 is destroyed [2007-12-24 07:08:34 xend 2342] ERROR (XendDomain:1043) Restore failed Traceback (most recent call last): File "/usr/lib/python2.4/site-packages/xen/xend/XendDomain.py", line 1038, in domain_restore_fd File "/usr/lib/python2.4/site-packages/xen/xend/XendCheckpoint.py", line 173, in restore File "/usr/lib/python2.4/site-packages/xen/xend/XendDomainInfo.py", line 518, in waitForDevices File "/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py", line 150, in waitForDevices File "/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py", line 165, in waitForDevice File "/usr/lib/python2.4/site-packages/xen/xend/server/blkif.py", line 146, in destroyDevice File "/usr/lib/python2.4/site-packages/xen/xend/server/DevController.py", line 237, in destroyDevice EnvironmentError ************************************************************************************************************** Thanks, --Navin David Edmondson wrote:> Check the "xend-relocation-*" properties of the xvm/xend service. They > limit the set of people from whom we accept incoming migration requests. > > dme. >