[Paul == i_p_a_u_l@yahoo.com on Tue, 08 Feb 2005 01:09:57 +0200]
Paul> I tried with XenCD 1.0 rc1 in both vmware environments, and it
Paul> did not work, but I suspect there is something else fishy,
Paul> because "xm save" did not work either.
Thanks for the report. I can duplicate the problem that xm save does
not work on XenCD. Currently, I''m treating this as a release blocker
for 1.0final.
I''m unable at determine a fix for the problem and am soliciting help
from anyone who thinks they could contribute.
After a fresh boot of XenCD 1.0rc01, if I do:
xm save 1 /tmp/ttylinux1.save
I get:
Error: Error: [Failure instance: Traceback: \
xen.xend.XendError.XendError, save failed
The /var/log/xend-debug.log in enclosed below. I see:
/dev/loop: Is a directory
ioctl: LOOP_SET_FD: Device or resource busy
ioctl: LOOP_SET_FD: Device or resource busy
Relevant to that, I''m using udev with a plain Debian sarge install.
/dev/loop does indeed exist as a directory. My other udev box,
Gentoo, also has a /dev/loop directory, with 8 block devices inside.
XenCD has on 1 block device. My Debian woody devfs box has no
/dev/loop directory entry.
dom0 is booted with max_loop=64.
Oddly, there are possible loop leaks in my startup as there are
processes and lsof entries for loop0, loop1, loop2, loop3, loop10, and
loop11, which seems like a few too many. Only ttylinux-1 and 2''s
loopback rootfs should be using up loops, I think.
Possibilities:
Do I need to patch sarge''s udev to create a /dev/loop device?
Should I be digging into xend code to see how it uses /dev/loop? I''m
still wrapping my head around xend and I can''t find "loop" in
a grep
of the tree.
Is there something about tmpfs that makes xm save work for others but
not XenCD? XenCD''s Xen installation is as absolutely vanilla as I can
make it, other than tmpfs. And ttylinux is running on a loopback
ext2.
Apologies if I''m missing something obvious.
-- begin /var/log/xend-debug.log --
network start bridge=xen-br0 netdev=eth0 antispoof=no
VIRTUAL MEMORY ARRANGEMENT:
Loaded kernel: c0100000->c034f6c4
Init. ramdisk: c0350000->c0350000
Phys-Mach map: c0350000->c0358000
Page tables: c0358000->c035a000
Start info: c035a000->c035b000
Boot stack: c035b000->c035c000
TOTAL: c0000000->c0400000
ENTRY ADDRESS: c0100000
VCPUS: 1
/dev/loop: Is a directory
ioctl: LOOP_SET_FD: Device or resource busy
ioctl: LOOP_SET_FD: Device or resource busy
vif-bridge up vif=vif1.0 domain=ttylinux-1 mac=aa:00:00:69:13:b5 bridge=xen-br0
recv_fe_driver_status> {''status'': 1}
recv_fe_driver_status>
recv_fe_interface_connect {''tx_shmem_frame'': 23907,
''rx_shmem_frame'': 23906, ''handle'': 0}
VIRTUAL MEMORY ARRANGEMENT:
Loaded kernel: c0100000->c034f6c4
Init. ramdisk: c0350000->c0350000
Phys-Mach map: c0350000->c0358000
Page tables: c0358000->c035a000
Start info: c035a000->c035b000
Boot stack: c035b000->c035c000
TOTAL: c0000000->c0400000
ENTRY ADDRESS: c0100000
VCPUS: 1
/dev/loop: Is a directory
ioctl: LOOP_SET_FD: Device or resource busy
ioctl: LOOP_SET_FD: Device or resource busy
ioctl: LOOP_SET_FD: Device or resource busy
vif-bridge up vif=vif2.0 domain=ttylinux-2 mac=aa:00:00:56:44:c6 bridge=xen-br0
recv_fe_driver_status> {''status'': 1}
recv_fe_driver_status>
recv_fe_interface_connect {''tx_shmem_frame'': 5204,
''rx_shmem_frame'': 5203, ''handle'': 0}
sync_session> <type ''str''> 1
[''save'', [''id'', ''1''],
[''state'', ''begin''],
[''domain'', ''1''], [''file'',
''/tmp/ttylinux1.save'']]
Started to connect self= <xen.xend.XendMigrate.XfrdClientFactory instance at
0xb78fbdec> connector= <twisted.internet.tcp.Connector instance at
0xb78fbe0c>
buildProtocol> IPv4Address(TCP, ''localhost'', 8002)
***request> (domain (id 1) (name ttylinux-1) (memory 31) (maxmem 32768)
(state -b---) (cpu 0) (cpu_time 81.206402435) (up_time 59103.1672821)
(start_time 1107920300.57) (console (status listening) (id 12) (domain 1)
(local_port 12) (remote_port 1) (console_port 9601)) (devices (vif (idx 0) (vif
0) (mac aa:00:00:69:13:b5) (evtchn 14 4) (index 0)) (vbd (idx 0) (vdev 2049)
(device 1802) (mode w) (dev sda1) (uname file:/tmp/xen-rootfs.ttylinux-1) (node
/dev/loop10) (index 0))) (config (vm (name ttylinux-1) (memory 32) (cpu -1)
(restart always) (image (linux (kernel /media/cdrom/boot/xenU-kernel) (root
''/dev/sda1 ro'') (vcpus 1))) (device (vbd (uname
file:/tmp/xen-rootfs.ttylinux-1) (dev sda1) (mode w))) (device (vif (mac
aa:00:00:69:13:b5))) (memmap) (device_model) (device_config))))
***request> begin
xfr_err> [''xfr.err'', ''0'']
xfr_err> <type ''str''> 0
Xfrd>connectionLost> [Failure instance: Traceback:
twisted.internet.error.ConnectionDone, Connection was closed cleanly.
]
XfrdSaveInfo>connectionLost> [Failure instance: Traceback:
twisted.internet.error.ConnectionDone, Connection was closed cleanly.
]
XfrdInfo>connectionLost> [Failure instance: Traceback:
twisted.internet.error.ConnectionDone, Connection was closed cleanly.
]
Error> save failed
Error> calling errback
***cbremove> [Failure instance: Traceback: xen.xend.XendError.XendError, save
failed
]
***_delete_session> 1
clientConnectionLost> connector= <twisted.internet.tcp.Connector instance
at 0xb78fbe0c> reason= [Failure instance: Traceback:
twisted.internet.error.ConnectionDone, Connection was closed cleanly.
]
-- end --
-- jared@wordzoo.com
"A black hole is where God is dividing by zero."
-- attributed to Roger Smith
-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
[Jared == jared@wordzoo.com on Wed, 09 Feb 2005 16:05:18 -0800] Jared> I can duplicate the problem that xm save does not work on Jared> XenCD. Others have reported success after switching to Xen 2.0.4, but my XenCD save problem still exists with a 2.0.4-built snapshot. From my last email: -- begin -- Possibilities: Do I need to patch sarge''s udev to create a /dev/loop device? Should I be digging into xend code to see how it uses /dev/loop? I''m still wrapping my head around xend and I can''t find "loop" in a grep of the tree. Is there something about tmpfs that makes xm save work for others but not XenCD? XenCD''s Xen installation is as absolutely vanilla as I can make it, other than tmpfs. And ttylinux is running on a loopback ext2. Apologies if I''m missing something obvious. -- end -- Details available in my last message. -- jared@wordzoo.com "Better to be of a rare breed than a long line." -- TDK ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_ide95&alloc_id396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On 10 Feb 2005, at 22:50, Jared Rhine wrote:> [Jared == jared@wordzoo.com on Wed, 09 Feb 2005 16:05:18 -0800] > > Jared> I can duplicate the problem that xm save does not work on > Jared> XenCD. > > Others have reported success after switching to Xen 2.0.4, but my > XenCD save problem still exists with a 2.0.4-built snapshot. > > From my last email:If you run xfrd under strace -fF does it help? What versions of python/twisted do you have on it? Cheers Arthur ----- CTO @ Fotango Ltd +447834716919 http://www.fotango.com/ ----- CTO @ Fotango Ltd +447834716919 http://www.fotango.com/ ----- CTO @ Fotango Ltd +447834716919 http://www.fotango.com/ ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
[Arthur == abergman@fotango.com on Thu, 10 Feb 2005 23:31:21 +0000]
Arthur> On 10 Feb 2005, at 22:50, Jared Rhine wrote:
Jared> Others have reported success after switching to Xen 2.0.4,
Jared> but my XenCD save problem still exists with a 2.0.4-built
Jared> snapshot.
Arthur> If you run xfrd under strace -fF does it help?
Ah, should have mentioned. No, it does not. At least on Xen
unstable, I will try one more time with 2.0.4+strace xfrd.
Arthur> What versions of python/twisted do you have on it?
Debian sarge. Looking...
Python 2.3.4
Twisted 1.3.0
I presume others can replicate my results exactly just by using the CD
and the command sequence posted. I replicated results reported by
someone else.
-- jared@wordzoo.com
"A pessimist is one who has been intimately acquainted with an
optimist."
-- Elbert Hubbard
-------------------------------------------------------
SF email is sponsored by - The IT Product Guide
Read honest & candid reviews on hundreds of IT Products from real users.
Discover which products truly live up to the hype. Start reading now.
http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel
On 11 Feb 2005, at 00:40, Jared Rhine wrote:> Arthur> What versions of python/twisted do you have on it? > > Debian sarge. Looking... > > Python 2.3.4 > Twisted 1.3.0 > > I presume others can replicate my results exactly just by using the CD > and the command sequence posted. I replicated results reported by > someone else. >Thanks, now can someone where save/migrate works flawless please tell us the python version, twisted version and gcc version :) (and possibly also distro) Cheers Arthur ----- CTO @ Fotango Ltd +447834716919 http://www.fotango.com/ ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
[Jared == jared@wordzoo.com on Thu, 10 Feb 2005 16:40:05 -0800] Jared> Others have reported success after switching to Xen 2.0.4, Jared> but my XenCD save problem still exists with a 2.0.4-built Jared> snapshot. My sincere thanks to Paul Ionescu who pointed out that my 2.0.4 build wasn''t actually 2.0.4 but rather xeno-unstable. After fixing this build bug and retrying the resulting XenCD inside qemu, xm save/restore now works properly! (Fresh out of the box (no strace xfrd), to both hard disks and tmpfs). I believe that now every recent report of xm save/restore actually works under 2.0.4, while current xen-unstable installations are still broken (perhaps in only some cases?) Hopefully, this community testing provides the Xen team a more clear problem report and points to xm save/restore reversion between 2.0.4 and current unstable. My box is chugging away on building some new XenCD snapshots and the resulting snapshot should be available for download in a few hours. (If I remember, I''ll report on xen-testing''s save/restore success/failure as well). This issue is important enough for demo purposes that XenCD 1.0rc01 is revoked as a release candidate, and I''ll need to release a 1.0rc02 within a few days. -- jared@wordzoo.com "We suffer primarily not from our vices or our weaknesses, but from our illusions. We are haunted, not by reality, but by those images we have put in place of reality." - Daniel J. Boorstin ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
Arthur Bergman
2005-Feb-14 13:32 UTC
Re: [Xen-devel] XenCD xm save problem: solved by 2.0.4
On 12 Feb 2005, at 07:56, Jared Rhine wrote:> > After fixing this build bug and retrying the resulting XenCD inside > qemu, xm save/restore now works properly! (Fresh out of the box (no > strace xfrd), to both hard disks and tmpfs). > > I believe that now every recent report of xm save/restore actually > works under 2.0.4, while current xen-unstable installations are still > broken (perhaps in only some cases?) >Please try it multiple times Compiling xfrd with debugging it took my 36 save restore cycles to hit the bug. Anything that SLOWS xfrd down seems to fix the bug. Which is odd since it is blocking in a read. Anyway... this is where it is blocking. It seems to be waiting for a response that it never gets. Looking closer at this. Cheers Arthur (gdb) bt #0 0x402acb48 in read () from /lib/libc.so.6 #1 0x403125e0 in ?? () from /lib/libc.so.6 #2 0x4024e30b in _IO_file_read () from /lib/libc.so.6 #3 0x4024e79f in _IO_file_xsputn () from /lib/libc.so.6 #4 0x4024f4e8 in _IO_sgetn () from /lib/libc.so.6 #5 0x402434b2 in fread () from /lib/libc.so.6 #6 0x0804fb22 in file_read (s=0x1, buf=0xfffffe00, n=4294966784) at ../../tools/libxutil/file_stream.c:77 #7 0x0804f499 in Conn_sxpr (conn=0xbffff900, sxpr=0xfffffe00) at iostream.h:114 #8 0x0804dd6c in xfr_vm_suspend (xend=0xbffff900, vmid=1) at xfrd.c:451 #9 0x40026524 in suspend_and_state () from /usr/lib/libxc.so.2.0 #10 0x40027c60 in xc_linux_save () from /usr/lib/libxc.so.2.0 #11 0x0804e9e4 in xen_domain_snd (xend=0xfffffe00, io=0xfffffe00, dom=4294966784, vmconfig=0xfffffe00 <Address 0xfffffe00 out of bounds>, vmconfig_n=-512, live=0, resource=-512) at xen_domain.c:39 #12 0x0804b79a in xfr_save (args=0x805d3b0, state=0xbffff820, xend=0xbffff900, file=0x805f9a8 "/tmp/foo") at xfrd.c:774 #13 0x0804c454 in xfrd_service (args=0x805d3b0, peersock=-512, peer_in {sin_family = 2, sin_port = 12294, sin_addr = {s_addr = 16777343}, sin_zero = "\000\000\000\000\000\000\000"}) at xfrd.c:950 #14 0x0804c8c8 in xfrd_accept (args=0xfffffe00, sock=2) at xfrd.c:1024 #15 0x0804c9b5 in xfrd_select (args=0x805d3b0, listen_sock=2) at xfrd.c:1054 #16 0x0804d1e8 in main (argc=1, argv=0xbffffe44) at xfrd.c:1210 ----- CTO @ Fotango Ltd +447834716919 http://www.fotango.com/ ------------------------------------------------------- SF email is sponsored by - The IT Product Guide Read honest & candid reviews on hundreds of IT Products from real users. Discover which products truly live up to the hype. Start reading now. http://ads.osdn.com/?ad_id=6595&alloc_id=14396&op=click _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel