Denis Chapligin
2009-Aug-10 08:04 UTC
[Xen-devel] Problem with migrating/saving 2.6.30 vanilla kernel with multiple VCPUs
Hi I have xen 3.4.1 installed, with dom0 running linux-2.6.18-xen.hg (four weeks old) I''ve made a domU with a vanilla 2.6.30.4 kernel and tried to save it to a file. xm save command stuck and resulting file was only 1503 bytes. Here are some details - xm list after xm save command: dom0:~# xm list Name ID Mem VCPUs State Time(s) Domain-0 0 254 1 r----- 90.7 migrating-test 9 256 4 -b---- 12.3 Snapshot file: dom0:/tmp# ls -l total 0 -rwxr-xr-x 1 root root 1503 2009-08-10 10:51 test.mem Contents of test.mem file: dom0:/tmp# cat test.mem LinuxGuestRecord�domain (domid 9) (on_crash restart) (uuid e3ac4f7e-d1e4-e9ce-8d9f-dfb229ced941) (bootloader_args ) (vcpus 4) (name test) (on_poweroff destroy) (on_reboot restart) (cpus (() () () ())) (bootloader ) (maxmem 256) (memory 256) (shadow_memory 0) (vcpu_avail 15) (features ) (on_xend_start ignore) (on_xend_stop ignore) (start_time 1249890685.85) (cpu_time 0.0) (online_vcpus 1) (image (linux (kernel /GFS/kernels/vmlinuz-2.6.30.4) (ramdisk /GFS/kernels/initrd.img-2.6.30.4) (args ''root=/dev/xvda clocksource=jiffies'') (videoram 4) (notes (HV_START_LOW 4118806528) (FEATURES ''!writable_page_tables|pae_pgdir_above_4gb'') (VIRT_BASE 3221225472) (GUEST_VERSION 2.6) (PADDR_OFFSET 0) (GUEST_OS linux) (HYPERCALL_PAGE 3222282240) (LOADER generic) (SUSPEND_CANCEL 1) (PAE_MODE yes) (ENTRY 3225649152) (XEN_VERSION xen-3.0)))) (status 2) (state --p---) (store_mfn 1165228) (console_mfn 1165227) (device (vif (bridge br6) (mac 00:16:3e:2e:4c:8b) (script /etc/xen/scripts/vif-bridge) (uuid 668ceb77-5657-79d0-a6d8-20442e718d79) (backend 0))) (device (vbd (protocol x86_32-abi) (uuid da6930eb-f7eb-893e-6d87-a27db331b957) (bootable 1) (dev xvda:disk) (uname phy:/dev/etherd/e1.2) (mode w) (backend 0) (bootable 1) (VDI ))) (device (console (protocol vt100) (location 2) (uuid becf1aa4-e58c-7404-f2ff-8e33ec7f788e))) (cpuid ((2147483648 ((eax 00000000000000000000000000000011) (edx 00000000000000000000000000000000) (ebx 00000000000000000000000000000000) (ecx 00000000000000000000000000000000)))))) There is nothing on xen''s dmesg and dom0 dmesg. Here are part of xend.log: [2009-08-10 10:51:45 2186] DEBUG (XendCheckpoint:110) [xc_save]: /usr/lib/xen-3.4/bin/xc_save 46 9 0 0 0 [2009-08-10 10:51:452186] DEBUG (XendCheckpoint:388) suspend [2009-08-10 10:51:45 2186]DEBUG (XendCheckpoint:113) In saveInputHandler suspend [2009-08-1010:51:45 2186] DEBUG (XendCheckpoint:115) Suspending 9 ... [2009-08-1010:51:45 2186] DEBUG (XendDomainInfo:511) XendDomainInfo.shutdown(suspend) [2009-08-10 10:51:45 2186] DEBUG (XendDomainInfo:1709) XendDomainInfo.handleShutdownWatch [2009-08-1010:51:45 2186] DEBUG (XendDomainInfo:1709) XendDomainInfo.handleShutdownWatch [2009-08-10 10:51:45 2186] INFO(XendCheckpoint:417) xc_save: failed to get the suspend evtchn port Xen''s info: dom0:~# xm info host : dom0 release : 2.6.18.8 version : #1 SMP Tue Jul 7 18:42:07 EEST 2009 machine : i686 nr_cpus : 4 nr_nodes : 1 cores_per_socket : 1 threads_per_core : 2 cpu_mhz : 2992 hw_caps : bfebfbff:20000800:00000000:00000180:0000641d:00000000:0 0000000:00000000 virt_caps : total_memory : 4095 free_memory : 3529 node_to_cpu : node0:0-3 node_to_memory : node0:3529 xen_major : 3 xen_minor : 4 xen_extra : xen_caps : xen-3.0-x86_64 xen-3.0-x86_32p xen_scheduler : credit xen_pagesize : 4096 platform_params : virt_start=0xff800000 xen_changeset : unavailable cc_compiler : gcc version 4.3.2 (Debian 4.3.2-1.1) cc_compile_by : chollya cc_compile_domain : satgate.net cc_compile_date : Fri Aug 7 11:54:19 EEST 2009 xend_config_format : 4 and the domain configuration file: dom0:~# cat /GFS/xen/test.cfg kernel="/GFS/kernels/vmlinuz-2.6.30.4" ramdisk="/GFS/kernels/initrd.img-2.6.30.4" root="/dev/xvda" extra="clocksource=jiffies" memory = 256 cpuid=[''0x80000000:eax=0x3,ebx=0x0,ecx=0x0,edx=0x0''] name = "test" vcpus = 4 vif = [''bridge=br6''] disk = [''phy:/dev/etherd/e1.2,xvda,w''] Save/migrate works fine when i use linux-2.6.18-xen.hg as a domU kernel, so i suppose it could be a bug somewhere, not a configuration issue. -- Denis Chapligin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Denis Chapligin
2009-Aug-10 08:16 UTC
Re: [Xen-devel] Problem with migrating/saving 2.6.30 vanilla kernel with multiple VCPUs
Hi On Mon, 10 Aug 2009 11:04:24 +0300 Denis Chapligin <chollya@satgate.net> wrote:> Hi > > I have xen 3.4.1 installed, with dom0 running linux-2.6.18-xen.hg > (four weeks old) I''ve made a domU with a vanilla 2.6.30.4 kernel and > tried to save it to a file. > xm save command stuck and resulting file was only 1503 bytes. >Even more - when i set vcpus to 1 on this domain, it is possible to save/restore it, but domain can migrate only one time. Second migratin fails with: dom0:~# xm migrate -l test otherdom0 Error: /usr/lib/xen-3.4/bin/xc_save 47 3 0 0 1 failed and following message inx xm dmesg: (XEN) mm.c:806:d0 Error getting mfn 10d679 (pfn 133b6) from L1 entry 800000010d679225 for l1e_owner=0, pg_owner=3 (XEN) mm.c:806:d0 Error getting mfn 10d679 (pfn 133b6) from L1 entry 800000010d679225 for l1e_owner=0, pg_owner=3 -- Denis Chapligin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Denis Chapligin
2009-Aug-12 06:14 UTC
Re: [Xen-devel] Problem with migrating/saving 2.6.30 vanilla kernel with multiple VCPUs
Hi On Mon, 10 Aug 2009 11:16:20 +0300 Denis Chapligin <chollya@satgate.net> wrote:> Even more - when i set vcpus to 1 on this domain, it is possible to > save/restore it, but domain can migrate only one time. Second migratin > fails with: > dom0:~# xm migrate -l test otherdom0 > Error: /usr/lib/xen-3.4/bin/xc_save 47 3 0 0 1 failed > > and following message inx xm dmesg: > > (XEN) mm.c:806:d0 Error getting mfn 10d679 (pfn 133b6) from L1 entry > 800000010d679225 for l1e_owner=0, pg_owner=3 > (XEN) mm.c:806:d0 Error getting mfn 10d679 (pfn 133b6) from L1 entry > 800000010d679225 for l1e_owner=0, pg_owner=3 >With domU kernel 2.6.30.2, taken from http://x17.eu/xen/, it is possible to migrate domain with 1 vcpu. With more then one, it migrates well, but hangs on disk operations in a couple of seconds after migration. Right after migration (before hang) next message is printed to the domU''s console: ------------[ cut here ]------------ WARNING: at kernel/irq/manage.c:259 __enable_irq+0x56/0x79() Unbalanced enable for IRQ 776 Modules linked in: ipv6 Pid: 2022, comm: suspend Tainted: G W 2.6.30.2 #1 Call Trace: [<c01564a3>] ? __enable_irq+0x56/0x79 [<c01564a3>] ? __enable_irq+0x56/0x79 [<c012561b>] warn_slowpath_common+0x59/0x94 [<c01564a3>] ? __enable_irq+0x56/0x79 [<c0125693>] warn_slowpath_fmt+0x26/0x28 [<c01564a3>] __enable_irq+0x56/0x79 [<c01584d4>] resume_device_irqs+0x59/0x6d [<c03d5790>] ? xen_resume_notifier+0x0/0x21 [<c03cd9f9>] device_power_up+0x8d/0x94 [<c03d6264>] __xen_suspend+0xaf/0x141 [<c01275dc>] ? daemonize+0x1cd/0x223 [<c03d5790>] ? xen_resume_notifier+0x0/0x21 [<c03d5c72>] xen_suspend+0x48/0xc6 [<c03d5c2a>] ? xen_suspend+0x0/0xc6 [<c010461b>] kernel_thread_helper+0x7/0x10 ---[ end trace 5ed28e66d100536c ]--- Is there any solution, that will help fix this issue? -- Denis Chapligin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel