I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that will leave the domain running at the end of the save operation. root@ubuntu:/etc/xen# xl -v save -c win7 win7chk win7 Saving to win7chk new xl format (info 0x0/0x0/232) xc: detail: delta 8875ms, dom0 20%, target 0%, sent 971Mb/s, dirtied 0Mb/s 0 pages xc: detail: Total pages sent= 263168 (0.25x) xc: detail: (of which 0 were fixups) xc: detail: All memory is saved xc: detail: Save exit rc=0 root@ubuntu:/etc/xen# xl list Name ID Mem VCPUs State Time(s) Domain-0 0 2914 4 r----- 604.5 win7 8 1019 2 ---ss- 9.8 At the end of this the domain is frozen. Is this a known issue? Any pointers as to how to debug this? Where does xl pipe its debug messages to? BTW, if I destroy the domain and bring it back up using "xl restore", it comes up successfully. Thanks, AP _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
George Dunlap
2011-May-03 11:07 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
Have you tried it with other operating systems and found it to work? I.e., is it something specific to Windows 7, or is it a general HVM / Windows problem? -George On Fri, Apr 29, 2011 at 8:28 PM, AP Xen <apxeng@gmail.com> wrote:> I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that > will leave the domain running at the end of the save operation. > > root@ubuntu:/etc/xen# xl -v save -c win7 win7chk win7 > Saving to win7chk new xl format (info 0x0/0x0/232) > xc: detail: delta 8875ms, dom0 20%, target 0%, sent 971Mb/s, dirtied > 0Mb/s 0 pages > xc: detail: Total pages sent= 263168 (0.25x) > xc: detail: (of which 0 were fixups) > xc: detail: All memory is saved > xc: detail: Save exit rc=0 > > root@ubuntu:/etc/xen# xl list > Name ID Mem VCPUs State Time(s) > Domain-0 0 2914 4 r----- 604.5 > win7 8 1019 2 > ---ss- 9.8 > > At the end of this the domain is frozen. Is this a known issue? Any > pointers as to how to debug this? Where does xl pipe its debug > messages to? > > BTW, if I destroy the domain and bring it back up using "xl restore", > it comes up successfully. > > Thanks, > AP > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-May-03 13:01 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
On Fri, 2011-04-29 at 20:28 +0100, AP Xen wrote:> I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that > will leave the domain running at the end of the save operation.Do you have pv drivers installed which support checkpoint suspends? I''m not sure if such a thing even exists for Windows. I''m also not entirely sure that checkpointing was ever supported for HVM domains without PV drivers (e.g. via emulated hibernation). Perhaps the Remus guys know? [...]> At the end of this the domain is frozen. Is this a known issue? Any > pointers as to how to debug this? Where does xl pipe its debug > messages to?/var/log/xen/xl-<domname>.log. You can also do "xl -vvv <command>" to get some additional debug output. Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Shriram Rajagopalan
2011-May-03 14:09 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
On Tue, May 3, 2011 at 6:01 AM, Ian Campbell <Ian.Campbell@citrix.com>wrote:> On Fri, 2011-04-29 at 20:28 +0100, AP Xen wrote: > > I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that > > will leave the domain running at the end of the save operation. > > Do you have pv drivers installed which support checkpoint suspends? I''m > not sure if such a thing even exists for Windows. > > I''m also not entirely sure that checkpointing was ever supported for HVM > domains without PV drivers (e.g. via emulated hibernation). Perhaps the > Remus guys know? > > Remus works with HVM domains via normal xenstore based suspend/resume.Only PV-HVM support is "disabled" for the moment.> [...] > > At the end of this the domain is frozen. Is this a known issue? Any > > pointers as to how to debug this? Where does xl pipe its debug > > messages to? > > /var/log/xen/xl-<domname>.log. You can also do "xl -vvv <command>" to > get some additional debug output. > > Yes. the logs would be great. Also, by frozen, do you mean the domainremains in "suspended" state? or is Windows hung? shriram> Ian. > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ian Campbell
2011-May-03 14:31 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
On Tue, 2011-05-03 at 15:09 +0100, Shriram Rajagopalan wrote:> Remus works with HVM domains via normal xenstore based suspend/resume. > Only PV-HVM support is "disabled" for the moment.I''m confused ;-) PV-HVM uses PV suspend, which can be triggered either by xenstore or the suspend evtchn, but the choice of which is mostly orthogonal to whether or not suspend cancellation is supported. The alternative to PV suspend is the inject an ACPI S3 (or whatever) event option, isn''t it? Ian. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tim Deegan
2011-May-03 14:42 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
At 15:31 +0100 on 03 May (1304436713), Ian Campbell wrote:> On Tue, 2011-05-03 at 15:09 +0100, Shriram Rajagopalan wrote: > > > Remus works with HVM domains via normal xenstore based suspend/resume. > > Only PV-HVM support is "disabled" for the moment. > > I''m confused ;-) > > PV-HVM uses PV suspend, which can be triggered either by xenstore or the > suspend evtchn, but the choice of which is mostly orthogonal to whether > or not suspend cancellation is supported. > > The alternative to PV suspend is the inject an ACPI S3 (or whatever) > event option, isn''t it?The alternative is not to involve the OS at all; just use the normal HVM save and restore functions. For Remus, where downtime will be low and storage consistency is handled elsewhere, that''s good enough. Tim. -- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, Xen Platform Team Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Shriram Rajagopalan
2011-May-03 16:32 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
On Tue, May 3, 2011 at 7:42 AM, Tim Deegan <Tim.Deegan@citrix.com> wrote:> At 15:31 +0100 on 03 May (1304436713), Ian Campbell wrote: > > On Tue, 2011-05-03 at 15:09 +0100, Shriram Rajagopalan wrote: > > > > > Remus works with HVM domains via normal xenstore based suspend/resume. > > > Only PV-HVM support is "disabled" for the moment. > > > > I''m confused ;-) > > > > PV-HVM uses PV suspend, which can be triggered either by xenstore or the > > suspend evtchn, but the choice of which is mostly orthogonal to whether > > or not suspend cancellation is supported. > > > > The alternative to PV suspend is the inject an ACPI S3 (or whatever) > > event option, isn''t it? > > The alternative is not to involve the OS at all; just use the normal HVM > save and restore functions. For Remus, where downtime will be low and > storage consistency is handled elsewhere, that''s good enough. > > Tim. > >Enabling it is a matter of just removing the check for pvhvm, which I ll remove and send out a patch soon. :-) I was waiting for the last remus patch (remus: fix incorrect error handling for switch_qemu_logdirty...) to get pulled in before sending the patch. shriram> -- > Tim Deegan <Tim.Deegan@citrix.com> > Principal Software Engineer, Xen Platform Team > Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Brendan Cully
2011-May-03 16:35 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
On 2011-05-03, at 9:32 AM, Shriram Rajagopalan wrote:> On Tue, May 3, 2011 at 7:42 AM, Tim Deegan <Tim.Deegan@citrix.com> wrote: > At 15:31 +0100 on 03 May (1304436713), Ian Campbell wrote: > > On Tue, 2011-05-03 at 15:09 +0100, Shriram Rajagopalan wrote: > > > > > Remus works with HVM domains via normal xenstore based suspend/resume. > > > Only PV-HVM support is "disabled" for the moment. > > > > I''m confused ;-) > > > > PV-HVM uses PV suspend, which can be triggered either by xenstore or the > > suspend evtchn, but the choice of which is mostly orthogonal to whether > > or not suspend cancellation is supported. > > > > The alternative to PV suspend is the inject an ACPI S3 (or whatever) > > event option, isn''t it? > > The alternative is not to involve the OS at all; just use the normal HVM > save and restore functions. For Remus, where downtime will be low and > storage consistency is handled elsewhere, that''s good enough. > > Tim. > > > Enabling it is a matter of just removing the check for pvhvm, which I ll remove > and send out a patch soon. :-) > I was waiting for the last remus patch (remus: fix incorrect error handling for > switch_qemu_logdirty...) to get pulled in before sending the patch.As we discussed earlier, please do not do this without actually testing that pvhvm checkpointing works. On windows. PV suspend is a cooperative process and the reason it is disabled is that it hasn''t been tested. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I have only tried with Windows 7 for now. Let me try with CentOS 5.6 and get back to you. Thanks, AP On Tue, May 3, 2011 at 4:07 AM, George Dunlap <George.Dunlap@eu.citrix.com> wrote:> Have you tried it with other operating systems and found it to work? > I.e., is it something specific to Windows 7, or is it a general HVM / > Windows problem? > > -George > > On Fri, Apr 29, 2011 at 8:28 PM, AP Xen <apxeng@gmail.com> wrote: >> I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that >> will leave the domain running at the end of the save operation. >> >> root@ubuntu:/etc/xen# xl -v save -c win7 win7chk win7 >> Saving to win7chk new xl format (info 0x0/0x0/232) >> xc: detail: delta 8875ms, dom0 20%, target 0%, sent 971Mb/s, dirtied >> 0Mb/s 0 pages >> xc: detail: Total pages sent= 263168 (0.25x) >> xc: detail: (of which 0 were fixups) >> xc: detail: All memory is saved >> xc: detail: Save exit rc=0 >> >> root@ubuntu:/etc/xen# xl list >> Name ID Mem VCPUs State Time(s) >> Domain-0 0 2914 4 r----- 604.5 >> win7 8 1019 2 >> ---ss- 9.8 >> >> At the end of this the domain is frozen. Is this a known issue? Any >> pointers as to how to debug this? Where does xl pipe its debug >> messages to? >> >> BTW, if I destroy the domain and bring it back up using "xl restore", >> it comes up successfully. >> >> Thanks, >> AP >> >> _______________________________________________ >> Xen-devel mailing list >> Xen-devel@lists.xensource.com >> http://lists.xensource.com/xen-devel >> >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
I am not running with any PV drivers. I can try with James Harper''s GPLPV drivers and see how it behaves. Thanks, AP On Tue, May 3, 2011 at 6:01 AM, Ian Campbell <Ian.Campbell@citrix.com> wrote:> On Fri, 2011-04-29 at 20:28 +0100, AP Xen wrote: >> I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that >> will leave the domain running at the end of the save operation. > > Do you have pv drivers installed which support checkpoint suspends? I''m > not sure if such a thing even exists for Windows. > > I''m also not entirely sure that checkpointing was ever supported for HVM > domains without PV drivers (e.g. via emulated hibernation). Perhaps the > Remus guys know? > > [...] >> At the end of this the domain is frozen. Is this a known issue? Any >> pointers as to how to debug this? Where does xl pipe its debug >> messages to? > > /var/log/xen/xl-<domname>.log. You can also do "xl -vvv <command>" to > get some additional debug output. > > Ian. > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, May 3, 2011 at 7:09 AM, Shriram Rajagopalan <rshriram@cs.ubc.ca> wrote:> On Tue, May 3, 2011 at 6:01 AM, Ian Campbell <Ian.Campbell@citrix.com> > wrote: >> >> On Fri, 2011-04-29 at 20:28 +0100, AP Xen wrote: >> > I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that >> > will leave the domain running at the end of the save operation. >> >> Do you have pv drivers installed which support checkpoint suspends? I''m >> not sure if such a thing even exists for Windows. >> >> I''m also not entirely sure that checkpointing was ever supported for HVM >> domains without PV drivers (e.g. via emulated hibernation). Perhaps the >> Remus guys know? >> > Remus works with HVM domains via normal xenstore based suspend/resume. > Only PV-HVM support is "disabled" for the moment. >> >> [...] >> > At the end of this the domain is frozen. Is this a known issue? Any >> > pointers as to how to debug this? Where does xl pipe its debug >> > messages to? >> >> /var/log/xen/xl-<domname>.log. You can also do "xl -vvv <command>" to >> get some additional debug output. >> > Yes. the logs would be great. Also, by frozen, do you mean the domain > remains > in "suspended" state? or is Windows hung?Not sure what the difference between Windows being suspended and hung is. Here is the xl list output: Name ID Mem VCPUs State Time(s) Domain-0 0 2914 4 r----- 1259.0 win7 15 1019 2 ---ss- 0.3 Here is the log: Saving to win7chk new xl format (info 0x0/0x0/255) libxl: debug: libxl_dom.c:378:libxl__domain_suspend_common_callback Calling xc_domain_shutdown on HVM domain libxl: debug: libxl_dom.c:438:libxl__domain_suspend_common_callback wait for the guest to suspend libxl: debug: libxl_dom.c:450:libxl__domain_suspend_common_callback guest has suspended xc: debug: outbuf_write: 4194304 > 90092@16687124 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: detail: delta 9991ms, dom0 27%, target 0%, sent 863Mb/s, dirtied 0Mb/s 0 pages xc: detail: Total pages sent= 263168 (0.25x) xc: detail: (of which 0 were fixups) xc: detail: All memory is saved xc: detail: Save exit rc=0 libxl: debug: libxl_dom.c:534:libxl__domain_save_device_model Saving device model state to /var/lib/xen/qemu-save.15 libxl: debug: libxl_dom.c:546:libxl__domain_save_device_model Qemu state is 7204 bytes> shriram >> >> Ian. >> >> > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Shriram Rajagopalan
2011-May-05 14:42 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
On Tue, May 3, 2011 at 2:17 PM, AP Xen <apxeng@gmail.com> wrote:> On Tue, May 3, 2011 at 7:09 AM, Shriram Rajagopalan <rshriram@cs.ubc.ca> > wrote: > > On Tue, May 3, 2011 at 6:01 AM, Ian Campbell <Ian.Campbell@citrix.com> > > wrote: > >> > >> On Fri, 2011-04-29 at 20:28 +0100, AP Xen wrote: > >> > I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that > >> > will leave the domain running at the end of the save operation. > >> > >> Do you have pv drivers installed which support checkpoint suspends? I''m > >> not sure if such a thing even exists for Windows. > >> > >> I''m also not entirely sure that checkpointing was ever supported for HVM > >> domains without PV drivers (e.g. via emulated hibernation). Perhaps the > >> Remus guys know? > >> > > Remus works with HVM domains via normal xenstore based suspend/resume. > > Only PV-HVM support is "disabled" for the moment. > >> > >> [...] > >> > At the end of this the domain is frozen. Is this a known issue? Any > >> > pointers as to how to debug this? Where does xl pipe its debug > >> > messages to? > >> > >> /var/log/xen/xl-<domname>.log. You can also do "xl -vvv <command>" to > >> get some additional debug output. > >> > > Yes. the logs would be great. Also, by frozen, do you mean the domain > > remains > > in "suspended" state? or is Windows hung? > > Not sure what the difference between Windows being suspended and hung > is. Here is the xl list output: > Name ID Mem VCPUs State > Time(s) > Domain-0 0 2914 4 r----- > 1259.0 > win7 15 1019 2 ---ss- > 0.3 > > Here is the log: > Saving to win7chk new xl format (info 0x0/0x0/255) > libxl: debug: libxl_dom.c:378:libxl__domain_suspend_common_callback > Calling xc_domain_shutdown on HVM domain > libxl: debug: libxl_dom.c:438:libxl__domain_suspend_common_callback > wait for the guest to suspend > libxl: debug: libxl_dom.c:450:libxl__domain_suspend_common_callback > guest has suspended > xc: debug: outbuf_write: 4194304 > 90092@16687124 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: debug: outbuf_write: 4194304 > 4169716@12607500 > xc: detail: delta 9991ms, dom0 27%, target 0%, sent 863Mb/s, dirtied > 0Mb/s 0 pages > xc: detail: Total pages sent= 263168 (0.25x) > xc: detail: (of which 0 were fixups) > xc: detail: All memory is saved > xc: detail: Save exit rc=0 > libxl: debug: libxl_dom.c:534:libxl__domain_save_device_model Saving > device model state to /var/lib/xen/qemu-save.15 > libxl: debug: libxl_dom.c:546:libxl__domain_save_device_model Qemu > state is 7204 bytes > > > Ok. I see a "HVM shutdown". But where is the resume?Going through the libxl code, one obvious difference I see between xl''s implementation of save and the old xm implementation is, xl calls "xc_domain_unpause" while the xm implementation calls xc_domain_resume. Just in case, have you tried the same with xm save -c ? shriram> > shriram > >> > >> Ian. > >> > >> > > > > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Thu, May 5, 2011 at 7:42 AM, Shriram Rajagopalan <rshriram@cs.ubc.ca> wrote:> On Tue, May 3, 2011 at 2:17 PM, AP Xen <apxeng@gmail.com> wrote: >> >> On Tue, May 3, 2011 at 7:09 AM, Shriram Rajagopalan <rshriram@cs.ubc.ca> >> wrote: >> > On Tue, May 3, 2011 at 6:01 AM, Ian Campbell <Ian.Campbell@citrix.com> >> > wrote: >> >> >> >> On Fri, 2011-04-29 at 20:28 +0100, AP Xen wrote: >> >> > I am trying to do an "xl save -c" on a Windows 7 Ultimate domain that >> >> > will leave the domain running at the end of the save operation. >> >> >> >> Do you have pv drivers installed which support checkpoint suspends? I''m >> >> not sure if such a thing even exists for Windows. >> >> >> >> I''m also not entirely sure that checkpointing was ever supported for >> >> HVM >> >> domains without PV drivers (e.g. via emulated hibernation). Perhaps the >> >> Remus guys know? >> >> >> > Remus works with HVM domains via normal xenstore based suspend/resume. >> > Only PV-HVM support is "disabled" for the moment. >> >> >> >> [...] >> >> > At the end of this the domain is frozen. Is this a known issue? Any >> >> > pointers as to how to debug this? Where does xl pipe its debug >> >> > messages to? >> >> >> >> /var/log/xen/xl-<domname>.log. You can also do "xl -vvv <command>" to >> >> get some additional debug output. >> >> >> > Yes. the logs would be great. Also, by frozen, do you mean the domain >> > remains >> > in "suspended" state? or is Windows hung? >> >> Not sure what the difference between Windows being suspended and hung >> is. Here is the xl list output: >> Name ID Mem VCPUs State >> Time(s) >> Domain-0 0 2914 4 r----- >> 1259.0 >> win7 15 1019 2 ---ss- >> 0.3 >> >> Here is the log: >> Saving to win7chk new xl format (info 0x0/0x0/255) >> libxl: debug: libxl_dom.c:378:libxl__domain_suspend_common_callback >> Calling xc_domain_shutdown on HVM domain >> libxl: debug: libxl_dom.c:438:libxl__domain_suspend_common_callback >> wait for the guest to suspend >> libxl: debug: libxl_dom.c:450:libxl__domain_suspend_common_callback >> guest has suspended >> xc: debug: outbuf_write: 4194304 > 90092@16687124 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 >> xc: detail: delta 9991ms, dom0 27%, target 0%, sent 863Mb/s, dirtied >> 0Mb/s 0 pages >> xc: detail: Total pages sent= 263168 (0.25x) >> xc: detail: (of which 0 were fixups) >> xc: detail: All memory is saved >> xc: detail: Save exit rc=0 >> libxl: debug: libxl_dom.c:534:libxl__domain_save_device_model Saving >> device model state to /var/lib/xen/qemu-save.15 >> libxl: debug: libxl_dom.c:546:libxl__domain_save_device_model Qemu >> state is 7204 bytes >> >> > Ok. I see a "HVM shutdown". But where is the resume? > Going through the libxl code, one obvious difference I see between xl''s > implementation of save > and the old xm implementation is, xl calls "xc_domain_unpause" while the xm > implementation > calls xc_domain_resume. > > Just in case, have you tried the same with xm save -c ? > shriramxm save -c doesn''t seem to work at all. root@tubuntu:/etc/xen# xm save -c win7 win7chk Error: Timeout waiting for domain 2 to suspend [2011-05-05 13:55:43 1615] DEBUG (XendCheckpoint:124) [xc_save]: /usr/lib/xen/bin/xc_save 22 2 0 0 0 [2011-05-05 13:55:43 1615] INFO (XendCheckpoint:423) xc_save: failed to get the suspend evtchn port [2011-05-05 13:55:43 1615] INFO (XendCheckpoint:423) [2011-05-05 13:55:43 1615] DEBUG (XendCheckpoint:394) suspend [2011-05-05 13:55:43 1615] DEBUG (XendCheckpoint:127) In saveInputHandler suspend [2011-05-05 13:55:43 1615] DEBUG (XendCheckpoint:129) Suspending 2 ... [2011-05-05 13:55:43 1615] DEBUG (XendDomainInfo:524) XendDomainInfo.shutdown(suspend) [2011-05-05 13:55:43 1615] DEBUG (XendDomainInfo:1881) XendDomainInfo.handleShutdownWatch [2011-05-05 13:56:44 1615] DEBUG (XendDomainInfo:1881) XendDomainInfo.handleShutdownWatch [2011-05-05 13:56:44 1615] INFO (XendCheckpoint:423) xc: error: Suspend request failed: Internal error [2011-05-05 13:56:44 1615] INFO (XendCheckpoint:423) xc: error: Domain appears not to have suspended: Internal error [2011-05-05 13:56:44 1615] ERROR (XendCheckpoint:185) Save failed on domain win7 (2) - resuming. Traceback (most recent call last): File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendCheckpoint.py", line 146, in save forkHelper(cmd, fd, saveInputHandler, False) File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendCheckpoint.py", line 395, in forkHelper inputHandler(line, child.tochild) File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendCheckpoint.py", line 131, in saveInputHandler dominfo.waitForSuspend() File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomainInfo.py", line 2998, in waitForSuspend raise XendError(msg) XendError: Timeout waiting for domain 2 to suspend [2011-05-05 13:56:44 1615] DEBUG (XendDomainInfo:3135) XendDomainInfo.resumeDomain(2) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Shriram Rajagopalan
2011-May-06 02:31 UTC
Re: [Xen-devel] xl save -c issues with Windows 7 Ultimate
On Thu, May 5, 2011 at 2:01 PM, AP Xen <apxeng@gmail.com> wrote:> On Thu, May 5, 2011 at 7:42 AM, Shriram Rajagopalan <rshriram@cs.ubc.ca> > wrote: > > On Tue, May 3, 2011 at 2:17 PM, AP Xen <apxeng@gmail.com> wrote: > >> > >> On Tue, May 3, 2011 at 7:09 AM, Shriram Rajagopalan <rshriram@cs.ubc.ca > > > >> wrote: > >> > On Tue, May 3, 2011 at 6:01 AM, Ian Campbell <Ian.Campbell@citrix.com > > > >> > wrote: > >> >> > >> >> On Fri, 2011-04-29 at 20:28 +0100, AP Xen wrote: > >> >> > I am trying to do an "xl save -c" on a Windows 7 Ultimate domain > that > >> >> > will leave the domain running at the end of the save operation. > >> >> > >> >> Do you have pv drivers installed which support checkpoint suspends? > I''m > >> >> not sure if such a thing even exists for Windows. > >> >> > >> >> I''m also not entirely sure that checkpointing was ever supported for > >> >> HVM > >> >> domains without PV drivers (e.g. via emulated hibernation). Perhaps > the > >> >> Remus guys know? > >> >> > >> > Remus works with HVM domains via normal xenstore based suspend/resume. > >> > Only PV-HVM support is "disabled" for the moment. > >> >> > >> >> [...] > >> >> > At the end of this the domain is frozen. Is this a known issue? Any > >> >> > pointers as to how to debug this? Where does xl pipe its debug > >> >> > messages to? > >> >> > >> >> /var/log/xen/xl-<domname>.log. You can also do "xl -vvv <command>" to > >> >> get some additional debug output. > >> >> > >> > Yes. the logs would be great. Also, by frozen, do you mean the domain > >> > remains > >> > in "suspended" state? or is Windows hung? > >> > >> Not sure what the difference between Windows being suspended and hung > >> is. Here is the xl list output: > >> Name ID Mem VCPUs State > >> Time(s) > >> Domain-0 0 2914 4 r----- > >> 1259.0 > >> win7 15 1019 2 ---ss- > >> 0.3 > >> > >> Here is the log: > >> Saving to win7chk new xl format (info 0x0/0x0/255) > >> libxl: debug: libxl_dom.c:378:libxl__domain_suspend_common_callback > >> Calling xc_domain_shutdown on HVM domain > >> libxl: debug: libxl_dom.c:438:libxl__domain_suspend_common_callback > >> wait for the guest to suspend > >> libxl: debug: libxl_dom.c:450:libxl__domain_suspend_common_callback > >> guest has suspended > >> xc: debug: outbuf_write: 4194304 > 90092@16687124 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: debug: outbuf_write: 4194304 > 4169716@12607500 > >> xc: detail: delta 9991ms, dom0 27%, target 0%, sent 863Mb/s, dirtied > >> 0Mb/s 0 pages > >> xc: detail: Total pages sent= 263168 (0.25x) > >> xc: detail: (of which 0 were fixups) > >> xc: detail: All memory is saved > >> xc: detail: Save exit rc=0 > >> libxl: debug: libxl_dom.c:534:libxl__domain_save_device_model Saving > >> device model state to /var/lib/xen/qemu-save.15 > >> libxl: debug: libxl_dom.c:546:libxl__domain_save_device_model Qemu > >> state is 7204 bytes > >> > >> > > Ok. I see a "HVM shutdown". But where is the resume? > > Going through the libxl code, one obvious difference I see between xl''s > > implementation of save > > and the old xm implementation is, xl calls "xc_domain_unpause" while the > xm > > implementation > > calls xc_domain_resume. > > > > Just in case, have you tried the same with xm save -c ? > > shriram > > xm save -c doesn''t seem to work at all. > > root@tubuntu:/etc/xen# xm save -c win7 win7chk > Error: Timeout waiting for domain 2 to suspend > > [2011-05-05 13:55:43 1615] DEBUG (XendCheckpoint:124) [xc_save]: > /usr/lib/xen/bin/xc_save 22 2 0 0 0 > [2011-05-05 13:55:43 1615] INFO (XendCheckpoint:423) xc_save: failed > to get the suspend evtchn port > [2011-05-05 13:55:43 1615] INFO (XendCheckpoint:423) > [2011-05-05 13:55:43 1615] DEBUG (XendCheckpoint:394) suspend > [2011-05-05 13:55:43 1615] DEBUG (XendCheckpoint:127) In > saveInputHandler suspend > [2011-05-05 13:55:43 1615] DEBUG (XendCheckpoint:129) Suspending 2 ... > [2011-05-05 13:55:43 1615] DEBUG (XendDomainInfo:524) > XendDomainInfo.shutdown(suspend) > [2011-05-05 13:55:43 1615] DEBUG (XendDomainInfo:1881) > XendDomainInfo.handleShutdownWatch > [2011-05-05 13:56:44 1615] DEBUG (XendDomainInfo:1881) > XendDomainInfo.handleShutdownWatch > [2011-05-05 13:56:44 1615] INFO (XendCheckpoint:423) xc: error: > Suspend request failed: Internal error > [2011-05-05 13:56:44 1615] INFO (XendCheckpoint:423) xc: error: Domain > appears not to have suspended: Internal error > [2011-05-05 13:56:44 1615] ERROR (XendCheckpoint:185) Save failed on > domain win7 (2) - resuming. > Traceback (most recent call last): > File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendCheckpoint.py", > line 146, in save > forkHelper(cmd, fd, saveInputHandler, False) > File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendCheckpoint.py", > line 395, in forkHelper > inputHandler(line, child.tochild) > File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendCheckpoint.py", > line 131, in saveInputHandler > dominfo.waitForSuspend() > File "/usr/local/lib/python2.6/dist-packages/xen/xend/XendDomainInfo.py", > line 2998, in waitForSuspend > raise XendError(msg) > XendError: Timeout waiting for domain 2 to suspend > [2011-05-05 13:56:44 1615] DEBUG (XendDomainInfo:3135) > XendDomainInfo.resumeDomain(2) > > Any debug output from xen? "xm dmesg"shriram _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On Tue, May 3, 2011 at 4:07 AM, George Dunlap <George.Dunlap@eu.citrix.com> wrote:> Have you tried it with other operating systems and found it to work? > I.e., is it something specific to Windows 7, or is it a general HVM / > Windows problem?I tried this with CentOS 5.6 and saw the same behavior. root@ubuntu:~# xl -vvv save -c centos /etc/xen/centoschk Saving to /etc/xen/centoschk new xl format (info 0x0/0x0/255) libxl: debug: libxl_dom.c:384:libxl__domain_suspend_common_callback issuing PVHVM suspend request via XenBus control node libxl: debug: libxl_dom.c:389:libxl__domain_suspend_common_callback wait for the guest to acknowledge suspend request libxl: debug: libxl_dom.c:434:libxl__domain_suspend_common_callback guest acknowledged suspend request libxl: debug: libxl_dom.c:438:libxl__domain_suspend_common_callback wait for the guest to suspend libxl: debug: libxl_dom.c:450:libxl__domain_suspend_common_callback guest has suspended xc: debug: outbuf_write: 4194304 > 90092@16687124 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: debug: outbuf_write: 4194304 > 4169716@12607500 xc: detail: type fail: page 0 mfn 000f2000 xc: detail: type fail: page 1 mfn 000f2001 xc: detail: type fail: page 2 mfn 000f2002 xc: detail: delta 9371ms, dom0 25%, target 0%, sent 920Mb/s, dirtied 0Mb/s 0 pages xc: detail: Total pages sent= 263168 (0.25x) xc: detail: (of which 0 were fixups) xc: detail: All memory is saved xc: detail: Save exit rc=0 libxl: debug: libxl_dom.c:534:libxl__domain_save_device_model Saving device model state to /var/lib/xen/qemu-save.7 libxl: debug: libxl_dom.c:546:libxl__domain_save_device_model Qemu state is 7204 bytes root@ubuntu:~# xl list Name ID Mem VCPUs State Time(s) Domain-0 0 2914 4 r----- 6576.4 centos 7 1019 2 ---ss- 575.4 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel