Brendan Cully
2007-Feb-27 02:36 UTC
[Xen-devel] [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint
The attached patches create a --checkpoint flag for xm save: xc_domain_resume lets the caller alert the guest that it is being resumed. Callers should not do this unless they know the guest supports the operation. When xend wants to resume a domain, it checks for the suspend_cancel feature in the elfnotes it has recorded from domain build or resume. If the flag is present, it simply lets the guest handle resuming, otherwise it tears down and rebuilds the domain and lets the guest act as if it is resuming in a new domain. A new suspend_cancel feature is added to the kernel ''features'' elfnote. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Brendan Cully
2007-Feb-27 02:36 UTC
[Xen-devel] [PATCH 1 of 3] Let xend choose to do hard or soft domain resumption depending on
# HG changeset patch
# User Brendan Cully <brendan@cs.ubc.ca>
# Date 1172543764 28800
# Node ID b2edb7f3393255f7b51af7fe688de687233b525e
# Parent e7b2a282c9e70fbadd38ccc7cc7d1fcb6a32b886
Let xend choose to do hard or soft domain resumption depending on
whether the domain advertises support for soft resumption in its
elfnotes.
Signed-off-by: Brendan Cully <brendan@kublai.com>
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/libxc/xc_resume.c
--- a/tools/libxc/xc_resume.c Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/libxc/xc_resume.c Mon Feb 26 18:36:04 2007 -0800
@@ -169,13 +169,9 @@ static int xc_domain_resume_any(int xc_h
* (2) should be used only for guests which cannot handle the special
* new return code. (1) is always safe (but slower).
*/
-int xc_domain_resume(int xc_handle, uint32_t domid)
+int xc_domain_resume(int xc_handle, uint32_t domid, int fast)
{
- /*
- * XXX: Implement a way to select between options (1) and (2).
- * Or expose the options as two different methods to Python.
- */
- return (0
+ return (fast
? xc_domain_resume_cooperative(xc_handle, domid)
: xc_domain_resume_any(xc_handle, domid));
}
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/libxc/xenctrl.h
--- a/tools/libxc/xenctrl.h Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/libxc/xenctrl.h Mon Feb 26 18:36:04 2007 -0800
@@ -243,10 +243,12 @@ int xc_domain_destroy(int xc_handle,
*
* @parm xc_handle a handle to an open hypervisor interface
* @parm domid the domain id to resume
+ * @parm fast use cooperative resume (guest must support this)
* return 0 on success, -1 on failure
*/
int xc_domain_resume(int xc_handle,
- uint32_t domid);
+ uint32_t domid,
+ int fast);
/**
* This function will shutdown a domain. This is intended for use in
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/python/xen/lowlevel/xc/xc.c
--- a/tools/python/xen/lowlevel/xc/xc.c Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/python/xen/lowlevel/xc/xc.c Mon Feb 26 18:36:04 2007 -0800
@@ -178,7 +178,17 @@ static PyObject *pyxc_domain_shutdown(Xc
static PyObject *pyxc_domain_resume(XcObject *self, PyObject *args)
{
- return dom_op(self, args, xc_domain_resume);
+ uint32_t dom;
+ int fast;
+
+ if (!PyArg_ParseTuple(args, "ii", &dom, &fast))
+ return NULL;
+
+ if (xc_domain_resume(self->xc_handle, dom, fast) != 0)
+ return pyxc_error_to_exception();
+
+ Py_INCREF(zero);
+ return zero;
}
static PyObject *pyxc_vcpu_setaffinity(XcObject *self,
@@ -1124,7 +1134,8 @@ static PyMethodDef pyxc_methods[] = {
(PyCFunction)pyxc_domain_resume,
METH_VARARGS, "\n"
"Resume execution of a suspended domain.\n"
- " dom [int]: Identifier of domain to be resumed.\n\n"
+ " dom [int]: Identifier of domain to be resumed.\n"
+ " fast [int]: Use cooperative resume.\n\n"
"Returns: [int] 0 on success; -1 on error.\n" },
{ "domain_shutdown",
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/python/xen/xend/XendCheckpoint.py
--- a/tools/python/xen/xend/XendCheckpoint.py Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/python/xen/xend/XendCheckpoint.py Mon Feb 26 18:36:04 2007 -0800
@@ -137,23 +137,6 @@ def save(fd, dominfo, network, live, dst
log.exception("Save failed on domain %s (%s).", domain_name,
dominfo.getDomid())
- dominfo._releaseDevices()
- dominfo.testDeviceComplete()
- dominfo.testvifsComplete()
- log.debug("XendCheckpoint.save: devices released")
-
- dominfo._resetChannels()
-
- dominfo._removeDom(''control/shutdown'')
- dominfo._removeDom(''device-misc/vif/nextDeviceID'')
-
- dominfo._createChannels()
- dominfo._introduceDomain()
- dominfo._storeDomDetails()
-
- dominfo._createDevices()
- log.debug("XendCheckpoint.save: devices created")
-
dominfo.resumeDomain()
log.debug("XendCheckpoint.save: resumeDomain")
diff -r e7b2a282c9e7 -r b2edb7f33932 tools/python/xen/xend/XendDomainInfo.py
--- a/tools/python/xen/xend/XendDomainInfo.py Mon Feb 26 17:20:36 2007 +0000
+++ b/tools/python/xen/xend/XendDomainInfo.py Mon Feb 26 18:36:04 2007 -0800
@@ -1662,10 +1662,31 @@ class XendDomainInfo:
def resumeDomain(self):
log.debug("XendDomainInfo.resumeDomain(%s)", str(self.domid))
+ if self.domid is None:
+ return
try:
- if self.domid is not None:
- xc.domain_resume(self.domid)
- ResumeDomain(self.domid)
+ # could also fetch a parsed key from xenstore
+ fast = ''suspend_cancel'' in
self.info.get_notes().get(''FEATURES'',
'''').split(''|'')
+ if not fast:
+ self._releaseDevices()
+ self.testDeviceComplete()
+ self.testvifsComplete()
+ log.debug("XendDomainInfo.resumeDomain: devices
released")
+
+ self._resetChannels()
+
+ self._removeDom(''control/shutdown'')
+
self._removeDom(''device-misc/vif/nextDeviceID'')
+
+ self._createChannels()
+ self._introduceDomain()
+ self._storeDomDetails()
+
+ self._createDevices()
+ log.debug("XendDomainInfo.resumeDomain: devices
created")
+
+ xc.domain_resume(self.domid, fast)
+ ResumeDomain(self.domid)
except:
log.exception("XendDomainInfo.resume: xc.domain_resume failed
on domain %s." % (str(self.domid)))
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Brendan Cully
2007-Feb-27 02:36 UTC
[Xen-devel] [PATCH 2 of 3] Add xm save -c/--checkpoint option
# HG changeset patch
# User Brendan Cully <brendan@cs.ubc.ca>
# Date 1172543764 28800
# Node ID b207ed258dcf3a6aee865ca82bc170f68e589342
# Parent b2edb7f3393255f7b51af7fe688de687233b525e
Add xm save -c/--checkpoint option
xm save --checkpoint leaves the domain running after creating the
snapshot.
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
diff -r b2edb7f33932 -r b207ed258dcf tools/python/xen/xend/XendCheckpoint.py
--- a/tools/python/xen/xend/XendCheckpoint.py Mon Feb 26 18:36:04 2007 -0800
+++ b/tools/python/xen/xend/XendCheckpoint.py Mon Feb 26 18:36:04 2007 -0800
@@ -54,7 +54,7 @@ def read_exact(fd, size, errmsg):
return buf
-def save(fd, dominfo, network, live, dst):
+def save(fd, dominfo, network, live, dst, checkpoint=False):
write_exact(fd, SIGNATURE, "could not write guest state file:
signature")
config = sxp.to_string(dominfo.sxpr())
@@ -121,9 +121,11 @@ def save(fd, dominfo, network, live, dst
os.close(qemu_fd)
os.remove("/tmp/xen.qemu-dm.%d" % dominfo.getDomid())
- dominfo.destroyDomain()
- dominfo.testDeviceComplete()
-
+ if checkpoint:
+ dominfo.resumeDomain()
+ else:
+ dominfo.destroyDomain()
+ dominfo.testDeviceComplete()
try:
dominfo.setName(domain_name)
except VmError:
diff -r b2edb7f33932 -r b207ed258dcf tools/python/xen/xend/XendDomain.py
--- a/tools/python/xen/xend/XendDomain.py Mon Feb 26 18:36:04 2007 -0800
+++ b/tools/python/xen/xend/XendDomain.py Mon Feb 26 18:36:04 2007 -0800
@@ -1172,7 +1172,7 @@ class XendDomain:
XendCheckpoint.save(sock.fileno(), dominfo, True, live, dst)
sock.close()
- def domain_save(self, domid, dst):
+ def domain_save(self, domid, dst, checkpoint):
"""Start saving a domain to file.
@param domid: Domain ID or Name
@@ -1196,8 +1196,8 @@ class XendDomain:
oflags |= os.O_LARGEFILE
fd = os.open(dst, oflags)
try:
- # For now we don''t support ''live
checkpoint''
- XendCheckpoint.save(fd, dominfo, False, False, dst)
+ XendCheckpoint.save(fd, dominfo, False, False, dst,
+ checkpoint=checkpoint)
finally:
os.close(fd)
except OSError, ex:
diff -r b2edb7f33932 -r b207ed258dcf tools/python/xen/xm/main.py
--- a/tools/python/xen/xm/main.py Mon Feb 26 18:36:04 2007 -0800
+++ b/tools/python/xen/xm/main.py Mon Feb 26 18:36:04 2007 -0800
@@ -102,7 +102,7 @@ SUBCOMMAND_HELP = {
''reboot'' : (''<Domain>
[-wa]'', ''Reboot a domain.''),
''restore'' : (''<CheckpointFile>
[-p]'',
''Restore a domain from a saved state.''),
- ''save'' : (''<Domain>
<CheckpointFile>'',
+ ''save'' : (''[-c] <Domain>
<CheckpointFile>'',
''Save a domain state to restore
later.''),
''shutdown'' : (''<Domain>
[-waRH]'', ''Shutdown a domain.''),
''top'' : ('''', ''Monitor
a host and the domains in real time.''),
@@ -232,6 +232,9 @@ SUBCOMMAND_OPTIONS = {
''resume'': (
(''-p'', ''--paused'', ''Do not
unpause domain after resuming it''),
),
+ ''save'': (
+ (''-c'', ''--checkpoint'',
''Leave domain running after creating snapshot''),
+ ),
''restore'': (
(''-p'', ''--paused'', ''Do not
unpause domain after restoring it''),
),
@@ -604,21 +607,37 @@ def xm_shell(args):
#########################################################################
def xm_save(args):
- arg_check(args, "save", 2)
-
- try:
- dominfo = parse_doms_info(server.xend.domain(args[0]))
+ arg_check(args, "save", 2, 3)
+
+ try:
+ (options, params) = getopt.gnu_getopt(args, ''c'',
[''checkpoint''])
+ except getopt.GetoptError, opterr:
+ err(opterr)
+ sys.exit(1)
+
+ checkpoint = False
+ for (k, v) in options:
+ if k in [''-c'', ''--checkpoint'']:
+ checkpoint = True
+
+ if len(params) != 2:
+ err("Wrong number of parameters")
+ usage(''save'')
+ sys.exit(1)
+
+ try:
+ dominfo = parse_doms_info(server.xend.domain(params[0]))
except xmlrpclib.Fault, ex:
raise ex
domid = dominfo[''domid'']
- savefile = os.path.abspath(args[1])
+ savefile = os.path.abspath(params[1])
if not os.access(os.path.dirname(savefile), os.W_OK):
err("xm save: Unable to create file %s" % savefile)
sys.exit(1)
- server.xend.domain.save(domid, savefile)
+ server.xend.domain.save(domid, savefile, checkpoint)
def xm_restore(args):
arg_check(args, "restore", 1, 2)
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Brendan Cully
2007-Feb-27 02:36 UTC
[Xen-devel] [PATCH 3 of 3] Add suspend_cancel flag to linux elf notes
# HG changeset patch
# User Brendan Cully <brendan@cs.ubc.ca>
# Date 1172543764 28800
# Node ID 728b370eaca01f5d7dd3e1ec59b34a0860372645
# Parent b207ed258dcf3a6aee865ca82bc170f68e589342
Add suspend_cancel flag to linux elf notes.
Signed-off-by: Brendan Cully <brendan@cs.ubc.ca>
diff -r b207ed258dcf -r 728b370eaca0
linux-2.6-xen-sparse/arch/i386/kernel/head-xen.S
--- a/linux-2.6-xen-sparse/arch/i386/kernel/head-xen.S Mon Feb 26 18:36:04 2007
-0800
+++ b/linux-2.6-xen-sparse/arch/i386/kernel/head-xen.S Mon Feb 26 18:36:04 2007
-0800
@@ -175,6 +175,7 @@ ENTRY(cpu_gdt_table)
.ascii "|auto_translated_physmap"
.ascii "|pae_pgdir_above_4gb"
.ascii "|supervisor_mode_kernel"
+ .ascii "|suspend_cancel"
#ifdef CONFIG_X86_PAE
.ascii ",PAE=yes[extended-cr3]"
#else
@@ -197,7 +198,7 @@ ENTRY(cpu_gdt_table)
ELFNOTE(Xen, XEN_ELFNOTE_ENTRY, .long, startup_32)
ELFNOTE(Xen, XEN_ELFNOTE_HYPERCALL_PAGE, .long, hypercall_page)
ELFNOTE(Xen, XEN_ELFNOTE_HV_START_LOW, .long, HYPERVISOR_VIRT_START)
- ELFNOTE(Xen, XEN_ELFNOTE_FEATURES, .asciz,
"writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel")
+ ELFNOTE(Xen, XEN_ELFNOTE_FEATURES, .asciz,
"writable_page_tables|writable_descriptor_tables|auto_translated_physmap|pae_pgdir_above_4gb|supervisor_mode_kernel|suspend_cancel")
#ifdef CONFIG_X86_PAE
ELFNOTE(Xen, XEN_ELFNOTE_PAE_MODE, .asciz, "yes")
ELFNOTE(Xen, XEN_ELFNOTE_L1_MFN_VALID, .quad, _PAGE_PRESENT,_PAGE_PRESENT)
diff -r b207ed258dcf -r 728b370eaca0 xen/common/libelf/libelf-dominfo.c
--- a/xen/common/libelf/libelf-dominfo.c Mon Feb 26 18:36:04 2007 -0800
+++ b/xen/common/libelf/libelf-dominfo.c Mon Feb 26 18:36:04 2007 -0800
@@ -12,7 +12,8 @@ const char *elf_xen_feature_names[] = {
[XENFEAT_writable_descriptor_tables] =
"writable_descriptor_tables",
[XENFEAT_auto_translated_physmap] = "auto_translated_physmap",
[XENFEAT_supervisor_mode_kernel] = "supervisor_mode_kernel",
- [XENFEAT_pae_pgdir_above_4gb] = "pae_pgdir_above_4gb"
+ [XENFEAT_pae_pgdir_above_4gb] = "pae_pgdir_above_4gb",
+ [XENFEAT_suspend_cancel] = "suspend_cancel"
};
const int elf_xen_features sizeof(elf_xen_feature_names) /
sizeof(elf_xen_feature_names[0]);
diff -r b207ed258dcf -r 728b370eaca0 xen/include/public/features.h
--- a/xen/include/public/features.h Mon Feb 26 18:36:04 2007 -0800
+++ b/xen/include/public/features.h Mon Feb 26 18:36:04 2007 -0800
@@ -56,6 +56,12 @@
*/
#define XENFEAT_pae_pgdir_above_4gb 4
+/*
+ * If set, the guest is checkpointable. If the suspend hypercall returns
+ * 1, the domain will undo its suspend prep work and continue.
+ */
+#define XENFEAT_suspend_cancel 5
+
#define XENFEAT_NR_SUBMAPS 1
#endif /* __XEN_PUBLIC_FEATURES_H__ */
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Keir Fraser
2007-Feb-27 16:10 UTC
Re: [Xen-devel] [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint
Basically okay, but please create a new elfnote rather than adding a feature. The feature stuff has kind of got a bit confused, but is meant to indicate features provided by Xen to a particular guest. The feature maps in the guest header indicate which Xen features it requires and/or supports. This new flag has nothing to do with the hypervisor itself so I think it belongs in its own elfnote (XEN_ELFNOTE_SUSPEND_CANCEL?). This can either be a boolean (so no elfnote description bytes) or perhaps we should map it to a long with value 1 for now (so we can add further save/restore revisions later if we like). Oh, also remember to add the elfnote to the x86/64 head-xen.S. -- Keir On 27/2/07 02:36, "Brendan Cully" <brendan@cs.ubc.ca> wrote:> The attached patches create a --checkpoint flag for xm save: > > xc_domain_resume lets the caller alert the guest that it is being > resumed. Callers should not do this unless they know the guest > supports the operation. > > When xend wants to resume a domain, it checks for the suspend_cancel > feature in the elfnotes it has recorded from domain build or > resume. If the flag is present, it simply lets the guest handle > resuming, otherwise it tears down and rebuilds the domain and lets the > guest act as if it is resuming in a new domain. > > A new suspend_cancel feature is added to the kernel ''features'' > elfnote. > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Brendan Cully
2007-Feb-28 06:42 UTC
Re: [Xen-devel] [PATCH 0 of 3] Enable domain checkpointing via xm save --checkpoint
On Tuesday, 27 February 2007 at 16:10, Keir Fraser wrote:> Basically okay, but please create a new elfnote rather than adding a > feature. The feature stuff has kind of got a bit confused, but is meant to > indicate features provided by Xen to a particular guest. The feature maps in > the guest header indicate which Xen features it requires and/or supports. > This new flag has nothing to do with the hypervisor itself so I think it > belongs in its own elfnote (XEN_ELFNOTE_SUSPEND_CANCEL?). This can either be > a boolean (so no elfnote description bytes) or perhaps we should map it to a > long with value 1 for now (so we can add further save/restore revisions > later if we like).Ok, I''ve resubmitted with this approach, making the note a long.> Oh, also remember to add the elfnote to the x86/64 head-xen.S.Done, thanks for the heads up. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel