Hi all,
this is the sixth version of the Xen save/restore patch series.
We have been discussing this issue for quite a while on #qemu and
qemu-devel:
http://marc.info/?l=qemu-devel&m=132346828427314&w=2
http://marc.info/?l=qemu-devel&m=132377734605464&w=2
Please review the second patch: Introduce "save_devices".
Changes in v6:
- remove the is_ram parameter from register_savevm_live and sets is_ram
if the device is a live_savevm device;
- introduce save_devices as a QAPI command, write a better description
for it;
- fix CODING_STYLE;
- introduce a new doc to explain the save format used by save_devices.
Changes in v5:
- rebased on b4bd0b168e9f4898b98308f4a8a089f647a86d16.
Changes in v4:
- following Anthony''s suggestion I have introduced a new monitor
command
to save the non-ram device state to file;
- I have also removed the hack not to reset the cirrus videoram on
restore, because it turns out that the videoram doesn''t need to be
reset in the reset handler at all (tested on Win2K, where the problem
was found in the first place).
This is the list of patches with a diffstat:
Anthony PERARD (2):
xen mapcache: check if memory region has moved.
xen: do not allocate RAM during INMIGRATE runstate
Stefano Stabellini (4):
cirrus_vga: do not reset videoram
Introduce "save_devices"
xen: record physmap changes to xenstore
Set runstate to INMIGRATE earlier
docs/save_devices.txt | 33 +++++++++++++++
hw/cirrus_vga.c | 4 --
qapi-schema.json | 18 ++++++++
qmp-commands.hx | 25 ++++++++++++
savevm.c | 71 +++++++++++++++++++++++++++++++++
vl.c | 2 +-
xen-all.c | 104 ++++++++++++++++++++++++++++++++++++++++++++++++-
xen-mapcache.c | 22 +++++++++-
xen-mapcache.h | 9 +++-
9 files changed, 276 insertions(+), 12 deletions(-)
git://xenbits.xen.org/people/sstabellini/qemu-dm.git saverestore-6
Cheers,
Stefano
There is no need to set the videoram to 0xff in cirrus_reset, because it
is the BIOS'' job.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Reviewed-by: Avi Kivity <avi@redhat.com>
---
hw/cirrus_vga.c | 4 ----
1 files changed, 0 insertions(+), 4 deletions(-)
diff --git a/hw/cirrus_vga.c b/hw/cirrus_vga.c
index 4edcb94..afedaa4 100644
--- a/hw/cirrus_vga.c
+++ b/hw/cirrus_vga.c
@@ -2767,10 +2767,6 @@ static void cirrus_reset(void *opaque)
}
s->vga.cr[0x27] = s->device_id;
- /* Win2K seems to assume that the pattern buffer is at 0xff
- initially ! */
- memset(s->vga.vram_ptr, 0xff, s->real_vram_size);
-
s->cirrus_hidden_dac_lockindex = 5;
s->cirrus_hidden_dac_data = 0;
}
--
1.7.2.5
- add an "is_ram" flag to SaveStateEntry;
- register_savevm_live sets is_ram for live_savevm devices;
- introduce a "save_devices" QAPI command that can be used to save
the state of all devices, but not the RAM or the block devices of the
VM.
Changes in v6:
- remove the is_ram parameter from register_savevm_live and sets is_ram
if the device is a live_savevm device;
- introduce save_devices as a QAPI command, write a better description
for it;
- fix CODING_STYLE;
- introduce a new doc to explain the save format used by save_devices.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
CC: Anthony Liguori <anthony@codemonkey.ws>
CC: Luiz Capitulino <lcapitulino@redhat.com>
---
docs/save_devices.txt | 33 ++++++++++++++++++++++
qapi-schema.json | 18 ++++++++++++
qmp-commands.hx | 25 +++++++++++++++++
savevm.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++++
4 files changed, 147 insertions(+), 0 deletions(-)
create mode 100644 docs/save_devices.txt
diff --git a/docs/save_devices.txt b/docs/save_devices.txt
new file mode 100644
index 0000000..79915d2
--- /dev/null
+++ b/docs/save_devices.txt
@@ -0,0 +1,33 @@
+= Save Devices +
+QEMU has code to load/save the state of the guest that it is running.
+These are two complementary operations. Saving the state just does
+that, saves the state for each device that the guest is running.
+
+These operations are normally used with migration (see migration.txt),
+however it is also possible to save the state of all devices to file,
+without saving the RAM or the block devices of the VM.
+
+This operation is called "save_devices" (see QMP/qmp-commands.txt).
+
+
+The binary format used in the file is the following:
+
+
+-------------------------------------------
+
+32 bit big endian: QEMU_VM_FILE_MAGIC
+32 bit big endian: QEMU_VM_FILE_VERSION
+
+for_each_device
+{
+ 8 bit: QEMU_VM_SECTION_FULL
+ 32 bit big endian: section_id
+ 8 bit: idstr (ID string) length
+ string: idstr (ID string)
+ 32 bit big endian: instance_id
+ 32 bit big endian: version_id
+ buffer: device specific data
+}
+
+8 bit: QEMU_VM_EOF
diff --git a/qapi-schema.json b/qapi-schema.json
index d0b6792..7f938ff 100644
--- a/qapi-schema.json
+++ b/qapi-schema.json
@@ -1593,3 +1593,21 @@
{ ''command'': ''qom-list-types'',
''data'': { ''*implements'':
''str'', ''*abstract'': ''bool''
},
''returns'': [ ''ObjectTypeInfo'' ] }
+
+##
+# @save_devices:
+#
+# Save the state of all devices to file. The RAM and the block devices
+# of the VM are not saved by this command.
+#
+# @filename: the file to save the state of the devices to as binary
+# data. See save_devices.txt for a description of the binary format.
+#
+# Returns: Nothing on success
+# If @filename cannot be opened, OpenFileFailed
+# If an I/O error occurs while writing the file, IOError
+#
+# Since: 1.0
+##
+{ ''command'': ''save_devices'',
''data'': {''filename'':
''str''} }
+
diff --git a/qmp-commands.hx b/qmp-commands.hx
index 705f704..064d2a4 100644
--- a/qmp-commands.hx
+++ b/qmp-commands.hx
@@ -444,6 +444,31 @@ Note: inject-nmi is only supported for x86 guest currently,
it will
EQMP
{
+ .name = "save_devices",
+ .args_type = "filename:F",
+ .mhandler.cmd_new = qmp_marshal_input_save_devices,
+ },
+
+SQMP
+save_devices
+-------
+
+Save the state of all devices to file. The RAM and the block devices
+of the VM are not saved by this command.
+
+Arguments:
+
+- "filename": the file to save the state of the devices to as binary
+data. See save_devices.txt for a description of the binary format.
+
+Example:
+
+-> { "execute": "save_devices", "arguments": {
"filename": "/tmp/save" } }
+<- { "return": {} }
+
+EQMP
+
+ {
.name = "migrate",
.args_type = "detach:-d,blk:-b,inc:-i,uri:s",
.params = "[-d] [-b] [-i] uri",
diff --git a/savevm.c b/savevm.c
index 80be1ff..148356a 100644
--- a/savevm.c
+++ b/savevm.c
@@ -84,6 +84,7 @@
#include "qemu-timer.h"
#include "cpus.h"
#include "memory.h"
+#include "qmp-commands.h"
#define SELF_ANNOUNCE_ROUNDS 5
@@ -1177,6 +1178,7 @@ typedef struct SaveStateEntry {
void *opaque;
CompatEntry *compat;
int no_migrate;
+ int is_ram;
} SaveStateEntry;
@@ -1241,6 +1243,10 @@ int register_savevm_live(DeviceState *dev,
se->opaque = opaque;
se->vmsd = NULL;
se->no_migrate = 0;
+ /* if this is a live_savem then set is_ram */
+ if (save_live_state != NULL) {
+ se->is_ram = 1;
+ }
if (dev && dev->parent_bus &&
dev->parent_bus->info->get_dev_path) {
char *id = dev->parent_bus->info->get_dev_path(dev);
@@ -1728,6 +1734,45 @@ out:
return ret;
}
+static int qemu_save_device_state(QEMUFile *f)
+{
+ SaveStateEntry *se;
+
+ qemu_put_be32(f, QEMU_VM_FILE_MAGIC);
+ qemu_put_be32(f, QEMU_VM_FILE_VERSION);
+
+ cpu_synchronize_all_states();
+
+ QTAILQ_FOREACH(se, &savevm_handlers, entry) {
+ int len;
+
+ if (se->is_ram) {
+ continue;
+ }
+ if (se->save_state == NULL && se->vmsd == NULL) {
+ continue;
+ }
+
+ /* Section type */
+ qemu_put_byte(f, QEMU_VM_SECTION_FULL);
+ qemu_put_be32(f, se->section_id);
+
+ /* ID string */
+ len = strlen(se->idstr);
+ qemu_put_byte(f, len);
+ qemu_put_buffer(f, (uint8_t *)se->idstr, len);
+
+ qemu_put_be32(f, se->instance_id);
+ qemu_put_be32(f, se->version_id);
+
+ vmstate_save(f, se);
+ }
+
+ qemu_put_byte(f, QEMU_VM_EOF);
+
+ return qemu_file_get_error(f);
+}
+
static SaveStateEntry *find_se(const char *idstr, int instance_id)
{
SaveStateEntry *se;
@@ -2109,6 +2154,32 @@ void do_savevm(Monitor *mon, const QDict *qdict)
vm_start();
}
+void qmp_save_devices(const char *filename, Error **errp)
+{
+ QEMUFile *f;
+ int saved_vm_running;
+ int ret;
+
+ saved_vm_running = runstate_is_running();
+ vm_stop(RUN_STATE_SAVE_VM);
+
+ f = qemu_fopen(filename, "wb");
+ if (!f) {
+ error_set(errp, QERR_OPEN_FILE_FAILED, filename);
+ goto the_end;
+ }
+ ret = qemu_save_device_state(f);
+ qemu_fclose(f);
+ if (ret < 0) {
+ error_set(errp, QERR_IO_ERROR);
+ }
+
+ the_end:
+ if (saved_vm_running)
+ vm_start();
+ return;
+}
+
int load_vmstate(const char *name)
{
BlockDriverState *bs, *bs_vm_state;
--
1.7.2.5
Set runstate to RUN_STATE_INMIGRATE as soon as we can on resume.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
Acked-by: Luiz Capitulino <lcapitulino@redhat.com>
---
vl.c | 2 +-
1 files changed, 1 insertions(+), 1 deletions(-)
diff --git a/vl.c b/vl.c
index 1d4c350..918177a 100644
--- a/vl.c
+++ b/vl.c
@@ -3099,6 +3099,7 @@ int main(int argc, char **argv, char **envp)
break;
case QEMU_OPTION_incoming:
incoming = optarg;
+ runstate_set(RUN_STATE_INMIGRATE);
break;
case QEMU_OPTION_nodefaults:
default_serial = 0;
@@ -3596,7 +3597,6 @@ int main(int argc, char **argv, char **envp)
}
if (incoming) {
- runstate_set(RUN_STATE_INMIGRATE);
int ret = qemu_start_incoming_migration(incoming);
if (ret < 0) {
fprintf(stderr, "Migration failed. Exit code %s(%d),
exiting.\n",
--
1.7.2.5
Stefano Stabellini
2012-Mar-15 18:19 UTC
[PATCH v6 4/6] xen: record physmap changes to xenstore
Write to xenstore any physmap changes so that the hypervisor can be
aware of them.
Read physmap changes from xenstore on boot.
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
xen-all.c | 78 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 files changed, 77 insertions(+), 1 deletions(-)
diff --git a/xen-all.c b/xen-all.c
index b0ed1ed..f2cad82 100644
--- a/xen-all.c
+++ b/xen-all.c
@@ -65,7 +65,7 @@ static inline ioreq_t *xen_vcpu_ioreq(shared_iopage_t
*shared_page, int vcpu)
typedef struct XenPhysmap {
target_phys_addr_t start_addr;
ram_addr_t size;
- MemoryRegion *mr;
+ char *name;
target_phys_addr_t phys_offset;
QLIST_ENTRY(XenPhysmap) list;
@@ -237,6 +237,7 @@ static int xen_add_to_physmap(XenIOState *state,
XenPhysmap *physmap = NULL;
target_phys_addr_t pfn, start_gpfn;
target_phys_addr_t phys_offset = memory_region_get_ram_addr(mr);
+ char path[80], value[17];
if (get_physmapping(state, start_addr, size)) {
return 0;
@@ -275,6 +276,7 @@ go_physmap:
physmap->start_addr = start_addr;
physmap->size = size;
+ physmap->name = (char *)mr->name;
physmap->phys_offset = phys_offset;
QLIST_INSERT_HEAD(&state->physmap, physmap, list);
@@ -283,6 +285,30 @@ go_physmap:
start_addr >> TARGET_PAGE_BITS,
(start_addr + size) >>
TARGET_PAGE_BITS,
XEN_DOMCTL_MEM_CACHEATTR_WB);
+
+ snprintf(path, sizeof(path),
+
"/local/domain/0/device-model/%d/physmap/%"PRIx64"/start_addr",
+ xen_domid, (uint64_t)phys_offset);
+ snprintf(value, sizeof(value), "%"PRIx64, (uint64_t)start_addr);
+ if (!xs_write(state->xenstore, 0, path, value, strlen(value))) {
+ return -1;
+ }
+ snprintf(path, sizeof(path),
+
"/local/domain/0/device-model/%d/physmap/%"PRIx64"/size",
+ xen_domid, (uint64_t)phys_offset);
+ snprintf(value, sizeof(value), "%"PRIx64, (uint64_t)size);
+ if (!xs_write(state->xenstore, 0, path, value, strlen(value))) {
+ return -1;
+ }
+ if (mr->name) {
+ snprintf(path, sizeof(path),
+
"/local/domain/0/device-model/%d/physmap/%"PRIx64"/name",
+ xen_domid, (uint64_t)phys_offset);
+ if (!xs_write(state->xenstore, 0, path, mr->name,
strlen(mr->name))) {
+ return -1;
+ }
+ }
+
return 0;
}
@@ -911,6 +937,55 @@ int xen_init(void)
return 0;
}
+static void xen_read_physmap(XenIOState *state)
+{
+ XenPhysmap *physmap = NULL;
+ unsigned int len, num, i;
+ char path[80], *value = NULL;
+ char **entries = NULL;
+
+ snprintf(path, sizeof(path),
+ "/local/domain/0/device-model/%d/physmap", xen_domid);
+ entries = xs_directory(state->xenstore, 0, path, &num);
+ if (entries == NULL)
+ return;
+
+ for (i = 0; i < num; i++) {
+ physmap = g_malloc(sizeof (XenPhysmap));
+ physmap->phys_offset = strtoull(entries[i], NULL, 16);
+ snprintf(path, sizeof(path),
+
"/local/domain/0/device-model/%d/physmap/%s/start_addr",
+ xen_domid, entries[i]);
+ value = xs_read(state->xenstore, 0, path, &len);
+ if (value == NULL) {
+ free(physmap);
+ continue;
+ }
+ physmap->start_addr = strtoull(value, NULL, 16);
+ free(value);
+
+ snprintf(path, sizeof(path),
+ "/local/domain/0/device-model/%d/physmap/%s/size",
+ xen_domid, entries[i]);
+ value = xs_read(state->xenstore, 0, path, &len);
+ if (value == NULL) {
+ free(physmap);
+ continue;
+ }
+ physmap->size = strtoull(value, NULL, 16);
+ free(value);
+
+ snprintf(path, sizeof(path),
+ "/local/domain/0/device-model/%d/physmap/%s/name",
+ xen_domid, entries[i]);
+ physmap->name = xs_read(state->xenstore, 0, path, &len);
+
+ QLIST_INSERT_HEAD(&state->physmap, physmap, list);
+ }
+ free(entries);
+ return;
+}
+
int xen_hvm_init(void)
{
int i, rc;
@@ -986,6 +1061,7 @@ int xen_hvm_init(void)
xen_be_register("console", &xen_console_ops);
xen_be_register("vkbd", &xen_kbdmouse_ops);
xen_be_register("qdisk", &xen_blkdev_ops);
+ xen_read_physmap(state);
return 0;
}
--
1.7.2.5
Stefano Stabellini
2012-Mar-15 18:19 UTC
[PATCH v6 5/6] xen mapcache: check if memory region has moved.
From: Anthony PERARD <anthony.perard@citrix.com>
This patch changes the xen_map_cache behavior. Before trying to map a guest
addr, mapcache will look into the list of range of address that have been moved
(physmap/set_memory). There is currently one memory space like this, the vram,
"moved" from were it''s allocated to were the guest will look
into.
This help to have a succefull migration.
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
xen-all.c | 18 +++++++++++++++++-
xen-mapcache.c | 22 +++++++++++++++++++---
xen-mapcache.h | 9 +++++++--
3 files changed, 43 insertions(+), 6 deletions(-)
diff --git a/xen-all.c b/xen-all.c
index f2cad82..972cffd 100644
--- a/xen-all.c
+++ b/xen-all.c
@@ -225,6 +225,22 @@ static XenPhysmap *get_physmapping(XenIOState *state,
return NULL;
}
+static target_phys_addr_t xen_phys_offset_to_gaddr(target_phys_addr_t
start_addr,
+ ram_addr_t size, void
*opaque)
+{
+ target_phys_addr_t addr = start_addr & TARGET_PAGE_MASK;
+ XenIOState *xen_io_state = opaque;
+ XenPhysmap *physmap = NULL;
+
+ QLIST_FOREACH(physmap, &xen_io_state->physmap, list) {
+ if (range_covers_byte(physmap->phys_offset, physmap->size, addr))
{
+ return physmap->start_addr;
+ }
+ }
+
+ return start_addr;
+}
+
#if CONFIG_XEN_CTRL_INTERFACE_VERSION >= 340
static int xen_add_to_physmap(XenIOState *state,
target_phys_addr_t start_addr,
@@ -1043,7 +1059,7 @@ int xen_hvm_init(void)
}
/* Init RAM management */
- xen_map_cache_init();
+ xen_map_cache_init(xen_phys_offset_to_gaddr, state);
xen_ram_init(ram_size);
qemu_add_vm_change_state_handler(xen_hvm_change_state_handler, state);
diff --git a/xen-mapcache.c b/xen-mapcache.c
index 585b559..a456479 100644
--- a/xen-mapcache.c
+++ b/xen-mapcache.c
@@ -78,6 +78,9 @@ typedef struct MapCache {
uint8_t *last_address_vaddr;
unsigned long max_mcache_size;
unsigned int mcache_bucket_shift;
+
+ phys_offset_to_gaddr_t phys_offset_to_gaddr;
+ void *opaque;
} MapCache;
static MapCache *mapcache;
@@ -91,13 +94,16 @@ static inline int test_bits(int nr, int size, const unsigned
long *addr)
return 0;
}
-void xen_map_cache_init(void)
+void xen_map_cache_init(phys_offset_to_gaddr_t f, void *opaque)
{
unsigned long size;
struct rlimit rlimit_as;
mapcache = g_malloc0(sizeof (MapCache));
+ mapcache->phys_offset_to_gaddr = f;
+ mapcache->opaque = opaque;
+
QTAILQ_INIT(&mapcache->locked_entries);
mapcache->last_address_index = -1;
@@ -193,9 +199,14 @@ uint8_t *xen_map_cache(target_phys_addr_t phys_addr,
target_phys_addr_t size,
uint8_t lock)
{
MapCacheEntry *entry, *pentry = NULL;
- target_phys_addr_t address_index = phys_addr >> MCACHE_BUCKET_SHIFT;
- target_phys_addr_t address_offset = phys_addr & (MCACHE_BUCKET_SIZE -
1);
+ target_phys_addr_t address_index;
+ target_phys_addr_t address_offset;
target_phys_addr_t __size = size;
+ bool translated = false;
+
+tryagain:
+ address_index = phys_addr >> MCACHE_BUCKET_SHIFT;
+ address_offset = phys_addr & (MCACHE_BUCKET_SIZE - 1);
trace_xen_map_cache(phys_addr);
@@ -237,6 +248,11 @@ uint8_t *xen_map_cache(target_phys_addr_t phys_addr,
target_phys_addr_t size,
if(!test_bits(address_offset >> XC_PAGE_SHIFT, size >>
XC_PAGE_SHIFT,
entry->valid_mapping)) {
mapcache->last_address_index = -1;
+ if (!translated && mapcache->phys_offset_to_gaddr) {
+ phys_addr = mapcache->phys_offset_to_gaddr(phys_addr, size,
mapcache->opaque);
+ translated = true;
+ goto tryagain;
+ }
trace_xen_map_cache_return(NULL);
return NULL;
}
diff --git a/xen-mapcache.h b/xen-mapcache.h
index da874ca..70301a5 100644
--- a/xen-mapcache.h
+++ b/xen-mapcache.h
@@ -11,9 +11,13 @@
#include <stdlib.h>
+typedef target_phys_addr_t (*phys_offset_to_gaddr_t)(target_phys_addr_t
start_addr,
+ ram_addr_t size,
+ void *opaque);
#ifdef CONFIG_XEN
-void xen_map_cache_init(void);
+void xen_map_cache_init(phys_offset_to_gaddr_t f,
+ void *opaque);
uint8_t *xen_map_cache(target_phys_addr_t phys_addr, target_phys_addr_t size,
uint8_t lock);
ram_addr_t xen_ram_addr_from_mapcache(void *ptr);
@@ -22,7 +26,8 @@ void xen_invalidate_map_cache(void);
#else
-static inline void xen_map_cache_init(void)
+static inline void xen_map_cache_init(phys_offset_to_gaddr_t f,
+ void *opaque)
{
}
--
1.7.2.5
Stefano Stabellini
2012-Mar-15 18:19 UTC
[PATCH v6 6/6] xen: do not allocate RAM during INMIGRATE runstate
From: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Anthony PERARD <anthony.perard@citrix.com>
Signed-off-by: Stefano Stabellini <stefano.stabellini@eu.citrix.com>
---
xen-all.c | 8 ++++++++
1 files changed, 8 insertions(+), 0 deletions(-)
diff --git a/xen-all.c b/xen-all.c
index 972cffd..10d53d1 100644
--- a/xen-all.c
+++ b/xen-all.c
@@ -190,6 +190,14 @@ void xen_ram_alloc(ram_addr_t ram_addr, ram_addr_t size,
MemoryRegion *mr)
xen_pfn_t *pfn_list;
int i;
+ if (runstate_check(RUN_STATE_INMIGRATE)) {
+ /* RAM already populated in Xen */
+ fprintf(stderr, "%s: do not alloc "RAM_ADDR_FMT
+ " bytes of ram at "RAM_ADDR_FMT" when runstate
is INMIGRATE\n",
+ __func__, size, ram_addr);
+ return;
+ }
+
if (mr == &ram_memory) {
return;
}
--
1.7.2.5
On 03/15/2012 01:19 PM, Stefano Stabellini wrote:> - add an "is_ram" flag to SaveStateEntry; > > - register_savevm_live sets is_ram for live_savevm devices; > > - introduce a "save_devices" QAPI command that can be used to save > the state of all devices, but not the RAM or the block devices of the > VM. > > Changes in v6: > > - remove the is_ram parameter from register_savevm_live and sets is_ram > if the device is a live_savevm device; > > - introduce save_devices as a QAPI command, write a better description > for it; > > - fix CODING_STYLE; > > - introduce a new doc to explain the save format used by save_devices. > > Signed-off-by: Stefano Stabellini<stefano.stabellini@eu.citrix.com> > CC: Anthony Liguori<anthony@codemonkey.ws> > CC: Luiz Capitulino<lcapitulino@redhat.com> > --- > docs/save_devices.txt | 33 ++++++++++++++++++++++ > qapi-schema.json | 18 ++++++++++++ > qmp-commands.hx | 25 +++++++++++++++++ > savevm.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++++ > 4 files changed, 147 insertions(+), 0 deletions(-) > create mode 100644 docs/save_devices.txt > > diff --git a/docs/save_devices.txt b/docs/save_devices.txt > new file mode 100644 > index 0000000..79915d2 > --- /dev/null > +++ b/docs/save_devices.txt > @@ -0,0 +1,33 @@ > += Save Devices > + > +QEMU has code to load/save the state of the guest that it is running. > +These are two complementary operations. Saving the state just does > +that, saves the state for each device that the guest is running. > + > +These operations are normally used with migration (see migration.txt), > +however it is also possible to save the state of all devices to file, > +without saving the RAM or the block devices of the VM. > + > +This operation is called "save_devices" (see QMP/qmp-commands.txt). > + > + > +The binary format used in the file is the following: > + > + > +------------------------------------------- > + > +32 bit big endian: QEMU_VM_FILE_MAGIC > +32 bit big endian: QEMU_VM_FILE_VERSION > + > +for_each_device > +{ > + 8 bit: QEMU_VM_SECTION_FULL > + 32 bit big endian: section_id > + 8 bit: idstr (ID string) length > + string: idstr (ID string) > + 32 bit big endian: instance_id > + 32 bit big endian: version_id > + buffer: device specific data > +} > + > +8 bit: QEMU_VM_EOF > diff --git a/qapi-schema.json b/qapi-schema.json > index d0b6792..7f938ff 100644 > --- a/qapi-schema.json > +++ b/qapi-schema.json > @@ -1593,3 +1593,21 @@ > { ''command'': ''qom-list-types'', > ''data'': { ''*implements'': ''str'', ''*abstract'': ''bool'' }, > ''returns'': [ ''ObjectTypeInfo'' ] } > + > +## > +# @save_devices: > +# > +# Save the state of all devices to file. The RAM and the block devices > +# of the VM are not saved by this command. > +# > +# @filename: the file to save the state of the devices to as binary > +# data. See save_devices.txt for a description of the binary format. > +# > +# Returns: Nothing on success > +# If @filename cannot be opened, OpenFileFailed > +# If an I/O error occurs while writing the file, IOError > +# > +# Since: 1.0Since: 1.1. Otherwise Reviewed-by: Anthony Liguori <aliguori@us.ibm.com> Regards, Anthony Liguori> +## > +{ ''command'': ''save_devices'', ''data'': {''filename'': ''str''} } > + > diff --git a/qmp-commands.hx b/qmp-commands.hx > index 705f704..064d2a4 100644 > --- a/qmp-commands.hx > +++ b/qmp-commands.hx > @@ -444,6 +444,31 @@ Note: inject-nmi is only supported for x86 guest currently, it will > EQMP > > { > + .name = "save_devices", > + .args_type = "filename:F", > + .mhandler.cmd_new = qmp_marshal_input_save_devices, > + }, > + > +SQMP > +save_devices > +------- > + > +Save the state of all devices to file. The RAM and the block devices > +of the VM are not saved by this command. > + > +Arguments: > + > +- "filename": the file to save the state of the devices to as binary > +data. See save_devices.txt for a description of the binary format. > + > +Example: > + > +-> { "execute": "save_devices", "arguments": { "filename": "/tmp/save" } } > +<- { "return": {} } > + > +EQMP > + > + { > .name = "migrate", > .args_type = "detach:-d,blk:-b,inc:-i,uri:s", > .params = "[-d] [-b] [-i] uri", > diff --git a/savevm.c b/savevm.c > index 80be1ff..148356a 100644 > --- a/savevm.c > +++ b/savevm.c > @@ -84,6 +84,7 @@ > #include "qemu-timer.h" > #include "cpus.h" > #include "memory.h" > +#include "qmp-commands.h" > > #define SELF_ANNOUNCE_ROUNDS 5 > > @@ -1177,6 +1178,7 @@ typedef struct SaveStateEntry { > void *opaque; > CompatEntry *compat; > int no_migrate; > + int is_ram; > } SaveStateEntry; > > > @@ -1241,6 +1243,10 @@ int register_savevm_live(DeviceState *dev, > se->opaque = opaque; > se->vmsd = NULL; > se->no_migrate = 0; > + /* if this is a live_savem then set is_ram */ > + if (save_live_state != NULL) { > + se->is_ram = 1; > + } > > if (dev&& dev->parent_bus&& dev->parent_bus->info->get_dev_path) { > char *id = dev->parent_bus->info->get_dev_path(dev); > @@ -1728,6 +1734,45 @@ out: > return ret; > } > > +static int qemu_save_device_state(QEMUFile *f) > +{ > + SaveStateEntry *se; > + > + qemu_put_be32(f, QEMU_VM_FILE_MAGIC); > + qemu_put_be32(f, QEMU_VM_FILE_VERSION); > + > + cpu_synchronize_all_states(); > + > + QTAILQ_FOREACH(se,&savevm_handlers, entry) { > + int len; > + > + if (se->is_ram) { > + continue; > + } > + if (se->save_state == NULL&& se->vmsd == NULL) { > + continue; > + } > + > + /* Section type */ > + qemu_put_byte(f, QEMU_VM_SECTION_FULL); > + qemu_put_be32(f, se->section_id); > + > + /* ID string */ > + len = strlen(se->idstr); > + qemu_put_byte(f, len); > + qemu_put_buffer(f, (uint8_t *)se->idstr, len); > + > + qemu_put_be32(f, se->instance_id); > + qemu_put_be32(f, se->version_id); > + > + vmstate_save(f, se); > + } > + > + qemu_put_byte(f, QEMU_VM_EOF); > + > + return qemu_file_get_error(f); > +} > + > static SaveStateEntry *find_se(const char *idstr, int instance_id) > { > SaveStateEntry *se; > @@ -2109,6 +2154,32 @@ void do_savevm(Monitor *mon, const QDict *qdict) > vm_start(); > } > > +void qmp_save_devices(const char *filename, Error **errp) > +{ > + QEMUFile *f; > + int saved_vm_running; > + int ret; > + > + saved_vm_running = runstate_is_running(); > + vm_stop(RUN_STATE_SAVE_VM); > + > + f = qemu_fopen(filename, "wb"); > + if (!f) { > + error_set(errp, QERR_OPEN_FILE_FAILED, filename); > + goto the_end; > + } > + ret = qemu_save_device_state(f); > + qemu_fclose(f); > + if (ret< 0) { > + error_set(errp, QERR_IO_ERROR); > + } > + > + the_end: > + if (saved_vm_running) > + vm_start(); > + return; > +} > + > int load_vmstate(const char *name) > { > BlockDriverState *bs, *bs_vm_state;
On Thu, 15 Mar 2012 15:16:15 -0500 Anthony Liguori <anthony@codemonkey.ws> wrote:> On 03/15/2012 01:19 PM, Stefano Stabellini wrote: > > - add an "is_ram" flag to SaveStateEntry; > > > > - register_savevm_live sets is_ram for live_savevm devices; > > > > - introduce a "save_devices" QAPI command that can be used to save > > the state of all devices, but not the RAM or the block devices of the > > VM. > > > > Changes in v6: > > > > - remove the is_ram parameter from register_savevm_live and sets is_ram > > if the device is a live_savevm device; > > > > - introduce save_devices as a QAPI command, write a better description > > for it; > > > > - fix CODING_STYLE; > > > > - introduce a new doc to explain the save format used by save_devices. > > > > Signed-off-by: Stefano Stabellini<stefano.stabellini@eu.citrix.com> > > CC: Anthony Liguori<anthony@codemonkey.ws> > > CC: Luiz Capitulino<lcapitulino@redhat.com> > > --- > > docs/save_devices.txt | 33 ++++++++++++++++++++++ > > qapi-schema.json | 18 ++++++++++++ > > qmp-commands.hx | 25 +++++++++++++++++ > > savevm.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++++ > > 4 files changed, 147 insertions(+), 0 deletions(-) > > create mode 100644 docs/save_devices.txt > > > > diff --git a/docs/save_devices.txt b/docs/save_devices.txt > > new file mode 100644 > > index 0000000..79915d2 > > --- /dev/null > > +++ b/docs/save_devices.txt > > @@ -0,0 +1,33 @@ > > += Save Devices > > + > > +QEMU has code to load/save the state of the guest that it is running. > > +These are two complementary operations. Saving the state just does > > +that, saves the state for each device that the guest is running. > > + > > +These operations are normally used with migration (see migration.txt), > > +however it is also possible to save the state of all devices to file, > > +without saving the RAM or the block devices of the VM. > > + > > +This operation is called "save_devices" (see QMP/qmp-commands.txt). > > + > > + > > +The binary format used in the file is the following: > > + > > + > > +------------------------------------------- > > + > > +32 bit big endian: QEMU_VM_FILE_MAGIC > > +32 bit big endian: QEMU_VM_FILE_VERSION > > + > > +for_each_device > > +{ > > + 8 bit: QEMU_VM_SECTION_FULL > > + 32 bit big endian: section_id > > + 8 bit: idstr (ID string) length > > + string: idstr (ID string) > > + 32 bit big endian: instance_id > > + 32 bit big endian: version_id > > + buffer: device specific data > > +} > > + > > +8 bit: QEMU_VM_EOF > > diff --git a/qapi-schema.json b/qapi-schema.json > > index d0b6792..7f938ff 100644 > > --- a/qapi-schema.json > > +++ b/qapi-schema.json > > @@ -1593,3 +1593,21 @@ > > { ''command'': ''qom-list-types'', > > ''data'': { ''*implements'': ''str'', ''*abstract'': ''bool'' }, > > ''returns'': [ ''ObjectTypeInfo'' ] } > > + > > +## > > +# @save_devices: > > +# > > +# Save the state of all devices to file. The RAM and the block devices > > +# of the VM are not saved by this command. > > +# > > +# @filename: the file to save the state of the devices to as binary > > +# data. See save_devices.txt for a description of the binary format. > > +# > > +# Returns: Nothing on success > > +# If @filename cannot be opened, OpenFileFailed > > +# If an I/O error occurs while writing the file, IOError > > +# > > +# Since: 1.0 > > Since: 1.1. > > Otherwise Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>Looks fine to me FWIW, my only nitpick is that we use an hyphen instead of the underline in qmp command names, but I''d call this save-devices-state. Don''t you want this in HMP too, btw?> > Regards, > > Anthony Liguori > > > > +## > > +{ ''command'': ''save_devices'', ''data'': {''filename'': ''str''} } > > + > > diff --git a/qmp-commands.hx b/qmp-commands.hx > > index 705f704..064d2a4 100644 > > --- a/qmp-commands.hx > > +++ b/qmp-commands.hx > > @@ -444,6 +444,31 @@ Note: inject-nmi is only supported for x86 guest currently, it will > > EQMP > > > > { > > + .name = "save_devices", > > + .args_type = "filename:F", > > + .mhandler.cmd_new = qmp_marshal_input_save_devices, > > + }, > > + > > +SQMP > > +save_devices > > +------- > > + > > +Save the state of all devices to file. The RAM and the block devices > > +of the VM are not saved by this command. > > + > > +Arguments: > > + > > +- "filename": the file to save the state of the devices to as binary > > +data. See save_devices.txt for a description of the binary format. > > + > > +Example: > > + > > +-> { "execute": "save_devices", "arguments": { "filename": "/tmp/save" } } > > +<- { "return": {} } > > + > > +EQMP > > + > > + { > > .name = "migrate", > > .args_type = "detach:-d,blk:-b,inc:-i,uri:s", > > .params = "[-d] [-b] [-i] uri", > > diff --git a/savevm.c b/savevm.c > > index 80be1ff..148356a 100644 > > --- a/savevm.c > > +++ b/savevm.c > > @@ -84,6 +84,7 @@ > > #include "qemu-timer.h" > > #include "cpus.h" > > #include "memory.h" > > +#include "qmp-commands.h" > > > > #define SELF_ANNOUNCE_ROUNDS 5 > > > > @@ -1177,6 +1178,7 @@ typedef struct SaveStateEntry { > > void *opaque; > > CompatEntry *compat; > > int no_migrate; > > + int is_ram; > > } SaveStateEntry; > > > > > > @@ -1241,6 +1243,10 @@ int register_savevm_live(DeviceState *dev, > > se->opaque = opaque; > > se->vmsd = NULL; > > se->no_migrate = 0; > > + /* if this is a live_savem then set is_ram */ > > + if (save_live_state != NULL) { > > + se->is_ram = 1; > > + } > > > > if (dev&& dev->parent_bus&& dev->parent_bus->info->get_dev_path) { > > char *id = dev->parent_bus->info->get_dev_path(dev); > > @@ -1728,6 +1734,45 @@ out: > > return ret; > > } > > > > +static int qemu_save_device_state(QEMUFile *f) > > +{ > > + SaveStateEntry *se; > > + > > + qemu_put_be32(f, QEMU_VM_FILE_MAGIC); > > + qemu_put_be32(f, QEMU_VM_FILE_VERSION); > > + > > + cpu_synchronize_all_states(); > > + > > + QTAILQ_FOREACH(se,&savevm_handlers, entry) { > > + int len; > > + > > + if (se->is_ram) { > > + continue; > > + } > > + if (se->save_state == NULL&& se->vmsd == NULL) { > > + continue; > > + } > > + > > + /* Section type */ > > + qemu_put_byte(f, QEMU_VM_SECTION_FULL); > > + qemu_put_be32(f, se->section_id); > > + > > + /* ID string */ > > + len = strlen(se->idstr); > > + qemu_put_byte(f, len); > > + qemu_put_buffer(f, (uint8_t *)se->idstr, len); > > + > > + qemu_put_be32(f, se->instance_id); > > + qemu_put_be32(f, se->version_id); > > + > > + vmstate_save(f, se); > > + } > > + > > + qemu_put_byte(f, QEMU_VM_EOF); > > + > > + return qemu_file_get_error(f); > > +} > > + > > static SaveStateEntry *find_se(const char *idstr, int instance_id) > > { > > SaveStateEntry *se; > > @@ -2109,6 +2154,32 @@ void do_savevm(Monitor *mon, const QDict *qdict) > > vm_start(); > > } > > > > +void qmp_save_devices(const char *filename, Error **errp) > > +{ > > + QEMUFile *f; > > + int saved_vm_running; > > + int ret; > > + > > + saved_vm_running = runstate_is_running(); > > + vm_stop(RUN_STATE_SAVE_VM); > > + > > + f = qemu_fopen(filename, "wb"); > > + if (!f) { > > + error_set(errp, QERR_OPEN_FILE_FAILED, filename); > > + goto the_end; > > + } > > + ret = qemu_save_device_state(f); > > + qemu_fclose(f); > > + if (ret< 0) { > > + error_set(errp, QERR_IO_ERROR); > > + } > > + > > + the_end: > > + if (saved_vm_running) > > + vm_start(); > > + return; > > +} > > + > > int load_vmstate(const char *name) > > { > > BlockDriverState *bs, *bs_vm_state; >
On Thu, 15 Mar 2012, Anthony Liguori wrote:> > diff --git a/qapi-schema.json b/qapi-schema.json > > index d0b6792..7f938ff 100644 > > --- a/qapi-schema.json > > +++ b/qapi-schema.json > > @@ -1593,3 +1593,21 @@ > > { ''command'': ''qom-list-types'', > > ''data'': { ''*implements'': ''str'', ''*abstract'': ''bool'' }, > > ''returns'': [ ''ObjectTypeInfo'' ] } > > + > > +## > > +# @save_devices: > > +# > > +# Save the state of all devices to file. The RAM and the block devices > > +# of the VM are not saved by this command. > > +# > > +# @filename: the file to save the state of the devices to as binary > > +# data. See save_devices.txt for a description of the binary format. > > +# > > +# Returns: Nothing on success > > +# If @filename cannot be opened, OpenFileFailed > > +# If an I/O error occurs while writing the file, IOError > > +# > > +# Since: 1.0 > > Since: 1.1. > > Otherwise Reviewed-by: Anthony Liguori <aliguori@us.ibm.com>Thanks!
On Thu, 15 Mar 2012, Luiz Capitulino wrote:> On Thu, 15 Mar 2012 15:16:15 -0500 > Anthony Liguori <anthony@codemonkey.ws> wrote: > > > On 03/15/2012 01:19 PM, Stefano Stabellini wrote: > > > - add an "is_ram" flag to SaveStateEntry; > > > > > > - register_savevm_live sets is_ram for live_savevm devices; > > > > > > - introduce a "save_devices" QAPI command that can be used to save > > > the state of all devices, but not the RAM or the block devices of the > > > VM. > > > > > > Changes in v6: > > > > > > - remove the is_ram parameter from register_savevm_live and sets is_ram > > > if the device is a live_savevm device; > > > > > > - introduce save_devices as a QAPI command, write a better description > > > for it; > > > > > > - fix CODING_STYLE; > > > > > > - introduce a new doc to explain the save format used by save_devices. > > > > > > Signed-off-by: Stefano Stabellini<stefano.stabellini@eu.citrix.com> > > > CC: Anthony Liguori<anthony@codemonkey.ws> > > > CC: Luiz Capitulino<lcapitulino@redhat.com> > > > --- > > > docs/save_devices.txt | 33 ++++++++++++++++++++++ > > > qapi-schema.json | 18 ++++++++++++ > > > qmp-commands.hx | 25 +++++++++++++++++ > > > savevm.c | 71 +++++++++++++++++++++++++++++++++++++++++++++++++ > > > 4 files changed, 147 insertions(+), 0 deletions(-) > > > create mode 100644 docs/save_devices.txt > > > > > > diff --git a/docs/save_devices.txt b/docs/save_devices.txt > > > new file mode 100644 > > > index 0000000..79915d2 > > > --- /dev/null > > > +++ b/docs/save_devices.txt > > > @@ -0,0 +1,33 @@ > > > += Save Devices > > > + > > > +QEMU has code to load/save the state of the guest that it is running. > > > +These are two complementary operations. Saving the state just does > > > +that, saves the state for each device that the guest is running. > > > + > > > +These operations are normally used with migration (see migration.txt), > > > +however it is also possible to save the state of all devices to file, > > > +without saving the RAM or the block devices of the VM. > > > + > > > +This operation is called "save_devices" (see QMP/qmp-commands.txt). > > > + > > > + > > > +The binary format used in the file is the following: > > > + > > > + > > > +------------------------------------------- > > > + > > > +32 bit big endian: QEMU_VM_FILE_MAGIC > > > +32 bit big endian: QEMU_VM_FILE_VERSION > > > + > > > +for_each_device > > > +{ > > > + 8 bit: QEMU_VM_SECTION_FULL > > > + 32 bit big endian: section_id > > > + 8 bit: idstr (ID string) length > > > + string: idstr (ID string) > > > + 32 bit big endian: instance_id > > > + 32 bit big endian: version_id > > > + buffer: device specific data > > > +} > > > + > > > +8 bit: QEMU_VM_EOF > > > diff --git a/qapi-schema.json b/qapi-schema.json > > > index d0b6792..7f938ff 100644 > > > --- a/qapi-schema.json > > > +++ b/qapi-schema.json > > > @@ -1593,3 +1593,21 @@ > > > { ''command'': ''qom-list-types'', > > > ''data'': { ''*implements'': ''str'', ''*abstract'': ''bool'' }, > > > ''returns'': [ ''ObjectTypeInfo'' ] } > > > + > > > +## > > > +# @save_devices: > > > +# > > > +# Save the state of all devices to file. The RAM and the block devices > > > +# of the VM are not saved by this command. > > > +# > > > +# @filename: the file to save the state of the devices to as binary > > > +# data. See save_devices.txt for a description of the binary format. > > > +# > > > +# Returns: Nothing on success > > > +# If @filename cannot be opened, OpenFileFailed > > > +# If an I/O error occurs while writing the file, IOError > > > +# > > > +# Since: 1.0 > > > > Since: 1.1. > > > > Otherwise Reviewed-by: Anthony Liguori <aliguori@us.ibm.com> > > Looks fine to me FWIW, my only nitpick is that we use an hyphen instead of the > underline in qmp command names, but I''d call this save-devices-state.OK, I''ll rename and resend.> Don''t you want this in HMP too, btw?It is basically useless because the data saved by save_devices needs to be packet together with other data before any tools can actually use it.
On 03/15/2012 02:34 PM, Luiz Capitulino wrote:> On Thu, 15 Mar 2012 15:16:15 -0500 > Anthony Liguori <anthony@codemonkey.ws> wrote: > >> On 03/15/2012 01:19 PM, Stefano Stabellini wrote: >>> - add an "is_ram" flag to SaveStateEntry; >>> >>> - register_savevm_live sets is_ram for live_savevm devices; >>> >>> - introduce a "save_devices" QAPI command that can be used to save >>> the state of all devices, but not the RAM or the block devices of the >>> VM. >>>>>> +SQMP >>> +save_devices >>> +------- >>> + >>> +Save the state of all devices to file. The RAM and the block devices >>> +of the VM are not saved by this command. >>> + >>> +Arguments: >>> + >>> +- "filename": the file to save the state of the devices to as binary >>> +data. See save_devices.txt for a description of the binary format. >>> + >>> +Example: >>> + >>> +-> { "execute": "save_devices", "arguments": { "filename": "/tmp/save" } }Why are we adding yet another command without support for passing in an fd instead of having qemu open the file name? Libvirt really does want to move to an ability to pass in fds, and the more commands we add that open files, the harder converting them all will be. -- Eric Blake eblake@redhat.com +1-919-301-3266 Libvirt virtualization library http://libvirt.org
On Fri, 16 Mar 2012, Eric Blake wrote:> On 03/15/2012 02:34 PM, Luiz Capitulino wrote: > > On Thu, 15 Mar 2012 15:16:15 -0500 > > Anthony Liguori <anthony@codemonkey.ws> wrote: > > > >> On 03/15/2012 01:19 PM, Stefano Stabellini wrote: > >>> - add an "is_ram" flag to SaveStateEntry; > >>> > >>> - register_savevm_live sets is_ram for live_savevm devices; > >>> > >>> - introduce a "save_devices" QAPI command that can be used to save > >>> the state of all devices, but not the RAM or the block devices of the > >>> VM. > >>> > > >>> +SQMP > >>> +save_devices > >>> +------- > >>> + > >>> +Save the state of all devices to file. The RAM and the block devices > >>> +of the VM are not saved by this command. > >>> + > >>> +Arguments: > >>> + > >>> +- "filename": the file to save the state of the devices to as binary > >>> +data. See save_devices.txt for a description of the binary format. > >>> + > >>> +Example: > >>> + > >>> +-> { "execute": "save_devices", "arguments": { "filename": "/tmp/save" } } > > Why are we adding yet another command without support for passing in an > fd instead of having qemu open the file name? Libvirt really does want > to move to an ability to pass in fds, and the more commands we add that > open files, the harder converting them all will be.This command is not meant to be used by libvirt directly: it is meant to be used by libxenlight. Libvirt is just going to call libxl_domain_suspend and behind the scenes libxl is going to take care of everything.