Xen devel - Dec 2008 - [PATCH 0/4] stubdom-dm: Support live migration of HVM domain with stubdom-dm

Hi,

This series of patches is an attempt to support live migration of hvm
domains with stubdom-dm.

For hvm domains to live-migrate, we should create a log-dirty shared
memory segment between xc_save and qemu-dm. for a normal qemu-dm,
memory sharing is done by inter process communication interface
(shmget/shmat/shmctl), but it obviously doesn''t work for a stubdom-dm
because it is not a dom0 process but a domain itself.

The idea here is to allocate log-dirty the shared memory segment in
the stubdom-dm and to map the memory region into the xc_save process
by using grant table interface.

[PATCH 1/4] mini-os: Make utility function get_self_id() public
[PATCH 2/4] Log-dirty shared memory between stubdom-dm and xc_save
[PATCH 3/4] Log-dirty shared memory between stubdom-dm and xc_save
            (ioemu-remote part)
[PATCH 4/4] xend: Give stubdom flag to xc_save

Thanks,
-----------------------
Yosuke Iwamatsu
        NEC Corporation



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Yosuke Iwamatsu, le Tue 16 Dec 2008 18:22:20 +0900, a écrit
:
> This series of patches is an attempt to support live migration of hvm
> domains with stubdom-dm.
Err, I''m a bit surprised. I haven''t tested it since then, but
I''m sure
live migration was working with stubdoms a few months ago, without
the need for a logdirty shared buffer.  This works thanks to the few
lines at the end of the cpu_physical_memory_rw() function that tell the
hypervisor about modified memory.  Actually, the plan was even to use
that bit of code in all cases (not only non-stubdom), so as to get rid
of the logdirty shared buffer completely.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Samuel Thibault, le Tue 16 Dec 2008 11:02:31 +0100, a écrit
:
> Actually, the plan was even to use that bit of code in all cases (not
> only non-stubdom), so as to get rid of the logdirty shared buffer
> completely.
(That even _helps_ the shadow code, which can thus know that a pagetable
was replaced by data from the disk, for instance.)

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Samuel Thibault writes ("[Xen-devel] Re: [PATCH 0/4] stubdom-dm: Support
live migration of HVM domain with stubdom-dm"):
> Yosuke Iwamatsu, le Tue 16 Dec 2008 18:22:20 +0900, a écrit :
> > This series of patches is an attempt to support live migration of hvm
> > domains with stubdom-dm.
> 
> Err, I''m a bit surprised. I haven''t tested it since then,
but I''m sure
> live migration was working with stubdoms a few months ago, without
> the need for a logdirty shared buffer.  This works thanks to the few
> lines at the end of the cpu_physical_memory_rw() function that tell the
> hypervisor about modified memory.  Actually, the plan was even to use
> that bit of code in all cases (not only non-stubdom), so as to get rid
> of the logdirty shared buffer completely.
I take it you''re referring to this:

#ifdef CONFIG_STUBDOM
    if (logdirty_bitmap != NULL)
        xc_hvm_modified_memory(xc_handle, domid, _addr >>
TARGET_PAGE_BITS,
                ((_addr + _len + TARGET_PAGE_SIZE - 1) >>
TARGET_PAGE_BITS)
                    - (_addr >> TARGET_PAGE_BITS));
#endif

Yosuke Iwamatsu: is this not working for you then ?  It''s possible
that I''ve broken it somehow.

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson, le Tue 16 Dec 2008 14:04:53 +0000, a écrit
:
> Samuel Thibault writes ("[Xen-devel] Re: [PATCH 0/4] stubdom-dm:
Support live migration of HVM domain with stubdom-dm"):
> > This works thanks to the few lines at the end of the
> > cpu_physical_memory_rw() function that tell the hypervisor about
> > modified memory.  Actually, the plan was even to use that bit of
> > code in all cases (not only non-stubdom), so as to get rid of the
> > logdirty shared buffer completely.
> 
> I take it you''re referring to this:
> 
> #ifdef CONFIG_STUBDOM
>     if (logdirty_bitmap != NULL)
>         xc_hvm_modified_memory(xc_handle, domid, _addr >>
TARGET_PAGE_BITS,
>                 ((_addr + _len + TARGET_PAGE_SIZE - 1) >>
TARGET_PAGE_BITS)
>                     - (_addr >> TARGET_PAGE_BITS));
> #endif
Yes, precisely.

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson wrote:
> Samuel Thibault writes ("[Xen-devel] Re: [PATCH 0/4] stubdom-dm:
Support live migration of HVM domain with stubdom-dm"):
>> Yosuke Iwamatsu, le Tue 16 Dec 2008 18:22:20 +0900, a écrit :
>>> This series of patches is an attempt to support live migration of
hvm
>>> domains with stubdom-dm.
>> Err, I''m a bit surprised. I haven''t tested it since
then, but I''m sure
>> live migration was working with stubdoms a few months ago, without
>> the need for a logdirty shared buffer.  This works thanks to the few
>> lines at the end of the cpu_physical_memory_rw() function that tell the
>> hypervisor about modified memory.  Actually, the plan was even to use
>> that bit of code in all cases (not only non-stubdom), so as to get rid
>> of the logdirty shared buffer completely.
> 
> I take it you''re referring to this:
> 
> #ifdef CONFIG_STUBDOM
>     if (logdirty_bitmap != NULL)
>         xc_hvm_modified_memory(xc_handle, domid, _addr >>
TARGET_PAGE_BITS,
>                 ((_addr + _len + TARGET_PAGE_SIZE - 1) >>
TARGET_PAGE_BITS)
>                     - (_addr >> TARGET_PAGE_BITS));
> #endif
Ah, I didn''t notice this.
> Yosuke Iwamatsu: is this not working for you then ?  It''s possible
> that I''ve broken it somehow.
Live-migration fails because stubdom-dm doesn''t respond to the request
from xc_save for the logdirty bitmap segment. The attached patch
simply generated from old ioemu tree will fix the issue, then.

Derived from old xen internal tools/ioemu tree.
Signed-off-by: Yosuke Iwamatsu <y-iwamatsu@ab.jp.nec.com>

Thanks,
-- Yosuke


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Yosuke Iwamatsu, le Wed 17 Dec 2008 10:06:10 +0900, a écrit
:
> The attached patch
> simply generated from old ioemu tree will fix the issue, then.
> Derived from old xen internal tools/ioemu tree.
> Signed-off-by: Yosuke Iwamatsu <y-iwamatsu@ab.jp.nec.com>
Acked-by: Samuel Thibault <samuel.thibault@ens-lyon.org>

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Yosuke Iwamatsu writes ("Re: [Xen-devel] Re: [PATCH 0/4] stubdom-dm:
Support live migration of	HVM domain with
stubdom-dm"):
> Live-migration fails because stubdom-dm doesn''t respond to the
request
> from xc_save for the logdirty bitmap segment. The attached patch
> simply generated from old ioemu tree will fix the issue, then.
Thanks, applied.  I take it this should be backported to 3.3 ?

Ian.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Ian Jackson, le Wed 17 Dec 2008 11:39:46 +0000, a écrit
:
> Yosuke Iwamatsu writes ("Re: [Xen-devel] Re: [PATCH 0/4] stubdom-dm:
Support live migration of	HVM domain with stubdom-dm"):
> > Live-migration fails because stubdom-dm doesn''t respond to
the request
> > from xc_save for the logdirty bitmap segment. The attached patch
> > simply generated from old ioemu tree will fix the issue, then.
> 
> Thanks, applied.  I take it this should be backported to 3.3 ?
Yes (though in 3.3 the in-tree qemu is used by default).

Samuel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel