RE: [Xen-devel] [PATCH] (resent) NUMA node migration

Ian Pratt
Sat, 22 Dec 2007 01:15:43 -0000

> > I am aware that using live migration isn''t the best approach
(takes
> > twice the memory and quite some time), but it''s less
intrusive and
> works
> > fine (given localhost migration stability...)
>
> Is this really using localhost live migration to move a domain from
one
> NUMA node to another on the same host?

Yep, it really is.

> Why isn''t there a simpler way?

Well, you can''t beat using live migration for code simplicity :)

Doing page migration for HVM guests is easy as you can just stop the
guest, copy a bunch of pages and update the p2m table, flush the shadow
page table cache and all VCPU''s TLBs, then resume. Doing it live is a
little tricky as you have to go from MFN''s to PTE''s which
currently
requires a full shadow page table scan. [Though some of the experimental
page sharing patches maintain linked lists of backpointers, and we could
switch to a shadow mode that supports this while doing page migration]

PV guests are a little more challenging as all references in the
direct-mode page tables need to be updated. We also need to make sure
that the guest isn''t holding MFNs outside of pagetables, so we need to
get all the VCPUs into a known state. The best way of handling this is
to use the PV fast checkpoint support to freeze the guest, copy the
badly located pages, scan and update all pagetables, resume from
checkpoint. This would make a nice little project for someone...

Ian