Hi,
When doing live migration, the shadow code is translated to log
dirty mode. However, in the shadow_clean_dirty_bitmap function, all the
in-use shadow page tables are blew down. I think it is too crude, just as
the comment of the function says. I am trying to find a better solution, but
it does not succeeds. Would you please help me check my source code and give
me some suggestions? My current solution is walking all the in-use L1 shadow
page tables and clearing each L1 entry''s R/W bit if it has set.
I don''t know whether it is enough to remove these R/W bits for
live
migration. If not, how to make all the pages of a PV domain read-only again?
Thanks very much!
My source code is here.
sl1mfn = shadow_l2e_get_mfn(*sl2e);
if ( mfn_valid(sl1mfn) )
{
SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, 0, {
flags = shadow_l1e_get_flags(*sl1e);
target_mfn = shadow_l1e_get_mfn(*sl1e);
if(mfn_valid(target_mfn))
{
pg = mfn_to_page(target_mfn);
if ((pg != NULL) || ((pg->u.inuse.type_info
& PGT_type_mask) == PGT_writable_page))
{
if ((flags & _PAGE_PRESENT)
&&
(flags & _PAGE_RW))
{
shadow_l1e_t ro_sl1e
shadow_l1e_remove_flags(*sl1e, _PAGE_RW);
(void) shadow_set_l1e(v,
sl1e, ro_sl1e, sl1mfn);
}
}
}
});
}
zhujun
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
Tim Deegan
2009-Jun-24 09:16 UTC
Re: [Xen-devel] shadow_clean_dirty_bitmap''s another solution
At 10:08 +0100 on 24 Jun (1245838118), zhujun wrote:> Hi, > When doing live migration, the shadow code is translated to log dirty mode. However, in the shadow_clean_dirty_bitmap function, all the in-use shadow page tables are blew down. I think it is too crude, just as the comment of the function says. I am trying to find a better solution, but it does not succeeds. Would you please help me check my source code and give me some suggestions? My current solution is walking all the in-use L1 shadow page tables and clearing each L1 entry?s R/W bit if it has set. > I don?t know whether it is enough to remove these R/W bits for live migration. If not, how to make all the pages of a PV domain read-only again? Thanks very much! > My source code is here. > > sl1mfn = shadow_l2e_get_mfn(*sl2e);Stop right there! :) Why are you looking in sl2es? You need to make _all_ the sl1es read-only for this to work. You should be using hash_foreach() to walk through every l1 shadow. Have a look at sh_remove_all_mappings for an example of code that walks every sl1 table. Also your code below seems a bit too cautious: it should be OK to just check for_PAGE_PRESENT and _PAGE_RW and then clean _PAGE_RW. Cheers, Tim.> if ( mfn_valid(sl1mfn) ) > { > SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, 0, { > flags = shadow_l1e_get_flags(*sl1e); > target_mfn = shadow_l1e_get_mfn(*sl1e); > if(mfn_valid(target_mfn)) > { > pg = mfn_to_page(target_mfn); > if ((pg != NULL) || ((pg->u.inuse.type_info & PGT_type_mask) == PGT_writable_page)) > { > if ((flags & _PAGE_PRESENT) && (flags & _PAGE_RW)) > { > shadow_l1e_t ro_sl1e = shadow_l1e_remove_flags(*sl1e, _PAGE_RW); > (void) shadow_set_l1e(v, sl1e, ro_sl1e, sl1mfn); > } > } > } > }); > } > > > zhujun >Content-Description: ATT00001.txt> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel-- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, Citrix Systems (R&D) Ltd. [Company #02300071, SL9 0DZ, UK.] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks for your advice. I am sorry my code was not complete in last mail. I forgot to mention that I was looking in sl2es because I wanted to only walk all the in-use L1 shadow page tables. Those not in-use L1 shadow page tables are not walked. That''s because when a L1 shadow page table is selected to be in-use, the entries will be propagated from the guest page table again. Then, the R/W bit will be cleared because of log dirty mode. Just as shadow_blow_tables, it only blows the in-use shadow page tables. Others in the hash table are not blew down. Am I right? Yes, my code is too cautious, because I have tried too many times to remove my faults. Unfortunately, I am not sure about my solution. -----邮件原件----- 发件人: Tim Deegan [mailto:Tim.Deegan@citrix.com] 发送时间: 2009年6月24日 17:16 收件人: zhujun 抄送: xen-devel@lists.xensource.com 主题: Re: [Xen-devel] shadow_clean_dirty_bitmap''s another solution At 10:08 +0100 on 24 Jun (1245838118), zhujun wrote:> Hi, > When doing live migration, the shadow code is translated to logdirty mode. However, in the shadow_clean_dirty_bitmap function, all the in-use shadow page tables are blew down. I think it is too crude, just as the comment of the function says. I am trying to find a better solution, but it does not succeeds. Would you please help me check my source code and give me some suggestions? My current solution is walking all the in-use L1 shadow page tables and clearing each L1 entry?s R/W bit if it has set.> I don?t know whether it is enough to remove these R/W bits forlive migration. If not, how to make all the pages of a PV domain read-only again? Thanks very much!> My source code is here. > > sl1mfn = shadow_l2e_get_mfn(*sl2e);Stop right there! :) Why are you looking in sl2es? You need to make _all_ the sl1es read-only for this to work. You should be using hash_foreach() to walk through every l1 shadow. Have a look at sh_remove_all_mappings for an example of code that walks every sl1 table. Also your code below seems a bit too cautious: it should be OK to just check for_PAGE_PRESENT and _PAGE_RW and then clean _PAGE_RW. Cheers, Tim.> if ( mfn_valid(sl1mfn) ) > { > SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, 0, { > flags = shadow_l1e_get_flags(*sl1e); > target_mfn = shadow_l1e_get_mfn(*sl1e); > if(mfn_valid(target_mfn)) > { > pg = mfn_to_page(target_mfn); > if ((pg != NULL) ||((pg->u.inuse.type_info & PGT_type_mask) == PGT_writable_page))> { > if ((flags & _PAGE_PRESENT) &&(flags & _PAGE_RW))> { > shadow_l1e_t ro_sl1e shadow_l1e_remove_flags(*sl1e, _PAGE_RW); > (void) shadow_set_l1e(v,sl1e, ro_sl1e, sl1mfn);> } > } > } > }); > } > > > zhujun >Content-Description: ATT00001.txt> _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel-- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, Citrix Systems (R&D) Ltd. [Company #02300071, SL9 0DZ, UK.] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tim Deegan
2009-Jun-24 10:10 UTC
Re: ????: [Xen-devel] shadow_clean_dirty_bitmap''s another solution
At 10:39 +0100 on 24 Jun (1245839974), zhujun wrote:> Thanks for your advice. > I am sorry my code was not complete in last mail. I forgot to mention that I > was looking in sl2es because I wanted to only walk all the in-use L1 shadow > page tables. Those not in-use L1 shadow page tables are not walked. That''s > because when a L1 shadow page table is selected to be in-use, the entries > will be propagated from the guest page table again.I''m afraid not. The shadows are kept around and re-used. Re-propagating the whole pagetables whenever CR3 is changed would be very expensive. So if the shadow already has the R/W bit set it will keep being used without any more pagefaults, and the log-dirty bitmap won''t get updated on the next write.> Then, the R/W bit will > be cleared because of log dirty mode. Just as shadow_blow_tables, it only > blows the in-use shadow page tables. Others in the hash table are not blew > down. Am I right?No; shadow_blow_tables discards _all_ shadows, except the currently in-use top-level pagetables, which it empties. Cheers, Tim.> Yes, my code is too cautious, because I have tried too many times to remove > my faults. Unfortunately, I am not sure about my solution. > > > -----????????----- > ??????: Tim Deegan [mailto:Tim.Deegan@citrix.com] > ????????: 2009??6??24?? 17:16 > ??????: zhujun > ????: xen-devel@lists.xensource.com > ????: Re: [Xen-devel] shadow_clean_dirty_bitmap''s another solution > > At 10:08 +0100 on 24 Jun (1245838118), zhujun wrote: > > Hi, > > When doing live migration, the shadow code is translated to log > dirty mode. However, in the shadow_clean_dirty_bitmap function, all the > in-use shadow page tables are blew down. I think it is too crude, just as > the comment of the function says. I am trying to find a better solution, but > it does not succeeds. Would you please help me check my source code and give > me some suggestions? My current solution is walking all the in-use L1 shadow > page tables and clearing each L1 entry?s R/W bit if it has set. > > I don?t know whether it is enough to remove these R/W bits for > live migration. If not, how to make all the pages of a PV domain read-only > again? Thanks very much! > > My source code is here. > > > > sl1mfn = shadow_l2e_get_mfn(*sl2e); > > Stop right there! :) Why are you looking in sl2es? You need to make _all_ > the sl1es read-only for this to work. You should be using > hash_foreach() to walk through every l1 shadow. Have a look at > sh_remove_all_mappings for an example of code that walks every sl1 table. > > Also your code below seems a bit too cautious: it should be OK to just check > for_PAGE_PRESENT and _PAGE_RW and then clean _PAGE_RW. > > Cheers, > > Tim. > > > if ( mfn_valid(sl1mfn) ) > > { > > SHADOW_FOREACH_L1E(sl1mfn, sl1e, 0, 0, { > > flags = shadow_l1e_get_flags(*sl1e); > > target_mfn = shadow_l1e_get_mfn(*sl1e); > > if(mfn_valid(target_mfn)) > > { > > pg = mfn_to_page(target_mfn); > > if ((pg != NULL) || > ((pg->u.inuse.type_info & PGT_type_mask) == PGT_writable_page)) > > { > > if ((flags & _PAGE_PRESENT) && > (flags & _PAGE_RW)) > > { > > shadow_l1e_t ro_sl1e > shadow_l1e_remove_flags(*sl1e, _PAGE_RW); > > (void) shadow_set_l1e(v, > sl1e, ro_sl1e, sl1mfn); > > } > > } > > } > > }); > > } > > > > > > zhujun > > > > Content-Description: ATT00001.txt > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > > -- > Tim Deegan <Tim.Deegan@citrix.com> > Principal Software Engineer, Citrix Systems (R&D) Ltd. > [Company #02300071, SL9 0DZ, UK.] >-- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, Citrix Systems (R&D) Ltd. [Company #02300071, SL9 0DZ, UK.] _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel