Hi All,
I have some queries regarding unpinning and unhooking top level shadow page
table for HVM guest on page fault.
whenever we try to make some space prior to allocating the shadow using
prealloc_shadaow() call
1. we do back traversing of per domain list of top level shadow pages and try to
unpin them using function call sh_unpin()
but in unpinning shadow we are unsetting pin bit in page->count_info and
decrement the reference count of shadow page
using call to sh_put_ref() [define in xen/arch/x86/mm/shadow/private.h]
but in this function I am not able to understand why this condition is unlikely?
if ( unlikely(nx == 0) )
sh_destroy_shadow(v, smfn);
as we are trying to make space for new shadow which would be created using
shadow_alloc()
so this sh_destroy_shadow must be called for any one of the entry in toplevel
list to free space of at least required order
and put back the pages in freelist of shadow pool ?
2. if we could not free space for shadow creation by this then why are we not
pinning all the top level pages again while we are
marking them unpinned while traversing the top level list?
Or is it not necessary to pinned all top level shadows in list? or it has some
other significance?
3. Also after this if still space is not free then we try to unhooking the same
toplevel list by going though each entry in list and
marking corresponding PML4 table''s entries as 0 if that entry was
marked PRESENT.
But I am not able to understand how this will return pages back to per domain
freelist of shadow pages?
Please correct if there is something wrong in my understanding or if I have
missed any piece of code in this flow
Plz provide your valuable replies to clear my queries/doubts.
Jeet
__________________________________________________________
Yahoo! India Answers: Share what you know. Learn something new
http://in.answers.yahoo.com/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
At 21:12 +0530 on 14 Mar (1173906759), jeet wrote:> 1. we do back traversing of per domain list of top level shadow pages and try to unpin them using function call sh_unpin() > > but in unpinning shadow we are unsetting pin bit in page->count_info and decrement the reference count of shadow page > using call to sh_put_ref() [define in xen/arch/x86/mm/shadow/private.h] > > but in this function I am not able to understand why this condition is unlikely? > > if ( unlikely(nx == 0) ) > sh_destroy_shadow(v, smfn);Because sh_put_ref is called in lots of other places too...> as we are trying to make space for new shadow which would be created using shadow_alloc() > so this sh_destroy_shadow must be called for any one of the entry in toplevel list to free space of at least required order > and put back the pages in freelist of shadow pool ?I don''t understand your question. When we need to free up some shadow memory, we walk the list of pinned shadows and unpin them. We walk them in reverse order because we hope that less useful shadows will be at the end. When we unpin a shadow, if it''s not in use right now, its refcount falls to zero, which causes the whole tree of shadows below it to be recursively destroyed. If that doesn''t free up enough memeory, we try the next one.> 3. Also after this if still space is not free then we try to unhooking the same toplevel list by going though each entry in list and > marking corresponding PML4 table''s entries as 0 if that entry was marked PRESENT. > > But I am not able to understand how this will return pages back to per domain freelist of shadow pages?Because it will drop the refcount on all the lower-level shadows that were being pointed to by those entries, which will cause some of them to be destroyed. When we have completed both scans of the pinned-shadows list, the only shadows still allocated will be the top-level shadows for each vcpu in the guest, and they will have no lower-level shadows attached to them. Cheers, Tim. -- Tim Deegan <Tim.Deegan@xensource.com>, XenSource UK Limited Registered office c/o EC2Y 5EB, UK; company number 05334508 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
HI Tim Thanks for reply> 1. we do back traversing of per domain list of top level shadow pages and try to unpin them using function call sh_unpin() > > but in unpinning shadow we are unsetting pin bit in page->count_info and decrement the reference count of shadow page > using call to sh_put_ref() [define in xen/arch/x86/mm/shadow/private.h] > > but in this function I am not able to understand why this condition is unlikely? > > if ( unlikely(nx == 0) ) > sh_destroy_shadow(v, smfn);| Because sh_put_ref is called in lots of other places too...> as we are trying to make space for new shadow which would be created using shadow_alloc() > so this sh_destroy_shadow must be called for any one of the entry in toplevel list to free space of at least required order > and put back the pages in freelist of shadow pool ?| I don''t understand your question. | | When we need to free up some shadow memory, we walk the list of pinned | shadows and unpin them. We walk them in reverse order because we hope | that less useful shadows will be at the end. | | When we unpin a shadow, if it''s not in use right now, its refcount falls | to zero, which causes the whole tree of shadows below it to be | recursively destroyed. My question was for confirming my understanding that during unpinning it should destroy shadow recursively. Thanks for clarifying my understanding | If that doesn''t free up enough memeory, we try the next one.> 3. Also after this if still space is not free then we try to unhooking the same toplevel list by going though each entry in list and > marking corresponding PML4 table''s entries as 0 if that entry was marked PRESENT. > > But I am not able to understand how this will return pages back to per domain freelist of shadow pages?| Because it will drop the refcount on all the lower-level shadows that | were being pointed to by those entries, which will cause some of them to | be destroyed. Sorry but this is still not clear to me how this will drop reference count of low level shadow pointed by shadow Top PT entries by putting 0 in shadow top level PT entries, which are PRESENT. And how & where these low level shadow would be destroyed?. please explain this more. | | When we have completed both scans of the pinned-shadows list, the only | shadows still allocated will be the top-level shadows for each vcpu in | the guest, and they will have no lower-level shadows attached to them. So when handling page fault after this unhooking (If as explained above this will remove all references of low level shadow ) we need to create all the low level shadow again recurrsively (shadow_get_and_create_l1e()) and put the entry in top level shadow of currently executing process. is my understanding correct? All also after both scan these Top level shadow will have pinned flag clear. As we are not pinning them again after unpinning in sh_unpin(). Right? Jeet ____ __________________________________________________________ Yahoo! India Answers: Share what you know. Learn something new http://in.answers.yahoo.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
HI Tim & all Sorry for resending the same mail again. Please provide your comment/answer. Hi Tim, Thanks for reply> 1. we do back traversing of per domain list of top level shadow pages and try to unpin them using function call sh_unpin() > > but in unpinning shadow we are unsetting pin bit in page->count_info and decrement the reference count of shadow page > using call to sh_put_ref() [define in xen/arch/x86/mm/shadow/private.h] > > but in this function I am not able to understand why this condition is unlikely? > > if ( unlikely(nx == 0) ) > sh_destroy_shadow(v, smfn);| Because sh_put_ref is called in lots of other places too...> as we are trying to make space for new shadow which would be created using shadow_alloc() > so this sh_destroy_shadow must be called for any one of the entry in toplevel list to free space of at least required order > and put back the pages in freelist of shadow pool ?| I don''t understand your question. | | When we need to free up some shadow memory, we walk the list of pinned | shadows and unpin them. We walk them in reverse order because we hope | that less useful shadows will be at the end. | | When we unpin a shadow, if it''s not in use right now, its refcount falls | to zero, which causes the whole tree of shadows below it to be | recursively destroyed. My question was for confirming my understanding that during unpinning it should destroy shadow recursively. Thanks for clarifying my understanding | If that doesn''t free up enough memeory, we try the next one.> 3. Also after this if still space is not free then we try to unhooking the same toplevel list by going though each entry in list and > marking corresponding PML4 table''s entries as 0 if that entry was marked PRESENT. > > But I am not able to understand how this will return pages back to per domain freelist of shadow pages?| Because it will drop the refcount on all the lower-level shadows that | were being pointed to by those entries, which will cause some of them to | be destroyed. Sorry but this is still not clear to me how this will drop reference count of low level shadow pointed by shadow Top PT entries by putting 0 in shadow top level PT entries, which are PRESENT. And how & where these low level shadow would be destroyed?. please explain this more. | | When we have completed both scans of the pinned-shadows list, the only | shadows still allocated will be the top-level shadows for each vcpu in | the guest, and they will have no lower-level shadows attached to them. So when handling page fault after this unhooking (If as explained above this will remove all references of low level shadow ) we need to create all the low level shadow again recurrsively (shadow_get_and_create_l1e()) and put the entry in top level shadow of currently executing process. is my understanding correct? All also after both scan these Top level shadow will have pinned flag clear. As we are not pinning them again after unpinning in sh_unpin(). Right? Jeet __________________________________________________________ Yahoo! India Answers: Share what you know. Learn something new http://in.answers.yahoo.com/ _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tim Deegan
2007-Mar-19 10:56 UTC
Re: [Xen-devel] Re: [Queries] Unpinning and Unhooking shadow
At 10:47 +0530 on 15 Mar (1173955665), jeet wrote:> Sorry but this is still not clear to me how this will drop reference > count of low level shadow pointed by shadow Top PT entries by putting > 0 in shadow top level PT entries, which are PRESENT.shadow_unhook_mappings() calls sh_unhook_*_mappings() calls shadow_set_l*e() with an empty entry calls sh_put_ref() on the MFN in the previous entry calls sh_destroy_shadow() if that was the last reference to that MFN (which recursively drops references to lower level page tables and so on).> So when handling page fault after this unhooking (If as explained > above this will remove all references of low level shadow ) we need to > create all the low level shadow again recurrsively > (shadow_get_and_create_l1e()) and put the entry in top level shadow of > currently executing process. is my understanding correct?Only if we managed somehow to get as far as unhooking the entries in a currently running process. In the vast majority of cases the first pass (unpinning) will free enough memory before we get anywhere near this.> All also after both scan these Top level shadow will have pinned flag > clear. As we are not pinning them again after unpinning in > sh_unpin(). Right?Yes. Though if the guest reloads CR3 with the same value it will get repinned then. And in any case if we hit this case very often performance will already be shot by the cost of all those recursive teardowns. And if the shadow memory is properly provisioned, it''s probably a sign that the guest is thrashing, so the shadow performance is the least of our problems. Cheers, Tim. -- Tim Deegan <Tim.Deegan@xensource.com>, XenSource UK Limited Registered office c/o EC2Y 5EB, UK; company number 05334508 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel