Xen devel - Aug 2009 - can dom0 modify Shadow PT of HVM domU?

Hi all,

Can Xen hypervisor modify HVM domU's Shadow page table, under the dom0's
context, like trapped from dom0's hypercall?

I think it have to call 2 functions at least: guest_walk_tables() and
flush_tlb_all(). Can these 2 functions called in dom0's context?

In my test, if hypervisor tries to modify HVM's shadow page table, it will
bring down the whole system. I am not sure what's the reason.

Thanks,
Wu

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

At 09:35 +0100 on 26 Aug (1251279335), Wu Bingzheng
wrote:
> Can Xen hypervisor modify HVM domU''s Shadow page table, under the
> dom0''s context, like trapped from dom0''s hypercall?
Yes, and it sometimes does (e.g. dom0 hypercalls that change domU''s p2m
tables cause changes indirectly in the shadows).
> I think it have to call 2 functions at least: guest_walk_tables() and
> flush_tlb_all(). Can these 2 functions called in dom0''s context?
Yes, but they''re not nearly enough to safely modify the shadow
pagetables.  There''s a lot of reference-counting and concurrency code
in
there.  The paging_* function calls are really the only sensible way to
interact with the shadow pagetables code.
> In my test, if hypervisor tries to modify HVM''s shadow page table,
it
> will bring down the whole system. I am not sure what''s the reason.
Why do you want to modify the shadow pagetables from dom0?  They''re
probably the wrong place to be trying to do things since (a) they don''t
exist on EPT/NPT-capable hardware, and (b) they can get discarded and
rebuilt by Xen at any time.

Cheers,

Tim.

-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

Thanks very much for your detailed answer.

What I am doing is that:

I add a monitor in hypervisor to monitor some behaviors (like system call) of
HVM domU.
If that behavior happens, the monitor 1) pauses this domU [1], 2) notify dom0.
When dom0 gets the notification, it makes a decision, like killing the process
(who calls system-call) in domU, or not.
Then hypervisor gets the killing request. To do the killing, I implement like
this:
 1) set the domU's page, which contains the EIP before paused,  NX
(none-execute) in shadow page table;
 2) then unpause domU
 3) then domU will invoke a #PF. Hypervisor can catch this #PF, and then kill
the process in domU [2].

So, I have to modify the domU's shadow PT in dom0's context, to make the
domU trap into hypervisor.
Am I right? or is there a better way to make a paused domU trap into hypervisor?

[1] To pause domU, I call domain_pause_for_debugger();
[2] To kill a process in domU, I call vmx_inject_hw_exception(v, TRAP_gp_fault,
0);

Thanks,
Wu


email:wubingzheng@163.com在2009-08-26，"Tim Deegan"
<Tim.Deegan@citrix.com> 写道：
>Hi,
>
>At 09:35 +0100 on 26 Aug (1251279335), Wu Bingzheng wrote:
>> Can Xen hypervisor modify HVM domU's Shadow page table, under the
>> dom0's context, like trapped from dom0's hypercall?
>
>Yes, and it sometimes does (e.g. dom0 hypercalls that change domU's p2m
>tables cause changes indirectly in the shadows).
>
>> I think it have to call 2 functions at least: guest_walk_tables() and
>> flush_tlb_all(). Can these 2 functions called in dom0's context?
>
>Yes, but they're not nearly enough to safely modify the shadow
>pagetables.  There's a lot of reference-counting and concurrency code in
>there.  The paging_* function calls are really the only sensible way to
>interact with the shadow pagetables code.
>
>> In my test, if hypervisor tries to modify HVM's shadow page table,
it
>> will bring down the whole system. I am not sure what's the reason.
>
>Why do you want to modify the shadow pagetables from dom0?  They're
>probably the wrong place to be trying to do things since (a) they don't
>exist on EPT/NPT-capable hardware, and (b) they can get discarded and
>rebuilt by Xen at any time.
>
>Cheers,
>
>Tim.
>
>-- 
>Tim Deegan <Tim.Deegan@citrix.com>
>Principal Software Engineer, Citrix Systems (R&D) Ltd.
>[Company #02300071, SL9 0DZ, UK.]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

At 10:30 +0100 on 26 Aug (1251282600), Wu Bingzheng
wrote:
> I add a monitor in hypervisor to monitor some behaviors (like system call)
of HVM domU.
> If that behavior happens, the monitor 1) pauses this domU [1], 2) notify
dom0.
> When dom0 gets the notification, it makes a decision, like killing the
process (who calls system-call) in domU, or not.
> Then hypervisor gets the killing request. To do the killing, I implement
like this:
>  1) set the domU''s page, which contains the EIP before paused,  NX
(none-execute) in shadow page table;
>  2) then unpause domU
>  3) then domU will invoke a #PF. Hypervisor can catch this #PF, and then
kill the process in domU [2].
I see.  That seems like a pretty convoluted way of doing it.  Why don''t
you just inject #GP (or whatever) straight into the guest from your dom0
hypercall?

Marking the guest page NX in the shadows isn''t guaranteed to work
anyway
since the shadow pagetables could get discarded and rebuilt before the
guest resumes running.

Cheers,

Tim.
> So, I have to modify the domU''s shadow PT in dom0''s
context, to make the domU trap into hypervisor.
> Am I right? or is there a better way to make a paused domU trap into
hypervisor?
> 
> [1] To pause domU, I call domain_pause_for_debugger();
> [2] To kill a process in domU, I call vmx_inject_hw_exception(v,
TRAP_gp_fault, 0);
> 
> Thanks,
> Wu
> 
> 
> email:wubingzheng@163.com??2009-08-26??"Tim Deegan"
<Tim.Deegan@citrix.com> ??????
> >Hi,
> >
> >At 09:35 +0100 on 26 Aug (1251279335), Wu Bingzheng wrote:
> >> Can Xen hypervisor modify HVM domU''s Shadow page table,
under the
> >> dom0''s context, like trapped from dom0''s
hypercall?
> >
> >Yes, and it sometimes does (e.g. dom0 hypercalls that change
domU''s p2m
> >tables cause changes indirectly in the shadows).
> >
> >> I think it have to call 2 functions at least: guest_walk_tables()
and
> >> flush_tlb_all(). Can these 2 functions called in dom0''s
context?
> >
> >Yes, but they''re not nearly enough to safely modify the shadow
> >pagetables.  There''s a lot of reference-counting and
concurrency code in
> >there.  The paging_* function calls are really the only sensible way to
> >interact with the shadow pagetables code.
> >
> >> In my test, if hypervisor tries to modify HVM''s shadow
page table, it
> >> will bring down the whole system. I am not sure what''s
the reason.
> >
> >Why do you want to modify the shadow pagetables from dom0? 
They''re
> >probably the wrong place to be trying to do things since (a) they
don''t
> >exist on EPT/NPT-capable hardware, and (b) they can get discarded and
> >rebuilt by Xen at any time.
> >
> >Cheers,
> >
> >Tim.
> >
> >--
> >Tim Deegan <Tim.Deegan@citrix.com>
> >Principal Software Engineer, Citrix Systems (R&D) Ltd.
> >[Company #02300071, SL9 0DZ, UK.]
> 
> ________________________________
> ??????????????????????,www.yeah.net<http://www.yeah.net/?from=footer>
-- 
Tim Deegan <Tim.Deegan@citrix.com>
Principal Software Engineer, Citrix Systems (R&D) Ltd.
[Company #02300071, SL9 0DZ, UK.]

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

在2009-08-26，"Tim Deegan" <Tim.Deegan@citrix.com>
写道：
>At 10:30 +0100 on 26 Aug (1251282600), Wu Bingzheng wrote:
>> I add a monitor in hypervisor to monitor some behaviors (like system
call) of HVM domU.
>> If that behavior happens, the monitor 1) pauses this domU [1], 2)
notify dom0.
>> When dom0 gets the notification, it makes a decision, like killing the
process (who calls system-call) in domU, or not.
>> Then hypervisor gets the killing request. To do the killing, I
implement like this:
>>  1) set the domU's page, which contains the EIP before paused,  NX
(none-execute) in shadow page table;
>>  2) then unpause domU
>>  3) then domU will invoke a #PF. Hypervisor can catch this #PF, and
then kill the process in domU [2].
>
>I see.  That seems like a pretty convoluted way of doing it.  Why don't
>you just inject #GP (or whatever) straight into the guest from your dom0
>hypercall?
I think vmx_inject_hw_exception() can't be called under dom0's context,
because it calls __vmread() to read VMCS.

Since you said that, it's wrong to modify HVM's shadow PT in dom0, I
have to find another way to do this.

Thanks,
Wu
>
>Marking the guest page NX in the shadows isn't guaranteed to work anyway
>since the shadow pagetables could get discarded and rebuilt before the
>guest resumes running.
>
>Cheers,
>
>Tim.
>
>> So, I have to modify the domU's shadow PT in dom0's context, to
make the domU trap into hypervisor.
>> Am I right? or is there a better way to make a paused domU trap into
hypervisor?
>> 
>> [1] To pause domU, I call domain_pause_for_debugger();
>> [2] To kill a process in domU, I call vmx_inject_hw_exception(v,
TRAP_gp_fault, 0);
>> 
>> Thanks,
>> Wu
>> 
>> 
>> email:wubingzheng@163.com??2009-08-26??"Tim Deegan"
<Tim.Deegan@citrix.com> ??????
>> >Hi,
>> >
>> >At 09:35 +0100 on 26 Aug (1251279335), Wu Bingzheng wrote:
>> >> Can Xen hypervisor modify HVM domU's Shadow page table,
under the
>> >> dom0's context, like trapped from dom0's hypercall?
>> >
>> >Yes, and it sometimes does (e.g. dom0 hypercalls that change
domU's p2m
>> >tables cause changes indirectly in the shadows).
>> >
>> >> I think it have to call 2 functions at least:
guest_walk_tables() and
>> >> flush_tlb_all(). Can these 2 functions called in dom0's
context?
>> >
>> >Yes, but they're not nearly enough to safely modify the shadow
>> >pagetables.  There's a lot of reference-counting and
concurrency code in
>> >there.  The paging_* function calls are really the only sensible
way to
>> >interact with the shadow pagetables code.
>> >
>> >> In my test, if hypervisor tries to modify HVM's shadow
page table, it
>> >> will bring down the whole system. I am not sure what's the
reason.
>> >
>> >Why do you want to modify the shadow pagetables from dom0? 
They're
>> >probably the wrong place to be trying to do things since (a) they
don't
>> >exist on EPT/NPT-capable hardware, and (b) they can get discarded
and
>> >rebuilt by Xen at any time.
>> >
>> >Cheers,
>> >
>> >Tim.
>> >
>> >--
>> >Tim Deegan <Tim.Deegan@citrix.com>
>> >Principal Software Engineer, Citrix Systems (R&D) Ltd.
>> >[Company #02300071, SL9 0DZ, UK.]
>> 
>> ________________________________
>>
??????????????????????,www.yeah.net<http://www.yeah.net/?from=footer>
>
>-- 
>Tim Deegan <Tim.Deegan@citrix.com>
>Principal Software Engineer, Citrix Systems (R&D) Ltd.
>[Company #02300071, SL9 0DZ, UK.]
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 26/08/2009 11:30, "Wu Bingzheng" <wubingzheng@163.com> wrote:
>> I see.  That seems like a pretty convoluted way of doing it.  Why
don''t
>> you just inject #GP (or whatever) straight into the guest from your
dom0
>> hypercall?
> 
> I think vmx_inject_hw_exception() can''t be called under
dom0''s context,
> because it calls __vmread() to read VMCS.
You can do vmx_vmcs_enter(v) at the very start of the function (before any
vmread or vmwrite) and vmx_vmcs_exit(v) at the very end. If you make this
change then it will be safe to call the function from dom0.

 -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Thanks very much. I will try that.

Wu


email:wubingzheng@163.com在2009-08-26，"Keir Fraser"
<keir.fraser@eu.citrix.com> 写道：
>On 26/08/2009 11:30, "Wu Bingzheng" <wubingzheng@163.com>
wrote:
>
>>> I see.  That seems like a pretty convoluted way of doing it.  Why
don't
>>> you just inject #GP (or whatever) straight into the guest from your
dom0
>>> hypercall?
>> 
>> I think vmx_inject_hw_exception() can't be called under dom0's
context,
>> because it calls __vmread() to read VMCS.
>
>You can do vmx_vmcs_enter(v) at the very start of the function (before any
>vmread or vmwrite) and vmx_vmcs_exit(v) at the very end. If you make this
>change then it will be safe to call the function from dom0.
>
> -- Keir
>
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel