Xen devel - May 2004 - How does hypervisor handle the hypercalls from guest OSes?

Hi All !!

I''m trying to encapsulate some operations into "hypercall"
and pass the hypercall into hypervisor.

But I cannot find out how the hypervisor to deal with hypercalls from guest
OSes.

I find a function "do_dom0_op" in xeno-1.2.bk/xen/common/dom0_ops.c.

It looks like the function which would deal with hypercalls .

and I try to modify the code in do_dom0_op to handle my new hypercall, but I
failed.

can someone tell me how does hypervisor handle the hypercalls from guest OSes?

Cheers

-- C.K. Chiu , Taiwan

> I''m trying to encapsulate some operations into
"hypercall" and pass
> the hypercall into hypervisor.
>
> But I cannot find out how the hypervisor to deal with hypercalls
> from guest OSes.
Hypercalls go from ring 1 (xenolinux kernel) to ring 0; they are
invoked by using "int $0x82".

You can see the "top half" (guest side) of the hypercall interface by
looking at the file

   xenolinux-2.4.2x-sparse/include/asm-xen/hypervisor.h 

which includes a bunch of inline C functions like: 

static inline int HYPERVISOR_mmu_update(mmu_update_t *req, int count)
{
    int ret;
    __asm__ __volatile__ (
        TRAP_INSTR
        : "=a" (ret) : "0" (__HYPERVISOR_mmu_update),
        "b" (req), "c" (count) : "memory" );

    return ret;
}



You can see the "bottom half" (xen side) of the hypercall interface 
by looking at the files 

   xen/include/hypervisor-ifs/hypervisor-if.h 
   xen/arch/i386/entry.S 

; the former includes definitions of all the numbers used for the 
various hypercalls while latter handles the int $0x82 and decides
which actual hypercall to invoke (by using hypervisor_call_table; 
see the end of entry.S). 

> I find a function "do_dom0_op" in
xeno-1.2.bk/xen/common/dom0_ops.c.
> 
> It looks like the function which would deal with hypercalls .
No; a dom0_op is just one of the hypercalls; it is the general method
used for control software (running in domain 0) to invoke various 
operations within Xen. 

Depending on what you''re actually trying to do, adding a new dom0_op 
rather than a new hypercall might be the right thing to do though. 
> and I try to modify the code in do_dom0_op to handle my new
> hypercall, but I failed.
Yes. 

> can someone tell me how does hypervisor handle the hypercalls from
> guest OSes?
Hopefully the above description is of some use. 

cheers, 

S.


-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

for a new hypercall here are the files you need to change:

1. define a hypercall number + params in
xen/include/hypervisor-ifs/hypervisor-if.h or similar

2. add an entry corresponding to the hypercall number in the
hypervisor_call_table in xen/arch/i386/entry.S

3. implement the hypercall in xen (function as in entry.S)

4. provide an assempbly stub in XenoLinux for your hypercall like in:
xenolinux-2.4.26-sparse/include/asm-xen/hypervisor.h

then you can use the function name you use in hypervisor.h in XenoLinux.

alternatively you piggy-back your hypercall onto an existing one, if it
matches the function of that hypercall. quite a few hypercalls already
multiplex multiple related functions onto on hypercall.

Rolf

On Mon, 2004-05-10 at 14:55, K.C. Chiu wrote:
> Hi All !!
>  
> I''m trying to encapsulate some operations into
"hypercall" and pass
> the hypercall into hypervisor.
>  
> But I cannot find out how the hypervisor to deal with hypercalls
> from guest OSes.
>  
> I find a function "do_dom0_op" in
xeno-1.2.bk/xen/common/dom0_ops.c.
>  
> It looks like the function which would deal with hypercalls .
>  
> and I try to modify the code in do_dom0_op to handle my new hypercall,
> but I failed.
>  
> can someone tell me how does hypervisor handle the hypercalls from
> guest OSes?
>  
> Cheers
>  
> -- C.K. Chiu ,
> Taiwan


-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Thanks for your help !!

I would try to handle my new hypercall !

But I have another question, how do I debug in the Xen hypervisor side?

Just "printk" ?

However I could not find out any print message in /var/log/message and the
xen_read_console ?

Cheers

-- C.K. Chiu , Taiwan

----- Original Message ----- 
From: "Rolf Neugebauer" <rolf.neugebauer@intel.com>
To: "K.C. Chiu" <B8844014@stmail.cgu.edu.tw>
Cc: <rolf.neugebauer@intel.com>; <xen-devel@lists.sourceforge.net>
Sent: Monday, May 10, 2004 10:22 PM
Subject: Re: [Xen-devel] How does hypervisor handle the hypercalls fromguest
OSes?

> for a new hypercall here are the files you need to change:
>
> 1. define a hypercall number + params in
> xen/include/hypervisor-ifs/hypervisor-if.h or similar
>
> 2. add an entry corresponding to the hypercall number in the
> hypervisor_call_table in xen/arch/i386/entry.S
>
> 3. implement the hypercall in xen (function as in entry.S)
>
> 4. provide an assempbly stub in XenoLinux for your hypercall like in:
> xenolinux-2.4.26-sparse/include/asm-xen/hypervisor.h
>
> then you can use the function name you use in hypervisor.h in XenoLinux.
>
> alternatively you piggy-back your hypercall onto an existing one, if it
> matches the function of that hypercall. quite a few hypercalls already
> multiplex multiple related functions onto on hypercall.
>
> Rolf
>
> On Mon, 2004-05-10 at 14:55, K.C. Chiu wrote:
> > Hi All !!
> >
> > I''m trying to encapsulate some operations into
"hypercall" and pass
> > the hypercall into hypervisor.
> >
> > But I cannot find out how the hypervisor to deal with hypercalls
> > from guest OSes.
> >
> > I find a function "do_dom0_op" in
xeno-1.2.bk/xen/common/dom0_ops.c.
> >
> > It looks like the function which would deal with hypercalls .
> >
> > and I try to modify the code in do_dom0_op to handle my new hypercall,
> > but I failed.
> >
> > can someone tell me how does hypervisor handle the hypercalls from
> > guest OSes?
> >
> > Cheers
> >
> > -- C.K. Chiu ,
> > Taiwan
>
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by Sleepycat Software
> Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to
deliver
> higher performing products faster, at low TCO.
> http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel



-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

> But I have another question, how do I debug in the Xen hypervisor side?
> 
> Just "printk" ?
Connect something (e.g. another computer) to the serial console,
and boot Xen with serial console enabled.

As well as printk, Xen has a GDB stub. This is quite powerful,
and can be used for debugging domains and processes too.
 
> However I could not find out any print message in /var/log/message and the
> xen_read_console ?
"xen_dmesg.py" in dom0 will show you the console output from
*Xen*. (//var/log/messages and xen_read_console are domain
console tools).

For any serious debugging, get yourself a serial line. xen_dmesg
isn''t much use if either Xen or domain 0 are already blown to bits.

BTW: What does your new hypercall do?

Ian


-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Thanks a lot !!

Well, I''m trying to enhance the operating system security through VMM
technology.

For guest OS''s Audit log, I want to put the audit log file in domain
0''s
file system to avoid any modification from guest OSes.

therefore, I need to encapsulate a new command into dom0''s operation
hypercall and hanlde the command in hypervisor to read the audit log file.

Now I''m trying to find out how to handle the hypercalls and the event
notifications from hypervisor to guest OSes.

And these works may need your helps !! ^ ^

Cheers

-- C.K. Chiu , Taiwan

----- Original Message ----- 
From: "Ian Pratt" <Ian.Pratt@cl.cam.ac.uk>
To: "K.C. Chiu" <B8844014@stmail.cgu.edu.tw>
Cc: <rolf.neugebauer@intel.com>; <xen-devel@lists.sourceforge.net>;
<Ian.Pratt@cl.cam.ac.uk>
Sent: Tuesday, May 11, 2004 2:18 PM
Subject: Re: [Xen-devel] How does hypervisor handle the hypercalls fromguest
OSes?

>
> > But I have another question, how do I debug in the Xen hypervisor
side?
> >
> > Just "printk" ?
>
> Connect something (e.g. another computer) to the serial console,
> and boot Xen with serial console enabled.
>
> As well as printk, Xen has a GDB stub. This is quite powerful,
> and can be used for debugging domains and processes too.
>
> > However I could not find out any print message in /var/log/message and
the
> > xen_read_console ?
>
> "xen_dmesg.py" in dom0 will show you the console output from
> *Xen*. (//var/log/messages and xen_read_console are domain
> console tools).
>
> For any serious debugging, get yourself a serial line. xen_dmesg
> isn''t much use if either Xen or domain 0 are already blown to
bits.
>
> BTW: What does your new hypercall do?
>
> Ian
>
>
> -------------------------------------------------------
> This SF.Net email is sponsored by Sleepycat Software
> Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to
deliver
> higher performing products faster, at low TCO.
> http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/xen-devel



-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

> Well, I''m trying to enhance the operating system security through
VMM
> technology.
> 
> For guest OS''s Audit log, I want to put the audit log file in
domain 0''s
> file system to avoid any modification from guest OSes.
> 
> therefore, I need to encapsulate a new command into dom0''s
operation
> hypercall and hanlde the command in hypervisor to read the audit log file.
Use xen 1.3 ("unstable") as this provides much better support for
doing this -- there are generic communication and console paths
for between domains.

You could either just use the console connection to domain 0 (and
have xend log security messages to disk), or create a separate
console connection for security messages (again, modifying xend
to log to the messages).

Ian


-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

This is just a thumbs up that I''ve successfully upgraded my DP AMD64
machine to head with nodev set in Xen and the various drivers enabled
in xenolinux. The only gotcha was that, on the first pass, I didn''t
realize I needed to enable ethernet bridging.

The interesting part will come tomorrow after I''ve upgraded the
firmware
on the iSCSI initiator and try exporting LUNs to domains other than DOM0
(assuming that is intended to work at this point).



				-Kip


-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

> This is just a thumbs up that I''ve successfully upgraded my DP
AMD64
> machine to head with nodev set in Xen and the various drivers enabled
> in xenolinux. The only gotcha was that, on the first pass, I
didn''t
> realize I needed to enable ethernet bridging.
Yes. If you run the device drivers in domain 0, rather than a
special-purpose device-driver domain, then your IP connectivity
disappears from DOM0. You need to configure device ''nbe-br''
with your
IP/netmask, and update your forwarding rules to refer to nbe-br rather
than eth0.

In future we will also support methods of connectivity other than
bridging -- e.g., device-driver domain configured as gateway
router. For this we''ll have to provide some convenient way to send
netlink messages over the device-driver control interface.
> The interesting part will come tomorrow after I''ve upgraded the
firmware
> on the iSCSI initiator and try exporting LUNs to domains other than DOM0
> (assuming that is intended to work at this point).
Yes, there are new inter-domain device drivers for disc and
network. Just select ''new IO world'' when configuring
Xenolinux, but
exclude support for real physical device drivers (of course).

These drivers have only been tested with one domain using them at a
time, and the interface setup/teardown control in xend is very ropey
(will need rewriting properly). But it should work.

 -- Keir


-------------------------------------------------------
This SF.Net email is sponsored by Sleepycat Software
Learn developer strategies Cisco, Motorola, Ericsson & Lucent use to deliver
higher performing products faster, at low TCO.
http://www.sleepycat.com/telcomwpreg.php?From=osdnemail3
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

Hello!

I''m trying to modified some codes in Xen VMM.

But I can not understand how Xen VMM to share memory regions with domains!

I tried to imitate the block device in Xen.

In block device, it would get free page memory for process structure
blk_ring_base

and call the macro SHARE_PFN_WITH_DOMAIN to share with domains.

code is like this:

p->blk_ring_base = (blk_ring_t *)get_free_page(GFP_KERNEL);
clear_page(p->blk_ring_base);
SHARE_PFN_WITH_DOMAIN(virt_to_page(p->blk_ring_base), p->domain);

But I can''t not understand how guest OS to get these i/o data from
blk_ring_base.

I found some codes as below:

op.cmd = BLOCK_IO_OP_RING_ADDRESS;
(void)HYPERVISOR_block_io_op(&op);

It would ask the VMM where the io ring memory ! and get the I/O ring address
!!

set_fixmap(FIX_BLKRING_BASE, op.u.ring_mfn << PAGE_SHIFT);
blk_ring = (blk_ring_t *)fix_to_virt(FIX_BLKRING_BASE);

However I don''t understand these codes means ! How Xen VMM get the
block I/O
ring''s data ?

And I tried to imitate these codes to do share memory with domains !

At the Xen VMM sides:

In task structure, I added :

void *temp;
unsigned int temp_size;

long hypervisor_temp(struct task_struct *p, dom0_tmp_t *tmp)
{
        int ret;
        unsigned long cpu_mask = 0;
        int p_size;

        if ( sizeof(*p->temp) > PAGE_SIZE ) BUG();

        p->temp = (void *)get_free_page(GFP_KERNEL);
        clear_page(p->temp);
        p->temp = tmp->test;
        SHARE_PFN_WITH_DOMAIN(virt_to_page(p->temp), p->domain);

        cpu_mask = mark_guest_event(p, _EVENT_TEMP);
        guest_event_notify(cpu_mask);
        put_task_struct(p);

        return 0;
}

In block_io_op_t structure, I added :

unsigned long temp_mfn;

In do_block_io_op function added:

 case POLICY_ADDRESS:
        op.u.temp_mfn = virt_to_phys(p->temp) >> PAGE_SHIFT;
        ret = copy_to_user(u_block_io_op, &op, sizeof(op)) ? -EFAULT : 0;
        break;

At the guest OS sides:

void * temp;

static int __init setup_temp_event(void)
{
    block_io_op_t op;

    op.cmd = TEMP_ADDRESS;
    (void)HYPERVISOR_block_io_op(&op);

    set_fixmap(FIX_TEMP_BASE, op.u.temp_mfn << PAGE_SHIFT);
    temp = (void *)fix_to_virt(FIX_TEMP_BASE);

    (void)request_irq(_EVENT_POLICY, temp_irq, SA_SAMPLE_RANDOM,
"temp",
NULL);
    return 0;
}

However I got the error message like this :

" Kernel panic: Failed mmu update: c01c4cc0, 14 "

How can I solve this ?

thanks a lot !

----- Original Message ----- 
From: "Ian Pratt" <Ian.Pratt@cl.cam.ac.uk>
To: "K.C. Chiu" <B8844014@stmail.cgu.edu.tw>
Cc: "Ian Pratt" <Ian.Pratt@cl.cam.ac.uk>;
<rolf.neugebauer@intel.com>;
<xen-devel@lists.sourceforge.net>; <Ian.Pratt@cl.cam.ac.uk>
Sent: Tuesday, May 11, 2004 3:15 PM
Subject: Re: [Xen-devel] How does hypervisor handle the hypercalls fromguest
OSes?

>
> > Well, I''m trying to enhance the operating system security
through VMM
> > technology.
> >
> > For guest OS''s Audit log, I want to put the audit log file in
domain 0''s
> > file system to avoid any modification from guest OSes.
> >
> > therefore, I need to encapsulate a new command into dom0''s
operation
> > hypercall and hanlde the command in hypervisor to read the audit log
file.
>
> Use xen 1.3 ("unstable") as this provides much better support for
> doing this -- there are generic communication and console paths
> for between domains.
>
> You could either just use the console connection to domain 0 (and
> have xend log security messages to disk), or create a separate
> console connection for security messages (again, modifying xend
> to log to the messages).
>
> Ian



-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we''ll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel

A couple of comments...
> long hypervisor_temp(struct task_struct *p, dom0_tmp_t *tmp)
> {
>         int ret;
>         unsigned long cpu_mask = 0;
>         int p_size;
> 
>         if ( sizeof(*p->temp) > PAGE_SIZE ) BUG();
> 
>         p->temp = (void *)get_free_page(GFP_KERNEL);
>         clear_page(p->temp);
>         p->temp = tmp->test;
          ^^^^^^^^^^^^^^^^^^^^
The line above looks like debugging you''ve forgotten to remove?
>         SHARE_PFN_WITH_DOMAIN(virt_to_page(p->temp), p->domain);
> 
>         cpu_mask = mark_guest_event(p, _EVENT_TEMP);
>         guest_event_notify(cpu_mask);
>         put_task_struct(p);
          ^^^^^^^^^^^^^^^^^^^
You probably don''t want to decrement p''s reference count here.
p was
passed into the function as an argument, which usually means that
you are "borrowing" the caller''s reference -- and he probably
expects to get it back!

Everything else looks like it is probably okay. The first comment
above could be the problem though -- it looks like the code is plain
wrong. 

 -- Keir


-------------------------------------------------------
This SF.Net email is sponsored by: Oracle 10g
Get certified on the hottest thing ever to hit the market... Oracle 10g. 
Take an Oracle 10g class now, and we''ll give you the exam FREE.
http://ads.osdn.com/?ad_id=3149&alloc_id=8166&op=click
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/xen-devel