thr3ads.net - Xen devel - [Xen-devel] Question: How to map granted pages into domU properly? [Sep 2008]

If this information is useful, please help other people find it:
Share via:

abc

2008-Sep-04 15:09 UTC

[Xen-devel] Question: How to map granted pages into domU properly?

Hi,

I''m writing a diploma-thesis based on paravirtualized XEN v3.2.1 and 
openSuSE 11.0 (32bit PAE).
The idea of my thesis is to use XEN as a research platform for on-die 
message passing interface hardware. Therefore several domUs should 
communicate via a configurable kernel-module over shared-memory.

Currently I encountered a problem in granting kernel-pages from dom0 to 
domUs.
The pages can be mapped successfully. But when the kernel-module in 
domU, tries to read the content of this page, a kernel oops occurs.

Can you explain me how to grant pages and how to map them into domUs 
properly?




These are my sources (shortened):


Dom0 -----------------------------------------------

static inline unsigned long vaddr2pfn(void *vaddr) {

  struct page* Page = vmalloc_to_page(vaddr);
  unsigned long PFN = page_to_pfn(Page);
 
  return PFN;
}

  unsigned int ForeignDomainID = 12; // <-- Hard coded
  unsigned long Page = get_zeroed_page(0);
  unsigned long PFN = vaddr2pfn( (void *) Page);
  unsigned long MFN = virt_to_phys( (void *) PFN);
      
  grant_ref = gnttab_grant_foreign_access( ForeignDomainID, MFN, 0 );

----------------------------------------------------

grant_ref then is magically being transported to domU..

DomU (domID=12) ------------------------------------

  struct vm_struct *Area = alloc_vm_area(PAGE_SIZE);
  struct gnttab_map_grant_ref operation;
  gnttab_set_map_op(&operation, (unsigned long) Area->addr, 
GNTMAP_host_map | GNTMAP_application_map, grant_ref, 0);
  HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &operation, 1);
  // print content of operation to syslog..

  Unsigned long* Dword = (Unsigned long*) Area->addr;
  unsigned long Content = *Dword  // <-- BUG: unable to handle kernel 
paging request at d928a000

----------------------------------------------------

Dom0 - /var/log/messages ---------------------------

Sep  4 16:43:19 mpibench1 kernel: sm_communicator: instanciating in dom0
Sep  4 16:43:19 mpibench1 kernel: sm_communicator: granting memory to 
guest #12
Sep  4 16:43:19 mpibench1 kernel: sm_communicator: granted page 
@0xe0738000 (mfn=0x40020738) to domain #12
Sep  4 16:43:19 mpibench1 kernel: sm_communicator: grant_refs[1]==21 (0x15)

----------------------------------------------------

DomU - /var/log/messages ---------------------------

Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: instanciating 
in domU
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: mapping page 
#21 from domain #0
Sep  4 16:43:37 mpibench_guest2 kernel: klogd 1.4.1, ---------- state 
change ----------
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: new 
area={next=0xd7671760, addr=0xd928a000, size=0x2000, flags=0x1, 
pages=0x0, nr_pages=0, phys_addr=0x0}
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: 
HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &Operation, 1);
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: struct 
gnttab_map_grant_ref Operation = {:
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   /* IN 
parameters. */
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint64_t 
host_addr=0x00000000d928a000;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint32_t 
flags=2=GNTMAP_host_map;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   grant_ref_t 
ref=21;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   domid_t  dom=0;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   /* OUT 
parameters. */
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   int16_t  
status=0;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   
grant_handle_t handle=0;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint64_t 
dev_bus_addr=0x0000040020738000;
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:}
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: successfully 
mapped page @0xd928a000 (handle=0, phys=0x0000040020738000)
Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: GrantedPage 
@0xd928a000
Sep  4 16:43:37 mpibench_guest2 kernel: BUG: unable to handle kernel 
paging request at d928a000
Sep  4 16:43:37 mpibench_guest2 kernel: IP: [<d9460896>] 
:sm_communicator:m_init+0x1e1/0x210
Sep  4 16:43:37 mpibench_guest2 kernel: *pdpt = 0000000002794001 *pde = 
0000000002861067 *pte = 0000040020738863
Sep  4 16:43:37 mpibench_guest2 kernel: Oops: 0009 [#1] SMP
Sep  4 16:43:37 mpibench_guest2 kernel: last sysfs file: 
/sys/devices/xen/vif-0/net/eth0/type
Sep  4 16:43:37 mpibench_guest2 kernel: Modules linked in: 
sm_communicator(+) nfs lockd nfs_acl sunrpc iptable_filter ip_tables 
ip6table_filter ip6_tables x_tables ipv6 af_packet 8250 serial_core loop 
dm_mod joydev reiserfs xenblk xennet
Sep  4 16:43:37 mpibench_guest2 kernel:
Sep  4 16:43:38 mpibench_guest2 kernel: Pid: 2383, comm: insmod Tainted: 
G        N (2.6.25.5-1.1-xen #1)
Sep  4 16:43:38 mpibench_guest2 kernel: EIP: 0061:[<d9460896>] EFLAGS: 
00010246 CPU: 0
Sep  4 16:43:38 mpibench_guest2 kernel: EIP is at m_init+0x1e1/0x210 
[sm_communicator]
Sep  4 16:43:38 mpibench_guest2 kernel: EAX: 0000002c EBX: 00000000 ECX: 
f5636000 EDX: f5636000
Sep  4 16:43:38 mpibench_guest2 kernel: ESI: d928a000 EDI: d9463a20 EBP: 
d6d49e88 ESP: d6d49e64
Sep  4 16:43:38 mpibench_guest2 kernel:  DS: 007b ES: 007b FS: 00d8 GS: 
0033 SS: 0069
Sep  4 16:43:38 mpibench_guest2 kernel: Process insmod (pid: 2383, 
ti=d6d48000 task=d750c160 task.ti=d6d48000)
Sep  4 16:43:38 mpibench_guest2 kernel: Stack: d9461590 d928a000 
d7532900 00000001 00000025 d6d49e88 d7532900 00000001
Sep  4 16:43:38 mpibench_guest2 kernel:        00000025 d6d49fb0 
c013be4d 00000024 000003e8 00000064 d6e27c00 0000e793
Sep  4 16:43:38 mpibench_guest2 kernel:        0000001f d6d49f88 
d7957718 00000000 d9462800 00000000 00000000 00000000
Sep  4 16:43:38 mpibench_guest2 kernel: Call Trace:
Sep  4 16:43:38 mpibench_guest2 kernel:  [<c013be4d>] 
sys_init_module+0x1989/0x1ab6
Sep  4 16:43:38 mpibench_guest2 kernel:  [<c0103f5a>] syscall_call+0x7/0xb
Sep  4 16:43:38 mpibench_guest2 kernel:  [<f57fe416>] 0xf57fe416
Sep  4 16:43:38 mpibench_guest2 kernel:  ======================Sep  4 16:43:38
mpibench_guest2 kernel: Code: ff eb 4a 8b 04 9f ba 00 00
00 00 e8 7e fc ff ff 89 c6 a1 98 39 46 d9 85 c0 78 32 89 74 24 04 c7 04 
24 90 15 46 d9 e8 66 a2 e7 e6 <8b> 06 89 44 24 04 c7 04 24 b8 15 46 d9 
e8 54 a2 e7 e6 83 c3 01
Sep  4 16:43:38 mpibench_guest2 kernel: EIP: [<d9460896>] 
m_init+0x1e1/0x210 [sm_communicator] SS:ESP 0069:d6d49e64
Sep  4 16:43:38 mpibench_guest2 kernel: ---[ end trace fc6d563289120224 ]---

------------------------------------------



Any help on this would be appreciated. I''m working on this for a while 
now and being currently stucked at this point.


Regards,

Gregor Rebel
Candidate Diplom Informatiker


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Asim

2008-Sep-04 15:35 UTC

head link

Re: [Xen-devel] Question: How to map granted pages into domU properly?

I don''t know the correct process to map pages but here is a guess :-

Did you check the permissions of the page and if domU has permissions
to read them?  From mm.h  this is the permissions legend.

28000001: L2 page, pinned
58000001: GDT page, pinned
e0000000: writable page, not pinned

You want your permissions to be e0000000. If no then one of the ways
to change permissions is:
  pgprot_val(vma->vm_page_prot) |= (_PAGE_DIRTY | _PAGE_VALID);

 You can also see mm.c in the xen source directory (xen-.3.3*/../../)
for better understanding of this process.

Regards,
Asim



On 9/4/08, abc <abc@mailbarrier.de> wrote:> Hi,
>
> I''m writing a diploma-thesis based on paravirtualized XEN v3.2.1
and
> openSuSE 11.0 (32bit PAE).
> The idea of my thesis is to use XEN as a research platform for on-die
> message passing interface hardware. Therefore several domUs should
> communicate via a configurable kernel-module over shared-memory.
>
> Currently I encountered a problem in granting kernel-pages from dom0 to
> domUs.
> The pages can be mapped successfully. But when the kernel-module in
> domU, tries to read the content of this page, a kernel oops occurs.
>
> Can you explain me how to grant pages and how to map them into domUs
> properly?
>
>
>
>
> These are my sources (shortened):
>
>
> Dom0 -----------------------------------------------
>
> static inline unsigned long vaddr2pfn(void *vaddr) {
>
>   struct page* Page = vmalloc_to_page(vaddr);
>   unsigned long PFN = page_to_pfn(Page);
>
>   return PFN;
> }
>
>   unsigned int ForeignDomainID = 12; // <-- Hard coded
>   unsigned long Page = get_zeroed_page(0);
>   unsigned long PFN = vaddr2pfn( (void *) Page);
>   unsigned long MFN = virt_to_phys( (void *) PFN);
>
>   grant_ref = gnttab_grant_foreign_access( ForeignDomainID, MFN, 0 );
>
> ----------------------------------------------------
>
> grant_ref then is magically being transported to domU..
>
> DomU (domID=12) ------------------------------------
>
>   struct vm_struct *Area = alloc_vm_area(PAGE_SIZE);
>   struct gnttab_map_grant_ref operation;
>   gnttab_set_map_op(&operation, (unsigned long) Area->addr,
> GNTMAP_host_map | GNTMAP_application_map, grant_ref, 0);
>   HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &operation, 1);
>   // print content of operation to syslog..
>
>   Unsigned long* Dword = (Unsigned long*) Area->addr;
>   unsigned long Content = *Dword  // <-- BUG: unable to handle kernel
> paging request at d928a000
>
> ----------------------------------------------------
>
> Dom0 - /var/log/messages ---------------------------
>
> Sep  4 16:43:19 mpibench1 kernel: sm_communicator: instanciating in dom0
> Sep  4 16:43:19 mpibench1 kernel: sm_communicator: granting memory to
> guest #12
> Sep  4 16:43:19 mpibench1 kernel: sm_communicator: granted page
> @0xe0738000 (mfn=0x40020738) to domain #12
> Sep  4 16:43:19 mpibench1 kernel: sm_communicator: grant_refs[1]==21 (0x15)
>
> ----------------------------------------------------
>
> DomU - /var/log/messages ---------------------------
>
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: instanciating
> in domU
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: mapping page
> #21 from domain #0
> Sep  4 16:43:37 mpibench_guest2 kernel: klogd 1.4.1, ---------- state
> change ----------
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: new
> area={next=0xd7671760, addr=0xd928a000, size=0x2000, flags=0x1,
> pages=0x0, nr_pages=0, phys_addr=0x0}
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:
> HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &Operation, 1);
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: struct
> gnttab_map_grant_ref Operation = {:
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   /* IN
> parameters. */
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint64_t
> host_addr=0x00000000d928a000;
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint32_t
> flags=2=GNTMAP_host_map;
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   grant_ref_t
> ref=21;
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   domid_t  dom=0;
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   /* OUT
> parameters. */
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   int16_t
> status=0;
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:
> grant_handle_t handle=0;
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:   uint64_t
> dev_bus_addr=0x0000040020738000;
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator:}
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: successfully
> mapped page @0xd928a000 (handle=0, phys=0x0000040020738000)
> Sep  4 16:43:37 mpibench_guest2 kernel: sm_communicator: GrantedPage
> @0xd928a000
> Sep  4 16:43:37 mpibench_guest2 kernel: BUG: unable to handle kernel
> paging request at d928a000
> Sep  4 16:43:37 mpibench_guest2 kernel: IP: [<d9460896>]
> :sm_communicator:m_init+0x1e1/0x210
> Sep  4 16:43:37 mpibench_guest2 kernel: *pdpt = 0000000002794001 *pde >
0000000002861067 *pte = 0000040020738863
> Sep  4 16:43:37 mpibench_guest2 kernel: Oops: 0009 [#1] SMP
> Sep  4 16:43:37 mpibench_guest2 kernel: last sysfs file:
> /sys/devices/xen/vif-0/net/eth0/type
> Sep  4 16:43:37 mpibench_guest2 kernel: Modules linked in:
> sm_communicator(+) nfs lockd nfs_acl sunrpc iptable_filter ip_tables
> ip6table_filter ip6_tables x_tables ipv6 af_packet 8250 serial_core loop
> dm_mod joydev reiserfs xenblk xennet
> Sep  4 16:43:37 mpibench_guest2 kernel:
> Sep  4 16:43:38 mpibench_guest2 kernel: Pid: 2383, comm: insmod Tainted:
> G        N (2.6.25.5-1.1-xen #1)
> Sep  4 16:43:38 mpibench_guest2 kernel: EIP: 0061:[<d9460896>]
EFLAGS:
> 00010246 CPU: 0
> Sep  4 16:43:38 mpibench_guest2 kernel: EIP is at m_init+0x1e1/0x210
> [sm_communicator]
> Sep  4 16:43:38 mpibench_guest2 kernel: EAX: 0000002c EBX: 00000000 ECX:
> f5636000 EDX: f5636000
> Sep  4 16:43:38 mpibench_guest2 kernel: ESI: d928a000 EDI: d9463a20 EBP:
> d6d49e88 ESP: d6d49e64
> Sep  4 16:43:38 mpibench_guest2 kernel:  DS: 007b ES: 007b FS: 00d8 GS:
> 0033 SS: 0069
> Sep  4 16:43:38 mpibench_guest2 kernel: Process insmod (pid: 2383,
> ti=d6d48000 task=d750c160 task.ti=d6d48000)
> Sep  4 16:43:38 mpibench_guest2 kernel: Stack: d9461590 d928a000
> d7532900 00000001 00000025 d6d49e88 d7532900 00000001
> Sep  4 16:43:38 mpibench_guest2 kernel:        00000025 d6d49fb0
> c013be4d 00000024 000003e8 00000064 d6e27c00 0000e793
> Sep  4 16:43:38 mpibench_guest2 kernel:        0000001f d6d49f88
> d7957718 00000000 d9462800 00000000 00000000 00000000
> Sep  4 16:43:38 mpibench_guest2 kernel: Call Trace:
> Sep  4 16:43:38 mpibench_guest2 kernel:  [<c013be4d>]
> sys_init_module+0x1989/0x1ab6
> Sep  4 16:43:38 mpibench_guest2 kernel:  [<c0103f5a>]
syscall_call+0x7/0xb
> Sep  4 16:43:38 mpibench_guest2 kernel:  [<f57fe416>] 0xf57fe416
> Sep  4 16:43:38 mpibench_guest2 kernel:  ======================> Sep  4
16:43:38 mpibench_guest2 kernel: Code: ff eb 4a 8b 04 9f ba 00 00
> 00 00 e8 7e fc ff ff 89 c6 a1 98 39 46 d9 85 c0 78 32 89 74 24 04 c7 04
> 24 90 15 46 d9 e8 66 a2 e7 e6 <8b> 06 89 44 24 04 c7 04 24 b8 15 46
d9
> e8 54 a2 e7 e6 83 c3 01
> Sep  4 16:43:38 mpibench_guest2 kernel: EIP: [<d9460896>]
> m_init+0x1e1/0x210 [sm_communicator] SS:ESP 0069:d6d49e64
> Sep  4 16:43:38 mpibench_guest2 kernel: ---[ end trace fc6d563289120224
]---
>
> ------------------------------------------
>
>
>
> Any help on this would be appreciated. I''m working on this for a
while
> now and being currently stucked at this point.
>
>
> Regards,
>
> Gregor Rebel
> Candidate Diplom Informatiker
>
>
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Xen devel - Sep 2008 - Question: How to map granted pages into domU properly?

[Xen-devel] Question: How to map granted pages into domU properly?

Re: [Xen-devel] Question: How to map granted pages into domU properly?