Kieran Mansley
2006-Nov-02 16:01 UTC
[Xen-devel] Getting a bus address for granted page in dom0
I''m having a couple of problems dealing with pages in dom0 that have been mapped via a grant from domU, and wondered if anyone can confirm my suspicions and suggest a way round this. I pass a grant from domU to dom0, and map it in to dom0 using xenbus_map_ring_valloc(). This gives me a "struct vm_struct" which isn''t terribly useful as most of it isn''t filled out, but it does have a virtual address for the page in dom0. I need to pass this page to a PCI device, and so would like to use something like pci_map_single() (or dma_map_page()) to convert this virtual address into a bus address. However, neither of these seem to be working as one might hope: they always return the same value regardless of the address you pass in (at least for the valid addresses that I''ve tried). Subsequent investigations using, for example, follow_page() yield the same result. Interestingly, to get from the virtual address in dom0 to a "struct vm_area_struct" that can be used for follow_page() I used find_vma(). It however claimed that it couldn''t find a vma for that address. The call to alloc_vm_area() in xenbus_map_ring_valloc() does a walk of the page tables (using apply_to_page_range()) to, I assume, set things up appropriately for this new page, and does seem to end up with a valid "struct page" in its hands. However, I now suspect that it and the hyperop that maps in the grant are doing something clever, such as only updating the real page tables in the hypervisor, and doing just enough to the page tables in dom0 to ensure that things don''t go horribly wrong. Is this correct? If so, when I try and walk the page tables (using something like follow_page()) in dom0 to get from the virtual address to a struct page, I find the hacked up entry rather than something that is useful, and this would explain the above behaviour of find_vma(). If this is right, any suggestions for how to get from a virtual address of a page that has been mapped using a grant from domU->dom0, to an address that is suitable for passing to a PCI device? Thanks Kieran _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kieran Mansley
2006-Nov-06 14:55 UTC
Re: [Xen-devel] Getting a bus address for granted page in dom0
On Thu, 2006-11-02 at 16:01 +0000, Kieran Mansley wrote:> I''m having a couple of problems dealing with pages in dom0 that have > been mapped via a grant from domU, and wondered if anyone can confirm my > suspicions and suggest a way round this. > > I pass a grant from domU to dom0, and map it in to dom0 using > xenbus_map_ring_valloc(). This gives me a "struct vm_struct" which > isn''t terribly useful as most of it isn''t filled out, but it does have a > virtual address for the page in dom0. I need to pass this page to a PCI > device, and so would like to use something like pci_map_single() (or > dma_map_page()) to convert this virtual address into a bus address. > However, neither of these seem to be working as one might hope: they > always return the same value regardless of the address you pass in (at > least for the valid addresses that I''ve tried).As a follow-up to my question (to which I would still welcome any ideas!), I''ve experimented with using the GNTMAP_contains_pte flag to the HYPERVISOR_grant_table_op by modifying xenbus_map_ring_valloc() as follows. After the call to alloc_vm_area, replace the newly allocated area->addr with a call to kzalloc() to get a region of virtual address space that actually has something backing it. I call create_lookup_pte_addr() at this point to ensure the page tables are set up properly and get the page table entry. Then, call the HYPERVISOR_grant_table_op as before, but add the GNTMAP_contains_pte flag and give it the address of the pte: struct vm_struct *xenbus_map_ring_valloc(struct xenbus_device *dev, int gnt_ref) { struct gnttab_map_grant_ref op; struct vm_struct *area; int ret; uint64_t ptep; area = alloc_vm_area(PAGE_SIZE); if (!area) return ERR_PTR(-ENOMEM); /* Added code starts here */ area->addr = kzalloc(PAGE_SIZE, GFP_KERNEL); ret = create_lookup_pte_addr(&init_mm, area->addr, &ptep); if(ret){ printk("%s: Couldn''t get a pte addr!\n", __FUNCTION__); return ERR_PTR(-ENOMEM); } /* Added code ends here */ /* Changed to modify pte rather than providing virtual address */ gnttab_set_map_op(&op, ptep, GNTMAP_host_map | GNTMAP_contains_pte, gnt_ref, dev->otherend_id); lock_vm_area(area); BUG_ON(HYPERVISOR_grant_table_op(GNTTABOP_map_grant_ref, &op, 1)); unlock_vm_area(area); The rest of the function is unchanged. This changes are clearly a dirty hack, but at the moment I''m interested in finding out what is wrong rather than getting something pretty! This change seems to result in the grant getting mapped in and me being able to get a different "struct page" and bus address for each granted page, rather than the same one each time. This seems to confirm my theory that the normal call to map the grant is deficient in the way it deals with the page tables in dom0. Before I file a bug for this, would anyone like to comment, or point out what I''m doing wrong? Thanks Kieran _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2006-Nov-07 11:26 UTC
Re: [Xen-devel] Getting a bus address for granted page in dom0
On 6/11/06 14:55, "Kieran Mansley" <kmansley@solarflare.com> wrote:>> I pass a grant from domU to dom0, and map it in to dom0 using >> xenbus_map_ring_valloc(). This gives me a "struct vm_struct" which >> isn''t terribly useful as most of it isn''t filled out, but it does have a >> virtual address for the page in dom0. I need to pass this page to a PCI >> device, and so would like to use something like pci_map_single() (or >> dma_map_page()) to convert this virtual address into a bus address. >> However, neither of these seem to be working as one might hope: they >> always return the same value regardless of the address you pass in (at >> least for the valid addresses that I''ve tried).Map_ring_valloc() maps the page into the vmalloc area of the kernel address space. Thus it has no ''struct page'' backing it and none of the DMA macros will work. Do you need to map the rings into dom0 addr space at all? If not just do a GNTMAP_device_map yourself, and this will return a dev_bus_addr that you can poke into hardware. If you go this route you avoid using the Linux DMA interfaces altogether. If you need to be able to use those macros then you need to map the grant into Linux lowmem. You''ll then have a ''struct page'' associated with the virtual address you mapped at, and so long as you update the physical_to_machine_mapping table you should find the DMA API starts to work for you. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kieran Mansley
2006-Nov-07 15:34 UTC
Re: [Xen-devel] Getting a bus address for granted page in dom0
On Tue, 2006-11-07 at 11:26 +0000, Keir Fraser wrote:> Do you need to map the rings into dom0 addr space at all? If not just do a > GNTMAP_device_map yourself, and this will return a dev_bus_addr that you can > poke into hardware. If you go this route you avoid using the Linux DMA > interfaces altogether.Thanks Keir, that was exactly what I was looking for. I''ve implemented this as an extra little helper function in xenbus_backend_client.c as an alternative to xenbus_map_ring() and xenbus_map_ring_valloc(). Is this something of wider use/interest? I''d be happy to submit a patch if so. Kieran _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
hi I try to read the granttable code of xen,I am confused about some fields in the struct grant_table,such as struct grant_entry **shared; struct active_grant_entry **active; struct grant_mapping **maptrack; what are the functions for the three structures ? I am confused about it could you help me Thanks in advance Kieran Mansley 写道:> On Tue, 2006-11-07 at 11:26 +0000, Keir Fraser wrote: > >> Do you need to map the rings into dom0 addr space at all? If not just do a >> GNTMAP_device_map yourself, and this will return a dev_bus_addr that you can >> poke into hardware. If you go this route you avoid using the Linux DMA >> interfaces altogether. >> > > Thanks Keir, that was exactly what I was looking for. > > I''ve implemented this as an extra little helper function in > xenbus_backend_client.c as an alternative to xenbus_map_ring() and > xenbus_map_ring_valloc(). Is this something of wider use/interest? I''d > be happy to submit a patch if so. > > Kieran > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
hi I try to read the granttable code of xen,I am confused about some fields in the struct grant_table,such as struct grant_entry **shared; struct active_grant_entry **active; struct grant_mapping **maptrack; what are the functions for the three structures ? in the grant_table_create() function,each entry of three structures is allocated a page frame,is it right? if it is,when we try to use a grant page ,three pages are needed to be allocated for matadata,or how does it work? I am confused about it could you help me Thanks in advance _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel