Sorry for my poor x86 memory model knowlege... In some code I am working on, I currently pass the address of a guest page from a domain via a hypercall and in Xen either copy to or from the guest page using: __copy_from_guest(xen_va, guest_va, PAGE_SIZE) or __copy_to_guest(guest_va, xen_va, PAGE_SIZE) where guest_va is declared as a XEN_GUEST_HANDLE(void) Now I''d like to switch the hypercall interface so that the domain passes a guest physical address (or guest physical frame number) instead of a guest virtual address. I see code that does get_gpfn_from_mfn() but not the opposite. What do I need to do to copy a page of data from a guest pfn to a Xen mfn (or vice versa)? And is a guest physical address still referenced as a XEN_GUEST_HANDLE? It''s probably very simple but I can''t find it... Thanks, Dan P.S. This has to work on 32- or 64-bit guests to both 32- and 64-bit hypervisors. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
You can try gfn_to_mfn... Thanks, Kevin>From: Dan Magenheimer >Sent: 2008年9月28日 5:33 > >Sorry for my poor x86 memory model knowlege... > >In some code I am working on, I currently pass the >address of a guest page from a domain via a hypercall >and in Xen either copy to or from the guest page >using: > >__copy_from_guest(xen_va, guest_va, PAGE_SIZE) > >or > >__copy_to_guest(guest_va, xen_va, PAGE_SIZE) > >where guest_va is declared as a XEN_GUEST_HANDLE(void) > >Now I'd like to switch the hypercall interface so >that the domain passes a guest physical address >(or guest physical frame number) instead of a guest >virtual address. > >I see code that does get_gpfn_from_mfn() but not >the opposite. What do I need to do to copy a >page of data from a guest pfn to a Xen mfn (or >vice versa)? And is a guest physical address >still referenced as a XEN_GUEST_HANDLE? > >It's probably very simple but I can't find it... > >Thanks, >Dan > >P.S. This has to work on 32- or 64-bit guests to >both 32- and 64-bit hypervisors. > >_______________________________________________ >Xen-devel mailing list >Xen-devel@lists.xensource.com >http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
> You can try gfn_to_mfn.Thanks! Then if I have a page_info pointer and a gpfn, will this code work for copying? Or can it be made simpler? Thanks, Dan #include <asm/p2m.h> static int copy_page_from_guest(struct page_info *dst_pi, XEN_GUEST_HANDLE(void) gpfn) { int i; unsigned long dst_mfn, guest_mfn; void *dst_va, *guest_va; p2m_type_t t; if (is_pv_32on64_vcpu(current)) gpfn.p = (void *)((unsigned long)gpfn.p & 0xfffffUL); dst_mfn = page_to_mfn(dst_pi); guest_mfn = gfn_to_mfn(current->domain,(unsigned long)gpfn.p,&t); if (t != p2m_ram_rw && t != p2m_ram_ro) return(0); /* do I need to worry about p2m_ram_logdirty? */ dst_va = map_domain_page_global(dst_mfn); guest_va = map_domain_page_global(guest_mfn); /* i = __copy_from_guest(dst_va, guest_va, PAGE_SIZE); */ i = __copy_from_user(dst_va, guest_va, PAGE_SIZE); unmap_domain_page_global(guest_va); unmap_domain_page_global(dst_va); return (!i); }> -----Original Message----- > From: > Sent: Saturday, September 27, 2008 8:38 PM > To: Dan Magenheimer; Xen-Devel (E-mail) > Subject: RE: [Xen-devel] guest physical to xen machine copy? > > > You can try gfn_to_mfn... > > Thanks, > Kevin > > >From: Dan Magenheimer > >Sent: 2008年9月28日 5:33 > > > >Sorry for my poor x86 memory model knowlege... > > > >In some code I am working on, I currently pass the > >address of a guest page from a domain via a hypercall > >and in Xen either copy to or from the guest page > >using: > > > >__copy_from_guest(xen_va, guest_va, PAGE_SIZE) > > > >or > > > >__copy_to_guest(guest_va, xen_va, PAGE_SIZE) > > > >where guest_va is declared as a XEN_GUEST_HANDLE(void) > > > >Now I''d like to switch the hypercall interface so > >that the domain passes a guest physical address > >(or guest physical frame number) instead of a guest > >virtual address. > > > >I see code that does get_gpfn_from_mfn() but not > >the opposite. What do I need to do to copy a > >page of data from a guest pfn to a Xen mfn (or > >vice versa)? And is a guest physical address > >still referenced as a XEN_GUEST_HANDLE? > > > >It''s probably very simple but I can''t find it... > > > >Thanks, > >Dan > > > >P.S. This has to work on 32- or 64-bit guests to > >both 32- and 64-bit hypervisors. > > > >_______________________________________________ > >Xen-devel mailing list > >Xen-devel@lists.xensource.com > >http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 28/9/08 21:19, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:>> You can try gfn_to_mfn. > > Thanks! Then if I have a page_info pointer and a gpfn, will > this code work for copying? Or can it be made simpler?Use p2m_is_ram() instead of checking individual p2m ram type enumerations. Use [un]map_domain_page instead of [un]map_domain_page_global. -- Keir> Thanks, > Dan > > #include <asm/p2m.h> > static int > copy_page_from_guest(struct page_info *dst_pi, XEN_GUEST_HANDLE(void) gpfn) > { > int i; > unsigned long dst_mfn, guest_mfn; > void *dst_va, *guest_va; > p2m_type_t t; > > if (is_pv_32on64_vcpu(current)) > gpfn.p = (void *)((unsigned long)gpfn.p & 0xfffffUL); > dst_mfn = page_to_mfn(dst_pi); > guest_mfn = gfn_to_mfn(current->domain,(unsigned long)gpfn.p,&t); > if (t != p2m_ram_rw && t != p2m_ram_ro) > return(0); /* do I need to worry about p2m_ram_logdirty? */ > dst_va = map_domain_page_global(dst_mfn); > guest_va = map_domain_page_global(guest_mfn); > /* i = __copy_from_guest(dst_va, guest_va, PAGE_SIZE); */ > i = __copy_from_user(dst_va, guest_va, PAGE_SIZE); > unmap_domain_page_global(guest_va); > unmap_domain_page_global(dst_va); > return (!i); > } > > >> -----Original Message----- >> From: >> Sent: Saturday, September 27, 2008 8:38 PM >> To: Dan Magenheimer; Xen-Devel (E-mail) >> Subject: RE: [Xen-devel] guest physical to xen machine copy? >> >> >> You can try gfn_to_mfn... >> >> Thanks, >> Kevin >> >>> From: Dan Magenheimer >>> Sent: 2008年9月28日 5:33 >>> >>> Sorry for my poor x86 memory model knowlege... >>> >>> In some code I am working on, I currently pass the >>> address of a guest page from a domain via a hypercall >>> and in Xen either copy to or from the guest page >>> using: >>> >>> __copy_from_guest(xen_va, guest_va, PAGE_SIZE) >>> >>> or >>> >>> __copy_to_guest(guest_va, xen_va, PAGE_SIZE) >>> >>> where guest_va is declared as a XEN_GUEST_HANDLE(void) >>> >>> Now I''d like to switch the hypercall interface so >>> that the domain passes a guest physical address >>> (or guest physical frame number) instead of a guest >>> virtual address. >>> >>> I see code that does get_gpfn_from_mfn() but not >>> the opposite. What do I need to do to copy a >>> page of data from a guest pfn to a Xen mfn (or >>> vice versa)? And is a guest physical address >>> still referenced as a XEN_GUEST_HANDLE? >>> >>> It''s probably very simple but I can''t find it... >>> >>> Thanks, >>> Dan >>> >>> P.S. This has to work on 32- or 64-bit guests to >>> both 32- and 64-bit hypervisors. >>> >>> _______________________________________________ >>> Xen-devel mailing list >>> Xen-devel@lists.xensource.com >>> http://lists.xensource.com/xen-devel >>> >> > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hmmm... gfn_to_mfn() when used as below seems to just return the original gfn, e.g. guest_mfn = gfn_to_mfn(current->domain,gpfn,&t); Is this correct? Note that current->domain is a 32-bit paravirtualized domain and xen is 64-bit, if that matters. (Is there a better way if one knows that the domain is always pv?)> Use p2m_is_ram() instead of checking individual p2m ram typeIn copying the other direction (e.g. copy_page_TO_guest), I think I need to check for t==p2m_ram_rw, right? If so, the "Not necessarily true" comment in static inline _gfn_to_mfn in include/asm-x86/p2m.h worries me. Should it? Dan> -----Original Message----- > From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] > Sent: Monday, September 29, 2008 12:49 AM > To: Dan Magenheimer; Xen-Devel (E-mail); Tian, Kevin > Subject: Re: [Xen-devel] guest physical to xen machine copy? > > > On 28/9/08 21:19, "Dan Magenheimer" > <dan.magenheimer@oracle.com> wrote: > > >> You can try gfn_to_mfn. > > > > Thanks! Then if I have a page_info pointer and a gpfn, will > > this code work for copying? Or can it be made simpler? > > Use p2m_is_ram() instead of checking individual p2m ram type > enumerations. > Use [un]map_domain_page instead of [un]map_domain_page_global. > > -- Keir > > > Thanks, > > Dan > > > > #include <asm/p2m.h> > > static int > > copy_page_from_guest(struct page_info *dst_pi, > XEN_GUEST_HANDLE(void) gpfn) > > { > > int i; > > unsigned long dst_mfn, guest_mfn; > > void *dst_va, *guest_va; > > p2m_type_t t; > > > > if (is_pv_32on64_vcpu(current)) > > gpfn.p = (void *)((unsigned long)gpfn.p & 0xfffffUL); > > dst_mfn = page_to_mfn(dst_pi); > > guest_mfn = gfn_to_mfn(current->domain,(unsigned > long)gpfn.p,&t); > > if (t != p2m_ram_rw && t != p2m_ram_ro) > > return(0); /* do I need to worry about p2m_ram_logdirty? */ > > dst_va = map_domain_page_global(dst_mfn); > > guest_va = map_domain_page_global(guest_mfn); > > /* i = __copy_from_guest(dst_va, guest_va, PAGE_SIZE); */ > > i = __copy_from_user(dst_va, guest_va, PAGE_SIZE); > > unmap_domain_page_global(guest_va); > > unmap_domain_page_global(dst_va); > > return (!i); > > } > > > > > >> -----Original Message----- > >> From: > >> Sent: Saturday, September 27, 2008 8:38 PM > >> To: Dan Magenheimer; Xen-Devel (E-mail) > >> Subject: RE: [Xen-devel] guest physical to xen machine copy? > >> > >> > >> You can try gfn_to_mfn... > >> > >> Thanks, > >> Kevin > >> > >>> From: Dan Magenheimer > >>> Sent: 2008年9月28日 5:33 > >>> > >>> Sorry for my poor x86 memory model knowlege... > >>> > >>> In some code I am working on, I currently pass the > >>> address of a guest page from a domain via a hypercall > >>> and in Xen either copy to or from the guest page > >>> using: > >>> > >>> __copy_from_guest(xen_va, guest_va, PAGE_SIZE) > >>> > >>> or > >>> > >>> __copy_to_guest(guest_va, xen_va, PAGE_SIZE) > >>> > >>> where guest_va is declared as a XEN_GUEST_HANDLE(void) > >>> > >>> Now I''d like to switch the hypercall interface so > >>> that the domain passes a guest physical address > >>> (or guest physical frame number) instead of a guest > >>> virtual address. > >>> > >>> I see code that does get_gpfn_from_mfn() but not > >>> the opposite. What do I need to do to copy a > >>> page of data from a guest pfn to a Xen mfn (or > >>> vice versa)? And is a guest physical address > >>> still referenced as a XEN_GUEST_HANDLE? > >>> > >>> It''s probably very simple but I can''t find it... > >>> > >>> Thanks, > >>> Dan > >>> > >>> P.S. This has to work on 32- or 64-bit guests to > >>> both 32- and 64-bit hypervisors. > >>> > >>> _______________________________________________ > >>> Xen-devel mailing list > >>> Xen-devel@lists.xensource.com > >>> http://lists.xensource.com/xen-devel > >>> > >> > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29/9/08 19:50, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:> Hmmm... gfn_to_mfn() when used as below seems to just > return the original gfn, e.g. > > guest_mfn = gfn_to_mfn(current->domain,gpfn,&t); > > Is this correct? Note that current->domain is a 32-bit > paravirtualized domain and xen is 64-bit, if that matters. > (Is there a better way if one knows that the domain is > always pv?)Xen doesn''t do pfn-to-mfn translation for PV guests. They do it themselves with their own p2m table. So, for a PV guest, gfn==mfn.>> Use p2m_is_ram() instead of checking individual p2m ram type > > In copying the other direction (e.g. copy_page_TO_guest), > I think I need to check for t==p2m_ram_rw, right?You need to check for not p2m_ram_ro. You also should page_mark_dirty() (__hvm_copy() does it unconditionally on pages it modifies). -- Keir> If so, the "Not necessarily true" comment in static > inline _gfn_to_mfn in include/asm-x86/p2m.h worries me. > Should it?_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
OK, thanks! Sorry to belabour this further, but getting this wrong has resulted in dom0 data corruption so I''m trying to get it right this time :-)> Xen doesn''t do pfn-to-mfn translation for PV guests...So there''s no way to do this from inside Xen then? Then I gather I need to be passing an mfn to Xen instead of a gpfn. To do so, it looks like I call pfn_to_mfn() in the paravirtualized guest. But how can this work on a 32-bit-PV-on-64-bit-Xen since pfn_to_mfn() returns an unsigned long? Also, once I''ve got an (pvguest-translated-gpfn-to-) mfn inside of Xen, do I still do a gfn_to_mfn() on it? (Shouldn''t matter because it just returns the gfn, right?) I''ve now tried various combinations of these with known data in the page being passed by the pv guest, but the wrong data is found on the xen side, so I''m clearly still missing something. BTW, all this works when passing the page as a guest virtual address, but the virtual addresss of the page isn''t always accessible on the guest side. Guest side: Xen side: if (is_pv_32on64_vcpu(current)) gpfn.p = (void *)((unsigned long)gpfn.p & 0xfffffUL); dst_mfn = page_to_mfn(xen_page); guest_mfn = gfn_to_mfn(current->domain,(unsigned long)gpfn.p,&t); if (!p2m_is_ram(t)) { bad_addr_copies++; printk("bad copy_from_guest: gpfn=%lu mfn=%lu\n",(unsigned long)gpfn.p,guest_mfn); return 0; } dst_va = map_domain_page(dst_mfn); guest_va = map_domain_page(guest_mfn); i = __copy_from_user(dst_va, guest_va, PAGE_SIZE); if (i) printk("copy_from_guest failed: gpfn=%lu mfn=%lu va=%p\n",(unsigned long)gpfn.p,guest_mfn,guest_va); if (*(uint32_t *)guest_va != 0x12345678) printk("copy_from_guest bad data: gpfn=%lu mfn=%lu va=%p\n",(unsigned long)gpfn.p,guest_mfn,guest_va); unmap_domain_page(guest_va); unmap_domain_page(dst_va);> -----Original Message----- > From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] > Sent: Monday, September 29, 2008 2:12 PM > To: Dan Magenheimer; Xen-Devel (E-mail); Tian, Kevin > Subject: Re: [Xen-devel] guest physical to xen machine copy? > > > On 29/9/08 19:50, "Dan Magenheimer" > <dan.magenheimer@oracle.com> wrote: > > > Hmmm... gfn_to_mfn() when used as below seems to just > > return the original gfn, e.g. > > > > guest_mfn = gfn_to_mfn(current->domain,gpfn,&t); > > > > Is this correct? Note that current->domain is a 32-bit > > paravirtualized domain and xen is 64-bit, if that matters. > > (Is there a better way if one knows that the domain is > > always pv?) > > Xen doesn''t do pfn-to-mfn translation for PV guests. They do > it themselves > with their own p2m table. So, for a PV guest, gfn==mfn. > > >> Use p2m_is_ram() instead of checking individual p2m ram type > > > > In copying the other direction (e.g. copy_page_TO_guest), > > I think I need to check for t==p2m_ram_rw, right? > > You need to check for not p2m_ram_ro. You also should > page_mark_dirty() > (__hvm_copy() does it unconditionally on pages it modifies). > > -- Keir > > > If so, the "Not necessarily true" comment in static > > inline _gfn_to_mfn in include/asm-x86/p2m.h worries me. > > Should it? > > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Oops, sorry, fat fingers... second try with guest code: =========OK, thanks! Sorry to belabour this further, but getting this wrong has resulted in dom0 data corruption so I''m trying to get it right this time :-)> Xen doesn''t do pfn-to-mfn translation for PV guests...So there''s no way to do this from inside Xen then? Then I gather I need to be passing an mfn to Xen instead of a gpfn. To do so, it looks like I call pfn_to_mfn() in the paravirtualized guest. But how can this work on a 32-bit-PV-on-64-bit-Xen since pfn_to_mfn() returns an unsigned long? Also, once I''ve got an (pvguest-translated-gpfn-to-) mfn inside of Xen, do I still do a gfn_to_mfn() on it? (Shouldn''t matter because it just returns the gfn, right?) I''ve now tried various combinations of these with known data in the page being passed by the pv guest, but the wrong data is found on the xen side, so I''m clearly still missing something. BTW, all this works when passing the page as a guest virtual address, but the virtual addresss of the page isn''t always accessible on the guest side. Guest side: unsigned long mfn = pfn_to_mfn(page_to_pfn(page)); void *va = PageHighMem(page) ? page_address(page) : lowmem_page_address(page); if (!va) return 0; else *(uint32_t *)va == 0x12345678; return my_hypercall(hcache_id, obj, ind, mfn); Xen side: if (is_pv_32on64_vcpu(current)) gpfn.p = (void *)((unsigned long)gpfn.p & 0xfffffUL); dst_mfn = page_to_mfn(xen_page); guest_mfn = gfn_to_mfn(current->domain, (unsigned long)gpfn.p,&t); if (!p2m_is_ram(t)) { bad_addr_copies++; printk("bad copy_from_guest: " "gpfn=%lu mfn=%lu\n", (unsigned long)gpfn.p,guest_mfn); return 0; } dst_va = map_domain_page(dst_mfn); guest_va = map_domain_page(guest_mfn); i = __copy_from_user(dst_va, guest_va, PAGE_SIZE); if (i) printk("copy_from_guest failed:" " gpfn=%lu mfn=%lu va=%p\n", (unsigned long)gpfn.p,guest_mfn,guest_va); if (*(uint32_t *)guest_va != 0x12345678) printk("copy_from_guest bad data:" " gpfn=%lu mfn=%lu va=%p\n" (unsigned long)gpfn.p,guest_mfn,guest_va); unmap_domain_page(guest_va); unmap_domain_page(dst_va);> > -----Original Message----- > > From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] > > Sent: Monday, September 29, 2008 2:12 PM > > To: Dan Magenheimer; Xen-Devel (E-mail); Tian, Kevin > > Subject: Re: [Xen-devel] guest physical to xen machine copy? > > > > > > On 29/9/08 19:50, "Dan Magenheimer" > > <dan.magenheimer@oracle.com> wrote: > > > > > Hmmm... gfn_to_mfn() when used as below seems to just > > > return the original gfn, e.g. > > > > > > guest_mfn = gfn_to_mfn(current->domain,gpfn,&t); > > > > > > Is this correct? Note that current->domain is a 32-bit > > > paravirtualized domain and xen is 64-bit, if that matters. > > > (Is there a better way if one knows that the domain is > > > always pv?) > > > > Xen doesn''t do pfn-to-mfn translation for PV guests. They do > > it themselves > > with their own p2m table. So, for a PV guest, gfn==mfn. > > > > >> Use p2m_is_ram() instead of checking individual p2m ram type > > > > > > In copying the other direction (e.g. copy_page_TO_guest), > > > I think I need to check for t==p2m_ram_rw, right? > > > > You need to check for not p2m_ram_ro. You also should > > page_mark_dirty() > > (__hvm_copy() does it unconditionally on pages it modifies). > > > > -- Keir > > > > > If so, the "Not necessarily true" comment in static > > > inline _gfn_to_mfn in include/asm-x86/p2m.h worries me. > > > Should it? > > > > > > > > > > _______________________________________________ > > Xen-devel mailing list > > Xen-devel@lists.xensource.com > > http://lists.xensource.com/xen-devel > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 29/9/08 23:30, "Dan Magenheimer" <dan.magenheimer@oracle.com> wrote:> Then I gather I need to be passing an mfn to Xen instead > of a gpfn. To do so, it looks like I call pfn_to_mfn() > in the paravirtualized guest. But how can this work > on a 32-bit-PV-on-64-bit-Xen since pfn_to_mfn() returns > an unsigned long?So what?> Also, once I''ve got an (pvguest-translated-gpfn-to-) mfn > inside of Xen, do I still do a gfn_to_mfn() on it? > (Shouldn''t matter because it just returns the gfn, right?)Yes, gfn here means what-the-guest-thinks-is-an-mfn. That''s pfn for an HVM guest; mfn for a PV guest. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Thanks for your help. I finally got all of this working. Interestingly, I think my problem was aliasing: I was writing from one virtual address in the guest and reading from another virtual address in Xen, with both mapped to the same machine address. Adding mb() appropriately seemed to resolve the problem. Thanks, Dan> -----Original Message----- > From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] > Sent: Tuesday, September 30, 2008 12:25 AM > To: Dan Magenheimer; Xen-Devel (E-mail); Tian, Kevin > Subject: Re: [Xen-devel] guest physical to xen machine copy? > > > On 29/9/08 23:30, "Dan Magenheimer" > <dan.magenheimer@oracle.com> wrote: > > > Then I gather I need to be passing an mfn to Xen instead > > of a gpfn. To do so, it looks like I call pfn_to_mfn() > > in the paravirtualized guest. But how can this work > > on a 32-bit-PV-on-64-bit-Xen since pfn_to_mfn() returns > > an unsigned long? > > So what? > > > Also, once I''ve got an (pvguest-translated-gpfn-to-) mfn > > inside of Xen, do I still do a gfn_to_mfn() on it? > > (Shouldn''t matter because it just returns the gfn, right?) > > Yes, gfn here means what-the-guest-thinks-is-an-mfn. That''s > pfn for an HVM > guest; mfn for a PV guest. > > -- Keir > > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel