Kay, Allen M
2010-Aug-30 23:57 UTC
[Xen-devel] error messages when doing VT-d device passthrough
I''m getting the following error messages when doing Vt-d device passthrough: ... (XEN) p2m.c:2626:d0 clear_mmio_p2m_entry: gfn_to_mfn failed! gfn=000f303b ... This message is printed out for each MMIO page used by the device. This seems to be cause by the code change from "if ( INVALID_MFN == mfn)" to "if (!mfn_valid(mfn) )" in p2m.c/clear_mmio_p2m_entry(). What is the reason for the change? Allen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kay, Allen M
2010-Aug-31 00:08 UTC
[Xen-devel] RE: error messages when doing VT-d device passthrough
Cc''ing Christoph Egger since this change was part of the "nested Virtualization: p2m infrastructure" in cs# 21940. From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Kay, Allen M Sent: Monday, August 30, 2010 4:57 PM To: xen-devel@lists.xensource.com Cc: tim.deegan@eu.citrix.com; Keir Fraser Subject: [Xen-devel] error messages when doing VT-d device passthrough I''m getting the following error messages when doing Vt-d device passthrough: ... (XEN) p2m.c:2626:d0 clear_mmio_p2m_entry: gfn_to_mfn failed! gfn=000f303b ... This message is printed out for each MMIO page used by the device. This seems to be cause by the code change from "if ( INVALID_MFN == mfn)" to "if (!mfn_valid(mfn) )" in p2m.c/clear_mmio_p2m_entry(). What is the reason for the change? Allen _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kay, Allen M
2010-Aug-31 01:58 UTC
[Xen-devel] [PATCH] re-enable HVM PCI passthrough functionality
Attached patch reverts back to INVALID_MFN. PCI passthrough functionality works with this patch. Signed-off-by: Allen Kay allen.m.kay@intel.com _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tim Deegan
2010-Aug-31 09:17 UTC
[Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough functionality
At 02:58 +0100 on 31 Aug (1283223494), Kay, Allen M wrote:> Attached patch reverts back to INVALID_MFN. PCI passthrough functionality works with this patch. >Apologies for missing this - I did ask for this enormous patch to contain only what it said on the tin (and I did read it to check!) but this one got through.> Signed-off-by: Allen Kay allen.m.kay@intel.com >This is fine, but please add a comment explaining the rather subtle use of INVALID_MFN here so the next tidy-minded person doesn''t "fix" it again. :) In fact would it be better to check the returned type == p2m_mmio_direct instead? That seems like it would fit better with the intention. Or do other callers use this to clean non-mmio entries too? Cheers, Tim. -- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, XenServer Engineering Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Christoph Egger
2010-Aug-31 09:35 UTC
[Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough functionality
On Tuesday 31 August 2010 11:17:27 Tim Deegan wrote:> At 02:58 +0100 on 31 Aug (1283223494), Kay, Allen M wrote: > > Attached patch reverts back to INVALID_MFN. PCI passthrough > > functionality works with this patch. > > Apologies for missing this - I did ask for this enormous patch to > contain only what it said on the tin (and I did read it to check!) but > this one got through.Also apologies from my side. I did what Tim requested for, but that one slipped through.> > Signed-off-by: Allen Kay allen.m.kay@intel.com > > This is fine, but please add a comment explaining the rather subtle use > of INVALID_MFN here so the next tidy-minded person doesn''t "fix" it > again. :) > > In fact would it be better to check the returned type == p2m_mmio_direct > instead? That seems like it would fit better with the intention. Or > do other callers use this to clean non-mmio entries too?I''m interested in if this patch fixes the SR-IOV issue. We are still about investigation... Christoph -- ---to satisfy European Law for business letters: Advanced Micro Devices GmbH Einsteinring 24, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Kay, Allen M
2010-Aug-31 16:13 UTC
[Xen-devel] RE: [PATCH] re-enable HVM PCI passthrough functionality
I have updated the patch to indicate valid_mfn() cannot be used here as MMIO pages are always above max_mem. Also added sanity checking for type == p2m_mmio_direct before clearing page table entry. This should fix both VT-d and SR-IOV issues reported recently as there are no SR-IOV specific code in Xen. As far as Xen is concern, SR-IOV virtual function is just another PCI device. Signed-off-by: Allen Kay allen.m.kay@intel.com -----Original Message----- From: Christoph Egger [mailto:Christoph.Egger@amd.com] Sent: Tuesday, August 31, 2010 2:35 AM To: Tim Deegan Cc: Kay, Allen M; xen-devel@lists.xensource.com; Keir Fraser Subject: Re: [PATCH] re-enable HVM PCI passthrough functionality On Tuesday 31 August 2010 11:17:27 Tim Deegan wrote:> At 02:58 +0100 on 31 Aug (1283223494), Kay, Allen M wrote: > > Attached patch reverts back to INVALID_MFN. PCI passthrough > > functionality works with this patch. > > Apologies for missing this - I did ask for this enormous patch to > contain only what it said on the tin (and I did read it to check!) but > this one got through.Also apologies from my side. I did what Tim requested for, but that one slipped through.> > Signed-off-by: Allen Kay allen.m.kay@intel.com > > This is fine, but please add a comment explaining the rather subtle use > of INVALID_MFN here so the next tidy-minded person doesn''t "fix" it > again. :) > > In fact would it be better to check the returned type == p2m_mmio_direct > instead? That seems like it would fit better with the intention. Or > do other callers use this to clean non-mmio entries too?I''m interested in if this patch fixes the SR-IOV issue. We are still about investigation... Christoph -- ---to satisfy European Law for business letters: Advanced Micro Devices GmbH Einsteinring 24, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tim Deegan
2010-Sep-01 08:56 UTC
[Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough functionality
At 17:13 +0100 on 31 Aug (1283274786), Kay, Allen M wrote:> I have updated the patch to indicate valid_mfn() cannot be used here > as MMIO pages are always above max_mem. Also added sanity checking > for type == p2m_mmio_direct before clearing page table entry. This > should fix both VT-d and SR-IOV issues reported recently as there are > no SR-IOV specific code in Xen. As far as Xen is concern, SR-IOV > virtual function is just another PCI device.Applied; thanks for the fix. Tim.> Signed-off-by: Allen Kay allen.m.kay@intel.com > > -----Original Message----- > From: Christoph Egger [mailto:Christoph.Egger@amd.com] > Sent: Tuesday, August 31, 2010 2:35 AM > To: Tim Deegan > Cc: Kay, Allen M; xen-devel@lists.xensource.com; Keir Fraser > Subject: Re: [PATCH] re-enable HVM PCI passthrough functionality > > On Tuesday 31 August 2010 11:17:27 Tim Deegan wrote: > > At 02:58 +0100 on 31 Aug (1283223494), Kay, Allen M wrote: > > > Attached patch reverts back to INVALID_MFN. PCI passthrough > > > functionality works with this patch. > > > > Apologies for missing this - I did ask for this enormous patch to > > contain only what it said on the tin (and I did read it to check!) but > > this one got through. > > Also apologies from my side. I did what Tim requested for, but that > one slipped through. > > > > Signed-off-by: Allen Kay allen.m.kay@intel.com > > > > This is fine, but please add a comment explaining the rather subtle use > > of INVALID_MFN here so the next tidy-minded person doesn''t "fix" it > > again. :) > > > > In fact would it be better to check the returned type == p2m_mmio_direct > > instead? That seems like it would fit better with the intention. Or > > do other callers use this to clean non-mmio entries too? > > I''m interested in if this patch fixes the SR-IOV issue. We are still about > investigation... > > Christoph > > > > -- > ---to satisfy European Law for business letters: > Advanced Micro Devices GmbH > Einsteinring 24, 85609 Dornach b. Muenchen > Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd > Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen > Registergericht Muenchen, HRB Nr. 43632 >-- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, XenServer Engineering Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Zhang, Yang Z
2010-Sep-02 02:26 UTC
RE: [Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough functionality
With allen''s patch, both vt-d and sr-iov can work now. Thanks allen. best regards yang> -----Original Message----- > From: xen-devel-bounces@lists.xensource.com > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tim Deegan > Sent: Wednesday, September 01, 2010 4:57 PM > To: Kay, Allen M > Cc: Christoph Egger; xen-devel@lists.xensource.com; Keir Fraser > Subject: [Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough functionality > > At 17:13 +0100 on 31 Aug (1283274786), Kay, Allen M wrote: > > I have updated the patch to indicate valid_mfn() cannot be used here > > as MMIO pages are always above max_mem. Also added sanity checking > > for type == p2m_mmio_direct before clearing page table entry. This > > should fix both VT-d and SR-IOV issues reported recently as there are > > no SR-IOV specific code in Xen. As far as Xen is concern, SR-IOV > > virtual function is just another PCI device. > > Applied; thanks for the fix. > > Tim. > > > Signed-off-by: Allen Kay allen.m.kay@intel.com > > > > -----Original Message----- > > From: Christoph Egger [mailto:Christoph.Egger@amd.com] > > Sent: Tuesday, August 31, 2010 2:35 AM > > To: Tim Deegan > > Cc: Kay, Allen M; xen-devel@lists.xensource.com; Keir Fraser > > Subject: Re: [PATCH] re-enable HVM PCI passthrough functionality > > > > On Tuesday 31 August 2010 11:17:27 Tim Deegan wrote: > > > At 02:58 +0100 on 31 Aug (1283223494), Kay, Allen M wrote: > > > > Attached patch reverts back to INVALID_MFN. PCI passthrough > > > > functionality works with this patch. > > > > > > Apologies for missing this - I did ask for this enormous patch to > > > contain only what it said on the tin (and I did read it to check!) but > > > this one got through. > > > > Also apologies from my side. I did what Tim requested for, but that > > one slipped through. > > > > > > Signed-off-by: Allen Kay allen.m.kay@intel.com > > > > > > This is fine, but please add a comment explaining the rather subtle use > > > of INVALID_MFN here so the next tidy-minded person doesn''t "fix" it > > > again. :) > > > > > > In fact would it be better to check the returned type == p2m_mmio_direct > > > instead? That seems like it would fit better with the intention. Or > > > do other callers use this to clean non-mmio entries too? > > > > I''m interested in if this patch fixes the SR-IOV issue. We are still about > > investigation... > > > > Christoph > > > > > > > > -- > > ---to satisfy European Law for business letters: > > Advanced Micro Devices GmbH > > Einsteinring 24, 85609 Dornach b. Muenchen > > Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd > > Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen > > Registergericht Muenchen, HRB Nr. 43632 > > > > > > -- > Tim Deegan <Tim.Deegan@citrix.com> > Principal Software Engineer, XenServer Engineering > Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xensource.com > http://lists.xensource.com/xen-devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Christoph Egger
2010-Sep-02 08:34 UTC
Re: [Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough functionality
On Thursday 02 September 2010 04:26:32 Zhang, Yang Z wrote:> With allen''s patch, both vt-d and sr-iov can work now. Thanks allen.We were not able to reproduce SR-IOV failures with and without the fix on AMD machines. I am wondering if there are differences in memory mappings between AMD and Intel PCI passthrough. Nonetheless, good to know this has been fixed. Thanks, Christoph> best regards > yang > > > -----Original Message----- > > From: xen-devel-bounces@lists.xensource.com > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tim Deegan > > Sent: Wednesday, September 01, 2010 4:57 PM > > To: Kay, Allen M > > Cc: Christoph Egger; xen-devel@lists.xensource.com; Keir Fraser > > Subject: [Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough > > functionality > > > > At 17:13 +0100 on 31 Aug (1283274786), Kay, Allen M wrote: > > > I have updated the patch to indicate valid_mfn() cannot be used here > > > as MMIO pages are always above max_mem. Also added sanity checking > > > for type == p2m_mmio_direct before clearing page table entry. This > > > should fix both VT-d and SR-IOV issues reported recently as there are > > > no SR-IOV specific code in Xen. As far as Xen is concern, SR-IOV > > > virtual function is just another PCI device. > > > > Applied; thanks for the fix. > > > > Tim. > > > > > Signed-off-by: Allen Kay allen.m.kay@intel.com > > > > > > -----Original Message----- > > > From: Christoph Egger [mailto:Christoph.Egger@amd.com] > > > Sent: Tuesday, August 31, 2010 2:35 AM > > > To: Tim Deegan > > > Cc: Kay, Allen M; xen-devel@lists.xensource.com; Keir Fraser > > > Subject: Re: [PATCH] re-enable HVM PCI passthrough functionality > > > > > > On Tuesday 31 August 2010 11:17:27 Tim Deegan wrote: > > > > At 02:58 +0100 on 31 Aug (1283223494), Kay, Allen M wrote: > > > > > Attached patch reverts back to INVALID_MFN. PCI passthrough > > > > > functionality works with this patch. > > > > > > > > Apologies for missing this - I did ask for this enormous patch to > > > > contain only what it said on the tin (and I did read it to check!) > > > > but this one got through. > > > > > > Also apologies from my side. I did what Tim requested for, but that > > > one slipped through. > > > > > > > > Signed-off-by: Allen Kay allen.m.kay@intel.com > > > > > > > > This is fine, but please add a comment explaining the rather subtle > > > > use of INVALID_MFN here so the next tidy-minded person doesn''t "fix" > > > > it again. :) > > > > > > > > In fact would it be better to check the returned type => > > > p2m_mmio_direct instead? That seems like it would fit better with > > > > the intention. Or do other callers use this to clean non-mmio > > > > entries too? > > > > > > I''m interested in if this patch fixes the SR-IOV issue. We are still > > > about investigation... > > > > > > Christoph-- ---to satisfy European Law for business letters: Advanced Micro Devices GmbH Einsteinring 24, 85609 Dornach b. Muenchen Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen Registergericht Muenchen, HRB Nr. 43632 _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Tim Deegan
2010-Sep-02 09:07 UTC
Re: [Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough functionality
At 09:34 +0100 on 02 Sep (1283420058), Christoph Egger wrote:> On Thursday 02 September 2010 04:26:32 Zhang, Yang Z wrote: > > With allen''s patch, both vt-d and sr-iov can work now. Thanks allen. > > We were not able to reproduce SR-IOV failures with and without the fix > on AMD machines. I am wondering if there are differences in memory > mappings between AMD and Intel PCI passthrough.I expect that you tested on machines with >4GiB of RAM, passing through 32-bit MMIO addresses (which would then pass mfn_valid()) and Intel either had less RAM or higher BARs in their test (so their MMIO addresses were !mfn_valid() but != INVALID_MFN). Cheers, Tim.> Nonetheless, good to know this has been fixed. > > Thanks, > Christoph > > > best regards > > yang > > > > > -----Original Message----- > > > From: xen-devel-bounces@lists.xensource.com > > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Tim Deegan > > > Sent: Wednesday, September 01, 2010 4:57 PM > > > To: Kay, Allen M > > > Cc: Christoph Egger; xen-devel@lists.xensource.com; Keir Fraser > > > Subject: [Xen-devel] Re: [PATCH] re-enable HVM PCI passthrough > > > functionality > > > > > > At 17:13 +0100 on 31 Aug (1283274786), Kay, Allen M wrote: > > > > I have updated the patch to indicate valid_mfn() cannot be used here > > > > as MMIO pages are always above max_mem. Also added sanity checking > > > > for type == p2m_mmio_direct before clearing page table entry. This > > > > should fix both VT-d and SR-IOV issues reported recently as there are > > > > no SR-IOV specific code in Xen. As far as Xen is concern, SR-IOV > > > > virtual function is just another PCI device. > > > > > > Applied; thanks for the fix. > > > > > > Tim. > > > > > > > Signed-off-by: Allen Kay allen.m.kay@intel.com > > > > > > > > -----Original Message----- > > > > From: Christoph Egger [mailto:Christoph.Egger@amd.com] > > > > Sent: Tuesday, August 31, 2010 2:35 AM > > > > To: Tim Deegan > > > > Cc: Kay, Allen M; xen-devel@lists.xensource.com; Keir Fraser > > > > Subject: Re: [PATCH] re-enable HVM PCI passthrough functionality > > > > > > > > On Tuesday 31 August 2010 11:17:27 Tim Deegan wrote: > > > > > At 02:58 +0100 on 31 Aug (1283223494), Kay, Allen M wrote: > > > > > > Attached patch reverts back to INVALID_MFN. PCI passthrough > > > > > > functionality works with this patch. > > > > > > > > > > Apologies for missing this - I did ask for this enormous patch to > > > > > contain only what it said on the tin (and I did read it to check!) > > > > > but this one got through. > > > > > > > > Also apologies from my side. I did what Tim requested for, but that > > > > one slipped through. > > > > > > > > > > Signed-off-by: Allen Kay allen.m.kay@intel.com > > > > > > > > > > This is fine, but please add a comment explaining the rather subtle > > > > > use of INVALID_MFN here so the next tidy-minded person doesn''t "fix" > > > > > it again. :) > > > > > > > > > > In fact would it be better to check the returned type => > > > > p2m_mmio_direct instead? That seems like it would fit better with > > > > > the intention. Or do other callers use this to clean non-mmio > > > > > entries too? > > > > > > > > I''m interested in if this patch fixes the SR-IOV issue. We are still > > > > about investigation... > > > > > > > > Christoph > > > -- > ---to satisfy European Law for business letters: > Advanced Micro Devices GmbH > Einsteinring 24, 85609 Dornach b. Muenchen > Geschaeftsfuehrer: Alberto Bozzo, Andrew Bowd > Sitz: Dornach, Gemeinde Aschheim, Landkreis Muenchen > Registergericht Muenchen, HRB Nr. 43632 >-- Tim Deegan <Tim.Deegan@citrix.com> Principal Software Engineer, XenServer Engineering Citrix Systems UK Ltd. (Company #02937203, SL9 0BG) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel