The patchset improves Xen x2APIC support. Patch 1/3: allocate iommu when create a drhd A drhd is created when parse ACPI DMAR table, but drhd->iommu is not allocated until iommu setup. But iommu is needed by x2APIC which will enable interrupt remapping before iommu setup. This patch allocates iommu when create drhd. And then drhd->ecap can be removed because it''s the same as iommu->ecap. Patch 2/3: improve interrupt remapping and queued invalidation enabling and disabling x2APIC depends on interrupt remapping, so interrupt remapping needs to be enabled before x2APIC. Usually x2APIC is not enabled (x2apic_enabled=0) when enable interrupt remapping, although x2APIC will be enabled later. So it needs to pass a parameter to set interrupt mode in intremap_enable, instead of checking x2apic_enable. This patch adds a parameter "eim" to intremap_enable to achieve it. Interrupt remapping and queued invalidation are already enabled when enable x2apic, so it needn''t to enable them again when setup iommu. This patch checks if interrupt remapping and queued invalidation are already enable or not, and won''t enable them if already enabled. It does the similar in disabling, that''s to say don''t disable them if already disabled. Patch 3/3: improve x2apic enabling logic This patch masks PIC and IOAPIC RTE''s before x2APIC enabling, unmask and restore them after x2APIC enabling. It also really enables interrupt remapping before x2APIC enabling instead of just checking interrupt remapping setting. This patch handles all configuration related to x2APIC. Especially, it handles that BIOS hands over in x2APIC mode (when there is apic id > 255). It checks if x2APIC is already enabled by BIOS. If already enabled, it will disable interrupt remapping and queued invalidation first, then enable them again. Signed-off-by: Weidong Han <weidong.han@intel.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jul-06 14:49 UTC
Re: [Xen-devel] [PATCH 0/3] x2APIC improvement
On Mon, Jul 05, 2010 at 03:07:58PM +0800, Han, Weidong wrote:> The patchset improves Xen x2APIC support. > > Patch 1/3: allocate iommu when create a drhd > A drhd is created when parse ACPI DMAR table, but drhd->iommu is not allocated until iommu setup. But iommu is needed by x2APIC which will enable interrupt remapping before iommu setup. This patch allocates iommu when create drhd. And then drhd->ecap can be removed because it''s the same as iommu->ecap. >Have these patches been tested on a machine where the number of DRHD !number of IOAPICs? _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk wrote:> On Mon, Jul 05, 2010 at 03:07:58PM +0800, Han, Weidong wrote: > >> The patchset improves Xen x2APIC support. >> >> Patch 1/3: allocate iommu when create a drhd >> A drhd is created when parse ACPI DMAR table, but drhd->iommu is not allocated until iommu setup. But iommu is needed by x2APIC which will enable interrupt remapping before iommu setup. This patch allocates iommu when create drhd. And then drhd->ecap can be removed because it''s the same as iommu->ecap. >> >> > > Have these patches been tested on a machine where the number of DRHD !> number of IOAPICs? > >Why need to care about number of DRHD != number of IOAPICs? One DRHD may cover more than one IOAPICs. And there is already check if all IOAPICs have corresponding DRHD. Regards, Weidong _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk
2010-Jul-07 13:56 UTC
Re: [Xen-devel] [PATCH 0/3] x2APIC improvement
On Wed, Jul 07, 2010 at 01:09:08PM +0800, Weidong Han wrote:> Konrad Rzeszutek Wilk wrote: > >On Mon, Jul 05, 2010 at 03:07:58PM +0800, Han, Weidong wrote: > >>The patchset improves Xen x2APIC support. > >> > >>Patch 1/3: allocate iommu when create a drhd > >> A drhd is created when parse ACPI DMAR table, but drhd->iommu is not allocated until iommu setup. But iommu is needed by x2APIC which will enable interrupt remapping before iommu setup. This patch allocates iommu when create drhd. And then drhd->ecap can be removed because it''s the same as iommu->ecap. > >> > > > >Have these patches been tested on a machine where the number of DRHD !> >number of IOAPICs? > > > Why need to care about number of DRHD != number of IOAPICs? One DRHD > may cover more than one IOAPICs. And there is already check if allAnd the inverse might be true as well - in which case we should disable x2APIC.> IOAPICs have corresponding DRHD.I understand. I am asking whether these patches have not changed that behavior and whether you have tested for that. _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Konrad Rzeszutek Wilk wrote:> On Wed, Jul 07, 2010 at 01:09:08PM +0800, Weidong Han wrote: > >> Konrad Rzeszutek Wilk wrote: >> >>> On Mon, Jul 05, 2010 at 03:07:58PM +0800, Han, Weidong wrote: >>> >>>> The patchset improves Xen x2APIC support. >>>> >>>> Patch 1/3: allocate iommu when create a drhd >>>> A drhd is created when parse ACPI DMAR table, but drhd->iommu is not allocated until iommu setup. But iommu is needed by x2APIC which will enable interrupt remapping before iommu setup. This patch allocates iommu when create drhd. And then drhd->ecap can be removed because it''s the same as iommu->ecap. >>>> >>>> >>> Have these patches been tested on a machine where the number of DRHD !>>> number of IOAPICs? >>> >>> >> Why need to care about number of DRHD != number of IOAPICs? One DRHD >> may cover more than one IOAPICs. And there is already check if all >> > > And the inverse might be true as well - in which case we should disable > x2APIC. > >Each IOAPIC must have a corresponding DRHD, but a DRHD maybe don''t have a IOAPIC in its device scope. So it won''t enable x2APIC if there is any IOAPIC which doesn''t have corresponding DRHD.>> IOAPICs have corresponding DRHD. >> > > I understand. I am asking whether these patches have not changed that > behavior and whether you have tested for that. >These patches don''t change that behavior. Regards, Weidong _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Hi Keir, In order to support that BIOS hands over in x2APIC mode, these patches are necessary. So it''s better to also back port them to xen 4.0. I already ported these patches to xen-4.0-testing tree, pls check them in. Thanks. Regards, Weidong -----Original Message----- From: xen-devel-bounces@lists.xensource.com [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Han, Weidong Sent: Monday, July 05, 2010 3:08 PM To: Xen-devel Cc: Keir Fraser Subject: [Xen-devel] [PATCH 0/3] x2APIC improvement The patchset improves Xen x2APIC support. Patch 1/3: allocate iommu when create a drhd A drhd is created when parse ACPI DMAR table, but drhd->iommu is not allocated until iommu setup. But iommu is needed by x2APIC which will enable interrupt remapping before iommu setup. This patch allocates iommu when create drhd. And then drhd->ecap can be removed because it''s the same as iommu->ecap. Patch 2/3: improve interrupt remapping and queued invalidation enabling and disabling x2APIC depends on interrupt remapping, so interrupt remapping needs to be enabled before x2APIC. Usually x2APIC is not enabled (x2apic_enabled=0) when enable interrupt remapping, although x2APIC will be enabled later. So it needs to pass a parameter to set interrupt mode in intremap_enable, instead of checking x2apic_enable. This patch adds a parameter "eim" to intremap_enable to achieve it. Interrupt remapping and queued invalidation are already enabled when enable x2apic, so it needn''t to enable them again when setup iommu. This patch checks if interrupt remapping and queued invalidation are already enable or not, and won''t enable them if already enabled. It does the similar in disabling, that''s to say don''t disable them if already disabled. Patch 3/3: improve x2apic enabling logic This patch masks PIC and IOAPIC RTE''s before x2APIC enabling, unmask and restore them after x2APIC enabling. It also really enables interrupt remapping before x2APIC enabling instead of just checking interrupt remapping setting. This patch handles all configuration related to x2APIC. Especially, it handles that BIOS hands over in x2APIC mode (when there is apic id > 255). It checks if x2APIC is already enabled by BIOS. If already enabled, it will disable interrupt remapping and queued invalidation first, then enable them again. Signed-off-by: Weidong Han <weidong.han@intel.com> _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel