Xen devel - Apr 2006 - [PATCH][FIX] Possible fix for spurious interrupts

Here is a patch that appears to fix the spurious interrupt problems for 
me. This is inspired by Jan''s observation that the IO-APIC mask/unmask 
that we do for ISR processing seems to cause corruption of other 
routing entries. So the fix here is to do the same as native Linux -- 
do not mask/unmask, but instead send EOI after doing ISR processing.

To confirm this really was the problem I modified native Linux to do 
the same as Xen currently does (mask before ISR processing; unmask 
after). I then got lots of spurious USB interrupts even on native 
Linux! :-(

Please test... I''ll put it in xen-unstable tomorrow if there are no 
complaints and it will be a strong candidate for the next 3.0.2 release 
too.

  -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Hi,

On Thu, 2006-04-13 at 19:12 +0100, Keir Fraser wrote:
> To confirm this really was the problem I modified native Linux to do 
> the same as Xen currently does (mask before ISR processing; unmask 
> after). I then got lots of spurious USB interrupts even on native 
> Linux! :-(
> 
> Please test... 
Looks like it cures it completely for me.  I''ve got the same box
running
the same load that used to send irq16 through the roof, and it''s
currently pegged at:

         16:  0   0   0   0  Phys-irq  uhci_hcd:usb2, uhci_hcd:usb5

Thanks!

I''m moving back to the UK shortly so won''t personally be able
to
double-check this again from the final xen-unstable tree, btw.

--Stephen



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 13 Apr 2006, at 20:59, Stephen C. Tweedie wrote:
> Looks like it cures it completely for me.  I''ve got the same box 
> running
> the same load that used to send irq16 through the roof, and it''s
> currently pegged at:
>
>          16:  0   0   0   0  Phys-irq  uhci_hcd:usb2, uhci_hcd:usb5
>
> Thanks!
Actually, it seems the patch doesn''t entirely work for multi-processor 
domain 0 as the wrong local APIC is sometimes prodded to do the EOI 
cycle. This was partially hidden by the fact that IRQ ffinity is also 
currently broken in our Linux tree (we need to call move_irq() in a few 
choice places). I''ll get a complete fix for all this stuff into 
xen-unstable today. Probably the meat of the patch should wait a while 
before going into the -testing tree, but we''ll want to get something in
there before 3.0.2-3.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>
>> Looks like it cures it completely for me.  I''ve got the same
box
>> running
>> the same load that used to send irq16 through the roof, and
it''s
>> currently pegged at:
>>
>>          16:  0   0   0   0  Phys-irq  uhci_hcd:usb2, uhci_hcd:usb5
>>
>> Thanks!
>
>Actually, it seems the patch doesn''t entirely work for
multi-processor
>domain 0 as the wrong local APIC is sometimes prodded to do the EOI 
>cycle. This was partially hidden by the fact that IRQ ffinity is also 
>currently broken in our Linux tree (we need to call move_irq() 
>in a few 
Yes, this patch seems hang the SMP dom0 system at all, even pressing
ctrl-A 3 times won''t response.
So, waiting for your complete fix :-)
-Xin

>choice places). I''ll get a complete fix for all this stuff into 
>xen-unstable today. Probably the meat of the patch should wait a while 
>before going into the -testing tree, but we''ll want to get 
>something in 
>there before 3.0.2-3.
>
>  -- Keir
>
>
>_______________________________________________
>Xen-devel mailing list
>Xen-devel@lists.xensource.com
>http://lists.xensource.com/xen-devel
>
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 14 Apr 2006, at 11:02, Li, Xin B wrote:
>> Actually, it seems the patch doesn''t entirely work for
multi-processor
>> domain 0 as the wrong local APIC is sometimes prodded to do the EOI
>> cycle. This was partially hidden by the fact that IRQ ffinity is also
>> currently broken in our Linux tree (we need to call move_irq()
>> in a few
>
> Yes, this patch seems hang the SMP dom0 system at all, even pressing
> ctrl-A 3 times won''t response.
> So, waiting for your complete fix :-)
There''s something going through our staging tree now, so should be on 
xenbits in the early afternoon (xen-unstable tree).

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 14 Apr 2006, at 12:07, Keir Fraser wrote:
>> Yes, this patch seems hang the SMP dom0 system at all, even pressing
>> ctrl-A 3 times won''t response.
>> So, waiting for your complete fix :-)
>
> There''s something going through our staging tree now, so should be
on
> xenbits in the early afternoon (xen-unstable tree).
The patch doesn''t seem to work well on some systems (e.g., Dell 1850) 
so I''ve applied another patch that defaults to the old ack method by 
default. If you want to try out the new method then for the time being 
you''ll need to put ''new_ack'' as a boot parameter to
Xen.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

This looks questionable to me (even after your follow-up mails). You are, in
this mode, ack-ing the interrupt before the
handler even gets invoked. I would expect level triggered interrupts to fire
again right away then if you don''t manage
to run the handling domain before re-enabling interrupts. Of course, I
haven''t seen your later attempts at fixing the
problem, but am I missing something here?


Jan
>>> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> 04/13/06 8:12 PM
>>>
Here is a patch that appears to fix the spurious interrupt problems for 
me. This is inspired by Jan''s observation that the IO-APIC mask/unmask 
that we do for ISR processing seems to cause corruption of other 
routing entries. So the fix here is to do the same as native Linux -- 
do not mask/unmask, but instead send EOI after doing ISR processing.

To confirm this really was the problem I modified native Linux to do 
the same as Xen currently does (mask before ISR processing; unmask 
after). I then got lots of spurious USB interrupts even on native 
Linux! :-(

Please test... I''ll put it in xen-unstable tomorrow if there are no 
complaints and it will be a strong candidate for the next 3.0.2 release 
too.

  -- Keir



_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 14 Apr 2006, at 18:42, Jan Beulich wrote:
> This looks questionable to me (even after your follow-up mails). You 
> are, in this mode, ack-ing the interrupt before the
> handler even gets invoked. I would expect level triggered interrupts 
> to fire again right away then if you don''t manage
> to run the handling domain before re-enabling interrupts. Of course, I 
> haven''t seen your later attempts at fixing the
> problem, but am I missing something here?
Yes, I ack *after* the handlers have executed. The ack is in the ->end 
handler.

This is dodgy too on Xen unfortunately. All lower priority interrupts 
are blocked until the driver domain sees fit to tell Xen it''s finished 
ISR processing. But if a large proportion of x86 hardware is screwed 
with respect to IO-APIC masking then we have no choice but to get this 
scheme to work. I look forward to the day when MSI is ubiquitous.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

I think you guys are running into the same problem that FreeBSD ran 
into on some Intel motherboards more than a year ago.

This explanation seems to make the most sense. 

http://thread.gmane.org/gmane.os.freebsd.current/67490/focus=67490

Because the problem happened on FreeBSD (which masks ioapic RTEs to
implement interrupt threads) and not on Linux, it was hard to get
attention from the hardware guys back then. I had suspected Xen
would run into it sooner or later.

	-Arun

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 15 Apr 2006, at 19:57, Arun Sharma wrote:
> I think you guys are running into the same problem that FreeBSD ran
> into on some Intel motherboards more than a year ago.
>
> This explanation seems to make the most sense.
>
> http://thread.gmane.org/gmane.os.freebsd.current/67490/focus=67490
>
> Because the problem happened on FreeBSD (which masks ioapic RTEs to
> implement interrupt threads) and not on Linux, it was hard to get
> attention from the hardware guys back then. I had suspected Xen
> would run into it sooner or later.
Thanks Arun, this is very informative although unfortunately not very 
helpful. Matt Dillon''s suggested alternatives to masking do not really 
work as they all cause spurious interrupts. Do you know if they ever 
found a good fix, or do they live with the problem?

I''d not heard of boot interrupt mode before, but it sounds like many 
chipsets cannot disable it and, even when it can be disabled, the 
method is chipset specific. The Intel legacy INTx model is so 
unbelievably crap. At least source-vectored interrupts are becoming 
more common.

Anyway, this the current status of my workaround for Xen:
  1. I added a new ioapic ack method which delays EOI until after ISR 
processing in the driver domain. This mode is enabled by default but 
can be disabled with ''ioapic_ack=old'' as a Xen boot parameter.
  2. The code to safely manage deferred EOI is quite complicated and has 
some weaknesses:
      * Must EOI on the CPU that received the interrupt
      * Must EOI in ''reverse'' order when interrupts have
nested
      * Un-EOIed interrupts block other guest-bound interrupts which 
happen to have lower priority
      * Right now, disable_irq() in a driver domain may potentially lock 
up all interrupt sources as it may prevent EOI ever happening (until 
enable_irq() or the interrupt is unbound from the domain)
      * All Xen-bound interrupts have strictly higher priority than any 
guest-bound IO-APIC interrupt. This should avoid deadlock issues.

Really it''s a messy solution. I think having both old and new ack 
methods makes sense, but I''m not sure how we will end up picking which 
to use automatically. Maybe using the old method is best, and let users 
pick the new one if they see spurious interrupt problems. Or maybe the 
problems with the new method are mostly theoretical and we should use 
that by default. Or maybe we should have a DMI table to pick between 
them. I''m not sure.

Another question is whether to put this in 3.0.2. I think it definitely 
needs more testing before that, but it might not make sense to do so at 
all as the patch is quite invasive.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

Keir Fraser wrote:
> 
> Thanks Arun, this is very informative although unfortunately not very 
> helpful. Matt Dillon''s suggested alternatives to masking do not
really
> work as they all cause spurious interrupts. Do you know if they ever 
> found a good fix, or do they live with the problem?
They never found a good fix. How they live with the problem is explained 
in this status report under "e1000 driver improvements":

http://www.freebsd.org/news/status/report-oct-2005-dec-2005.html

	-Arun

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> Keir Fraser <Keir.Fraser@cl.cam.ac.uk> 14.04.06 20:42:18
>>>
>
>On 14 Apr 2006, at 18:42, Jan Beulich wrote:
>
>> This looks questionable to me (even after your follow-up mails). You 
>> are, in this mode, ack-ing the interrupt before the
>> handler even gets invoked. I would expect level triggered interrupts 
>> to fire again right away then if you don''t manage
>> to run the handling domain before re-enabling interrupts. Of course, I 
>> haven''t seen your later attempts at fixing the
>> problem, but am I missing something here?
>
>Yes, I ack *after* the handlers have executed. The ack is in the ->end 
>handler.
Oh, I see. I was looking at the patch only (at home), and missed that its main
change was to the end routine.
>This is dodgy too on Xen unfortunately. All lower priority interrupts 
>are blocked until the driver domain sees fit to tell Xen it''s
finished
>ISR processing. But if a large proportion of x86 hardware is screwed 
>with respect to IO-APIC masking then we have no choice but to get this 
>scheme to work. I look forward to the day when MSI is ubiquitous.
... and even worse when the interrupt belongs to a domU.

Have you heard anything from Intel on this? I am still hoping that there is a
pattern which interrupt gets triggered by
masking a certain other one, so a (perhaps more complex, but less impacting) fix
could be derived...

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 20 Apr 2006, at 08:45, Jan Beulich wrote:
> ... and even worse when the interrupt belongs to a domU.
>
> Have you heard anything from Intel on this? I am still hoping that 
> there is a pattern which interrupt gets triggered by
> masking a certain other one, so a (perhaps more complex, but less 
> impacting) fix could be derived...
Arun Sharma pointed out its the same problem that FreeBSD has been 
having (they also mask interrupt lines when the ISR is scheduled in an 
interrupt thread). The problem is caused *by design* on many chipsets 
-- the idea is that if the bootloader or OS does not know about ACPI 
and cannot find the IO-APICs then interrupt lines routed to those 
IO-APICs should have their interrupt fired on a legacy line (it so 
happens that this usually is also the line used by the USB 
controllers). They detect this by seeing if the IO-APIC RTE is masked. 
Many chipsets have no way to disable this mode of operation (no need 
for Linux and Windows -- when they detect and enable IO-APICs, they 
never mask active RTEs). So, basically, we''re screwed and have to work 
around by delaying the EOI.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>Arun Sharma pointed out its the same problem that FreeBSD has been 
>having (they also mask interrupt lines when the ISR is scheduled in an 
>interrupt thread). The problem is caused *by design* on many chipsets 
>-- the idea is that if the bootloader or OS does not know about ACPI 
>and cannot find the IO-APICs then interrupt lines routed to those 
>IO-APICs should have their interrupt fired on a legacy line (it so 
>happens that this usually is also the line used by the USB 
>controllers). They detect this by seeing if the IO-APIC RTE is masked. 
>Many chipsets have no way to disable this mode of operation (no need 
>for Linux and Windows -- when they detect and enable IO-APICs, they 
>never mask active RTEs). So, basically, we''re screwed and have to
work
>around by delaying the EOI.
Hmm, I read that mail, but it seemed to indicate that this would affect only a
single line. We had been seeing this on
multiple lines, however. Also, to me lines 16 and above aren''t really
''legacy'', so the implication might rather be for
routing in systems with multiple IO-APICs where older software only finds one.
As this would then seem to be
deterministic, the question would be if there is a way to know which IO-APIC
triggers which line (this must be possible,
or else the ''older software'' wouldn''t be able to know
where the interrupts arrive at the first IO-APIC), and use this
knowledge, or at least restrict the fix currently in use to all but the first
(''legacy'') IO-APIC (which should improve
things namely on single-IO-APIC systems).

Jan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi


I am having toubles bluiding openafs inside a virtual machine.

1. The kernel headers inside the xen-3.0.0/linux-2.6.12-xenU tree are
not good. So the ./configure works, but the build fails with:

/root/openafs-1.4.0/src/afs/LINUX/osi_machdep.h:55:2: #error Not sure
what to do about rlim (should be in the Linux task struct somewhere....)

2. Forcing the ./configure agains linux-kernel-2.6.12 (not xen''s)
headers, I get the build to finish just nice, but them insmod returns:

insmod: error inserting
''./src/libafs/MODLOAD-2.6.12.6-MP/libafs.ko'': -1
Invalid module format

3. I have sucessfully buil the module inside the dom0 (using xen source
kernel headers) and loaded it OK, also.

Does someone know if this is expected?
Xen kernel headers are much diferent from the original ones?
How far can they be replaced by other headers?
How to build something that is not xen aware, thus is expecting normal
kernel headers?

Thanks for your time

Regards
- --
Tiago Teresa Teodódio
tiago@rnl.ist.utl.pt

Administração da Rede das Novas Licenciaturas
Instituto Superior Técnico

web: http://www.rnl.ist.utl.pt
email: rnl@rnl.ist.utl.pt
tel: +351 218 41 77 71

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2.2 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFER1OoQO/Tafxh35cRAtljAKC5PuoSmDZxI2jM1qoyAJf+hHKacQCglVhL
3Sy9qV+Sy8W4qvDAVe1CPFw=lS4i
-----END PGP SIGNATURE-----

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 20 Apr 2006, at 09:53, Jan Beulich wrote:
> Hmm, I read that mail, but it seemed to indicate that this would 
> affect only a single line. We had been seeing this on
> multiple lines, however. Also, to me lines 16 and above aren''t
really
> ''legacy'', so the implication might rather be for
> routing in systems with multiple IO-APICs where older software only 
> finds one. As this would then seem to be
> deterministic, the question would be if there is a way to know which 
> IO-APIC triggers which line (this must be possible,
> or else the ''older software'' wouldn''t be able to
know where the
> interrupts arrive at the first IO-APIC), and use this
> knowledge, or at least restrict the fix currently in use to all but 
> the first (''legacy'') IO-APIC (which should improve
> things namely on single-IO-APIC systems).
I am certain that we have been hitting *at least* this issue. Maybe 
there are others, but I''ll be surprised if you continue to hit spurious
interrupt problems with the workaround in place.

Here is my understanding of this boot interrupt mode and why it affects 
us:
  1. PCI-X bridge interrupt inputs are not configurable directly as 
inputs to the 859 PICs (I/O hubs usually allow that for normal PCI 
interrupt inputs)
  2. So any OS not aware of APICs cannot get at those interrupt lines 
unless a hardware hack is introduced
  3. Intel introduce a ''boot interrupt'' line by wire-ORing
together all
the interrupt inputs to an IO-APIC that services PCI-X interrupts, and 
forwarding the resulting ''boot interrupt'' line to some INTx
line or pin
on the 8259 (note: I''m unsure where this boot interrupt actually gets 
wired to).
  4. I guess that if you have more than one PCI-X IO-APIC, they may 
forward their boot interrupts to different INTx lines and so appear as 
different interrupts to the OS.

What I''m not sure about:
  1. How are legacy OSes supposed to detect these devices and know that 
the interrupt lines are routed to a particular 8259 pin?
  2. Why does this alias with an IOAPIC0 input? Perhaps the boot 
interrupt fires on an INTx line that is then routed to both 8259 and 
IOAPIC0 -- so you get it on whichever happens to be enabled, and it 
ends up looking like an interrupt from whichever device is bound to 
that INTx line. Or perhaps the 8259 output is wired into IOAPIC0. I''m 
not sure.

The workaround shouldn''t hurt performance, assuming interrupts are 
usually delivered to the CPU on which they are serviced. I''m also 
confident of its correctness, assuming driver domains service their 
interrupts in a reasonably timely fashion (a shame to have to make that 
assumption, but once we have MSI support we can recommend people use 
that if they want better driver-domain isolation).

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On Thu, Apr 20, 2006 at  3:35 AM, in message
<41b22a1a50ce8da0b406f088e78d14f1@cl.cam.ac.uk>, Keir Fraser
<Keir.Fraser@cl.cam.ac.uk> wrote: 
> I am certain that we have been hitting *at least* this issue. Maybe 
> there are others, but I''ll be surprised if you continue to hit
spurious 
> interrupt problems with the workaround in place.
> 
> Here is my understanding of this boot interrupt mode and why it
affects 
> us:
>   1. PCI- X bridge interrupt inputs are not configurable directly as
> inputs to the 8259 PICs (I/O hubs usually allow that for normal PCI 
> interrupt inputs)
>   2. So any OS not aware of APICs cannot get at those interrupt lines
> unless a hardware hack is introduced
True.  But it''s worse than that.  Even before PCI X some systems only
routed some interrupts to a specific GSIs > 15. These devices were never
available to a legacy OS at all.  Only if the OS was in full symmetric
IO mode.  These systems are rare however and devices that fall into this
category should never be setup by the manufacturer as boot devices so
its not an issue unless your trying to run a legacy OS.
>   3. Intel introduce a ''boot interrupt'' line by wire-
ORing together
all 
> the interrupt inputs to an IO- APIC that services PCI- X interrupts,
and 
> forwarding the resulting ''boot interrupt'' line to some
INTx line or
pin 
> on the 8259 
TRUE, but my understanding was that its not necessarily just one line.
>(note: I''m unsure where this boot interrupt actually gets 
> wired to).
It''s manufacturer dependent and you don''t have to care.  See
PCI config
space discussion below.
>   4. I guess that if you have more than one PCI- X IO- APIC, they may
> forward their boot interrupts to different INTx lines and so appear
as 
> different interrupts to the OS.
TRUE.  There is lots of IRQ sharing until you switch to symmetric IO
mode.  On some systems the event that causes the firmware to reroute
interrupts is when the OS unmasks or masks an IO APIC RTE entry.  (Sorry
I couldn''t remember that earlier I would have saved you some grief.) 
> What I''m not sure about:
>   1. How are legacy OSes supposed to detect these devices and know
that 
> the interrupt lines are routed to a particular 8259 pin?
For PCI the BIOS or utility modifying the interrupt route is required
to write the legacy IRQ information into the "Interrupt Line" register
of the Devices Configuration Space Header.  Values of 0x00 through 0x0F
are valid.  0xFF is also valid and means that there is no interrupt
route or that the device does use interrupts.
>   2. Why does this alias with an IOAPIC0 input? 
In my opinion once you''ve switched to full symmetric mode it
shouldn''t.
 But according to ACPI Systems that support both APIC and dual 8259
interrupt models must map global system interrupts 0-15 to the 8259 IRQs
0-15, except where Interrupt Source Overrides are provided as is almost
always the case with the 8254 timer IRQ0 which almost always goes to
GSI2 (IOAPIC0 INTIN2).  

This aliasing issue can get rather messy.  Since you don''t know if dual
routes exist once interrupts are fanned out to other IOAPIC lines (other
than GSI 0-15) you have to assume the duplicate routes can and do exist.
 My experience tell me that they almost always exist. 

If duplicate routes don''t exist then the BIOS can leave some interrupts
on the legacy IRQ and still pull others off to a separate IO APIC line
otherwise they all have to stay on the legacy GSI or they all have to
move somewhere else.  This is why we have to trust MPS or ACPI to be
correct.  Its usually when a PCI interrupt entry in ACPI or MPS is not
properly described that you run into this problem.  
> Perhaps the boot 
> interrupt fires on an INTx line that is then routed to both 8259 and
> IOAPIC0
TRUE in most cases.
> so you get it on whichever happens to be enabled, 
TRUE
> and it 
> ends up looking like an interrupt from whichever device is bound to 
> that INTx line. 
Yes, and the machine will may lock up at that point due to an pending
interrupt that never get serviced and keeps firing over and over.
> Or perhaps the 8259 output is wired into IOAPIC0. I''m 
> not sure.
The output of the 8259 is typically routed to INTIN0 of IOAPIC0 or
directly to the processor local APIC LINTIN0.  MPS was kind enough to
allow the BIOS to describe the available routes.  ACPI was less helpful
in describing virtual wire mode routes.  

So with ACPI if you need a 8259 virtual wire mode route you choose one
and then look for the 8254 timer interrupt coming through.  Or you can
refer to MPS tables.  The 8254 timer is typically routed to INTIN2 of
IOAPIC0 but be aware that some early systems failed to route the 8254
timer directly to IOAPIC0 INTIN2 so mixed mode operation (IOACPI/PIC)
would be required.  There are a whole set of issues related to mixed
mode.  Don''t go there unless you have to.
> The workaround shouldn''t hurt performance, assuming interrupts are
> usually delivered to the CPU on which they are serviced. 
It will hurt performance.  How much?  I don''t know.  Leaving an
interrupt "in service" until the right domain is scheduled to service
the interrupt will cause all interrupts of a equal or lower class
priority on that processor to wait until EOI.  Further interrupts of the
same class will also be held off.  So never put IO interrupts in the
same APIC interrupt class as timer and IPIs.

I would recommend that for performance reasons we might want to only
apply this fix for systems which have GSIs > 15.  Meaning that for
systems with only one IO APIC with only 16 lines we may be okay without
it.  

There may be other reasons for not masking IO APICs besides this
interrupt routing issue I just can''t remember what they are.  I
couldn''t
remember this whole duplicate route issue until you figured it out based
on my suggestion to Jan Beulich that I was pretty sure it was masking
related.  The other issue I do remember is that sometimes we saw level
triggered interrupt assertions get lost as a result of masking.  Not
sure why.  Hence I really don''t like this idea of touching the IO APIC
at all after its once setup.

Other suggestions:
If a spurious interrupt problems persists you might try just flipping
bit 16 instead of both 15 and 16 when you mask/unmask GSIs 0-15.  But I
am not sure this is a correct thing to do for newer chipsets.  Flipping
both 15 and 16 may be correct.
Be aware that an interrupt could be in flight even though masked.
Never mask an EXTINT entry if you have any.  EXTINT is treated like an
edge triggered interrupt.  Mask the inputs to the 8259 instead.
>I''m also 
> confident of its correctness, assuming driver domains service their 
> interrupts in a reasonably timely fashion (a shame to have to make
that 
> assumption, but once we have MSI support we can recommend people use
> that if they want better driver- domain isolation).
Agreed.  But there are lots or reasons for spurious interrupts beside
duplicate routes.  The main thing is that we don''t get too many  Linux
shuts them off.

Hope at least some of this helps.

Clyde R. Griffin
Virtualization Platform Team
Novell, Inc.
> 
>   --  Keir
> 
> 
> _______________________________________________
> Xen- devel mailing list
> Xen- devel@lists.xensource.com
> http://lists.xensource.com/xen- devel

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel