Hi folks, When I tried to run HVM pv drivers on non-ACPI SMP guest, The guest crashes complaining pv drivers using edged triggered interrupt. When ACPI is disabled, Linux guest uses MP table to set up interrupt, so I checked out mp table in hvmloader. Per my understanding, it''s a dynamically generated version of original BOCH rom''s static mp table. However, there is only one bus entry in that table, ISA. All the interrupts is related to ISA bus and their trigger mode is set to `follow bus'', which Linux sets to edge triggered later. I don''t know whether lacking a PCI bus entry was OK, but now it turns out to have problem with the pv driver. Is this going to be fixed? Qing _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2006-Oct-27 07:50 UTC
Re: [Xen-devel] edge-triggered interrupts in non-ACPI SMP HVM
On 27/10/06 3:20 am, "He, Qing" <qing.he@intel.com> wrote:> However, there is only one bus entry in that table, ISA. All the > interrupts is related to ISA bus and their trigger mode is set to > `follow bus'', which Linux sets to edge triggered later. > > I don''t know whether lacking a PCI bus entry was OK, but now it turns > out to have problem with the pv driver. Is this going to be fixed?That''s odd, since the guest should see the PV platform device as a PCI device just like any other (IDE, rtl8139, ...). So I wonder why the guest only complains about the PV platform device? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2006-Oct-27 08:11 UTC
Re: [Xen-devel] edge-triggered interrupts in non-ACPI SMP HVM
On 27/10/06 9:17 am, "He, Qing" <qing.he@intel.com> wrote:> The guest crashed here when insmod platform-pci driver. So that means guest > considers the irq as edge triggered. (in fact, if hypervisor does not > intentionally crash, the guest is likely to work happily). > > Later I checked Linux source, when ACPI is disabled, Linux read MP table to > decide irq to I/O APIC binding. This is where it sets the trigger mode of > irqs. See arch/i386/kernel/mpparse.c and arch/i386/kernel/io_apic.c > > Mp talbe is constructed in hvmloader/mp_table.c, which has only ISA bus...Yes, but why does the guest set only this PCI device''s IRQ to edge-triggered? Why not the rtl8139, for example? Neither is explicitly listed in the MP tables, so where does this differing behaviour come from? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
He, Qing
2006-Oct-27 08:17 UTC
RE: [Xen-devel] edge-triggered interrupts in non-ACPI SMP HVM
Keir,
It''s not guest complaining, it''s hypervisor who complains.
There are such lines in vioapic.c:
DPRINTK("Forcing edge triggered APIC irq %d?\n", irq);
domain_crash(d);
The guest crashed here when insmod platform-pci driver. So that means guest
considers the irq as edge triggered. (in fact, if hypervisor does not
intentionally crash, the guest is likely to work happily).
Later I checked Linux source, when ACPI is disabled, Linux read MP table to
decide irq to I/O APIC binding. This is where it sets the trigger mode of irqs.
See arch/i386/kernel/mpparse.c and arch/i386/kernel/io_apic.c
Mp talbe is constructed in hvmloader/mp_table.c, which has only ISA bus...
Thanks,
Qing>-----Original Message-----
>From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk]
>Sent: 2006年10月27日 15:50
>To: He, Qing; xen-devel@lists.xensource.com
>Subject: Re: [Xen-devel] edge-triggered interrupts in non-ACPI SMP HVM
>
>
>
>
>On 27/10/06 3:20 am, "He, Qing" <qing.he@intel.com> wrote:
>
>> However, there is only one bus entry in that table, ISA. All the
>> interrupts is related to ISA bus and their trigger mode is set to
>> `follow bus'', which Linux sets to edge triggered later.
>>
>> I don''t know whether lacking a PCI bus entry was OK, but now
it turns
>> out to have problem with the pv driver. Is this going to be fixed?
>
>That''s odd, since the guest should see the PV platform device as a
PCI
>device just like any other (IDE, rtl8139, ...). So I wonder why the guest
>only complains about the PV platform device?
>
> -- Keir
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel
He, Qing
2006-Oct-27 08:30 UTC
RE: [Xen-devel] edge-triggered interrupts in non-ACPI SMP HVM
>-----Original Message----- >From: Keir Fraser [mailto:Keir.Fraser@cl.cam.ac.uk] >Sent: 2006年10月27日 16:11 >To: He, Qing; xen-devel@lists.xensource.com >Subject: Re: [Xen-devel] edge-triggered interrupts in non-ACPI SMP HVM > > > > >On 27/10/06 9:17 am, "He, Qing" <qing.he@intel.com> wrote: > >> The guest crashed here when insmod platform-pci driver. So that means guest >> considers the irq as edge triggered. (in fact, if hypervisor does not >> intentionally crash, the guest is likely to work happily). >> >> Later I checked Linux source, when ACPI is disabled, Linux read MP table to >> decide irq to I/O APIC binding. This is where it sets the trigger mode of >> irqs. See arch/i386/kernel/mpparse.c and arch/i386/kernel/io_apic.c >> >> Mp talbe is constructed in hvmloader/mp_table.c, which has only ISA bus... > >Yes, but why does the guest set only this PCI device''s IRQ to >edge-triggered? Why not the rtl8139, for example? Neither is explicitly >listed in the MP tables, so where does this differing behaviour come from? > > -- KeirPer my understanding, rtl8139 works fine even if it is edge-triggered, because there is no interrupt sharing. So if there is no platform pci device, it is possible for guest to set all devices to edge triggered without complaining anything. (I do know that if I comment out sanity checks in HV, platform pci device works fine in edge triggered mode, and there will be no complaints.) Qing _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser
2006-Oct-27 09:03 UTC
Re: [Xen-devel] edge-triggered interrupts in non-ACPI SMP HVM
On 27/10/06 9:30 am, "He, Qing" <qing.he@intel.com> wrote:>> Yes, but why does the guest set only this PCI device''s IRQ to >> edge-triggered? Why not the rtl8139, for example? Neither is explicitly >> listed in the MP tables, so where does this differing behaviour come from? >> >> -- Keir > > Per my understanding, rtl8139 works fine even if it is edge-triggered, because > there is no interrupt sharing. So if there is no platform pci device, it is > possible for guest to set all devices to edge triggered without complaining > anything. (I do know that if I comment out sanity checks in HV, platform pci > device works fine in edge triggered mode, and there will be no complaints.)Oh sorry, I see that check is specific to the platform-device IRQ. I guess it should be reduced just to a printk warning, and we should work out what needs adding to the MP tables to get the PCI devices set up with level-triggering. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Ross Maxfield
2006-Oct-27 14:52 UTC
Re: [Xen-devel] edge-triggered interrupts in non-ACPI SMP HVM
Could this be the cause of the spurious interrupts that I reported yesterday? -- Ross>>> On Fri, Oct 27, 2006 at 3:03 AM, in message<C16789EF.3611%Keir.Fraser@cl.cam.ac.uk>, Keir Fraser <Keir.Fraser@cl.cam.ac.uk> wrote:> > > On 27/10/06 9:30 am, "He, Qing" <qing.he@intel.com> wrote: > >>> Yes, but why does the guest set only this PCI device''s IRQ to >>> edge- triggered? Why not the rtl8139, for example? Neither is explicitly >>> listed in the MP tables, so where does this differing behaviour come from? >>> >>> -- Keir >> >> Per my understanding, rtl8139 works fine even if it is edge- triggered, > because >> there is no interrupt sharing. So if there is no platform pci device, it is >> possible for guest to set all devices to edge triggered without complaining >> anything. (I do know that if I comment out sanity checks in HV, platform pci >> device works fine in edge triggered mode, and there will be no complaints.) > > Oh sorry, I see that check is specific to the platform- device IRQ. I guess > it should be reduced just to a printk warning, and we should work out what > needs adding to the MP tables to get the PCI devices set up with > level- triggering. > > -- Keir > > > > _______________________________________________ > Xen- devel mailing list > Xen- devel@lists.xensource.com > http://lists.xensource.com/xen- devel_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel