We seem to find an interesting bug, but not for sure. Each time before xen returns to hvm domain, it checks local_events_need_delivery to see whether any events pending and if yes, inject event by callback_irq into the virtual interrupt controller. However, the interesting point is, we never found any place, either in xen or in PV drivers, to clear evtchn_upcall_mask at any time. The initial value of evtchn_upcall_mask is 1 which makes local_events_need_delivery always return 0. One possible reason why PV drivers still works in HVM is that platform pci device happens to share irq line with another pci device within Qemu. In that case, the irq from other pci device may still bring event back into PV drivers. But can this condition always promise? Anyway, we''d like to know whether this is a real bug. If yes, it may have some impact on performance, and then people may clear the evtchn_upcall_mask either at domain creation time or at callback_irq setting time. In any case, this field does nothing for HVM domain since the latter already has ''rflags.if'' for same purpose. Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30/11/06 07:05, "Tian, Kevin" <kevin.tian@intel.com> wrote:> Anyway, we''d like to know whether this is a real bug. If yes, it may > have some impact on performance, and then people may clear the > evtchn_upcall_mask either at domain creation time or at callback_irq > setting time. In any case, this field does nothing for HVM domain since > the latter already has ''rflags.if'' for same purpose.Local_events_need_delivery() should check for pending APIC/PIC interrupts on HVM. That''ll fix the block() wakeup-waiting race too. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: Keir Fraser [mailto:keir@xensource.com] >Sent: 2006年11月30日 17:09 > >On 30/11/06 07:05, "Tian, Kevin" <kevin.tian@intel.com> wrote: > >> Anyway, we''d like to know whether this is a real bug. If yes, it may >> have some impact on performance, and then people may clear the >> evtchn_upcall_mask either at domain creation time or at callback_irq >> setting time. In any case, this field does nothing for HVM domain since >> the latter already has ''rflags.if'' for same purpose. > >Local_events_need_delivery() should check for pending APIC/PIC >interrupts on >HVM. That''ll fix the block() wakeup-waiting race too. > > -- KeirHi, Keir, Sorry that I didn''t quite catch you. Did you mean check for pending interrupts for same gsi line (used as callback irq)? Whether to actually pending an irq instance is decided by APIC/PIC themselves, right? Vmx_intr_assit just needs to check status of evtchn_upcall_pending which serves as a virtual wire state. BTW, what''s the block() wakeup-waiting race above? :-) Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 30/11/06 13:56, "Tian, Kevin" <kevin.tian@intel.com> wrote:>> Local_events_need_delivery() should check for pending APIC/PIC >> interrupts on >> HVM. That''ll fix the block() wakeup-waiting race too. >> >> -- Keir > > Hi, Keir, > Sorry that I didn''t quite catch you. Did you mean check for > pending interrupts for same gsi line (used as callback irq)? Whether > to actually pending an irq instance is decided by APIC/PIC themselves, > right? Vmx_intr_assit just needs to check status of > evtchn_upcall_pending which serves as a virtual wire state. BTW, > what''s the block() wakeup-waiting race above? :-)I mean that the evtchn_upcall_pending flag is not the master flag indicating whether there are pending interrupts for an HVM guest. So it is not the correct thing to test in local_events_need_delivery(). The correct thing to test is whether there are any interrupts pending from the PIC/IOAPIC. The wakeup-waiting race is that a VCPU may block just when, for example, the qemu-dm rtl8139 device model is asserting an interrupt. It will try to vcpu_kick() and send an IPI to the CPU that is running the HVM VCPU. However, before that IPI is received the other CPU may block the HVM VCPU (after checking for local_events_need_delivery() which will in this case return 0). So the rtl8139 interrupt does not prevent the HVM VCPU from blocking, which is a bug. - Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: Keir Fraser [mailto:keir@xensource.com] >Sent: 2006年11月30日 22:51 >> what''s the block() wakeup-waiting race above? :-) > >I mean that the evtchn_upcall_pending flag is not the master flag >indicating >whether there are pending interrupts for an HVM guest. So it is not the >correct thing to test in local_events_need_delivery(). The correct thing to >test is whether there are any interrupts pending from the PIC/IOAPIC. > >The wakeup-waiting race is that a VCPU may block just when, for >example, the >qemu-dm rtl8139 device model is asserting an interrupt. It will try to >vcpu_kick() and send an IPI to the CPU that is running the HVM VCPU. >However, before that IPI is received the other CPU may block the HVM >VCPU >(after checking for local_events_need_delivery() which will in this case >return 0). So the rtl8139 interrupt does not prevent the HVM VCPU from >blocking, which is a bug. > > - KeirOK, I see. That''s a real performance bug.. Thanks for explanation. :-) Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel