Xen devel - May 2011 - [PATCH v2 1/2] x86: skip migrating IRQF_PER_CPU irq in fixup_irqs

x86: skip migrating IRQF_PER_CPU irq in fixup_irqs

IRQF_PER_CPU marks a irq binding to a specific cpu, and can never be
moved away from that cpu. So it shouldn''t be migrated when fixup irqs
to offline a cpu. Xen pvops guest is one source using IRQF_PER_CPU
on a set of virtual interrupts. Previously no error is observed 
because Xen event chip silently fails the set_affinity ops, and
logically IRQF_PER_CPU should be recognized here.

Signed-off-by: Fengzhe Zhang <fengzhe.zhang@intel.com>
Signed-off-by: Kevin Tian <kevin.tian@intel.com>
CC: Thomas Gleixner <tglx@linutronix.de>
CC: Ingo Molnar <mingo@redhat.com>
CC: H. Peter Anvin <hpa@zytor.com>
CC: Ian Campbell <Ian.Campbell@citrix.com>
CC: Jan Beulich <JBeulich@novell.com>

--- linux-2.6.39-rc6.orig/arch/x86/kernel/irq.c	2011-05-04 10:59:13.000000000
+0800
+++ linux-2.6.39-rc6/arch/x86/kernel/irq.c	2011-05-06 09:20:25.563963000 +0800
@@ -249,7 +250,7 @@ void fixup_irqs(void)
 
 		data = irq_desc_get_irq_data(desc);
 		affinity = data->affinity;
-		if (!irq_has_action(irq) ||
+		if (!irq_has_action(irq) || irqd_is_per_cpu(data) ||
 		    cpumask_subset(affinity, cpu_online_mask)) {
 			raw_spin_unlock(&desc->lock);
 			continue;

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, May 06, 2011 at 02:43:36PM +0800, Tian, Kevin
wrote:
> x86: skip migrating IRQF_PER_CPU irq in fixup_irqs
> 
> IRQF_PER_CPU marks a irq binding to a specific cpu, and can never be
> moved away from that cpu. So it shouldn''t be migrated when fixup
irqs
> to offline a cpu. Xen pvops guest is one source using IRQF_PER_CPU
  ^- are called 
> on a set of virtual interrupts. Previously no error is observed 
                                                      ^^- was
 Which ones? Can you be more specific here of which type of virtual interrupts?
 spinlock? timer? 
> because Xen event chip silently fails the set_affinity ops, and
> logically IRQF_PER_CPU should be recognized here.
OK, so what if the set_affinity ops was implemented?
> 
> Signed-off-by: Fengzhe Zhang <fengzhe.zhang@intel.com>
> Signed-off-by: Kevin Tian <kevin.tian@intel.com>
> CC: Thomas Gleixner <tglx@linutronix.de>
> CC: Ingo Molnar <mingo@redhat.com>
> CC: H. Peter Anvin <hpa@zytor.com>
> CC: Ian Campbell <Ian.Campbell@citrix.com>
> CC: Jan Beulich <JBeulich@novell.com>
> 
> --- linux-2.6.39-rc6.orig/arch/x86/kernel/irq.c	2011-05-04
10:59:13.000000000 +0800
> +++ linux-2.6.39-rc6/arch/x86/kernel/irq.c	2011-05-06 09:20:25.563963000
+0800
> @@ -249,7 +250,7 @@ void fixup_irqs(void)
>  
>  		data = irq_desc_get_irq_data(desc);
>  		affinity = data->affinity;
> -		if (!irq_has_action(irq) ||
> +		if (!irq_has_action(irq) || irqd_is_per_cpu(data) ||
>  		    cpumask_subset(affinity, cpu_online_mask)) {
>  			raw_spin_unlock(&desc->lock);
>  			continue;
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> From: Konrad Rzeszutek Wilk [mailto:konrad.wilk@oracle.com]
> Sent: Friday, May 06, 2011 9:58 PM
> 
> On Fri, May 06, 2011 at 02:43:36PM +0800, Tian, Kevin wrote:
> > x86: skip migrating IRQF_PER_CPU irq in fixup_irqs
> >
> > IRQF_PER_CPU marks a irq binding to a specific cpu, and can never be
> > moved away from that cpu. So it shouldn''t be migrated when
fixup irqs
> > to offline a cpu. Xen pvops guest is one source using IRQF_PER_CPU
>   ^- are called
> > on a set of virtual interrupts. Previously no error is observed
>                                                       ^^- was
> Which ones? Can you be more specific here of which type of virtual
interrupts?
>  spinlock? timer?
all of them: spinlock, timer, resched, callfunc, ...
> > because Xen event chip silently fails the set_affinity ops, and
> > logically IRQF_PER_CPU should be recognized here.
> 
> OK, so what if the set_affinity ops was implemented?
it was implemented: (drivers/xen/event.c, rebind_irq_to_cpu)
        /*
         * If this fails, it usually just indicates that we''re dealing
with a
         * virq or IPI channel, which don''t actually need to be
rebound. Ignore
         * it, but don''t do the xenlinux-level rebind in that case.
         */
        if (HYPERVISOR_event_channel_op(EVTCHNOP_bind_vcpu, &bind_vcpu)
>= 0)
                bind_evtchn_to_cpu(evtchn, tcpu);
Hypervisor doesn''t allow to change affinity for virq and ipi.

Thanks,
Kevin
> >
> > Signed-off-by: Fengzhe Zhang <fengzhe.zhang@intel.com>
> > Signed-off-by: Kevin Tian <kevin.tian@intel.com>
> > CC: Thomas Gleixner <tglx@linutronix.de>
> > CC: Ingo Molnar <mingo@redhat.com>
> > CC: H. Peter Anvin <hpa@zytor.com>
> > CC: Ian Campbell <Ian.Campbell@citrix.com>
> > CC: Jan Beulich <JBeulich@novell.com>
> >
> > --- linux-2.6.39-rc6.orig/arch/x86/kernel/irq.c	2011-05-04
> 10:59:13.000000000 +0800
> > +++ linux-2.6.39-rc6/arch/x86/kernel/irq.c	2011-05-06
09:20:25.563963000
> +0800
> > @@ -249,7 +250,7 @@ void fixup_irqs(void)
> >
> >  		data = irq_desc_get_irq_data(desc);
> >  		affinity = data->affinity;
> > -		if (!irq_has_action(irq) ||
> > +		if (!irq_has_action(irq) || irqd_is_per_cpu(data) ||
> >  		    cpumask_subset(affinity, cpu_online_mask)) {
> >  			raw_spin_unlock(&desc->lock);
> >  			continue;
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> > linux-kernel" in the body of a message to
majordomo@vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Fri, 6 May 2011, Konrad Rzeszutek Wilk wrote:
> On Fri, May 06, 2011 at 02:43:36PM +0800, Tian, Kevin wrote:
> > because Xen event chip silently fails the set_affinity ops, and
> > logically IRQF_PER_CPU should be recognized here.
> 
> OK, so what if the set_affinity ops was implemented?
An interrupt chip which has a set_affinity op should not mark
something per cpu, which implies that the irq CANNOT be moved.

Thanks,

	tglx

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

>>> On 09.05.11 at 14:39, Thomas Gleixner <tglx@linutronix.de>
wrote:
> On Fri, 6 May 2011, Konrad Rzeszutek Wilk wrote:
>> On Fri, May 06, 2011 at 02:43:36PM +0800, Tian, Kevin wrote:
>> > because Xen event chip silently fails the set_affinity ops, and
>> > logically IRQF_PER_CPU should be recognized here.
>> 
>> OK, so what if the set_affinity ops was implemented?
> 
> An interrupt chip which has a set_affinity op should not mark
> something per cpu, which implies that the irq CANNOT be moved.
Why shouldn''t it be possible o use the same "chip" for both
per-CPU
and "normal" IRQs?

Jan


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> From: Thomas Gleixner [mailto:tglx@linutronix.de]
> Sent: Monday, May 09, 2011 8:39 PM
> 
> On Fri, 6 May 2011, Konrad Rzeszutek Wilk wrote:
> > On Fri, May 06, 2011 at 02:43:36PM +0800, Tian, Kevin wrote:
> > > because Xen event chip silently fails the set_affinity ops, and
> > > logically IRQF_PER_CPU should be recognized here.
> >
> > OK, so what if the set_affinity ops was implemented?
> 
> An interrupt chip which has a set_affinity op should not mark something per
> cpu, which implies that the irq CANNOT be moved.
> 
If this is the hard requirement, why not throwing out an error when a chip
is registered?

Thanks
Kevin

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel