Linux Virtualization - Aug 2014 - [RFC PATCH 00/11] Refactor MSI to support Non-PCI device

Hi Yijing
> -----Original Message-----
> From: Yijing Wang [mailto:wangyijing at huawei.com]
> Sent: Saturday, July 26, 2014 8:39 AM
> To: linux-kernel at vger.kernel.org
> Cc: Xinwei Hu; Wuyun; Bjorn Helgaas; linux-pci at vger.kernel.org;
> Paul.Mundt at huawei.com; James E.J. Bottomley; Marc Zyngier; linux-arm-
> kernel at lists.infradead.org; Russell King; linux-arch at vger.kernel.org;
> Basu Arnab-B45036; virtualization at lists.linux-foundation.org; Hanjun
Guo;
> Yijing Wang
> Subject: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
> 
> Hi all,
> 	The series is a draft of generic MSI driver that supports PCI and
> Non-PCI device which have MSI capability. If you're not interested it,
> sorry for the noise.
> 
Thanks for sending out these patches, I have some (very basic) questions.
> The series is based on Linux-3.16-rc1.
> 
> MSI was introduced in PCI Spec 2.2. Currently, kernel MSI driver codes
> are bonding with PCI device. Because MSI has a lot advantages in design.
> More and more non-PCI devices want to use MSI as their default interrupt.
> The existing MSI device include HPET. HPET driver provide its own MSI
> code to initialize and process MSI interrupts. In the latest GIC v3 spec,
> legacy device can deliver MSI by the help of a relay device named
> consolidator.
> Consolidator can translate the legacy interrupts connected to it to
> MSI/MSI-X. And new non-PCI device will be designed to support MSI in
> future. So make the MSI driver code be generic will help the non-PCI
> device use MSI more simply.
As per my understanding the GICv3 provides a service that will convert writes to
a specified address to IRQs delivered to the core and as you mention above MSIs
are part of the PCI spec. So I can see a strong case for non-PCI devices to want
MSI like functionality without being fully compliant with the requirements of
the MSI spec.

My question is do we necessarily want to rework so much of the PCI-MSI layer to
support non PCI devices? Or will it be sufficient to create a framework to allow
non PCI devices to hook up with a device that can convert their writes to an IRQ
to the core.

As I understand it, the msi_chip is (almost) such a framework. The only problem
being that it makes some PCI specific assumptions (such as PCI specific writes
from within msi_chip functions). Won't it be sufficient to make the msi_chip
framework generic enough to be used by non-PCI devices and let each bus/device
manage any additional requirements (such as configuration flow, bit definitions
etc) that it places on message based interrupts?

Thanks
Arnab

Hi Yijing
> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-owner at
vger.kernel.org]
> On Behalf Of Yijing Wang
> Sent: Monday, August 04, 2014 8:34 AM
> To: Basu Arnab-B45036
> Cc: Xinwei Hu; Wuyun; Bjorn Helgaas; linux-pci at vger.kernel.org;
> Paul.Mundt at huawei.com; James E.J. Bottomley; Marc Zyngier; linux-arm-
> kernel at lists.infradead.org; Russell King; linux-arch at vger.kernel.org;
> virtualization at lists.linux-foundation.org; Hanjun Guo; linux-
> kernel at vger.kernel.org
> Subject: Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
> 
> >> MSI was introduced in PCI Spec 2.2. Currently, kernel MSI driver
> >> codes are bonding with PCI device. Because MSI has a lot
advantages in
> design.
> >> More and more non-PCI devices want to use MSI as their default
interrupt.
> >> The existing MSI device include HPET. HPET driver provide its own
MSI
> >> code to initialize and process MSI interrupts. In the latest GIC
v3
> >> spec, legacy device can deliver MSI by the help of a relay device
> >> named consolidator.
> >> Consolidator can translate the legacy interrupts connected to it
to
> >> MSI/MSI-X. And new non-PCI device will be designed to support MSI
in
> >> future. So make the MSI driver code be generic will help the
non-PCI
> >> device use MSI more simply.
> >
> > As per my understanding the GICv3 provides a service that will convert
writes
> to a specified address to IRQs delivered to the core and as you mention
above
> MSIs are part of the PCI spec. So I can see a strong case for non-PCI
devices to
> want MSI like functionality without being fully compliant with the
requirements
> of the MSI spec.
> 
> In GICv3, MBI is named for the service, but there is no more detailed
> information about it, only we can know is MBI is analogous to MSI, MBI
devices
> must have address/data registers, but other registers like enable/mask/ctrl
are
> not mandatory requirement.
> I don't know whether the MBI spec will be release, but anyway I think
MSI
> refactoring is make sense, there are some existing Non-PCI MSI device like
hpet,
> dmar.
> For simplicity, let name MSI and MBI to MSI temporarily.
> >
> > My question is do we necessarily want to rework so much of the PCI-MSI
layer
> to support non PCI devices? Or will it be sufficient to create a framework
to
> allow non PCI devices to hook up with a device that can convert their
writes to
> an IRQ to the core.
> >
> > As I understand it, the msi_chip is (almost) such a framework. The
only
> problem being that it makes some PCI specific assumptions (such as PCI
specific
> writes from within msi_chip functions). Won't it be sufficient to make
the
> msi_chip framework generic enough to be used by non-PCI devices and let
each
> bus/device manage any additional requirements (such as configuration flow,
bit
> definitions etc) that it places on message based interrupts?
> 
> msi_chip framework is important to support that, but I think maybe it's
not
> enough, msi_chip is only responsible for IRQ allocation, teardown, etc..
> 
> The key difference between PCI device and Non-PCI MSI is the interfaces to
> access hardware MSI registers.
> for instance, currently, msi_chip->setup_irq() to setup MSI irq and
configure
> the MSI address/data registers, so we need to provide device specific
> write_msi_msg() interface, then when we call msi_chip->setup_irq(), the
device
> MSI registers can be configured appropriately.
What if we can register/override the setup_irq() from bus-driver (not sure, but
may be device-driver itself). Example PCI bus-driver will provide setup_irq()
(or the part of setup_irq which set address and data in h/w) by PCI bus, which
configure address/data in h/w as per PCI standard.

We in Freescale will be using MSI for the devices behind a new-bus (which is not
PCI based), We have a separate bus driver for same. And this new bus driver
register/provide its own address/data write function which is based on that
specific bus protocol.

Thanks
-Bharat
> 
> My patchset is just a RFC draft, I will update it later, all we want to do
is
> make kernel support Non-PCI MSI devices.
> 
> Thanks!
> Yijing.
> 
> 
> >
> > Thanks
> > Arnab
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> > linux-kernel" in the body of a message to majordomo at
vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> >
> > .
> >
> 
> 
> --
> Thanks!
> Yijing
> 
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-pci" in the body
> of a message to majordomo at vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html