Linux Virtualization - Aug 2014 - [RFC PATCH 00/11] Refactor MSI to support Non-PCI device

Hi Yijing
> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-owner at
vger.kernel.org]
> On Behalf Of Yijing Wang
> Sent: Monday, August 04, 2014 8:34 AM
> To: Basu Arnab-B45036
> Cc: Xinwei Hu; Wuyun; Bjorn Helgaas; linux-pci at vger.kernel.org;
> Paul.Mundt at huawei.com; James E.J. Bottomley; Marc Zyngier; linux-arm-
> kernel at lists.infradead.org; Russell King; linux-arch at vger.kernel.org;
> virtualization at lists.linux-foundation.org; Hanjun Guo; linux-
> kernel at vger.kernel.org
> Subject: Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
> 
> >> MSI was introduced in PCI Spec 2.2. Currently, kernel MSI driver
> >> codes are bonding with PCI device. Because MSI has a lot
advantages in
> design.
> >> More and more non-PCI devices want to use MSI as their default
interrupt.
> >> The existing MSI device include HPET. HPET driver provide its own
MSI
> >> code to initialize and process MSI interrupts. In the latest GIC
v3
> >> spec, legacy device can deliver MSI by the help of a relay device
> >> named consolidator.
> >> Consolidator can translate the legacy interrupts connected to it
to
> >> MSI/MSI-X. And new non-PCI device will be designed to support MSI
in
> >> future. So make the MSI driver code be generic will help the
non-PCI
> >> device use MSI more simply.
> >
> > As per my understanding the GICv3 provides a service that will convert
writes
> to a specified address to IRQs delivered to the core and as you mention
above
> MSIs are part of the PCI spec. So I can see a strong case for non-PCI
devices to
> want MSI like functionality without being fully compliant with the
requirements
> of the MSI spec.
> 
> In GICv3, MBI is named for the service, but there is no more detailed
> information about it, only we can know is MBI is analogous to MSI, MBI
devices
> must have address/data registers, but other registers like enable/mask/ctrl
are
> not mandatory requirement.
> I don't know whether the MBI spec will be release, but anyway I think
MSI
> refactoring is make sense, there are some existing Non-PCI MSI device like
hpet,
> dmar.
> For simplicity, let name MSI and MBI to MSI temporarily.
> >
> > My question is do we necessarily want to rework so much of the PCI-MSI
layer
> to support non PCI devices? Or will it be sufficient to create a framework
to
> allow non PCI devices to hook up with a device that can convert their
writes to
> an IRQ to the core.
> >
> > As I understand it, the msi_chip is (almost) such a framework. The
only
> problem being that it makes some PCI specific assumptions (such as PCI
specific
> writes from within msi_chip functions). Won't it be sufficient to make
the
> msi_chip framework generic enough to be used by non-PCI devices and let
each
> bus/device manage any additional requirements (such as configuration flow,
bit
> definitions etc) that it places on message based interrupts?
> 
> msi_chip framework is important to support that, but I think maybe it's
not
> enough, msi_chip is only responsible for IRQ allocation, teardown, etc..
> 
> The key difference between PCI device and Non-PCI MSI is the interfaces to
> access hardware MSI registers.
> for instance, currently, msi_chip->setup_irq() to setup MSI irq and
configure
> the MSI address/data registers, so we need to provide device specific
> write_msi_msg() interface, then when we call msi_chip->setup_irq(), the
device
> MSI registers can be configured appropriately.
What if we can register/override the setup_irq() from bus-driver (not sure, but
may be device-driver itself). Example PCI bus-driver will provide setup_irq()
(or the part of setup_irq which set address and data in h/w) by PCI bus, which
configure address/data in h/w as per PCI standard.

We in Freescale will be using MSI for the devices behind a new-bus (which is not
PCI based), We have a separate bus driver for same. And this new bus driver
register/provide its own address/data write function which is based on that
specific bus protocol.

Thanks
-Bharat
> 
> My patchset is just a RFC draft, I will update it later, all we want to do
is
> make kernel support Non-PCI MSI devices.
> 
> Thanks!
> Yijing.
> 
> 
> >
> > Thanks
> > Arnab
> > --
> > To unsubscribe from this list: send the line "unsubscribe
> > linux-kernel" in the body of a message to majordomo at
vger.kernel.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html
> > Please read the FAQ at  http://www.tux.org/lkml/
> >
> > .
> >
> 
> 
> --
> Thanks!
> Yijing
> 
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-pci" in the body
> of a message to majordomo at vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html

>> The key difference between PCI device and Non-PCI MSI is the interfaces
to
>> access hardware MSI registers.
>> for instance, currently, msi_chip->setup_irq() to setup MSI irq and
configure
>> the MSI address/data registers, so we need to provide device specific
>> write_msi_msg() interface, then when we call msi_chip->setup_irq(),
the device
>> MSI registers can be configured appropriately.
> 
> What if we can register/override the setup_irq() from bus-driver (not sure,
but may be device-driver itself). Example PCI bus-driver will provide
setup_irq() (or the part of setup_irq which set address and data in h/w) by PCI
bus, which configure address/data in h/w as per PCI standard.
> 
> We in Freescale will be using MSI for the devices behind a new-bus (which
is not PCI based), We have a separate bus driver for same. And this new bus
driver register/provide its own address/data write function which is based on
that specific bus protocol.
Hi Bharat, I'm glad to know your MSI device working mode.
Provide the private MSI setup functions in bus-driver layer can't apply to
all Non-PCI MSI devices,
because we can not guarantee Non-PCI MSI devices are always on a bus. The
existing HPET, DMAR device both
have no bus bind. I'm working on a new MSI setup framework, as you mentioned
before, in device-driver model.

I abstracted a new virtual device (called struct msi_dev), this msi_dev will
manage all MSI info, and a new bus
named msi_bus, also introduced a new driver msi_driver, msi_bus is responsible
for binding msi_dev and msi_driver.
All MSI devices will be classified into different MSI device types, like
MSI_TYPE_PCI, MSI_TYPE_HPET, MSI_TYPE_DMAR, etc..

Each MSI type device should provide a private struct msi_driver. msi_driver
should contain the type specific
MSI ops functions to help setup and enable MSI device, request MSI irq.

I almost finish the first draft, and will post out next week in plan :)


Thanks!
Yijing.
> 
> Thanks
> -Bharat
> 
>>
>> My patchset is just a RFC draft, I will update it later, all we want to
do is
>> make kernel support Non-PCI MSI devices.
>>
>> Thanks!
>> Yijing.
>>
>>
>>>
>>> Thanks
>>> Arnab
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe
>>> linux-kernel" in the body of a message to majordomo at
vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>> Please read the FAQ at  http://www.tux.org/lkml/
>>>
>>> .
>>>
>>
>>
>> --
>> Thanks!
>> Yijing
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe
linux-pci" in the body
>> of a message to majordomo at vger.kernel.org More majordomo info at
>> http://vger.kernel.org/majordomo-info.html
> 
> .
> 

-- 
Thanks!
Yijing

> -----Original Message-----
> From: linux-pci-owner at vger.kernel.org [mailto:linux-pci-owner at
vger.kernel.org]
> On Behalf Of Yijing Wang
> Sent: Wednesday, August 20, 2014 11:59 AM
> To: Bhushan Bharat-R65777; Basu Arnab-B45036
> Cc: Xinwei Hu; Wuyun; Bjorn Helgaas; linux-pci at vger.kernel.org;
> Paul.Mundt at huawei.com; James E.J. Bottomley; Marc Zyngier; linux-arm-
> kernel at lists.infradead.org; Russell King; linux-arch at vger.kernel.org;
> virtualization at lists.linux-foundation.org; Hanjun Guo; linux-
> kernel at vger.kernel.org
> Subject: Re: [RFC PATCH 00/11] Refactor MSI to support Non-PCI device
> 
> >> The key difference between PCI device and Non-PCI MSI is the
> >> interfaces to access hardware MSI registers.
> >> for instance, currently, msi_chip->setup_irq() to setup MSI irq
and
> >> configure the MSI address/data registers, so we need to provide
> >> device specific
> >> write_msi_msg() interface, then when we call
msi_chip->setup_irq(),
> >> the device MSI registers can be configured appropriately.
> >
> > What if we can register/override the setup_irq() from bus-driver (not
sure,
> but may be device-driver itself). Example PCI bus-driver will provide
> setup_irq() (or the part of setup_irq which set address and data in h/w) by
PCI
> bus, which configure address/data in h/w as per PCI standard.
> >
> > We in Freescale will be using MSI for the devices behind a new-bus
(which is
> not PCI based), We have a separate bus driver for same. And this new bus
driver
> register/provide its own address/data write function which is based on that
> specific bus protocol.
> 
> Hi Bharat, I'm glad to know your MSI device working mode.
> Provide the private MSI setup functions in bus-driver layer can't apply
to all
> Non-PCI MSI devices, because we can not guarantee Non-PCI MSI devices are
always
> on a bus. The existing HPET, DMAR device both have no bus bind.
Yes, that's why I was not sure of bus-driver or device-driver model.
> I'm working on a
> new MSI setup framework, as you mentioned before, in device-driver model.
> 
> I abstracted a new virtual device (called struct msi_dev), this msi_dev
will
> manage all MSI info,
Will this "struct msi_dev" will be part of "struct device"?
> and a new bus named msi_bus, also introduced a new driver
> msi_driver, msi_bus is responsible for binding msi_dev and msi_driver.
> All MSI devices will be classified into different MSI device types, like
> MSI_TYPE_PCI, MSI_TYPE_HPET, MSI_TYPE_DMAR, etc..
> 
> Each MSI type device should provide a private struct msi_driver. msi_driver
> should contain the type specific MSI ops functions to help setup and enable
MSI
> device, request MSI irq.
> 
> I almost finish the first draft, and will post out next week in plan :)
Will be looking forward to next version.

Thanks
-Bharat
> 
> 
> Thanks!
> Yijing.
> 
> >
> > Thanks
> > -Bharat
> >
> >>
> >> My patchset is just a RFC draft, I will update it later, all we
want
> >> to do is make kernel support Non-PCI MSI devices.
> >>
> >> Thanks!
> >> Yijing.
> >>
> >>
> >>>
> >>> Thanks
> >>> Arnab
> >>> --
> >>> To unsubscribe from this list: send the line "unsubscribe
> >>> linux-kernel" in the body of a message to majordomo at
vger.kernel.org
> >>> More majordomo info at 
http://vger.kernel.org/majordomo-info.html
> >>> Please read the FAQ at  http://www.tux.org/lkml/
> >>>
> >>> .
> >>>
> >>
> >>
> >> --
> >> Thanks!
> >> Yijing
> >>
> >> --
> >> To unsubscribe from this list: send the line "unsubscribe
linux-pci"
> >> in the body of a message to majordomo at vger.kernel.org More
majordomo
> >> info at http://vger.kernel.org/majordomo-info.html
> >
> > .
> >
> 
> 
> --
> Thanks!
> Yijing
> 
> --
> To unsubscribe from this list: send the line "unsubscribe
linux-pci" in the body
> of a message to majordomo at vger.kernel.org More majordomo info at
> http://vger.kernel.org/majordomo-info.html