This patch changes the pirq to be per-domain in xen tree. Signed-off-by: Jiang Yunhong <yunhong.jiang@intel.com> Signed-off-by: Shan Haitao <haitao.shan@intel.com> Best Regards Shan Haitao _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 27/3/08 07:00, "Shan, Haitao" <haitao.shan@intel.com> wrote:> This patch changes the pirq to be per-domain in xen tree. > > Signed-off-by: Jiang Yunhong <yunhong.jiang@intel.com> > Signed-off-by: Shan Haitao <haitao.shan@intel.com>I''m not sure why this would be a prerequisite for the rest of the MSI support. Still I have a feeling that I may have asked for this a long time ago on a previous iteration of this patchset... :-) It looks pretty sensible, but PHYSDEVOP_map_irq shouldn''t take an IRQ_TYPE_IRQ -- ''IRQ'' is a meaningless thing architecturally-speaking, and I think instead we should allow to specify a ''GSI'' or an ''ISA IRQ''. As for mapping pirq to MSI, I''m not sure about making real interrupt vectors visible to the guest. But maybe that''s unavoidable. The way I could imagine this working is to teach Xen a bit about accessing PCI space, and then have the guest relinquish control of critical MSI control fields in the config space to Xen. The guest would tell Xen where the fields are, and then Xen can freely configure the target APIC, mask, etc. Seems neater to me, but is this a nuts idea? -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] >Sent: 2008年3月28日 16:52 > >I''m not sure why this would be a prerequisite for the rest of the MSI >support. Still I have a feeling that I may have asked for this >a long time >ago on a previous iteration of this patchset... :-) It looks pretty >sensible, but PHYSDEVOP_map_irq shouldn''t take an IRQ_TYPE_IRQ >-- ''IRQ'' is a >meaningless thing architecturally-speaking, and I think >instead we should >allow to specify a ''GSI'' or an ''ISA IRQ''.I think the reverse. :-) Here IRQ is just a namespace which is allocatable and not bound to platform hard-wired logic. Each MSI just requires one IRQ placeholder to gear to evtchn core with the latter on top of IRQ name- space. However GSI or ISA IRQ more indicates platform attribute which doesn''t fit the purpose here, though GSI can be also tweaked in some version of Linux kernel.> >As for mapping pirq to MSI, I''m not sure about making real >interrupt vectors >visible to the guest. But maybe that''s unavoidable. The way I >could imagine >this working is to teach Xen a bit about accessing PCI space, >and then have >the guest relinquish control of critical MSI control fields in >the config >space to Xen. The guest would tell Xen where the fields are, >and then Xen >can freely configure the target APIC, mask, etc. Seems neater >to me, but is >this a nuts idea? >This should work, and may solve the issue Yunhong described in another mail by giving Xen ability to mask device directly upon spurious interrupts. And... seems like less change to Linux code? The only concern is how complex the interface may finally go, and in this case Xen still needs to sync PCI config space access for port I/O style. Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 28/3/08 09:23, "Tian, Kevin" <kevin.tian@intel.com> wrote:> I think the reverse. :-) Here IRQ is just a namespace which is allocatable > and not bound to platform hard-wired logic. Each MSI just requires one > IRQ placeholder to gear to evtchn core with the latter on top of IRQ name- > space. However GSI or ISA IRQ more indicates platform attribute which > doesn''t fit the purpose here, though GSI can be also tweaked in some > version of Linux kernel.I don''t understand you. Do you mean *PIRQ* is just a namespace which is allocatable? I fully agree with that. I was talking about the MAO_IRQ_TYPE_IRQ binding type -- here I believe ''IRQ'' does refer to a real platform resource, but ''IRQ'' is not a well-defined architectural namespace like ''GSI'' or ''ISA IRQ''. So the interface should be fixed imo.> This should work, and may solve the issue Yunhong described in > another mail by giving Xen ability to mask device directly upon > spurious interrupts. And... seems like less change to Linux code? > The only concern is how complex the interface may finally go, > and in this case Xen still needs to sync PCI config space access > for port I/O style.Yes, the synchronisation is pretty easy though. We just have to add a layer of emulation to PV guest accesses to 0xcf8/0xcfc. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
Keir Fraser <mailto:keir.fraser@eu.citrix.com> wrote:> On 27/3/08 07:00, "Shan, Haitao" <haitao.shan@intel.com> wrote: > >> This patch changes the pirq to be per-domain in xen tree. >> >> Signed-off-by: Jiang Yunhong <yunhong.jiang@intel.com> >> Signed-off-by: Shan Haitao <haitao.shan@intel.com> > > I''m not sure why this would be a prerequisite for the rest of the MSI > support. Still I have a feeling that I may have asked for this > a long time > ago on a previous iteration of this patchset... :-) It looks pretty > sensible, but PHYSDEVOP_map_irq shouldn''t take an IRQ_TYPE_IRQ --''IRQ'' is a> meaningless thing architecturally-speaking, and I think > instead we should > allow to specify a ''GSI'' or an ''ISA IRQ''. > > As for mapping pirq to MSI, I''m not sure about making real > interrupt vectors > visible to the guest. But maybe that''s unavoidable. The way I > could imagine > this working is to teach Xen a bit about accessing PCI space, > and then have > the guest relinquish control of critical MSI control fields in > the config > space to Xen. The guest would tell Xen where the fields are, > and then Xen > can freely configure the target APIC, mask, etc. Seems neater > to me, but is > this a nuts idea?DomainU (PV and hvm) should have no idea of vector. Do you think it will matter if domain0 have such idea? one thing missed here is, if domainU want to access the MSI config spafce, pci backend should return 0xff. Then it should be secure if domain0 can have idea of vector.> > -- Keir_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 28/3/08 09:37, "Jiang, Yunhong" <yunhong.jiang@intel.com> wrote:> DomainU (PV and hvm) should have no idea of vector. Do you think it > will matter if domain0 have such idea? > one thing missed here is, if domainU want to access the MSI config > spafce, pci backend should return 0xff. Then it should be secure if > domain0 can have idea of vector.No, it''s not a security risk for dom0 kernel to know about real vectors. It''s already part of the TCB. It''s just a question of which is the cleanest design. And letting Xen get some access to PCI config space (just a little -- not a lot -- and under direction of dom0 kernel) will let it properly mask MSIs, which would be a nicer and deadlock-free alternative to the ''ACK-NEW'' masking method. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] >Sent: 2008年3月28日 17:33 > >On 28/3/08 09:23, "Tian, Kevin" <kevin.tian@intel.com> wrote: > >> I think the reverse. :-) Here IRQ is just a namespace which >is allocatable >> and not bound to platform hard-wired logic. Each MSI just >requires one >> IRQ placeholder to gear to evtchn core with the latter on >top of IRQ name- >> space. However GSI or ISA IRQ more indicates platform attribute which >> doesn''t fit the purpose here, though GSI can be also tweaked in some >> version of Linux kernel. > >I don''t understand you. Do you mean *PIRQ* is just a namespace which is >allocatable? I fully agree with that. I was talking about the >MAO_IRQ_TYPE_IRQ binding type -- here I believe ''IRQ'' does >refer to a real >platform resource, but ''IRQ'' is not a well-defined >architectural namespace >like ''GSI'' or ''ISA IRQ''. So the interface should be fixed imo.Oh, I jumped in too early. Sorry and you''re right.> >> This should work, and may solve the issue Yunhong described in >> another mail by giving Xen ability to mask device directly upon >> spurious interrupts. And... seems like less change to Linux code? >> The only concern is how complex the interface may finally go, >> and in this case Xen still needs to sync PCI config space access >> for port I/O style. > >Yes, the synchronisation is pretty easy though. We just have >to add a layer >of emulation to PV guest accesses to 0xcf8/0xcfc.Yes, a small tricky however is, the owner vcpu may be scheduled out between 0xcf8 and 0xcfc access, when another pcpu is trying to access PCI config space like masking MSI within do_IRQ... Maybe Xen need to emulate 0xcf8/0xcfc pair without return to guest in the middle. :-) Thanks, Kevin _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 28/3/08 09:49, "Tian, Kevin" <kevin.tian@intel.com> wrote:>> Yes, the synchronisation is pretty easy though. We just have >> to add a layer >> of emulation to PV guest accesses to 0xcf8/0xcfc. > > Yes, a small tricky however is, the owner vcpu may be scheduled > out between 0xcf8 and 0xcfc access, when another pcpu is trying > to access PCI config space like masking MSI within do_IRQ... > Maybe Xen need to emulate 0xcf8/0xcfc pair without return to > guest in the middle. :-)Not at all. You track the guest''s 0xcf8 value in the vcpu structure, and issue the physical 0xcf8/0xcfc access as a pair, under a spinlock, whenever the guest writes 0xcfc. -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
>> From: Keir Fraser [mailto:keir.fraser@eu.citrix.com] >> Sent: 2008年3月28日 17:33 >> >> On 28/3/08 09:23, "Tian, Kevin" <kevin.tian@intel.com> wrote: >> >>> I think the reverse. :-) Here IRQ is just a namespace which is >>> allocatable and not bound to platform hard-wired logic. Each MSI >>> just requires one IRQ placeholder to gear to evtchn core with the >>> latter on top of IRQ name- space. However GSI or ISA IRQ more >>> indicates platform attribute which doesn''t fit the purpose here, >>> though GSI can be also tweaked in some version of Linux kernel. >> >> I don''t understand you. Do you mean *PIRQ* is just a namespace which >> is allocatable? I fully agree with that. I was talking about the >> MAO_IRQ_TYPE_IRQ binding type -- here I believe ''IRQ'' does >> refer to a real >> platform resource, but ''IRQ'' is not a well-defined >> architectural namespace >> like ''GSI'' or ''ISA IRQ''. So the interface should be fixed imo. > > Oh, I jumped in too early. Sorry and you''re right.I will change that name.> >> >>> This should work, and may solve the issue Yunhong described in >>> another mail by giving Xen ability to mask device directly upon >>> spurious interrupts. And... seems like less change to Linux code? >>> The only concern is how complex the interface may finally go, >>> and in this case Xen still needs to sync PCI config space access >>> for port I/O style. >> >> Yes, the synchronisation is pretty easy though. We just have >> to add a layer >> of emulation to PV guest accesses to 0xcf8/0xcfc. > > Yes, a small tricky however is, the owner vcpu may be scheduled > out between 0xcf8 and 0xcfc access, when another pcpu is trying > to access PCI config space like masking MSI within do_IRQ... > Maybe Xen need to emulate 0xcf8/0xcfc pair without return to > guest in the middle. :-)Can we add hypercall to combine the two-time access to one?> > Thanks, > Kevin_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
On 28/3/08 10:03, "Shan, Haitao" <haitao.shan@intel.com> wrote:>>> like ''GSI'' or ''ISA IRQ''. So the interface should be fixed imo. >> >> Oh, I jumped in too early. Sorry and you''re right. > I will change that name.I think you should add both. I believe there''s not necessarily a guaranteed mapping of every ISA IRQ into GSI namespace. I don''t strongly insist on this now if you know what you''re doing and only want to include MAP_IRQ_TYPE_GSI. The names should be MAP_PIRQ_* and PHYSDEVOP_[un]map_pirq, by the way. Another comment I have is that it doesn''t look like the per-domain-pirq patch is compatible with old dom0 kernels if msi_irq_enable is specified on the Xen command line. That''s a bit sucky, especially since we''re sure to want to make msi_irq_enable the default after these patches have had some testing. I''m not sure what to suggest as the better way to achieve backward compatibility. I''ll have a think about it... -- Keir _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
OK. I will update the patch and send it out again. Best Regards Haitao Shan Keir Fraser wrote:> On 28/3/08 09:49, "Tian, Kevin" <kevin.tian@intel.com> wrote: > >>> Yes, the synchronisation is pretty easy though. We just have to add >>> a layer of emulation to PV guest accesses to 0xcf8/0xcfc. >> >> Yes, a small tricky however is, the owner vcpu may be scheduled >> out between 0xcf8 and 0xcfc access, when another pcpu is trying >> to access PCI config space like masking MSI within do_IRQ... >> Maybe Xen need to emulate 0xcf8/0xcfc pair without return to >> guest in the middle. :-) > > Not at all. You track the guest''s 0xcf8 value in the vcpu structure, > and issue the physical 0xcf8/0xcfc access as a pair, under a > spinlock, whenever the guest writes 0xcfc. > > -- Keir_______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel
[Keir Fraser]> On 28/3/08 09:37, "Jiang, Yunhong" <yunhong.jiang@intel.com> wrote:>> DomainU (PV and hvm) should have no idea of vector. Do you think it >> will matter if domain0 have such idea? one thing missed here is, >> if domainU want to access the MSI config spafce, pci backend should >> return 0xff. Then it should be secure if domain0 can have idea of >> vector.> No, it''s not a security risk for dom0 kernel to know about real > vectors. It''s already part of the TCB.> It''s just a question of which is the cleanest design. And letting > Xen get some access to PCI config space (just a little -- not a lot > -- and under direction of dom0 kernel) will let it properly mask > MSIs, which would be a nicer and deadlock-free alternative to the > ''ACK-NEW'' masking method.With the introduction of VT-d interrupt remapping you might want to relinquish some more control of the PCI config space to Xen anyway. More precisely, the interrupt address and message data written into the MSI capability structure or MSI-X Table will no longer be the destination APIC id, interrupt type, vector, etc., for delivering the interrupt. Instead, the information goes into the interrupt remapping table, and a special remappable message type goes into the capabilty structure/MSI-X table. This already happens for IOAPIC entries. The alternative is to put the interrupt remapping table under the control of dom0. eSk _______________________________________________ Xen-devel mailing list Xen-devel@lists.xensource.com http://lists.xensource.com/xen-devel