On Mon, Jul 30, 2018 at 01:28:03PM +0300, Michael S. Tsirkin wrote:> Let me reply to the "crappy" part first: > So virtio devices can run on another CPU or on a PCI bus. Configuration > can happen over mupltiple transports. There is a discovery protocol to > figure out where it is. It has some warts but any real system has warts. > > So IMHO virtio running on another CPU isn't "legacy virtual crappy > virtio". virtio devices that actually sit on a PCI bus aren't "sane" > simply because the DMA is more convoluted on some architectures.All of what you said would be true if virtio didn't claim to be a PCI device. Once it claims to be a PCI device and we also see real hardware written to the interface I stand to all what I said above.> With this out of my system: > I agree these approaches are hacky. I think it is generally better to > have virtio feature negotiation tell you whether device runs on a CPU or > not rather than rely on platform specific ways for this. To this end > there was a recent proposal to rename VIRTIO_F_IO_BARRIER to > VIRTIO_F_REAL_DEVICE. It got stuck since "real" sounds vague to people, > e.g. what if it's a VF - is that real or not? But I can see something > like e.g. VIRTIO_F_PLATFORM_DMA gaining support. > > We would then rename virtio_has_iommu_quirk to virtio_has_dma_quirk > and test VIRTIO_F_PLATFORM_DMA in addition to the IOMMU thing.I don't really care about the exact naming, and indeed a device that sets the flag doesn't have to be a 'real' device - it just has to act like one. I explained all the issues that this means (at least relating to DMA) in one of the previous threads. The important bit is that we can specify exact behavior for both devices that sets the "I'm real!" flag and that ones that don't exactly in the spec. And that very much excludes arch-specific (or Xen-specific) overrides.
On Mon, Jul 30, 2018 at 04:18:02AM -0700, Christoph Hellwig wrote:> On Mon, Jul 30, 2018 at 01:28:03PM +0300, Michael S. Tsirkin wrote: > > Let me reply to the "crappy" part first: > > So virtio devices can run on another CPU or on a PCI bus. Configuration > > can happen over mupltiple transports. There is a discovery protocol to > > figure out where it is. It has some warts but any real system has warts. > > > > So IMHO virtio running on another CPU isn't "legacy virtual crappy > > virtio". virtio devices that actually sit on a PCI bus aren't "sane" > > simply because the DMA is more convoluted on some architectures. > > All of what you said would be true if virtio didn't claim to be > a PCI device.There's nothing virtio claims to be. It's a PV device that uses PCI for its configuration. Configuration is enumerated on the virtual PCI bus. That part of the interface is emulated PCI. Data path is through a PV device enumerated on the virtio bus.> Once it claims to be a PCI device and we also see > real hardware written to the interface I stand to all what I said > above.Real hardware would reuse parts of the interface but by necessity it needs to behave slightly differently on some platforms. However for some platforms (such as x86) a PV virtio driver will by luck work with a PCI device backend without changes. As these platforms and drivers are widely deployed, some people will deploy hardware like that. Should be a non issue as by definition it's transparent to guests.> > With this out of my system: > > I agree these approaches are hacky. I think it is generally better to > > have virtio feature negotiation tell you whether device runs on a CPU or > > not rather than rely on platform specific ways for this. To this end > > there was a recent proposal to rename VIRTIO_F_IO_BARRIER to > > VIRTIO_F_REAL_DEVICE. It got stuck since "real" sounds vague to people, > > e.g. what if it's a VF - is that real or not? But I can see something > > like e.g. VIRTIO_F_PLATFORM_DMA gaining support. > > > > We would then rename virtio_has_iommu_quirk to virtio_has_dma_quirk > > and test VIRTIO_F_PLATFORM_DMA in addition to the IOMMU thing. > > I don't really care about the exact naming, and indeed a device that > sets the flag doesn't have to be a 'real' device - it just has to act > like one. I explained all the issues that this means (at least relating > to DMA) in one of the previous threads.I believe you refer to this: https://lkml.org/lkml/2018/6/7/15 that was a very helpful list outlining the problems we need to solve, thanks a lot for that!> The important bit is that we can specify exact behavior for both > devices that sets the "I'm real!" flag and that ones that don't exactly > in the spec.I would very much like that, yes.> And that very much excludes arch-specific (or > Xen-specific) overrides.We already committed to a xen specific hack but generally I prefer devices that describe how they work instead of platforms magically guessing, yes. However the question people raise is that DMA API is already full of arch-specific tricks the likes of which are outlined in your post linked above. How is this one much worse? -- MST
On Mon, Jul 30, 2018 at 04:26:32PM +0300, Michael S. Tsirkin wrote:> Real hardware would reuse parts of the interface but by necessity it > needs to behave slightly differently on some platforms. However for > some platforms (such as x86) a PV virtio driver will by luck work with a > PCI device backend without changes. As these platforms and drivers are > widely deployed, some people will deploy hardware like that. Should be > a non issue as by definition it's transparent to guests.On some x86. As soon as you have an iommu or strange PCI root ports things are going to start breaking apart.> > And that very much excludes arch-specific (or > > Xen-specific) overrides. > > We already committed to a xen specific hack but generally I prefer > devices that describe how they work instead of platforms magically > guessing, yes.For legacy reasons I guess we'll have to keep it, but we really need to avoid adding more junk than this.> However the question people raise is that DMA API is already full of > arch-specific tricks the likes of which are outlined in your post linked > above. How is this one much worse?None of these warts is visible to the driver, they are all handled in the architecture (possibly on a per-bus basis). So for virtio we really need to decide if it has one set of behavior as specified in the virtio spec, or if it behaves exactly as if it was on a PCI bus, or in fact probably both as you lined up. But no magic arch specific behavior inbetween.