Benjamin Herrenschmidt
2018-Aug-02 16:01 UTC
[RFC 0/4] Virtio uses DMA API for all devices
On Thu, 2018-08-02 at 18:41 +0300, Michael S. Tsirkin wrote:> > > I don't completely agree: > > > > 1 - VIRTIO_F_IOMMU_PLATFORM is a property of the "other side", ie qemu > > for example. It indicates that the peer bypasses the normal platform > > iommu. The platform code in the guest has no real way to know that this > > is the case, this is a specific "feature" of the qemu implementation. > > > > 2 - VIRTIO_F_PLATFORM_DMA (or whatever you want to call it), is a > > property of the guest platform itself (not qemu), there's no way the > > "peer" can advertize it via the virtio negociated flags. At least for > > us. I don't know for sure whether that would be workable for the ARM > > case. In our case, qemu has no idea at VM creation time that the VM > > will turn itself into a secure VM and thus will require bounce > > buffering for IOs (including virtio). > > > > So unless we have another hook for the arch code to set > > VIRTIO_F_PLATFORM_DMA on selected (or all) virtio devices from the > > guest itself, I don't see that as a way to deal with it. > > > > > The other issue is VIRTIO_F_IO_BARRIER > > > which is very vaguely defined, and which needs a better definition. > > > And last but not least we'll need some text explaining the challenges > > > of hardware devices - I think VIRTIO_F_PLATFORM_DMA + VIRTIO_F_IO_BARRIER > > > is what would basically cover them, but a good description including > > > an explanation of why these matter. > > > > Ben. > > > > So is it true that from qemu point of view there is nothing special > going on? You pass in a PA, host writes there.Yes, qemu doesn't see a different. It's the guest that will bounce the pages via a pool of "insecure" pages that qemu can access. Normal pages in a secure VM come from PAs that qemu cannot physically access. Cheers, Ben.
On Thu, Aug 02, 2018 at 11:01:26AM -0500, Benjamin Herrenschmidt wrote:> On Thu, 2018-08-02 at 18:41 +0300, Michael S. Tsirkin wrote: > > > > > I don't completely agree: > > > > > > 1 - VIRTIO_F_IOMMU_PLATFORM is a property of the "other side", ie qemu > > > for example. It indicates that the peer bypasses the normal platform > > > iommu. The platform code in the guest has no real way to know that this > > > is the case, this is a specific "feature" of the qemu implementation. > > > > > > 2 - VIRTIO_F_PLATFORM_DMA (or whatever you want to call it), is a > > > property of the guest platform itself (not qemu), there's no way the > > > "peer" can advertize it via the virtio negociated flags. At least for > > > us. I don't know for sure whether that would be workable for the ARM > > > case. In our case, qemu has no idea at VM creation time that the VM > > > will turn itself into a secure VM and thus will require bounce > > > buffering for IOs (including virtio). > > > > > > So unless we have another hook for the arch code to set > > > VIRTIO_F_PLATFORM_DMA on selected (or all) virtio devices from the > > > guest itself, I don't see that as a way to deal with it. > > > > > > > The other issue is VIRTIO_F_IO_BARRIER > > > > which is very vaguely defined, and which needs a better definition. > > > > And last but not least we'll need some text explaining the challenges > > > > of hardware devices - I think VIRTIO_F_PLATFORM_DMA + VIRTIO_F_IO_BARRIER > > > > is what would basically cover them, but a good description including > > > > an explanation of why these matter. > > > > > > Ben. > > > > > > > So is it true that from qemu point of view there is nothing special > > going on? You pass in a PA, host writes there. > > Yes, qemu doesn't see a different. It's the guest that will bounce the > pages via a pool of "insecure" pages that qemu can access. Normal pages > in a secure VM come from PAs that qemu cannot physically access. > > Cheers, > Ben. >I see. So yes, given that device does not know or care, using virtio features is an awkward fit. So let's say as a quick fix for you maybe we could generalize the xen_domain hack, instead of just checking xen_domain check some static branch. Then teach xen and others to enable that. OK but problem then becomes this: if you do this and virtio device appears behind a vIOMMU and it does not advertize the IOMMU flag, the code will try to use the vIOMMU mappings and fail. It does look like even with trick above, you need a special version of DMA ops that does just swiotlb but not any of the other things DMA API might do. Thoughts? -- MST
Benjamin Herrenschmidt
2018-Aug-02 17:53 UTC
[RFC 0/4] Virtio uses DMA API for all devices
On Thu, 2018-08-02 at 20:19 +0300, Michael S. Tsirkin wrote:> > I see. So yes, given that device does not know or care, using > virtio features is an awkward fit. > > So let's say as a quick fix for you maybe we could generalize the > xen_domain hack, instead of just checking xen_domain check some static > branch. Then teach xen and others to enable that.>> OK but problem then becomes this: if you do this and virtio device appears > behind a vIOMMU and it does not advertize the IOMMU flag, the > code will try to use the vIOMMU mappings and fail. > > It does look like even with trick above, you need a special version of > DMA ops that does just swiotlb but not any of the other things DMA API > might do. > > Thoughts?Yes, this is the purpose of Anshuman original patch (I haven't looked at the details of the patch in a while but that's what I told him to implement ;-) : - Make virtio always use DMA ops to simplify the code path (with a set of "transparent" ops for legacy) and - Provide an arch hook allowing us to "override" those "transparent" DMA ops with some custom ones that do the appropriate swiotlb gunk. Cheers, Ben.