Jason Wang
2023-Aug-15 02:25 UTC
[PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit
On Tue, Aug 15, 2023 at 9:45?AM Si-Wei Liu <si-wei.liu at oracle.com> wrote:> > Signed-off-by: Si-Wei Liu <si-wei.liu at oracle.com> > --- > drivers/vhost/vdpa.c | 16 +++++++++++++++- > include/uapi/linux/vhost_types.h | 2 ++ > 2 files changed, 17 insertions(+), 1 deletion(-) > > diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c > index 62b0a01..75092a7 100644 > --- a/drivers/vhost/vdpa.c > +++ b/drivers/vhost/vdpa.c > @@ -406,6 +406,14 @@ static bool vhost_vdpa_can_resume(const struct vhost_vdpa *v) > return ops->resume; > } > > +static bool vhost_vdpa_has_persistent_map(const struct vhost_vdpa *v) > +{ > + struct vdpa_device *vdpa = v->vdpa; > + const struct vdpa_config_ops *ops = vdpa->config; > + > + return (!ops->set_map && !ops->dma_map) || ops->reset_map;So this means the IOTLB/IOMMU mappings have already been decoupled from the vdpa reset. So it should have been noticed by the userspace. I guess we can just fix the simulator and mlx5 then we are fine? Thanks
Si-Wei Liu
2023-Aug-15 22:30 UTC
[PATCH RFC 4/4] vhost-vdpa: introduce IOTLB_PERSIST backend feature bit
On 8/14/2023 7:25 PM, Jason Wang wrote:> On Tue, Aug 15, 2023 at 9:45?AM Si-Wei Liu <si-wei.liu at oracle.com> wrote: >> Signed-off-by: Si-Wei Liu <si-wei.liu at oracle.com> >> --- >> drivers/vhost/vdpa.c | 16 +++++++++++++++- >> include/uapi/linux/vhost_types.h | 2 ++ >> 2 files changed, 17 insertions(+), 1 deletion(-) >> >> diff --git a/drivers/vhost/vdpa.c b/drivers/vhost/vdpa.c >> index 62b0a01..75092a7 100644 >> --- a/drivers/vhost/vdpa.c >> +++ b/drivers/vhost/vdpa.c >> @@ -406,6 +406,14 @@ static bool vhost_vdpa_can_resume(const struct vhost_vdpa *v) >> return ops->resume; >> } >> >> +static bool vhost_vdpa_has_persistent_map(const struct vhost_vdpa *v) >> +{ >> + struct vdpa_device *vdpa = v->vdpa; >> + const struct vdpa_config_ops *ops = vdpa->config; >> + >> + return (!ops->set_map && !ops->dma_map) || ops->reset_map; > So this means the IOTLB/IOMMU mappings have already been decoupled > from the vdpa reset.Not in the sense of API, it' been coupled since day one from the implementations of every on-chip IOMMU parent driver, namely mlx5_vdpa and vdpa_sim. Because of that, later on the (improper) support for virtio-vdpa, from commit 6f5312f80183 ("vdpa/mlx5: Add support for running with virtio_vdpa") and 6c3d329e6486 ("vdpa_sim: get rid of DMA ops") misused the .reset() op to realize 1:1 mapping, rendering strong coupling between device reset and reset of iotlb mappings. This series try to rectify that implementation deficiency, while keep userspace continuing to work with older kernel behavior.> So it should have been noticed by the userspace.Yes, userspace had noticed this no-chip IOMMU discrepancy since day one I suppose. Unfortunately there's already code in userspace with this assumption in mind that proactively tears down and sets up iotlb mapping around vdpa device reset...> I guess we can just fix the simulator and mlx5 then we are fine?Only IF we don't care about running new QEMU on older kernels with flawed on-chip iommu behavior around reset. But that's a big IF... Regards, -Siwei> > Thanks >