Linux Virtualization - Jan 2021 - [RFC 3/3] vfio: Share the KVM instance with Vdmabuf

On Tue, 19 Jan 2021 00:28:12 -0800
Vivek Kasireddy <vivek.kasireddy at intel.com> wrote:
> Getting a copy of the KVM instance is necessary for mapping Guest
> pages in the Host.
> 
> TODO: Instead of invoking the symbol directly, there needs to be a
> better way of getting a copy of the KVM instance probably by using
> other notifiers. However, currently, KVM shares its instance only
> with VFIO and therefore we are compelled to bind the passthrough'd
> device to vfio-pci.
Yeah, this is a bad solution, sorry, vfio is not going to gratuitously
call out to vhost to share a kvm pointer.  I'd prefer to get rid of
vfio having any knowledge or visibility of the kvm pointer.  Thanks,

Alex
 
> Signed-off-by: Vivek Kasireddy <vivek.kasireddy at intel.com>
> ---
>  drivers/vfio/vfio.c | 9 +++++++++
>  1 file changed, 9 insertions(+)
> 
> diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> index 4ad8a35667a7..9fb11b1ad3cd 100644
> --- a/drivers/vfio/vfio.c
> +++ b/drivers/vfio/vfio.c
> @@ -2213,11 +2213,20 @@ static int vfio_unregister_iommu_notifier(struct
vfio_group *group,
>  	return ret;
>  }
>  
> +extern void vhost_vdmabuf_get_kvm(unsigned long action, void *data);
>  void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
>  {
> +	void (*fn)(unsigned long, void *);
> +
>  	group->kvm = kvm;
>  	blocking_notifier_call_chain(&group->notifier,
>  				VFIO_GROUP_NOTIFY_SET_KVM, kvm);
> +
> +	fn = symbol_get(vhost_vdmabuf_get_kvm);
> +	if (fn) {
> +		fn(VFIO_GROUP_NOTIFY_SET_KVM, kvm);
> +		symbol_put(vhost_vdmabuf_get_kvm);
> +	}
>  }
>  EXPORT_SYMBOL_GPL(vfio_group_set_kvm);
>

Hi Alex,
> -----Original Message-----
> From: Alex Williamson <alex.williamson at redhat.com>
> Sent: Tuesday, January 19, 2021 7:40 AM
> To: Kasireddy, Vivek <vivek.kasireddy at intel.com>
> Cc: virtualization at lists.linux-foundation.org; Kim, Dongwon
<dongwon.kim at intel.com>
> Subject: Re: [RFC 3/3] vfio: Share the KVM instance with Vdmabuf
> 
> On Tue, 19 Jan 2021 00:28:12 -0800
> Vivek Kasireddy <vivek.kasireddy at intel.com> wrote:
> 
> > Getting a copy of the KVM instance is necessary for mapping Guest
> > pages in the Host.
> >
> > TODO: Instead of invoking the symbol directly, there needs to be a
> > better way of getting a copy of the KVM instance probably by using
> > other notifiers. However, currently, KVM shares its instance only
> > with VFIO and therefore we are compelled to bind the passthrough'd
> > device to vfio-pci.
> 
> Yeah, this is a bad solution, sorry, vfio is not going to gratuitously
> call out to vhost to share a kvm pointer.  I'd prefer to get rid of
> vfio having any knowledge or visibility of the kvm pointer.  Thanks,
[Kasireddy, Vivek] I agree that this is definitely not ideal as I recognize it
in the TODO. However, it looks like VFIO also gets a copy of the KVM 
pointer in a similar manner:

virt/kvm/vfio.c

static void kvm_vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
{
        void (*fn)(struct vfio_group *, struct kvm *);

        fn = symbol_get(vfio_group_set_kvm);
        if (!fn)
                return;

        fn(group, kvm);

        symbol_put(vfio_group_set_kvm);
}

With this patch, I am not suggesting that this is a precedent that should be
followed
but it appears there doesn't seem to be an alternative way of getting a copy
of the KVM
pointer that is clean and elegant -- unless I have not looked hard enough. I
guess we
could create a notifier chain with callbacks for VFIO and Vhost that KVM would
call
but this would mean modifying KVM.

Also, if I understand correctly, if VFIO does not want to share the KVM pointer
with
VFIO groups, then I think it would break stuff like mdev which counts on it. 

Thanks,
Vivek
> Alex
> 
> > Signed-off-by: Vivek Kasireddy <vivek.kasireddy at intel.com>
> > ---
> >  drivers/vfio/vfio.c | 9 +++++++++
> >  1 file changed, 9 insertions(+)
> >
> > diff --git a/drivers/vfio/vfio.c b/drivers/vfio/vfio.c
> > index 4ad8a35667a7..9fb11b1ad3cd 100644
> > --- a/drivers/vfio/vfio.c
> > +++ b/drivers/vfio/vfio.c
> > @@ -2213,11 +2213,20 @@ static int
vfio_unregister_iommu_notifier(struct vfio_group
> *group,
> >  	return ret;
> >  }
> >
> > +extern void vhost_vdmabuf_get_kvm(unsigned long action, void *data);
> >  void vfio_group_set_kvm(struct vfio_group *group, struct kvm *kvm)
> >  {
> > +	void (*fn)(unsigned long, void *);
> > +
> >  	group->kvm = kvm;
> >  	blocking_notifier_call_chain(&group->notifier,
> >  				VFIO_GROUP_NOTIFY_SET_KVM, kvm);
> > +
> > +	fn = symbol_get(vhost_vdmabuf_get_kvm);
> > +	if (fn) {
> > +		fn(VFIO_GROUP_NOTIFY_SET_KVM, kvm);
> > +		symbol_put(vhost_vdmabuf_get_kvm);
> > +	}
> >  }
> >  EXPORT_SYMBOL_GPL(vfio_group_set_kvm);
> >