On Wed, Oct 27, 2021 at 03:21:15PM +0000, Parav Pandit wrote:> Hi Stefano, > > > From: Stefano Garzarella <sgarzare at redhat.com> > > Sent: Wednesday, October 27, 2021 8:04 PM > > > > Hi folks, > > I was trying to understand if we have a way to specify which vDPA bus driver > > (e.g. vhost-vdpa, virtio-vdpa) a device should use. > > IIUC we don't have it, and the first registered driver is used when a new device > > is registered. > > > > I was thinking if it makes sense to extend the management API to specify which > > bus driver to use for a device. A use case could be for example a single host > > handling VMs and bare-metal containers, so we would have both virtio-vdpa > > and vhost-vdpa loaded and we want to attach some devices to VMs through > > vhost-vdpa and others to containers through virtio-vdpa. > > > > What do you think? > > > One option is, user keeps the drivers_autoprobe disabled for the vdpa bus using, > > $ vdpa/vdpa dev add mgmtdev vdpasim_net name vdpa0 mac 00:11:22:33:44:55 > $ echo 0 > /sys/bus/vdpa/drivers_autoprobe > > And after vdpa device creation, it manually binds to the desired driver such as, > > $ echo vdpa0 > /sys/bus/vdpa/drivers/virtio_vdpa/bind > Or > $ echo vdpa0 > /sys/bus/vdpa/drivers/vhost_vdpa/bind > > In an case of VDUSE, it makes more sense to bind to the one of the above driver after user space has connected the use space backend to the kernel device.The only annoying thing is that manual bind is not validated. E.g. if one makes a mistake and binds an incorrect device, it just tends to crash IIRC. Another is that it all needs to be root. -- MST
On Thu, Oct 28, 2021 at 4:16 AM Michael S. Tsirkin <mst at redhat.com> wrote:> > On Wed, Oct 27, 2021 at 03:21:15PM +0000, Parav Pandit wrote: > > Hi Stefano, > > > > > From: Stefano Garzarella <sgarzare at redhat.com> > > > Sent: Wednesday, October 27, 2021 8:04 PM > > > > > > Hi folks, > > > I was trying to understand if we have a way to specify which vDPA bus driver > > > (e.g. vhost-vdpa, virtio-vdpa) a device should use. > > > IIUC we don't have it, and the first registered driver is used when a new device > > > is registered. > > > > > > I was thinking if it makes sense to extend the management API to specify which > > > bus driver to use for a device.Actually, we want to support this in the first version of vDPA bus. But for some reason it was dropped. The idea is to specify the device type 'virtio' or 'vhost'. But a concern is that, it may encourage vendor to implement e.g virtio specific device (without DMA isolation).>A use case could be for example a single host > > > handling VMs and bare-metal containers, so we would have both virtio-vdpa > > > and vhost-vdpa loaded and we want to attach some devices to VMs through > > > vhost-vdpa and others to containers through virtio-vdpa. > > > > > > What do you think? > > > > > One option is, user keeps the drivers_autoprobe disabled for the vdpa bus using, > > > > $ vdpa/vdpa dev add mgmtdev vdpasim_net name vdpa0 mac 00:11:22:33:44:55 > > $ echo 0 > /sys/bus/vdpa/drivers_autoprobe > > > > And after vdpa device creation, it manually binds to the desired driver such as, > > > > $ echo vdpa0 > /sys/bus/vdpa/drivers/virtio_vdpa/bind > > Or > > $ echo vdpa0 > /sys/bus/vdpa/drivers/vhost_vdpa/bind > > > > In an case of VDUSE, it makes more sense to bind to the one of the above driver after user space has connected the use space backend to the kernel device. > > The only annoying thing is that manual bind is not validated. > E.g. if one makes a mistake and binds an incorrect device, > it just tends to crash IIRC. > Another is that it all needs to be root.I'm not sure it's worth bothering with. As discussed, switching between vendor drivers and vfio requires manually bind/unbind as well. Thanks> > -- > MST >
> From: Michael S. Tsirkin <mst at redhat.com> > Sent: Thursday, October 28, 2021 1:46 AM > > On Wed, Oct 27, 2021 at 03:21:15PM +0000, Parav Pandit wrote: > > Hi Stefano, > > > > > From: Stefano Garzarella <sgarzare at redhat.com> > > > Sent: Wednesday, October 27, 2021 8:04 PM > > > > > > Hi folks, > > > I was trying to understand if we have a way to specify which vDPA > > > bus driver (e.g. vhost-vdpa, virtio-vdpa) a device should use. > > > IIUC we don't have it, and the first registered driver is used when > > > a new device is registered. > > > > > > I was thinking if it makes sense to extend the management API to > > > specify which bus driver to use for a device. A use case could be > > > for example a single host handling VMs and bare-metal containers, so > > > we would have both virtio-vdpa and vhost-vdpa loaded and we want to > > > attach some devices to VMs through vhost-vdpa and others to containers > through virtio-vdpa. > > > > > > What do you think? > > > > > One option is, user keeps the drivers_autoprobe disabled for the vdpa > > bus using, > > > > $ vdpa/vdpa dev add mgmtdev vdpasim_net name vdpa0 mac > > 00:11:22:33:44:55 $ echo 0 > /sys/bus/vdpa/drivers_autoprobe > > > > And after vdpa device creation, it manually binds to the desired > > driver such as, > > > > $ echo vdpa0 > /sys/bus/vdpa/drivers/virtio_vdpa/bind > > Or > > $ echo vdpa0 > /sys/bus/vdpa/drivers/vhost_vdpa/bind > > > > In an case of VDUSE, it makes more sense to bind to the one of the above > driver after user space has connected the use space backend to the kernel > device. > > The only annoying thing is that manual bind is not validated. > E.g. if one makes a mistake and binds an incorrect device, it just tends to crash > IIRC.Only a vdpa device can be bind/unbind to a vdpa bus driver. Such checks are done by the kernel core. I didn't follow when can it crash. Can you please share an example when can it crash?