Virtualization - Dec 2021 - [RFC PATCH] virtio: make sure legacy pci device gain 32bit-pfn vq

On Tue, Dec 07, 2021 at 05:09:56PM +0800, ?? wrote:
> 
> 
> ? 2021/12/7 ??4:13, Michael S. Tsirkin ??:
> > On Tue, Dec 07, 2021 at 03:51:45PM +0800, ?? wrote:
> > > We observed issues like:
> > >    virtio-pci 0000:14:00.0: platform bug: legacy virtio-mmio must
> > >    not be used with RAM above 0x4000GB
> > > 
> > > when we have a legacy pci device which desired 32bit-pfn vq
> > > but gain 64bit-pfn instead, lead into the failure of probe.
> > > 
> > > vring_use_dma_api() is playing the key role in here, to help the
> > > allocation process understand which kind of vq it should alloc,
> > > however, it failed to take care the legacy pci device, which only
> > > have 32bit feature flag and can never have
VIRTIO_F_ACCESS_PLATFORM
> > > setted.
> > > 
> > > This patch introduce force_dma flag to help vring_use_dma_api()
> > > understanding the requirement better, to avoid the failing.
> > > 
> > > Signed-off-by: Michael Wang <yun.wang at linux.alibaba.com>
> > 
> > This will break configs where the device appears behind
> > a virtual iommu, so this won't fly.
> > Just make your device support 1.0, eh?
> 
> Hi, Michael
> 
> Thanks for the comment, unfortunately modify device is not an option for us
> :-(
> 
> Is there any idea on how to solve this issue properly?
> 
> Regards,
> Michael Wang
There's an old idea:
https://www.spinics.net/lists/kernel/msg3248800.html

> > 
> > > ---
> > >   drivers/virtio/virtio_pci_legacy.c | 10 ++++++++++
> > >   drivers/virtio/virtio_ring.c       |  3 +++
> > >   include/linux/virtio.h             |  1 +
> > >   3 files changed, 14 insertions(+)
> > > 
> > > diff --git a/drivers/virtio/virtio_pci_legacy.c
> > > b/drivers/virtio/virtio_pci_legacy.c
> > > index d62e983..11f2ebf 100644
> > > --- a/drivers/virtio/virtio_pci_legacy.c
> > > +++ b/drivers/virtio/virtio_pci_legacy.c
> > > @@ -263,6 +263,16 @@ int virtio_pci_legacy_probe(struct
virtio_pci_device
> > > *vp_dev)
> > >   	vp_dev->setup_vq = setup_vq;
> > >   	vp_dev->del_vq = del_vq;
> > > 
> > > +	/*
> > > +	 * The legacy pci device requre 32bit-pfn vq,
> > > +	 * or setup_vq() will failed.
> > > +	 *
> > > +	 * Thus we make sure vring_use_dma_api() will
> > > +	 * return true during the allocation by marking
> > > +	 * force_dma here.
> > > +	 */
> > > +	vp_dev->vdev.force_dma = true;
> > > +
> > >   	return 0;
> > > 
> > >   err_iomap:
> > > diff --git a/drivers/virtio/virtio_ring.c
b/drivers/virtio/virtio_ring.c
> > > index 3035bb6..6562e01 100644
> > > --- a/drivers/virtio/virtio_ring.c
> > > +++ b/drivers/virtio/virtio_ring.c
> > > @@ -245,6 +245,9 @@ static inline bool
virtqueue_use_indirect(struct
> > > virtqueue *_vq,
> > > 
> > >   static bool vring_use_dma_api(struct virtio_device *vdev)
> > >   {
> > > +	if (vdev->force_dma)
> > > +		return true;
> > > +
> > >   	if (!virtio_has_dma_quirk(vdev))
> > >   		return true;
> > > 
> > > diff --git a/include/linux/virtio.h b/include/linux/virtio.h
> > > index 41edbc0..a4eb29d 100644
> > > --- a/include/linux/virtio.h
> > > +++ b/include/linux/virtio.h
> > > @@ -109,6 +109,7 @@ struct virtio_device {
> > >   	bool failed;
> > >   	bool config_enabled;
> > >   	bool config_change_pending;
> > > +	bool force_dma;
> > >   	spinlock_t config_lock;
> > >   	spinlock_t vqs_list_lock; /* Protects VQs list access */
> > >   	struct device dev;
> > > -- 
> > > 1.8.3.1