Hi Gerd,> -----Original Message----- > From: Gerd Hoffmann <kraxel at redhat.com> > Sent: Tuesday, February 09, 2021 12:45 AM > To: Kasireddy, Vivek <vivek.kasireddy at intel.com> > Cc: Daniel Vetter <daniel at ffwll.ch>; virtualization at lists.linux-foundation.org; dri- > devel at lists.freedesktop.org; Vetter, Daniel <daniel.vetter at intel.com>; > daniel.vetter at ffwll.ch; Kim, Dongwon <dongwon.kim at intel.com>; > sumit.semwal at linaro.org; christian.koenig at amd.com; linux-media at vger.kernel.org > Subject: Re: [RFC v3 2/3] virtio: Introduce Vdmabuf driver > > Hi, > > > > > > Nack, this doesn't work on dma-buf. And it'll blow up at runtime > > > > > when you enable the very recently merged CONFIG_DMABUF_DEBUG (would > > > > > be good to test with that, just to make sure). > > [Kasireddy, Vivek] Although, I have not tested it yet but it looks like this will > > throw a wrench in our solution as we use sg_next to iterate over all the struct page * > > and get their PFNs. I wonder if there is any other clean way to get the PFNs of all > > the pages associated with a dmabuf. > > Well, there is no guarantee that dma-buf backing storage actually has > struct page ...[Kasireddy, Vivek] What if I do mmap() on the fd followed by mlock() or mmap() followed by get_user_pages()? If it still fails, would ioremapping the device memory and poking at the backing storage be an option? Or, if I bind the passthrough'd GPU device to vfio-pci and tap into the memory region associated with the device memory, can it be made to work? And, I noticed that for PFNs that do not have valid struct page associated with it, KVM does a memremap() to access/map them. Is this an option?> > > [Kasireddy, Vivek] To exclude such cases, would it not be OK to limit the scope > > of this solution (Vdmabuf) to make it clear that the dma-buf has to live in Guest RAM? > > Or, are there any ways to pin the dma-buf pages in Guest RAM to make this > > solution work? > > At that point it becomes (i915) driver-specific. If you go that route > it doesn't look that useful to use dma-bufs in the first place ...[Kasireddy, Vivek] I prefer not to make this driver specific if possible.> > > IIUC, Virtio GPU is used to present a virtual GPU to the Guest and all the rendering > > commands are captured and forwarded to the Host GPU via Virtio. > > You don't have to use the rendering pipeline. You can let the i915 gpu > render into a dma-buf shared with virtio-gpu, then use virtio-gpu only for > buffer sharing with the host.[Kasireddy, Vivek] Is this the most viable path forward? I am not sure how complex or feasible it would be but I'll look into it. Also, not using the rendering capabilities of virtio-gpu and turning it into a sharing only device means there would be a giant mode switch with a lot of if() conditions sprinkled across. Are you OK with that? Thanks, Vivek> > take care, > Gerd
Hi Vivek, Am 10.02.21 um 05:47 schrieb Kasireddy, Vivek:> Hi Gerd, > >> -----Original Message----- >> From: Gerd Hoffmann <kraxel at redhat.com> >> Sent: Tuesday, February 09, 2021 12:45 AM >> To: Kasireddy, Vivek <vivek.kasireddy at intel.com> >> Cc: Daniel Vetter <daniel at ffwll.ch>; virtualization at lists.linux-foundation.org; dri- >> devel at lists.freedesktop.org; Vetter, Daniel <daniel.vetter at intel.com>; >> daniel.vetter at ffwll.ch; Kim, Dongwon <dongwon.kim at intel.com>; >> sumit.semwal at linaro.org; christian.koenig at amd.com; linux-media at vger.kernel.org >> Subject: Re: [RFC v3 2/3] virtio: Introduce Vdmabuf driver >> >> Hi, >> >>>>>> Nack, this doesn't work on dma-buf. And it'll blow up at runtime >>>>>> when you enable the very recently merged CONFIG_DMABUF_DEBUG (would >>>>>> be good to test with that, just to make sure). >>> [Kasireddy, Vivek] Although, I have not tested it yet but it looks like this will >>> throw a wrench in our solution as we use sg_next to iterate over all the struct page * >>> and get their PFNs. I wonder if there is any other clean way to get the PFNs of all >>> the pages associated with a dmabuf. >> Well, there is no guarantee that dma-buf backing storage actually has >> struct page ... > [Kasireddy, Vivek] What if I do mmap() on the fd followed by mlock() or mmap() > followed by get_user_pages()? If it still fails, would ioremapping the device memory > and poking at the backing storage be an option? Or, if I bind the passthrough'd GPU device > to vfio-pci and tap into the memory region associated with the device memory, can it be > made to work?get_user_pages() is not allowed on mmaped DMA-bufs in the first place. Daniel is currently adding code to make sure that this is never ever used.> And, I noticed that for PFNs that do not have valid struct page associated with it, KVM > does a memremap() to access/map them. Is this an option?No, even for system memory which has a valid struct page touching it when it is part of a DMA-buf is illegal since the reference count and mapping fields in struct page might be used for something different. Keep in mind that struct page is a heavily overloaded structure for different use cases. You can't just use it for a different use case than what the owner of the page has intended it. Regards, Christian.> >>> [Kasireddy, Vivek] To exclude such cases, would it not be OK to limit the scope >>> of this solution (Vdmabuf) to make it clear that the dma-buf has to live in Guest RAM? >>> Or, are there any ways to pin the dma-buf pages in Guest RAM to make this >>> solution work? >> At that point it becomes (i915) driver-specific. If you go that route >> it doesn't look that useful to use dma-bufs in the first place ... > [Kasireddy, Vivek] I prefer not to make this driver specific if possible. > >>> IIUC, Virtio GPU is used to present a virtual GPU to the Guest and all the rendering >>> commands are captured and forwarded to the Host GPU via Virtio. >> You don't have to use the rendering pipeline. You can let the i915 gpu >> render into a dma-buf shared with virtio-gpu, then use virtio-gpu only for >> buffer sharing with the host. > [Kasireddy, Vivek] Is this the most viable path forward? I am not sure how complex or > feasible it would be but I'll look into it. > Also, not using the rendering capabilities of virtio-gpu and turning it into a sharing only > device means there would be a giant mode switch with a lot of if() conditions sprinkled > across. Are you OK with that? > > Thanks, > Vivek >> take care, >> Gerd
> > You don't have to use the rendering pipeline. You can let the i915 gpu > > render into a dma-buf shared with virtio-gpu, then use virtio-gpu only for > > buffer sharing with the host. > [Kasireddy, Vivek] Is this the most viable path forward? I am not sure how complex or > feasible it would be but I'll look into it. > Also, not using the rendering capabilities of virtio-gpu and turning it into a sharing only > device means there would be a giant mode switch with a lot of if() conditions sprinkled > across. Are you OK with that?Hmm, why a big mode switch? You should be able to do that without modifying the virtio-gpu guest driver. On the host side qemu needs some work to support the most recent virtio-gpu features like the buffer uuids (assuming you use qemu userspace), right now those are only supported by crosvm. It might be useful to add support for display-less virtio-gpu, i.e. "qemu -device virtio-gpu-pci,max_outputs=0". Right now the linux driver throws an error in case no output (crtc) is present. Should be fixable without too much effort though, effectively the sanity check would have to be moved from driver initialization to commands like SET_SCANOUT which manage the outputs. take care, Gerd