search for: clenqueuesvmmigratemem

On 2020-06-22 16:38, Ralph Campbell wrote: > The OpenCL function clEnqueueSVMMigrateMem(), without any flags, will > migrate memory in the given address range to device private memory. The > source pages might already have been migrated to device private memory. > In that case, the source struct page is not checked to see if it is > a device private page and incorrectly co...

...his feature. > There is various code related to it in nouveau, but as far as I can tell > it never actually got turned on, and the only changes since the initial > commit are global cleanups. This is not actually true. OpenCL 2.x does support SVM with nouveau and device private memory via clEnqueueSVMMigrateMem(). Also, Ben Skeggs has accepted a set of patches to map GPU memory after being migrated and this change would conflict with that. > Signed-off-by: Christoph Hellwig <hch at lst.de> > --- > drivers/gpu/drm/amd/amdgpu/amdgpu_ttm.c | 1 - > drivers/gpu/drm/nouveau/nouveau_dmem...

On Mon, Jun 22, 2020 at 04:38:53PM -0700, Ralph Campbell wrote: > The OpenCL function clEnqueueSVMMigrateMem(), without any flags, will > migrate memory in the given address range to device private memory. The > source pages might already have been migrated to device private memory. > In that case, the source struct page is not checked to see if it is > a device private page and incorrectly co...

When calling OpenCL clEnqueueSVMMigrateMem() on a region of memory that is backed by pte_none() or zero pages, migrate_vma_setup() will fill the source PFN array with an entry indicating the source page is zero. Use this to optimize migration to device private memory by allocating GPU memory and zero filling it instead of failing to migrate...

On 5/20/20 12:20 PM, Jason Gunthorpe wrote: > On Wed, May 20, 2020 at 11:36:52AM -0700, Ralph Campbell wrote: >> When calling OpenCL clEnqueueSVMMigrateMem() on a region of memory that >> is backed by pte_none() or zero pages, migrate_vma_setup() will fill the >> source PFN array with an entry indicating the source page is zero. >> Use this to optimize migration to device private memory by allocating >> GPU memory and zero fill...

...(args->va_end == 0) >> + args->va_end = args->va_start + PAGE_SIZE; > > That is really weird, how is it useful for the kernel to map a region > of unknown size and alignment to the GPU? > > Jason I agree it is somewhat weird. The OpenCL 2.2 specification says that clEnqueueSVMMigrateMem() takes an array of pointers and sizes (in bytes) but the size is optional. There is no alignment required. This "works" because the pointers have to be allocated with clSVMAlloc() and presumably, the implementation for clSVMAlloc() keeps track of the length allocated and can fill that in...

Making sure to include linux-mm and Bharata B Rao for IBM's use of migrate_vma*(). On 6/24/20 11:10 AM, Ralph Campbell wrote: > > On 6/24/20 12:23 AM, Christoph Hellwig wrote: >> On Mon, Jun 22, 2020 at 04:38:53PM -0700, Ralph Campbell wrote: >>> The OpenCL function clEnqueueSVMMigrateMem(), without any flags, will >>> migrate memory in the given address range to device private memory. The >>> source pages might already have been migrated to device private memory. >>> In that case, the source struct page is not checked to see if it is >>> a device...

...to it in nouveau, but as far as I can tell > >> it never actually got turned on, and the only changes since the initial > >> commit are global cleanups. > > > > This is not actually true. OpenCL 2.x does support SVM with nouveau and > > device private memory via clEnqueueSVMMigrateMem(). > > Also, Ben Skeggs has accepted a set of patches to map GPU memory after being > > migrated and this change would conflict with that. > > Can you explain me how we actually invoke this code? > > For that we'd need HMM_PFN_DEVICE_PRIVATE NVIF_VMM_PFNMAP_V0_VRAM &gt...

These are based on 5.8.0-rc2 and intended for Ben Skeggs' nouveau tree. I believe the changes can be queued for 5.8-rcX after being reviewed. These were part of a larger series but I'm resending them separately as suggested by Jason Gunthorpe. https://lore.kernel.org/linux-mm/20200619215649.32297-1-rcampbell at nvidia.com/ Note that in order to exercise/test patch 2 here, you will need a

...e is various code related to it in nouveau, but as far as I can tell >> it never actually got turned on, and the only changes since the initial >> commit are global cleanups. > > This is not actually true. OpenCL 2.x does support SVM with nouveau and > device private memory via clEnqueueSVMMigrateMem(). > Also, Ben Skeggs has accepted a set of patches to map GPU memory after being > migrated and this change would conflict with that. Can you explain me how we actually invoke this code? For that we'd need HMM_PFN_DEVICE_PRIVATE NVIF_VMM_PFNMAP_V0_VRAM set in ->pfns before calling h...

...s various code related to it in nouveau, but as far as I can tell > > it never actually got turned on, and the only changes since the initial > > commit are global cleanups. > > This is not actually true. OpenCL 2.x does support SVM with nouveau and > device private memory via clEnqueueSVMMigrateMem(). > Also, Ben Skeggs has accepted a set of patches to map GPU memory after being > migrated and this change would conflict with that. Other than the page_to_dmem() possibly doing container_of on the wrong type pgmap, are there other bugs here to fix? Something like this is probably close t...

On Wed, May 20, 2020 at 11:36:52AM -0700, Ralph Campbell wrote: > When calling OpenCL clEnqueueSVMMigrateMem() on a region of memory that > is backed by pte_none() or zero pages, migrate_vma_setup() will fill the > source PFN array with an entry indicating the source page is zero. > Use this to optimize migration to device private memory by allocating > GPU memory and zero filling it instead o...

The OpenCL function clEnqueueSVMMigrateMem(), without any flags, will migrate memory in the given address range to device private memory. The source pages might already have been migrated to device private memory. In that case, the source struct page is not checked to see if it is a device private page and incorrectly computes the GPU's...

On 6/24/20 12:23 AM, Christoph Hellwig wrote: > On Mon, Jun 22, 2020 at 04:38:53PM -0700, Ralph Campbell wrote: >> The OpenCL function clEnqueueSVMMigrateMem(), without any flags, will >> migrate memory in the given address range to device private memory. The >> source pages might already have been migrated to device private memory. >> In that case, the source struct page is not checked to see if it is >> a device private page an...

...au, but as far as I can tell >>>> it never actually got turned on, and the only changes since the initial >>>> commit are global cleanups. >>> >>> This is not actually true. OpenCL 2.x does support SVM with nouveau and >>> device private memory via clEnqueueSVMMigrateMem(). >>> Also, Ben Skeggs has accepted a set of patches to map GPU memory after being >>> migrated and this change would conflict with that. >> >> Can you explain me how we actually invoke this code? >> >> For that we'd need HMM_PFN_DEVICE_PRIVATE NVIF_VM...

On Fri, Jun 19, 2020 at 02:56:41PM -0700, Ralph Campbell wrote: > The SVM page fault handler groups faults into a range of contiguous > virtual addresses and requests hmm_range_fault() to populate and > return the page frame number of system memory mapped by the CPU. > In preparation for supporting large pages to be mapped by the GPU, > process faults one page at a time. In

...and Bharata B Rao for IBM's > use of migrate_vma*(). > > On 6/24/20 11:10 AM, Ralph Campbell wrote: > > > > On 6/24/20 12:23 AM, Christoph Hellwig wrote: > > > On Mon, Jun 22, 2020 at 04:38:53PM -0700, Ralph Campbell wrote: > > > > The OpenCL function clEnqueueSVMMigrateMem(), without any flags, will > > > > migrate memory in the given address range to device private memory. The > > > > source pages might already have been migrated to device private memory. > > > > In that case, the source struct page is not checked to see if it is...

...->va_end = args->va_start + PAGE_SIZE; > > > > That is really weird, how is it useful for the kernel to map a region > > of unknown size and alignment to the GPU? > > > > Jason > > I agree it is somewhat weird. The OpenCL 2.2 specification says that > clEnqueueSVMMigrateMem() takes an array of pointers and sizes (in bytes) > but the size is optional. There is no alignment required. > This "works" because the pointers have to be allocated with clSVMAlloc() > and presumably, the implementation for clSVMAlloc() > keeps track of the length allocated...

The user level OpenCL code shouldn't have to align start and end addresses to a page boundary. That is better handled in the nouveau driver. The npages field is also redundant since it can be computed from the start and end addresses. Signed-off-by: Ralph Campbell <rcampbell at nvidia.com> --- This is for Ben Skegg's nouveau tree. I have been working with Karol Herbst on the

When acting on device private mappings a driver needs to know if the device (or other entity in case of kvmppc) actually owns this private mapping. This series adds an owner field and converts the migrate_vma code over to check it. I looked into doing the same for hmm_range_fault, but as far as I can tell that code has never been wired up to actually work for device private memory, so instead of