search for: migrate_vma_finalize

...were still copied to a new page and thus we wasted -bandwidth but this is considered as a rare event and a price that we are -willing to pay to keep all the code simpler. +engine to perform copy from and to device memory. For this we need a new to +use migrate_vma_setup(), migrate_vma_pages(), and migrate_vma_finalize() +helpers. Memory cgroup (memcg) and rss accounting diff --git a/drivers/gpu/drm/nouveau/nouveau_dmem.c b/drivers/gpu/drm/nouveau/nouveau_dmem.c index 345c63cb752a..38416798abd4 100644 --- a/drivers/gpu/drm/nouveau/nouveau_dmem.c +++ b/drivers/gpu/drm/nouveau/nouveau_dmem.c @@ -131,9 +131,8...

...were still copied to a new page and thus we wasted -bandwidth but this is considered as a rare event and a price that we are -willing to pay to keep all the code simpler. +engine to perform copy from and to device memory. For this we need a new to +use migrate_vma_setup(), migrate_vma_pages(), and migrate_vma_finalize() +helpers. Memory cgroup (memcg) and rss accounting diff --git a/drivers/gpu/drm/nouveau/nouveau_dmem.c b/drivers/gpu/drm/nouveau/nouveau_dmem.c index 345c63cb752a..38416798abd4 100644 --- a/drivers/gpu/drm/nouveau/nouveau_dmem.c +++ b/drivers/gpu/drm/nouveau/nouveau_dmem.c @@ -131,9 +131,8 @...

...o a new page and thus we wasted > -bandwidth but this is considered as a rare event and a price that we are > -willing to pay to keep all the code simpler. > +engine to perform copy from and to device memory. For this we need a new to > +use migrate_vma_setup(), migrate_vma_pages(), and migrate_vma_finalize() > +helpers. > > > Memory cgroup (memcg) and rss accounting > diff --git a/drivers/gpu/drm/nouveau/nouveau_dmem.c b/drivers/gpu/drm/nouveau/nouveau_dmem.c > index 345c63cb752a..38416798abd4 100644 > --- a/drivers/gpu/drm/nouveau/nouveau_dmem.c > +++ b/drivers/gpu/dr...

Hi Jérôme, Ben and Jason, below is a series against the hmm tree which starts revamping the migrate_vma functionality. The prime idea is to export three slightly lower level functions and thus avoid the need for migrate_vma_ops callbacks. Diffstat: 7 files changed, 282 insertions(+), 614 deletions(-) A git tree is also available at: git://git.infradead.org/users/hch/misc.git

Hi Jérôme, Ben and Jason, below is a series against the hmm tree which starts revamping the migrate_vma functionality. The prime idea is to export three slightly lower level functions and thus avoid the need for migrate_vma_ops callbacks. Diffstat: 4 files changed, 285 insertions(+), 602 deletions(-) A git tree is also available at: git://git.infradead.org/users/hch/misc.git

Hi Jérôme, Ben and Jason, below is a series against the hmm tree which starts revamping the migrate_vma functionality. The prime idea is to export three slightly lower level functions and thus avoid the need for migrate_vma_ops callbacks. Diffstat: 5 files changed, 281 insertions(+), 607 deletions(-) A git tree is also available at: git://git.infradead.org/users/hch/misc.git

...migrate_misplaced_transhuge_page(struct mm_struct *mm, #endif /* CONFIG_NUMA */ -#if defined(CONFIG_MIGRATE_VMA_HELPER) +#ifdef CONFIG_DEVICE_PRIVATE static int migrate_vma_collect_hole(unsigned long start, unsigned long end, struct mm_walk *walk) @@ -2942,4 +2942,4 @@ void migrate_vma_finalize(struct migrate_vma *migrate) } } EXPORT_SYMBOL(migrate_vma_finalize); -#endif /* defined(MIGRATE_VMA_HELPER) */ +#endif /* CONFIG_DEVICE_PRIVATE */ -- 2.20.1

...-339,9 +339,8 @@ Migration to and from device memory =================================== Because the CPU cannot access device memory, migration must use the device DMA -engine to perform copy from and to device memory. For this we need a new to -use migrate_vma_setup(), migrate_vma_pages(), and migrate_vma_finalize() -helpers. +engine to perform copy from and to device memory. For this we need to use +migrate_vma_setup(), migrate_vma_pages(), and migrate_vma_finalize() helpers. Memory cgroup (memcg) and rss accounting diff --git a/mm/migrate.c b/mm/migrate.c index 993386cb53358d..0e78ebd720c0e4 100644 --...

...t; > migrate_vma_pages() > // moves source struct page info to destination struct page info. > // clears migration flag for pages that can't be migrated. > > <driver updates device page tables for pages still migrating, rollback pages not migrating> > > migrate_vma_finalize() > // replaces migration page table entry with destination page PFN. > > mmap_read_unlock() > > Since the address range is invalidated in the migrate_vma_setup() stage, > and the page is isolated from the LRU cache, locked, unmapped, and the page table > holds a migrat...

On Thu, Jul 23, 2020 at 03:30:04PM -0700, Ralph Campbell wrote: > When migrating the special zero page, migrate_vma_pages() calls > mmu_notifier_invalidate_range_start() before replacing the zero page > PFN in the CPU page tables. This is unnecessary since the range was > invalidated in migrate_vma_setup() and the page table entry is checked > to be sure it hasn't changed

...he approach taken is to introduce a new mmu notifier invalidation event type and use that in the device driver to skip invalidation callbacks from migrate_vma_setup(). The device driver is also then expected to handle device MMU invalidations as part of the migrate_vma_setup(), migrate_vma_pages(), migrate_vma_finalize() process. Note that this is opt-in. A device driver can simply invalidate its MMU in the mmu notifier callback and not handle MMU invalidations in the migration sequence. This series is based on Jason Gunthorpe's HMM tree (linux-5.8.0-rc4). Also, this replaces the need for the following two...

...RRAY_SIZE(src_pfns) << PAGE_SHIFT)); + next = min(end, addr + (PTRS_PER_PTE << PAGE_SHIFT)); if (next > vma->vm_end) next = vma->vm_end; @@ -725,6 +851,8 @@ static int dmirror_migrate(struct dmirror *dmirror, dmirror_migrate_finalize_and_map(&args, dmirror); migrate_vma_finalize(&args); } + kfree(dst_pfns); + kfree(src_pfns); mmap_read_unlock(mm); mmput(mm); @@ -746,6 +874,10 @@ static int dmirror_migrate(struct dmirror *dmirror, out: mmap_read_unlock(mm); + kfree(dst_pfns); +out_free_src: + kfree(src_pfns); +out_put: mmput(mm); return ret; } @@ -986,...

...to introduce a new mmu notifier > invalidation event type and use that in the device driver to skip > invalidation callbacks from migrate_vma_setup(). The device driver is > also then expected to handle device MMU invalidations as part of the > migrate_vma_setup(), migrate_vma_pages(), migrate_vma_finalize() process. > Note that this is opt-in. A device driver can simply invalidate its MMU > in the mmu notifier callback and not handle MMU invalidations in the > migration sequence. > > This series is based on Jason Gunthorpe's HMM tree (linux-5.8.0-rc4). > > Also, this repla...

...he approach taken is to introduce a new mmu notifier invalidation event type and use that in the device driver to skip invalidation callbacks from migrate_vma_setup(). The device driver is also then expected to handle device MMU invalidations as part of the migrate_vma_setup(), migrate_vma_pages(), migrate_vma_finalize() process. Note that this is opt-in. A device driver can simply invalidate its MMU in the mmu notifier callback and not handle MMU invalidations in the migration sequence. This series is based on linux-5.8.0-rc4 and the patches I sent for ("mm/hmm/nouveau: add PMD system memory mapping")...

...pages and copies source to dest> migrate_vma_pages() // moves source struct page info to destination struct page info. // clears migration flag for pages that can't be migrated. <driver updates device page tables for pages still migrating, rollback pages not migrating> migrate_vma_finalize() // replaces migration page table entry with destination page PFN. mmap_read_unlock() Since the address range is invalidated in the migrate_vma_setup() stage, and the page is isolated from the LRU cache, locked, unmapped, and the page table holds a migration entry (so the page can't b...

...(&args, dmirror); if (ret) return ret; migrate_vma_pages(&args); - dmirror_devmem_fault_finalize_and_map(&args, dmirror); + /* + * No device finalize step is needed since + * dmirror_devmem_fault_alloc_and_copy() will have already + * invalidated the device page table. + */ migrate_vma_finalize(&args); return 0; } diff --git a/tools/testing/selftests/vm/hmm-tests.c b/tools/testing/selftests/vm/hmm-tests.c index b533dd08da1d..91d38a29956b 100644 --- a/tools/testing/selftests/vm/hmm-tests.c +++ b/tools/testing/selftests/vm/hmm-tests.c @@ -881,8 +881,9 @@ TEST_F(hmm, migrate) } /*...

...he approach taken is to introduce a new mmu notifier invalidation event type and use that in the device driver to skip invalidation callbacks from migrate_vma_setup(). The device driver is also then expected to handle device MMU invalidations as part of the migrate_vma_setup(), migrate_vma_pages(), migrate_vma_finalize() process. Note that this is opt-in. A device driver can simply invalidate its MMU in the mmu notifier callback and not handle MMU invalidations in the migration sequence. This series is based on Jason Gunthorpe's HMM tree (linux-5.8.0-rc4). Also, this replaces the need for the following two...

...e(drm, vmf, &args, &dma_addr); + if (ret || dst == 0) + goto done; + nouveau_fence_new(dmem->migrate.chan, false, &fence); + migrate_vma_pages(&args); + nouveau_dmem_fence_done(&fence); + dma_unmap_page(drm->dev->dev, dma_addr, PAGE_SIZE, DMA_BIDIRECTIONAL); +done: migrate_vma_finalize(&args); - if (dst[0] == MIGRATE_PFN_ERROR) - return VM_FAULT_SIGBUS; - - return 0; + return ret; } static const struct dev_pagemap_ops nouveau_dmem_pagemap_ops = { -- 2.20.1

...ss, src, + &dst, &dma_addr); + if (ret || dst == 0) + goto done; + nouveau_fence_new(dmem->migrate.chan, false, &fence); + migrate_vma_pages(&args); + nouveau_dmem_fence_done(&fence); + dma_unmap_page(drm->dev->dev, dma_addr, PAGE_SIZE, DMA_BIDIRECTIONAL); +done: migrate_vma_finalize(&args); - if (dst[0] == MIGRATE_PFN_ERROR) - return VM_FAULT_SIGBUS; - - return 0; + return ret; } static const struct dev_pagemap_ops nouveau_dmem_pagemap_ops = { -- 2.20.1

...f (ret || dst == 0) > + goto done; > > + nouveau_fence_new(dmem->migrate.chan, false, &fence); > + migrate_vma_pages(&args); > + nouveau_dmem_fence_done(&fence); > + dma_unmap_page(drm->dev->dev, dma_addr, PAGE_SIZE, DMA_BIDIRECTIONAL); > +done: > migrate_vma_finalize(&args); > - if (dst[0] == MIGRATE_PFN_ERROR) > - return VM_FAULT_SIGBUS; > - > - return 0; > + return ret; > } > > static const struct dev_pagemap_ops nouveau_dmem_pagemap_ops = { >