search for: migrate_vma_collect

...device private memory. > If a page isn't migrating, it seems inefficient to invalidate those TLB > entries. > > Any other suggestions? After a night's sleep, I think this might work. What do others think? 1) Add a new MMU_NOTIFY_MIGRATE enum to mmu_notifier_event. 2) Change migrate_vma_collect() to use the new MMU_NOTIFY_MIGRATE event type. 3) Modify nouveau_svmm_invalidate_range_start() to simply return (no invalidations) for MMU_NOTIFY_MIGRATE mmu notifier callbacks. 4) Leave the src_owner check in migrate_vma_collect_pmd() for normal pages so if the device driver is migrating normal...

...de/linux/mmu_notifier.h > @@ -38,6 +38,10 @@ struct mmu_interval_notifier; > * > * @MMU_NOTIFY_RELEASE: used during mmu_interval_notifier invalidate to signal > * that the mm refcount is zero and the range is no longer accessible. > + * > + * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal > + * a device driver to possibly ignore the invalidation if the > + * migrate_pgmap_owner field matches the driver's device private pgmap owner. > */ > enum mmu_notifier_event { > MMU_NOTIFY_UNMAP = 0, > @@ -46,6 +50,7 @@ enum mmu_notifier_event {...

...nux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -38,6 +38,10 @@ struct mmu_interval_notifier; * * @MMU_NOTIFY_RELEASE: used during mmu_interval_notifier invalidate to signal * that the mm refcount is zero and the range is no longer accessible. + * + * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal + * a device driver to possibly ignore the invalidation if the + * migrate_pgmap_owner field matches the driver's device private pgmap owner. */ enum mmu_notifier_event { MMU_NOTIFY_UNMAP = 0, @@ -46,6 +50,7 @@ enum mmu_notifier_event { MMU_NOTIFY_PROTECTION_PAGE,...

...nux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -38,6 +38,10 @@ struct mmu_interval_notifier; * * @MMU_NOTIFY_RELEASE: used during mmu_interval_notifier invalidate to signal * that the mm refcount is zero and the range is no longer accessible. + * + * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal + * a device driver to possibly ignore the invalidation if the src_own + * field matches. */ enum mmu_notifier_event { MMU_NOTIFY_UNMAP = 0, @@ -46,6 +50,7 @@ enum mmu_notifier_event { MMU_NOTIFY_PROTECTION_PAGE, MMU_NOTIFY_SOFT_DIRTY, MMU_NOTIFY_RELEASE, + MMU_NOT...

...9;t migrating, it seems inefficient to invalidate those TLB > > entries. > > > > Any other suggestions? > > After a night's sleep, I think this might work. What do others think? > > 1) Add a new MMU_NOTIFY_MIGRATE enum to mmu_notifier_event. > > 2) Change migrate_vma_collect() to use the new MMU_NOTIFY_MIGRATE event type. > > 3) Modify nouveau_svmm_invalidate_range_start() to simply return (no invalidations) > for MMU_NOTIFY_MIGRATE mmu notifier callbacks. Isn't it a bit of an assumption that migrate_vma_collect() is only used by nouveau itself? What if...

...nux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -38,6 +38,10 @@ struct mmu_interval_notifier; * * @MMU_NOTIFY_RELEASE: used during mmu_interval_notifier invalidate to signal * that the mm refcount is zero and the range is no longer accessible. + * + * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal + * a device driver to possibly ignore the invalidation if the + * migrate_pgmap_owner field matches the driver's device private pgmap owner. */ enum mmu_notifier_event { MMU_NOTIFY_UNMAP = 0, @@ -46,6 +50,7 @@ enum mmu_notifier_event { MMU_NOTIFY_PROTECTION_PAGE,...

...nux/mmu_notifier.h +++ b/include/linux/mmu_notifier.h @@ -38,6 +38,10 @@ struct mmu_interval_notifier; * * @MMU_NOTIFY_RELEASE: used during mmu_interval_notifier invalidate to signal * that the mm refcount is zero and the range is no longer accessible. + * + * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal + * a device driver to possibly ignore the invalidation if the + * migrate_pgmap_owner field matches the driver's device private pgmap owner. */ enum mmu_notifier_event { MMU_NOTIFY_UNMAP = 0, @@ -46,6 +50,7 @@ enum mmu_notifier_event { MMU_NOTIFY_PROTECTION_PAGE,...

...de/linux/mmu_notifier.h > @@ -38,6 +38,10 @@ struct mmu_interval_notifier; > * > * @MMU_NOTIFY_RELEASE: used during mmu_interval_notifier invalidate to signal > * that the mm refcount is zero and the range is no longer accessible. > + * > + * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal > + * a device driver to possibly ignore the invalidation if the src_own > + * field matches. > */ > enum mmu_notifier_event { > MMU_NOTIFY_UNMAP = 0, > @@ -46,6 +50,7 @@ enum mmu_notifier_event { > MMU_NOTIFY_PROTECTION_PAGE, > MMU_NOTIFY_SOF...

The goal for this series is to avoid device private memory TLB invalidations when migrating a range of addresses from system memory to device private memory and some of those pages have already been migrated. The approach taken is to introduce a new mmu notifier invalidation event type and use that in the device driver to skip invalidation callbacks from migrate_vma_setup(). The device driver is

...;> @@ -38,6 +38,10 @@ struct mmu_interval_notifier; >> * >> * @MMU_NOTIFY_RELEASE: used during mmu_interval_notifier invalidate to signal >> * that the mm refcount is zero and the range is no longer accessible. >> + * >> + * @MMU_NOTIFY_MIGRATE: used during migrate_vma_collect() invalidate to signal >> + * a device driver to possibly ignore the invalidation if the src_own >> + * field matches. >> */ >> enum mmu_notifier_event { >> MMU_NOTIFY_UNMAP = 0, >> @@ -46,6 +50,7 @@ enum mmu_notifier_event { >> MMU_NOTIFY_PROTEC...

The goal for this series is to avoid device private memory TLB invalidations when migrating a range of addresses from system memory to device private memory and some of those pages have already been migrated. The approach taken is to introduce a new mmu notifier invalidation event type and use that in the device driver to skip invalidation callbacks from migrate_vma_setup(). The device driver is

...(hence no one can unmap the range being migrated). * * Once the caller is done cleaning up things and updating its page table (if it * chose to do so, this is not an obligation) it finally calls @@ -2657,10 +2656,11 @@ int migrate_vma_setup(struct migrate_vma *args) args->npages = 0; migrate_vma_collect(args); - if (args->cpages) - migrate_vma_prepare(args); - if (args->cpages) - migrate_vma_unmap(args); + if (!args->cpages) + return 0; + + migrate_vma_prepare(args); + migrate_vma_unmap(args); /* * At this point pages are locked and unmapped, and thus they have

On Mon, Jun 22, 2020 at 04:38:53PM -0700, Ralph Campbell wrote: > The OpenCL function clEnqueueSVMMigrateMem(), without any flags, will > migrate memory in the given address range to device private memory. The > source pages might already have been migrated to device private memory. > In that case, the source struct page is not checked to see if it is > a device private page and

The goal for this series is to avoid device private memory TLB invalidations when migrating a range of addresses from system memory to device private memory and some of those pages have already been migrated. The approach taken is to introduce a new mmu notifier invalidation event type and use that in the device driver to skip invalidation callbacks from migrate_vma_setup(). The device driver is

...ruct mm_struct *mm, #endif /* CONFIG_NUMA */ #if defined(CONFIG_MIGRATE_VMA_HELPER) -struct migrate_vma { - struct vm_area_struct *vma; - unsigned long *dst; - unsigned long *src; - unsigned long cpages; - unsigned long npages; - unsigned long start; - unsigned long end; -}; - static int migrate_vma_collect_hole(unsigned long start, unsigned long end, struct mm_walk *walk) @@ -2578,6 +2568,108 @@ static void migrate_vma_unmap(struct migrate_vma *migrate) } } +/** + * migrate_vma_setup() - prepare to migrate a range of memory + * @args: contains the vma, start, and and pfns array...

...ruct mm_struct *mm, #endif /* CONFIG_NUMA */ #if defined(CONFIG_MIGRATE_VMA_HELPER) -struct migrate_vma { - struct vm_area_struct *vma; - unsigned long *dst; - unsigned long *src; - unsigned long cpages; - unsigned long npages; - unsigned long start; - unsigned long end; -}; - static int migrate_vma_collect_hole(unsigned long start, unsigned long end, struct mm_walk *walk) @@ -2578,6 +2568,110 @@ static void migrate_vma_unmap(struct migrate_vma *migrate) } } +/** + * migrate_vma_setup() - prepare to migrate a range of memory + * @args: contains the vma, start, and and pfns array...

...ined(CONFIG_MIGRATE_VMA_HELPER) > -struct migrate_vma { > - struct vm_area_struct *vma; > - unsigned long *dst; > - unsigned long *src; > - unsigned long cpages; > - unsigned long npages; > - unsigned long start; > - unsigned long end; > -}; > - > static int migrate_vma_collect_hole(unsigned long start, > unsigned long end, > struct mm_walk *walk) > @@ -2578,6 +2568,108 @@ static void migrate_vma_unmap(struct migrate_vma *migrate) > } > } > > +/** > + * migrate_vma_setup() - prepare to migrate a range of memory > + *...

The goal for this series is to avoid device private memory TLB invalidations when migrating a range of addresses from system memory to device private memory and some of those pages have already been migrated. The approach taken is to introduce a new mmu notifier invalidation event type and use that in the device driver to skip invalidation callbacks from migrate_vma_setup(). The device driver is

Hi Jérôme, Ben and Jason, below is a series against the hmm tree which starts revamping the migrate_vma functionality. The prime idea is to export three slightly lower level functions and thus avoid the need for migrate_vma_ops callbacks. Diffstat: 7 files changed, 282 insertions(+), 614 deletions(-) A git tree is also available at: git://git.infradead.org/users/hch/misc.git

Hi Jérôme, Ben and Jason, below is a series against the hmm tree which starts revamping the migrate_vma functionality. The prime idea is to export three slightly lower level functions and thus avoid the need for migrate_vma_ops callbacks. Diffstat: 4 files changed, 285 insertions(+), 602 deletions(-) A git tree is also available at: git://git.infradead.org/users/hch/misc.git