Nouveau - Feb 2021 - [PATCH 1/9] mm/migrate.c: Always allow device private pages to migrate

On Tue, Feb 09, 2021 at 12:07:14PM +1100, Alistair Popple
wrote:
> Device private pages are used to represent device memory that is not
> directly accessible from the CPU. Extra references to a device private
> page are only used to ensure the struct page itself remains valid whilst
> waiting for migration entries. Therefore extra references should not
> prevent device private page migration as this can lead to failures to
> migrate pages back to the CPU which are fatal to the user process.
This should identify the extra references in expected_count, just
disabling this protection seems unsafe, ZONE_DEVICE is not so special
that the refcount means nothing

Is this a side effect of the extra refcounts that Ralph was trying to
get rid of? I'd rather see that work finished :)

Jason

On Wednesday, 10 February 2021 12:39:32 AM AEDT Jason Gunthorpe
wrote:
> On Tue, Feb 09, 2021 at 12:07:14PM +1100, Alistair Popple wrote:
> > Device private pages are used to represent device memory that is not
> > directly accessible from the CPU. Extra references to a device private
> > page are only used to ensure the struct page itself remains valid
whilst
> > waiting for migration entries. Therefore extra references should not
> > prevent device private page migration as this can lead to failures to
> > migrate pages back to the CPU which are fatal to the user process.
> 
> This should identify the extra references in expected_count, just
> disabling this protection seems unsafe, ZONE_DEVICE is not so special
> that the refcount means nothing
This is similar to what migarte_vma_check_page() does now. The issue is that a 
migration wait takes a reference on the device private page so you can end up 
with one thread stuck waiting for migration whilst the other can't migrate
due
to the extra refcount.

Given device private pages can't undergo GUP and that it's not possible
to
differentiate the migration wait refcount from any other refcount we assume 
any possible extra reference must be from migration wait.
> Is this a side effect of the extra refcounts that Ralph was trying to
> get rid of? I'd rather see that work finished :)
I'd like to see that finished too but I don't think it would help here
as this
is not a side effect of that.

 - Alistair
> Jason