Nouveau - Jun 2021 - [PATCH v10 07/10] mm: Device exclusive memory access

On Friday, 11 June 2021 4:04:35 AM AEST Peter Xu wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Thu, Jun 10, 2021 at 10:18:25AM +1000, Alistair Popple wrote:
> > > > The main problem is split_huge_pmd_address() unconditionally
calls a mmu
> > > > notifier so I would need to plumb in passing an owner
everywhere which could
> > > > get messy.
> > >
> > > Could I ask why?  split_huge_pmd_address() will notify with
CLEAR, so I'm a bit
> > > confused why we need to pass over the owner.
> >
> > Sure, it is the same reason we need to pass it for the exclusive
notifier.
> > Any invalidation during the make exclusive operation will break the
mmu read
> > side critical section forcing a retry of the operation. The owner
field is what
> > is used to filter out invalidations (such as the exclusive
invalidation) that
> > don't need to be retried.
> 
> Do you mean the mmu_interval_read_begin|retry() calls?
Yep.
> Hmm, the thing is.. to me FOLL_SPLIT_PMD should have similar effect to
explicit
> call split_huge_pmd_address(), afaict.  Since both of them use
__split_huge_pmd()
> internally which will generate that unwanted CLEAR notify.
Agree that gup calls __split_huge_pmd() via split_huge_pmd_address()
which will always CLEAR. However gup only calls split_huge_pmd_address() if it
finds a thp pmd. In follow_pmd_mask() we have:

	if (likely(!pmd_trans_huge(pmdval)))
		return follow_page_pte(vma, address, pmd, flags, &ctx->pgmap);

So I don't think we have a problem here.
> If that's the case, I think it fails because split_huge_pmd_address()
will
> trigger that CLEAR notify unconditionally (even if it's not a thp; not
sure
> whether it should be optimized to not notify at all... definitely another
> story), while FOLL_SPLIT_PMD will skip the notify as it calls
split_huge_pmd()
> instead, who checks the pmd before calling __split_huge_pmd().
> 
> Does it also mean that if there's a real THP it won't really work? 
As then
> FOLL_SPLIT_PMD will start to trigger that CLEAR notify too, I think..
> 
> --
> Peter Xu
>

On Fri, Jun 11, 2021 at 12:21:26AM +1000, Alistair Popple
wrote:
> > Hmm, the thing is.. to me FOLL_SPLIT_PMD should have similar effect to
explicit
> > call split_huge_pmd_address(), afaict.  Since both of them use
__split_huge_pmd()
> > internally which will generate that unwanted CLEAR notify.
> 
> Agree that gup calls __split_huge_pmd() via split_huge_pmd_address()
> which will always CLEAR. However gup only calls split_huge_pmd_address() if
it
> finds a thp pmd. In follow_pmd_mask() we have:
> 
> 	if (likely(!pmd_trans_huge(pmdval)))
> 		return follow_page_pte(vma, address, pmd, flags, &ctx->pgmap);
> 
> So I don't think we have a problem here.
Sorry I didn't follow here..  We do FOLL_SPLIT_PMD after this check, right? 
I
mean, if it's a thp for the current mm, afaict pmd_trans_huge() should
return
true above, so we'll skip follow_page_pte(); then we'll check
FOLL_SPLIT_PMD
and do the split, then the CLEAR notify.  Hmm.. Did I miss something?

-- 
Peter Xu

On Friday, 11 June 2021 9:04:19 AM AEST Peter Xu wrote:
> External email: Use caution opening links or attachments
> 
> 
> On Fri, Jun 11, 2021 at 12:21:26AM +1000, Alistair Popple wrote:
> > > Hmm, the thing is.. to me FOLL_SPLIT_PMD should have similar
effect to explicit
> > > call split_huge_pmd_address(), afaict.  Since both of them use
__split_huge_pmd()
> > > internally which will generate that unwanted CLEAR notify.
> >
> > Agree that gup calls __split_huge_pmd() via split_huge_pmd_address()
> > which will always CLEAR. However gup only calls
split_huge_pmd_address() if it
> > finds a thp pmd. In follow_pmd_mask() we have:
> >
> >       if (likely(!pmd_trans_huge(pmdval)))
> >               return follow_page_pte(vma, address, pmd, flags,
&ctx->pgmap);
> >
> > So I don't think we have a problem here.
> 
> Sorry I didn't follow here..  We do FOLL_SPLIT_PMD after this check,
right?  I
> mean, if it's a thp for the current mm, afaict pmd_trans_huge() should
return
> true above, so we'll skip follow_page_pte(); then we'll check
FOLL_SPLIT_PMD
> and do the split, then the CLEAR notify.  Hmm.. Did I miss something?
That seems correct - if the thp is not mapped with a pmd we won't split and
we
won't CLEAR. If there is a thp pmd we will split and CLEAR, but in that case
it
is fine - we will retry, but the retry will won't CLEAR because the pmd has
already been split.

The issue arises with doing it unconditionally in make device exclusive is that
you *always* CLEAR even if there is no thp pmd to split. Or at least that's
my
understanding, please let me know if it doesn't make sense.

 - Alistair
> --
> Peter Xu
>