Nouveau - Jan 2025 - [PATCH v1 08/12] mm/rmap: handle device-exclusive entries correctly in try_to_unmap_one()

On 30.01.25 11:10, Simona Vetter wrote:
> On Wed, Jan 29, 2025 at 12:54:06PM +0100, David Hildenbrand wrote:
>> Ever since commit b756a3b5e7ea ("mm: device exclusive memory
access")
>> we can return with a device-exclusive entry from
page_vma_mapped_walk().
>>
>> try_to_unmap_one() is not prepared for that, so teach it about these
>> non-present nonswap PTEs.
>>
>> Before that, could we also have triggered this case with device-private
>> entries? Unlikely.
> 
> Just quick comment on this, I'm still pondering all the other aspects.
> 
> device-private memory is entirely owned by the driver, the core mm
isn't
> supposed to touch these beyond migrating it back to system memory in
> do_swap_page. Plus using rmap when the driver asks for invalidating
> mappings as needed.
> 
> So no lru, thp, migration or anything initiated by core mm should ever
> happen on these device private pages. If it does, it'd be a bug.
I was not 100% sure about HWPoison handling, that's why I added that 
comment. In other regards I agree: reclaim etc. does not apply.

-- 
Cheers,

David / dhildenb

On Thu, Jan 30, 2025 at 12:08:42PM +0100, David Hildenbrand
wrote:
> On 30.01.25 11:10, Simona Vetter wrote:
> > On Wed, Jan 29, 2025 at 12:54:06PM +0100, David Hildenbrand wrote:
> > > Ever since commit b756a3b5e7ea ("mm: device exclusive memory
access")
> > > we can return with a device-exclusive entry from
page_vma_mapped_walk().
> > > 
> > > try_to_unmap_one() is not prepared for that, so teach it about
these
> > > non-present nonswap PTEs.
> > > 
> > > Before that, could we also have triggered this case with
device-private
> > > entries? Unlikely.
> > 
> > Just quick comment on this, I'm still pondering all the other
aspects.
> > 
> > device-private memory is entirely owned by the driver, the core mm
isn't
> > supposed to touch these beyond migrating it back to system memory in
> > do_swap_page. Plus using rmap when the driver asks for invalidating
> > mappings as needed.
> > 
> > So no lru, thp, migration or anything initiated by core mm should ever
> > happen on these device private pages. If it does, it'd be a bug.
> 
> I was not 100% sure about HWPoison handling, that's why I added that
> comment. In other regards I agree: reclaim etc. does not apply.
So maybe I'm just entirely lost, but unless you have a coherent
interconnect I don't think hwpoisin should get involved with device
private memory? And for a coherent interconnect it's just device memory,
which isn't treated very special.

Also to clarify, I meant this as a general comment for all subsequent
patches that have the same paragraph.
-Sima
-- 
Simona Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch

On Thu, Jan 30, 2025 at 02:06:12PM +0100, Simona Vetter
wrote:
> On Thu, Jan 30, 2025 at 12:08:42PM +0100, David Hildenbrand wrote:
> > On 30.01.25 11:10, Simona Vetter wrote:
> > > On Wed, Jan 29, 2025 at 12:54:06PM +0100, David Hildenbrand
wrote:
> > > > Ever since commit b756a3b5e7ea ("mm: device exclusive
memory access")
> > > > we can return with a device-exclusive entry from
page_vma_mapped_walk().
> > > > 
> > > > try_to_unmap_one() is not prepared for that, so teach it
about these
> > > > non-present nonswap PTEs.
> > > > 
> > > > Before that, could we also have triggered this case with
device-private
> > > > entries? Unlikely.
> > > 
> > > Just quick comment on this, I'm still pondering all the other
aspects.
> > > 
> > > device-private memory is entirely owned by the driver, the core
mm isn't
> > > supposed to touch these beyond migrating it back to system memory
in
> > > do_swap_page. Plus using rmap when the driver asks for
invalidating
> > > mappings as needed.
> > > 
> > > So no lru, thp, migration or anything initiated by core mm should
ever
> > > happen on these device private pages. If it does, it'd be a
bug.
> > 
> > I was not 100% sure about HWPoison handling, that's why I added
that
> > comment. In other regards I agree: reclaim etc. does not apply.
> 
> So maybe I'm just entirely lost, but unless you have a coherent
> interconnect I don't think hwpoisin should get involved with device
> private memory? And for a coherent interconnect it's just device
memory,
> which isn't treated very special.
I'm not sure it is meaningful, but in principle a driver could keep
track of the poisoned private memory using that struct page
bit. Perhaps in that sense it is more of a driver private flag than
something the core MM would touch.

If you have a coherent interconnect then you should not be using
device private.

Jason

On 30.01.25 14:06, Simona Vetter wrote:
> On Thu, Jan 30, 2025 at 12:08:42PM +0100, David Hildenbrand wrote:
>> On 30.01.25 11:10, Simona Vetter wrote:
>>> On Wed, Jan 29, 2025 at 12:54:06PM +0100, David Hildenbrand wrote:
>>>> Ever since commit b756a3b5e7ea ("mm: device exclusive
memory access")
>>>> we can return with a device-exclusive entry from
page_vma_mapped_walk().
>>>>
>>>> try_to_unmap_one() is not prepared for that, so teach it about
these
>>>> non-present nonswap PTEs.
>>>>
>>>> Before that, could we also have triggered this case with
device-private
>>>> entries? Unlikely.
>>>
>>> Just quick comment on this, I'm still pondering all the other
aspects.
>>>
>>> device-private memory is entirely owned by the driver, the core mm
isn't
>>> supposed to touch these beyond migrating it back to system memory
in
>>> do_swap_page. Plus using rmap when the driver asks for invalidating
>>> mappings as needed.
>>>
>>> So no lru, thp, migration or anything initiated by core mm should
ever
>>> happen on these device private pages. If it does, it'd be a
bug.
>>
>> I was not 100% sure about HWPoison handling, that's why I added
that
>> comment. In other regards I agree: reclaim etc. does not apply.
> 
> So maybe I'm just entirely lost, but unless you have a coherent
> interconnect I don't think hwpoisin should get involved with device
> private memory? And for a coherent interconnect it's just device
memory,
> which isn't treated very special.
I would have thought that in a scenario Jason describes, that you would 
still want to zap the page from the page table (try_to_unmap()) and 
install a hwpoison entry instead.

But yes, right now this should never ever happen: memory_failure() does 
some ZONE_DEVICE specific things, but likely doesn't call try_to_unmap() 
on these folios.
> 
> Also to clarify, I meant this as a general comment for all subsequent
> patches that have the same paragraph.
Yeah, I'll rephrase that to "We'll never hit that case for special 
device-private pages."

-- 
Cheers,

David / dhildenb