Nouveau - Feb 2025 - [RFC PATCH 0/3] gpu: nova-core: add basic timer subdevice implementation

On Wed, Feb 26, 2025 at 12:45:45AM +0100, Danilo Krummrich
wrote:
> On Tue, Feb 25, 2025 at 06:57:56PM -0400, Jason Gunthorpe wrote:
> > The common driver shutdown process in the kernel, that is well tested
> > and copied, makes the driver single threaded during the remove()
> > callback.
> 
> All devres callbacks run in the same callchain, __device_release_driver()
first
> calls remove() and then all the devres callbacks, where we revoke the
pci::Bar,
> by which gets dropped and hence the bar is unmapped and resource regions
are
> free. It's not different to C drivers. Except that in C you don't
lose access to
> the void pointer that still points to the (unmapped) MMIO address.
I understand how devm works.

I'm pointing out the fundamental different in approachs. The typical
widely used pattern results in __device_release_driver() completing
with no concurrent driver code running.

DRM achieves this, in part, by using drm_dev_unplug().

The Rust approach ends up with __device_release_driver() completing
and leaving driver code still running in other threads.

This is a significant different outcome that must be mitigated somehow
to prevent an Execute After Free failure for Rust. DRM happens to be
safe because it ends up linking the driver module refcount to FD
lifetime. This also prevents unloading the driver (bad!!).

However, for instance, you can't rely on the module reference count
with work queues, so this scheme is not generally applicable.

Jason

On Tue, Feb 25, 2025 at 08:49:16PM -0400, Jason Gunthorpe
wrote:
> I'm pointing out the fundamental different in approachs. The typical
> widely used pattern results in __device_release_driver() completing
> with no concurrent driver code running.
Typically yes, but there are exceptions, such as DRM.
> 
> DRM achieves this, in part, by using drm_dev_unplug().
No, DRM can have concurrent driver code running, which is why drm_dev_enter()
returns whether the device is unplugged already, such that subsequent
operations, (e.g. I/O) can be omitted.
> 
> The Rust approach ends up with __device_release_driver() completing
> and leaving driver code still running in other threads.
No, this has nothing to do with Rust device / driver or I/O abstractions.

It entirely depends on the driver you implement. If you register a DRM device,
then yes, there may be concurrent driver code running after
__device_release_driver() completes. But this is specific to the DRM
implementation, *not* to Rust.

Again, the reason a pci::Bar needs to be revocable in Rust is that we can't
have
the driver potentially keep the pci::Bar alive (or even access it) after the
device is unbound.

A driver can also be unbound without the module being removed, and if the driver
would be able to keep the pci::Bar alive, it would mean that the resource region
is not freed and the MMIO mapping is not unmapped, because the resource region
and the MMIO mapping is bound to the lifetime of the pci::Bar object. This would
not be acceptable for a Rust driver.

That this also comes in handy for subsystems like DRM, where we could have
attempts to access to the pci::Bar object after the device is unbound by design,
can be seen as a nice side effect.