Nouveau - Jul 2025 - [PATCH v3 2/7] gpu: nova-core: Clarify sysmembar operations

From: Joel Fernandes <joelagnelf at nvidia.com>

sysmembar is a critical operation that the GSP falcon needs to perform
in the reset sequence. Add some code comments to clarify.

[acourbot at nvdidia.com: move relevant documentation to SysmemFlush type]

Signed-off-by: Joel Fernandes <joelagnelf at nvidia.com>
Signed-off-by: Alexandre Courbot <acourbot at nvidia.com>
---
 drivers/gpu/nova-core/fb.rs   | 10 ++++++++++
 drivers/gpu/nova-core/gpu.rs  |  3 +--
 drivers/gpu/nova-core/regs.rs |  2 ++
 3 files changed, 13 insertions(+), 2 deletions(-)

diff --git a/drivers/gpu/nova-core/fb.rs b/drivers/gpu/nova-core/fb.rs
index
172b4a12ba2afc05860cc004fd1f0154402f467a..02c44c79cef500462bab10ea18678749f72404ee
100644
--- a/drivers/gpu/nova-core/fb.rs
+++ b/drivers/gpu/nova-core/fb.rs
@@ -17,6 +17,16 @@
 /// Type holding the sysmem flush memory page, a page of memory to be written
into the
 /// `NV_PFB_NISO_FLUSH_SYSMEM_ADDR*` registers and used to maintain memory
coherency.
 ///
+/// A system memory page is required for `sysmembar`, which is a GPU-initiated
hardware
+/// memory-barrier operation that flushes all pending GPU-side memory writes
that were done through
+/// PCIE to system memory. It is required for falcons to be reset as the reset
operation involves a
+/// reset handshake. When the falcon acknowledges a reset, it writes into
system memory. To ensure
+/// this write is visible to the host and prevent driver timeouts, the falcon
must perform a
+/// sysmembar operation to flush its writes.
+///
+/// Because of this, the sysmem flush memory page must be registered as early
as possible during
+/// driver initialization, and before any falcon is reset.
+//
 /// Users are responsible for manually calling [`Self::unregister`] before
dropping this object,
 /// otherwise the GPU might still use it even after it has been freed.
 pub(crate) struct SysmemFlush {
diff --git a/drivers/gpu/nova-core/gpu.rs b/drivers/gpu/nova-core/gpu.rs
index
8e32af16b669ca773e63e184d34c3e0427bc9b76..72d40b0124f0c1a2a381484172c289af523511df
100644
--- a/drivers/gpu/nova-core/gpu.rs
+++ b/drivers/gpu/nova-core/gpu.rs
@@ -170,7 +170,7 @@ pub(crate) struct Gpu {
     bar: Devres<Bar0>,
     fw: Firmware,
     /// System memory page required for flushing all pending GPU-side memory
writes done through
-    /// PCIE into system memory.
+    /// PCIE into system memory, via sysmembar (A GPU-initiated HW
memory-barrier operation).
     sysmem_flush: SysmemFlush,
 }
 
@@ -283,7 +283,6 @@ pub(crate) fn new(
         gfw::wait_gfw_boot_completion(bar)
             .inspect_err(|_| dev_err!(pdev.as_ref(), "GFW boot did not
complete"))?;
 
-        // System memory page required for sysmembar to properly flush into
system memory.
         let sysmem_flush = SysmemFlush::register(pdev.as_ref(), bar,
spec.chipset)?;
 
         let gsp_falcon = Falcon::<Gsp>::new(
diff --git a/drivers/gpu/nova-core/regs.rs b/drivers/gpu/nova-core/regs.rs
index
ce08fab8fa96fcacf6947512ebdf4975ebd8772c..e94c1d2190f352e9316937dd3165cc2a390e0ed2
100644
--- a/drivers/gpu/nova-core/regs.rs
+++ b/drivers/gpu/nova-core/regs.rs
@@ -51,6 +51,8 @@ pub(crate) fn chipset(self) -> Result<Chipset> {
 
 /* PFB */
 
+// These two registers together hold the physical system memory address that is
used by the GPU for
+// perform sysmembar operation (see [`crate::fb::SysmemFlush`]).
 register!(NV_PFB_NISO_FLUSH_SYSMEM_ADDR @ 0x00100c10 {
     31:0    adr_39_08 as u32;
 });

-- 
2.50.0

On Wed, Jul 2, 2025 at 1:05?PM Alexandre Courbot <acourbot at nvidia.com>
wrote:
>
> +/// driver initialization, and before any falcon is reset.
> +//
>  /// Users are responsible for manually calling [`Self::unregister`] before
dropping this object,
Missing slash.

Added the possibility of catching this automatically in e.g. `rustdoc` to:

    https://github.com/Rust-for-Linux/linux/issues/350
> +// These two registers together hold the physical system memory address
that is used by the GPU for
> +// perform sysmembar operation (see [`crate::fb::SysmemFlush`]).
This is a normal comment, so the intra-doc link will (sadly) not do anything.

Cheers,
Miguel