John Hubbard
2025-Oct-20 19:36 UTC
[PATCH 4/7] docs: gpu: nova-core: Document the PRAMIN aperture mechanism
On 10/20/25 11:55 AM, Joel Fernandes wrote: ...> +Logically, the PRAMIN aperture mechanism is implemented by the GPU's PBUS (PCIe Bus Controller Unit)> +and provides a CPU-accessible window into VRAM through the PCIe interface:: > + > + +-----------------+ PCIe +------------------------------+ > + | CPU |<----------->| GPU | > + +-----------------+ | | > + | +----------------------+ | > + | | PBUS | | > + | | (Bus Controller) | | > + | | | | > + | | +--------------.<------------ (window always starts at > + | | | PRAMIN | | | BAR0 + 0x700000)Quick question: does "window always starts at" actually mean "windows is always initialized to" ? Or something else? thanks, -- John Hubbard> + | | | Window | | | > + | | | (1MB) | | | > + | | +--------------+ | | > + | | | | | > + | +---------|------------+ | > + | | | > + | v | > + | .----------------------.<------------ (Program PRAMIN to any > + | | VRAM | | 64KB VRAM physical boundary) > + | | (Several GBs) | | > + | | | | > + | | FB[0x000000000000] | | > + | | ... | | > + | | FB[0x7FFFFFFFFFF] | | > + | +----------------------+ | > + +------------------------------+ > + > +PBUS (PCIe Bus Controller) among other things is responsible in the GPU for handling MMIO > +accesses to the BAR registers. > + > +PRAMIN Window Operation > +======================> + > +The PRAMIN window provides a 1MB sliding aperture that can be repositioned over > +the entire VRAM address space using the NV_PBUS_BAR0_WINDOW register. > + > +Window Control Mechanism > +------------------------- > + > +The window position is controlled via the PBUS BAR0_WINDOW register:: > + > + NV_PBUS_BAR0_WINDOW Register > + +-----+-----+--------------------------------------+ > + |31-26|25-24| 23-0 | > + | |TARG | BASE_ADDR | > + | | ET | (bits 39:16 of VRAM address) | > + +-----+-----+--------------------------------------+ > + > + TARGET field values: > + - 0x0: VID_MEM (Video Memory / VRAM) > + - 0x1: SYS_MEM_COHERENT (Coherent system memory) > + - 0x2: SYS_MEM_NONCOHERENT (Non-coherent system memory) > + > +64KB Alignment Requirement > +--------------------------- > + > +The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is enforced > +by the BASE_ADDR field representing bits [39:16] of the target address:: > + > + VRAM Address Calculation: > + actual_vram_addr = (BASE_ADDR << 16) + pramin_offset > + Where: > + - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0] > + - pramin_offset: 20-bit offset within PRAMIN window [0x00000-0xFFFFF] > + Example Window Positioning: > + +---------------------------------------------------------+ > + | VRAM Space | > + | | > + | 0x000000000 +-----------------+ <-- 64KB aligned | > + | | PRAMIN Window | | > + | | (1MB) | | > + | 0x0000FFFFF +-----------------+ | > + | | > + | | ^ | > + | | | Window can slide | > + | v | to any 64KB boundary | > + | | > + | 0x123400000 +-----------------+ <-- 64KB aligned | > + | | PRAMIN Window | | > + | | (1MB) | | > + | 0x1234FFFFF +-----------------+ | > + | | > + | ... | > + | | > + | 0x7FFFF0000 +-----------------+ <-- 64KB aligned | > + | | PRAMIN Window | | > + | | (1MB) | | > + | 0x7FFFFFFFF +-----------------+ | > + +---------------------------------------------------------+ > diff --git a/Documentation/gpu/nova/index.rst b/Documentation/gpu/nova/index.rst > index 46302daace34..e77d3ee336a4 100644 > --- a/Documentation/gpu/nova/index.rst > +++ b/Documentation/gpu/nova/index.rst > @@ -33,3 +33,4 @@ vGPU manager VFIO driver and the nova-drm driver. > core/fwsec > core/falcon > core/msgq > + core/pramin
Joel Fernandes
2025-Oct-20 19:48 UTC
[PATCH 4/7] docs: gpu: nova-core: Document the PRAMIN aperture mechanism
> On Oct 20, 2025, at 3:36?PM, John Hubbard <jhubbard at nvidia.com> wrote: > > ?On 10/20/25 11:55 AM, Joel Fernandes wrote: > ...> +Logically, the PRAMIN aperture mechanism is implemented by the GPU's PBUS (PCIe Bus Controller Unit) >> +and provides a CPU-accessible window into VRAM through the PCIe interface:: >> + >> + +-----------------+ PCIe +------------------------------+ >> + | CPU |<----------->| GPU | >> + +-----------------+ | | >> + | +----------------------+ | >> + | | PBUS | | >> + | | (Bus Controller) | | >> + | | | | >> + | | +--------------.<------------ (window always starts at >> + | | | PRAMIN | | | BAR0 + 0x700000) > > Quick question: does "window always starts at" actually mean "windows > is always initialized to" ? Or something else?It means start of the 1MB window (offset 0) is accessed from the BAR at that location. Thanks.> > > thanks, > -- > John Hubbard > >> + | | | Window | | | >> + | | | (1MB) | | | >> + | | +--------------+ | | >> + | | | | | >> + | +---------|------------+ | >> + | | | >> + | v | >> + | .----------------------.<------------ (Program PRAMIN to any >> + | | VRAM | | 64KB VRAM physical boundary) >> + | | (Several GBs) | | >> + | | | | >> + | | FB[0x000000000000] | | >> + | | ... | | >> + | | FB[0x7FFFFFFFFFF] | | >> + | +----------------------+ | >> + +------------------------------+ >> + >> +PBUS (PCIe Bus Controller) among other things is responsible in the GPU for handling MMIO >> +accesses to the BAR registers. >> + >> +PRAMIN Window Operation >> +======================>> + >> +The PRAMIN window provides a 1MB sliding aperture that can be repositioned over >> +the entire VRAM address space using the NV_PBUS_BAR0_WINDOW register. >> + >> +Window Control Mechanism >> +------------------------- >> + >> +The window position is controlled via the PBUS BAR0_WINDOW register:: >> + >> + NV_PBUS_BAR0_WINDOW Register >> + +-----+-----+--------------------------------------+ >> + |31-26|25-24| 23-0 | >> + | |TARG | BASE_ADDR | >> + | | ET | (bits 39:16 of VRAM address) | >> + +-----+-----+--------------------------------------+ >> + >> + TARGET field values: >> + - 0x0: VID_MEM (Video Memory / VRAM) >> + - 0x1: SYS_MEM_COHERENT (Coherent system memory) >> + - 0x2: SYS_MEM_NONCOHERENT (Non-coherent system memory) >> + >> +64KB Alignment Requirement >> +--------------------------- >> + >> +The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is enforced >> +by the BASE_ADDR field representing bits [39:16] of the target address:: >> + >> + VRAM Address Calculation: >> + actual_vram_addr = (BASE_ADDR << 16) + pramin_offset >> + Where: >> + - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0] >> + - pramin_offset: 20-bit offset within PRAMIN window [0x00000-0xFFFFF] >> + Example Window Positioning: >> + +---------------------------------------------------------+ >> + | VRAM Space | >> + | | >> + | 0x000000000 +-----------------+ <-- 64KB aligned | >> + | | PRAMIN Window | | >> + | | (1MB) | | >> + | 0x0000FFFFF +-----------------+ | >> + | | >> + | | ^ | >> + | | | Window can slide | >> + | v | to any 64KB boundary | >> + | | >> + | 0x123400000 +-----------------+ <-- 64KB aligned | >> + | | PRAMIN Window | | >> + | | (1MB) | | >> + | 0x1234FFFFF +-----------------+ | >> + | | >> + | ... | >> + | | >> + | 0x7FFFF0000 +-----------------+ <-- 64KB aligned | >> + | | PRAMIN Window | | >> + | | (1MB) | | >> + | 0x7FFFFFFFF +-----------------+ | >> + +---------------------------------------------------------+ >> diff --git a/Documentation/gpu/nova/index.rst b/Documentation/gpu/nova/index.rst >> index 46302daace34..e77d3ee336a4 100644 >> --- a/Documentation/gpu/nova/index.rst >> +++ b/Documentation/gpu/nova/index.rst >> @@ -33,3 +33,4 @@ vGPU manager VFIO driver and the nova-drm driver. >> core/fwsec >> core/falcon >> core/msgq >> + core/pramin > >