John Hubbard
2025-Sep-30 22:07 UTC
[PATCH 0/2] rust: pci: expose is_virtfn() and reject VFs in nova-core
Post-Kangrejos, the approach for NovaCore + VFIO has changed a bit: the idea now is that VFIO drivers, for NVIDIA GPUs that are supported by NovaCore, should bind directly to the GPU's VFs. (An earlier idea was to let NovaCore bind to the VFs, and then have NovaCore call into the upper (VFIO) module via Aux Bus, but this turns out to be awkward and is no longer in favor.) So, in order to support that: Nova-core must only bind to Physical Functions (PFs) and regular PCI devices, not to Virtual Functions (VFs) created through SR-IOV. Add a method to check if a PCI device is a Virtual Function (VF). This allows Rust drivers to determine whether a device is a VF created through SR-IOV. This is required in order to implement VFIO, because drivers such as NovaCore must only bind to Physical Functions (PFs) or regular PCI devices. The VFs must be left unclaimed, so that a VFIO kernel module can claim them. Use is_virtfn() in NovaCore, in preparation for it to be used in a VFIO scenario. I've based this on top of today's driver-core-next [1], because the first patch belongs there, and the second patch applies cleanly to either driver-core-next or drm-rust-next. So this seems like the easiest to work with. [1] https://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core.git/ John Hubbard (2): rust: pci: add is_virtfn(), to check for VFs gpu: nova-core: reject binding to SR-IOV Virtual Functions drivers/gpu/nova-core/driver.rs | 5 +++++ rust/kernel/pci.rs | 6 ++++++ 2 files changed, 11 insertions(+) base-commit: 6d97171ac6585de698df019b0bfea3f123fd8385 -- 2.51.0
John Hubbard
2025-Sep-30 22:07 UTC
[PATCH 1/2] rust: pci: add is_virtfn(), to check for VFs
Add a method to check if a PCI device is a Virtual Function (VF). This allows Rust drivers to determine whether a device is a VF created through SR-IOV. This is required in order to implement VFIO, because drivers such as NovaCore must only bind to Physical Functions (PFs) or regular PCI devices. The VFs must be left unclaimed, so that a VFIO kernel module can claim them. is_virtfn() returns true if the device's is_virtfn flag is set, matching the behavior of the C code. Signed-off-by: John Hubbard <jhubbard at nvidia.com> --- rust/kernel/pci.rs | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs index 7fcc5f6022c1..476b80f05905 100644 --- a/rust/kernel/pci.rs +++ b/rust/kernel/pci.rs @@ -496,6 +496,12 @@ pub fn resource_start(&self, bar: u32) -> Result<bindings::resource_size_t> { Ok(unsafe { bindings::pci_resource_start(self.as_raw(), bar.try_into()?) }) } + /// Returns true if this device is a Virtual Function (VF). + pub fn is_virtfn(&self) -> bool { + // SAFETY: `self.as_raw` is a valid pointer to a `struct pci_dev`. + unsafe { (*self.as_raw()).is_virtfn() != 0 } + } + /// Returns the size of the given PCI bar resource. pub fn resource_len(&self, bar: u32) -> Result<bindings::resource_size_t> { if !Bar::index_is_valid(bar) { -- 2.51.0
John Hubbard
2025-Sep-30 22:07 UTC
[PATCH 2/2] gpu: nova-core: reject binding to SR-IOV Virtual Functions
Nova-core must only bind to Physical Functions (PFs) and regular PCI devices, not to Virtual Functions (VFs) created through SR-IOV. Detect VFs using the newly added is_virtfn() method, and leave them unclaimed. This allows a VFIO kernel module to claim the VFs instead. Signed-off-by: John Hubbard <jhubbard at nvidia.com> --- drivers/gpu/nova-core/driver.rs | 5 +++++ 1 file changed, 5 insertions(+) diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs index 5d23a91f51dd..004c15763694 100644 --- a/drivers/gpu/nova-core/driver.rs +++ b/drivers/gpu/nova-core/driver.rs @@ -54,6 +54,11 @@ impl pci::Driver for NovaCore { fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self>>> { dev_dbg!(pdev.as_ref(), "Probe Nova Core GPU driver.\n"); + // NovaCore must only bind to Physical Functions (PFs), not Virtual Functions (VFs) + if pdev.is_virtfn() { + return Err(ENODEV); + } + pdev.enable_device_mem()?; pdev.set_master(); -- 2.51.0
Alexandre Courbot
2025-Oct-01 00:26 UTC
[PATCH 0/2] rust: pci: expose is_virtfn() and reject VFs in nova-core
On Wed Oct 1, 2025 at 7:07 AM JST, John Hubbard wrote:> Post-Kangrejos, the approach for NovaCore + VFIO has changed a bit: the > idea now is that VFIO drivers, for NVIDIA GPUs that are supported by > NovaCore, should bind directly to the GPU's VFs. (An earlier idea was to > let NovaCore bind to the VFs, and then have NovaCore call into the upper > (VFIO) module via Aux Bus, but this turns out to be awkward and is no > longer in favor.) So, in order to support that: > > Nova-core must only bind to Physical Functions (PFs) and regular PCI > devices, not to Virtual Functions (VFs) created through SR-IOV.Naive question: will guests also see the passed-through VF as a VF? If so, wouldn't this change also prevents guests from using Nova?
Alistair Popple
2025-Oct-01 00:29 UTC
[PATCH 0/2] rust: pci: expose is_virtfn() and reject VFs in nova-core
On 2025-10-01 at 08:07 +1000, John Hubbard <jhubbard at nvidia.com> wrote...> Post-Kangrejos, the approach for NovaCore + VFIO has changed a bit: the > idea now is that VFIO drivers, for NVIDIA GPUs that are supported by > NovaCore, should bind directly to the GPU's VFs. (An earlier idea was to > let NovaCore bind to the VFs, and then have NovaCore call into the upper > (VFIO) module via Aux Bus, but this turns out to be awkward and is no > longer in favor.) So, in order to support that: > > Nova-core must only bind to Physical Functions (PFs) and regular PCI > devices, not to Virtual Functions (VFs) created through SR-IOV. > > Add a method to check if a PCI device is a Virtual Function (VF). This > allows Rust drivers to determine whether a device is a VF created > through SR-IOV. This is required in order to implement VFIO, because > drivers such as NovaCore must only bind to Physical Functions (PFs) or > regular PCI devices. The VFs must be left unclaimed, so that a VFIO > kernel module can claim them.Curiously based on a quick glance I didn't see any other drivers doing this which makes me wonder why we're different here. But it seems likely their virtual functions are supported by the same driver rather than requiring a different VF specific driver (or I glanced too quickly!). I'm guessing the proposal is to fail the probe() function in nova-core for the VFs - I'm not sure but does the driver core continue to try probing other drivers if one fails probe()? It seems like this would be something best filtered on in the device id table, although I understand that's not possible today.> Use is_virtfn() in NovaCore, in preparation for it to be used in a VFIO > scenario. > > I've based this on top of today's driver-core-next [1], because the > first patch belongs there, and the second patch applies cleanly to either > driver-core-next or drm-rust-next. So this seems like the easiest to > work with. > > > [1] https://git.kernel.org/pub/scm/linux/kernel/git/driver-core/driver-core.git/ > > John Hubbard (2): > rust: pci: add is_virtfn(), to check for VFs > gpu: nova-core: reject binding to SR-IOV Virtual Functions > > drivers/gpu/nova-core/driver.rs | 5 +++++ > rust/kernel/pci.rs | 6 ++++++ > 2 files changed, 11 insertions(+) > > > base-commit: 6d97171ac6585de698df019b0bfea3f123fd8385 > -- > 2.51.0 >
Alistair Popple
2025-Oct-01 00:30 UTC
[PATCH 1/2] rust: pci: add is_virtfn(), to check for VFs
On 2025-10-01 at 08:07 +1000, John Hubbard <jhubbard at nvidia.com> wrote...> Add a method to check if a PCI device is a Virtual Function (VF). This > allows Rust drivers to determine whether a device is a VF created > through SR-IOV. This is required in order to implement VFIO, because > drivers such as NovaCore must only bind to Physical Functions (PFs) or > regular PCI devices. The VFs must be left unclaimed, so that a VFIO > kernel module can claim them. > > is_virtfn() returns true if the device's is_virtfn flag is set, matching > the behavior of the C code.Seems fine to me so please add: Reviewed-by: Alistair Popple <apopple at nvidia.com>> Signed-off-by: John Hubbard <jhubbard at nvidia.com> > --- > rust/kernel/pci.rs | 6 ++++++ > 1 file changed, 6 insertions(+) > > diff --git a/rust/kernel/pci.rs b/rust/kernel/pci.rs > index 7fcc5f6022c1..476b80f05905 100644 > --- a/rust/kernel/pci.rs > +++ b/rust/kernel/pci.rs > @@ -496,6 +496,12 @@ pub fn resource_start(&self, bar: u32) -> Result<bindings::resource_size_t> { > Ok(unsafe { bindings::pci_resource_start(self.as_raw(), bar.try_into()?) }) > } > > + /// Returns true if this device is a Virtual Function (VF). > + pub fn is_virtfn(&self) -> bool { > + // SAFETY: `self.as_raw` is a valid pointer to a `struct pci_dev`. > + unsafe { (*self.as_raw()).is_virtfn() != 0 } > + } > + > /// Returns the size of the given PCI bar resource. > pub fn resource_len(&self, bar: u32) -> Result<bindings::resource_size_t> { > if !Bar::index_is_valid(bar) { > -- > 2.51.0 >
Alistair Popple
2025-Oct-01 00:33 UTC
[PATCH 2/2] gpu: nova-core: reject binding to SR-IOV Virtual Functions
On 2025-10-01 at 08:07 +1000, John Hubbard <jhubbard at nvidia.com> wrote...> Nova-core must only bind to Physical Functions (PFs) and regular PCI > devices, not to Virtual Functions (VFs) created through SR-IOV. > > Detect VFs using the newly added is_virtfn() method, and leave them > unclaimed. This allows a VFIO kernel module to claim the VFs instead.See my comments on the cover letter, but does the kernel driver core continue trying to probe other drivers if this one failed? Or does it just give up? If it just gives up I'm not sure this really helps much, although I suppose if NovaCore can't use the VFs for anything useful anyway then it probably should fail.> Signed-off-by: John Hubbard <jhubbard at nvidia.com> > --- > drivers/gpu/nova-core/driver.rs | 5 +++++ > 1 file changed, 5 insertions(+) > > diff --git a/drivers/gpu/nova-core/driver.rs b/drivers/gpu/nova-core/driver.rs > index 5d23a91f51dd..004c15763694 100644 > --- a/drivers/gpu/nova-core/driver.rs > +++ b/drivers/gpu/nova-core/driver.rs > @@ -54,6 +54,11 @@ impl pci::Driver for NovaCore { > fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self>>> { > dev_dbg!(pdev.as_ref(), "Probe Nova Core GPU driver.\n"); > > + // NovaCore must only bind to Physical Functions (PFs), not Virtual Functions (VFs) > + if pdev.is_virtfn() { > + return Err(ENODEV); > + } > + > pdev.enable_device_mem()?; > pdev.set_master(); > > -- > 2.51.0 >