Nouveau - Oct 2025 - [PATCH v2 1/2] rust: pci: skip probing VFs if driver doesn't support VFs

On 10/2/25 8:05 PM, Jason Gunthorpe wrote:
> On Thu, Oct 02, 2025 at 10:49:21AM -0700, John Hubbard wrote:
>>> Forgot to add: But I think Zhi explained that this is not necessary
and can be
>>> controlled by the VFIO driver, i.e. the PCI driver that binds to
the VF itself.
>>
>> Yes, this is the direction that I originally (3 whole days ago, haha)
had in mind,
>> after talking with Zhi and a few others: nova-core handles PFs, and the
VFIO driver
>> handles the VFs, and use the "is virtual" logic to sort them
out.
> 
> To be clear, no matter what the VFIO driver bound to the VF should not
> become entangled with any aux devices.
> 
> The VFIO VF driver uses pci_iov_get_pf_drvdata() to reach into the PF
> to request the PF's help. Eg for live migration or things of that
> nature.
Ick! The VF driver should never mess with the PF driver's private data.

Instead the PF driver should provide an API for the VF driver to get things done
on behalf.

It also has the implication that we need to guarantee that PF driver unbind will
also unbind all VFs, but we need this guarantee anyways. I.e. when the VFIO
driver calls into nova-core we want the guarantee that we're in a scope
where
the PF driver is bound.
> My point here is that generally we don't put profiling code in the
> VFIO driver and then use pci_iov_get_pf_drvdata() to access the PF do
> actually do the profiling.
> 
> The VF cannot/should not control profiling of itself - that would be a
> security problem once it is assigned to a VM.
As mentioned above, if at all I think the PF driver has to provide an API for
that.
> So the profiling resides entirely inside the PF world and should
> operate without VFIO.
Perfectly fine with me, I'm open to both approaches.

On Thu, Oct 02, 2025 at 08:17:10PM +0200, Danilo Krummrich
wrote:
> On 10/2/25 8:05 PM, Jason Gunthorpe wrote:
> > On Thu, Oct 02, 2025 at 10:49:21AM -0700, John Hubbard wrote:
> >>> Forgot to add: But I think Zhi explained that this is not
necessary and can be
> >>> controlled by the VFIO driver, i.e. the PCI driver that binds
to the VF itself.
> >>
> >> Yes, this is the direction that I originally (3 whole days ago,
haha) had in mind,
> >> after talking with Zhi and a few others: nova-core handles PFs,
and the VFIO driver
> >> handles the VFs, and use the "is virtual" logic to sort
them out.
> > 
> > To be clear, no matter what the VFIO driver bound to the VF should not
> > become entangled with any aux devices.
> > 
> > The VFIO VF driver uses pci_iov_get_pf_drvdata() to reach into the PF
> > to request the PF's help. Eg for live migration or things of that
> > nature.
> 
> Ick! The VF driver should never mess with the PF driver's private data.
> Instead the PF driver should provide an API for the VF driver to get things
done
> on behalf.
This exactly how this function is used.

The core PF driver provides an API:

struct mlx5_core_dev *mlx5_vf_get_core_dev(struct pci_dev *pdev)

Which takes in the VF as pdev and internally it invokes:

	mdev = pci_iov_get_pf_drvdata(pdev, &mlx5_core_driver);

Which asserts the PF is bound to the right driver, which asserts the
format of the drvdata is known, and now we have a proper handle to use
in the rest of API surface the VF driver will use.

By forcing the ops into the signature we strongly encourage people to
follow this design pattern and provide APIs, otherwise you have to
export the ops to call pci_iov_get_pf_drvdata()

There really isn't another good way to obtain the 'struct
mlx5_core_dev' from a simple VF pci_dev.
> It also has the implication that we need to guarantee that PF driver unbind
will
> also unbind all VFs, but we need this guarantee anyways. 
This is another reason why pci_iov_get_pf_drvdata() exists.

/**
 * pci_iov_get_pf_drvdata - Return the drvdata of a PF
 * @dev: VF pci_dev
 * @pf_driver: Device driver required to own the PF
 *
 * This must be called from a context that ensures that a VF driver is attached.
 * The value returned is invalid once the VF driver completes its remove()
 * callback.
 *
 * Locking is achieved by the driver core. A VF driver cannot be probed until
 * pci_enable_sriov() is called and pci_disable_sriov() does not return until
 * all VF drivers have completed their remove().
 *
 * The PF driver must call pci_disable_sriov() before it begins to destroy the
 * drvdata.
 */

Meaning nova-core has to guarentee to call pci_disable_sriov() before
remove completes or before a failing probe returns as part of
implementing SRIOV support.

Userspace cannot create VFs until those calls are done.

Jason