Virtualization - Sep 2023 - [PATCH vfio 11/11] vfio/virtio: Introduce a vfio driver over virtio devices

On Fri, Sep 22, 2023 at 8:11?PM Jason Gunthorpe <jgg at nvidia.com>
wrote:
>
> On Fri, Sep 22, 2023 at 11:01:23AM +0800, Jason Wang wrote:
>
> > > Even when it does, there is no real use case to live migrate a
> > > virtio-net function from, say, AWS to GCP.
> >
> > It can happen inside a single cloud vendor. For some reasons, DPU must
> > be purchased from different vendors. And vDPA has been used in that
> > case.
>
> Nope, you misunderstand the DPU scenario.
>
> Look at something like vmware DPU enablement. vmware runs the software
> side of the DPU and all their supported DPU HW, from every vendor,
> generates the same PCI functions on the x86. They are the same because
> the same software on the DPU side is creating them.
>
> There is no reason to put a mediation layer in the x86 if you also
> control the DPU.
>
> Cloud vendors will similarly use DPUs to create a PCI functions that
> meet the cloud vendor's internal specification.
This can only work if:

1) the internal specification has finer garin than virtio spec
2) so it can define what is not implemented in the virtio spec (like
migration and compatibility)

All of the above doesn't seem to be possible or realistic now, and it
actually has a risk to be not compatible with virtio spec. In the
future when virtio has live migration supported, they want to be able
to migrate between virtio and vDPA.

As I said, vDPA has been used for cross vendor live migration for a while.
> Regardless of DPU
> vendor.
>
> Fundamentally if you control the DPU SW and the hypervisor software
> you do not need hypervisor meditation because everything you could do
> in hypervisor mediation can just be done in the DPU. Putting it in the
> DPU is better in every regard.
>
> So, as I keep saying, in this scenario the goal is no mediation in the
> hypervisor.
That's pretty fine, but I don't think trapping + relying is not
mediation. Does it really matter what happens after trapping?
> It is pointless, everything you think you need to do there
> is actually already being done in the DPU.
Well, migration or even Qemu could be offloaded to DPU as well. If
that's the direction that's pretty fine.

Thanks
>
> Jason
>