Virtualization - Oct 2023 - [PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct

On Tue, 2023-09-26 at 12:44 +0800, Jason Wang wrote:
> On Tue, Sep 12, 2023 at 9:02?PM Dragos Tatulea <dtatulea at
nvidia.com> wrote:
> > 
> > This patch adapts the mr creation/deletion code to be able to work
with
> > any given mr struct pointer. All the APIs are adapted to take an extra
> > parameter for the mr.
> > 
> > mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore.
The
> > check is done in the caller instead (mlx5_set_map).
> > 
> > This change is needed for a followup patch which will introduce an
> > additional mr for the vq descriptor data.
> > 
> > Signed-off-by: Dragos Tatulea <dtatulea at nvidia.com>
> > ---
> 
> Thinking of this decoupling I think I have a question.
> 
> We advertise 2 address spaces and 2 groups. So we actually don't know
> for example which address spaces will be used by dvq.
> 
> And actually we allow the user space to do something like
> 
> set_group_asid(dvq_group, 0)
> set_map(0)
> set_group_asid(dvq_group, 1)
> set_map(1)
> 
> I wonder if the decoupling like this patch can work and why.
> 
This scenario could indeed work. Especially if you look at the 13'th patch
[0]
where hw support is added. Are you wondering if this should work at all or if it
should be blocked?
> It looks to me the most easy way is to let each AS be backed by an MR.
> Then we don't even need to care about the dvq, cvq.
That's what this patch series dowes.

Thanks,
Dragos

[0]https://lore.kernel.org/virtualization/20230912130132.561193-14-dtatulea at
nvidia.com/T/#u

On Tue, Sep 26, 2023 at 3:21?PM Dragos Tatulea <dtatulea at nvidia.com>
wrote:
>
> On Tue, 2023-09-26 at 12:44 +0800, Jason Wang wrote:
> > On Tue, Sep 12, 2023 at 9:02?PM Dragos Tatulea <dtatulea at
nvidia.com> wrote:
> > >
> > > This patch adapts the mr creation/deletion code to be able to
work with
> > > any given mr struct pointer. All the APIs are adapted to take an
extra
> > > parameter for the mr.
> > >
> > > mlx5_vdpa_create/delete_mr doesn't need a ASID parameter
anymore. The
> > > check is done in the caller instead (mlx5_set_map).
> > >
> > > This change is needed for a followup patch which will introduce
an
> > > additional mr for the vq descriptor data.
> > >
> > > Signed-off-by: Dragos Tatulea <dtatulea at nvidia.com>
> > > ---
> >
> > Thinking of this decoupling I think I have a question.
> >
> > We advertise 2 address spaces and 2 groups. So we actually don't
know
> > for example which address spaces will be used by dvq.
> >
> > And actually we allow the user space to do something like
> >
> > set_group_asid(dvq_group, 0)
> > set_map(0)
> > set_group_asid(dvq_group, 1)
> > set_map(1)
> >
> > I wonder if the decoupling like this patch can work and why.
> >
> This scenario could indeed work. Especially if you look at the 13'th
patch [0]
> where hw support is added. Are you wondering if this should work at all or
if it
> should be blocked?
It would be great if it can work with the following patches. But at
least for this patch, it seems not:

For example, what happens if we switch back to group 0 for dvq?

set_group_asid(dvq_group, 0)
set_map(0)
set_group_asid(dvq_group, 1)
set_map(1)
// here we destroy the mr created for asid 0
set_group_asid(dvq_group, 0)

Btw, if this is a new issue, I haven't checked whether or not it
exists before this series (if yes, we can fix on top).
>
> > It looks to me the most easy way is to let each AS be backed by an MR.
> > Then we don't even need to care about the dvq, cvq.
> That's what this patch series dowes.
Good to know this, I will review the series.

Thanks
>
> Thanks,
> Dragos
>
> [0]https://lore.kernel.org/virtualization/20230912130132.561193-14-dtatulea
at nvidia.com/T/#u