Dragos Tatulea
2023-Oct-08 12:04 UTC
[PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct
On Sun, 2023-10-08 at 12:25 +0800, Jason Wang wrote:> On Tue, Sep 26, 2023 at 3:21?PM Dragos Tatulea <dtatulea at nvidia.com> wrote: > > > > On Tue, 2023-09-26 at 12:44 +0800, Jason Wang wrote: > > > On Tue, Sep 12, 2023 at 9:02?PM Dragos Tatulea <dtatulea at nvidia.com> > > > wrote: > > > > > > > > This patch adapts the mr creation/deletion code to be able to work with > > > > any given mr struct pointer. All the APIs are adapted to take an extra > > > > parameter for the mr. > > > > > > > > mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore. The > > > > check is done in the caller instead (mlx5_set_map). > > > > > > > > This change is needed for a followup patch which will introduce an > > > > additional mr for the vq descriptor data. > > > > > > > > Signed-off-by: Dragos Tatulea <dtatulea at nvidia.com> > > > > --- > > > > > > Thinking of this decoupling I think I have a question. > > > > > > We advertise 2 address spaces and 2 groups. So we actually don't know > > > for example which address spaces will be used by dvq. > > > > > > And actually we allow the user space to do something like > > > > > > set_group_asid(dvq_group, 0) > > > set_map(0) > > > set_group_asid(dvq_group, 1) > > > set_map(1) > > > > > > I wonder if the decoupling like this patch can work and why. > > > > > This scenario could indeed work. Especially if you look at the 13'th patch > > [0] > > where hw support is added. Are you wondering if this should work at all or > > if it > > should be blocked? > > It would be great if it can work with the following patches. But at > least for this patch, it seems not: > > For example, what happens if we switch back to group 0 for dvq? > > set_group_asid(dvq_group, 0) > set_map(0) > set_group_asid(dvq_group, 1) > set_map(1) > // here we destroy the mr created for asid 0 > set_group_asid(dvq_group, 0) >If by destroy you mean .reset, I think it works: During .reset the mapping in ASID 0 is reset back to the DMA/pysical map (mlx5_vdpa_create_dma_mr). Am I missing something?> Btw, if this is a new issue, I haven't checked whether or not it > exists before this series (if yes, we can fix on top).> > > > > It looks to me the most easy way is to let each AS be backed by an MR. > > > Then we don't even need to care about the dvq, cvq. > > That's what this patch series dowes. > > Good to know this, I will review the series. >I was planning to spin a v3 with Eugenio's suggestions. Should I wait for your feedback before doing that? Thanks, Dragos
Jason Wang
2023-Oct-09 06:39 UTC
[PATCH 09/16] vdpa/mlx5: Allow creation/deletion of any given mr struct
On Sun, Oct 8, 2023 at 8:05?PM Dragos Tatulea <dtatulea at nvidia.com> wrote:> > On Sun, 2023-10-08 at 12:25 +0800, Jason Wang wrote: > > On Tue, Sep 26, 2023 at 3:21?PM Dragos Tatulea <dtatulea at nvidia.com> wrote: > > > > > > On Tue, 2023-09-26 at 12:44 +0800, Jason Wang wrote: > > > > On Tue, Sep 12, 2023 at 9:02?PM Dragos Tatulea <dtatulea at nvidia.com> > > > > wrote: > > > > > > > > > > This patch adapts the mr creation/deletion code to be able to work with > > > > > any given mr struct pointer. All the APIs are adapted to take an extra > > > > > parameter for the mr. > > > > > > > > > > mlx5_vdpa_create/delete_mr doesn't need a ASID parameter anymore. The > > > > > check is done in the caller instead (mlx5_set_map). > > > > > > > > > > This change is needed for a followup patch which will introduce an > > > > > additional mr for the vq descriptor data. > > > > > > > > > > Signed-off-by: Dragos Tatulea <dtatulea at nvidia.com> > > > > > --- > > > > > > > > Thinking of this decoupling I think I have a question. > > > > > > > > We advertise 2 address spaces and 2 groups. So we actually don't know > > > > for example which address spaces will be used by dvq. > > > > > > > > And actually we allow the user space to do something like > > > > > > > > set_group_asid(dvq_group, 0) > > > > set_map(0) > > > > set_group_asid(dvq_group, 1) > > > > set_map(1) > > > > > > > > I wonder if the decoupling like this patch can work and why. > > > > > > > This scenario could indeed work. Especially if you look at the 13'th patch > > > [0] > > > where hw support is added. Are you wondering if this should work at all or > > > if it > > > should be blocked? > > > > It would be great if it can work with the following patches. But at > > least for this patch, it seems not: > > > > For example, what happens if we switch back to group 0 for dvq? > > > > set_group_asid(dvq_group, 0) > > set_map(0) > > set_group_asid(dvq_group, 1) > > set_map(1) > > // here we destroy the mr created for asid 0 > > set_group_asid(dvq_group, 0) > > > If by destroy you mean .reset,It's not rest. During the second map, the mr is destroyed by mlx5_vdpa_change_map(). I think it works: During .reset the mapping in> ASID 0 is reset back to the DMA/pysical map (mlx5_vdpa_create_dma_mr). Am I > missing something? > > > Btw, if this is a new issue, I haven't checked whether or not it > > exists before this series (if yes, we can fix on top). > > > > > > > > It looks to me the most easy way is to let each AS be backed by an MR. > > > > Then we don't even need to care about the dvq, cvq. > > > That's what this patch series dowes. > > > > Good to know this, I will review the series. > > > I was planning to spin a v3 with Eugenio's suggestions. Should I wait for your > feedback before doing that?You can post v3 and we can move the discussion there if you wish. Thanks> > Thanks, > Dragos