Virtualization - Aug 2021 - [PATCH linux-next v3 0/6] vdpa: enable user to set mac, mtu

On Wed, Aug 18, 2021 at 11:15 AM Parav Pandit <parav at nvidia.com>
wrote:
>
>
>
> > From: Michael S. Tsirkin <mst at redhat.com>
> > Sent: Tuesday, August 17, 2021 2:24 AM
> >
> > On Mon, Aug 09, 2021 at 09:51:49AM +0000, Parav Pandit wrote:
> > > > From: Michael S. Tsirkin <mst at redhat.com>
> > > > Sent: Monday, August 9, 2021 3:10 PM
> > > >
> > > > On Fri, Aug 06, 2021 at 08:55:56AM +0000, Parav Pandit
wrote:
> > > > >
> > > > >
> > > > > >
> > > > > > The point is to try and not reinvent a dedicated
vpda interface
> > > > > > where a generic one exits.
> > > > > > E.g. for phy things such as mac speed etc, I think
most people
> > > > > > are using ethtool things right?
> > > > >
> > > > > As you know vdpa is the backend device for the
front-end netdevice
> > > > accessed by the ethtool.
> > > > > vdpa management tool here is composing the vdpa device.
> > > > >
> > > > > For example creator (hypervisor) of the vdpa devices
knows that a
> > > > > guest VM is given 4 vcpus, So hypervisor creates a vdpa
devices
> > > > > with config space layout as, max_virtqueue_pairs = 4.
> > > > > And the MAC address chosen by hypervisor in mac[6].
> > > > >
> > > > > Guest VM ethtool can still chose to use less number of
channels.
> > > > >
> > > > > Typically,
> > > > > ethtool is for guest VM.
> > > > > vdpa device is in hypevisor.
> > > > >
> > > > > How can hypervisor compose a vdpa device without any
tool?
> > > > > How can it tell ethtool, what is supported and what are
the defaults?
> > > > >
> > > > > I must be misunderstanding your comment about ethtool.
> > > > > Can you please explain?
> > > >
> > > >
> > > > I am basically saying that we probably want to be able to
change MAC
> > > > of a VDPA device on the host without desroying and
recreating the
> > > > device as long as it's not in use.
> > > Ok. I understood your comment now.
> > > Yes, this was the objective which is why they are present as
independent
> > config knob.
> > > Jason was suggesting to have them as creation only knobs, which
requires
> > recreate.
> > >
> > > I don't have strong opinion for either method.
> > >
> > > Passing them at creation time is simpler for user.
> > > If user needs the ability to modify and reuse same device with
different
> > config, extending such support in future like this patch should
possible.
> > >
> > > So there are two questions to close.
> > > 1. Can we start with config params at vdpa device creation time?
> >
> > I'm not sure whether we need both but I'd like to see a full
API and I think we
> > all agree host wants ability to tweak mac after device creation even
if guest is
> > not allowed to change mac, right?
> >
> Yes.
> $ vdpa dev add name foo mgmtdev pci/0000:03:00.0 mac 00:11:22:33:44:55
maxvqs 8 mtu 9000
>
> Above API if we do at creation time. It is likely simpler for user to pass
necessary params during creation time.
>
> > > 2. Is it ok to have these config params as individual fields at
netlink U->K
> > UAPI level?
> > > This is the method proposed in this patch series.
> > > (Similar to incrementally growing vxlan ip link command).
> > >
> > > Or
> > > They should be packed in a structure between U-> K and deal
with
> > typecasting based on size and more?
> > > (Jason's input).
> >
> > I'm inclined to say vxlan is closer to a model to follow.
> Ok. thanks for the feedback. We are using the model close to vxlan.
> Lets resolve should we have it at creation time, post creation or both?
> (a) Creation time
> Pros:
> - simpler single api for user
> - eliminates needs of inventing stats reset in future series
> Cons:
> - inability to reuse the device with different config
This can be solved by destroying the instance and re-creating it with
a different params?
> - This may not be of great advantage, and it is probably fine to have
creation time params
>
> (b) post creation time:
> Pros:
> - able to reuse the device with different config for say different VM.
> - will require stats reset in future once stats are implemented
Any reason for doing this other than re-creating the device?
> Cons:
> - more commands for users to config a device, better to have the ability at
create time.
We probably need to support post creation but it should be device specific.

E.g we may support device resize for virtio-blk devices.

But it can be done on top I think.

Thanks
>
> >
> > --
> > MST
>

> From: Jason Wang <jasowang at redhat.com>
> Sent: Wednesday, August 18, 2021 10:02 AM
> On Wed, Aug 18, 2021 at 11:15 AM Parav Pandit <parav at nvidia.com>
wrote:
[..]
> > > I'm inclined to say vxlan is closer to a model to follow.
> > Ok. thanks for the feedback. We are using the model close to vxlan.
> > Lets resolve should we have it at creation time, post creation or
both?
> > (a) Creation time
> > Pros:
> > - simpler single api for user
> > - eliminates needs of inventing stats reset in future series
> > Cons:
> > - inability to reuse the device with different config
> 
> This can be solved by destroying the instance and re-creating it with a
> different params?
> 
Yes, which is what I tried be say below.
> > - This may not be of great advantage, and it is probably fine to have
creation time params
      ^^^^^ here.
> >
> > (b) post creation time:
> > Pros:
> > - able to reuse the device with different config for say different VM.
> > - will require stats reset in future once stats are implemented
> 
> Any reason for doing this other than re-creating the device?
> 
No. Only reason I can think of is, device reconfig may be faster than recreate.
But I weigh user simplicity more at the beginning and optimizations to bring
later if required.
> > Cons:
> > - more commands for users to config a device, better to have the
ability at
> create time.
> 
> We probably need to support post creation but it should be device specific.
True. Your below device resize is good example of it.
> 
> E.g we may support device resize for virtio-blk devices.
> 
> But it can be done on top I think.
I think so too.

On Wed, Aug 18, 2021 at 12:31:39PM +0800, Jason Wang
wrote:
> On Wed, Aug 18, 2021 at 11:15 AM Parav Pandit <parav at nvidia.com>
wrote:
> >
> >
> >
> > > From: Michael S. Tsirkin <mst at redhat.com>
> > > Sent: Tuesday, August 17, 2021 2:24 AM
> > >
> > > On Mon, Aug 09, 2021 at 09:51:49AM +0000, Parav Pandit wrote:
> > > > > From: Michael S. Tsirkin <mst at redhat.com>
> > > > > Sent: Monday, August 9, 2021 3:10 PM
> > > > >
> > > > > On Fri, Aug 06, 2021 at 08:55:56AM +0000, Parav Pandit
wrote:
> > > > > >
> > > > > >
> > > > > > >
> > > > > > > The point is to try and not reinvent a
dedicated vpda interface
> > > > > > > where a generic one exits.
> > > > > > > E.g. for phy things such as mac speed etc, I
think most people
> > > > > > > are using ethtool things right?
> > > > > >
> > > > > > As you know vdpa is the backend device for the
front-end netdevice
> > > > > accessed by the ethtool.
> > > > > > vdpa management tool here is composing the vdpa
device.
> > > > > >
> > > > > > For example creator (hypervisor) of the vdpa
devices knows that a
> > > > > > guest VM is given 4 vcpus, So hypervisor creates a
vdpa devices
> > > > > > with config space layout as, max_virtqueue_pairs =
4.
> > > > > > And the MAC address chosen by hypervisor in
mac[6].
> > > > > >
> > > > > > Guest VM ethtool can still chose to use less
number of channels.
> > > > > >
> > > > > > Typically,
> > > > > > ethtool is for guest VM.
> > > > > > vdpa device is in hypevisor.
> > > > > >
> > > > > > How can hypervisor compose a vdpa device without
any tool?
> > > > > > How can it tell ethtool, what is supported and
what are the defaults?
> > > > > >
> > > > > > I must be misunderstanding your comment about
ethtool.
> > > > > > Can you please explain?
> > > > >
> > > > >
> > > > > I am basically saying that we probably want to be able
to change MAC
> > > > > of a VDPA device on the host without desroying and
recreating the
> > > > > device as long as it's not in use.
> > > > Ok. I understood your comment now.
> > > > Yes, this was the objective which is why they are present as
independent
> > > config knob.
> > > > Jason was suggesting to have them as creation only knobs,
which requires
> > > recreate.
> > > >
> > > > I don't have strong opinion for either method.
> > > >
> > > > Passing them at creation time is simpler for user.
> > > > If user needs the ability to modify and reuse same device
with different
> > > config, extending such support in future like this patch should
possible.
> > > >
> > > > So there are two questions to close.
> > > > 1. Can we start with config params at vdpa device creation
time?
> > >
> > > I'm not sure whether we need both but I'd like to see a
full API and I think we
> > > all agree host wants ability to tweak mac after device creation
even if guest is
> > > not allowed to change mac, right?
> > >
> > Yes.
> > $ vdpa dev add name foo mgmtdev pci/0000:03:00.0 mac 00:11:22:33:44:55
maxvqs 8 mtu 9000
> >
> > Above API if we do at creation time. It is likely simpler for user to
pass necessary params during creation time.
> >
> > > > 2. Is it ok to have these config params as individual fields
at netlink U->K
> > > UAPI level?
> > > > This is the method proposed in this patch series.
> > > > (Similar to incrementally growing vxlan ip link command).
> > > >
> > > > Or
> > > > They should be packed in a structure between U-> K and
deal with
> > > typecasting based on size and more?
> > > > (Jason's input).
> > >
> > > I'm inclined to say vxlan is closer to a model to follow.
> > Ok. thanks for the feedback. We are using the model close to vxlan.
> > Lets resolve should we have it at creation time, post creation or
both?
> > (a) Creation time
> > Pros:
> > - simpler single api for user
> > - eliminates needs of inventing stats reset in future series
> > Cons:
> > - inability to reuse the device with different config
> 
> This can be solved by destroying the instance and re-creating it with
> a different params?
> 
> > - This may not be of great advantage, and it is probably fine to have
creation time params
> >
> > (b) post creation time:
> > Pros:
> > - able to reuse the device with different config for say different VM.
> > - will require stats reset in future once stats are implemented
> 
> Any reason for doing this other than re-creating the device?
Permissions.

> > Cons:
> > - more commands for users to config a device, better to have the
ability at create time.
> 
> We probably need to support post creation but it should be device specific.
> 
> E.g we may support device resize for virtio-blk devices.
> 
> But it can be done on top I think.
> 
> Thanks
> 
> >
> > >
> > > --
> > > MST
> >