Jason Wang
2022-Jul-28 05:53 UTC
[PATCH V3 5/6] vDPA: answer num of queue pairs = 1 to userspace when VIRTIO_NET_F_MQ == 0
On Thu, Jul 28, 2022 at 11:47 AM Zhu, Lingshan <lingshan.zhu at intel.com> wrote:> > > > On 7/28/2022 9:21 AM, Jason Wang wrote: > > On Wed, Jul 27, 2022 at 11:45 PM Michael S. Tsirkin <mst at redhat.com> wrote: > >> On Wed, Jul 27, 2022 at 05:50:59PM +0800, Jason Wang wrote: > >>> On Wed, Jul 27, 2022 at 5:03 PM Michael S. Tsirkin <mst at redhat.com> wrote: > >>>> On Wed, Jul 27, 2022 at 02:54:13PM +0800, Jason Wang wrote: > >>>>> On Wed, Jul 27, 2022 at 2:01 PM Michael S. Tsirkin <mst at redhat.com> wrote: > >>>>>> On Wed, Jul 27, 2022 at 03:47:35AM +0000, Parav Pandit wrote: > >>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > >>>>>>>> Sent: Tuesday, July 26, 2022 10:53 PM > >>>>>>>> > >>>>>>>> On 7/27/2022 10:17 AM, Parav Pandit wrote: > >>>>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > >>>>>>>>>> Sent: Tuesday, July 26, 2022 10:15 PM > >>>>>>>>>> > >>>>>>>>>> On 7/26/2022 11:56 PM, Parav Pandit wrote: > >>>>>>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > >>>>>>>>>>>> Sent: Tuesday, July 12, 2022 11:46 PM > >>>>>>>>>>>>> When the user space which invokes netlink commands, detects that > >>>>>>>>>> _MQ > >>>>>>>>>>>> is not supported, hence it takes max_queue_pair = 1 by itself. > >>>>>>>>>>>> I think the kernel module have all necessary information and it is > >>>>>>>>>>>> the only one which have precise information of a device, so it > >>>>>>>>>>>> should answer precisely than let the user space guess. The kernel > >>>>>>>>>>>> module should be reliable than stay silent, leave the question to > >>>>>>>>>>>> the user space > >>>>>>>>>> tool. > >>>>>>>>>>> Kernel is reliable. It doesn?t expose a config space field if the > >>>>>>>>>>> field doesn?t > >>>>>>>>>> exist regardless of field should have default or no default. > >>>>>>>>>> so when you know it is one queue pair, you should answer one, not try > >>>>>>>>>> to guess. > >>>>>>>>>>> User space should not guess either. User space gets to see if _MQ > >>>>>>>>>> present/not present. If _MQ present than get reliable data from kernel. > >>>>>>>>>>> If _MQ not present, it means this device has one VQ pair. > >>>>>>>>>> it is still a guess, right? And all user space tools implemented this > >>>>>>>>>> feature need to guess > >>>>>>>>> No. it is not a guess. > >>>>>>>>> It is explicitly checking the _MQ feature and deriving the value. > >>>>>>>>> The code you proposed will be present in the user space. > >>>>>>>>> It will be uniform for _MQ and 10 other features that are present now and > >>>>>>>> in the future. > >>>>>>>> MQ and other features like RSS are different. If there is no _RSS_XX, there > >>>>>>>> are no attributes like max_rss_key_size, and there is not a default value. > >>>>>>>> But for MQ, we know it has to be 1 wihtout _MQ. > >>>>>>> "we" = user space. > >>>>>>> To keep the consistency among all the config space fields. > >>>>>> Actually I looked and the code some more and I'm puzzled: > >>>>>> > >>>>>> > >>>>>> struct virtio_net_config config = {}; > >>>>>> u64 features; > >>>>>> u16 val_u16; > >>>>>> > >>>>>> vdpa_get_config_unlocked(vdev, 0, &config, sizeof(config)); > >>>>>> > >>>>>> if (nla_put(msg, VDPA_ATTR_DEV_NET_CFG_MACADDR, sizeof(config.mac), > >>>>>> config.mac)) > >>>>>> return -EMSGSIZE; > >>>>>> > >>>>>> > >>>>>> Mac returned even without VIRTIO_NET_F_MAC > >>>>>> > >>>>>> > >>>>>> val_u16 = le16_to_cpu(config.status); > >>>>>> if (nla_put_u16(msg, VDPA_ATTR_DEV_NET_STATUS, val_u16)) > >>>>>> return -EMSGSIZE; > >>>>>> > >>>>>> > >>>>>> status returned even without VIRTIO_NET_F_STATUS > >>>>>> > >>>>>> val_u16 = le16_to_cpu(config.mtu); > >>>>>> if (nla_put_u16(msg, VDPA_ATTR_DEV_NET_CFG_MTU, val_u16)) > >>>>>> return -EMSGSIZE; > >>>>>> > >>>>>> > >>>>>> MTU returned even without VIRTIO_NET_F_MTU > >>>>>> > >>>>>> > >>>>>> What's going on here? > >>>>> Probably too late to fix, but this should be fine as long as all > >>>>> parents support STATUS/MTU/MAC. > >>>> Why is this too late to fix. > >>> If we make this conditional on the features. This may break the > >>> userspace that always expects VDPA_ATTR_DEV_NET_CFG_MTU? > >>> > >>> Thanks > >> Well only on devices without MTU. I'm saying said userspace > >> was reading trash on such devices anyway. > > It depends on the parent actually. For example, mlx5 query the lower > > mtu unconditionally: > > > > err = query_mtu(mdev, &mtu); > > if (err) > > goto err_alloc; > > > > ndev->config.mtu = cpu_to_mlx5vdpa16(mvdev, mtu); > > > > Supporting MTU features seems to be a must for real hardware. > > Otherwise the driver may not work correctly. > > > >> We don't generally maintain bug for bug compatiblity on a whim, > >> only if userspace is actually known to break if we fix a bug. > > So I think it should be fine to make this conditional then we should > > have a consistent handling of other fields like MQ. > For some fields that have a default value, like MQ =1, we can return the > default value. > For other fields without a default value, like MAC, we return nothing. > > Does this sounds good? So, for MTU, if without _F_MTU, I think we can > return 1500 by default.Or we can just read MTU from the device. But It looks to me Michael wants it conditional. Thanks> > Thanks, > Zhu Lingshan > > > > Thanks > > > >> > >>>>> I wonder if we can add a check in the core and fail the device > >>>>> registration in this case. > >>>>> > >>>>> Thanks > >>>>> > >>>>>> > >>>>>> -- > >>>>>> MST > >>>>>> >
Michael S. Tsirkin
2022-Jul-28 06:41 UTC
[PATCH V3 5/6] vDPA: answer num of queue pairs = 1 to userspace when VIRTIO_NET_F_MQ == 0
On Thu, Jul 28, 2022 at 01:53:51PM +0800, Jason Wang wrote:> On Thu, Jul 28, 2022 at 11:47 AM Zhu, Lingshan <lingshan.zhu at intel.com> wrote: > > > > > > > > On 7/28/2022 9:21 AM, Jason Wang wrote: > > > On Wed, Jul 27, 2022 at 11:45 PM Michael S. Tsirkin <mst at redhat.com> wrote: > > >> On Wed, Jul 27, 2022 at 05:50:59PM +0800, Jason Wang wrote: > > >>> On Wed, Jul 27, 2022 at 5:03 PM Michael S. Tsirkin <mst at redhat.com> wrote: > > >>>> On Wed, Jul 27, 2022 at 02:54:13PM +0800, Jason Wang wrote: > > >>>>> On Wed, Jul 27, 2022 at 2:01 PM Michael S. Tsirkin <mst at redhat.com> wrote: > > >>>>>> On Wed, Jul 27, 2022 at 03:47:35AM +0000, Parav Pandit wrote: > > >>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > > >>>>>>>> Sent: Tuesday, July 26, 2022 10:53 PM > > >>>>>>>> > > >>>>>>>> On 7/27/2022 10:17 AM, Parav Pandit wrote: > > >>>>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > > >>>>>>>>>> Sent: Tuesday, July 26, 2022 10:15 PM > > >>>>>>>>>> > > >>>>>>>>>> On 7/26/2022 11:56 PM, Parav Pandit wrote: > > >>>>>>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > > >>>>>>>>>>>> Sent: Tuesday, July 12, 2022 11:46 PM > > >>>>>>>>>>>>> When the user space which invokes netlink commands, detects that > > >>>>>>>>>> _MQ > > >>>>>>>>>>>> is not supported, hence it takes max_queue_pair = 1 by itself. > > >>>>>>>>>>>> I think the kernel module have all necessary information and it is > > >>>>>>>>>>>> the only one which have precise information of a device, so it > > >>>>>>>>>>>> should answer precisely than let the user space guess. The kernel > > >>>>>>>>>>>> module should be reliable than stay silent, leave the question to > > >>>>>>>>>>>> the user space > > >>>>>>>>>> tool. > > >>>>>>>>>>> Kernel is reliable. It doesn?t expose a config space field if the > > >>>>>>>>>>> field doesn?t > > >>>>>>>>>> exist regardless of field should have default or no default. > > >>>>>>>>>> so when you know it is one queue pair, you should answer one, not try > > >>>>>>>>>> to guess. > > >>>>>>>>>>> User space should not guess either. User space gets to see if _MQ > > >>>>>>>>>> present/not present. If _MQ present than get reliable data from kernel. > > >>>>>>>>>>> If _MQ not present, it means this device has one VQ pair. > > >>>>>>>>>> it is still a guess, right? And all user space tools implemented this > > >>>>>>>>>> feature need to guess > > >>>>>>>>> No. it is not a guess. > > >>>>>>>>> It is explicitly checking the _MQ feature and deriving the value. > > >>>>>>>>> The code you proposed will be present in the user space. > > >>>>>>>>> It will be uniform for _MQ and 10 other features that are present now and > > >>>>>>>> in the future. > > >>>>>>>> MQ and other features like RSS are different. If there is no _RSS_XX, there > > >>>>>>>> are no attributes like max_rss_key_size, and there is not a default value. > > >>>>>>>> But for MQ, we know it has to be 1 wihtout _MQ. > > >>>>>>> "we" = user space. > > >>>>>>> To keep the consistency among all the config space fields. > > >>>>>> Actually I looked and the code some more and I'm puzzled: > > >>>>>> > > >>>>>> > > >>>>>> struct virtio_net_config config = {}; > > >>>>>> u64 features; > > >>>>>> u16 val_u16; > > >>>>>> > > >>>>>> vdpa_get_config_unlocked(vdev, 0, &config, sizeof(config)); > > >>>>>> > > >>>>>> if (nla_put(msg, VDPA_ATTR_DEV_NET_CFG_MACADDR, sizeof(config.mac), > > >>>>>> config.mac)) > > >>>>>> return -EMSGSIZE; > > >>>>>> > > >>>>>> > > >>>>>> Mac returned even without VIRTIO_NET_F_MAC > > >>>>>> > > >>>>>> > > >>>>>> val_u16 = le16_to_cpu(config.status); > > >>>>>> if (nla_put_u16(msg, VDPA_ATTR_DEV_NET_STATUS, val_u16)) > > >>>>>> return -EMSGSIZE; > > >>>>>> > > >>>>>> > > >>>>>> status returned even without VIRTIO_NET_F_STATUS > > >>>>>> > > >>>>>> val_u16 = le16_to_cpu(config.mtu); > > >>>>>> if (nla_put_u16(msg, VDPA_ATTR_DEV_NET_CFG_MTU, val_u16)) > > >>>>>> return -EMSGSIZE; > > >>>>>> > > >>>>>> > > >>>>>> MTU returned even without VIRTIO_NET_F_MTU > > >>>>>> > > >>>>>> > > >>>>>> What's going on here? > > >>>>> Probably too late to fix, but this should be fine as long as all > > >>>>> parents support STATUS/MTU/MAC. > > >>>> Why is this too late to fix. > > >>> If we make this conditional on the features. This may break the > > >>> userspace that always expects VDPA_ATTR_DEV_NET_CFG_MTU? > > >>> > > >>> Thanks > > >> Well only on devices without MTU. I'm saying said userspace > > >> was reading trash on such devices anyway. > > > It depends on the parent actually. For example, mlx5 query the lower > > > mtu unconditionally: > > > > > > err = query_mtu(mdev, &mtu); > > > if (err) > > > goto err_alloc; > > > > > > ndev->config.mtu = cpu_to_mlx5vdpa16(mvdev, mtu); > > > > > > Supporting MTU features seems to be a must for real hardware. > > > Otherwise the driver may not work correctly. > > > > > >> We don't generally maintain bug for bug compatiblity on a whim, > > >> only if userspace is actually known to break if we fix a bug. > > > So I think it should be fine to make this conditional then we should > > > have a consistent handling of other fields like MQ. > > For some fields that have a default value, like MQ =1, we can return the > > default value. > > For other fields without a default value, like MAC, we return nothing. > > > > Does this sounds good? So, for MTU, if without _F_MTU, I think we can > > return 1500 by default. > > Or we can just read MTU from the device. > > But It looks to me Michael wants it conditional. > > ThanksI'm fine either way but let's keep it consistent. And I think Parav wants it conditional.> > > > Thanks, > > Zhu Lingshan > > > > > > Thanks > > > > > >> > > >>>>> I wonder if we can add a check in the core and fail the device > > >>>>> registration in this case. > > >>>>> > > >>>>> Thanks > > >>>>> > > >>>>>> > > >>>>>> -- > > >>>>>> MST > > >>>>>> > >
Jason Wang
2022-Aug-01 04:50 UTC
[PATCH V3 5/6] vDPA: answer num of queue pairs = 1 to userspace when VIRTIO_NET_F_MQ == 0
On Thu, Jul 28, 2022 at 2:41 PM Michael S. Tsirkin <mst at redhat.com> wrote:> > On Thu, Jul 28, 2022 at 01:53:51PM +0800, Jason Wang wrote: > > On Thu, Jul 28, 2022 at 11:47 AM Zhu, Lingshan <lingshan.zhu at intel.com> wrote: > > > > > > > > > > > > On 7/28/2022 9:21 AM, Jason Wang wrote: > > > > On Wed, Jul 27, 2022 at 11:45 PM Michael S. Tsirkin <mst at redhat.com> wrote: > > > >> On Wed, Jul 27, 2022 at 05:50:59PM +0800, Jason Wang wrote: > > > >>> On Wed, Jul 27, 2022 at 5:03 PM Michael S. Tsirkin <mst at redhat.com> wrote: > > > >>>> On Wed, Jul 27, 2022 at 02:54:13PM +0800, Jason Wang wrote: > > > >>>>> On Wed, Jul 27, 2022 at 2:01 PM Michael S. Tsirkin <mst at redhat.com> wrote: > > > >>>>>> On Wed, Jul 27, 2022 at 03:47:35AM +0000, Parav Pandit wrote: > > > >>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > > > >>>>>>>> Sent: Tuesday, July 26, 2022 10:53 PM > > > >>>>>>>> > > > >>>>>>>> On 7/27/2022 10:17 AM, Parav Pandit wrote: > > > >>>>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > > > >>>>>>>>>> Sent: Tuesday, July 26, 2022 10:15 PM > > > >>>>>>>>>> > > > >>>>>>>>>> On 7/26/2022 11:56 PM, Parav Pandit wrote: > > > >>>>>>>>>>>> From: Zhu, Lingshan <lingshan.zhu at intel.com> > > > >>>>>>>>>>>> Sent: Tuesday, July 12, 2022 11:46 PM > > > >>>>>>>>>>>>> When the user space which invokes netlink commands, detects that > > > >>>>>>>>>> _MQ > > > >>>>>>>>>>>> is not supported, hence it takes max_queue_pair = 1 by itself. > > > >>>>>>>>>>>> I think the kernel module have all necessary information and it is > > > >>>>>>>>>>>> the only one which have precise information of a device, so it > > > >>>>>>>>>>>> should answer precisely than let the user space guess. The kernel > > > >>>>>>>>>>>> module should be reliable than stay silent, leave the question to > > > >>>>>>>>>>>> the user space > > > >>>>>>>>>> tool. > > > >>>>>>>>>>> Kernel is reliable. It doesn?t expose a config space field if the > > > >>>>>>>>>>> field doesn?t > > > >>>>>>>>>> exist regardless of field should have default or no default. > > > >>>>>>>>>> so when you know it is one queue pair, you should answer one, not try > > > >>>>>>>>>> to guess. > > > >>>>>>>>>>> User space should not guess either. User space gets to see if _MQ > > > >>>>>>>>>> present/not present. If _MQ present than get reliable data from kernel. > > > >>>>>>>>>>> If _MQ not present, it means this device has one VQ pair. > > > >>>>>>>>>> it is still a guess, right? And all user space tools implemented this > > > >>>>>>>>>> feature need to guess > > > >>>>>>>>> No. it is not a guess. > > > >>>>>>>>> It is explicitly checking the _MQ feature and deriving the value. > > > >>>>>>>>> The code you proposed will be present in the user space. > > > >>>>>>>>> It will be uniform for _MQ and 10 other features that are present now and > > > >>>>>>>> in the future. > > > >>>>>>>> MQ and other features like RSS are different. If there is no _RSS_XX, there > > > >>>>>>>> are no attributes like max_rss_key_size, and there is not a default value. > > > >>>>>>>> But for MQ, we know it has to be 1 wihtout _MQ. > > > >>>>>>> "we" = user space. > > > >>>>>>> To keep the consistency among all the config space fields. > > > >>>>>> Actually I looked and the code some more and I'm puzzled: > > > >>>>>> > > > >>>>>> > > > >>>>>> struct virtio_net_config config = {}; > > > >>>>>> u64 features; > > > >>>>>> u16 val_u16; > > > >>>>>> > > > >>>>>> vdpa_get_config_unlocked(vdev, 0, &config, sizeof(config)); > > > >>>>>> > > > >>>>>> if (nla_put(msg, VDPA_ATTR_DEV_NET_CFG_MACADDR, sizeof(config.mac), > > > >>>>>> config.mac)) > > > >>>>>> return -EMSGSIZE; > > > >>>>>> > > > >>>>>> > > > >>>>>> Mac returned even without VIRTIO_NET_F_MAC > > > >>>>>> > > > >>>>>> > > > >>>>>> val_u16 = le16_to_cpu(config.status); > > > >>>>>> if (nla_put_u16(msg, VDPA_ATTR_DEV_NET_STATUS, val_u16)) > > > >>>>>> return -EMSGSIZE; > > > >>>>>> > > > >>>>>> > > > >>>>>> status returned even without VIRTIO_NET_F_STATUS > > > >>>>>> > > > >>>>>> val_u16 = le16_to_cpu(config.mtu); > > > >>>>>> if (nla_put_u16(msg, VDPA_ATTR_DEV_NET_CFG_MTU, val_u16)) > > > >>>>>> return -EMSGSIZE; > > > >>>>>> > > > >>>>>> > > > >>>>>> MTU returned even without VIRTIO_NET_F_MTU > > > >>>>>> > > > >>>>>> > > > >>>>>> What's going on here? > > > >>>>> Probably too late to fix, but this should be fine as long as all > > > >>>>> parents support STATUS/MTU/MAC. > > > >>>> Why is this too late to fix. > > > >>> If we make this conditional on the features. This may break the > > > >>> userspace that always expects VDPA_ATTR_DEV_NET_CFG_MTU? > > > >>> > > > >>> Thanks > > > >> Well only on devices without MTU. I'm saying said userspace > > > >> was reading trash on such devices anyway. > > > > It depends on the parent actually. For example, mlx5 query the lower > > > > mtu unconditionally: > > > > > > > > err = query_mtu(mdev, &mtu); > > > > if (err) > > > > goto err_alloc; > > > > > > > > ndev->config.mtu = cpu_to_mlx5vdpa16(mvdev, mtu); > > > > > > > > Supporting MTU features seems to be a must for real hardware. > > > > Otherwise the driver may not work correctly. > > > > > > > >> We don't generally maintain bug for bug compatiblity on a whim, > > > >> only if userspace is actually known to break if we fix a bug. > > > > So I think it should be fine to make this conditional then we should > > > > have a consistent handling of other fields like MQ. > > > For some fields that have a default value, like MQ =1, we can return the > > > default value. > > > For other fields without a default value, like MAC, we return nothing. > > > > > > Does this sounds good? So, for MTU, if without _F_MTU, I think we can > > > return 1500 by default. > > > > Or we can just read MTU from the device. > > > > But It looks to me Michael wants it conditional. > > > > Thanks > > I'm fine either way but let's keep it consistent. And I think > Parav wants it conditional.Parav, what's your opinion here? Michale spots some in-consistent stuffs, so I think we should either 1) make all conditional, so we should change both MTU and MAC or 2) make them unconditional, so we should only change MQ Thanks> > > > > > > Thanks, > > > Zhu Lingshan > > > > > > > > Thanks > > > > > > > >> > > > >>>>> I wonder if we can add a check in the core and fail the device > > > >>>>> registration in this case. > > > >>>>> > > > >>>>> Thanks > > > >>>>> > > > >>>>>> > > > >>>>>> -- > > > >>>>>> MST > > > >>>>>> > > > >