Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 00/10] devlink: Control auxiliary devices
(Resend to CC RDMA and vdpa mailing lists). Hi Dave, Jakub, Currently, for mlx5 multi-function device, a user is not able to control which functionality to enable/disable. For example, each PCI PF, VF, SF function by default has netdevice, RDMA and vdpa-net devices always enabled. Hence, enable user to control which device functionality to enable/disable. This is achieved by using existing devlink params [1] to enable/disable eth, rdma and vdpa net functionality control knob. For example user interested in only vdpa device function: performs, $ devlink dev param set pci/0000:06:00.0 name enable_rdma value false \ cmode driverinit $ devlink dev param set pci/0000:06:00.0 name enable_eth value false \ cmode driverinit $ devlink dev param set pci/0000:06:00.0 name enable_vnet value true \ cmode driverinit $ devlink dev reload pci/0000:06:00.0 Reload command honors parameters set, initializes the device that user has composed using devlink dev params and resources. Devices before reload: mlx5_core.sf.4 (subfunction device) /\ /| \ / | \ / | \ mlx5_core.eth.4 | mlx5_core.rdma.4 (SF eth aux dev) | (SF rdma aux dev) | | | | | | enp6s0f0s88 | mlx5_0 (SF netdev) | (SF rdma device) | mlx5_core.vnet.4 (SF vnet aux dev) | | auxiliary/mlx5_core.sf.4 (vdpa net mgmt device) Above example reconfigures the device with only VDPA functionality. Devices after reload: mlx5_core.sf.4 (subfunction device) /\ / \ / \ / \ mlx5_core.vnet.4 no eth, no rdma aux devices (SF vnet aux dev) Above parameters enable user to compose the device as needed based on the use case. Since devlink params are done on the devlink instance, these knobs are uniformly usable for PCI PF, VF and SF devices. Patch summary: patch-1 adds generic enable_eth devlink parameter to control Ethernet auxiliary device function patch-2 adds generic enable_rdma devlink parameter to control RDMA auxiliary device function patch-3 adds generic enable_vnet devlink parameter to control VDPA net auxilariy device function patch-4 rework the code to register single device parameter patch-5 added APIs to register, unregister single device parameter patch-6 added APIs to publish, unpublishe single device parameter patch-7 Fixed missing parameter unpublish call in mlx5 driver patch-8 extends mlx5 driver to support enable_eth devlink parameter patch-9 extends mlx5 driver to support enable_rdma devlink parameter patch-10 extends mlx5 driver to support enable_vnet devlink parameter Subsequent to this series, in future mlx5 driver will be updated to use single device parameter API for metadata enable/disable knob which is only applicable on the eswitch manager device. [1] https://www.kernel.org/doc/html/latest/networking/devlink/devlink-params.html Parav Pandit (10): devlink: Add new "enable_eth" generic device param devlink: Add new "enable_rdma" generic device param devlink: Add new "enable_vnet" generic device param devlink: Create a helper function for one parameter registration devlink: Add API to register and unregister single parameter devlink: Add APIs to publish, unpublish individual parameter net/mlx5: Fix unpublish devlink parameters net/mlx5: Support enable_eth devlink dev param net/mlx5: Support enable_rdma devlink dev param net/mlx5: Support enable_vnet devlink dev param .../networking/devlink/devlink-params.rst | 12 ++ drivers/net/ethernet/mellanox/mlx5/core/dev.c | 74 +++++++- .../net/ethernet/mellanox/mlx5/core/devlink.c | 159 ++++++++++++++++++ .../ethernet/mellanox/mlx5/core/mlx5_core.h | 5 + include/net/devlink.h | 20 +++ net/core/devlink.c | 124 +++++++++++++- 6 files changed, 382 insertions(+), 12 deletions(-) -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 01/10] devlink: Add new "enable_eth" generic device param
Add new device generic parameter to enable/disable creation of Ethernet auxiliary device and associated device functionality in the devlink instance. User who prefers to disable such functionality can disable it using below example. $ devlink dev param set pci/0000:06:00.0 \ name enable_eth value false cmode driverinit $ devlink dev reload pci/0000:06:00.0 At this point devlink instance do not create auxiliary device for the Ethernet functionality. Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Jiri Pirko <jiri at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- Documentation/networking/devlink/devlink-params.rst | 4 ++++ include/net/devlink.h | 4 ++++ net/core/devlink.c | 5 +++++ 3 files changed, 13 insertions(+) diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst index 54c9f107c4b0..219c1272f2d6 100644 --- a/Documentation/networking/devlink/devlink-params.rst +++ b/Documentation/networking/devlink/devlink-params.rst @@ -97,6 +97,10 @@ own name. * - ``enable_roce`` - Boolean - Enable handling of RoCE traffic in the device. + * - ``enable_eth`` + - Boolean + - When enabled, the device driver will instantiate Ethernet specific + auxiliary device of the devlink device. * - ``internal_err_reset`` - Boolean - When enabled, the device driver will reset the device on internal diff --git a/include/net/devlink.h b/include/net/devlink.h index 0236c77f2fd0..1e3e183bb2c2 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -519,6 +519,7 @@ enum devlink_param_generic_id { DEVLINK_PARAM_GENERIC_ID_RESET_DEV_ON_DRV_PROBE, DEVLINK_PARAM_GENERIC_ID_ENABLE_ROCE, DEVLINK_PARAM_GENERIC_ID_ENABLE_REMOTE_DEV_RESET, + DEVLINK_PARAM_GENERIC_ID_ENABLE_ETH, /* add new param generic ids above here*/ __DEVLINK_PARAM_GENERIC_ID_MAX, @@ -559,6 +560,9 @@ enum devlink_param_generic_id { #define DEVLINK_PARAM_GENERIC_ENABLE_REMOTE_DEV_RESET_NAME "enable_remote_dev_reset" #define DEVLINK_PARAM_GENERIC_ENABLE_REMOTE_DEV_RESET_TYPE DEVLINK_PARAM_TYPE_BOOL +#define DEVLINK_PARAM_GENERIC_ENABLE_ETH_NAME "enable_eth" +#define DEVLINK_PARAM_GENERIC_ENABLE_ETH_TYPE DEVLINK_PARAM_TYPE_BOOL + #define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \ { \ .id = DEVLINK_PARAM_GENERIC_ID_##_id, \ diff --git a/net/core/devlink.c b/net/core/devlink.c index b02d54ab59ac..9a59f45c8bf9 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -4277,6 +4277,11 @@ static const struct devlink_param devlink_param_generic[] = { .name = DEVLINK_PARAM_GENERIC_ENABLE_REMOTE_DEV_RESET_NAME, .type = DEVLINK_PARAM_GENERIC_ENABLE_REMOTE_DEV_RESET_TYPE, }, + { + .id = DEVLINK_PARAM_GENERIC_ID_ENABLE_ETH, + .name = DEVLINK_PARAM_GENERIC_ENABLE_ETH_NAME, + .type = DEVLINK_PARAM_GENERIC_ENABLE_ETH_TYPE, + }, }; static int devlink_param_generic_verify(const struct devlink_param *param) -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 02/10] devlink: Add new "enable_rdma" generic device param
Add new device generic parameter to enable/disable creation of RDMA auxiliary device and associated device functionality in the devlink instance. User who prefers to disable such functionality can disable it using below example. $ devlink dev param set pci/0000:06:00.0 \ name enable_rdma value false cmode driverinit $ devlink dev reload pci/0000:06:00.0 At this point devlink instance do not create auxiliary device for the RDMA functionality. Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Jiri Pirko <jiri at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- Documentation/networking/devlink/devlink-params.rst | 4 ++++ include/net/devlink.h | 4 ++++ net/core/devlink.c | 5 +++++ 3 files changed, 13 insertions(+) diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst index 219c1272f2d6..a49da0b049b6 100644 --- a/Documentation/networking/devlink/devlink-params.rst +++ b/Documentation/networking/devlink/devlink-params.rst @@ -101,6 +101,10 @@ own name. - Boolean - When enabled, the device driver will instantiate Ethernet specific auxiliary device of the devlink device. + * - ``enable_rdma`` + - Boolean + - When enabled, the device driver will instantiate RDMA specific + auxiliary device of the devlink device. * - ``internal_err_reset`` - Boolean - When enabled, the device driver will reset the device on internal diff --git a/include/net/devlink.h b/include/net/devlink.h index 1e3e183bb2c2..6f4f0416e598 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -520,6 +520,7 @@ enum devlink_param_generic_id { DEVLINK_PARAM_GENERIC_ID_ENABLE_ROCE, DEVLINK_PARAM_GENERIC_ID_ENABLE_REMOTE_DEV_RESET, DEVLINK_PARAM_GENERIC_ID_ENABLE_ETH, + DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA, /* add new param generic ids above here*/ __DEVLINK_PARAM_GENERIC_ID_MAX, @@ -563,6 +564,9 @@ enum devlink_param_generic_id { #define DEVLINK_PARAM_GENERIC_ENABLE_ETH_NAME "enable_eth" #define DEVLINK_PARAM_GENERIC_ENABLE_ETH_TYPE DEVLINK_PARAM_TYPE_BOOL +#define DEVLINK_PARAM_GENERIC_ENABLE_RDMA_NAME "enable_rdma" +#define DEVLINK_PARAM_GENERIC_ENABLE_RDMA_TYPE DEVLINK_PARAM_TYPE_BOOL + #define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \ { \ .id = DEVLINK_PARAM_GENERIC_ID_##_id, \ diff --git a/net/core/devlink.c b/net/core/devlink.c index 9a59f45c8bf9..b68d6921d34f 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -4282,6 +4282,11 @@ static const struct devlink_param devlink_param_generic[] = { .name = DEVLINK_PARAM_GENERIC_ENABLE_ETH_NAME, .type = DEVLINK_PARAM_GENERIC_ENABLE_ETH_TYPE, }, + { + .id = DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA, + .name = DEVLINK_PARAM_GENERIC_ENABLE_RDMA_NAME, + .type = DEVLINK_PARAM_GENERIC_ENABLE_RDMA_TYPE, + }, }; static int devlink_param_generic_verify(const struct devlink_param *param) -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 03/10] devlink: Add new "enable_vnet" generic device param
Add new device generic parameter to enable/disable creation of VDPA net auxiliary device and associated device functionality in the devlink instance. User who prefers to disable such functionality can disable it using below example. $ devlink dev param set pci/0000:06:00.0 \ name enable_vnet value false cmode driverinit $ devlink dev reload pci/0000:06:00.0 At this point devlink instance do not create auxiliary device for the VDPA net functionality. Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Jiri Pirko <jiri at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- Documentation/networking/devlink/devlink-params.rst | 4 ++++ include/net/devlink.h | 4 ++++ net/core/devlink.c | 5 +++++ 3 files changed, 13 insertions(+) diff --git a/Documentation/networking/devlink/devlink-params.rst b/Documentation/networking/devlink/devlink-params.rst index a49da0b049b6..4878907e9232 100644 --- a/Documentation/networking/devlink/devlink-params.rst +++ b/Documentation/networking/devlink/devlink-params.rst @@ -105,6 +105,10 @@ own name. - Boolean - When enabled, the device driver will instantiate RDMA specific auxiliary device of the devlink device. + * - ``enable_vnet`` + - Boolean + - When enabled, the device driver will instantiate VDPA networking + specific auxiliary device of the devlink device. * - ``internal_err_reset`` - Boolean - When enabled, the device driver will reset the device on internal diff --git a/include/net/devlink.h b/include/net/devlink.h index 6f4f0416e598..0a0becbcdc49 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -521,6 +521,7 @@ enum devlink_param_generic_id { DEVLINK_PARAM_GENERIC_ID_ENABLE_REMOTE_DEV_RESET, DEVLINK_PARAM_GENERIC_ID_ENABLE_ETH, DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA, + DEVLINK_PARAM_GENERIC_ID_ENABLE_VNET, /* add new param generic ids above here*/ __DEVLINK_PARAM_GENERIC_ID_MAX, @@ -567,6 +568,9 @@ enum devlink_param_generic_id { #define DEVLINK_PARAM_GENERIC_ENABLE_RDMA_NAME "enable_rdma" #define DEVLINK_PARAM_GENERIC_ENABLE_RDMA_TYPE DEVLINK_PARAM_TYPE_BOOL +#define DEVLINK_PARAM_GENERIC_ENABLE_VNET_NAME "enable_vnet" +#define DEVLINK_PARAM_GENERIC_ENABLE_VNET_TYPE DEVLINK_PARAM_TYPE_BOOL + #define DEVLINK_PARAM_GENERIC(_id, _cmodes, _get, _set, _validate) \ { \ .id = DEVLINK_PARAM_GENERIC_ID_##_id, \ diff --git a/net/core/devlink.c b/net/core/devlink.c index b68d6921d34f..867ae7e39788 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -4287,6 +4287,11 @@ static const struct devlink_param devlink_param_generic[] = { .name = DEVLINK_PARAM_GENERIC_ENABLE_RDMA_NAME, .type = DEVLINK_PARAM_GENERIC_ENABLE_RDMA_TYPE, }, + { + .id = DEVLINK_PARAM_GENERIC_ID_ENABLE_VNET, + .name = DEVLINK_PARAM_GENERIC_ENABLE_VNET_NAME, + .type = DEVLINK_PARAM_GENERIC_ENABLE_VNET_TYPE, + }, }; static int devlink_param_generic_verify(const struct devlink_param *param) -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 04/10] devlink: Create a helper function for one parameter registration
Create and use a helper function for one parameter registration. Subsequent patch also will reuse this for driver facing routine to register a single parameter. Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Jiri Pirko <jiri at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- net/core/devlink.c | 24 ++++++++++++++++++------ 1 file changed, 18 insertions(+), 6 deletions(-) diff --git a/net/core/devlink.c b/net/core/devlink.c index 867ae7e39788..050dd7271a45 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -9800,6 +9800,22 @@ static int devlink_param_verify(const struct devlink_param *param) return devlink_param_driver_verify(param); } +static int __devlink_param_register_one(struct devlink *devlink, + unsigned int port_index, + struct list_head *param_list, + const struct devlink_param *param, + enum devlink_command reg_cmd) +{ + int err; + + err = devlink_param_verify(param); + if (err) + return err; + + return devlink_param_register_one(devlink, port_index, + param_list, param, reg_cmd); +} + static int __devlink_params_register(struct devlink *devlink, unsigned int port_index, struct list_head *param_list, @@ -9814,12 +9830,8 @@ static int __devlink_params_register(struct devlink *devlink, mutex_lock(&devlink->lock); for (i = 0; i < params_count; i++, param++) { - err = devlink_param_verify(param); - if (err) - goto rollback; - - err = devlink_param_register_one(devlink, port_index, - param_list, param, reg_cmd); + err = __devlink_param_register_one(devlink, port_index, + param_list, param, reg_cmd); if (err) goto rollback; } -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 05/10] devlink: Add API to register and unregister single parameter
Currently device configuration parameters can be registered as an array. Due to this a constant array must be registered. A single driver supporting multiple devices each with different device capabilities end up registering all parameters even if it doesn't support it. One possible workaround a driver can do is, it registers multiple single entry arrays to overcome such limitation. Better is to provide a API that enables driver to register/unregister a single parameter. This also further helps in two ways. (1) to reduce the memory of devlink_param_entry by avoiding in registering parameters which are not supported by the device. (2) avoid generating multiple parameter add, delete, publish, unpublish, init value notifications for such unsupported parameters Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Jiri Pirko <jiri at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- include/net/devlink.h | 4 ++++ net/core/devlink.c | 37 +++++++++++++++++++++++++++++++++++++ 2 files changed, 41 insertions(+) diff --git a/include/net/devlink.h b/include/net/devlink.h index 0a0becbcdc49..f6459ee77114 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -1645,6 +1645,10 @@ int devlink_params_register(struct devlink *devlink, void devlink_params_unregister(struct devlink *devlink, const struct devlink_param *params, size_t params_count); +int devlink_param_register(struct devlink *devlink, + const struct devlink_param *param); +void devlink_param_unregister(struct devlink *devlink, + const struct devlink_param *param); void devlink_params_publish(struct devlink *devlink); void devlink_params_unpublish(struct devlink *devlink); int devlink_port_params_register(struct devlink_port *devlink_port, diff --git a/net/core/devlink.c b/net/core/devlink.c index 050dd7271a45..629291175af3 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -9903,6 +9903,43 @@ void devlink_params_unregister(struct devlink *devlink, } EXPORT_SYMBOL_GPL(devlink_params_unregister); +/** + * devlink_param_register - register one configuration parameter + * + * @devlink: devlink + * @param: one configuration parameter + * + * Register the configuration parameter supported by the driver. + * Return: returns 0 on successful registration or error code otherwise. + */ +int devlink_param_register(struct devlink *devlink, + const struct devlink_param *param) +{ + int err; + + mutex_lock(&devlink->lock); + err = __devlink_param_register_one(devlink, 0, &devlink->param_list, + param, DEVLINK_CMD_PARAM_NEW); + mutex_unlock(&devlink->lock); + return err; +} +EXPORT_SYMBOL_GPL(devlink_param_register); + +/** + * devlink_param_unregister - unregister one configuration parameter + * @devlink: devlink + * @param: configuration parameter to unregister + */ +void devlink_param_unregister(struct devlink *devlink, + const struct devlink_param *param) +{ + mutex_lock(&devlink->lock); + devlink_param_unregister_one(devlink, 0, &devlink->param_list, param, + DEVLINK_CMD_PARAM_DEL); + mutex_unlock(&devlink->lock); +} +EXPORT_SYMBOL_GPL(devlink_param_unregister); + /** * devlink_params_publish - publish configuration parameters * -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 06/10] devlink: Add APIs to publish, unpublish individual parameter
Enable drivers to publish/unpublish individual parameter. Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Jiri Pirko <jiri at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- include/net/devlink.h | 4 ++++ net/core/devlink.c | 48 +++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 52 insertions(+) diff --git a/include/net/devlink.h b/include/net/devlink.h index f6459ee77114..1151497c0ec5 100644 --- a/include/net/devlink.h +++ b/include/net/devlink.h @@ -1651,6 +1651,10 @@ void devlink_param_unregister(struct devlink *devlink, const struct devlink_param *param); void devlink_params_publish(struct devlink *devlink); void devlink_params_unpublish(struct devlink *devlink); +void devlink_param_publish(struct devlink *devlink, + const struct devlink_param *param); +void devlink_param_unpublish(struct devlink *devlink, + const struct devlink_param *param); int devlink_port_params_register(struct devlink_port *devlink_port, const struct devlink_param *params, size_t params_count); diff --git a/net/core/devlink.c b/net/core/devlink.c index 629291175af3..ee9787314cff 100644 --- a/net/core/devlink.c +++ b/net/core/devlink.c @@ -9982,6 +9982,54 @@ void devlink_params_unpublish(struct devlink *devlink) } EXPORT_SYMBOL_GPL(devlink_params_unpublish); +/** + * devlink_param_publish - publish one configuration parameter + * + * @devlink: devlink + * @param: one configuration parameter + * + * Publish previously registered configuration parameter. + */ +void devlink_param_publish(struct devlink *devlink, + const struct devlink_param *param) +{ + struct devlink_param_item *param_item; + + list_for_each_entry(param_item, &devlink->param_list, list) { + if (param_item->param != param || param_item->published) + continue; + param_item->published = true; + devlink_param_notify(devlink, 0, param_item, + DEVLINK_CMD_PARAM_NEW); + break; + } +} +EXPORT_SYMBOL_GPL(devlink_param_publish); + +/** + * devlink_param_unpublish - unpublish one configuration parameter + * + * @devlink: devlink + * @param: one configuration parameter + * + * Unpublish previously registered configuration parameter. + */ +void devlink_param_unpublish(struct devlink *devlink, + const struct devlink_param *param) +{ + struct devlink_param_item *param_item; + + list_for_each_entry(param_item, &devlink->param_list, list) { + if (param_item->param != param || !param_item->published) + continue; + param_item->published = false; + devlink_param_notify(devlink, 0, param_item, + DEVLINK_CMD_PARAM_DEL); + break; + } +} +EXPORT_SYMBOL_GPL(devlink_param_unpublish); + /** * devlink_port_params_register - register port configuration parameters * -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 07/10] net/mlx5: Fix unpublish devlink parameters
Cleanup routine missed to unpublish the parameters. Add it. Fixes: e890acd5ff18 ("net/mlx5: Add devlink flow_steering_mode parameter") Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Jiri Pirko <jiri at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- drivers/net/ethernet/mellanox/mlx5/core/devlink.c | 1 + 1 file changed, 1 insertion(+) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index f38553ff538b..0ec446d0fd6a 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -671,6 +671,7 @@ int mlx5_devlink_register(struct devlink *devlink) void mlx5_devlink_unregister(struct devlink *devlink) { mlx5_devlink_traps_unregister(devlink); + devlink_params_unpublish(devlink); devlink_params_unregister(devlink, mlx5_devlink_params, ARRAY_SIZE(mlx5_devlink_params)); devlink_unregister(devlink); -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 08/10] net/mlx5: Support enable_eth devlink dev param
Enable user to disable Ethernet auxiliary device so that when it is not required, user can disable it. For example, $ devlink dev param set pci/0000:06:00.0 \ name enable_eth value false cmode driverinit $ devlink dev reload pci/0000:06:00.0 At this point devlink instance do not create mlx5_core.eth.2 auxiliary device for the Ethernet functionality. Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- drivers/net/ethernet/mellanox/mlx5/core/dev.c | 42 ++++++++++++++- .../net/ethernet/mellanox/mlx5/core/devlink.c | 53 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/mlx5_core.h | 3 ++ 3 files changed, 96 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c index def2156e50ee..10c4309f29be 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c @@ -53,7 +53,7 @@ static bool is_eth_rep_supported(struct mlx5_core_dev *dev) return true; } -static bool is_eth_supported(struct mlx5_core_dev *dev) +bool mlx5_eth_supported(struct mlx5_core_dev *dev) { if (!IS_ENABLED(CONFIG_MLX5_CORE_EN)) return false; @@ -105,6 +105,17 @@ static bool is_eth_supported(struct mlx5_core_dev *dev) return true; } +static bool is_eth_enabled(struct mlx5_core_dev *dev) +{ + union devlink_param_value val; + int err; + + err = devlink_param_driverinit_value_get(priv_to_devlink(dev), + DEVLINK_PARAM_GENERIC_ID_ENABLE_ETH, + &val); + return err ? false : val.vbool; +} + static bool is_vnet_supported(struct mlx5_core_dev *dev) { if (!IS_ENABLED(CONFIG_MLX5_VDPA_NET)) @@ -201,13 +212,15 @@ enum { static const struct mlx5_adev_device { const char *suffix; bool (*is_supported)(struct mlx5_core_dev *dev); + bool (*is_enabled)(struct mlx5_core_dev *dev); } mlx5_adev_devices[] = { [MLX5_INTERFACE_PROTOCOL_VNET] = { .suffix = "vnet", .is_supported = &is_vnet_supported }, [MLX5_INTERFACE_PROTOCOL_IB] = { .suffix = "rdma", .is_supported = &is_ib_supported }, [MLX5_INTERFACE_PROTOCOL_ETH] = { .suffix = "eth", - .is_supported = &is_eth_supported }, + .is_supported = &mlx5_eth_supported, + .is_enabled = &is_eth_enabled }, [MLX5_INTERFACE_PROTOCOL_ETH_REP] = { .suffix = "eth-rep", .is_supported = &is_eth_rep_supported }, [MLX5_INTERFACE_PROTOCOL_IB_REP] = { .suffix = "rdma-rep", @@ -308,6 +321,14 @@ int mlx5_attach_device(struct mlx5_core_dev *dev) if (!priv->adev[i]) { bool is_supported = false; + if (mlx5_adev_devices[i].is_enabled) { + bool enabled; + + enabled = mlx5_adev_devices[i].is_enabled(dev); + if (!enabled) + continue; + } + if (mlx5_adev_devices[i].is_supported) is_supported = mlx5_adev_devices[i].is_supported(dev); @@ -360,6 +381,14 @@ void mlx5_detach_device(struct mlx5_core_dev *dev) if (!priv->adev[i]) continue; + if (mlx5_adev_devices[i].is_enabled) { + bool enabled; + + enabled = mlx5_adev_devices[i].is_enabled(dev); + if (!enabled) + goto skip_suspend; + } + adev = &priv->adev[i]->adev; /* Auxiliary driver was unbind manually through sysfs */ if (!adev->dev.driver) @@ -447,12 +476,21 @@ static void delete_drivers(struct mlx5_core_dev *dev) if (!priv->adev[i]) continue; + if (mlx5_adev_devices[i].is_enabled) { + bool enabled; + + enabled = mlx5_adev_devices[i].is_enabled(dev); + if (!enabled) + goto del_adev; + } + if (mlx5_adev_devices[i].is_supported && !delete_all) is_supported = mlx5_adev_devices[i].is_supported(dev); if (is_supported) continue; +del_adev: del_adev(&priv->adev[i]->adev); priv->adev[i] = NULL; } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index 0ec446d0fd6a..557973c9212f 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -596,6 +596,52 @@ static void mlx5_devlink_set_params_init_values(struct devlink *devlink) #endif } +static const struct devlink_param enable_eth_param + DEVLINK_PARAM_GENERIC(ENABLE_ETH, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT), + NULL, NULL, NULL); + +static int mlx5_devlink_eth_param_register(struct devlink *devlink) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + union devlink_param_value value; + int err; + + if (!mlx5_eth_supported(dev)) + return 0; + + err = devlink_param_register(devlink, &enable_eth_param); + if (err) + return err; + + value.vbool = true; + devlink_param_driverinit_value_set(devlink, + DEVLINK_PARAM_GENERIC_ID_ENABLE_ETH, + value); + devlink_param_publish(devlink, &enable_eth_param); + return 0; +} + +static void mlx5_devlink_eth_param_unregister(struct devlink *devlink) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + + if (!mlx5_eth_supported(dev)) + return; + + devlink_param_unpublish(devlink, &enable_eth_param); + devlink_param_unregister(devlink, &enable_eth_param); +} + +static int mlx5_devlink_auxdev_params_register(struct devlink *devlink) +{ + return mlx5_devlink_eth_param_register(devlink); +} + +static void mlx5_devlink_auxdev_params_unregister(struct devlink *devlink) +{ + mlx5_devlink_eth_param_unregister(devlink); +} + #define MLX5_TRAP_DROP(_id, _group_id) \ DEVLINK_TRAP_GENERIC(DROP, DROP, _id, \ DEVLINK_TRAP_GROUP_GENERIC_ID_##_group_id, \ @@ -654,6 +700,10 @@ int mlx5_devlink_register(struct devlink *devlink) mlx5_devlink_set_params_init_values(devlink); devlink_params_publish(devlink); + err = mlx5_devlink_auxdev_params_register(devlink); + if (err) + goto auxdev_reg_err; + err = mlx5_devlink_traps_register(devlink); if (err) goto traps_reg_err; @@ -661,6 +711,8 @@ int mlx5_devlink_register(struct devlink *devlink) return 0; traps_reg_err: + mlx5_devlink_auxdev_params_unregister(devlink); +auxdev_reg_err: devlink_params_unregister(devlink, mlx5_devlink_params, ARRAY_SIZE(mlx5_devlink_params)); params_reg_err: @@ -671,6 +723,7 @@ int mlx5_devlink_register(struct devlink *devlink) void mlx5_devlink_unregister(struct devlink *devlink) { mlx5_devlink_traps_unregister(devlink); + mlx5_devlink_auxdev_params_unregister(devlink); devlink_params_unpublish(devlink); devlink_params_unregister(devlink, mlx5_devlink_params, ARRAY_SIZE(mlx5_devlink_params)); diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index 343807ac2036..b3dfecf4f433 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -270,4 +270,7 @@ static inline u32 mlx5_sriov_get_vf_total_msix(struct pci_dev *pdev) return MLX5_CAP_GEN_MAX(dev, num_total_dynamic_vf_msix); } + +bool mlx5_eth_supported(struct mlx5_core_dev *dev); + #endif /* __MLX5_CORE_H__ */ -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 09/10] net/mlx5: Support enable_rdma devlink dev param
Enable user to disable RDMA auxiliary device so that when it is not required, user can disable it. For example, $ devlink dev param set pci/0000:06:00.0 \ name enable_rdma value false cmode driverinit $ devlink dev reload pci/0000:06:00.0 At this point devlink instance do not create auxiliary device mlx5_core.rdma.2 for the RDMA functionality. Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- drivers/net/ethernet/mellanox/mlx5/core/dev.c | 16 ++++- .../net/ethernet/mellanox/mlx5/core/devlink.c | 65 ++++++++++++++++++- .../ethernet/mellanox/mlx5/core/mlx5_core.h | 1 + 3 files changed, 79 insertions(+), 3 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c index 10c4309f29be..cb86844099c0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c @@ -181,7 +181,7 @@ static bool is_mp_supported(struct mlx5_core_dev *dev) return true; } -static bool is_ib_supported(struct mlx5_core_dev *dev) +bool mlx5_rdma_supported(struct mlx5_core_dev *dev) { if (!IS_ENABLED(CONFIG_MLX5_INFINIBAND)) return false; @@ -198,6 +198,17 @@ static bool is_ib_supported(struct mlx5_core_dev *dev) return true; } +static bool is_ib_enabled(struct mlx5_core_dev *dev) +{ + union devlink_param_value val; + int err; + + err = devlink_param_driverinit_value_get(priv_to_devlink(dev), + DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA, + &val); + return err ? false : val.vbool; +} + enum { MLX5_INTERFACE_PROTOCOL_ETH, MLX5_INTERFACE_PROTOCOL_ETH_REP, @@ -217,7 +228,8 @@ static const struct mlx5_adev_device { [MLX5_INTERFACE_PROTOCOL_VNET] = { .suffix = "vnet", .is_supported = &is_vnet_supported }, [MLX5_INTERFACE_PROTOCOL_IB] = { .suffix = "rdma", - .is_supported = &is_ib_supported }, + .is_supported = &mlx5_rdma_supported, + .is_enabled = &is_ib_enabled }, [MLX5_INTERFACE_PROTOCOL_ETH] = { .suffix = "eth", .is_supported = &mlx5_eth_supported, .is_enabled = &is_eth_enabled }, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index 557973c9212f..f247ffb325a9 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -632,13 +632,76 @@ static void mlx5_devlink_eth_param_unregister(struct devlink *devlink) devlink_param_unregister(devlink, &enable_eth_param); } +static int mlx5_devlink_enable_rdma_validate(struct devlink *devlink, u32 id, + union devlink_param_value val, + struct netlink_ext_ack *extack) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + bool new_state = val.vbool; + + if (new_state && !mlx5_rdma_supported(dev)) + return -EOPNOTSUPP; + return 0; +} + +static const struct devlink_param enable_rdma_param + DEVLINK_PARAM_GENERIC(ENABLE_RDMA, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT), + NULL, NULL, mlx5_devlink_enable_rdma_validate); + +static int mlx5_devlink_rdma_param_register(struct devlink *devlink) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + union devlink_param_value value; + int err; + + if (!IS_ENABLED(CONFIG_MLX5_INFINIBAND) || MLX5_ESWITCH_MANAGER(dev)) + return 0; + + err = devlink_param_register(devlink, &enable_rdma_param); + if (err) + return err; + + value.vbool = true; + devlink_param_driverinit_value_set(devlink, + DEVLINK_PARAM_GENERIC_ID_ENABLE_RDMA, + value); + devlink_param_publish(devlink, &enable_rdma_param); + return 0; +} + +static void mlx5_devlink_rdma_param_unregister(struct devlink *devlink) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + + if (!IS_ENABLED(CONFIG_MLX5_INFINIBAND) || MLX5_ESWITCH_MANAGER(dev)) + return; + + devlink_param_unpublish(devlink, &enable_rdma_param); + devlink_param_unregister(devlink, &enable_rdma_param); +} + static int mlx5_devlink_auxdev_params_register(struct devlink *devlink) { - return mlx5_devlink_eth_param_register(devlink); + int err; + + err = mlx5_devlink_eth_param_register(devlink); + if (err) + return err; + + err = mlx5_devlink_rdma_param_register(devlink); + if (err) + goto rdma_err; + + return 0; + +rdma_err: + mlx5_devlink_eth_param_unregister(devlink); + return err; } static void mlx5_devlink_auxdev_params_unregister(struct devlink *devlink) { + mlx5_devlink_rdma_param_unregister(devlink); mlx5_devlink_eth_param_unregister(devlink); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index b3dfecf4f433..b36fbcdc048e 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -272,5 +272,6 @@ static inline u32 mlx5_sriov_get_vf_total_msix(struct pci_dev *pdev) } bool mlx5_eth_supported(struct mlx5_core_dev *dev); +bool mlx5_rdma_supported(struct mlx5_core_dev *dev); #endif /* __MLX5_CORE_H__ */ -- 2.26.2
Parav Pandit
2021-Aug-10 13:24 UTC
[PATCH RESEND net-next 10/10] net/mlx5: Support enable_vnet devlink dev param
Enable user to disable VDPA net auxiliary device so that when it is not required, user can disable it. For example, $ devlink dev param set pci/0000:06:00.0 \ name enable_vnet value false cmode driverinit $ devlink dev reload pci/0000:06:00.0 At this point devlink instance do not create auxiliary device mlx5_core.vnet.2 for the VDPA net functionality. Signed-off-by: Parav Pandit <parav at nvidia.com> Reviewed-by: Leon Romanovsky <leonro at nvidia.com> --- drivers/net/ethernet/mellanox/mlx5/core/dev.c | 16 ++++++- .../net/ethernet/mellanox/mlx5/core/devlink.c | 42 +++++++++++++++++++ .../ethernet/mellanox/mlx5/core/mlx5_core.h | 1 + 3 files changed, 57 insertions(+), 2 deletions(-) diff --git a/drivers/net/ethernet/mellanox/mlx5/core/dev.c b/drivers/net/ethernet/mellanox/mlx5/core/dev.c index cb86844099c0..ff6b03dc7e32 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/dev.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/dev.c @@ -116,7 +116,7 @@ static bool is_eth_enabled(struct mlx5_core_dev *dev) return err ? false : val.vbool; } -static bool is_vnet_supported(struct mlx5_core_dev *dev) +bool mlx5_vnet_supported(struct mlx5_core_dev *dev) { if (!IS_ENABLED(CONFIG_MLX5_VDPA_NET)) return false; @@ -138,6 +138,17 @@ static bool is_vnet_supported(struct mlx5_core_dev *dev) return true; } +static bool is_vnet_enabled(struct mlx5_core_dev *dev) +{ + union devlink_param_value val; + int err; + + err = devlink_param_driverinit_value_get(priv_to_devlink(dev), + DEVLINK_PARAM_GENERIC_ID_ENABLE_VNET, + &val); + return err ? false : val.vbool; +} + static bool is_ib_rep_supported(struct mlx5_core_dev *dev) { if (!IS_ENABLED(CONFIG_MLX5_INFINIBAND)) @@ -226,7 +237,8 @@ static const struct mlx5_adev_device { bool (*is_enabled)(struct mlx5_core_dev *dev); } mlx5_adev_devices[] = { [MLX5_INTERFACE_PROTOCOL_VNET] = { .suffix = "vnet", - .is_supported = &is_vnet_supported }, + .is_supported = &mlx5_vnet_supported, + .is_enabled = &is_vnet_enabled }, [MLX5_INTERFACE_PROTOCOL_IB] = { .suffix = "rdma", .is_supported = &mlx5_rdma_supported, .is_enabled = &is_ib_enabled }, diff --git a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c index f247ffb325a9..6f4d7c7f06e0 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/devlink.c +++ b/drivers/net/ethernet/mellanox/mlx5/core/devlink.c @@ -680,6 +680,42 @@ static void mlx5_devlink_rdma_param_unregister(struct devlink *devlink) devlink_param_unregister(devlink, &enable_rdma_param); } +static const struct devlink_param enable_vnet_param + DEVLINK_PARAM_GENERIC(ENABLE_VNET, BIT(DEVLINK_PARAM_CMODE_DRIVERINIT), + NULL, NULL, NULL); + +static int mlx5_devlink_vnet_param_register(struct devlink *devlink) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + union devlink_param_value value; + int err; + + if (!mlx5_vnet_supported(dev)) + return 0; + + err = devlink_param_register(devlink, &enable_vnet_param); + if (err) + return err; + + value.vbool = true; + devlink_param_driverinit_value_set(devlink, + DEVLINK_PARAM_GENERIC_ID_ENABLE_VNET, + value); + devlink_param_publish(devlink, &enable_rdma_param); + return 0; +} + +static void mlx5_devlink_vnet_param_unregister(struct devlink *devlink) +{ + struct mlx5_core_dev *dev = devlink_priv(devlink); + + if (!mlx5_vnet_supported(dev)) + return; + + devlink_param_unpublish(devlink, &enable_vnet_param); + devlink_param_unregister(devlink, &enable_vnet_param); +} + static int mlx5_devlink_auxdev_params_register(struct devlink *devlink) { int err; @@ -692,8 +728,13 @@ static int mlx5_devlink_auxdev_params_register(struct devlink *devlink) if (err) goto rdma_err; + err = mlx5_devlink_vnet_param_register(devlink); + if (err) + goto vnet_err; return 0; +vnet_err: + mlx5_devlink_rdma_param_unregister(devlink); rdma_err: mlx5_devlink_eth_param_unregister(devlink); return err; @@ -701,6 +742,7 @@ static int mlx5_devlink_auxdev_params_register(struct devlink *devlink) static void mlx5_devlink_auxdev_params_unregister(struct devlink *devlink) { + mlx5_devlink_vnet_param_unregister(devlink); mlx5_devlink_rdma_param_unregister(devlink); mlx5_devlink_eth_param_unregister(devlink); } diff --git a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h index b36fbcdc048e..2059b7319867 100644 --- a/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h +++ b/drivers/net/ethernet/mellanox/mlx5/core/mlx5_core.h @@ -273,5 +273,6 @@ static inline u32 mlx5_sriov_get_vf_total_msix(struct pci_dev *pdev) bool mlx5_eth_supported(struct mlx5_core_dev *dev); bool mlx5_rdma_supported(struct mlx5_core_dev *dev); +bool mlx5_vnet_supported(struct mlx5_core_dev *dev); #endif /* __MLX5_CORE_H__ */ -- 2.26.2