Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time
From: Sui Jingfeng <suijingfeng at loongson.cn> On a machine with multiple GPUs, a Linux user has no control over which one is primary at boot time. This series tries to solve above mentioned problem by introduced the ->be_primary() function stub. The specific device drivers can provide an implementation to hook up with this stub by calling the vga_client_register() function. Once the driver bound the device successfully, VGAARB will call back to the device driver. To query if the device drivers want to be primary or not. Device drivers can just pass NULL if have no such needs. Please note that: 1) The ARM64, Loongarch, Mips servers have a lot PCIe slot, and I would like to mount at least three video cards. 2) Typically, those non-86 machines don't have a good UEFI firmware support, which doesn't support select primary GPU as firmware stage. Even on x86, there are old UEFI firmwares which already made undesired decision for you. 3) This series is attempt to solve the remain problems at the driver level, while another series[1] of me is target to solve the majority of the problems at device level. Tested (limited) on x86 with four video card mounted, Intel UHD Graphics 630 is the default boot VGA, successfully override by ast2400 with ast.modeset=10 append at the kernel cmd line. $ lspci | grep VGA 00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos XTX [Radeon HD 8490 / R5 235X OEM] 04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30) 05:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 720] (rev a1) $ sudo dmesg | grep vgaarb pci 0000:00:02.0: vgaarb: setting as boot VGA device pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none pci 0000:04:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none pci 0000:05:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none vgaarb: loaded ast 0000:04:00.0: vgaarb: Override as primary by driver i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem radeon 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none ast 0000:04:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none v2: * Add a simple implemment for drm/i915 and drm/ast * Pick up all tags (Mario) v3: * Fix a mistake for drm/i915 implement * Fix patch can not be applied problem because of merge conflect. v4: * Focus on solve the real problem. v1,v2 at https://patchwork.freedesktop.org/series/120059/ v3 at https://patchwork.freedesktop.org/series/120562/ [1] https://patchwork.freedesktop.org/series/122845/ Sui Jingfeng (9): PCI/VGA: Allowing the user to select the primary video adapter at boot time drm/nouveau: Implement .be_primary() callback drm/radeon: Implement .be_primary() callback drm/amdgpu: Implement .be_primary() callback drm/i915: Implement .be_primary() callback drm/loongson: Implement .be_primary() callback drm/ast: Register as a VGA client by calling vga_client_register() drm/hibmc: Register as a VGA client by calling vga_client_register() drm/gma500: Register as a VGA client by calling vga_client_register() drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 +++- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++- drivers/gpu/drm/ast/ast_drv.c | 31 ++++++++++ drivers/gpu/drm/gma500/psb_drv.c | 57 ++++++++++++++++++- .../gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c | 15 +++++ drivers/gpu/drm/i915/display/intel_vga.c | 15 ++++- drivers/gpu/drm/loongson/loongson_module.c | 2 +- drivers/gpu/drm/loongson/loongson_module.h | 1 + drivers/gpu/drm/loongson/lsdc_drv.c | 10 +++- drivers/gpu/drm/nouveau/nouveau_vga.c | 11 +++- drivers/gpu/drm/radeon/radeon_device.c | 10 +++- drivers/pci/vgaarb.c | 43 ++++++++++++-- drivers/vfio/pci/vfio_pci_core.c | 2 +- include/linux/vgaarb.h | 8 ++- 14 files changed, 210 insertions(+), 19 deletions(-) -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 1/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time
From: Sui Jingfeng <suijingfeng at loongson.cn> On a machine with multiple GPUs, a Linux user has no control over which one is primary at boot time. This series tries to solve above mentioned problem by introduced the ->be_primary() function stub. The specific device drivers can provide an implementation to hook up with this stub by calling the vga_client_register() function. Once the driver bound the device successfully, VGAARB will call back to the device driver. To query if the device drivers want to be primary or not. Device drivers can just pass NULL if have no such needs. Acked-by: Jani Nikula <jani.nikula at intel.com> # i915 Reviewed-by: Lyude Paul <lyude at redhat.com> # nouveau Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 +- drivers/gpu/drm/i915/display/intel_vga.c | 3 +- drivers/gpu/drm/loongson/lsdc_drv.c | 2 +- drivers/gpu/drm/nouveau/nouveau_vga.c | 2 +- drivers/gpu/drm/radeon/radeon_device.c | 2 +- drivers/pci/vgaarb.c | 43 +++++++++++++++++++--- drivers/vfio/pci/vfio_pci_core.c | 2 +- include/linux/vgaarb.h | 8 ++-- 8 files changed, 49 insertions(+), 15 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index e77f048c99d8..ecc4564ceac0 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3916,7 +3916,7 @@ int amdgpu_device_init(struct amdgpu_device *adev, * ignore it */ if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA) - vga_client_register(adev->pdev, amdgpu_device_vga_set_decode); + vga_client_register(adev->pdev, amdgpu_device_vga_set_decode, NULL); px = amdgpu_device_supports_px(ddev); diff --git a/drivers/gpu/drm/i915/display/intel_vga.c b/drivers/gpu/drm/i915/display/intel_vga.c index 286a0bdd28c6..98d7d4dffe9f 100644 --- a/drivers/gpu/drm/i915/display/intel_vga.c +++ b/drivers/gpu/drm/i915/display/intel_vga.c @@ -115,7 +115,6 @@ intel_vga_set_decode(struct pci_dev *pdev, bool enable_decode) int intel_vga_register(struct drm_i915_private *i915) { - struct pci_dev *pdev = to_pci_dev(i915->drm.dev); int ret; @@ -127,7 +126,7 @@ int intel_vga_register(struct drm_i915_private *i915) * then we do not take part in VGA arbitration and the * vga_client_register() fails with -ENODEV. */ - ret = vga_client_register(pdev, intel_vga_set_decode); + ret = vga_client_register(pdev, intel_vga_set_decode, NULL); if (ret && ret != -ENODEV) return ret; diff --git a/drivers/gpu/drm/loongson/lsdc_drv.c b/drivers/gpu/drm/loongson/lsdc_drv.c index 188ec82afcfb..d10a28c2c494 100644 --- a/drivers/gpu/drm/loongson/lsdc_drv.c +++ b/drivers/gpu/drm/loongson/lsdc_drv.c @@ -289,7 +289,7 @@ static int lsdc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) pci_set_drvdata(pdev, ddev); - vga_client_register(pdev, lsdc_vga_set_decode); + vga_client_register(pdev, lsdc_vga_set_decode, NULL); drm_kms_helper_poll_init(ddev); diff --git a/drivers/gpu/drm/nouveau/nouveau_vga.c b/drivers/gpu/drm/nouveau/nouveau_vga.c index f8bf0ec26844..162b4f4676c7 100644 --- a/drivers/gpu/drm/nouveau/nouveau_vga.c +++ b/drivers/gpu/drm/nouveau/nouveau_vga.c @@ -92,7 +92,7 @@ nouveau_vga_init(struct nouveau_drm *drm) return; pdev = to_pci_dev(dev->dev); - vga_client_register(pdev, nouveau_vga_set_decode); + vga_client_register(pdev, nouveau_vga_set_decode, NULL); /* don't register Thunderbolt eGPU with vga_switcheroo */ if (pci_is_thunderbolt_attached(pdev)) diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c index afbb3a80c0c6..71f2ff39d6a1 100644 --- a/drivers/gpu/drm/radeon/radeon_device.c +++ b/drivers/gpu/drm/radeon/radeon_device.c @@ -1425,7 +1425,7 @@ int radeon_device_init(struct radeon_device *rdev, /* if we have > 1 VGA cards, then disable the radeon VGA resources */ /* this will fail for cards that aren't VGA class devices, just * ignore it */ - vga_client_register(rdev->pdev, radeon_vga_set_decode); + vga_client_register(rdev->pdev, radeon_vga_set_decode, NULL); if (rdev->flags & RADEON_IS_PX) runtime = true; diff --git a/drivers/pci/vgaarb.c b/drivers/pci/vgaarb.c index 5a696078b382..552ac7df10ee 100644 --- a/drivers/pci/vgaarb.c +++ b/drivers/pci/vgaarb.c @@ -53,6 +53,7 @@ struct vga_device { bool bridge_has_one_vga; bool is_firmware_default; /* device selected by firmware */ unsigned int (*set_decode)(struct pci_dev *pdev, bool decode); + bool (*be_primary)(struct pci_dev *pdev); }; static LIST_HEAD(vga_list); @@ -956,6 +957,10 @@ EXPORT_SYMBOL(vga_set_legacy_decoding); * @set_decode callback: If a client can disable its GPU VGA resource, it * will get a callback from this to set the encode/decode state. * + * @be_primary callback: Callback to the device driver, query if a device + * want to be the primary display. This callback is optional, device drivers + * who have no special needs can simply pass a NULL. + * * Rationale: we cannot disable VGA decode resources unconditionally some single * GPU laptops seem to require ACPI or BIOS access to the VGA registers to * control things like backlights etc. Hopefully newer multi-GPU laptops do @@ -971,7 +976,8 @@ EXPORT_SYMBOL(vga_set_legacy_decoding); * Returns: 0 on success, -1 on failure */ int vga_client_register(struct pci_dev *pdev, - unsigned int (*set_decode)(struct pci_dev *pdev, bool decode)) + unsigned int (*set_decode)(struct pci_dev *pdev, bool decode), + bool (*be_primary)(struct pci_dev *pdev)) { int ret = -ENODEV; struct vga_device *vgadev; @@ -983,6 +989,7 @@ int vga_client_register(struct pci_dev *pdev, goto bail; vgadev->set_decode = set_decode; + vgadev->be_primary = be_primary; ret = 0; bail: @@ -1493,6 +1500,21 @@ static void vga_arbiter_notify_clients(void) spin_unlock_irqrestore(&vga_lock, flags); } +static void vga_arbiter_do_arbitration(struct pci_dev *pdev) +{ + struct vga_device *vgadev; + + if (pdev == vga_default_device()) + return; + + vgadev = vgadev_find(pdev); + if (vgadev && vgadev->be_primary && vgadev->be_primary(pdev)) { + vga_set_default_device(pdev); + + vgaarb_info(&pdev->dev, "Override as primary by driver\n"); + } +} + static int pci_notify(struct notifier_block *nb, unsigned long action, void *data) { @@ -1505,13 +1527,24 @@ static int pci_notify(struct notifier_block *nb, unsigned long action, /* For now we're only intereted in devices added and removed. I didn't * test this thing here, so someone needs to double check for the * cases of hotplugable vga cards. */ - if (action == BUS_NOTIFY_ADD_DEVICE) + switch (action) { + case BUS_NOTIFY_ADD_DEVICE: notify = vga_arbiter_add_pci_device(pdev); - else if (action == BUS_NOTIFY_DEL_DEVICE) + if (notify) + vga_arbiter_notify_clients(); + break; + case BUS_NOTIFY_DEL_DEVICE: notify = vga_arbiter_del_pci_device(pdev); + if (notify) + vga_arbiter_notify_clients(); + break; + case BUS_NOTIFY_BOUND_DRIVER: + vga_arbiter_do_arbitration(pdev); + break; + default: + break; + } - if (notify) - vga_arbiter_notify_clients(); return 0; } diff --git a/drivers/vfio/pci/vfio_pci_core.c b/drivers/vfio/pci/vfio_pci_core.c index 20d7b69ea6ff..531c4d8ef26e 100644 --- a/drivers/vfio/pci/vfio_pci_core.c +++ b/drivers/vfio/pci/vfio_pci_core.c @@ -2108,7 +2108,7 @@ static int vfio_pci_vga_init(struct vfio_pci_core_device *vdev) if (ret) return ret; - ret = vga_client_register(pdev, vfio_pci_set_decode); + ret = vga_client_register(pdev, vfio_pci_set_decode, NULL); if (ret) return ret; vga_set_legacy_decoding(pdev, vfio_pci_set_decode(pdev, false)); diff --git a/include/linux/vgaarb.h b/include/linux/vgaarb.h index b4b9137f9792..291d82b9f05a 100644 --- a/include/linux/vgaarb.h +++ b/include/linux/vgaarb.h @@ -52,7 +52,8 @@ struct pci_dev *vga_default_device(void); void vga_set_default_device(struct pci_dev *pdev); int vga_remove_vgacon(struct pci_dev *pdev); int vga_client_register(struct pci_dev *pdev, - unsigned int (*set_decode)(struct pci_dev *pdev, bool state)); + unsigned int (*set_decode)(struct pci_dev *pdev, bool state), + bool (*be_primary)(struct pci_dev *pdev)); #else /* CONFIG_VGA_ARB */ static inline void vga_set_legacy_decoding(struct pci_dev *pdev, unsigned int decodes) @@ -78,7 +79,8 @@ static inline int vga_remove_vgacon(struct pci_dev *pdev) return 0; } static inline int vga_client_register(struct pci_dev *pdev, - unsigned int (*set_decode)(struct pci_dev *pdev, bool state)) + unsigned int (*set_decode)(struct pci_dev *pdev, bool state), + bool (*be_primary)(struct pci_dev *pdev)) { return 0; } @@ -116,7 +118,7 @@ static inline int vga_get_uninterruptible(struct pci_dev *pdev, static inline void vga_client_unregister(struct pci_dev *pdev) { - vga_client_register(pdev, NULL); + vga_client_register(pdev, NULL, NULL); } #endif /* LINUX_VGA_H */ -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 2/9] drm/nouveau: Implement .be_primary() callback
From: Sui Jingfeng <suijingfeng at loongson.cn> On a machine with multiple GPUs, a Linux user has no control over which one is primary at boot time. This patch tries to solve the mentioned problem by implementing the .be_primary() callback. VGAARB will call back to Nouveau when the drm/nouveau gets loaded successfully. Pass nouveau.modeset=10 on the kernel cmd line if you really want the device bound by Nouveau to be the primary video adapter. This overrides whatever boot device selected by VGAARB. Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/nouveau/nouveau_vga.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/nouveau/nouveau_vga.c b/drivers/gpu/drm/nouveau/nouveau_vga.c index 162b4f4676c7..4242188667e2 100644 --- a/drivers/gpu/drm/nouveau/nouveau_vga.c +++ b/drivers/gpu/drm/nouveau/nouveau_vga.c @@ -80,6 +80,15 @@ nouveau_switcheroo_ops = { .can_switch = nouveau_switcheroo_can_switch, }; +static bool +nouveau_want_to_be_primary(struct pci_dev *pdev) +{ + if (nouveau_modeset == 10) + return true; + + return false; +} + void nouveau_vga_init(struct nouveau_drm *drm) { @@ -92,7 +101,7 @@ nouveau_vga_init(struct nouveau_drm *drm) return; pdev = to_pci_dev(dev->dev); - vga_client_register(pdev, nouveau_vga_set_decode, NULL); + vga_client_register(pdev, nouveau_vga_set_decode, nouveau_want_to_be_primary); /* don't register Thunderbolt eGPU with vga_switcheroo */ if (pci_is_thunderbolt_attached(pdev)) -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 3/9] drm/radeon: Implement .be_primary() callback
From: Sui Jingfeng <suijingfeng at loongson.cn> On a machine with multiple GPUs, a Linux user has no control over which one is primary at boot time. This patch tries to solve the mentioned problem by implementing the .be_primary() callback. Pass radeon.modeset=10 on the kernel cmd line if you really want the device bound by radeon to be the primary video adapter, no matter what VGAARB say. Cc: Alex Deucher <alexander.deucher at amd.com> Cc: Christian Koenig <christian.koenig at amd.com> Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/radeon/radeon_device.c | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/radeon/radeon_device.c b/drivers/gpu/drm/radeon/radeon_device.c index 71f2ff39d6a1..b661cd3a8dc2 100644 --- a/drivers/gpu/drm/radeon/radeon_device.c +++ b/drivers/gpu/drm/radeon/radeon_device.c @@ -1263,6 +1263,14 @@ static const struct vga_switcheroo_client_ops radeon_switcheroo_ops = { .can_switch = radeon_switcheroo_can_switch, }; +static bool radeon_want_to_be_primary(struct pci_dev *pdev) +{ + if (radeon_modeset == 10) + return true; + + return false; +} + /** * radeon_device_init - initialize the driver * @@ -1425,7 +1433,7 @@ int radeon_device_init(struct radeon_device *rdev, /* if we have > 1 VGA cards, then disable the radeon VGA resources */ /* this will fail for cards that aren't VGA class devices, just * ignore it */ - vga_client_register(rdev->pdev, radeon_vga_set_decode, NULL); + vga_client_register(rdev->pdev, radeon_vga_set_decode, radeon_want_to_be_primary); if (rdev->flags & RADEON_IS_PX) runtime = true; -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 4/9] drm/amdgpu: Implement .be_primary() callback
From: Sui Jingfeng <suijingfeng at loongson.cn> On a machine with multiple GPUs, a Linux user has no control over which one is primary at boot time. This patch tries to solve the mentioned problem by implementing the .be_primary() callback. Pass amdgpu.modeset=10 on the kernel cmd line if you really want the device bound by amdgpu drm driver to be the primary video adapter, no matter what VGAARB say. Cc: Alex Deucher <alexander.deucher at amd.com> Cc: Christian Konig <christian.koenig at amd.com> Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 ++++++++++- drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++++++++++- 2 files changed, 22 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c index ecc4564ceac0..59bde6972a8b 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c @@ -3507,6 +3507,14 @@ static void amdgpu_device_set_mcbp(struct amdgpu_device *adev) DRM_INFO("MCBP is enabled\n"); } +static bool amdgpu_want_to_be_primary(struct pci_dev *pdev) +{ + if (amdgpu_modeset == 10) + return true; + + return false; +} + /** * amdgpu_device_init - initialize the driver * @@ -3916,7 +3924,8 @@ int amdgpu_device_init(struct amdgpu_device *adev, * ignore it */ if ((adev->pdev->class >> 8) == PCI_CLASS_DISPLAY_VGA) - vga_client_register(adev->pdev, amdgpu_device_vga_set_decode, NULL); + vga_client_register(adev->pdev, amdgpu_device_vga_set_decode, + amdgpu_want_to_be_primary); px = amdgpu_device_supports_px(ddev); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c index 81edf66dbea8..2592e24ce62c 100644 --- a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c +++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c @@ -118,6 +118,7 @@ #define KMS_DRIVER_MINOR 54 #define KMS_DRIVER_PATCHLEVEL 0 +int amdgpu_modeset = -1; unsigned int amdgpu_vram_limit = UINT_MAX; int amdgpu_vis_vram_limit; int amdgpu_gart_size = -1; /* auto */ @@ -223,6 +224,13 @@ struct amdgpu_watchdog_timer amdgpu_watchdog_timer = { .period = 0x0, /* default to 0x0 (timeout disable) */ }; +/** + * DOC: modeset (int) + * Disable/Enable kernel modesetting (1 = enable, 0 = disable, -1 = auto (default)). + */ +MODULE_PARM_DESC(modeset, "Disable/Enable kernel modesetting"); +module_param_named(modeset, amdgpu_modeset, int, 0600); + /** * DOC: vramlimit (int) * Restrict the total amount of VRAM in MiB for testing. The default is 0 (Use full VRAM). @@ -2872,7 +2880,10 @@ static int __init amdgpu_init(void) { int r; - if (drm_firmware_drivers_only()) + if (drm_firmware_drivers_only() && amdgpu_modeset == -1) + return -EINVAL; + + if (amdgpu_modeset == 0) return -EINVAL; r = amdgpu_sync_init(); -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 5/9] drm/i915: Implement .be_primary() callback
From: Sui Jingfeng <suijingfeng at loongson.cn> On a machine with multiple GPUs, a Linux user has no control over which one is primary at boot time. This patch tries to solve the mentioned problem by implementing the .be_primary() callback. Pass i915.modeset=10 on the kernel cmd line if you really want the device bound by i915 drm driver to be the primary video adapter, no matter what VGAARB say. Cc: Jani Nikula <jani.nikula at linux.intel.com> Cc: David Airlie <airlied at gmail.com> Cc: Daniel Vetter <daniel at ffwll.ch> Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/i915/display/intel_vga.c | 14 +++++++++++++- 1 file changed, 13 insertions(+), 1 deletion(-) diff --git a/drivers/gpu/drm/i915/display/intel_vga.c b/drivers/gpu/drm/i915/display/intel_vga.c index 98d7d4dffe9f..e3f78ba2668b 100644 --- a/drivers/gpu/drm/i915/display/intel_vga.c +++ b/drivers/gpu/drm/i915/display/intel_vga.c @@ -113,6 +113,17 @@ intel_vga_set_decode(struct pci_dev *pdev, bool enable_decode) return VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM; } +static bool intel_want_to_be_primary(struct pci_dev *pdev) +{ + struct drm_i915_private *i915 = pdev_to_i915(pdev); + struct i915_params *params = &i915->params; + + if (params->modeset == 10) + return true; + + return false; +} + int intel_vga_register(struct drm_i915_private *i915) { struct pci_dev *pdev = to_pci_dev(i915->drm.dev); @@ -126,7 +137,8 @@ int intel_vga_register(struct drm_i915_private *i915) * then we do not take part in VGA arbitration and the * vga_client_register() fails with -ENODEV. */ - ret = vga_client_register(pdev, intel_vga_set_decode, NULL); + ret = vga_client_register(pdev, intel_vga_set_decode, + intel_want_to_be_primary); if (ret && ret != -ENODEV) return ret; -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 6/9] drm/loongson: Implement .be_primary() callback
From: Sui Jingfeng <suijingfeng at loongson.cn> On a machine with multiple GPUs, a Linux user has no control over which one is primary at boot time. This patch tries to solve the mentioned problem by implementing the .be_primary() callback. Pass loongson.modeset=10 on the kernel cmd line if you really want the device bound by loongson drm driver to be the primary video adapter, no matter what VGAARB say. Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/loongson/loongson_module.c | 2 +- drivers/gpu/drm/loongson/loongson_module.h | 1 + drivers/gpu/drm/loongson/lsdc_drv.c | 10 +++++++++- 3 files changed, 11 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/loongson/loongson_module.c b/drivers/gpu/drm/loongson/loongson_module.c index d2a51bd395f6..12f2a453adff 100644 --- a/drivers/gpu/drm/loongson/loongson_module.c +++ b/drivers/gpu/drm/loongson/loongson_module.c @@ -9,7 +9,7 @@ #include "loongson_module.h" -static int loongson_modeset = -1; +int loongson_modeset = -1; MODULE_PARM_DESC(modeset, "Disable/Enable modesetting"); module_param_named(modeset, loongson_modeset, int, 0400); diff --git a/drivers/gpu/drm/loongson/loongson_module.h b/drivers/gpu/drm/loongson/loongson_module.h index 931c17521bf0..afff51e7f34f 100644 --- a/drivers/gpu/drm/loongson/loongson_module.h +++ b/drivers/gpu/drm/loongson/loongson_module.h @@ -6,6 +6,7 @@ #ifndef __LOONGSON_MODULE_H__ #define __LOONGSON_MODULE_H__ +extern int loongson_modeset; extern int loongson_vblank; extern struct pci_driver lsdc_pci_driver; diff --git a/drivers/gpu/drm/loongson/lsdc_drv.c b/drivers/gpu/drm/loongson/lsdc_drv.c index d10a28c2c494..7183b0666167 100644 --- a/drivers/gpu/drm/loongson/lsdc_drv.c +++ b/drivers/gpu/drm/loongson/lsdc_drv.c @@ -257,6 +257,14 @@ static unsigned int lsdc_vga_set_decode(struct pci_dev *pdev, bool state) return VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM; } +static bool lsdc_want_to_be_primary(struct pci_dev *pdev) +{ + if (loongson_modeset == 10) + return true; + + return false; +} + static int lsdc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { const struct lsdc_desc *descp; @@ -289,7 +297,7 @@ static int lsdc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) pci_set_drvdata(pdev, ddev); - vga_client_register(pdev, lsdc_vga_set_decode, NULL); + vga_client_register(pdev, lsdc_vga_set_decode, lsdc_want_to_be_primary); drm_kms_helper_poll_init(ddev); -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 7/9] drm/ast: Register as a VGA client by calling vga_client_register()
From: Sui Jingfeng <suijingfeng at loongson.cn> Becasuse the display controller in the ASpeed BMC chip is a VGA-compatible device, the software programming guide of AST2400 say that it is fully IBM VGA compliant. Thus, it should also participate in the arbitration. Cc: Thomas Zimmermann <tzimmermann at suse.de> Cc: Jocelyn Falempe <jfalempe at redhat.com> Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/ast/ast_drv.c | 31 +++++++++++++++++++++++++++++++ 1 file changed, 31 insertions(+) diff --git a/drivers/gpu/drm/ast/ast_drv.c b/drivers/gpu/drm/ast/ast_drv.c index e1224ef4ad83..1349f7bb5dfb 100644 --- a/drivers/gpu/drm/ast/ast_drv.c +++ b/drivers/gpu/drm/ast/ast_drv.c @@ -28,6 +28,7 @@ #include <linux/module.h> #include <linux/pci.h> +#include <linux/vgaarb.h> #include <drm/drm_aperture.h> #include <drm/drm_atomic_helper.h> @@ -89,6 +90,34 @@ static const struct pci_device_id ast_pciidlist[] = { MODULE_DEVICE_TABLE(pci, ast_pciidlist); +static bool ast_want_to_be_primary(struct pci_dev *pdev) +{ + if (ast_modeset == 10) + return true; + + return false; +} + +static unsigned int ast_vga_set_decode(struct pci_dev *pdev, bool state) +{ + struct drm_device *drm = pci_get_drvdata(pdev); + struct ast_device *ast = to_ast_device(drm); + unsigned int decode; + + if (state) { + /* Enable standard VGA decode and Enable normal VGA decode */ + ast_set_index_reg(ast, AST_IO_CRTC_PORT, 0xa1, 0x04); + + decode = VGA_RSRC_LEGACY_IO | VGA_RSRC_LEGACY_MEM | + VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM; + } else { + ast_set_index_reg(ast, AST_IO_CRTC_PORT, 0xa1, 0x07); + decode = VGA_RSRC_NORMAL_IO | VGA_RSRC_NORMAL_MEM; + } + + return decode; +} + static int ast_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { struct ast_device *ast; @@ -112,6 +141,8 @@ static int ast_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) if (ret) return ret; + vga_client_register(pdev, ast_vga_set_decode, ast_want_to_be_primary); + drm_fbdev_generic_setup(dev, 32); return 0; -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 8/9] drm/hibmc: Register as a VGA client by calling vga_client_register()
From: Sui Jingfeng <suijingfeng at loongson.cn> Because the display controller in the Hibmc chip is a VGA compatible display controller. Because ARM64 doesn't need the VGA console. It does not need to worry about the side effects that come with the VGA compatible. However, the real problem is that some ARM64 PCs and servers do not have good UEFI firmware support. At least, it is not as good as UEFI firmware for x86. The Huawei KunPeng 920 PC and Taishan 100 server are examples. When a discrete GPU is mounted on such machines, the UEFI firmware still selects the integrated display controller (in the BMC) as the primary GPU. It is hardcoded, no options are provided for selection. A Linux user has no control at all. Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c | 15 +++++++++++++++ 1 file changed, 15 insertions(+) diff --git a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c index 8a98fa276e8a..73a3f1cb109a 100644 --- a/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c +++ b/drivers/gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c @@ -13,6 +13,7 @@ #include <linux/module.h> #include <linux/pci.h> +#include <linux/vgaarb.h> #include <drm/drm_aperture.h> #include <drm/drm_atomic_helper.h> @@ -27,6 +28,10 @@ #include "hibmc_drm_drv.h" #include "hibmc_drm_regs.h" +static int hibmc_modeset = -1; +MODULE_PARM_DESC(modeset, "Disable/Enable modesetting"); +module_param_named(modeset, hibmc_modeset, int, 0400); + DEFINE_DRM_GEM_FOPS(hibmc_fops); static irqreturn_t hibmc_interrupt(int irq, void *arg) @@ -299,6 +304,14 @@ static int hibmc_load(struct drm_device *dev) return ret; } +static bool hibmc_want_to_be_primary(struct pci_dev *pdev) +{ + if (hibmc_modeset == 10) + return true; + + return false; +} + static int hibmc_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { @@ -339,6 +352,8 @@ static int hibmc_pci_probe(struct pci_dev *pdev, goto err_unload; } + vga_client_register(pdev, NULL, hibmc_want_to_be_primary); + drm_fbdev_generic_setup(dev, 32); return 0; -- 2.34.1
Sui Jingfeng
2023-Sep-04 19:57 UTC
[Nouveau] [RFC, drm-misc-next v4 9/9] drm/gma500: Register as a VGA client by calling vga_client_register()
From: Sui Jingfeng <suijingfeng at loongson.cn> Because the display controller in N2000/D2000 series can be VGA-compatible, so let's register gma500 as a VGA client, despite the firmware may alter the PCI class code of IGD on a multiple GPU co-exist configuration. But this commit no crime, because VGAARB only cares about VGA devices. Noticed that the display controller in N2000/D2000 processor don't has a valid VRAM BAR, the firmware put the EFI firmware framebuffer into the stolen memory, so the commit <86fd887b7fe3> ("vgaarb: Don't default exclusively to first video device with mem+io") is not effictive on such a case. But the benefits of the stolen memory is that it will not suffer from PCI resource relocation. Becase the stolen memory is carved out by the firmware and reside in system RAM. Therefore, while at it, provided a naive version of firmware framebuffer identification function and use the new machanism just created. Signed-off-by: Sui Jingfeng <suijingfeng at loongson.cn> --- drivers/gpu/drm/gma500/psb_drv.c | 57 ++++++++++++++++++++++++++++++-- 1 file changed, 55 insertions(+), 2 deletions(-) diff --git a/drivers/gpu/drm/gma500/psb_drv.c b/drivers/gpu/drm/gma500/psb_drv.c index 8b64f61ffaf9..eb95d030d981 100644 --- a/drivers/gpu/drm/gma500/psb_drv.c +++ b/drivers/gpu/drm/gma500/psb_drv.c @@ -14,7 +14,7 @@ #include <linux/pm_runtime.h> #include <linux/spinlock.h> #include <linux/delay.h> - +#include <linux/vgaarb.h> #include <asm/set_memory.h> #include <acpi/video.h> @@ -36,6 +36,11 @@ #include "psb_irq.h" #include "psb_reg.h" +static int gma500_modeset = -1; + +MODULE_PARM_DESC(modeset, "Disable/Enable modesetting"); +module_param_named(modeset, gma500_modeset, int, 0400); + static const struct drm_driver driver; static int psb_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent); @@ -446,6 +451,49 @@ static int gma_remove_conflicting_framebuffers(struct pci_dev *pdev, return __aperture_remove_legacy_vga_devices(pdev); } +static bool gma_contain_firmware_fb(u64 ap_start, u64 ap_end) +{ + u64 fb_start; + u64 fb_size; + u64 fb_end; + + if (screen_info.capabilities & VIDEO_CAPABILITY_64BIT_BASE) + fb_start = (u64)screen_info.ext_lfb_base << 32 | screen_info.lfb_base; + else + fb_start = screen_info.lfb_base; + + fb_size = screen_info.lfb_size; + fb_end = fb_start + fb_size - 1; + + /* No firmware framebuffer support */ + if (!fb_start || !fb_size) + return false; + + if (fb_start >= ap_start && fb_end <= ap_end) + return true; + + return false; +} + +static bool gma_want_to_be_primary(struct pci_dev *pdev) +{ + struct drm_device *drm = pci_get_drvdata(pdev); + struct drm_psb_private *priv = to_drm_psb_private(drm); + u64 vram_base = priv->stolen_base; + u64 vram_size = priv->vram_stolen_size; + + if (gma500_modeset == 10) + return true; + + /* Stolen memory are not going to be moved */ + if (gma_contain_firmware_fb(vram_base, vram_base + vram_size)) { + drm_dbg(drm, "Contains firmware FB in the stolen memory\n"); + return true; + } + + return false; +} + static int psb_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) { struct drm_psb_private *dev_priv; @@ -475,6 +523,8 @@ static int psb_pci_probe(struct pci_dev *pdev, const struct pci_device_id *ent) if (ret) return ret; + vga_client_register(pdev, NULL, gma_want_to_be_primary); + psb_fbdev_setup(dev_priv); return 0; @@ -526,7 +576,10 @@ static struct pci_driver psb_pci_driver = { static int __init psb_init(void) { - if (drm_firmware_drivers_only()) + if (drm_firmware_drivers_only() && (gma500_modeset == -1)) + return -ENODEV; + + if (!gma500_modeset) return -ENODEV; return pci_register_driver(&psb_pci_driver); -- 2.34.1
Jani Nikula
2023-Sep-05 10:38 UTC
[Nouveau] [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time
On Tue, 05 Sep 2023, Sui Jingfeng <sui.jingfeng at linux.dev> wrote:> From: Sui Jingfeng <suijingfeng at loongson.cn> > > On a machine with multiple GPUs, a Linux user has no control over which > one is primary at boot time. This series tries to solve above mentioned > problem by introduced the ->be_primary() function stub. The specific > device drivers can provide an implementation to hook up with this stub by > calling the vga_client_register() function. > > Once the driver bound the device successfully, VGAARB will call back to > the device driver. To query if the device drivers want to be primary or > not. Device drivers can just pass NULL if have no such needs. > > Please note that: > > 1) The ARM64, Loongarch, Mips servers have a lot PCIe slot, and I would > like to mount at least three video cards. > > 2) Typically, those non-86 machines don't have a good UEFI firmware > support, which doesn't support select primary GPU as firmware stage. > Even on x86, there are old UEFI firmwares which already made undesired > decision for you. > > 3) This series is attempt to solve the remain problems at the driver level, > while another series[1] of me is target to solve the majority of the > problems at device level. > > Tested (limited) on x86 with four video card mounted, Intel UHD Graphics > 630 is the default boot VGA, successfully override by ast2400 with > ast.modeset=10 append at the kernel cmd line.The value 10 is incredibly arbitrary, and multiplied as a magic number all over the place.> $ lspci | grep VGA > > 00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] > 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos XTX [Radeon HD 8490 / R5 235X OEM] > 04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30) > 05:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 720] (rev a1)In this example, all of the GPUs are driven by different drivers. What good does a module parameter do if you have multiple GPUs of the same model, all driven by the same driver module? BR, Jani.> > $ sudo dmesg | grep vgaarb > > pci 0000:00:02.0: vgaarb: setting as boot VGA device > pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none > pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > pci 0000:04:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > pci 0000:05:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > vgaarb: loaded > ast 0000:04:00.0: vgaarb: Override as primary by driver > i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem > radeon 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none > ast 0000:04:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none > > v2: > * Add a simple implemment for drm/i915 and drm/ast > * Pick up all tags (Mario) > v3: > * Fix a mistake for drm/i915 implement > * Fix patch can not be applied problem because of merge conflect. > v4: > * Focus on solve the real problem. > > v1,v2 at https://patchwork.freedesktop.org/series/120059/ > v3 at https://patchwork.freedesktop.org/series/120562/ > > [1] https://patchwork.freedesktop.org/series/122845/ > > Sui Jingfeng (9): > PCI/VGA: Allowing the user to select the primary video adapter at boot > time > drm/nouveau: Implement .be_primary() callback > drm/radeon: Implement .be_primary() callback > drm/amdgpu: Implement .be_primary() callback > drm/i915: Implement .be_primary() callback > drm/loongson: Implement .be_primary() callback > drm/ast: Register as a VGA client by calling vga_client_register() > drm/hibmc: Register as a VGA client by calling vga_client_register() > drm/gma500: Register as a VGA client by calling vga_client_register() > > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 +++- > drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++- > drivers/gpu/drm/ast/ast_drv.c | 31 ++++++++++ > drivers/gpu/drm/gma500/psb_drv.c | 57 ++++++++++++++++++- > .../gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c | 15 +++++ > drivers/gpu/drm/i915/display/intel_vga.c | 15 ++++- > drivers/gpu/drm/loongson/loongson_module.c | 2 +- > drivers/gpu/drm/loongson/loongson_module.h | 1 + > drivers/gpu/drm/loongson/lsdc_drv.c | 10 +++- > drivers/gpu/drm/nouveau/nouveau_vga.c | 11 +++- > drivers/gpu/drm/radeon/radeon_device.c | 10 +++- > drivers/pci/vgaarb.c | 43 ++++++++++++-- > drivers/vfio/pci/vfio_pci_core.c | 2 +- > include/linux/vgaarb.h | 8 ++- > 14 files changed, 210 insertions(+), 19 deletions(-)-- Jani Nikula, Intel Open Source Graphics Center
Thomas Zimmermann
2023-Sep-05 10:45 UTC
[Nouveau] [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time
Hi Am 04.09.23 um 21:57 schrieb Sui Jingfeng:> From: Sui Jingfeng <suijingfeng at loongson.cn> > > On a machine with multiple GPUs, a Linux user has no control over which > one is primary at boot time. This series tries to solve above mentionedIf anything, the primary graphics adapter is the one initialized by the firmware. I think our boot-up graphics also make this assumption implicitly. But what's the use case for overriding this setting? Best regards Thomas> problem by introduced the ->be_primary() function stub. The specific > device drivers can provide an implementation to hook up with this stub by > calling the vga_client_register() function. > > Once the driver bound the device successfully, VGAARB will call back to > the device driver. To query if the device drivers want to be primary or > not. Device drivers can just pass NULL if have no such needs. > > Please note that: > > 1) The ARM64, Loongarch, Mips servers have a lot PCIe slot, and I would > like to mount at least three video cards. > > 2) Typically, those non-86 machines don't have a good UEFI firmware > support, which doesn't support select primary GPU as firmware stage. > Even on x86, there are old UEFI firmwares which already made undesired > decision for you. > > 3) This series is attempt to solve the remain problems at the driver level, > while another series[1] of me is target to solve the majority of the > problems at device level. > > Tested (limited) on x86 with four video card mounted, Intel UHD Graphics > 630 is the default boot VGA, successfully override by ast2400 with > ast.modeset=10 append at the kernel cmd line. > > $ lspci | grep VGA > > 00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] > 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos XTX [Radeon HD 8490 / R5 235X OEM] > 04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30) > 05:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 720] (rev a1) > > $ sudo dmesg | grep vgaarb > > pci 0000:00:02.0: vgaarb: setting as boot VGA device > pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none > pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > pci 0000:04:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > pci 0000:05:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > vgaarb: loaded > ast 0000:04:00.0: vgaarb: Override as primary by driver > i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem > radeon 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none > ast 0000:04:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none > > v2: > * Add a simple implemment for drm/i915 and drm/ast > * Pick up all tags (Mario) > v3: > * Fix a mistake for drm/i915 implement > * Fix patch can not be applied problem because of merge conflect. > v4: > * Focus on solve the real problem. > > v1,v2 at https://patchwork.freedesktop.org/series/120059/ > v3 at https://patchwork.freedesktop.org/series/120562/ > > [1] https://patchwork.freedesktop.org/series/122845/ > > Sui Jingfeng (9): > PCI/VGA: Allowing the user to select the primary video adapter at boot > time > drm/nouveau: Implement .be_primary() callback > drm/radeon: Implement .be_primary() callback > drm/amdgpu: Implement .be_primary() callback > drm/i915: Implement .be_primary() callback > drm/loongson: Implement .be_primary() callback > drm/ast: Register as a VGA client by calling vga_client_register() > drm/hibmc: Register as a VGA client by calling vga_client_register() > drm/gma500: Register as a VGA client by calling vga_client_register() > > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 +++- > drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++- > drivers/gpu/drm/ast/ast_drv.c | 31 ++++++++++ > drivers/gpu/drm/gma500/psb_drv.c | 57 ++++++++++++++++++- > .../gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c | 15 +++++ > drivers/gpu/drm/i915/display/intel_vga.c | 15 ++++- > drivers/gpu/drm/loongson/loongson_module.c | 2 +- > drivers/gpu/drm/loongson/loongson_module.h | 1 + > drivers/gpu/drm/loongson/lsdc_drv.c | 10 +++- > drivers/gpu/drm/nouveau/nouveau_vga.c | 11 +++- > drivers/gpu/drm/radeon/radeon_device.c | 10 +++- > drivers/pci/vgaarb.c | 43 ++++++++++++-- > drivers/vfio/pci/vfio_pci_core.c | 2 +- > include/linux/vgaarb.h | 8 ++- > 14 files changed, 210 insertions(+), 19 deletions(-) >-- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstrasse 146, 90461 Nuernberg, Germany GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman HRB 36809 (AG Nuernberg) -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: <https://lists.freedesktop.org/archives/nouveau/attachments/20230905/a648f930/attachment.sig>
Thomas Zimmermann
2023-Sep-05 10:49 UTC
[Nouveau] [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time
Hi Am 04.09.23 um 21:57 schrieb Sui Jingfeng:> From: Sui Jingfeng <suijingfeng at loongson.cn> > > On a machine with multiple GPUs, a Linux user has no control over which > one is primary at boot time. This series tries to solve above mentioned > problem by introduced the ->be_primary() function stub. The specific > device drivers can provide an implementation to hook up with this stub by > calling the vga_client_register() function. > > Once the driver bound the device successfully, VGAARB will call back to > the device driver. To query if the device drivers want to be primary or > not. Device drivers can just pass NULL if have no such needs. > > Please note that: > > 1) The ARM64, Loongarch, Mips servers have a lot PCIe slot, and I would > like to mount at least three video cards. > > 2) Typically, those non-86 machines don't have a good UEFI firmware > support, which doesn't support select primary GPU as firmware stage. > Even on x86, there are old UEFI firmwares which already made undesired > decision for you. > > 3) This series is attempt to solve the remain problems at the driver level, > while another series[1] of me is target to solve the majority of the > problems at device level. > > Tested (limited) on x86 with four video card mounted, Intel UHD Graphics > 630 is the default boot VGA, successfully override by ast2400 with > ast.modeset=10 append at the kernel cmd line.FYI: per-driver modeset parameters are deprecated and not to be used. Please don't promote them. You can use modprobe.blacklist or initcall_blacklist on the kernel command line. Best regards Thomas> > $ lspci | grep VGA > > 00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] > 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos XTX [Radeon HD 8490 / R5 235X OEM] > 04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30) > 05:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 720] (rev a1) > > $ sudo dmesg | grep vgaarb > > pci 0000:00:02.0: vgaarb: setting as boot VGA device > pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none > pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > pci 0000:04:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > pci 0000:05:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > vgaarb: loaded > ast 0000:04:00.0: vgaarb: Override as primary by driver > i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem > radeon 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none > ast 0000:04:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none > > v2: > * Add a simple implemment for drm/i915 and drm/ast > * Pick up all tags (Mario) > v3: > * Fix a mistake for drm/i915 implement > * Fix patch can not be applied problem because of merge conflect. > v4: > * Focus on solve the real problem. > > v1,v2 at https://patchwork.freedesktop.org/series/120059/ > v3 at https://patchwork.freedesktop.org/series/120562/ > > [1] https://patchwork.freedesktop.org/series/122845/ > > Sui Jingfeng (9): > PCI/VGA: Allowing the user to select the primary video adapter at boot > time > drm/nouveau: Implement .be_primary() callback > drm/radeon: Implement .be_primary() callback > drm/amdgpu: Implement .be_primary() callback > drm/i915: Implement .be_primary() callback > drm/loongson: Implement .be_primary() callback > drm/ast: Register as a VGA client by calling vga_client_register() > drm/hibmc: Register as a VGA client by calling vga_client_register() > drm/gma500: Register as a VGA client by calling vga_client_register() > > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 +++- > drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++- > drivers/gpu/drm/ast/ast_drv.c | 31 ++++++++++ > drivers/gpu/drm/gma500/psb_drv.c | 57 ++++++++++++++++++- > .../gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c | 15 +++++ > drivers/gpu/drm/i915/display/intel_vga.c | 15 ++++- > drivers/gpu/drm/loongson/loongson_module.c | 2 +- > drivers/gpu/drm/loongson/loongson_module.h | 1 + > drivers/gpu/drm/loongson/lsdc_drv.c | 10 +++- > drivers/gpu/drm/nouveau/nouveau_vga.c | 11 +++- > drivers/gpu/drm/radeon/radeon_device.c | 10 +++- > drivers/pci/vgaarb.c | 43 ++++++++++++-- > drivers/vfio/pci/vfio_pci_core.c | 2 +- > include/linux/vgaarb.h | 8 ++- > 14 files changed, 210 insertions(+), 19 deletions(-) >-- Thomas Zimmermann Graphics Driver Developer SUSE Software Solutions Germany GmbH Frankenstrasse 146, 90461 Nuernberg, Germany GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman HRB 36809 (AG Nuernberg) -------------- next part -------------- A non-text attachment was scrubbed... Name: OpenPGP_signature Type: application/pgp-signature Size: 840 bytes Desc: OpenPGP digital signature URL: <https://lists.freedesktop.org/archives/nouveau/attachments/20230905/21cea2d7/attachment-0001.sig>
Alex Williamson
2023-Sep-05 14:52 UTC
[Nouveau] [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time
On Tue, 5 Sep 2023 03:57:15 +0800 Sui Jingfeng <sui.jingfeng at linux.dev> wrote:> From: Sui Jingfeng <suijingfeng at loongson.cn> > > On a machine with multiple GPUs, a Linux user has no control over which > one is primary at boot time. This series tries to solve above mentioned > problem by introduced the ->be_primary() function stub. The specific > device drivers can provide an implementation to hook up with this stub by > calling the vga_client_register() function. > > Once the driver bound the device successfully, VGAARB will call back to > the device driver. To query if the device drivers want to be primary or > not. Device drivers can just pass NULL if have no such needs. > > Please note that: > > 1) The ARM64, Loongarch, Mips servers have a lot PCIe slot, and I would > like to mount at least three video cards. > > 2) Typically, those non-86 machines don't have a good UEFI firmware > support, which doesn't support select primary GPU as firmware stage. > Even on x86, there are old UEFI firmwares which already made undesired > decision for you. > > 3) This series is attempt to solve the remain problems at the driver level, > while another series[1] of me is target to solve the majority of the > problems at device level. > > Tested (limited) on x86 with four video card mounted, Intel UHD Graphics > 630 is the default boot VGA, successfully override by ast2400 with > ast.modeset=10 append at the kernel cmd line. > > $ lspci | grep VGA > > 00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630]In all my previous experiments with VGA routing and IGD I found that IGD can't actually release VGA routing and Intel confirmed the hardware doesn't have the ability to do so. It will always be primary from a VGA routing perspective. Was this actually tested with non-UEFI? I suspect it might only work in UEFI mode where we probably don't actually have a dependency on VGA routing. This is essentially why vfio requires UEFI ROMs when assigning GPUs to VMs, VGA routing is too broken to use on Intel systems with IGD. Thanks, Alex> 01:00.0 VGA compatible controller: Advanced Micro Devices, Inc. [AMD/ATI] Caicos XTX [Radeon HD 8490 / R5 235X OEM] > 04:00.0 VGA compatible controller: ASPEED Technology, Inc. ASPEED Graphics Family (rev 30) > 05:00.0 VGA compatible controller: NVIDIA Corporation GK208B [GeForce GT 720] (rev a1) > > $ sudo dmesg | grep vgaarb > > pci 0000:00:02.0: vgaarb: setting as boot VGA device > pci 0000:00:02.0: vgaarb: VGA device added: decodes=io+mem,owns=io+mem,locks=none > pci 0000:01:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > pci 0000:04:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > pci 0000:05:00.0: vgaarb: VGA device added: decodes=io+mem,owns=none,locks=none > vgaarb: loaded > ast 0000:04:00.0: vgaarb: Override as primary by driver > i915 0000:00:02.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=io+mem > radeon 0000:01:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none > ast 0000:04:00.0: vgaarb: changed VGA decodes: olddecodes=io+mem,decodes=none:owns=none > > v2: > * Add a simple implemment for drm/i915 and drm/ast > * Pick up all tags (Mario) > v3: > * Fix a mistake for drm/i915 implement > * Fix patch can not be applied problem because of merge conflect. > v4: > * Focus on solve the real problem. > > v1,v2 at https://patchwork.freedesktop.org/series/120059/ > v3 at https://patchwork.freedesktop.org/series/120562/ > > [1] https://patchwork.freedesktop.org/series/122845/ > > Sui Jingfeng (9): > PCI/VGA: Allowing the user to select the primary video adapter at boot > time > drm/nouveau: Implement .be_primary() callback > drm/radeon: Implement .be_primary() callback > drm/amdgpu: Implement .be_primary() callback > drm/i915: Implement .be_primary() callback > drm/loongson: Implement .be_primary() callback > drm/ast: Register as a VGA client by calling vga_client_register() > drm/hibmc: Register as a VGA client by calling vga_client_register() > drm/gma500: Register as a VGA client by calling vga_client_register() > > drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 11 +++- > drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c | 13 ++++- > drivers/gpu/drm/ast/ast_drv.c | 31 ++++++++++ > drivers/gpu/drm/gma500/psb_drv.c | 57 ++++++++++++++++++- > .../gpu/drm/hisilicon/hibmc/hibmc_drm_drv.c | 15 +++++ > drivers/gpu/drm/i915/display/intel_vga.c | 15 ++++- > drivers/gpu/drm/loongson/loongson_module.c | 2 +- > drivers/gpu/drm/loongson/loongson_module.h | 1 + > drivers/gpu/drm/loongson/lsdc_drv.c | 10 +++- > drivers/gpu/drm/nouveau/nouveau_vga.c | 11 +++- > drivers/gpu/drm/radeon/radeon_device.c | 10 +++- > drivers/pci/vgaarb.c | 43 ++++++++++++-- > drivers/vfio/pci/vfio_pci_core.c | 2 +- > include/linux/vgaarb.h | 8 ++- > 14 files changed, 210 insertions(+), 19 deletions(-) >
suijingfeng
2023-Sep-05 16:21 UTC
[Nouveau] [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time
Hi, On 2023/9/5 22:52, Alex Williamson wrote:> On Tue, 5 Sep 2023 03:57:15 +0800 > Sui Jingfeng <sui.jingfeng at linux.dev> wrote: > >> From: Sui Jingfeng <suijingfeng at loongson.cn> >> >> On a machine with multiple GPUs, a Linux user has no control over which >> one is primary at boot time. This series tries to solve above mentioned >> problem by introduced the ->be_primary() function stub. The specific >> device drivers can provide an implementation to hook up with this stub by >> calling the vga_client_register() function. >> >> Once the driver bound the device successfully, VGAARB will call back to >> the device driver. To query if the device drivers want to be primary or >> not. Device drivers can just pass NULL if have no such needs. >> >> Please note that: >> >> 1) The ARM64, Loongarch, Mips servers have a lot PCIe slot, and I would >> like to mount at least three video cards. >> >> 2) Typically, those non-86 machines don't have a good UEFI firmware >> support, which doesn't support select primary GPU as firmware stage. >> Even on x86, there are old UEFI firmwares which already made undesired >> decision for you. >> >> 3) This series is attempt to solve the remain problems at the driver level, >> while another series[1] of me is target to solve the majority of the >> problems at device level. >> >> Tested (limited) on x86 with four video card mounted, Intel UHD Graphics >> 630 is the default boot VGA, successfully override by ast2400 with >> ast.modeset=10 append at the kernel cmd line. >> >> $ lspci | grep VGA >> >> 00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] > In all my previous experiments with VGA routing and IGD I found that > IGD can't actually release VGA routing and Intel confirmed the hardware > doesn't have the ability to do so. It will always be primary from a > VGA routing perspective. Was this actually tested with non-UEFI?Yes, I have tested on my aspire e471 notebook (i5 5200U), because that notebook using legacy firmware (also have UEFI, double firmware). But this machine have difficult in install ubuntu under UEFI firmware in the past. So I keep it using the legacy firmware. It have two video card, IGD and nvidia video card(GFORCE 840M). nvidia call its video card as 3D controller (pci->class = 0x030200) I have tested this patch and another patch mention at [1] together. I can tell you that the firmware framebuffer of this notebook using vesafb, not efifb. And the framebuffer size (lfb.size) is very small. This is very strange, but I don't have enough time to look in details. But still works. I'm using and tesing my patch whenever and wherever possible.> I suspect it might only work in UEFI mode where we probably don't > actually have a dependency on VGA routing. This is essentially why > vfio requires UEFI ROMs when assigning GPUs to VMs, VGA routing is too > broken to use on Intel systems with IGD. Thanks,What you tell me here is the side effect come with the VGA-compatible, but I'm focus on the arbitration itself. I think there no need to keep the VGA routing hardware features nowadays except that hardware vendor want keep the backward compatibility and/or comply the PCI VGA compatible spec.> Alex >
Sui Jingfeng
2023-Sep-06 03:51 UTC
[Nouveau] [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time
Hi, On 2023/9/5 22:52, Alex Williamson wrote:> On Tue, 5 Sep 2023 03:57:15 +0800 > Sui Jingfeng <sui.jingfeng at linux.dev> wrote: > >> From: Sui Jingfeng <suijingfeng at loongson.cn> >> >> On a machine with multiple GPUs, a Linux user has no control over which >> one is primary at boot time. This series tries to solve above mentioned >> problem by introduced the ->be_primary() function stub. The specific >> device drivers can provide an implementation to hook up with this stub by >> calling the vga_client_register() function. >> >> Once the driver bound the device successfully, VGAARB will call back to >> the device driver. To query if the device drivers want to be primary or >> not. Device drivers can just pass NULL if have no such needs. >> >> Please note that: >> >> 1) The ARM64, Loongarch, Mips servers have a lot PCIe slot, and I would >> like to mount at least three video cards. >> >> 2) Typically, those non-86 machines don't have a good UEFI firmware >> support, which doesn't support select primary GPU as firmware stage. >> Even on x86, there are old UEFI firmwares which already made undesired >> decision for you. >> >> 3) This series is attempt to solve the remain problems at the driver level, >> while another series[1] of me is target to solve the majority of the >> problems at device level. >> >> Tested (limited) on x86 with four video card mounted, Intel UHD Graphics >> 630 is the default boot VGA, successfully override by ast2400 with >> ast.modeset=10 append at the kernel cmd line. >> >> $ lspci | grep VGA >> >> 00:02.0 VGA compatible controller: Intel Corporation CoffeeLake-S GT2 [UHD Graphics 630] > In all my previous experiments with VGA routing and IGD I found that > IGD can't actually release VGA routing and Intel confirmed the hardware > doesn't have the ability to do so.Which model of the IGD you are using??even for the IGD in Atom D2550, the legacy 128KB VGA memory range can be?tuned to be mapped to IGD or to the DMI Interface. See the 1.7.3.2 section of the N2000 datasheet[1]. If a specific model of Intel has a bug in the VGA routing hardware logic unit, I would like to ignore it. Or switch to the UEFI firmware on such hardware. It is the hardware engineer's responsibility, I will not worry about it. Thanks for you tell this. [1] https://www.intel.com/content/dam/doc/datasheet/atom-d2000-n2000-vol-2-datasheet.pdf> It will always be primary from a > VGA routing perspective. Was this actually tested with non-UEFI?As you already said,?the generous Intel already have confirmed that the hardware defect. So probably this is a good chance to switch to UEFI to solve the problem. Then, no testing for legacy is needed.> I suspect it might only work in UEFI mode where we probably don't > actually have a dependency on VGA routing. This is essentially why > vfio requires UEFI ROMs when assigning GPUs to VMs, VGA routing is too > broken to use on Intel systems with IGD. Thanks,Thanks for you tell me this. To be honest, I have only tested my patch on machines with UEFI?firmware. Since UEFI because the main stream, but if this patch is really useful for majority machine, I'm satisfied. The results is not too bad. Thanks.> Alex >