Nouveau - Sep 2023 - [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time

Hi,


On 2023/9/5 18:45, Thomas Zimmermann wrote:
> Hi
>
> Am 04.09.23 um 21:57 schrieb Sui Jingfeng:
>> From: Sui Jingfeng <suijingfeng at loongson.cn>
>>
>> On a machine with multiple GPUs, a Linux user has no control over which
>> one is primary at boot time. This series tries to solve above mentioned
>
> If anything, the primary graphics adapter is the one initialized by 
> the firmware. I think our boot-up graphics also make this assumption 
> implicitly.
>
Yes, but by the time of DRM drivers get loaded successfully,the boot-up graphics
already finished.
Firmware framebuffer device already get killed by the
drm_aperture_remove_conflicting_pci_framebuffers()
function (or its siblings). So, this series is definitely not to interact with
the firmware framebuffer
(or more intelligent framebuffer drivers).  It is for user space program, such
as X server and Wayland
compositor. Its for Linux user or drm drivers testers, which allow them to
direct graphic display server
using right hardware of interested as primary video card.

Also, I believe that X server and Wayland compositor are the best test examples.
If a specific DRM driver can't work with X server as a primary,
then there probably have something wrong.

> But what's the use case for overriding this setting?
>
On a specific machine with multiple GPUs mounted,
only the primary graphics get POST-ed (initialized) by the firmware.
Therefore, the DRM drivers for the rest video cards, have to choose to
work without the prerequisite setups done by firmware, This is called as POST.

One of the use cases of this series is to test if a specific DRM driver could
works properly,
even though there is no prerequisite works have been done by firmware at all.
And it seems that the results is not satisfying in all cases.

drm/ast is the first drm drivers which refused to work if not being POST-ed by
the firmware.

Before apply this series, I was unable make drm/ast as the primary video card
easily. On a
multiple video card configuration, the monitor connected with the AST2400 not
light up.
While confusing, a naive programmer may suspect the PRIME is not working.

After applied this series and passing ast.modeset=10 on the kernel cmd line,
I found that the monitor connected with my ast2400 video card still black,
It doesn't display and doesn't show image to me.

While in the process of study drm/ast, I know that drm/ast driver has the POST
code shipped.
See the ast_post_gpu() function,?then, I was wondering why this function
doesn't works.
After a short-time (hasty) debugging, I found that the the ast_post_gpu()
function
didn't get run. Because it have something to do with the
ast->config_mode.

Without thinking too much, I hardcoded the ast->config_mode as ast_use_p2a to
force the ast_post_gpu() function get run.

```

--- a/drivers/gpu/drm/ast/ast_main.c
+++ b/drivers/gpu/drm/ast/ast_main.c
@@ -132,6 +132,8 @@ static int ast_device_config_init(struct ast_device 
*ast)
 ??????????????? }
 ??????? }

+?????? ast->config_mode = ast_use_p2a;
+
 ??????? switch (ast->config_mode) {
 ??????? case ast_use_defaults:
 ??????????????? drm_info(dev, "Using default configuration\n");

```

Then, the monitor light up, it display the Ubuntu greeter to me.
Therefore, my patch is helpful, at lease for the Linux drm driver tester and
developer.
It allow programmers to test the specific part of the specific drive
without changing a line of the source code and without the need of sudo
authority.
It helps to improve efficiency of the testing and patch verification.

I know the PrimaryGPU option of Xorg conf, but this approach will remember the
setup
have been made, you need modify it with root authority each time you want to
switch
the primary. But on rapid developing and/or testing multiple video drivers, with
only one computer hardware resource available. What we really want probably is a
one-shoot command as this series provide.

So, this is the first use case. This probably also help to test full modeset,
PRIME and reverse PRIME on multiple video card machine.

> Best regards
> Thomas
>

Hi

Am 05.09.23 um 15:30 schrieb suijingfeng:
> Hi,
> 
> 
> On 2023/9/5 18:45, Thomas Zimmermann wrote:
>> Hi
>>
>> Am 04.09.23 um 21:57 schrieb Sui Jingfeng:
>>> From: Sui Jingfeng <suijingfeng at loongson.cn>
>>>
>>> On a machine with multiple GPUs, a Linux user has no control over
which
>>> one is primary at boot time. This series tries to solve above
mentioned
>>
>> If anything, the primary graphics adapter is the one initialized by 
>> the firmware. I think our boot-up graphics also make this assumption 
>> implicitly.
>>
> 
> Yes, but by the time of DRM drivers get loaded successfully,the boot-up 
> graphics already finished.
> Firmware framebuffer device already get killed by the 
> drm_aperture_remove_conflicting_pci_framebuffers()
> function (or its siblings). So, this series is definitely not to 
> interact with the firmware framebuffer
Yes and no. The helpers you mention will attempt to remove the firmware 
framebuffer on the given PCI device. If you have multiple PCI devices, 
the other devices would not be affected.

This also means that probing a non-primary card will not affect the 
firmware framebuffer on the primary card. You can have all these drivers 
co-exist next to each other. If you link a full DRM driver into the 
kernel image, it might even be loaded before the firmware-framebuffer's 
driver.  We had some funny bugs from these interactions.

> (or more intelligent framebuffer drivers).? It is for user space 
> program, such as X server and Wayland
> compositor. Its for Linux user or drm drivers testers, which allow them 
> to direct graphic display server
> using right hardware of interested as primary video card.
> 
> Also, I believe that X server and Wayland compositor are the best test 
> examples.
> If a specific DRM driver can't work with X server as a primary,
> then there probably have something wrong.
If you want to run a userspace compositor or X11 on a certain device, 
you best configure this in the program's config files. But not on the 
kernel command line.

The whole concept of a 'primary' display is bogus IMHO. It only exists 
because old VGA and BIOS (and their equivalents on non-PC systems) were 
unable to use more than one graphics device. Hence, as you write below, 
only the first device got POSTed by the BIOS. If you had an additional 
card, the device driver needed to perform the POSTing.

However, on modern Linux systems the primary display does not really 
exist. 'Primary' is the device that is available via VGA, VESA or EFI. 
Our drivers don't use these interfaces, but the native registers. As you 
said yourself, these firmware devices (VGA, VESA, EFI) are removed ASAP 
by the native drivers.
> 
> 
>> But what's the use case for overriding this setting?
>>
> 
> On a specific machine with multiple GPUs mounted,
> only the primary graphics get POST-ed (initialized) by the firmware.
> Therefore, the DRM drivers for the rest video cards, have to choose to
> work without the prerequisite setups done by firmware, This is called as 
> POST.
> 
> One of the use cases of this series is to test if a specific DRM driver 
> could works properly,
> even though there is no prerequisite works have been done by firmware at 
> all.
> And it seems that the results is not satisfying in all cases.
> 
> drm/ast is the first drm drivers which refused to work if not being 
> POST-ed by the firmware.
You might have found a bug in the ast driver. Ast has means to detect if 
the device has been POSTed and maybe do that. If this doesn't work 
correctly, it needs a fix.

As Christian mentioned, if anything, you might add an option to specify 
the default card to vgaarb (e.g., as PCI slot). But userspace should 
avoid the idea of a primary card IMHO.

Best regards
Thomas
> 
> Before apply this series, I was unable make drm/ast as the primary video 
> card easily. On a
> multiple video card configuration, the monitor connected with the 
> AST2400 not light up.
> While confusing, a naive programmer may suspect the PRIME is not working.
> 
> After applied this series and passing ast.modeset=10 on the kernel cmd 
> line,
> I found that the monitor connected with my ast2400 video card still black,
> It doesn't display and doesn't show image to me.
> 
> While in the process of study drm/ast, I know that drm/ast driver has 
> the POST code shipped.
> See the ast_post_gpu() function,?then, I was wondering why this function 
> doesn't works.
> After a short-time (hasty) debugging, I found that the the 
> ast_post_gpu() function
> didn't get run. Because it have something to do with the
ast->config_mode.
> 
> Without thinking too much, I hardcoded the ast->config_mode as 
> ast_use_p2a to
> force the ast_post_gpu() function get run.
> 
> ```
> 
> --- a/drivers/gpu/drm/ast/ast_main.c
> +++ b/drivers/gpu/drm/ast/ast_main.c
> @@ -132,6 +132,8 @@ static int ast_device_config_init(struct ast_device 
> *ast)
>  ??????????????? }
>  ??????? }
> 
> +?????? ast->config_mode = ast_use_p2a;
> +
>  ??????? switch (ast->config_mode) {
>  ??????? case ast_use_defaults:
>  ??????????????? drm_info(dev, "Using default configuration\n");
> 
> ```
> 
> Then, the monitor light up, it display the Ubuntu greeter to me.
> Therefore, my patch is helpful, at lease for the Linux drm driver tester 
> and developer.
> It allow programmers to test the specific part of the specific drive
> without changing a line of the source code and without the need of sudo 
> authority.
> It helps to improve efficiency of the testing and patch verification.
> 
> I know the PrimaryGPU option of Xorg conf, but this approach will 
> remember the setup
> have been made, you need modify it with root authority each time you 
> want to switch
> the primary. But on rapid developing and/or testing multiple video 
> drivers, with
> only one computer hardware resource available. What we really want 
> probably is a
> one-shoot command as this series provide.
> 
> So, this is the first use case. This probably also help to test full 
> modeset,
> PRIME and reverse PRIME on multiple video card machine.
> 
> 
>> Best regards
>> Thomas
>>
> 
-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstrasse 146, 90461 Nuernberg, Germany
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman
HRB 36809 (AG Nuernberg)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL:
<https://lists.freedesktop.org/archives/nouveau/attachments/20230905/cb16d308/attachment-0001.sig>

Am 05.09.23 um 15:30 schrieb suijingfeng:
> Hi,
>
>
> On 2023/9/5 18:45, Thomas Zimmermann wrote:
>> Hi
>>
>> Am 04.09.23 um 21:57 schrieb Sui Jingfeng:
>>> From: Sui Jingfeng <suijingfeng at loongson.cn>
>>>
>>> On a machine with multiple GPUs, a Linux user has no control over
which
>>> one is primary at boot time. This series tries to solve above
mentioned
>>
>> If anything, the primary graphics adapter is the one initialized by 
>> the firmware. I think our boot-up graphics also make this assumption 
>> implicitly.
>>
>
> Yes, but by the time of DRM drivers get loaded successfully,the 
> boot-up graphics already finished.
This is an incorrect assumption.

drm_aperture_remove_conflicting_pci_framebuffers() and co don't kill the 
framebuffer, they just remove the current framebuffer driver to avoid 
further updates.

So what happens (at least for amdgpu) is that we take over the 
framebuffer, including both mode and it's contents, and provide a new 
framebuffer interface until DRM masters like X or Wayland take over.
> Firmware framebuffer device already get killed by the 
> drm_aperture_remove_conflicting_pci_framebuffers()
> function (or its siblings). So, this series is definitely not to 
> interact with the firmware framebuffer
> (or more intelligent framebuffer drivers).? It is for user space 
> program, such as X server and Wayland
> compositor. Its for Linux user or drm drivers testers, which allow 
> them to direct graphic display server
> using right hardware of interested as primary video card.
>
> Also, I believe that X server and Wayland compositor are the best test 
> examples.
> If a specific DRM driver can't work with X server as a primary,
> then there probably have something wrong.
>
>
>> But what's the use case for overriding this setting?
>>
>
> On a specific machine with multiple GPUs mounted,
> only the primary graphics get POST-ed (initialized) by the firmware.
> Therefore, the DRM drivers for the rest video cards, have to choose to
> work without the prerequisite setups done by firmware, This is called 
> as POST.
Well, you don't seem to understand the background here. This is 
perfectly normal behavior.

Secondary cards are posted after loading the appropriate DRM driver. At 
least for amdgpu this is done by calling the appropriate functions in 
the BIOS.
>
> One of the use cases of this series is to test if a specific DRM 
> driver could works properly,
> even though there is no prerequisite works have been done by firmware 
> at all.
> And it seems that the results is not satisfying in all cases.
>
> drm/ast is the first drm drivers which refused to work if not being 
> POST-ed by the firmware.
As far as I know this is expected as well. AST is a relatively simple 
driver and when it's not the primary one during boot the assumption is 
that it isn't used at all.

Regards,
Christian.
>
> Before apply this series, I was unable make drm/ast as the primary 
> video card easily. On a
> multiple video card configuration, the monitor connected with the 
> AST2400 not light up.
> While confusing, a naive programmer may suspect the PRIME is not working.
>
> After applied this series and passing ast.modeset=10 on the kernel cmd 
> line,
> I found that the monitor connected with my ast2400 video card still 
> black,
> It doesn't display and doesn't show image to me.
>
> While in the process of study drm/ast, I know that drm/ast driver has 
> the POST code shipped.
> See the ast_post_gpu() function,?then, I was wondering why this 
> function doesn't works.
> After a short-time (hasty) debugging, I found that the the 
> ast_post_gpu() function
> didn't get run. Because it have something to do with the 
> ast->config_mode.
>
> Without thinking too much, I hardcoded the ast->config_mode as 
> ast_use_p2a to
> force the ast_post_gpu() function get run.
>
> ```
>
> --- a/drivers/gpu/drm/ast/ast_main.c
> +++ b/drivers/gpu/drm/ast/ast_main.c
> @@ -132,6 +132,8 @@ static int ast_device_config_init(struct 
> ast_device *ast)
> ??????????????? }
> ??????? }
>
> +?????? ast->config_mode = ast_use_p2a;
> +
> ??????? switch (ast->config_mode) {
> ??????? case ast_use_defaults:
> ??????????????? drm_info(dev, "Using default configuration\n");
>
> ```
>
> Then, the monitor light up, it display the Ubuntu greeter to me.
> Therefore, my patch is helpful, at lease for the Linux drm driver 
> tester and developer.
> It allow programmers to test the specific part of the specific drive
> without changing a line of the source code and without the need of 
> sudo authority.
> It helps to improve efficiency of the testing and patch verification.
>
> I know the PrimaryGPU option of Xorg conf, but this approach will 
> remember the setup
> have been made, you need modify it with root authority each time you 
> want to switch
> the primary. But on rapid developing and/or testing multiple video 
> drivers, with
> only one computer hardware resource available. What we really want 
> probably is a
> one-shoot command as this series provide.
>
> So, this is the first use case. This probably also help to test full 
> modeset,
> PRIME and reverse PRIME on multiple video card machine.
>
>
>> Best regards
>> Thomas
>>
>