Nouveau - Sep 2023 - [RFC, drm-misc-next v4 0/9] PCI/VGA: Allowing the user to select the primary video adapter at boot time

Hi

Am 05.09.23 um 15:30 schrieb suijingfeng:
> Hi,
> 
> 
> On 2023/9/5 18:45, Thomas Zimmermann wrote:
>> Hi
>>
>> Am 04.09.23 um 21:57 schrieb Sui Jingfeng:
>>> From: Sui Jingfeng <suijingfeng at loongson.cn>
>>>
>>> On a machine with multiple GPUs, a Linux user has no control over
which
>>> one is primary at boot time. This series tries to solve above
mentioned
>>
>> If anything, the primary graphics adapter is the one initialized by 
>> the firmware. I think our boot-up graphics also make this assumption 
>> implicitly.
>>
> 
> Yes, but by the time of DRM drivers get loaded successfully,the boot-up 
> graphics already finished.
> Firmware framebuffer device already get killed by the 
> drm_aperture_remove_conflicting_pci_framebuffers()
> function (or its siblings). So, this series is definitely not to 
> interact with the firmware framebuffer
Yes and no. The helpers you mention will attempt to remove the firmware 
framebuffer on the given PCI device. If you have multiple PCI devices, 
the other devices would not be affected.

This also means that probing a non-primary card will not affect the 
firmware framebuffer on the primary card. You can have all these drivers 
co-exist next to each other. If you link a full DRM driver into the 
kernel image, it might even be loaded before the firmware-framebuffer's 
driver.  We had some funny bugs from these interactions.

> (or more intelligent framebuffer drivers).? It is for user space 
> program, such as X server and Wayland
> compositor. Its for Linux user or drm drivers testers, which allow them 
> to direct graphic display server
> using right hardware of interested as primary video card.
> 
> Also, I believe that X server and Wayland compositor are the best test 
> examples.
> If a specific DRM driver can't work with X server as a primary,
> then there probably have something wrong.
If you want to run a userspace compositor or X11 on a certain device, 
you best configure this in the program's config files. But not on the 
kernel command line.

The whole concept of a 'primary' display is bogus IMHO. It only exists 
because old VGA and BIOS (and their equivalents on non-PC systems) were 
unable to use more than one graphics device. Hence, as you write below, 
only the first device got POSTed by the BIOS. If you had an additional 
card, the device driver needed to perform the POSTing.

However, on modern Linux systems the primary display does not really 
exist. 'Primary' is the device that is available via VGA, VESA or EFI. 
Our drivers don't use these interfaces, but the native registers. As you 
said yourself, these firmware devices (VGA, VESA, EFI) are removed ASAP 
by the native drivers.
> 
> 
>> But what's the use case for overriding this setting?
>>
> 
> On a specific machine with multiple GPUs mounted,
> only the primary graphics get POST-ed (initialized) by the firmware.
> Therefore, the DRM drivers for the rest video cards, have to choose to
> work without the prerequisite setups done by firmware, This is called as 
> POST.
> 
> One of the use cases of this series is to test if a specific DRM driver 
> could works properly,
> even though there is no prerequisite works have been done by firmware at 
> all.
> And it seems that the results is not satisfying in all cases.
> 
> drm/ast is the first drm drivers which refused to work if not being 
> POST-ed by the firmware.
You might have found a bug in the ast driver. Ast has means to detect if 
the device has been POSTed and maybe do that. If this doesn't work 
correctly, it needs a fix.

As Christian mentioned, if anything, you might add an option to specify 
the default card to vgaarb (e.g., as PCI slot). But userspace should 
avoid the idea of a primary card IMHO.

Best regards
Thomas
> 
> Before apply this series, I was unable make drm/ast as the primary video 
> card easily. On a
> multiple video card configuration, the monitor connected with the 
> AST2400 not light up.
> While confusing, a naive programmer may suspect the PRIME is not working.
> 
> After applied this series and passing ast.modeset=10 on the kernel cmd 
> line,
> I found that the monitor connected with my ast2400 video card still black,
> It doesn't display and doesn't show image to me.
> 
> While in the process of study drm/ast, I know that drm/ast driver has 
> the POST code shipped.
> See the ast_post_gpu() function,?then, I was wondering why this function 
> doesn't works.
> After a short-time (hasty) debugging, I found that the the 
> ast_post_gpu() function
> didn't get run. Because it have something to do with the
ast->config_mode.
> 
> Without thinking too much, I hardcoded the ast->config_mode as 
> ast_use_p2a to
> force the ast_post_gpu() function get run.
> 
> ```
> 
> --- a/drivers/gpu/drm/ast/ast_main.c
> +++ b/drivers/gpu/drm/ast/ast_main.c
> @@ -132,6 +132,8 @@ static int ast_device_config_init(struct ast_device 
> *ast)
>  ??????????????? }
>  ??????? }
> 
> +?????? ast->config_mode = ast_use_p2a;
> +
>  ??????? switch (ast->config_mode) {
>  ??????? case ast_use_defaults:
>  ??????????????? drm_info(dev, "Using default configuration\n");
> 
> ```
> 
> Then, the monitor light up, it display the Ubuntu greeter to me.
> Therefore, my patch is helpful, at lease for the Linux drm driver tester 
> and developer.
> It allow programmers to test the specific part of the specific drive
> without changing a line of the source code and without the need of sudo 
> authority.
> It helps to improve efficiency of the testing and patch verification.
> 
> I know the PrimaryGPU option of Xorg conf, but this approach will 
> remember the setup
> have been made, you need modify it with root authority each time you 
> want to switch
> the primary. But on rapid developing and/or testing multiple video 
> drivers, with
> only one computer hardware resource available. What we really want 
> probably is a
> one-shoot command as this series provide.
> 
> So, this is the first use case. This probably also help to test full 
> modeset,
> PRIME and reverse PRIME on multiple video card machine.
> 
> 
>> Best regards
>> Thomas
>>
> 
-- 
Thomas Zimmermann
Graphics Driver Developer
SUSE Software Solutions Germany GmbH
Frankenstrasse 146, 90461 Nuernberg, Germany
GF: Ivo Totev, Andrew Myers, Andrew McDonald, Boudien Moerman
HRB 36809 (AG Nuernberg)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: OpenPGP_signature
Type: application/pgp-signature
Size: 840 bytes
Desc: OpenPGP digital signature
URL:
<https://lists.freedesktop.org/archives/nouveau/attachments/20230905/cb16d308/attachment-0001.sig>

Hi,


On 2023/9/5 23:05, Thomas Zimmermann wrote:
> However, on modern Linux systems the primary display does not really 
> exist.

No, it do exist.? X server need to know which one is the primary GPU.
The '*' character at the of (4 at 0:0:0) PCI device?is the Primary.
The '*' denote primary, see the log below.

(II) xfree86: Adding drm device (/dev/dri/card2)
(II) xfree86: Adding drm device (/dev/dri/card0)
(II) Platform probe for 
/sys/devices/pci0000:00/0000:00:1c.5/0000:003:00.0/0000:04:00.0/drm/card0
(II) xfree86: Adding drm device (/dev/dri/card3)
(II) Platform probe for 
/sys/devices/pci0000:00/0000:00:1c.6/0000:005:00.0/drm/card3
(--) PCI: (0 at 0:2:0) 8086:3e91:8086:3e91 rev 0, Mem @ 
0xdb000000/167777216, 0xa0000000/536870912, I/O @ 0x0000f000/64, BIOS @ 
0x????????/131072
(--) PCI: (1 at 0:0:0) 1002:6771:1043:8636 rev 0, Mem @ 
0xc0000000/2688435456, 0xdf220000/131072, I/O @ 0x0000e000/256, BIOS @ 
0x????????/131072
(--) PCI:*(4 at 0:0:0) 1a03:2000:1a03:2000 rev 48, Mem @ 
0xde000000/166777216, 0xdf020000/131072, I/O @ 0x0000c000/128, BIOS @ 
0x????????/131072
(--) PCI: (5 at 0:0:0) 10de:1288:174b:b324 rev 161, Mem @ 
0xdc000000/116777216, 0xd0000000/134217728, 0xd8000000/33554432, I/O @ 
0x0000b000/128, BIOS @@0x????????/524288

The modesetting driver of X server will create framebuffer on the primary video
adapter.
If a 2D video adapter (like the aspeed BMC) is not the primary, then it probably
will not
be used. The only chance to be able to display something is to functional as a
output slave.
But the output slave technology need the PRIME support for cross driver buffer
sharing.

So, there do have some difference between the primary and non-primary video
adapters.

> 'Primary' is the device that is available via VGA, VESA or EFI. Our
> drivers don't use these interfaces, but the native registers. As you 
> said yourself, these firmware devices (VGA, VESA, EFI) are removed 
> ASAP by the native drivers.

On 2023/9/5 23:05, Thomas Zimmermann wrote:
> Hi
>
> Am 05.09.23 um 15:30 schrieb suijingfeng:
>> Hi,
>>
>>
>> On 2023/9/5 18:45, Thomas Zimmermann wrote:
>>> Hi
>>>
>>> Am 04.09.23 um 21:57 schrieb Sui Jingfeng:
>>>> From: Sui Jingfeng <suijingfeng at loongson.cn>
>>>>
>>>> On a machine with multiple GPUs, a Linux user has no control
over
>>>> which
>>>> one is primary at boot time. This series tries to solve above 
>>>> mentioned
>>>
>>> If anything, the primary graphics adapter is the one initialized by
>>> the firmware. I think our boot-up graphics also make this
assumption
>>> implicitly.
>>>
>>
>> Yes, but by the time of DRM drivers get loaded successfully,the 
>> boot-up graphics already finished.
>> Firmware framebuffer device already get killed by the 
>> drm_aperture_remove_conflicting_pci_framebuffers()
>> function (or its siblings). So, this series is definitely not to 
>> interact with the firmware framebuffer
>
> Yes and no. The helpers you mention will attempt to remove the 
> firmware framebuffer on the given PCI device. If you have multiple PCI 
> devices, the other devices would not be affected.
>
Yes and no.


For the yes part: drm_aperture_remove_conflicting_pci_framebuffers() only kill
the conflict one.
But for a specific machine with the modern UEFI firmware,
there should be only one firmware framebuffer driver.
That shoudd be the EFIFB(UEFI GOP). I do have multiple PCI devices,
but I don't understand when and why a system will have more than one
firmware framebuffer.

Even for the machines with the legacy BIOS, the fixed VGA aperture address range
can only be owned by one firmware driver. It is just that we need to handle the
routing, the ->set_decode() callback of vga_client_register() is used to do
such
work. Am I correct?

Hi,


On 2023/9/5 23:05, Thomas Zimmermann wrote:
> However, on modern Linux systems the primary display does not really 
> exist. 'Primary' is the device that is available via VGA, VESA or
EFI.
I may miss the point, what do you means by choose the word "modern"?
Are you trying to tell me that X server is too old and Wayland is the modern
display server?

> Our drivers don't use these interfaces, but the native registers.

Yes and no?

Yes for the machine with the UEFI firmware,
but I not sure if this statement is true for the machine with the?legacy
firmware.

As the display controller in the ASpeed BMC is VGA compatible.
Therefore, in theory, it should works with the VGA console on the machine
with another VGA compatible video card. So the ast_vga_set_decode() function
provided in the 0007 patch probably useful on legacy firmware environment.

To be honest, I have tested this on various machine with UEFI firmware.
But I didn't realized that I should do the testing on legacy firmware
environment
before sending this patch. It seems that the testing effort needed are quite
exhausting, since all my machines come with the UEFI firmware.

So is it OK to leave the legacy part to someone else who interested in it?
Probably Alex is more professional at legacy VGA routing stuff?
:-)

Hi,

On 2023/9/5 23:05, Thomas Zimmermann wrote:
> You might have found a bug in the ast driver. Ast has means to detect 
> if the device has been POSTed and maybe do that. If this doesn't work 
> correctly, it needs a fix.
>
That sounds fine.

The bug is not a big deal, I'm just take it as an example and report it to
you.
But a real fix can be complex, because there are quite a lot of servers
ship with ASpeed BMC hardware.

Honestly I don't have the time fix it on formal way.
I have already tons patches in pending and I will focus on solve VGAARB related
problem.


Because I want to test your patch occasionally.
So this series is useful for myself at corner cases.