Nouveau - Mar 2014 - PROM vbios fetching issues

Hello,

One of my GPU (GK107/NVE7) fails to properly fetch its vbios from PROM 
at boot time but, if I blacklist the module and load it myself later on, 
it always succeeds. To make things weirder, the same card works great on 
another computer.

Here is the relevant code in Nouveau to fetch the vbios from PROM:
http://code.woboq.org/linux/linux/drivers/gpu/drm/nouveau/core/subdev/bios/base.c.html#nouveau_bios_shadow_prom

The only solution I found to this problem is to try up to 16 times in a 
row to fetch the vbios using PROM. If the 16 tries yield a bad signature 
or checksum, I just give up and let it try other techniques. Here is the 
patch[1].

With this hack in place, the number of retries needed to be able to 
correctly fetch the vbios ranges from 0 to 13. I see no consistency in 
this. I tried reading the vbios 32 bits by 32 bits (instead of 8 by 8), 
disabling local IRQs and generating more activity on the PCIe port 
before fetching (and making sure the value read is always the same), all 
solutions yielded no improvement.

It could be a brownout problem but this would be the only manifestation 
of this problem. Moreover, some cards are known to have problems with
PROM[2] so mine is definitely not the only one.

I'm at a loss here, should we wait on anything before reading from PROM? 
The mmiotrace I have doesn't seem to suggest we should :s I wonder if 
you have had this issue before and what the nvkm does here.

Thanks,
Martin

[1] http://lists.freedesktop.org/archives/nouveau/2014-March/016590.html
[2] 
http://code.woboq.org/linux/linux/drivers/gpu/drm/nouveau/core/subdev/bios/base.c.html#144

On Mon, Mar 24, 2014 at 11:59:46AM -0700, Martin Peres
wrote:
>
> Hello,
> 
> One of my GPU (GK107/NVE7) fails to properly fetch its vbios from PROM 
> at boot time but, if I blacklist the module and load it myself later on, 
> it always succeeds. To make things weirder, the same card works great on 
> another computer.
> 
> Here is the relevant code in Nouveau to fetch the vbios from PROM:
>
http://code.woboq.org/linux/linux/drivers/gpu/drm/nouveau/core/subdev/bios/base.c.html#nouveau_bios_shadow_prom
> 
> The only solution I found to this problem is to try up to 16 times in a 
> row to fetch the vbios using PROM. If the 16 tries yield a bad signature 
> or checksum, I just give up and let it try other techniques. Here is the 
> patch[1].
> 
> With this hack in place, the number of retries needed to be able to 
> correctly fetch the vbios ranges from 0 to 13. I see no consistency in 
> this. I tried reading the vbios 32 bits by 32 bits (instead of 8 by 8), 
> disabling local IRQs and generating more activity on the PCIe port 
> before fetching (and making sure the value read is always the same), all 
> solutions yielded no improvement.
> 
> It could be a brownout problem but this would be the only manifestation 
> of this problem. Moreover, some cards are known to have problems with
> PROM[2] so mine is definitely not the only one.
> 
> I'm at a loss here, should we wait on anything before reading from
PROM?
> The mmiotrace I have doesn't seem to suggest we should :s I wonder if 
> you have had this issue before and what the nvkm does here.
> 
Hi Martin,

a functional regression in GK10x prevents your GK107 GPU
from properly decoding 8-bit and 16-bit accesses to
the PROM registers. While these unaligned accesses work on
most GPUs, it is best not to rely on them. You can ensure
reliable retrieval of the VBIOS via the PROM registers
by using 32-bit register accesses across all GPUs.

Thank you, Christian.

> Thanks,
> Martin
> 
> [1] http://lists.freedesktop.org/archives/nouveau/2014-March/016590.html
> [2]
http://code.woboq.org/linux/linux/drivers/gpu/drm/nouveau/core/subdev/bios/base.c.html#144
-- 
christian zander
ch?zander at nvidia.com

On 25/03/2014 17:17, Christian Zander wrote:
> On Mon, Mar 24, 2014 at 11:59:46AM -0700, Martin Peres wrote:
>> Hello,
>>
>> One of my GPU (GK107/NVE7) fails to properly fetch its vbios from PROM
>> at boot time but, if I blacklist the module and load it myself later
on,
>> it always succeeds. To make things weirder, the same card works great
on
>> another computer.
>>
>> Here is the relevant code in Nouveau to fetch the vbios from PROM:
>>
http://code.woboq.org/linux/linux/drivers/gpu/drm/nouveau/core/subdev/bios/base.c.html#nouveau_bios_shadow_prom
>>
>> The only solution I found to this problem is to try up to 16 times in a
>> row to fetch the vbios using PROM. If the 16 tries yield a bad
signature
>> or checksum, I just give up and let it try other techniques. Here is
the
>> patch[1].
>>
>> With this hack in place, the number of retries needed to be able to
>> correctly fetch the vbios ranges from 0 to 13. I see no consistency in
>> this. I tried reading the vbios 32 bits by 32 bits (instead of 8 by 8),
>> disabling local IRQs and generating more activity on the PCIe port
>> before fetching (and making sure the value read is always the same),
all
>> solutions yielded no improvement.
>>
>> It could be a brownout problem but this would be the only manifestation
>> of this problem. Moreover, some cards are known to have problems with
>> PROM[2] so mine is definitely not the only one.
>>
>> I'm at a loss here, should we wait on anything before reading from
PROM?
>> The mmiotrace I have doesn't seem to suggest we should :s I wonder
if
>> you have had this issue before and what the nvkm does here.
>>
> Hi Martin,
>
> a functional regression in GK10x prevents your GK107 GPU
> from properly decoding 8-bit and 16-bit accesses to
> the PROM registers. While these unaligned accesses work on
> most GPUs, it is best not to rely on them. You can ensure
> reliable retrieval of the VBIOS via the PROM registers
> by using 32-bit register accesses across all GPUs.
>
> Thank you, Christian.
Hi Christian,

Thanks, that was what went wrong. I thought I had tested 32-bit accesses 
only but
Imust have forgotten to update the vbios size read...

Thanks again for this quick answer,

Martin