Speex dev - Feb 2008 - Re: Problem with Blackfin assembly optimizations -- bug in fixed_bfin.h / resampler saturation???

Hi,

I tried to figure out what the problem is -- but it seems to be totally
different from what I expected.
My status at the moment is:
- computing results for "generic" and "Blackfin ASM"
versions of the DIV32_16 function are the same, there is no "algorithmic
bug"
- Instead, there seems some sort of memory corruption:
  When I comment out the DIV32_16 function inside "fixed_bfin.h",
everything works fine.
  When I replace the DIV32_16 function inside "fixed_bfin.h" with a
function, which is just a wrapper of the generic function:

static inline spx_word16_t DIV32_16(spx_word32_t a, spx_word16_t b)
{
    return DIV32_16_GENERIC ( a, b );
}

(DIV32_16_GENERIC is just the renamed DIV32_16 preprocessor macro inside
fixed_generic.h),

the the encoder seems to be broken again...

Yes, this is another point: I wrote that the resampler breaks, but this is not
the case. Instead, the speex encoder corrupts its input buffer when the
"DIV32_16 bug" described above happens.
Do you have any idea how to proceed?

best regards,
Frank


-----Urspr?ngliche Nachricht-----
Von: "Jean-Marc Valin" <jean-marc.valin@usherbrooke.ca>
Gesendet: 06.02.08 11:07:46
An: Frank Lorenz <Frank_wtal@web.de>
CC: speex-dev@xiph.org
Betreff: Re: [Speex-dev] Problem with Blackfin assembly optimizations -- bug in
fixed_bfin.h / resampler saturation???


Frank Lorenz a ?crit :
> I just started to examine the DIV32_16 function (Blackfin ASM
> version), and wondered why the return value of the function inside
> 'fixed_bfin.h' is of type 'spx_word16_t', but the local
variable
> 'res' which is returned by this function is of type
'spx_word32_t'.
> Is this a trick of optimization or a bug? (Same question for
> PDIV32_16 and MAX16, too!)
Neither (AFAIK). The idea is that sometimes gcc does
silly/counter-intuitive things when I specify a 16-bit variable as a
constraint. That way, I force the top 16 bits to be zero.

	Jean-Marc
> best regards, Frank
> 
> 
> -----Urspr?ngliche Nachricht----- Von: "Jean-Marc Valin"
> <jean-marc.valin@usherbrooke.ca> Gesendet: 02.02.08 06:26:08 An:
> speex-dev <speex-dev@xiph.org> Betreff: Re: FW: Re: [Speex-dev]
> Problem with Blackfin assembly optimizations -- bug in fixed_bfin.h /
> resampler saturation???
> 
> 
> Frank Lorenz a ?crit :
>> And yes, the same "overflow" happens even when I disable
Blackfin
>> ASM optimizations.
> 
> Indeed, that shouldn't happen. Just to make sure I understand, so far
>  there's two problems: 1) DIV32_16() in Blackfin assembly causes
> problems 2) The resampler overflows
> 
> When you fix/workaround those two, is the encoder/decoder working 
> correctly or are there other issues?
> 
> About the DIV32_16() issue, could you try and replace it with a
> function that compares the Blackfin-asm result with the real division
> and dumps the args and output when they don't match. I'd like to
> understand for what inputs that functions fails. That would help in
> fixing the problem.
> 
> Thanks,
> 
> Jean-Marc
> 
>> best regards, Frank
>> 
>> -----Urspr?ngliche Nachricht----- Von: "Jean-Marc Valin" 
>> <jean-marc.valin@usherbrooke.ca> Gesendet: 01.02.08 12:11:39 An: 
>> Frank Lorenz <Frank_wtal@web.de> CC: speex-dev@xiph.org Betreff:
>> Re: FW: Re: [Speex-dev] Problem with Blackfin assembly
>> optimizations -- bug in fixed_bfin.h / resampler saturation???
>> 
>> 
>> Frank Lorenz a ?crit :
>>> didn't get a reply to my last post (see below) -- do you have
no
>>> idea what happens here?
>> Sorry, been quite busy lately. Still have a backlog of things to 
>> respond to (sorry to others as well), some of it quite old.
>> 
>>> After some more tests, I disabled the DIV32_16 Blackfin 
>>> optimizations and now get good quality on the Blackfin.
>> OK, I'll need to investigate that one. It's mostly taken from
the
>> ADI manual, so I don't see what could have happened.
>> 
>>> But when I have overdrive on the input, things become very bad
>> Does that happen without any assembly. Some amount of badness is 
>> expected from the fixed-point because of saturation issues. It 
>> shouldn't overflow though (can you check?)
>> 
>>> -- I'm not sure if this is really a filter stability issue like
I
>>>  wrote some weeks ago.
>> I don't think it has to do with filters.
>> 
>>> I use the speex resampler to downsample from 48 to 16 kHz. When I
>>>  look to the downsampled signal before passing it to the encoder,
>>>  there seems to be a bug in saturation. On overdrive, instead of 
>>> holding a too big signal value at maximum (e.g. 32767) for some 
>>> samples, the sample values I get from the resampler are 
>>> [-32768,32767,-32768,...], i.e. the output of the resampler
>>> alters between the biggest possible negative value and the
>>> biggest possible positive value.
>> This is definitely bad. Can you send me an example file. Also, can 
>> you reproduce that problem without the Blackfin assembly?
>> 
>>> Does the resampler use the hardware saturation available on 
>>> Blackfin, or does it saturate by software? Can you point me to
>>> the place I have to look at inside the resampler function for 
>>> saturation?
>> Saturation is entirely done in software. The resampler does the 
>> conversion using the WORD2INT macro.
>> 
>> Jean-Marc
>> 
>> 
>> 
>> _______________________________________________________________________
>>  Jetzt neu! Sch?tzen Sie Ihren PC mit McAfee und WEB.DE. 30 Tage 
>> kostenlos testen. 
>> http://www.pc-sicherheit.web.de/startseite/?mc=022220
>> 
>> 
>> 
> 
> 
> 
> _________________________________________________________________________
>  In 5 Schritten zur eigenen Homepage. Jetzt Domain sichern und
> gestalten! Nur 3,99 EUR/Monat!
> http://www.maildomain.web.de/?mc=021114
> 
> 
> 


_______________________________________________________________________
Jetzt neu! Sch?tzen Sie Ihren PC mit McAfee und WEB.DE. 30 Tage
kostenlos testen. http://www.pc-sicherheit.web.de/startseite/?mc=022220

Frank Lorenz a ?crit :
> Yes, this is another point: I wrote that the resampler breaks, but
> this is not the case. Instead, the speex encoder corrupts its input
> buffer when the "DIV32_16 bug" described above happens. Do you
have
> any idea how to proceed?
Can you try enabling --enable-fixed-point-debug (FIXED_DEBUG) and see if
any error gets printed. I suspect there might be a place where the
assumptions of DIV32_16 are violated, i.e. either the right hand side
doesn't fit in 16 bits or the result doesn't fit in 16 bits. Can you
check?

	Jean-Marc
> best regards, Frank
> 
> 
> -----Urspr?ngliche Nachricht----- Von: "Jean-Marc Valin"
> <jean-marc.valin@usherbrooke.ca> Gesendet: 06.02.08 11:07:46 An:
> Frank Lorenz <Frank_wtal@web.de> CC: speex-dev@xiph.org Betreff: Re:
> [Speex-dev] Problem with Blackfin assembly optimizations -- bug in
> fixed_bfin.h / resampler saturation???
> 
> 
> Frank Lorenz a ?crit :
>> I just started to examine the DIV32_16 function (Blackfin ASM 
>> version), and wondered why the return value of the function inside 
>> 'fixed_bfin.h' is of type 'spx_word16_t', but the local
variable
>> 'res' which is returned by this function is of type
'spx_word32_t'.
>>  Is this a trick of optimization or a bug? (Same question for 
>> PDIV32_16 and MAX16, too!)
> 
> Neither (AFAIK). The idea is that sometimes gcc does 
> silly/counter-intuitive things when I specify a 16-bit variable as a 
> constraint. That way, I force the top 16 bits to be zero.
> 
> Jean-Marc
> 
>> best regards, Frank
>> 
>> 
>> -----Urspr?ngliche Nachricht----- Von: "Jean-Marc Valin" 
>> <jean-marc.valin@usherbrooke.ca> Gesendet: 02.02.08 06:26:08 An: 
>> speex-dev <speex-dev@xiph.org> Betreff: Re: FW: Re: [Speex-dev] 
>> Problem with Blackfin assembly optimizations -- bug in fixed_bfin.h
>> / resampler saturation???
>> 
>> 
>> Frank Lorenz a ?crit :
>>> And yes, the same "overflow" happens even when I disable
Blackfin
>>>  ASM optimizations.
>> Indeed, that shouldn't happen. Just to make sure I understand, so
>> far there's two problems: 1) DIV32_16() in Blackfin assembly causes
>>  problems 2) The resampler overflows
>> 
>> When you fix/workaround those two, is the encoder/decoder working 
>> correctly or are there other issues?
>> 
>> About the DIV32_16() issue, could you try and replace it with a 
>> function that compares the Blackfin-asm result with the real
>> division and dumps the args and output when they don't match.
I'd
>> like to understand for what inputs that functions fails. That would
>> help in fixing the problem.
>> 
>> Thanks,
>> 
>> Jean-Marc
>> 
>>> best regards, Frank
>>> 
>>> -----Urspr?ngliche Nachricht----- Von: "Jean-Marc Valin" 
>>> <jean-marc.valin@usherbrooke.ca> Gesendet: 01.02.08 12:11:39
An:
>>>  Frank Lorenz <Frank_wtal@web.de> CC: speex-dev@xiph.org
Betreff:
>>>  Re: FW: Re: [Speex-dev] Problem with Blackfin assembly 
>>> optimizations -- bug in fixed_bfin.h / resampler saturation???
>>> 
>>> 
>>> Frank Lorenz a ?crit :
>>>> didn't get a reply to my last post (see below) -- do you
have
>>>> no idea what happens here?
>>> Sorry, been quite busy lately. Still have a backlog of things to
>>>  respond to (sorry to others as well), some of it quite old.
>>> 
>>>> After some more tests, I disabled the DIV32_16 Blackfin 
>>>> optimizations and now get good quality on the Blackfin.
>>> OK, I'll need to investigate that one. It's mostly taken
from the
>>>  ADI manual, so I don't see what could have happened.
>>> 
>>>> But when I have overdrive on the input, things become very bad
>>> Does that happen without any assembly. Some amount of badness is
>>>  expected from the fixed-point because of saturation issues. It 
>>> shouldn't overflow though (can you check?)
>>> 
>>>> -- I'm not sure if this is really a filter stability issue
like
>>>> I wrote some weeks ago.
>>> I don't think it has to do with filters.
>>> 
>>>> I use the speex resampler to downsample from 48 to 16 kHz. When
>>>> I look to the downsampled signal before passing it to the
>>>> encoder, there seems to be a bug in saturation. On overdrive,
>>>> instead of holding a too big signal value at maximum (e.g.
>>>> 32767) for some samples, the sample values I get from the
>>>> resampler are [-32768,32767,-32768,...], i.e. the output of the
>>>> resampler alters between the biggest possible negative value
>>>> and the biggest possible positive value.
>>> This is definitely bad. Can you send me an example file. Also,
>>> can you reproduce that problem without the Blackfin assembly?
>>> 
>>>> Does the resampler use the hardware saturation available on 
>>>> Blackfin, or does it saturate by software? Can you point me to 
>>>> the place I have to look at inside the resampler function for 
>>>> saturation?
>>> Saturation is entirely done in software. The resampler does the 
>>> conversion using the WORD2INT macro.
>>> 
>>> Jean-Marc
>>> 
>>> 
>>> 
>>>
_______________________________________________________________________
>>>  Jetzt neu! Sch?tzen Sie Ihren PC mit McAfee und WEB.DE. 30 Tage
>>>  kostenlos testen. 
>>> http://www.pc-sicherheit.web.de/startseite/?mc=022220
>>> 
>>> 
>>> 
>> 
>> 
>>
_________________________________________________________________________
>>  In 5 Schritten zur eigenen Homepage. Jetzt Domain sichern und 
>> gestalten! Nur 3,99 EUR/Monat! 
>> http://www.maildomain.web.de/?mc=021114
>> 
>> 
>> 
> 
> 
> 
> _______________________________________________________________________
>  Jetzt neu! Sch?tzen Sie Ihren PC mit McAfee und WEB.DE. 30 Tage 
> kostenlos testen.
> http://www.pc-sicherheit.web.de/startseite/?mc=022220
> 
> 
>