Speex dev - Jan 2010 - Fwd: Re: Fixed Point on wideband-mode: Single Frame loss on 2000 Hz sine causes "freak off"

Hi Jean-Marc,

yes, problem exists in narrowband-mode, too.

I already twiddled with max_gain, but did not have real success. I changed line
337 of ltp.c (function pitch_gain_search_3tap_vq)

    if (sum>best_sum && gain_sum<=max_gain) {

to
    if (sum>best_sum && gain_sum<max_gain) {

-- that stabilizes speex for 2000 Hz and 2200 Hz input on quality setting 7
(23800 bits/s) complexity 2, while it does not help much for quality setting 9
((34400 bits/s) (just slightly reduces the duration of the "freak
out").


best regards,
Frank



Jean-Marc Valin <jean-marc.valin at usherbrooke.ca> hat am 14. Januar 2010
um 02:27 geschrieben:
> Hi,
>
> That's very useful information. From what you tell me, it may be
related
> to the handling of max_gain in pitch_gain_search_3tap() (ltp.c). Also,
> are you able to reproduce the problem in narrowband mode?
>
> Cheers,
>
>         Jean-Marc
>
> On 2010-01-13 07:34, Frank Lorenz wrote:
> > Hi Jean-Marc,
> >
> > yes, I tested with floating point. It is only a fixed point problem,
> > floating point works fine. Even a fixed point decoder is o.k. --
problem
> > is inside encoder.
> >
> > I had the idea of an "above unity" pitch gain, too, but I
did not manage
> > to modifiy the encoder in a way to fix it. Because the encoder's
pitch
> > handling is some kind of "feedback system" (i.e. a change in
output
> > parameters of frame N will influence input parameters in frame N+1),
> > it's not so easy for me to reduce gain to test for stability. All
> > modifications I tried made the situation worse, so I need some adivce
> > how to proceed...
> >
> > The complexity parameter has some influence, too: For a sine input of
> > 2200 Hz, the instability is present only for complexity values up to
3,
> > but for the 2000 Hz input, instability can be observed even for the
> > highest complexity values.
> >
> > As already written, I observed that the pitch and pitch gain values
> > computed with the fixed point encoder seem quite odd and are
absolutely
> > different from the ones computed with floating point. This seems to
> > stabilize for higher complexity values, so I'm not sure if this is
the
> > cause of the problem or only a side effect.
> >
> > best regards,
> >
> > Frank
> >
> >
> > Jean-Marc Valin <jean-marc.valin at usherbrooke.ca> hat am 13.
Januar 2010
> > um 12:48 geschrieben:
> >
> >  > Hi Frank,
> >  >
> >  > Sorry, I *do* care about the problem and just happen to be
overworked at
> >  > the moment. What I suspect is that the pitch gain gets close
enough to
> >  > unity that the loss makes it bust. Did you test with the
floating-point
> >  > code?
> >  >
> >  > Jean-Marc
> >  >
> >  > On 2010-01-13 03:45, Frank Lorenz wrote:
> >  > > Hi,
> >  > >
> >  > > is no one willing to spent some effort on this topic? At
least it would
> >  > > be good
> >  > > to know if anybody can reproduce this problem.
> >  > >
> >  > > I did further investigation on the topic and found out that
the encoder
> >  > > in fixed
> >  > > point mode produces absolutely different pitch and pitch
gain
> > parameters
> >  > > than in
> >  > > floating point mode (for a 2000 Hz input signal). The codec
breaks on
> >  > > frame loss
> >  > > if the encoder is in fixed point mode, decoder doesn't
matter, so I
> >  > > assume the
> >  > > pitch parameters are "bad".
> >  > > Maybe its caused by accuracy/rounding problems for this
periodic
> > signal?
> >  > >
> >  > > Because I do not know much about the internals of Speex,
it's hard
> > for me to
> >  > > analyse the pitch / pitch gain computation. Some help would
be really
> >  > > nice...
> >  > >
> >  > > best regards,
> >  > > Frank
> >  > >
> >  > >
> >  > >
> >  > > ---------- Urspr?ngliche Nachricht ----------
> >  > > Von: Frank Lorenz <Frank_wtal at web.de>
> >  > > An: speex-dev at xiph.org
> >  > > Datum: 4. Januar 2010 um 09:57
> >  > > Betreff: Re: [Speex-dev] Fixed Point on wideband-mode:
Single Frame
> > loss
> >  > > on 2000
> >  > > Hz sine causes "freak off"
> >  > >
> >  > >
> >  > > Hi Jean-Marc, all,
> >  > >
> >  > > I didn't get any response to my issue up to now and
would like to
> > now if
> >  > > anyone
> >  > > can reproduce this behaviour and if there is some idea what
> > happens. I am
> >  > > willing to fix this issue, but because I do not know the
internals of
> >  > > speex, I
> >  > > need some advice on how to proceed...
> >  > >
> >  > > best regards,
> >  > > Frank
> >  > >
> >  > >
> >  > >
> >  > > Frank Lorenz <Frank_wtal at web.de> hat am 21.
Dezember 2009 um 10:49
> >  > > geschrieben:
> >  > >
> >  > > > An update:
> >  > > >
> >  > > > I found that the "ADD32: output is not int:"
messages are caused
> > by the
> >  > > > enhancer. When I turn the enhancer off, the messages
vanish, but the
> >  > > "freak
> >  > > > out" of the codec is still there - so the problem
seems not to be
> >  > > related to
> >  > > > the overflow messages.
> >  > > >
> >  > > > best regards,
> >  > > > Frank
> >  > > >
> >  > > >
> >  > > > ---------- Urspr?ngliche Nachricht ----------
> >  > > > Von: Frank Lorenz <Frank_wtal at web.de>
> >  > > > An: speex-dev at xiph.org
> >  > > > Datum: 18. Dezember 2009 um 13:36
> >  > > > Betreff: [Speex-dev] Fixed Point on wideband-mode:
Single Frame loss
> >  > > on 2000
> >  > > > Hz sine causes "freak off"
> >  > > >
> >  > > > Hi,
> >  > > >
> >  > > > I'm quite sure we found a bug inside the codec
(1.2rc1, wideband
> >  > > mode). It
> >  > > > only
> >  > > > occurs when speex is compiled with the FIXED_POINT
flag.
> >  > > >
> >  > > > I feed in a 2000 Hz sine wave (-15 dBFS). When I
simulate a single
> >  > > frame loss
> >  > > > (i.e. pass a null pointer to speex_decode_int as
second parameter for
> >  > > just one
> >  > > > frame), the decoder
> >  > > > starts to freak out a second later: It produces
horrible noise with 0
> >  > > dBFS.
> >  > > > This
> >  > > > horrible noise will not vanish until you turn off the
input signal.
> >  > > >
> >  > > > I did some investigation on this with FIXED_DEBUG. The
error
> > messages are
> >  > > > (lots
> >  > > > of this):
> >  > > > ADD32: output is not int: 1702286034 in ltp.c: line 68
> >  > > > ADD32: output is not int: 1455757562 in ltp.c: line 69
> >  > > >
> >  > > > Call stack for this (only set the break point once,
may be there are
> >  > > other
> >  > > > call
> >  > > > stacks on error, too):
> >  > > >
> >  > > > sb_decode ( ) at sb_celp.c:898
> >  > > > nb_decode ( ) at nb_celp.c:1471
> >  > > > multicomb ( ) at filter.c:709
> >  > > > interp_pitch ( ) at filter.c:603
> >  > > > inner_prod ( ) at ltp.c:68
> >  > > >
> >  > > > Can you help please?
> >  > > >
> >  > > > best regards,
> >  > > > Frank
> >  > > > ______________________________________________________
> >  > > > GRATIS f?r alle WEB.DE-Nutzer: Die maxdome Movie-FLAT!
> >  > > > Jetzt freischalten unter http://movieflat.web.de
> >  > > <http://movieflat.web.de/>
> >  > > >
> >  > > > _______________________________________________
> >  > > > Speex-dev mailing list
> >  > > > Speex-dev at xiph.org
> >  > > > http://lists.xiph.org/mailman/listinfo/speex-dev
> >  > > ______________________________________________________
> >  > > GRATIS f?r alle WEB.DE-Nutzer: Die maxdome Movie-FLAT!
> >  > > Jetzt freischalten unter http://movieflat.web.de
> > <http://movieflat.web.de/>
> >  > >
> >  > > _______________________________________________
> >  > > Speex-dev mailing list
> >  > > Speex-dev at xiph.org
> >  > > http://lists.xiph.org/mailman/listinfo/speex-dev
> >  > >
> >  > > Web Bug from https://img.web.de/p.gif
> >  > > WEB.DE MillionenKlick: Kostenlos tippen, t?glich 1 Million
gewinnen!
> >  > > http://produkte.web.de/go/07/
> >  > >
> >  > >
> >  > >
> >  > > _______________________________________________
> >  > > Speex-dev mailing list
> >  > > Speex-dev at xiph.org
> >  > > http://lists.xiph.org/mailman/listinfo/speex-dev
> >  >
> >
> >
> >
> > Web Bug from https://img.web.de/p.gif
> > GRATIS f?r alle WEB.DE-Nutzer: Die maxdome Movie-FLAT!
> > Jetzt freischalten unter http://movieflat.web.de
> >
> >
> >
> > _______________________________________________
> > Speex-dev mailing list
> > Speex-dev at xiph.org
> > http://lists.xiph.org/mailman/listinfo/speex-dev
>
___________________________________________________________
Preisknaller: WEB.DE DSL Flatrate f?r nur 16,99 Euro/mtl.! 
http://produkte.web.de/go/02/

What happens if you change that line:

if (cumul_gain > 262144)

to use a smaller value? What value works OK (if any)?

One more thing, when things go wrong, do they eventually go back to 
normal or does the codec never recover? It's unavoidable that the audio 
goes bad for a short period of time because of the long-term predictor.

	Jean-Marc

On 2010-01-14 05:57, Frank Lorenz wrote:
> Hi Jean-Marc,
>
> yes, problem exists in narrowband-mode, too.
>
> I already twiddled with max_gain, but did not have real success. I
> changed line 337 of ltp.c (function pitch_gain_search_3tap_vq)
>
> if (sum>best_sum&&  gain_sum<=max_gain) {
>
> to if (sum>best_sum&&  gain_sum<max_gain) {
>
> -- that stabilizes speex for 2000 Hz and 2200 Hz input on quality
> setting 7 (23800 bits/s) complexity 2, while it does not help much
> for quality setting 9 ((34400 bits/s) (just slightly reduces the
> duration of the "freak out").
>
>
> best regards, Frank
>
>
>
> Jean-Marc Valin<jean-marc.valin at usherbrooke.ca>  hat am 14. Januar
> 2010 um 02:27 geschrieben:
>
>> Hi,
>>
>> That's very useful information. From what you tell me, it may be
>> related to the handling of max_gain in pitch_gain_search_3tap()
>> (ltp.c). Also, are you able to reproduce the problem in narrowband
>> mode?
>>
>> Cheers,
>>
>> Jean-Marc
>>
>> On 2010-01-13 07:34, Frank Lorenz wrote:
>>> Hi Jean-Marc,
>>>
>>> yes, I tested with floating point. It is only a fixed point
>>> problem, floating point works fine. Even a fixed point decoder is
>>> o.k. -- problem is inside encoder.
>>>
>>> I had the idea of an "above unity" pitch gain, too, but I
did not
>>> manage to modifiy the encoder in a way to fix it. Because the
>>> encoder's pitch handling is some kind of "feedback
system" (i.e.
>>> a change in output parameters of frame N will influence input
>>> parameters in frame N+1), it's not so easy for me to reduce
gain
>>> to test for stability. All modifications I tried made the
>>> situation worse, so I need some adivce how to proceed...
>>>
>>> The complexity parameter has some influence, too: For a sine
>>> input of 2200 Hz, the instability is present only for complexity
>>> values up to 3, but for the 2000 Hz input, instability can be
>>> observed even for the highest complexity values.
>>>
>>> As already written, I observed that the pitch and pitch gain
>>> values computed with the fixed point encoder seem quite odd and
>>> are absolutely different from the ones computed with floating
>>> point. This seems to stabilize for higher complexity values, so
>>> I'm not sure if this is the cause of the problem or only a side
>>> effect.
>>>
>>> best regards,
>>>
>>> Frank
>>>
>>>
>>> Jean-Marc Valin<jean-marc.valin at usherbrooke.ca>  hat am
13.
>>> Januar 2010 um 12:48 geschrieben:
>>>
>>>> Hi Frank,
>>>>
>>>> Sorry, I *do* care about the problem and just happen to be
>>>> overworked at the moment. What I suspect is that the pitch gain
>>>> gets close enough to unity that the loss makes it bust. Did you
>>>> test with the floating-point code?
>>>>
>>>> Jean-Marc
>>>>
>>>> On 2010-01-13 03:45, Frank Lorenz wrote:
>>>>> Hi,
>>>>>
>>>>> is no one willing to spent some effort on this topic? At
>>>>> least it would be good to know if anybody can reproduce
this
>>>>> problem.
>>>>>
>>>>> I did further investigation on the topic and found out that
>>>>> the encoder in fixed point mode produces absolutely
different
>>>>> pitch and pitch gain
>>> parameters
>>>>> than in floating point mode (for a 2000 Hz input signal).
The
>>>>> codec breaks on frame loss if the encoder is in fixed point
>>>>> mode, decoder doesn't matter, so I assume the pitch
>>>>> parameters are "bad". Maybe its caused by
accuracy/rounding
>>>>> problems for this periodic
>>> signal?
>>>>>
>>>>> Because I do not know much about the internals of Speex,
it's
>>>>> hard
>>> for me to
>>>>> analyse the pitch / pitch gain computation. Some help would
>>>>> be really nice...
>>>>>
>>>>> best regards, Frank
>>>>>
>>>>>
>>>>>
>>>>> ---------- Urspr?ngliche Nachricht ---------- Von: Frank
>>>>> Lorenz<Frank_wtal at web.de> An: speex-dev at
xiph.org Datum: 4.
>>>>> Januar 2010 um 09:57 Betreff: Re: [Speex-dev] Fixed Point
on
>>>>> wideband-mode: Single Frame
>>> loss
>>>>> on 2000 Hz sine causes "freak off"
>>>>>
>>>>>
>>>>> Hi Jean-Marc, all,
>>>>>
>>>>> I didn't get any response to my issue up to now and
would
>>>>> like to
>>> now if
>>>>> anyone can reproduce this behaviour and if there is some
idea
>>>>> what
>>> happens. I am
>>>>> willing to fix this issue, but because I do not know the
>>>>> internals of speex, I need some advice on how to proceed...
>>>>>
>>>>> best regards, Frank
>>>>>
>>>>>
>>>>>
>>>>> Frank Lorenz<Frank_wtal at web.de>  hat am 21.
Dezember 2009 um
>>>>> 10:49 geschrieben:
>>>>>
>>>>>> An update:
>>>>>>
>>>>>> I found that the "ADD32: output is not int:"
messages are
>>>>>> caused
>>> by the
>>>>>> enhancer. When I turn the enhancer off, the messages
>>>>>> vanish, but the
>>>>> "freak
>>>>>> out" of the codec is still there - so the problem
seems not
>>>>>> to be
>>>>> related to
>>>>>> the overflow messages.
>>>>>>
>>>>>> best regards, Frank
>>>>>>
>>>>>>
>>>>>> ---------- Urspr?ngliche Nachricht ---------- Von:
Frank
>>>>>> Lorenz<Frank_wtal at web.de> An: speex-dev at
xiph.org Datum: 18.
>>>>>> Dezember 2009 um 13:36 Betreff: [Speex-dev] Fixed Point
on
>>>>>> wideband-mode: Single Frame loss
>>>>> on 2000
>>>>>> Hz sine causes "freak off"
>>>>>>
>>>>>> Hi,
>>>>>>
>>>>>> I'm quite sure we found a bug inside the codec
(1.2rc1,
>>>>>> wideband
>>>>> mode). It
>>>>>> only occurs when speex is compiled with the FIXED_POINT
>>>>>> flag.
>>>>>>
>>>>>> I feed in a 2000 Hz sine wave (-15 dBFS). When I
simulate a
>>>>>> single
>>>>> frame loss
>>>>>> (i.e. pass a null pointer to speex_decode_int as second
>>>>>> parameter for
>>>>> just one
>>>>>> frame), the decoder starts to freak out a second later:
It
>>>>>> produces horrible noise with 0
>>>>> dBFS.
>>>>>> This horrible noise will not vanish until you turn off
the
>>>>>> input signal.
>>>>>>
>>>>>> I did some investigation on this with FIXED_DEBUG. The
>>>>>> error
>>> messages are
>>>>>> (lots of this): ADD32: output is not int: 1702286034 in
>>>>>> ltp.c: line 68 ADD32: output is not int: 1455757562 in
>>>>>> ltp.c: line 69
>>>>>>
>>>>>> Call stack for this (only set the break point once, may
be
>>>>>> there are
>>>>> other
>>>>>> call stacks on error, too):
>>>>>>
>>>>>> sb_decode ( ) at sb_celp.c:898 nb_decode ( ) at
>>>>>> nb_celp.c:1471 multicomb ( ) at filter.c:709
interp_pitch (
>>>>>> ) at filter.c:603 inner_prod ( ) at ltp.c:68
>>>>>>
>>>>>> Can you help please?
>>>>>>
>>>>>> best regards, Frank
>>>>>> ______________________________________________________
>>>>>> GRATIS f?r alle WEB.DE-Nutzer: Die maxdome Movie-FLAT!
>>>>>> Jetzt freischalten unter http://movieflat.web.de
>>>>> <http://movieflat.web.de/>
>>>>>>
>>>>>> _______________________________________________
Speex-dev
>>>>>> mailing list Speex-dev at xiph.org
>>>>>> http://lists.xiph.org/mailman/listinfo/speex-dev
>>>>> ______________________________________________________
GRATIS
>>>>> f?r alle WEB.DE-Nutzer: Die maxdome Movie-FLAT! Jetzt
>>>>> freischalten unter http://movieflat.web.de
>>> <http://movieflat.web.de/>
>>>>>
>>>>> _______________________________________________ Speex-dev
>>>>> mailing list Speex-dev at xiph.org
>>>>> http://lists.xiph.org/mailman/listinfo/speex-dev
>>>>>
>>>>> Web Bug from https://img.web.de/p.gif WEB.DE
MillionenKlick:
>>>>> Kostenlos tippen, t?glich 1 Million gewinnen!
>>>>> http://produkte.web.de/go/07/
>>>>>
>>>>>
>>>>>
>>>>> _______________________________________________ Speex-dev
>>>>> mailing list Speex-dev at xiph.org
>>>>> http://lists.xiph.org/mailman/listinfo/speex-dev
>>>>
>>>
>>>
>>>
>>> Web Bug from https://img.web.de/p.gif GRATIS f?r alle
>>> WEB.DE-Nutzer: Die maxdome Movie-FLAT! Jetzt freischalten unter
>>> http://movieflat.web.de
>>>
>>>
>>>
>>> _______________________________________________ Speex-dev mailing
>>> list Speex-dev at xiph.org
>>> http://lists.xiph.org/mailman/listinfo/speex-dev
>>
>
> ___________________________________________________________
> Preisknaller: WEB.DE DSL Flatrate f?r nur 16,99 Euro/mtl.!
> http://produkte.web.de/go/02/
>
>
>

Dear Jean-Mark,

I am using your really amazing speex codec with success for a year in my
narrow-band, cryptographic voip application (it is in testing phase only).
In literature, I have found some so called "IP Over Voice" (IOV, IPOV)
theory, which utilizes a CELP codec in inverse, transmitting any random
digital data as CELP synthetized voice though any voice transmitter
channels (CB radio, GSM voice call, telephone etc.), then converts it
back into bytes with low distortion/loss.

This is an example:
http://info.ee.surrey.ac.uk/Personal/N.Katugampala/pubs/iee04.pdf

My first question is that if you have ever tried to make such an
application.
My second one is that if you authorize me to use the source code of
Speex to create such means, and under what licence.
The third one, if you can give some initial tips to start with this project.

Best regards,
Tom