Speex dev - Dec 2011 - Decoding only a certain frame results in different values than when decoding the entire file

Sorry, it seems I have only replied to Lakhdar, not to the newsgroup.
Below is my reply to Lakhdar, and I would like to make it more clear 
now, using some pseudo values for simplicity:

I read bytes 1 to 124 from my encoded spx file.
I decode themt and get the values:
---Frame 1----
-293
-8234
2134
17
---Frame 2----
-9323
-732
189
2329

Both frames are just perfect as I need them.

But now when I read bytes 63 to 125 from my encoded spx and decode it, I 
get the values
732
9273
-799
-723

This is weird because I expected to get the encoded values
-9323
-732
189
2329

I hope I could explain it well.
Am I missing something here?

Thank you very much.
Hermann



Hi Lakhdar,

I am not sure if I do anything wrong with the dec state memory.
Could you perhaps take a look at what I am (perhaps wrongly) doing?

int __stdcall SpxDecode(unsigned char* DinBuf, float* DoutBuf, unsigned 
int Dinlen)
{

     //char *testFile;
     //FILE *ftest;
     //testFile = "test2";
     //ftest = fopen(testFile, "wb");
     //fwrite(inBuf,1,inlen,ftest);
     //fwrite(outBuf, 1, outBufpos, ftest);
     //fclose(ftest);

     //take every 62 bytes
     //decode, then write to outbuf

     char cbits[MAX_FRAME_BYTES];
     float output[MAX_FRAME_SIZE];
     short out[MAX_FRAME_SIZE];
     unsigned int i,j;

     unsigned char* tout; tout = (unsigned char*)&out;

     for (i=0; i<(Dinlen/62); i++)
         {
         for (j=0;j<62;j++)    cbits[j] = DinBuf[(i*62)+j];

         speex_bits_read_from(&bits, cbits, 62);
         speex_decode(decstate, &bits, output);

         for (j=0;j<160;j++) DoutBuf[j+(i*160)] = output[j];
         }

     return 43;
}

Btw, just if you wondering what I am doing... I am feeding (62 * 
NumberOfFramesThatIWantToRead) bytes from VB6 and return it as a float 
array.

Greetings,
Hermann

Am 21.12.2011 15:26, schrieb Lakhdar Bourokba:
 > Hi Hermann,
 > Could it be your decoder state memory that is not longer sync ?
 > When a decoder processes a frame, it uses that frame and its memory 
state (affected by the previous frames) to generate the output samples. 
If you skip the previous frames, the decoder state memory will not be 
the same (as when you do full decoding) which causes the output to be 
different.
 > I hope this helps.
 > L.
 >
 >
 > On Wed, Dec 21, 2011 at 5:17 AM, Hermann Weber <hermie.weber at
gmx.de>
wrote:
 >
 >     Hello!
 >
 >     I am still using version 1.0.4.
 >
 >     When I have encoded a larger file and then only want to decode 
only one
 >     frame from it (for example the 2nd frame), is there anything special
 >     that I have to keep in mind?
 >
 >     I thought that I could simply read the bytes from the encoded 
file for
 >     example from the start position of the 2nd frame and then decode 
it. I
 >     did that, and I can play it play, but the values in the decoded 2nd
 >     frame are different than they are when I decode the entire file 
and then
 >     look at the values for the 2nd frame.
 >     I hope I could explain it well.
 >
 >     I noticed that there is a pattern in the beginning of the encoded 
file,
 >     and I am not sure if I have to take it into account somehow.
 >
 >     It would be nice if someone could tell me if something comes to 
his/her
 >     mind that I might be doing wrong.
 >
 >     Thank you very much.
 >
 >     Hermann
 >     _______________________________________________
 >     Speex-dev mailing list
 >     Speex-dev at xiph.org
 >     http://lists.xiph.org/mailman/listinfo/speex-dev
 >
 >


Hello!

I am still using version 1.0.4.

When I have encoded a larger file and then only want to decode only one 
frame from it (for example the 2nd frame), is there anything special 
that I have to keep in mind?

I thought that I could simply read the bytes from the encoded file for 
example from the start position of the 2nd frame and then decode it. I 
did that, and I can play it play, but the values in the decoded 2nd 
frame are different than they are when I decode the entire file and then 
look at the values for the 2nd frame.
I hope I could explain it well.

I noticed that there is a pattern in the beginning of the encoded file, 
and I am not sure if I have to take it into account somehow.

It would be nice if someone could tell me if something comes to his/her 
mind that I might be doing wrong.

Thank you very much.

Hermann

HI Hermann,
I am not sure I understand exactly what you are trying to do.
Let me try to describe what I understood (I assumed your byte array starts
@ 1 and not 0 as it is supposed to be in C) :

##############################
CASE 1: (output is good)
##############################
   - INITILIAZE_DECODER (reset state memory)
   - READ bytes 1 to 124
   - SpxDecode(DinBuf, DoutBuf, unsigned int Dinlen)
   - DECODE bytes 1 to 62
             ---OUTPUT Frame 1----
             -293
             -8234
             2134
             17
   - DECODE bytes 63 to 124
             ---OUTPUT Frame 2----
        -9323
       -732
       189
       2329

##############################
CASE 2: (output is not as expected)
##############################
   - INITILIAZE_DECODER (reset state memory)
   - SKIP bytes 1 to 62 (This is where I get confused, do you actually skip
FRAME 1 ?)
   - READ bytes 63 to 124
   - DECODE bytes 63 to 124 (and not 125 as you mentioned, I think it's a
typo but I just want to make sure)
             ---OUTPUT Frame 2----
       732
       9273
       -799
       -723



If what is described above is what you are doing, you can not expect both
outputs to be identical.
The reason is because the decoder has a different state prior to processing
FRAME 2.
    - In case 2, the decoder has its state reset to the default state.
    - In case 1, the decoder memory state has been modified by the
processing of FRAME 1.

If I got it wrong, please provide more details.
L.


On Wed, Dec 21, 2011 at 6:03 PM, Hermann Weber <hermie.weber at gmx.de>
wrote:
> Sorry, it seems I have only replied to Lakhdar, not to the newsgroup.
> Below is my reply to Lakhdar, and I would like to make it more clear
> now, using some pseudo values for simplicity:
>
> I read bytes 1 to 124 from my encoded spx file.
> I decode themt and get the values:
> ---Frame 1----
> -293
> -8234
> 2134
> 17
> ---Frame 2----
> -9323
> -732
> 189
> 2329
>
> Both frames are just perfect as I need them.
>
> But now when I read bytes 63 to 125 from my encoded spx and decode it, I
> get the values
> 732
> 9273
> -799
> -723
>
> This is weird because I expected to get the encoded values
> -9323
> -732
> 189
> 2329
>
> I hope I could explain it well.
> Am I missing something here?
>
> Thank you very much.
> Hermann
>
>
>
> Hi Lakhdar,
>
> I am not sure if I do anything wrong with the dec state memory.
> Could you perhaps take a look at what I am (perhaps wrongly) doing?
>
> int __stdcall SpxDecode(unsigned char* DinBuf, float* DoutBuf, unsigned
> int Dinlen)
> {
>
>     //char *testFile;
>     //FILE *ftest;
>     //testFile = "test2";
>     //ftest = fopen(testFile, "wb");
>     //fwrite(inBuf,1,inlen,ftest);
>     //fwrite(outBuf, 1, outBufpos, ftest);
>     //fclose(ftest);
>
>     //take every 62 bytes
>     //decode, then write to outbuf
>
>     char cbits[MAX_FRAME_BYTES];
>     float output[MAX_FRAME_SIZE];
>     short out[MAX_FRAME_SIZE];
>     unsigned int i,j;
>
>     unsigned char* tout; tout = (unsigned char*)&out;
>
>     for (i=0; i<(Dinlen/62); i++)
>         {
>         for (j=0;j<62;j++)    cbits[j] = DinBuf[(i*62)+j];
>
>         speex_bits_read_from(&bits, cbits, 62);
>         speex_decode(decstate, &bits, output);
>
>         for (j=0;j<160;j++) DoutBuf[j+(i*160)] = output[j];
>         }
>
>     return 43;
> }
>
> Btw, just if you wondering what I am doing... I am feeding (62 *
> NumberOfFramesThatIWantToRead) bytes from VB6 and return it as a float
> array.
>
> Greetings,
> Hermann
>
> Am 21.12.2011 15:26, schrieb Lakhdar Bourokba:
>  > Hi Hermann,
>  > Could it be your decoder state memory that is not longer sync ?
>  > When a decoder processes a frame, it uses that frame and its memory
> state (affected by the previous frames) to generate the output samples.
> If you skip the previous frames, the decoder state memory will not be
> the same (as when you do full decoding) which causes the output to be
> different.
>  > I hope this helps.
>  > L.
>  >
>  >
>  > On Wed, Dec 21, 2011 at 5:17 AM, Hermann Weber <hermie.weber at
gmx.de>
> wrote:
>  >
>  >     Hello!
>  >
>  >     I am still using version 1.0.4.
>  >
>  >     When I have encoded a larger file and then only want to decode
> only one
>  >     frame from it (for example the 2nd frame), is there anything
special
>  >     that I have to keep in mind?
>  >
>  >     I thought that I could simply read the bytes from the encoded
> file for
>  >     example from the start position of the 2nd frame and then decode
> it. I
>  >     did that, and I can play it play, but the values in the decoded
2nd
>  >     frame are different than they are when I decode the entire file
> and then
>  >     look at the values for the 2nd frame.
>  >     I hope I could explain it well.
>  >
>  >     I noticed that there is a pattern in the beginning of the encoded
> file,
>  >     and I am not sure if I have to take it into account somehow.
>  >
>  >     It would be nice if someone could tell me if something comes to
> his/her
>  >     mind that I might be doing wrong.
>  >
>  >     Thank you very much.
>  >
>  >     Hermann
>  >     _______________________________________________
>  >     Speex-dev mailing list
>  >     Speex-dev at xiph.org
>  >     http://lists.xiph.org/mailman/listinfo/speex-dev
>  >
>  >
>
>
> Hello!
>
> I am still using version 1.0.4.
>
> When I have encoded a larger file and then only want to decode only one
> frame from it (for example the 2nd frame), is there anything special
> that I have to keep in mind?
>
> I thought that I could simply read the bytes from the encoded file for
> example from the start position of the 2nd frame and then decode it. I
> did that, and I can play it play, but the values in the decoded 2nd
> frame are different than they are when I decode the entire file and then
> look at the values for the 2nd frame.
> I hope I could explain it well.
>
> I noticed that there is a pattern in the beginning of the encoded file,
> and I am not sure if I have to take it into account somehow.
>
> It would be nice if someone could tell me if something comes to his/her
> mind that I might be doing wrong.
>
> Thank you very much.
>
> Hermann
> _______________________________________________
> Speex-dev mailing list
> Speex-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/speex-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/speex-dev/attachments/20111222/1b850142/attachment.htm

Hi Lakhdar,

yes, sorry, it was a typo. I am decoding bytes 63 to 124, not 63 to 125.
Yes, Case 1 is good.
Yes, what you said is correct: As soon as I reset the memory state, the 
output is not as expected anymore.
In Case 1 (which is good), I don't reset the memory state after decoding 
1 frame.

In Case 2 I am not skipping the frame by skipping some bytes.
Instead I only read the bytes 63 to 124 from the encoded file and decode it.

Personally, I do not care if the results are binarily the same, but for 
some reason, the output in Case 2 is so that I can not work with it.
I can play it, but processing it further is not possible because the 
values are different than what I expected.

In the encoded file (when viewed with WinHex) I see something like a 
header. Is it really a header? Does it affect the decoding somehow? I am 
asking because when I decode bytes 63 to 124 only, I skip this header of 
course, and it might be an explanation why the results are different.

In which way does the previous frame affect the next frame?

Thank you so much for the help.

Greetings,
Hermann

Am 22.12.2011 19:53, schrieb Lakhdar Bourokba:
> HI Hermann,
> I am not sure I understand exactly what you are trying to do.
> Let me try to describe what I understood (I assumed your byte array 
> starts @ 1 and not 0 as it is supposed to be in C) :
>
> ##############################
> CASE 1: (output is good)
> ##############################
>    - INITILIAZE_DECODER (reset state memory)
>    - READ bytes 1 to 124
>    - SpxDecode(DinBuf, DoutBuf, unsigned int Dinlen)
>    - DECODE bytes 1 to 62
>              ---OUTPUT Frame 1----
>              -293
>              -8234
>              2134
>              17
>    - DECODE bytes 63 to 124
>              ---OUTPUT Frame 2----
>         -9323
>        -732
>        189
>        2329
>
> ##############################
> CASE 2: (output is not as expected)
> ##############################
>    - INITILIAZE_DECODER (reset state memory)
>    - SKIP bytes 1 to 62 (This is where I get confused, do you actually 
> skip FRAME 1 ?)
>    - READ bytes 63 to 124
>    - DECODE bytes 63 to 124 (and not 125 as you mentioned, I think 
> it's a typo but I just want to make sure)
>              ---OUTPUT Frame 2----
>        732
>        9273
>        -799
>        -723
>
>
>
> If what is described above is what you are doing, you can not expect 
> both outputs to be identical.
> The reason is because the decoder has a different state prior to 
> processing FRAME 2.
>     - In case 2, the decoder has its state reset to the default state.
>     - In case 1, the decoder memory state has been modified by the 
> processing of FRAME 1.
>
> If I got it wrong, please provide more details.
> L.
>
>
> On Wed, Dec 21, 2011 at 6:03 PM, Hermann Weber <hermie.weber at gmx.de 
> <mailto:hermie.weber at gmx.de>> wrote:
>
>     Sorry, it seems I have only replied to Lakhdar, not to the newsgroup.
>     Below is my reply to Lakhdar, and I would like to make it more clear
>     now, using some pseudo values for simplicity:
>
>     I read bytes 1 to 124 from my encoded spx file.
>     I decode themt and get the values:
>     ---Frame 1----
>     -293
>     -8234
>     2134
>     17
>     ---Frame 2----
>     -9323
>     -732
>     189
>     2329
>
>     Both frames are just perfect as I need them.
>
>     But now when I read bytes 63 to 125 from my encoded spx and decode
>     it, I
>     get the values
>     732
>     9273
>     -799
>     -723
>
>     This is weird because I expected to get the encoded values
>     -9323
>     -732
>     189
>     2329
>
>     I hope I could explain it well.
>     Am I missing something here?
>
>     Thank you very much.
>     Hermann
>
>
>
>     Hi Lakhdar,
>
>     I am not sure if I do anything wrong with the dec state memory.
>     Could you perhaps take a look at what I am (perhaps wrongly) doing?
>
>     int __stdcall SpxDecode(unsigned char* DinBuf, float* DoutBuf,
>     unsigned
>     int Dinlen)
>     {
>
>         //char *testFile;
>         //FILE *ftest;
>         //testFile = "test2";
>         //ftest = fopen(testFile, "wb");
>         //fwrite(inBuf,1,inlen,ftest);
>         //fwrite(outBuf, 1, outBufpos, ftest);
>         //fclose(ftest);
>
>         //take every 62 bytes
>         //decode, then write to outbuf
>
>         char cbits[MAX_FRAME_BYTES];
>         float output[MAX_FRAME_SIZE];
>         short out[MAX_FRAME_SIZE];
>         unsigned int i,j;
>
>         unsigned char* tout; tout = (unsigned char*)&out;
>
>         for (i=0; i<(Dinlen/62); i++)
>             {
>             for (j=0;j<62;j++)    cbits[j] = DinBuf[(i*62)+j];
>
>             speex_bits_read_from(&bits, cbits, 62);
>             speex_decode(decstate, &bits, output);
>
>             for (j=0;j<160;j++) DoutBuf[j+(i*160)] = output[j];
>             }
>
>         return 43;
>     }
>
>     Btw, just if you wondering what I am doing... I am feeding (62 *
>     NumberOfFramesThatIWantToRead) bytes from VB6 and return it as a float
>     array.
>
>     Greetings,
>     Hermann
>
>     Am 21.12.2011 15:26, schrieb Lakhdar Bourokba:
>     > Hi Hermann,
>     > Could it be your decoder state memory that is not longer sync ?
>     > When a decoder processes a frame, it uses that frame and its
memory
>     state (affected by the previous frames) to generate the output
>     samples.
>     If you skip the previous frames, the decoder state memory will not be
>     the same (as when you do full decoding) which causes the output to be
>     different.
>     > I hope this helps.
>     > L.
>     >
>     >
>     > On Wed, Dec 21, 2011 at 5:17 AM, Hermann Weber
>     <hermie.weber at gmx.de <mailto:hermie.weber at gmx.de>>
>     wrote:
>     >
>     >     Hello!
>     >
>     >     I am still using version 1.0.4.
>     >
>     >     When I have encoded a larger file and then only want to decode
>     only one
>     >     frame from it (for example the 2nd frame), is there anything
>     special
>     >     that I have to keep in mind?
>     >
>     >     I thought that I could simply read the bytes from the encoded
>     file for
>     >     example from the start position of the 2nd frame and then
decode
>     it. I
>     >     did that, and I can play it play, but the values in the
>     decoded 2nd
>     >     frame are different than they are when I decode the entire
file
>     and then
>     >     look at the values for the 2nd frame.
>     >     I hope I could explain it well.
>     >
>     >     I noticed that there is a pattern in the beginning of the
>     encoded
>     file,
>     >     and I am not sure if I have to take it into account somehow.
>     >
>     >     It would be nice if someone could tell me if something comes
to
>     his/her
>     >     mind that I might be doing wrong.
>     >
>     >     Thank you very much.
>     >
>     >     Hermann
>     >     _______________________________________________
>     >     Speex-dev mailing list
>     > Speex-dev at xiph.org <mailto:Speex-dev at xiph.org>
>     > http://lists.xiph.org/mailman/listinfo/speex-dev
>     >
>     >
>
>
>     Hello!
>
>     I am still using version 1.0.4.
>
>     When I have encoded a larger file and then only want to decode
>     only one
>     frame from it (for example the 2nd frame), is there anything special
>     that I have to keep in mind?
>
>     I thought that I could simply read the bytes from the encoded file for
>     example from the start position of the 2nd frame and then decode it. I
>     did that, and I can play it play, but the values in the decoded 2nd
>     frame are different than they are when I decode the entire file
>     and then
>     look at the values for the 2nd frame.
>     I hope I could explain it well.
>
>     I noticed that there is a pattern in the beginning of the encoded
>     file,
>     and I am not sure if I have to take it into account somehow.
>
>     It would be nice if someone could tell me if something comes to
>     his/her
>     mind that I might be doing wrong.
>
>     Thank you very much.
>
>     Hermann
>     _______________________________________________
>     Speex-dev mailing list
>     Speex-dev at xiph.org <mailto:Speex-dev at xiph.org>
>     http://lists.xiph.org/mailman/listinfo/speex-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/speex-dev/attachments/20111222/679ba232/attachment-0001.htm

To make it complete, here is the code that I am using to encode a large 
file:

int __stdcall SpxEncode(unsigned char* inBuf, unsigned char* outBuf, 
unsigned int inlen)
{
     //char *testFile;
     //FILE *ftest;
     //testFile = "test";
     //ftest = fopen(testFile, "wb");
     //fwrite(inBuf,1,inlen,ftest);

     //take every 320 bytes
     //copy every short to float array, and make sure to put null in 
unused area

     float input[MAX_FRAME_SIZE];
     char cbits[MAX_FRAME_BYTES];
     unsigned int inBufpos = 0, j, outBufpos = 0;
     short *sinBuf;

     sinBuf = (short*) inBuf;

     do
     {
         j = 0;
         do
             {
             //copy to float array
             input[j] = sinBuf[inBufpos];
             inBufpos++; j++;
             }
         while ((inBufpos % 160 != 0) && (inBufpos != (inlen/2)));

         //fill the rest with null
         if (inBufpos == (inlen/2))
             for (j=(inBufpos%160); j<160; j++)    input[j] = 0;

         speex_bits_reset(&bits);
         speex_encode(encstate, input, &bits);

         /*Copy the bits to an array of char that can be written*/
         speex_bits_write(&bits, cbits, MAX_FRAME_BYTES);
         for (j=0; j<62; j++)
             {
             outBuf[outBufpos] = cbits[j];
             outBufpos++;
             }

     }
     while (inBufpos != (inlen/2));

     //fwrite(outBuf, 1, outBufpos, ftest);
     //fclose(ftest);
     return 42;
}