opus - Jun 2013 - conformance testing of custom implementation

Hi, 

I'm working on porting libopus and I'd like to get information how to
test Opus Encoder properly.

There is information in RFC6716, that Opus Decoder's test files
(http://opus-codec.org/testvectors/opus_testvectors.tar.gz)  were created
specifically to exercise all aspects of the decoder. Using these files, a custom
implementation of libopus can be verified to be sure the Opus decoder operates
properly.

Is there any test files created specifically to exercise all aspects of the
encoder? If not, can you recommend some steps to verify a custom implementation
of libopus in encoding mode ?

Thanks,
Sergey

On 06/12/2013 12:21 PM, SG wrote:
> Is there any test files created specifically to exercise all aspects
> of the encoder? If not, can you recommend some steps to verify a
> custom implementation of libopus in encoding mode ?
There is no test for the encoder because unlike the decoder, it is not
normatively specified and it keeps improving over time. The best way to
check your encoder for non-bit-exact numerical issues is to use a tool
like PQevalAudio:
http://www-mmsp.ece.mcgill.ca/documents/Downloads/PQevalAudio/

Cheers,

	Jean-Marc

Thanks for the answer. I'll take a look at this tool and verify my
implementation.

But I'd like to clarify the moment. If the following is applied for a test
vector

Reference Float Point Encoder --> Reference Float Point Decoder --> Float
Point Output
Reference Fixed Point Encoder --> Reference Float Point Decoder --> Fixed
Point Output

And then, if Float Point Output and Fixed Point Output are compared using
opus_compare.c tool, the result is 'Test vector FAILS'.

Is it as supposed to be ? Or maybe there is some inaccuracy in reference fixed
point implementation (libopus 1.0.2) ?

Thanks,
Sergey


15.06.2013, 09:26, "Jean-Marc Valin" <jmvalin at jmvalin.ca>:
> ?On 06/12/2013 12:21 PM, SG wrote:
>> ??Is there any test files created specifically to exercise all aspects
>> ??of the encoder? If not, can you recommend some steps to verify a
>> ??custom implementation of libopus in encoding mode ?
> ?There is no test for the encoder because unlike the decoder, it is not
> ?normatively specified and it keeps improving over time. The best way to
> ?check your encoder for non-bit-exact numerical issues is to use a tool
> ?like PQevalAudio:
> ?http://www-mmsp.ece.mcgill.ca/documents/Downloads/PQevalAudio/
>
> ?Cheers,
>
> ?????????Jean-Marc
> ?_______________________________________________
> ?opus mailing list
> ?opus at xiph.org
> ?http://lists.xiph.org/mailman/listinfo/opus