opus - Apr 2015 - Opus multi-stream/surround: Audio corruption on decoded content

Hello Everyone,

I am using the opus 1.1 multistream APIs to encode a 5.1 surround stream on the
server, stream it to client, decode it and capture the pcm data. I noticed that
there was severe corruption/attenuation on one of the channels(specifically
Back/Rear Right). This would appear to be the last channel in the stream. I am
attaching an image of the PCM dumps from the original and the one decoded on the
client which shows the channel corruption. If required I can also provide the
source and destination wav files. The track I am trying to play is purely speech
data, however, it has been formatted as Hi-Definition content (16 bit LPCM, 48
KHz). I'd like to add that I have also tried a simple loopback test; where I
read from 'the' source file encode it, immediately decode the encoded
buffer and write to a output file and I am seeing the same kind of corruption.
Since my actual use-case will involve mixed media content, i.e. typically voice
and music, I was a bit concerned about this.

If the file purely has music(and no speech), I do notice some attenuation,
however, it does not suffer from any distortions.

Following is the code I am using for initializing the opus multi-stream encoder:
-

<code>

<defines and vars>

#define CHANNELS           6
#define OPUS_SURROUND_MAPPING_FAMILY 1
#define SAMPLE_RATE 48000

typedef union t_SurroundInfo
{
    unsigned char surroundInfo[1];
    struct
    {
        unsigned char    channels;
        unsigned char    streams;
        unsigned char    coupled_streams;
        unsigned char    channel_mapping[MAX_SURROUND_CHANNELS];
    }s;
}SurroundInfo;

</defines and vars>

<encoder>

        int streams = 0; int coupled_streams = 0;
        m_SurroundInfo.s.channels = CHANNELS; // set to 6
        m_MSEnc = opus_multistream_surround_encoder_create(SAMPLE_RATE, //
SAMPLE_RATE = 48000
            m_SurroundInfo.s.channels,
            OPUS_SURROUND_MAPPING_FAMILY,
            &streams,
            &coupled_streams,
            m_SurroundInfo.s.channel_mapping,
            OPUS_APPLICATION_RESTRICTED_LOWDELAY,
            &err
            );
        if (err != OPUS_OK)
            return;

        m_SurroundInfo.s.streams = (unsigned char)streams;
        m_SurroundInfo.s.coupled_streams = (unsigned char)coupled_streams;

        opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_BITRATE(bitRate));
        opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_BANDWIDTH(OPUS_AUTO));
        opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_VBR(0));
        opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_VBR_CONSTRAINT(0));
        opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_COMPLEXITY(10));
        opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_INBAND_FEC(0));
        opus_multistream_encoder_ctl(m_MSEnc,
OPUS_SET_FORCE_CHANNELS(OPUS_AUTO));
        opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_DTX(0));
        opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_PACKET_LOSS_PERC(0));
        opus_multistream_encoder_ctl(m_MSEnc,
OPUS_SET_FORCE_MODE(MODE_CELT_ONLY));

</encoder>

<decoder>

    int streams = (int)m_SurroundInfo.s.streams; int coupled_streams =
(int)m_SurroundInfo.s.coupled_streams;
    m_MSDec = opus_multistream_decoder_create(SAMPLE_RATE,
        m_SurroundInfo.s.channels,
        streams,
        coupled_streams,
        m_SurroundInfo.s.channel_mapping,
        &err
        );

</decoder>

</code>


I'd appreciate it, if someone can shed some light on why the observed
phenomenon is occurring. Could it just be that, I am playing a content that only
has speech and actually forcing opus mode to CELT?

Thanks in advance.

Warm regards,

Mukund.

-----------------------------------------------------------------------------------
This email message is for the sole use of the intended recipient(s) and may
contain
confidential information.  Any unauthorized review, use, disclosure or
distribution
is prohibited.  If you are not the intended recipient, please contact the sender
by
reply email and destroy all copies of the original message.
-----------------------------------------------------------------------------------
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/opus/attachments/20150402/caf1e4fd/attachment-0001.htm

The Opus multistream API uses the Vorbis channel ordering. The order is:
L, R, C, RL, RR, LFE
So the last channel is the LFE, which Opus encodes with a much narrower
bandwidth and a much lower bitrate. I suspect you are just feeding the
data in the wav order, which is different. In wav 5.1, the last channel
is RR, which is probably the channel that got attenuated for you.

	Jean-Marc

On 01/04/15 09:12 PM, Mukund Raman wrote:
> Hello Everyone,
> 
>  
> 
> I am using the opus 1.1 multistream APIs to encode a 5.1 surround stream
> on the server, stream it to client, decode it and capture the pcm data.
> I noticed that there was severe corruption/attenuation on one of the
> channels(specifically Back/Rear Right). This would appear to be the last
> channel in the stream. I am attaching an image of the PCM dumps from the
> original and the one decoded on the client which shows the channel
> corruption. If required I can also provide the source and destination
> wav files. The track I am trying to play is purely speech data, however,
> it has been formatted as Hi-Definition content (16 bit LPCM, 48 KHz).
> I?d like to add that I have also tried a simple loopback test; where I
> read from ?the? source file encode it, immediately decode the encoded
> buffer and write to a output file and I am seeing the same kind of
> corruption. Since my actual use-case will involve mixed media content,
> i.e. typically voice and music, I was a bit concerned about this.
> 
>  
> 
> If the file purely has music(and no speech), I do notice some
> attenuation, however, it does not suffer from any distortions.
> 
>  
> 
> Following is the code I am using for initializing the opus multi-stream
> encoder: -
> 
>  
> 
> <code>
> 
>  
> 
> <defines and vars>
> 
>  
> 
> #defineCHANNELS           6
> 
> #defineOPUS_SURROUND_MAPPING_FAMILY1
> 
> #defineSAMPLE_RATE48000
> 
>  
> 
> typedefuniont_SurroundInfo
> 
> {
> 
>     unsignedcharsurroundInfo[1];
> 
>     struct
> 
>     {
> 
>         unsignedchar    channels;
> 
>         unsignedchar    streams;
> 
>         unsignedchar    coupled_streams;
> 
>         unsignedchar    channel_mapping[MAX_SURROUND_CHANNELS];
> 
>     }s;
> 
> }SurroundInfo;
> 
>  
> 
> </defines and vars>
> 
>  
> 
> <encoder>
> 
>  
> 
>         intstreams = 0; intcoupled_streams = 0;
> 
>         m_SurroundInfo.s.channels = CHANNELS; // set to 6
> 
>         m_MSEnc = opus_multistream_surround_encoder_create(SAMPLE_RATE,
> // SAMPLE_RATE = 48000
> 
>             m_SurroundInfo.s.channels,
> 
>             OPUS_SURROUND_MAPPING_FAMILY,
> 
>             &streams,
> 
>             &coupled_streams,
> 
>             m_SurroundInfo.s.channel_mapping,
> 
>             OPUS_APPLICATION_RESTRICTED_LOWDELAY,
> 
>             &err
> 
>             );
> 
>         if(err != OPUS_OK)
> 
>             return;
> 
>  
> 
>         m_SurroundInfo.s.streams = (unsignedchar)streams;
> 
>         m_SurroundInfo.s.coupled_streams = (unsignedchar)coupled_streams;
> 
>  
> 
>         opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_BITRATE(bitRate));
> 
>         opus_multistream_encoder_ctl(m_MSEnc,
> OPUS_SET_BANDWIDTH(OPUS_AUTO));
> 
>         opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_VBR(0));
> 
>         opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_VBR_CONSTRAINT(0));
> 
>         opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_COMPLEXITY(10));
> 
>         opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_INBAND_FEC(0));
> 
>         opus_multistream_encoder_ctl(m_MSEnc,
> OPUS_SET_FORCE_CHANNELS(OPUS_AUTO));
> 
>         opus_multistream_encoder_ctl(m_MSEnc, OPUS_SET_DTX(0));
> 
>         opus_multistream_encoder_ctl(m_MSEnc,
OPUS_SET_PACKET_LOSS_PERC(0));
> 
>         opus_multistream_encoder_ctl(m_MSEnc,
> OPUS_SET_FORCE_MODE(MODE_CELT_ONLY));
> 
>  
> 
> </encoder>
> 
>  
> 
> <decoder>
> 
>  
> 
>     intstreams = (int)m_SurroundInfo.s.streams; intcoupled_streams >
(int)m_SurroundInfo.s.coupled_streams;
> 
>     m_MSDec = opus_multistream_decoder_create(SAMPLE_RATE,
> 
>         m_SurroundInfo.s.channels,
> 
>         streams,
> 
>         coupled_streams,
> 
>         m_SurroundInfo.s.channel_mapping,
> 
>         &err
> 
>         );
> 
>  
> 
> </decoder>
> 
>  
> 
> </code>
> 
>  
> 
>  
> 
> I?d appreciate it, if someone can shed some light on why the observed
> phenomenon is occurring. Could it just be that, I am playing a content
> that only has speech and actually forcing opus mode to CELT?
> 
>  
> 
> Thanks in advance.
> 
>  
> 
> Warm regards,
> 
>  
> 
> Mukund.
> 
> ------------------------------------------------------------------------
> This email message is for the sole use of the intended recipient(s) and
> may contain confidential information.  Any unauthorized review, use,
> disclosure or distribution is prohibited.  If you are not the intended
> recipient, please contact the sender by reply email and destroy all
> copies of the original message.
> ------------------------------------------------------------------------
> 
> 
> _______________________________________________
> opus mailing list
> opus at xiph.org
> http://lists.xiph.org/mailman/listinfo/opus
>