Speex dev - Oct 2010 - how to play multiple speex audio streams simultaneously

Hello,

In a way similar to speexclient voip example, I want to have a voip
application where 3 or more users can interact. This creates the following
problems "1. How to playback 2 or more audio streams such that they appear
overlayed (parallel)". The audio streams are speex encoded.

I have few approaches in mind.

1. Create multiple decoder states and multiple ALSA audio playback device.
But if I do this in the same thread, voice will appear sequential (slightly
displaced in time) rather than overlayed. This issue will get more deep as
the number of users increase. Moreover creating multiple OS threads will
also not solve the issue due to latency issues.

2. Use multiple channels in decoder but not sure how this can be done.

3. Using intermediate server to merge/overlay multiple speex audio streams
encoded at same sampling rate. I can't find an appropriate
function/algorithm in speex code to do so.

Please suggest an appropriate solution.

Regards,
Vishal
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/speex-dev/attachments/20101014/174cdc40/attachment.htm

Another approach that I have come across is to decode all speex audio
streams and add their 16 bit signed PCM values clipping values above 2^15-1
and below -2^15. Is this a reliable appproach?

On Thu, Oct 14, 2010 at 3:54 PM, Vishal Jain <cse.vishal at gmail.com>
wrote:
> Hello,
>
> In a way similar to speexclient voip example, I want to have a voip
> application where 3 or more users can interact. This creates the following
> problems "1. How to playback 2 or more audio streams such that they
appear
> overlayed (parallel)". The audio streams are speex encoded.
>
> I have few approaches in mind.
>
> 1. Create multiple decoder states and multiple ALSA audio playback device.
> But if I do this in the same thread, voice will appear sequential (slightly
> displaced in time) rather than overlayed. This issue will get more deep as
> the number of users increase. Moreover creating multiple OS threads will
> also not solve the issue due to latency issues.
>
> 2. Use multiple channels in decoder but not sure how this can be done.
>
> 3. Using intermediate server to merge/overlay multiple speex audio streams
> encoded at same sampling rate. I can't find an appropriate
> function/algorithm in speex code to do so.
>
> Please suggest an appropriate solution.
>
> Regards,
> Vishal
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/speex-dev/attachments/20101014/7faf5b35/attachment.htm

That's the approach we're using, and yes, it's reliable.  I believe
it's the
standard way of doing a mix with PCM audio data.

You can do this mixing either on an intermediate server, or on the client,
and each have their benefits.  Doing it on the client requires less
server-side horsepower, but it also means more bandwidth requirements (since
you have n number of inbound streams instead of one, along with n outbound
streams).  Doing it on an intermediate server means less bandwidth is
required, but it requires more processing (since you have to first decode
the packets, then mix them, then re-encode them).  It also means that it's
somewhat more difficult on the client to control the volume of each stream
independently, since once they're mixed they can't easily be separated
out
again.

Ken Smith
Cell: 425-443-2359
Email: ken at alanta.com
Blog: http://blog.wouldbetheologian.com/


On Thu, Oct 14, 2010 at 7:07 AM, Vishal Jain <cse.vishal at gmail.com>
wrote:
> Another approach that I have come across is to decode all speex audio
> streams and add their 16 bit signed PCM values clipping values above 2^15-1
> and below -2^15. Is this a reliable appproach?
>
>
> On Thu, Oct 14, 2010 at 3:54 PM, Vishal Jain <cse.vishal at
gmail.com> wrote:
>
>> Hello,
>>
>> In a way similar to speexclient voip example, I want to have a voip
>> application where 3 or more users can interact. This creates the
following
>> problems "1. How to playback 2 or more audio streams such that
they appear
>> overlayed (parallel)". The audio streams are speex encoded.
>>
>> I have few approaches in mind.
>>
>> 1. Create multiple decoder states and multiple ALSA audio playback
device.
>> But if I do this in the same thread, voice will appear sequential
(slightly
>> displaced in time) rather than overlayed. This issue will get more deep
as
>> the number of users increase. Moreover creating multiple OS threads
will
>> also not solve the issue due to latency issues.
>>
>> 2. Use multiple channels in decoder but not sure how this can be done.
>>
>> 3. Using intermediate server to merge/overlay multiple speex audio
streams
>> encoded at same sampling rate. I can't find an appropriate
>> function/algorithm in speex code to do so.
>>
>> Please suggest an appropriate solution.
>>
>> Regards,
>> Vishal
>>
>
>
> _______________________________________________
> Speex-dev mailing list
> Speex-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/speex-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/speex-dev/attachments/20101014/216c4f68/attachment.htm