opus - Jul 2015 - mjr to opus audio conversion - corrupted results

Hi,

In our project we use janus-gateway (http://janus.conf.meetecho.com/) as a
webRTC gateway and also as a stream recorder. We are on the tests stage of
our project, and after very long development time we have ecountered a bug
that is a blocker for whole project. After real tests of recording streams
using janus we realized that audio and video are out of sync in recordings,
despite of fact, that live they are perfectly in sync.

Janus records two *.mjr files for each recording - one for video, and one
for audio. Janus also provides own converter to convert this files to
*.webm for video and *.opus for audio. After that, we use ffmpeg to merge
this files. By recording stopwatch view on the stream we concluded, that
video has correct length and speed, so most probably this is something
wrong with audio recording or converting.

Janus uses libogg for converting *.mjr audio to *.opus. You can check how
it's done in this file
https://github.com/meetecho/janus-gateway/blob/master/postprocessing/pp-opus.c

There are two kinds of problems regarding converted audio:

In in some cases media lengths differ much (ie 30s difference between audio
and video duration in 30 min recording) and can't be synchronized,
stretching audio does not helps.

In other cases audio is exactly 1s shorter than video, but shifting media
by 1s not always fixes delay - sometimes it's 0.5s, sometimes it's 0.75s
-
you have to do some attempts, so it can't be automated, what is also a big
problem for us.


Is this because of improper use of api, or maybe it is some bug? How could
we possibly overcome this problems?

For more details you can read our whole conversation with janus author here:
https://groups.google.com/forum/#!topic/meetecho-janus/lMCtBm7VbMY


Regards,

Mateusz Ko?lacz
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/opus/attachments/20150708/b4e384e7/attachment.htm

Hi Mateusz,

The Ogg Opus container does not have explicit per-packet timestamps,
and so doesn't allow gaps in the audio.  PLC packets may be generated
to replace any packets that were lost or corrupted, as described in
section 4.1 of the specification:
https://tools.ietf.org/html/draft-ietf-codec-oggopus-08#section-4.1
These packets usually consist of just a single TOC byte.  It looks
like the tool you are using does not do this, and does not replace
gaps with anything.  Unfortunately FFmpeg has a similar issue:
https://trac.ffmpeg.org/ticket/3391

 - Mark

Hi Mark,

Well, so do you think that any of this libraries probably won't provide
correct audio conversion? Do you know any other possible solutions for
described problems, ommiting trying to fix on my own any of this libraries
or waiting for fix? Will you mind if I quote you in disscussion I
mentioned? Sorry for asking you so many questions at once.


Regards,

Mateusz

2015-07-08 17:42 GMT+02:00 Mark Harris <mark.hsj at gmail.com>:
> Hi Mateusz,
>
> The Ogg Opus container does not have explicit per-packet timestamps,
> and so doesn't allow gaps in the audio.  PLC packets may be generated
> to replace any packets that were lost or corrupted, as described in
> section 4.1 of the specification:
> https://tools.ietf.org/html/draft-ietf-codec-oggopus-08#section-4.1
> These packets usually consist of just a single TOC byte.  It looks
> like the tool you are using does not do this, and does not replace
> gaps with anything.  Unfortunately FFmpeg has a similar issue:
> https://trac.ffmpeg.org/ticket/3391
>
>  - Mark
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.xiph.org/pipermail/opus/attachments/20150708/1846a92e/attachment.htm