Philip-Dylan Gleonec
2021-Jan-11 16:55 UTC
[opus] Opus FEC/loss concealment on multiple packets ?
Hello, I'm currently working on enabling opus FEC/loss concealment in Jami (GNU VoIP application) through ffmpeg. We have integrated this patch (https://patchwork.ffmpeg.org/project/ffmpeg/patch/E1fIUGT-0003tH-Ql at pannekake.samfundet.no/) to enable FEC on the decoder. I traced the codec execution while debugging and noticed that, when multiple consecutive packets are lost, the 'opus_multistream_decode' function is called with a frame size equal to the total number of samples lost, instead of just the size of the last packet lost. What happens in this case ? From my understanding, FEC can only be used when a single packet is missing. However, is it possible to do loss concealment on multiple consecutive packets this way ? For the record, here is below a commented trace of ffmpeg using opus: [sdp @ 0x7f72a4001000] max delay reached. need to consume packet [sdp @ 0x7f72a4001000] RTP: missed 1 packets [libopus @ 0x7f72a4006640] FEC: calculating audio mismatch : 251472 | 252432 [libopus @ 0x7f72a4006640] FEC: calculating nb_lost_samples : 960 [libopus @ 0x7f72a4006640] FEC: calculated lost samples : 960 [libopus @ 0x7f72a4006640] FEC: lost 960 samples // Here, opus_multistream_decode is called with frame_size = 960, as expected [libopus @ 0x7f72a4006640] FEC: Recovered 960 samples [libopus @ 0x7f72a4006640] FEC: no mismatch detected [libopus @ 0x7f72a4006640] FEC: no mismatch detected [libopus @ 0x7f72a4006640] FEC: no mismatch detected [libopus @ 0x7f72a4006640] FEC: no mismatch detected [libopus @ 0x7f72a4006640] FEC: no mismatch detected [libopus @ 0x7f72a4006640] FEC: no mismatch detected [sdp @ 0x7f72a4001000] max delay reached. need to consume packet [sdp @ 0x7f72a4001000] RTP: missed 2 packets [libopus @ 0x7f72a4006640] FEC: calculating audio mismatch : 259152 | 261072 [libopus @ 0x7f72a4006640] FEC: calculating nb_lost_samples : 1920 [libopus @ 0x7f72a4006640] FEC: calculated lost samples : 1920 [libopus @ 0x7f72a4006640] FEC: lost 1920 samples // Here, opus_multistream_decode is called with frame_size = 1920. This seems functional and seems to work, but I'm not sure how/why [libopus @ 0x7f72a4006640] FEC: Recovered 1920 samples <- Is it possible to recover 1920 samples through loss concealment when a packet is 960 samples ? [libopus @ 0x7f72a4006640] FEC: no mismatch detected [libopus @ 0x7f72a4006640] FEC: no mismatch detected [libopus @ 0x7f72a4006640] FEC: no mismatch detected [libopus @ 0x7f72a4006640] FEC: no mismatch detected Thanks a lot for your time and assistance (and for the codec, of course). Best regards, Philip-Dylan Gleonec -- Logo Savoir-faire Linux inc. Philip-Dylan Gleonec Consultant en logiciel libre / Ingénieur systèmes embarqués | Rennes, Fr Bureau (+33) 9 72 46 89 80 <tel:+14185257354> Logo Jami sfl-pgleonec new-year Site web <https://www.savoirfairelinux.com> | Blog <https://blog.savoirfairelinux.com/fr-ca/> | Jami <https://jami.net> Suivez-nous Facebook <https://www.facebook.com/savoirfairelinux/> LinkedIn <https://www.linkedin.com/company/savoir-faire-linux> Twitter <https://twitter.com/sflinux> gitLab <https://gitlab.savoirfairelinux.com> Messages de confidentialité : Ce courriel (de même que les fichiers joints) est strictement réservé à l'usage de la personne ou de l'entité à qui il est adressé et peut contenir de l'information privilégiée et confidentielle. Toute divulgation, distribution ou copie de ce courriel est strictement prohibée. Si vous avez reçu ce courriel par erreur, veuillez nous en aviser sur-le-champ, détruire toutes les copies et le supprimer de votre système informatique. -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/opus/attachments/20210111/d7306a9e/attachment.html>