Juan Gerardo Ruelas Jr
2020-Dec-03 17:56 UTC
[opus] How to get started with opus and C++/JavaScript/WebAsm
Sorry, forgot to cc the opus mailing list... What is the difference between Ogg-encapsulated Opus packets vs Raw packets? How does one generate one or the other programmatically? - Juan On Thu, Dec 3, 2020 at 9:30 AM Juan Gerardo Ruelas Jr <jruel006 at ucr.edu> wrote:> What is the difference between Ogg-encapsulated Opus packets vs Raw > packets? > > How does one generate one or the other programmatically? > > - Juan > > On Thu, Dec 3, 2020 at 2:10 AM Emily Bowman <silverbacknet at gmail.com> > wrote: > >> Small correction, I just realized while re-reading this that where I said >> raw Opus packets, I should have said Ogg-encapsulated Opus packets, which >> is at least a much simpler format than MP4. Raw packets are useful in some >> contexts but more awkward to work with. >> >> - Em >> >> On Thu, Dec 3, 2020 at 12:15 AM Emily Bowman <silverbacknet at gmail.com> >> wrote: >> >>> Since everything goes back to the C library anyway (or a variant of it, >>> if you use, say, emscripten), you're free to use any language you want to >>> interface with the API. Third-parties have created wrappers for a number of >>> other languages, like Python, since C is a simple and universal binding. >>> >>> While most web streams use chunking via DASH these days, which involves >>> encapsulating packets in ISO-BMFF (MP4), you're perfectly free to just send >>> raw Opus packets sliced up and the decoder will handle them gracefully. >>> Even if packets are lost and you have no error correction, the decoding >>> will resynchronize within one packet, and sound a lot less ugly than the >>> chirps you get with many other codecs. That was a specific design goal of >>> the format. The format on disk is exactly identical to the streamed format, >>> so just concatenating the received packets is a valid and simple way to >>> save to disk. >>> >>> For a basic example of using the API from C/C++, check out >>> https://opus-codec.org/docs/opus_api-1.3.1/group__opus__encoder.html . >>> There's a bit of boilerplate to initialize it, but not too much, and from >>> there you can broadcast the encoded packets however you like. DASH is much >>> more complex, though if you have a library for your platforms of choice >>> it'll either support Opus natively or be fairly simple to retrofit in. >>> >>> -Em >>> >>> On Wed, Dec 2, 2020 at 5:36 PM Juan Gerardo Ruelas Jr <jruel006 at ucr.edu> >>> wrote: >>> >>>> I am interested in using the opus codec to stream audio files to a >>>> mobile app. >>>> >>>> I have a couple of questions about how to get started? >>>> >>>> 1. How do I generate opus files programmatically with C++? What >>>> inputs does it need? Is there a special file format because all the >>>> examples I see only have .wav files as inputs. >>>> 2. Does opus support chunking? For example, breaking a 24 hour long >>>> audio into a series of sequential smaller files (say 5 seconds long) to >>>> stream them down to a user? >>>> 3. Does opus support stitching together opus files? Say, after a >>>> user receives the 5 second long stream, can I programmatically stitch the >>>> files together as they come so as to save the entire 24 hour long audio >>>> file for offline use at a later time? >>>> 4. How do you stream these files programmatically? What is the >>>> general approach. Can you do it with node.js? C++? Python? What is most >>>> efficient? >>>> >>>> p.s. (when I say audio files I generally mean just voice audio like >>>> in audio books) >>>> >>>> >>>> Thank you for your help, I am honestly confused as to how to get >>>> started. >>>> >>>> - Juan Gerardo Ruelas Jr. >>>> _______________________________________________ >>>> opus mailing list >>>> opus at xiph.org >>>> http://lists.xiph.org/mailman/listinfo/opus >>>> >>>-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/opus/attachments/20201203/f8f2c899/attachment.html>
Andreas Stöckel
2020-Dec-03 18:20 UTC
[opus] How to get started with opus and C++/JavaScript/WebAsm
Hi Juan, raw Opus packets can be thought of the minimal unit processed by the Opus encoder/decoder. They do contain all side-channel information necessary to decode a chunk of audio and potentially error correction data, but nothing more. Encapsulating Opus packets in an Ogg stream is simply a standardised way for adding more metadata to the stream, such as an arbitrary metadata header and timestamps for each packet. Also, Ogg can be used to multiplex multiple streams. Regarding your previous questions, I had similar questions a while back [1]. I wanted to (independently) pre-encode small chunks of audio data and splice them together in the browser for playback. This turned out to be a little bit more complicated. Long story short, for proper gap-less concatenation of *independently* encoded Opus chunks you need to add a lead-in/lead-out region to each chunk and interpolate between the chunk boundaries. See my C++/JS code here: https://github.com/astoeckel/opus_gapless This code has no dependencies except for libopus and ffmpeg being installed as an executable. It also comes with its own Ogg muxer. I'm using WebAudio for decoding the opus packets on the client-side. Note that using DASH as suggested by Emily is the correct solution if you can pre-encode the entire stream up-front---which wasn't an option in my case, since I generate the Opus chunks on-demand; i.e., the user requests the chunk for one hour thirty minutes into the stream, and I can neither pre-encode the entire stream, nor live with the latency of encoding the entire file first once the request arrives. I hope this helps! All the best, Andreas [1] http://lists.xiph.org/pipermail/opus/2017-November/004031.html On 03.12.20 12:56, Juan Gerardo Ruelas Jr wrote:> Sorry, forgot to cc the opus mailing list... > > What is the difference between Ogg-encapsulated Opus packets vs > Raw packets? > > > > > > How does one generate one or the other programmatically? > > > - Juan > > On Thu, Dec 3, 2020 at 9:30 AM Juan Gerardo Ruelas Jr > <jruel006 at ucr.edu <mailto:jruel006 at ucr.edu>> wrote: > > What is the difference between Ogg-encapsulated Opus packets vs > Raw packets? > > How does one generate one or the other programmatically? > > - Juan > > On Thu, Dec 3, 2020 at 2:10 AM Emily Bowman > <silverbacknet at gmail.com <mailto:silverbacknet at gmail.com>> wrote: > > Small correction, I just realized while re-reading this that > where I said raw Opus packets, I should have said > Ogg-encapsulated Opus packets, which is at least a much > simpler format than MP4. Raw packets are useful in some > contexts but more awkward to work with. > > - Em > > On Thu, Dec 3, 2020 at 12:15 AM Emily Bowman > <silverbacknet at gmail.com <mailto:silverbacknet at gmail.com>> wrote: > > Since everything goes back to the C library anyway (or a > variant of it, if you use, say, emscripten), you're free > to use any language you want to interface with the API. > Third-parties have created wrappers for a number of other > languages, like Python, since C is a simple and universal > binding. > > While most web streams use chunking via DASH these days, > which involves encapsulating packets in ISO-BMFF (MP4), > you're perfectly free to just send raw Opus packets sliced > up and the decoder will handle them gracefully. Even if > packets are lost and you have no error correction, the > decoding will resynchronize within one packet, and sound a > lot less ugly than the chirps you get with many other > codecs. That was a specific design goal of the format. The > format on disk is exactly identical to the streamed > format, so just concatenating the received packets is a > valid and simple way to save to disk. > > For a basic example of using the API from C/C++, check out > https://opus-codec.org/docs/opus_api-1.3.1/group__opus__encoder.html > <https://opus-codec.org/docs/opus_api-1.3.1/group__opus__encoder.html> > . There's a bit of boilerplate to initialize it, but not > too much, and from there you can broadcast the encoded > packets however you like. DASH is much more complex, > though if you have a library for your platforms of choice > it'll either support Opus natively or be fairly simple to > retrofit in. > > -Em > > On Wed, Dec 2, 2020 at 5:36 PM Juan Gerardo Ruelas Jr > <jruel006 at ucr.edu <mailto:jruel006 at ucr.edu>> wrote: > > I am interested in using the opus codec to stream > audio files to a mobile app. > > I have a couple of questions about how to get started? > > 1. How do I generate opus files programmatically with > C++? What inputs does it need? Is there a special > file format because all the examples I see only > have .wav files as inputs. > 2. Does opus support chunking? For example, breaking > a 24 hour long audio into a series of sequential > smaller files (say 5 seconds long) to stream them > down to a user? > 3. Does opus support stitching together opus files? > Say, after a user receives the 5 second long > stream, can I programmatically stitch the files > together as they come so as to save the entire 24 > hour long audio file for offline use at a later time? > 4. How do you stream these files programmatically? > What is the general approach. Can you do it with > node.js? C++? Python? What is most efficient? > > p.s. (when I say audio files I generally mean just > voice audio like in audio books) > > Thank you for your help, I am honestly confused as to > how to get started. > > - Juan Gerardo Ruelas Jr. > _______________________________________________ > opus mailing list > opus at xiph.org <mailto:opus at xiph.org> > http://lists.xiph.org/mailman/listinfo/opus > <http://lists.xiph.org/mailman/listinfo/opus> > > > _______________________________________________ > opus mailing list > opus at xiph.org > http://lists.xiph.org/mailman/listinfo/opus >