flac dev - Jan 2011 - Synchronizing a streaming client to the server Was: Idea to possibly improve flac?

I just thought of something: Given the maximum supported network  
packet size, and the minimum number of channels (probably stereo) for  
a FLAC broadcast stream, it should be possible to calculate the  
absolute longest time that a single network packet could span.  Once  
you know that time, you could simply double it, and then make sure  
the streaming client always buffers up at least that much time before  
playback is started.  Voila - instant protection against starvation  
due to silent frames being compressed to ultra-tiny packets with a  
long delay.

Some of the comments here have talked about low latency, but I would  
say that low latency has no place in an internet streaming  
broadcast.  I mean, the listened have no frame of reference for  
latency anyway, so what does it matter if the latency is really high?

Now that I think about it this way, I'd say that FLAC and OggFLAC  
should not have any real problems due to compression of silent  
frames.  Any place there is a problem should be blamed on bad  
streaming client / player code, not on the format itself.

Brian Willoughby
Sound Consulting

The main problem is in the Ogg layer, in my opinion.

Imagine this extreme use-case with __completely made up__ numbers.  This
is a scenario where the server is encoding to FLAC on-the-fly from a raw
PCM input, either from disk or a live stream.

Let's say the FLAC block size is 1024 samples, or 23ms at 44100 Hz.
Let's say each silent block compresses to 1 byte.  Let's also say that
the
Ogg packeting layer wants 4096 bytes before creating a page.  Again -
these numbers are completely made up, but illustrate the point.  In this
example, it would take 95 seconds of digital silence before Ogg decided to
send another page out over the network.

If the input audio on the server is coming from a live-source (such as
simulcast of an FM station) or if the disk I/O is very slow, this can be
extremely problematic.

Ben Allison
Principal Software Engineer
Nullsoft, Inc.
> I just thought of something: Given the maximum supported network
> packet size, and the minimum number of channels (probably stereo) for
> a FLAC broadcast stream, it should be possible to calculate the
> absolute longest time that a single network packet could span.  Once
> you know that time, you could simply double it, and then make sure
> the streaming client always buffers up at least that much time before
> playback is started.  Voila - instant protection against starvation
> due to silent frames being compressed to ultra-tiny packets with a
> long delay.
>
> Some of the comments here have talked about low latency, but I would
> say that low latency has no place in an internet streaming
> broadcast.  I mean, the listened have no frame of reference for
> latency anyway, so what does it matter if the latency is really high?
>
> Now that I think about it this way, I'd say that FLAC and OggFLAC
> should not have any real problems due to compression of silent
> frames.  Any place there is a problem should be blamed on bad
> streaming client / player code, not on the format itself.
>
> Brian Willoughby
> Sound Consulting
>
> _______________________________________________
> Flac-dev mailing list
> Flac-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/flac-dev
>

On Fri, Jan 7, 2011 at 11:25 PM, Brian Willoughby <brianw at
sounds.wa.com> wrote:
> I just thought of something: Given the maximum supported network
> packet size, and the minimum number of channels (probably stereo) for
> a FLAC broadcast stream, it should be possible to calculate the
> absolute longest time that a single network packet could span. ?Once
> you know that time, you could simply double it, and then make sure
> the streaming client always buffers up at least that much time before
> playback is started. ?Voila - instant protection against starvation
> due to silent frames being compressed to ultra-tiny packets with a
> long delay.
>
> Some of the comments here have talked about low latency, but I would
> say that low latency has no place in an internet streaming
> broadcast. ?I mean, the listened have no frame of reference for
> latency anyway, so what does it matter if the latency is really high?
You can say that but I can also stick my fingers in ears and whistle
whilst you do. 2kb of ram is enough for anyone...
> Now that I think about it this way, I'd say that FLAC and OggFLAC
> should not have any real problems due to compression of silent
> frames. ?Any place there is a problem should be blamed on bad
> streaming client / player code, not on the format itself.
Its not in the format or the client / player code. Its in libflac as I
have pointed out.

-David
>
> Brian Willoughby
> Sound Consulting
>
> _______________________________________________
> Flac-dev mailing list
> Flac-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/flac-dev
>

The actual non made up number for 44100 is 23 seconds. :D

4096 samples, 254 packets in an ogg page.

-David

On Fri, Jan 7, 2011 at 11:44 PM, Ben Allison <benski at winamp.com>
wrote:
> The main problem is in the Ogg layer, in my opinion.
>
> Imagine this extreme use-case with __completely made up__ numbers. ?This
> is a scenario where the server is encoding to FLAC on-the-fly from a raw
> PCM input, either from disk or a live stream.
>
> Let's say the FLAC block size is 1024 samples, or 23ms at 44100 Hz.
> Let's say each silent block compresses to 1 byte. ?Let's also say
that the
> Ogg packeting layer wants 4096 bytes before creating a page. ?Again -
> these numbers are completely made up, but illustrate the point. ?In this
> example, it would take 95 seconds of digital silence before Ogg decided to
> send another page out over the network.
>
> If the input audio on the server is coming from a live-source (such as
> simulcast of an FM station) or if the disk I/O is very slow, this can be
> extremely problematic.
>
> Ben Allison
> Principal Software Engineer
> Nullsoft, Inc.
>
>> I just thought of something: Given the maximum supported network
>> packet size, and the minimum number of channels (probably stereo) for
>> a FLAC broadcast stream, it should be possible to calculate the
>> absolute longest time that a single network packet could span. ?Once
>> you know that time, you could simply double it, and then make sure
>> the streaming client always buffers up at least that much time before
>> playback is started. ?Voila - instant protection against starvation
>> due to silent frames being compressed to ultra-tiny packets with a
>> long delay.
>>
>> Some of the comments here have talked about low latency, but I would
>> say that low latency has no place in an internet streaming
>> broadcast. ?I mean, the listened have no frame of reference for
>> latency anyway, so what does it matter if the latency is really high?
>>
>> Now that I think about it this way, I'd say that FLAC and OggFLAC
>> should not have any real problems due to compression of silent
>> frames. ?Any place there is a problem should be blamed on bad
>> streaming client / player code, not on the format itself.
>>
>> Brian Willoughby
>> Sound Consulting
>>
>> _______________________________________________
>> Flac-dev mailing list
>> Flac-dev at xiph.org
>> http://lists.xiph.org/mailman/listinfo/flac-dev
>>
>
> _______________________________________________
> Flac-dev mailing list
> Flac-dev at xiph.org
> http://lists.xiph.org/mailman/listinfo/flac-dev
>

On Jan 7, 2011, at 20:44, Ben Allison wrote:
> The main problem is in the Ogg layer, in my opinion.
I think you are right.

What is the reason that the Ogg layer was chosen for streaming  
broadcast?  My admittedly naive assumption was that Ogg is a file  
format, not a streaming format.

> Imagine this extreme use-case with __completely made up__ numbers.   
> This
> is a scenario where the server is encoding to FLAC on-the-fly from  
> a raw
> PCM input, either from disk or a live stream.
>
> Let's say the FLAC block size is 1024 samples, or 23ms at 44100 Hz.
> Let's say each silent block compresses to 1 byte.  Let's also say  
> that the
> Ogg packeting layer wants 4096 bytes before creating a page.  Again -
> these numbers are completely made up, but illustrate the point.  In  
> this
> example, it would take 95 seconds of digital silence before Ogg  
> decided to
> send another page out over the network.
I realize that these are made-up numbers, and I wouldn't expect a  
great deal of precision, but it would seem worthwhile to at least  
look at realistic estimates.

The minimum FRAME blocking in FLAC would be 12 bytes for stereo, not  
just 1 byte.  That's with 40-bit minimum FRAME_HEADER, 8-bit  
SUBFRAME_HEADER, 2 silent CONSTANT 16-bit audio samples, and 16-bit  
FRAME_FOOTER.

This means the 4096-byte Ogg packeting layer would only represent 7.9  
seconds of latency.  I think it's perfectly reasonable for a client  
tuning in to a lossless broadcast to wait just under 8 seconds for  
pre-buffering of the stream.

Better yet, ditch the Ogg packeting layer entirely, and use raw FLAC  
streaming.  That should mean that any size packet is possible.  TCP/ 
IP should allow shorter packets than the maximum, although it may be  
more efficient for broadcast traffic to have packets closer to the  
normal size for file transfers, which would be more like 3.4  
seconds.  I admit that I don't know for sure what restrictions there  
are on internet broadcast packets, but it can't be much worse.

3.4 seconds is a long time to go without a packet, but buffering of  
6.7 seconds or more should easily take care of that.

> If the input audio on the server is coming from a live-source (such as
> simulcast of an FM station) or if the disk I/O is very slow, this  
> can be
> extremely problematic.
>
> Ben Allison
> Principal Software Engineer
> Nullsoft, Inc.
I don't see why live-source audio would be a problem.  The system  
would simply buffer the required amount of uncompressed audio data  
before calling the FLAC encoder.  The only cost would be latency, and  
all digital broadcast systems that involve codecs involve a serious  
amount of latency.  I can hear a distinct delay between my HDTV set  
tuner and computer USB tuner, since each has a different amount of  
buffering in its pipeline, exacerbated by the digital audio system.

Brian Willoughby
Sound Consulting