Icecast - Mar 2024 - Education - 1, 000s, 100, 000's, Millions of listeners. (What kind of infrastructure)

Hi Wayne

 

Yep, HLS is something we would like to use with Icecast. As you mentioned,
Liquidsoap made a great step in the right direction if it comes to HLS. It seem
that icecast might not get this support soon, as you might read in the
discussion list. I don?t share these arguments, because we have real world use
cases (we have many of them) where we  would like use icecast with HLS support.
But unfortunately we needed to go with another streaming server application
which supports HLS to fulfill our requirements.

 

But coming back to the topic of this thread.

I was looking into using NGINX as a reverse proxy for doing the load balancing
(single point of SSL and centralized monitoring). NGINX is awesome in doing
this. And there are some valid arguments using the approach to put a single
NGINX server (or a two node Nginx Cluster) in front of a multi node icecast
cluster. At that time, 5 years ago,  we decided to go with the ? poor man
solution ? and use the RR DNS approach. We did not had enough time for
investigation, development and testing. Additional, we eliminated another single
point of failure. Sometimes I like way of KISS (keep it simple stupid) ?

 

if you need to serve static files globaly, I would consider the CDN approach. We
use CF for this, but any other CDN provider would do the job. They are kind of
?same same - but different?

Cheers

Tom

 

Von: Icecast <icecast-bounces at xiph.org> Im Auftrag von Wayne Barron
Gesendet: Mittwoch, 20. M?rz 2024 23:06
An: Icecast streaming server user discussions <icecast at xiph.org>
Betreff: Re: [Icecast] Education - 1, 000s, 100, 000's, Millions of
listeners. (What kind of infrastructure)

 

Tom and Frederick.

Thank you both for your input. It is greatly appreciated, not only by me, but I
am sure for many others who find this thread.

 

Tom - using Linux server for icecast.

 

HLS was brought to my attention a while back on the liquidsoap forum.

I have not had a chance to completely look in on it, but do plan on checking it
out.

 

Tom. You have your icecast servers in a round robin configuration.

Are you using NGINX for that?

I have watched several YouTube videos on using it for doing round robin as well
as ssl and other things.

 

I understand that once a scenario of say 10,000 or more, that I will have to
look into either a bigger infrastructure on my side or leasing space on the
cloud somewhere.

 

Now, if I did do the cloud.

Would that be to host the servers or just the files or everything?

 

Wayne 

 

On Wed, Mar 20, 2024, 4:50?PM <thomas.zumbrunnen at gmail.com
<mailto:thomas.zumbrunnen at gmail.com> > wrote:

Dear all

 

My 5 cents (or Rappen in CH) if it comes to serving many clients.

We are running a 4 node cluster since several years ? rock solid and w/o any
issues. This cluster serves many thousands of listeners from all over the world.
Our source transcoder sending the audio streams to each Node.  Hence,
transcoding power is not an issue here. The four Nodes a geographically
dispersed in 3 countries in Europe. In our case each Node is running Debian with
icecast and has 10Gbit connectivity with brilliant worldwide peerings. Good
peering is key, choose your ISP wisely ?.Each icecast servers has the same multi
domain ssl cert. which allows us to deliver to several customers (each customer
a subdomain) the cluster is round robin load balanced by using AWS Route53. This
approach may can be achived also with other DNS Providers like Cloudflare. For
example, if one node need to be taken down for maintenance, Route53 throws the
Node out of the DNS automatically. This will be achived with ?health checks?
This mechanism is pretty fast and responsive. If a client gets disconnected and
tries a reconnect, the RR DNS is passing the client immediately to a working
Node. No issues here as well. It just works.

 

In the beginning we?ve experienced similar issues even though, the bandwidth
capacity of the VM?s was never the root cause. We?ve identified some solutions:

 

1.	Linux TCP stack tuning. Cloudflare has many studies published in their Blogs
about this. But you will find a lot about this tuning also in the interweb
2.	Consider to bake your own kernel, which is tuned for high throughput ? goes
in line with TCP Stack tunning
3.	Tune the Linux open file limits and adjust the init start script for the
icecast server. Example : start icecast with  ulimit -c unlimited and ulimit -n
32768
4.	Consider to use FreeBSD instead of Linux. FreeBSD has the better TCP stack
out of the box.
5.	If all of this is not feasible for you, just add a new Node to the cluster
and level the amount of clients to more Nodes.

 

For the points 1. and 2. I can?t give you a ?out of the box? solution or default
settings. It?s an iteration process: adjusting, trial, load testing, monitoring
and <repeat>

Because the result will need to fit your requirements, therefore every setup
might need a different tuning. And btw. do not try using icecast on Windows
Servers, if you need to serve a lot of clients ?

 

Happy icecasting

tom

 

Von: Icecast <icecast-bounces at xiph.org <mailto:icecast-bounces at
xiph.org> > Im Auftrag von Fred Gleason
Gesendet: Mittwoch, 20. M?rz 2024 20:53
An: Icecast streaming server user discussions <icecast at xiph.org
<mailto:icecast at xiph.org> >
Betreff: Re: [Icecast] Education - 1, 000s, 100, 000's, Millions of
listeners. (What kind of infrastructure)

 

On Mar 20, 2024, at 13:16, Wayne Barron <wayne at cffcs.com <mailto:wayne
at cffcs.com> > wrote:

 

In Windows and Linux web servers, we can create a forest for our web servers.
Send traffic to different servers to even the workload.

Can we do something like this with the Icecast servers?
(or)
Will we have to install new VMs, add the heavy stations on that one,
and send the new traffic there?

 

Ok, I?m going to be ?that guy??

 

I would argue that, as soon as you?ve hit an audience size of 10,000 or more
(especially if that audience is at all geographically dispersed), IceCast is
basically off the table. The reason why can be summarized in three letters:
?CDN? [Content Distribution Networks].

 

To fan out to large, geographically dispersed audiences of 10,000 or more (not
to mention 100k?s or, Lord help us, 1M?s or more), you need to get content
cached in locations that are geographically close to your listeners. By far the
easiest (read: most cost effective) way to do this at scale is to leverage the
already existing infrastructure of CDNs (companies like Akamai or CloudFlare,
that have a world-wide footprint). That means using streaming formats that
utilize segmented distribution mechanisms, such as HLS or DASH. You can
kinda-sorta do this sort of thing with IceCast by using relays, but it?s complex
to configure and monitor while being not well supported at many CDNs (Akamai for
example discontinued their IceCast product offering several years ago). HLS OTOH
plays very well with that infrastructure because it?s effectively just a bunch
of static files that get replicated via HTTP[S]. No special ?server? software is
required; bog-standard Apache or Nginx work just fine, because the complex
?media handling? bits have been intentionally pushed to the endpoints; namely
the encoder and (especially) the players. Today though, when FOSS HLS audio
encoders are available and pretty much every browser supports playing HLS
content natively, the complexity angle can be largely ignored by content
creators.

 

Just my take. That, and 2 ? will get you a (cheap) cup of coffee?

 

Cheers!

 

 

|---------------------------------------------------------------------|

| Frederick F. Gleason, Jr. |             Chief Developer             |

|                           |             Paravel Systems             |

|---------------------------------------------------------------------|

|    All progress is based upon a universal innate desire of every    |

|    organism to live beyond its income.                              |

|                                                                     |

|                                                  -- Samuel Butler   |

|                                                  "Notebooks"       
|

|---------------------------------------------------------------------|

 

_______________________________________________
Icecast mailing list
Icecast at xiph.org <mailto:Icecast at xiph.org> 
http://lists.xiph.org/mailman/listinfo/icecast


-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<http://lists.xiph.org/pipermail/icecast/attachments/20240321/636ef45e/attachment.htm>

Looking through HLS, I found this.
https://github.com/mbugeia/srt2hls
For what I can un
As in the liquidsoap link I shared, this is what my links look like.
https://github.com/savonet/liquidsoap/discussions/3024
Liquisoap link.
music =
playlist("https://www.domain.com/Panels/DJ/101.asp",mode="normal")

On Thu, Mar 21, 2024 at 4:08?AM <thomas.zumbrunnen at gmail.com>
wrote:
>
> Hi Wayne
>
>
>
> Yep, HLS is something we would like to use with Icecast. As you mentioned,
Liquidsoap made a great step in the right direction if it comes to HLS. It seem
that icecast might not get this support soon, as you might read in the
discussion list. I don?t share these arguments, because we have real world use
cases (we have many of them) where we  would like use icecast with HLS support.
But unfortunately we needed to go with another streaming server application
which supports HLS to fulfill our requirements.
>
>
>
> But coming back to the topic of this thread.
>
> I was looking into using NGINX as a reverse proxy for doing the load
balancing (single point of SSL and centralized monitoring). NGINX is awesome in
doing this. And there are some valid arguments using the approach to put a
single NGINX server (or a two node Nginx Cluster) in front of a multi node
icecast cluster. At that time, 5 years ago,  we decided to go with the ? poor
man solution ? and use the RR DNS approach. We did not had enough time for
investigation, development and testing. Additional, we eliminated another single
point of failure. Sometimes I like way of KISS (keep it simple stupid) ?
>
>
>
> if you need to serve static files globaly, I would consider the CDN
approach. We use CF for this, but any other CDN provider would do the job. They
are kind of ?same same - but different?
>
> Cheers
>
> Tom
>
>
>
> Von: Icecast <icecast-bounces at xiph.org> Im Auftrag von Wayne
Barron
> Gesendet: Mittwoch, 20. M?rz 2024 23:06
> An: Icecast streaming server user discussions <icecast at xiph.org>
> Betreff: Re: [Icecast] Education - 1, 000s, 100, 000's, Millions of
listeners. (What kind of infrastructure)
>
>
>
> Tom and Frederick.
>
> Thank you both for your input. It is greatly appreciated, not only by me,
but I am sure for many others who find this thread.
>
>
>
> Tom - using Linux server for icecast.
>
>
>
> HLS was brought to my attention a while back on the liquidsoap forum.
>
> I have not had a chance to completely look in on it, but do plan on
checking it out.
>
>
>
> Tom. You have your icecast servers in a round robin configuration.
>
> Are you using NGINX for that?
>
> I have watched several YouTube videos on using it for doing round robin as
well as ssl and other things.
>
>
>
> I understand that once a scenario of say 10,000 or more, that I will have
to look into either a bigger infrastructure on my side or leasing space on the
cloud somewhere.
>
>
>
> Now, if I did do the cloud.
>
> Would that be to host the servers or just the files or everything?
>
>
>
> Wayne
>
>
>
> On Wed, Mar 20, 2024, 4:50?PM <thomas.zumbrunnen at gmail.com> wrote:
>
> Dear all
>
>
>
> My 5 cents (or Rappen in CH) if it comes to serving many clients.
>
> We are running a 4 node cluster since several years ? rock solid and w/o
any issues. This cluster serves many thousands of listeners from all over the
world. Our source transcoder sending the audio streams to each Node.  Hence,
transcoding power is not an issue here. The four Nodes a geographically
dispersed in 3 countries in Europe. In our case each Node is running Debian with
icecast and has 10Gbit connectivity with brilliant worldwide peerings. Good
peering is key, choose your ISP wisely ?.Each icecast servers has the same multi
domain ssl cert. which allows us to deliver to several customers (each customer
a subdomain) the cluster is round robin load balanced by using AWS Route53. This
approach may can be achived also with other DNS Providers like Cloudflare. For
example, if one node need to be taken down for maintenance, Route53 throws the
Node out of the DNS automatically. This will be achived with ?health checks?
This mechanism is pretty fast and responsive. If a client gets disconnected and
tries a reconnect, the RR DNS is passing the client immediately to a working
Node. No issues here as well. It just works.
>
>
>
> In the beginning we?ve experienced similar issues even though, the
bandwidth capacity of the VM?s was never the root cause. We?ve identified some
solutions:
>
>
>
> Linux TCP stack tuning. Cloudflare has many studies published in their
Blogs about this. But you will find a lot about this tuning also in the interweb
> Consider to bake your own kernel, which is tuned for high throughput ? goes
in line with TCP Stack tunning
> Tune the Linux open file limits and adjust the init start script for the
icecast server. Example : start icecast with  ulimit -c unlimited and ulimit -n
32768
> Consider to use FreeBSD instead of Linux. FreeBSD has the better TCP stack
out of the box.
> If all of this is not feasible for you, just add a new Node to the cluster
and level the amount of clients to more Nodes.
>
>
>
> For the points 1. and 2. I can?t give you a ?out of the box? solution or
default settings. It?s an iteration process: adjusting, trial, load testing,
monitoring and <repeat>
>
> Because the result will need to fit your requirements, therefore every
setup might need a different tuning. And btw. do not try using icecast on
Windows Servers, if you need to serve a lot of clients ?
>
>
>
> Happy icecasting
>
> tom
>
>
>
> Von: Icecast <icecast-bounces at xiph.org> Im Auftrag von Fred
Gleason
> Gesendet: Mittwoch, 20. M?rz 2024 20:53
> An: Icecast streaming server user discussions <icecast at xiph.org>
> Betreff: Re: [Icecast] Education - 1, 000s, 100, 000's, Millions of
listeners. (What kind of infrastructure)
>
>
>
> On Mar 20, 2024, at 13:16, Wayne Barron <wayne at cffcs.com> wrote:
>
>
>
> In Windows and Linux web servers, we can create a forest for our web
servers.
> Send traffic to different servers to even the workload.
>
> Can we do something like this with the Icecast servers?
> (or)
> Will we have to install new VMs, add the heavy stations on that one,
> and send the new traffic there?
>
>
>
> Ok, I?m going to be ?that guy??
>
>
>
> I would argue that, as soon as you?ve hit an audience size of 10,000 or
more (especially if that audience is at all geographically dispersed), IceCast
is basically off the table. The reason why can be summarized in three letters:
?CDN? [Content Distribution Networks].
>
>
>
> To fan out to large, geographically dispersed audiences of 10,000 or more
(not to mention 100k?s or, Lord help us, 1M?s or more), you need to get content
cached in locations that are geographically close to your listeners. By far the
easiest (read: most cost effective) way to do this at scale is to leverage the
already existing infrastructure of CDNs (companies like Akamai or CloudFlare,
that have a world-wide footprint). That means using streaming formats that
utilize segmented distribution mechanisms, such as HLS or DASH. You can
kinda-sorta do this sort of thing with IceCast by using relays, but it?s complex
to configure and monitor while being not well supported at many CDNs (Akamai for
example discontinued their IceCast product offering several years ago). HLS OTOH
plays very well with that infrastructure because it?s effectively just a bunch
of static files that get replicated via HTTP[S]. No special ?server? software is
required; bog-standard Apache or Nginx work just fine, because the complex
?media handling? bits have been intentionally pushed to the endpoints; namely
the encoder and (especially) the players. Today though, when FOSS HLS audio
encoders are available and pretty much every browser supports playing HLS
content natively, the complexity angle can be largely ignored by content
creators.
>
>
>
> Just my take. That, and 2 ? will get you a (cheap) cup of coffee?
>
>
>
> Cheers!
>
>
>
>
>
> |---------------------------------------------------------------------|
>
> | Frederick F. Gleason, Jr. |             Chief Developer             |
>
> |                           |             Paravel Systems             |
>
> |---------------------------------------------------------------------|
>
> |    All progress is based upon a universal innate desire of every    |
>
> |    organism to live beyond its income.                              |
>
> |                                                                     |
>
> |                                                  -- Samuel Butler   |
>
> |                                                  "Notebooks"   
|
>
> |---------------------------------------------------------------------|
>
>
>
> _______________________________________________
> Icecast mailing list
> Icecast at xiph.org
> http://lists.xiph.org/mailman/listinfo/icecast
>
> _______________________________________________
> Icecast mailing list
> Icecast at xiph.org
> http://lists.xiph.org/mailman/listinfo/icecast

>From what I understand, I should be able to take a Dynamic Source.
Which will be from an ASP file with a list of songs.
Example of the list within the ASP File.

https://domain/Asylum/01082021184726649.mp3
https://domain/Asylum/01082021184727383.mp3
https://domain/Asylum/01082021184727946.mp3

And HLS will create a live one.m3u8 files in the /tmp/HLS directory.
Every time I update the ASP page, it updates the live.m3u8 file in the
HLS directory.

I found this page. (But it is NOT active)
https://github.com/mbugeia/srt2hls
Toots has implemented HLS Metadata in Liquidsoap for rolling-release-v2.2.x!
https://github.com/savonet/liquidsoap/issues/1487#issuecomment-1605797838

It took a while to get used to working with Icecast and Liquidsoap,
and now I will have to learn HLS and Liquidsoap and how to make them
work together.

It would be great if anyone knew of a page that gives instructions on
how to set up a radio station with HLS and Liquidsoap that can at
least get me started.

This is pretty much the way my liq file is set up with Liquidsoap and Icecast.

# !/home/iceadmin/.opam/default/bin/liquidsoap
# log dir
set ("log.file.path","/var/log/liquidsoap/Radio.log")
set("scheduler.fast_queues",1)
# Music
music =
playlist("https://www.example.com/Panels/DJ/101.asp",mode="normal")

# Start building the feed with music
radio = music
# Now add some jingles
#radio = random(weights = [1, 4],[radio, jingles])

# Stream it out
output.icecast(%mp3(bitrate=256,samplerate=44100,internal_quality=0,id3v2=true,stereo=true,stereo_mode="stereo"),
name="Radio-101",
encoding="UTF-8",
host="192.168.2.203",
port=8000,
password="hackme",
icy_metadata="true",
description="Where I come to Rock!",
mount="Radio-101",
mksafe(radio))

Can I still use this, or will I have to add something to it to make it
work with HLS?
I see that there is one area that would need to be changed.
output.icecast
And this makes me think that the entire script will not work with HLS,
or will it?

A lot to learn and a lot to think about.
I just launched for Linux VM for the first time in over a year, and I
am trying to remember where everything is at and how to work it again
with Icecast and Liquidsoap.
Luckily, I have a lot of the things written down for reference.

On Thu, Mar 21, 2024 at 4:08?AM <thomas.zumbrunnen at gmail.com>
wrote:
>
> Hi Wayne
>
>
>
> Yep, HLS is something we would like to use with Icecast. As you mentioned,
Liquidsoap made a great step in the right direction if it comes to HLS. It seem
that icecast might not get this support soon, as you might read in the
discussion list. I don?t share these arguments, because we have real world use
cases (we have many of them) where we  would like use icecast with HLS support.
But unfortunately we needed to go with another streaming server application
which supports HLS to fulfill our requirements.
>
>
>
> But coming back to the topic of this thread.
>
> I was looking into using NGINX as a reverse proxy for doing the load
balancing (single point of SSL and centralized monitoring). NGINX is awesome in
doing this. And there are some valid arguments using the approach to put a
single NGINX server (or a two node Nginx Cluster) in front of a multi node
icecast cluster. At that time, 5 years ago,  we decided to go with the ? poor
man solution ? and use the RR DNS approach. We did not had enough time for
investigation, development and testing. Additional, we eliminated another single
point of failure. Sometimes I like way of KISS (keep it simple stupid) ?
>
>
>
> if you need to serve static files globaly, I would consider the CDN
approach. We use CF for this, but any other CDN provider would do the job. They
are kind of ?same same - but different?
>
> Cheers
>
> Tom
>
>
>
> Von: Icecast <icecast-bounces at xiph.org> Im Auftrag von Wayne
Barron
> Gesendet: Mittwoch, 20. M?rz 2024 23:06
> An: Icecast streaming server user discussions <icecast at xiph.org>
> Betreff: Re: [Icecast] Education - 1, 000s, 100, 000's, Millions of
listeners. (What kind of infrastructure)
>
>
>
> Tom and Frederick.
>
> Thank you both for your input. It is greatly appreciated, not only by me,
but I am sure for many others who find this thread.
>
>
>
> Tom - using Linux server for icecast.
>
>
>
> HLS was brought to my attention a while back on the liquidsoap forum.
>
> I have not had a chance to completely look in on it, but do plan on
checking it out.
>
>
>
> Tom. You have your icecast servers in a round robin configuration.
>
> Are you using NGINX for that?
>
> I have watched several YouTube videos on using it for doing round robin as
well as ssl and other things.
>
>
>
> I understand that once a scenario of say 10,000 or more, that I will have
to look into either a bigger infrastructure on my side or leasing space on the
cloud somewhere.
>
>
>
> Now, if I did do the cloud.
>
> Would that be to host the servers or just the files or everything?
>
>
>
> Wayne
>
>
>
> On Wed, Mar 20, 2024, 4:50?PM <thomas.zumbrunnen at gmail.com> wrote:
>
> Dear all
>
>
>
> My 5 cents (or Rappen in CH) if it comes to serving many clients.
>
> We are running a 4 node cluster since several years ? rock solid and w/o
any issues. This cluster serves many thousands of listeners from all over the
world. Our source transcoder sending the audio streams to each Node.  Hence,
transcoding power is not an issue here. The four Nodes a geographically
dispersed in 3 countries in Europe. In our case each Node is running Debian with
icecast and has 10Gbit connectivity with brilliant worldwide peerings. Good
peering is key, choose your ISP wisely ?.Each icecast servers has the same multi
domain ssl cert. which allows us to deliver to several customers (each customer
a subdomain) the cluster is round robin load balanced by using AWS Route53. This
approach may can be achived also with other DNS Providers like Cloudflare. For
example, if one node need to be taken down for maintenance, Route53 throws the
Node out of the DNS automatically. This will be achived with ?health checks?
This mechanism is pretty fast and responsive. If a client gets disconnected and
tries a reconnect, the RR DNS is passing the client immediately to a working
Node. No issues here as well. It just works.
>
>
>
> In the beginning we?ve experienced similar issues even though, the
bandwidth capacity of the VM?s was never the root cause. We?ve identified some
solutions:
>
>
>
> Linux TCP stack tuning. Cloudflare has many studies published in their
Blogs about this. But you will find a lot about this tuning also in the interweb
> Consider to bake your own kernel, which is tuned for high throughput ? goes
in line with TCP Stack tunning
> Tune the Linux open file limits and adjust the init start script for the
icecast server. Example : start icecast with  ulimit -c unlimited and ulimit -n
32768
> Consider to use FreeBSD instead of Linux. FreeBSD has the better TCP stack
out of the box.
> If all of this is not feasible for you, just add a new Node to the cluster
and level the amount of clients to more Nodes.
>
>
>
> For the points 1. and 2. I can?t give you a ?out of the box? solution or
default settings. It?s an iteration process: adjusting, trial, load testing,
monitoring and <repeat>
>
> Because the result will need to fit your requirements, therefore every
setup might need a different tuning. And btw. do not try using icecast on
Windows Servers, if you need to serve a lot of clients ?
>
>
>
> Happy icecasting
>
> tom
>
>
>
> Von: Icecast <icecast-bounces at xiph.org> Im Auftrag von Fred
Gleason
> Gesendet: Mittwoch, 20. M?rz 2024 20:53
> An: Icecast streaming server user discussions <icecast at xiph.org>
> Betreff: Re: [Icecast] Education - 1, 000s, 100, 000's, Millions of
listeners. (What kind of infrastructure)
>
>
>
> On Mar 20, 2024, at 13:16, Wayne Barron <wayne at cffcs.com> wrote:
>
>
>
> In Windows and Linux web servers, we can create a forest for our web
servers.
> Send traffic to different servers to even the workload.
>
> Can we do something like this with the Icecast servers?
> (or)
> Will we have to install new VMs, add the heavy stations on that one,
> and send the new traffic there?
>
>
>
> Ok, I?m going to be ?that guy??
>
>
>
> I would argue that, as soon as you?ve hit an audience size of 10,000 or
more (especially if that audience is at all geographically dispersed), IceCast
is basically off the table. The reason why can be summarized in three letters:
?CDN? [Content Distribution Networks].
>
>
>
> To fan out to large, geographically dispersed audiences of 10,000 or more
(not to mention 100k?s or, Lord help us, 1M?s or more), you need to get content
cached in locations that are geographically close to your listeners. By far the
easiest (read: most cost effective) way to do this at scale is to leverage the
already existing infrastructure of CDNs (companies like Akamai or CloudFlare,
that have a world-wide footprint). That means using streaming formats that
utilize segmented distribution mechanisms, such as HLS or DASH. You can
kinda-sorta do this sort of thing with IceCast by using relays, but it?s complex
to configure and monitor while being not well supported at many CDNs (Akamai for
example discontinued their IceCast product offering several years ago). HLS OTOH
plays very well with that infrastructure because it?s effectively just a bunch
of static files that get replicated via HTTP[S]. No special ?server? software is
required; bog-standard Apache or Nginx work just fine, because the complex
?media handling? bits have been intentionally pushed to the endpoints; namely
the encoder and (especially) the players. Today though, when FOSS HLS audio
encoders are available and pretty much every browser supports playing HLS
content natively, the complexity angle can be largely ignored by content
creators.
>
>
>
> Just my take. That, and 2 ? will get you a (cheap) cup of coffee?
>
>
>
> Cheers!
>
>
>
>
>
> |---------------------------------------------------------------------|
>
> | Frederick F. Gleason, Jr. |             Chief Developer             |
>
> |                           |             Paravel Systems             |
>
> |---------------------------------------------------------------------|
>
> |    All progress is based upon a universal innate desire of every    |
>
> |    organism to live beyond its income.                              |
>
> |                                                                     |
>
> |                                                  -- Samuel Butler   |
>
> |                                                  "Notebooks"   
|
>
> |---------------------------------------------------------------------|
>
>
>
> _______________________________________________
> Icecast mailing list
> Icecast at xiph.org
> http://lists.xiph.org/mailman/listinfo/icecast
>
> _______________________________________________
> Icecast mailing list
> Icecast at xiph.org
> http://lists.xiph.org/mailman/listinfo/icecast