Raymond Lutz
2011-Jun-27 21:33 UTC
[Icecast] Multiple mount points without dynamic reencoding
Hello:
I am running the icecast2.3.2 server and ices0.4 with a perl script
which provides the filename of the next track to play. I am reencoding
in advance using ffmpeg, which has been very reliable. My main stream is
mp3, 64kbps / 44.1 kHz (mono) and I have pre-reencoded the tracks to
32kbps / 22.05 kHz (mono), for a lower data rate stream. At this time, I
have not figured out how to get a second mount point to work for the 32k
stream.
The stream is at http://www.AirProgressive.org.
My content is "News/Talk" and tend to be very long, usually at least
1/2
hour or 1 hour (or more). Fidelity is not as important as compatibility,
so we are using mp3 format, and therefore, ices 0.4 version. (plus most
of the files we get to play are mp3 and it is not under our control.)
It is pretty clear that icecast can deal with multiple mount points.
Ices says it can too, but apparently to do so, it will require that I
deal with dynamic reencoding, which thus far I have not been able to
successfully run as it encounters a runtime error. (See the end of this
email for the dump of the error.) So I have worked around this
difficulty by pre-reencoding the audio using ffmpeg, which is very
robust, but can take significant time to reencode a track of 30 to 60
minutes (taking a number of minutes to do so). But this can be easily
done in advance and cached as separate files. Doing this has eliminated
glitches that will occur at track boundaries when going from track to
track (although mp3 describes the bitrate and freq on a frame-by-frame
basis, the assumption is that each track will be uniform, and thus, the
entire stream will be uniform).
1. Is it possible to serve two file formats using ices, destined to two
different mount points?
a. Will I need to run two instances of ices to do it?
b. Is there a way to specify which mount point a given file should
be sent to, when ices_get_next() perl function is called? (i.e. if I
return a list of filenames, will each filename be sent to the list of
streams if reencoding is not enabled, for example??)
2. You say in your documentation that there is no interest in mp3 format
files, which I find to be astonishing, since most audio files available
are mp3 and almost none are Ogg/Vorbis or anything else. You can add me
to the list of people who are interested, so you will no longer say
there is no interest (or is this some sort of a joke that I don't
understand?) Since most of the files we get access to are mp3,
reencoding into Ogg/Vorbis will only decrease fidelity, and so it sounds
silly to do that.
3. If you happen to know, is there any benefit to changing the sample
rate to a lower value when reencoding, or is it just as good to use
32K/44.1K as 32K/22.05K.
4. Currently, what is the best approach to get into the Shoutcast
directory. Can I treat shoutcast as a slave and relay content to it?
That mean I need to install shoutcast on my (linux) server, right? What
are the configuration details?
Thank you for your kind assistance.
--Ray Lutz
=============================================ERROR DUMP
*** glibc detected *** ices: free(): invalid pointer: 0xb7ddd008 ***
======= Backtrace: ========/lib/libc.so.6[0xa965a5]
/lib/libc.so.6(cfree+0x59)[0xa969e9]
ices[0x804fcbb]
ices(ices_stream_loop+0x6d1)[0x804d431]
ices(main+0x33)[0x804b793]
/lib/libc.so.6(__libc_start_main+0xdc)[0xa42e9c]
ices[0x804b6a1]
======= Memory map: =======00110000-00228000 r-xp 00000000 00:26 157335553
/usr/local/lib/libxml2.so.2.7.8
00228000-0022e000 rw-p 00117000 00:26 157335553 /usr/local/lib/libxml2.so.2.7.8
0022e000-00236000 r-xp 00000000 00:26 157335612
/usr/local/lib/libvorbisfile.so.3.3.4
00236000-00237000 rw-p 00007000 00:26 157335612
/usr/local/lib/libvorbisfile.so.3.3.4
00237000-0023c000 r-xp 00000000 00:26 157335602 /usr/local/lib/libogg.so.0.7.1
0023c000-0023d000 rw-p 00004000 00:26 157335602 /usr/local/lib/libogg.so.0.7.1
0040d000-00428000 r-xp 00000000 00:26 26871825 /lib/ld-2.5.so
00428000-00429000 r--p 0001a000 00:26 26871825 /lib/ld-2.5.so
00429000-0042a000 rw-p 0001b000 00:26 26871825 /lib/ld-2.5.so
00543000-0056a000 r-xp 00000000 00:26 26871781 /lib/libm-2.5.so
0056a000-0056b000 r--p 00026000 00:26 26871781 /lib/libm-2.5.so
0056b000-0056c000 rw-p 00027000 00:26 26871781 /lib/libm-2.5.so
005a1000-005e8000 r-xp 00000000 00:26 157335629
/usr/local/lib/libmp3lame.so.0.0.0
005e8000-005e9000 rw-p 00046000 00:26 157335629
/usr/local/lib/libmp3lame.so.0.0.0
005e9000-0061c000 rw-p 005e9000 00:00 0
0068a000-0069f000 r-xp 00000000 00:26 26871789 /lib/libnsl-2.5.so
0069f000-006a0000 r--p 00014000 00:26 26871789 /lib/libnsl-2.5.so
006a0000-006a1000 rw-p 00015000 00:26 26871789 /lib/libnsl-2.5.so
006a1000-006a3000 rw-p 006a1000 00:00 0
0072e000-00737000 r-xp 00000000 00:26 26871796 /lib/libcrypt-2.5.so
00737000-00738000 r--p 00008000 00:26 26871796 /lib/libcrypt-2.5.so
00738000-00739000 rw-p 00009000 00:26 26871796 /lib/libcrypt-2.5.so
00739000-00760000 rw-p 00739000 00:00 0
00799000-007a4000 r-xp 00000000 00:26 26871882
/lib/libgcc_s-4.1.2-20080825.so.1
007a4000-007a5000 rw-p 0000a000 00:26 26871882
/lib/libgcc_s-4.1.2-20080825.so.1
007e0000-007f0000 r-xp 00000000 00:26 26871874 /lib/libresolv-2.5.so
007f0000-007f1000 r--p 0000f000 00:26 26871874 /lib/libresolv-2.5.so
007f1000-007f2000 rw-p 00010000 00:26 26871874 /lib/libresolv-2.5.so
007f2000-007f4000 rw-p 007f2000 00:00 0
0084f000-00876000 r-xp 00000000 00:26 157335607
/usr/local/lib/libvorbis.so.0.4.5
00876000-00877000 rw-p 00027000 00:26 157335607
/usr/local/lib/libvorbis.so.0.4.5
008ee000-008f8000 r-xp 00000000 00:26 26871832 /lib/libnss_files-2.5.so
008f8000-008f9000 r--p 00009000 00:26 26871832 /lib/libnss_files-2.5.so
008f9000-008fa000 rw-p 0000a000 00:26 26871832 /lib/libnss_files-2.5.so
00a2d000-00b7f000 r-xp 00000000 00:26 26871861 /lib/libc-2.5.so
00b7f000-00b81000 r--p 00152000 00:26 26871861 /lib/libc-2.5.so
00b81000-00b82000 rw-p 00154000 00:26 26871861 /lib/libc-2.5.so
00b82000-00b85000 rw-p 00b82000 00:00 0
00c76000-00c77000 r-xp 00c76000 00:00 0 [vdso]
00d7f000-00d8c000 r-xp 00000000 00:26 157335622
/usr/local/lib/libshout.so.3.2.0
00d8c000-00d8d000 rw-p 0000c000 00:26 157335622
/usr/local/lib/libshout.so.3.2.0
00e2d000-00e30000 r-xp 00000000 00:26 26871871 /lib/libdl-2.5.so
00e30000-00e31000 r--p 00002000 00:26 26871871 /lib/libdl-2.5.so
00e31000-00e32000 rw-p 00003000 00:26 26871871 /lib/libdl-2.5.so
00e6d000-00e6f000 r-xp 00000000 00:26 26871795 /lib/libutil-2.5.so
00e6f000-00e70000 r--p 00001000 00:26 26871795 /lib/libutil-2.5.so
00e70000-00e71000 rw-p 00002000 00:26 26871795 /lib/libutil-2.5.so
00e7f000-00e94000 r-xp 00000000 00:26 26871820 /lib/libpthread-2.5.so
00e94000-00e95000 r--p 00015000 00:26 26871820 /lib/libpthread-2.5.so
00e95000-00e96000 rw-p 00016000 00:26 26871820 /lib/libpthread-2.5.so
00e96000-00e98000 rw-p 00e96000 00:00 0
00ebf000-00fea000 r-xp 00000000 00:26 174100243
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
00fea000-00fef000 rw-p 0012a000 00:26 174100243
/usr/lib/perl5/5.8.8/i386-linux-thread-multi/CORE/libperl.so
00fef000-00ff1000 rw-p 00fef000 00:00 0
08048000-08055000 r-xp 00000000 00:26 157302894 /usr/local/bin/ices
08055000-08056000 rw-p 0000d000 00:26 157302894 /usr/local/bin/ices
08056000-0810c000 rw-p 08056000 00:00 0
09893000-09b0b000 rw-p 09893000 00:00 0 [heap]
b7d1a000-b7e1e000 rw-p b7d1a000 00:00 0
b7ea0000-b7ec1000 rw-p b7ea0000 00:00 0
b7ec1000-b7efc000 r--p 00000000 00:26 36622587
/usr/lib/locale/en_US.utf8/LC_CTYPE
b7efc000-b7efd000 r--p 00000000 00:26 36622639
/usr/lib/locale/en_US.utf8/LC_NUMERIC
b7efd000-b7efe000 r--p 00000000 00:26 36623942
/usr/lib/locale/en_US.utf8/LC_TIME
b7efe000-b7fd5000 r--p 00000000 00:26 36622592
/usr/lib/locale/en_US.utf8/LC_COLLATE
b7fd5000-b7fd6000 r--p 00000000 00:26 36623941
/usr/lib/locale/en_US.utf8/LC_MONETARY
b7fd6000-b7fd7000 r--p 00000000 00:26 36622735
/usr/lib/locale/en_US.utf8/LC_MESSAGES/SYS_LC_MESSAGES
b7fd7000-b7fd8000 r--p 00000000 00:26 36623713
/usr/lib/locale/en_US.utf8/LC_PAPER
b7fd8000-b7fd9000 r--p 00000000 00:26 36623399
/usr/lib/locale/en_US.utf8/LC_NAME
b7fd9000-b7fe0000 r--s 00000000 00:26 139657355
/usr/lib/gconv/gconv-modules.cache
b7fe0000-b7fe5000 rw-p b7fe0000 00:00 0
b7fe5000-b7fe6000 r--p 00000000 00:26 36623939
/usr/lib/locale/en_US.utf8/LC_ADDRESS
b7fe6000-b7fe7000 r--p 00000000 00:26 36624033
/usr/lib/locale/en_US.utf8/LC_TELEPHONE
b7fe7000-b7fe8000 r--p 00000000 00:26 36624044
/usr/lib/locale/en_US.utf8/LC_MEASUREMENT
b7fe8000-b7fe9000 r--p 00000000 00:26 36623940
/usr/lib/locale/en_US.utf8/LC_IDENTIFICATION
b7fe9000-b7feb000 rw-p b7fe9000 00:00 0
bfa27000-bfa3c000 rw-p bffe9000 00:00 0 [stack]
Aborted
--
---------------------------------------
Raymond Lutz
Cognisys, Inc.
1010 Old Chase Ave., Bldg B
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com
Thomas.Rucker at tieto.com
2011-Jun-28 06:01 UTC
[Icecast] Multiple mount points without dynamic reencoding
Hi, I'll skip the ices0 part of the email as I don't know that software. I only dealt with ices2 in that respect. So that is question is left to be answered by someone else (though I'm not sure if it will still apply). And here we arrive also at the most likely source of an underlying misconception: You do not have to have the same codec for the files as for the resulting stream! There are plenty source clients that will happily take almost any file format and codec in and encode it into a nice ogg/vorbis stream. (Random examples: mpd, rpld, liquidsoap,...) Also today's CPUs will happily do that without consuming too many of their cycles. It just happens to be a limitation of a few source clients that do not allow for this (e.g. ices2) that is fuelling this misconception. Also note that icecast and the shoutcast directory do not mix due to the terms of service of the latter. As to your question 32kbit at 44kHz vs 32kbit at 22kHz Think of two pictures: 44100x16px and 22050x16px (which is only one half of the former) both run through a JPEG encoder at the same setting. Which one will look better? It depends what you're after. Quality or spectral bandwith? When in doubt, try it. Cheers Thomas
Raymond Lutz
2011-Jun-28 20:00 UTC
[Icecast] Multiple mount points without dynamic reencoding
Thomas:
Thank you for your response. (I hope someone else will comment on how to
deal with multiple source files without dynamic encoding, but the
question may be moot if I can get dynamic encoding to work properly). It
was some usefulness to write the response below, so I apologize for the
length.
Perhaps you can correct/ my thinking here.
Reencoding from mp3 to ogg/vorbis will result in reduced fidelity,
because both are lossy encoders and are both assuming they are starting
with the original audio content, and it is well documented that
reencoding mp3 to ogg-vorbis only reduces the ultimate fidelity. In my
situation, we have mp3s coming from various sources, and their original
encoding varies across the map,
but the most common is 64k/44.1. So, we are re-encoding to that as our
standard so that in some cases, no reencoding will be necessary and the
original fidelity will be preserved.
(I find it is essential to have a continuous stream with the same format
and changes from format to format will break the connection with some
players, and some try to play the new data at the old rate, etc. So the
stream MUST be one encoding and data rate.)
It would be great if ices2 would be able to accept mp3 as input and
perhaps reencode it to ogg/vorbis (although in my mind, that is a stupid
thing to do, per the above argument). So my "interest" in mp3
continues,
and the statement that no one is interested in mp3 is ridiculous.
With ices2, the documentation says about the configuration parameter
"resample":
<resample>
<in-rate>44100</in-rate>
<out-rate>22050</out-rate>
</resample>
> When encoding or re-encoding, there is a point where you take PCM
> audio and encode to Ogg Vorbis. In some situations a particular
> encoded stream may require a lower samplerate to achieve a lower
> bitrate. The resample will modifiy the audio data before it enters the
> encoder, but does not affect other instances.
>
> The most common values used are 48000, 44100, 22050 and 11025, and is
> really only used to resample to a lower samplerate, going to a higher
> rate serves no purpose within IceS.
>
This implies that we have a PCM input stream of fixed format, and we can
set this just once and be happy. This is not the case in my situation,
because the mp3s are coming from various sources, with differing sample
rates, and obviously are not PCM. Therefore, it appears that ices2 is
not sufficient solution.
On your "source client" comment, I looked these over,
1. mpd is oriented to playing music and is not suitable for what I need
to do. My application pulls in mp3 "podcasts" from various websites
and
queues these up for specific time slots during the day. mpd requires a
playlist which I cannot provide since the content may change minutes
before the timeslot arrives.
2. rpld (RoarAudio) -- looks interesting, but is a layer I'm not sure I
need unless I need to interface with other sound input devices. I will
have to research this more since it looks like a good building block,
and may be a good solution for dynamic reencoding. (Of course, some of
these may encounter the same runtime error as ices0.4).
3. liquidsoap -- looks very interesting, but also looks very new and
unstable, and also looks like it is trying to do a lot more than I need
right now, with a complete programming language to learn to boot. The
good news is it looks like there is something going on here, unlike many
audio projects that are largely defunct and abandoned. I see many
outstanding issues before they get to version 1.0, with a stream of
messages and difficulties implying that this might be a bottomless pit
of time if I get started using it.
I'm not ready to take on the big project of adopting liquidsoap just to
deal with this encoding issue.
As I see it, I still have the following options:
1. debug the runtime error that occurs when trying to use liblame to
reencode mp3 using ices0.4. Not sure how long this will take or if I
will be successful at determining what is wrong. This would be a nice
solution but it still relies on the old version of ices.
2. Use a non-ices2 and non-ices0.4 source client that does support
dynamic reencoding, so I can start from an arbitrary source format and
produce a desired result format. From the list you suggested above, the
choice is not clear. I looked over liquidsoap for about an hour and I
still don't know how to use it to fill that need, and even if I worked
on it for a week, I may not understand what is needed.
3. Figure out how to use ices2 or 0.4 to access multiple files at the
same time, so that dynamic reencoding is not necessary.
3a. I am using mp3 now because but it is possible to produce ogg-vorbis
using asynchronous reencoding (using ffmpeg), and then the question is
the same -- if I have multiple sources, can ices2, for example, with
ogg-vorbis input, run two sources simultaneously?
3b. If I continue to use ices0.4 because it supports mp3, then it may be
possible to solve the problem by running multiple instances of ices and
providing a different format mp3 to each, and connecting them to
separate mount points on icecast.
BTW, I went through several versions of re-encoding packages before I
was able to get ffmpeg to work correctly for my needs. I think it is too
bad that ices dropped the ball and decided mp3 was irrelevant just
because they have a vested interest in ogg/vorbis.
--Ray
On 6/27/2011 11:01 PM, Thomas.Rucker at tieto.com wrote:> Hi,
>
> I'll skip the ices0 part of the email as I don't know that
software.
> I only dealt with ices2 in that respect. So that is question
> is left to be answered by someone else (though I'm not sure if it
> will still apply).
>
> And here we arrive also at the most likely source of an underlying
> misconception:
> You do not have to have the same codec for the files as for
> the resulting stream! There are plenty source clients that will
> happily take almost any file format and codec in and encode it into
> a nice ogg/vorbis stream. (Random examples: mpd, rpld, liquidsoap,...)
> Also today's CPUs will happily do that without consuming too many
> of their cycles.
>
> It just happens to be a limitation of a few source clients that
> do not allow for this (e.g. ices2) that is fuelling this misconception.
>
> Also note that icecast and the shoutcast directory do not mix due
> to the terms of service of the latter.
>
> As to your question 32kbit at 44kHz vs 32kbit at 22kHz
> Think of two pictures: 44100x16px and 22050x16px (which is
> only one half of the former) both run through a JPEG encoder at the
> same setting. Which one will look better?
> It depends what you're after. Quality or spectral bandwith?
> When in doubt, try it.
>
> Cheers
>
> Thomas
> _______________________________________________
> Icecast mailing list
> Icecast at xiph.org
> http://lists.xiph.org/mailman/listinfo/icecast
>
--
---------------------------------------
Raymond Lutz
Cognisys, Inc.
1010 Old Chase Ave., Bldg B
El Cajon (San Diego Cty), CA 92020 USA
Voice 619-447-3246
http//www.cognisys.com