Icecast - Oct 2004 - [Jackit-devel] Re: ices-kh dropping jack ports unexpectedly

I've been having a problem where ices-kh (the jack'ified version)
disconnects from its jack input source unexpectedly. This happens mainly
while other jack clients are being started/stopped, or
connected/disconnected, but also at other times (e.g. switching between
different X sessions). I'm planning to do a bit more work on tuning up the
jack setup to see if I can get rid of the problem that way, but it would
be good to know in the meantime if this is a known bug, and whether there
is anything I can do about it.

It happened just now when I connected two other jack sources. There were
two xruns at the same time, of 0.8 and 1.8 msecs, so I'm guessing this
happens when there is an xrun. I've tried increasing the client timeout to
1000 msec, but this doesn't help. I can't increase the buffer size any
more - it's at 2048 and increasing it to 4096 stops ices from starting at
all.

The setup I'm working with is an ecasound session relaying audio between
its input and output ports, with its output connected to all the ices
clients for the different streams we're running. Then I'm
switching the ecasound input between different stream sources (line in,
recorded audio, http relay etc.) I want to be able to do this switching
without breaking the stream, which is why I've set things up this way.

Andy Baxter wrote:
> I've been having a problem where ices-kh (the jack'ified version)
> disconnects from its jack input source unexpectedly. This happens mainly
> while other jack clients are being started/stopped, or
> connected/disconnected, but also at other times (e.g. switching between
> different X sessions). I'm planning to do a bit more work on tuning up
the
> jack setup to see if I can get rid of the problem that way, but it would
> be good to know in the meantime if this is a known bug, and whether there
> is anything I can do about it.
> 
> It happened just now when I connected two other jack sources. There were
> two xruns at the same time, of 0.8 and 1.8 msecs, so I'm guessing this
> happens when there is an xrun. I've tried increasing the client timeout
to
> 1000 msec, but this doesn't help. I can't increase the buffer size
any
> more - it's at 2048 and increasing it to 4096 stops ices from starting
at
> all.
> 
> The setup I'm working with is an ecasound session relaying audio
between
> its input and output ports, with its output connected to all the ices
> clients for the different streams we're running. Then I'm
> switching the ecasound input between different stream sources (line in,
> recorded audio, http relay etc.) I want to be able to do this switching
> without breaking the stream, which is why I've set things up this way.
> 
This sounds like an incredible amount of effort when all you need is a 4 
channel mixing desk and a live encoder...

Is that not an option?

Stephen
--
LiveIce Project http://liveice.sourceforge.net/

On Mon, 2004-10-18 at 16:06, Andy Baxter wrote:
> I've been having a problem where ices-kh (the jack'ified version)
> disconnects from its jack input source unexpectedly. This happens mainly
> while other jack clients are being started/stopped, or
> connected/disconnected, but also at other times (e.g. switching between
> different X sessions). I'm planning to do a bit more work on tuning up
the
> jack setup to see if I can get rid of the problem that way, but it would
> be good to know in the meantime if this is a known bug, and whether there
> is anything I can do about it.
Are you running with realtime privileges, for this you need to start as
root if you want that. Even with realtime privileges there may be odd
cases where the scheduling latency is a bit too high, it all depends on
the drivers and kernel version but the current state is not that bad and
getting better.
> It happened just now when I connected two other jack sources. There were
> two xruns at the same time, of 0.8 and 1.8 msecs, so I'm guessing this
> happens when there is an xrun. I've tried increasing the client timeout
to
> 1000 msec, but this doesn't help. I can't increase the buffer size
any
> more - it's at 2048 and increasing it to 4096 stops ices from starting
at
> all.
xrun is just when the input thread cannot get scheduled quickly enough,
this may be resolved enabling realtime.
> The setup I'm working with is an ecasound session relaying audio
between
> its input and output ports, with its output connected to all the ices
> clients for the different streams we're running. Then I'm
> switching the ecasound input between different stream sources (line in,
> recorded audio, http relay etc.) I want to be able to do this switching
> without breaking the stream, which is why I've set things up this way.
As already mentioned it may be better using a different approach (eg
jackEQ). The ices package itself will also allow for defining multiple
input sections, not for mixing but for cascading in a round robin
fashion (switchable on input ending or USR2 signal).

karl.

On Mon, 18 Oct 2004 17:41:59 +0100, Karl Heyes wrote:
> On Mon, 2004-10-18 at 16:06, Andy Baxter wrote:
>> I've been having a problem where ices-kh (the jack'ified
version)
>> disconnects from its jack input source unexpectedly. This happens
mainly
>> while other jack clients are being started/stopped, or
>> connected/disconnected, but also at other times (e.g. switching between
>> different X sessions). I'm planning to do a bit more work on tuning
up the
>> jack setup to see if I can get rid of the problem that way, but it
would
>> be good to know in the meantime if this is a known bug, and whether
there
>> is anything I can do about it.
> 
> Are you running with realtime privileges, for this you need to start as
> root if you want that. Even with realtime privileges there may be odd
> cases where the scheduling latency is a bit too high, it all depends on
> the drivers and kernel version but the current state is not that bad and
> getting better.
I'm using the realtime-lsm module with the 2.6 kernel, loaded with gid=29,
which is the 'audio' group on my system. This seems to be working OK,
but
I'm thinking of switching to the 2.4 patched kernel instead.
>> It happened just now when I connected two other jack sources. There
were
>> two xruns at the same time, of 0.8 and 1.8 msecs, so I'm guessing
this
>> happens when there is an xrun. I've tried increasing the client
timeout to
>> 1000 msec, but this doesn't help. I can't increase the buffer
size any
>> more - it's at 2048 and increasing it to 4096 stops ices from
starting at
>> all.
> 
> xrun is just when the input thread cannot get scheduled quickly enough,
> this may be resolved enabling realtime.
OK.
>> The setup I'm working with is an ecasound session relaying audio
between
>> its input and output ports, with its output connected to all the ices
>> clients for the different streams we're running. Then I'm
>> switching the ecasound input between different stream sources (line in,
>> recorded audio, http relay etc.) I want to be able to do this switching
>> without breaking the stream, which is why I've set things up this
way.
> 
> As already mentioned it may be better using a different approach (eg
> jackEQ). The ices package itself will also allow for defining multiple
> input sections, not for mixing but for cascading in a round robin
> fashion (switchable on input ending or USR2 signal).
> 
maybe - if I can get it working, this setup would be good because it means
the sources can be switched under program control. (E.g. switching to a
repeat of the day's shows at the end of every day.) switching on USR2
sounds like a recipe for losing track of where it's got to in the round
robin, unless there's some way to signal it with a definite source name.

On Mon, 18 Oct 2004 16:15:44 +0100, EvilOverlord wrote:
>> The setup I'm working with is an ecasound session relaying audio
between
>> its input and output ports, with its output connected to all the ices
>> clients for the different streams we're running. Then I'm
>> switching the ecasound input between different stream sources (line in,
>> recorded audio, http relay etc.) I want to be able to do this switching
>> without breaking the stream, which is why I've set things up this
way.
>> 
> 
> This sounds like an incredible amount of effort when all you need is a 4 
> channel mixing desk and a live encoder...
> 
> Is that not an option?
> 
Maybe, but I'd rather try and get the system I thought of working before I
try something else. thanks.

Andy Baxter wrote:
> The setup I'm working with is an ecasound session relaying audio
between
> its input and output ports, with its output connected to all the ices
> clients for the different streams we're running.
This doesn't address the issue you're asking about, but why are you 
running multiple Ices instances when a single instance can produce multiple 
streams?

Geoff.

On Tue, 19 Oct 2004 10:57:12 +1000, Geoff Shang wrote:
> Andy Baxter wrote:
> 
>> The setup I'm working with is an ecasound session relaying audio
between
>> its input and output ports, with its output connected to all the ices
>> clients for the different streams we're running.
> 
> This doesn't address the issue you're asking about, but why are you
> running multiple Ices instances when a single instance can produce multiple
> streams?
> 
> Geoff.
Mainly I didn't take enough time to think through different ways of doing
it before I started writing the scripts. Also there might be times when it
would be good to be able to stop some streams but not others. I think I
was trying to design the system to cover all eventualities in
advance rather than waiting for them to come up and dealing with them then.

Karl Heyes <karl@xiph.org> writes:
> Are you running with realtime privileges, for this you need to start as
> root if you want that. Even with realtime privileges there may be odd
> cases where the scheduling latency is a bit too high, it all depends on
> the drivers and kernel version but the current state is not that bad and
> getting better.
There are several ways to gain realtime privileges.  Running as root
is the simplest, but perhaps the worst solution.  

This is explained in the JACK FAQ...

  http://jackit.sourceforge.net/docs/faq.php#a52
-- 
  joq

On Mon, 18 Oct 2004 16:06:00 +0100, Andy Baxter wrote:
> I've been having a problem where ices-kh (the jack'ified version)
> disconnects from its jack input source unexpectedly. This happens mainly
> while other jack clients are being started/stopped, or
> connected/disconnected, but also at other times (e.g. switching between
> different X sessions). I'm planning to do a bit more work on tuning up
the
> jack setup to see if I can get rid of the problem that way, but it would
> be good to know in the meantime if this is a known bug, and whether there
> is anything I can do about it.
> 
> It happened just now when I connected two other jack sources. There were
> two xruns at the same time, of 0.8 and 1.8 msecs, so I'm guessing this
> happens when there is an xrun. I've tried increasing the client timeout
to
> 1000 msec, but this doesn't help. I can't increase the buffer size
any
> more - it's at 2048 and increasing it to 4096 stops ices from starting
at
> all.
> 
> The setup I'm working with is an ecasound session relaying audio
between
> its input and output ports, with its output connected to all the ices
> clients for the different streams we're running. Then I'm
> switching the ecasound input between different stream sources (line in,
> recorded audio, http relay etc.) I want to be able to do this switching
> without breaking the stream, which is why I've set things up this way.
I've been testing this some more and the ices logs are showing a
ringbuffer full error at the time the port drops. I looked through the
source and the 'sleep time' parameter seems to have something to do with
setting the ring buffer size. I've tried recompiling the im_jack module
with the line:

rb_size=(size_t)((s->sleep / 2000.0 ) * ......

changed to ... s->sleep / 1000.0

I'm running another test at the moment, but it would be good to know
whether this is actually likely to help, and if it's going to have any
other effects on the way ices works (apart from increasing the stream
latency.)

I'll let you know how the test goes.

On Sun, 31 Oct 2004 22:32:17 +0000, andy wrote:
> On Mon, 18 Oct 2004 16:06:00 +0100, Andy Baxter wrote:
> 
>> I've been having a problem where ices-kh (the jack'ified
version)
>> disconnects from its jack input source unexpectedly. This happens
mainly
>> while other jack clients are being started/stopped, or
>> connected/disconnected, but also at other times (e.g. switching between
>> different X sessions). I'm planning to do a bit more work on tuning
up the
>> jack setup to see if I can get rid of the problem that way, but it
would
>> be good to know in the meantime if this is a known bug, and whether
there
>> is anything I can do about it.
>> 
>> It happened just now when I connected two other jack sources. There
were
>> two xruns at the same time, of 0.8 and 1.8 msecs, so I'm guessing
this
>> happens when there is an xrun. I've tried increasing the client
timeout to
>> 1000 msec, but this doesn't help. I can't increase the buffer
size any
>> more - it's at 2048 and increasing it to 4096 stops ices from
starting at
>> all.
>> 
>> The setup I'm working with is an ecasound session relaying audio
between
>> its input and output ports, with its output connected to all the ices
>> clients for the different streams we're running. Then I'm
>> switching the ecasound input between different stream sources (line in,
>> recorded audio, http relay etc.) I want to be able to do this switching
>> without breaking the stream, which is why I've set things up this
way.
> 
> I've been testing this some more and the ices logs are showing a
> ringbuffer full error at the time the port drops. I looked through the
> source and the 'sleep time' parameter seems to have something to do
with
> setting the ring buffer size. I've tried recompiling the im_jack module
> with the line:
> 
> rb_size=(size_t)((s->sleep / 2000.0 ) * ......
> 
> changed to ... s->sleep / 1000.0
> 
> I'm running another test at the moment, but it would be good to know
> whether this is actually likely to help, and if it's going to have any
> other effects on the way ices works (apart from increasing the stream
> latency.)
> 
> I'll let you know how the test goes.
I just had a look at it and it's still dropping ports - I set a test
running which has ices-kh running from an ecasound process which is
connected to a new source every 10 minutes, and after about 3 hours it had
lost the jack port connection to ecasound.

this is the bit from the log:

2004-11-01  01:28:09] DBUG stream/_output_oggpacket seen new stream, better get
headers
[2004-11-01  01:28:09] DBUG stream/_output_oggpacket Clearing output
info/comment settings
[2004-11-01  01:28:09] DBUG stream/_output_oggpacket samplerate is 44100,
channels is 2
[2004-11-01  01:28:09] DBUG om_shout/output_ogg_shout initialising output stream
[2004-11-01  01:44:05] EROR input-jack/jack_callback_process ringbuffer full
[2004-11-01  01:44:05] INFO input/input_loop Closing input module
[2004-11-01
  01:44:05] DBUG input-jack/jack_close_module closing JACK module
[2004-11-01  01:44:05] DBUG input/open_next_input_module checking module 1
[2004-11-01  01:44:05] INFO input-jack/jack_open_module Registering as
ices64:in_1,ices64:in_2

[2004-11-01  01:44:05] DBUG input-jack/jack_open_module creating 2 ringbuffers,
one per channel, of 81920 bytes each
[2004-11-01  01:44:05] INFO input-jack/jack_open_module Channels 2 / Samplerate
48000
[2004-11-01  01:44:05] DBUG input-jack/jack_read metadata updates flagged
[2004-11-01  01:44:05] INFO metadata/metadata_thread_signal Updating metadata
with 9 comments

I think the problem is that ices is losing the jack connection because of
an xrun, trying to recover by recreating the jack port, but then ices
doesn't know which other clients were connected to it, so when the new
port is created it has no connections.  I could get round this by using
jack_lsp to monitor the ports and reconnect ices when it drops, but I
would rather not do that if there's a better way to do it. Another way
might be to be able to tell the ices jack client which ports it should
connect to on startup, and then have it
reconnect itself when it makes the new port? If you have any idea why ices
is more sensitive to xruns than the other jack programs I'm using, that
would be a help as well.

I've just tried rewriting the im-jack module so it doesn't shutdown when
the ringbuffer fills up (but still logs the error), which ought to solve
the problem (I'd rather have occasional breaks in the audio than lose
the connection altogether) but maybe isn't the best way to do things.