Good afternoon Philipp Many thanks for your reply. Sorry for not being clear. I think the problem I have would be in the implementation. How do I run two versions of Icecast on two servers, with load balancing between the two (perhaps using RR-DNS), but I present my client with one unified log file for the audience statistics? Should the two servers write their logs via a network file share to a third logging server, for example? With many thanks and best regards Chip Scooter On Mon, 13 Jan 2020 at 13:36, Philipp Schafft <phschafft at de.loewenfelsen.net> wrote:> Good afternoon, > > On Mon, 2020-01-13 at 13:30 +0000, Chip wrote: > > Hi > > > > I have a potential project for which my client requests that we load > > balance the streaming service. > > > > Of course, the Icecast server scales very well. > > > > - http://icecast.org/loadtest/ > > > > However, the client requests high-availability and, due to the scale of > the > > potential project, we would like to load balance the service over two or > > more servers. > > > > I think the load balancing aspect is not my major issue at this stage but > > how to aggregate the listener logs in one place so that my client has an > > accurate representation of the audience stats. > > > > Hope someone can help and many thanks in advance > > I hardly see any question in your mail so it is a bit hard to answer. > What exactly is the problem you face? Is it finding a good architecture > for your setup or is it in actually implementing it? > > Please feel free to reply to me off-list for professional Icecast > support. > > With best regards, > > > -- > Philipp Schafft (CEO/Geschäftsführer) > Telephon: +49.3535 490 17 92 > > Löwenfelsen UG (haftungsbeschränkt) Registration number: > Bickinger Straße 21 HRB 12308 CB > 04916 Herzberg (Elster) VATIN/USt-ID: > Germany DE305133015 >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/icecast/attachments/20200113/cba76c0f/attachment.html>
Good afternoon, On Mon, 2020-01-13 at 13:41 +0000, Chip wrote:> Good afternoon Philipp > > Many thanks for your reply.> Sorry for not being clear. > > I think the problem I have would be in the implementation.Ok.> How do I run two versions of Icecast on two servers, with load balancing > between the two (perhaps using RR-DNS),Generally I recommend RR-DNS as it is a very good solution for most problems. Even if something more advanced is used it is a layer that should be added. If there are no special requirements, it's the way to go.> but I present my client with one unified log file for the audience > statistics?> Should the two servers write their logs via a network file share to a > third logging server, for example?I would recommend against that. It just adds another single point of failure. Most software I know for that kind of analysis can in fact read multiple files anyway. (e.g. some wants to import the data into it's own database first, other just want to support log rotation.) So it depends on what software you want to use. Maybe it can already read multiple log files. If not you could join the files. Check if the software will be happy with a file that is basically 'cat server-A.log server-B.log > both.log'. That should be the case as the log lines are not guaranteed to be in chronological order anyway (to support multithreading). If it does not work this way some more advanced solution must be found. But I think this should be possible. You could implement cat-ing the logs together as part of logrotate: rotate the log and send it to a common server for arching and processing. That way the servers themself could also keep a copy in case that processing server can not be reached (e.g. for maintenance). For live statistics consider using the STATS interface. With best regards,> On Mon, 13 Jan 2020 at 13:36, Philipp Schafft <phschafft at de.loewenfelsen.net> > wrote: > > > Good afternoon, > > > > On Mon, 2020-01-13 at 13:30 +0000, Chip wrote: > > > Hi > > > > > > I have a potential project for which my client requests that we load > > > balance the streaming service. > > > > > > Of course, the Icecast server scales very well. > > > > > > - http://icecast.org/loadtest/ > > > > > > However, the client requests high-availability and, due to the scale of > > the > > > potential project, we would like to load balance the service over two or > > > more servers. > > > > > > I think the load balancing aspect is not my major issue at this stage but > > > how to aggregate the listener logs in one place so that my client has an > > > accurate representation of the audience stats. > > > > > > Hope someone can help and many thanks in advance > > > > I hardly see any question in your mail so it is a bit hard to answer. > > What exactly is the problem you face? Is it finding a good architecture > > for your setup or is it in actually implementing it? > > > > Please feel free to reply to me off-list for professional Icecast > > support.-- Philipp Schafft (CEO/Geschäftsführer) Telephon: +49.3535 490 17 92 Löwenfelsen UG (haftungsbeschränkt) Registration number: Bickinger Straße 21 HRB 12308 CB 04916 Herzberg (Elster) VATIN/USt-ID: Germany DE305133015 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 490 bytes Desc: This is a digitally signed message part URL: <http://lists.xiph.org/pipermail/icecast/attachments/20200113/0a3a3b6a/attachment.sig>
Thank you very much Philipp. I will think this over. If I have any problems, I'll get back to you. With best regards for now Chip Scooter On Mon, 13 Jan 2020 at 14:01, Philipp Schafft <phschafft at de.loewenfelsen.net> wrote:> Good afternoon, > > On Mon, 2020-01-13 at 13:41 +0000, Chip wrote: > > Good afternoon Philipp > > > > Many thanks for your reply. > > > Sorry for not being clear. > > > > I think the problem I have would be in the implementation. > > Ok. > > > > How do I run two versions of Icecast on two servers, with load balancing > > between the two (perhaps using RR-DNS), > > Generally I recommend RR-DNS as it is a very good solution for most > problems. Even if something more advanced is used it is a layer that > should be added. > > If there are no special requirements, it's the way to go. > > > > but I present my client with one unified log file for the audience > > statistics? > > > Should the two servers write their logs via a network file share to a > > third logging server, for example? > > I would recommend against that. It just adds another single point of > failure. Most software I know for that kind of analysis can in fact read > multiple files anyway. (e.g. some wants to import the data into it's own > database first, other just want to support log rotation.) So it depends > on what software you want to use. Maybe it can already read multiple log > files. > > If not you could join the files. Check if the software will be happy > with a file that is basically 'cat server-A.log server-B.log > > both.log'. That should be the case as the log lines are not guaranteed > to be in chronological order anyway (to support multithreading). > > If it does not work this way some more advanced solution must be found. > But I think this should be possible. > > You could implement cat-ing the logs together as part of logrotate: > rotate the log and send it to a common server for arching and > processing. That way the servers themself could also keep a copy in case > that processing server can not be reached (e.g. for maintenance). > > For live statistics consider using the STATS interface. > > With best regards, > > > > > On Mon, 13 Jan 2020 at 13:36, Philipp Schafft < > phschafft at de.loewenfelsen.net> > > wrote: > > > > > Good afternoon, > > > > > > On Mon, 2020-01-13 at 13:30 +0000, Chip wrote: > > > > Hi > > > > > > > > I have a potential project for which my client requests that we load > > > > balance the streaming service. > > > > > > > > Of course, the Icecast server scales very well. > > > > > > > > - http://icecast.org/loadtest/ > > > > > > > > However, the client requests high-availability and, due to the scale > of > > > the > > > > potential project, we would like to load balance the service over > two or > > > > more servers. > > > > > > > > I think the load balancing aspect is not my major issue at this > stage but > > > > how to aggregate the listener logs in one place so that my client > has an > > > > accurate representation of the audience stats. > > > > > > > > Hope someone can help and many thanks in advance > > > > > > I hardly see any question in your mail so it is a bit hard to answer. > > > What exactly is the problem you face? Is it finding a good architecture > > > for your setup or is it in actually implementing it? > > > > > > Please feel free to reply to me off-list for professional Icecast > > > support. > > > -- > Philipp Schafft (CEO/Geschäftsführer) > Telephon: +49.3535 490 17 92 > > Löwenfelsen UG (haftungsbeschränkt) Registration number: > Bickinger Straße 21 HRB 12308 CB > 04916 Herzberg (Elster) VATIN/USt-ID: > Germany DE305133015 >-------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/icecast/attachments/20200113/ad362eb9/attachment.html>
Many thanks for this Philipp. On Mon, 13 Jan 2020 at 14:01, Philipp Schafft <phschafft at de.loewenfelsen.net> wrote:> I would recommend against that. It just adds another single point of > failure. Most software I know for that kind of analysis can in fact read > multiple files anyway. (e.g. some wants to import the data into it's own > database first, other just want to support log rotation.) So it depends > on what software you want to use. Maybe it can already read multiple log > files. >May I ask which applications do you know of that can aggregate and interpret Icecast log files from multiple servers in this way? Free and open source solutions, of course, are good but I need not be restricted to those in this regard. With many thanks and best regards Chip Scooter -------------- next part -------------- An HTML attachment was scrubbed... URL: <http://lists.xiph.org/pipermail/icecast/attachments/20200114/a75a896e/attachment.html>