On 2017-03-03 13:37, Marvin Scholz wrote:> >> I'd like to suggest adding history to status-json.xsl or to reduce >> the size of the json to the bare minimum that a Web base mediaplayer >> might need >> Add a new one at http://127.0.0.1:8000/playing-json.xsl?mount=/live >> Which will only hold the current info: > > Right now I personally don't think there should be more .xsl json > things be added, as it's just a transform from XML to JSON and > has proven to sometimes cause some weird bugs with malformed json in > the past, so proper JSON generation capabilities in Icecast > would be preferred.In that case I'd like to suggest http://127.0.0.1:8000/playing.json?mount=/live> >> [example]... > > More suitable would it be, to use the STATS streaming API that Icecast > offers, server side, and then use a websocket connection to update the > players.Um. Where is this documented? I took a look at http://icecast.org/docs/icecast-trunk/server_stats/ but can't find it (I'm probably looking in the wrong place, and if this is supported in Icecast 2.4 then shame on me for not knowing). And does that require authentication? Since (in my case) the caching of the info for the webplayer(s) is done using a plain PHP script I'd rater not plop password/credentials in it. Also websockets are not the best choice for PHP scripts etc. The way it all works is like this: The webplayer requests the url (PHP script) and get back the JSON data. However the script will do one of two things, if the cache is not stale it will provide the cached JSON data, if he cache is still it will request it from the streaming server (Icecast/Shoutcast etc), there is some delay while this occurs but the next webplayer will get the cached JSON instead. This solution works great on most hosting systems. Websocket would require a always running service. The benefit there obviously is that the service would generate the json file and the webserver (or possibly a CDN) would just server a static JSON file with no script overhead. But PHP scripts can not normally run continuously/never ending (in theory they can but no webservers allow that normally). I'm certainly interested in learning more about STATS streaming API (it's news to me).> >> The server_name and server_description is included since those could >> change between DJs (and are nice to display in the webplayer to the >> listener). >> The date for each song is somewhat useful for a webplayer, not only >> can a webplayer show the start time for each song to the listener but >> (with the exeception of the current song) it can calculate and show >> the duration of songs. > > While this in theory works, it requires that you do _not_ use ICY > metadata (which is used for MP3 and AAC), as ICY metadata can't be > very accurate, > due to the way ICY works.Not sure why you are mentioning ICY data? The source sends a title update to the stream server, the script (or webplayer) fetches the json data fro the stream server. Not sure how the Shoutcast protocol is involved with this? Also no metadata (ICY or not) can be reliable. For example the station I'm the tech guy at has live playlist and live DJs. The accuracy of titles is beholden to whatever source client is used. Also, it is possible that such is time delayed on purpose to mess wit stream rippers so it will obviously not accurate to the second. But for displaying to a listener as long as it's not more than half a minute off is OK enough, the test setup I have here is accurate to within a few seconds which makes calculating a duration in the webplayer actually useful.> >> With Icecast there is luckily no need to get a server datetime from >> anywhere else as Icecast has proper HTTP headers, but providing the >> date in the JSON would be easier to code in a script/webplayer, HTTP >> headers can be a bit fiddly, although with a server side script PHP >> or something else should have no issues handling the HTTP Date value, >> but still providing the date in the json simplifies the >> implementation (no parsing needed, just passthrough and cache >> basically). > > Using the Date headers is the right thing to do.But it requires some fiddling around with date reformatting from HTTP date to the one shown in the example JSON, if Icecast provided the server date in it's JSON then that is one less piece of code needed in the script/webplayers. There also seems to be some issues with getting the Date header when doing XHR in javascript (possibly CORS related). Again, it sounds lazy but having less "end programmer" coded needed would be a plus.> >> I believe the lack of a song history has been pointed out before, now >> with Webplayers possible (which do not support in-stream metadata >> "out-of-the-box") this is very useful. If Shoutcast v2 had a proper >> server date provided it would still be two calls (to get the history >> and the stream name/description). Icecast can easily do this better, >> right? > > When streaming in modern formats, like Ogg or WebM, browser players > should support metadata.I need to look into that deeper (for webplayers) then, I'm currently working on adding Ogg (and possibly WebM) metadata support for a Windows based player I'm maintaining so if browsers can provide that info to javascript then that is awesome. BTW! Does this require CORS or is the metadata available without CORS?> > Could you maybe register at trac.xiph.org and create a feature request > there with the information you included in this E-mail? > (If you do, don't forget to choose Icecast as the component)Sure, once you answer this email and clarify/answer some of my questions I'll re-think my feature request if needed and then create it. -- Roger Hågensen, Freelancer, Norway.
On 3 Mar 2017, at 14:53, Roger Hågensen wrote:> On 2017-03-03 13:37, Marvin Scholz wrote: >> >>> I'd like to suggest adding history to status-json.xsl or to reduce >>> the size of the json to the bare minimum that a Web base mediaplayer >>> might need >>> Add a new one at http://127.0.0.1:8000/playing-json.xsl?mount=/live >>> Which will only hold the current info: >> >> Right now I personally don't think there should be more .xsl json >> things be added, as it's just a transform from XML to JSON and >> has proven to sometimes cause some weird bugs with malformed json in >> the past, so proper JSON generation capabilities in Icecast >> would be preferred. > In that case I'd like to suggest > http://127.0.0.1:8000/playing.json?mount=/live >Well that's not really the relevant thing that I was trying to say, the problem is that Icecast currently has no way to natively generate JSON and adding this is not trivial, given that all internals use XML right now.>> >>> [example]... >> >> More suitable would it be, to use the STATS streaming API that >> Icecast offers, server side, and then use a websocket connection to >> update the players. > > Um. Where is this documented? I took a look at > http://icecast.org/docs/icecast-trunk/server_stats/ but can't find it > (I'm probably looking in the wrong place, and if this is supported in > Icecast 2.4 then shame on me for not knowing).This is supported since a very long time, but not documented in a good way anywhere, unfortunately. And the output format of STATS is a bit weird too.> And does that require authentication? Since (in my case) the caching > of the info for the webplayer(s) is done using a plain PHP script I'd > rater not plop password/credentials in it.Yes, right now it still does, for 2.5 it was planned to allow different auth just for STATS, so that no admin credentials would be required for it.> > Also websockets are not the best choice for PHP scripts etc. The way > it all works is like this: > The webplayer requests the url (PHP script) and get back the JSON > data. However the script will do one of two things, if the cache is > not stale it will provide the cached JSON data, if he cache is still > it will request it from the streaming server (Icecast/Shoutcast etc), > there is some delay while this occurs but the next webplayer will get > the cached JSON instead.I agree that websockets is not really easy to do with PHP.> > This solution works great on most hosting systems. > Websocket would require a always running service. The benefit there > obviously is that the service would generate the json file and the > webserver (or possibly a CDN) would just server a static JSON file > with no script overhead. > But PHP scripts can not normally run continuously/never ending (in > theory they can but no webservers allow that normally). > > I'm certainly interested in learning more about STATS streaming API > (it's news to me). > >> >>> The server_name and server_description is included since those could >>> change between DJs (and are nice to display in the webplayer to the >>> listener). >>> The date for each song is somewhat useful for a webplayer, not only >>> can a webplayer show the start time for each song to the listener >>> but (with the exeception of the current song) it can calculate and >>> show the duration of songs. >> >> While this in theory works, it requires that you do _not_ use ICY >> metadata (which is used for MP3 and AAC), as ICY metadata can't be >> very accurate, >> due to the way ICY works. > > Not sure why you are mentioning ICY data? The source sends a title > update to the stream server, the script (or webplayer) fetches the > json data fro the stream server. Not sure how the Shoutcast protocol > is involved with this?Because "The source sends a title update to the stream server" is not how things should be done, usually. This is a "hack" for MP3 and AAC, introduced for the ICY protocol to work.> > Also no metadata (ICY or not) can be reliable.With Ogg it can be very reliable, at least timing-wise. Everything else depends on correct metadata of the source files, of course.> For example the station I'm the tech guy at has live playlist and live > DJs. The accuracy of titles is beholden to whatever source client is > used. Also, it is possible that such is time delayed on purpose to > mess wit stream rippers so it will obviously not accurate to the > second. But for displaying to a listener as long as it's not more than > half a minute off is OK enough, the test setup I have here is accurate > to within a few seconds which makes calculating a duration in the > webplayer actually useful. > >> >>> With Icecast there is luckily no need to get a server datetime from >>> anywhere else as Icecast has proper HTTP headers, but providing the >>> date in the JSON would be easier to code in a script/webplayer, HTTP >>> headers can be a bit fiddly, although with a server side script PHP >>> or something else should have no issues handling the HTTP Date >>> value, but still providing the date in the json simplifies the >>> implementation (no parsing needed, just passthrough and cache >>> basically). >> >> Using the Date headers is the right thing to do. > > But it requires some fiddling around with date reformatting from HTTP > date to the one shown in the example JSON, if Icecast provided the > server date in it's JSON then that is one less piece of code needed in > the script/webplayers. There also seems to be some issues with getting > the Date header when doing XHR in javascript (possibly CORS related). > Again, it sounds lazy but having less "end programmer" coded needed > would be a plus.Just because something requires more code because Javascript XHR is crap, I don't see that as reason why we should include the Date in the JSON, that would make caching the JSON impossible for Icecast.>> >>> I believe the lack of a song history has been pointed out before, >>> now with Webplayers possible (which do not support in-stream >>> metadata "out-of-the-box") this is very useful. If Shoutcast v2 had >>> a proper server date provided it would still be two calls (to get >>> the history and the stream name/description). Icecast can easily do >>> this better, right?It's of course possible to include the stream name/desc in the JSON, yes.>> >> When streaming in modern formats, like Ogg or WebM, browser players >> should support metadata. > > I need to look into that deeper (for webplayers) then, I'm currently > working on adding Ogg (and possibly WebM) metadata support for a > Windows based player I'm maintaining so if browsers can provide that > info to javascript then that is awesome. > BTW! Does this require CORS or is the metadata available without CORS? > >> >> Could you maybe register at trac.xiph.org and create a feature >> request there with the information you included in this E-mail? >> (If you do, don't forget to choose Icecast as the component) > > Sure, once you answer this email and clarify/answer some of my > questions I'll re-think my feature request if needed and then create > it. > > > -- > Roger Hågensen, > Freelancer, Norway. > > _______________________________________________ > Icecast-dev mailing list > Icecast-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/icecast-dev
On 2017-03-03 15:43, Marvin Scholz wrote:> On 3 Mar 2017, at 14:53, Roger Hågensen wrote: > >> In that case I'd like to suggest >> http://127.0.0.1:8000/playing.json?mount=/live > > Well that's not really the relevant thing that I was trying to say, > the problem is that Icecast currently has no way to natively generate > JSON > and adding this is not trivial, given that all internals use XML right > now.Ah! Gotcha.> >>> More suitable would it be, to use the STATS streaming API that >>> Icecast offers, server side, and then use a websocket connection to >>> update the players. >> Um. Where is this documented? I took a look at >> http://icecast.org/docs/icecast-trunk/server_stats/ but can't find it >> (I'm probably looking in the wrong place, and if this is supported in >> Icecast 2.4 then shame on me for not knowing). > > This is supported since a very long time, but not documented in a good > way anywhere, unfortunately. And the output format of STATS is a bit > weird too.I'll just have to test and maybe peek in source code then.> >> And does that require authentication? Since (in my case) the caching >> of the info for the webplayer(s) is done using a plain PHP script I'd >> rater not plop password/credentials in it. > > Yes, right now it still does, for 2.5 it was planned to allow > different auth just for STATS, so that no admin credentials would be > required for it.But that would still require some form of authentication? Right now the station I do tech stuff for ran into a situation where "Bose Monitoring Service" showed up as a listener useragent, they are basically tuned in to the stream and fetching metadata from shoutcast. What they Exactly do with this data I have no idea but they provide it to the listening devices user use. This was a issue, not only did they take up a listner slot but royalties for all the songs would be charged as well since this is a 24/7 listener even if it's not a listener. I exchanged a few emails with them and informed them about the JSON stuff Shoutcast v2 servers had and how to get the history info. But I had to say that there was no way to get the same info from Icecast in a simple way that I was aware of. With authentication you can get more info but why would my station hand admin access to Bose? And Bose is not unique in this. Over the years when I had to lookup info on retrieving current song/history etc I often see someone suggesting parsing the audio stream. And my first though is that these must be amateurs that can thing straight. (a simple XML or JSON request if gzip compression is applied could fit in a single TCP packet or two, compared to a stream) Basically Bose did a very un-clever thing and consumes almost 1GB of data to do what a simple HTTP fetch could do once per 30 minutes, which would be barely a blip o the daily bandwidth use), and since they probably have thousands of users/devices and there are tens of thousands of stations that's a lot of Bose Monitoring Service's running out there. Bose responded back and Said they will change the way they fetch this info but they can't give any ETA on that. Which gave me the idea that hey Icecast could provide public history info like Shoutcast 2 does, only you know, better. I'm in the process of re-designing the webplayer for our station and though two birds one hand (or however that expression goes).> >> Not sure why you are mentioning ICY data? The source sends a title >> update to the stream server, the script (or webplayer) fetches the >> json data fro the stream server. Not sure how the Shoutcast protocol >> is involved with this? > > Because "The source sends a title update to the stream server" is not > how things should be done, usually. This is a "hack" for MP3 and AAC, > introduced for the ICY protocol to work. > >> >> Also no metadata (ICY or not) can be reliable. > > With Ogg it can be very reliable, at least timing-wise. Everything > else depends on correct metadata of the source files, of course.I'll make doubly sure that my own stream player, as well as a Source program I'm making do it the proper way then. I'm aware of the hack nature of the GET based title update, I once experience two source clients (both Shoutcast Source DSPs) where one was a Live DJ and the other a playlist source but it bugged out for some reason and kept sending title updates even though it wasn't the connected stream, causing wrong titles shown for the live DJ, how fun. I'm assuming just standard Ogg (and similar for WebM) meta sidestream is OK?> >> But it requires some fiddling around with date reformatting from HTTP >> date to the one shown in the example JSON, if Icecast provided the >> server date in it's JSON then that is one less piece of code needed >> in the script/webplayers. There also seems to be some issues with >> getting the Date header when doing XHR in javascript (possibly CORS >> related). Again, it sounds lazy but having less "end programmer" >> coded needed would be a plus. > > Just because something requires more code because Javascript XHR is > crap, I don't see that as reason why we should include the Date in the > JSON, that would make caching the JSON impossible for Icecast.Ah. I forgot about that, very good point. (I didn't know Icecast cached the info, I assumed it generated it live).> >>> >>>> I believe the lack of a song history has been pointed out before, >>>> now with Webplayers possible (which do not support in-stream >>>> metadata "out-of-the-box") this is very useful. If Shoutcast v2 had >>>> a proper server date provided it would still be two calls (to get >>>> the history and the stream name/description). Icecast can easily do >>>> this better, right? > > It's of course possible to include the stream name/desc in the JSON, yes.Thanks. I'll re-think the feature request then reg/submit it later today (I certainly will drop the date idea). -- Roger Hågensen, Freelancer, Norway.
On 2017-03-03 15:43, Marvin Scholz wrote:>>> Could you maybe register at trac.xiph.org and create a feature >>> request there with the information you included in this E-mail? >>> (If you do, don't forget to choose Icecast as the component)Done! https://trac.xiph.org/ticket/2313 I broadened the request to include XML as well. -- Roger Hågensen, Freelancer, Norway.