Hi all, 1. I recently noticed that yp.xml is still limited to 1,000 entries only while the web directory says there are over 21K streams. I remember some discussions regarding the bandwidth required to serve this file, so is it possible to get it in compressed format? Here are some figures: - the yp.xml as served now (1K entries) is 343 KB. This means 21K entries will be around 7 MB. - compressing yp.xml with gzip (default options on Linux, not even -9) gives a yp.xml.gz of 14 KB. This means 21K compressed entries will be less than 300 KB, or less than the current uncompressed 1K! gzip is fast enough to be used on-the-fly (bzip2 gives even smaller size, around 8 KB for 1K entries, but is an order of magnitude slower). If CPU usage is a concern, then to avoid on-the-fly compression, yp.xml can be compressed, say, once every minute, cached and served from the cache. 2. Also, I noticed at hat all 1K entries in yp.xml right now come from the same web site, radionomy.com. It claims ti has over 3K streams. However, there are some issues with their entries which make them nearly useless: - all of their streams are listed with the "server_name" tag set to "Unspecified name". Having 3,000 stations named "Unspecified name" is not very useful. Every station at radionomy.com has a distinguished "name" which is actually part of the URL - could they send this actual name as "server_name"? - all of their streams have the "genre" tag set to "various". Again, not very helpful; their web site says there are clearly different genres, about a dozen, and each station belongs to one; why cannot they send the proper genre name? The only email I could find on their web site is faq at radionomy.com, all other contact info is in the form of web forms. If I could be of help on any of these two topic, please, let me know. With my best regards, Assen Totin -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast-dev/attachments/20120710/55551ce8/attachment.htm
Lol well good luck with this I have been asking for this for well over a year I don't know if anyone even reads this list man... Certainly nobody who has access to make the 5 - 10 minute change to the yp.xml feed is responding anyway Best Victor On Jul 10, 2012 11:53 AM, "Assen Totin" <assen.totin at gmail.com> wrote:> Hi all, > > 1. I recently noticed that yp.xml is still limited to 1,000 entries only > while the web directory says there are over 21K streams. I remember some > discussions regarding the bandwidth required to serve this file, so is it > possible to get it in compressed format? Here are some figures: > > - the yp.xml as served now (1K entries) is 343 KB. This means 21K entries > will be around 7 MB. > > - compressing yp.xml with gzip (default options on Linux, not even -9) > gives a yp.xml.gz of 14 KB. This means 21K compressed entries will be less > than 300 KB, or less than the current uncompressed 1K! gzip is fast enough > to be used on-the-fly (bzip2 gives even smaller size, around 8 KB for 1K > entries, but is an order of magnitude slower). If CPU usage is a concern, > then to avoid on-the-fly compression, yp.xml can be compressed, say, once > every minute, cached and served from the cache. > > 2. Also, I noticed at hat all 1K entries in yp.xml right now come from the > same web site, radionomy.com. It claims ti has over 3K streams. However, > there are some issues with their entries which make them nearly useless: > > - all of their streams are listed with the "server_name" tag set to > "Unspecified name". Having 3,000 stations named "Unspecified name" is not > very useful. Every station at radionomy.com has a distinguished "name" > which is actually part of the URL - could they send this actual name as > "server_name"? > > - all of their streams have the "genre" tag set to "various". Again, not > very helpful; their web site says there are clearly different genres, about > a dozen, and each station belongs to one; why cannot they send the proper > genre name? > > The only email I could find on their web site is faq at radionomy.com, all > other contact info is in the form of web forms. > > If I could be of help on any of these two topic, please, let me know. > > With my best regards, > > Assen Totin > > > > > _______________________________________________ > Icecast-dev mailing list > Icecast-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/icecast-dev > >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast-dev/attachments/20120711/08886362/attachment.htm
Hi everyone, On 10/07/12 18:52, Assen Totin wrote:> 1. I recently noticed that yp.xml is still limited to 1,000 entries > only while the web directory says there are over 21K streams. I > remember some discussions regarding the bandwidth required to serve > this file, so is it possible to get it in compressed format? Here are > some figures: > > - the yp.xml as served now (1K entries) is 343 KB. This means 21K > entries will be around 7 MB. > > - compressing yp.xml with gzip (default options on Linux, not even > -9) gives a yp.xml.gz of 14 KB. This means 21K compressed entries > will be less than 300 KB, or less than the current uncompressed 1K! > gzip is fast enough to be used on-the-fly (bzip2 gives even smaller > size, around 8 KB for 1K entries, but is an order of magnitude > slower). If CPU usage is a concern, then to avoid on-the-fly > compression, yp.xml can be compressed, say, once every minute, cached > and served from the cache.We're aware of those issues and as much as we wanted to see this resolved earlier it was not possible as we rely on server and foremost bandwidth capacity provided to us for free. I don't have the time to venture into details right now, but just to give you an idea of the scale of this unusual setup: - hundreds of hits per second to apache - hundreds of queries per second to the database - transient database in RAM - egress of about 2Megabit per second, 24/7 That said we're currently working on moving hosting of dir.xiph.org back to xiph servers as we have now the capacity for this and also want to resolve the problem with the limit. Along the way we'll most likely implement further changes to the service. If you are maintaining software that accesses dir.xiph.org/yp.xml - please stay tuned for updates!> 2. Also, I noticed at hat all 1K entries in yp.xml right now come from > the same web site, radionomy.com <http://radionomy.com>. It claims ti > has over 3K streams. However, there are some issues with their entries > which make them nearly useless: > > - all of their streams are listed with the "server_name" tag set to > "Unspecified name". Having 3,000 stations named "Unspecified name" is > not very useful. Every station at radionomy.com <http://radionomy.com> > has a distinguished "name" which is actually part of the URL - could > they send this actual name as "server_name"? > > - all of their streams have the "genre" tag set to "various". Again, > not very helpful; their web site says there are clearly different > genres, about a dozen, and each station belongs to one; why cannot > they send the proper genre name? > > The only email I could find on their web site is faq at radionomy.com > <mailto:faq at radionomy.com>, all other contact info is in the form of > web forms.That is indeed 'fascinating'. We'll have to have a closer look after the transition. Thanks for bringing this up! Cheers Thomas
Hi,
I've just updated ticket 1890 - https://trac.xiph.org/ticket/1890#comment:8
==As we have moved to the Xiph server at OSUOSL - we have now disabled
this limitation and will observe the situation. Personally I'm very
disappointed that most clients downloading the yp.xml are not accepting
compressed encoding. Right now the file is 4818606 Bytes in size, most
clients download it uncompressed, but when transferred using Firefox
will only use 436859(sic!) Bytes of actual traffic!
Please encourage client software projects to fix their clients! (see
also RFC 2616 section 3.5)
Possible outcomes are that we will have to:
* black-list clients that do not support compression and feed them
again a highly reduced list.
* go back to a truncated list all together (preferably randomized
content with preference for free formats)
* provide a full list under different URL
* deprecate the list and implement a query API
(leaving open as this is only a test and no final decision has been made)
==
This is a wakeup call to all client software maintainers accessing
yp.xml! Fix your software! Please!
You unnecessarily annoy your users with long waiting times for the
directory content and waste OSUOSL's bandwidth.
Cheers
Thomas