Hi, Emilis, I have two working Icecast players which implement client-side parsing of yp.xml and searching then through it: - Python, can either transform the yp.xml into SQLite database and then search inside, or can store the yp.xml locally and transfer in into DOM, then search it. http://www.zavedil.com/software-xbmc-icecast - Java (Android), same algorithm as Python but only the SQLite version. Originally was done in DOM (fast parsing, but uses too much memory), then switched to SAX (slow parsing, but works always). http://www.zavedil.com/software-android-icecast-player Feel free to use ideas and/code. If help is needed, let me know. WWell, Assen On 22 May 2013 15:13, "Thomas B. R?cker" <thomas at ruecker.fi> wrote:> Hi, > > On 05/17/2013 12:12 PM, Emilis Dambauskas wrote: > > I have pushed my radio player for FxOS to GitHub: > > https://github.com/emilis/worldradioplayer > > Finally got around to cloning your repo and pushing the app to my Unagi. > Simple and functional. I like that. > Once you get access to a larger list you'll have to accommodate some > filtering though. > > > Currently it uses a precompiled data from http://dir.xiph.org/yp.xml: > > > https://raw.github.com/emilis/worldradioplayer/master/static/js/data/xiph.org.stations.json > > > > I will have to switch to asynchronous station list loading and filtering > > in the near future and then will nag you again about the JSON API :-). > > Yeah, I'll try to work something out for you to play with soon. > It will be experimental for now, so that nobody comes after me if I > break their app by changing the API here or there to improve it or > reduce the system load on the database. > > > Thank you very much for your help! > > You're welcome. I'm happy, that finally Firefox and Icecast start to go > hand in hand. > > Cheers > > Thomas > _______________________________________________ > Icecast-dev mailing list > Icecast-dev at xiph.org > http://lists.xiph.org/mailman/listinfo/icecast-dev >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast-dev/attachments/20130522/ef403592/attachment.htm
Hi, On 05/22/2013 12:57 PM, Assen Totin wrote:> Hi, Emilis, > > I have two working Icecast players which implement client-side parsing > of yp.xml and searching then through it: [...]The idea is to finally offer a better API towards the data. Also to reduce the bandwidth usage. The XML file causes a LOT of traffic, especially as many clients don't implement HTTP compression. :-( This is also bad for users as they have to wait longer for: - a large download (700k compressed, 7MByte uncompressed) - parsing of the full XML file Whereas this could be broken down into only delivering the necessary data. More requests, less total transfer volume.> On 22 May 2013 15:13, "Thomas B. R?cker" <thomas at ruecker.fi > <mailto:thomas at ruecker.fi>> wrote: > > Hi, > > On 05/17/2013 12:12 PM, Emilis Dambauskas wrote: > > I have pushed my radio player for FxOS to GitHub: > > https://github.com/emilis/worldradioplayer > > Finally got around to cloning your repo and pushing the app to my > Unagi. > Simple and functional. I like that. > Once you get access to a larger list you'll have to accommodate some > filtering though. > > > Currently it uses a precompiled data from > http://dir.xiph.org/yp.xml: > > > https://raw.github.com/emilis/worldradioplayer/master/static/js/data/xiph.org.stations.json > > > > I will have to switch to asynchronous station list loading and > filtering > > in the near future and then will nag you again about the JSON > API :-). > > Yeah, I'll try to work something out for you to play with soon. > It will be experimental for now, so that nobody comes after me if I > break their app by changing the API here or there to improve it or > reduce the system load on the database. > > > Thank you very much for your help! > > You're welcome. I'm happy, that finally Firefox and Icecast start > to go > hand in hand. >Cheers Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast-dev/attachments/20130522/d288db84/attachment.htm
Hi, On 05/22/2013 01:07 PM, "Thomas B. R?cker" wrote:> Hi, > > On 05/22/2013 12:57 PM, Assen Totin wrote: >> Hi, Emilis, >> >> I have two working Icecast players which implement client-side >> parsing of yp.xml and searching then through it: [...] > > The idea is to finally offer a better API towards the data. Also to > reduce the bandwidth usage. The XML file causes a LOT of traffic, > especially as many clients don't implement HTTP compression. :-( > This is also bad for users as they have to wait longer for: > - a large download (700k compressed, 7MByte uncompressed) > - parsing of the full XML file > > Whereas this could be broken down into only delivering the necessary > data. More requests, less total transfer volume.I've cobbled together something. For now there is only the JSON version of the full XML output. http://api.dir.xiph.org/experimental/full Note, that I will change the output without notice, but will increment the API-revision HTTP header. Also in case I add more experimental queries, they'll be listed automatically at: http://api.dir.xiph.org/experimental/ Notes / known problems: - UTF-8 is broken, badly (this is a problem in the legacy code which is on my TODO list, for now please ignore the ugliness, or send patches for the YP code) - Each stream URL is listed separately for clusters - I'll try to return them as an array in the URL field instead - clustering doesn't work reliably, this is a legacy code limitation for the whole YP and on TODO Cheers Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast-dev/attachments/20130610/048e7c6e/attachment.htm
Hi, On 06/10/2013 02:46 PM, Emilis Dambauskas wrote:> this looks very promising. I see the compressed version is now ~250kB.Yeah, that's a start and there is probably still some room for improvement.> My Firefox OS app is using this data on its first run to create a > browser-side database of stations. I am currently looking into ways to > shorten this process to improve user experience. I am not 100% sure I > will be able to supply a snapshot of dir.xiph.org database with the > app before the first start, so using the official dir.xiph.org API > would be a logical first choice.Well, for the moment this is quite experimental.> Any way, every kilobyte shaved off the data transfer or processing > time on a mobile phone helps very much.Yes, that's the reason for this API rewrite in the first place. Also more flexible access will help to reduce transfer in favor of targeted access.> I am now waiting for API to get out of the experimental state :-)I can give you a stable snapshot for your app explicitly, until the real API pans out. But I won't support that ad infinity, basically until you then release an updated version for final API and usage of the 'snapshot' tapers off. Cheers Thomas> On 06/10/2013 08:17 AM, "Thomas B. R?cker" wrote: >> Hi, >> >> On 05/22/2013 01:07 PM, "Thomas B. R?cker" wrote: >>> Hi, >>> >>> On 05/22/2013 12:57 PM, Assen Totin wrote: >>>> Hi, Emilis, >>>> >>>> I have two working Icecast players which implement client-side >>>> parsing of yp.xml and searching then through it: [...] >>> >>> The idea is to finally offer a better API towards the data. Also to >>> reduce the bandwidth usage. The XML file causes a LOT of traffic, >>> especially as many clients don't implement HTTP compression. :-( >>> This is also bad for users as they have to wait longer for: >>> - a large download (700k compressed, 7MByte uncompressed) >>> - parsing of the full XML file >>> >>> Whereas this could be broken down into only delivering the necessary >>> data. More requests, less total transfer volume. >> >> I've cobbled together something. For now there is only the JSON >> version of the full XML output. >> http://api.dir.xiph.org/experimental/full >> >> Note, that I will change the output without notice, but will >> increment the API-revision HTTP header. >> >> Also in case I add more experimental queries, they'll be listed >> automatically at: >> http://api.dir.xiph.org/experimental/ >> >> Notes / known problems: >> - UTF-8 is broken, badly (this is a problem in the legacy code which >> is on my TODO list, for now please ignore the ugliness, or send >> patches for the YP code) >> - Each stream URL is listed separately for clusters >> - I'll try to return them as an array in the URL field instead >> - clustering doesn't work reliably, this is a legacy code >> limitation for the whole YP and on TODO >> >> Cheers >> >> Thomas > > > -- > Emilis Dambauskas > > emilis.d at gmail.com > gsm: +370-686-07732 > http://emilis.info/ > > -----BEGIN GEEK CODE BLOCK----- > Version: 3.12 > GAT/CC/MC/GG/O dpu(-) s:+ a C++ UBLHS++ P(+) L+++ E-(++) W+++$ N+ o-- K? !w O? M-@ V? PS+(--) PE Y+>++ PGP>+ t- 5? X+@ R- !tv b+ DI++++ D+ G e++ h----(*) r+++ y++++ > ------END GEEK CODE BLOCK------ >-------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast-dev/attachments/20130610/17ffb501/attachment.htm
On 06/10/2013 07:31 PM, "Thomas B. R?cker" wrote:> >> I am now waiting for API to get out of the experimental state :-) > > I can give you a stable snapshot for your app explicitly, until the > real API pans out. > But I won't support that ad infinity, basically until you then release > an updated version for final API and usage of the 'snapshot' tapers off.Thank you for the offer! I will check if I need it that bad in the next few days :-). I still need to check my app bundling options, so I am not sure right now if I will need to let my app use the API directly or I will use it via a build script. My current Makefile uses yp.xml which is good enough in this case. -- Emilis Dambauskas emilis.d at gmail.com gsm: +370-686-07732 http://emilis.info/ -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GAT/CC/MC/GG/O dpu(-) s:+ a C++ UBLHS++ P(+) L+++ E-(++) W+++$ N+ o-- K? !w O? M-@ V? PS+(--) PE Y+>++ PGP>+ t- 5? X+@ R- !tv b+ DI++++ D+ G e++ h----(*) r+++ y++++ ------END GEEK CODE BLOCK------ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast-dev/attachments/20130610/0e219d55/attachment.htm
Hi, again, is there any URL where I could POST to suggest streams for dir.xiph.org? My users are demanding the ability to add their own stations to the app: https://marketplace.firefox.com/app/world-radio-player/ratings I would like to use their input (with their explicit consent) to push data about new streams to dir.xiph.org. On 06/10/2013 07:31 PM, "Thomas B. R?cker" wrote:> Hi, > > On 06/10/2013 02:46 PM, Emilis Dambauskas wrote: >> this looks very promising. I see the compressed version is now ~250kB. > > Yeah, that's a start and there is probably still some room for > improvement. > > >> My Firefox OS app is using this data on its first run to create a >> browser-side database of stations. I am currently looking into ways >> to shorten this process to improve user experience. I am not 100% >> sure I will be able to supply a snapshot of dir.xiph.org database >> with the app before the first start, so using the official >> dir.xiph.org API would be a logical first choice. > > Well, for the moment this is quite experimental. > > >> Any way, every kilobyte shaved off the data transfer or processing >> time on a mobile phone helps very much. > > Yes, that's the reason for this API rewrite in the first place. > Also more flexible access will help to reduce transfer in favor of > targeted access. > > >> I am now waiting for API to get out of the experimental state :-) > > I can give you a stable snapshot for your app explicitly, until the > real API pans out. > But I won't support that ad infinity, basically until you then release > an updated version for final API and usage of the 'snapshot' tapers off. > > > Cheers > > Thomas > >> On 06/10/2013 08:17 AM, "Thomas B. R?cker" wrote: >>> Hi, >>> >>> On 05/22/2013 01:07 PM, "Thomas B. R?cker" wrote: >>>> Hi, >>>> >>>> On 05/22/2013 12:57 PM, Assen Totin wrote: >>>>> Hi, Emilis, >>>>> >>>>> I have two working Icecast players which implement client-side >>>>> parsing of yp.xml and searching then through it: [...] >>>> >>>> The idea is to finally offer a better API towards the data. Also to >>>> reduce the bandwidth usage. The XML file causes a LOT of traffic, >>>> especially as many clients don't implement HTTP compression. :-( >>>> This is also bad for users as they have to wait longer for: >>>> - a large download (700k compressed, 7MByte uncompressed) >>>> - parsing of the full XML file >>>> >>>> Whereas this could be broken down into only delivering the >>>> necessary data. More requests, less total transfer volume. >>> >>> I've cobbled together something. For now there is only the JSON >>> version of the full XML output. >>> http://api.dir.xiph.org/experimental/full >>> >>> Note, that I will change the output without notice, but will >>> increment the API-revision HTTP header. >>> >>> Also in case I add more experimental queries, they'll be listed >>> automatically at: >>> http://api.dir.xiph.org/experimental/ >>> >>> Notes / known problems: >>> - UTF-8 is broken, badly (this is a problem in the legacy code which >>> is on my TODO list, for now please ignore the ugliness, or send >>> patches for the YP code) >>> - Each stream URL is listed separately for clusters >>> - I'll try to return them as an array in the URL field instead >>> - clustering doesn't work reliably, this is a legacy code >>> limitation for the whole YP and on TODO >>> >>> Cheers >>> >>> Thomas >> >> >> -- >> Emilis Dambauskas >> >> emilis.d at gmail.com >> gsm: +370-686-07732 >> http://emilis.info/ >> >> -----BEGIN GEEK CODE BLOCK----- >> Version: 3.12 >> GAT/CC/MC/GG/O dpu(-) s:+ a C++ UBLHS++ P(+) L+++ E-(++) W+++$ N+ o-- K? !w O? M-@ V? PS+(--) PE Y+>++ PGP>+ t- 5? X+@ R- !tv b+ DI++++ D+ G e++ h----(*) r+++ y++++ >> ------END GEEK CODE BLOCK------ >> >-- Emilis Dambauskas emilis.d at gmail.com gsm: +370-686-07732 http://emilis.info/ -----BEGIN GEEK CODE BLOCK----- Version: 3.12 GAT/CC/MC/GG/O dpu(-) s:+ a C++ UBLHS++ P(+) L+++ E-(++) W+++$ N+ o-- K? !w O? M-@ V? PS+(--) PE Y+>++ PGP>+ t- 5? X+@ R- !tv b+ DI++++ D+ G e++ h----(*) r+++ y++++ ------END GEEK CODE BLOCK------ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.xiph.org/pipermail/icecast-dev/attachments/20130625/12ca8840/attachment.htm