Lars Kurth
2013-Nov-20 18:00 UTC
Xen Developer Summit BoF Notes : Seeding a Xen + Android / Embedded ecosystem BoF
Hi all, this is a copy of the notes I took at the Bof. I may have missed quite a few pices here. I also don''t know who was at the BoF (we had about 40 people there) . I have an action to make this thread available on http://xenproject.org/component/content/article/9-uncategorised/159-xen-project-developer-summit-2013-videos-and-presentations.html and to a mailshot to summit attendees. I also added these at http://wiki.xenproject.org/wiki/XPDS13_BoF_Notes_:_Seeding_an_Android_and_Embedded_ecosystem Please feel free to add stuff. Regards Lars == Platforms =# The overall agreement in the BoF was that Android on Xen should be platform agnostic # aka support ARM as well as Intel (e.g. many Android tablets in China are Intel based) == Missing Pieces for a complete Android system on top of Xen =# Backend ION memory allocator (manages allocation of additional buffers). My understanding that what was needed is an ION memory allocator for PV drivers ## Samsung noted that a ION PV allocator may be need to be specialized for a particular SOC ## There was a bit of debate on whether it would be valuable to have a reference ION PV allocator # Wrapper user space Device Drivers for Graphic, Sound, USB, Giros, GPS, etc. ## These are not Linux drivers but user space drivers ## In most cases these will be shims, but some may be device specific ## A common place in the Xen Project (or even better in Android) where these could be put would help everyone ## The Xen Project could host a repo initially; as more momentum builds the case to approach Google is likely to increase == Pieces missing for Embedded (non-Android) =# Xen Dom0 support in [https://openwrt.org/ OpenWrt]; [http://wiki.openwrt.org/doc/howto/xen DomU] support exists # Soft-Real-time support - note that this was in the original Xen ARM port - see [[Xen_ARM_(PV)]]. But it didn''t make it across to the Xen on ARM support. Also see a related more general thread at [http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02536.html xen-devel]. # Hard-Real-time support: a US government standard / RT kernel / OS was mentioned. In my notes it says ''''''Arine7'''''' ... but this is obviously wrong as google does not find anything. # Longer term: GPU virtualization - Video Card fragmenbtation is a huge issue ## Establish a list of GPUs vendors (note that this would be a very long list) ## Wait for Intel and Samsung to start submitting patches for their technology (see [http://www.youtube.com/watch?v=Km6gBnIqaWo <http://www.youtube.com/watch?v=Km6gBnIqaWo&feature=youtu.be>] and [http://www.youtube.com/watch?v=P5c7B9lk_Q0]) ## Are there any software alternatives? ## Stefano (I think) made the point that the Android GPU interface would lend itself for a neat standard interface for GPU virt in Xen == Mechanics of Bootstrapping a Community =# Lists ## The general consensus was that it would be a bad idea to create a new list. Discussions should happen on xen-devel ## However some attendees were concerned about the volume of traffic ## Threads related to ANDROID on xen-devel should be tagged with ANDROID [Aside: I am wondering, whether a more low-volume list for bigger and longer-term developer discussions may be helpful. We have done this in the past and all the lists in question (IA64, ARM, ...), but over time these lists had to be culled when the developed features became mainstream. There have also been some discussion recently about the amount of traffic on xen-devel becoming a little painful. I am just putting this one up for debate, but this would probably grant a separate discussion. ] # Meetings / Calls ## A monthly or less frequent open call would help ## Action Lars: write up minutes {{Tick}}, send out to all attendees and get the ones who''d be interested to reply whether''d be interested == Missing non-code pieces to speed up adoption = # Sharing of information andf helping each other ## This comes back down to the list / meeting /calls part # Lack of documentation ## Step 1: create an embedded and android category on the wiki (and post these notes). Created http://wiki.xenproject.org/wiki/Category:Embedded and http://wiki.xenproject.org/wiki/Category:Android {{Tick}} ## Somebody to Volunteer to create a ''''''Getting Started with Xen on Android'''''' on the wiki ## A Hackathon hosted by a vendor that cares about Android / Embedded in 2014 ## Have a central place / repo for Android user space drivers ### Open Question (for Android experts): is there something like an upstream for Android user space drivers today? ### How does one contribute (assuming such a place exists)?
Dario Faggioli
2013-Nov-21 09:52 UTC
Re: Xen Developer Summit BoF Notes : Seeding a Xen + Android / Embedded ecosystem BoF
On mer, 2013-11-20 at 18:00 +0000, Lars Kurth wrote:> == Platforms => # The overall agreement in the BoF was that Android on Xen should be > platform agnostic > # aka support ARM as well as Intel (e.g. many Android tablets in China > are Intel based) >32 or 64 bits (I''m talking mostly of the Intel ones)?> == Pieces missing for Embedded (non-Android) => # Soft-Real-time support - note that this was in the original Xen ARM > port - see [[Xen_ARM_(PV)]]. But it didn''t make it across to the Xen on > ARM support. >Was it? Didn''t know that, and I don''t see much about it in the archived Wiki page. It''s not that important as the project has been archived, but it''d be nice to see what was there, and what and why it didn''t survive, I think.> Also see a related more general thread at > [http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg02536.html > xen-devel]. >Yep, have a look and share you thoughts there too! ;-P> # Hard-Real-time support: a US government standard / RT kernel / OS was > mentioned. In my notes it says ''''''Arine7'''''' ... but this is obviously > wrong as google does not find anything. >Would it be possible for it to be ARINC653 (http://en.wikipedia.org/wiki/ARINC_653)? If yes, it''s an hard real time scheduler we have in Xen. It''s very special purpose and static so, although it''s very good at its job, I doubt it would fit here. In any case, although it''s definitely good to have ARINC (typically for avionics workloads), what I think we should aim at is something more general, more flexible, and yet no less hard real-time. Some (non mutually exclusive) alternatives are putting SEDF back in operation and looking at what the Real-Time Xen people have and try to upstream it (and both these are being worked on and discussed in the thread referenced above).> == Mechanics of Bootstrapping a Community => # Lists > ## The general consensus was that it would be a bad idea to create a new > list. Discussions should happen on xen-devel > ## However some attendees were concerned about the volume of traffic > ## Threads related to ANDROID on xen-devel should be tagged with ANDROID > > [Aside: I am wondering, whether a more low-volume list for bigger and > longer-term developer discussions may be helpful. We have done this in > the past and all the lists in question (IA64, ARM, ...), but over time > these lists had to be culled when the developed features became > mainstream. There have also been some discussion recently about the > amount of traffic on xen-devel becoming a little painful. I am just > putting this one up for debate, but this would probably grant a separate > discussion. ] >I thought about something similar (having a dedicated list) when starting the thread about real-time and embedded too. Personally, I like everything happening in xen-devel, but, if the main purpose is bootstrapping, and if that, for instance, would buy us people that wouldn''t subscribe to xen-devel anyway (because of the amount of traffic), well, I''d seriously consider having the list.> # Meetings / Calls > ## A monthly or less frequent open call would help > ## Action Lars: write up minutes {{Tick}}, send out to all attendees and > get the ones who''d be interested to reply whether''d be interested >If we''re doing it, I''d be interested.> == Missing non-code pieces to speed up adoption => > # Sharing of information andf helping each other > ## This comes back down to the list / meeting /calls part >Indeed.> # Lack of documentation > ## Step 1: create an embedded and android category on the wiki (and post > these notes). Created http://wiki.xenproject.org/wiki/Category:Embedded > and http://wiki.xenproject.org/wiki/Category:Android {{Tick}} > ## Somebody to Volunteer to create a ''''''Getting Started with Xen on > Android'''''' on the wiki >Zero experience with Android, but I''m willing to learn, so, I at least volunteer for helping. Then, if nobody more experienced steps up, I''ll upgrade my commitment. :-) Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Lars Kurth
2013-Nov-21 13:45 UTC
Re: Xen Developer Summit BoF Notes : Seeding a Xen + Android / Embedded ecosystem BoF
Dario, I added a few things where there was an open question. Also pointing back to the original mail at http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg03004.html ... On 21/11/2013 09:52, Dario Faggioli wrote:>> == Pieces missing for Embedded (non-Android) =>> # Soft-Real-time support - note that this was in the original Xen ARM >> port - see [[Xen_ARM_(PV)]]. But it didn''t make it across to the Xen on >> ARM support. >> > Was it? Didn''t know that, and I don''t see much about it in the archived > Wiki page. It''s not that important as the project has been archived, but > it''d be nice to see what was there, and what and why it didn''t survive, > I think.I may be wrong on this - there was certainly some research (see http://www.computer.org/csdl/trans/tm/preprint/06583914-abs.html, http://www.slideshare.net/xen_com_mgr/real-time-support-for-xen). I added See-Hwan Yoo to this thread who has done the original work. Maybe he can correct me and shed light on what was done. The issue was that there wasn''t any demand for real-time when we started the new Xen ARM port. Also the Citrix folks didn''t understand the existing code: we were waiting for somebody who knew the code to take the initiative and do the porting or guide us. However, it may make sense to have a solution that works on x86 and ARM. Not sure whether this is a pipe dream though.>> # Hard-Real-time support: a US government standard / RT kernel / OS was >> mentioned. In my notes it says ''''''Arine7'''''' ... but this is obviously >> wrong as google does not find anything. >> > Would it be possible for it to be ARINC653 > (http://en.wikipedia.org/wiki/ARINC_653)?Reading my notes again, it was saying "ARINC 7" (bad hand-writing). I can''t remember who raised this point. Maybe the raiser can step up? There certainly seems to be an ARINC 700 standard series.>> == Mechanics of Bootstrapping a Community =>> # Lists >> ## The general consensus was that it would be a bad idea to create a new >> list. Discussions should happen on xen-devel >> ## However some attendees were concerned about the volume of traffic >> ## Threads related to ANDROID on xen-devel should be tagged with ANDROID >> >> [Aside: I am wondering, whether a more low-volume list for bigger and >> longer-term developer discussions may be helpful. We have done this in >> the past and all the lists in question (IA64, ARM, ...), but over time >> these lists had to be culled when the developed features became >> mainstream. There have also been some discussion recently about the >> amount of traffic on xen-devel becoming a little painful. I am just >> putting this one up for debate, but this would probably grant a separate >> discussion. ] >> > I thought about something similar (having a dedicated list) when > starting the thread about real-time and embedded too. Personally, I like > everything happening in xen-devel, but, if the main purpose is > bootstrapping, and if that, for instance, would buy us people that > wouldn''t subscribe to xen-devel anyway (because of the amount of > traffic), well, I''d seriously consider having the list.I am not sure I would want a dedicated new list which lasts for a year only. This is what we used to have: there is a long history of one-off lists such as ova-devel, xci-devel, xen-ia64-devel, ... to name just a few. But maybe a list for researchy, longer term stuff may work that we can direct people to that are scared by the amount of xen-devel traffic. We can use the list whenever bigger and new ideas come along that still need to be formed and take some time to form. We still have xen-research (archived), that we could resurrect, but maybe the name isn''t quite right. Lars
Dario Faggioli
2013-Nov-22 07:08 UTC
Re: Xen Developer Summit BoF Notes : Seeding a Xen + Android / Embedded ecosystem BoF
On gio, 2013-11-21 at 13:45 +0000, Lars Kurth wrote:> Dario, > I added a few things where there was an open question. Also pointing > back to the original mail at > http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg03004.html ... >Ok.> On 21/11/2013 09:52, Dario Faggioli wrote: > >> == Pieces missing for Embedded (non-Android) => >> # Soft-Real-time support - note that this was in the original Xen ARM > >> port - see [[Xen_ARM_(PV)]]. But it didn''t make it across to the Xen on > >> ARM support. > >> > > Was it? Didn''t know that, and I don''t see much about it in the archived > > Wiki page. It''s not that important as the project has been archived, but > > it''d be nice to see what was there, and what and why it didn''t survive, > > I think. > I may be wrong on this - there was certainly some research (see > http://www.computer.org/csdl/trans/tm/preprint/06583914-abs.html, > http://www.slideshare.net/xen_com_mgr/real-time-support-for-xen). I > added See-Hwan Yoo to this thread who has done the original work. Maybe > he can correct me and shed light on what was done. >Well, I know there was ongoing research about real-time in virtualization, mostly because, at that time (before joining Citrix), I was among the ones doing it! :-)> However, it may make sense to have a solution that works on x86 and ARM. > Not sure whether this is a pipe dream though. >Sure. According to my experience, we can go pretty far in enabling real-time performances, without architecture specific trick. I mean, there are for sure low-level stuff, but the most of it needs to happen in common code. As, for example, you can see here (still from a research point of view): http://bugs.xenproject.org/xen/mid/%3CCAPqOm-pz8jW3Dd2k9RWu3FFZi7rJTJpAbb6mACRWUXuY7eiFKQ@mail.gmail.com%3E We need real-time capable schedulers in Xen, a real-time aware driver model (which means Dom0 and the backends), etc, and that''s all doable in such a way that all supported archs will benefit from it.> > I thought about something similar (having a dedicated list) when > > starting the thread about real-time and embedded too. Personally, I like > > everything happening in xen-devel, but, if the main purpose is > > bootstrapping, and if that, for instance, would buy us people that > > wouldn''t subscribe to xen-devel anyway (because of the amount of > > traffic), well, I''d seriously consider having the list. > I am not sure I would want a dedicated new list which lasts for a year > only. This is what we used to have: there is a long history of one-off > lists such as ova-devel, xci-devel, xen-ia64-devel, ... to name just a > few. But maybe a list for researchy, longer term stuff may work that we > can direct people to that are scared by the amount of xen-devel traffic. > We can use the list whenever bigger and new ideas come along that still > need to be formed and take some time to form. We still have xen-research > (archived), that we could resurrect, but maybe the name isn''t quite right. >Right. Well, my point here is: at this stage, we need to be able to reach out to as many as possible of the people interested and active in this area, and have all of us share thoughts and collaborate. Whatever (well, almost :-P) it is required to make this happen, I''m all for it. Thanks and Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
See-Hwan Yoo
2013-Nov-22 15:53 UTC
Re: Xen Developer Summit BoF Notes : Seeding a Xen + Android / Embedded ecosystem BoF
Dear Faggioli and Lars, Thanks for putting me into the thread, I am eager to join here. I''ve had a great time in Edinburgh, meeting friends that are working on the subject that is closely related with me. (Xen-arm and rt-scheduling issues) My thought about real-time things are becoming clear, and here are some. 1. Simply running a real-time OS is definitely not enough for guaranteeing deadline under virtualization. 2. To support hard real-time over Xen, we need the followings: a. WCET analysis for time-critical Xen services (hypcalls). Based upon this WCET, RTOS and apps (tasks) can calculate WCET, accordingly. b. Hierarchical scheduling theory buys you a real-time guarantee, and what we want here is an interface and algorithm for finding VM (or vcpu) scheduling parameters for rt schedulers (such as EDF or RM). Of course we need a scheduler implementation (I think ARINC/SEDF provides a basic format). c. a, and b would provide us real-time guarantee for cpu-bound real-time tasks. d. Maximum latency guarantee for aperiodic/sporadic tasks (such as I/O tasks) are difficult because We have to address scheduling model under split drivers. If inter-VM scheduling latency is introduced, then the latency goes up to tens of milliseconds, which is definitely not what we want. A possible simplification is prioritize the driver domain, however note that the driver domain should run a real-time OS, instead of Linux. In addition, we should implement backend drivers (and physical drivers) as real-time tasks. 3. To support soft real-time over Xen, there are several studies out there. However, most of them do not make code contribution. That is one of key huddles. 4. To concurrently support both real-time and non-real-time guest OSs, the scheduler framework has to be hierarchically structured. I am specifically interested in bigLITTLE cpus as a means of supporting soft real-time applications. However, I don''t have time schedules or written codes; so I just want to looking up when it begins to get started. If you have another idea, please tell me. I am observing the lists, and would be glad if I can help it to make progress. Best regards, Seehwan Yoo Ph.D. On Fri, Nov 22, 2013 at 4:08 PM, Dario Faggioli <dario.faggioli@citrix.com>wrote:> On gio, 2013-11-21 at 13:45 +0000, Lars Kurth wrote: > > Dario, > > I added a few things where there was an open question. Also pointing > > back to the original mail at > > > http://lists.xenproject.org/archives/html/xen-devel/2013-11/msg03004.html... > > > Ok. > > > On 21/11/2013 09:52, Dario Faggioli wrote: > > >> == Pieces missing for Embedded (non-Android) => > >> # Soft-Real-time support - note that this was in the original Xen ARM > > >> port - see [[Xen_ARM_(PV)]]. But it didn''t make it across to the Xen > on > > >> ARM support. > > >> > > > Was it? Didn''t know that, and I don''t see much about it in the archived > > > Wiki page. It''s not that important as the project has been archived, > but > > > it''d be nice to see what was there, and what and why it didn''t survive, > > > I think. > > I may be wrong on this - there was certainly some research (see > > http://www.computer.org/csdl/trans/tm/preprint/06583914-abs.html, > > http://www.slideshare.net/xen_com_mgr/real-time-support-for-xen). I > > added See-Hwan Yoo to this thread who has done the original work. Maybe > > he can correct me and shed light on what was done. > > > Well, I know there was ongoing research about real-time in > virtualization, mostly because, at that time (before joining Citrix), I > was among the ones doing it! :-) > > > However, it may make sense to have a solution that works on x86 and ARM. > > Not sure whether this is a pipe dream though. > > > Sure. According to my experience, we can go pretty far in enabling > real-time performances, without architecture specific trick. I mean, > there are for sure low-level stuff, but the most of it needs to happen > in common code. > > As, for example, you can see here (still from a research point of view): > > http://bugs.xenproject.org/xen/mid/%3CCAPqOm-pz8jW3Dd2k9RWu3FFZi7rJTJpAbb6mACRWUXuY7eiFKQ@mail.gmail.com%3E > > We need real-time capable schedulers in Xen, a real-time aware driver > model (which means Dom0 and the backends), etc, and that''s all doable in > such a way that all supported archs will benefit from it. > > > > I thought about something similar (having a dedicated list) when > > > starting the thread about real-time and embedded too. Personally, I > like > > > everything happening in xen-devel, but, if the main purpose is > > > bootstrapping, and if that, for instance, would buy us people that > > > wouldn''t subscribe to xen-devel anyway (because of the amount of > > > traffic), well, I''d seriously consider having the list. > > I am not sure I would want a dedicated new list which lasts for a year > > only. This is what we used to have: there is a long history of one-off > > lists such as ova-devel, xci-devel, xen-ia64-devel, ... to name just a > > few. But maybe a list for researchy, longer term stuff may work that we > > can direct people to that are scared by the amount of xen-devel traffic. > > We can use the list whenever bigger and new ideas come along that still > > need to be formed and take some time to form. We still have xen-research > > (archived), that we could resurrect, but maybe the name isn''t quite > right. > > > Right. Well, my point here is: at this stage, we need to be able to > reach out to as many as possible of the people interested and active in > this area, and have all of us share thoughts and collaborate. Whatever > (well, almost :-P) it is required to make this happen, I''m all for it. > > Thanks and Regards, > Dario > > -- > <<This happens because I choose it to happen!>> (Raistlin Majere) > ----------------------------------------------------------------- > Dario Faggioli, Ph.D, http://about.me/dario.faggioli > Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
Dario Faggioli
2013-Nov-23 17:01 UTC
Re: Xen Developer Summit BoF Notes : Seeding a Xen + Android / Embedded ecosystem BoF
On sab, 2013-11-23 at 00:53 +0900, See-Hwan Yoo wrote:> Dear Faggioli and Lars, >Hi! Nice to talk to you here too... :-)> Thanks for putting me into the thread, I am eager to join here. >Sure, BTW, have a look at this other one too, it''s a lot related to this and even more to what you''re talking about in this email of yours: http://bugs.xenproject.org/xen/mid/%3C1384802050.16918.219.camel@Solace%3E> My thought about real-time things are becoming clear, and here are > some. > > > 1. Simply running a real-time OS is definitely not enough for > guaranteeing deadline under virtualization. > 2. To support hard real-time over Xen, we need the followings: > a. WCET analysis for time-critical Xen services (hypcalls). > Based upon this WCET, RTOS and apps (tasks) can calculate WCET, > accordingly. >Yeah... WCET is particularly nasty, even in general, and virtualization would only make it even more of a nightmare! :-/ I agree that some need-to-be-certified / super-critical / ultra-hard real-time workload would need it, but at the same time I think there is a lot to improve before even getting close to that. Don''t get me wrong, not that I object or won''t be interested in someone wanting to do it, just a matter of priorities and of trying to start with the simple. :-) Speaking of RTOS on Xen, it would be nice to investigate whether there are someone that have been already ported...> b. Hierarchical scheduling theory buys you a real-time guarantee, and > what we want here is > an interface and algorithm for finding VM (or vcpu) scheduling > parameters for rt schedulers (such as EDF or RM). > Of course we need a scheduler implementation (I think ARINC/SEDF > provides a basic format). >Which is quite similar to what Sisu is saying in the other thread referenced above: http://bugs.xenproject.org/xen/mid/%3CCAPqOm-pz8jW3Dd2k9RWu3FFZi7rJTJpAbb6mACRWUXuY7eiFKQ@mail.gmail.com%3E And to which I agree too. :-) I''ll avoid listing here some (more) research paper on how real-time hierarchical scheduling could be applied here... Perhaps I''ll put them in the Wiki page.> d. Maximum latency guarantee for aperiodic/sporadic tasks (such as I/O > tasks) are difficult because > We have to address scheduling model under split drivers. If > inter-VM scheduling latency is introduced, > then the latency goes up to tens of milliseconds, which is > definitely not what we want. A possible simplification is > prioritize the driver domain, however note that the driver domain > should run a real-time OS, instead of Linux. >Yep, that would also be quite interesting. Perhaps we can actually start (at least for doing some measurements) with a real-time variant of Linux, either something like Xenomai or Linux+PREEMPT_RT patch and trheaded interrupt (and, of course, see how well threaded interrupt plays with our backends).> In addition, we should implement backend drivers (and physical > drivers) as real-time tasks. > > > 3. To support soft real-time over Xen, there are several studies out > there. > However, most of them do not make code contribution. That is one > of key huddles. >Indeed.> I am specifically interested in bigLITTLE cpus as a means of > supporting soft real-time applications. However, I don''t have time > schedules or written codes; so I just want to looking up when it > begins to get started. >Well, we''ll surely have to cross that bridge at some point!> If you have another idea, please tell me. > I am observing the lists, and would be glad if I can help it to make > progress. >Cool... Let''s see how these discussion evolve, and, sometime in the near future, if we can come up with a more concrete plan. Thanks and Regards, Dario -- <<This happens because I choose it to happen!>> (Raistlin Majere) ----------------------------------------------------------------- Dario Faggioli, Ph.D, http://about.me/dario.faggioli Senior Software Engineer, Citrix Systems R&D Ltd., Cambridge (UK) _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel