This took me a while to post, but given that we are not starting 4.4 just yet, this may be appropriate now. I may have misrepresented some stuff as it has been 4 weeks since I wrote these. Cheers Lars = Purpose of Roadmap * Set a vision for interesting features * Track items * Help consumers of Xen with their planning = Release Models that work well There was a brief discussion on two different release models * Train leaves the station (Linux) * Release when ready (Debian) == Stefano''s Proposal =We should aim to reduce the release cycle to 4 months (or maybe 6 months as an intermediate step) from the current 9 months. A 4 months relase cycle should help accelerate development and lead to fewer patches being queued up. The implications are that we would have to operate a 2-3 weeks merge window. To do this, we would need to resolve a number of issues * It is likely that code reviews for such a short merge window would become a bottleneck. We are not sure whether this would be a major issue : the review bandwith would depend on the patches submitted (and their complexity) * [I can''t remember who raised this] The concern was raised that we would not have enough x86 Xen reviewers got a 2-3 weeks merge window * [Konrad] Stated that in PVOPS for Linux contributions we don''t have a review bottleneck, but we should make sure that the Xen and PVOPS/Linux merge window don''t overlap (as this would require the same set of people to review patches in two projects) * The rest of the time (approx 3 months) would be used for stabilizing the release * If we had a more extensive testing framework, and thus better testing, we could tighten the RC period (making this happen is also being discussed by the Advisory Board) Additional conerns raised: * [Matt Wilson]: if we had shorter merge windows, there is a risk that we would end up with unnused code (uncompleted features) in mainline. Something which we''d rather not have * [I can''t remember who raised this] But we already have a buffer via staging : we could make more use of this [Conclusion] We aLL agreed, that a release cycle of less than 9 months is desirable. Maybe we can go to 6 months for Xen 4.4 (before being more aggressive). = 4.3 Release cycle : what worked well / didn''t work well * The 4.3 release updates and criteria went well * BUT : 50% of what was supposed to be in 4.3 didn''t make it ** In some cases, we simply underestimated the effort that is needed. Concrete example : QEMU/stubdomain was a combination of under-estimating the size and over-estimating the development bandwidth that was available ** Some of the high-impact features (e.g. PVH) came in too late in the dev cycle. Mitigation : break contributions into smaller parts and submit earlier in the merge window. The same applies to changes to generic code. * BUT : Some patches were lost (i.e. when there are spikes of activity it becomes hard for some maintainers/committers to keep on top of their queue). ** [Ian Campell] said that we should rely on submitter to resend the patch: the assumption is that if the patch is not important, the submitter will badger and resend. ** [Lars] raised the point that this can alienate contributors and get them to look at other projects instead. ** [Can''t remember who said this] Maybe use patchwork (http://jk.ozlabs.org/projects/patchwork/) to track patches ** [Ian Campbell]: patchwork looks like a good idea, but may not work well in practice [Note] We should probably have a discussion or some sort of trial. It may also be possible to use the http://bugs.xenproject.org prototype (or add a "deferred patch" attribute) [Note] We typically start opening the dev branch at RC5/6 (not sure I quite got this) [Note] We don''t actually have a list of patches that got lost in the 4.3 release cycle )-: [ACTION]: George write up a proposal for the beginning of the 4.4 release cycle = 4.4 Content George volunteers to be the Release co-ordinator for Xen 4.4 (to apply what he learned) * Big features that did not make it in 4.3 * PVH? What will make it into 4.4 * Missed patches (don''t have a list) * "User" features that look interesting ** Network effects ** We shouldn''t have broken feature ** What about XenClient / VirtualComputer being able to help out in adding mopre support for PCI / VGA cards ** Other features: Sharing dom0 keyboard / mouse * GPU passthrough ** We have an issue with graphics card support ** A lot of users care about GPU passthrough (but not many vendors) ** Maybe we could mentor somebody about GPU passthrough? ** Maybe we could get XenClient, VirtualComputer or Qubes to pick this up (or partly do so)?
>>> On 13.06.13 at 16:00, Lars Kurth <lars.kurth@xen.org> wrote: > = Release Models that work well > There was a brief discussion on two different release models > * Train leaves the station (Linux) > * Release when ready (Debian) > > == Stefano''s Proposal => We should aim to reduce the release cycle to 4 months (or maybe 6 months > as an intermediate step) from the current 9 months. A 4 months relase > cycle should help accelerate development and lead to fewer patches being > queued up. The implications are that we would have to operate a 2-3 > weeks merge window. > > To do this, we would need to resolve a number of issues > * It is likely that code reviews for such a short merge window would > become a bottleneck. We are not sure whether this would be a major issue > : the review bandwith would depend on the patches submitted (and their > complexity) > * [I can''t remember who raised this] The concern was raised that we > would not have enough x86 Xen reviewers got a 2-3 weeks merge window > * [Konrad] Stated that in PVOPS for Linux contributions we don''t have a > review bottleneck, but we should make sure that the Xen and PVOPS/Linux > merge window don''t overlap (as this would require the same set of people > to review patches in two projects) > * The rest of the time (approx 3 months) would be used for stabilizing > the release > * If we had a more extensive testing framework, and thus better testing, > we could tighten the RC period (making this happen is also being > discussed by the Advisory Board) > > Additional conerns raised: > * [Matt Wilson]: if we had shorter merge windows, there is a risk that > we would end up with unnused code (uncompleted features) in mainline. > Something which we''d rather not have > * [I can''t remember who raised this] But we already have a buffer via > staging : we could make more use of this > > [Conclusion] We aLL agreed, that a release cycle of less than 9 months > is desirable. Maybe we can go to 6 months for Xen 4.4 (before being more > aggressive).I sincerely hope that we won''t switch to this (Linux) model. Having just a couple of weeks to get new code added means that everyone halfway deeply involved in the development will have to permanently carry a huge bag of patches. I''m thinking this is painful enough already for the freeze periods we currently have. That implies, however, that with an approximate freeze period of 3 months, shortening the release cycle to or even below 6 months is going to be an issue. From the summary above I didn''t really get what''s wrong with the current 9 month cycle, which - perhaps for the first time since I joined the project - it seems like we can actually meet with 4.3. The only option I would see is to branch before RC1, thus keeping the development tree open irrespective of the ongoing release. Of course that''s having the drawback of possibly/likely pulling away the attention of some of us from the being released branch (I would be very surprised if I wouldn''t end up among those "some"). Jan
On Thu, 2013-06-13 at 15:00 +0100, Lars Kurth wrote:> == Stefano''s Proposal => We should aim to reduce the release cycle to 4 months (or maybe 6 months > as an intermediate step) from the current 9 months. A 4 months relase > cycle should help accelerate development and lead to fewer patches being > queued up. The implications are that we would have to operate a 2-3 > weeks merge window. > > To do this, we would need to resolve a number of issues > * It is likely that code reviews for such a short merge window would > become a bottleneck. We are not sure whether this would be a major issue > : the review bandwith would depend on the patches submitted (and their > complexity)If we do end up going for this model then I think we need to also inherit the principal from Linux that the merge window is for *merging* and that the code in question should have already been posted and reviewed *before* the window opens (i.e. in the other 3 months of the cycle). This would be something of a cultural change because at the minute people seem to assume that a freeze means that development and review must stop. Personally I don''t think that should be the case anyway, although I appreciate the need to focus peoples attention on the release as well. This could end up with subsystem maintainers queueing patches into their own tree for merge into mainline during the window (this is what Linux does). This has its own downsides WRT the merging work during window and there would need to be a change since we would also have to ask people to test those trees, which they aren''t used to doing.> * [I can''t remember who raised this] The concern was raised that we > would not have enough x86 Xen reviewers got a 2-3 weeks merge window > * [Konrad] Stated that in PVOPS for Linux contributions we don''t have a > review bottleneck,Largely due to Linux having the properties I described above, IMHO. Although I''m not a Linux maintainer of a large subsystem so what would I know ;-)> but we should make sure that the Xen and PVOPS/Linux > merge window don''t overlap (as this would require the same set of people > to review patches in two projects) > * The rest of the time (approx 3 months) would be used for stabilizing > the release > * If we had a more extensive testing framework, and thus better testing, > we could tighten the RC period (making this happen is also being > discussed by the Advisory Board) > > Additional conerns raised: > * [Matt Wilson]: if we had shorter merge windows, there is a risk that > we would end up with unnused code (uncompleted features) in mainline. > Something which we''d rather not have > * [I can''t remember who raised this] But we already have a buffer via > staging : we could make more use of thisstaging is simply the pre-test branch, I don''t think it can serve as a buffer of the type you seem to be suggesting. We could encourage people to make more use of personal git trees, including maintainers.> > [Conclusion] We aLL agreed, that a release cycle of less than 9 months > is desirable. Maybe we can go to 6 months for Xen 4.4 (before being more > aggressive). > > = 4.3 Release cycle : what worked well / didn''t work well > * The 4.3 release updates and criteria went well > * BUT : 50% of what was supposed to be in 4.3 didn''t make itI think this is simply a normal part of the "train leaves the station" model. There will always be stuff which misses, but then there is always the next train, and the one after that. To some extent I''m not sure how useful it is with the train model to try and enumerate in advance the things which are going to be in a particular release . A list of things which people are working on and their current *target* release seems more useful, with the knowledge that the targets can and will change, probably more often than not for bigger items.> ** In some cases, we simply underestimated the effort that is needed. > Concrete example : QEMU/stubdomain was a combination of under-estimating > the size and over-estimating the development bandwidth that was available > ** Some of the high-impact features (e.g. PVH) came in too late in the > dev cycle. Mitigation : break contributions into smaller parts and > submit earlier in the merge window.I would say "submit well in advance of the merge window".> The same applies to changes to > generic code. > * BUT : Some patches were lost (i.e. when there are spikes of activity > it becomes hard for some maintainers/committers to keep on top of their > queue). > ** [Ian Campell] said that we should rely on submitter to resend the > patch: the assumption is that if the patch is not important, the > submitter will badger and resend.You have an extra "not" in there. I consider the "badger and resend" as more of a fallback/backstop rather than an intentional part of the workflow. For my own part I do keep an explicit queue of mails containing patches and I don''t think I tend to miss many due to them simply falling through the cracks, even during busts of activity. That''s not to say I always manage to process the queue in a timely manner though. That said a little bit of badgering has the secondary effect of reminding me that I need to look at the patch in my queue as well as the primary affect of me enqueuing something I may have missed.> ** [Lars] raised the point that this can alienate contributors and get > them to look at other projects instead. > ** [Can''t remember who said this] Maybe use patchwork > (http://jk.ozlabs.org/projects/patchwork/) to track patches > ** [Ian Campbell]: patchwork looks like a good idea, but may not work > well in practice > [Note] We should probably have a discussion or some sort of trial. It > may also be possible to use the http://bugs.xenproject.org prototype (or > add a "deferred patch" attribute) > [Note] We typically start opening the dev branch at RC5/6 (not sure I > quite got this)That''s right, we usually fork around the late RCs when we are reaching the end game.> [Note] We don''t actually have a list of patches that got lost in the 4.3 > release cycle )-:If they were on a list they by definition wouldn''t be lost ;-) FWIW I do have a separate queue of mails with patches which were deferred to 4.4 (but only for stuff that I would have committed myself had the tree not been frozen). Ian.
On 13/06/13 15:00, Lars Kurth wrote:> This took me a while to post, but given that we are not starting 4.4 > just yet, this may be appropriate now. I may have misrepresented some > stuff as it has been 4 weeks since I wrote these. > Cheers > Lars > > = Purpose of Roadmap > * Set a vision for interesting features > * Track items > * Help consumers of Xen with their planning > > = Release Models that work well > There was a brief discussion on two different release models > * Train leaves the station (Linux) > * Release when ready (Debian) > > == Stefano''s Proposal => We should aim to reduce the release cycle to 4 months (or maybe 6 > months as an intermediate step) from the current 9 months. A 4 months > relase cycle should help accelerate development and lead to fewer > patches being queued up. The implications are that we would have to > operate a 2-3 weeks merge window. > > To do this, we would need to resolve a number of issues > * It is likely that code reviews for such a short merge window would > become a bottleneck. We are not sure whether this would be a major > issue : the review bandwith would depend on the patches submitted (and > their complexity) > * [I can''t remember who raised this] The concern was raised that we > would not have enough x86 Xen reviewers got a 2-3 weeks merge window > * [Konrad] Stated that in PVOPS for Linux contributions we don''t have > a review bottleneck, but we should make sure that the Xen and > PVOPS/Linux merge window don''t overlap (as this would require the same > set of people to review patches in two projects) > * The rest of the time (approx 3 months) would be used for stabilizing > the releaseWhy would we need 3 months to do testing for 2 weeks of merging (or 4 months of development, depending on how you look at it), when 6 weeks has been just about right for 9 months of "merging"? I think it takes that long for Linux because it''s such a gigantic project. I would think 3 months development / 1 month stabilization would be OK for Xen. I would rather avoid the "merge window" workflow until it becomes really necessary, as it adds all kinds of other issues (e.g., needing something like linux-next to detect and sort out merge conflicts ahead of the merge window). -George
On 13/06/13 15:31, Ian Campbell wrote:> On Thu, 2013-06-13 at 15:00 +0100, Lars Kurth wrote: >> == Stefano''s Proposal =>> We should aim to reduce the release cycle to 4 months (or maybe 6 months >> as an intermediate step) from the current 9 months. A 4 months relase >> cycle should help accelerate development and lead to fewer patches being >> queued up. The implications are that we would have to operate a 2-3 >> weeks merge window. >> >> To do this, we would need to resolve a number of issues >> * It is likely that code reviews for such a short merge window would >> become a bottleneck. We are not sure whether this would be a major issue >> : the review bandwith would depend on the patches submitted (and their >> complexity) > If we do end up going for this model then I think we need to also > inherit the principal from Linux that the merge window is for *merging* > and that the code in question should have already been posted and > reviewed *before* the window opens (i.e. in the other 3 months of the > cycle). > > This would be something of a cultural change because at the minute > people seem to assume that a freeze means that development and review > must stop. Personally I don''t think that should be the case anyway, > although I appreciate the need to focus peoples attention on the release > as well.I think these are interdependent: Because we view having a code freeze (and having people with out-of-tree patches) as the exception, we try to keep the code freeze as short as possible; which means prioritizing working on the bugs. If we mixed development, review, and bug fixing, then the code freeze would necessarily be longer, as fixing bugs would be delayed. However, this wouldn''t necessarily be a terrible thing if everyone was expecting to keep out-of-tree branches anyway.> I think this is simply a normal part of the "train leaves the station" > model. There will always be stuff which misses, but then there is always > the next train, and the one after that.Yes -- I think this time we ended up sort of trying to do both; "The train is leaving at X time, what can we fit on before that time?" Having more frequent trains means less need for trying to match up features with releases, as the impact of slipping one release is much lower. -George
On Thu, 2013-06-13 at 15:52 +0100, George Dunlap wrote:> On 13/06/13 15:31, Ian Campbell wrote: > > On Thu, 2013-06-13 at 15:00 +0100, Lars Kurth wrote: > >> == Stefano''s Proposal => >> We should aim to reduce the release cycle to 4 months (or maybe 6 months > >> as an intermediate step) from the current 9 months. A 4 months relase > >> cycle should help accelerate development and lead to fewer patches being > >> queued up. The implications are that we would have to operate a 2-3 > >> weeks merge window. > >> > >> To do this, we would need to resolve a number of issues > >> * It is likely that code reviews for such a short merge window would > >> become a bottleneck. We are not sure whether this would be a major issue > >> : the review bandwith would depend on the patches submitted (and their > >> complexity) > > If we do end up going for this model then I think we need to also > > inherit the principal from Linux that the merge window is for *merging* > > and that the code in question should have already been posted and > > reviewed *before* the window opens (i.e. in the other 3 months of the > > cycle). > > > > This would be something of a cultural change because at the minute > > people seem to assume that a freeze means that development and review > > must stop. Personally I don''t think that should be the case anyway, > > although I appreciate the need to focus peoples attention on the release > > as well. > > I think these are interdependent: Because we view having a code freeze > (and having people with out-of-tree patches) as the exception, we try to > keep the code freeze as short as possible; which means prioritizing > working on the bugs. If we mixed development, review, and bug fixing, > then the code freeze would necessarily be longer, as fixing bugs would > be delayed. However, this wouldn''t necessarily be a terrible thing if > everyone was expecting to keep out-of-tree branches anyway.Agreed. And as you say in your other mail 2 weeks dev + 3 months stabilisation isn''t a good fit for us, something more like 3 months dev and 1 month stabilisation is a much better fit and avoids most of the issues I was thinking of.> > I think this is simply a normal part of the "train leaves the station" > > model. There will always be stuff which misses, but then there is always > > the next train, and the one after that. > > Yes -- I think this time we ended up sort of trying to do both; "The > train is leaving at X time, what can we fit on before that time?" > > Having more frequent trains means less need for trying to match up > features with releases, as the impact of slipping one release is much lower.Agreed. Ian.
On 13/06/13 15:22, Jan Beulich wrote:>>>> On 13.06.13 at 16:00, Lars Kurth <lars.kurth@xen.org> wrote: >> = Release Models that work well >> There was a brief discussion on two different release models >> * Train leaves the station (Linux) >> * Release when ready (Debian) >> >> == Stefano''s Proposal =>> We should aim to reduce the release cycle to 4 months (or maybe 6 months >> as an intermediate step) from the current 9 months. A 4 months relase >> cycle should help accelerate development and lead to fewer patches being >> queued up. The implications are that we would have to operate a 2-3 >> weeks merge window. >> >> To do this, we would need to resolve a number of issues >> * It is likely that code reviews for such a short merge window would >> become a bottleneck. We are not sure whether this would be a major issue >> : the review bandwith would depend on the patches submitted (and their >> complexity) >> * [I can''t remember who raised this] The concern was raised that we >> would not have enough x86 Xen reviewers got a 2-3 weeks merge window >> * [Konrad] Stated that in PVOPS for Linux contributions we don''t have a >> review bottleneck, but we should make sure that the Xen and PVOPS/Linux >> merge window don''t overlap (as this would require the same set of people >> to review patches in two projects) >> * The rest of the time (approx 3 months) would be used for stabilizing >> the release >> * If we had a more extensive testing framework, and thus better testing, >> we could tighten the RC period (making this happen is also being >> discussed by the Advisory Board) >> >> Additional conerns raised: >> * [Matt Wilson]: if we had shorter merge windows, there is a risk that >> we would end up with unnused code (uncompleted features) in mainline. >> Something which we''d rather not have >> * [I can''t remember who raised this] But we already have a buffer via >> staging : we could make more use of this >> >> [Conclusion] We aLL agreed, that a release cycle of less than 9 months >> is desirable. Maybe we can go to 6 months for Xen 4.4 (before being more >> aggressive). > I sincerely hope that we won''t switch to this (Linux) model. Having > just a couple of weeks to get new code added means that everyone > halfway deeply involved in the development will have to permanently > carry a huge bag of patches. I''m thinking this is painful enough > already for the freeze periods we currently have.This would be much less painful if you used git branches. :-) Seriously, check out stackgit -- it will make keeping track of all those patches, and of rebasing them, much much easier. (Not to mention make it easier to send them, share them, merge them, &c.)> That implies, however, that with an approximate freeze period of 3 > months, shortening the release cycle to or even below 6 months is > going to be an issue. From the summary above I didn''t really get > what''s wrong with the current 9 month cycle, which - perhaps for > the first time since I joined the project - it seems like we can actually > meet with 4.3.The problem with the current cycle I think is that if you miss getting a feature into this release, it''s going to be a full year and a half until the next release. This means that there is increased pressure to try to "shove things in" at the last minute, and to ask for feature freeze exceptions. If I was sure that 4.4 would be another 9-month cycle, I would have pushed much harder to make USB hot-plug support a blocker, for example. That in turn causes less review, longer code freezes, and so on. If there''s a 4-month cadence, then if you miss one release, it''s only another 4 months until the next release; not a big deal to wait.> The only option I would see is to branch before RC1, thus keeping > the development tree open irrespective of the ongoing release. Of > course that''s having the drawback of possibly/likely pulling away > the attention of some of us from the being released branch (I would > be very surprised if I wouldn''t end up among those "some").Ian C in fact proposed that this (doing bug-fixing, development, and review at the same time) might not be a bad thing. It would make the "freeze" longer, but if we had shorter release cycles, and if people could still get review time and maybe even items merged, then having a long freeze might not actually be all that terrible. Branching before RC1 would essentially be similar to the Linux model of "maintainers having their own branches", except that we''d just have one branch. Given that there are only a handful of developers compared to Linux, that might not be such a bad model to start with. Then there wouldn''t actually be a merge window per se -- we''d just fork off a branch every N months and start stabilizing it. -George
>>> On 13.06.13 at 17:11, George Dunlap <george.dunlap@eu.citrix.com> wrote: > The problem with the current cycle I think is that if you miss getting a > feature into this release, it''s going to be a full year and a half until > the next release.9 months != 1.5 years> This means that there is increased pressure to try to > "shove things in" at the last minute, and to ask for feature freeze > exceptions. If I was sure that 4.4 would be another 9-month cycle, I > would have pushed much harder to make USB hot-plug support a blocker, > for example. That in turn causes less review, longer code freezes, and > so on. > > If there''s a 4-month cadence, then if you miss one release, it''s only > another 4 months until the next release; not a big deal to wait.As long as releases are halfway predictable, I don''t see 4 months this significantly much better than 9 - those who don''t get their stuff ready in 4 months aren''t that unlikely to also not finish it in 8 or 9. But yes, I can see a 3+1 month cycle as still reasonable, albeit I''d personally prefer longer cycles. That''s not the least because at least till now the window of older branches being maintained is coupled to the main release cycle, and would hence also get shortened, or the amount of work to maintain these branches would increase simply because their number would grow - neither of which is really desirable imo. Jan
On Thu, 2013-06-13 at 16:30 +0100, Jan Beulich wrote:> But yes, I can see a 3+1 month cycle as still reasonable, albeit I''d > personally prefer longer cycles. That''s not the least because at > least till now the window of older branches being maintained is > coupled to the main release cycle, and would hence also get > shortened, or the amount of work to maintain these branches > would increase simply because their number would grow - neither > of which is really desirable imo.I raised this at the hackathon but didn''t spot that it wasn''t in the minutes. I think if we increase the release cadence we also need to tweak the stable release policy, perhaps by nominating a fraction of releases for longer term instead of the current "last 2". i.e perhaps we would support the previous stable release in the normal way and every second or third for a longer period (I''d need to sit down with a calendar to figure out which cadence I really thought was sensible). That of course undermines "there''s a next release soon" since 1/N releases somewhat become "special"... Ian
On Thu, Jun 13, 2013 at 10:00 AM, Lars Kurth <lars.kurth@xen.org> wrote:> This took me a while to post, but given that we are not starting 4.4 just > yet, this may be appropriate now. I may have misrepresented some stuff as it > has been 4 weeks since I wrote these. > Cheers > Lars > > = Purpose of Roadmap > * Set a vision for interesting features > * Track items > * Help consumers of Xen with their planning > > = Release Models that work well > There was a brief discussion on two different release models > * Train leaves the station (Linux) > * Release when ready (Debian) > > == Stefano''s Proposal => We should aim to reduce the release cycle to 4 months (or maybe 6 months as > an intermediate step) from the current 9 months. A 4 months relase cycle > should help accelerate development and lead to fewer patches being queued > up. The implications are that we would have to operate a 2-3 weeks merge > window. > > To do this, we would need to resolve a number of issues > * It is likely that code reviews for such a short merge window would become > a bottleneck. We are not sure whether this would be a major issue : the > review bandwith would depend on the patches submitted (and their complexity) > * [I can''t remember who raised this] The concern was raised that we would > not have enough x86 Xen reviewers got a 2-3 weeks merge window > * [Konrad] Stated that in PVOPS for Linux contributions we don''t have a > review bottleneck, but we should make sure that the Xen and PVOPS/Linux > merge window don''t overlap (as this would require the same set of people to > review patches in two projects) > * The rest of the time (approx 3 months) would be used for stabilizing the > release > * If we had a more extensive testing framework, and thus better testing, we > could tighten the RC period (making this happen is also being discussed by > the Advisory Board) > > Additional conerns raised: > * [Matt Wilson]: if we had shorter merge windows, there is a risk that we > would end up with unnused code (uncompleted features) in mainline. Something > which we''d rather not have > * [I can''t remember who raised this] But we already have a buffer via > staging : we could make more use of this > > [Conclusion] We aLL agreed, that a release cycle of less than 9 months is > desirable. Maybe we can go to 6 months for Xen 4.4 (before being more > aggressive). > > = 4.3 Release cycle : what worked well / didn''t work well > * The 4.3 release updates and criteria went well > * BUT : 50% of what was supposed to be in 4.3 didn''t make it > ** In some cases, we simply underestimated the effort that is needed. > Concrete example : QEMU/stubdomain was a combination of under-estimating the > size and over-estimating the development bandwidth that was available > ** Some of the high-impact features (e.g. PVH) came in too late in the dev > cycle. Mitigation : break contributions into smaller parts and submit > earlier in the merge window. The same applies to changes to generic code. > * BUT : Some patches were lost (i.e. when there are spikes of activity it > becomes hard for some maintainers/committers to keep on top of their queue). > ** [Ian Campell] said that we should rely on submitter to resend the patch: > the assumption is that if the patch is not important, the submitter will > badger and resend. > ** [Lars] raised the point that this can alienate contributors and get them > to look at other projects instead. > ** [Can''t remember who said this] Maybe use patchwork > (http://jk.ozlabs.org/projects/patchwork/) to track patches > ** [Ian Campbell]: patchwork looks like a good idea, but may not work well > in practice > [Note] We should probably have a discussion or some sort of trial. It may > also be possible to use the http://bugs.xenproject.org prototype (or add a > "deferred patch" attribute) > [Note] We typically start opening the dev branch at RC5/6 (not sure I quite > got this) > [Note] We don''t actually have a list of patches that got lost in the 4.3 > release cycle )-: > > [ACTION]: George write up a proposal for the beginning of the 4.4 release > cycle > > = 4.4 Content > George volunteers to be the Release co-ordinator for Xen 4.4 (to apply what > he learned) > > * Big features that did not make it in 4.3 > > * PVH? What will make it into 4.4 > > * Missed patches (don''t have a list) > > * "User" features that look interesting > > ** Network effects > > ** We shouldn''t have broken feature > > ** What about XenClient / VirtualComputer being able to help out in adding > mopre support for PCI / VGA cards >Did you have any more info on this bullet, and the other one below Re: XenClient / VirtualComputer? FWIW, the XenClient Enterprise product (what was VirtualComputer NxTop) doesn''t currently do PCI / VGA passthrough, in an attempt to get a larger HCL. The XenClient XT product (developed primarily in Citrix Cambridge) does rely on this functionality, however. It is a subtle (but sizable) architectural difference, that made each team come to some different conclusions when solving similar problems. As such, this is an area that the two teams have remained separate, despite sharing a XenClient moniker. There may be some areas that we may be able to help out, but we''d need to enumerate the issues, and work with management to get it properly staffed - naturally balancing against the product delivery requirements.> ** Other features: Sharing dom0 keyboard / mouse > > > * GPU passthrough > ** We have an issue with graphics card support > ** A lot of users care about GPU passthrough (but not many vendors) > ** Maybe we could mentor somebody about GPU passthrough? > > ** Maybe we could get XenClient, VirtualComputer or Qubes to pick this up > (or partly do so)? > > > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel
On Thu, Jun 13, 2013 at 01:09:46PM -0400, Ben Guthro wrote:> > > > ** What about XenClient / VirtualComputer being able to help out in adding > > mopre support for PCI / VGA cards > > > > Did you have any more info on this bullet, and the other one below Re: > XenClient / VirtualComputer? > > FWIW, the XenClient Enterprise product (what was VirtualComputer > NxTop) doesn''t currently do PCI / VGA passthrough, in an attempt to > get a larger HCL. > The XenClient XT product (developed primarily in Citrix Cambridge) > does rely on this functionality, however. > > It is a subtle (but sizable) architectural difference, that made each > team come to some different conclusions when solving similar problems. > As such, this is an area that the two teams have remained separate, > despite sharing a XenClient moniker. > > There may be some areas that we may be able to help out, but we''d need > to enumerate the issues, and work with management to get it properly > staffed - naturally balancing against the product delivery > requirements. >I''m not sure what the "more support for PCI / VGA cards" above meant, but Xen would at least need someone to be the "GPU passthrough" maintainer, there are many unmerged patches laying around on the mailinglists, for all the three major GPU vendors: Intel, Nvidia and AMD.> > > ** Other features: Sharing dom0 keyboard / mouse >This should be a pretty small task actually, we did some planning and analysis of various options available at last year''s XenSummit with Gianluca and Stefano.> > > > > > * GPU passthrough > > ** We have an issue with graphics card support > > ** A lot of users care about GPU passthrough (but not many vendors) > > ** Maybe we could mentor somebody about GPU passthrough? > > > > ** Maybe we could get XenClient, VirtualComputer or Qubes to pick this up > > (or partly do so)? > > > >GPU passthrough is an important feature, and currently it''s not well supported out-of-the-box with Xen. It is being discussed very often on xen related mailinglists and irc channels, so it seems to be "hot" feature for users. -- Pasi
--On 13 June 2013 15:00:46 +0100 Lars Kurth <lars.kurth@xen.org> wrote:> We should aim to reduce the release cycle to 4 months (or maybe 6 months > as an intermediate step) from the current 9 months. A 4 months relase > cycle should help accelerate development and lead to fewer patches being > queued up. The implications are that we would have to operate a 2-3 weeks > merge window.Disclaimer: I don''t claim to be a xen developer, despite writing a few patches. But we are a heavy libxl API user. This perspective may or may not be useful from a ''consumer of your development'' perspective. Delete or ignore if not. The thing we like the least about Xen is (was?), stuff breaking between major releases. If you have to drop 90% of your new features in order not to break stuff, please do just that. Xen 3.x -> Xen 4.1 (we mostly skipped 4.0) was a huge change, requiring a lot of rewriting. Xen 4.1 to Xen 4.2 was far, far worse; had we not needed it to support qemu-upstream, we''d have probably stuck with 4.1. We talk to 4 hypervisors, and have never had this difficulty with any other hypervisor. You can imagine our joy when we found we could compile against 4.3 (we haven''t tried running yet) without a single line code changed. Please, please, keep it like this. If we can also run unchanged against 4.3, this will be even better. The thing we like the second least about Xen is how long it seems to take to get what we count as serious bugs fixed, even in stable releases. We like it even less if we have to find them and fix them. By ''serious'' I mean basic functionality not working, crashes dom0, etc. I don''t know if this says something bad about us, but we''ve not found any such bug ever in kvm (in well over 5 years). On the positive side, we''ve found xen-devel pretty friendly and receptive. The perception is, however, that new development takes priority over stable releases. I recognise that I have a bias here, so this may not be fair. The result of this is two-fold. Firstly, we''ve never (yet) been able to run a production version of xen which is a standard xen release. We''ve always had to maintain our own patches even on ''stable'' releases. Frankly, this is a pain. Secondly, there is a perception that moving versions of Xen is going to be a huge PITA; that perception does not exist for other hypervisors. I''m really really hoping Xen 4.3 dispels this, and signs are good so far (we''ve done a lot of testing at xl level, not so much at libxl level). What does this mean for the development process? 1. I think more testing would be useful, particularly against API driven stuff. I find the current test stuff a bit confusing. What I wan''t to know is whether commit X works or not - if things fail randomly, they aren''t useful tests, particularly if it''s difficult to distinguish them from other failures. Spoken as someone who''s broken things :-( 2. My concern about early branching of -next (at -rc1 for instance) is that developing new stuff is far more interesting than fixing bugs. We liked the way stuff we raised with 4.3 (e.g. tsc issues) got looked at, and would hope that continues. 3. I would quite like a slightly shorter development cycle IFF it doesn''t impact on stability. EG I thought we were probably bending the rules to get live migrate on qemu-upstream backported into 4.2, and had 4.3 been available sooner, we wouldn''t have pushed. At a guess, 6 months would be about right, 4 months would be too short. 4. Following xen has been 10 times easier since you''ve moved to git. I agree with George''s statement that you aren''t using it enough - particularly branches. 5. At risk of repetition, we don''t really care, so long as you don''t break stuff. -- Alex Bligh
On 13/06/13 22:03, Alex Bligh wrote:> > > --On 13 June 2013 15:00:46 +0100 Lars Kurth <lars.kurth@xen.org> wrote: > >> We should aim to reduce the release cycle to 4 months (or maybe 6 months >> as an intermediate step) from the current 9 months. A 4 months relase >> cycle should help accelerate development and lead to fewer patches being >> queued up. The implications are that we would have to operate a 2-3 >> weeks >> merge window. > > Disclaimer: I don''t claim to be a xen developer, despite writing a few > patches. But we are a heavy libxl API user. This perspective may or may > not be useful from a ''consumer of your development'' perspective. Delete > or ignore if not. > > The thing we like the least about Xen is (was?), stuff breaking between > major releases. If you have to drop 90% of your new features in order > not to break stuff, please do just that. Xen 3.x -> Xen 4.1 (we mostly > skipped 4.0) was a huge change, requiring a lot of rewriting. Xen 4.1 > to Xen 4.2 was far, far worse; had we not needed it to support > qemu-upstream, we''d have probably stuck with 4.1. We talk to 4 > hypervisors, > and have never had this difficulty with any other hypervisor. You can > imagine our joy when we found we could compile against 4.3 (we haven''t > tried running yet) without a single line code changed. Please, please, > keep it like this. If we can also run unchanged against 4.3, this will > be even better. >When you talk about "stuff" breaking between major releases, are you talking about Xen code not functioning or your code failing because of changes in Xen? If the latter, are we talking designed changes in Xen''s behaviour or non-designed ones (=bugs)?> The thing we like the second least about Xen is how long it seems to take > to get what we count as serious bugs fixed, even in stable releases. We > like it even less if we have to find them and fix them. By ''serious'' > I mean basic functionality not working, crashes dom0, etc. I don''t know > if this says something bad about us, but we''ve not found any such bug > ever in kvm (in well over 5 years). On the positive side, we''ve found > xen-devel pretty friendly and receptive. The perception is, however, that > new development takes priority over stable releases. I recognise that I > have a bias here, so this may not be fair.I am at a loss as to what is wrong with contributing a few bug fixes back if you''re technically capable of finding and fixing... I am not feeling the community spirit here.> > The result of this is two-fold. Firstly, we''ve never (yet) been able to > run a production version of xen which is a standard xen release. We''ve > always had to maintain our own patches even on ''stable'' releases. > Frankly, this is a pain. Secondly, there is a perception that moving > versions of Xen is going to be a huge PITA; that perception does not > exist for other hypervisors. I''m really really hoping Xen 4.3 dispels > this, and signs are good so far (we''ve done a lot of testing at xl > level, not so much at libxl level).I thought Xen was a "project" not a "product". Also, a little bit of Googling tells me that Flexiant (that is the "we" in all of this, right?) provides cloud software to third-parties and does not provide cloud services itself. To make your product work reliably with Xen for your customers, are you distributing your own patches for Xen to these third-parties?> > What does this mean for the development process? > > 1. I think more testing would be useful, particularly against API driven > stuff. I find the current test stuff a bit confusing. What I wan''t > to know is whether commit X works or not - if things fail randomly, > they aren''t useful tests, particularly if it''s difficult to > distinguish them from other failures. Spoken as someone who''s broken > things :-( > > 2. My concern about early branching of -next (at -rc1 for instance) is > that developing new stuff is far more interesting than fixing bugs. > We liked the way stuff we raised with 4.3 (e.g. tsc issues) got looked > at, and would hope that continues. > > 3. I would quite like a slightly shorter development cycle IFF it > doesn''t impact on stability. EG I thought we were probably bending the > rules to get live migrate on qemu-upstream backported into 4.2, > and had 4.3 been available sooner, we wouldn''t have pushed. At > a guess, 6 months would be about right, 4 months would be too short. > > 4. Following xen has been 10 times easier since you''ve moved to git. > I agree with George''s statement that you aren''t using it enough - > particularly branches. > > 5. At risk of repetition, we don''t really care, so long as you don''t > break > stuff. >Again, what "stuff"? IMHO woolly language isn''t wonderfully helpful in most cases, especially on a technical list. Thanks for reading. Disclosure: An interested individual who was delighted to recently have had a tincy-wincy patch accepted into unstable, fixing a bug that has annoyed me for donkey''s.
Ian, --On 14 June 2013 00:56:14 +0100 Ian Murray <murrayie@yahoo.co.uk> wrote:> When you talk about "stuff" breaking between major releases, are you > talking about Xen code not functioning or your code failing because of > changes in Xen? If the latter, are we talking designed changes in Xen''s > behaviour or non-designed ones (=bugs)?Both. As an example of the first, the API changed very significantly between 3.x and 4.1, and 4.1 and 4.2. Some of the API changes were subtle (e.g. what you had to do across a fork()). As an example of the second, see the very long thread with O_DIRECT in the subject line.> I am at a loss as to what is wrong with contributing a few bug fixes back > if you''re technically capable of finding and fixing... I am not feeling > the community spirit here.There''s nothing wrong with that, and we have done. However this thread is about development cycles. Critical stuff breaks, we know that. That should happen in the unstable version. Patches that fix critical stuff should be made in the unstable version, not the stable version.> Also, a little bit of Googling tells me that Flexiant (that is the "we" > in all of this, right?) provides cloud software to third-parties and does > not provide cloud services itself. To make your product work reliably > with Xen for your customers, are you distributing your own patches for > Xen to these third-parties?Yes. We ship with a patched version of xen. All our patches have been sent here and are on github. If the implication is that we''re somehow keeping these to ourselves, you have that totally wrong. We would like nothing better than all our patches to be in mainline. For 4.2 we are currently carrying one patch for fixing the tsc (see ''clock stalled on live migration'') which is a backport of a 4.3 patch here, and my minideb patch which is a packaging change which (for quite understandable reasons) people don''t want in mainline. In the past, we''ve carried tens of patches - see the live migrate on qemu-upstream-dm for 4.2 patch series I posted here, since accepted. -- Alex Bligh
>>> On 13.06.13 at 23:03, Alex Bligh <alex@alex.org.uk> wrote: > The thing we like the second least about Xen is how long it seems to take > to get what we count as serious bugs fixed, even in stable releases.I think applicable bug fixes get applied to the stable branches in quite timely a manner, at least on the hypervisor side. Hence I can only assume that you''re unhappy with the rate of stable releases. Yet I don''t think we''re going to get anywhere near of the almost weekly stable releases that get done for Linux. As you also didn''t say what expectations you really have, it''s hard to take out anything useful from that complaint.> We like it even less if we have to find them and fix them.But isn''t that how open source projects work - everyone contributes and fixes bugs. If you don''t want to help fixing bugs, I''m afraid there''s also no good reason for you to complain they don''t get fixed.> By ''serious'' I mean basic functionality not working, crashes dom0, etc.When did we last break Dom0, and not fix it in a timely manner?> The result of this is two-fold. Firstly, we''ve never (yet) been able to > run a production version of xen which is a standard xen release. We''ve > always had to maintain our own patches even on ''stable'' releases. > Frankly, this is a pain.But if you don''t contribute back your patches, how do you expect them to get accepted/merged? Or are you saying that it''s _far_ more than occasional that patches from you get entirely lost? Jan
----- Original Message -----> From: Alex Bligh <alex@alex.org.uk> > To: Ian Murray <murrayie@yahoo.co.uk>; xen-devel@lists.xen.org > Cc: Alex Bligh <alex@alex.org.uk> > Sent: Friday, 14 June 2013, 8:01 > Subject: Re: [Xen-devel] [Hackathon Minutes] Xen 4.4 Planning > > Ian, > > --On 14 June 2013 00:56:14 +0100 Ian Murray <murrayie@yahoo.co.uk> wrote: > >> When you talk about "stuff" breaking between major releases, are > you >> talking about Xen code not functioning or your code failing because of >> changes in Xen? If the latter, are we talking designed changes in Xen''s >> behaviour or non-designed ones (=bugs)? > > Both. > > As an example of the first, the API changed very significantly between > 3.x and 4.1, and 4.1 and 4.2. Some of the API changes were subtle (e.g. > what you had to do across a fork()).But surely, this is your business. This is what you do. Sounds like you want API changes to be held back for the entire community so your company has to do less work.> > As an example of the second, see the very long thread with O_DIRECT > in the subject line. > >> I am at a loss as to what is wrong with contributing a few bug fixes back >> if you''re technically capable of finding and fixing... I am not feeling >> the community spirit here. > > There''s nothing wrong with that, and we have done. However this thread > is about development cycles. Critical stuff breaks, we know that. > That should happen in the unstable version. Patches that fix critical > stuff should be made in the unstable version, not the stable version.Do *you* test unstable and point out the critical bugs BEFORE it goes stable? If you don''t, then don''t be surprised when bugs crop up...> >> Also, a little bit of Googling tells me that Flexiant (that is the > "we" >> in all of this, right?) provides cloud software to third-parties and does >> not provide cloud services itself. To make your product work reliably >> with Xen for your customers, are you distributing your own patches for >> Xen to these third-parties? > > Yes. We ship with a patched version of xen. All our patches have been sent > here and are on github. If the implication is that we''re somehow keeping > these to ourselves, you have that totally wrong. We would like nothing > better than all our patches to be in mainline. For 4.2 we are currently > carrying one patch for fixing the tsc (see ''clock stalled on live > migration'') which is a backport of a 4.3 patch here, and my minideb > patch which is a packaging change which (for quite understandable > reasons) people don''t want in mainline. In the past, we''ve carried > tens of patches - see the live migrate on qemu-upstream-dm for 4.2 > patch series I posted here, since accepted.I looked on your website and couldn''t find any download section for patches or a pointer to github or anything. I am not a lawyer but I didn''t get the impression that sticking patches on a mailing list was fulfilling ones obligation under the appropriate licence. So please can you link from your company website to your github were these patches are available. One aspect from your previous email I''d like to pickup on, when you mentioned KVM not having these issues, are you compiling from the source or using a distribution version? If the latter, then I think that is an unfair comparison.> > -- > Alex Bligh > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >
On 14/06/13 09:15, Jan Beulich wrote:> But isn''t that how open source projects work - everyone contributes > and fixes bugs. If you don''t want to help fixing bugs, I''m afraid > there''s also no good reason for you to complain they don''t get fixed.Well, Jan, that''s how the market works -- people use the project that has the best code and the least bugs; if people find more stability with KVM, then people are likely to switch to it even if they don''t think the performance or architecture is as good as they want. It''s no good complaining that your users aren''t helping to fix bugs if there are other projects with fewer bugs to fix. :-) Alex and his team has been very helpful in helping test and track down some issues in 4.3, and we really appreciate the effort they''ve put in. -George
Hi all, I will go through the thread and try and answer question (where they have not been answered and discussed), but will need the help of the people who actually attended and drove the discussion at the Hackathon. I merely took notes of what was said, and at times I missed things. And no decision of any kind has been made : for this we do have a decision making process. But this thread seems to show that we do have a number of problems that need fixing. Maybe we need to crystallize this a little more. From Jan:> From the summary above I didn''t really get what''s wrong with the current9 month cycle, which ... I don''t think this was indeed covered. I think the rationale was really covered in the thread. I also think I minuted the 2-3 weeks + 3 months wrong (which seems to have been clarified). From Ben Guthro:> Did you have any more info on this bullet, and the other one below Re: XenClient/ VirtualComputer? The only thing I remember was that this discussion was originally started by George and that IanC, Konrad, Stefano and Matt mostly covered it. We do seem to have a problem with PCI passthrough - Pasi''s points. Maybe they recall more detail. I also wanted to add to the point that Alex has made on serious bugs in xen vs. kvm : this gets raised regularly by Xen users when I attend conferences. And I also heard a few times now that we appear to focus more on sexy features rather than the basics. This is relatively new though: the first time I noticed was towards the end of last year. Which does not mean that this is new: it may just mean that top issues/concerns that were frequently raised before (mainly trust in the future of the project) have disappeared. Lars On Fri, Jun 14, 2013 at 9:15 AM, Jan Beulich <JBeulich@suse.com> wrote:> >>> On 13.06.13 at 23:03, Alex Bligh <alex@alex.org.uk> wrote: > > The thing we like the second least about Xen is how long it seems to take > > to get what we count as serious bugs fixed, even in stable releases. > > I think applicable bug fixes get applied to the stable branches in > quite timely a manner, at least on the hypervisor side. Hence I > can only assume that you''re unhappy with the rate of stable > releases. Yet I don''t think we''re going to get anywhere near of > the almost weekly stable releases that get done for Linux. As > you also didn''t say what expectations you really have, it''s hard > to take out anything useful from that complaint. > > > We like it even less if we have to find them and fix them. > > But isn''t that how open source projects work - everyone contributes > and fixes bugs. If you don''t want to help fixing bugs, I''m afraid > there''s also no good reason for you to complain they don''t get fixed. > > > By ''serious'' I mean basic functionality not working, crashes dom0, etc. > > When did we last break Dom0, and not fix it in a timely manner? > > > The result of this is two-fold. Firstly, we''ve never (yet) been able to > > run a production version of xen which is a standard xen release. We''ve > > always had to maintain our own patches even on ''stable'' releases. > > Frankly, this is a pain. > > But if you don''t contribute back your patches, how do you expect > them to get accepted/merged? Or are you saying that it''s _far_ > more than occasional that patches from you get entirely lost? > > Jan > >_______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On 13/06/13 22:03, Alex Bligh wrote:> > > --On 13 June 2013 15:00:46 +0100 Lars Kurth <lars.kurth@xen.org> wrote: > >> We should aim to reduce the release cycle to 4 months (or maybe 6 months >> as an intermediate step) from the current 9 months. A 4 months relase >> cycle should help accelerate development and lead to fewer patches being >> queued up. The implications are that we would have to operate a 2-3 >> weeks >> merge window. > > Disclaimer: I don''t claim to be a xen developer, despite writing a few > patches. But we are a heavy libxl API user. This perspective may or may > not be useful from a ''consumer of your development'' perspective. Delete > or ignore if not. > > The thing we like the least about Xen is (was?), stuff breaking between > major releases. If you have to drop 90% of your new features in order > not to break stuff, please do just that. Xen 3.x -> Xen 4.1 (we mostly > skipped 4.0) was a huge change, requiring a lot of rewriting. Xen 4.1 > to Xen 4.2 was far, far worse; had we not needed it to support > qemu-upstream, we''d have probably stuck with 4.1. We talk to 4 > hypervisors, > and have never had this difficulty with any other hypervisor. You can > imagine our joy when we found we could compile against 4.3 (we haven''t > tried running yet) without a single line code changed. Please, please, > keep it like this. If we can also run unchanged against 4.3, this will > be even better.I think we''ve been aware of this for some time, and have only in the last few years really put an effort into sorting out some of these issues. Having libxl as a library, with an API compatibility promise, was one step in that direction. Another step in that was having a better testing infrastructure, which has helped a great deal. If it''s any comfort, the problems you had upgrading 3.x->4.1 and 4.1->4.2 have been shared by the Citrix XenServer team, so there''s a lot of desire to make the process better. The testing of the upstream Xen is still really lacking, however -- this is a known problem, and is actually being discussed at the moment by the Linux Foundation project members. At the moment, the real hard-core testing of Xen happens by the downstream users -- XenServer, OracleVM, SuSE, plus cloud vendors (including you) -- but this is a rather wasteful duplication of effort. Having a more thorough testing infrastructure and set of test cases for upstream Xen would mean the standard Xen releases were of much higher quality; this would benefit everyone. In any case, thanks for sharing your thoughts -- it''s useful to have your perspective. We''ll have to keep these in mind going forward. -George
>>> On 14.06.13 at 11:59, Lars Kurth <lars.kurth.xen@gmail.com> wrote: > I also wanted to add to the point that Alex has made on serious bugs in xen > vs. kvm : this gets raised regularly by Xen users when I attend > conferences. And I also heard a few times now that we appear to focus more > on sexy features rather than the basics. This is relatively new though: the > first time I noticed was towards the end of last year. Which does not mean > that this is new: it may just mean that top issues/concerns that were > frequently raised before (mainly trust in the future of the project) have > disappeared.I''d really like to see examples of this - there ought to be quite a few according to what you write, yet I don''t seem to recall any that got plainly ignored. Of course there are always bugs which take longer than others to figure out and fix. Jan
On 14/06/13 11:45, Jan Beulich wrote:>>>> On 14.06.13 at 11:59, Lars Kurth <lars.kurth.xen@gmail.com> wrote: >> I also wanted to add to the point that Alex has made on serious bugs in xen >> vs. kvm : this gets raised regularly by Xen users when I attend >> conferences. And I also heard a few times now that we appear to focus more >> on sexy features rather than the basics. This is relatively new though: the >> first time I noticed was towards the end of last year. Which does not mean >> that this is new: it may just mean that top issues/concerns that were >> frequently raised before (mainly trust in the future of the project) have >> disappeared. > I''d really like to see examples of this - there ought to be quite a > few according to what you write, yet I don''t seem to recall any > that got plainly ignored. Of course there are always bugs which > take longer than others to figure out and fix.Well one example was that from the beginning of time until the 4.3 release, you could only specify a single USB device in the config file for an HVM domain. Another one is the qemu one we''re discussing right now -- it was reported back in February I think, but it''s only recently actually getting the attention that it needs to be sorted out. The root cause was known by March, at which point there would have been plenty of time for a "proper fix". And there''s random things like cd-eject or cd-insert not working in certain circumstances (namely, when blktap was available). It''s something basic, but something we don''t really use very much, and never got flagged up. So it was broken in the 4.2 release. I don''t really know if this is the kind of thing that users are talking about, but if I used a release of a (supposedly) mature product, and something as basic as "insert a CD" crashed, I wouldn''t be very impressed. -George
On Fri, 14 Jun 2013 11:45:52 +0100, "Jan Beulich" <JBeulich@suse.com> wrote:>>>> On 14.06.13 at 11:59, Lars Kurth <lars.kurth.xen@gmail.com> wrote: >> I also wanted to add to the point that Alex has made on serious bugs >> in xen >> vs. kvm : this gets raised regularly by Xen users when I attend >> conferences. And I also heard a few times now that we appear to >> focus more >> on sexy features rather than the basics. This is relatively new >> though: the >> first time I noticed was towards the end of last year. Which does >> not mean >> that this is new: it may just mean that top issues/concerns that >> were >> frequently raised before (mainly trust in the future of the project) >> have >> disappeared. > > I''d really like to see examples of this - there ought to be quite a > few according to what you write, yet I don''t seem to recall any > that got plainly ignored. Of course there are always bugs which > take longer than others to figure out and fix.Not a bug per se, but arguably a relatively important feature; does the lack of feature to pass multiple USB devices to domU count as an example of this? Gordan
Jan, --On 14 June 2013 09:15:35 +0100 Jan Beulich <JBeulich@suse.com> wrote:>>>> On 13.06.13 at 23:03, Alex Bligh <alex@alex.org.uk> wrote: >> The thing we like the second least about Xen is how long it seems to take >> to get what we count as serious bugs fixed, even in stable releases. > > I think applicable bug fixes get applied to the stable branches in > quite timely a manner, at least on the hypervisor side. Hence I > can only assume that you''re unhappy with the rate of stable > releases. Yet I don''t think we''re going to get anywhere near of > the almost weekly stable releases that get done for Linux. As > you also didn''t say what expectations you really have, it''s hard > to take out anything useful from that complaint.I obviously miswrote some of this, as it wasn''t intended as a complaint, rather as observation. As with all open source code, the answer to ''it doesn''t work'' is (a) try to find out why, and (b) send code. Which we have on occasions tried to do. I''m not unhappy with the rate of stable releases, I am/was unhappy about the quality of stable releases (particularly 4.2). All our testing to date on 4.3 indicates it''s a lot higher quality release, at least for what we want it for. It''s possible that our expectations were incorrect, because we were using 4.2 to support qemu-upstream dm, and this was (IIRC) marked as a ''preview'' feature; however we were surprised things like live migrate were missing, and we had several ''killer'' bugs. We''ve not found that on 4.3. Saying that, we''ve tested 4.3 quite a lot on its own, but not with our agent code, because we''ve been trying to get 4.2 working properly first.>> We like it even less if we have to find them and fix them. > > But isn''t that how open source projects work - everyone contributes > and fixes bugs. If you don''t want to help fixing bugs, I''m afraid > there''s also no good reason for you to complain they don''t get fixed.I should have explicitly added the words ''in stable releases''. We''re perfectly happy to test, find bugs, etc. and indeed contribute fixes as we have done. However, we shouldn''t (in my opinion) be finding these in stable releases.>> By ''serious'' I mean basic functionality not working, crashes dom0, etc. > > When did we last break Dom0, and not fix it in a timely manner?Well, the ''fatal crash on Xen4.2 HVM'' thread that started on 14 Dec 2012 had the last fix committed on 5 Apr 2013, and I think came out in 4.2.2 on 23 April. Between those points, as far as I''m concerned anything running with network backed VMs was likely to crash dom0. That''s about half a 9 month release cycle.>> The result of this is two-fold. Firstly, we''ve never (yet) been able to >> run a production version of xen which is a standard xen release. We''ve >> always had to maintain our own patches even on ''stable'' releases. >> Frankly, this is a pain. > > But if you don''t contribute back your patches, how do you expect > them to get accepted/merged?We contributed back ALL of our patches. Every one. And we work pretty hard to get them merged too. The only one that hasn''t been merged or superseded by a better patch is the minideb patch, which I fully understand why it wasn''t merged, and is just packaging. I''m not complaining you drop my patches on the floor. In fact I''m not really complaining at all. What I''m pointing out is that if a ''stable'' is released with a nasty bug in, and it takes a while to find the right solution, it also takes a while to get a point release out that fixes it. The solution here is not to change the rate of acceptance of patches into a stable tree, but to ensure the bugs are caught before they make their way into a stable release.> Or are you saying that it''s _far_ > more than occasional that patches from you get entirely lost?Nope. -- Alex Bligh
Ian, --On 14 June 2013 10:46:31 +0100 Ian Murray <murrayie@yahoo.co.uk> wrote:>> As an example of the first, the API changed very significantly between >> 3.x and 4.1, and 4.1 and 4.2. Some of the API changes were subtle (e.g. >> what you had to do across a fork()). > > But surely, this is your business. This is what you do. Sounds like you > want API changes to be held back for the entire community so your company > has to do less work.I think forward compatible APIs are a good thing. This is not controversial. I believe there are few people in the Xen community who think not having forward compatible APIs was anything but a mistake.>> There''s nothing wrong with that, and we have done. However this thread >> is about development cycles. Critical stuff breaks, we know that. >> That should happen in the unstable version. Patches that fix critical >> stuff should be made in the unstable version, not the stable version. > > Do *you* test unstable and point out the critical bugs BEFORE it goes > stable? If you don''t, then don''t be surprised when bugs crop up...Yes we do. If you want a concrete example, go search for the migration wallclock lockup thread this month.> I looked on your website and couldn''t find any download section for > patches or a pointer to github or anything. I am not a lawyer but I > didn''t get the impression that sticking patches on a mailing list was > fulfilling ones obligation under the appropriate licence. So please can > you link from your company website to your github were these patches are > available.The fact it is not immediately obvious to you on a marketing web site does not mean it isn''t there. You''ll find it in our repo too (though not as a git repository which is actually what people want). Saying that, I have no objection at all to making this more obvious, and will do so. We''re proud of what we do with open source.> One aspect from your previous email I''d like to pickup on, when you > mentioned KVM not having these issues, are you compiling from the source > or using a distribution version? If the latter, then I think that is an > unfair comparison.We use a distribution version (we run Ubuntu throughout). I''m not quite sure why it''s an unfair comparison. If there was an Ubuntu version of 4.2.x for an LTS release, we''d be using that. Are you implying that the STABLE Xen version on xenbits should not be expected to be as reliable as the same version packaged by a maintainer? If so, why? -- Alex Bligh
Friday, June 14, 2013, 11:59:35 AM, you wrote:> Hi all,> I will go through the thread and try and answer question (where they have > not been answered and discussed), but will need the help of the people who > actually attended and drove the discussion at the Hackathon. I merely took > notes of what was said, and at times I missed things.> And no decision of any kind has been made : for this we do have a decision > making process. But this thread seems to show that we do have a number of > problems that need fixing. Maybe we need to crystallize this a little more.>>From Jan: >> From the summary above I didn''t really get what''s wrong with the current > 9 month cycle, which ... > I don''t think this was indeed covered. I think the rationale was really > covered in the thread. I also think I minuted the 2-3 weeks + 3 months > wrong (which seems to have been clarified).>>From Ben Guthro: >> Did you have any more info on this bullet, and the other one below Re: XenClient > / VirtualComputer? > The only thing I remember was that this discussion was originally started > by George and that IanC, Konrad, Stefano and Matt mostly covered it. We do > seem to have a problem with PCI passthrough - Pasi''s points. Maybe they > recall more detail.> I also wanted to add to the point that Alex has made on serious bugs in xen > vs. kvm : this gets raised regularly by Xen users when I attend > conferences. And I also heard a few times now that we appear to focus more > on sexy features rather than the basics. This is relatively new though: the > first time I noticed was towards the end of last year. Which does not mean > that this is new: it may just mean that top issues/concerns that were > frequently raised before (mainly trust in the future of the project) have > disappeared.One of the things mentioned below is the "break dom0", i think KVM has some advantage here. They could learn from the "mistakes" from Xen and make a new implementation from the ground up. One of the causes could be that the Xen project has had to make a giant leap on multiple fronts from the "(forced) choice / mistake (in hind sight perhaps)" to fork the Kernel and Qemu. Not to forget the tool-stack change that are also underway ... Especially since that leap had to be made on multiple fronts at once and has it''s interdepencies everywhere. In the mean time there is still the wish and effort to not go and lag behind feature-wise as well (arm port). Xen still seems to lag somewhat behind, although it doesn''t need a seperate "special" Xen kernel anymore since the PV parts have been upstreamed. But the giant leap, limited resource and wish to keep PV and functionality on par at one side, and the views and interest of the Linux community and maintainers on the other side seems to have: 1) Disgruntle some Linux maintainers 2) Made code and interfaces not always clean and well documented 3) Not always uses the current kernel interfaces in the ways it was expected (by the authors/maintainers) Which causes: 1) It makes patches from authors not familiar with Xen break things, this often happens in the merge window. 2) Maintenance burden, it requires quite some effort to keep up with the impact of (non primarily xen-targeted) changes in Linux and Qemu (which can''t be invested in nice new features) 3) A lot of legacy stuff perhaps hinders the way forward ... I must say Konrad does a tremendous job of keeping things together here and is *very* responsive ! I know Konrad is trying to clean this all up, refactor and document, but i think his time is split between quite some tasks and kernel parts. One of the things I question is the need to still support machines without hardware virtualization ? - Yes, it''s one of the main features compared to f.e. KVM - No, 32bit support seems to be on it''s way out, although there are (intel) 64bit processors without hardware virtualiztion, is that worth the effort ? But there doesn''t seem to be a "architectural-roadmap", which not only tells "WHAT features", but is more orientated on (desirably) "HOW to implement / refactor / reimplement" ? (so the grant-scheme of things so to say, and perhaps there can be things learned from KVM now ?) Could a shorter development cycle, together with more focus, get things going faster ? Perhaps the key is in more focus, focus can be seen in two ways: 1) focus of the complete release cycle (as a "theme"), f.e. do a tick-tock, tick=feature release cycle, tock=cleanup, code refactoring. 2) focus at front of which (limited number) of main features or cleanups the cycle focusses on, and make sure there is enough man-power in reserve to get these done. Any other feature can be included, but won''t be commited unless in such a state that is well tested and will "guarantee" to not make the release date shift. For the chosen mean features, it could be a debian style release when ready and happy. The cleanup/refactoring release cycle could be shorter, but will probably need more synchronization with the 2 main dependencies (linux/qemu) projects. So one release cycle would focus on new Xen-features and implements them in Xen and related projects (inside->out), the second release cycle focuses on cleaning up legacy and refactoring, cleaning up code and tries to relieve the maintenance burden especially with the related projects (outside->in). Perhaps this could suppress incomplete code and incomplete features between releases somewhat. Unfortunately, i''m not that good of a low-level programmer, that my efforts would contribute much code wise, but i try to contribute by testing new kernels early during the development cycle. (in the hope it''s easier to pinpoint the changes causing the bug and maximizing the time to fix things up) Just my few cents ... P.S.: I think the Xen community is quite responsive and nice to work with, and Xen as a whole works great for me :-) -- Sander> Lars> On Fri, Jun 14, 2013 at 9:15 AM, Jan Beulich <JBeulich@suse.com> wrote:>> >>> On 13.06.13 at 23:03, Alex Bligh <alex@alex.org.uk> wrote: >> > The thing we like the second least about Xen is how long it seems to take >> > to get what we count as serious bugs fixed, even in stable releases. >> >> I think applicable bug fixes get applied to the stable branches in >> quite timely a manner, at least on the hypervisor side. Hence I >> can only assume that you''re unhappy with the rate of stable >> releases. Yet I don''t think we''re going to get anywhere near of >> the almost weekly stable releases that get done for Linux. As >> you also didn''t say what expectations you really have, it''s hard >> to take out anything useful from that complaint. >> >> > We like it even less if we have to find them and fix them. >> >> But isn''t that how open source projects work - everyone contributes >> and fixes bugs. If you don''t want to help fixing bugs, I''m afraid >> there''s also no good reason for you to complain they don''t get fixed. >> >> > By ''serious'' I mean basic functionality not working, crashes dom0, etc. >> >> When did we last break Dom0, and not fix it in a timely manner? >> >> > The result of this is two-fold. Firstly, we''ve never (yet) been able to >> > run a production version of xen which is a standard xen release. We''ve >> > always had to maintain our own patches even on ''stable'' releases. >> > Frankly, this is a pain. >> >> But if you don''t contribute back your patches, how do you expect >> them to get accepted/merged? Or are you saying that it''s _far_ >> more than occasional that patches from you get entirely lost? >> >> Jan >> >>
>>> On 14.06.13 at 13:46, Alex Bligh <alex@alex.org.uk> wrote: > I''m not unhappy with the rate of stable releases, I am/was unhappy about > the quality of stable releases (particularly 4.2).Oh, okay, so we seem to mean slightly different things with "stable releases" - you seem to mean x.y.0 (in the current numbering scheme), whereas to me this means x.y.z, with z != 0. Indeed, there had been many bad bugs in 4.2.0, presumably at least partly because of its overlong development cycle. But I think we got most of them sorted out by now, with 4.2.2 being out.> All our testing to > date on 4.3 indicates it''s a lot higher quality release, at least > for what we want it for.Let''s hope for this to be true - a lower rate of fixes going into what I call stable releases would be very much to my taste.>>> We like it even less if we have to find them and fix them. >> >> But isn''t that how open source projects work - everyone contributes >> and fixes bugs. If you don''t want to help fixing bugs, I''m afraid >> there''s also no good reason for you to complain they don''t get fixed. > > I should have explicitly added the words ''in stable releases''. We''re > perfectly happy to test, find bugs, etc. and indeed contribute fixes > as we have done. However, we shouldn''t (in my opinion) be finding these > in stable releases.Which implies that the features you were after weren''t tested sufficiently before the major release went out. Which may (but doesn''t have to) be an indicator that you didn''t get to test them early enough.>>> By ''serious'' I mean basic functionality not working, crashes dom0, etc. >> >> When did we last break Dom0, and not fix it in a timely manner? > > Well, the ''fatal crash on Xen4.2 HVM'' thread that started on 14 Dec 2012 > had the last fix committed on 5 Apr 2013, and I think came out in 4.2.2 > on 23 April. Between those points, as far as I''m concerned anything > running with network backed VMs was likely to crash dom0. That''s about > half a 9 month release cycle.I participated in this discussion only very early on, and hence don''t really know which fixes for this got committed where and when. Which may mean that none of them were actually to the hypervisor. Jan
> > I think forward compatible APIs are a good thing. This is not > controversial. I believe there are few people in the Xen community > who think not having forward compatible APIs was anything but a mistake. >It certainly is a good thing, but if there is a need to change them then a major release is a good place to do it. Surely this is a governance issue, not a release cycle issue? I have no idea whether the API changes you descrbe were a "mistake" or not as I don''t use them.>>> There''s nothing wrong with that, and we have done. However this > thread >>> is about development cycles. Critical stuff breaks, we know that. >>> That should happen in the unstable version. Patches that fix critical >>> stuff should be made in the unstable version, not the stable version. >> >> Do *you* test unstable and point out the critical bugs BEFORE it goes >> stable? If you don''t, then don''t be surprised when bugs crop up... > > Yes we do. If you want a concrete example, go search for the migration > wallclock lockup thread this month. > >> I looked on your website and couldn''t find any download section for >> patches or a pointer to github or anything. I am not a lawyer but I >> didn''t get the impression that sticking patches on a mailing list was >> fulfilling ones obligation under the appropriate licence. So please can >> you link from your company website to your github were these patches are >> available. > > The fact it is not immediately obvious to you on a marketing web site > does not mean it isn''t there. You''ll find it in our repo too (though > not as a git repository which is actually what people want). > > Saying that, I have no objection at all to making this more obvious, > and will do so. We''re proud of what we do with open source. >Well, I spent more time than "immediate" to try to find them and if it''s not obvious where they are or they aren''t linked to, then they might as well not be there tbh. so where is this publically available repo?>> One aspect from your previous email I''d like to pickup on, when you >> mentioned KVM not having these issues, are you compiling from the source >> or using a distribution version? If the latter, then I think that is an >> unfair comparison. > > We use a distribution version (we run Ubuntu throughout). I''m not > quite sure why it''s an unfair comparison. If there was an Ubuntu version > of 4.2.x for an LTS release, we''d be using that. > > Are you implying that the STABLE Xen version on xenbits should not be > expected to be as reliable as the same version packaged by a maintainer? > If so, why?I seem to remember that a (perhaps old) Xen mantra was that it is a project and not a product. Ubuntu is a product, Xen is a project. RedHat 5 is a product and I seem to remember Xen being stable for my limited purposes. Your experience may have differed.> > -- > Alex Bligh > > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.xen.org > http://lists.xen.org/xen-devel >
Jan, --On 14 June 2013 13:26:59 +0100 Jan Beulich <JBeulich@suse.com> wrote:>>>> On 14.06.13 at 13:46, Alex Bligh <alex@alex.org.uk> wrote: >> I''m not unhappy with the rate of stable releases, I am/was unhappy about >> the quality of stable releases (particularly 4.2). > > Oh, okay, so we seem to mean slightly different things with "stable > releases" - you seem to mean x.y.0 (in the current numbering > scheme), whereas to me this means x.y.z, with z != 0. Indeed, > there had been many bad bugs in 4.2.0, presumably at least partly > because of its overlong development cycle. But I think we got most > of them sorted out by now, with 4.2.2 being out.Yes. I meant anything other than -unstable. And as you say, it may be (let''s hope!) that this is an echo from the past (i.e. 4.2) and 4.3 will be great.> Which implies that the features you were after weren''t tested > sufficiently before the major release went out. Which may (but > doesn''t have to) be an indicator that you didn''t get to test them > early enough.That''s partly the case, yes, and a fair point. We assumed the move to 4.2 was far easier than it turn out to be (because of API changes) and concentrated on getting our own stuff to work with 4.2, rather than trying 4.2 with xl etc. in anger. In our defence, all our torture tests test our own software controlling xen. However, I still feel the internal testing (automated or whatever) should have picked some of these up.>> Well, the ''fatal crash on Xen4.2 HVM'' thread that started on 14 Dec 2012 >> had the last fix committed on 5 Apr 2013, and I think came out in 4.2.2 >> on 23 April. Between those points, as far as I''m concerned anything >> running with network backed VMs was likely to crash dom0. That''s about >> half a 9 month release cycle. > > I participated in this discussion only very early on, and hence > don''t really know which fixes for this got committed where and > when. Which may mean that none of them were actually to the > hypervisor.I can''t remember now either. And the root of this is a kernel bug, which we are working around. However, this was a known problem: Ian Campbell (apologies if this was Ian J) discovered it in 2007 I think! (links at the top of the original thread if people are interested in the gory details). -- Alex Bligh
Ian, --On 14 June 2013 13:32:36 +0100 Ian Murray <murrayie@yahoo.co.uk> wrote:> I have no idea whether the API changes you descrbe were a "mistake" or > not as I don''t use them.I''m not saying the API changes were a mistake. I''m saying that it was a mistake that before 4.2 there was not a forward compatible API. Clearly there had to be some pain to fix that.> Well, I spent more time than "immediate" to try to find them and if it''s > not obvious where they are or they aren''t linked to, then they might as > well not be there tbh. > > so where is this publically available repo?repo.flexiant.com/ ..., which is installed by default for anyone using our binaries. You''ll obviously have found: http://docs.flexiant.com/display/DOCS/Open+Source+Software -- Alex Bligh
>> >> so where is this publically available repo? > > repo.flexiant.com/ ..., which is installed by default for anyone using > our binaries. > > You''ll obviously have found: > http://docs.flexiant.com/display/DOCS/Open+Source+SoftwareNope. Still can''t find a link from the main site. Besides, I am not sure if providing source via a repo who''s address isn''t publically published is really complying with your resposibilities, is it? Again, not a lawyer, just IMHO.> > -- > Alex Bligh >
On Fri, 2013-06-14 at 14:34 +0100, Ian Murray wrote:> > >> > >> so where is this publically available repo? > > > > repo.flexiant.com/ ..., which is installed by default for anyone using > > our binaries. > > > > You''ll obviously have found: > > http://docs.flexiant.com/display/DOCS/Open+Source+Software > > Nope. Still can''t find a link from the main site. > > Besides, I am not sure if providing source via a repo who''s address > isn''t publically published is really complying with your > resposibilities, is it?Assuming you are referring to the responsibilities of the GPL you should probably read the license (or at least do some basic investigation of the terms) to see what it actually requires before making these sorts of accusations. There are options for applying the GPL which don''t require anything like what you seem to expect. There''s certainly no requirement for a link on ones website when one uses GPL software. The text of the GPL is freely available. I strongly suggest you go read it, it''s not all that long nor is it particularly full of legalese. Accusing someone of a license infringement is rather a serious matter and you really should make sure you know what you are talking about before doing so. I think you are totally out of order hounding Alex like this.> Again, not a lawyer, just IMHO.Lawyer or otherwise I think you would be well advised to keep your opinions to yourself until you''ve actually read the license in question. Ian.
----- Original Message -----> From: Ian Campbell <Ian.Campbell@citrix.com> > To: Ian Murray <murrayie@yahoo.co.uk> > Cc: Alex Bligh <alex@alex.org.uk>; "xen-devel@lists.xen.org" <xen-devel@lists.xen.org> > Sent: Friday, 14 June 2013, 14:55 > Subject: Re: [Xen-devel] [Hackathon Minutes] Xen 4.4 Planning > > On Fri, 2013-06-14 at 14:34 +0100, Ian Murray wrote: >> >> >> >> >> so where is this publically available repo? >> > >> > repo.flexiant.com/ ..., which is installed by default for anyone using >> > our binaries. >> > >> > You''ll obviously have found: >> > http://docs.flexiant.com/display/DOCS/Open+Source+Software >> >> Nope. Still can''t find a link from the main site. >> >> Besides, I am not sure if providing source via a repo who''s address >> isn''t publically published is really complying with your >> resposibilities, is it? > > Assuming you are referring to the responsibilities of the GPL you should > probably read the license (or at least do some basic investigation of > the terms) to see what it actually requires before making these sorts of > accusations. > > There are options for applying the GPL which don''t require anything like > what you seem to expect. There''s certainly no requirement for a link on > ones website when one uses GPL software. > > The text of the GPL is freely available. I strongly suggest you go read > it, it''s not all that long nor is it particularly full of legalese. > > Accusing someone of a license infringement is rather a serious matter > and you really should make sure you know what you are talking about > before doing so. > > I think you are totally out of order hounding Alex like this. > >> Again, not a lawyer, just IMHO. > > Lawyer or otherwise I think you would be well advised to keep your > opinions to yourself until you''ve actually read the license in question. > > Ian. >My understanding is that source should be made available in a reasonable format to any 3rd party if you distribute modified code that is licenced to you via the GPL. Do I have that wrong? I was curious to see his patches to see what he was referring to and I seem to have had to go through hoops a bit. Not accusing anybody of anything... I merely asked the question. No meaning to hounding. Just trying to get an understanding. I am sure Alex has had to deal with worse than me in his time! :)
On Fri, 14 Jun 2013 15:44:50 +0100 (BST), Ian Murray <murrayie@yahoo.co.uk> wrote:> ----- Original Message ----- >> From: Ian Campbell <Ian.Campbell@citrix.com> >> To: Ian Murray <murrayie@yahoo.co.uk> >> Cc: Alex Bligh <alex@alex.org.uk>; "xen-devel@lists.xen.org" >> <xen-devel@lists.xen.org> >> Sent: Friday, 14 June 2013, 14:55 >> Subject: Re: [Xen-devel] [Hackathon Minutes] Xen 4.4 Planning >> >> On Fri, 2013-06-14 at 14:34 +0100, Ian Murray wrote: >>> >>> >> >>> >> so where is this publically available repo? >>> > >>> > repo.flexiant.com/ ..., which is installed by default for anyone >>> using >>> > our binaries. >>> > >>> > You'll obviously have found: >>> > http://docs.flexiant.com/display/DOCS/Open+Source+Software >>> >>> Nope. Still can't find a link from the main site. >>> >>> Besides, I am not sure if providing source via a repo who's >>> address >>> isn't publically published is really complying with your >>> resposibilities, is it? >> >> Assuming you are referring to the responsibilities of the GPL you >> should >> probably read the license (or at least do some basic investigation >> of >> the terms) to see what it actually requires before making these >> sorts of >> accusations. >> >> There are options for applying the GPL which don't require anything >> like >> what you seem to expect. There's certainly no requirement for a link >> on >> ones website when one uses GPL software. >> >> The text of the GPL is freely available. I strongly suggest you go >> read >> it, it's not all that long nor is it particularly full of legalese. >> >> Accusing someone of a license infringement is rather a serious >> matter >> and you really should make sure you know what you are talking about >> before doing so. >> >> I think you are totally out of order hounding Alex like this. >> >>> Again, not a lawyer, just IMHO. >> >> Lawyer or otherwise I think you would be well advised to keep your >> opinions to yourself until you've actually read the license in >> question. >> >> Ian. >> > > My understanding is that source should be made available in a > reasonable format to any 3rd party if you distribute modified code > that is licenced to you via the GPL. Do I have that wrong?IANAL, but: Reasonable format can mean anything. For example the kernel source for the Toshiba AC100 (ARM based Android laptop) was provided by Toshiba upon request on a CD by snail mail when those of us playing with the device requested it. If nobody is requesting the source, it arguably doesn't matter. But if somebody is asking for it, it must be provided. That's about the size of it as far as I can work out. Gordan _______________________________________________ Xen-devel mailing list Xen-devel@lists.xen.org http://lists.xen.org/xen-devel
On Fri, Jun 14, 2013 at 3:55 PM, Gordan Bobic <gordan@bobich.net> wrote:> Reasonable format can mean anything. For example the kernel source > for the Toshiba AC100 (ARM based Android laptop) was provided by > Toshiba upon request on a CD by snail mail when those of us playing > with the device requested it.In fact, when the GPL first came out, the internet was only available to a select few; it might have been considered an *unreasonble* format. Typically how you got the source code was writing a snail-mail letter, and being shipped a stack of floppies. And the requester had to pay for the floppies, the postage, and a reasonable handling fee. -George
On Fri, 2013-06-14 at 15:44 +0100, Ian Murray wrote:> My understanding is that source should be made available in a > reasonable format to any 3rd party if you distribute modified code > that is licenced to you via the GPL. Do I have that wrong?That is (roughly, although not exactly) one of the options for complying with the GPL, but it is not the only one. So, yes, you have it wrong. Again, the text of the license is freely available, so you could easily have checked what it says for yourself. Ian.
--On 14 June 2013 14:55:07 +0100 Ian Campbell <Ian.Campbell@citrix.com> wrote:> I think you are totally out of order hounding Alex like this.Despite which, and despite there being no legal obligation to do so, we actually like publishing the open source stuff we do, and the fact it''s not fantastically easy to find is well taken. I''m trying to find out who (hopefully someone internal) registered github.com/flexiant - if not I''ll use something else - and we will be putting all our open source stuff up there, and have a link from the open source page in the manual and the open source page on the website. This plus at least one of those links should be up in the next few days. Note this is because we want to publish stuff, not because we think there is a compliance issue. Ian M: had you said "please could you publish this more obviously" you would have got the same response. -- Alex Bligh
--On 14 June 2013 14:34:00 +0100 Ian Murray <murrayie@yahoo.co.uk> wrote:> Besides, I am not sure if providing source via a repo who''s address isn''t > publically published is really complying with your resposibilities, is > it? Again, not a lawyer, just IMHO.I suggest you read the requirements of the licence. -- Alex Bligh
On Fri, Jun 14, 2013 at 10:46:31AM +0100, Ian Murray wrote:> > > > > ----- Original Message ----- > > From: Alex Bligh <alex@alex.org.uk> > > To: Ian Murray <murrayie@yahoo.co.uk>; xen-devel@lists.xen.org > > Cc: Alex Bligh <alex@alex.org.uk> > > Sent: Friday, 14 June 2013, 8:01 > > Subject: Re: [Xen-devel] [Hackathon Minutes] Xen 4.4 Planning > > > > Ian, > > > > --On 14 June 2013 00:56:14 +0100 Ian Murray <murrayie@yahoo.co.uk> wrote: > > > >> When you talk about "stuff" breaking between major releases, are > > you > >> talking about Xen code not functioning or your code failing because of > >> changes in Xen? If the latter, are we talking designed changes in Xen''s > >> behaviour or non-designed ones (=bugs)? > > > > Both. > > > > As an example of the first, the API changed very significantly between > > 3.x and 4.1, and 4.1 and 4.2. Some of the API changes were subtle (e.g. > > what you had to do across a fork()). > > But surely, this is your business. This is what you do. Sounds like you want API changes to be held back for the entire community so your company has to do less work.That is not true. I know what Alex is referring. I used to work for VirtualIron and we had our own C based ''agent'' that would interface with libxc. Meaning we did not have ''xend'', nor any ''xl''. Launching of guests, etc was all done via our ''agent''. What we had issues was that when compiling against a new hypervisor API it all compiled, but launching guests was broken. What we found was that some structures had changed and their corresponding XEN_VERSION_SOMETHING changed as well. But it was not clear _which_ of the structures changed so it took a while to figure out that the cpumap had been changed (I think that is what it was). There was also some HVM parameter field that had to be added otherwise the guest would not boot (can''t remember the details). The XEN_VERSION_someting in the header doesn''t really say - what had changed. It is incremented to guard against this (so the call to the hypervisor will error out with -ENOSYS if you don''t use the right version)- but worst it does not provide very much a backwards compatible way of having a runtime work against Xen 4.1 vs Xen 4.3 (or Xen 3.4). And now looking back at Xen 4.3 I have to admin that I did this as well - so the xcinfo has an extra unsigned long in it expanding it. I am not really sure how to solve that. The ''libxl'' should provide a nice isolation layer for this kind of stuff and guard the consumers of the API with the #NEW_SOMETHING_FEATURE_ADDED defines.
On Thu, Jun 13, 2013 at 03:31:27PM +0100, Ian Campbell wrote:> On Thu, 2013-06-13 at 15:00 +0100, Lars Kurth wrote: > > == Stefano''s Proposal => > We should aim to reduce the release cycle to 4 months (or maybe 6 months > > as an intermediate step) from the current 9 months. A 4 months relase > > cycle should help accelerate development and lead to fewer patches being > > queued up. The implications are that we would have to operate a 2-3 > > weeks merge window. > > > > To do this, we would need to resolve a number of issues > > * It is likely that code reviews for such a short merge window would > > become a bottleneck. We are not sure whether this would be a major issue > > : the review bandwith would depend on the patches submitted (and their > > complexity) > > If we do end up going for this model then I think we need to also > inherit the principal from Linux that the merge window is for *merging* > and that the code in question should have already been posted and > reviewed *before* the window opens (i.e. in the other 3 months of the > cycle).Which would imply that if we use this for Xen 4.4, we would have almost no patches ready :-) That would be a painless release - just fixing the carryover bugs from Xen 4.3.> > This would be something of a cultural change because at the minute > people seem to assume that a freeze means that development and review > must stop. Personally I don''t think that should be the case anyway, > although I appreciate the need to focus peoples attention on the release > as well. > > This could end up with subsystem maintainers queueing patches into their > own tree for merge into mainline during the window (this is what Linux > does). This has its own downsides WRT the merging work during window and > there would need to be a change since we would also have to ask people > to test those trees, which they aren''t used to doing.That in the Linux ecosystem was solved with the linux-next which would take all of the subsystems''s tree.> > > * [I can''t remember who raised this] The concern was raised that we > > would not have enough x86 Xen reviewers got a 2-3 weeks merge window > > * [Konrad] Stated that in PVOPS for Linux contributions we don''t have a > > review bottleneck, > > Largely due to Linux having the properties I described above, IMHO. > Although I''m not a Linux maintainer of a large subsystem so what would I > know ;-)This is also b/c of the down-time - those three months can be used to thrash out bugs and be pedantic about peoples patches. A nice Linux release is when at rc1 there are no regressions that I know of and I can concentrate on developing a feature or clear some of my backlog. But that would imply that the maintainer (release maintainer?) of Xen would now be primarily responsible for chasing down folks to fix bugs. And if one does not have bugs, then there are nice three months to develop that killer feature. But that does not solve the problem of various bugs that exists or are detected by users and chasing those down. They unfortunatly take more time to troubleshoot and figure out than I personally would like. Perhaps the way to think about this is what we need to make sure works 100% between releases. There are a lot of pieces in it. I appreciate very much that there are folks who are willing to test it out during that time, report and bear with us trying to figure what got broken. But it would also help to have a regression test. I am very excited about the ''extensive testing framework'', but it is not ready yet. The osstest is pretty awesome too - is there other things we can add in to the testing framework to be able to do more of the regression tests? More OS-es?
--On 14 June 2013 13:25:38 -0400 Konrad Rzeszutek Wilk <konrad.wilk@oracle.com> wrote:> What we had issues was that when compiling against a new hypervisor > API it all compiled, but launching guests was broken. What we found was > that some structures had changed and their corresponding > XEN_VERSION_SOMETHING changed as well. But it was not clear _which_ of > the structures changed so it took a while to figure out that the cpumap > had been changed (I think that is what it was). There was also some HVM > parameter field that had to be added otherwise the guest would not boot > (can''t remember the details).That''s precisely it. Actually the one that REALLY bit us was far more subtle. We have a multithreaded, forking agent. In Xen 4.2 (but not 4.1) you have to call some post-fork function (whose name escapes me) in order to tell xen about a fork. Signal handlers have to be in a particular state (in particular SIGCHLD being set to ignore causes subtle problems). And to get various xl calls working (mainly server creation), it turns out you need to do a fork (assuming your client is multithreaded and other threads call other xen things even with different contexts); again this wasn''t necessary in 4.1. We have this working reliably now, but it was a voyage of discovery particularly as there are no multithreaded client examples, and whilst this is (sort of) explained in the header files, it''s a bit opaque. Seeing xl launch stuff, and our agent fail when it was passing byte for byte the same values was frustrating. If I felt I really understood how this worked (rather than discovered from trial and error and a lot of strace), I''d have sent a documentation patch. So this wasn''t a change in the API in the sense of structs changing or parameters changing (we caught that early at compile time), but in how the thing gets used in practice. I fully understand why the API was changed 4.2 -> 4.3, and really appreciate the fact it''s now stable (or at least advertised as such). Initial indications (i.e. it built first time without warnings) are good. -- Alex Bligh
On 14/06/13 16:43, Alex Bligh wrote:> > > --On 14 June 2013 14:55:07 +0100 Ian Campbell > <Ian.Campbell@citrix.com> wrote: > >> I think you are totally out of order hounding Alex like this. > > Despite which, and despite there being no legal obligation to do so, > we actually like publishing the open source stuff we do, and the > fact it''s not fantastically easy to find is well taken. I''m trying > to find out who (hopefully someone internal) registered > github.com/flexiant - if not I''ll use something else - and we will > be putting all our open source stuff up there, and have a link from > the open source page in the manual and the open source page on > the website. This plus at least one of those links should be up > in the next few days. > > Note this is because we want to publish stuff, not because we > think there is a compliance issue. Ian M: had you said "please > could you publish this more obviously" you would have got the > same response. >Thanks very much for that. As a casual browser, it will be a lot easier than the repo. Absolutely no rush. After reflecting on the conversation earlier, I realise it was ill-conceived to take it in the direction I did, so please accept my apologies for any distress that may have caused. Please also accept my apologies if you felt "hounded" (granted, not your word). This wasn''t my intention. Thanks again and have a good weekend. Ian (M).
About release time, I think probably a good choice could be the 3 months of development + 1 of testing but with Debian style: a minimum of 1 month for testing and, if there are critical/important bugs, one more week until it will be solved. About features for 4.4, I think that in the first post list some very good features already discussed are missing: - Improve hvm domUs integration adding configure parameters to allow usage of packaged qemu and seabios instead of compiling the internal one. About seabios and qemu upstream patch for configure already discussed with Ian Campbell, I tried to make a quick patch but failed (I couldn''t find the right configure parameters to do it and the exact part to change on libxl for qemu upstream binary path), I can''t find the discussion about it at the moment. - Add support for SSE and AVX for the hvm domU to improve performance and solve qxl problem (qxl requires SSE): http://bugs.xenproject.org/xen/bug/11 - Improve Spice support in libxl: * vdagent, working and tested, patch already posted: http://lists.xen.org/archives/html/xen-devel/2013-05/msg00484.html * usbredirection (different from usb passthrough: from spice client to qemu instead of from dom0 to qemu), working and tested, I want improve the patch before posting it, it doesn''t look good but I''ll post it anyway. * support auto select of port similar to vnc, I''ll probably try to make the patch. * support spice also on pv domU as an alternative to vnc, I don''t have enough knowledge to try to do it now. What do you think? Thanks for any reply
On Mon, 2013-06-17 at 10:27 +0200, Fabio Fantoni wrote:> - Improve hvm domUs integration adding configure parameters to allow > usage of packaged qemu and seabios instead of compiling the internal one. > About seabios and qemu upstream patch for configure already discussed > with Ian Campbell, I tried to make a quick patch but failed (I couldn''t > find the right configure parameters to do it and the exact part to > change on libxl for qemu upstream binary path), I can''t find the > discussion about it at the moment.I have a patch like this on my laptop (which is currently turned off at home) which I wrote on the train to the last hackathon. It needs cleaning up etc but I hope to post it in the 4.4 dev cycle. Ian.
--On 14 June 2013 16:43:41 +0100 Alex Bligh <alex@alex.org.uk> wrote:> Despite which, and despite there being no legal obligation to do so, > we actually like publishing the open source stuff we do, and the > fact it''s not fantastically easy to find is well taken. I''m trying > to find out who (hopefully someone internal) registered > github.com/flexiant - if not I''ll use something else - and we will > be putting all our open source stuff up there, and have a link from > the open source page in the manual and the open source page on > the website. This plus at least one of those links should be up > in the next few days.Just to close the loop on this, https://github.com/flexiant now is a single point for all our (that''s work''s - i.e. Flexiant''s) open source components, in git format (rather than tarballs), which is linked from our main open source page on our web site, and will be (as of mid next week) linked from our product manual. Probably more relevant to xen-devel, if I''m sticking patches up for review etc. they will still be in https://github.com/abligh which carries more branches, e.g. including code that doesn''t actually get released. Constructive criticism welcomed, but I would suggest it is off topic for xen-devel, so please send off list. -- Alex Bligh