Hi Ian, team, In the week of Feb 23 (time flies!:), we had a very productive day \o/ working on the stretch security/stable package and some irc discussions in oftc #debian-xen which ended with an open end. Ian, you were a bit surprised that 1) my debian-xen repo started with an import of a 4.8 package 2) it was not a git-pbuilder style repo with upstream source included, and asked me to at least make a conscious decision about why not doing those two things instead. To start with the conclusion: I indeed want to continue in the same way as the current repository on salsa is set up, since I believe it fits the type of package and work (and workflow for contributors) quite well. Summarizing that's: * packaging-only repo * debian changelog version is an unambiguous reference to an upstream version/commit * generated files (4.x version dependent) are in the git ignore * branching done like explained at https://salsa.debian.org/xen-team/debian-xen/wikis/home I'd appreciate if you just would let me do this for a while and see what track record we achieve while rolling forward. The rest of this wall of text is a collection of thoughts and things that happened in the last months. Feel free to read or ignore (but not ask questions which are answered here already) or let me know how I could have provided the same information using less words. -------- >8 -------- For Xen packaging, there are two different types of work running in parallel: * Following the latest greatest releases and putting them in Debian unstable (yes, there's actually an audience for that, and for stable-backports) * Provide fixes for security and stability issues for Debian stable. Also, for Xen packaging, there are two peculiarities which make it look a bit different from many other packages in Debian: * Instead of simple a linear development model, upstream supports multiple releases in parallel (e.g. 4.8, 4.9, 4.10). There is no path forward from a stable branch to the next release (e.g 4.10-stable -> 4.11) in upstream version control. * There are quite some binary packages, and a bunch of them have a version number in them, which changes when moving to a new 4.x release. In Nov/Dec 2017, the first thing I tried was searching for an existing packaging git repo. I could find an old repository on alioth, which had some packaging from 2014 and 2015, but nothing recent. Since the way of working in there was very different again (git-dpm I guess), and about 75% of the full commit history is the same set of patches over and over and over and over and over again, and since I could find only exacty one commit in there that has a bit more explanation than a single meaningless line of ("Update XYZ") which describes why the change is done and what it solves (092785a0d8), I respectfully ignored this and later archived it at https://salsa.debian.org/xen-team/xen-2014-2015 Since I couldn't find anything else, I assumed the current packages were just "based on changing the previous version" and then uploading that. Ubuntu already had a Xen 4.9 package, and based on that work, the first thing I tried was to create a git-pbuilder style repo with upstream + debian packaging. But, I quickly ran into a set of problems and inconveniences: * git-pbuilder can't handle the git-attributes replacement magic when creating an orig tar, so it always ends up with a failing build because of unexpected upstream changes. I couldn't figure out a way to work around this. * Having the upstream source attached to the packaging is not a good match for the parallel-development style repo that Xen upstream has. There is no path forward from e.g 4.10-stable to 4.11. Merge commits of upstream source make wip-rebasing, branching and merging of packaging changes between versions very awkward, noisy and highly meaningless. * The ubuntu package obviously had been constructed by modifying generated content (all the generated files and files with the 4.x version in their name) in place, and having these files around meant I had to rename them from 4.9 to 4.10, resulting in noisy error-prone non-optimal git commits with no default history for the new files. At that moment I discovered these version-dependent files could be generated, and when taking a closer look, I also discovered that the packaging looked much alike the packaging repo of the debian linux kernel team. Since I've been following and using their packaging for quite a bunch of years, I was pleasantly surprised about this. BOOM! What I did directly after this is weeding out all generated content, and that's where the repo started. I documented a few things in messages to the team mailing list: https://alioth-lists.debian.net/pipermail/pkg-xen-devel/2017-December/007118.html https://alioth-lists.debian.net/pipermail/pkg-xen-devel/2018-January/007153.html Those messages got zero feedback from anyone(!!), so I just continued working in that way. It was only in March 2018 when I learned about the dgit system and discovered the part of the packaging for the current Stretch release. However, after playing around for a few hours with dgit, I wasn't able to construct a local repository which made much sense. What I ended up with was multiple initial commits and unrelated branches based without finding out how to combine them into a meaningful history. There's undoubtedly a lot more to say, but this message is already too long :) Let useful discussions continue, but aside from that, I would really like to end up with an upload of Xen 4.10 to Debian Unstable soon. Hans
Hi Hans, Thanks for the detailed message, while I haven’t actually done any work on Xen in Debian (yet, well, aside from that small UEFI stuff), it does make a lot of sense to me to use the style currently used in Salsa, the old(er) system doesn’t track well with upstream indeed, and since Debian uses the same structure for the kernel packages I don’t see why continuing with the older style helps anyone. From my perspective: good job! John> On 30 Apr 2018, at 21:36, Hans van Kranenburg <hans at knorrie.org> wrote: > > Hi Ian, team, > > In the week of Feb 23 (time flies!:), we had a very productive day > \o/ working on the stretch security/stable package and some irc > discussions in oftc #debian-xen which ended with an open end. > > Ian, you were a bit surprised that 1) my debian-xen repo started with an > import of a 4.8 package 2) it was not a git-pbuilder style repo with > upstream source included, and asked me to at least make a conscious > decision about why not doing those two things instead. > > To start with the conclusion: I indeed want to continue in the same way > as the current repository on salsa is set up, since I believe it fits > the type of package and work (and workflow for contributors) quite well. > > Summarizing that's: > * packaging-only repo > * debian changelog version is an unambiguous reference to an upstream > version/commit > * generated files (4.x version dependent) are in the git ignore > * branching done like explained at > https://salsa.debian.org/xen-team/debian-xen/wikis/home > > I'd appreciate if you just would let me do this for a while and see what > track record we achieve while rolling forward. > > The rest of this wall of text is a collection of thoughts and things > that happened in the last months. Feel free to read or ignore (but not > ask questions which are answered here already) or let me know how I > could have provided the same information using less words. > > -------- >8 -------- > > For Xen packaging, there are two different types of work running in > parallel: > * Following the latest greatest releases and putting them in Debian > unstable (yes, there's actually an audience for that, and for > stable-backports) > * Provide fixes for security and stability issues for Debian stable. > > Also, for Xen packaging, there are two peculiarities which make it look > a bit different from many other packages in Debian: > * Instead of simple a linear development model, upstream supports > multiple releases in parallel (e.g. 4.8, 4.9, 4.10). There is no path > forward from a stable branch to the next release (e.g 4.10-stable -> > 4.11) in upstream version control. > * There are quite some binary packages, and a bunch of them have a > version number in them, which changes when moving to a new 4.x release. > > In Nov/Dec 2017, the first thing I tried was searching for an > existing packaging git repo. > > I could find an old repository on alioth, which had some packaging from > 2014 and 2015, but nothing recent. Since the way of working in there was > very different again (git-dpm I guess), and about 75% of the full commit > history is the same set of patches over and over and over and over and > over again, and since I could find only exacty one commit in there that > has a bit more explanation than a single meaningless line of ("Update > XYZ") which describes why the change is done and what it solves > (092785a0d8), I respectfully ignored this and later archived it at > https://salsa.debian.org/xen-team/xen-2014-2015 > > Since I couldn't find anything else, I assumed the current packages > were just "based on changing the previous version" and then uploading > that. Ubuntu already had a Xen 4.9 package, and based on that work, the > first thing I tried was to create a git-pbuilder style repo with > upstream + debian packaging. > > But, I quickly ran into a set of problems and inconveniences: > * git-pbuilder can't handle the git-attributes replacement magic when > creating an orig tar, so it always ends up with a failing build because > of unexpected upstream changes. I couldn't figure out a way to work > around this. > * Having the upstream source attached to the packaging is not a good > match for the parallel-development style repo that Xen upstream has. > There is no path forward from e.g 4.10-stable to 4.11. Merge commits of > upstream source make wip-rebasing, branching and merging of packaging > changes between versions very awkward, noisy and highly meaningless. > * The ubuntu package obviously had been constructed by modifying > generated content (all the generated files and files with the 4.x > version in their name) in place, and having these files around meant I > had to rename them from 4.9 to 4.10, resulting in noisy error-prone > non-optimal git commits with no default history for the new files. > > At that moment I discovered these version-dependent files could be > generated, and when taking a closer look, I also discovered that the > packaging looked much alike the packaging repo of the debian linux > kernel team. Since I've been following and using their packaging for > quite a bunch of years, I was pleasantly surprised about this. BOOM! > > What I did directly after this is weeding out all generated content, and > that's where the repo started. > > I documented a few things in messages to the team mailing list: > > https://alioth-lists.debian.net/pipermail/pkg-xen-devel/2017-December/007118.html > https://alioth-lists.debian.net/pipermail/pkg-xen-devel/2018-January/007153.html > > Those messages got zero feedback from anyone(!!), so I just continued > working in that way. > > It was only in March 2018 when I learned about the dgit system and > discovered the part of the packaging for the current Stretch release. > However, after playing around for a few hours with dgit, I wasn't able > to construct a local repository which made much sense. What I ended up > with was multiple initial commits and unrelated branches based without > finding out how to combine them into a meaningful history. > > There's undoubtedly a lot more to say, but this message is already too > long :) > > Let useful discussions continue, but aside from that, I would really > like to end up with an upload of Xen 4.10 to Debian Unstable soon. > > Hans > > _______________________________________________ > Pkg-xen-devel mailing list > Pkg-xen-devel at alioth-lists.debian.net > https://alioth-lists.debian.net/cgi-bin/mailman/listinfo/pkg-xen-devel
Hans van Kranenburg writes ("debian-xen git workflow"):> I'd appreciate if you just would let me do this for a while and see what > track record we achieve while rolling forward.OK.> The rest of this wall of text is a collection of thoughts and things > that happened in the last months. Feel free to read or ignore (but not > ask questions which are answered here already) or let me know how I > could have provided the same information using less words. > > -------- >8 --------I have read everything you wrote. Thanks for the writeup. I will only reply to the bits I had any kind of comment on: ...> But, I quickly ran into a set of problems and inconveniences: > * git-pbuilder can't handle the git-attributes replacement magic when > creating an orig tar, so it always ends up with a failing build because > of unexpected upstream changes. I couldn't figure out a way to work > around this.The gitattributes are very annoying. My approach with the security releases has been to (i) completely disable them (ii) generate my own orig tarballs using appropriate `git archive' runes.> * Having the upstream source attached to the packaging is not a good > match for the parallel-development style repo that Xen upstream has. > There is no path forward from e.g 4.10-stable to 4.11. Merge commits of > upstream source make wip-rebasing, branching and merging of packaging > changes between versions very awkward, noisy and highly meaningless.I have a new tool (`git-debrebase') in development which may make some of this easier, but I don't get the impression that you want to be an early adopter of novel git tools, so think it would be best for you to avoid it until it's more mature.> It was only in March 2018 when I learned about the dgit system and > discovered the part of the packaging for the current Stretch release. > However, after playing around for a few hours with dgit, I wasn't able > to construct a local repository which made much sense. What I ended up > with was multiple initial commits and unrelated branches based without > finding out how to combine them into a meaningful history.The stretch branches are not in a very convenient state for anyone but the most hardcore git expert. My apologies for that.> Let useful discussions continue, but aside from that, I would really > like to end up with an upload of Xen 4.10 to Debian Unstable soon.Yes. Regards, Ian. -- Ian Jackson <ijackson at chiark.greenend.org.uk> These opinions are my own. If I emailed you from an address @fyvzl.net or @evade.org.uk, that is a private address which bypasses my fierce spamfilter.
On 05/14/2018 04:52 PM, Ian Jackson wrote:> Hans van Kranenburg writes ("debian-xen git workflow"): >> I'd appreciate if you just would let me do this for a while and see what >> track record we achieve while rolling forward. > > OK.Thanks for the feedback.> I have read everything you wrote. Thanks for the writeup. I will > only reply to the bits I had any kind of comment on:Doing the same now.>> * Having the upstream source attached to the packaging is not a good >> match for the parallel-development style repo that Xen upstream has. >> There is no path forward from e.g 4.10-stable to 4.11. Merge commits of >> upstream source make wip-rebasing, branching and merging of packaging >> changes between versions very awkward, noisy and highly meaningless. > > I have a new tool (`git-debrebase') in development which may make some > of this easier, but I don't get the impression that you want to be an > early adopter of novel git tools, so think it would be best for you to > avoid it until it's more mature.For sure I'm interested in learning more about this, and if I can, contribute some testing time and ideas to help. One of the things I really like about the dgit approach is that you work towards getting rid of manual actions needed to manage debian/patches. I think we agree that just using git is less of a headache than vim and quilt if you want to make custom changes to upstream. However, the downside of this approach is that it instantly gets less obvious for someone browsing the history to understand which git commits are part of the real upstream code or were cherry-picked by a maintainer, because they look exactly the same. This works when you're a sole maintainer and have this information in your head, or you can reverse engineer this information by looking at debian/patches again, but meh... :| And, updating to a new upstream version means that you can end up with conflicts caused by changes to upstream added by yourself... In the case of Xen, I've been looking at and thinking about the current patches a bit. * Most of the patches are related to the abiname/prefix thing, and we already know that we'd like to find a solution for this which can be upstreamed. I'm not an expert in this area, but I will try to help in whatever way I can when it's going to happen. Also: [0] * There might be some cherry-picked patches, like including the xen-diag tool for 4.8 now (#880554), but those kind of patches are not the painful ones to add manually to debian/patches. Just use the right git format patch command and boom!, it's done. * And, I'd really like to support a "fewer patches is better" culture, trying to build a good relationship with upstream, so that instead of piling up patches, there can be some discussion back and forth about what issues users run into and how they can be solved upstream. Adding patches to the debian packaging should be a last resort.>> It was only in March 2018 when I learned about the dgit system and >> discovered the part of the packaging for the current Stretch release. >> However, after playing around for a few hours with dgit, I wasn't able >> to construct a local repository which made much sense. What I ended up >> with was multiple initial commits and unrelated branches based without >> finding out how to combine them into a meaningful history. > > The stretch branches are not in a very convenient state for anyone but > the most hardcore git expert. My apologies for that.Well, as you told me earlier, you kind of adopted the packaging because it got abandoned, and at least made sure there were new security updates for users. Yay for that! If I was in the same position and also developing new tools for package management at the same time, I think I would also have gone the way of trying them on whatever I was working on. :)>> Let useful discussions continue, but aside from that, I would really >> like to end up with an upload of Xen 4.10 to Debian Unstable soon. > > Yes.\o/ Can you share your thoughts about what work we need to do to reach this goal? I'm currently already running the Xen 4.10 packages at work in a pre-production environment. Except for some serious major breakage [1] it just works (tm) as long as we don't try to use live migrate. One thing I noticed is that there are an awful lot of warnings and interesting messages spewed out during the build, also related to debian-specific things. And, the packages I end up also report a quite interesting list of lintian errors and warnings. I think the lintian report is at least something that we should have a look at. I can run this on my most recent packages and compose a message to the list in which I try to triage them. Or, I could just split them out and create gitlab issues for every one of them, in which anyone can add a thought about how important it is and how it could be solved? What do you think? Hans [0] I have been wondering... and I haven't tried to test this myself yet... This hypervisor/utils ABI which the debian packaging relies on... Is there some upstream test in the test suite to make sure this does not change during a 4.x release? I can imagine that making huge changes like the spectre/meltdown stuff might accidentally change this. How is this managed? [1] At Mendix, we've ran into a very nasty live migration problem, leading to corruption in the dom0 network stack (or worse things looking like random memory corruption causing spectacular crashes and even disk curruption in the domU) and haven't been able to trace down the exact cause yet. However, we can reproduce the same issue with Xen 4.8 from Debian Stable and are really puzzled about what's so different about our setup that causes this. Since we can reproduce using the current stretch xen 4.8 and kernel in dom0, I don't think this should be a total blocker for 4.10 to enter unstable, because all other users of it are probably not hitting this problem? (insert puzzled face icon here)
Hi, On 05/01/2018 12:59 AM, John Keates wrote:> > Thanks for the detailed message, while I haven’t actually done any > work on Xen in Debian (yet, well, aside from that small UEFI stuff), > it does make a lot of sense to me to use the style currently used in > Salsa, the old(er) system doesn’t track well with upstream indeed, > and since Debian uses the same structure for the kernel packages I > don’t see why continuing with the older style helps anyone. From my > perspective: good job!Thanks! Did you try to build and run 4.10 Xen packages with your change in it already? "Doing work on Xen in Debian" might start with just asking around if there's something you could help with, or, mentioning things that are missing and getting them added, like you did already. Don't underestimate the fact that you're a user who is actually using that "small UEFI stuff". There's a huge amount of different use cases and there's a huge amount of configuration options and all of them result in a specific user doing specific things with it, which could result in some weird specific bug that they're running into. So, don't assume someone else will also be running the exact same Xen setup as you do, and let your voice be heard if you encounter anything. :) Have fun, Hans