This may be more work than is worthwhile, but I just had an idea to make the build process a lot more flexible. What if the Makefile looped through all config-* files found in install/boot and built a kernel based on each config? That way you could mix and match kernel versions simply by having prepared configs, eg 2.6.8.1-xen0, 2.6.8.1-xenU, 2.4.27-xenU. Also, whatever happened to the windows XP port? James ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
> What if the Makefile looped through all config-* files found in > install/boot and built a kernel based on each config? That way you could > mix and match kernel versions simply by having prepared configs, eg > 2.6.8.1-xen0, 2.6.8.1-xenU, 2.4.27-xenU.I''d certainly like to nuke the LINUX_RELEASE mechanism and have a list of image types that should be built e.g. "linux-2.4 linux-2.6 netbsd-2.0 freebsd". For each of these, I guess we could see whether there are appropriate config file(s) present in install/boot and then loop building them all. If there are no suitable config files, it could generate some defaults. However, this scheme wouldn''t deal with applying different patches to each kernel. We''d certainly welcome suggestions (and patches!) as to how to improve the build process and better integrate with building deb/rpm packages. However, we should avoid "over-automization": it should still be straightforward for someone who''s familiar with running ''make xconfig'' and customising their own Linux kernel to still do so. I haven''t had a chance to look at Brian''s working in building deb packages, but is there stuff here which we should be pulling into the top-level makefile?> Also, whatever happened to the windows XP port?The existing windows XP port was done within the University using windows source code, which has meant that even binaries of the port can only be released to people that have signed the same source licence. Since there wasn''t any likelihood of a more general release of Xen-XP being possible, we haven''t kept the port up to date with current releases of Xen. We''d obviously like to get XP running on Xen in a form that can be generally released, and are looking into a number of options. We hope to add Xen support for the forthcoming Intel VT hardware extensions, which we believe should enable Xen to support vanilla unmodified operating systems images. Even with the extra hardware support, there''ll still be an inevitable performance penalty of not doing proper para-virtualisation, but at least it will enable legacy images to be supported. We''d hope that all new OS installationss would use kernels containing para-virtualised extensions. Ian ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Tue, 2004-09-21 at 02:25, Ian Pratt wrote:> > What if the Makefile looped through all config-* files found in > > install/boot and built a kernel based on each config? That way you could > > mix and match kernel versions simply by having prepared configs, eg > > 2.6.8.1-xen0, 2.6.8.1-xenU, 2.4.27-xenU. > > I''d certainly like to nuke the LINUX_RELEASE mechanism and have a > list of image types that should be built e.g. > "linux-2.4 linux-2.6 netbsd-2.0 freebsd".This is sort of what I am working towards with xenlinux-builder. I''ve based it off of make, so it should be easy to integrate it into the main package as a utility program. The idea was to create and manage individual config files for discretely separate kernel images during and post xen compilation. Right now it''s only in a deb format, and only works with the debian xen-kernel-patch .deb package for applying the sparse tree. (xen-kernel-patch is a unified diff of the sparse tree patches.) I would like to argue for the benifit of converting the sparse patches to diffs in the build process since the sparse patch uses symlinks, which get damaged once you start applying 3rd party patches to the resulting kernel source tree. Due to the size of the diffs, I don''t think we want to distribute 4+ diffs with lots of duplicate patching. Maybe we can split the diff into the globally common portion and the kernel and version specific portions. This would make it feasible to generate unified diffs instead of sparse patches without bulking up the xen source tree too much. The second reason for the unified diffs vs sparse patches is that everyone understands them. True, everyone understands the sparse patches, but if a distro builds both a 2.4.27 AND a 2.6.8.1 kernel, then the sparse patching with symlinks WILL walk on each other when the distro applies it''s own patches (see the debianizing stuff of the xen source diff for debian for the temp solution of converting to unified diff prior to creating the kernel images).> > For each of these, I guess we could see whether there are > appropriate config file(s) present in install/boot and then loop > building them all. If there are no suitable config files, it > could generate some defaults. However, this scheme wouldn''t deal > with applying different patches to each kernel. > > We''d certainly welcome suggestions (and patches!) as to how to > improve the build process and better integrate with building > deb/rpm packages. However, we should avoid "over-automization": > it should still be straightforward for someone who''s familiar > with running ''make xconfig'' and customising their own Linux > kernel to still do so. > > I haven''t had a chance to look at Brian''s working in building deb > packages, but is there stuff here which we should be pulling into > the top-level makefile?IMHO yes. I''m still fairly new at packaging, so don''t take anythign that I create as debian gospel. ;) I rely on Adam to brow beat me anytime I fsck things up. *grin* Though I am getting better. I would suggest the first step of examining the sparse patch conversion to unified diff and xenlinux-builder package. These are the primary weak points in making xen portable between distros and work cleaner with packaging. It is NOT cleanly integrated -yet-. I need to study BK some more and determine how to easilly manage importing the main tree patch sets into a local repository until I get something decently stable for inclusion into the main tree.> > > Also, whatever happened to the windows XP port? > > The existing windows XP port was done within the University using > windows source code, which has meant that even binaries of the > port can only be released to people that have signed the same > source licence. Since there wasn''t any likelihood of a more > general release of Xen-XP being possible, we haven''t kept the > port up to date with current releases of Xen. > > We''d obviously like to get XP running on Xen in a form that can > be generally released, and are looking into a number of > options. We hope to add Xen support for the forthcoming Intel VT > hardware extensions, which we believe should enable Xen to > support vanilla unmodified operating systems images. Even with > the extra hardware support, there''ll still be an inevitable > performance penalty of not doing proper para-virtualisation, but > at least it will enable legacy images to be supported. We''d hope > that all new OS installationss would use kernels containing > para-virtualised extensions.This is going to take someone that is familiar with the business process of getting MS to play ball. 8-( I don''t think that I would trust them to do anything internally just yet. In my experience and info from contacts they aren''t that great at making their stuff play ball with other people''s work. They prefer the reverse by far and thus have far more experience (and tendancy to do it this way). 8-P> > Ian > > > ------------------------------------------------------- > This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 > Project Admins to receive an Apple iPod Mini FREE for your judgement on > who ports your project to Linux PPC the best. Sponsored by IBM. > Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php > _______________________________________________ > Xen-devel mailing list > Xen-devel@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/xen-devel------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Tue, Sep 21, 2004 at 12:46:35PM -0500, Brian Wolfe wrote:> The second reason for the unified diffs vs sparse patches is that > everyone understands them. True, everyone understands the sparse > patches, but if a distro builds both a 2.4.27 AND a 2.6.8.1 kernel, then > the sparse patching with symlinks WILL walk on each other when the > distro applies it''s own patches (see the debianizing stuff of the xen > source diff for debian for the temp solution of converting to unified > diff prior to creating the kernel images).You could change the mkbuildtree scripts to copy the files instead of creating symlinks. That should allow you to apply different patches to multiple trees. christian ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Tue, 2004-09-21 at 15:42, Christian Limpach wrote:> On Tue, Sep 21, 2004 at 12:46:35PM -0500, Brian Wolfe wrote: > > The second reason for the unified diffs vs sparse patches is that > > everyone understands them. True, everyone understands the sparse > > patches, but if a distro builds both a 2.4.27 AND a 2.6.8.1 kernel, then > > the sparse patching with symlinks WILL walk on each other when the > > distro applies it''s own patches (see the debianizing stuff of the xen > > source diff for debian for the temp solution of converting to unified > > diff prior to creating the kernel images). > > You could change the mkbuildtree scripts to copy the files instead > of creating symlinks. That should allow you to apply different patches > to multiple trees. > > christianThat still doesn''t fix the issue of the sparse patch replacing files that may have been altered by the distro. 8-( As it stands now you have to sparse-patch fist, THEN apply any other patches that you want. This is an undesireable situation to be in when there are many other things to patch. (bug fixes, stadard features specific to this distro, etc). IMO replacing files and adding new ones is a bad way of distributing changes to a standard software package when a beter way IMHO exists. The only dificulty is in that if we had a diff for each tree, then they would be large, and splitting the diffs based on what''s common, and what isn''t, is tough. 8-P I wonder if I could make a script that takes the individual diffs nd pulls out the unique alterations into their own files and leaves a single large common diff.... ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Tue, Sep 21, 2004 at 04:04:28PM -0500, Brian Wolfe wrote:> That still doesn''t fix the issue of the sparse patch replacing files > that may have been altered by the distro. 8-( As it stands now you have > to sparse-patch fist, THEN apply any other patches that you want. This > is an undesireable situation to be in when there are many other things > to patch. (bug fixes, stadard features specific to this distro, etc).Ah I see, I''ve never used a distribution patched kernel. Won''t you then also need to apply patches which the distribution applies to files in arch/i386 to the corresponding files in arch/xen?> IMO replacing files and adding new ones is a bad way of distributing > changes to a standard software package when a beter way IMHO exists. The > only dificulty is in that if we had a diff for each tree, then they > would be large, and splitting the diffs based on what''s common, and what > isn''t, is tough. 8-P I wonder if I could make a script that takes the > individual diffs nd pulls out the unique alterations into their own > files and leaves a single large common diff....To reduce size, you want to have diffs for: - changes we''ve made to files outside of arch/xen, include/asm-xen and drivers/xen. (for 2.6 that''s 9.9K) - files in (2.6'') arch/xen/i386 and in include/asm-xen/asm-i386, diff against arch/i386 and include/asm-i386 respectively (254K, 75K gzip''ed, 41K of the uncompressed size is the diff for arch/i386/Kconfig) All these will be quite specific to the tree they are supposed to be applied to. Everything else is new files which won''t have distribution applied patches and thus it''s probably preferable to use sparse files. If you apply the 2nd diff after copying the files from your distribution patched kernel, you should be able to pull their changes to arch/i386 and include/asm-i386 over to arch/xen/i386 and include/asm-xen/asm-i386 (unless they conflict). christian ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Tue, 21 Sep 2004, Brian Wolfe wrote:> IMO replacing files and adding new ones is a bad way of distributing > changes to a standard software package when a beter way IMHO exists. The > only dificulty is in that if we had a diff for each tree, then they > would be large, and splitting the diffs based on what''s common, and what > isn''t, is tough. 8-P I wonder if I could make a script that takes the > individual diffs nd pulls out the unique alterations into their own > files and leaves a single large common diff....interdiff ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel
On Tue, 21 Sep 2004, Christian Limpach wrote:> On Tue, Sep 21, 2004 at 04:04:28PM -0500, Brian Wolfe wrote: > > That still doesn''t fix the issue of the sparse patch replacing files > > that may have been altered by the distro. 8-( As it stands now you have > > to sparse-patch fist, THEN apply any other patches that you want. This > > is an undesireable situation to be in when there are many other things > > to patch. (bug fixes, stadard features specific to this distro, etc). > > Ah I see, I''ve never used a distribution patched kernel. Won''t you > then also need to apply patches which the distribution applies > to files in arch/i386 to the corresponding files in arch/xen?I''ve used csplit in the past to break up diffs. I had a conversion script that could split a debian diff.gz into upstream changes, and everything in debian/. I then recombined the latter part, and turned it into a tarball, while moving the former part into debian/patches/(I''m the original author of dbs). ------------------------------------------------------- This SF.Net email is sponsored by: YOU BE THE JUDGE. Be one of 170 Project Admins to receive an Apple iPod Mini FREE for your judgement on who ports your project to Linux PPC the best. Sponsored by IBM. Deadline: Sept. 24. Go here: http://sf.net/ppc_contest.php _______________________________________________ Xen-devel mailing list Xen-devel@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/xen-devel