Fedora kernel list - Mar 2007 - Spinning kernel-vanilla packages via standard spec

A little follow-on to the "Longing for git-bisect" thread, in a
newly-created/subjected thread...

So, I''ve been poking around in the kernel spec file extensively the
past
few days already... Any objections to my attempting to implement the
building of a kernel-vanilla package as well?

Roland, not sure if you were already planning to do this in conjunction
with digging up your old patches, or if you were just throwing the
patches out there, but either way, I''m happy to take on the task, but
I''ll defer to anyone else that really wants to do it.

-- 
Jarod Wilson
jwilson@redhat.com

I already got it working and posted the spec and Makefile patch here.
(I''ve also got my stuff for building from git branches working now.)
Nobody answered about whether they wanted it committed.

I didn''t do it in a way intended to produce different variant rpms
called
kernel-vanilla-V-R, if that''s what you had in mind.  That requires
basically duplicating all the innards of the spec file.  I don''t think
that
should be attempted without a complete cleanup of the spec file to become
sane.

What I''ve done is sufficient for making a vanilla build like this one:
http://porkchop.devel.redhat.com/brewroot/scratch/roland/task_700227/


Thanks,
Roland

Roland McGrath wrote:
> I already got it working and posted the spec and Makefile patch here.
Gah, sorry, I saw the patch, but missed that the spec mods were included 
in it too.
> (I''ve also got my stuff for building from git branches working
now.)
> Nobody answered about whether they wanted it committed.
> 
> I didn''t do it in a way intended to produce different variant rpms
called
> kernel-vanilla-V-R, if that''s what you had in mind.
It is. And I was under the impression that''s what Dave was thinking
too,
but I''ll let him speak for himself.
> That requires
> basically duplicating all the innards of the spec file.  I don''t
think that
> should be attempted without a complete cleanup of the spec file to become
> sane.
More specifically, what I had in mind was to add just one more variant, 
so you''d have kernel, kernel-xen, kernel-kdump, kernel-PAE, etc., plus 
now a kernel-vanilla, which would be analogous to kernel minus patches 
-- i.e., don''t bother with kernel-vanilla-xen, kernel-vanilla-kudmp,
etc.

Not to say that a spec file sanitization/sane-itization wouldn''t be
good
to do as well (which I''m also perfectly willing to take on -- a few 
steps in this direction were taken earlier today).
> What I''ve done is sufficient for making a vanilla build like this
one:
> http://porkchop.devel.redhat.com/brewroot/scratch/roland/task_700227/
Cool, I''ll read your patch more carefully now. :)

-- 
Jarod Wilson
jwilson@redhat.com

> It is. And I was under the impression that''s what Dave was
thinking too,
> but I''ll let him speak for himself.
Well, do we want it in the package set?  I figured we were talking about
"informal" builds that might be put up for ftp, but not be an
integrated
part of the Fedora world.  If one rpmbuild from the spec kicks out -vanilla
packages along with other variants, and we turn that on only for rawhide,
still it winds up "in the system", people install it as
"normal" and wind
up with upgrade issues and all that.  Eventually we have clueless Fedora
users saying "you use -PAE? well -vanilla is the Fedora kernel I like
best!"
and reporting in bugs "why this fix isn''t in -vanilla too?"

It''s easy to tweak my changes to put "vanilla" into %name
instead of
%release.  I went with "kernel" and a long, funky %release because
that''s
the tradition I''ve experienced for one-off and test builds people
distribute.  From my own experience juggling a dozen installations on two
or three test boxes with many hack kernels installed each in an utterly
disorganized fashion, it seems more likely to get resolved usefully when
doing upgrades and such than an rpm named "kernel-foobar".  That is,
yum
and rpm will get in your face all the time in the way that people are used
to for juggling kernel rpms.  (Also, I always shudder in superstitious fear
that yum or who knows what will assume things about the kernel rpm being
the way it always has been.)

With whatever name, separate vanilla builds could easily be made usable as
their own little yum repository.

What is more work, and I think is more dubious, is having it be the same
srpm that produces both fedora and vanilla builds.  What will -bp do, make
two source trees patched differently?
> > That requires
> > basically duplicating all the innards of the spec file.  I
don''t think that
> > should be attempted without a complete cleanup of the spec file to
become
> > sane.
> 
> More specifically, what I had in mind was to add just one more variant, 
> so you''d have kernel, kernel-xen, kernel-kdump, kernel-PAE, etc.,
plus
> now a kernel-vanilla, which would be analogous to kernel minus patches 
> -- i.e., don''t bother with kernel-vanilla-xen,
kernel-vanilla-kudmp, etc.
That''s no fun!  Seriously, I think we need the full suite of variants
(modulo xen until that''s uptsream, etc.) for it really to be useful.
Plenty of people are using -PAE and the point is to compare the foo
you''re
using to foo-vanilla on a one-to-one basis, i.e. different source, same
.config settings.  And you want -debuginfo for all of those too.
> Not to say that a spec file sanitization/sane-itization wouldn''t
be good
> to do as well (which I''m also perfectly willing to take on -- a
few
> steps in this direction were taken earlier today).
I despise how much duplication there is in there.  The debuginfo stuff I
originally did with rpm macros got hand-expanded because older rpmbuild
couldn''t cope with it.  Maybe nowadays there is nothing with an
rpmbuild
older than FC-5 we are trying to work with, so we could use more macros for
new stuff.  (I think brew got fixed to use the buildroot''s rpmbuild, so
anything that''s fine for the target systems sharing the kernel spec
source
should be fine.)  rpm macros are a hairy mess, but cleaning things up with
clean new macros is a better bet than anything else, IMHO.
> Cool, I''ll read your patch more carefully now. :)
Here''s the current version of it.  You still need the nonintconfig
patch
variant I posted last time to go with this.  I also attached below the
shell script used by the GIT-using hacks that are in this version.
http://porkchop.devel.redhat.com/brewroot/scratch/roland/task_703101/ shows
what "make git/roland/scratch-build" does (the build magic works for
"make
git/roland-fedora/..." too, but the fedora patches don''t apply to
current
upstream so it didn''t build).  This (the non-fedora flavor) is similar
to
the vanilla build, but automagically prepends:

%changelog
* Thu Mar 29 2007 Roland McGrath <roland@redhat.com>
- Experimental build from git sources (no Fedora patches)
- git tag:    v2.6.21-rc5             6fb04ccf5c5e054c4107090bed6e866489f1089f
- git branch: upstream                28defbea64622f69d65a6079bf800cedb9915a5f
- git branch: roland                  0ca00c859e7ac4d9d053abd06376233cb4f3bf84

and includes:

Patch1: patch-2.6.21-rc5.bz2
Patch2: linux-2.6.21-rc5-upstream.patch
Patch3: linux-2.6.21-rc5-roland.patch


Thanks,
Roland


Index: Makefile
==================================================================RCS file:
/cvs/dist/rpms/kernel/devel/Makefile,v
retrieving revision 1.45
diff -u -B -p -r1.45 Makefile
--- Makefile	19 Mar 2007 21:32:30 -0000	1.45
+++ Makefile	30 Mar 2007 21:45:57 -0000
@@ -68,3 +68,119 @@ reconfig:
 
 # since i386 isn''t a target...
 compile compile-short: DIST_DEFINES += --target $(shell uname -m)
+
+#
+# Hacks for building vanilla (unpatched) kernel rpms.
+# Use "make vanilla-TARGET" like "make TARGET" (make
vanilla-scratch-build).
+#
+vanilla-%: $(SPECFILE:.spec=-vanilla.spec)
+	@$(MAKE) $* SPECFILE=$<
+
+$(SPECFILE:.spec=-vanilla.spec): $(SPECFILE)
+	@rm -f $@
+	(echo %define nopatches 1; cat $<) > $@
+
+scratch-build: test-srpm
+	$(BUILD_CLIENT) $(BUILD_FLAGS) --scratch $(COLLECTION) \
+			$(SRCRPMDIR)/$(NAME)-$(VERSION)-$(RELEASE).src.rpm
+
+# Dismal kludge for building via brew from cvs after "make
vanilla-tag".
+ifdef BEEHIVE_SRPM_BUILD
+export CHECKOUT_TAG ?= $(shell sed s/^.// CVS/Tag)
+tag-pattern = $(TAG_NAME)-$(TAG_VERSION)-0_%_$(TAG_RELEASE)
+ifeq (,$(filter-out $(tag-pattern),$(CHECKOUT_TAG)))
+variant := $(patsubst $(tag-pattern),%,$(CHECKOUT_TAG))
+srpm: SPECFILE := $(wildcard $(SPECFILE:.spec=-$(variant).spec) \
+			     $(SPECFILE:.spec=-t.$(variant).spec))
+srpm beehive-sprm: RELEASE := 0.$(variant).$(RELEASE)
+endif
+endif
+
+
+#
+# Hacks for building kernel rpms from upstream code plus local GIT branches.
+# Use "make git/BRANCH/TARGET" like "make TARGET".
+# Use "make git/BRANCH-fedora/TARGET" to include Fedora patches on
top.
+#
+ifndef GIT_SPEC
+git/%:
+	@$(MAKE) GIT_SPEC=$(subst /,-,$(*D)) git-$(*F)
+else
+git-%: $(SPECFILE:.spec=-t.$(GIT_SPEC).spec)
+	@$(MAKE) GIT_SPEC= $* SPECFILE=$<
+endif
+
+#
+# Your git-branches.mk file can define GIT_DIR, e.g.:
+#	GIT_DIR = ${HOME}/kernel/.git
+# Make sure GIT_AUTHOR_NAME and GIT_AUTHOR_EMAIL are also set
+# or your rpm changelogs will look like crap.
+#
+# For each branch it can define a variable branch-BRANCH or tag-BRANCH
+# giving the parent of BRANCH to diff against in a separate patch.  If
+# the parent is unknown, it will use $(branch-upstream) defaulting to
+# "upstream".
+#
+# Defining tag-BRANCH means the tag corresponds to an upstream patch in
+# the sources file, so that is used instead of generating a patch with
+# git.  If there is no tag-upstream defined, it will figure out a vNNN
+# tag or vNNN-gitN pseudo-tag from the last patch in the sources file.
+# For example:
+#	tag-some-hacks = v2.6.21-rc5
+#	branch-more-hacks = first-hacks
+# Leads to patches:
+#	git diff v2.6.21-rc5..more-hacks > linux-2.6.21-rc5-some-hacks.patch
+#	git diff some-hacks..more-hacks > linux-2.6.21-rc5-more-hacks.patch
+# Whereas having no git-branches.mk at all but doing
+# "make GIT_DIR=... git/mybranch/test-srpm" does:
+#	id=`cat patch-2.6.21-rc5-git4.id` # auto-fetched via upstream file
+#	git diff $id..upstream > linux-2.6.21-rc5-git4-upstream.patch
+#	git diff upstream..mybranch > linux-2.6.21-rc5-git4-mybranch.patch
+# If the upstream patch (or any branch patch) is empty it''s left out.
+#
+git-branches.mk:;
+-include git-branches.mk
+
+branch-upstream ?= upstream
+
+ifdef GIT_DIR
+export GIT_DIR
+export GIT_AUTHOR_NAME
+export GIT_AUTHOR_EMAIL
+gen-patches ?= gen-patches
+
+ifndef havespec
+$(SPECFILE:.spec=-t.%-fedora.spec): $(SPECFILE) $(gen-patches) FORCE
+	./$(gen-patches) --fedora < $< > $@ $(gen-patches-args)
+$(SPECFILE:.spec=-t.%.spec): $(SPECFILE) $(gen-patches) FORCE
+	./$(gen-patches) < $< > $@ $(gen-patches-args)
+.PRECIOUS: $(SPECFILE:.spec=-t.%.spec) $(SPECFILE:.spec=-t.%-fedora.spec)
+endif
+
+spec-%: $(SPECFILE:.spec=-t.%.spec) ;
+$(SPECFILE):;
+FORCE:;
+
+gen-patches-args = ${srcdir} v$(VERSION) $(call heads,$*)
+define heads
+$(if $(tag-$1),$(filter-out v$(VERSION),$(tag-$1)),\
+     $(call heads,$(firstword $(branch-$1) $(branch-upstream)))) $1
+endef
+
+files-%-fedora:
+	@echo $(SPECFILE:.spec=-t.$*-fedora.spec)
+	@$(call list-patches,$*)
+files-%:
+	@echo $(SPECFILE:.spec=-t.$*.spec)
+	@$(call list-patches,$*)
+define list-patches
+$(if $(tag-$1),version=$(patsubst v%,%,$(tag-$1)),\
+     $(call list-patches,$(firstword $(branch-$1) $(branch-upstream)))); \
+echo linux-$${version}-$1.patch
+endef
+
+ifndef tag-$(branch-upstream)
+tag-$(branch-upstream) := $(shell \
+	sed -n ''s/^.*  *//;s/\.bz2$$//;s/patch-/v/p'' sources)
+endif
+endif
Index: kernel-2.6.spec
==================================================================RCS file:
/cvs/dist/rpms/kernel/devel/kernel-2.6.spec,v
retrieving revision 1.3025
diff -u -B -p -r1.3025 kernel-2.6.spec
--- kernel-2.6.spec	29 Mar 2007 00:00:55 -0000	1.3025
+++ kernel-2.6.spec	30 Mar 2007 21:45:57 -0000
@@ -32,7 +32,8 @@ Summary: The Linux kernel (the core of t
 %define sublevel 20
 %define kversion 2.6.%{sublevel}
 %define rpmversion 2.6.%{sublevel}
-%define release %(R="$Revision: 1.3025 $"; RR="${R##: }";
echo ${RR%%?})%{?dist}
+%define specrelease %(R="$Revision: 1.3025 $"; RR="${R##:
}"; echo ${RR%%?})%{?dist}
+%define release %{specrelease}
 
 %define make_target bzImage
 %define kernel_image x86
@@ -45,6 +46,28 @@ Summary: The Linux kernel (the core of t
 %define KVERREL %{PACKAGE_VERSION}-%{PACKAGE_RELEASE}
 %define hdrarch %_target_cpu
 
+%if 0%{!?nopatches:1}
+%define nopatches 0
+%endif
+
+%if %{nopatches}
+%define includexen 0
+%else
+%define relsuffix .fedora
+%endif
+
+%define using_upstream_branch 0
+%if 0%{?upstream_branch:1}
+%define using_upstream_branch 1
+%define release 0.%{upstream_branch}%{?relsuffix}.%{specrelease}
+%define buildxen 0
+%define buildxenPAE 0
+%else
+%if %{nopatches}
+%define release 0.vanilla.%{specrelease}
+%endif
+%endif
+
 # groups of related archs
 #OLPC stuff
 %if 0%{?olpc}
@@ -169,6 +192,15 @@ Summary: The Linux kernel (the core of t
 %define xen_image vmlinux.gz
 %endif
 
+%if %{nopatches}
+%define signmodules 0
+# Ignore unknown options in our config-* files.
+# Some options go with patches we''re not applying.
+%define oldconfig_target loose_nonint_oldconfig
+%else
+%define oldconfig_target nonint_oldconfig
+%endif
+
 # To temporarily exclude an architecture from being built, add it to
 # %nobuildarches. Do _NOT_ use the ExclusiveArch: line, because if we
 # don''t build kernel-headers then the new build system will no longer
let
@@ -304,7 +336,15 @@ Source82: config-olpc-generic
 #
 # Patches 0 through 100 are meant for core subsystem upgrades
 #
+
+%if %{using_upstream_branch}
+### BRANCH PATCH ###
+%else
+# Here should be only the patches up to the upstream canonical Linus tree.
 Patch1: patch-2.6.21-rc5.bz2
+%endif
+
+%if !%{nopatches}
 Patch3: git-geode.patch
 
 # Patches 10 through 99 are for things that are going upstream really soon.
@@ -360,7 +400,9 @@ Patch341: linux-2.6-mpc52xx-fec.patch
 # Patches 800 through 899 are reserved for bugfixes to the core system
 # and patches related to how RPMs are build
 #
-Patch800: linux-2.6-build-nonintconfig.patch
+%endif
+Patch800: linux-2.6.17-nonintconfig.patch
+%if !%{nopatches}
 
 # Exec-shield.
 Patch810: linux-2.6-execshield.patch
@@ -495,6 +537,7 @@ Patch20001: xen-11668-hvm_disable_fix.pa
 Patch20002: xen-dom0-reboot.patch
 
 # END OF PATCH DEFINITIONS
+%endif
 
 BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root-%{_target_cpu}
 
@@ -867,9 +910,16 @@ cp -rl vanilla linux-%{kversion}.%{_targ
 
 cd linux-%{kversion}.%{_target_cpu}
 
+%if %{using_upstream_branch}
+### BRANCH APPLY ###
+%else
+
 # Update to latest upstream.
 %patch1 -p1
 
+%endif
+%if !%{nopatches}
+
 # Patches 10 through 100 are meant for core subsystem upgrades
 
 # Roland''s utrace ptrace replacement.
@@ -945,13 +995,14 @@ cd linux-%{kversion}.%{_target_cpu}
 # Patches 800 through 899 are reserved for bugfixes to the core system
 # and patches related to how RPMs are build
 #
-
+%endif
 
 # This patch adds a "make nonint_oldconfig" which is non-interactive
and
 # also gives a list of missing options at the end. Useful for automated
 # builds (as used in the buildsystem).
 %patch800 -p1
 
+%if !%{nopatches}
 # Exec shield
 %patch810 -p1
 
@@ -1171,6 +1222,7 @@ mv drivers/xen/blktap/blktap.c drivers/x
 %patch10001 -p1
 
 # END OF PATCH APPLICATIONS
+%endif
 
 cp %{SOURCE10} Documentation/
 
@@ -1201,7 +1253,7 @@ for i in *.config
 do
   mv $i .config
   Arch=`head -1 .config | cut -b 3-`
-  make ARCH=$Arch nonint_oldconfig > /dev/null
+  make ARCH=$Arch %{oldconfig_target} > /dev/null
   echo "# $Arch" > configs/$i
   cat .config >> configs/$i
 done
@@ -1295,7 +1347,7 @@ BuildKernel() {
        KernelImage=arch/$Arch/boot/bzImage
     fi
 
-    make -s ARCH=$Arch nonint_oldconfig > /dev/null
+    make -s ARCH=$Arch %{oldconfig_target} > /dev/null
     make -s ARCH=$Arch %{?_smp_mflags} $MakeTarget %{?sparse_mflags}
     make -s ARCH=$Arch %{?_smp_mflags} modules %{?sparse_mflags} || exit 1

Roland McGrath wrote:
>> It is. And I was under the impression that''s what Dave was
thinking too,
>> but I''ll let him speak for himself.
> 
> Well, do we want it in the package set?  I figured we were talking about
> "informal" builds that might be put up for ftp, but not be an
integrated
> part of the Fedora world.
This would appear to be question #1 we need to get answered. :)
> If one rpmbuild from the spec kicks out -vanilla
> packages along with other variants, and we turn that on only for rawhide,
> still it winds up "in the system", people install it as
"normal" and wind
> up with upgrade issues and all that.  Eventually we have clueless Fedora
> users saying "you use -PAE? well -vanilla is the Fedora kernel I like
best!"
> and reporting in bugs "why this fix isn''t in -vanilla
too?"
I don''t think that''s too big a concern, we could be 100% firm
that we
don''t apply any patches whatsoever (outside of nonintconfig), its pure 
upstream.

The main benefit I see is that it would be really nice to spin them off 
the exact same rpmbuild, since that would mean the only possible 
variance is the set of patches applied.

However, yeah, it being "in the system" could indeed be annoying. And
it
could be a bit of unnecessary overhead, resulting in an identical kernel 
if from one kernel build to another, all that changes is the patches we add.

So perhaps, building "unofficial" builds, of all varieties, does make 
more sense, and then we just point folks at them when they have issues 
with our normal kernels and want to compare w/an upstream build.
> It''s easy to tweak my changes to put "vanilla" into
%name instead of
> %release.  I went with "kernel" and a long, funky %release
because that''s
> the tradition I''ve experienced for one-off and test builds people
> distribute.  From my own experience juggling a dozen installations on two
> or three test boxes with many hack kernels installed each in an utterly
> disorganized fashion, it seems more likely to get resolved usefully when
> doing upgrades and such than an rpm named "kernel-foobar".  That
is, yum
> and rpm will get in your face all the time in the way that people are used
> to for juggling kernel rpms.  (Also, I always shudder in superstitious fear
> that yum or who knows what will assume things about the kernel rpm being
> the way it always has been.)
I kinda like it in %name whichever route we go here. Along the same 
lines as Ingo''s kernel-rt packages, it makes it easier to install them 
in parallel with normal kernels for testing. Otherwise, a vanilla build 
we only wanted someone to install for testing would trigger one of the 
user''s two other installed kernels being un-installed with our current 
default yum setup.
> With whatever name, separate vanilla builds could easily be made usable as
> their own little yum repository.
Okay, I think that''s what I''m leaning toward now. :)
> What is more work, and I think is more dubious, is having it be the same
> srpm that produces both fedora and vanilla builds.  What will -bp do, make
> two source trees patched differently?
Good question...
>>> That requires
>>> basically duplicating all the innards of the spec file.  I
don''t think that
>>> should be attempted without a complete cleanup of the spec file to
become
>>> sane.
>> More specifically, what I had in mind was to add just one more variant,
>> so you''d have kernel, kernel-xen, kernel-kdump, kernel-PAE,
etc., plus
>> now a kernel-vanilla, which would be analogous to kernel minus patches 
>> -- i.e., don''t bother with kernel-vanilla-xen,
kernel-vanilla-kudmp, etc.
> 
> That''s no fun!  Seriously, I think we need the full suite of
variants
> (modulo xen until that''s uptsream, etc.) for it really to be
useful.
> Plenty of people are using -PAE and the point is to compare the foo
you''re
> using to foo-vanilla on a one-to-one basis, i.e. different source, same
> .config settings.  And you want -debuginfo for all of those too.
I''d have had vanilla-debuginfo built off the main spec too (plus 
vanilla-devel and vanilla-headers). But I''m persuaded to go all-out.
>> Not to say that a spec file sanitization/sane-itization
wouldn''t be good
>> to do as well (which I''m also perfectly willing to take on --
a few
>> steps in this direction were taken earlier today).
> 
> I despise how much duplication there is in there.  The debuginfo stuff I
> originally did with rpm macros got hand-expanded because older rpmbuild
> couldn''t cope with it.  Maybe nowadays there is nothing with an
rpmbuild
> older than FC-5 we are trying to work with
And we only have to care about FC-5 for another month or so, in which 
case I wouldn''t bother worrying about anything but FC-6 and later.
> so we could use more macros for
> new stuff.  (I think brew got fixed to use the buildroot''s
rpmbuild, so
> anything that''s fine for the target systems sharing the kernel
spec source
> should be fine.)
Correct, the buildroot should be using the target system''s rpmbuild.
> rpm macros are a hairy mess, but cleaning things up with
> clean new macros is a better bet than anything else, IMHO.
Absolutely.
>> Cool, I''ll read your patch more carefully now. :)
> 
> Here''s the current version of it.  You still need the nonintconfig
patch
> variant I posted last time to go with this.  I also attached below the
> shell script used by the GIT-using hacks that are in this version.
> http://porkchop.devel.redhat.com/brewroot/scratch/roland/task_703101/ shows
> what "make git/roland/scratch-build" does (the build magic works
for "make
> git/roland-fedora/..." too, but the fedora patches don''t
apply to current
> upstream so it didn''t build).  This (the non-fedora flavor) is
similar to
> the vanilla build, but automagically prepends:
> 
> %changelog
> * Thu Mar 29 2007 Roland McGrath <roland@redhat.com>
> - Experimental build from git sources (no Fedora patches)
> - git tag:    v2.6.21-rc5            
6fb04ccf5c5e054c4107090bed6e866489f1089f
> - git branch: upstream               
28defbea64622f69d65a6079bf800cedb9915a5f
> - git branch: roland                 
0ca00c859e7ac4d9d053abd06376233cb4f3bf84
> 
> and includes:
> 
> Patch1: patch-2.6.21-rc5.bz2
> Patch2: linux-2.6.21-rc5-upstream.patch
> Patch3: linux-2.6.21-rc5-roland.patch
Thanks much, I''ll try to digest all this over the weekend, and/or early
Monday...

Still interested to hear what approach others think we should take to 
providing some "vanilla" kernels, but it would seem I''m now
on board for
your idea, but with "vanilla" (or "upstream" or
"whatever") moved from
%release to %name.

-- 
Jarod Wilson
jwilson@redhat.com

> I kinda like it in %name whichever route we go here. Along the same 
> lines as Ingo''s kernel-rt packages, it makes it easier to install
them
> in parallel with normal kernels for testing. 
I was just remembering about Ingo''s -rt builds.  I hadn''t
looked.  What he
uses is nearly identical to what I wind up with.  It should be easy to use
a unified spec file for his -rt patch version too.

I''ve tweaked my hacks slightly to use name: kernel-vanilla and
kernel-branch for the git stuff.  For git builds, it now produces
e.g. kernel-upstream-2.6.20-2.6.21.rc5.97.1.3025
or kernel-roland-fedora-PAE-2.6.20-2.6.18.rc6.256.1.3025
using the tag and "commit number" from git (1.5) describe, so upgrades
for
the same branch package should work right without the ginormous date string
release numbers I had before.  (Those changes are small but I don''t
have
intermediate diffs on hand since I didn''t commit the old version.  I
can
send you the new files if you want.)


Thanks,
Roland

Roland McGrath wrote:
>> I kinda like it in %name whichever route we go here. Along the same 
>> lines as Ingo''s kernel-rt packages, it makes it easier to
install them
>> in parallel with normal kernels for testing. 
> 
> I was just remembering about Ingo''s -rt builds.  I hadn''t
looked.  What he
> uses is nearly identical to what I wind up with.  It should be easy to use
> a unified spec file for his -rt patch version too.
Definitely doable. Probably lots more %if''age in the spec, but hey,
there''s already a ton...
> I''ve tweaked my hacks slightly to use name: kernel-vanilla and
> kernel-branch for the git stuff.  For git builds, it now produces
> e.g. kernel-upstream-2.6.20-2.6.21.rc5.97.1.3025
> or kernel-roland-fedora-PAE-2.6.20-2.6.18.rc6.256.1.3025
> using the tag and "commit number" from git (1.5) describe, so
upgrades for
> the same branch package should work right without the ginormous date string
> release numbers I had before.
If I''m understanding correctly, my current implementation
isn''t quite as
flexible as yours, as it only adds the ability to build kernel-vanilla
(and all sub-variants/bits), but it does do so for both released and
in-development -rc and -rc-git vanilla kernels. I''ve been toying around
with a few different variations, shuffling %name, %version and %release
around a bit, and I think I''ve come upon one that I''m liking.

My current version outputs a kernel-vanilla-2.6.21-0.rc5.git9.1.fc7 from
a 1.3040-based spec, which is (hopefully) obviously 2.6.21-rc5-git9. A
2.6.20.4-based build should wind up being kernel-vanilla-2.6.20-4.1.fc6
using similar spec tweaks, and a forthcoming F7 2.6.21 kernel ought to
wind up being kernel-vanilla-2.6.21-0.1.fc7.
> (Those changes are small but I don''t have
> intermediate diffs on hand since I didn''t commit the old version. 
I can
> send you the new files if you want.)
Absolutely, shoot me whatcha got. I still have your prior changes in
patch form (as well as a good portion merged into a working copy),
extracting the diffs is no prob.

Renaming to kernel-vanilla while still using the same spec does
introduce a few problems I hadn''t considered...

1) it breaks some build setups, where spec and sources are both
somewhere like %_topdir/sources/%name. The build gets confused if %name
is now kernel-vanilla. I can work around it locally with a symlink in my
build root, but something programmatic would be good. Haven''t looked
into this much just yet.

2) kernel-xyz-headers is going to conflict with kernel-headers. Perhaps
solved by simply ''if kernel-xyz, kernel-xyz-headers Provides:
kernel-headers''. I''ve got this in my latest edition, will see
how it
behaves...

Despite those issues, I still vote for the %name change.

-- 
Jarod Wilson
jwilson@redhat.com

> Definitely doable. Probably lots more %if''age in the spec, but
hey,
> there''s already a ton...
Nope, pretty much just the same ones I added.
> Absolutely, shoot me whatcha got. I still have your prior changes in
> patch form (as well as a good portion merged into a working copy),
> extracting the diffs is no prob.
Index: Makefile
==================================================================RCS file:
/cvs/dist/rpms/kernel/devel/Makefile,v
retrieving revision 1.45
diff -B -p -u -r1.45 Makefile
--- Makefile	19 Mar 2007 21:32:30 -0000	1.45
+++ Makefile	3 Apr 2007 21:02:40 -0000
@@ -68,3 +68,120 @@ reconfig:
 
 # since i386 isn''t a target...
 compile compile-short: DIST_DEFINES += --target $(shell uname -m)
+
+#
+# Hacks for building vanilla (unpatched) kernel rpms.
+# Use "make vanilla-TARGET" like "make TARGET" (make
vanilla-scratch-build).
+#
+vanilla-%: $(SPECFILE:.spec=-vanilla.spec)
+	@$(MAKE) $* SPECFILE=$<
+
+$(SPECFILE:.spec=-vanilla.spec): $(SPECFILE)
+	@rm -f $@
+	(echo %define nopatches 1; cat $<) > $@
+
+scratch-build: test-srpm
+	$(BUILD_CLIENT) $(BUILD_FLAGS) --scratch $(COLLECTION) \
+			$(SRCRPMDIR)/$(NAME)-$(VERSION)-$(RELEASE).src.rpm
+
+# Dismal kludge for building via brew from cvs after "make
vanilla-tag".
+ifdef BEEHIVE_SRPM_BUILD
+export CHECKOUT_TAG ?= $(shell sed s/^.// CVS/Tag)
+tag-pattern = $(TAG_NAME)-$(TAG_VERSION)-0_%_$(TAG_RELEASE)
+ifeq (,$(filter-out $(tag-pattern),$(CHECKOUT_TAG)))
+variant := $(patsubst $(tag-pattern),%,$(CHECKOUT_TAG))
+srpm: SPECFILE := $(wildcard $(SPECFILE:.spec=-$(variant).spec) \
+			     $(SPECFILE:.spec=-t.$(variant).spec))
+srpm beehive-sprm: RELEASE := 0.$(variant).$(RELEASE)
+endif
+endif
+
+
+#
+# Hacks for building kernel rpms from upstream code plus local GIT branches.
+# Use "make git/BRANCH/TARGET" like "make TARGET".
+# Use "make git/BRANCH-fedora/TARGET" to include Fedora patches on
top.
+#
+ifndef GIT_SPEC
+git/%:
+	@$(MAKE) GIT_SPEC=$(subst /,-,$(*D)) git-$(*F)
+else
+git-%: $(SPECFILE:.spec=-t.$(GIT_SPEC).spec)
+	@$(MAKE) GIT_SPEC= $* SPECFILE=$<
+endif
+
+#
+# Your git-branches.mk file can define GIT_DIR, e.g.:
+#	GIT_DIR = ${HOME}/kernel/.git
+# Make sure GIT_AUTHOR_NAME and GIT_AUTHOR_EMAIL are also set
+# or your rpm changelogs will look like crap.
+#
+# For each branch it can define a variable branch-BRANCH or tag-BRANCH
+# giving the parent of BRANCH to diff against in a separate patch.  If
+# the parent is unknown, it will use $(branch-upstream) defaulting to
+# "upstream".
+#
+# Defining tag-BRANCH means the tag corresponds to an upstream patch in
+# the sources file, so that is used instead of generating a patch with
+# git.  If there is no tag-upstream defined, it will figure out a vNNN
+# tag or vNNN-gitN pseudo-tag from the last patch in the sources file.
+# For example:
+#	tag-some-hacks = v2.6.21-rc5
+#	branch-more-hacks = first-hacks
+# Leads to patches:
+#	git diff v2.6.21-rc5..more-hacks > linux-2.6.21-rc5-some-hacks.patch
+#	git diff some-hacks..more-hacks > linux-2.6.21-rc5-more-hacks.patch
+# Whereas having no git-branches.mk at all but doing
+# "make GIT_DIR=... git/mybranch/test-srpm" does:
+#	id=`cat patch-2.6.21-rc5-git4.id` # auto-fetched via upstream file
+#	git diff $id..upstream > linux-2.6.21-rc5-git4-upstream.patch
+#	git diff upstream..mybranch > linux-2.6.21-rc5-git4-mybranch.patch
+# If the upstream patch (or any branch patch) is empty it''s left out.
+#
+git-branches.mk:;
+-include git-branches.mk
+
+branch-upstream ?= upstream
+
+ifdef GIT_DIR
+export GIT_DIR
+export GIT_AUTHOR_NAME
+export GIT_AUTHOR_EMAIL
+gen-patches ?= gen-patches
+
+ifndef havespec
+$(SPECFILE:.spec=-t.%-fedora.spec): $(SPECFILE) $(gen-patches) FORCE
+	./$(gen-patches) --fedora < $< > $@ $(gen-patches-args)
+$(SPECFILE:.spec=-t.%.spec): $(SPECFILE) $(gen-patches) FORCE
+	./$(gen-patches) < $< > $@ $(gen-patches-args)
+.PRECIOUS: $(SPECFILE:.spec=-t.%.spec) $(SPECFILE:.spec=-t.%-fedora.spec)
+endif
+
+spec-%: $(SPECFILE:.spec=-t.%.spec) ;
+$(SPECFILE):;
+FORCE:;
+
+branch-of-* = $(firstword $(head-$*) $*)
+gen-patches-args = --name $* v$(VERSION) $(call heads,$(branch-of-*))
+define heads
+$(if $(tag-$1),$(filter-out v$(VERSION),$(tag-$1)),\
+     $(call heads,$(firstword $(branch-$1) $(branch-upstream)))) $1
+endef
+
+files-%-fedora:
+	@echo $(SPECFILE:.spec=-t.$*-fedora.spec)
+	@$(call list-patches,$(branch-of-*))
+files-%:
+	@echo $(SPECFILE:.spec=-t.$*.spec)
+	@$(call list-patches,$(branch-of-*))
+define list-patches
+$(if $(tag-$1),version=$(patsubst v%,%,$(tag-$1)),\
+     $(call list-patches,$(firstword $(branch-$1) $(branch-upstream)))); \
+echo linux-$${version}-$1.patch
+endef
+
+ifndef tag-$(branch-upstream)
+tag-$(branch-upstream) := $(shell \
+	sed -n ''s/^.*  *//;s/\.bz2$$//;s/patch-/v/p'' sources)
+endif
+endif
Index: kernel-2.6.spec
==================================================================RCS file:
/cvs/dist/rpms/kernel/devel/kernel-2.6.spec,v
retrieving revision 1.3040
diff -B -p -u -r1.3040 kernel-2.6.spec
--- kernel-2.6.spec	2 Apr 2007 19:39:38 -0000	1.3040
+++ kernel-2.6.spec	3 Apr 2007 21:02:40 -0000
@@ -60,7 +60,8 @@ Summary: The Linux kernel (the core of t
 %define sublevel 20
 %define kversion 2.6.%{sublevel}
 %define rpmversion 2.6.%{sublevel}
-%define release %(R="$Revision: 1.3040 $"; RR="${R##: }";
echo ${RR%%?})%{?dist}%{?buildid}
+%define specrelease %(R="$Revision: 1.3040 $"; RR="${R##:
}"; echo ${RR%%?})%{?dist}%{?buildid}
+%define release %{specrelease}
 
 %define make_target bzImage
 %define kernel_image x86
@@ -73,6 +74,24 @@ Summary: The Linux kernel (the core of t
 %define KVERREL %{PACKAGE_VERSION}-%{PACKAGE_RELEASE}
 %define hdrarch %_target_cpu
 
+%if 0%{!?nopatches:1}
+%define nopatches 0
+%endif
+
+%if %{nopatches}
+%define includexen 0
+%define variant -vanilla
+%else
+%define variant_fedora -fedora
+%endif
+
+%define using_upstream_branch 0
+%if 0%{?upstream_branch:1}
+%define using_upstream_branch 1
+%define variant -%{upstream_branch}%{?variant_fedora}
+%define release %{upstream_branch_release}.%{specrelease}
+%endif
+
 # if requested, only build base kernel
 %if %{with_baseonly}
 %define with_smp 0
@@ -237,6 +256,15 @@ Summary: The Linux kernel (the core of t
 %define xen_image vmlinux.gz
 %endif
 
+%if %{nopatches}
+%define signmodules 0
+# Ignore unknown options in our config-* files.
+# Some options go with patches we''re not applying.
+%define oldconfig_target loose_nonint_oldconfig
+%else
+%define oldconfig_target nonint_oldconfig
+%endif
+
 # To temporarily exclude an architecture from being built, add it to
 # %nobuildarches. Do _NOT_ use the ExclusiveArch: line, because if we
 # don''t build kernel-headers then the new build system will no longer
let
@@ -285,7 +313,7 @@ Summary: The Linux kernel (the core of t
 #
 %define kernel_prereq  fileutils, module-init-tools, initscripts >=
8.11.1-1, mkinitrd >= 6.0.4-1
 
-Name: kernel
+Name: kernel%{?variant}
 Group: System Environment/Kernel
 License: GPLv2
 Version: %{rpmversion}
@@ -373,8 +401,16 @@ Source82: config-olpc-generic
 #
 # Patches 0 through 100 are meant for core subsystem upgrades
 #
+
+%if %{using_upstream_branch}
+### BRANCH PATCH ###
+%else
+# Here should be only the patches up to the upstream canonical Linus tree.
 Patch1: patch-2.6.21-rc5.bz2
 Patch2: patch-2.6.21-rc5-git9.bz2
+%endif
+
+%if !%{nopatches}
 Patch3: git-geode.patch
 
 # Patches 10 through 99 are for things that are going upstream really soon.
@@ -448,7 +484,9 @@ Patch366: linux-2.6-ps3av-export-header.
 # Patches 800 through 899 are reserved for bugfixes to the core system
 # and patches related to how RPMs are build
 #
-Patch800: linux-2.6-build-nonintconfig.patch
+%endif
+Patch800: linux-2.6.17-nonintconfig.patch
+%if !%{nopatches}
 
 # Exec-shield.
 Patch810: linux-2.6-execshield.patch
@@ -582,6 +620,7 @@ Patch20001: xen-11668-hvm_disable_fix.pa
 Patch20002: xen-dom0-reboot.patch
 
 # END OF PATCH DEFINITIONS
+%endif
 
 BuildRoot: %{_tmppath}/kernel-%{KVERREL}-root-%{_target_cpu}
 
@@ -959,10 +998,17 @@ cp -rl vanilla linux-%{kversion}.%{_targ
 
 cd linux-%{kversion}.%{_target_cpu}
 
+%if %{using_upstream_branch}
+### BRANCH APPLY ###
+%else
+
 # Update to latest upstream.
 %patch1 -p1
 %patch2 -p1
 
+%endif
+%if !%{nopatches}
+
 # Patches 10 through 100 are meant for core subsystem upgrades
 
 # Roland''s utrace ptrace replacement.
@@ -1056,13 +1102,14 @@ cd linux-%{kversion}.%{_target_cpu}
 # Patches 800 through 899 are reserved for bugfixes to the core system
 # and patches related to how RPMs are build
 #
-
+%endif
 
 # This patch adds a "make nonint_oldconfig" which is non-interactive
and
 # also gives a list of missing options at the end. Useful for automated
 # builds (as used in the buildsystem).
 %patch800 -p1
 
+%if !%{nopatches}
 # Exec shield
 %patch810 -p1
 
@@ -1280,6 +1327,7 @@ mv drivers/xen/blktap/blktap.c drivers/x
 %patch10001 -p1
 
 # END OF PATCH APPLICATIONS
+%endif
 
 cp %{SOURCE10} Documentation/
 
@@ -1310,7 +1358,7 @@ for i in *.config
 do
   mv $i .config
   Arch=`head -1 .config | cut -b 3-`
-  make ARCH=$Arch nonint_oldconfig > /dev/null
+  make ARCH=$Arch %{oldconfig_target} > /dev/null
   echo "# $Arch" > configs/$i
   cat .config >> configs/$i
 done
@@ -1404,7 +1452,7 @@ BuildKernel() {
        KernelImage=arch/$Arch/boot/bzImage
     fi
 
-    make -s ARCH=$Arch nonint_oldconfig > /dev/null
+    make -s ARCH=$Arch %{oldconfig_target} > /dev/null
     make -s ARCH=$Arch %{?_smp_mflags} $MakeTarget %{?sparse_mflags}
     make -s ARCH=$Arch %{?_smp_mflags} modules %{?sparse_mflags} || exit 1

Roland McGrath wrote:
>> Definitely doable. Probably lots more %if''age in the spec, but
hey,
>> there''s already a ton...
> 
> Nope, pretty much just the same ones I added.
Ah, sorry, I see where the disconnect is here... I''ve not taken some of
the build system side of things into account, and was trying to do
things solely via spec mods. I think I now understand the necessity of
Makefile hacks too. :\

-- 
Jarod Wilson
jwilson@redhat.com