Xen devel - Nov 2005 - compiler errors in latest unstable from 20051027

If CONFIG_SMP_ALTERNATIVES is selected, arch/xen/kernel/smpboot.c attempts to
include <asm/smp_boot.h>, which does not exist anywhere.  Which means
selecting that is not allowed.

However, there are several additional unprotected includes of that same file
in other places; include/asm-xen/asm-i386/system.h and
include/asm-xen/asm-i386/spinlock.h.

Plus, there is a complaint about LOCK not being defined in
include/asm-xen/asm/system.h.  If I add include <asm/atomic.h>, it gets by
the
first such error, but then I get this later on:

=In file included from include/asm-xen/asm/processor.h:18,
                 from include/asm/atomic.h:6,
                 from kernel/audit.c:45:
include/asm-xen/asm/system.h: In function `__cmpxchg'':
include/asm-xen/asm/system.h:259: error: `LOCK'' undeclared (first use
in this
function)
=
This is because asm/system.h is being included recursively, so none of the
defines in asm/atomic.h are defined yet.

The steps I took to repeat this:

1: apt-get install lkinux-tree-2.6.12
2: /usr/src/kernel-patches/all/2.6.12/unpatch/debian 2.6.12
   # this gives a real 2.6.12 tree
3: cd linux-2.6-sparse
4: ./mkbuildtree $path_to_linux
   # this applies the 2.6.12.6 patch, then overlays the xen code on top of it.
5: create diff between xen-patched linux and plain 2.6.12.6
6: Go to another machine
7: extract 2.6.12, apply 2.6.12.6; no warnings from diff during this
8: apply xen patch; no warnings from diff during this
9: make menuconfig ARCH=xen
10: error as above


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

> -----Original Message-----
> From: xen-devel-bounces@lists.xensource.com 
> [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Adam Heath
> Sent: 09 November 2005 17:23
> To: xen-devel@lists.xensource.com
> Subject: [Xen-devel] compiler errors in latest unstable from 20051027
Why are you using unstable from 20051027 rather than today? A week is a
long time in the unstable tree, and there have definitely been patches
in this area.

Ian
> If CONFIG_SMP_ALTERNATIVES is selected, 
> arch/xen/kernel/smpboot.c attempts to include 
> <asm/smp_boot.h>, which does not exist anywhere.  Which means 
> selecting that is not allowed.
> 
> However, there are several additional unprotected includes of 
> that same file in other places; 
> include/asm-xen/asm-i386/system.h and 
> include/asm-xen/asm-i386/spinlock.h.
> 
> Plus, there is a complaint about LOCK not being defined in 
> include/asm-xen/asm/system.h.  If I add include 
> <asm/atomic.h>, it gets by the first such error, but then I 
> get this later on:
> 
> => In file included from include/asm-xen/asm/processor.h:18,
>                  from include/asm/atomic.h:6,
>                  from kernel/audit.c:45:
> include/asm-xen/asm/system.h: In function `__cmpxchg'':
> include/asm-xen/asm/system.h:259: error: `LOCK'' undeclared 
> (first use in this
> function)
> => 
> This is because asm/system.h is being included recursively, 
> so none of the defines in asm/atomic.h are defined yet.
> 
> The steps I took to repeat this:
> 
> 1: apt-get install lkinux-tree-2.6.12
> 2: /usr/src/kernel-patches/all/2.6.12/unpatch/debian 2.6.12
>    # this gives a real 2.6.12 tree
> 3: cd linux-2.6-sparse
> 4: ./mkbuildtree $path_to_linux
>    # this applies the 2.6.12.6 patch, then overlays the xen 
> code on top of it.
> 5: create diff between xen-patched linux and plain 2.6.12.6
> 6: Go to another machine
> 7: extract 2.6.12, apply 2.6.12.6; no warnings from diff during this
> 8: apply xen patch; no warnings from diff during this
> 9: make menuconfig ARCH=xen
> 10: error as above
> 
> 
> _______________________________________________
> Xen-devel mailing list
> Xen-devel@lists.xensource.com
> http://lists.xensource.com/xen-devel
> 
_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, 9 Nov 2005, Ian Pratt wrote:
>
>
> > -----Original Message-----
> > From: xen-devel-bounces@lists.xensource.com
> > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Adam Heath
> > Sent: 09 November 2005 17:23
> > To: xen-devel@lists.xensource.com
> > Subject: [Xen-devel] compiler errors in latest unstable from 20051027
>
> Why are you using unstable from 20051027 rather than today? A week is a
> long time in the unstable tree, and there have definitely been patches
> in this area.
Er, no.  I verified that smp_alt.h doesn''t exist in the latest before
sending
this mail.  As for the other problems, that I didn''t verify.

A. Because it breaks up the logical flow of conversation.
Q. Why is "top-posting" bad?

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, 9 Nov 2005, Adam Heath wrote:
> On Wed, 9 Nov 2005, Ian Pratt wrote:
>
> >
> >
> > > -----Original Message-----
> > > From: xen-devel-bounces@lists.xensource.com
> > > [mailto:xen-devel-bounces@lists.xensource.com] On Behalf Of Adam
Heath
> > > Sent: 09 November 2005 17:23
> > > To: xen-devel@lists.xensource.com
> > > Subject: [Xen-devel] compiler errors in latest unstable from
20051027
> >
> > Why are you using unstable from 20051027 rather than today? A week is
a
> > long time in the unstable tree, and there have definitely been patches
> > in this area.
>
> Er, no.  I verified that smp_alt.h doesn''t exist in the latest
before sending
> this mail.  As for the other problems, that I didn''t verify.
And I just verified that it doesn''t work with latest unstable either.

=purple2:~/kernel/xen/linux-2.6.12.6-xen0# cd ..
purple2:~/kernel/xen# mv linux-2.6.12.6-xen0/ foo
purple2:~/kernel/xen# rm foo -rf &
purple2:~/kernel/xen# cp linux-2.6.12.6 linux-2.6.12.6-xen0 -a
purple2:~/kernel/xen# wget
http://www.cl.cam.ac.uk/Research/SRG/netos/xen/downloads/xen-unstable-src.tgz
purple2:~/kernel/xen# tar zxf xen-unstable-src.tgz
purple2:~/kernel/xen# cd xen-unstable/linux-2.6-xen-sparse/
purple2:~/kernel/xen/xen-unstable/linux-2.6-xen-sparse# ./mkbuildtree
~/kernel/xen/linux-2.6.12.6-xen0/
purple2:~/kernel/xen/xen-unstable/linux-2.6-xen-sparse# cd
~/kernel/xen/linux-2.6.12.6-xen0/
purple2:~/kernel/xen/linux-2.6.12.6-xen0# cp ../config-xen0 .config
purple2:~/kernel/xen/linux-2.6.12.6-xen0# make oldconfig ARCH=xen
purple2:~/kernel/xen/linux-2.6.12.6-xen0# make bzImage modules ARCH=xen
CC=gcc-3.3
....
 CC      arch/i386/kernel/asm-offsets.s
In file included from include/asm-xen/asm/processor.h:18,
                 from include/asm/thread_info.h:17,
                 from include/linux/thread_info.h:21,
                 from include/linux/spinlock.h:12,
                 from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm-xen/asm/system.h:11:25: asm/smp_alt.h: No such file or directory
In file included from include/asm-xen/asm/processor.h:18,
                 from include/asm/thread_info.h:17,
                 from include/linux/thread_info.h:21,
                 from include/linux/spinlock.h:12,
                 from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm-xen/asm/system.h: In function `__cmpxchg'':
include/asm-xen/asm/system.h:256: error: `LOCK'' undeclared (first use
in this function)
include/asm-xen/asm/system.h:256: error: (Each undeclared identifier is reported
only once
include/asm-xen/asm/system.h:256: error: for each function it appears in.)
include/asm-xen/asm/system.h:256: error: syntax error before string constant
include/asm-xen/asm/system.h:262: error: syntax error before string constant
include/asm-xen/asm/system.h:268: error: syntax error before string constant
In file included from include/linux/spinlock.h:43,
                 from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm-xen/asm/spinlock.h:9:25: asm/smp_alt.h: No such file or directory
make[1]: *** [arch/i386/kernel/asm-offsets.s] Error 1
make: *** [arch/i386/kernel/asm-offsets.s] Error 2
=

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, 2005-11-09 at 15:59 -0600, Adam Heath wrote:
> > Er, no.  I verified that smp_alt.h doesn''t exist in the
latest before sending
> > this mail.  As for the other problems, that I didn''t verify.
> 
> And I just verified that it doesn''t work with latest unstable
either.
It looks like a few places aren''t making sure that the smp_alternatives
patch is applied... does the attached fix things?

Jeremy


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Wed, 9 Nov 2005, Jeremy Katz wrote:
> On Wed, 2005-11-09 at 15:59 -0600, Adam Heath wrote:
> > > Er, no.  I verified that smp_alt.h doesn''t exist in the
latest before sending
> > > this mail.  As for the other problems, that I didn''t
verify.
> >
> > And I just verified that it doesn''t work with latest unstable
either.
>
> It looks like a few places aren''t making sure that the
smp_alternatives
> patch is applied... does the attached fix things?
=purple2:~/kernel/xen/linux-2.6.12.6-xen0# patch -p2 < ~/sparse-smp-alt.patch
patching file include/asm-xen/asm-i386/spinlock.h
patching file include/asm-xen/asm-i386/system.h
purple2:~/kernel/xen/linux-2.6.12.6-xen0# make bzImage modules ARCH=xen
CC=gcc-3.3
  CHK     include/linux/version.h
  CC      arch/i386/kernel/asm-offsets.s
In file included from include/asm-xen/asm/processor.h:18,
                 from include/asm/thread_info.h:17,
                 from include/linux/thread_info.h:21,
                 from include/linux/spinlock.h:12,
                 from include/linux/capability.h:45,
                 from include/linux/sched.h:7,
                 from arch/i386/kernel/asm-offsets.c:7:
include/asm-xen/asm/system.h: In function `__cmpxchg'':
include/asm-xen/asm/system.h:259: error: `LOCK'' undeclared (first use
in this function)
include/asm-xen/asm/system.h:259: error: (Each undeclared identifier is reported
only once
include/asm-xen/asm/system.h:259: error: for each function it appears in.)
include/asm-xen/asm/system.h:259: error: syntax error before string constant
include/asm-xen/asm/system.h:265: error: syntax error before string constant
include/asm-xen/asm/system.h:271: error: syntax error before string constant
make[1]: *** [arch/i386/kernel/asm-offsets.s] Error 1
make: *** [arch/i386/kernel/asm-offsets.s] Error 2
=
It may fix the one error, but not the big one.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On 9 Nov 2005, at 23:04, Adam Heath wrote:
> It may fix the one error, but not the big one.
You *must* apply the patches in patches/linux-2.6.x before building. 
They are not optional.

  -- Keir


_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 10 Nov 2005, Keir Fraser wrote:
>
> On 9 Nov 2005, at 23:04, Adam Heath wrote:
>
> > It may fix the one error, but not the big one.
>
> You *must* apply the patches in patches/linux-2.6.x before building.
> They are not optional.
Well, see, that''s the funny thing.

I was testing out the debs of unstable I made.  The xen patch that is produced
calls the existing makefile targets.  I take the patched 2.6.12 debian source,
revert that back to plain 2.6.12, then name the directory to what the makefile
expects.  I then generate a diff from the patched tree against the original.

And this patch still failed.

However, I''ll dig more into it.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 10 Nov 2005, Adam Heath wrote:
> On Thu, 10 Nov 2005, Keir Fraser wrote:
>
> >
> > On 9 Nov 2005, at 23:04, Adam Heath wrote:
> >
> > > It may fix the one error, but not the big one.
> >
> > You *must* apply the patches in patches/linux-2.6.x before building.
> > They are not optional.
>
> Well, see, that''s the funny thing.
>
> I was testing out the debs of unstable I made.  The xen patch that is
produced
> calls the existing makefile targets.  I take the patched 2.6.12 debian
source,
> revert that back to plain 2.6.12, then name the directory to what the
makefile
> expects.  I then generate a diff from the patched tree against the
original.
>
> And this patch still failed.
>
> However, I''ll dig more into it.
Hmm.  I see something a bit odd.  In buildconfigs/Rules.mk, the shell snippet
that applies the patches in a loop, contains a || exit 1 inside a ()
construct.  This doesn''t work.  Please observe the following:

=adam@gradall:~$ cat /tmp/test.sh
#!/bin/sh
set -x

if [ -d /tmp ]; then
        for i in $(seq 1 10); do ( ! [ $i -eq 9 ] || exit 1 ); done
fi
exit 0
adam@gradall:~$ /tmp/test.sh
+ ''['' -d /tmp '']''
++ seq 1 10
+ for i in ''$(seq 1 10)''
+ ''['' 1 -eq 9 '']''
+ for i in ''$(seq 1 10)''
+ ''['' 2 -eq 9 '']''
+ for i in ''$(seq 1 10)''
+ ''['' 3 -eq 9 '']''
+ for i in ''$(seq 1 10)''
+ ''['' 4 -eq 9 '']''
+ for i in ''$(seq 1 10)''
+ ''['' 5 -eq 9 '']''
+ for i in ''$(seq 1 10)''
+ ''['' 6 -eq 9 '']''
+ for i in ''$(seq 1 10)''
+ ''['' 7 -eq 9 '']''
+ for i in ''$(seq 1 10)''
+ ''['' 8 -eq 9 '']''
+ for i in ''$(seq 1 10)''
+ ''['' 9 -eq 9 '']''
+ exit 1
+ for i in ''$(seq 1 10)''
+ ''['' 10 -eq 9 '']''
+ exit 0
=
Note how the inner exit 1 ran, but it didn''t actually abort the loop. 
This
means that the patch might fail to apply, or half apply, or have fuzz/offsets,
but the other patches will still be applied, and the makefile won''t
abort.

The fix is to just remove the ().

This explains why I had part of the smpalts patch applied.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel

On Thu, 10 Nov 2005, Adam Heath wrote:
> On Thu, 10 Nov 2005, Adam Heath wrote:
>
> > On Thu, 10 Nov 2005, Keir Fraser wrote:
> >
> > >
> > > On 9 Nov 2005, at 23:04, Adam Heath wrote:
> > >
> > > > It may fix the one error, but not the big one.
> > >
> > > You *must* apply the patches in patches/linux-2.6.x before
building.
> > > They are not optional.
> >
> > Well, see, that''s the funny thing.
> >
> > I was testing out the debs of unstable I made.  The xen patch that is
produced
> > calls the existing makefile targets.  I take the patched 2.6.12 debian
source,
> > revert that back to plain 2.6.12, then name the directory to what the
makefile
> > expects.  I then generate a diff from the patched tree against the
original.
> >
> > And this patch still failed.
> >
> > However, I''ll dig more into it.
>
> Hmm.  I see something a bit odd.  In buildconfigs/Rules.mk, the shell
snippet
> that applies the patches in a loop, contains a || exit 1 inside a ()
> construct.  This doesn''t work.  Please observe the following:
Even more problems with the build system.

When building a patch, the makefiles create a pristine-linux-$ver.  Then they
create a new ref-linux-$ver, apply everything in patches/linux-$ver/*.  Then,
that is copied tmp-linux-$ver.patch.  Finally, a diff is generated between
ref-linux-$ver and tmp-linux-$ver.patch.

This is bad, as the patch does not contain anything applied between
pristine-linux-$ver and ref-linux-$ver.  That is my problem.  I then took the
generated patch, placed it into a deb, installed the deb, then attempted to
apply the patch.  That''s how I got the error from before.

However, just generated a full patch between pristine-* and tmp-* seems wrong,
as that would include things that are not actually xen.  I''ll probably
end up
taking everything out of patches/linux-$ver, and placing them all into the deb
directly.

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xensource.com
http://lists.xensource.com/xen-devel