I now have a Debian PPC64 execution environment set up on my mac
cluster, so am trying to get Lustre built (no cross compilation
required).
I have the 1.5.95 kernel patches installed and compiled against a
2.6.18 vanilla kernel, no problems there.
When I try to build the lustre distribution itself, I find references
to <asm/segment.h> which does not exist in the powerpc architecture
includes; I worked around this by touching the file to get an empty
include. I then get a number of warnings in multiple files of the form:
CC [M] /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/socklnd.o
In file included from /home/lustre/lustre-1.5.95/lnet/include/libcfs/
kp30.h:12,
from /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/
socklnd_lib-linux.h:42,
from /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/
socklnd.h:34,
from /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/
socklnd.c:26:
/home/lustre/lustre-1.5.95/lnet/include/libcfs/linux/kp30.h:184:6:
warning: "KLWT_SUPPORT" is not defined
/home/lustre/lustre-1.5.95/lnet/include/libcfs/linux/kp30.h:315:31:
warning: "__WORDSIZE" is not defined
In file included from /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/
socklnd.c:26:
/home/lustre/lustre-1.5.95/lnet/klnds/socklnd/socklnd.h:144:5:
warning: "SOCKNAL_BACKOFF" is not defined
and a fatal error:
CC [M] /home/lustre/lustre-1.5.95/lustre/ldiskfs/super.o
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:45:16: error: al.h:
No such file or directory
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c: In function
''kalived'':
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1466: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1466: error:
''ALIVE_MAGIC'' undeclared (first use in this function)
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1466: error: (Each
undeclared identifier is reported only once
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1466: error: for
each function it appears in.)
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1467: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1469: error:
''uts_sem'' undeclared (first use in this function)
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1470: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1470: error:
''system_utsname'' undeclared (first use in this function)
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1477: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1478: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1490: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1491: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c: In function
''check_alive'':
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1596: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1596: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1596: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1596: error:
''ALIVE_MAGIC'' undeclared (first use in this function)
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1603: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1603: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1603: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1605: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1605: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1605: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1605: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1628: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1628: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1628: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1631: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1638: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1662: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1662: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1662: error:
dereferencing pointer to incomplete type
/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:1665: error:
dereferencing pointer to incomplete type
make[6]: *** [/home/lustre/lustre-1.5.95/lustre/ldiskfs/super.o] Error 1
make[5]: *** [/home/lustre/lustre-1.5.95/lustre/ldiskfs] Error 2
make[4]: *** [/home/lustre/lustre-1.5.95/lustre] Error 2
make[3]: *** [_module_/home/lustre/lustre-1.5.95] Error 2
make[3]: Leaving directory `/usr/src/linux-2.6.18''
make[2]: *** [modules] Error 2
make[2]: Leaving directory `/home/lustre/lustre-1.5.95''
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/lustre/lustre-1.5.95''
make: *** [all] Error 2
al.h does exist somewhere:
/home/lustre/lustre-1.5.95# find . -name al.h
./lustre/ldiskfs/linux-stage/fs/ext3/al.h
./lustre/ldiskfs/linux-stage/.pc/ext3-multi-mount-protection-2.6.18-
vanilla.patch/fs/ext3/al.h
Thanks in advance for any help,
--bob
On Oct 13, 2006 15:53 -0500, Robert Olson wrote:> I then get a number of warnings in multiple files of the form: > > CC [M] /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/socklnd.o > In file included from /home/lustre/lustre-1.5.95/lnet/include/libcfs/ > kp30.h:12, > from /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/ > socklnd_lib-linux.h:42, > from /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/ > socklnd.h:34, > from /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/ > socklnd.c:26: > /home/lustre/lustre-1.5.95/lnet/include/libcfs/linux/kp30.h:184:6: > warning: "KLWT_SUPPORT" is not defined > /home/lustre/lustre-1.5.95/lnet/include/libcfs/linux/kp30.h:315:31: > warning: "__WORDSIZE" is not defined > In file included from /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/ > socklnd.c:26: > /home/lustre/lustre-1.5.95/lnet/klnds/socklnd/socklnd.h:144:5: > warning: "SOCKNAL_BACKOFF" is not definedYes, these are new warnings from gcc4 from using "#if FOO" instead of "#ifdef FOO". Harmless, but annoying. We are fixing those.> and a fatal error: > > CC [M] /home/lustre/lustre-1.5.95/lustre/ldiskfs/super.o > /home/lustre/lustre-1.5.95/lustre/ldiskfs/super.c:45:16: error: al.h: > No such file or directory > /home/lustre/lustre-1.5.95# find . -name al.h > ./lustre/ldiskfs/linux-stage/fs/ext3/al.h > ./lustre/ldiskfs/linux-stage/.pc/ext3-multi-mount-protection-2.6.18- > vanilla.patch/fs/ext3/al.hNeed to copy al.h to include dir, see previous post with same problem. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
Thanks - I found the older messages and made those changes and the compile
went further. I''m now to:
make[4]: Entering directory `/home/lustre/lustre-1.5.95/lnet/libcfs''
if gcc -m64 -DHAVE_CONFIG_H -I. -I. -I../.. -D__arch_lib__
-D_LARGEFILE64_SOURCE=1 -include ../../config.h -I../../lnet/include
-I../../lnet/include -I../../lustre/include -I../../lustre/include -g
-Wall -fPIC -g -O2 -MT libcfs_a-user-prim.o -MD -MP -MF
".deps/libcfs_a-user-prim.Tpo" \
-c -o libcfs_a-user-prim.o `test -f ''user-prim.c'' ||
echo
''./''`user-prim.c; \
then mv -f ".deps/libcfs_a-user-prim.Tpo"
".deps/libcfs_a-user-prim.Po"; \
else rm -f ".deps/libcfs_a-user-prim.Tpo"; exit 1; \
fi
user-prim.c: In function ''cfs_alloc_page'':
user-prim.c:152: error: ''PAGE_SIZE'' undeclared (first use in
this
function)
user-prim.c:152: error: (Each undeclared identifier is reported only once
user-prim.c:152: error: for each function it appears in.)
which is odd because PAGE_SIZE is indeed defined in <asm/page.h>, and
HAVE_ASM_PAGE_H is defined in config.h.
--bob
Aha - asm/page.h is only active if __KERNEL__ defined; had to bracket the include with #define/#undef. --bob On Sat, 14 Oct 2006, Robert Olson wrote:> Thanks - I found the older messages and made those changes and the compile > went further. I''m now to: > > make[4]: Entering directory `/home/lustre/lustre-1.5.95/lnet/libcfs'' > if gcc -m64 -DHAVE_CONFIG_H -I. -I. -I../.. -D__arch_lib__ > -D_LARGEFILE64_SOURCE=1 -include ../../config.h -I../../lnet/include > -I../../lnet/include -I../../lustre/include -I../../lustre/include -g > -Wall -fPIC -g -O2 -MT libcfs_a-user-prim.o -MD -MP -MF > ".deps/libcfs_a-user-prim.Tpo" \ > -c -o libcfs_a-user-prim.o `test -f ''user-prim.c'' || echo > ''./''`user-prim.c; \ > then mv -f ".deps/libcfs_a-user-prim.Tpo" > ".deps/libcfs_a-user-prim.Po"; \ > else rm -f ".deps/libcfs_a-user-prim.Tpo"; exit 1; \ > fi > user-prim.c: In function ''cfs_alloc_page'': > user-prim.c:152: error: ''PAGE_SIZE'' undeclared (first use in this > function) > user-prim.c:152: error: (Each undeclared identifier is reported only once > user-prim.c:152: error: for each function it appears in.) > > which is odd because PAGE_SIZE is indeed defined in <asm/page.h>, and > HAVE_ASM_PAGE_H is defined in config.h. > > --bob > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > >
On Oct 14, 2006 08:55 -0500, Robert Olson wrote:> Aha - asm/page.h is only active if __KERNEL__ defined; had to bracket the > include with #define/#undef.Strange - in the kernels I''m looking at (2.6.9, 2.6.17) the PAGE_SIZE is outsize any "#ifdef __KERNEL__".> On Sat, 14 Oct 2006, Robert Olson wrote: > > > Thanks - I found the older messages and made those changes and the compile > > went further. I''m now to: > > > > make[4]: Entering directory `/home/lustre/lustre-1.5.95/lnet/libcfs'' > > if gcc -m64 -DHAVE_CONFIG_H -I. -I. -I../.. -D__arch_lib__ > > -D_LARGEFILE64_SOURCE=1 -include ../../config.h -I../../lnet/include > > -I../../lnet/include -I../../lustre/include -I../../lustre/include -g > > -Wall -fPIC -g -O2 -MT libcfs_a-user-prim.o -MD -MP -MF > > ".deps/libcfs_a-user-prim.Tpo" \ > > -c -o libcfs_a-user-prim.o `test -f ''user-prim.c'' || echo > > ''./''`user-prim.c; \ > > then mv -f ".deps/libcfs_a-user-prim.Tpo" > > ".deps/libcfs_a-user-prim.Po"; \ > > else rm -f ".deps/libcfs_a-user-prim.Tpo"; exit 1; \ > > fi > > user-prim.c: In function ''cfs_alloc_page'': > > user-prim.c:152: error: ''PAGE_SIZE'' undeclared (first use in this > > function) > > user-prim.c:152: error: (Each undeclared identifier is reported only once > > user-prim.c:152: error: for each function it appears in.) > > > > which is odd because PAGE_SIZE is indeed defined in <asm/page.h>, and > > HAVE_ASM_PAGE_H is defined in config.h.Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
On Oct 16, 2006, at 12:29 PM, Andreas Dilger wrote:> On Oct 14, 2006 08:55 -0500, Robert Olson wrote: >> Aha - asm/page.h is only active if __KERNEL__ defined; had to >> bracket the >> include with #define/#undef. > > Strange - in the kernels I''m looking at (2.6.9, 2.6.17) the > PAGE_SIZE is > outsize any "#ifdef __KERNEL__". >Maybe it''s a 2.6.18 change. Now that I''ve got a reasonable build environment I may go back to one of the earlier kernel versions that''s officially supported. --bob
This reminds me to ask - what is the latest officially supported kernel? I tried 2.6.18 since there is a 2.6.18-vanilla.series file in the lustre/kernel_patches directory. I see the betas release with 2.6.9, I''ll see if I can get my PPC64 machines running with that. --bob
On Oct 16, 2006 13:12 -0500, Robert Olson wrote:> This reminds me to ask - what is the latest officially supported > kernel? I tried 2.6.18 since there is a 2.6.18-vanilla.series file in > the lustre/kernel_patches directory. I see the betas release with > 2.6.9, I''ll see if I can get my PPC64 machines running with that.The SLES10 kernel will soon be supported officially, it is 2.6.16.8 or something like that. The 2.6.18-vanilla kernel is not officially supported. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
Andreas Dilger wrote:> On Oct 14, 2006 08:55 -0500, Robert Olson wrote: >> Aha - asm/page.h is only active if __KERNEL__ defined; had to bracket the >> include with #define/#undef. > > Strange - in the kernels I''m looking at (2.6.9, 2.6.17) the PAGE_SIZE is > outsize any "#ifdef __KERNEL__".Look at include/asm-powerpc/page.h and you will see PAGE_SIZE inside #ifdef __KERNEL__. -- Qi Yong System Software Engineer Cluster File Systems, Inc.
On Oct 17, 2006 12:11 +0800, Qi Yong wrote:> Andreas Dilger wrote: > >On Oct 14, 2006 08:55 -0500, Robert Olson wrote: > >>Aha - asm/page.h is only active if __KERNEL__ defined; had to bracket the > >>include with #define/#undef. > > > >Strange - in the kernels I''m looking at (2.6.9, 2.6.17) the PAGE_SIZE is > >outsize any "#ifdef __KERNEL__". > > Look at include/asm-powerpc/page.h and you will see PAGE_SIZE inside #ifdef > __KERNEL__.Sigh, I was looking at include/asm-ppc/page.h (and also include/asm-{i386,ia64,x86_64}/page.h too). Isn''t it great that the kernel is so consistent. For that matter, you should also be able to --disable-liblustre, since it isn''t needed and probably won''t work on a ppc system anyways. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
> > For that matter, you should also be able to --disable-liblustre, since > it isn''t needed and probably won''t work on a ppc system anyways.That helped; I also had to remove test and utils from the autoMakefile in the lustre dir, as --disable-utils seemed to not do that for me. WIth those changes, it built. I think I''m going to try to get one of the earlier supported kernels working on this machine since there were notes about runtime troubles with 2.6.18. --bob
On Oct 18, 2006 11:40 -0500, Robert Olson wrote:> >For that matter, you should also be able to --disable-liblustre, since > >it isn''t needed and probably won''t work on a ppc system anyways. > > That helped; I also had to remove test and utils from the > autoMakefile in the lustre dir, as --disable-utils seemed to not do > that for me.Hmm, you''re not going to get very far without some of the utils... Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.