Malcolm Cowe
2008-May-15 10:37 UTC
[Lustre-discuss] Lustre 1.6.4.3 + OFED 1.2.5.5 + RHEL 4u4 AS
Hi Folks,
Having some trouble with building Lustre 1.6.4.3 with OFED 1.2.5.5 on a
RHEL 4u4 AS server. Could somebody please help me to understand where
I''ve gone wrong? Here''s what I have done so far:
1. Install RHEL 4u4 AS (full installation).
2. Download the Lustre RPMs from sun.com:
e2fsprogs-1.40.4.cfs1-0redhat.x86_64.rpm
e2fsprogs-devel-1.40.4.cfs1-0redhat.x86_64.rpm
kernel-lustre-smp-2.6.9-67.0.4.EL_lustre.1.6.4.3.x86_64.rpm
kernel-lustre-source-2.6.9-67.0.4.EL_lustre.1.6.4.3.x86_64.rpm
lustre-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
lustre-modules-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
lustre-source-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
plus: lustre-1.6.4.3.tar.gz
3. Install kernel-lustre-smp and kernel-lustre-source rpms.
- Change grub to boot from lustre patched kernel by default.
- Reboot.
4. Download OFED distribution from openib.org:
OFED-1.2.5.5.tgz
5. Extract OFED distribution.
6. Install OFED:
cd OFED-1.2.5.5/
./install.sh
2) Install OFED Software
3) All packages (all of Basic, HPC)
[accept defaults for everything, configure IPoIB IP address].
7. Reboot.
8. Modify "Module.symvers", removing all references to Infiniband
modules supplied with the kernel distribution. N.B. Could not find this
file in the lustre kernel source tree, only in the "-obj" tree.
vi
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/x86_64/smp/Module.symvers
9. Run "/usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh" and
append the resulting file to the existing Module.symvers file:
/usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh
cat Module.symvers >>
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/x86_64/smp/Module.symvers
10. Change into the lustre kernel source and edit the Makefile. Change
"custom" suffix to "smp" in the variable
"EXTRAVERSION".
11. Change into the lustre kernel source and run the setup commands:
cd /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
[linux]$ cp /boot/config-`uname -r` .config
[linux]$ make oldconfig || make menuconfig
# For 2.6 kernels
[linux]$ make include/asm
[linux]$ make include/linux/version.h
[linux]$ make SUBDIRS=scripts
12. Extract the lustre source distribution (using lustre-1.6.4.3.tar.gz
rather than the RPM):
tar zxf lustre-1.6.4.3.tar.gz
13. Run the configure script:
cd lustre-1.6.4.3/
./configure --with-linux=/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
--with-o2ib=/usr/src/ofa_kernel
N.B. Cannot include "--with-linux-obj=" option as the configure
script
exits with an error and a recommendation to run "make config" in
the
linux src tree:
checking for
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/include/linux/autoconf.h... no
configure: error: Run make config in
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3.
If I do this, the build fails very early on:
[root at node006 lustre-1.6.4.3]# make
test -d CVS || exit 0; \
list=""; for mod in $list; do \
perl ./build/kabi -v archive $HOME/nonfree $mod || exit $?; \
done
make all-recursive
make[1]: Entering directory `/root/HPC/build/lustre-1.6.4.3''
Making all in ldiskfs
make[2]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs''
test -d CVS || exit 0; \
list=""; for mod in $list; do \
perl ./build/kabi -v archive $HOME/nonfree $mod || exit $?; \
done
make all-recursive
make[3]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs''
Making all in .
make[4]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs''
for dir in ldiskfs ; do \
make sources -C $dir || exit $? ; \
done
make[5]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
rm -rf linux-stage linux sources
mkdir -p linux-stage/fs/ext3 linux-stage/include/linux
cp /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/acl.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/balloc.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/bitmap.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/dir.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/file.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/fsync.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/hash.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/ialloc.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/inode.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/ioctl.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/namei.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/resize.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/super.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/symlink.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_security.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_trusted.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_user.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/acl.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/Makefile
linux-stage/fs/ext3
cp /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs_i.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs_sb.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_jbd.h
linux-stage/include/linux
Applying ext3 patches: ext3-wantedi-2.6-rhel4.patch iopen-2.6-rhel4.patch
export_symbols-ext3-2.6-suse.patch ext3-map_inode_page-2.6-suse.patch
ext3-ea-in-inode-2.6-rhel4.patch export-ext3-2.6-rhel4.patch
ext3-include-fixes-2.6-rhel4.patch
ext3-extents-2.6.9-rhel4.patch ext3-extents-fixes-2.6.9-rhel4.patch
ext3-extents-multiblock-directio-2.6.9-rhel4.patch
ext3-extents-search-2.6.9-rhel4.patch ext3-extents-sanity-checks.patch
ext3-mballoc3-core.patch ext3-mballoc3-rhel4.patch
ext3-nlinks-2.6.9.patch ext3-ialloc-2.6.patch ext3-lookup-dotdot-2.6.9.patch
ext3-sector_t-overflow-2.6.9-rhel4.patch ext3-check-jbd-errors-2.6.9.patch
ext3-uninit-2.6.9.patch ext3-nanosecond-2.6-rhel4.patch ext3-unlink-race.patch
ext3-mmp-2.6-rhel4.patch
ext3-fiemap-2.6-sles10.patch
mkdir linux
Replacing ''ext3'' with ''ldiskfs'': acl.h
xattr.h acl.c balloc.c bitmap.c dir.c file.c fsync.c hash.c ialloc.c inode.c
ioctl.c namei.c resize.c super.c symlink.c xattr.c xattr_security.c
xattr_trusted.c xattr_user.c iopen.c iopen.h extents.c mballoc.c group.h
fiemap.h ext3_fs.h ext3_fs_i.h ext3_fs_sb.h ext3_jbd.h ext3_extents.h
touch sources
make[5]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
make CC="gcc -m64" -C /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
\
-f /root/HPC/build/lustre-1.6.4.3/ldiskfs/build/Makefile
LUSTRE_LINUX_CONFIG=/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/.config \
LINUXINCLUDE='' -Iinclude $(if $(KBUILD_SRC),-Iinclude2
-I$(srctree)/include) -include include/linux/autoconf.h'' \
M=/root/HPC/build/lustre-1.6.4.3/ldiskfs -o tmp_include_depends -o scripts -o \
include/config/MARKER modules
make[5]: Entering directory
`/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3''
LD /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/built-in.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/acl.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/balloc.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/bitmap.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/dir.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/file.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/fsync.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/hash.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ialloc.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/inode.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ioctl.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/namei.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/resize.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/super.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/symlink.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_security.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_trusted.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_user.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/iopen.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/extents.o
CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/mballoc.o
LD [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.o
Building modules, stage 2.
MODPOST
CC /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.mod.o
LD [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.ko
make[5]: Leaving directory
`/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3''
make[4]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs''
Making all in ldiskfs
make[4]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
cd .. ; /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/scripts/mod/modpost -m -i
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/Module.symvers \
-o /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/Module.symvers
ldiskfs/ldiskfs.o
test -d CVS || exit 0; \
list=""; for mod in $list; do \
perl ../build/kabi -v archive $HOME/nonfree $mod || exit $?; \
done
make[4]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
make[3]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs''
make[2]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs''
Making all in .
make[2]: Entering directory `/root/HPC/build/lustre-1.6.4.3''
for dir in ldiskfs lnet lustre ; do \
make sources -C $dir || exit $? ; \
done
make[3]: Entering directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs''
for dir in ldiskfs ; do \
make sources -C $dir || exit $? ; \
done
make[4]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
make[4]: `sources'' is up to date.
make[4]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
make[3]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/ldiskfs''
make[3]: Entering directory `/root/HPC/build/lustre-1.6.4.3/lnet''
make sources -C libcfs
make[4]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/lnet/libcfs''
make[4]: Nothing to be done for `sources''.
make[4]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/lnet/libcfs''
make[3]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/lnet''
make[3]: Entering directory `/root/HPC/build/lustre-1.6.4.3/lustre''
make sources -C lvfs
make[4]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/lustre/lvfs''
sed -e "s/dx_hash_info/ext3_dx_hash_info/g" -e
"s/dir_private_info/ext3_dir_private_info/g" -e
"s/DX_HASH/EXT3_DX_HASH/g" -e
"s/reserve_window/ext3_reserve_window/g" -e
"s/rsv_window_add/ext3_rsv_window_add/g" -e
"s/EXT3/LDISKFS/g" -e "s/ext3/ldiskfs/g"
fsfilt_ext3.c > fsfilt_ldiskfs.c
touch sources
make[4]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/lustre/lvfs''
make sources -C obdclass
make[4]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/lustre/obdclass''
make[4]: Nothing to be done for `sources''.
make[4]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/lustre/obdclass''
perl ../lustre/scripts/version_tag.pl .. .. > tmpver
echo "#define LUSTRE_RELEASE
2.6.9_67.0.4.EL_lustre.1.6.4.3custom_200805151022" >> tmpver
cmp -s ../lustre/include/lustre/lustre_build_version.h tmpver > tmpdiff 2>
/dev/null && \
rm -f tmpver tmpdiff || \
mv -f tmpver ../lustre/include/lustre/lustre_build_version.h
make[3]: Leaving directory `/root/HPC/build/lustre-1.6.4.3/lustre''
make CC="gcc -m64" -C /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
\
-f /root/HPC/build/lustre-1.6.4.3/build/Makefile
LUSTRE_LINUX_CONFIG=/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/.config \
LINUXINCLUDE=''-I/usr/src/ofa_kernel/include -Iinclude $(if
$(KBUILD_SRC),-Iinclude2 -I$(srctree)/include) -include
include/linux/autoconf.h'' \
M=/root/HPC/build/lustre-1.6.4.3 -o tmp_include_depends -o scripts -o \
include/config/MARKER modules
make[3]: Entering directory
`/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3''
LD /root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/built-in.o
CC [M] /root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.o
In file included from
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:24:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:330: error: syntax
error before "DECLARE_PCI_UNMAP_ADDR"
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:330: warning: no
semicolon at end of struct or union
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:333: error: syntax
error before ''}'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:333: warning: type
defaults to `int'' in declaration of `kib_rx_t''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:333: warning: data
definition has no type or storage class
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:353: error: syntax
error before "DECLARE_PCI_UNMAP_ADDR"
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:353: warning: no
semicolon at end of struct or union
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:369: error: syntax
error before ''}'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:369: warning: type
defaults to `int'' in declaration of `kib_tx_t''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:369: warning: data
definition has no type or storage class
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:400: error: syntax
error before "kib_rx_t"
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:400: warning: no
semicolon at end of struct or union
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:407: error: syntax
error before ''}'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:407: warning: type
defaults to `int'' in declaration of `kib_conn_t''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:407: warning: data
definition has no type or storage class
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:492: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:494: warning: return
type defaults to `int''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h: In function
`kiblnd_get_conn_locked'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:498: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:498: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:498: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:498: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:498: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:502: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:503: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h: In function
`kiblnd_send_keepalive'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:505: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:505: error: (Each
undeclared identifier is reported only once
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:505: error: for each
function it appears in.)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:505: warning: type
defaults to `int'' in declaration of `__dummy2''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:505: warning:
comparison of distinct pointer types lacks a cast
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:510: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:511: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h: In function
`kiblnd_abort_receives'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:512: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:547: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:548: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h: In function
`kiblnd_set_conn_state'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:549: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:549: error:
`state'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:703: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:703: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:704: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:705: warning: type
defaults to `int'' in declaration of `kiblnd_create_conn''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:705: warning: data
definition has no type or storage class
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:706: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:706: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:707: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:707: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:708: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:708: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:710: error: syntax
error before "kib_tx_t"
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:711: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:713: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:713: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:714: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:714: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:715: error: syntax
error before "kib_tx_t"
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:715: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:717: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:717: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:727: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.h:727: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_del_peer_locked'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:452: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:458: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:458: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:458: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:458: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:458: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:521: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:523: warning: return
type defaults to `int''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_get_conn_by_idx'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:526: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:527: warning: ISO
C90 forbids mixed declarations and code
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:548: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:548: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:548: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:548: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:548: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:562: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:563: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_debug_rx'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:564: error:
`rx'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:570: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:571: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_debug_tx'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:572: error:
`tx'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:582: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:583: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_debug_conn'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:587: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:598: warning:
left-hand operand of comma expression has no effect
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:599: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:599: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:599: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:599: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:599: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:602: warning:
left-hand operand of comma expression has no effect
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:603: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:603: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:603: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:603: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:603: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:606: warning:
left-hand operand of comma expression has no effect
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:607: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:607: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:607: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:607: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:607: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:610: warning:
left-hand operand of comma expression has no effect
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:611: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:611: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:611: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:611: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:611: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:614: warning:
left-hand operand of comma expression has no effect
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:615: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:615: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:615: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:615: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:615: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:624: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:626: warning: return
type defaults to `int''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_create_conn'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:634: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:635: warning: ISO
C90 forbids mixed declarations and code
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:695: error:
`rx'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:721: error: too few
arguments to function `ib_create_cq''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: At top level:
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:837: error: syntax
error before ''*'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:838: warning:
function declaration isn''t a prototype
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_destroy_conn'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:839: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:880: error:
`rx'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_close_peer_conns_locked'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:916: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:917: warning: ISO
C90 forbids mixed declarations and code
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:922: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:922: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:922: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:922: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:922: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_close_stale_conns_locked'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:934: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:935: warning: ISO
C90 forbids mixed declarations and code
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:940: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:940: warning: type
defaults to `int'' in declaration of `__mptr''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:940: warning:
initialization from incompatible pointer type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:940: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:940: error: syntax
error before '')'' token
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_ctl'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1027: error:
`conn'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_free_tx_descs'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1102: error:
`tx'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1102: error: invalid
use of undefined type `struct kib_tx''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1102: error:
dereferencing pointer to incomplete type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_alloc_tx_descs'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1166: error:
`tx'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1166: error: invalid
use of undefined type `struct kib_tx''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1166: error:
dereferencing pointer to incomplete type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_unmap_tx_descs'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1210: error:
`tx'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1211: warning: ISO
C90 forbids mixed declarations and code
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1216: error: invalid
use of undefined type `struct kib_tx''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1216: error:
dereferencing pointer to incomplete type
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c: In function
`kiblnd_map_tx_descs'':
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1232: error:
`tx'' undeclared (first use in this function)
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1233: warning: ISO
C90 forbids mixed declarations and code
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1245: error: invalid
use of undefined type `struct kib_tx''
/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.c:1245: error:
dereferencing pointer to incomplete type
make[7]: *** [/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd/o2iblnd.o] Error
1
make[6]: *** [/root/HPC/build/lustre-1.6.4.3/lnet/klnds/o2iblnd] Error 2
make[5]: *** [/root/HPC/build/lustre-1.6.4.3/lnet/klnds] Error 2
make[4]: *** [/root/HPC/build/lustre-1.6.4.3/lnet] Error 2
make[3]: *** [_module_/root/HPC/build/lustre-1.6.4.3] Error 2
make[3]: Leaving directory
`/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3''
make[2]: *** [modules] Error 2
make[2]: Leaving directory `/root/HPC/build/lustre-1.6.4.3''
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/root/HPC/build/lustre-1.6.4.3''
make: *** [all] Error 2
Brian J. Murrell
2008-May-15 13:25 UTC
[Lustre-discuss] Lustre 1.6.4.3 + OFED 1.2.5.5 + RHEL 4u4 AS
A search of bugzilla yields bug 15315 which identifies bug 15030 as well. Please read through those two bugs. b. -------------- next part -------------- A non-text attachment was scrubbed... Name: not available Type: application/pgp-signature Size: 189 bytes Desc: This is a digitally signed message part Url : http://lists.lustre.org/pipermail/lustre-discuss/attachments/20080515/53941848/attachment.bin
Andreas Dilger
2008-May-15 16:49 UTC
[Lustre-discuss] Lustre 1.6.4.3 + OFED 1.2.5.5 + RHEL 4u4 AS
On May 15, 2008 11:37 +0100, Malcolm Cowe wrote:> Having some trouble with building Lustre 1.6.4.3 with OFED 1.2.5.5 on a > RHEL 4u4 AS server. Could somebody please help me to understand where > I''ve gone wrong? Here''s what I have done so far:If you are interested in trying an easier way of getting Lustre + OFED working, you could try downloading the 1.6.5 RC build, as it has all of the support for building the o2iblnd module built in. Check out lustre from CVS and then use "./lustrecvs -r b_release_1_6_5", and http://wiki.lustre.org/index.php?title=Open_CVS http://wiki.lustre.org/index.php?title=BuildLustre Cheers, Andreas -- Andreas Dilger Sr. Staff Engineer, Lustre Group Sun Microsystems of Canada, Inc.
cxmtnbike
2008-Jul-01 12:32 UTC
[Lustre-discuss] Lustre 1.6.4.3 + OFED 1.2.5.5 + RHEL 4u4 AS
It doesn''t appear as if you''ve come up with a solution to this
problem. I too have run into the same set of issues as you appear to
have here, but I think I have resolved them. I am running SLES 10 SP1
but this should also work for you.
first patch lnet/klnds/o2iblnd/o2iblnd.h You''ll need to add the
following line somewhere near the top:
#include <linux/pci.h>
Next you''ll need to patch lnet/klnds/o2iblnd/o2iblnd.c. ib_create_cq
uses 6 args in OFED-1.2.5.5 but o2iblnd.c only has 5 in the call. The
new line should look like this:
cq = ib_create_cq(cmid->device,
kiblnd_cq_completion, kiblnd_cq_event, conn,
IBLND_CQ_ENTRIES(), 0);
After applying the two changes mentioned above I have been able to
make though ''make rpms'' finishes it is a tad chatty with
warning
messages.
On May 15, 12:37 pm, Malcolm Cowe <Malcolm.C... at Sun.COM>
wrote:> Hi Folks,
>
> Having some trouble with building Lustre 1.6.4.3 with OFED 1.2.5.5 on a
> RHEL 4u4 AS server. Could somebody please help me to understand where
> I''ve gone wrong? Here''s what I have done so far:
>
> 1. Install RHEL 4u4 AS (full installation).
>
> 2. Download the Lustre RPMs from sun.com:
>
> e2fsprogs-1.40.4.cfs1-0redhat.x86_64.rpm
> e2fsprogs-devel-1.40.4.cfs1-0redhat.x86_64.rpm
> kernel-lustre-smp-2.6.9-67.0.4.EL_lustre.1.6.4.3.x86_64.rpm
> kernel-lustre-source-2.6.9-67.0.4.EL_lustre.1.6.4.3.x86_64.rpm
> lustre-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
> lustre-modules-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
> lustre-source-1.6.4.3-2.6.9_67.0.4.EL_lustre.1.6.4.3smp.x86_64.rpm
> plus: lustre-1.6.4.3.tar.gz
>
> 3. Install kernel-lustre-smp and kernel-lustre-source rpms.
> - Change grub to boot from lustre patched kernel by default.
> - Reboot.
>
> 4. Download OFED distribution from openib.org:
>
> OFED-1.2.5.5.tgz
>
> 5. Extract OFED distribution.
>
> 6. Install OFED:
>
> cd OFED-1.2.5.5/
> ./install.sh
> 2) Install OFED Software
> 3) All packages (all of Basic, HPC)
> [accept defaults for everything, configure IPoIB IP address].
>
> 7. Reboot.
>
> 8. Modify "Module.symvers", removing all references to Infiniband
> modules supplied with the kernel distribution. N.B. Could not find this
> file in the lustre kernel source tree, only in the "-obj"
tree.
>
> vi
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/x86_64/smp/Module.symvers
>
> 9. Run
"/usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh" and
> append the resulting file to the existing Module.symvers file:
>
> /usr/share/doc/ofed-docs-1.2.5.5/create_Module.symvers.sh
> cat Module.symvers >>
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/x86_64/smp/Module.symvers
>
> 10. Change into the lustre kernel source and edit the Makefile. Change
> "custom" suffix to "smp" in the variable
"EXTRAVERSION".
>
> 11. Change into the lustre kernel source and run the setup commands:
>
> cd /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
> [linux]$ cp /boot/config-`uname -r` .config
> [linux]$ make oldconfig || make menuconfig
> # For 2.6 kernels
> [linux]$ make include/asm
> [linux]$ make include/linux/version.h
> [linux]$ make SUBDIRS=scripts
>
> 12. Extract the lustre source distribution (using lustre-1.6.4.3.tar.gz
> rather than the RPM):
>
> tar zxf lustre-1.6.4.3.tar.gz
>
> 13. Run the configure script:
>
> cd lustre-1.6.4.3/
> ./configure --with-linux=/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3
--with-o2ib=/usr/src/ofa_kernel
>
> N.B. Cannot include "--with-linux-obj=" option as the
configure script
> exits with an error and a recommendation to run "make config"
in the
> linux src tree:
>
> checking for
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3-obj/include/linux/autoconf.h... no
> configure: error: Run make config in
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3.
>
> If I do this, the build fails very early on:
>
> [root at node006 lustre-1.6.4.3]# make
> test -d CVS || exit 0; \
> list=""; for mod in $list; do \
> perl ./build/kabi -v archive $HOME/nonfree $mod || exit $?; \
> done
> make all-recursive
> make[1]: Entering directory `/root/HPC/build/lustre-1.6.4.3''
> Making all in ldiskfs
> make[2]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs''
> test -d CVS || exit 0; \
> list=""; for mod in $list; do \
> perl ./build/kabi -v archive $HOME/nonfree $mod || exit $?; \
> done
> make all-recursive
> make[3]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs''
> Making all in .
> make[4]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs''
> for dir in ldiskfs ; do \
> make sources -C $dir || exit $? ; \
> done
> make[5]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
> rm -rf linux-stage linux sources
> mkdir -p linux-stage/fs/ext3 linux-stage/include/linux
> cp /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/acl.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/balloc.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/bitmap.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/dir.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/file.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/fsync.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/hash.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/ialloc.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/inode.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/ioctl.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/namei.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/resize.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/super.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/symlink.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_security.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_trusted.c
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr_user.c
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/acl.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/xattr.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/fs/ext3/Makefile
linux-stage/fs/ext3
> cp /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs_i.h
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_fs_sb.h
> /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/include/linux/ext3_jbd.h
linux-stage/include/linux
> Applying ext3 patches: ext3-wantedi-2.6-rhel4.patch iopen-2.6-rhel4.patch
export_symbols-ext3-2.6-suse.patch ext3-map_inode_page-2.6-suse.patch
ext3-ea-in-inode-2.6-rhel4.patch export-ext3-2.6-rhel4.patch
ext3-include-fixes-2.6-rhel4.patch
> ext3-extents-2.6.9-rhel4.patch ext3-extents-fixes-2.6.9-rhel4.patch
ext3-extents-multiblock-directio-2.6.9-rhel4.patch
ext3-extents-search-2.6.9-rhel4.patch ext3-extents-sanity-checks.patch
ext3-mballoc3-core.patch ext3-mballoc3-rhel4.patch
> ext3-nlinks-2.6.9.patch ext3-ialloc-2.6.patch
ext3-lookup-dotdot-2.6.9.patch ext3-sector_t-overflow-2.6.9-rhel4.patch
ext3-check-jbd-errors-2.6.9.patch ext3-uninit-2.6.9.patch
ext3-nanosecond-2.6-rhel4.patch ext3-unlink-race.patch ext3-mmp-2.6-rhel4.patch
> ext3-fiemap-2.6-sles10.patch
> mkdir linux
> Replacing ''ext3'' with ''ldiskfs'': acl.h
xattr.h acl.c balloc.c bitmap.c dir.c file.c fsync.c hash.c ialloc.c inode.c
ioctl.c namei.c resize.c super.c symlink.c xattr.c xattr_security.c
xattr_trusted.c xattr_user.c iopen.c iopen.h extents.c mballoc.c group.h
> fiemap.h ext3_fs.h ext3_fs_i.h ext3_fs_sb.h ext3_jbd.h ext3_extents.h
> touch sources
> make[5]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
> make CC="gcc -m64" -C
/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3 \
> -f /root/HPC/build/lustre-1.6.4.3/ldiskfs/build/Makefile
LUSTRE_LINUX_CONFIG=/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/.config \
> LINUXINCLUDE='' -Iinclude $(if $(KBUILD_SRC),-Iinclude2
-I$(srctree)/include) -include include/linux/autoconf.h'' \
> M=/root/HPC/build/lustre-1.6.4.3/ldiskfs -o tmp_include_depends -o scripts
-o \
> include/config/MARKER modules
> make[5]: Entering directory
`/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3''
> LD /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/built-in.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/acl.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/balloc.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/bitmap.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/dir.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/file.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/fsync.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/hash.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ialloc.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/inode.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ioctl.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/namei.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/resize.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/super.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/symlink.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_security.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_trusted.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/xattr_user.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/iopen.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/extents.o
> CC [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/mballoc.o
> LD [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.o
> Building modules, stage 2.
> MODPOST
> CC /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.mod.o
> LD [M] /root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs/ldiskfs.ko
> make[5]: Leaving directory
`/usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3''
> make[4]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs''
> Making all in ldiskfs
> make[4]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
> cd .. ; /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/scripts/mod/modpost
-m -i /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/Module.symvers \
> -o /usr/src/linux-2.6.9-67.0.4.EL_lustre.1.6.4.3/Module.symvers
ldiskfs/ldiskfs.o
> test -d CVS || exit 0; \
> list=""; for mod in $list; do \
> perl ../build/kabi -v archive $HOME/nonfree $mod || exit $?; \
> done
> make[4]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs/ldiskfs''
> make[3]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs''
> make[2]: Leaving directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs''
> Making all in .
> make[2]: Entering directory `/root/HPC/build/lustre-1.6.4.3''
> for dir in ldiskfs lnet lustre ; do \
> make sources -C $dir || exit $? ; \
> done
> make[3]: Entering directory
`/root/HPC/build/lustre-1.6.4.3/ldiskfs''
> for ...
>
> read more ?