Dear All, I try to build kernel for a patchless client. The system is an uptodate FC7. Linux fedora7 2.6.22.1-41.fc7 #1 SMP Fri Jul 27 18:10:34 EDT 2007 i686 i686 i386 GNU/Linux Considering to the docs a kernel for a client supposed to be built something like this: ./configure --with-linux=/usr/src/lustre/linux-2.6.22 But I get this message: checking that modules can be built at all... no configure: WARNING: Consult config.log for details. configure: WARNING: If you are trying to build with a kernel-source rpm, consult build/README.kernel-source configure: error: Kernel modules cannot be built. Why? What is this? Than I installed kernel-devel package and tried this: ./configure --with-linux=/usr/src/kernels/2.6.22.1-41.fc7-i686 checking which ldiskfs series to use... configure: WARNING: Unknown kernel version 2.6.22.1-41.fc7, fix ldiskfs/configure.ac Even if I use --disable-ldiskfs. So make rpms doesn''t work. The make;make install commands work, lustre.ko go to the right place, but when I try to insert it, I get this messages. # modprobe lustre WARNING: Error inserting libcfs (/lib/modules/2.6.22.1-41.fc7/kernel/net/lustre/libcfs.ko): Operation not permitted WARNING: Error inserting lvfs (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/lvfs.ko): Unknown symbol in module, or unknown parameter (see dmesg) WARNING: Error inserting lnet (/lib/modules/2.6.22.1-41.fc7/kernel/net/lustre/lnet.ko): Unknown symbol in module, or unknown parameter (see dmesg) WARNING: Error inserting obdclass (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/obdclass.ko): Unknown symbol in module, or unknown parameter (see dmesg) WARNING: Error inserting ptlrpc (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/ptlrpc.ko): Unknown symbol in module, or unknown parameter (see dmesg) WARNING: Error inserting mdc (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/mdc.ko): Unknown symbol in module, or unknown parameter (see dmesg) WARNING: Error inserting lov (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/lov.ko): Unknown symbol in module, or unknown parameter (see dmesg) FATAL: Error inserting lustre (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/lustre.ko): Unknown symbol in module, or unknown parameter (see dmesg) What''s wrong, what do I miss? The docs are not always clear to me. Maybe is there any step-by-step howto for building kernel for a patchless client? Thank you very much, tamas
On Aug 17, 2007 18:45 +0200, Papp Tamas wrote:> I try to build kernel for a patchless client. > > Than I installed kernel-devel package and tried this: > > ./configure --with-linux=/usr/src/kernels/2.6.22.1-41.fc7-i686 > > checking which ldiskfs series to use... configure: WARNING: Unknown > kernel version 2.6.22.1-41.fc7, fix ldiskfs/configure.ac > > Even if I use --disable-ldiskfs.Try also --disable-server. Cheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
On Fri, Aug 17, 2007 at 01:03:41PM -0600, Andreas Dilger wrote:> On Aug 17, 2007 18:45 +0200, Papp Tamas wrote: > > I try to build kernel for a patchless client. > > > > Than I installed kernel-devel package and tried this: > > > > ./configure --with-linux=/usr/src/kernels/2.6.22.1-41.fc7-i686 > > > > checking which ldiskfs series to use... configure: WARNING: Unknown > > kernel version 2.6.22.1-41.fc7, fix ldiskfs/configure.ac > > > > Even if I use --disable-ldiskfs. > > Try also --disable-server.Of course, but it didn''t helped. tamas
On Sat, Aug 18, 2007 at 03:08:54AM -0400, Aaron Knister wrote:> Can you post anything in dmesg or /var/log/messages relative to lustre?I can see: LustreError: 22171:0:(linux-proc.c:249:insert_proc()) couldn''t register dump_kernel LustreError: 22171:0:(module.c:378:init_libcfs_module()) insert_proc: error -1 lvfs: Unknown symbol lbug_with_loc lvfs: Unknown symbol libcfs_assertion_failed lvfs: Unknown symbol cfs_free lvfs: Unknown symbol cfs_alloc lvfs: Unknown symbol libcfs_kmemory lvfs: Unknown symbol libcfs_debug_vmsg2 [...and so on...] Also in /var/log/messages I can see only this messages. tamas
Can you post anything in dmesg or /var/log/messages relative to lustre?> WARNING: Error inserting libcfs (/lib/modules/2.6.22.1-41.fc7/ > kernel/net/lustre/libcfs.ko): Operation > not permittedThat line looks fishy. I wonder what''s going on. The other modules are failing to load as a result. -Aaron On Aug 17, 2007, at 12:45 PM, Papp Tamas wrote:> Dear All, > > I try to build kernel for a patchless client. > > The system is an uptodate FC7. > > Linux fedora7 2.6.22.1-41.fc7 #1 SMP Fri Jul 27 18:10:34 EDT 2007 i686 > i686 i386 GNU/Linux > > > Considering to the docs a kernel for a client supposed to be built > something like this: > > ./configure --with-linux=/usr/src/lustre/linux-2.6.22 > > But I get this message: > > checking that modules can be built at all... no > configure: WARNING: Consult config.log for details. > configure: WARNING: If you are trying to build with a kernel-source > rpm, consult build/README.kernel-source > configure: error: Kernel modules cannot be built. > > Why? What is this? > > > Than I installed kernel-devel package and tried this: > > ./configure --with-linux=/usr/src/kernels/2.6.22.1-41.fc7-i686 > > checking which ldiskfs series to use... configure: WARNING: Unknown > kernel version 2.6.22.1-41.fc7, fix ldiskfs/configure.ac > > Even if I use --disable-ldiskfs. > > So make rpms doesn''t work. > > The make;make install commands work, lustre.ko go to the right place, > but when I try to insert it, I get this messages. > > # modprobe lustre > WARNING: Error inserting libcfs (/lib/modules/2.6.22.1-41.fc7/ > kernel/net/lustre/libcfs.ko): Operation > not permitted > WARNING: Error inserting lvfs (/lib/modules/2.6.22.1-41.fc7/kernel/ > fs/lustre/lvfs.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > WARNING: Error inserting lnet (/lib/modules/2.6.22.1-41.fc7/kernel/ > net/lustre/lnet.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > WARNING: Error inserting obdclass (/lib/modules/2.6.22.1-41.fc7/ > kernel/fs/lustre/obdclass.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > WARNING: Error inserting ptlrpc (/lib/modules/2.6.22.1-41.fc7/ > kernel/fs/lustre/ptlrpc.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > WARNING: Error inserting mdc (/lib/modules/2.6.22.1-41.fc7/kernel/ > fs/lustre/mdc.ko): Unknown symbol > in module, or unknown parameter (see dmesg) > WARNING: Error inserting lov (/lib/modules/2.6.22.1-41.fc7/kernel/ > fs/lustre/lov.ko): Unknown symbol > in module, or unknown parameter (see dmesg) > FATAL: Error inserting lustre (/lib/modules/2.6.22.1-41.fc7/kernel/ > fs/lustre/lustre.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > > > What''s wrong, what do I miss? > > The docs are not always clear to me. > > Maybe is there any step-by-step howto for building kernel for a > patchless client? > > Thank you very much, > > tamas > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discussAaron Knister Systems Administrator/ Center for Research on Environment and Water (301) 595-7001 aaron@iges.org -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20070818/ea402821/attachment.html
Could you post 5 or 6 lines before that? On Aug 19, 2007, at 4:57 AM, Papp Tamas wrote:> On Sat, Aug 18, 2007 at 03:08:54AM -0400, Aaron Knister wrote: >> Can you post anything in dmesg or /var/log/messages relative to >> lustre? > > I can see: > > LustreError: 22171:0:(linux-proc.c:249:insert_proc()) couldn''t > register dump_kernel > LustreError: 22171:0:(module.c:378:init_libcfs_module()) > insert_proc: error -1 > lvfs: Unknown symbol lbug_with_loc > lvfs: Unknown symbol libcfs_assertion_failed > lvfs: Unknown symbol cfs_free > lvfs: Unknown symbol cfs_alloc > lvfs: Unknown symbol libcfs_kmemory > lvfs: Unknown symbol libcfs_debug_vmsg2 > [...and so on...] > > Also in /var/log/messages I can see only this messages. > > tamasAaron Knister Systems Administrator/ Center for Research on Environment and Water (301) 595-7001 aaron@iges.org -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20070819/3c9a65c1/attachment-0001.html
Hello Papp Tamas ,>>> I try to build kernel for a patchless client. >>> >>> Than I installed kernel-devel package and tried this: >>> >>> ./configure --with-linux=/usr/src/kernels/2.6.22.1-41.fc7-i686 >>> >>> checking which ldiskfs series to use... configure: WARNING: Unknown >>> kernel version 2.6.22.1-41.fc7, fix ldiskfs/configure.ac >>> >>> Even if I use --disable-ldiskfs. >>> >> Try also --disable-server. >> > > Of course, but it didn''t helped. >Did you try ''make oldconfig && make'' in /usr/src/kernels/2.6.22.1-41.fc7-i686 before compiling patchless client?
On Mon, Aug 20, 2007 at 10:37:58AM +0300, Valentyn Klindukh wrote:> > Did you try ''make oldconfig && make'' in > /usr/src/kernels/2.6.22.1-41.fc7-i686 before compiling patchless client?Yes, of course. tamas
On Sun, Aug 19, 2007 at 10:36:52PM -0400, Aaron Knister wrote:> Could you post 5 or 6 lines before that?There is no lustre related message. EXT3 FS on sda2, internal journal kjournald starting. Commit interval 5 seconds EXT3 FS on sda1, internal journal EXT3-fs: mounted filesystem with ordered data mode. Adding 4096564k swap on /dev/sda3. Priority:-1 extents:1 across:4096564k IA-32 Microcode Update Driver: v1.14a <tigran@aivazian.fsnet.co.uk> r8169: eth0: link up r8169: eth0: link up audit(1187361362.175:3): audit_pid=1685 old=0 by auid=4294967295 Bluetooth: Core ver 2.11 NET: Registered protocol family 31 Bluetooth: HCI device and connection manager initialized Bluetooth: HCI socket layer initialized Bluetooth: L2CAP ver 2.8 Bluetooth: L2CAP socket layer initialized Bluetooth: RFCOMM socket layer initialized Bluetooth: RFCOMM TTY layer initialized Bluetooth: RFCOMM ver 1.8 Bluetooth: HIDP (Human Interface Emulation) ver 1.2 eth0: no IPv6 routers present nvidia: module license ''NVIDIA'' taints kernel. PCI: Setting latency timer of device 0000:01:00.0 to 64 NVRM: loading NVIDIA UNIX x86 Kernel Module 100.14.11 Wed Jun 13 18:21:22 PDT 2007 PCI: Setting latency timer of device 0000:01:00.0 to 64 NVRM: loading NVIDIA UNIX x86 Kernel Module 100.14.11 Wed Jun 13 18:21:22 PDT 2007 LustreError: 4538:0:(linux-proc.c:249:insert_proc()) couldn''t register dump_kernel LustreError: 4538:0:(module.c:378:init_libcfs_module()) insert_proc: error -1 lvfs: Unknown symbol lbug_with_loc tamas
On Fri, 2007-08-17 at 18:45 +0200, Papp Tamas wrote:> > # modprobe lustre > WARNING: Error inserting libcfs (/lib/modules/2.6.22.1-41.fc7/kernel/net/lustre/libcfs.ko): Operation > not permittedHrm. Why this? Your prompt indicates you are root when you do the modprobe. Verify that your uid==0 with "id" before you modprobe. Alternatively do you have some security framework installed that is preventing the module from being installed? Does your kernel require signed modules perhaps, or is something like selinux preventing module loading? I think FC defaults to having selinux enabled. If you are running selinux or some other MAC/capabilities enforcement framework try disabling it/them before you attempt your modprobe.> WARNING: Error inserting lvfs (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/lvfs.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > WARNING: Error inserting lnet (/lib/modules/2.6.22.1-41.fc7/kernel/net/lustre/lnet.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > WARNING: Error inserting obdclass (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/obdclass.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > WARNING: Error inserting ptlrpc (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/ptlrpc.ko): Unknown > symbol in module, or unknown parameter (see dmesg) > WARNING: Error inserting mdc (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/mdc.ko): Unknown symbol > in module, or unknown parameter (see dmesg) > WARNING: Error inserting lov (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/lov.ko): Unknown symbol > in module, or unknown parameter (see dmesg) > FATAL: Error inserting lustre (/lib/modules/2.6.22.1-41.fc7/kernel/fs/lustre/lustre.ko): Unknown > symbol in module, or unknown parameter (see dmesg)These are all just fallout from not being able to load libcfs.ko.> Maybe is there any step-by-step howto for building kernel for a > patchless client?Just to get your nomenclature correct so that your explanation is more clear, by definition you don''t build a kernel for the patchless client. You use the kernel supplied with your system and what you are building is the (kernel modules for the) patchless client.> Thank you very much,NP. b.
On Sun, Aug 19, 2007 at 10:57:52AM +0200, Papp Tamas wrote:>On Sat, Aug 18, 2007 at 03:08:54AM -0400, Aaron Knister wrote: >> Can you post anything in dmesg or /var/log/messages relative to lustre? >LustreError: 22171:0:(linux-proc.c:249:insert_proc()) couldn''t register dump_kernel >LustreError: 22171:0:(module.c:378:init_libcfs_module()) insert_proc: error -1you have hit this: https://bugzilla.lustre.org/show_bug.cgi?id=12782 Bernd (I think) wrote a "2.6.21-sysctl.patch" to fix this and it looks like about half of this patch made it into 1.6.1. I''ve attached an updated version of the rest of the patch that applies cleanly to 1.6.1. also attached is a buildfix patch for 1.6.1 which is necessary for building with OpenIB/o2ib. most of this was from the TACC guys in this thread: https://bugzilla.lustre.org/show_bug.cgi?id=13378 and it appears that the same fixes are also necesary for using the standard in-kernel InfiniBand stack. eg. ./configure --with-linux=/usr/src/linux-2.6.22.4 --with-o2ib=/usr/src/linux-2.6.22.4 --disable-server --enable-quota both patches are so far only lightly tested against 2.6.22.4 on x86_64. cheers, robin -- Dr Robin Humble, HPC Systems Analyst, Australian National Facility -------------- next part -------------- diff -ru lustre-1.6.1/lnet/include/libcfs/linux/linux-prim.h lustre-1.6.1.rjh/lnet/include/libcfs/linux/linux-prim.h --- lustre-1.6.1/lnet/include/libcfs/linux/linux-prim.h 2007-05-17 16:46:58.000000000 +1000 +++ lustre-1.6.1.rjh/lnet/include/libcfs/linux/linux-prim.h 2007-08-21 18:33:34.000000000 +1000 @@ -39,6 +39,8 @@ #include <linux/proc_fs.h> #include <linux/mm.h> #include <linux/timer.h> +#include <linux/signal.h> +#include <linux/sched.h> #include <linux/miscdevice.h> #include <libcfs/linux/portals_compat25.h> diff -ru lustre-1.6.1/lnet/klnds/o2iblnd/o2iblnd.c lustre-1.6.1.rjh/lnet/klnds/o2iblnd/o2iblnd.c --- lustre-1.6.1/lnet/klnds/o2iblnd/o2iblnd.c 2007-06-21 03:43:52.000000000 +1000 +++ lustre-1.6.1.rjh/lnet/klnds/o2iblnd/o2iblnd.c 2007-08-21 18:09:58.000000000 +1000 @@ -718,7 +718,7 @@ cq = ib_create_cq(cmid->device, kiblnd_cq_completion, kiblnd_cq_event, conn, - IBLND_CQ_ENTRIES()); + IBLND_CQ_ENTRIES(), 0); if (!IS_ERR(cq)) { conn->ibc_cq = cq; } else { diff -ru lustre-1.6.1/lnet/klnds/o2iblnd/o2iblnd.h lustre-1.6.1.rjh/lnet/klnds/o2iblnd/o2iblnd.h --- lustre-1.6.1/lnet/klnds/o2iblnd/o2iblnd.h 2007-06-21 15:02:47.000000000 +1000 +++ lustre-1.6.1.rjh/lnet/klnds/o2iblnd/o2iblnd.h 2007-08-21 18:08:39.000000000 +1000 @@ -41,6 +41,7 @@ #include <asm/uaccess.h> #include <asm/io.h> +#include <linux/pci.h> #include <linux/init.h> #include <linux/fs.h> #include <linux/file.h> -------------- next part -------------- diff -ru lustre-1.6.1.orig/lnet/libcfs/linux/linux-proc.c lustre-1.6.1/lnet/libcfs/linux/linux-proc.c --- lustre-1.6.1.orig/lnet/libcfs/linux/linux-proc.c 2007-06-14 19:16:49.000000000 +1000 +++ lustre-1.6.1/lnet/libcfs/linux/linux-proc.c 2007-08-21 20:10:22.000000000 +1000 @@ -62,6 +62,8 @@ static struct ctl_table_header *lnet_table_header = NULL; extern char lnet_upcall[1024]; +extern struct ctl_table top_table[2]; + #define PSDEV_LNET (0x100) enum { PSDEV_DEBUG = 1, /* control debugging */ @@ -239,10 +241,24 @@ { struct proc_dir_entry *ent; + ent = proc_mkdir("sys/lnet", NULL); + if (ent == NULL) { + CERROR("couldn''t create /proc/sys/lnet \n"); + return -1; + } + #ifdef CONFIG_SYSCTL if (!lnet_table_header) +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,21) + lnet_table_header = register_sysctl_table(top_table); +#else lnet_table_header = cfs_register_sysctl_table(top_table, 0); #endif + if (!lnet_table_header) { + CERROR("registering top_table failed\n"); + return -1; + } +#endif ent = create_proc_entry("sys/lnet/dump_kernel", 0, NULL); if (ent == NULL) { diff -ru lustre-1.6.1.orig/lustre/obdclass/linux/linux-sysctl.c lustre-1.6.1/lustre/obdclass/linux/linux-sysctl.c --- lustre-1.6.1.orig/lustre/obdclass/linux/linux-sysctl.c 2007-07-06 09:42:32.000000000 +1000 +++ lustre-1.6.1/lustre/obdclass/linux/linux-sysctl.c 2007-08-21 19:59:47.000000000 +1000 @@ -83,6 +83,26 @@ return rc; } +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,21) +static ctl_table obd_table[] = { + {OBD_FAIL_LOC, "fail_loc", &obd_fail_loc, sizeof(int), 0644, NULL, + NULL, &proc_fail_loc}, + {OBD_TIMEOUT, "timeout", &obd_timeout, sizeof(int), 0644, NULL, + NULL, &proc_set_timeout}, + {OBD_DEBUG_PEER_ON_TIMEOUT, "debug_peer_on_timeout", + &obd_debug_peer_on_timeout, + sizeof(int), 0644, NULL, NULL, &proc_dointvec}, + {OBD_DUMP_ON_TIMEOUT, "dump_on_timeout", &obd_dump_on_timeout, + sizeof(int), 0644, NULL, NULL, &proc_dointvec}, + {OBD_DUMP_ON_EVICTION, "dump_on_eviction", &obd_dump_on_eviction, + sizeof(int), 0644, NULL, NULL, &proc_dointvec}, + {OBD_MEMUSED, "memused", (int *)&obd_memory.counter, + sizeof(int), 0644, NULL, NULL, &proc_dointvec}, + {OBD_LDLM_TIMEOUT, "ldlm_timeout", &ldlm_timeout, sizeof(int), 0644, + NULL, NULL, &proc_set_timeout}, + { 0 } +}; +#else static ctl_table obd_table[] = { {OBD_FAIL_LOC, "fail_loc", &obd_fail_loc, sizeof(int), 0644, NULL, &proc_fail_loc}, @@ -103,6 +123,7 @@ NULL, &proc_set_timeout}, { 0 } }; +#endif static ctl_table parent_table[] = { {OBD_SYSCTL, "lustre", NULL, 0, 0555, obd_table}, @@ -113,8 +134,12 @@ { #ifdef CONFIG_SYSCTL if ( !obd_table_header ) +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,21) + obd_table_header = register_sysctl_table(parent_table); +#else obd_table_header = cfs_register_sysctl_table(parent_table, 0); #endif +#endif } void obd_sysctl_clean (void)
On Tue, Aug 21, 2007 at 06:42:01AM -0400, Robin Humble wrote:> On Sun, Aug 19, 2007 at 10:57:52AM +0200, Papp Tamas wrote: > >On Sat, Aug 18, 2007 at 03:08:54AM -0400, Aaron Knister wrote: > >> Can you post anything in dmesg or /var/log/messages relative to lustre? > >LustreError: 22171:0:(linux-proc.c:249:insert_proc()) couldn''t register dump_kernel > >LustreError: 22171:0:(module.c:378:init_libcfs_module()) insert_proc: error -1 > > you have hit this: > https://bugzilla.lustre.org/show_bug.cgi?id=12782 > > Bernd (I think) wrote a "2.6.21-sysctl.patch" to fix this and it looks > like about half of this patch made it into 1.6.1. > I''ve attached an updated version of the rest of the patch that applies > cleanly to 1.6.1. > > also attached is a buildfix patch for 1.6.1 which is necessary for > building with OpenIB/o2ib. most of this was from the TACC guys in this > thread: > https://bugzilla.lustre.org/show_bug.cgi?id=13378 > and it appears that the same fixes are also necesary for using the > standard in-kernel InfiniBand stack. eg. > ./configure --with-linux=/usr/src/linux-2.6.22.4 --with-o2ib=/usr/src/linux-2.6.22.4 --disable-server --enable-quota > > both patches are so far only lightly tested against 2.6.22.4 on x86_64.Thank you. But now I cannot build it at all. ./configure --with-linux=/usr/src/kernels/2.6.22.1-41.fc7-i686/ --with-o2ib=/usr/src/kernels/2.6.22.1-41.fc7-i686/ --disable-server --enable-quota make: [...] CC [M] /usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-fs.o CC [M] /usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-sync.o CC [M] /usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-tcpip.o CC [M] /usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-lwt.o CC [M] /usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-proc.o /usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-proc.c:163: error: static declaration of .top_table. follows non-static declaration /usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-proc.c:65: error: previous declaration of .top_table. was here make[6]: *** [/usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-proc.o] Error 1 make[5]: *** [/usr/src/lustre/lustre-1.6.1/lnet/libcfs] Error 2 make[4]: *** [/usr/src/lustre/lustre-1.6.1/lnet] Error 2 make[3]: *** [_module_/usr/src/lustre/lustre-1.6.1] Error 2 make[3]: Leaving directory `/usr/src/kernels/2.6.22.1-41.fc7-i686'' make[2]: *** [modules] Error 2 make[2]: Leaving directory `/usr/src/lustre/lustre-1.6.1'' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/usr/src/lustre/lustre-1.6.1'' make: *** [all] Error 2 tamas
On Tue, Aug 21, 2007 at 01:11:20PM +0200, Papp Tamas wrote:>On Tue, Aug 21, 2007 at 06:42:01AM -0400, Robin Humble wrote: >> On Sun, Aug 19, 2007 at 10:57:52AM +0200, Papp Tamas wrote: >> >On Sat, Aug 18, 2007 at 03:08:54AM -0400, Aaron Knister wrote: >> >> Can you post anything in dmesg or /var/log/messages relative to lustre? >> >LustreError: 22171:0:(linux-proc.c:249:insert_proc()) couldn''t register dump_kernel >> >LustreError: 22171:0:(module.c:378:init_libcfs_module()) insert_proc: error -1 >> you have hit this: >> https://bugzilla.lustre.org/show_bug.cgi?id=12782 >But now I cannot build it at all. >/usr/src/lustre/lustre-1.6.1/lnet/libcfs/linux/linux-proc.c:163: >error: static declaration of .top_table. follows non-static >declarationah, my bad. looks like your gcc4 is pickier than my gcc3. just delete the line: extern struct ctl_table top_table[2]; alternatively, updated patch attached. cheers, robin -------------- next part -------------- diff -ru lustre-1.6.1.orig/lnet/libcfs/linux/linux-proc.c lustre-1.6.1/lnet/libcfs/linux/linux-proc.c --- lustre-1.6.1.orig/lnet/libcfs/linux/linux-proc.c 2007-06-14 19:16:49.000000000 +1000 +++ lustre-1.6.1/lnet/libcfs/linux/linux-proc.c 2007-08-21 21:31:25.000000000 +1000 @@ -239,10 +239,24 @@ { struct proc_dir_entry *ent; + ent = proc_mkdir("sys/lnet", NULL); + if (ent == NULL) { + CERROR("couldn''t create /proc/sys/lnet \n"); + return -1; + } + #ifdef CONFIG_SYSCTL if (!lnet_table_header) +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,21) + lnet_table_header = register_sysctl_table(top_table); +#else lnet_table_header = cfs_register_sysctl_table(top_table, 0); #endif + if (!lnet_table_header) { + CERROR("registering top_table failed\n"); + return -1; + } +#endif ent = create_proc_entry("sys/lnet/dump_kernel", 0, NULL); if (ent == NULL) { diff -ru lustre-1.6.1.orig/lustre/obdclass/linux/linux-sysctl.c lustre-1.6.1/lustre/obdclass/linux/linux-sysctl.c --- lustre-1.6.1.orig/lustre/obdclass/linux/linux-sysctl.c 2007-07-06 09:42:32.000000000 +1000 +++ lustre-1.6.1/lustre/obdclass/linux/linux-sysctl.c 2007-08-21 20:18:21.000000000 +1000 @@ -83,6 +83,26 @@ return rc; } +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,21) +static ctl_table obd_table[] = { + {OBD_FAIL_LOC, "fail_loc", &obd_fail_loc, sizeof(int), 0644, NULL, + NULL, &proc_fail_loc}, + {OBD_TIMEOUT, "timeout", &obd_timeout, sizeof(int), 0644, NULL, + NULL, &proc_set_timeout}, + {OBD_DEBUG_PEER_ON_TIMEOUT, "debug_peer_on_timeout", + &obd_debug_peer_on_timeout, + sizeof(int), 0644, NULL, NULL, &proc_dointvec}, + {OBD_DUMP_ON_TIMEOUT, "dump_on_timeout", &obd_dump_on_timeout, + sizeof(int), 0644, NULL, NULL, &proc_dointvec}, + {OBD_DUMP_ON_EVICTION, "dump_on_eviction", &obd_dump_on_eviction, + sizeof(int), 0644, NULL, NULL, &proc_dointvec}, + {OBD_MEMUSED, "memused", (int *)&obd_memory.counter, + sizeof(int), 0644, NULL, NULL, &proc_dointvec}, + {OBD_LDLM_TIMEOUT, "ldlm_timeout", &ldlm_timeout, sizeof(int), 0644, + NULL, NULL, &proc_set_timeout}, + { 0 } +}; +#else static ctl_table obd_table[] = { {OBD_FAIL_LOC, "fail_loc", &obd_fail_loc, sizeof(int), 0644, NULL, &proc_fail_loc}, @@ -103,6 +123,7 @@ NULL, &proc_set_timeout}, { 0 } }; +#endif static ctl_table parent_table[] = { {OBD_SYSCTL, "lustre", NULL, 0, 0555, obd_table}, @@ -113,8 +134,12 @@ { #ifdef CONFIG_SYSCTL if ( !obd_table_header ) +#if LINUX_VERSION_CODE >= KERNEL_VERSION(2,6,21) + obd_table_header = register_sysctl_table(parent_table); +#else obd_table_header = cfs_register_sysctl_table(parent_table, 0); #endif +#endif } void obd_sysctl_clean (void)
On Tue, Aug 21, 2007 at 07:39:19AM -0400, Robin Humble wrote:> > ah, my bad. looks like your gcc4 is pickier than my gcc3. > just delete the line: > extern struct ctl_table top_table[2]; > > alternatively, updated patch attached.Excellent! Thank you very much. tamas
On Tue, Aug 21, 2007 at 02:57:44PM +0200, Papp Tamas wrote:> > Excellent! > > Thank you very much.So, thank you, it''s working now. But as I earlier wrote, I have other problems. First of all, I cannot build the rpm, because of the lack of the right ldiskfs support. Even if I use configure --disable-utils --disable-server. checking which ldiskfs series to use... configure: WARNING: Unknown kernel version 2.6.22.1-41.fc7, fix ldiskfs/configure.ac In the file ldiskfs/configure.ac I can see the kernels only up to 2.6.18. So I used --prefix=/usr/local/lustre, simple make (without the rpms switch) works. Now everything (I guess) is there, but mount.lustre. It''s in /sbin. Why? I like to keep every lustre related file at one place. I guess, it would be really great, if there will be configure options to build only one part of the whole source (eg. --only-client, --only-modules, etc.). Is it possible to make a request?:) Thank you for your help. tamas
On Tue, Aug 21, 2007 at 04:01:22PM +0200, Papp Tamas wrote:>First of all, I cannot build the rpm, because of the lack of the >right ldiskfs support. >Even if I use configure --disable-utils --disable-server.hmmmm... works for me :-/ ./configure --with-linux=/usr/src/linux-2.6.22.4 --with-o2ib=/usr/src/linux-2.6.22.4 --disable-server --enable-quota make make rpms and 6 rpms arrived. this is CentOS 4.5 x86_64. I tend not to use those rpms though - I just copy the kernel modules from the patchless build and then use the standard CFS rpms for everything else. lets me use more lustre version with random kernels.>So I used --prefix=/usr/local/lustre, simple make (without the rpms >switch) works.so I guess my question is whether a make ; make rpms works for you?>Now everything (I guess) is there, but mount.lustre. It''s in /sbin. >Why? I like to keep every lustre related file at one place./sbin is the standard linux place for all mount.* utilities. it''s also where all mkfs.* files go, but actually, for some reason mkfs.lustre is in /usr/sbin/ - that''s wrong. cheers, robin>I guess, it would be really great, if there will be configure options >to build only one part of the whole source (eg. --only-client, >--only-modules, etc.). > >Is it possible to make a request?:) > > > >Thank you for your help. > >tamas
On Tue, Aug 21, 2007 at 10:25:54AM -0400, Robin Humble wrote:> > hmmmm... works for me :-/ > ./configure --with-linux=/usr/src/linux-2.6.22.4 --with-o2ib=/usr/src/linux-2.6.22.4 --disable-server --enable-quota > make > make rpms > and 6 rpms arrived. this is CentOS 4.5 x86_64. > > I tend not to use those rpms though - I just copy the kernel modules > from the patchless build and then use the standard CFS rpms for > everything else. lets me use more lustre version with random kernels.This is an FC7 system with readline 5, but readline 4 is needed by the official the rpm. The problem is probably a kernel version typo, I gues. But what exactly?> >So I used --prefix=/usr/local/lustre, simple make (without the rpms > >switch) works. > > so I guess my question is whether a > make ; make rpms > works for you?No, because the make rpms command wants to build the ldiskfs rpm too (not just modules and client utilities)> /sbin is the standard linux place for all mount.* utilities. it''s also > where all mkfs.* files go, but actually, for some reason mkfs.lustre is > in /usr/sbin/ - that''s wrong.That''s right. But it would be nice, let me choose, what I want to do. By the way, in the most systems the local filesystems have pactical reason to keep them there. Regards, tamas
On Tue, Aug 21, 2007 at 05:32:32PM +0200, Papp Tamas wrote:>On Tue, Aug 21, 2007 at 10:25:54AM -0400, Robin Humble wrote: >> so I guess my question is whether a >> make ; make rpms >> works for you? >No, because the make rpms command wants to build the ldiskfs rpm too >(not just modules and client utilities)yeah, you''re right, seems like ''make rpms'' is busted on f7. but I don''t see it being broken in the way you describe. % make rpms ... rpmbuild -ta lustre-1.6.1.tar.gz error: line 261: second %prep make[1]: *** [rpms-real] Error 1 make[1]: Leaving directory `/home/rjh/lustre/lustre-1.6.1'' make: *** [rpms] Error 2 this works though: untar, configure copy tarball to <rpmdir>/SOURCES/ rpmbuild -ba lustre.spec and rpms arrive. applying patches and rebuilding the tarball is a possible necessary extra step. cheers, robin
On Tue, Aug 21, 2007 at 06:42:01AM -0400, Robin Humble wrote:>On Sun, Aug 19, 2007 at 10:57:52AM +0200, Papp Tamas wrote: >>LustreError: 22171:0:(linux-proc.c:249:insert_proc()) couldn''t register dump_kernel >>LustreError: 22171:0:(module.c:378:init_libcfs_module()) insert_proc: error -1 >you have hit this: > https://bugzilla.lustre.org/show_bug.cgi?id=12782 >Bernd (I think) wrote a "2.6.21-sysctl.patch" to fix this and it looks >like about half of this patch made it into 1.6.1. >I''ve attached an updated version of the rest of the patch that applies >cleanly to 1.6.1. > >also attached is a buildfix patch for 1.6.1 which is necessary for >building with OpenIB/o2ib. most of this was from the TACC guys in this >thread: > https://bugzilla.lustre.org/show_bug.cgi?id=13378 >and it appears that the same fixes are also necesary for using the >standard in-kernel InfiniBand stack. eg.looks like these fixes slipped through the cracks and didn''t make it into Lustre 1.6.2. updated for 1.6.2 patchless client build fixes are here: https://bugzilla.lustre.org/show_bug.cgi?id=13510 cheers, robin