Lustre discuss - Oct 2006 - Lustre 1.6-beta Debian package with 2.6.18 a success

Hi,

big thanks go to Alastair McKinstry for gathering the patches from the
mailinglist and my patches for 1.4.6 Lustre into the Debian package of
Lustre 1.6-beta. They worked like a charm.

Big thanks to Cluster File Systems, Inc. too for Lustre itself. :)


I fetched the package source from svn.debian.org (pkg-lustre),
compiled it, installed lustre-source and linux-patch-lustre, unpacked
/usr/src/lustre.tar.bz2 and then build myself a kernel and module debs
with make-kpkg the Debian way without a hitch.

I then installed all the debs on a test system and configured it
following https://mail.clusterfs.com/wikis/lustre/MountConf. One combo
MDT/MGS and one OST on the same system and then mounted on the same
system. Again everything went without a problem.

It never was so easy to install Lustre on Debian.

I started bonnie to give the system a workout and so far it hasn''t
crashed. Lets see if bonnie is finished and the system still running
in the morning.

MfG
        Goswin

Do you know what I might be missing to get the patchless client working
on debian with vservers.

I can get the patchless client working without vservers and I had heard
that someone had it working with vservers.
I can build the lustre source patchless client with the vservers patched
in the kernel but then when I go to mount lustre FS I get a bunch of
unknown symbols reported to dmesg.

Here is a sample of a few:

libcfs: Unknown symbol vx_rmap_pid
lvfs: Unknown symbol lbug_with_loc
lvfs: Unknown symbol libcfs_assertion_failed
lvfs: Unknown symbol libcfs_debug_msg
lvfs: Unknown symbol cfs_free
lvfs: Unknown symbol cfs_alloc
lvfs: Unknown symbol libcfs_kmemory
lvfs: Unknown symbol libcfs_catastrophe
lvfs: Unknown symbol libcfs_subsystem_debug
lvfs: Unknown symbol libcfs_debug
lnet: Unknown symbol cfs_free_large
lnet: Unknown symbol libcfs_sock_connect
lnet: Unknown symbol libcfs_ipif_free_enumeration
lnet: Unknown symbol libcfs_isknown_lnd
lnet: Unknown symbol libcfs_sock_release
lnet: Unknown symbol libcfs_register_ioctl
lnet: Unknown symbol libcfs_str2net
lnet: Unknown symbol libcfs_ipif_enumerate
lnet: Unknown symbol cfs_restore_sigs
lnet: Unknown symbol lbug_with_loc
lnet: Unknown symbol libcfs_assertion_failed
lnet: Unknown symbol libcfs_debug_msg
lnet: Unknown symbol libcfs_sock_write
lnet: Unknown symbol libcfs_lnd2modname
lnet: Unknown symbol libcfs_str2nid
lnet: Unknown symbol cfs_free
lnet: Unknown symbol libcfs_sock_accept
lnet: Unknown symbol libcfs_run_upcall
lnet: Unknown symbol libcfs_sock_read
lnet: Unknown symbol cfs_alloc
lnet: Unknown symbol cfs_alloc_large
lnet: Unknown symbol cfs_daemonize
lnet: Unknown symbol libcfs_net2str
lnet: Unknown symbol libcfs_kmemory
lnet: Unknown symbol libcfs_sock_getaddr
lnet: Unknown symbol libcfs_deregister_ioctl
lnet: Unknown symbol cfs_block_allsigs
lnet: Unknown symbol libcfs_id2str
..... rest omitted for sake of space.

Any thoughts would be greatly appreciated.

Thanks,

Jon


On Wed, 2006-10-25 at 17:32 +0200, Goswin von Brederlow wrote:
> Hi,
> 
> big thanks go to Alastair McKinstry for gathering the patches from the
> mailinglist and my patches for 1.4.6 Lustre into the Debian package of
> Lustre 1.6-beta. They worked like a charm.
> 
> Big thanks to Cluster File Systems, Inc. too for Lustre itself. :)
> 
> 
> I fetched the package source from svn.debian.org (pkg-lustre),
> compiled it, installed lustre-source and linux-patch-lustre, unpacked
> /usr/src/lustre.tar.bz2 and then build myself a kernel and module debs
> with make-kpkg the Debian way without a hitch.
> 
> I then installed all the debs on a test system and configured it
> following https://mail.clusterfs.com/wikis/lustre/MountConf. One combo
> MDT/MGS and one OST on the same system and then mounted on the same
> system. Again everything went without a problem.
> 
> It never was so easy to install Lustre on Debian.
> 
> I started bonnie to give the system a workout and so far it hasn''t
> crashed. Lets see if bonnie is finished and the system still running
> in the morning.
> 
> MfG
>         Goswin
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
> 
Jon Scottorn
Systems Administrator
The Possibility Forge, Inc.
http://www.possibilityforge.com
435.635.0591 x.1004
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20061025/aa98dc0e/attachment.html

On Oct 25, 2006  13:48 -0600, Jon Scottorn wrote:
> Do you know what I might be missing to get the patchless client working
> on debian with vservers.
> 
> I can get the patchless client working without vservers and I had heard
> that someone had it working with vservers.
> I can build the lustre source patchless client with the vservers patched
> in the kernel but then when I go to mount lustre FS I get a bunch of
> unknown symbols reported to dmesg.
> 
> Here is a sample of a few:
> 
> libcfs: Unknown symbol vx_rmap_pid
This is caused by the vserver patch, I''d guess.  Maybe it adds a macro
that
Lustre calls somewhere that accesses this symbol?  Can''t help you
there.
> lvfs: Unknown symbol lbug_with_loc
> [etc]
Caused by the failure to load libcfs.


Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

Jon Scottorn <jscottorn@possibilityforge.com> writes:
> Do you know what I might be missing to get the patchless client working on
> debian with vservers.
Sorry, never tried patchless nor vserver.

That is something low on the todo for me. Before that I will try xen.

MfG
        Goswin

Goswin von Brederlow wrote:
> Jon Scottorn <jscottorn@possibilityforge.com> writes:
>
>   
>> Do you know what I might be missing to get the patchless client working
on
>> debian with vservers.
>>     
>
> Sorry, never tried patchless nor vserver.
>
> That is something low on the todo for me. Before that I will try xen.
>
> MfG
>         Goswin
>   
Ok, Now seeing an oops when I try to umount a Lustre FS (either client or
mgt / mgs  / ost server ):

general protection fault: 0000 [1] SMP
CPU 0
Modules linked in: osc mgc lustre lov lquota mdc ksocklnd ptlrpc obdclass lnet l
vfs libcfs ipv6 dm_snapshot dm_mirror dm_mod usbhid usbkbd ipmi_watchdog ipmi_de
vintf ipmi_poweroff ipmi_si ipmi_msghandler dummy loop ide_generic ide_disk evde
v psmouse i2c_piix4 i2c_core serio_raw pcspkr shpchp pci_hotplug ext3 jbd mbcach
e ide_cd cdrom sd_mod serverworks sata_svw libata scsi_mod ohci_hcd ehci_hcd gen
eric ide_core tg3 fan
Pid: 2721, comm: umount Not tainted 2.6.18-1-xen-amd64 #1
RIP: e030:[<ffffffff8825bed3>]  [<ffffffff8825bed3>]
:obdclass:class_exp2obd+0x7 /0xc
RSP: e02b:ffff880057af1c40  EFLAGS: 00010206
RAX: 0000000000000000 RBX: ffff88006362a800 RCX: 0000000000000010
RDX: ffff880057b0c209 RSI: ffffffff883d3c30 RDI: 76653a7475706e69
RBP: ffff88006362a6c0 R08: 00000000fffffffe R09: 0000000000000020
R10: 0000000000000000 R11: 0000000000000000 R12: 0000000000000000
R13: ffff88006194fe20 R14: 0000000000511b70 R15: 0000000000511c00
FS:  00002b3c0923c1d0(0000) GS:ffffffff804c3000(0000) knlGS:0000000000000000
CS:  e033 DS: 0000 ES: 0000
Process umount (pid: 2721, threadinfo ffff880057af0000, task ffff88005dc330c0)
Stack:  ffffffff883bfdfd  0000000000000000  0000000000000000  0000000000000000
 0000000000000000  0000000000000000  0000000000000000  0000000000000000
 0000000000000000  0000000000000000
Call Trace:
 [<ffffffff883bfdfd>] :lustre:ll_umount_begin+0xc8/0x69f
 [<ffffffff802bdd89>] sys_umount+0x11e/0x27b
 [<ffffffff802238a2>] sys_newstat+0x19/0x31
 [<ffffffff8026078a>] _spin_lock_irqsave+0x9/0x14
 [<ffffffff80230fa8>] __up_write+0x21/0x10d
 [<ffffffff8025c67e>] system_call+0x86/0x8b
 [<ffffffff8025c5f8>] system_call+0x0/0x8b


Code: 48 8b 47 70 c3 48 8b 57 70 31 c0 48 85 d2 74 07 48 8b 82 f0
RIP  [<ffffffff8825bed3>] :obdclass:class_exp2obd+0x7/0xc
 RSP <ffff880057af1c40>

Has this bug been seen before?

- Alastair

http://www.paul.sladen.org/vserver/archives/200506/0058.html

----- Original Message ----- 
From: "Andreas Dilger" <adilger@clusterfs.com>
To: "Jon Scottorn" <jscottorn@possibilityforge.com>
Cc: "lustre-discuss" <lustre-discuss@clusterfs.com>
Sent: Thursday, October 26, 2006 12:46 AM
Subject: Re: [Lustre-discuss] Lustre 1.6-beta Debian package with 2.6.18 
asuccess

> On Oct 25, 2006  13:48 -0600, Jon Scottorn wrote:
>> Do you know what I might be missing to get the patchless client working
>> on debian with vservers.
>>
>> I can get the patchless client working without vservers and I had heard
>> that someone had it working with vservers.
>> I can build the lustre source patchless client with the vservers
patched
>> in the kernel but then when I go to mount lustre FS I get a bunch of
>> unknown symbols reported to dmesg.
>>
>> Here is a sample of a few:
>>
>> libcfs: Unknown symbol vx_rmap_pid
>
> This is caused by the vserver patch, I''d guess.  Maybe it adds a
macro
> that
> Lustre calls somewhere that accesses this symbol?  Can''t help you
there.
>
>> lvfs: Unknown symbol lbug_with_loc
>> [etc]
>
> Caused by the failure to load libcfs.
>
>
> Cheers, Andreas
> --
> Andreas Dilger
> Principal Software Engineer
> Cluster File Systems, Inc.
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>
>

On Oct 26, 2006  16:21 +0200, konrad.gutkowski@ffs.pl
wrote:
> http://www.paul.sladen.org/vserver/archives/200506/0058.html
> During compilation you should have gotten warnings about implicit
> declarations of vx_rmap_pid. Take the .c files for which those warnings
> show up, and add an #include <linux/vs_cvirt.h> to them. That should
fix
> your issues.

It would make more sense to include this header into whatever defines the
macros that affect the compiled files.  That way no need to change the
compiled sources at all?
> ----- Original Message ----- 
> From: "Andreas Dilger" <adilger@clusterfs.com>
> To: "Jon Scottorn" <jscottorn@possibilityforge.com>
> Cc: "lustre-discuss" <lustre-discuss@clusterfs.com>
> Sent: Thursday, October 26, 2006 12:46 AM
> Subject: Re: [Lustre-discuss] Lustre 1.6-beta Debian package with 2.6.18 
> asuccess
> 
> 
> >On Oct 25, 2006  13:48 -0600, Jon Scottorn wrote:
> >>Do you know what I might be missing to get the patchless client
working
> >>on debian with vservers.
> >>
> >>I can get the patchless client working without vservers and I had
heard
> >>that someone had it working with vservers.
> >>I can build the lustre source patchless client with the vservers
patched
> >>in the kernel but then when I go to mount lustre FS I get a bunch
of
> >>unknown symbols reported to dmesg.
> >>
> >>Here is a sample of a few:
> >>
> >>libcfs: Unknown symbol vx_rmap_pid
> >
> >This is caused by the vserver patch, I''d guess.  Maybe it adds
a macro
> >that
> >Lustre calls somewhere that accesses this symbol?  Can''t help
you there.
Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.

On Thu, Oct 26, 2006 at 02:04:01PM -0600, Andreas Dilger
wrote:
> On Oct 26, 2006  16:21 +0200, konrad.gutkowski@ffs.pl wrote:
> > http://www.paul.sladen.org/vserver/archives/200506/0058.html
> > During compilation you should have gotten warnings about implicit
> > declarations of vx_rmap_pid. Take the .c files for which those
> > warnings show up, and add an #include <linux/vs_cvirt.h> to
them.
> > That should fix your issues.
> 
> It would make more sense to include this header into whatever defines
> the macros that affect the compiled files. That way no need to change
> the compiled sources at all?
not so trivial, for various reasons, but this has
been fixed in a different way in recent patches

best,
Herbert
> > ----- Original Message ----- 
> > From: "Andreas Dilger" <adilger@clusterfs.com>
> > To: "Jon Scottorn" <jscottorn@possibilityforge.com>
> > Cc: "lustre-discuss" <lustre-discuss@clusterfs.com>
> > Sent: Thursday, October 26, 2006 12:46 AM
> > Subject: Re: [Lustre-discuss] Lustre 1.6-beta Debian package with
2.6.18
> > asuccess
> > 
> > 
> > >On Oct 25, 2006  13:48 -0600, Jon Scottorn wrote:
> > >>Do you know what I might be missing to get the patchless
client
> > >>working on debian with vservers.
> > >>
> > >>I can get the patchless client working without vservers and I
had
> > >>heard that someone had it working with vservers. I can build
the
> > >>lustre source patchless client with the vservers patched in
the
> > >>kernel but then when I go to mount lustre FS I get a bunch of
> > >>unknown symbols reported to dmesg.
> > >>
> > >>Here is a sample of a few:
> > >>
> > >>libcfs: Unknown symbol vx_rmap_pid
> > >
> > >This is caused by the vserver patch, I''d guess. Maybe it
adds a
> > >macro that Lustre calls somewhere that accesses this symbol?
Can''t
> > >help you there.
> 
> Cheers, Andreas
> --
> Andreas Dilger
> Principal Software Engineer
> Cluster File Systems, Inc.
> 
> _______________________________________________
> Vserver mailing list
> Vserver@list.linux-vserver.org
> http://list.linux-vserver.org/mailman/listinfo/vserver