Hi, all Does anyone install Lustre 1.6.0.1 on Ubuntu 7.04 (kernel version 2.6.20) sucessfully? I googled a lot of times, but still cannot find any information. Does any have similar experiment or something that might be useful? Thanks , Chris -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20070624/971865b3/attachment.html
On Monday 25 June 2007 06:33:01 chris wrote:> Hi, all > > Does anyone install Lustre 1.6.0.1 on Ubuntu 7.04 (kernel version 2.6.20) > sucessfully? I googled a lot of times, but still cannot find any > information. > Does any have similar experiment or something that might be useful?See here: http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/1.6/ This should support up to linux-2.6.21, though I still do most tests with 2.6.20. I''m using Ubuntu Feisty as testing/development system and with unionfs-fuse (shameless self-advertisement ;) ) also as diskless server. If you are using an amd64-system I could also upload debs of the utils. Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH
----- Original Message ----- From: "Bernd Schubert" <bs@q-leap.de> To: <lustre-discuss@clusterfs.com> Cc: "chris" <chris@nchc.org.tw> Sent: Wednesday, June 27, 2007 12:57 AM Subject: Re: [Lustre-discuss] Lustre 1.6.0.1 in Ubuntu Fiesty> See here: > > http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/1.6/ > > This should support up to linux-2.6.21, though I still do most tests with > 2.6.20.After I finished the install instructions above, I tried to mount the lustre file system on MDS server, but failed.. Here are the commands and the error messages: ----------------------------------------------------------------------- # mkfs.lustre --fsname=spfs --mdt --mgs /dev/hdb Permanent disk data: Target: spfs-MDTffff Index: unassigned Lustre FS: spfs Mount type: ldiskfs Flags: 0x75 (MDT MGS needs_index first_time update ) Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr Parameters: checking for existing Lustre data: not found device size = 20480MB formatting backing filesystem ldiskfs on /dev/hdb target name spfs-MDTffff 4k blocks 0 options -J size=400 -i 4096 -I 512 -q -O dir_index -F mkfs_cmd = mkfs.ext2 -j -b 4096 -L spfs-MDTffff -J size=400 -i 4096 -I 512 -q -O dir_index -F /dev/hdb mkfs.lustre: Unable to mount /dev/hdb: No such device Is the ldiskfs module available? mkfs.lustre FATAL: failed to write local files mkfs.lustre: exiting with 19 (No such device) ------------------------------------------------------------------------- It seems the ldiskfs module does not load correctly. Could anyone help? Thanks~~~> > I''m using Ubuntu Feisty as testing/development system and with > unionfs-fuse > (shameless self-advertisement ;) ) also as diskless server. > > If you are using an amd64-system I could also upload debs of the utils. > > > Cheers, > Bernd > > -- > Bernd Schubert > Q-Leap Networks GmbH
Apologies for my reply to this fairly dated thread, however I would like to report that I have successfully built 1.6.0.1 on Feisty 7.04 (amd64) using 2.6.18-vanilla and modifying only lustre/obdfilter/ filter_io_26.c as described here: https://mail.clusterfs.com/pipermail/lustre-discuss/2006-October/ 002263.html Lustre works well, but there are numerous errors in dmesg on the OST''s and MGST that I can only speculate on. I do have a question regarding throughput in terms of what I should expect from a simple dd copy from /dev/zero to a striped file across 4 OST''s. I am new to working with Lustre and I am very excited about what has been accomplished by this project so far. Would it be possible for me to get some Deb packages if anyone has made any? so I can compare what I have to learn a bit more. -Joel Robison
Joel Robison wrote:> Apologies for my reply to this fairly dated thread, however I would > like to report that I have successfully built 1.6.0.1 on Feisty 7.04 > (amd64) using 2.6.18-vanilla and modifying only lustre/obdfilter/ > filter_io_26.c as described here: > https://mail.clusterfs.com/pipermail/lustre-discuss/2006-October/ > 002263.html > > Lustre works well, but there are numerous errors in dmesg on the > OST''s and MGST that I can only speculate on.Be carefull, probably your filesystem will suffer from silent data corruption. Just copy the hole linux source on your lustre filesystem and then umount and e2fsck the OSSs. If you want to use 2.6.0.1 as it is, at least you will need to disable extents,mballoc.> > I do have a question regarding throughput in terms of what I should > expect from a simple dd copy from /dev/zero to a striped file across > 4 OST''s. I am new to working with Lustre and I am very excited aboutThat stronlgy depends on the speed of your OSTs ;)> what has been accomplished by this project so far. > > Would it be possible for me to get some Deb packages if anyone has > made any? so I can compare what I have to learn a bit more.I have amd64 debs for the utils, but not for the kernel modules. Cheers, Bernd
Thanks for the quick reply Bernd,
I have not put any valuable data on the filesystem yet, so no worries
about data loss, just 1.2T of zeros.
How can I disable extents and mballoc? is it a ../configure option?
I am away from the machines right now or I would check myself.
As for my speed question, I apologize for not adding any info, I have
the following setup:
6 Supermicro servers each has:
2x SATA2 320GB drives with 16MB cache
2x Intel(R) Xeon(R) CPU 3060 @ 2.40GHz
2X Intel e1000 cards with TSO turned on, and a gigabit switch (I
am thinking of bonding the interfaces if that would help me with
performance)
I can also set the MTU''s up to 9000 on my switch.
-Joel Robison
On Jul 5, 2007, at 11:30 AM, Bernd Schubert wrote:
> Joel Robison wrote:
>
>> Apologies for my reply to this fairly dated thread, however I would
>> like to report that I have successfully built 1.6.0.1 on Feisty 7.04
>> (amd64) using 2.6.18-vanilla and modifying only lustre/obdfilter/
>> filter_io_26.c as described here:
>> https://mail.clusterfs.com/pipermail/lustre-discuss/2006-October/
>> 002263.html
>>
>> Lustre works well, but there are numerous errors in dmesg on the
>> OST''s and MGST that I can only speculate on.
>
> Be carefull, probably your filesystem will suffer from silent data
> corruption. Just copy the hole linux source on your lustre
> filesystem and
> then umount and e2fsck the OSSs.
> If you want to use 2.6.0.1 as it is, at least you will need to disable
> extents,mballoc.
>
>>
>> I do have a question regarding throughput in terms of what I should
>> expect from a simple dd copy from /dev/zero to a striped file across
>> 4 OST''s. I am new to working with Lustre and I am very
excited about
>
> That stronlgy depends on the speed of your OSTs ;)
>
>> what has been accomplished by this project so far.
>>
>> Would it be possible for me to get some Deb packages if anyone has
>> made any? so I can compare what I have to learn a bit more.
>
> I have amd64 debs for the utils, but not for the kernel modules.
>
> Cheers,
> Bernd
>
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20070705/c27f140f/attachment.html
Bernd Schubert
2007-Jul-05 15:23 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Joel Robison wrote:> Thanks for the quick reply Bernd, > > I have not put any valuable data on the filesystem yet, so no worries > about data loss, just 1.2T of zeros. > How can I disable extents and mballoc? is it a ../configure option? > I am away from the machines right now or I would check myself.On the OSSs 1.) umount {ost-mountpoint} 2.) tunefs.lustre --mountfsoptions="" /dev/{ost-device} 3.) mount {ost-mountpoint}> > As for my speed question, I apologize for not adding any info, I have > the following setup: > 6 Supermicro servers each has: > 2x SATA2 320GB drives with 16MB cache > 2x Intel(R) Xeon(R) CPU 3060 @ 2.40GHz > 2X Intel e1000 cards with TSO turned on, and a gigabit switch (I > am thinking of bonding the interfaces if that would help me with > performance) > > I can also set the MTU''s up to 9000 on my switch.I will answer to this tomorrow, it already late here and the answer will take some time. Cheers, Bernd
Brian Elliott Finley
2007-Jul-05 15:31 UTC
[Lustre-discuss] Lustre 1.6.0.1 in Ubuntu Fiesty
Joel, Very glad to hear this. I eagerly anticipate hearing more about your experience. We are an Ubuntu amd64 shop, and I am in the process of doing a deployment using SuSE based kernels, but would eventually like to be able to use very recent vanilla, or even Ubuntu based kernels. Cheers, -Brian Joel Robison wrote:> Apologies for my reply to this fairly dated thread, however I would > like to report that I have successfully built 1.6.0.1 on Feisty 7.04 > (amd64) using 2.6.18-vanilla and modifying only > lustre/obdfilter/filter_io_26.c as described here: > https://mail.clusterfs.com/pipermail/lustre-discuss/2006-October/002263.html > > > Lustre works well, but there are numerous errors in dmesg on the OST''s > and MGST that I can only speculate on. > > I do have a question regarding throughput in terms of what I should > expect from a simple dd copy from /dev/zero to a striped file across 4 > OST''s. I am new to working with Lustre and I am very excited about > what has been accomplished by this project so far. > > Would it be possible for me to get some Deb packages if anyone has > made any? so I can compare what I have to learn a bit more. > > -Joel Robison > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss-- Brian Elliott Finley Linux Strategist, CIS Desk: 630.252.4742 Cell: 630.631.6621
Harald van Pee
2007-Jul-05 15:42 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
On Thursday 05 July 2007 11:23 pm, Bernd Schubert wrote:> Joel Robison wrote: > > Thanks for the quick reply Bernd, > > > > I have not put any valuable data on the filesystem yet, so no worries > > about data loss, just 1.2T of zeros. > > How can I disable extents and mballoc? is it a ../configure option? > > I am away from the machines right now or I would check myself. > > On the OSSs > > 1.) umount {ost-mountpoint} > 2.) tunefs.lustre --mountfsoptions="" /dev/{ost-device} > 3.) mount {ost-mountpoint}would there a big performance loss and if so does the CFS team suggests a patch instead?> > > Cheers, > Bernd
Joel Robison
2007-Jul-05 16:30 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
After doing the steps below ( I actually reformatted all of the
drives used (sdb) in my case) the OST''s join the MDT/MGS just fine
but then when my client mounts the target it will not write data to it:
root@woolsey:/mnt/testfs# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/md1 455G 44G 389G 10% /
varrun 1005M 52K 1005M 1% /var/run
varlock 1005M 4.0K 1005M 1% /var/lock
procbususb 1005M 96K 1005M 1% /proc/bus/usb
udev 1005M 96K 1005M 1% /dev
devshm 1005M 0 1005M 0% /dev/shm
/dev/md0 89M 44M 40M 53% /boot
10.10.10.200@tcp:/testfs
1.2T 1.9G 1.1T 1% /mnt/testfs
root@woolsey:/mnt/testfs#
root@woolsey:/mnt/testfs# dd if=/dev/zero of=/mnt/testfs/striped/
pain.img bs=65536 count=100000
dd: writing `/mnt/testfs/striped/pain.img'': Invalid argument
1+0 records in
0+0 records out
0 bytes (0 B) copied, 0.004042 seconds, 0.0 kB/s
I have tried records of different sizes and still the same, here are
the errors on the client:
LustreError: 2568:0:(client.c:574:ptlrpc_check_status()) @@@ type ==
PTL_RPC_MSG_ERR, err == 1 req@ffff81007e0eca00 x69/t0 o4->testfs-
OST0000_UUID@10.10.10.201@tcp:28 lens 384/352 ref 2 fl Rpc:R/0/0 rc 0/1
LustreError: 2568:0:(client.c:574:ptlrpc_check_status()) @@@ type ==
PTL_RPC_MSG_ERR, err == 1 req@ffff81007e0ee600 x70/t0 o4->testfs-
OST0001_UUID@10.10.10.202@tcp:28 lens 384/352 ref 2 fl Rpc:R/0/0 rc 0/1
LustreError: 2568:0:(client.c:574:ptlrpc_check_status()) @@@ type ==
PTL_RPC_MSG_ERR, err == 1 req@ffff81007e0e2000 x68/t0 o4->testfs-
OST0003_UUID@10.10.10.205@tcp:28 lens 384/352 ref 2 fl Rpc:R/0/0 rc 0/1
these repeat several times and all OST''s are mentioned, not just
0,1,3 as above.
Here is the log from the MGS/MDT:
LustreError: 30032:0:(ldlm_lib.c:1363:target_send_reply_msg()) @@@
processing error (-107) req@ffff810076e64450 x590075/t0 o101-><?>@<?
>:-1 lens 232/0 ref 0 fl Interpret:/0/0 rc -107/0
LustreError: 30032:0:(ldlm_lib.c:1363:target_send_reply_msg())
Skipped 1 previous similar message
alot of these.
I dont think I didnt anything differently, can anyone point whats
going wrong here?
-Joel Robison
On Jul 5, 2007, at 2:42 PM, Harald van Pee wrote:
> On Thursday 05 July 2007 11:23 pm, Bernd Schubert wrote:
>> Joel Robison wrote:
>>> Thanks for the quick reply Bernd,
>>>
>>> I have not put any valuable data on the filesystem yet, so no
>>> worries
>>> about data loss, just 1.2T of zeros.
>>> How can I disable extents and mballoc? is it a ../configure option?
>>> I am away from the machines right now or I would check myself.
>>
>> On the OSSs
>>
>> 1.) umount {ost-mountpoint}
>> 2.) tunefs.lustre --mountfsoptions="" /dev/{ost-device}
>> 3.) mount {ost-mountpoint}
>
> would there a big performance loss and if so does the CFS team
> suggests a
> patch instead?
>
>>
>>
>> Cheers,
>> Bernd
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20070705/6b88ae5f/attachment-0001.html
Joel Robison
2007-Jul-05 16:40 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Sorry I forgot to include the OST logs, these help quite a bit: ldiskfs_map_inode_page: error reading block 0 LustreError: 3168:0:(fsfilt-ldiskfs.c: 1088:fsfilt_ldiskfs_map_bm_inode_pages()) ino 14303278, blk 0 cr 4294967295 create 1: rc 1 LustreError: 3168:0:(fsfilt-ldiskfs.c: 1088:fsfilt_ldiskfs_map_bm_inode_pages()) Skipped 57 previous similar messages ldiskfs_map_inode_page: error reading block 256 ldiskfs_map_inode_page: error reading block 512 ldiskfs_map_inode_page: error reading block 768 ldiskfs_map_inode_page: error reading block 1 if I am missing something obvious please, someone let me know, as previously mentioned I am very new to Lustre.do I want to use a different backingfstype? right now it just defaults to ldiskfs. -Joel Robison On Jul 5, 2007, at 3:29 PM, Joel Robison wrote:> After doing the steps below ( I actually reformatted all of the > drives used (sdb) in my case) the OST''s join the MDT/MGS just fine > but then when my client mounts the target it will not write data to > it: > > root@woolsey:/mnt/testfs# df -h > Filesystem Size Used Avail Use% Mounted on > /dev/md1 455G 44G 389G 10% / > varrun 1005M 52K 1005M 1% /var/run > varlock 1005M 4.0K 1005M 1% /var/lock > procbususb 1005M 96K 1005M 1% /proc/bus/usb > udev 1005M 96K 1005M 1% /dev > devshm 1005M 0 1005M 0% /dev/shm > /dev/md0 89M 44M 40M 53% /boot > 10.10.10.200@tcp:/testfs > 1.2T 1.9G 1.1T 1% /mnt/testfs > root@woolsey:/mnt/testfs# > root@woolsey:/mnt/testfs# dd if=/dev/zero of=/mnt/testfs/striped/ > pain.img bs=65536 count=100000 > dd: writing `/mnt/testfs/striped/pain.img'': Invalid argument > 1+0 records in > 0+0 records out > 0 bytes (0 B) copied, 0.004042 seconds, 0.0 kB/s > > I have tried records of different sizes and still the same, here > are the errors on the client: > LustreError: 2568:0:(client.c:574:ptlrpc_check_status()) @@@ type > == PTL_RPC_MSG_ERR, err == 1 req@ffff81007e0eca00 x69/t0 o4->testfs- > OST0000_UUID@10.10.10.201@tcp:28 lens 384/352 ref 2 fl Rpc:R/0/0 rc > 0/1 > LustreError: 2568:0:(client.c:574:ptlrpc_check_status()) @@@ type > == PTL_RPC_MSG_ERR, err == 1 req@ffff81007e0ee600 x70/t0 o4->testfs- > OST0001_UUID@10.10.10.202@tcp:28 lens 384/352 ref 2 fl Rpc:R/0/0 rc > 0/1 > LustreError: 2568:0:(client.c:574:ptlrpc_check_status()) @@@ type > == PTL_RPC_MSG_ERR, err == 1 req@ffff81007e0e2000 x68/t0 o4->testfs- > OST0003_UUID@10.10.10.205@tcp:28 lens 384/352 ref 2 fl Rpc:R/0/0 rc > 0/1 > > these repeat several times and all OST''s are mentioned, not just > 0,1,3 as above. > Here is the log from the MGS/MDT: > LustreError: 30032:0:(ldlm_lib.c:1363:target_send_reply_msg()) @@@ > processing error (-107) req@ffff810076e64450 x590075/t0 o101-><?>@<? > >:-1 lens 232/0 ref 0 fl Interpret:/0/0 rc -107/0 > LustreError: 30032:0:(ldlm_lib.c:1363:target_send_reply_msg()) > Skipped 1 previous similar message > > alot of these. > > I dont think I didnt anything differently, can anyone point whats > going wrong here? > > > > -Joel Robison > > > > > On Jul 5, 2007, at 2:42 PM, Harald van Pee wrote: > >> On Thursday 05 July 2007 11:23 pm, Bernd Schubert wrote: >>> Joel Robison wrote: >>>> Thanks for the quick reply Bernd, >>>> >>>> I have not put any valuable data on the filesystem yet, so no >>>> worries >>>> about data loss, just 1.2T of zeros. >>>> How can I disable extents and mballoc? is it a ../configure option? >>>> I am away from the machines right now or I would check myself. >>> >>> On the OSSs >>> >>> 1.) umount {ost-mountpoint} >>> 2.) tunefs.lustre --mountfsoptions="" /dev/{ost-device} >>> 3.) mount {ost-mountpoint} >> >> would there a big performance loss and if so does the CFS team >> suggests a >> patch instead? >> >>> >>> >>> Cheers, >>> Bernd >> >> _______________________________________________ >> Lustre-discuss mailing list >> Lustre-discuss@clusterfs.com >> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss-------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20070705/f71a94f8/attachment.html
Bernd Schubert
2007-Jul-06 02:55 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
On Thursday 05 July 2007 23:42:38 Harald van Pee wrote:> On Thursday 05 July 2007 11:23 pm, Bernd Schubert wrote: > > Joel Robison wrote: > > > Thanks for the quick reply Bernd, > > > > > > I have not put any valuable data on the filesystem yet, so no worries > > > about data loss, just 1.2T of zeros. > > > How can I disable extents and mballoc? is it a ../configure option? > > > I am away from the machines right now or I would check myself. > > > > On the OSSs > > > > 1.) umount {ost-mountpoint} > > 2.) tunefs.lustre --mountfsoptions="" /dev/{ost-device} > > 3.) mount {ost-mountpoint} > > would there a big performance loss and if so does the CFS team suggests a > patch instead?Some numbers you might find here: http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/performance/ Patches are in bug#11039 Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH
Bernd Schubert
2007-Jul-06 02:59 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
On Friday 06 July 2007 00:29:58 Joel Robison wrote:> After doing the steps below ( I actually reformatted all of the > drives used (sdb) in my case) the OST''s join the MDT/MGS just fine > but then when my client mounts the target it will not write data to it:[...]> dd: writing `/mnt/testfs/striped/pain.img'': Invalid argumentSorry, entirely forgot that, it happend here, too. You will also need to remove extents and mballoc patches from the ldiskfs series and to recompile lustre. It might be easier then to take our recent release, though its for 2.6.20+ only. http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/1.6/ Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH
Joel Robison
2007-Jul-06 07:47 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Bernd, Thank you very much for your assistance/guidance. I will recompile as you directed and update with my results, I may not get back to the list until tomorrow. lots to do! -Joel Robison On Jul 6, 2007, at 1:59 AM, Bernd Schubert wrote:> On Friday 06 July 2007 00:29:58 Joel Robison wrote: >> After doing the steps below ( I actually reformatted all of the >> drives used (sdb) in my case) the OST''s join the MDT/MGS just fine >> but then when my client mounts the target it will not write data >> to it: > > [...] > >> dd: writing `/mnt/testfs/striped/pain.img'': Invalid argument > > Sorry, entirely forgot that, it happend here, too. You will also > need to > remove extents and mballoc patches from the ldiskfs series and to > recompile > lustre. > > It might be easier then to take our recent release, though its for > 2.6.20+ > only. > > http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/1.6/ > > > Cheers, > Bernd > > -- > Bernd Schubert > Q-Leap Networks GmbH > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
Harald van Pee
2007-Jul-06 09:02 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
On Friday 06 July 2007 10:55 am, Bernd Schubert wrote:> On Thursday 05 July 2007 23:42:38 Harald van Pee wrote: > > On Thursday 05 July 2007 11:23 pm, Bernd Schubert wrote: > > > Joel Robison wrote: > > > > Thanks for the quick reply Bernd, > > > > > > > > I have not put any valuable data on the filesystem yet, so no worries > > > > about data loss, just 1.2T of zeros. > > > > How can I disable extents and mballoc? is it a ../configure option? > > > > I am away from the machines right now or I would check myself. > > > > > > On the OSSs > > > > > > 1.) umount {ost-mountpoint} > > > 2.) tunefs.lustre --mountfsoptions="" /dev/{ost-device} > > > 3.) mount {ost-mountpoint} > > > > would there a big performance loss and if so does the CFS team suggests a > > patch instead? > > Some numbers you might find here: > http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/performance/thanks for your reply the performance loss is not small, but can I be sure that than do not run into bmalloc related file system crashes? Just to be sure because in the last discussions there are a lot of informations and I''m not sure what I have to do next.> > Patches are in bug#11039 > > > Cheers, > BerndI have looked into this but there for me its not easy to find out which patch should I realy use for vanilla 2.6.18, some are outdated and I want to avoid to do something wrong. Harald
Joel Robison
2007-Jul-09 11:14 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Bernd, I have downloaded the file you suggested for 2.6.20+ and I am at a stand still with the following issues: 1. After patching the 2.6.20 vanilla kernel and configuring lustre: ./ configure --disable-liblustre --disable-tests and then running I make I recieve: make[3]: Entering directory `/usr/src/linux-2.6.20'' CC [M] /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.o /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.c:168: error: static declaration of ???top_table??? follows non-static declaration /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.c:65: error: previous declaration of ???top_table??? was here make[6]: *** [/root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux- proc.o] Error 1 make[5]: *** [/root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs] Error 2 make[4]: *** [/root/lustre-1.6.0.1-rc1-ql6/lnet] Error 2 make[3]: *** [_module_/root/lustre-1.6.0.1-rc1-ql6] Error 2 make[3]: Leaving directory `/usr/src/linux-2.6.20'' make[2]: *** [modules] Error 2 make[2]: Leaving directory `/root/lustre-1.6.0.1-rc1-ql6'' make[1]: *** [all-recursive] Error 1 make[1]: Leaving directory `/root/lustre-1.6.0.1-rc1-ql6'' make: *** [all] Error 2make[3]: Entering directory `/usr/src/ linux-2.6.20'' I tried patching against 2.6.22 but I received some errors in the quilt push -av process which I will post later. Additionally the configure process complains about not having a definition for 2.6.22 in "lustre/autoconf/lustre-core.m4" Am I doing something wrong? -Joel Robison On Jul 6, 2007, at 8:01 AM, Harald van Pee wrote:> On Friday 06 July 2007 10:55 am, Bernd Schubert wrote: >> On Thursday 05 July 2007 23:42:38 Harald van Pee wrote: >>> On Thursday 05 July 2007 11:23 pm, Bernd Schubert wrote: >>>> Joel Robison wrote: >>>>> Thanks for the quick reply Bernd, >>>>> >>>>> I have not put any valuable data on the filesystem yet, so no >>>>> worries >>>>> about data loss, just 1.2T of zeros. >>>>> How can I disable extents and mballoc? is it a ../configure >>>>> option? >>>>> I am away from the machines right now or I would check myself. >>>> >>>> On the OSSs >>>> >>>> 1.) umount {ost-mountpoint} >>>> 2.) tunefs.lustre --mountfsoptions="" /dev/{ost-device} >>>> 3.) mount {ost-mountpoint} >>> >>> would there a big performance loss and if so does the CFS team >>> suggests a >>> patch instead? >> >> Some numbers you might find here: >> http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/ >> performance/ > > thanks for your reply the performance loss is not small, but > can I be sure that than do not run into bmalloc related file system > crashes? > Just to be sure because in the last discussions there are a lot of > informations and I''m not sure what I have to do next. > >> >> Patches are in bug#11039 >> >> >> Cheers, >> Bernd > > I have looked into this but there for me its not easy to find out > which patch > should I realy use for vanilla 2.6.18, > some are outdated and I want to avoid to do something wrong. > > Harald > > _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss-------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20070709/0acf4e98/attachment.html
Bernd Schubert
2007-Jul-09 13:40 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Hi Joel, On Monday 09 July 2007 19:14:40 Joel Robison wrote:> Bernd, > > I have downloaded the file you suggested for 2.6.20+ and I am at a > stand still with the following issues: > > 1. After patching the 2.6.20 vanilla kernel and configuring lustre: ./ > configure --disable-liblustre --disable-tests and then running I make > I recieve: > make[3]: Entering directory `/usr/src/linux-2.6.20'' > CC [M] /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.o > /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.c:168: > error: static declaration of ???top_table??? follows non-static > declaration > /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.c:65: > error: previous declaration of ???top_table??? was hereinteresting with gcc-3.4 this works, whats your compiler version? I was from the very beginning a bit sceptical if its legal first to declare a variable as extern and then again static. Anyway you can safely remove the entire line 65, it was only required for a previous version of the 2.6.21-sysctl.patch. Its good your compiler complained about it, so I can remove this useless forgotten line :)> I tried patching against 2.6.22 but I received some errors in the > quilt push -av process which I will post later. Additionally the > configure process complains about not having a definition for 2.6.22 > in "lustre/autoconf/lustre-core.m4"Well, 2.6.22 was only released today and I only did run a very very basic test if it works at all. If you want to try it, you will need mercurial and need to run "hg qpush -af" before running configure. The 2.6.22 patches so far do not have any version tests, so applying them and using a kernel <2.6.22 won''t work, thus I didn''t apply them to -rc1. The configure script also still doesn''t know about 2.6.22, so you need to compile with "make LDISKFS_SERIES=2.6.21-vanilla.series ...". Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH
/* Chris */
2007-Jul-10 22:53 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Hi, Bernd,
I did all the instructions as
http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/1.6/Readme.install
described.
Everything is fine, but when I created lustre FS on MDS, I got the error
messages:
mkfs.lustre --fsname=spfs --mdt --mgs /dev/hdb
Permanent disk data:
Target: spfs-MDTffff
Index: unassigned
Lustre FS: spfs
Mount type: ldiskfs
Flags: 0x75
(MDT MGS needs_index first_time update )
Persistent mount opts: errors=remount-ro,iopen_nopriv,user_xattr
Parameters:
checking for existing Lustre data: not found
device size = 10240MB
formatting backing filesystem ldiskfs on /dev/hdb
target name spfs-MDTffff
4k blocks 0
options -J size=400 -i 4096 -I 512 -q -O dir_index -F
mkfs_cmd = mkfs.ext2 -j -b 4096 -L spfs-MDTffff -J size=400 -i 4096 -I
512 -q -O dir_index -F /dev/hdb
mkfs.lustre: Unable to mount /dev/hdb: No such device
Is the ldiskfs module available?
mkfs.lustre FATAL: failed to write local files
mkfs.lustre: exiting with 19 (No such device)
-------------------------------------------------
I am using a linux-source-2.6.20 which is patched by ubuntu, not from
www.kernel.org. Of course, I made some modifications on the lustre-patched
files(vfs_intend-2.6.20 & vfs_nointend-2.6.20) you released.
When I using modprobe to check the ldiskfs module, it says "FATAL : Module
ldiskfs not found." How should install the ldiskfs module?
Any responses will be appreciated.
Chris
----- Original Message -----
From: "Bernd Schubert" <bs@q-leap.de>
To: <lustre-discuss@clusterfs.com>
Sent: Tuesday, July 10, 2007 3:39 AM
Subject: Re: [Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
> Hi Joel,
>
> On Monday 09 July 2007 19:14:40 Joel Robison wrote:
>> Bernd,
>>
>> I have downloaded the file you suggested for 2.6.20+ and I am at a
>> stand still with the following issues:
>>
>> 1. After patching the 2.6.20 vanilla kernel and configuring lustre: ./
>> configure --disable-liblustre --disable-tests and then running I make
>> I recieve:
>> make[3]: Entering directory `/usr/src/linux-2.6.20''
>> CC [M] /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.o
>> /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.c:168:
>> error: static declaration of ???top_table??? follows non-static
>> declaration
>> /root/lustre-1.6.0.1-rc1-ql6/lnet/libcfs/linux/linux-proc.c:65:
>> error: previous declaration of ???top_table??? was here
>
> interesting with gcc-3.4 this works, whats your compiler version? I was
> from
> the very beginning a bit sceptical if its legal first to declare a
> variable
> as extern and then again static. Anyway you can safely remove the entire
> line
> 65, it was only required for a previous version of the
> 2.6.21-sysctl.patch.
> Its good your compiler complained about it, so I can remove this useless
> forgotten line :)
>
>> I tried patching against 2.6.22 but I received some errors in the
>> quilt push -av process which I will post later. Additionally the
>> configure process complains about not having a definition for 2.6.22
>> in "lustre/autoconf/lustre-core.m4"
>
> Well, 2.6.22 was only released today and I only did run a very very basic
> test
> if it works at all. If you want to try it, you will need mercurial and
> need
> to run "hg qpush -af" before running configure. The 2.6.22
patches so far
> do
> not have any version tests, so applying them and using a kernel <2.6.22
> won''t
> work, thus I didn''t apply them to -rc1.
> The configure script also still doesn''t know about 2.6.22, so you
need to
> compile with "make LDISKFS_SERIES=2.6.21-vanilla.series ...".
>
> Cheers,
> Bernd
>
> --
> Bernd Schubert
> Q-Leap Networks GmbH
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>
>
> __________ NOD32 2387 (20070710) Information __________
>
> This message was checked by NOD32 antivirus system.
> http://www.eset.com
>
>
Bernd Schubert
2007-Jul-11 02:51 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Hi Chris, On Wednesday 11 July 2007 06:53:20 you wrote:> Hi, Bernd, > > I did all the instructions as > http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/1.6/Readme.i >nstall described. > Everything is fine, but when I created lustre FS on MDS, I got the error > messages:[...]> > > I am using a linux-source-2.6.20 which is patched by ubuntu, not from > www.kernel.org. Of course, I made some modifications on the lustre-patched > files(vfs_intend-2.6.20 & vfs_nointend-2.6.20) you released. > > When I using modprobe to check the ldiskfs module, it says "FATAL : Module > ldiskfs not found." How should install the ldiskfs module? > Any responses will be appreciated.did you also run "make install"? So far you probably only compiled lustre, but didn''t install it. Just added this step. Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH
/* Chris */
2007-Jul-11 09:27 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Hi Bernd, I did also run "make install" after "make", and there is no any warnnings or error messages. The problem probably is the 2.6.20-kernel which is patched by Ubuntu Feisty. Maybe I should try a new, and clean 2.6.20-kernel from kernel.org. Anyway, thanks for your response! ----- Original Message ----- From: "Bernd Schubert" <bs@q-leap.de> To: "/* Chris */" <chris@nchc.org.tw> Sent: Wednesday, July 11, 2007 4:49 PM Subject: Re: [Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty> Hi Chris, > > On Wednesday 11 July 2007 06:53:20 you wrote: >> Hi, Bernd, >> >> I did all the instructions as >> http://www.pci.uni-heidelberg.de/tc/usr/bernd/downloads/lustre/1.6/Readme.i >>nstall described. >> Everything is fine, but when I created lustre FS on MDS, I got the error >> messages: > > [...] > >> >> >> I am using a linux-source-2.6.20 which is patched by ubuntu, not from >> www.kernel.org. Of course, I made some modifications on the >> lustre-patched >> files(vfs_intend-2.6.20 & vfs_nointend-2.6.20) you released. >> >> When I using modprobe to check the ldiskfs module, it says "FATAL : >> Module >> ldiskfs not found." How should install the ldiskfs module? >> Any responses will be appreciated. > > did you also run "make install"? So far you probably only compiled lustre, > but > didn''t install it. Just added this step. > > Cheers, > Bernd > > -- > Bernd Schubert > Q-Leap Networks GmbH > > __________ NOD32 2390 (20070710) Information __________ > > This message was checked by NOD32 antivirus system. > http://www.eset.com > >
Bernd Schubert
2007-Jul-11 09:47 UTC
[Lustre-discuss] Re: Re: Lustre 1.6.0.1 in Ubuntu Fiesty
Hi Chris, On Wednesday 11 July 2007 17:27:06 you wrote:> Hi Bernd, > > I did also run "make install" after "make", and there is no any warnnings > or error messages. > The problem probably is the 2.6.20-kernel which is patched by Ubuntu > Feisty. Maybe I should try a new, and clean 2.6.20-kernel from kernel.org. > Anyway, thanks for your response!generally it would be better to take vanilla kernels for our present releases, since there are not yet any sanity checks for backported features from newer releases. Anyway, can you check the modules have been installed into the proper dir in /lib/modules/...? Also, did you run "depmod -a"? If everything is fine, what does dmesg say? Cheers, Bernd -- Bernd Schubert Q-Leap Networks GmbH