Lustre devel - Feb 2012 - Lustre 1.8.7 - Setup prototype in Research field - STUCK !

Hello Everyone,
 
being the local crafty busy admin for a neuroscience research branch, Lustre
seems the only way to go however I''m a bit stuck and need some
thoughtful guidance.
 
My goal  is to setup a virtual OS environment which is a replica of our 
Direct attached storage head node running SLES 11.0 x86 64   Kernel: 
2.6.27.19-5 default #1 SMP
and our (2) Dell blade clusters running CentOS 5.3 x86 64   Kernel:
2.6.18-128.el5 #1 SMP
which I now have running as a) SLES 11 same kernel MDS  b) SLES 11 same kernel
OSS   and   c) CentOS 5.3 x86 65 same kernel
and then get Lustre running across it.
 
The trouble began when i was informed that the Lustre rpm kernel numbers MUST
match the OS kernel number EXACTLY due to modprobe errors and mount errors on
the client,
and some known messages on the servers after the rpm installs.
 
My only direct access to Oracle Lustre downloads is through another person with
an Oracle ID who''s not very willing to help - i.e. this route is
painful
 
So to explain why I''m stuck:
 
a) access to oracle downloads is not easy
b) there is so much risk with altering kernels, given all the applications and
stability of the environment you could literally trash the server and spend days
recovering - in addition to it being the main storage / resource for research
c) I can''t seem to find after looking Lustre RPMs that match my kernel
environment specifically, i.e. the SLES 11 AND CENTOS 5.3
d) I''ve never created rpms to a specific kernel version and that would
be a deep dive into new territory and frankly another gamble
 
What''s the least painful and least risky to get Lustre working in this
prototype which will then lend to production (equally least painful) given these
statements - Help !
Cliff, I could use some details on how specifically wamcloud can fit this
scenero - and thanks for all the enlightenment.
 
 
thanks for your help
Charles 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20120203/abbe092a/attachment.html

You should download from the Whamcloud download site, for a start:
http://downloads.whamcloud.com/public/lustre/
Typically, the Lustre server does nothing but run Lustre. For that reason
there is generally little risk
from using our current version on the server platforms. If your clients
require a particular kernel version, you
would have to build lustre-client and lustre-client modules only.

I would reccomend
http://wiki.whamcloud.com/display/PUB/Getting+started+with+Lustre
as a useful resource.
cliffw


On Fri, Feb 3, 2012 at 2:30 PM, Charles Cummings <Ccummings at
harthosp.org>wrote:
>  Hello Everyone,
>
> being the local crafty busy admin for a neuroscience research branch,
> Lustre seems the only way to go however I''m a bit stuck and need
some
> thoughtful guidance.
>
> My goal  is to setup a virtual OS environment which is a replica of our
> Direct attached storage head node running SLES 11.0 x86 64   Kernel:
> 2.6.27.19-5 default #1 SMP
> and our (2) Dell blade clusters running CentOS 5.3 x86 64   Kernel:
> 2.6.18-128.el5 #1 SMP
> which I now have running as a) SLES 11 same kernel MDS  b) SLES 11 same
> kernel OSS   and   c) CentOS 5.3 x86 65 same kernel
> and then get Lustre running across it.
>
> The trouble began when i was informed that the Lustre rpm kernel numbers
> MUST match the OS kernel number EXACTLY due to modprobe errors and mount
> errors on the client,
> and some known messages on the servers after the rpm installs.
>
> My only direct access to Oracle Lustre downloads is through another person
> with an Oracle ID who''s not very willing to help - i.e. this route
is
> painful
>
> So to explain why I''m stuck:
>
> a) access to oracle downloads is not easy
> b) there is so much risk with altering kernels, given all the applications
> and stability of the environment you could literally trash the server and
> spend days recovering - in addition to it being the main storage / resource
> for research
> c) I can''t seem to find after looking Lustre RPMs that match my
kernel
> environment specifically, i.e. the SLES 11 AND CENTOS 5.3
> d) I''ve never created rpms to a specific kernel version and that
would be
> a deep dive into new territory and frankly another gamble
>
> What''s the least painful and least risky to get Lustre working in
this
> prototype which will then lend to production (equally least painful) given
> these statements - Help !
> Cliff, I could use some details on how specifically wamcloud can fit this
> scenero - and thanks for all the enlightenment.
>
>
> thanks for your help
> Charles
>


-- 
cliffw
Support Guy
WhamCloud, Inc.
www.whamcloud.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20120203/34c5a337/attachment-0001.html

On 2012-02-03, at 3:30 PM, Charles Cummings wrote:
> being the local crafty busy admin for a neuroscience research branch,
Lustre seems the only way to go however I''m a bit stuck and need some
thoughtful guidance.
>  
> My goal  is to setup a virtual OS environment which is a replica of our
> Direct attached storage head node running SLES 11.0 x86 64   Kernel: 
2.6.27.19-5 default #1 SMP
> and our (2) Dell blade clusters running CentOS 5.3 x86 64   Kernel:
2.6.18-128.el5 #1 SMP
> which I now have running as a) SLES 11 same kernel MDS  b) SLES 11 same
kernel OSS   and   c) CentOS 5.3 x86 65 same kernel
> and then get Lustre running across it.
>  
> The trouble began when i was informed that the Lustre rpm kernel numbers
MUST match the OS kernel number EXACTLY due to modprobe errors and mount errors
on the client, and some known messages on the servers after the rpm installs.
>  
> My only direct access to Oracle Lustre downloads is through another person
with an Oracle ID who''s not very willing to help - i.e. this route is
painful
You can download the community Lustre releases at:
http://downloads.whamcloud.com/public/lustre/

Even so, they may not match your kernel versions exactly.
> So to explain why I''m stuck:
>  
> a) access to oracle downloads is not easy
> b) there is so much risk with altering kernels, given all the applications
and stability of the environment you could literally trash the server and spend
days recovering - in addition to it being the main storage / resource for
research
> c) I can''t seem to find after looking Lustre RPMs that match my
kernel environment specifically, i.e. the SLES 11 AND CENTOS 5.3
> d) I''ve never created rpms to a specific kernel version and that
would be a deep dive into new territory and frankly another gamble
There is a good walk-through for building your own Lustre client RPMs at:
http://wiki.whamcloud.com/display/PUB/Building+Lustre+from+Source

You don''t need to patch the kernel to run Lustre on the client.
> What''s the least painful and least risky to get Lustre working in
this prototype which will then lend to production (equally least painful) given
these statements - Help !
> Cliff, I could use some details on how specifically wamcloud can fit this
scenero - and thanks for all the enlightenment.
>  
>  
> thanks for your help
> Charles 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Engineer                   http://www.whamcloud.com/

You can get the Oracle downloads from:
http://downloads.lustre.org/public/lustre/v1.8/lustre_1.8.7/

Basically, build Lustre for your kernel on the clients, but use the Lustre
server kernel on the servers.

Kevin


On Feb 7, 2012, at 9:09 AM, "Charles Cummings" <Ccummings at
harthosp.org> wrote:
> Hello Everyone,
>  
> being the local crafty busy admin for a neuroscience research branch,
Lustre seems the only way to go however I''m a bit stuck and need some
thoughtful guidance.
>  
> My goal  is to setup a virtual OS environment which is a replica of our
> Direct attached storage head node running SLES 11.0 x86 64   Kernel: 
2.6.27.19-5 default #1 SMP
> and our (2) Dell blade clusters running CentOS 5.3 x86 64   Kernel:
2.6.18-128.el5 #1 SMP
> which I now have running as a) SLES 11 same kernel MDS  b) SLES 11 same
kernel OSS   and   c) CentOS 5.3 x86 65 same kernel
> and then get Lustre running across it.
>  
> The trouble began when i was informed that the Lustre rpm kernel numbers
MUST match the OS kernel number EXACTLY due to modprobe errors and mount errors
on the client,
> and some known messages on the servers after the rpm installs.
>  
> My only direct access to Oracle Lustre downloads is through another person
with an Oracle ID who''s not very willing to help - i.e. this route is
painful
>  
> So to explain why I''m stuck:
>  
> a) access to oracle downloads is not easy
> b) there is so much risk with altering kernels, given all the applications
and stability of the environment you could literally trash the server and spend
days recovering - in addition to it being the main storage / resource for
research
> c) I can''t seem to find after looking Lustre RPMs that match my
kernel environment specifically, i.e. the SLES 11 AND CENTOS 5.3
> d) I''ve never created rpms to a specific kernel version and that
would be a deep dive into new territory and frankly another gamble
>  
> What''s the least painful and least risky to get Lustre working in
this prototype which will then lend to production (equally least painful) given
these statements - Help !
> Cliff, I could use some details on how specifically wamcloud can fit this
scenero - and thanks for all the enlightenment.
>  
>  
> thanks for your help
> Charles 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
Confidentiality Notice: This e-mail message, its contents and any attachments to
it are confidential to the intended recipient, and may contain information that
is privileged and/or exempt from disclosure under applicable law. If you are not
the intended recipient, please immediately notify the sender and destroy the
original e-mail message and any attachments (and any copies that may have been
made) from your system or otherwise. Any unauthorized use, copying, disclosure
or distribution of this information is strictly prohibited.  Email addresses
that end with a ?-c? identify the sender as a Fusion-io contractor.

as far as I can see client rebuilding is not working via rpmbuild.

there appears to be no SRPMS directory in the 1.8.7-wc1 client tree and

rpmbuild --rebuild lustre-client-source-1.8.7-wc1_2.6.18_274.3.1.el5.x86_64.rpm
Installing lustre-client-source-1.8.7-wc1_2.6.18_274.3.1.el5.x86_64.rpm
error: source package expected, binary found
error: lustre-client-source-1.8.7-wc1_2.6.18_274.3.1.el5.x86_64.rpm cannot be
installed

suggests this is not the correct rpm - (looks like a source tree install binary
..??)

we are stuck - need to compile the1.8.7-wc1 client against our own kernel ...
any ideas welcome.
i think i once did this by compiling the lustre source on a tree without the
presence of a patched kernel, which defaults to client only build.
would this work??


Professor Samuel Aparicio BM BCh PhD FRCPath
Nan and Lorraine Robertson Chair UBC/BC Cancer Agency
675 West 10th, Vancouver V5Z 1L3, Canada.
office: +1 604 675 8200 lab website http://molonc.bccrc.ca







On Feb 7, 2012, at 4:30 AM, Andreas Dilger wrote:
> On 2012-02-03, at 3:30 PM, Charles Cummings wrote:
>> being the local crafty busy admin for a neuroscience research branch,
Lustre seems the only way to go however I''m a bit stuck and need some
thoughtful guidance.
>> 
>> My goal  is to setup a virtual OS environment which is a replica of our
>> Direct attached storage head node running SLES 11.0 x86 64   Kernel: 
2.6.27.19-5 default #1 SMP
>> and our (2) Dell blade clusters running CentOS 5.3 x86 64   Kernel:
2.6.18-128.el5 #1 SMP
>> which I now have running as a) SLES 11 same kernel MDS  b) SLES 11 same
kernel OSS   and   c) CentOS 5.3 x86 65 same kernel
>> and then get Lustre running across it.
>> 
>> The trouble began when i was informed that the Lustre rpm kernel
numbers MUST match the OS kernel number EXACTLY due to modprobe errors and mount
errors on the client, and some known messages on the servers after the rpm
installs.
>> 
>> My only direct access to Oracle Lustre downloads is through another
person with an Oracle ID who''s not very willing to help - i.e. this
route is painful
> 
> You can download the community Lustre releases at:
> http://downloads.whamcloud.com/public/lustre/
> 
> Even so, they may not match your kernel versions exactly.
> 
>> So to explain why I''m stuck:
>> 
>> a) access to oracle downloads is not easy
>> b) there is so much risk with altering kernels, given all the
applications and stability of the environment you could literally trash the
server and spend days recovering - in addition to it being the main storage /
resource for research
>> c) I can''t seem to find after looking Lustre RPMs that match
my kernel environment specifically, i.e. the SLES 11 AND CENTOS 5.3
>> d) I''ve never created rpms to a specific kernel version and
that would be a deep dive into new territory and frankly another gamble
> 
> There is a good walk-through for building your own Lustre client RPMs at:
> http://wiki.whamcloud.com/display/PUB/Building+Lustre+from+Source
> 
> You don''t need to patch the kernel to run Lustre on the client.
> 
>> What''s the least painful and least risky to get Lustre working
in this prototype which will then lend to production (equally least painful)
given these statements - Help !
>> Cliff, I could use some details on how specifically wamcloud can fit
this scenero - and thanks for all the enlightenment.
>> 
>> 
>> thanks for your help
>> Charles 
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
> 
> 
> Cheers, Andreas
> --
> Andreas Dilger                       Whamcloud, Inc.
> Principal Engineer                   http://www.whamcloud.com/
> 
> 
> 
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss