Lustre discuss - Aug 2009 - Lustre 1.8.1 distribution missing kernel source?

I downloaded Lustre 1.8.1 today, but didn''t see any kernel-source rpm.
There were a few new src.rpm packages whose equivalent wasn''t in
1.8.0.1.
Looking at the ChangeLog there''s no mention of changing the way
everything
is bundled. Where can I find the patched server kernel source?

-- 
===========================================================================   
,-_|\   Richard Smith Staff Engineer PAE
   /     \  Sun Microsystems                   Phone : +61 3 9869 6200
richard.smith at Sun.COM                         Direct : +61 3 9869 6224
   \_,-._/  476 St Kilda Road                    Fax : +61 3 9869 6290
        v   Melbourne Vic 3004 Australia
===========================================================================

On Wed, 2009-08-12 at 15:08 +1000, Richard Smith wrote:
> I downloaded Lustre 1.8.1 today, but didn''t see any kernel-source
rpm.
Which distro?
> There were a few new src.rpm packages whose equivalent wasn''t in
1.8.0.1.
Right.  Starting with 1.8.1, we are packaging our kernels just like the
vendor does, so that a RH user will get our packages and be immediately
familiar with them and what he has to do to use them because it''s
exactly the same as the stock packages.
> Looking at the ChangeLog there''s no mention of changing the way
everything
> is bundled. Where can I find the patched server kernel source?
Because you didn''t find a kernel-source, I am guessing you are using
the
RHEL5 packages.  RH doesn''t package a kernel-source package, only Suse
does.  RH requires that you get the source from the src.rpm.  There''s
tonnes of info on the web on how to build a kernel from source for RHEL.
Google will find it pretty quickly.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090812/131c6a30/attachment.bin

I think the issues is this is all that is listed:

Base Linux Kernel Source - unpatched (installable RPM)
 kernel-2.6.18-128.1.14.el5.src.rpm

Where is the version with the Lustre patches?

Kevin


Brian J. Murrell wrote:
> On Wed, 2009-08-12 at 15:08 +1000, Richard Smith wrote:
>   
>> I downloaded Lustre 1.8.1 today, but didn''t see any
kernel-source rpm.
>>     
>
> Which distro?
>
>   
>> There were a few new src.rpm packages whose equivalent wasn''t
in 1.8.0.1.
>>     
>
> Right.  Starting with 1.8.1, we are packaging our kernels just like the
> vendor does, so that a RH user will get our packages and be immediately
> familiar with them and what he has to do to use them because it''s
> exactly the same as the stock packages.
>
>   
>> Looking at the ChangeLog there''s no mention of changing the
way everything
>> is bundled. Where can I find the patched server kernel source?
>>     
>
> Because you didn''t find a kernel-source, I am guessing you are
using the
> RHEL5 packages.  RH doesn''t package a kernel-source package, only
Suse
> does.  RH requires that you get the source from the src.rpm. 
There''s
> tonnes of info on the web on how to build a kernel from source for RHEL.
> Google will find it pretty quickly.
>
> b.
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>

On Wed, 2009-08-12 at 12:01 -0600, Kevin Van Maren
wrote:
> I think the issues is this is all that is listed:
> 
> Base Linux Kernel Source - unpatched (installable RPM)
>  kernel-2.6.18-128.1.14.el5.src.rpm
> 
> Where is the version with the Lustre patches?
The Lustre patch is in the RPM along with all of RH''s own patches and
will be applied when one turns the src.rpm into a source tree with RH''s
usual process for this (i.e. unpacking the RPM and running the prep) --
just like one has to do with a stock RH kernel src.rpm.

As for the description "unpatched (installable RPM)
kernel-2.6.18-128.1.14.el5.src.rpm" I''m not really sure if that is
accurate or not.  Technically it''s not patched but the patches are in
the src.rpm.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090812/a9417ade/attachment.bin

Maybe (if so that is extremely confusing), but under Platform "Source"
(rather than RedHat/x86_64) there is this package:

     Lustre-patched kernel source (installable RPM)
 kernel-lustre-2.6.18-128.1.14.el5_lustre.1.8.1.src.rpm

which is a bit larger is size.

A bit confusing because there is some source code under the normal 
platform selection, but some is under the Source selection.

Kevin


Brian J. Murrell wrote:
> On Wed, 2009-08-12 at 12:01 -0600, Kevin Van Maren wrote:
>   
>> I think the issues is this is all that is listed:
>>
>> Base Linux Kernel Source - unpatched (installable RPM)
>>  kernel-2.6.18-128.1.14.el5.src.rpm
>>
>> Where is the version with the Lustre patches?
>>     
>
> The Lustre patch is in the RPM along with all of RH''s own patches
and
> will be applied when one turns the src.rpm into a source tree with
RH''s
> usual process for this (i.e. unpacking the RPM and running the prep) --
> just like one has to do with a stock RH kernel src.rpm.
>
> As for the description "unpatched (installable RPM)
> kernel-2.6.18-128.1.14.el5.src.rpm" I''m not really sure if
that is
> accurate or not.  Technically it''s not patched but the patches are
in
> the src.rpm.
>
> b.
>
>   
> ------------------------------------------------------------------------
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>

Brian J. Murrell wrote:
> Because you didn''t find a kernel-source, I am guessing you are
using the
> RHEL5 packages. RH doesn''t package a kernel-source package, only
Suse
> does. RH requires that you get the source from the src.rpm.
There''s
> tonnes of info on the web on how to build a kernel from source for RHEL.
> Google will find it pretty quickly.
Yes, I was using RHEL5 x86_64 packages. FWIW I did go and compile the
kernel after installing the src.rpm, via rpmbuild -bb kernel-2.6.spec.
This appeared to apply a lot of patches, but didn''t create any rpms
with lustre in their name, so presumably there''s an extra step required
somewhere along the way.

I stopped pursuing this when I realised for the purposes of compiling
drivers, what I really needed was to install
kernel-lustre-devel-2.6.18-128.1.14.el5_lustre.1.8.1.x86_64.rpm

I also had to resolve a header file conflict to get myri10ge driver to
build, for which I hacked its check_headers.sh script to inspect
include/linux/inet_lro.h and set MYRI10GE_HAVE_WSUM appropriately.

-- 
===========================================================================   
,-_|\   Richard Smith Staff Engineer PAE
   /     \  Sun Microsystems                   Phone : +61 3 9869 6200
richard.smith at Sun.COM                         Direct : +61 3 9869 6224
   \_,-._/  476 St Kilda Road                    Fax : +61 3 9869 6290
        v   Melbourne Vic 3004 Australia
===========================================================================

On Mon, 2009-08-17 at 17:31 +1000, Richard Smith wrote:
> 
> Yes, I was using RHEL5 x86_64 packages. FWIW I did go and compile the
> kernel after installing the src.rpm, via rpmbuild -bb kernel-2.6.spec.
Yes, IIRC that is one of the possible paths that RH describe.  If you
just wanted patched source which you could then further patch/tweak
before you executed the build, you could just use rpmbuild -bp, which I
believe is one of the RH recommended paths.
> This appeared to apply a lot of patches, but didn''t create any
rpms
> with lustre in their name, so presumably there''s an extra step
required
> somewhere along the way.
Nope.  rpmbuild -bb should have built the lustre kernel RPMs.  What did
get created?
> I stopped pursuing this when I realised for the purposes of compiling
> drivers, what I really needed was to install
> kernel-lustre-devel-2.6.18-128.1.14.el5_lustre.1.8.1.x86_64.rpm
Indeed.  Compiling drivers is exactly what the kernel-*devel packages
are for.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090817/bcf3e0d0/attachment.bin

On Aug 17, 2009  09:44 -0400, Brian J. Murrell wrote:
> On Mon, 2009-08-17 at 17:31 +1000, Richard Smith wrote:
> > Yes, I was using RHEL5 x86_64 packages. FWIW I did go and compile the
> > kernel after installing the src.rpm, via rpmbuild -bb kernel-2.6.spec.
> 
> Yes, IIRC that is one of the possible paths that RH describe.  If you
> just wanted patched source which you could then further patch/tweak
> before you executed the build, you could just use rpmbuild -bp, which I
> believe is one of the RH recommended paths.
This will build the kernel RPMs.
> > This appeared to apply a lot of patches, but didn''t create
any rpms
> > with lustre in their name, so presumably there''s an extra
step required
> > somewhere along the way.
> 
> Nope.  rpmbuild -bb should have built the lustre kernel RPMs.  What did
> get created?
I would be surprised if building the kernel also built the Lustre RPMs
(which is what I think is the issue here).  To rebuild the Lustre RPMs
for the new kernel you need to rpmbuild -bp the lustre-{version}.src.rpm.


Cheers, Andreas
--
Andreas Dilger
Sr. Staff Engineer, Lustre Group
Sun Microsystems of Canada, Inc.

On Mon, 2009-08-17 at 12:18 -0600, Andreas Dilger wrote:
> > 
> > Yes, IIRC that is one of the possible paths that RH describe.  If you
> > just wanted patched source which you could then further patch/tweak
> > before you executed the build, you could just use rpmbuild -bp, which
I
> > believe is one of the RH recommended paths.
> 
> This will build the kernel RPMs.
rpmbuild -bb will build RPMs.  rpmbuild -bp will just "%prep" the
package.  IOW, it stops after %prep and before %build.
> 
> > > This appeared to apply a lot of patches, but didn''t
create any rpms
> > > with lustre in their name, so presumably there''s an
extra step required
> > > somewhere along the way.
> > 
> > Nope.  rpmbuild -bb should have built the lustre kernel RPMs.  What
did
> > get created?
> 
> I would be surprised if building the kernel also built the Lustre RPMs
> (which is what I think is the issue here).
Right.  rpmbuild -bb on the kernel src.rpm will not build the Lustre
RPMs, but he said he found no RPMs at all with lustre in the name, which
is why I was interested in seeing what he did get.  We don''t remove the
"-lustre" from the kernel packages until an upcoming (1.8.2 probably)
release and the 1.8.1 kernel src.rpm should have built kernel-lustre-*
RPMs.
> To rebuild the Lustre RPMs
> for the new kernel you need to rpmbuild -bp the lustre-{version}.src.rpm.
                                          ^^^
-bb.  -bp only does %prep.

b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090817/6a9cbb0b/attachment.bin

Brian J. Murrell wrote:
> Nope.  rpmbuild -bb should have built the lustre kernel RPMs.  What did
> get created?
Here''s the list of artifacts that was created in 
/usr/src/redhat/RPMS/x86_64:

kernel-2.6.18-128.1.14.el5.x86_64.rpm
kernel-debug-2.6.18-128.1.14.el5.x86_64.rpm
kernel-debug-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
kernel-debug-devel-2.6.18-128.1.14.el5.x86_64.rpm
kernel-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
kernel-debuginfo-common-2.6.18-128.1.14.el5.x86_64.rpm
kernel-devel-2.6.18-128.1.14.el5.x86_64.rpm
kernel-headers-2.6.18-128.1.14.el5.x86_64.rpm
kernel-xen-2.6.18-128.1.14.el5.x86_64.rpm
kernel-xen-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
kernel-xen-devel-2.6.18-128.1.14.el5.x86_64.rpm

I was concerned when I didn''t see something named
kernel-lustre-2.6.18-128.1.14.el5_lustre.1.8.1.x86_64.rpm

Checking the contents of both the original binary rpm, and the kernel rpm
I freshly compiled, they have different module directories:

/lib/modules/2.6.18-128.1.14.el5_lustre.1.8.1 vs
/lib/modules/2.6.18-128.1.14.el5

So may be I have to use an intermediate step and copy over the appropriate
config file from /boot. It left a question in my mind about whether the two
rpms were compiled from the same patched source, or whether there are 
additional
patches that should be applied as part of the process of creating 
kernel-lustre
rpm.

For the moment I''m ok, because with kernel-devel I''ve been
able to compile
the drivers I needed. I''m still interested in the steps required to
compile
a lustre-patched kernel though, in the pursuit of MB-sized i/o.

-- 
===========================================================================   
,-_|\   Richard Smith Staff Engineer PAE
   /     \  Sun Microsystems                   Phone : +61 3 9869 6200
richard.smith at Sun.COM                         Direct : +61 3 9869 6224
   \_,-._/  476 St Kilda Road                    Fax : +61 3 9869 6290
        v   Melbourne Vic 3004 Australia
===========================================================================

Yes, those look to be the _unpatched_ rhel RPMs.

Try under Platform "Source" (rather than RedHat/x86_64) this package:

    Lustre-patched kernel source (installable RPM)
kernel-lustre-2.6.18-128.1.14.el5_lustre.1.8.1.src.rpm

Kevin


Richard Smith wrote:
> Brian J. Murrell wrote:
>
>   
>> Nope.  rpmbuild -bb should have built the lustre kernel RPMs.  What did
>> get created?
>>     
>
> Here''s the list of artifacts that was created in 
> /usr/src/redhat/RPMS/x86_64:
>
> kernel-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debug-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debug-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debug-devel-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debuginfo-common-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-devel-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-headers-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-xen-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-xen-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-xen-devel-2.6.18-128.1.14.el5.x86_64.rpm
>
> I was concerned when I didn''t see something named
> kernel-lustre-2.6.18-128.1.14.el5_lustre.1.8.1.x86_64.rpm
>
> Checking the contents of both the original binary rpm, and the kernel rpm
> I freshly compiled, they have different module directories:
>
> /lib/modules/2.6.18-128.1.14.el5_lustre.1.8.1 vs
> /lib/modules/2.6.18-128.1.14.el5
>
> So may be I have to use an intermediate step and copy over the appropriate
> config file from /boot. It left a question in my mind about whether the two
> rpms were compiled from the same patched source, or whether there are 
> additional
> patches that should be applied as part of the process of creating 
> kernel-lustre
> rpm.
>
> For the moment I''m ok, because with kernel-devel I''ve
been able to compile
> the drivers I needed. I''m still interested in the steps required
to compile
> a lustre-patched kernel though, in the pursuit of MB-sized i/o.
>
>

On Tue, 2009-08-18 at 11:15 +1000, Richard Smith wrote:
> Brian J. Murrell wrote:
> 
> > Nope.  rpmbuild -bb should have built the lustre kernel RPMs.  What
did
> > get created?
> 
> Here''s the list of artifacts that was created in 
> /usr/src/redhat/RPMS/x86_64:
> 
> kernel-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debug-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debug-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debug-devel-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-debuginfo-common-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-devel-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-headers-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-xen-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-xen-debuginfo-2.6.18-128.1.14.el5.x86_64.rpm
> kernel-xen-devel-2.6.18-128.1.14.el5.x86_64.rpm
> 
> I was concerned when I didn''t see something named
> kernel-lustre-2.6.18-128.1.14.el5_lustre.1.8.1.x86_64.rpm
Indeed, that is what you should have gotten.

Can you send me a transcript of everything you do from the point of
having downloaded the .src.rpm from the Sun Download Centre?  Once you
have the .src.rpm local, type this:

$ script

It will report:

        Script started, file is typescript

and give you a shell prompt.  At that point, go through your whole
process of trying to rebuild the kernel (i.e. as you did to get the
above build artifacts).  When you are all done type:

$ exit

and it will report:

        Script done, file is typescript

Send me the file named "typescript".

Thanx
b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20090818/56ad700e/attachment-0009.bin

Brian J. Murrell wrote:
>> I was concerned when I didn''t see something named
>> kernel-lustre-2.6.18-128.1.14.el5_lustre.1.8.1.x86_64.rpm
>
> Indeed, that is what you should have gotten.
>
> Can you send me a transcript of everything you do from the point of
> having downloaded the .src.rpm from the Sun Download Centre?  Once you
> have the .src.rpm local, type this:
<snip>

This may not be worth pursuing any further. I was using the kernel src.rpm
that can be found underneath the RHEL5 x86_64 tab, which is named
kernel-2.6.18-128.1.14.el5.src.rpm

Kevin Van Maren pointed out that under the Sources tab there is another 
bundle,
kernel-lustre-2.6.18-128.1.14.el5_lustre.1.8.1.src.rpm, which is explicitly
labelled as "Lustre-patched kernel source (installable RPM)".

It does seem confusing to have the various artifacts distributed as they are
across two tabs. I''d prefer to see all artifacts for a particular
kernel
grouped together under the one tab. The Sources tab has quite a mixture of
files which you have to wade through to identify the few for RHEL5 x86_64.


-- 
===========================================================================   
,-_|\   Richard Smith Staff Engineer PAE
   /     \  Sun Microsystems                   Phone : +61 3 9869 6200
richard.smith at Sun.COM                         Direct : +61 3 9869 6224
   \_,-._/  476 St Kilda Road                    Fax : +61 3 9869 6290
        v   Melbourne Vic 3004 Australia
===========================================================================