Lustre devel - Oct 2009 - Lustre version number and rpm "Release"

I am in the process of creating our LLNL branch of Lustre which is based
on b1_8.  Our build system for our branch of 1.6.6 was an additional top
level directory of make files, scripts, and our own custom lustre.spec
file.  As part of moving to 1.8, I want to try to get rid of that upper
layer and make the upstream 1.8 code our top level directory.

In bug 16424 there was discussion about setting custom version strings.
 The recommended method seems to be to set the LUSTRE_VERS string.  I
gave this a try, but it doesn''t appear to be of much use to us as it
stands.

As far as I can tell, setting LUSTRE_VERS will change the "Build
Version:" string that you see on the console upon loading the lustre
modules, and not much else.

Mainly here at LLNL we just want to append our own local version number
extension to Lustre''s version number.  If the upstream version is
"1.8.1.50", we want our local build to have the version
"1.8.1.50-3chaos".  LUSTRE_VERS mostly covers this part.

But we also want to prefix the rpm Release string with the same number.

In other words, when we build rpms we get:

$ rpm -qpi lustre-1.8.1.50-2.6.18_70chaos_200910011803.x86_64.rpm
Name        : lustre                       Relocations: (not relocatable)
Version     : 1.8.1.50                          Vendor: (none)
Release     : 2.6.18_70chaos_200910011803   Build Date: Thu Oct  1
18:11:30 2009

And instead we want this:

$ rpm -qpi lustre-1.8.1.50-3chaos_2.6.18_70chaos_200910011803.x86_64.rpm
Name        : lustre                       Relocations: (not relocatable)
Version     : 1.8.1.50                          Vendor: (none)
Release     : 3chaos_2.6.18_70chaos_200910011803   Build Date: Thu Oct
1 18:11:30 2009

Is there current a method for doing that?

Chris

On Thu, 2009-10-01 at 19:14 -0700, Christopher J. Morrone
wrote:
> As far as I can tell, setting LUSTRE_VERS will change the "Build
> Version:" string that you see on the console upon loading the lustre
> modules, and not much else.
Right.
> Mainly here at LLNL we just want to append our own local version number
> extension to Lustre''s version number.  If the upstream version is
> "1.8.1.50", we want our local build to have the version
> "1.8.1.50-3chaos".  LUSTRE_VERS mostly covers this part.
For the version embedded in the lustre modules (only) yes?
> But we also want to prefix the rpm Release string with the same number.
> 
> In other words, when we build rpms we get:
> 
> $ rpm -qpi lustre-1.8.1.50-2.6.18_70chaos_200910011803.x86_64.rpm
> Name        : lustre                       Relocations: (not relocatable)
> Version     : 1.8.1.50                          Vendor: (none)
> Release     : 2.6.18_70chaos_200910011803   Build Date: Thu Oct  1
Then presumably the kernel you built against is 2.6.18_70chaos.  Is that
correct?
> And instead we want this:
> 
> $ rpm -qpi lustre-1.8.1.50-3chaos_2.6.18_70chaos_200910011803.x86_64.rpm
> Name        : lustre                       Relocations: (not relocatable)
> Version     : 1.8.1.50                          Vendor: (none)
> Release     : 3chaos_2.6.18_70chaos_200910011803   Build Date: Thu Oct
> 1 18:11:30 2009
> 
> Is there current a method for doing that?
Well, backtracking to see how we get to the value for Release that''s in
the lustre.spec, we start with lustre.spec.in:

%{!?release: %define release @RELEASE@}
...
Release: %{release}

and @RELEASE@ is a subst''d variable from configure which is generated
with:

RELEASE="`echo ${LINUXRELEASE} | tr ''-''
''_''`_`date +%Y%m%d%H%M`"

and $LINUXRELEASE is determined by grabbing the value of the UTS_RELEASE
macro from either of utsrelease.h or version.h in
$LINUX_TREE/include/linux/.

That''s how the *default* value is determined.  However!  Notice that in
the lustre.spec, we use:

%{!?release: %define release @RELEASE@}

That means you can define Release for your rpm build on the command line
with:

$ rpmbuild --define ''release whatever_you_want'' ...
lustre.spec

I made this change a while ago as a first step toward getting rid of the
lustre.spec.in and passing all external values via rpmbuild macros to
facilitate just this sort of thing.  I have not gotten back to getting
more of that task complete, unfortunately.  But it''s complete enough
currently for you to do what you want.

Cheers,
b.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-devel/attachments/20091002/52c3900e/attachment.bin

Brian J. Murrell wrote:
>> $ rpm -qpi lustre-1.8.1.50-2.6.18_70chaos_200910011803.x86_64.rpm
>> Name        : lustre                       Relocations: (not
relocatable)
>> Version     : 1.8.1.50                          Vendor: (none)
>> Release     : 2.6.18_70chaos_200910011803   Build Date: Thu Oct  1
> 
> Then presumably the kernel you built against is 2.6.18_70chaos.  Is that
> correct?
That is correct.
> %{!?release: %define release @RELEASE@}
> 
> That means you can define Release for your rpm build on the command line
> with:
> 
> $ rpmbuild --define ''release whatever_you_want'' ...
lustre.spec
Well, that is true, except that we just want to prefix the release, not
replace it.  If we replace it with our own definition, we have to repeat
the kernel detection, and timestamp part.  So really, we''ll want an
additional "release_prefix" rpm variable.

There is an additional wrinkle that I didn''t mention.  Our automated
build system is designed to take source rpms and build the binary rpms
from that.  If we just set "release" at source rpm build time, I
don''t
think that it will be remember later when the binary rpms are built.

Part of the reason that our build farm works that way is that we would
like other users of our linux distribution to be able to easily rebuild
lustre (and any other kernel modules) from the source rpm.

In the end though, it looks like the most reasonable way to handle this
is to have our own spec file for the chaos linux distribution.

Chris

On Fri, 2009-10-02 at 17:36 -0700, Christopher J. Morrone
wrote:
> 
> Well, that is true, except that we just want to prefix the release, not
> replace it.  If we replace it with our own definition, we have to repeat
> the kernel detection, and timestamp part.
Yes, that''s true.
> So really, we''ll want an
> additional "release_prefix" rpm variable.
That would be a trivial patch.  Feel free to open a bug and submit it.
I can''t see any reason why we''d not land that.
> There is an additional wrinkle that I didn''t mention.  Our
automated
> build system is designed to take source rpms and build the binary rpms
> from that.  If we just set "release" at source rpm build time, I
don''t
> think that it will be remember later when the binary rpms are built.
Yes, I think you are correct about that.

FWIW, (I think[1]) I''d like us (Sun) to get out of the business of
putting anything meaningful in the Release tag and leave that for
distributors to use.  I''m not sure if or how that might help you
though.

b.

[1] I''ve not really investigated the possibility yet, so it''s
really
just a seed of an idea.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part
Url :
http://lists.lustre.org/pipermail/lustre-devel/attachments/20091005/a8039ca2/attachment.bin

Just curious, why do people build their own releases if everything is
available in release RPMs. Do most people or organizations build their
own version of Lustre or do they use the RPMs?

I am asking because I would like to build my own version and wanted to
see the benefits of it.

TIA

2009/10/5 Brian J. Murrell <Brian.Murrell at
sun.com>:
> On Fri, 2009-10-02 at 17:36 -0700, Christopher J. Morrone wrote:
>>
>> Well, that is true, except that we just want to prefix the release, not
>> replace it. ?If we replace it with our own definition, we have to
repeat
>> the kernel detection, and timestamp part.
>
> Yes, that''s true.
>
>> So really, we''ll want an
>> additional "release_prefix" rpm variable.
>
> That would be a trivial patch. ?Feel free to open a bug and submit it.
> I can''t see any reason why we''d not land that.
>
>> There is an additional wrinkle that I didn''t mention. ?Our
automated
>> build system is designed to take source rpms and build the binary rpms
>> from that. ?If we just set "release" at source rpm build
time, I don''t
>> think that it will be remember later when the binary rpms are built.
>
> Yes, I think you are correct about that.
>
> FWIW, (I think[1]) I''d like us (Sun) to get out of the business of
> putting anything meaningful in the Release tag and leave that for
> distributors to use. ?I''m not sure if or how that might help you
though.
>
> b.
>
> [1] I''ve not really investigated the possibility yet, so
it''s really
> just a seed of an idea.
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>
>

Mag Gam wrote:
> Just curious, why do people build their own releases if everything is
> available in release RPMs. Do most people or organizations build their
> own version of Lustre or do they use the RPMs?
>
We need to keep up to date with the latest kernel security errata - so 
I''ve been building Lustre clients.
> I am asking because I would like to build my own version and wanted to
> see the benefits of it.
Knowing that you can, even if you don''t currently need to, is an
advantage.
> TIA
> 
> 2009/10/5 Brian J. Murrell <Brian.Murrell at sun.com>:
>> On Fri, 2009-10-02 at 17:36 -0700, Christopher J. Morrone wrote:
>>> Well, that is true, except that we just want to prefix the release,
not
>>> replace it.  If we replace it with our own definition, we have to
repeat
>>> the kernel detection, and timestamp part.
>> Yes, that''s true.
>>
>>> So really, we''ll want an
>>> additional "release_prefix" rpm variable.
>> That would be a trivial patch.  Feel free to open a bug and submit it.
>> I can''t see any reason why we''d not land that.
>>
>>> There is an additional wrinkle that I didn''t mention.  Our
automated
>>> build system is designed to take source rpms and build the binary
rpms
>>> from that.  If we just set "release" at source rpm build
time, I don''t
>>> think that it will be remember later when the binary rpms are
built.
>> Yes, I think you are correct about that.
>>
>> FWIW, (I think[1]) I''d like us (Sun) to get out of the
business of
>> putting anything meaningful in the Release tag and leave that for
>> distributors to use.  I''m not sure if or how that might help
you though.
>>
>> b.
>>
>> [1] I''ve not really investigated the possibility yet, so
it''s really
>> just a seed of an idea.
>>
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>>
>>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel