Lustre discuss - May 2006 - Building user-space tools which don't depend on a kernel version.

On Mon, 2004-11-29 at 17:16 +0000, Simon Kelley wrote:
> I''m currently working on packaging Lustre as Debian packages.
Sounds really interesting. 

For which version(s) of Lustre? Anything you want to share and/or
something you want me to test. 

We are in the middle of deploying our own Lustre installation and as we
are running Debian on almost everything, I am really interested in your
work.

Hopefully we can persuade upstream to adopt some of your work :-)

/torkel

On Sat, 11 Dec 2004, Andreas Dilger wrote:
> None of our support customers use Debian, so we can''t really make
a huge
> effort in that direction.  If you have patches and they seem reasonable
> this might be considered.
Well, we have a support contract and are using Debian. We don''t want 
to "waste" support time fixing building-problems tho, but rather save 
it to fix more serius issues that probably will occur.

In general the user-space-building-issues affects all platforms, since 
it''s not feasible to have user-space tools that depends on 
kernel-stuff in the long run.

/Nikke
-- 
-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
  Niklas Edmundsson, Admin @ {acc,hpc2n}.umu.se      |     nikke@acc.umu.se
---------------------------------------------------------------------------
  *   <- Tribble     ®*¯ <- Sergeant tribble
=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=

--pAwQNkOnpTn9IO2O
Content-Type: text/plain; charset=us-ascii
Content-Disposition: inline
Content-Transfer-Encoding: quoted-printable

On Nov 29, 2004  17:16 +0000, Simon Kelley wrote:
> I would be useful to be able to build and package the lustre user-space=20
> tools without needing a kernel tree to build against and in a way which=20
> means that the same (binary) package is usable with all kernels.
When you say "all" kernels, what do you mean?  Kernels within e.g. 2.4
series or 2.6 also?  Do you intend the i386 user tools to work on
x86_64 too?  The former is probable, the latter isn''t really.
> So, my questions are:
>=20
> 1) Where does OBD_MAX_IOCTL_BUFFER come from, can I assume that will be=20
> 8192 always?
It defaults to 8192, but is configurable with a configure option.  Having
a larger buffer is not normally needed but for some configurations (e.g.
more than 200 OSTs) it might be needed again.
> 2) Can I get around the need to do a top-level configure in order to=20
> create utils/Makefile and portals/utils/Makefile
It may or may not be possible to put the Makefile for e.g. the utils
directory into CVS (I defer to our build/configure experts on that),
or at least put the generated ones into our release tarballs to avoid
the constant changing of the gigantic autoconf-generated files in CVS.
> 3) Would the Lustre developers consider rearanging things to make this=20
> easier in the future? Making heavy patches to my source tree is not=20
> really tenable, since the Debian packaging work has to be transferable=20
> to subsequent CFS releases with the minimum effort.
None of our support customers use Debian, so we can''t really make a
huge
effort in that direction.  If you have patches and they seem reasonable
this might be considered.

Cheers, Andreas
--
Andreas Dilger
http://sourceforge.net/projects/ext2resize/
http://members.shaw.ca/adilger/             http://members.shaw.ca/golinux/


--pAwQNkOnpTn9IO2O
Content-Type: application/pgp-signature
Content-Disposition: inline

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQFBurX0pIg59Q01vtYRAtZzAJ9OuKv5quFAUjMdXJ/4KDYz4wiNOgCfXJ4W
pw1BnTeq6fkbyxQFiyHwW8k=fqWt
-----END PGP SIGNATURE-----

--pAwQNkOnpTn9IO2O--

I''m currently working on packaging Lustre as Debian packages.

I would be useful to be able to build and package the lustre user-space 
tools without needing a kernel tree to build against and in a way which 
means that the same (binary) package is usable with all kernels.

I think that the kernel-independence is already there, or almost there. 
Everything in utils and portals/utils compiles fine as long as the 
pre-processor symbol OBD_MAX_IOCTL_BUFFER is defined in 
include/config.h, nothing else from config.h is needed. The compile/link 
commands run by the makefiles don''t appear to rely on anything in the 
kernel which the system was configured against.

The major fly in the ointment is that to create the Makefiles for those 
two directories needs a top-level configure step. There doesn''t seem to
be an easy way around that (though I''m no automake/autoconf guru, so I 
might have missed something.)

So, my questions are:

1) Where does OBD_MAX_IOCTL_BUFFER come from, can I assume that will be 
8192 always?

2) Can I get around the need to do a top-level configure in order to 
create utils/Makefile and portals/utils/Makefile

3) Would the Lustre developers consider rearanging things to make this 
easier in the future? Making heavy patches to my source tree is not 
really tenable, since the Debian packaging work has to be transferable 
to subsequent CFS releases with the minimum effort.


Cheers,

Simon.

Andreas Dilger wrote:
> On Nov 29, 2004  17:16 +0000, Simon Kelley wrote:
> 
>>I would be useful to be able to build and package the lustre user-space 
>>tools without needing a kernel tree to build against and in a way which 
>>means that the same (binary) package is usable with all kernels.
> 
> 
> When you say "all" kernels, what do you mean?  Kernels within
e.g. 2.4
> series or 2.6 also?  Do you intend the i386 user tools to work on
> x86_64 too?  The former is probable, the latter isn''t really.
> 
Cross-architecture compatibility is not required.

Kernel independence is a continuum between that achieved by
''ls'' (works
on any kernel that supports ELF binaries) and ''modutils''
(different
versions for 2.4.x and 2.6.x) I guess in an ideal world it would be 
possible to build the user-space components of Lustre against the kernel 
headers in /usr/include/linux and have them work on any kernel.

The main  obstacle to this at the moment is the the autoconf/automake 
process, which assumes access to a kernel tree. I could disentangle 
that, but my guess is that I''d create a huge and unwieldy diff from the
CFS sources which would be a nightmare to keep current. Hence the post 
to see if CFS are interested in doing this upstream.
>>3) Would the Lustre developers consider rearanging things to make this 
>>easier in the future? Making heavy patches to my source tree is not 
>>really tenable, since the Debian packaging work has to be transferable 
>>to subsequent CFS releases with the minimum effort.
> 
> 
> None of our support customers use Debian, so we can''t really make
a huge
> effort in that direction.  If you have patches and they seem reasonable
> this might be considered.
Debian aside, this would be a general win for maintainability and 
usability. All the rpms I''ve seen for Lustre suffer from the same 
problems of absolute dependency  on a kernel version and need to upgrade 
all the components in lockstep.


As a step towards the ultimate goal, I''ve currently implemented a
scheme
in which the source package includes a kernel tarball with the Lustre 
patches applied. In general there would one of these per supported 
kernel version. The included kernel is used both to as a target to 
configure against, and to generate a patch file against the standard 
Debian kernel. (Debian kernel-patch packaging techniques don''t appear
to
be amenable to using quilt, as far as I can see.)



Cheers,

Simon.

On Sat, 2004-12-11 at 01:55 -0700, Andreas Dilger wrote:
> None of our support customers use Debian, so we can''t really make
a huge
> effort in that direction.  If you have patches and they seem reasonable
> this might be considered.
I''m a big debian user, and all my development work for lustre and
Active
Storage has been done on debian boxes.

Evan