Lustre devel - Oct 2011 - Moving ldiskfs external to the Lustre tree

We would like to see the ldiskfs tree removed from the lustre tree and 
made an independent package.  I have been floating this idea 
unofficially for a while, but I would like to now officially propose 
this for Lustre 2.2.

With the OBD changes that are taking place on the Orion branch, we want 
to make lustre be able to use any of a number of backend filesystems. 
The current tree and configure tools make it very hard to build without 
ldiskfs (instead using zfs, btrfs, or something we haven''t thought of 
yet).  As part of cleaning that up, moving ldiskfs external to lustre 
will help ensure that we don''t have unnecessary dependencies crop up in
the future.

We have created an external package of the ldiskfs tree and put it up on 
github:

   https://github.com/chaos/ldiskfs

To use the new external ldiskfs package with lustre, you will also need 
a few patches to lustre itself.  There are links to the patches in this 
jira ticket:

   http://jira.whamcloud.com/browse/LU-723

On a RHEL system, ldiskfs has a build dependency on the kernel-debuginfo 
packages by default.  That is where we find the ext4 source code.

Building should be fairly straight forward.  Build and install ldiskfs 
(in particular the lustre-ldiskfs-devel package), and then build lustre. 
  The main new change to lustre is the addition of the configure option 
"--with-ldiskfs-devel".  On a RHEL system if you have the 
lustre-ldiskfs-devel package installed, you won''t need to give a path.

Note that we have not yet removed the in-tree ldiskfs.  Our first step 
was to get this working.  Once this is accepted, we will be happy to 
submit the patches to remove lustre''s copy of ldiskfs and generally 
clean up lustre''s autoconf checks involving ldiskfs.

We attempted to keep the changes minimal, so we didn''t change the name 
of the ldiskfs rpm packages.  But we think is would be nice to change 
the name from "lustre-ldiskfs" to simply "ldiskfs".  If we
want to make
that change, now is the time to do it.

Chris

Chris,
I''m not against this in principle, but I think it may be more difficult
to actually implement before the OSD changes from Orion are landed to master. 
In the past we also thought about making ldiskfs as a separate package, before
ext4 started in the kernel.

I''ve seen Prakash working on those patches but will not have time to
look at them until at least next week.

Are those patches against the master branch or against Orion?  Also, what
kernels are supported?

Cheers, Andreas

On 2011-10-12, at 3:44 PM, "Christopher J. Morrone" <morrone2 at
llnl.gov> wrote:
> We would like to see the ldiskfs tree removed from the lustre tree and 
> made an independent package.  I have been floating this idea 
> unofficially for a while, but I would like to now officially propose 
> this for Lustre 2.2.
> 
> With the OBD changes that are taking place on the Orion branch, we want 
> to make lustre be able to use any of a number of backend filesystems. 
> The current tree and configure tools make it very hard to build without 
> ldiskfs (instead using zfs, btrfs, or something we haven''t thought
of
> yet).  As part of cleaning that up, moving ldiskfs external to lustre 
> will help ensure that we don''t have unnecessary dependencies crop
up in
> the future.
> 
> We have created an external package of the ldiskfs tree and put it up on 
> github:
> 
>  https://github.com/chaos/ldiskfs
> 
> To use the new external ldiskfs package with lustre, you will also need 
> a few patches to lustre itself.  There are links to the patches in this 
> jira ticket:
> 
>  http://jira.whamcloud.com/browse/LU-723
> 
> On a RHEL system, ldiskfs has a build dependency on the kernel-debuginfo 
> packages by default.  That is where we find the ext4 source code.
> 
> Building should be fairly straight forward.  Build and install ldiskfs 
> (in particular the lustre-ldiskfs-devel package), and then build lustre. 
> The main new change to lustre is the addition of the configure option 
> "--with-ldiskfs-devel".  On a RHEL system if you have the 
> lustre-ldiskfs-devel package installed, you won''t need to give a
path.
> 
> Note that we have not yet removed the in-tree ldiskfs.  Our first step 
> was to get this working.  Once this is accepted, we will be happy to 
> submit the patches to remove lustre''s copy of ldiskfs and
generally
> clean up lustre''s autoconf checks involving ldiskfs.
> 
> We attempted to keep the changes minimal, so we didn''t change the
name
> of the ldiskfs rpm packages.  But we think is would be nice to change 
> the name from "lustre-ldiskfs" to simply "ldiskfs".  If
we want to make
> that change, now is the time to do it.
> 
> Chris
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

On Wed, Oct 12, 2011 at 07:05:51PM -0700, Andreas Dilger
wrote:
> Chris,
> I''m not against this in principle, but I think it may be more
difficult to actually implement before the OSD changes from Orion are landed to
master.  In the past we also thought about making ldiskfs as a separate package,
before ext4 started in the kernel.
> 
> I''ve seen Prakash working on those patches but will not have time
to look at them until at least next week.
> 
> Are those patches against the master branch or against Orion?  Also, what
kernels are supported?
The patches for Lustre are against master. I believe our plan is to
have the patches landed and working on master, before we officially
integrate them into Orion.

As far as supported kernels, I''ve tried to keep that consistent with
what is already supported; although I admit most testing so far has
been done on RHEL.

What kernels are officially supported by Whamcloud? What kernels does it
need to support?
> 
> Cheers, Andreas
> 
> On 2011-10-12, at 3:44 PM, "Christopher J. Morrone" <morrone2
at llnl.gov> wrote:
> 
> > We would like to see the ldiskfs tree removed from the lustre tree and
> > made an independent package.  I have been floating this idea 
> > unofficially for a while, but I would like to now officially propose 
> > this for Lustre 2.2.
> > 
> > With the OBD changes that are taking place on the Orion branch, we
want
> > to make lustre be able to use any of a number of backend filesystems. 
> > The current tree and configure tools make it very hard to build
without
> > ldiskfs (instead using zfs, btrfs, or something we haven''t
thought of
> > yet).  As part of cleaning that up, moving ldiskfs external to lustre 
> > will help ensure that we don''t have unnecessary dependencies
crop up in
> > the future.
> > 
> > We have created an external package of the ldiskfs tree and put it up
on
> > github:
> > 
> >  https://github.com/chaos/ldiskfs
> > 
> > To use the new external ldiskfs package with lustre, you will also
need
> > a few patches to lustre itself.  There are links to the patches in
this
> > jira ticket:
> > 
> >  http://jira.whamcloud.com/browse/LU-723
> > 
> > On a RHEL system, ldiskfs has a build dependency on the
kernel-debuginfo
> > packages by default.  That is where we find the ext4 source code.
> > 
> > Building should be fairly straight forward.  Build and install ldiskfs
> > (in particular the lustre-ldiskfs-devel package), and then build
lustre.
> > The main new change to lustre is the addition of the configure option 
> > "--with-ldiskfs-devel".  On a RHEL system if you have the 
> > lustre-ldiskfs-devel package installed, you won''t need to
give a path.
> > 
> > Note that we have not yet removed the in-tree ldiskfs.  Our first step
> > was to get this working.  Once this is accepted, we will be happy to 
> > submit the patches to remove lustre''s copy of ldiskfs and
generally
> > clean up lustre''s autoconf checks involving ldiskfs.
> > 
> > We attempted to keep the changes minimal, so we didn''t change
the name
> > of the ldiskfs rpm packages.  But we think is would be nice to change 
> > the name from "lustre-ldiskfs" to simply
"ldiskfs".  If we want to make
> > that change, now is the time to do it.
> > 
> > Chris
> > 
> > _______________________________________________
> > Lustre-devel mailing list
> > Lustre-devel at lists.lustre.org
> > http://lists.lustre.org/mailman/listinfo/lustre-devel
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
-- 
Cheers,
Prakash

On Oct 12, 2011, at 7:05 PM, Andreas Dilger wrote:
> Chris,
> I''m not against this in principle, but I think it may be more
difficult to actually implement before the OSD changes from Orion are landed to
master.
You mean, it will make it more difficult to land Orion changes, right?  
Chris already has implemented the external build.

> In the past we also thought about making ldiskfs as a separate package,
before ext4 started in the kernel.
> 
> I''ve seen Prakash working on those patches but will not have time
to look at them until at least next week.
> 
> Are those patches against the master branch or against Orion?  Also, what
kernels are supported?
> 
> Cheers, Andreas
> 
> On 2011-10-12, at 3:44 PM, "Christopher J. Morrone" <morrone2
at llnl.gov> wrote:
> 
>> We would like to see the ldiskfs tree removed from the lustre tree and 
>> made an independent package.  I have been floating this idea 
>> unofficially for a while, but I would like to now officially propose 
>> this for Lustre 2.2.
>> 
>> With the OBD changes that are taking place on the Orion branch, we want
>> to make lustre be able to use any of a number of backend filesystems. 
>> The current tree and configure tools make it very hard to build without
>> ldiskfs (instead using zfs, btrfs, or something we haven''t
thought of
>> yet).  As part of cleaning that up, moving ldiskfs external to lustre 
>> will help ensure that we don''t have unnecessary dependencies
crop up in
>> the future.
>> 
>> We have created an external package of the ldiskfs tree and put it up
on
>> github:
>> 
>> https://github.com/chaos/ldiskfs
>> 
>> To use the new external ldiskfs package with lustre, you will also need
>> a few patches to lustre itself.  There are links to the patches in this
>> jira ticket:
>> 
>> http://jira.whamcloud.com/browse/LU-723
>> 
>> On a RHEL system, ldiskfs has a build dependency on the
kernel-debuginfo
>> packages by default.  That is where we find the ext4 source code.
>> 
>> Building should be fairly straight forward.  Build and install ldiskfs 
>> (in particular the lustre-ldiskfs-devel package), and then build
lustre.
>> The main new change to lustre is the addition of the configure option 
>> "--with-ldiskfs-devel".  On a RHEL system if you have the 
>> lustre-ldiskfs-devel package installed, you won''t need to give
a path.
>> 
>> Note that we have not yet removed the in-tree ldiskfs.  Our first step 
>> was to get this working.  Once this is accepted, we will be happy to 
>> submit the patches to remove lustre''s copy of ldiskfs and
generally
>> clean up lustre''s autoconf checks involving ldiskfs.
>> 
>> We attempted to keep the changes minimal, so we didn''t change
the name
>> of the ldiskfs rpm packages.  But we think is would be nice to change 
>> the name from "lustre-ldiskfs" to simply "ldiskfs".
If we want to make
>> that change, now is the time to do it.
>> 
>> Chris
>> 
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
______________________________________________________________________
This email may contain privileged or confidential information, which should only
be used for the purpose for which it was sent by Xyratex. No further rights or
licenses are granted to use such information. If you are not the intended
recipient of this message, please notify the sender by return and delete it. You
may not use, copy, disclose or rely on the information contained in it.
 
Internet email is susceptible to data corruption, interception and unauthorised
amendment for which Xyratex does not accept liability. While we have taken
reasonable precautions to ensure that this email is free of viruses, Xyratex
does not accept liability for the presence of any computer viruses in this
email, nor for any losses caused as a result of viruses.
 
Xyratex Technology Limited (03134912), Registered in England & Wales,
Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA.
 
The Xyratex group of companies also includes, Xyratex Ltd, registered in
Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia)
Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in
The People''s Republic of China and Xyratex Japan Limited registered in
Japan.
______________________________________________________________________

On 2011-10-13, at 4:30 PM, "Nathan Rutman" <Nathan_Rutman at
xyratex.com> wrote:
> On Oct 12, 2011, at 7:05 PM, Andreas Dilger wrote:
>> I''m not against this in principle, but I think it may be more
difficult to actually implement before the OSD changes from Orion are landed to
master.
> 
> You mean, it will make it more difficult to land Orion changes, right?  
> Chris already has implemented the external build.
No, I mean it will make it harder to maintain Lustre and ldiskfs separately
before we get to the OSD API that has a better abstraction than the current
fsfilt/lvfs interface that is used in the current Lustre code.

Granted that we don''t have to change this code very often recently, it
does change on occasion, and maintaining compatibility between arbitrary
versions of ldiskfs and Lustre is a headache that we don''t really need
right now.

The fact that Chris has a point-in-time version of ldiskfs and Lustre that are
working together is great, but given that the OSD code migration is already
underway it doesn''t make sense to add a lot of code churn before that
happens.

I''m not against making this change if there is no expectation that
there is lomg-term compatibility between different ldiskfs and Lustre builds. If
it works, great, but if there is an interface change needed then both packages
will need to be upgraded.

Hopefully once we have the OSD API in use for the OSS, MGS, and the ZFS is
working there will be fewer API changes needed and the number of dependent
package updates can be reduced.

Cheers, Andreas
>> In the past we also thought about making ldiskfs as a separate package,
before ext4 started in the kernel.
>> 
>> I''ve seen Prakash working on those patches but will not have
time to look at them until at least next week.
>> 
>> Are those patches against the master branch or against Orion?  Also,
what kernels are supported?
>> 
>> Cheers, Andreas
>> 
>> On 2011-10-12, at 3:44 PM, "Christopher J. Morrone"
<morrone2 at llnl.gov> wrote:
>> 
>>> We would like to see the ldiskfs tree removed from the lustre tree
and
>>> made an independent package.  I have been floating this idea 
>>> unofficially for a while, but I would like to now officially
propose
>>> this for Lustre 2.2.
>>> 
>>> With the OBD changes that are taking place on the Orion branch, we
want
>>> to make lustre be able to use any of a number of backend
filesystems.
>>> The current tree and configure tools make it very hard to build
without
>>> ldiskfs (instead using zfs, btrfs, or something we haven''t
thought of
>>> yet).  As part of cleaning that up, moving ldiskfs external to
lustre
>>> will help ensure that we don''t have unnecessary
dependencies crop up in
>>> the future.
>>> 
>>> We have created an external package of the ldiskfs tree and put it
up on
>>> github:
>>> 
>>> https://github.com/chaos/ldiskfs
>>> 
>>> To use the new external ldiskfs package with lustre, you will also
need
>>> a few patches to lustre itself.  There are links to the patches in
this
>>> jira ticket:
>>> 
>>> http://jira.whamcloud.com/browse/LU-723
>>> 
>>> On a RHEL system, ldiskfs has a build dependency on the
kernel-debuginfo
>>> packages by default.  That is where we find the ext4 source code.
>>> 
>>> Building should be fairly straight forward.  Build and install
ldiskfs
>>> (in particular the lustre-ldiskfs-devel package), and then build
lustre.
>>> The main new change to lustre is the addition of the configure
option
>>> "--with-ldiskfs-devel".  On a RHEL system if you have the
>>> lustre-ldiskfs-devel package installed, you won''t need to
give a path.
>>> 
>>> Note that we have not yet removed the in-tree ldiskfs.  Our first
step
>>> was to get this working.  Once this is accepted, we will be happy
to
>>> submit the patches to remove lustre''s copy of ldiskfs and
generally
>>> clean up lustre''s autoconf checks involving ldiskfs.
>>> 
>>> We attempted to keep the changes minimal, so we didn''t
change the name
>>> of the ldiskfs rpm packages.  But we think is would be nice to
change
>>> the name from "lustre-ldiskfs" to simply
"ldiskfs".  If we want to make
>>> that change, now is the time to do it.
>>> 
>>> Chris
>>> 
>>> _______________________________________________
>>> Lustre-devel mailing list
>>> Lustre-devel at lists.lustre.org
>>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>> 
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
> ______________________________________________________________________
> This email may contain privileged or confidential information, which should
only be used for the purpose for which it was sent by Xyratex. No further rights
or licenses are granted to use such information. If you are not the intended
recipient of this message, please notify the sender by return and delete it. You
may not use, copy, disclose or rely on the information contained in it.
> 
> Internet email is susceptible to data corruption, interception and
unauthorised amendment for which Xyratex does not accept liability. While we
have taken reasonable precautions to ensure that this email is free of viruses,
Xyratex does not accept liability for the presence of any computer viruses in
this email, nor for any losses caused as a result of viruses.
> 
> Xyratex Technology Limited (03134912), Registered in England & Wales,
Registered Office, Langstone Road, Havant, Hampshire, PO9 1SA.
> 
> The Xyratex group of companies also includes, Xyratex Ltd, registered in
Bermuda, Xyratex International Inc, registered in California, Xyratex (Malaysia)
Sdn Bhd registered in Malaysia, Xyratex Technology (Wuxi) Co Ltd registered in
The People''s Republic of China and Xyratex Japan Limited registered in
Japan.
> ______________________________________________________________________
> 
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-devel/attachments/20111013/6bc1b75a/attachment-0001.html

On 2011-10-13, at 9:41 AM, Prakash Surya <surya1 at llnl.gov>
wrote:
> On Wed, Oct 12, 2011 at 07:05:51PM -0700, Andreas Dilger wrote:
>> 
>> I''m not against this in principle, but I think it may be more
difficult to actually implement before the OSD changes from Orion are landed to
master.  In the past we also thought about making ldiskfs as a separate package,
before ext4 started in the kernel.
>> 
>> I''ve seen Prakash working on those patches but will not have
time to look at them until at least next week.
>> 
>> Are those patches against the master branch or against Orion?  Also,
what kernels are supported?
> 
> The patches for Lustre are against master. I believe our plan is to
> have the patches landed and working on master, before we officially
> integrate them into Orion.
> 
> As far as supported kernels, I''ve tried to keep that consistent
with
> what is already supported; although I admit most testing so far has
> been done on RHEL.
> 
> What kernels are officially supported by Whamcloud? What kernels does it
> need to support?
So far we only support servers with RHEL-like kernels, but I''ve also
seen patches for SLES kernels submitted, so I don''t want to rule those
out.

Like I previously wrote, I''m not against the patches in principle, but
I haven''t had a chance to look at the changes themselves yet since
I''m on vacation and not supposed to be working...

Have you had the ldiskfs module externally working for a while?  How much effort
is it to coordinate changes in ldiskfs (exports, API changes, etc) with Lustre?

Cheers, Andreas
>> 
>> On 2011-10-12, at 3:44 PM, "Christopher J. Morrone"
<morrone2 at llnl.gov> wrote:
>> 
>>> We would like to see the ldiskfs tree removed from the lustre tree
and
>>> made an independent package.  I have been floating this idea 
>>> unofficially for a while, but I would like to now officially
propose
>>> this for Lustre 2.2.
>>> 
>>> With the OBD changes that are taking place on the Orion branch, we
want
>>> to make lustre be able to use any of a number of backend
filesystems.
>>> The current tree and configure tools make it very hard to build
without
>>> ldiskfs (instead using zfs, btrfs, or something we haven''t
thought of
>>> yet).  As part of cleaning that up, moving ldiskfs external to
lustre
>>> will help ensure that we don''t have unnecessary
dependencies crop up in
>>> the future.
>>> 
>>> We have created an external package of the ldiskfs tree and put it
up on
>>> github:
>>> 
>>> https://github.com/chaos/ldiskfs
>>> 
>>> To use the new external ldiskfs package with lustre, you will also
need
>>> a few patches to lustre itself.  There are links to the patches in
this
>>> jira ticket:
>>> 
>>> http://jira.whamcloud.com/browse/LU-723
>>> 
>>> On a RHEL system, ldiskfs has a build dependency on the
kernel-debuginfo
>>> packages by default.  That is where we find the ext4 source code.
>>> 
>>> Building should be fairly straight forward.  Build and install
ldiskfs
>>> (in particular the lustre-ldiskfs-devel package), and then build
lustre.
>>> The main new change to lustre is the addition of the configure
option
>>> "--with-ldiskfs-devel".  On a RHEL system if you have the
>>> lustre-ldiskfs-devel package installed, you won''t need to
give a path.
>>> 
>>> Note that we have not yet removed the in-tree ldiskfs.  Our first
step
>>> was to get this working.  Once this is accepted, we will be happy
to
>>> submit the patches to remove lustre''s copy of ldiskfs and
generally
>>> clean up lustre''s autoconf checks involving ldiskfs.
>>> 
>>> We attempted to keep the changes minimal, so we didn''t
change the name
>>> of the ldiskfs rpm packages.  But we think is would be nice to
change
>>> the name from "lustre-ldiskfs" to simply
"ldiskfs".  If we want to make
>>> that change, now is the time to do it.
>>> 
>>> Chris
>>> 
>>> _______________________________________________
>>> Lustre-devel mailing list
>>> Lustre-devel at lists.lustre.org
>>> http://lists.lustre.org/mailman/listinfo/lustre-devel
>> 
>> _______________________________________________
>> Lustre-devel mailing list
>> Lustre-devel at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-devel
> 
> -- 
> Cheers,
> Prakash
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-devel/attachments/20111013/83dc2b90/attachment.html