Lustre discuss - Mar 2012 - Lustre and cross-platform portability

Whamcloud and EMC are jointly investigating how to be able to contribute the
Lustre client code into the upstream Linux kernel.

As a prerequisite to this, EMC is working to clean up the Lustre client code to
better match the kernel coding style, and one of the anticipated major obstacles
to upstream kernel submission is the heavy use of code abstraction via libcfs
for portability to other operating systems (most notably MacOS and WinNT, but
also for liblustre, and potentially *BSD).

I have no information that the WinNT project will ever be released by Oracle,
and as yet there has not been any code released from the MacOS port, so the
libcfs portability layer is potentially exacting a high cost in code maintenance
and complexity (CLIO being a prime example) for no apparent benefit.  Similarly,
the liblustre client needs a portability layer for userspace, and suffers from
the same apparent lack of interest or users.

I''d like to get some feedback from the Lustre community about removing
the libcfs abstraction entirely, or possibly restructuring it to look like the
Linux kernel API, and having the other platforms code against it as a Linux
portability layer, like ZFS on Linux uses the Solaris Portability Layer (SPL) to
avoid changing the core ZFS code.  A related topic is whether it would be better
to replace all cfs_* functions with standard Linux kernel functions en-masse, or
migrate away from cfs_* functions slowly?

Also, we''re planning on deprecating the liblustre client code, due to
lack of interest/usage.  The current code is in disrepair, and we''ve
been keeping it around for years without any benefit, and while I was one of the
strongest advocates for keeping it in our back pocket in case of future needs, I
don''t see that materializing in the future.

The liblustre code would be left in the tree for now, in case someone from the
community is interested to get it working and maintain it, and it may be updated
on a best effort basis.  If nobody steps forward to do this work, the liblustre
code would be deleted from the development branch in a year or so.


Unfortunately, after starting this thread, I may not be able to reply to
questions in a timely manner due to vacation.  I look forward to a thread that
concludes with unanimous agreement from all parties. :-)

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/

Having a native Windows & Mac client is by far our company''s #1
most
important feature for the future. We have been seriously considering how we
could help make this happen, including putting funds and/or developers
toward the cause.

I know we''re in the minority on this, but I believe it''s
because we are not
using Lustre for HPC. We use it for post-production of TV/film. There are a
few others companies in our industry who have started doing this as well.
My point is that, while Lustre currently is focused on the HPC crowd who
seem not to care about Windows/Mac, Lustre''s maturity is giving it the
potential to grow into other uses besides HPC. I wouldn''t call them
general
use, but other high-performance uses. In our industry, where there are a
lot of Windows & Mac workstations that we want to connect to the Lustre
storage, the Linux-only client is a major obstacle to that. I''m sure
there
are other industries that would benefit from this.

If the community wants to keep Lustre strictly HPC focused and discourage
other industries from joining in, then abandoning the bridge (albeit the
half-built bridge that it is) to Linux/Windows is a good way to do that.
If, on the other hand, there is a desire to get some other industries
involved, perhaps with more resources and contribution coming from them,
then I think it''s important to build on the work that has been done. In
that regard, upstream Linux kernel inclusion seems like a very low priority
to me.


2012/3/15 <lustre-discuss-request at lists.lustre.org>
> ---------- Forwarded message ----------
> From: Andreas Dilger <adilger at whamcloud.com>
> To: twg at lists.opensfs.org
> Cc: wc-discuss <wc-discuss at whamcloud.com>, lustre-discuss discuss
<
> lustre-discuss at lists.lustre.org>, Lustre Devel <
> lustre-devel at lists.lustre.org>
> Date: Wed, 14 Mar 2012 18:31:29 -0600
> Subject: [Lustre-discuss] Lustre and cross-platform portability
> Whamcloud and EMC are jointly investigating how to be able to contribute
> the Lustre client code into the upstream Linux kernel.
>
> As a prerequisite to this, EMC is working to clean up the Lustre client
> code to better match the kernel coding style, and one of the anticipated
> major obstacles to upstream kernel submission is the heavy use of code
> abstraction via libcfs for portability to other operating systems (most
> notably MacOS and WinNT, but also for liblustre, and potentially *BSD).
>
> I have no information that the WinNT project will ever be released by
> Oracle, and as yet there has not been any code released from the MacOS
> port, so the libcfs portability layer is potentially exacting a high cost
> in code maintenance and complexity (CLIO being a prime example) for no
> apparent benefit.  Similarly, the liblustre client needs a portability
> layer for userspace, and suffers from the same apparent lack of interest or
> users.
>
> I''d like to get some feedback from the Lustre community about
removing the
> libcfs abstraction entirely, or possibly restructuring it to look like the
> Linux kernel API, and having the other platforms code against it as a Linux
> portability layer, like ZFS on Linux uses the Solaris Portability Layer
> (SPL) to avoid changing the core ZFS code.  A related topic is whether it
> would be better to replace all cfs_* functions with standard Linux kernel
> functions en-masse, or migrate away from cfs_* functions slowly?
>
> Also, we''re planning on deprecating the liblustre client code, due
to lack
> of interest/usage.  The current code is in disrepair, and we''ve
been
> keeping it around for years without any benefit, and while I was one of the
> strongest advocates for keeping it in our back pocket in case of future
> needs, I don''t see that materializing in the future.
>
> The liblustre code would be left in the tree for now, in case someone from
> the community is interested to get it working and maintain it, and it may
> be updated on a best effort basis.  If nobody steps forward to do this
> work, the liblustre code would be deleted from the development branch in a
> year or so.
>
>
> Unfortunately, after starting this thread, I may not be able to reply to
> questions in a timely manner due to vacation.  I look forward to a thread
> that concludes with unanimous agreement from all parties. :-)
>
> Cheers, Andreas
> --
> Andreas Dilger                       Whamcloud, Inc.
> Principal Lustre Engineer            http://www.whamcloud.com/
>
>
>
>
>
>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20120315/9287d0ff/attachment.html

On 03/14/2012 08:31 PM, Andreas Dilger wrote:
> Whamcloud and EMC are jointly investigating how to be able to
> contribute the Lustre client code into the upstream Linux kernel.
Would be good to have an active discussion of this at LUG as well.
> As a prerequisite to this, EMC is working to clean up the Lustre
> client code to better match the kernel coding style, and one of the
> anticipated major obstacles to upstream kernel submission is the
> heavy use of code abstraction via libcfs for portability to other
> operating systems (most notably MacOS and WinNT, but also for
> liblustre, and potentially *BSD).
Breaking the code into bite sized patches that could go into the kernel 
would be a *good* thing.  Continuing to maintain a set of out of core 
patches (with compatibility/portability issues) is not a good thing.
> I have no information that the WinNT project will ever be released by
> Oracle, and as yet there has not been any code released from the
> MacOS port, so the libcfs portability layer is potentially exacting a
> high cost in code maintenance and complexity (CLIO being a prime
> example) for no apparent benefit.  Similarly, the liblustre client
> needs a portability layer for userspace, and suffers from the same
> apparent lack of interest or users.
>
> I''d like to get some feedback from the Lustre community about
> removing the libcfs abstraction entirely, or possibly restructuring
> it to look like the Linux kernel API, and having the other platforms
(Re)Using as much of the existing kernel API as possible (reduce 
friction for inclusion), without re-inventing wheels (reduce friction 
for inclusion), and insisting "our wheels are rounder" versus 
fixing/updating existing subsystems (reduce friction for inclusion) 
would be advised.  Many a good project has lost out on kernel inclusion 
to other competitive projects over these issues (and interaction with 
the kernel community).  SCST (scsi target layer) is a prime example, and 
one we use, even though LIO was just included about a year ago.  The LIO 
team does kernel developer interaction far better than the SCST team. 
Has nothing to do with code quality/functionality.
> code against it as a Linux portability layer, like ZFS on Linux uses
> the Solaris Portability Layer (SPL) to avoid changing the core ZFS
> code.  A related topic is whether it would be better to replace all
> cfs_* functions with standard Linux kernel functions en-masse, or
> migrate away from cfs_* functions slowly?
Do the change en-masse.  Get it done, once.  The sooner the better, so 
we can test.  Hopefully this means we will also be able to use 
alternative OST file systems other than ext4.






-- 
Joseph Landman, Ph.D
Founder and CEO
Scalable Informatics Inc.
email: landman at scalableinformatics.com
web  : http://scalableinformatics.com
        http://scalableinformatics.com/sicluster
phone: +1 734 786 8423 x121
fax  : +1 866 888 3112
cell : +1 734 612 4615

On 2012-03-15, at 8:22 AM, Tyler Hawes wrote:
> Having a native Windows & Mac client is by far our company''s
#1 most important feature for the future. We have been seriously considering how
we could help make this happen, including putting funds and/or developers toward
the cause.
Tyler, thanks for your feedback.

While at Oracle, there was serious effort put toward having a Windows Native
Client.  Unfortunately, this died along with other Oracle Lustre development
projects, and the proprietary code has been sitting unused inside Oracle since
then.

Similarly, the MacOS native client project started a couple of years ago, but as
yet none of the code has been released.  While some work was done for a Solaris
server, I don''t see much hope in that making progress at all.

In both cases, I would be supportive of these efforts if there was a likelihood
of them bearing fruit any time in the future, but so far I haven''t seen
any progress in that direction.  If Oracle could somehow release the WNC code
and/or NRL can overcome the government roadblocks in their path for releasing
the MacOS client (MLC?), it would definitely change the direction of my
thinking.

As it stands now, we have portability code for Windows/Mac without any ability
to even build the code, let alone test it, so it is just a burden for Linux.  If
there was real value being derived from that code (e.g. actual users), then it
might be an acceptable burden.
> I know we''re in the minority on this, but I believe it''s
because we are not using Lustre for HPC. We use it for post-production of
TV/film. There are a few others companies in our industry who have started doing
this as well. My point is that, while Lustre currently is focused on the HPC
crowd who seem not to care about Windows/Mac, Lustre''s maturity is
giving it the potential to grow into other uses besides HPC. I wouldn''t
call them general use, but other high-performance uses. In our industry, where
there are a lot of Windows & Mac workstations that we want to connect to the
Lustre storage, the Linux-only client is a major obstacle to that. I''m
sure there are other industries that would benefit from this.
Yes, the TV/film industry was one of the prime motivators for WNC.  There have
always been a small number of users from this market, and growth in this
industry (and others) is definitely welcomed.
> If the community wants to keep Lustre strictly HPC focused and discourage
other industries from joining in, then abandoning the bridge (albeit the
half-built bridge that it is) to Linux/Windows is a good way to do that. If, on
the other hand, there is a desire to get some other industries involved, perhaps
with more resources and contribution coming from them, then I think
it''s important to build on the work that has been done. In that regard,
upstream Linux kernel inclusion seems like a very low priority to me.
I think even for upstream kernel submission, if it were clear that the layering
in Lustre was for a valid purpose (i.e. existing Mac/Win clients) then it might
be accepted.  As it stands now, we can''t honestly make that argument to
the kernel maintainers.
> 2012/3/15 <lustre-discuss-request at lists.lustre.org>
> ---------- Forwarded message ----------
> From: Andreas Dilger <adilger at whamcloud.com>
> To: twg at lists.opensfs.org
> Cc: wc-discuss <wc-discuss at whamcloud.com>, lustre-discuss discuss
<lustre-discuss at lists.lustre.org>, Lustre Devel <lustre-devel at
lists.lustre.org>
> Date: Wed, 14 Mar 2012 18:31:29 -0600
> Subject: [Lustre-discuss] Lustre and cross-platform portability
> Whamcloud and EMC are jointly investigating how to be able to contribute
the Lustre client code into the upstream Linux kernel.
> 
> As a prerequisite to this, EMC is working to clean up the Lustre client
code to better match the kernel coding style, and one of the anticipated major
obstacles to upstream kernel submission is the heavy use of code abstraction via
libcfs for portability to other operating systems (most notably MacOS and WinNT,
but also for liblustre, and potentially *BSD).
> 
> I have no information that the WinNT project will ever be released by
Oracle, and as yet there has not been any code released from the MacOS port, so
the libcfs portability layer is potentially exacting a high cost in code
maintenance and complexity (CLIO being a prime example) for no apparent benefit.
Similarly, the liblustre client needs a portability layer for userspace, and
suffers from the same apparent lack of interest or users.
> 
> I''d like to get some feedback from the Lustre community about
removing the libcfs abstraction entirely, or possibly restructuring it to look
like the Linux kernel API, and having the other platforms code against it as a
Linux portability layer, like ZFS on Linux uses the Solaris Portability Layer
(SPL) to avoid changing the core ZFS code.  A related topic is whether it would
be better to replace all cfs_* functions with standard Linux kernel functions
en-masse, or migrate away from cfs_* functions slowly?
> 
> Also, we''re planning on deprecating the liblustre client code, due
to lack of interest/usage.  The current code is in disrepair, and we''ve
been keeping it around for years without any benefit, and while I was one of the
strongest advocates for keeping it in our back pocket in case of future needs, I
don''t see that materializing in the future.
> 
> The liblustre code would be left in the tree for now, in case someone from
the community is interested to get it working and maintain it, and it may be
updated on a best effort basis.  If nobody steps forward to do this work, the
liblustre code would be deleted from the development branch in a year or so.
> 
> 
> Unfortunately, after starting this thread, I may not be able to reply to
questions in a timely manner due to vacation.  I look forward to a thread that
concludes with unanimous agreement from all parties. :-)
> 
> Cheers, Andreas
> --
> Andreas Dilger                       Whamcloud, Inc.
> Principal Lustre Engineer            http://www.whamcloud.com/
> 
> 
> 
> 
> 
> 
> 

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/

>I have no information that the WinNT project will ever be released
>by Oracle, and as yet there has not been any code released from the
>MacOS port, so the libcfs portability layer is potentially exacting
>a high cost in code maintenance and complexity (CLIO being a prime
>example) for no apparent benefit.  Similarly, the liblustre client needs
>a portability layer for userspace, and suffers from the same apparent
>lack of interest or users.
In terms of the MacOS X port, I don''t think that''s quite fair
...
the code I did is available and anyone can download it.  It was
functional in a very basic way but needed some additonal love.
Okay, I haven''t rolled that stuff into the Whamcloud release ...
what happened there was when there was all the uncertainty with
Oracle & Lustre development I lost momentum and got caught up in
other things.  I''ve talked with the guys at Whamcloud about bringing
at least the portability changes over, and that''s all been on me;
it''s certainly on my list to work on.

I can say that at least for MacOS X, there has been interest; I can''t
speak for the amount of interest, and there''s a bit of a chicken and
egg problem ... people don''t plan their Lustre use around MacOS X
clients because there isn''t one that works well, and people
don''t put
work into it because there isn''t people who plan their Lustre use
around it.
>I''d like to get some feedback from the Lustre community about
removing
>the libcfs abstraction entirely, or possibly restructuring it to look
>like the Linux kernel API, and having the other platforms code against
>it as a Linux portability layer, like ZFS on Linux uses the Solaris
>Portability Layer (SPL) to avoid changing the core ZFS code.  A related
>topic is whether it would be better to replace all cfs_* functions with
>standard Linux kernel functions en-masse, or migrate away from cfs_*
>functions slowly?
The only thing I can think of is that if this is done, then officially
Lustre is going to be a Linux-only filesystem.  I understand there are
real costs to maintaining the cfs layer, and I can''t speak to whether
or
not the loss would be worth the gains.

--Ken

On 2012-03-15, at 12:45 PM, Ken Hornstein wrote:
>> I have no information that the WinNT project will ever be released
>> by Oracle, and as yet there has not been any code released from the
>> MacOS port, so the libcfs portability layer is potentially exacting
>> a high cost in code maintenance and complexity (CLIO being a prime
>> example) for no apparent benefit.  Similarly, the liblustre client
>> needs a portability layer for userspace, and suffers from the same
>> apparent lack of interest or users.
> 
> In terms of the MacOS X port, I don''t think that''s quite
fair ...
> the code I did is available and anyone can download it.
Ken, my apologies for this misstatement.  I guess that my faulty memory is to
blame for the fact that I didn''t recall the MacOS code was made
publicly available for download.
>  It was
> functional in a very basic way but needed some additonal love.
> Okay, I haven''t rolled that stuff into the Whamcloud release ...
I don''t think I''ve ever seen patches sent from you to either
Oracle or Whamcloud, and unfortunately nobody on our side has had the bandwidth
or user demand/funding to be pulling such changes either.
> what happened there was when there was all the uncertainty with
> Oracle & Lustre development I lost momentum and got caught up in
> other things.  I''ve talked with the guys at Whamcloud about
bringing
> at least the portability changes over, and that''s all been on me;
> it''s certainly on my list to work on.
> 
> I can say that at least for MacOS X, there has been interest; I
can''t
> speak for the amount of interest, and there''s a bit of a chicken
and
> egg problem ... people don''t plan their Lustre use around MacOS X
> clients because there isn''t one that works well, and people
don''t put
> work into it because there isn''t people who plan their Lustre use
> around it.
Another alternative that has been proposed is FUSE for MacOS/Solaris. 
I''m not sure if FUSE exists for Windows or not.  Of course, this is
only useful for casual access, and not for high performance usage.
>> I''d like to get some feedback from the Lustre community about
removing
>> the libcfs abstraction entirely, or possibly restructuring it to look
>> like the Linux kernel API, and having the other platforms code against
>> it as a Linux portability layer, like ZFS on Linux uses the Solaris
>> Portability Layer (SPL) to avoid changing the core ZFS code.  A related
>> topic is whether it would be better to replace all cfs_* functions with
>> standard Linux kernel functions en-masse, or migrate away from cfs_*
>> functions slowly?
> 
> The only thing I can think of is that if this is done, then officially
> Lustre is going to be a Linux-only filesystem.  I understand there are
> real costs to maintaining the cfs layer, and I can''t speak to
whether or
> not the loss would be worth the gains.
This isn''t strictly correct.  It would be possible to change the libcfs
portability layer to export the same API as the Linux kernel to MacOS and
Windows.  This would simplify getting the client into the Linux kernel, but
still allow a native client on MacOS.

I''m fine with keeping the abstraction layer if there is legitimate use
for it.  Having the MacOS client in the Lustre tree in a state where it can at
least be built would allow us to make changes to the code (e.g. function
renaming) with some chance of it not being totally broken immediately.

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1



On 03/15/2012 03:39 PM, Andreas Dilger wrote:
> Another alternative that has been proposed is FUSE for
> MacOS/Solaris. I''m not sure if FUSE exists for Windows or not.  Of
> course, this is only useful for casual access, and not for high
> performance usage.
> 
I think the FUSE for Windows project is defunct, but there is a user
space filesystem project for Windows called Dokan
(http://dokan-dev.net/en/) that had a release a little more than a
year ago.

For my part, a Lustre client on Windows or OS X would be used mostly
for visualization of data, rather than being computed against so a
slower user space implementation would be more than sufficient. There
are a few applications that are using an SMB exported Lustre
filesystem for data collection, but those applications are similarly
low bandwidth. It would be nice to remove the SMB server in the middle.

- -Josh

- --
Joshua Walgenbach, High Performance File Systems, Indiana University
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/

iEYEARECAAYFAk9iSFEACgkQcqyJPuRTYp8dRgCePbaKA2P1Km80KBJM/YG6Nky1
8pgAn3RQW1i5omkhJfFZOG+0yhtXKJFB
=08vB
-----END PGP SIGNATURE-----

We had a thought that a FUSE-based client based on liblustre might make sense to
avoid the NFS/Samba re-export problems (scalability, coherency), at the
potential price of some performance.  You mentioned on the phone this morning
that someone had already done something like this in the past?  If we revive
this work, presumably we can drop the Mac native client.
On the other hand, I agree that the baggage is large, and it will take much more
work to see this through, and without a real champion and/or a real community
need, I understand the desire to drop it.

On Mar 15, 2012, at 11:38 AM, Andreas Dilger wrote:
> On 2012-03-15, at 8:22 AM, Tyler Hawes wrote:
>> Having a native Windows & Mac client is by far our
company''s #1 most important feature for the future. We have been
seriously considering how we could help make this happen, including putting
funds and/or developers toward the cause.
> 
> Tyler, thanks for your feedback.
> 
> While at Oracle, there was serious effort put toward having a Windows
Native Client.  Unfortunately, this died along with other Oracle Lustre
development projects, and the proprietary code has been sitting unused inside
Oracle since then.
> 
> Similarly, the MacOS native client project started a couple of years ago,
but as yet none of the code has been released.  While some work was done for a
Solaris server, I don''t see much hope in that making progress at all.
> 
> In both cases, I would be supportive of these efforts if there was a
likelihood of them bearing fruit any time in the future, but so far I
haven''t seen any progress in that direction.  If Oracle could somehow
release the WNC code and/or NRL can overcome the government roadblocks in their
path for releasing the MacOS client (MLC?), it would definitely change the
direction of my thinking.
> 
> As it stands now, we have portability code for Windows/Mac without any
ability to even build the code, let alone test it, so it is just a burden for
Linux.  If there was real value being derived from that code (e.g. actual
users), then it might be an acceptable burden.
> 
>> I know we''re in the minority on this, but I believe
it''s because we are not using Lustre for HPC. We use it for
post-production of TV/film. There are a few others companies in our industry who
have started doing this as well. My point is that, while Lustre currently is
focused on the HPC crowd who seem not to care about Windows/Mac,
Lustre''s maturity is giving it the potential to grow into other uses
besides HPC. I wouldn''t call them general use, but other
high-performance uses. In our industry, where there are a lot of Windows &
Mac workstations that we want to connect to the Lustre storage, the Linux-only
client is a major obstacle to that. I''m sure there are other industries
that would benefit from this.
> 
> Yes, the TV/film industry was one of the prime motivators for WNC.  There
have always been a small number of users from this market, and growth in this
industry (and others) is definitely welcomed.
> 
>> If the community wants to keep Lustre strictly HPC focused and
discourage other industries from joining in, then abandoning the bridge (albeit
the half-built bridge that it is) to Linux/Windows is a good way to do that. If,
on the other hand, there is a desire to get some other industries involved,
perhaps with more resources and contribution coming from them, then I think
it''s important to build on the work that has been done. In that regard,
upstream Linux kernel inclusion seems like a very low priority to me.
> 
> I think even for upstream kernel submission, if it were clear that the
layering in Lustre was for a valid purpose (i.e. existing Mac/Win clients) then
it might be accepted.  As it stands now, we can''t honestly make that
argument to the kernel maintainers.
> 
>> 2012/3/15 <lustre-discuss-request at lists.lustre.org>
>> ---------- Forwarded message ----------
>> From: Andreas Dilger <adilger at whamcloud.com>
>> To: twg at lists.opensfs.org
>> Cc: wc-discuss <wc-discuss at whamcloud.com>, lustre-discuss
discuss <lustre-discuss at lists.lustre.org>, Lustre Devel
<lustre-devel at lists.lustre.org>
>> Date: Wed, 14 Mar 2012 18:31:29 -0600
>> Subject: [Lustre-discuss] Lustre and cross-platform portability
>> Whamcloud and EMC are jointly investigating how to be able to
contribute the Lustre client code into the upstream Linux kernel.
>> 
>> As a prerequisite to this, EMC is working to clean up the Lustre client
code to better match the kernel coding style, and one of the anticipated major
obstacles to upstream kernel submission is the heavy use of code abstraction via
libcfs for portability to other operating systems (most notably MacOS and WinNT,
but also for liblustre, and potentially *BSD).
>> 
>> I have no information that the WinNT project will ever be released by
Oracle, and as yet there has not been any code released from the MacOS port, so
the libcfs portability layer is potentially exacting a high cost in code
maintenance and complexity (CLIO being a prime example) for no apparent benefit.
Similarly, the liblustre client needs a portability layer for userspace, and
suffers from the same apparent lack of interest or users.
>> 
>> I''d like to get some feedback from the Lustre community about
removing the libcfs abstraction entirely, or possibly restructuring it to look
like the Linux kernel API, and having the other platforms code against it as a
Linux portability layer, like ZFS on Linux uses the Solaris Portability Layer
(SPL) to avoid changing the core ZFS code.  A related topic is whether it would
be better to replace all cfs_* functions with standard Linux kernel functions
en-masse, or migrate away from cfs_* functions slowly?
>> 
>> Also, we''re planning on deprecating the liblustre client code,
due to lack of interest/usage.  The current code is in disrepair, and
we''ve been keeping it around for years without any benefit, and while I
was one of the strongest advocates for keeping it in our back pocket in case of
future needs, I don''t see that materializing in the future.
>> 
>> The liblustre code would be left in the tree for now, in case someone
from the community is interested to get it working and maintain it, and it may
be updated on a best effort basis.  If nobody steps forward to do this work, the
liblustre code would be deleted from the development branch in a year or so.
>> 
>> 
>> Unfortunately, after starting this thread, I may not be able to reply
to questions in a timely manner due to vacation.  I look forward to a thread
that concludes with unanimous agreement from all parties. :-)
>> 
>> Cheers, Andreas
>> --
>> Andreas Dilger                       Whamcloud, Inc.
>> Principal Lustre Engineer            http://www.whamcloud.com/
>> 
>> 
>> 
>> 
>> 
>> 
>> 
> 
> 
> Cheers, Andreas
> --
> Andreas Dilger                       Whamcloud, Inc.
> Principal Lustre Engineer            http://www.whamcloud.com/
> 
> 
> 
>

On 15/03/12 19:51, Joshua Walgenbach wrote:
> For my part, a Lustre client on Windows or OS X would be used mostly
> for visualization of data, rather than being computed against so a
> slower user space implementation would be more than sufficient.
> There are a few applications that are using an SMB exported Lustre
> filesystem for data collection, but those applications are similarly
> low bandwidth. It would be nice to remove the SMB server in the
> middle.
why bother having a windows client if you lose the performance? We have
windows based detectors and proprietary windows based analysis software
that would definitely benefit from higher performance access to lustre
file systems but replacing existing CIFS servers for no gain seems a bit
pointless.

I understand the problems that a native MS client would have, without
buy-in from Microsoft it would be a nightmare to keep the MS tree and
the Linux tree in sync. One possible future workaround proposed by
Whamcloud at last years LUG is pNFS.

GREG
>
> - -Josh

-- 
Greg Matthews            01235 778658
Senior Computer Systems Administrator
Diamond Light Source, Oxfordshire, UK

-- 
This e-mail and any attachments may contain confidential, copyright and or
privileged material, and are for the use of the intended addressee only. If you
are not the intended addressee or an authorised recipient of the addressee
please notify us of receipt by returning the e-mail and do not use, copy,
retain, distribute or disclose the information in or attached to the e-mail.
Any opinions expressed within this e-mail are those of the individual and not
necessarily of Diamond Light Source Ltd.
Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments
are free from viruses and we cannot accept liability for any damage which you
may sustain as a result of software viruses which may be transmitted in or with
the message.
Diamond Light Source Limited (company no. 4375679). Registered in England and
Wales with its registered office at Diamond House, Harwell Science and
Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

Windows don''t have a good OFED port so we anyway will lost a
performance.
but using a full client will able to use a striping information and accept a
data from a several OST''s
with comparing a get a single data stream from a SMB.

Also fuse client will able to run on any OS have a FUSE porting that is any BSD,
OpenSolaris, MacOS, in additional to the windows.
That is easy way to maintain a single client for many OS.


On Mar 16, 2012, at 14:11, Gregory Matthews wrote:
> On 15/03/12 19:51, Joshua Walgenbach wrote:
>> For my part, a Lustre client on Windows or OS X would be used mostly
>> for visualization of data, rather than being computed against so a
>> slower user space implementation would be more than sufficient.
>> There are a few applications that are using an SMB exported Lustre
>> filesystem for data collection, but those applications are similarly
>> low bandwidth. It would be nice to remove the SMB server in the
>> middle.
> 
> why bother having a windows client if you lose the performance? We have
> windows based detectors and proprietary windows based analysis software
> that would definitely benefit from higher performance access to lustre
> file systems but replacing existing CIFS servers for no gain seems a bit
> pointless.
> 
> I understand the problems that a native MS client would have, without
> buy-in from Microsoft it would be a nightmare to keep the MS tree and
> the Linux tree in sync. One possible future workaround proposed by
> Whamcloud at last years LUG is pNFS.
> 
> GREG
> 
>> 
>> - -Josh
> 
> 
> -- 
> Greg Matthews            01235 778658
> Senior Computer Systems Administrator
> Diamond Light Source, Oxfordshire, UK
> 
> -- 
> This e-mail and any attachments may contain confidential, copyright and or
privileged material, and are for the use of the intended addressee only. If you
are not the intended addressee or an authorised recipient of the addressee
please notify us of receipt by returning the e-mail and do not use, copy,
retain, distribute or disclose the information in or attached to the e-mail.
> Any opinions expressed within this e-mail are those of the individual and
not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot
guarantee that this e-mail or any attachments are free from viruses and we
cannot accept liability for any damage which you may sustain as a result of
software viruses which may be transmitted in or with the message.
> Diamond Light Source Limited (company no. 4375679). Registered in England
and Wales with its registered office at Diamond House, Harwell Science and
Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
> 
> 
>

Cray no longer deploys liblustre.  Therefore, we probably take a
''don''t
care'' position in the discussion about the future of liblustre or
libcfs.

Thanks,
-Cory

On 03/16/2012 01:45 AM, Alexey Lyashkov wrote:
> If cray don''t wan''t to use a liblustre single thread
client - liblustre is good start to create a multi platform client.
> As i know it''s work now on Mac and FreeBSD, as Windows have a FUSE
port - it''s should be work in Windows also.
> 
> 
> On Mar 16, 2012, at 02:53, Nathan Rutman wrote:
> 
>> We had a thought that a FUSE-based client based on liblustre might make
sense to avoid the NFS/Samba re-export problems (scalability, coherency), at the
potential price of some performance.  You mentioned on the phone this morning
that someone had already done something like this in the past?  If we revive
this work, presumably we can drop the Mac native client.
>> On the other hand, I agree that the baggage is large, and it will take
much more work to see this through, and without a real champion and/or a real
community need, I understand the desire to drop it.
>>
>> On Mar 15, 2012, at 11:38 AM, Andreas Dilger wrote:
>>
>>> On 2012-03-15, at 8:22 AM, Tyler Hawes wrote:
>>>> Having a native Windows & Mac client is by far our
company''s #1 most important feature for the future. We have been
seriously considering how we could help make this happen, including putting
funds and/or developers toward the cause.
>>>
>>> Tyler, thanks for your feedback.
>>>
>>> While at Oracle, there was serious effort put toward having a
Windows Native Client.  Unfortunately, this died along with other Oracle Lustre
development projects, and the proprietary code has been sitting unused inside
Oracle since then.
>>>
>>> Similarly, the MacOS native client project started a couple of
years ago, but as yet none of the code has been released.  While some work was
done for a Solaris server, I don''t see much hope in that making
progress at all.
>>>
>>> In both cases, I would be supportive of these efforts if there was
a likelihood of them bearing fruit any time in the future, but so far I
haven''t seen any progress in that direction.  If Oracle could somehow
release the WNC code and/or NRL can overcome the government roadblocks in their
path for releasing the MacOS client (MLC?), it would definitely change the
direction of my thinking.
>>>
>>> As it stands now, we have portability code for Windows/Mac without
any ability to even build the code, let alone test it, so it is just a burden
for Linux.  If there was real value being derived from that code (e.g. actual
users), then it might be an acceptable burden.
>>>
>>>> I know we''re in the minority on this, but I believe
it''s because we are not using Lustre for HPC. We use it for
post-production of TV/film. There are a few others companies in our industry who
have started doing this as well. My point is that, while Lustre currently is
focused on the HPC crowd who seem not to care about Windows/Mac,
Lustre''s maturity is giving it the potential to grow into other uses
besides HPC. I wouldn''t call them general use, but other
high-performance uses. In our industry, where there are a lot of Windows &
Mac workstations that we want to connect to the Lustre storage, the Linux-only
client is a major obstacle to that. I''m sure there are other industries
that would benefit from this.
>>>
>>> Yes, the TV/film industry was one of the prime motivators for WNC. 
There have always been a small number of users from this market, and growth in
this industry (and others) is definitely welcomed.
>>>
>>>> If the community wants to keep Lustre strictly HPC focused and
discourage other industries from joining in, then abandoning the bridge (albeit
the half-built bridge that it is) to Linux/Windows is a good way to do that. If,
on the other hand, there is a desire to get some other industries involved,
perhaps with more resources and contribution coming from them, then I think
it''s important to build on the work that has been done. In that regard,
upstream Linux kernel inclusion seems like a very low priority to me.
>>>
>>> I think even for upstream kernel submission, if it were clear that
the layering in Lustre was for a valid purpose (i.e. existing Mac/Win clients)
then it might be accepted.  As it stands now, we can''t honestly make
that argument to the kernel maintainers.
>>>
>>>> 2012/3/15 <lustre-discuss-request at lists.lustre.org>
>>>> ---------- Forwarded message ----------
>>>> From: Andreas Dilger <adilger at whamcloud.com>
>>>> To: twg at lists.opensfs.org
>>>> Cc: wc-discuss <wc-discuss at whamcloud.com>,
lustre-discuss discuss <lustre-discuss at lists.lustre.org>, Lustre Devel
<lustre-devel at lists.lustre.org>
>>>> Date: Wed, 14 Mar 2012 18:31:29 -0600
>>>> Subject: [Lustre-discuss] Lustre and cross-platform portability
>>>> Whamcloud and EMC are jointly investigating how to be able to
contribute the Lustre client code into the upstream Linux kernel.
>>>>
>>>> As a prerequisite to this, EMC is working to clean up the
Lustre client code to better match the kernel coding style, and one of the
anticipated major obstacles to upstream kernel submission is the heavy use of
code abstraction via libcfs for portability to other operating systems (most
notably MacOS and WinNT, but also for liblustre, and potentially *BSD).
>>>>
>>>> I have no information that the WinNT project will ever be
released by Oracle, and as yet there has not been any code released from the
MacOS port, so the libcfs portability layer is potentially exacting a high cost
in code maintenance and complexity (CLIO being a prime example) for no apparent
benefit.  Similarly, the liblustre client needs a portability layer for
userspace, and suffers from the same apparent lack of interest or users.
>>>>
>>>> I''d like to get some feedback from the Lustre
community about removing the libcfs abstraction entirely, or possibly
restructuring it to look like the Linux kernel API, and having the other
platforms code against it as a Linux portability layer, like ZFS on Linux uses
the Solaris Portability Layer (SPL) to avoid changing the core ZFS code.  A
related topic is whether it would be better to replace all cfs_* functions with
standard Linux kernel functions en-masse, or migrate away from cfs_* functions
slowly?
>>>>
>>>> Also, we''re planning on deprecating the liblustre
client code, due to lack of interest/usage.  The current code is in disrepair,
and we''ve been keeping it around for years without any benefit, and
while I was one of the strongest advocates for keeping it in our back pocket in
case of future needs, I don''t see that materializing in the future.
>>>>
>>>> The liblustre code would be left in the tree for now, in case
someone from the community is interested to get it working and maintain it, and
it may be updated on a best effort basis.  If nobody steps forward to do this
work, the liblustre code would be deleted from the development branch in a year
or so.
>>>>
>>>>
>>>> Unfortunately, after starting this thread, I may not be able to
reply to questions in a timely manner due to vacation.  I look forward to a
thread that concludes with unanimous agreement from all parties. :-)
>>>>
>>>> Cheers, Andreas
>>>> --
>>>> Andreas Dilger                       Whamcloud, Inc.
>>>> Principal Lustre Engineer            http://www.whamcloud.com/
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>>>
>>> Cheers, Andreas
>>> --
>>> Andreas Dilger                       Whamcloud, Inc.
>>> Principal Lustre Engineer            http://www.whamcloud.com/
>>>
>>>
>>>
>>>
>

>Ken, my apologies for this misstatement.  I guess that my faulty memory
>is to blame for the fact that I didn''t recall the MacOS code was
made
>publicly available for download.
No problem.  Back when I gave the talk at LUG the source wasn''t
available
yet due to issues here, but we got that worked out and I was pushing
my changes to a publically available Oracle git repo.  I did send out
email to everyone about that, but I''m sure it was easy to miss.
>I don''t think I''ve ever seen patches sent from you to
either Oracle or
>Whamcloud, and unfortunately nobody on our side has had the bandwidth or
>user demand/funding to be pulling such changes either.
Well, I did actually submit patches to Oracle to start the process of
working out at least the portability issues, but I believe that was
when Oracle started to implode the Lustre group so things sort of
stalled.  I''ll take 75% of the blame for that if we assign 25% to
Larry Ellison :-)
>This isn''t strictly correct.  It would be possible to change the
libcfs
>portability layer to export the same API as the Linux kernel to MacOS
>and Windows.  This would simplify getting the client into the Linux
>kernel, but still allow a native client on MacOS.
Well ... that shifts the burden to cross-platform people basically having
to re-implment the Linux kernel.  For some things, that''s possible
without
too much pain.  For other things, it''s not.

--Ken

>Also fuse client will able to run on any OS have a FUSE porting that is
>any BSD, OpenSolaris, MacOS, in additional to the windows.  That is
>easy way to maintain a single client for many OS.
It is, unfortunately, not quite that simple.

I can''t claim to be a FUSE expert, but I''ve been paying
attention
to it on other platforms.  From what I can tell, FUSE works great
on Linux, but on other platforms the support is iffy.  Also, it''s
not quite implemented the same on other operating systems as it is
on Linux, making porting a Linux FUSE module to other platforms not
trivial; from what I''ve seen, this is due to the Linux filesystem
interface versus the vnode interface used by every Unix except Linux
(and this is part of what makes Lustre hard to port).

I guess what I''m saying is that don''t fall into the underwear
gnomes
trap of thinking:

1) Get liblustre working with FUSE
2) ???
3) Lustre client everywhere!

It might make it easier, but I doubt it will make it easy.

--Ken

On Friday, March 16, 2012 6:11 AM, Gregory Matthews
wrote:
> why bother having a windows client if you lose the performance? We have
> windows based detectors and proprietary windows based analysis software
> that would definitely benefit from higher performance access to lustre
> file systems but replacing existing CIFS servers for no gain seems a bit
> pointless.
>From the perspective of my firm, the benefit of a windows lustre client
(native or fuse-based), even one that performs at 10% or 20% of the linux
client, is the improved scalability that it offers.  Our current solution uses 4
samba gateways per hundred windows servers to achieve acceptable bandwidth, but
that bandwidth can be cut to an unacceptable trickle if a large wave of native
linux clients simultaneously accesses the filesystem.
We have repeatedly asked our Microsoft HPC contacts to intervene to either fund
a native windows client or to get oracle to release the existing one, since
scalable storage is a big hole in the Microsoft HPC toolkit.  Obviously, nothing
has come out of that.  It seems Microsoft is banking on pNFS, eventually working
in this space.

Allen Todd




________________________________

IMPORTANT: The information contained in this email and/or its attachments is
confidential. If you are not the intended recipient, please notify the sender
immediately by reply and immediately delete this message and all its
attachments. Any review, use, reproduction, disclosure or dissemination of this
message or any attachment by an unintended recipient is strictly prohibited.
Neither this message nor any attachment is intended as or should be construed as
an offer, solicitation or recommendation to buy or sell any security or other
financial instrument. Neither the sender, his or her employer nor any of their
respective affiliates makes any warranties as to the completeness or accuracy of
any of the information contained herein or that this message or any of its
attachments is free of viruses.

On Mar 14, 2012, at 6:31 PM, Andreas Dilger wrote:
> Whamcloud and EMC are jointly investigating how to be able to contribute
the Lustre client code into the upstream Linux kernel.
> 
> As a prerequisite to this, EMC is working to clean up the Lustre client
code to better match the kernel coding style, and one of the anticipated major
obstacles to upstream kernel submission is the heavy use of code abstraction via
libcfs for portability to other operating systems (most notably MacOS and WinNT,
but also for liblustre, and potentially *BSD).
> 
> I have no information that the WinNT project will ever be released by
Oracle, and as yet there has not been any code released from the MacOS port, so
the libcfs portability layer is potentially exacting a high cost in code
maintenance and complexity (CLIO being a prime example) for no apparent benefit.
Similarly, the liblustre client needs a portability layer for userspace, and
suffers from the same apparent lack of interest or users.
> 
> I''d like to get some feedback from the Lustre community about
removing the libcfs abstraction entirely, or possibly restructuring it to look
like the Linux kernel API, and having the other platforms code against it as a
Linux portability layer, like ZFS on Linux uses the Solaris Portability Layer
(SPL) to avoid changing the core ZFS code.  A related topic is whether it would
be better to replace all cfs_* functions with standard Linux kernel functions
en-masse, or migrate away from cfs_* functions slowly?
> 
> Also, we''re planning on deprecating the liblustre client code, due
to lack of interest/usage.  The current code is in disrepair, and we''ve
been keeping it around for years without any benefit, and while I was one of the
strongest advocates for keeping it in our back pocket in case of future needs, I
don''t see that materializing in the future.
Relatively true I think, unfortunately. In the past, we provided direct support
of Lustre for processes running on light-weight kernels via liblustre. I am
aware that DOE/NNSA (more than just Sandia) believes light-weight kernels are
the future but it seems that it may be quite a while, yet, before we would be
forced into that choice. I don''t see NNSA having some sort of
heart-attack over your obfuscating liblustre then.

--Lee
> 
> The liblustre code would be left in the tree for now, in case someone from
the community is interested to get it working and maintain it, and it may be
updated on a best effort basis.  If nobody steps forward to do this work, the
liblustre code would be deleted from the development branch in a year or so.
> 
> 
> Unfortunately, after starting this thread, I may not be able to reply to
questions in a timely manner due to vacation.  I look forward to a thread that
concludes with unanimous agreement from all parties. :-)
> 
> Cheers, Andreas
> --
> Andreas Dilger                       Whamcloud, Inc.
> Principal Lustre Engineer            http://www.whamcloud.com/
> 
> 
> 
> 
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel
>

Trust me.
I (with two friends) write  a FUSE client for a FreeBSD in past.
Most issues related to a porting libcfs/libsysio to FreeBSD, but anyway patch
isn''''t huge ~200kb.
I have plan to send that patch in near time, as 2.3 landing window open.



On Mar 16, 2012, at 18:46, Ken Hornstein wrote:
>> Also fuse client will able to run on any OS have a FUSE porting that is
>> any BSD, OpenSolaris, MacOS, in additional to the windows.  That is
>> easy way to maintain a single client for many OS.
> 
> It is, unfortunately, not quite that simple.
> 
> I can''t claim to be a FUSE expert, but I''ve been paying
attention
> to it on other platforms.  From what I can tell, FUSE works great
> on Linux, but on other platforms the support is iffy.  Also, it''s
> not quite implemented the same on other operating systems as it is
> on Linux, making porting a Linux FUSE module to other platforms not
> trivial; from what I''ve seen, this is due to the Linux filesystem
> interface versus the vnode interface used by every Unix except Linux
> (and this is part of what makes Lustre hard to port).
> 
> I guess what I''m saying is that don''t fall into the
underwear gnomes
> trap of thinking:
> 
> 1) Get liblustre working with FUSE
> 2) ???
> 3) Lustre client everywhere!
> 
> It might make it easier, but I doubt it will make it easy.
> 
> --Ken

On Mar 16, 2012, at 8:06 AM, Todd, Allen wrote:
> 
> On Friday, March 16, 2012 6:11 AM, Gregory Matthews wrote:
>> why bother having a windows client if you lose the performance? We have
>> windows based detectors and proprietary windows based analysis software
>> that would definitely benefit from higher performance access to lustre
>> file systems but replacing existing CIFS servers for no gain seems a
bit
>> pointless.
1. Re-exporting Lustre via CIFS or NFS isn''t scalable to very large
numbers of Windows clients
2. Re-exporting Lustre via CIFS or NFS can have coherency problems when multiple
re-exporters are involved
3. Wide-striped file access would likely have greater performance on a native
client than via a re-exported single pipe, for some value of "wide".
> 
>> From the perspective of my firm, the benefit of a windows lustre client
(native or fuse-based), even one that performs at 10% or 20% of the linux
client, is the improved scalability that it offers.  Our current solution uses 4
samba gateways per hundred windows servers to achieve acceptable bandwidth, but
that bandwidth can be cut to an unacceptable trickle if a large wave of native
linux clients simultaneously accesses the filesystem.
> 
> We have repeatedly asked our Microsoft HPC contacts to intervene to either
fund a native windows client or to get oracle to release the existing one, since
scalable storage is a big hole in the Microsoft HPC toolkit.  Obviously, nothing
has come out of that.  It seems Microsoft is banking on pNFS, eventually working
in this space.
> 
> Allen Todd
> 
> 
> 
> 
> ________________________________
> 
> IMPORTANT: The information contained in this email and/or its attachments
is confidential. If you are not the intended recipient, please notify the sender
immediately by reply and immediately delete this message and all its
attachments. Any review, use, reproduction, disclosure or dissemination of this
message or any attachment by an unintended recipient is strictly prohibited.
Neither this message nor any attachment is intended as or should be construed as
an offer, solicitation or recommendation to buy or sell any security or other
financial instrument. Neither the sender, his or her employer nor any of their
respective affiliates makes any warranties as to the completeness or accuracy of
any of the information contained herein or that this message or any of its
attachments is free of viruses.
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

Hi Andreas,

Thank you very much for bringing it up in LUG and getting all these positive
support from community.
> -----Original Message-----
> From: Andreas Dilger [mailto:aedilger at gmail.com]
> Sent: Friday, April 27, 2012 7:48 AM
> To: wc-discuss
> Cc: Lustre Devel; lustre-discuss discuss
> Subject: [wc-discuss] Re: Lustre and cross-platform portability
> 
> To revive this thread, based on discussion at the LUG TWG:
> - there was general consensus that cleaning up the Lustre client
>   (and server) code was very desirable to do
> - migrating libcfs to emulate the Linux kernel APIs is also usable.
>   Ken mentioned that there is relatively little conflict between
>   the Linux kernel and the MacOS kernel, and the same for WinNT, so
>   they could use the same function names as Linux without problems.
I created LU-1346 (http://jira.whamcloud.com/browse/LU-1346) to track libcfs
cleanup work.
> - there was no objection to converting the Lustre code from spaces
>   to tabs.  My proposal was that build/checkpatch.pl could require
>   tabs immediately, and new patches should be submitted with tabs
>   on all new/modified lines (and optionally all lines on small
>   functions to avoid messy formatting).  This will avoid issues
>   with current patches in flight, and also avoid "git annotate"
>   showing the jumbo replace-all-spaces-with-tabs patch for every
>   line in Lustre, and I think a good fraction of lines will be
>   updated in the next 9-12 months or more.  As we approach the
>   actual time for upstream kernel submission is close, then we can
>   make a final effort to clean up remaining lines in idle code
>   (which will also be unlikely to conflict with existing work).
While tabs are the main coding style difference between Lustre and kernel, there
are a few minor change that is needed as well. I think we need to do following
to match kernel coding style:
1. Lustre uses expandtab while kernel requires tabs
2. Lustre has vim syntax rules in most source files, which need to be removed
3. Lustre uses a slightly different comment style, which need to be changed to
kernel style

I created LU-1347 (http://jira.whamcloud.com/browse/LU-1347) to track the coding
style changes.
> 
> There is some hope that the kernel maintainers will not require
> all of the Lustre macros to be removed, but we can deal with this
> on a case-by-case basis.
> 
IMO, we can divide macros to three groups (or more?):
1. Old kernel support macros, kernel maintainers are clear that they
won''t accept it.
2. For macros to mark out server code, kernel maintainers can accept it. But I
think we need to make sure we don''t do it too intrusive.
3. Lustre feature macros, we can convert them to Kconfig macros.

Cheers,
Tao
> Cheers, Andreas
> 
> On 2012-03-14, at 7:31 PM, Andreas Dilger wrote:
> > Whamcloud and EMC are jointly investigating how to be able to
contribute the Lustre client code into
> the upstream Linux kernel.
> >
> > As a prerequisite to this, EMC is working to clean up the Lustre
client code to better match the
> kernel coding style, and one of the anticipated major obstacles to upstream
kernel submission is the
> heavy use of code abstraction via libcfs for portability to other operating
systems (most notably
> MacOS and WinNT, but also for liblustre, and potentially *BSD).
> >
> > I have no information that the WinNT project will ever be released by
Oracle,
> 
> [revised] and the MacOS client needs significant work to update it to the
latest version of Lustre,
> and to get it landed into the Lustre repo,
> 
> > so the libcfs portability layer is potentially exacting a high cost in
code maintenance and
> complexity (CLIO being a prime example) for no apparent benefit. 
Similarly, the liblustre client
> needs a portability layer for userspace, and suffers from the same apparent
lack of interest or users.
> >
> > I''d like to get some feedback from the Lustre community about
removing the libcfs abstraction
> entirely, or possibly restructuring it to look like the Linux kernel API,
and having the other
> platforms code against it as a Linux portability layer, like ZFS on Linux
uses the Solaris Portability
> Layer (SPL) to avoid changing the core ZFS code.  A related topic is
whether it would be better to
> replace all cfs_* functions with standard Linux kernel functions en-masse,
or migrate away from cfs_*
> functions slowly?
> >
> > Also, we''re planning on deprecating the liblustre client
code, due to lack of interest/usage.  The
> current code is in disrepair, and we''ve been keeping it around for
years without any benefit, and
> while I was one of the strongest advocates for keeping it in our back
pocket in case of future needs,
> I don''t see that materializing in the future.
> >
> > The liblustre code would be left in the tree for now, in case someone
from the community is
> interested to get it working and maintain it, and it may be updated on a
best effort basis.  If nobody
> steps forward to do this work, the liblustre code would be deleted from the
development branch in a
> year or so.
> >
> >
> > Unfortunately, after starting this thread, I may not be able to reply
to questions in a timely
> manner due to vacation.  I look forward to a thread that concludes with
unanimous agreement from all
> parties. :-)
> >
> > Cheers, Andreas
> > --
> > Andreas Dilger                       Whamcloud, Inc.
> > Principal Lustre Engineer            http://www.whamcloud.com/
> >
> >
> >
> >
> 
> 
> Cheers, Andreas
> 
> 
> 
> 
>

On 2012-04-26, at 20:23, <tao.peng at emc.com>
wrote:
> Thank you very much for bringing it up in LUG and getting all these
positive support from community.
Tao,
Yes it does look promising. 
>> To revive this thread, based on discussion at the LUG TWG:
>> - there was general consensus that cleaning up the Lustre client
>>  (and server) code was very desirable to do
>> - migrating libcfs to emulate the Linux kernel APIs is also usable.
>>  Ken mentioned that there is relatively little conflict between
>>  the Linux kernel and the MacOS kernel, and the same for WinNT, so
>>  they could use the same function names as Linux without problems.
> I created LU-1346 (http://jira.whamcloud.com/browse/LU-1346) to track
libcfs cleanup work.
OK
>> - there was no objection to converting the Lustre code from spaces
>>  to tabs.  My proposal was that build/checkpatch.pl could require
>>  tabs immediately, and new patches should be submitted with tabs
>>  on all new/modified lines (and optionally all lines on small
>>  functions to avoid messy formatting).  This will avoid issues
>>  with current patches in flight, and also avoid "git
annotate"
>>  showing the jumbo replace-all-spaces-with-tabs patch for every
>>  line in Lustre, and I think a good fraction of lines will be
>>  updated in the next 9-12 months or more.  As we approach the
>>  actual time for upstream kernel submission is close, then we can
>>  make a final effort to clean up remaining lines in idle code
>>  (which will also be unlikely to conflict with existing work).
> While tabs are the main coding style difference between Lustre and kernel,
there are a few minor change that is needed as well. I think we need to do
following to match kernel coding style:
> 1. Lustre uses expandtab while kernel requires tabs
Right.
> 2. Lustre has vim syntax rules in most source files, which need to be
removed
They should be replaced with explicit vim and enacts syntax rules that have the
kernel indent style instead.  If we could get syntax rules that embodied more of
the coding style than just indentation, that would be even better.
> 3. Lustre uses a slightly different comment style, which need to be changed
to kernel style
This is DOxygen style formatting.  I had forgotten about this. We had in the
past used this inline formatting for producing some documentation, but
I''d need to ask about whether there is still a need for this today. In
the meantime, please leave the comment style as-is.

> I created LU-1347 (http://jira.whamcloud.com/browse/LU-1347) to track the
coding style changes.
> 
>> There is some hope that the kernel maintainers will not require
>> all of the Lustre macros to be removed, but we can deal with this
>> on a case-by-case basis.
>> 
> IMO, we can divide macros to three groups (or more?):
> 1. Old kernel support macros, kernel maintainers are clear that they
won''t accept it.
Yes, but we need to maintain this in the external Lustre tree for years after
Lustre would be accepted into mainline, since it would not be in vendor kernels
(which a majority of Lustre users would be using).

For such compat macros we need to make an effort to keep the upstream code as
close as possible to the external tree, so patches have the most chance of
applying.
> 2. For macros to mark out server code, kernel maintainers can accept it.
But I think we need to make sure we don''t do it too intrusive.
Sure, and we also need to make sure the ongoing maintenance effort to keep the
code working is not too much either.

I''m OK with incremental patches that more cleanly split the client and
server code (structures, headers, etc) if that improves the code structure and
readability.
> 3. Lustre feature macros, we can convert them to Kconfig macros.
Sure. Note that some of them may be obsolete, so before you spend too much time
cleaning them up, please post a list to Jira. Maybe some of them can be dropped
entirely.
>> On 2012-03-14, at 7:31 PM, Andreas Dilger wrote:
>>> Whamcloud and EMC are jointly investigating how to be able to
contribute the Lustre client code into
>> the upstream Linux kernel.
>>> 
>>> As a prerequisite to this, EMC is working to clean up the Lustre
client code to better match the
>> kernel coding style, and one of the anticipated major obstacles to
upstream kernel submission is the
>> heavy use of code abstraction via libcfs for portability to other
operating systems (most notably
>> MacOS and WinNT, but also for liblustre, and potentially *BSD).
>>> 
>>> I have no information that the WinNT project will ever be released
by Oracle,
>> 
>> [revised] and the MacOS client needs significant work to update it to
the latest version of Lustre,
>> and to get it landed into the Lustre repo,
>> 
>>> so the libcfs portability layer is potentially exacting a high cost
in code maintenance and
>> complexity (CLIO being a prime example) for no apparent benefit. 
Similarly, the liblustre client
>> needs a portability layer for userspace, and suffers from the same
apparent lack of interest or users.
>>> 
>>> I''d like to get some feedback from the Lustre community
about removing the libcfs abstraction
>> entirely, or possibly restructuring it to look like the Linux kernel
API, and having the other
>> platforms code against it as a Linux portability layer, like ZFS on
Linux uses the Solaris Portability
>> Layer (SPL) to avoid changing the core ZFS code.  A related topic is
whether it would be better to
>> replace all cfs_* functions with standard Linux kernel functions
en-masse, or migrate away from cfs_*
>> functions slowly?
>>> 
>>> Also, we''re planning on deprecating the liblustre client
code, due to lack of interest/usage.  The
>> current code is in disrepair, and we''ve been keeping it around
for years without any benefit, and
>> while I was one of the strongest advocates for keeping it in our back
pocket in case of future needs,
>> I don''t see that materializing in the future.
>>> 
>>> The liblustre code would be left in the tree for now, in case
someone from the community is
>> interested to get it working and maintain it, and it may be updated on
a best effort basis.  If nobody
>> steps forward to do this work, the liblustre code would be deleted from
the development branch in a
>> year or so.
>>> 
>>> 
>>> Unfortunately, after starting this thread, I may not be able to
reply to questions in a timely
>> manner due to vacation.  I look forward to a thread that concludes with
unanimous agreement from all
>> parties. :-)
>>> 
>>> Cheers, Andreas
>>> --
>>> Andreas Dilger                       Whamcloud, Inc.
>>> Principal Lustre Engineer            http://www.whamcloud.com/
>>> 
>>> 
>>> 
>>> 
>> 
>> 
>> Cheers, Andreas
>> 
>> 
>> 
>> 
>> 
>

Hi Andreas,
> -----Original Message-----
> From: Andreas Dilger [mailto:adilger at whamcloud.com]
> Sent: Friday, April 27, 2012 11:54 AM
> To: Peng, Tao
> Cc: <wc-discuss at whamcloud.com>; <lustre-devel at
lists.lustre.org>; <lustre-discuss at lists.lustre.org>
> Subject: Re: [wc-discuss] Re: Lustre and cross-platform portability
> 
> On 2012-04-26, at 20:23, <tao.peng at emc.com> wrote:
> > Thank you very much for bringing it up in LUG and getting all these
positive support from community.
> 
> Tao,
> Yes it does look promising.
> 
> >> To revive this thread, based on discussion at the LUG TWG:
> >> - there was general consensus that cleaning up the Lustre client
> >>  (and server) code was very desirable to do
> >> - migrating libcfs to emulate the Linux kernel APIs is also
usable.
> >>  Ken mentioned that there is relatively little conflict between
> >>  the Linux kernel and the MacOS kernel, and the same for WinNT, so
> >>  they could use the same function names as Linux without problems.
> > I created LU-1346 (http://jira.whamcloud.com/browse/LU-1346) to track
libcfs cleanup work.
> 
> OK
> 
> >> - there was no objection to converting the Lustre code from spaces
> >>  to tabs.  My proposal was that build/checkpatch.pl could require
> >>  tabs immediately, and new patches should be submitted with tabs
> >>  on all new/modified lines (and optionally all lines on small
> >>  functions to avoid messy formatting).  This will avoid issues
> >>  with current patches in flight, and also avoid "git
annotate"
> >>  showing the jumbo replace-all-spaces-with-tabs patch for every
> >>  line in Lustre, and I think a good fraction of lines will be
> >>  updated in the next 9-12 months or more.  As we approach the
> >>  actual time for upstream kernel submission is close, then we can
> >>  make a final effort to clean up remaining lines in idle code
> >>  (which will also be unlikely to conflict with existing work).
> > While tabs are the main coding style difference between Lustre and
kernel, there are a few minor
> change that is needed as well. I think we need to do following to match
kernel coding style:
> > 1. Lustre uses expandtab while kernel requires tabs
> 
> Right.
> 
> > 2. Lustre has vim syntax rules in most source files, which need to be
removed
> 
> They should be replaced with explicit vim and enacts syntax rules that have
the kernel indent style
> instead.  If we could get syntax rules that embodied more of the coding
style than just indentation,
> that would be even better.
> 
But we do need to remove them before submitting to kernel, right? And if we
enforce checkpatch.pl on every patch applied, IMHO there is not much benefit to
have syntax rules on every file, not to mention that it is explicitly forbidden
in kernel coding style (Documentation/CodingStyle, Chapter 18:  Editor modelines
and other cruft).

BTW, instead of just enabling tabs, how about changing checkpatch.pl to latest
kernel version to make sure all future patches follow kernel coding styles?
> > 3. Lustre uses a slightly different comment style, which need to be
changed to kernel style
> 
> This is DOxygen style formatting.  I had forgotten about this. We had in
the past used this inline
> formatting for producing some documentation, but I''d need to ask
about whether there is still a need
> for this today. In the meantime, please leave the comment style as-is.
> 
OK.
> 
> > I created LU-1347 (http://jira.whamcloud.com/browse/LU-1347) to track
the coding style changes.
> >
> >> There is some hope that the kernel maintainers will not require
> >> all of the Lustre macros to be removed, but we can deal with this
> >> on a case-by-case basis.
> >>
> > IMO, we can divide macros to three groups (or more?):
> > 1. Old kernel support macros, kernel maintainers are clear that they
won''t accept it.
> 
> Yes, but we need to maintain this in the external Lustre tree for years
after Lustre would be accepted
> into mainline, since it would not be in vendor kernels (which a majority of
Lustre users would be
> using).
> 
> For such compat macros we need to make an effort to keep the upstream code
as close as possible to the
> external tree, so patches have the most chance of applying.
> 
I agree. We should minimize maintenance effort for it. And as you suggested, we
can put as many of these compat macros into places like linux_compat.h as
possible and have Lustre code use latest kernel APIs, so that most maintenance
effort for old kernel support would be to keep linux_compat.h uptodate. For
compact macros that cannot be cleaned up this way, we will have to pay the extra
efforts. And of course the cleanup will be an incremental process and macros
will be dealt with in a case-by-case basis.
> > 2. For macros to mark out server code, kernel maintainers can accept
it. But I think we need to make
> sure we don''t do it too intrusive.
> 
> Sure, and we also need to make sure the ongoing maintenance effort to keep
the code working is not too
> much either.
> 
> I''m OK with incremental patches that more cleanly split the client
and server code (structures,
> headers, etc) if that improves the code structure and readability.
> 
I agree that we can do some incremental patches to split client and server code.
But I hope we only do it when it is trivial and simple. IMHO if we want to
entirely split client/server code, it will be large code structure change. Since
kernel maintainers already agreed on HAVE_SERVER_SUPPORT, how about we keep
going that way at first? And we can make some split wherever it is simple and
clear. And we will try to make code structure as clear/readable as possible.
Then when we summit code for review, if kernel maintainers still don''t
like it, we do the large restructuring. Does it make sense?
> > 3. Lustre feature macros, we can convert them to Kconfig macros.
> 
> Sure. Note that some of them may be obsolete, so before you spend too much
time cleaning them up,
> please post a list to Jira. Maybe some of them can be dropped entirely.
> 
Thanks. I will do as you suggested when it comes to converting them to Kconfig
macros.

Cheers,
Tao
> >> On 2012-03-14, at 7:31 PM, Andreas Dilger wrote:
> >>> Whamcloud and EMC are jointly investigating how to be able to
contribute the Lustre client code
> into
> >> the upstream Linux kernel.
> >>>
> >>> As a prerequisite to this, EMC is working to clean up the
Lustre client code to better match the
> >> kernel coding style, and one of the anticipated major obstacles to
upstream kernel submission is
> the
> >> heavy use of code abstraction via libcfs for portability to other
operating systems (most notably
> >> MacOS and WinNT, but also for liblustre, and potentially *BSD).
> >>>
> >>> I have no information that the WinNT project will ever be
released by Oracle,
> >>
> >> [revised] and the MacOS client needs significant work to update it
to the latest version of Lustre,
> >> and to get it landed into the Lustre repo,
> >>
> >>> so the libcfs portability layer is potentially exacting a high
cost in code maintenance and
> >> complexity (CLIO being a prime example) for no apparent benefit. 
Similarly, the liblustre client
> >> needs a portability layer for userspace, and suffers from the same
apparent lack of interest or
> users.
> >>>
> >>> I''d like to get some feedback from the Lustre
community about removing the libcfs abstraction
> >> entirely, or possibly restructuring it to look like the Linux
kernel API, and having the other
> >> platforms code against it as a Linux portability layer, like ZFS
on Linux uses the Solaris
> Portability
> >> Layer (SPL) to avoid changing the core ZFS code.  A related topic
is whether it would be better to
> >> replace all cfs_* functions with standard Linux kernel functions
en-masse, or migrate away from
> cfs_*
> >> functions slowly?
> >>>
> >>> Also, we''re planning on deprecating the liblustre
client code, due to lack of interest/usage.  The
> >> current code is in disrepair, and we''ve been keeping it
around for years without any benefit, and
> >> while I was one of the strongest advocates for keeping it in our
back pocket in case of future
> needs,
> >> I don''t see that materializing in the future.
> >>>
> >>> The liblustre code would be left in the tree for now, in case
someone from the community is
> >> interested to get it working and maintain it, and it may be
updated on a best effort basis.  If
> nobody
> >> steps forward to do this work, the liblustre code would be deleted
from the development branch in a
> >> year or so.
> >>>
> >>>
> >>> Unfortunately, after starting this thread, I may not be able
to reply to questions in a timely
> >> manner due to vacation.  I look forward to a thread that concludes
with unanimous agreement from
> all
> >> parties. :-)
> >>>
> >>> Cheers, Andreas
> >>> --
> >>> Andreas Dilger                       Whamcloud, Inc.
> >>> Principal Lustre Engineer            http://www.whamcloud.com/
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >> Cheers, Andreas
> >>
> >>
> >>
> >>
> >>
> >

Tao,Andreas,all,

What is your plan in test/test framework changes from the point of view
of integration to kernel? As i know, kernel.org has his own test
infrastructure and his own test framework.

I''m sorry if it''s incorrect place for this question.

Thanks,
	Roman

On 04/27/2012 02:15 PM, tao.peng at emc.com wrote:
> Hi Andreas,
> 
>> -----Original Message-----
>> From: Andreas Dilger [mailto:adilger at whamcloud.com]
>> Sent: Friday, April 27, 2012 11:54 AM
>> To: Peng, Tao
>> Cc: <wc-discuss at whamcloud.com>; <lustre-devel at
lists.lustre.org>; <lustre-discuss at lists.lustre.org>
>> Subject: Re: [wc-discuss] Re: Lustre and cross-platform portability
>>
>> On 2012-04-26, at 20:23, <tao.peng at emc.com> wrote:
>>> Thank you very much for bringing it up in LUG and getting all these
positive support from community.
>>
>> Tao,
>> Yes it does look promising.
>>
>>>> To revive this thread, based on discussion at the LUG TWG:
>>>> - there was general consensus that cleaning up the Lustre
client
>>>>  (and server) code was very desirable to do
>>>> - migrating libcfs to emulate the Linux kernel APIs is also
usable.
>>>>  Ken mentioned that there is relatively little conflict between
>>>>  the Linux kernel and the MacOS kernel, and the same for WinNT,
so
>>>>  they could use the same function names as Linux without
problems.
>>> I created LU-1346 (http://jira.whamcloud.com/browse/LU-1346) to
track libcfs cleanup work.
>>
>> OK
>>
>>>> - there was no objection to converting the Lustre code from
spaces
>>>>  to tabs.  My proposal was that build/checkpatch.pl could
require
>>>>  tabs immediately, and new patches should be submitted with
tabs
>>>>  on all new/modified lines (and optionally all lines on small
>>>>  functions to avoid messy formatting).  This will avoid issues
>>>>  with current patches in flight, and also avoid "git
annotate"
>>>>  showing the jumbo replace-all-spaces-with-tabs patch for every
>>>>  line in Lustre, and I think a good fraction of lines will be
>>>>  updated in the next 9-12 months or more.  As we approach the
>>>>  actual time for upstream kernel submission is close, then we
can
>>>>  make a final effort to clean up remaining lines in idle code
>>>>  (which will also be unlikely to conflict with existing work).
>>> While tabs are the main coding style difference between Lustre and
kernel, there are a few minor
>> change that is needed as well. I think we need to do following to match
kernel coding style:
>>> 1. Lustre uses expandtab while kernel requires tabs
>>
>> Right.
>>
>>> 2. Lustre has vim syntax rules in most source files, which need to
be removed
>>
>> They should be replaced with explicit vim and enacts syntax rules that
have the kernel indent style
>> instead.  If we could get syntax rules that embodied more of the coding
style than just indentation,
>> that would be even better.
>>
> But we do need to remove them before submitting to kernel, right? And if we
enforce checkpatch.pl on every patch applied, IMHO there is not much benefit to
have syntax rules on every file, not to mention that it is explicitly forbidden
in kernel coding style (Documentation/CodingStyle, Chapter 18:  Editor modelines
and other cruft).
> 
> BTW, instead of just enabling tabs, how about changing checkpatch.pl to
latest kernel version to make sure all future patches follow kernel coding
styles?
> 
>>> 3. Lustre uses a slightly different comment style, which need to be
changed to kernel style
>>
>> This is DOxygen style formatting.  I had forgotten about this. We had
in the past used this inline
>> formatting for producing some documentation, but I''d need to
ask about whether there is still a need
>> for this today. In the meantime, please leave the comment style as-is.
>>
> OK.
> 
>>
>>> I created LU-1347 (http://jira.whamcloud.com/browse/LU-1347) to
track the coding style changes.
>>>
>>>> There is some hope that the kernel maintainers will not require
>>>> all of the Lustre macros to be removed, but we can deal with
this
>>>> on a case-by-case basis.
>>>>
>>> IMO, we can divide macros to three groups (or more?):
>>> 1. Old kernel support macros, kernel maintainers are clear that
they won''t accept it.
>>
>> Yes, but we need to maintain this in the external Lustre tree for years
after Lustre would be accepted
>> into mainline, since it would not be in vendor kernels (which a
majority of Lustre users would be
>> using).
>>
>> For such compat macros we need to make an effort to keep the upstream
code as close as possible to the
>> external tree, so patches have the most chance of applying.
>>
> I agree. We should minimize maintenance effort for it. And as you
suggested, we can put as many of these compat macros into places like
linux_compat.h as possible and have Lustre code use latest kernel APIs, so that
most maintenance effort for old kernel support would be to keep linux_compat.h
uptodate. For compact macros that cannot be cleaned up this way, we will have to
pay the extra efforts. And of course the cleanup will be an incremental process
and macros will be dealt with in a case-by-case basis.
> 
>>> 2. For macros to mark out server code, kernel maintainers can
accept it. But I think we need to make
>> sure we don''t do it too intrusive.
>>
>> Sure, and we also need to make sure the ongoing maintenance effort to
keep the code working is not too
>> much either.
>>
>> I''m OK with incremental patches that more cleanly split the
client and server code (structures,
>> headers, etc) if that improves the code structure and readability.
>>
> I agree that we can do some incremental patches to split client and server
code. But I hope we only do it when it is trivial and simple. IMHO if we want to
entirely split client/server code, it will be large code structure change. Since
kernel maintainers already agreed on HAVE_SERVER_SUPPORT, how about we keep
going that way at first? And we can make some split wherever it is simple and
clear. And we will try to make code structure as clear/readable as possible.
Then when we summit code for review, if kernel maintainers still don''t
like it, we do the large restructuring. Does it make sense?
> 
>>> 3. Lustre feature macros, we can convert them to Kconfig macros.
>>
>> Sure. Note that some of them may be obsolete, so before you spend too
much time cleaning them up,
>> please post a list to Jira. Maybe some of them can be dropped entirely.
>>
> Thanks. I will do as you suggested when it comes to converting them to
Kconfig macros.
> 
> Cheers,
> Tao
> 
>>>> On 2012-03-14, at 7:31 PM, Andreas Dilger wrote:
>>>>> Whamcloud and EMC are jointly investigating how to be able
to contribute the Lustre client code
>> into
>>>> the upstream Linux kernel.
>>>>>
>>>>> As a prerequisite to this, EMC is working to clean up the
Lustre client code to better match the
>>>> kernel coding style, and one of the anticipated major obstacles
to upstream kernel submission is
>> the
>>>> heavy use of code abstraction via libcfs for portability to
other operating systems (most notably
>>>> MacOS and WinNT, but also for liblustre, and potentially *BSD).
>>>>>
>>>>> I have no information that the WinNT project will ever be
released by Oracle,
>>>>
>>>> [revised] and the MacOS client needs significant work to update
it to the latest version of Lustre,
>>>> and to get it landed into the Lustre repo,
>>>>
>>>>> so the libcfs portability layer is potentially exacting a
high cost in code maintenance and
>>>> complexity (CLIO being a prime example) for no apparent
benefit.  Similarly, the liblustre client
>>>> needs a portability layer for userspace, and suffers from the
same apparent lack of interest or
>> users.
>>>>>
>>>>> I''d like to get some feedback from the Lustre
community about removing the libcfs abstraction
>>>> entirely, or possibly restructuring it to look like the Linux
kernel API, and having the other
>>>> platforms code against it as a Linux portability layer, like
ZFS on Linux uses the Solaris
>> Portability
>>>> Layer (SPL) to avoid changing the core ZFS code.  A related
topic is whether it would be better to
>>>> replace all cfs_* functions with standard Linux kernel
functions en-masse, or migrate away from
>> cfs_*
>>>> functions slowly?
>>>>>
>>>>> Also, we''re planning on deprecating the liblustre
client code, due to lack of interest/usage.  The
>>>> current code is in disrepair, and we''ve been keeping
it around for years without any benefit, and
>>>> while I was one of the strongest advocates for keeping it in
our back pocket in case of future
>> needs,
>>>> I don''t see that materializing in the future.
>>>>>
>>>>> The liblustre code would be left in the tree for now, in
case someone from the community is
>>>> interested to get it working and maintain it, and it may be
updated on a best effort basis.  If
>> nobody
>>>> steps forward to do this work, the liblustre code would be
deleted from the development branch in a
>>>> year or so.
>>>>>
>>>>>
>>>>> Unfortunately, after starting this thread, I may not be
able to reply to questions in a timely
>>>> manner due to vacation.  I look forward to a thread that
concludes with unanimous agreement from
>> all
>>>> parties. :-)
>>>>>
>>>>> Cheers, Andreas
>>>>> --
>>>>> Andreas Dilger                       Whamcloud, Inc.
>>>>> Principal Lustre Engineer           
http://www.whamcloud.com/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> Cheers, Andreas
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

Hi Roman,

Not sure if I misunderstand your question, but we won''t push
lustre/tests/* files to kernel. We only push client code and any tests will be
executed out side of kernel.

Cheers,
Tao
________________________________________
From: Roman Grigoryev [Roman_Grigoryev at xyratex.com]
Sent: Friday, April 27, 2012 6:25 PM
To: Peng, Tao
Cc: adilger at whamcloud.com; wc-discuss at whamcloud.com; lustre-discuss at
lists.lustre.org; lustre-devel at lists.lustre.org
Subject: Re: [Lustre-discuss] [wc-discuss] Re: Lustre and cross-platform
portability

Tao,Andreas,all,

What is your plan in test/test framework changes from the point of view
of integration to kernel? As i know, kernel.org has his own test
infrastructure and his own test framework.

I''m sorry if it''s incorrect place for this question.

Thanks,
        Roman

On 04/27/2012 02:15 PM, tao.peng at emc.com wrote:
> Hi Andreas,
>
>> -----Original Message-----
>> From: Andreas Dilger [mailto:adilger at whamcloud.com]
>> Sent: Friday, April 27, 2012 11:54 AM
>> To: Peng, Tao
>> Cc: <wc-discuss at whamcloud.com>; <lustre-devel at
lists.lustre.org>; <lustre-discuss at lists.lustre.org>
>> Subject: Re: [wc-discuss] Re: Lustre and cross-platform portability
>>
>> On 2012-04-26, at 20:23, <tao.peng at emc.com> wrote:
>>> Thank you very much for bringing it up in LUG and getting all these
positive support from community.
>>
>> Tao,
>> Yes it does look promising.
>>
>>>> To revive this thread, based on discussion at the LUG TWG:
>>>> - there was general consensus that cleaning up the Lustre
client
>>>>  (and server) code was very desirable to do
>>>> - migrating libcfs to emulate the Linux kernel APIs is also
usable.
>>>>  Ken mentioned that there is relatively little conflict between
>>>>  the Linux kernel and the MacOS kernel, and the same for WinNT,
so
>>>>  they could use the same function names as Linux without
problems.
>>> I created LU-1346 (http://jira.whamcloud.com/browse/LU-1346) to
track libcfs cleanup work.
>>
>> OK
>>
>>>> - there was no objection to converting the Lustre code from
spaces
>>>>  to tabs.  My proposal was that build/checkpatch.pl could
require
>>>>  tabs immediately, and new patches should be submitted with
tabs
>>>>  on all new/modified lines (and optionally all lines on small
>>>>  functions to avoid messy formatting).  This will avoid issues
>>>>  with current patches in flight, and also avoid "git
annotate"
>>>>  showing the jumbo replace-all-spaces-with-tabs patch for every
>>>>  line in Lustre, and I think a good fraction of lines will be
>>>>  updated in the next 9-12 months or more.  As we approach the
>>>>  actual time for upstream kernel submission is close, then we
can
>>>>  make a final effort to clean up remaining lines in idle code
>>>>  (which will also be unlikely to conflict with existing work).
>>> While tabs are the main coding style difference between Lustre and
kernel, there are a few minor
>> change that is needed as well. I think we need to do following to match
kernel coding style:
>>> 1. Lustre uses expandtab while kernel requires tabs
>>
>> Right.
>>
>>> 2. Lustre has vim syntax rules in most source files, which need to
be removed
>>
>> They should be replaced with explicit vim and enacts syntax rules that
have the kernel indent style
>> instead.  If we could get syntax rules that embodied more of the coding
style than just indentation,
>> that would be even better.
>>
> But we do need to remove them before submitting to kernel, right? And if we
enforce checkpatch.pl on every patch applied, IMHO there is not much benefit to
have syntax rules on every file, not to mention that it is explicitly forbidden
in kernel coding style (Documentation/CodingStyle, Chapter 18:  Editor modelines
and other cruft).
>
> BTW, instead of just enabling tabs, how about changing checkpatch.pl to
latest kernel version to make sure all future patches follow kernel coding
styles?
>
>>> 3. Lustre uses a slightly different comment style, which need to be
changed to kernel style
>>
>> This is DOxygen style formatting.  I had forgotten about this. We had
in the past used this inline
>> formatting for producing some documentation, but I''d need to
ask about whether there is still a need
>> for this today. In the meantime, please leave the comment style as-is.
>>
> OK.
>
>>
>>> I created LU-1347 (http://jira.whamcloud.com/browse/LU-1347) to
track the coding style changes.
>>>
>>>> There is some hope that the kernel maintainers will not require
>>>> all of the Lustre macros to be removed, but we can deal with
this
>>>> on a case-by-case basis.
>>>>
>>> IMO, we can divide macros to three groups (or more?):
>>> 1. Old kernel support macros, kernel maintainers are clear that
they won''t accept it.
>>
>> Yes, but we need to maintain this in the external Lustre tree for years
after Lustre would be accepted
>> into mainline, since it would not be in vendor kernels (which a
majority of Lustre users would be
>> using).
>>
>> For such compat macros we need to make an effort to keep the upstream
code as close as possible to the
>> external tree, so patches have the most chance of applying.
>>
> I agree. We should minimize maintenance effort for it. And as you
suggested, we can put as many of these compat macros into places like
linux_compat.h as possible and have Lustre code use latest kernel APIs, so that
most maintenance effort for old kernel support would be to keep linux_compat.h
uptodate. For compact macros that cannot be cleaned up this way, we will have to
pay the extra efforts. And of course the cleanup will be an incremental process
and macros will be dealt with in a case-by-case basis.
>
>>> 2. For macros to mark out server code, kernel maintainers can
accept it. But I think we need to make
>> sure we don''t do it too intrusive.
>>
>> Sure, and we also need to make sure the ongoing maintenance effort to
keep the code working is not too
>> much either.
>>
>> I''m OK with incremental patches that more cleanly split the
client and server code (structures,
>> headers, etc) if that improves the code structure and readability.
>>
> I agree that we can do some incremental patches to split client and server
code. But I hope we only do it when it is trivial and simple. IMHO if we want to
entirely split client/server code, it will be large code structure change. Since
kernel maintainers already agreed on HAVE_SERVER_SUPPORT, how about we keep
going that way at first? And we can make some split wherever it is simple and
clear. And we will try to make code structure as clear/readable as possible.
Then when we summit code for review, if kernel maintainers still don''t
like it, we do the large restructuring. Does it make sense?
>
>>> 3. Lustre feature macros, we can convert them to Kconfig macros.
>>
>> Sure. Note that some of them may be obsolete, so before you spend too
much time cleaning them up,
>> please post a list to Jira. Maybe some of them can be dropped entirely.
>>
> Thanks. I will do as you suggested when it comes to converting them to
Kconfig macros.
>
> Cheers,
> Tao
>
>>>> On 2012-03-14, at 7:31 PM, Andreas Dilger wrote:
>>>>> Whamcloud and EMC are jointly investigating how to be able
to contribute the Lustre client code
>> into
>>>> the upstream Linux kernel.
>>>>>
>>>>> As a prerequisite to this, EMC is working to clean up the
Lustre client code to better match the
>>>> kernel coding style, and one of the anticipated major obstacles
to upstream kernel submission is
>> the
>>>> heavy use of code abstraction via libcfs for portability to
other operating systems (most notably
>>>> MacOS and WinNT, but also for liblustre, and potentially *BSD).
>>>>>
>>>>> I have no information that the WinNT project will ever be
released by Oracle,
>>>>
>>>> [revised] and the MacOS client needs significant work to update
it to the latest version of Lustre,
>>>> and to get it landed into the Lustre repo,
>>>>
>>>>> so the libcfs portability layer is potentially exacting a
high cost in code maintenance and
>>>> complexity (CLIO being a prime example) for no apparent
benefit.  Similarly, the liblustre client
>>>> needs a portability layer for userspace, and suffers from the
same apparent lack of interest or
>> users.
>>>>>
>>>>> I''d like to get some feedback from the Lustre
community about removing the libcfs abstraction
>>>> entirely, or possibly restructuring it to look like the Linux
kernel API, and having the other
>>>> platforms code against it as a Linux portability layer, like
ZFS on Linux uses the Solaris
>> Portability
>>>> Layer (SPL) to avoid changing the core ZFS code.  A related
topic is whether it would be better to
>>>> replace all cfs_* functions with standard Linux kernel
functions en-masse, or migrate away from
>> cfs_*
>>>> functions slowly?
>>>>>
>>>>> Also, we''re planning on deprecating the liblustre
client code, due to lack of interest/usage.  The
>>>> current code is in disrepair, and we''ve been keeping
it around for years without any benefit, and
>>>> while I was one of the strongest advocates for keeping it in
our back pocket in case of future
>> needs,
>>>> I don''t see that materializing in the future.
>>>>>
>>>>> The liblustre code would be left in the tree for now, in
case someone from the community is
>>>> interested to get it working and maintain it, and it may be
updated on a best effort basis.  If
>> nobody
>>>> steps forward to do this work, the liblustre code would be
deleted from the development branch in a
>>>> year or so.
>>>>>
>>>>>
>>>>> Unfortunately, after starting this thread, I may not be
able to reply to questions in a timely
>>>> manner due to vacation.  I look forward to a thread that
concludes with unanimous agreement from
>> all
>>>> parties. :-)
>>>>>
>>>>> Cheers, Andreas
>>>>> --
>>>>> Andreas Dilger                       Whamcloud, Inc.
>>>>> Principal Lustre Engineer           
http://www.whamcloud.com/
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>>
>>>> Cheers, Andreas
>>>>
>>>>
>>>>
>>>>
>>>>
>>>
>
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-discuss

On 2012-04-27, at 4:15 AM, <tao.peng at emc.com> <tao.peng at
emc.com> wrote:
> Andreas Dilger <adilger at whamcloud.com> wrote:
>> On 2012-04-26, at 20:23, <tao.peng at emc.com> wrote:
>>> 2. Lustre has vim syntax rules in most source files, which need
>>> to be removed
>> 
>> They should be replaced with explicit vim and enacts syntax rules that
have the kernel indent style instead.  If we could get syntax rules that
embodied more of the coding style than just indentation, that would be even
better.
>> 
> But we do need to remove them before submitting to kernel, right? And if we
enforce checkpatch.pl on every patch applied, IMHO there is not much benefit to
have syntax rules on every file, not to mention that it is explicitly forbidden
in kernel coding style (Documentation/CodingStyle, Chapter 18:  Editor modelines
and other cruft).
> 
> BTW, instead of just enabling tabs, how about changing checkpatch.pl to
latest kernel version to make sure all future patches follow kernel coding
styles?
The Lustre checkpatch.pl is already based on the kernel one, but with some small
modifications.  It will default to checking for spaces vs. tabs, default to
"--no-tree", does not require signoffs (since this is added at commit
time after the patch is checked).  One other change is to allow no spaces after
commas in function parameters if the line is 79 or 80 columns long.  That avoids
a line wrap for just a couple of characters.

I have no objection to updating to a newer version of checkpatch.pl if it
improves the checking.  Please run it against
>>> IMO, we can divide macros to three groups (or more?):
>>> 1. Old kernel support macros, kernel maintainers are clear that
they won''t accept it.
> 
>>> 2. For macros to mark out server code, kernel maintainers can
accept it. But I think we need to make sure we don''t do it too
intrusive.
>> 
>> Sure, and we also need to make sure the ongoing maintenance effort to
keep the code working is not too much either.
>> 
>> I''m OK with incremental patches that more cleanly split the
client and server code (structures, headers, etc) if that improves the code
structure and readability.
> 
> I agree that we can do some incremental patches to split client and server
code. But I hope we only do it when it is trivial and simple. IMHO if we want to
entirely split client/server code, it will be large code structure change. Since
kernel maintainers already agreed on HAVE_SERVER_SUPPORT, how about we keep
going that way at first?
By all means, we can stick with HAVE_SERVER_SUPPORT for now.  I was just
commenting that I''m not against other changes if they improve the code
in the long run.

Cheers, Andreas
--
Andreas Dilger                       Whamcloud, Inc.
Principal Lustre Engineer            http://www.whamcloud.com/

On Apr 27, 2012, at 7:48 AM, Andreas Dilger wrote:
> 
> - there was no objection to converting the Lustre code from spaces
>  to tabs.  My proposal was that build/checkpatch.pl could require
>  tabs immediately, and new patches should be submitted with tabs
>  on all new/modified lines (and optionally all lines on small
>  functions to avoid messy formatting).  This will avoid issues
>  with current patches in flight, and also avoid "git annotate"
>  showing the jumbo replace-all-spaces-with-tabs patch for every
>  line in Lustre, and I think a good fraction of lines will be
>  updated in the next 9-12 months or more.  As we approach the
>  actual time for upstream kernel submission is close, then we can
>  make a final effort to clean up remaining lines in idle code
>  (which will also be unlikely to conflict with existing work).
Lustre also requires to align the first alphabet of members while defining a
structure, but linux kernel only aligns any first character (for example, * for
pointer), then which way we should choose?
we probably should update our coding guidelines ASAP.

Liang

-------------- next part --------------
An HTML attachment was scrubbed...
URL:
http://lists.lustre.org/pipermail/lustre-discuss/attachments/20120428/8dadc549/attachment.html

On Sat, Apr 28, 2012 at 4:23 AM, Andreas Dilger <adilger at whamcloud.com>
wrote:
> On 2012-04-27, at 4:15 AM, <tao.peng at emc.com> <tao.peng at
emc.com> wrote:
>> Andreas Dilger <adilger at whamcloud.com> wrote:
>>> On 2012-04-26, at 20:23, <tao.peng at emc.com> wrote:
>>>> 2. Lustre has vim syntax rules in most source files, which need
>>>> to be removed
>>>
>>> They should be replaced with explicit vim and enacts syntax rules
that have the kernel indent style instead. ?If we could get syntax rules that
embodied more of the coding style than just indentation, that would be even
better.
>>>
>> But we do need to remove them before submitting to kernel, right? And
if we enforce checkpatch.pl on every patch applied, IMHO there is not much
benefit to have syntax rules on every file, not to mention that it is explicitly
forbidden in kernel coding style (Documentation/CodingStyle, Chapter 18: ?Editor
modelines and other cruft).
>>
>> BTW, instead of just enabling tabs, how about changing checkpatch.pl to
latest kernel version to make sure all future patches follow kernel coding
styles?
>
> The Lustre checkpatch.pl is already based on the kernel one, but with some
small modifications. ?It will default to checking for spaces vs. tabs, default
to "--no-tree", does not require signoffs (since this is added at
commit time after the patch is checked). ?One other change is to allow no spaces
after commas in function parameters if the line is 79 or 80 columns long. ?That
avoids a line wrap for just a couple of characters.
>
> I have no objection to updating to a newer version of checkpatch.pl if it
improves the checking. ?Please run it against
>
I hope we can use latest kernel version of checkpatch.pl so that it is
more likely future patches can be applied to both kernel and Whamcloud
tree.

I did a diff against kernel checkpatch.pl and it creates ~2k lines of
change. So I think it is worth updating Lustre version of
checkpatch.pl.

I submitted http://review.whamcloud.com/2610 for the change. Please
help to review. Thank you.
>>>> IMO, we can divide macros to three groups (or more?):
>>>> 1. Old kernel support macros, kernel maintainers are clear that
they won''t accept it.
>>
>>>> 2. For macros to mark out server code, kernel maintainers can
accept it. But I think we need to make sure we don''t do it too
intrusive.
>>>
>>> Sure, and we also need to make sure the ongoing maintenance effort
to keep the code working is not too much either.
>>>
>>> I''m OK with incremental patches that more cleanly split
the client and server code (structures, headers, etc) if that improves the code
structure and readability.
>>
>> I agree that we can do some incremental patches to split client and
server code. But I hope we only do it when it is trivial and simple. IMHO if we
want to entirely split client/server code, it will be large code structure
change. Since kernel maintainers already agreed on HAVE_SERVER_SUPPORT, how
about we keep going that way at first?
>
> By all means, we can stick with HAVE_SERVER_SUPPORT for now. ?I was just
commenting that I''m not against other changes if they improve the code
in the long run.
>
Thanks. We will do the split whenever it is easy. :)

Best,
Tao
> Cheers, Andreas
> --
> Andreas Dilger ? ? ? ? ? ? ? ? ? ? ? Whamcloud, Inc.
> Principal Lustre Engineer ? ? ? ? ? ?http://www.whamcloud.com/
>
>
>
>
> _______________________________________________
> Lustre-devel mailing list
> Lustre-devel at lists.lustre.org
> http://lists.lustre.org/mailman/listinfo/lustre-devel

Hi Tao,

I''m not know how linux developers approve patches. But, logically,
because Lustre is big enough project and use kernel subsystems then it
should be tested after kernel changes on kernel side too. In  other case
(without testing) we can observer situation when Lustre built ok but
doesn''t work.

I don''t know goal of adding Lustre to kernel and, possible,this
situation could be acceptable from your point of view.

Thanks,
	Roman

On 04/27/2012 04:33 PM, tao.peng at emc.com wrote:
> Hi Roman,
> 
> Not sure if I misunderstand your question, but we won''t push
lustre/tests/* files to kernel.
>We only push client code and any tests will be executed out side of kernel
> 
> Cheers,
> Tao
> ________________________________________
> From: Roman Grigoryev [Roman_Grigoryev at xyratex.com]
> Sent: Friday, April 27, 2012 6:25 PM
> To: Peng, Tao
> Cc: adilger at whamcloud.com; wc-discuss at whamcloud.com; lustre-discuss
at lists.lustre.org; lustre-devel at lists.lustre.org
> Subject: Re: [Lustre-discuss] [wc-discuss] Re: Lustre and cross-platform
portability
> 
> Tao,Andreas,all,
> 
> What is your plan in test/test framework changes from the point of view
> of integration to kernel? As i know, kernel.org has his own test
> infrastructure and his own test framework.
> 
> I''m sorry if it''s incorrect place for this question.
> 
> Thanks,
>         Roman
> 
> On 04/27/2012 02:15 PM, tao.peng at emc.com wrote:
>> Hi Andreas,
>>
>>> -----Original Message-----
>>> From: Andreas Dilger [mailto:adilger at whamcloud.com]
>>> Sent: Friday, April 27, 2012 11:54 AM
>>> To: Peng, Tao
>>> Cc: <wc-discuss at whamcloud.com>; <lustre-devel at
lists.lustre.org>; <lustre-discuss at lists.lustre.org>
>>> Subject: Re: [wc-discuss] Re: Lustre and cross-platform portability
>>>
>>> On 2012-04-26, at 20:23, <tao.peng at emc.com> wrote:
>>>> Thank you very much for bringing it up in LUG and getting all
these positive support from community.
>>>
>>> Tao,
>>> Yes it does look promising.
>>>
>>>>> To revive this thread, based on discussion at the LUG TWG:
>>>>> - there was general consensus that cleaning up the Lustre
client
>>>>>  (and server) code was very desirable to do
>>>>> - migrating libcfs to emulate the Linux kernel APIs is also
usable.
>>>>>  Ken mentioned that there is relatively little conflict
between
>>>>>  the Linux kernel and the MacOS kernel, and the same for
WinNT, so
>>>>>  they could use the same function names as Linux without
problems.
>>>> I created LU-1346 (http://jira.whamcloud.com/browse/LU-1346) to
track libcfs cleanup work.
>>>
>>> OK
>>>
>>>>> - there was no objection to converting the Lustre code from
spaces
>>>>>  to tabs.  My proposal was that build/checkpatch.pl could
require
>>>>>  tabs immediately, and new patches should be submitted with
tabs
>>>>>  on all new/modified lines (and optionally all lines on
small
>>>>>  functions to avoid messy formatting).  This will avoid
issues
>>>>>  with current patches in flight, and also avoid "git
annotate"
>>>>>  showing the jumbo replace-all-spaces-with-tabs patch for
every
>>>>>  line in Lustre, and I think a good fraction of lines will
be
>>>>>  updated in the next 9-12 months or more.  As we approach
the
>>>>>  actual time for upstream kernel submission is close, then
we can
>>>>>  make a final effort to clean up remaining lines in idle
code
>>>>>  (which will also be unlikely to conflict with existing
work).
>>>> While tabs are the main coding style difference between Lustre
and kernel, there are a few minor
>>> change that is needed as well. I think we need to do following to
match kernel coding style:
>>>> 1. Lustre uses expandtab while kernel requires tabs
>>>
>>> Right.
>>>
>>>> 2. Lustre has vim syntax rules in most source files, which need
to be removed
>>>
>>> They should be replaced with explicit vim and enacts syntax rules
that have the kernel indent style
>>> instead.  If we could get syntax rules that embodied more of the
coding style than just indentation,
>>> that would be even better.
>>>
>> But we do need to remove them before submitting to kernel, right? And
if we enforce checkpatch.pl on every patch applied, IMHO there is not much
benefit to have syntax rules on every file, not to mention that it is explicitly
forbidden in kernel coding style (Documentation/CodingStyle, Chapter 18:  Editor
modelines and other cruft).
>>
>> BTW, instead of just enabling tabs, how about changing checkpatch.pl to
latest kernel version to make sure all future patches follow kernel coding
styles?
>>
>>>> 3. Lustre uses a slightly different comment style, which need
to be changed to kernel style
>>>
>>> This is DOxygen style formatting.  I had forgotten about this. We
had in the past used this inline
>>> formatting for producing some documentation, but I''d need
to ask about whether there is still a need
>>> for this today. In the meantime, please leave the comment style
as-is.
>>>
>> OK.
>>
>>>
>>>> I created LU-1347 (http://jira.whamcloud.com/browse/LU-1347) to
track the coding style changes.
>>>>
>>>>> There is some hope that the kernel maintainers will not
require
>>>>> all of the Lustre macros to be removed, but we can deal
with this
>>>>> on a case-by-case basis.
>>>>>
>>>> IMO, we can divide macros to three groups (or more?):
>>>> 1. Old kernel support macros, kernel maintainers are clear that
they won''t accept it.
>>>
>>> Yes, but we need to maintain this in the external Lustre tree for
years after Lustre would be accepted
>>> into mainline, since it would not be in vendor kernels (which a
majority of Lustre users would be
>>> using).
>>>
>>> For such compat macros we need to make an effort to keep the
upstream code as close as possible to the
>>> external tree, so patches have the most chance of applying.
>>>
>> I agree. We should minimize maintenance effort for it. And as you
suggested, we can put as many of these compat macros into places like
linux_compat.h as possible and have Lustre code use latest kernel APIs, so that
most maintenance effort for old kernel support would be to keep linux_compat.h
uptodate. For compact macros that cannot be cleaned up this way, we will have to
pay the extra efforts. And of course the cleanup will be an incremental process
and macros will be dealt with in a case-by-case basis.
>>
>>>> 2. For macros to mark out server code, kernel maintainers can
accept it. But I think we need to make
>>> sure we don''t do it too intrusive.
>>>
>>> Sure, and we also need to make sure the ongoing maintenance effort
to keep the code working is not too
>>> much either.
>>>
>>> I''m OK with incremental patches that more cleanly split
the client and server code (structures,
>>> headers, etc) if that improves the code structure and readability.
>>>
>> I agree that we can do some incremental patches to split client and
server code. But I hope we only do it when it is trivial and simple. IMHO if we
want to entirely split client/server code, it will be large code structure
change. Since kernel maintainers already agreed on HAVE_SERVER_SUPPORT, how
about we keep going that way at first? And we can make some split wherever it is
simple and clear. And we will try to make code structure as clear/readable as
possible. Then when we summit code for review, if kernel maintainers still
don''t like it, we do the large restructuring. Does it make sense?
>>
>>>> 3. Lustre feature macros, we can convert them to Kconfig
macros.
>>>
>>> Sure. Note that some of them may be obsolete, so before you spend
too much time cleaning them up,
>>> please post a list to Jira. Maybe some of them can be dropped
entirely.
>>>
>> Thanks. I will do as you suggested when it comes to converting them to
Kconfig macros.
>>
>> Cheers,
>> Tao
>>
>>>>> On 2012-03-14, at 7:31 PM, Andreas Dilger wrote:
>>>>>> Whamcloud and EMC are jointly investigating how to be
able to contribute the Lustre client code
>>> into
>>>>> the upstream Linux kernel.
>>>>>>
>>>>>> As a prerequisite to this, EMC is working to clean up
the Lustre client code to better match the
>>>>> kernel coding style, and one of the anticipated major
obstacles to upstream kernel submission is
>>> the
>>>>> heavy use of code abstraction via libcfs for portability to
other operating systems (most notably
>>>>> MacOS and WinNT, but also for liblustre, and potentially
*BSD).
>>>>>>
>>>>>> I have no information that the WinNT project will ever
be released by Oracle,
>>>>>
>>>>> [revised] and the MacOS client needs significant work to
update it to the latest version of Lustre,
>>>>> and to get it landed into the Lustre repo,
>>>>>
>>>>>> so the libcfs portability layer is potentially exacting
a high cost in code maintenance and
>>>>> complexity (CLIO being a prime example) for no apparent
benefit.  Similarly, the liblustre client
>>>>> needs a portability layer for userspace, and suffers from
the same apparent lack of interest or
>>> users.
>>>>>>
>>>>>> I''d like to get some feedback from the Lustre
community about removing the libcfs abstraction
>>>>> entirely, or possibly restructuring it to look like the
Linux kernel API, and having the other
>>>>> platforms code against it as a Linux portability layer,
like ZFS on Linux uses the Solaris
>>> Portability
>>>>> Layer (SPL) to avoid changing the core ZFS code.  A related
topic is whether it would be better to
>>>>> replace all cfs_* functions with standard Linux kernel
functions en-masse, or migrate away from
>>> cfs_*
>>>>> functions slowly?
>>>>>>
>>>>>> Also, we''re planning on deprecating the
liblustre client code, due to lack of interest/usage.  The
>>>>> current code is in disrepair, and we''ve been
keeping it around for years without any benefit, and
>>>>> while I was one of the strongest advocates for keeping it
in our back pocket in case of future
>>> needs,
>>>>> I don''t see that materializing in the future.
>>>>>>
>>>>>> The liblustre code would be left in the tree for now,
in case someone from the community is
>>>>> interested to get it working and maintain it, and it may be
updated on a best effort basis.  If
>>> nobody
>>>>> steps forward to do this work, the liblustre code would be
deleted from the development branch in a
>>>>> year or so.
>>>>>>
>>>>>>
>>>>>> Unfortunately, after starting this thread, I may not be
able to reply to questions in a timely
>>>>> manner due to vacation.  I look forward to a thread that
concludes with unanimous agreement from
>>> all
>>>>> parties. :-)
>>>>>>
>>>>>> Cheers, Andreas
>>>>>> --
>>>>>> Andreas Dilger                       Whamcloud, Inc.
>>>>>> Principal Lustre Engineer           
http://www.whamcloud.com/
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>>
>>>>> Cheers, Andreas
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>
>> _______________________________________________
>> Lustre-discuss mailing list
>> Lustre-discuss at lists.lustre.org
>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> 
>

> -----Original Message-----
> From: Roman Grigoryev [mailto:Roman_Grigoryev at xyratex.com]
> Sent: Thursday, May 03, 2012 5:45 PM
> To: Peng, Tao
> Cc: adilger at whamcloud.com; wc-discuss at whamcloud.com; lustre-discuss
at lists.lustre.org; lustre-
> devel at lists.lustre.org
> Subject: Re: [Lustre-discuss] [wc-discuss] Re: Lustre and cross-platform
portability
> 
> Hi Tao,
Hi Roman,
> 
> I''m not know how linux developers approve patches. But, logically,
> because Lustre is big enough project and use kernel subsystems then it
> should be tested after kernel changes on kernel side too. In  other case
> (without testing) we can observer situation when Lustre built ok but
> doesn''t work.
Now I see what you are worrying about. All code submitted to Linux kernel will
be tested, and also patches later merged. It''s just that kernel is a
different place than Lustre git tree. We are not allowed to put test scripts
into kernel tree. So all test code will remain in Whamcloud tree. But we will
run the same tests against kernel code as well. One of our goal is to keep
kernel client code in sync with Whamcloud tree as much as possible, so that the
same code will be tested with older kernels as well.
> 
> I don''t know goal of adding Lustre to kernel and, possible,this
> situation could be acceptable from your point of view.
> 
Untested code is buggy and useless. We certainly don''t want it either.

Thanks,
Tao
> Thanks,
> 	Roman
> 
> On 04/27/2012 04:33 PM, tao.peng at emc.com wrote:
> > Hi Roman,
> >
> > Not sure if I misunderstand your question, but we won''t push
lustre/tests/* files to kernel.
> >We only push client code and any tests will be executed out side of
kernel
> >
> > Cheers,
> > Tao
> > ________________________________________
> > From: Roman Grigoryev [Roman_Grigoryev at xyratex.com]
> > Sent: Friday, April 27, 2012 6:25 PM
> > To: Peng, Tao
> > Cc: adilger at whamcloud.com; wc-discuss at whamcloud.com;
lustre-discuss at lists.lustre.org; lustre-
> devel at lists.lustre.org
> > Subject: Re: [Lustre-discuss] [wc-discuss] Re: Lustre and
cross-platform portability
> >
> > Tao,Andreas,all,
> >
> > What is your plan in test/test framework changes from the point of
view
> > of integration to kernel? As i know, kernel.org has his own test
> > infrastructure and his own test framework.
> >
> > I''m sorry if it''s incorrect place for this question.
> >
> > Thanks,
> >         Roman
> >
> > On 04/27/2012 02:15 PM, tao.peng at emc.com wrote:
> >> Hi Andreas,
> >>
> >>> -----Original Message-----
> >>> From: Andreas Dilger [mailto:adilger at whamcloud.com]
> >>> Sent: Friday, April 27, 2012 11:54 AM
> >>> To: Peng, Tao
> >>> Cc: <wc-discuss at whamcloud.com>; <lustre-devel at
lists.lustre.org>; <lustre-discuss at lists.lustre.org>
> >>> Subject: Re: [wc-discuss] Re: Lustre and cross-platform
portability
> >>>
> >>> On 2012-04-26, at 20:23, <tao.peng at emc.com> wrote:
> >>>> Thank you very much for bringing it up in LUG and getting
all these positive support from
> community.
> >>>
> >>> Tao,
> >>> Yes it does look promising.
> >>>
> >>>>> To revive this thread, based on discussion at the LUG
TWG:
> >>>>> - there was general consensus that cleaning up the
Lustre client
> >>>>>  (and server) code was very desirable to do
> >>>>> - migrating libcfs to emulate the Linux kernel APIs is
also usable.
> >>>>>  Ken mentioned that there is relatively little
conflict between
> >>>>>  the Linux kernel and the MacOS kernel, and the same
for WinNT, so
> >>>>>  they could use the same function names as Linux
without problems.
> >>>> I created LU-1346
(http://jira.whamcloud.com/browse/LU-1346) to track libcfs cleanup work.
> >>>
> >>> OK
> >>>
> >>>>> - there was no objection to converting the Lustre code
from spaces
> >>>>>  to tabs.  My proposal was that build/checkpatch.pl
could require
> >>>>>  tabs immediately, and new patches should be submitted
with tabs
> >>>>>  on all new/modified lines (and optionally all lines
on small
> >>>>>  functions to avoid messy formatting).  This will
avoid issues
> >>>>>  with current patches in flight, and also avoid
"git annotate"
> >>>>>  showing the jumbo replace-all-spaces-with-tabs patch
for every
> >>>>>  line in Lustre, and I think a good fraction of lines
will be
> >>>>>  updated in the next 9-12 months or more.  As we
approach the
> >>>>>  actual time for upstream kernel submission is close,
then we can
> >>>>>  make a final effort to clean up remaining lines in
idle code
> >>>>>  (which will also be unlikely to conflict with
existing work).
> >>>> While tabs are the main coding style difference between
Lustre and kernel, there are a few minor
> >>> change that is needed as well. I think we need to do following
to match kernel coding style:
> >>>> 1. Lustre uses expandtab while kernel requires tabs
> >>>
> >>> Right.
> >>>
> >>>> 2. Lustre has vim syntax rules in most source files, which
need to be removed
> >>>
> >>> They should be replaced with explicit vim and enacts syntax
rules that have the kernel indent
> style
> >>> instead.  If we could get syntax rules that embodied more of
the coding style than just
> indentation,
> >>> that would be even better.
> >>>
> >> But we do need to remove them before submitting to kernel, right?
And if we enforce checkpatch.pl
> on every patch applied, IMHO there is not much benefit to have syntax rules
on every file, not to
> mention that it is explicitly forbidden in kernel coding style
(Documentation/CodingStyle, Chapter 18:
> Editor modelines and other cruft).
> >>
> >> BTW, instead of just enabling tabs, how about changing
checkpatch.pl to latest kernel version to
> make sure all future patches follow kernel coding styles?
> >>
> >>>> 3. Lustre uses a slightly different comment style, which
need to be changed to kernel style
> >>>
> >>> This is DOxygen style formatting.  I had forgotten about this.
We had in the past used this inline
> >>> formatting for producing some documentation, but I''d
need to ask about whether there is still a
> need
> >>> for this today. In the meantime, please leave the comment
style as-is.
> >>>
> >> OK.
> >>
> >>>
> >>>> I created LU-1347
(http://jira.whamcloud.com/browse/LU-1347) to track the coding style changes.
> >>>>
> >>>>> There is some hope that the kernel maintainers will
not require
> >>>>> all of the Lustre macros to be removed, but we can
deal with this
> >>>>> on a case-by-case basis.
> >>>>>
> >>>> IMO, we can divide macros to three groups (or more?):
> >>>> 1. Old kernel support macros, kernel maintainers are clear
that they won''t accept it.
> >>>
> >>> Yes, but we need to maintain this in the external Lustre tree
for years after Lustre would be
> accepted
> >>> into mainline, since it would not be in vendor kernels (which
a majority of Lustre users would be
> >>> using).
> >>>
> >>> For such compat macros we need to make an effort to keep the
upstream code as close as possible to
> the
> >>> external tree, so patches have the most chance of applying.
> >>>
> >> I agree. We should minimize maintenance effort for it. And as you
suggested, we can put as many of
> these compat macros into places like linux_compat.h as possible and have
Lustre code use latest kernel
> APIs, so that most maintenance effort for old kernel support would be to
keep linux_compat.h uptodate.
> For compact macros that cannot be cleaned up this way, we will have to pay
the extra efforts. And of
> course the cleanup will be an incremental process and macros will be dealt
with in a case-by-case
> basis.
> >>
> >>>> 2. For macros to mark out server code, kernel maintainers
can accept it. But I think we need to
> make
> >>> sure we don''t do it too intrusive.
> >>>
> >>> Sure, and we also need to make sure the ongoing maintenance
effort to keep the code working is not
> too
> >>> much either.
> >>>
> >>> I''m OK with incremental patches that more cleanly
split the client and server code (structures,
> >>> headers, etc) if that improves the code structure and
readability.
> >>>
> >> I agree that we can do some incremental patches to split client
and server code. But I hope we only
> do it when it is trivial and simple. IMHO if we want to entirely split
client/server code, it will be
> large code structure change. Since kernel maintainers already agreed on
HAVE_SERVER_SUPPORT, how about
> we keep going that way at first? And we can make some split wherever it is
simple and clear. And we
> will try to make code structure as clear/readable as possible. Then when we
summit code for review, if
> kernel maintainers still don''t like it, we do the large
restructuring. Does it make sense?
> >>
> >>>> 3. Lustre feature macros, we can convert them to Kconfig
macros.
> >>>
> >>> Sure. Note that some of them may be obsolete, so before you
spend too much time cleaning them up,
> >>> please post a list to Jira. Maybe some of them can be dropped
entirely.
> >>>
> >> Thanks. I will do as you suggested when it comes to converting
them to Kconfig macros.
> >>
> >> Cheers,
> >> Tao
> >>
> >>>>> On 2012-03-14, at 7:31 PM, Andreas Dilger wrote:
> >>>>>> Whamcloud and EMC are jointly investigating how to
be able to contribute the Lustre client code
> >>> into
> >>>>> the upstream Linux kernel.
> >>>>>>
> >>>>>> As a prerequisite to this, EMC is working to clean
up the Lustre client code to better match
> the
> >>>>> kernel coding style, and one of the anticipated major
obstacles to upstream kernel submission is
> >>> the
> >>>>> heavy use of code abstraction via libcfs for
portability to other operating systems (most
> notably
> >>>>> MacOS and WinNT, but also for liblustre, and
potentially *BSD).
> >>>>>>
> >>>>>> I have no information that the WinNT project will
ever be released by Oracle,
> >>>>>
> >>>>> [revised] and the MacOS client needs significant work
to update it to the latest version of
> Lustre,
> >>>>> and to get it landed into the Lustre repo,
> >>>>>
> >>>>>> so the libcfs portability layer is potentially
exacting a high cost in code maintenance and
> >>>>> complexity (CLIO being a prime example) for no
apparent benefit.  Similarly, the liblustre
> client
> >>>>> needs a portability layer for userspace, and suffers
from the same apparent lack of interest or
> >>> users.
> >>>>>>
> >>>>>> I''d like to get some feedback from the
Lustre community about removing the libcfs abstraction
> >>>>> entirely, or possibly restructuring it to look like
the Linux kernel API, and having the other
> >>>>> platforms code against it as a Linux portability
layer, like ZFS on Linux uses the Solaris
> >>> Portability
> >>>>> Layer (SPL) to avoid changing the core ZFS code.  A
related topic is whether it would be better
> to
> >>>>> replace all cfs_* functions with standard Linux kernel
functions en-masse, or migrate away from
> >>> cfs_*
> >>>>> functions slowly?
> >>>>>>
> >>>>>> Also, we''re planning on deprecating the
liblustre client code, due to lack of interest/usage.
> The
> >>>>> current code is in disrepair, and we''ve been
keeping it around for years without any benefit,
> and
> >>>>> while I was one of the strongest advocates for keeping
it in our back pocket in case of future
> >>> needs,
> >>>>> I don''t see that materializing in the future.
> >>>>>>
> >>>>>> The liblustre code would be left in the tree for
now, in case someone from the community is
> >>>>> interested to get it working and maintain it, and it
may be updated on a best effort basis.  If
> >>> nobody
> >>>>> steps forward to do this work, the liblustre code
would be deleted from the development branch
> in a
> >>>>> year or so.
> >>>>>>
> >>>>>>
> >>>>>> Unfortunately, after starting this thread, I may
not be able to reply to questions in a timely
> >>>>> manner due to vacation.  I look forward to a thread
that concludes with unanimous agreement from
> >>> all
> >>>>> parties. :-)
> >>>>>>
> >>>>>> Cheers, Andreas
> >>>>>> --
> >>>>>> Andreas Dilger                       Whamcloud,
Inc.
> >>>>>> Principal Lustre Engineer           
http://www.whamcloud.com/
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>>
> >>>>> Cheers, Andreas
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>
> >>
> >> _______________________________________________
> >> Lustre-discuss mailing list
> >> Lustre-discuss at lists.lustre.org
> >> http://lists.lustre.org/mailman/listinfo/lustre-discuss
> >
> >
> 
> ______________________________________________________________________
> This email may contain privileged or confidential information, which should
only be used for the
> purpose for which it was sent by Xyratex. No further rights or licenses are
granted to use such
> information. If you are not the intended recipient of this message, please
notify the sender by return
> and delete it. You may not use, copy, disclose or rely on the information
contained in it.
> 
> Internet email is susceptible to data corruption, interception and
unauthorised amendment for which
> Xyratex does not accept liability. While we have taken reasonable
precautions to ensure that this
> email is free of viruses, Xyratex does not accept liability for the
presence of any computer viruses
> in this email, nor for any losses caused as a result of viruses.
> 
> Xyratex Technology Limited (03134912), Registered in England & Wales,
Registered Office, Langstone
> Road, Havant, Hampshire, PO9 1SA.
> 
> The Xyratex group of companies also includes, Xyratex Ltd, registered in
Bermuda, Xyratex
> International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd
registered in Malaysia,
> Xyratex Technology (Wuxi) Co Ltd registered in The People''s
Republic of China and Xyratex Japan
> Limited registered in Japan.
> ______________________________________________________________________
> 
>

On 05/03/2012 02:03 PM, tao.peng at emc.com wrote:
>>
>> I''m not know how linux developers approve patches. But,
logically,
>> because Lustre is big enough project and use kernel subsystems then it
>> should be tested after kernel changes on kernel side too. In  other
case
>> (without testing) we can observer situation when Lustre built ok but
>> doesn''t work.
> Now I see what you are worrying about. All code submitted to Linux kernel 
> will be tested, and also patches later merged. It''s just that
kernel is a
> different place than Lustre git tree. We are not allowed to put test
scripts
> into kernel tree. So all test code will remain in Whamcloud tree. But we
will
> run the same tests against kernel code as well. One of our goal is to keep
kernel
> client code in sync with Whamcloud tree as much as possible, so that the
same
> code will be tested with older kernels as well.
I just want to point, as I understand situation, keeping kernel client
code in sync with Whamcloud tree without testing on latest
kernels(include unstable) could lead to situation when client on new
kernel doesn''t work but built ok. Testing on older kernels
doesn''t say
about work on new kernels, only give possibility.
Thanks,
	Roman
> 
>>
>> I don''t know goal of adding Lustre to kernel and,
possible,this
>> situation could be acceptable from your point of view.
>>
> Untested code is buggy and useless. We certainly don''t want it
either.
> 
> Thanks,
> Tao
> 
>> Thanks,
>> 	Roman
>>
>> On 04/27/2012 04:33 PM, tao.peng at emc.com wrote:
>>> Hi Roman,
>>>
>>> Not sure if I misunderstand your question, but we won''t
push lustre/tests/* files to kernel.
>>> We only push client code and any tests will be executed out side of
kernel
>>>
>>> Cheers,
>>> Tao
>>> ________________________________________
>>> From: Roman Grigoryev [Roman_Grigoryev at xyratex.com]
>>> Sent: Friday, April 27, 2012 6:25 PM
>>> To: Peng, Tao
>>> Cc: adilger at whamcloud.com; wc-discuss at whamcloud.com;
lustre-discuss at lists.lustre.org; lustre-
>> devel at lists.lustre.org
>>> Subject: Re: [Lustre-discuss] [wc-discuss] Re: Lustre and
cross-platform portability
>>>
>>> Tao,Andreas,all,
>>>
>>> What is your plan in test/test framework changes from the point of
view
>>> of integration to kernel? As i know, kernel.org has his own test
>>> infrastructure and his own test framework.
>>>
>>> I''m sorry if it''s incorrect place for this
question.
>>>
>>> Thanks,
>>>         Roman
>>>
>>> On 04/27/2012 02:15 PM, tao.peng at emc.com wrote:
>>>> Hi Andreas,
>>>>
>>>>> -----Original Message-----
>>>>> From: Andreas Dilger [mailto:adilger at whamcloud.com]
>>>>> Sent: Friday, April 27, 2012 11:54 AM
>>>>> To: Peng, Tao
>>>>> Cc: <wc-discuss at whamcloud.com>; <lustre-devel
at lists.lustre.org>; <lustre-discuss at lists.lustre.org>
>>>>> Subject: Re: [wc-discuss] Re: Lustre and cross-platform
portability
>>>>>
>>>>> On 2012-04-26, at 20:23, <tao.peng at emc.com> wrote:
>>>>>> Thank you very much for bringing it up in LUG and
getting all these positive support from
>> community.
>>>>>
>>>>> Tao,
>>>>> Yes it does look promising.
>>>>>
>>>>>>> To revive this thread, based on discussion at the
LUG TWG:
>>>>>>> - there was general consensus that cleaning up the
Lustre client
>>>>>>>  (and server) code was very desirable to do
>>>>>>> - migrating libcfs to emulate the Linux kernel APIs
is also usable.
>>>>>>>  Ken mentioned that there is relatively little
conflict between
>>>>>>>  the Linux kernel and the MacOS kernel, and the
same for WinNT, so
>>>>>>>  they could use the same function names as Linux
without problems.
>>>>>> I created LU-1346
(http://jira.whamcloud.com/browse/LU-1346) to track libcfs cleanup work.
>>>>>
>>>>> OK
>>>>>
>>>>>>> - there was no objection to converting the Lustre
code from spaces
>>>>>>>  to tabs.  My proposal was that build/checkpatch.pl
could require
>>>>>>>  tabs immediately, and new patches should be
submitted with tabs
>>>>>>>  on all new/modified lines (and optionally all
lines on small
>>>>>>>  functions to avoid messy formatting).  This will
avoid issues
>>>>>>>  with current patches in flight, and also avoid
"git annotate"
>>>>>>>  showing the jumbo replace-all-spaces-with-tabs
patch for every
>>>>>>>  line in Lustre, and I think a good fraction of
lines will be
>>>>>>>  updated in the next 9-12 months or more.  As we
approach the
>>>>>>>  actual time for upstream kernel submission is
close, then we can
>>>>>>>  make a final effort to clean up remaining lines in
idle code
>>>>>>>  (which will also be unlikely to conflict with
existing work).
>>>>>> While tabs are the main coding style difference between
Lustre and kernel, there are a few minor
>>>>> change that is needed as well. I think we need to do
following to match kernel coding style:
>>>>>> 1. Lustre uses expandtab while kernel requires tabs
>>>>>
>>>>> Right.
>>>>>
>>>>>> 2. Lustre has vim syntax rules in most source files,
which need to be removed
>>>>>
>>>>> They should be replaced with explicit vim and enacts syntax
rules that have the kernel indent
>> style
>>>>> instead.  If we could get syntax rules that embodied more
of the coding style than just
>> indentation,
>>>>> that would be even better.
>>>>>
>>>> But we do need to remove them before submitting to kernel,
right? And if we enforce checkpatch.pl
>> on every patch applied, IMHO there is not much benefit to have syntax
rules on every file, not to
>> mention that it is explicitly forbidden in kernel coding style
(Documentation/CodingStyle, Chapter 18:
>> Editor modelines and other cruft).
>>>>
>>>> BTW, instead of just enabling tabs, how about changing
checkpatch.pl to latest kernel version to
>> make sure all future patches follow kernel coding styles?
>>>>
>>>>>> 3. Lustre uses a slightly different comment style,
which need to be changed to kernel style
>>>>>
>>>>> This is DOxygen style formatting.  I had forgotten about
this. We had in the past used this inline
>>>>> formatting for producing some documentation, but
I''d need to ask about whether there is still a
>> need
>>>>> for this today. In the meantime, please leave the comment
style as-is.
>>>>>
>>>> OK.
>>>>
>>>>>
>>>>>> I created LU-1347
(http://jira.whamcloud.com/browse/LU-1347) to track the coding style changes.
>>>>>>
>>>>>>> There is some hope that the kernel maintainers will
not require
>>>>>>> all of the Lustre macros to be removed, but we can
deal with this
>>>>>>> on a case-by-case basis.
>>>>>>>
>>>>>> IMO, we can divide macros to three groups (or more?):
>>>>>> 1. Old kernel support macros, kernel maintainers are
clear that they won''t accept it.
>>>>>
>>>>> Yes, but we need to maintain this in the external Lustre
tree for years after Lustre would be
>> accepted
>>>>> into mainline, since it would not be in vendor kernels
(which a majority of Lustre users would be
>>>>> using).
>>>>>
>>>>> For such compat macros we need to make an effort to keep
the upstream code as close as possible to
>> the
>>>>> external tree, so patches have the most chance of applying.
>>>>>
>>>> I agree. We should minimize maintenance effort for it. And as
you suggested, we can put as many of
>> these compat macros into places like linux_compat.h as possible and
have Lustre code use latest kernel
>> APIs, so that most maintenance effort for old kernel support would be
to keep linux_compat.h uptodate.
>> For compact macros that cannot be cleaned up this way, we will have to
pay the extra efforts. And of
>> course the cleanup will be an incremental process and macros will be
dealt with in a case-by-case
>> basis.
>>>>
>>>>>> 2. For macros to mark out server code, kernel
maintainers can accept it. But I think we need to
>> make
>>>>> sure we don''t do it too intrusive.
>>>>>
>>>>> Sure, and we also need to make sure the ongoing maintenance
effort to keep the code working is not
>> too
>>>>> much either.
>>>>>
>>>>> I''m OK with incremental patches that more cleanly
split the client and server code (structures,
>>>>> headers, etc) if that improves the code structure and
readability.
>>>>>
>>>> I agree that we can do some incremental patches to split client
and server code. But I hope we only
>> do it when it is trivial and simple. IMHO if we want to entirely split
client/server code, it will be
>> large code structure change. Since kernel maintainers already agreed on
HAVE_SERVER_SUPPORT, how about
>> we keep going that way at first? And we can make some split wherever it
is simple and clear. And we
>> will try to make code structure as clear/readable as possible. Then
when we summit code for review, if
>> kernel maintainers still don''t like it, we do the large
restructuring. Does it make sense?
>>>>
>>>>>> 3. Lustre feature macros, we can convert them to
Kconfig macros.
>>>>>
>>>>> Sure. Note that some of them may be obsolete, so before you
spend too much time cleaning them up,
>>>>> please post a list to Jira. Maybe some of them can be
dropped entirely.
>>>>>
>>>> Thanks. I will do as you suggested when it comes to converting
them to Kconfig macros.
>>>>
>>>> Cheers,
>>>> Tao
>>>>
>>>>>>> On 2012-03-14, at 7:31 PM, Andreas Dilger wrote:
>>>>>>>> Whamcloud and EMC are jointly investigating how
to be able to contribute the Lustre client code
>>>>> into
>>>>>>> the upstream Linux kernel.
>>>>>>>>
>>>>>>>> As a prerequisite to this, EMC is working to
clean up the Lustre client code to better match
>> the
>>>>>>> kernel coding style, and one of the anticipated
major obstacles to upstream kernel submission is
>>>>> the
>>>>>>> heavy use of code abstraction via libcfs for
portability to other operating systems (most
>> notably
>>>>>>> MacOS and WinNT, but also for liblustre, and
potentially *BSD).
>>>>>>>>
>>>>>>>> I have no information that the WinNT project
will ever be released by Oracle,
>>>>>>>
>>>>>>> [revised] and the MacOS client needs significant
work to update it to the latest version of
>> Lustre,
>>>>>>> and to get it landed into the Lustre repo,
>>>>>>>
>>>>>>>> so the libcfs portability layer is potentially
exacting a high cost in code maintenance and
>>>>>>> complexity (CLIO being a prime example) for no
apparent benefit.  Similarly, the liblustre
>> client
>>>>>>> needs a portability layer for userspace, and
suffers from the same apparent lack of interest or
>>>>> users.
>>>>>>>>
>>>>>>>> I''d like to get some feedback from the
Lustre community about removing the libcfs abstraction
>>>>>>> entirely, or possibly restructuring it to look like
the Linux kernel API, and having the other
>>>>>>> platforms code against it as a Linux portability
layer, like ZFS on Linux uses the Solaris
>>>>> Portability
>>>>>>> Layer (SPL) to avoid changing the core ZFS code.  A
related topic is whether it would be better
>> to
>>>>>>> replace all cfs_* functions with standard Linux
kernel functions en-masse, or migrate away from
>>>>> cfs_*
>>>>>>> functions slowly?
>>>>>>>>
>>>>>>>> Also, we''re planning on deprecating
the liblustre client code, due to lack of interest/usage.
>> The
>>>>>>> current code is in disrepair, and we''ve
been keeping it around for years without any benefit,
>> and
>>>>>>> while I was one of the strongest advocates for
keeping it in our back pocket in case of future
>>>>> needs,
>>>>>>> I don''t see that materializing in the
future.
>>>>>>>>
>>>>>>>> The liblustre code would be left in the tree
for now, in case someone from the community is
>>>>>>> interested to get it working and maintain it, and
it may be updated on a best effort basis.  If
>>>>> nobody
>>>>>>> steps forward to do this work, the liblustre code
would be deleted from the development branch
>> in a
>>>>>>> year or so.
>>>>>>>>
>>>>>>>>
>>>>>>>> Unfortunately, after starting this thread, I
may not be able to reply to questions in a timely
>>>>>>> manner due to vacation.  I look forward to a thread
that concludes with unanimous agreement from
>>>>> all
>>>>>>> parties. :-)
>>>>>>>>
>>>>>>>> Cheers, Andreas
>>>>>>>> --
>>>>>>>> Andreas Dilger                       Whamcloud,
Inc.
>>>>>>>> Principal Lustre Engineer           
http://www.whamcloud.com/
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Cheers, Andreas
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>
>>>> _______________________________________________
>>>> Lustre-discuss mailing list
>>>> Lustre-discuss at lists.lustre.org
>>>> http://lists.lustre.org/mailman/listinfo/lustre-discuss
>>>
>>>
>>
>> ______________________________________________________________________
>> This email may contain privileged or confidential information, which
should only be used for the
>> purpose for which it was sent by Xyratex. No further rights or licenses
are granted to use such
>> information. If you are not the intended recipient of this message,
please notify the sender by return
>> and delete it. You may not use, copy, disclose or rely on the
information contained in it.
>>
>> Internet email is susceptible to data corruption, interception and
unauthorised amendment for which
>> Xyratex does not accept liability. While we have taken reasonable
precautions to ensure that this
>> email is free of viruses, Xyratex does not accept liability for the
presence of any computer viruses
>> in this email, nor for any losses caused as a result of viruses.
>>
>> Xyratex Technology Limited (03134912), Registered in England &
Wales, Registered Office, Langstone
>> Road, Havant, Hampshire, PO9 1SA.
>>
>> The Xyratex group of companies also includes, Xyratex Ltd, registered
in Bermuda, Xyratex
>> International Inc, registered in California, Xyratex (Malaysia) Sdn Bhd
registered in Malaysia,
>> Xyratex Technology (Wuxi) Co Ltd registered in The People''s
Republic of China and Xyratex Japan
>> Limited registered in Japan.
>> ______________________________________________________________________
>>
>>
>

________________________________________
From: Roman Grigoryev [Roman_Grigoryev at xyratex.com]
Sent: Thursday, May 03, 2012 6:45 PM
To: Peng, Tao
Cc: adilger at whamcloud.com; wc-discuss at whamcloud.com; lustre-discuss at
lists.lustre.org; lustre-devel at lists.lustre.org
Subject: Re: [Lustre-discuss] [wc-discuss] Re: Lustre and cross-platform
portability

On 05/03/2012 02:03 PM, tao.peng at emc.com wrote:
>>
>> I''m not know how linux developers approve patches. But,
logically,
>> because Lustre is big enough project and use kernel subsystems then it
>> should be tested after kernel changes on kernel side too. In  other
case
>> (without testing) we can observer situation when Lustre built ok but
>> doesn''t work.
> Now I see what you are worrying about. All code submitted to Linux kernel
> will be tested, and also patches later merged. It''s just that
kernel is a
> different place than Lustre git tree. We are not allowed to put test
scripts
> into kernel tree. So all test code will remain in Whamcloud tree. But we
will
> run the same tests against kernel code as well. One of our goal is to keep
kernel
> client code in sync with Whamcloud tree as much as possible, so that the
same
> code will be tested with older kernels as well.
> I just want to point, as I understand situation, keeping kernel client
> code in sync with Whamcloud tree without testing on latest
> kernels(include unstable) could lead to situation when client on new
> kernel doesn''t work but built ok. Testing on older kernels
doesn''t say
> about work on new kernels, only give possibility.
I agree. We need to test latest kernel from time to time, and it can be a
community effort. We will certainly test every patch we put in. And anyone
interested can test the code as well. If you are suggesting some public testing
infrastructure, maybe Whamcloud will be interested? IMHO, testing every patch in
upstream kernel like Maloo does for Whamcloud tree, is a little luxury through.

Thanks,
Tao