twanjari at andrew.cmu.edu
2013-May-10 22:03 UTC
[Lustre-devel] Interested in contributing to Lustre
Hi all, I am a grad student at Carnegie Mellon University. I had my course work in advanced storage systems in previous semester, and I am interested to work on Lustre. I prefer to take up a project that could be completed in a duration of a month or two. Since I am a novice w.r.t. my familiarity with Lustre code base, I seek your opinion to choose a project from the list: http://wiki.lustre.org/index.php/Lustre_Project_List My preferences (in order) are: 1. fallocate() API - https://bugzilla.lustre.org/show_bug.cgi?id=15064 2. kernel patch removal - https://bugzilla.lustre.org/show_bug.cgi?id=21524 3. ioctl() number cleanups - https://bugzilla.lustre.org/show_bug.cgi?id=20731 I am also open to other suggestion. Any suggestions regarding code walkthrough or documentation to help ramping up are also welcome. Thanks, Tejas Wanjari
I wish you luck in your endeavour! Here are some links which might prove useful to you: * https://jira.hpdd.intel.com - New issues are opened here * http://review.whamcloud.com - New patches are reviewed here * https://wiki.hpdd.intel.com/display/PUB/Submitting+Changes - Some info on the development cycle used by the Lustre project. Unfortunately, there isn''t nearly enough documentation as there should be in the Lustre tree (especially for new comers), so feel free to ask questions. As far as potential projects, in my opinion, the project list you linked to is a bit dated. There are *many* open issues which need to be fixed, so I would suggest finding something that interests you on the JIRA bug tracker that I linked to above, since that is a more up to date source of information. If you are completely new, a good place to start is just downloading the git tree, building, and running the unit tests locally to get familiar with the project. Then you can start to modify the source and explore some of the areas that interest you. -- Cheers, Prakash On Fri, May 10, 2013 at 06:03:27PM -0400, twanjari at andrew.cmu.edu wrote:> Hi all, > > I am a grad student at Carnegie Mellon University. I had my course work in > advanced storage systems in previous semester, and I am interested to work > on Lustre. I prefer to take up a project that could be completed in a > duration of a month or two. > > Since I am a novice w.r.t. my familiarity with Lustre code base, I seek > your opinion to choose a project from the list: > http://wiki.lustre.org/index.php/Lustre_Project_List > > My preferences (in order) are: > 1. fallocate() API - https://bugzilla.lustre.org/show_bug.cgi?id=15064 > 2. kernel patch removal - https://bugzilla.lustre.org/show_bug.cgi?id=21524 > 3. ioctl() number cleanups - > https://bugzilla.lustre.org/show_bug.cgi?id=20731 > I am also open to other suggestion. > > Any suggestions regarding code walkthrough or documentation to help > ramping up are also welcome. > > Thanks, > Tejas Wanjari > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel
On 2013-05-13, at 12:44, "twanjari at andrew.cmu.edu<mailto:twanjari at andrew.cmu.edu>" <twanjari at andrew.cmu.edu<mailto:twanjari at andrew.cmu.edu>> wrote: I am a grad student at Carnegie Mellon University. I had my course work in advanced storage systems in previous semester, and I am interested to work on Lustre. I prefer to take up a project that could be completed in a duration of a month or two. Since I am a novice w.r.t. my familiarity with Lustre code base, I seek your opinion to choose a project from the list: http://wiki.lustre.org/index.php/Lustre_Project_List A more up-to-date list is at: https://wiki.hpdd.intel.com/display/PUB/Project+Ideas My preferences (in order) are: 1. fallocate() API - https://bugzilla.lustre.org/show_bug.cgi?id=15064 This one is a bit tricky, but benefits an increasing number of users and would be the most "filesystem-y" of the listed projects. Bonus marks if you tie this in with the Lustre MPIIO-ADIO driver. 2. kernel patch removal - https://bugzilla.lustre.org/show_bug.cgi?id=21524 The main project here is to replace the "dev-read-only" patch with hooks into the "dm-flakey" driver or similar, that accepts reads, but does not write anything to disk. It is partly Lustre and partly in the kernel. 3. ioctl() number cleanups - https://bugzilla.lustre.org/show_bug.cgi?id=20731 This would help to get Lustre into the upstream kernel, but is not necessarily going to get you distributed filesystem experience. I am also open to other suggestion. Any suggestions regarding code walkthrough or documentation to help ramping up are also welcome. Thanks, Tejas Wanjari _______________________________________________ Lustre-devel mailing list Lustre-devel at lists.lustre.org<mailto:Lustre-devel at lists.lustre.org> http://lists.lustre.org/mailman/listinfo/lustre-devel
Additionally there''s this pretty good (though stale in many parts now, but still some other parts are ok and high level is certainly ok) Lustre internals book: users.nccs.gov/~fwang2/papers/lustre_report.pdf? On May 13, 2013, at 5:34 PM, Prakash Surya wrote:> I wish you luck in your endeavour! > > Here are some links which might prove useful to you: > > * https://jira.hpdd.intel.com - New issues are opened here > > * http://review.whamcloud.com - New patches are reviewed here > > * https://wiki.hpdd.intel.com/display/PUB/Submitting+Changes - Some > info on the development cycle used by the Lustre project. > > Unfortunately, there isn''t nearly enough documentation as there should > be in the Lustre tree (especially for new comers), so feel free to ask > questions. > > As far as potential projects, in my opinion, the project list you > linked to is a bit dated. There are *many* open issues which need to be > fixed, so I would suggest finding something that interests you on the > JIRA bug tracker that I linked to above, since that is a more up to date > source of information. > > If you are completely new, a good place to start is just downloading the > git tree, building, and running the unit tests locally to get familiar > with the project. Then you can start to modify the source and explore > some of the areas that interest you. > > -- > Cheers, Prakash > > On Fri, May 10, 2013 at 06:03:27PM -0400, twanjari at andrew.cmu.edu wrote: >> Hi all, >> >> I am a grad student at Carnegie Mellon University. I had my course work in >> advanced storage systems in previous semester, and I am interested to work >> on Lustre. I prefer to take up a project that could be completed in a >> duration of a month or two. >> >> Since I am a novice w.r.t. my familiarity with Lustre code base, I seek >> your opinion to choose a project from the list: >> http://wiki.lustre.org/index.php/Lustre_Project_List >> >> My preferences (in order) are: >> 1. fallocate() API - https://bugzilla.lustre.org/show_bug.cgi?id=15064 >> 2. kernel patch removal - https://bugzilla.lustre.org/show_bug.cgi?id=21524 >> 3. ioctl() number cleanups - >> https://bugzilla.lustre.org/show_bug.cgi?id=20731 >> I am also open to other suggestion. >> >> Any suggestions regarding code walkthrough or documentation to help >> ramping up are also welcome. >> >> Thanks, >> Tejas Wanjari >> >> _______________________________________________ >> Lustre-devel mailing list >> Lustre-devel at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-devel > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel
Tejas, In addition to the Lustre_Project_List at wiki.lustre.org, there is another project list at https://wiki.hpdd.intel.com/display/PUB/Project+Ideas. Thanks, -Cory On 5/10/13 5:03 PM, "twanjari at andrew.cmu.edu" <twanjari at andrew.cmu.edu> wrote:>Hi all, > >I am a grad student at Carnegie Mellon University. I had my course work in >advanced storage systems in previous semester, and I am interested to work >on Lustre. I prefer to take up a project that could be completed in a >duration of a month or two. > >Since I am a novice w.r.t. my familiarity with Lustre code base, I seek >your opinion to choose a project from the list: >http://wiki.lustre.org/index.php/Lustre_Project_List > >My preferences (in order) are: >1. fallocate() API - https://bugzilla.lustre.org/show_bug.cgi?id=15064 >2. kernel patch removal - >https://bugzilla.lustre.org/show_bug.cgi?id=21524 >3. ioctl() number cleanups - >https://bugzilla.lustre.org/show_bug.cgi?id=20731 >I am also open to other suggestion. > >Any suggestions regarding code walkthrough or documentation to help >ramping up are also welcome. > >Thanks, >Tejas Wanjari > >_______________________________________________ >Lustre-devel mailing list >Lustre-devel at lists.lustre.org >http://lists.lustre.org/mailman/listinfo/lustre-devel
Same disclaimer applies for http://wiki.lustre.org/lid/agi/agi.html. The OpenSFS CDWG has discussed bringing these documents up-to-date. Nothing concrete yet though. Thanks, -Cory On 5/14/13 1:49 AM, "Drokin, Oleg" <oleg.drokin at intel.com> wrote:>Additionally there''s this pretty good (though stale in many parts now, >but still some other parts are ok and high level is certainly ok) >Lustre internals book: users.nccs.gov/~fwang2/papers/lustre_report.pdf? > >On May 13, 2013, at 5:34 PM, Prakash Surya wrote: > >> I wish you luck in your endeavour! >> >> Here are some links which might prove useful to you: >> >> * https://jira.hpdd.intel.com - New issues are opened here >> >> * http://review.whamcloud.com - New patches are reviewed here >> >> * https://wiki.hpdd.intel.com/display/PUB/Submitting+Changes - Some >> info on the development cycle used by the Lustre project. >> >> Unfortunately, there isn''t nearly enough documentation as there should >> be in the Lustre tree (especially for new comers), so feel free to ask >> questions. >> >> As far as potential projects, in my opinion, the project list you >> linked to is a bit dated. There are *many* open issues which need to be >> fixed, so I would suggest finding something that interests you on the >> JIRA bug tracker that I linked to above, since that is a more up to date >> source of information. >> >> If you are completely new, a good place to start is just downloading the >> git tree, building, and running the unit tests locally to get familiar >> with the project. Then you can start to modify the source and explore >> some of the areas that interest you. >> >> -- >> Cheers, Prakash >> >> On Fri, May 10, 2013 at 06:03:27PM -0400, twanjari at andrew.cmu.edu wrote: >>> Hi all, >>> >>> I am a grad student at Carnegie Mellon University. I had my course >>>work in >>> advanced storage systems in previous semester, and I am interested to >>>work >>> on Lustre. I prefer to take up a project that could be completed in a >>> duration of a month or two. >>> >>> Since I am a novice w.r.t. my familiarity with Lustre code base, I seek >>> your opinion to choose a project from the list: >>> http://wiki.lustre.org/index.php/Lustre_Project_List >>> >>> My preferences (in order) are: >>> 1. fallocate() API - https://bugzilla.lustre.org/show_bug.cgi?id=15064 >>> 2. kernel patch removal - >>>https://bugzilla.lustre.org/show_bug.cgi?id=21524 >>> 3. ioctl() number cleanups - >>> https://bugzilla.lustre.org/show_bug.cgi?id=20731 >>> I am also open to other suggestion. >>> >>> Any suggestions regarding code walkthrough or documentation to help >>> ramping up are also welcome. >>> >>> Thanks, >>> Tejas Wanjari >>> >>> _______________________________________________ >>> Lustre-devel mailing list >>> Lustre-devel at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-devel >> _______________________________________________ >> Lustre-devel mailing list >> Lustre-devel at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-devel > >_______________________________________________ >Lustre-devel mailing list >Lustre-devel at lists.lustre.org >http://lists.lustre.org/mailman/listinfo/lustre-devel
Nikitas Angelinas
2013-May-14 15:23 UTC
[Lustre-devel] Interested in contributing to Lustre
In addition to the document Oleg has pointed out (which imho is probably the best resource for a newcomer developer to start with), and the link that Cory has provided which also includes documentation on some more recent additions to the source (and is also a good source of information for newcomers imo), some older HLDs can be found at http://wiki.lustre.org/index.php/Architecture_-_Lustre_HLDs, and some newer HLDs/architectural docs at https://wiki.hpdd.intel.com/display/PUB/Whamcloud+Designs. The Operations Manual is also quite useful i think, at https://wiki.hpdd.intel.com/display/PUB/Documentation. Added to all this, there are quite a few presentation slides and av recordings from previous LUG/LAD or other events around the web that you might find useful. As Oleg has hinted, some of the information you may come across might be a bit stale, as it applies to previous versions of the source, so please keep in mind that whole subsystems that are referenced in a document you are reading may have been completely replaced (by newer and better versions). But don''t let this be an issue, please ask here if something does not make sense; people are usually quite keen to help. Cheers, Nikitas On 14 May 2013 15:58, Cory Spitz <spitzcor at cray.com> wrote:> Same disclaimer applies for http://wiki.lustre.org/lid/agi/agi.html. > > The OpenSFS CDWG has discussed bringing these documents up-to-date. > Nothing concrete yet though. > > Thanks, > -Cory > > > On 5/14/13 1:49 AM, "Drokin, Oleg" <oleg.drokin at intel.com> wrote: > > >Additionally there''s this pretty good (though stale in many parts now, > >but still some other parts are ok and high level is certainly ok) > >Lustre internals book: users.nccs.gov/~fwang2/papers/lustre_report.pdf? > > > >On May 13, 2013, at 5:34 PM, Prakash Surya wrote: > > > >> I wish you luck in your endeavour! > >> > >> Here are some links which might prove useful to you: > >> > >> * https://jira.hpdd.intel.com - New issues are opened here > >> > >> * http://review.whamcloud.com - New patches are reviewed here > >> > >> * https://wiki.hpdd.intel.com/display/PUB/Submitting+Changes - Some > >> info on the development cycle used by the Lustre project. > >> > >> Unfortunately, there isn''t nearly enough documentation as there should > >> be in the Lustre tree (especially for new comers), so feel free to ask > >> questions. > >> > >> As far as potential projects, in my opinion, the project list you > >> linked to is a bit dated. There are *many* open issues which need to be > >> fixed, so I would suggest finding something that interests you on the > >> JIRA bug tracker that I linked to above, since that is a more up to date > >> source of information. > >> > >> If you are completely new, a good place to start is just downloading the > >> git tree, building, and running the unit tests locally to get familiar > >> with the project. Then you can start to modify the source and explore > >> some of the areas that interest you. > >> > >> -- > >> Cheers, Prakash > >> > >> On Fri, May 10, 2013 at 06:03:27PM -0400, twanjari at andrew.cmu.eduwrote: > >>> Hi all, > >>> > >>> I am a grad student at Carnegie Mellon University. I had my course > >>>work in > >>> advanced storage systems in previous semester, and I am interested to > >>>work > >>> on Lustre. I prefer to take up a project that could be completed in a > >>> duration of a month or two. > >>> > >>> Since I am a novice w.r.t. my familiarity with Lustre code base, I seek > >>> your opinion to choose a project from the list: > >>> http://wiki.lustre.org/index.php/Lustre_Project_List > >>> > >>> My preferences (in order) are: > >>> 1. fallocate() API - https://bugzilla.lustre.org/show_bug.cgi?id=15064 > >>> 2. kernel patch removal - > >>>https://bugzilla.lustre.org/show_bug.cgi?id=21524 > >>> 3. ioctl() number cleanups - > >>> https://bugzilla.lustre.org/show_bug.cgi?id=20731 > >>> I am also open to other suggestion. > >>> > >>> Any suggestions regarding code walkthrough or documentation to help > >>> ramping up are also welcome. > >>> > >>> Thanks, > >>> Tejas Wanjari > >>> > >>> _______________________________________________ > >>> Lustre-devel mailing list > >>> Lustre-devel at lists.lustre.org > >>> http://lists.lustre.org/mailman/listinfo/lustre-devel > >> _______________________________________________ > >> Lustre-devel mailing list > >> Lustre-devel at lists.lustre.org > >> http://lists.lustre.org/mailman/listinfo/lustre-devel > > > >_______________________________________________ > >Lustre-devel mailing list > >Lustre-devel at lists.lustre.org > >http://lists.lustre.org/mailman/listinfo/lustre-devel > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel >-- ------------------------------ For additional information including the registered office and the treatment of Xyratex confidential information please visit www.xyratex.com ------------------------------ -------------- next part -------------- An HTML attachment was scrubbed... URL: http://lists.lustre.org/pipermail/lustre-devel/attachments/20130514/5e6a529b/attachment.html
I am looking into some Lustre development but want to build everything from scratch. I looked around to see how the linux kernel is configured but don''t seem to find anything. any ideas? thanks, Ron
hello, I am looking into some Lustre development but want to build everything from scratch. I looked around to see how the linux kernel is configured but don''t seem to find anything. any ideas? thanks, Ron
If you use the zfs osd for your servers, there is no need to modify your Linux kernel. That is probably a much easier way to get started with Lustre development. If you are set on using the ldiskfs osd, Intel has some instructions on their wiki here: https://wiki.hpdd.intel.com/display/PUB/Building+Lustre+from+Source Chris On 05/21/2013 07:11 AM, Ron Croonenberg wrote:> hello, > > I am looking into some Lustre development but want to build everything > from scratch. I looked around to see how the linux kernel is configured > but don''t seem to find anything. > > any ideas? > > thanks, > > Ron > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel >
Hi Chris, thanks for the link. What I was looking for was the Linux kernel config but couldn''t find any documentation foor (would have been nice to find the ones that are actuallu used in the binaries). Basically what I did is take my RHEL6 config and configured the kernel wit and compiling that seemed to work. (I did compile the rpms, just a regular make. That seemed to work for 1.8.7. I haven''t installed anything yet., so we''ll see on that part. thanks, Ron On 05/21/2013 12:16 PM, Christopher J. Morrone wrote:> If you use the zfs osd for your servers, there is no need to modify your > Linux kernel. That is probably a much easier way to get started with > Lustre development. > > If you are set on using the ldiskfs osd, Intel has some instructions on > their wiki here: > > https://wiki.hpdd.intel.com/display/PUB/Building+Lustre+from+Source > > Chris > > On 05/21/2013 07:11 AM, Ron Croonenberg wrote: >> hello, >> >> I am looking into some Lustre development but want to build everything >> from scratch. I looked around to see how the linux kernel is configured >> but don''t seem to find anything. >> >> any ideas? >> >> thanks, >> >> Ron >> _______________________________________________ >> Lustre-devel mailing list >> Lustre-devel at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-devel >> > > _______________________________________________ > Lustre-devel mailing list > Lustre-devel at lists.lustre.org > http://lists.lustre.org/mailman/listinfo/lustre-devel >
Kernel configs are kept in the tree. You can find master''s set at http://git.whamcloud.com/?p=fs/lustre-release.git;a=tree;f=lustre/kernel_pa tches/kernel_configs;h=88ca1484fa6597a7d786f3f09f03612f6d039db2;hb=HEAD -Cory On 5/21/13 2:04 PM, "Ron Croonenberg" <ronc at lanl.gov> wrote:>Hi Chris, > >thanks for the link. What I was looking for was the Linux kernel config >but couldn''t find any documentation foor (would have been nice to find >the ones that are actuallu used in the binaries). > >Basically what I did is take my RHEL6 config and configured the kernel >wit and compiling that seemed to work. (I did compile the rpms, just a >regular make. That seemed to work for 1.8.7. >I haven''t installed anything yet., so we''ll see on that part. > >thanks, > >Ron > >On 05/21/2013 12:16 PM, Christopher J. Morrone wrote: >> If you use the zfs osd for your servers, there is no need to modify your >> Linux kernel. That is probably a much easier way to get started with >> Lustre development. >> >> If you are set on using the ldiskfs osd, Intel has some instructions on >> their wiki here: >> >> https://wiki.hpdd.intel.com/display/PUB/Building+Lustre+from+Source >> >> Chris >> >> On 05/21/2013 07:11 AM, Ron Croonenberg wrote: >>> hello, >>> >>> I am looking into some Lustre development but want to build everything >>> from scratch. I looked around to see how the linux kernel is configured >>> but don''t seem to find anything. >>> >>> any ideas? >>> >>> thanks, >>> >>> Ron >>> _______________________________________________ >>> Lustre-devel mailing list >>> Lustre-devel at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-devel >>> >> >> _______________________________________________ >> Lustre-devel mailing list >> Lustre-devel at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-devel >> >_______________________________________________ >Lustre-devel mailing list >Lustre-devel at lists.lustre.org >http://lists.lustre.org/mailman/listinfo/lustre-devel
ok, cool thanks.. that probably will work better. (there were still a bunch of config options that were ''NEW'' but a lot less Ron On 05/21/2013 01:24 PM, Cory Spitz wrote:> Kernel configs are kept in the tree. You can find master''s set at > http://git.whamcloud.com/?p=fs/lustre-release.git;a=tree;f=lustre/kernel_pa > tches/kernel_configs;h=88ca1484fa6597a7d786f3f09f03612f6d039db2;hb=HEAD > > -Cory > > On 5/21/13 2:04 PM, "Ron Croonenberg" <ronc at lanl.gov> wrote: > >> Hi Chris, >> >> thanks for the link. What I was looking for was the Linux kernel config >> but couldn''t find any documentation foor (would have been nice to find >> the ones that are actuallu used in the binaries). >> >> Basically what I did is take my RHEL6 config and configured the kernel >> wit and compiling that seemed to work. (I did compile the rpms, just a >> regular make. That seemed to work for 1.8.7. >> I haven''t installed anything yet., so we''ll see on that part. >> >> thanks, >> >> Ron >> >> On 05/21/2013 12:16 PM, Christopher J. Morrone wrote: >>> If you use the zfs osd for your servers, there is no need to modify your >>> Linux kernel. That is probably a much easier way to get started with >>> Lustre development. >>> >>> If you are set on using the ldiskfs osd, Intel has some instructions on >>> their wiki here: >>> >>> https://wiki.hpdd.intel.com/display/PUB/Building+Lustre+from+Source >>> >>> Chris >>> >>> On 05/21/2013 07:11 AM, Ron Croonenberg wrote: >>>> hello, >>>> >>>> I am looking into some Lustre development but want to build everything >>>> from scratch. I looked around to see how the linux kernel is configured >>>> but don''t seem to find anything. >>>> >>>> any ideas? >>>> >>>> thanks, >>>> >>>> Ron >>>> _______________________________________________ >>>> Lustre-devel mailing list >>>> Lustre-devel at lists.lustre.org >>>> http://lists.lustre.org/mailman/listinfo/lustre-devel >>>> >>> >>> _______________________________________________ >>> Lustre-devel mailing list >>> Lustre-devel at lists.lustre.org >>> http://lists.lustre.org/mailman/listinfo/lustre-devel >>> >> _______________________________________________ >> Lustre-devel mailing list >> Lustre-devel at lists.lustre.org >> http://lists.lustre.org/mailman/listinfo/lustre-devel >
Apparently Analagous Threads
- Lustre 1.8 client on EL 6.5?
- configure error using Lustre 2.3 and OFED 3.5
- Lustre系统管理员培训,中国北京, 2012年10月30日至11月2日/ Lustre Administration and Installation training, Beijing, China, 10/30~11/02 2012
- Lustre and cross-platform portability
- Lustre and cross-platform portability