Lustre discuss - Jul 2006 - Meeting Summary for lustre.org Community Forum 2006-06-28

lustre.org Community Forum
2006-06-28 - SESSION 2 - Lustre Roadmap/Development Activities

Prepared by: Peter Jones
Date: 2006-06-28


MEETING NOTES

1. Myricom - Myrinet MX LND - Scott Atchley

pbojanic: what were your experiences with documentation/support on  
LND development?
Scott: high ramp up because new to Myricom as well as Lustre. eeb was  
great. Docs were Email and code review. Going forward, wants to doc  
for any future developers.

Q: performance?
Scott: eeb gave scripts. mem to mem operations between server and  
client.

Q: also tests routing?
Scott: did not try but would think so
Sandia: thinks that they have done that

Q: will CFS include Myricom code?
pbojanic: yes, will go into standard packages. Myricom''s understanding?
Scott: yes, happy to do that.

pbojanic: copyright?
Scott: no, but we should
ACTION: joint copyright assignment


2.  LLNL - Lustre monitoring tools - Herb Wartens
http://sourceforge.net/projects/lmt/

Herb:
- Written in Python.
- daemon runs on all servers.
- tree-view shows which file system details
- testbed file system has three clusters all talking to same file  
system.
- treeview shows status of MDS, portals
- could potentially have multiple file systems displayed
- portal routers show bandwidth and status info e.g. router goes down

Q: client statistics?
Herb: could maybe add another pane. code is straightforward

Q: difficult migration to Lustre 1.6.0 with mountconf?
Herb: no, as long as config file present. the tool uses the old file.  
if had formatted file system with 1.4.6 then just hacked xml. wants  
to remove this dependency which should make things easier for 1.6.0

Q: trending in database?
Herb: yes, would like to do that. dump everything into mysql to see  
trends. can dump to text file and then import those. but big, because  
shows all read\writes etc

pbojanic: what is best venue to discuss this tool?
Herb: on SourceForge so use forum there


3. CEA - HSM support - Jacques-Charles Lafoucriere

JC:
- want to keep namespace in Lustre. LRU in Lustre.
- userspace tool between Lustre and HSM
- free space in Lustre to avoid Lustre becoming full
- recent design in collaboration with CFS
- HPSS: fast copy between Lustre and HSM

Q: what timeframe?
JC: less than one year.
Indiana: please email if they can help

JC: resources?
several people: interested, but not in HPSS

pbojanic: best way to continue discussion? working group?

JC: share design paper with community to see if can converge on design.
several people: yes!
pbojanic: Lustre devel? If contention then could have separate list,  
but traffic is light there at present

Q: Needs to be someone overseeing vision of whole development community
pbojanic: tabled for tomorrow


4. Bull - Lustre management tools - Philippe Couv?e
http://www.bullopensource.org/lustre/

Phillipe:
- just released this morning
- Tools designed to make Lustre work on large cluster 500+ client
- single point of management
- install, start, stop, etc
- tuning
- command line interface
- designed for one cluster, so some limitations (e.g.1.4.6 Lustre only)
- Quadrics\Elan interconnect or TCP not both simultaneously
- fast formatting
- more info on Bull OpenSource website
- will be working on 1.6 support
- Openfabric support
- HA docs integration\database
- source code as patch on Lustre source tree because two patches
- easier for RPMs or TAR files
- would like community to contribute
- also know that problems with other architecture (currently ia64 only)
- kernel IO scheduler needs  testing for  other distros
- two-three months community review  then  another  discussion in  
Sept timeframe to discuss new features

pbojanic: most features related to config, but also mentioned  
monitoring elements. How does this compare to the LLNL offering?
Phillipe: stats from diff file systems then send to management tool.  
Cannot compare to LLNL tool. Thinks tool more interesting from  
management perspective rather than monitoring perspective

pbojanic: maybe CFS could roll those patches into mainline.
Philipee: yes, that would be good to remove that dependency and think  
that this should be feasible

pbojanic: start discussion on Lustre devel?
Philippe: planned to publish earlier rather than after code cleanup.  
so prepared for feedback
pbojanic: praised publishing decision despite concerns about review

pbojanic: intitiatives within CFS on monitoring. Will be sure to  
direct engineers in CFS to Bull\LLNL efforts to benefit from their  
experiences. Will publish Bull site in minutes

Q: preferred licence for Lustre.org
pbojanic: since Lustre is GPL, that would be the default. If you have  
reservations about using GPL then happy to discuss reservations and  
other options