Lustre discuss - Jan 2007 - Lustre with gigE

For testing, I have convert 5 of our compute nodes to 1 channel gigabit to
run with Lustre. If I mount the /lustre filesystem on the frontend and
copy data to it, it works, however if I try to use rsync the frontend
immediately locks up.

Lustre - 1.6beta5
Kernel - 2.6.9-42.EL_lustre.1.5.95smp x86_64 x86_64 x86_64 GNU/Linux

Frontend - MGS and MDT
mkfs.lustre --fsname=bcffs --mdt --mgs /dev/sda2
mount -t lustre /dev/sda2 /lustre-meta
mount -t lustre bcf@tcp0:/bcffs /lustre

Compute nodes - OST
mkfs.lustre --fsname=bcffs --ost --mgsnode=bcf@tcp0 /dev/md0
mount -t lustre /dev/md0 /lustre-storage

I''m wondering if there isn''t something wrong with the 64bit
version of
beta5. I have an older 11 node i686 cluster with the same 1.6beta5
installation and setup, and it works perfectly.


-- 
Jeremy Mann
jeremy@biochem.uthscsa.edu

University of Texas Health Science Center
Bioinformatics Core Facility
http://www.bioinformatics.uthscsa.edu
Phone: (210) 567-2672

Just out of curiosity, is there a significant performance gain using
gigE vs 10/100M Ethernet?

lin   
> -----Original Message-----
> From: lustre-discuss-bounces@clusterfs.com 
> [mailto:lustre-discuss-bounces@clusterfs.com] On Behalf Of Jeremy Mann
> Sent: Monday, January 22, 2007 2:19 PM
> To: lustre-discuss@clusterfs.com
> Subject: [Lustre-discuss] Lustre with gigE
> 
> For testing, I have convert 5 of our compute nodes to 1 
> channel gigabit to run with Lustre. If I mount the /lustre 
> filesystem on the frontend and copy data to it, it works, 
> however if I try to use rsync the frontend immediately locks up.
> 
> Lustre - 1.6beta5
> Kernel - 2.6.9-42.EL_lustre.1.5.95smp x86_64 x86_64 x86_64 GNU/Linux
> 
> Frontend - MGS and MDT
> mkfs.lustre --fsname=bcffs --mdt --mgs /dev/sda2 mount -t 
> lustre /dev/sda2 /lustre-meta mount -t lustre bcf@tcp0:/bcffs /lustre
> 
> Compute nodes - OST
> mkfs.lustre --fsname=bcffs --ost --mgsnode=bcf@tcp0 /dev/md0 
> mount -t lustre /dev/md0 /lustre-storage
> 
> I''m wondering if there isn''t something wrong with the
64bit
> version of beta5. I have an older 11 node i686 cluster with 
> the same 1.6beta5 installation and setup, and it works perfectly.
> 
> 
> --
> Jeremy Mann
> jeremy@biochem.uthscsa.edu
> 
> University of Texas Health Science Center Bioinformatics Core 
> Facility http://www.bioinformatics.uthscsa.edu
> Phone: (210) 567-2672
> 
> _______________________________________________
> Lustre-discuss mailing list
> Lustre-discuss@clusterfs.com
> https://mail.clusterfs.com/mailman/listinfo/lustre-discuss
>

On Jan 29, 2007  11:15 -0800, Lin Shen (lshen) wrote:
> Just out of curiosity, is there a significant performance gain using
> gigE vs 10/100M Ethernet?
Yes, Lustre+LNET can saturate a GigE link (i.e. > 100Mbytes/sec if the
disk can handle this).

Cheers, Andreas
--
Andreas Dilger
Principal Software Engineer
Cluster File Systems, Inc.