hi:
I tried to run Lustre over Myrinet interface.
First experiment, Lustre over tcp/ip through Myrinet.
Second experiment, try to compile Lustre with
--with-gm option. Both experiment failed. The firt
experiment fails because TCP connection can not be
established through Myrinet interface. The second
experiment failed due to compilation errors when
compiling Lustre.
Environment:
Kernel version: Linux version 2.4.21-20.ELsmp
Patched and compiled.
2 PCs, connected via fast ethernet and Myrinet.
Lustre can be run successfully through fast ethernet
interface.
First experiment:
node0 is acting as MDS, OST.
node1 is only acting as a client.
node0 can be loaded using lconf utility.
node1 hangs. can not connect to node0.
node0 and node1 can not make TCP connections.
#netstat -n --all
tcp 0 1 192.168.1.252:32860
192.168.1.1:988 SYN_SENT
node1 is sending SYN to node0, but never receive acks.
this also can be confirmed by tcpdump.
/usr/sbin/tcpdump -i myri0 tcp port 988
07:34:35.728455 192.168.1.252.32860 > 192.168.1.1.988:
S 3177422378:3177422378(0) win 17920 <mss
8960,sackOK,timestamp 3085021 0,nop,wscale 8> (DF)
07:34:38.728163 192.168.1.252.32860 > 192.168.1.1.988:
S 3177422378:3177422378(0) win 17920 <mss
8960,sackOK,timestamp 3085021 0,nop,wscale 8> (DF)
node0 receives SYN and never sends back ack. I don''t
know if it is the error of TCP/IP stack or portals.
But I can run MPICH and PVFS2 over TCP/IP on Myrinet.
Second experiment:
./configure --with-linux=/usr/src/linux-2.4.21-20.EL/
--with-gm= ../gm-2.0.21_Linux/
make
gcc -D__KERNEL__ -I/usr/src/linux-2.4.21-20.EL/include
-Wall -Wstrict-prototypes -Wno-trigraphs -O2
-fno-strict-aliasing -fno-common -Wno-unused
-fomit-frame-pointer -pipe -freorder-blocks
-mpreferred-stack-boundary=2 -march=i686 -DMODULE
-DMODVERSIONS -include
/usr/src/linux-2.4.21-20.EL/include/linux/modversions.h
-I/home/yxz12/gm-2.0.21_Linux/include
-I/home/yxz12/gm-2.0.21_Linux/drivers
-I/home/yxz12/gm-2.0.21_Linux/drivers/linux/gm
-include /home/yxz12/lustre-1.2.4/include/config.h -g
-I/home/yxz12/lustre-1.2.4/portals/include
-I/home/yxz12/lustre-1.2.4/include -nostdinc
-iwithprefix include -DKBUILD_BASENAME=gmnal_api
-DEXPORT_SYMTAB -c gmnal_api.c
In file included from
/home/yxz12/gm-2.0.21_Linux/include/gm_io.h:23,
from
/home/yxz12/gm-2.0.21_Linux/include/gm_internal.h:62,
from gmnal.h:63,
from gmnal_api.c:26:
/home/yxz12/gm-2.0.21_Linux/drivers/linux/gm/gm_arch_io.h:13:23:
sys/errno.h: No such file or directory
In file included from
/home/yxz12/gm-2.0.21_Linux/include/gm_internal.h:62,
from gmnal.h:63,
from gmnal_api.c:26:
/home/yxz12/gm-2.0.21_Linux/include/gm_io.h:26:23:
sys/types.h: No such file or directory
gmnal_api.c: In function `gmnal_init'':
gmnal_api.c:261: warning: assignment from incompatible
pointer type
gmnal_api.c:261: warning: assignment from incompatible
pointer type
gmnal_api.c:261: warning: assignment from incompatible
pointer type
gmnal_api.c:261: warning: assignment from incompatible
pointer type
gmnal_api.c:261: warning: assignment from incompatible
pointer type
gmnal_api.c:261: warning: assignment from incompatible
pointer type
gmnal_api.c:261: warning: assignment from incompatible
pointer type
make[6]: *** [gmnal_api.o] Error 1
__________________________________________________
Do You Yahoo!?
Tired of spam? Yahoo! Mail has the best spam protection around
http://mail.yahoo.com