hi: I tried to run Lustre over Myrinet interface. First experiment, Lustre over tcp/ip through Myrinet. Second experiment, try to compile Lustre with --with-gm option. Both experiment failed. The firt experiment fails because TCP connection can not be established through Myrinet interface. The second experiment failed due to compilation errors when compiling Lustre. Environment: Kernel version: Linux version 2.4.21-20.ELsmp Patched and compiled. 2 PCs, connected via fast ethernet and Myrinet. Lustre can be run successfully through fast ethernet interface. First experiment: node0 is acting as MDS, OST. node1 is only acting as a client. node0 can be loaded using lconf utility. node1 hangs. can not connect to node0. node0 and node1 can not make TCP connections. #netstat -n --all tcp 0 1 192.168.1.252:32860 192.168.1.1:988 SYN_SENT node1 is sending SYN to node0, but never receive acks. this also can be confirmed by tcpdump. /usr/sbin/tcpdump -i myri0 tcp port 988 07:34:35.728455 192.168.1.252.32860 > 192.168.1.1.988: S 3177422378:3177422378(0) win 17920 <mss 8960,sackOK,timestamp 3085021 0,nop,wscale 8> (DF) 07:34:38.728163 192.168.1.252.32860 > 192.168.1.1.988: S 3177422378:3177422378(0) win 17920 <mss 8960,sackOK,timestamp 3085021 0,nop,wscale 8> (DF) node0 receives SYN and never sends back ack. I don''t know if it is the error of TCP/IP stack or portals. But I can run MPICH and PVFS2 over TCP/IP on Myrinet. Second experiment: ./configure --with-linux=/usr/src/linux-2.4.21-20.EL/ --with-gm= ../gm-2.0.21_Linux/ make gcc -D__KERNEL__ -I/usr/src/linux-2.4.21-20.EL/include -Wall -Wstrict-prototypes -Wno-trigraphs -O2 -fno-strict-aliasing -fno-common -Wno-unused -fomit-frame-pointer -pipe -freorder-blocks -mpreferred-stack-boundary=2 -march=i686 -DMODULE -DMODVERSIONS -include /usr/src/linux-2.4.21-20.EL/include/linux/modversions.h -I/home/yxz12/gm-2.0.21_Linux/include -I/home/yxz12/gm-2.0.21_Linux/drivers -I/home/yxz12/gm-2.0.21_Linux/drivers/linux/gm -include /home/yxz12/lustre-1.2.4/include/config.h -g -I/home/yxz12/lustre-1.2.4/portals/include -I/home/yxz12/lustre-1.2.4/include -nostdinc -iwithprefix include -DKBUILD_BASENAME=gmnal_api -DEXPORT_SYMTAB -c gmnal_api.c In file included from /home/yxz12/gm-2.0.21_Linux/include/gm_io.h:23, from /home/yxz12/gm-2.0.21_Linux/include/gm_internal.h:62, from gmnal.h:63, from gmnal_api.c:26: /home/yxz12/gm-2.0.21_Linux/drivers/linux/gm/gm_arch_io.h:13:23: sys/errno.h: No such file or directory In file included from /home/yxz12/gm-2.0.21_Linux/include/gm_internal.h:62, from gmnal.h:63, from gmnal_api.c:26: /home/yxz12/gm-2.0.21_Linux/include/gm_io.h:26:23: sys/types.h: No such file or directory gmnal_api.c: In function `gmnal_init'': gmnal_api.c:261: warning: assignment from incompatible pointer type gmnal_api.c:261: warning: assignment from incompatible pointer type gmnal_api.c:261: warning: assignment from incompatible pointer type gmnal_api.c:261: warning: assignment from incompatible pointer type gmnal_api.c:261: warning: assignment from incompatible pointer type gmnal_api.c:261: warning: assignment from incompatible pointer type gmnal_api.c:261: warning: assignment from incompatible pointer type make[6]: *** [gmnal_api.o] Error 1 __________________________________________________ Do You Yahoo!? Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com