Prajith Lal
2006-Jul-02 21:36 UTC
[Lustre-discuss] Lustre Configuration with Multiple nodes
Hi, I''m working with a new Lustre installation and have some problems. I installed Lustre 1.4.6 in two machines and configured successfully with Client, MDS, and two OSTs on a single node on each machine. Now I want to configure lustre with multiple nodes means MDS, OST on one machine ( dev1.testfs.com) and client on the second(dev2.testfs.com). I created a sample script as follows. #!/bin/sh # config.sh # Create nodes rm -f config.xml lmc -m config.xml --add net --node dev1.testfs.com --nid dev1.testfs.com--nettype tcp lmc -m config.xml --add net --node dev2.testfs.com --nid dev2.testfs.com--nettype tcp # Cofigure MDS lmc -m config.xml --add mds --node dev1.testfs.com --mds mds-test --fstype ext3 --dev /tmp/mds-test --size 50000 # Configures OSTs lmc -m config.xml --add lov --lov lov-test --mds mds-test --stripe_sz 1048576 --stripe_cnt 0 --stripe_pattern 0 lmc -m config.xml --add ost --node dev1.testfs.com --lov lov-test --ost ost1-test --fstype ext3 --dev /tmp/ost1-test --size 100000 lmc -m config.xml --add ost --node dev1.testfs.com --lov lov-test --ost ost2-test --fstype ext3 --dev /tmp/ost2-test --size 100000 lmc -m config.xml --add ost --node dev1.testfs.com --lov lov-test --ost ost3-test --fstype ext3 --dev /tmp/ost3-test --size 100000 # Configure client lmc -m config.xml --add mtpt --node dev2.testfs.com --path /mnt/lustre --mds mds-test --lov lov-test I created the config.xml Now I have a doubt how will I run the lconf command for the config.xml Can I run the config.xml command in the client(dev2.testfs.com) as well as in the MDS(dev1.testfs.com)? Is there any need for mapping the client IP in ''/etc/hosts'' file of MDS? Thanks, Prajith -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20060630/e4782222/attachment.html
Wang Yibin
2006-Jul-02 22:12 UTC
[Lustre-discuss] Lustre Configuration with Multiple nodes
Prajith Lal wrote:> > I created the config.xml > Now I have a doubt how will I run the lconf command for the config.xml > Can I run the config.xml command in the client(dev2.testfs.com > <http://dev2.testfs.com>) as well as in the MDS(dev1.testfs.com > <http://dev1.testfs.com>)? > Is there any need for mapping the client IP in ''/etc/hosts'' file of MDS?You can start servers with lconf. The recommended order is the OSSs then the MDS lconf config.xml. #lconf --reformat --node dev1.testfs.com config.xml Then mount the lustre fs: #lconf --reformat --node dev2.testfs.com config.xml mount -t lustre ... also works. -- Cheers, Wang Yibin
Prajith Lal
2006-Jul-03 01:06 UTC
[Lustre-discuss] Lustre Configuration with Multiple nodes
When I try to execute lconf --node dev1.testfs.com config.xml it hangs and the dmesg says kjournald starting. Commit interval 5 seconds LDISKFS FS on loop5, internal journal LDISKFS-fs: mounted filesystem with ordered data mode. SELinux: initialized (dev loop5, type ldiskfs), not configured for labeling LustreError: 5389:0:(linux-tcpip.c:666:libcfs_sock_connect()) Error -113 connecting 0.0.0.0/1023 -> 69.10.150.203/988 LustreError: Connection to 69.10.150.203@tcp at host 69.10.150.203 was unreachable: the network or that node may be down, or Lustre may be misconfigured. I don''t know why it is, the IP(69.10.150.203) is dev2.testfs.com and I mapped the IP in the /etc/hosts file Thanks, Prajtih On 7/3/06, Wang Yibin <wangyb@clusterfs.com> wrote:> > Prajith Lal wrote: > > > > I created the config.xml > > Now I have a doubt how will I run the lconf command for the config.xml > > Can I run the config.xml command in the client(dev2.testfs.com > > <http://dev2.testfs.com>) as well as in the MDS(dev1.testfs.com > > <http://dev1.testfs.com>)? > > Is there any need for mapping the client IP in ''/etc/hosts'' file of MDS? > > You can start servers with lconf. > The recommended order is the OSSs then the MDS lconf config.xml. > #lconf --reformat --node dev1.testfs.com config.xml > > Then mount the lustre fs: > #lconf --reformat --node dev2.testfs.com config.xml > mount -t lustre ... also works. > > -- > Cheers, > Wang Yibin > >-- Lals -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20060703/bd9be3fa/attachment.html
Prajith Lal
2006-Jul-04 21:41 UTC
[Lustre-discuss] Lustre Configuration with Multiple nodes
Hi, It was the firewall issue . I stopped IPtables service and run the script it worked smoothly. Now the thing is I want to add the rules for Lustre FS in my IPtables. What are the deafult ports used by Lustre? Thanks, Prajith On 7/3/06, Prajith Lal <prajithlal@gmail.com> wrote:> > When I try to execute lconf --node dev1.testfs.com config.xml it hangs and > the dmesg says > > kjournald starting. Commit interval 5 seconds > LDISKFS FS on loop5, internal journal > LDISKFS-fs: mounted filesystem with ordered data mode. > SELinux: initialized (dev loop5, type ldiskfs), not configured for > labeling > LustreError: 5389:0:(linux-tcpip.c:666:libcfs_sock_connect()) Error -113 > connecting 0.0.0.0/1023 -> 69.10.150.203/988 > LustreError: Connection to 69.10.150.203@tcp at host 69.10.150.203 was > unreachable: the network or that node may be down, or Lustre may be > misconfigured. > > I don''t know why it is, the IP(69.10.150.203) is dev2.testfs.com and I > mapped the IP in the /etc/hosts file > > Thanks, > Prajtih > > > On 7/3/06, Wang Yibin <wangyb@clusterfs.com> wrote: > > > > Prajith Lal wrote: > > > > > > I created the config.xml > > > Now I have a doubt how will I run the lconf command for the config.xml > > > Can I run the config.xml command in the client( dev2.testfs.com > > > <http://dev2.testfs.com>) as well as in the MDS(dev1.testfs.com > > > <http://dev1.testfs.com >)? > > > Is there any need for mapping the client IP in ''/etc/hosts'' file of > > MDS? > > > > You can start servers with lconf. > > The recommended order is the OSSs then the MDS lconf config.xml. > > #lconf --reformat --node dev1.testfs.com config.xml > > > > Then mount the lustre fs: > > #lconf --reformat --node dev2.testfs.com config.xml > > mount -t lustre ... also works. > > > > -- > > Cheers, > > Wang Yibin > > > > > > > -- > Lals >-- Lals -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20060704/31afa704/attachment.html
Andreas Dilger
2006-Jul-04 21:51 UTC
[Lustre-discuss] Lustre Configuration with Multiple nodes
On Jul 04, 2006 15:16 +0530, Prajith Lal wrote:> It was the firewall issue . I stopped IPtables service and run the script it > worked smoothly. Now the thing is I want to add the rules for Lustre FS in > my IPtables. > > What are the deafult ports used by Lustre?Port 988 for incoming connections (unless specified otherwise in your configuration), and a source port < 1024 for return connections.> Thanks, > Prajith > > On 7/3/06, Prajith Lal <prajithlal@gmail.com> wrote: > > > >When I try to execute lconf --node dev1.testfs.com config.xml it hangs and > >the dmesg says > > > >kjournald starting. Commit interval 5 seconds > >LDISKFS FS on loop5, internal journal > >LDISKFS-fs: mounted filesystem with ordered data mode. > >SELinux: initialized (dev loop5, type ldiskfs), not configured for > >labeling > >LustreError: 5389:0:(linux-tcpip.c:666:libcfs_sock_connect()) Error -113 > >connecting 0.0.0.0/1023 -> 69.10.150.203/988 > >LustreError: Connection to 69.10.150.203@tcp at host 69.10.150.203 was > >unreachable: the network or that node may be down, or Lustre may be > >misconfigured. > > > > I don''t know why it is, the IP(69.10.150.203) is dev2.testfs.com and I > >mapped the IP in the /etc/hosts file > > > >Thanks, > >Prajtih > > > > > >On 7/3/06, Wang Yibin <wangyb@clusterfs.com> wrote: > >> > >> Prajith Lal wrote: > >> > > >> > I created the config.xml > >> > Now I have a doubt how will I run the lconf command for the config.xml > >> > Can I run the config.xml command in the client( dev2.testfs.com > >> > <http://dev2.testfs.com>) as well as in the MDS(dev1.testfs.com > >> > <http://dev1.testfs.com >)? > >> > Is there any need for mapping the client IP in ''/etc/hosts'' file of > >> MDS? > >> > >> You can start servers with lconf. > >> The recommended order is the OSSs then the MDS lconf config.xml. > >> #lconf --reformat --node dev1.testfs.com config.xml > >> > >> Then mount the lustre fs: > >> #lconf --reformat --node dev2.testfs.com config.xml > >> mount -t lustre ... also works. > >> > >> -- > >> Cheers, > >> Wang Yibin > >> > >> > > > > > >-- > >Lals > > > > > > -- > Lals> _______________________________________________ > Lustre-discuss mailing list > Lustre-discuss@clusterfs.com > https://mail.clusterfs.com/mailman/listinfo/lustre-discussCheers, Andreas -- Andreas Dilger Principal Software Engineer Cluster File Systems, Inc.
Prajith Lal
2006-Jul-05 00:57 UTC
[Lustre-discuss] Lustre Configuration with Multiple nodes
You are right, Prot 988 is used for incoming connections!. I had added a new rule in iptables to open a port 988 in the MDS, OST servers and its ok, the server is now accepting connections from the clients but what is this 1024? After disable firewall temperrory I listed all the ports used and get the port 988 (used the nmap command) but I can not able to see the port 1024 anywhere . where I want to add this port? in the client or in the MDS, OST server Thanks, Prajith On 7/5/06, Andreas Dilger <adilger@clusterfs.com> wrote:> > On Jul 04, 2006 15:16 +0530, Prajith Lal wrote: > > It was the firewall issue . I stopped IPtables service and run the > script it > > worked smoothly. Now the thing is I want to add the rules for Lustre FS > in > > my IPtables. > > > > What are the deafult ports used by Lustre? > > Port 988 for incoming connections (unless specified otherwise in your > configuration), and a source port < 1024 for return connections. > > > Thanks, > > Prajith > > > > On 7/3/06, Prajith Lal <prajithlal@gmail.com> wrote: > > > > > >When I try to execute lconf --node dev1.testfs.com config.xml it hangs > and > > >the dmesg says > > > > > >kjournald starting. Commit interval 5 seconds > > >LDISKFS FS on loop5, internal journal > > >LDISKFS-fs: mounted filesystem with ordered data mode. > > >SELinux: initialized (dev loop5, type ldiskfs), not configured for > > >labeling > > >LustreError: 5389:0:(linux-tcpip.c:666:libcfs_sock_connect()) Error > -113 > > >connecting 0.0.0.0/1023 -> 69.10.150.203/988 > > >LustreError: Connection to 69.10.150.203@tcp at host 69.10.150.203 was > > >unreachable: the network or that node may be down, or Lustre may be > > >misconfigured. > > > > > > I don''t know why it is, the IP(69.10.150.203) is dev2.testfs.com and I > > >mapped the IP in the /etc/hosts file > > > > > >Thanks, > > >Prajtih > > > > > > > > >On 7/3/06, Wang Yibin <wangyb@clusterfs.com> wrote: > > >> > > >> Prajith Lal wrote: > > >> > > > >> > I created the config.xml > > >> > Now I have a doubt how will I run the lconf command for the > config.xml > > >> > Can I run the config.xml command in the client( dev2.testfs.com > > >> > <http://dev2.testfs.com>) as well as in the MDS(dev1.testfs.com > > >> > <http://dev1.testfs.com >)? > > >> > Is there any need for mapping the client IP in ''/etc/hosts'' file of > > >> MDS? > > >> > > >> You can start servers with lconf. > > >> The recommended order is the OSSs then the MDS lconf config.xml. > > >> #lconf --reformat --node dev1.testfs.com config.xml > > >> > > >> Then mount the lustre fs: > > >> #lconf --reformat --node dev2.testfs.com config.xml > > >> mount -t lustre ... also works. > > >> > > >> -- > > >> Cheers, > > >> Wang Yibin > > >> > > >> > > > > > > > > >-- > > >Lals > > > > > > > > > > > -- > > Lals > > > _______________________________________________ > > Lustre-discuss mailing list > > Lustre-discuss@clusterfs.com > > https://mail.clusterfs.com/mailman/listinfo/lustre-discuss > > > Cheers, Andreas > -- > Andreas Dilger > Principal Software Engineer > Cluster File Systems, Inc. > >-- Lals -------------- next part -------------- An HTML attachment was scrubbed... URL: http://mail.clusterfs.com/pipermail/lustre-discuss/attachments/20060704/db345d00/attachment.html